APIs for all

Last week I talked about GOV.UK’s APIs and how we use them to build and monitor the site. Now it’s time to say a little more about what they mean for other developers who might want to use our content and data.

As with everything we do at GDS, our approach to APIs is driven by user need. Our core focus with APIs has been to approach them as a tool to help serve the vast majority of user needs that we’re serving with web pages. But we know there are many opportunities to serve needs by working with other people to provide the right APIs.

The same APIs that GOV.UK uses internally are available to anyone using the site. For most of our content-rich pages you can get the full content of the page and some supporting information (such as which categories it lives in and whether it’s designed for businesses) with a simple change of URL. So for:

https://www.gov.uk/passport-fees

You’d use:

https://www.gov.uk/api/passport-fees.json

As on the beta, we’ve included links in the page headers pointing to the API version. A similar approach works for getting search results.

There are still quite a few rough edges in these APIs (eg. a little inconsistency in some results and that annoying .json on the end of the URLs above). We’ve been so highly focussed on getting the main site ready that there’s not been time to add the extra polish we’d like. Over the next couple of months we’ll be making time to remove a few inconsistencies and pull together some documentation and an explorer to help developers find their way around.

What we really need now is feedback to help us understand the needs of those partners who might build things with our APIs. If you’ve got ideas or questions please do get in touch and help us shape the next iteration of GOV.UK APIs.

47 comments

  1. Great stuff James, and should lead to some interesting mashups, especially when combined with much of GOV.UK’s content being published under the liberal Open Government Licence.

    Of course with URIs, there is no consensus on style, which is actually fine as none is needed for The Web to work, but I actually like the .json suffix because it’s useful when testing in a browser and I’m not so keen on the word “/api/” in the path of the URL, but you already know that ;^)

    1. Good points, Paul.

      The /api/ prefix is an unfortunate reflection of our internal architecture. I hope we can make it (and .json vs. an Accept header) optional in a future release.

      1. +1 for removal of “/api/” being in the path.

        As well as the Accept header and implied extension, I’ve also seen X- headers such as X-PJAX used successfully to imply how the client would like content back.

  2. Not keen on /api in the URI, looking forward to the docs I think I have a few cool ideas to make a few apps

  3. +1 for API removal, I also like the .json extension for the same reasons as already mentioned. It should certainly generate some interesting re-uses of the data and content you hold. Further as a web developer myself it also helps me to sell in the concept of API’s to my clients as well, maybe they won’t need them now (although useful for single page style web apps) but certainly useful later and saves time spent re-engineering. GDS’s approach helps make a lot of things easier for selling into clients (as I see it) Great work.

      1. Is there any plan for a search widget to be made, similar to that on DirectGov site?

        The DirectGov site offered a simple search widget which allowed a user enter a location and to select from a number of categories such as “Jobs” or “Bin collections”. When run, the user was taken to an appropriate page on the DirectGov site.

        An example of the DirectGov search widget may be seen in the left hand menu of
        http://www.hertsdirect.org/your-community/

  4. Sounds great. Do you have a Guardian-style micro-apps model planned for the future, to help roll smaller third party apps/APIs into services delivered with the scalability of GOV.UK behind it?

    1. That’d definitely be an option, but we’ve not got any specific plans at the moment. There’s a lot more work and thinking to do as we look at the next stages of GOV.UK and other government services.

  5. Looks like you did manage to remove “/api” from the path already, but that’s now a 403 — while the non-“/api”.json URL does work. A redirect would have been nice ;)

    1. Hmm.. and now, seconds later, both URLs work. Though the two URLs give different results.

      Never mind!

  6. Firstly, thanks for all the work you have already done – we’re already using things like the bank holiday calendars in our back-end to automate things that we’ve previously had to do by hand.

    We’d find it hugely useful if the JSON API endpoints for the following pages returned the actual data on the page, not just the HTML content of the page wrapped in JSON:

    https://www.gov.uk/vehicle-tax-rate-tables
    https://www.gov.uk/vat-rates

    Great work guys. Can’t wait to see what’s coming next.

    1. Already responded to this on twitter, but for completeness…

      We’re definitely hoping to make that kind of data more accessible but it’s not been a top priority for the launch of GOV.UK. Hopefully we’ll get there soon!

      Please do keep sending feedback like this – it’s really helpful as we work out where to focus as we try to make the API more useful.

  7. Great post. A smallish addition that I’d like to see is an API that provides a complete listing of all the content URIs (answers etc.).

  8. Is there an API that lets you know all elected positions on a national and local levels in the UK. i.e. All councils, (Parish, local, district, GLA), MP, Scottish, and Welsh assembly’s? I want to know if this is kept by GOV.uk or will I have to try to find it all from various sources? Any help would be appreciated.

  9. So the passport-fees example you give currently results in a 403 and dropping /api/ results in it being directed back to the +/api/ URL and the same 403.

    Would be nice to have some API docs rather than just a link from the gov.uk FAQ to here, and then an example that doesn’t currently work :-( any idea when those kind of lists of information for developers will be made available?

    1. We’re aware of the 403 issue and working to fix it. Sorry it caught you. You can usually get around it by adding a query string (any query string) to the request – does that work?

      We’re also planning to get the API better documented, provide an explorer, and so on later in the year. It’s really important to us to do that, but not as high a priority as the ongoing migration of departments onto GOV.UK so we’ve had to hold off for a while.

      1. Hi James,
        I was using the FCO’s XML Country index, now using https://www.gov.uk/api/foreign-travel-advice.json much easier. Thanks for the link.

        Just one query:
        will there be a maximum of one image element for an individual country like https://www.gov.uk/api/foreign-travel-advice/afghanistan.json ? I hope so, but if not it would useful to have an “{image caption “and “{image parts_slug” to identify to which part of the content each image relates.

        1. Thanks Andy. Glad you’re liking the new .json API for foreign travel advice.

          There _should_ reliably be a maximum of one image per country. (We’ll definitely consider further improvements to the API if that situations ever looks likely to change though).

  10. Do you have some example javascript for accessing the travel advice please? I can download for example afghanistan.json and access it with:
    $.getJSON(“afghanistan.json”,function(result){
    alert(result);

    But using:
    $.getJSON(“https://www.gov.uk/api/foreign-travel-advice/afghanistan.json”,function(result){
    alert(result);

    Returns nothing.

    Many thanks.

    1. I have spent a bit more time on this and realised that I should have been using JSONP.

      However when I use:
      $.getJSON(“https://www.gov.uk/api/foreign-travel-advice/afghanistan.json?jsoncallback=?”,{})
      I get the error:
      Message: Expected ‘;’
      Line: 1
      Char: 18
      Code: 0
      URI: https://www.gov.uk/api/foreign-travel-advice/afghanistan.json?jsoncallback=jQuery19105645641892865305_1368006944041&_=1368006944042

      This seems to happen for any country page.

      Could anyone tell me if I am doing somethign wrong, or is there a problem with the JSON syntax?

      Many thanks

      1. Hi Tim,

        The JSON’s valid (I just checked it with jsonlint.com). I’d suspect you’re actually getting a 403 Forbidden HTTP response? We’ve not done any work yet to make the API work well for JSONP and it may well conflict with some of our security policies.

        That’s something we’ll be revisiting over the next few months, but in the meantime you’re probably best off setting up a proxy on your server so you can make the API requests server-side and then use regular ajax requests to connect back to your own server. That approach will also give you more control of your caching so is often preferable anyway.

        James.

  11. It would be nice if you updated the example in your original document to remove the /api/ in the URL rather than requiring users to read through the comments to find out this is now unnecessary. Any plans to create a developer portal?

    1. Hi Ian,

      The URLs with /api/ are still the better way to access the JSON versions of content. That’s under review but for now it’s just that the other URLs are designed to redirect to the /api/ equivalent.

      We’re doing a bit of work at the moment to get a better understanding of who the users are for our APIs and how we can best provide support. I’d love to hear more about how you’re using the API and what you’d be looking for from a developer portal if we were to launch one. Comments here are great or you can reach me at james.stewart@digital.cabinet-office.gov.uk

  12. Hi James.

    I am trying to understand the API and I really love how simple it is. The “lang” field made me very happy, because I am translating GOV’s publications into different language. This simple field allows me to use the same API for both languages. No need to write two separate applications. The API explains itself, and it’s easy to “copy” your database’s structure and translate the data very quickly.

    I found the “/api/” prefix easy to handle, just use kind of StringReplace function that exists in every programming language, so I can’t complain about that. Personally, I think its good idea.
    Can’t think about any ‘more convenient’ way to access the API that will be really more convenient.

    To compare if my translated content is up-to-date, I use the field ‘updated_at’ so I don’t have to compare all of the data, and that’s ‘like’ for your API.

    There are other nice things like ‘other relevant content’, sections and tags… Unfortunately it is there, where my problem with the API starts.

    For example, https://www.gov.uk/api/with_tag.json?section=benefits%2Fchild provides invalid JSON.

    ‘https://www.gov.uk/api/with_tag.json?section=benefits’ provide only one link about PIP.

    I had to use (unpublished) version of that query:
    https://www.gov.uk/api/tags.json?type=section&parent_id=benefits
    I discovered that when tried:
    https://www.gov.uk/browse/xyz.json – it redirected me to the https://www.gov.uk/api/tags.json?type=section&parent_id=xyz (note, https://www.gov.uk/api/browse/xyz.json will not work).

    However https://www.gov.uk/api/tags.json?type=section&parent_id=benefits%2Fchild still doesn’t work.

    ‘Content with tag’: like https://www.gov.uk/api/tags/sections/benefits%2Fchild.json doesn’t work too, though https://www.gov.uk/api/tags/sections/benefits.json do.

    Since you use the tags and sections on GOV’s website, I understand it is working somehow – I assume you use the same API for GOV. I am wonder, how to extract those things without manual handling. It holds my project in the alpha (covered with dust on my localhost), because I dislike the idea of handwriting tags and sections.

    Would be nice to have access to tags, sections and other things directly, providing as parameter numeric id or ‘+’ separated words (not benefits%2Fchild but benefits+child).

    I am sorry if my post looks somehow chaotic, just wanted to share as much, as I can in short and simple way.

    Kind Regards
    P. Hass

    1. Hi Przemyslaw,

      I’m sorry to hear you’re having trouble with the JSON you’re getting from several pages. The JSON I get when I use the URLs you listed is valid so we clearly need to do a bit more investigation to see why you’re getting different results.

      These comments aren’t really the best place for detailed discussions of this sort. If you could complete our support form https://www.gov.uk/feedback/contact that’ll make sure your request goes to the right place and we’ll get back to you asap.

      James.

      1. Thank you James for quick answer.
        Of course, I’ll use the ‘contact’ form. I have revised all my scripts to check for possible errors on my site to avoid commenting things that doesn’t work because of my fault.

        By the way, do you plan to allow access (through the API) to tools like Benefits Advisor? I guess yes, and can’t wait for that.

        I see that your team works hard, and would like to say I really appreciate that. All of you can be proud of job well done. Can only guess how much work you had and still have. The longer I browse GOV.UK, the more I am impressed, not only by the API but the whole content you provide in one place.

        Best Regards
        P. Hass

  13. It’s good practice to keep the “api” directory reference esp if the URL mirrors a sites URL plus it’s good practice to keep the json exstention to allow a developer using the API to see what script to expect but most importantly it allows the sites developers to add other scripts for more compatibility across other languages.

  14. Does anyone know what has happend to the json data for countries?
    I have been querying json.details.alert_status for a country page to check if there is an alert. But as of yesterday this array now seems to be empty for all countries, even when there is an alert.
    There is json.details.summary, but this includes html tags, so not valid json.

    1. Hi Tim,

      Sorry to hear that’s not working for you. I just tested it for Afghanistan and got an appropriate JSON response but it may be that there’s an intermittent fault.

      I’ll let the relevant team know, but it would be really helpful if you could fill in https://www.gov.uk/contact/govuk with the details if you run into the problem again as that’ll get all the info to someone who can really dig into the problem.

    2. I don’t know how long this has been the case but by pure chance I noticed this on 10 Jan.

      Hi @James. Yes, json works BUT “details.alert_status” data is no longer included for ANY country (or checking today is present but empty).

      This element identified whether the travel advice contained any “advise against travel” notice; and if so its type. – very useful for developers wishing to add value. I have altered my code to scrape (falible due to wording variations) the ‘summary’ for this information.

      I’ve raised it before – is it possible to set up a mailing list or rss feed that notifies us of upcoming changes to Govt data? If developers are unaware of changes you run the risk of your feeds resulting in output of dangerously misleading information e.g. an empty list of countries where normal travel insurance will not cover you.

      Best

      1. We’ve done some investigation and it appears that there were some editorial changes made to the content which means there aren’t currently any alert statuses showing up – in this case it’s not an API change per se as they will still show up if they’re populated so even if there were an announcements list for API changes we wouldn’t have been able to catch this one.

        We’re following up with the team responsible and should have an update soon. In the meantime I’d really advise using the form at https://www.gov.uk/contact/govuk to register any issues like this as that will make sure feedback quickly gets to the relevant product teams and they can provide updates.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s