Coding in the open

We talk a lot about Open Source at GDS. GOV.UK is built on open source software and, to a degree, built as open source software. It’s a topic we care passionately about because it helps us maintain our focus on user needs by helping us to quickly test and iterate software and systems.

The software world has a rich and growing range of tools to build your own software on. From operating systems such as GNU/Linux, web servers such as Apache or nginx, databases like MySQL, Postgres or MongoDB, the tools are there so we don’t have to do it all ourselves.

The challenge is often in knowing which of those tools are right for the job you’re focussed on.

Solving problems with the right tools

By choosing open source we can quickly try the tools that seem right, test them out, and make a decision about whether this is the right solution and start using the software straight away. A good example of that is the Elasticsearch search engine that Rob wrote about a few months back. From what we were reading it seemed like it’d be simpler for us to work with than solr, and we were able to dive in and try it out without a lengthy pre-sales process or costly and complex licensing arrangements. Instead, we test, we decide, we get on with tailoring our choice to best serve the user needs we’re focussed on.

There are open source tools that cover most of what we’re doing. We can analyse which are going to fit together in the ways we want, and where we should write our own code to make sure we’re building the best product we can in a way we can maintain. One of my fellow GDS architects, Paul Downey, has often likened our approach to that advocated by JP Rangaswami in his “open source pyramid”:

For common problems use Opensource.
For rare problems use Buy.
For unique problems use Build.

Make things open: it makes things better

We’re also making our code available for others to use. We tend to talk about that as Coding In The Open rather than Open Source. Partly because we’re not currently in a position to build and support a community around that code, and partly because most of it’s so tailored to our system that we’re not sure how useful it will be to others right now. But rest assured, if you see something useful you can use it – it’s published under an MIT licence.

We don’t want to jump to conclusions about what people will want to use, but we’re eager to hear from other people who might have overlapping needs. Together we can work out which components would be good to extract and package up in a way that makes them easier to share.

There are, of course, a few pieces that were so clearly standalone components that we think they’d be great for others to use. In his first week with us Nick Stenning put together unicorn_herder to help us manage the unicorn application server that our ruby applications rely on. It’s one of a few examples you can find in our github account.


  1. Coding needs to be more simple to follow. The only people who consider coding is easy are people who work in the industry itself. Good blog.

  2. We’re building a new dev team here at Registers of Scotland. (Thanks for this resource, which gives us good ideas.) We would like to use, develop – and therefore contribute – open source wherever appropriate. Do government guidelines encourage contributing open source? (I’m assuming so, as you have a github account.)

  3. How are security concerns met where open source software isn’t widely distributed and subject to the ‘thousand eyes’ argument?

    How is provenance and assurance of the software used gleaned?

    1. Hi Kevin – sorry for the exceedingly delayed response. Where software isn’t widely distributed the onus is very much on our development team to do due diligence, reviewing the code, making sure it’s subjected to testing and working out how to keep track of changes. That’s everything from tracking github repositories, to building relationships with the authors, to detailed penetration testing. Depending on how and where we’re using the code that’s an important consideration as we consider the cost of ownership.

      This is also one of the reasons we’re cautious about how we frame the release of our own code. There will be some pieces of code where we can commit to all the support that we would want ourselves, including defined ways of advertising new releases, disclosing vulnerabilities, etc.

    1. MIT felt more appropriate as a license for code as it’s designed for that purpose. It’s also more widely recognised in the software world and so makes it easier for other developers to make decisions about whether or not to use our code.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s