Further adventures in treemapping the UK's migrant population

9 Apr 2016 14:05 GMT

Earlier this week I posted a treemap showing the UK's migrant population by country of birth. A common reaction among people who saw it was to wonder what the size of the UK's foreign-born population was relative to the size of the UK-born population.

To help put that in context I have produced a new nested treemap showing the population of the UK broken down by region and broad country of birth. The population in each region is grouped into those born in the UK, those born in other EU countries, and those born in countries outside the EU.

While it's interesting to see the data visualised in this way, the advantages of using a treemap rather than a traditional bar chart are much less obvious in this case. When showing the migrant population by individual country of birth, a treemap lets you compare data for a very large number of countries in a way that is much easier to gloss than a bar chart. It allows you to group countries into common geographical regions, which represent the group's aggregate size. And the arrangement of countries from largest to smallest in each group provides a good visual representation of the distribution of the population within the group.

In this case, the UK's migrant population is too small as a proportion of the total population to break down into individual countries of birth, or anything more than two or three groups. Arguably the most interesting thing about the visualisation is how small the migrant population appears relative to the size of the UK-born population in every region outside London. On the other hand, a treemap makes it harder to make exact comparisons between the size of the migrant population in each region.

In short, I don't think this treemap is as effective as the last one, but that is probably because it is less well suited to the data being presented. But the new version of D3 made it just as easy to produce this treemap as the last one, and I thought it was worth sharing for those that asked to see it.

Treemapping the UK's migrant population

5 Apr 2016 08:04 GMT

It's been exciting seeing version 4.0 of D3 develop, and last week Mike Bostock announced that d3-hierarchy was now included in the alpha. This module provides various ways of visualising hierarchical data, including treemaps.

I have wanted to explore using treemaps for some time, as much of the data I work with could potentially be presented in this way, so this seemed like an ideal opportunity to start experimenting.

My first attempt is a treemap showing the foreign-born population of the UK broken down by country of birth. The countries are grouped into broad global regions, using a particular arrangement of the new country groupings the Office for National Statistics has recently introduced in reporting migration statistics. Within the group for the European Union, the EU14, EU8, and EU2 are grouped separately.

The figures are taken from the most recent quarterly Labour Force Survey, which is for the fourth quarter of 2015. They are estimates of all people born abroad who were living in the UK at the time of the survey, excluding two small groups: those born in British overseas territories, and those who did not fully specify their country of birth. (The latter group consists of people who, for example, said they were born in the USSR but did not say which current country that would be.)

The treemap itself is relatively simple and leans heavily on Mike Bostock's example code, with just a few presentational tweaks. But it's the first time I have seen this data (which is very familiar to me) laid out in this way. It was extremely simple to get this working and I am looking forward to delving deeper into d3-hierarchy to explore what else is possible.

Update on Population Builder

11 Feb 2016 20:32 GMT

I haven't had enough time to post about work on Population Builder in recent months, but there have been a few developments that I think are worth sharing.

New features

I had a few requests from regular users for features that seemed worthwhile and relatively easy to implement. By far the most requested feature was a button to deselect all the currently selected areas with a single click, without having to reload the page. This has now been added. Clicking or tapping “Clear Map” does what you would expect.

More interestingly, a couple of people have told me they are using the app as a general purpose tool for selecting LSOAs and data zones, in order to identify the area codes they need for geographical analysis of other data. To help with this, I have added a feature that allows you to inspect the geographical code of each area.

On a computer with a mouse pointer, hovering over an area highlights its boundary and displays its area code in the overlay control in the bottom right-hand corner.

A screenshot from the application showing a highlighted boundary.

You can highlight an area on a touchscreen device by long-pressing it, which also displays the area code in the overlay control. Another long-press on the same area dismisses the highlight, while long-pressing another area highlights the new area instead. This makes it possible to visually match an area to its code.

New data

The app has been updated to use the latest mid-year population estimates for small areas, which are for mid-2014. I updated the stats in November shortly after the ONS published the latest figures.

Updating the figures for Scotland also meant updating the maps to use 2011-based data zones. So if the boundaries in Scotland seem different, that's because they are.

Open source

The complete source code for Population Builder is now available on GitHub. Of all the things covered here, this was the most work, as I wanted to port the server-side code from PHP to Go before sharing it online. This is partly because Go makes it much easier to download and run the software locally as a standalone application, but also because I've wanted to move to a more modern web-development stack for a few years now and this seemed like a good place to start. Now it's done I can concentrate on new things.

An alternative file server for Go

28 Dec 2015 16:19 GMT

Go's net/http package contains a simple FileServer that lets you map a request path to a directory and serves files under that path like a normal webserver. It's a handy way to serve static files from inside an otherwise dynamic web application.

Go's built-in FileServer is fine if you're just using it for development, but it has two shortcomings if you want to use it in production. First, when it can't find a requested file it responds with a very basic plain text 404 page. Second, when the user requests a path to a directory which does not contain an index page, it shows the directory listing, with no easy way to turn that off.

The FileHandler in the handlers package is an alternative file server that addresses both of these problems. FileHandler lets you specify your own handler for serving 404 pages, and it responds to any request for a directory that does not contain an index page with that handler, rather than show a directory listing. FileHandler is just a wrapper around Go's ServeFile function, so it still has all the goodness of Go's built-in file handling.

The handlers package also contains a simple NotFoundHandler which you can use with FileHandler. But you don't have to. FileHandler can use any type that satisfies the Handler interface to serve custom 404 pages when no matching file is found.


16 Nov 2015 22:51 GMT

Decimals is a small library of functions for rounding and formatting base ten numbers in Go. Most of the software I write involves presenting numerical data, and Go's standard library lacks convenient methods for displaying numbers in a way humans find easy to read.

Decimals uses simple and consistent rules for rounding and formatting numbers across all its functions. There are other formatting libraries that offer a wider range of output formats, but decimals aims to make it as easy as possible to deal flexibly and extensibly with the most common case.

See the read me for a full write-up or jump straight into the GoDoc.