olihawkins

The FT makes a statistical mistake

16 Feb 2014 10:32 GMT

Update, 28 Feb 2014: Since publishing this post, and thanks to an intervention by the FT's stats and data team, the paper has now corrected the error discussed below.

Last week, the FT published an article suggesting that the number of British nationals living in other EU countries was roughly equal to the number of other EU nationals living in the UK. The story was based on two sets of figures:

1. Estimates of the number of British nationals living in other EU countries published by the Institute for Public Policy Research (IPPR) in 2010, which were recently cited in a response to a Parliamentary Question. These figures suggest that there are around 2.2 million British nationals living in other EU countries in total.

2. Office for National Statistics (ONS) estimates of the UK population by nationality. These figures show that there are around 2.34 million nationals of other EU countries living in the UK.1

However, these two figures are not comparable because the IPPR figure includes people living abroad for only a part of the year, while the ONS population estimates do not include such people, and only count people usually resident in the country for 12 months or more.

This makes a significant difference to the comparison, as there are 410,000 British nationals included in the IPPR figure that are only living abroad for a part of the year. When these people are removed and the figures are compared on the same basis, the IPPR figure is more than half a million (555,000) or 24% smaller than the ONS figure.2 The two figures are therefore not equal and the premise of the story is false.

I tried contacting the FT both through Twitter and by sending an email to their corrections department. The paper has said it is not going to correct the story, on the grounds that it was simply reporting two official figures in good faith.

This sidesteps the question of whether the ONS figure really does exclude people living in the UK for only a part of the year (it does — see page 4 of this, or page 1 of this, or section 1 of this for example); and it overlooks the fact that, while the Goverment does seem to have signed up to the IPPR estimates for now, it never said they were comparable with ONS data. The FT said that.

This is a big disappointment to say the least. Partly because it is unusual for the FT to overlook such a mistake once it has been pointed out. But more because of the paper's lack of intellectual curiosity on this occassion. Because here's the thing: no-one knows with any certainty how many British nationals are currently living in the EU.

The European Union's official statistics agency Eurostat publishes estimates of the number of British nationals living in each EU country; and the sum of the most recent available estimates suggests that there are around 870,000 British nationals living in other EU countries. Data is missing for some countries, and data for other countries is old, so 870,000 is probably an underestimate; but we don't know by how much.

Because of gaps like this in the data, when the IPPR did their study — which estimated the number of British nationals living abroad in 2008 — they applied various upward adjustments to the official figures to compensate for the possibility of undercounting. But we have no way of testing the validity of these adjustments, because we would need to know the actual number of British nationals living in each country to see if they were justified, and if we knew that we wouldn't have to guess. We simply don't know how accurate the IPPR's adjustments are because (unlike the ONS figures) their figures are not the product of a statistical method that allows you to quantify the degree of uncertainty surrounding each estimate. There are no confidence intervals here.

It would have been great to see the FT explaining all of this, and setting out the considerable uncertainty surrounding this known unknown; instead of pretending to know something it doesn't.

Footnotes

1. They're in Table 2.1 of the reference tables if you can't find them.

2. If you want to see the actual numbers involved in this calculation you can download a spreadsheet of the relevant IPPR figures here. They are taken from Appendix C of the IPPR Report.