olihawkins

A bestiary of undead statistics

30 Oct 2018 19:47 GMT

For some time now, statisticians and fact-checkers have talked about zombie statistics: false statistical claims which have found their way into public debate that are repeated endlessly and uncritically. They are called zombies because no matter how many times you beat them to death with evidence, they keep coming back to life.

A few months ago I described a particular false claim as a vampire statistic, because it survives in the shadows of semi-public discourse and avoids the daylight of scrutiny that could kill it.

I've been wondering what other kinds of undead statistics there are, and seeing as it's Halloween I've had a go at a typology. This is a little schematic, so I'd welcome any suggestions to flesh it out.

  • Zombie statistic — The classic undead statistic; it survives all attempts to destroy it with facts and keeps on claiming victims.
  • Vampire statistic — A statistic that never dies because it is never exposed to daylight. It is too wrong to appear in the usual arenas of public debate, so it keeps circulating through viral channels. Vampire statistics survive in the dark corners of the internet where paranoia and conspiracy theories flourish.
  • Phantom statistic — A statistical claim with no apparent source. It is either asserted without evidence or attributed to a source that does not contain the statistic.
  • Skeleton statistic — The bare bones of a statistical claim that has been removed from the body of knowledge that gives it life and meaning. This kind of statistic is often true in a narrow or technical sense, but is untrue in the way it is presented without context.1
  • Frankenstein statistic — a false statistical claim produced by stitching together statistics from different sources that shouldn't be combined.2
  • Werewolf statistic — A statistical howler that comes up with predictable regularity at certain events or times of year.3
  • Mummy statistic — A statistic that was once true but is no longer true. It has somehow been embalmed in the public imagination and keeps coming back to life when it should have died a long time ago.4

Happy Halloween!

Footnotes

1. An example of a skeleton statistic is the claim that more than 90% of communication is non-verbal. Albert Mehrabian's finding was that, in certain experimental settings, more than 90% of the content of communications about feelings and attitudes was non-verbal.

2. An example I can recall is this article, which compared the number of EU migrants living in the UK with the number of British migrants living in the EU using datasets that had different definitions of a migrant. To the FT's credit they quickly corrected the story (because they have a brilliant data team).

3. See Blue Monday, for example. See also the recurring confusion between the net change in the number of people in work and the number of new jobs that accompanies ONS's regular labour market statistics.

4. I once accidentally created a mummy statistic. In 2014, I tweeted that the Telegraph was wrong to say net migration was above 250,000. The tweet was picked up by a fact-checking bot, which intermittently cited me saying this over the next two years, as net migration rose to around 330,000.