You’d think it would be hard to miss half a million people, but the Office for National Statistics (ONS) managed it nevertheless. Realisation of this problem dawned just over a decade ago, when the results of the 2011 census were published. The census revealed that there were nearly five million foreign nationals in the country — 464,000 more than the ONS thought. Why had nobody noticed?
One school of thought blames a Hungarian entrepreneur called Jozsef Varadi. Varadi did nothing wrong, to be clear, but he did participate in a chain of events that wrongfooted the ONS.
In 2003, Varadi co-founded Wizz Air, a budget airline that followed the well-established model of flying people inexpensively to smaller regional airports.
Not long afterwards, Hungary, the Czech Republic, Poland and seven other countries joined the EU, giving their citizens the right to live and work anywhere in the union. Many of them decided to settle in the UK, and thanks to Wizz Air, they would often arrive at an airport such as Leeds or Luton.
This was good news for anyone looking to hire workers in the UK, but proved the undoing of the International Passenger Survey (IPS), the mainstay of immigration and emigration estimates in the UK for many years. The IPS is a bit like an opinion poll: IPS surveyors politely stop a sample of people in ports and airports and ask them if they’d be willing to answer a few questions. (Remarkably, almost everyone agrees.)
These questions vary from “How much did your plane ticket cost?” to “How long are you planning to stay?” Many of the IPS questions are really about tourism, but the survey generated enough data to estimate migration into and out of the country . . . barely. The problem, explains Georgina Sturge in her excellent book Bad Data, is that while hundreds of thousands of people are interviewed for the IPS, most of them are tourists and only a few thousand are migrants. The number from any particular country will often be tiny.
It is perilous enough to extrapolate from this small sample, but what really confounds any survey is an unnoticed change that flips the sample from being fairly representative of the background population to not representative at all. Wizz Air delivered that unnoticed change. To oversimplify a little, the IPS enumerators were standing at Heathrow, Gatwick and Manchester, while the people looking forward to making a new life in Britain were arriving at Luton.
Pro-Brexit campaigners were quick to highlight the problem, as Sturge explains. Not only could we not control immigration, they said, but we couldn’t even count it. But that’s not quite right. We could have counted it. But we decided not to.
“The choice to use a survey rather than other data is increasingly just that — a choice,” says Anna Powell-Smith, director of the Centre for Public Data, a non-profit campaigning for better data and statistics. There are now other ways to produce migration data, or indeed most of the statistics we see around us in the news or in policy discussions.
One alternative would be to require new arrivals to register, as they do in Germany, before they had access to basics such as a bank account or a doctor. There are pros and cons to this idea, but as Sturge observes, “Germany has a better grasp of its immigration statistics despite having no border control with 25 other European countries.”
The ONS has no power to introduce such a requirement, but in the wake of the pandemic it has moved from estimating migration with the smattered sampling of the IPS to using administrative data that aims to track every immigrant. This includes now-commonplace visas and information from the tax and benefit systems. (Thankfully, there are privacy protections built into the way the ONS uses this information.) The first such estimates were produced in May 2022, and the IPS is now only used to estimate the coming and going of British citizens.
There will be no more Wizz Air-induced statistical errors, says Jen Woolford, ONS’s director of population statistics, adding: “If the exact situation happened today, it would have no impact at all on the accuracy of our figures.” This is good to hear.
The point is not that we should introduce ID cards. It is that both the lawmakers in Westminster and the wonderful nerds in the Government Statistical Service are making choices about what to count, and how to count it. Those choices matter, and they could be different if our priorities were different.
As so often, we ordinary civilians notice statistical and administrative infrastructure in the same situations that we notice the sewers or the electricity supply: when something has gone wrong, or some new challenge is testing the system to destruction. The Wizz Air affair was one prominent example. The scramble to create Covid-19 testing capacity was another. The decision to destroy the arrival records of the “Windrush generation” — on the untested assumption that those records were superfluous or redundant — was a third. (It was a reminder that archivists are as taken for granted as statisticians, perhaps even more so.)
Can we do better? Undoubtedly. Nerdland contains all sorts of ideas, from better estimates of the harms from gambling, to trusted health data research environments that can prevent a privacy apocalypse while saving more lives.
But to unleash those ideas we need to take data seriously. Much of the discourse about data focuses on misleading presentation rather than where the data itself comes from. It is true, of course, that dodgy labels on a graph, or a slogan on a bus, can mislead. But so can statistical work that is underpowered, underfunded and undervalued.