Statistician and political forecaster Nate Silver won renown for the accuracy with which he predicted the 2008 US election results, a feat he repeated spectacularly in 2012. His book The Signal and the Noise came out in September that year; his celebrated FiveThirtyEight blog recently moved to ESPN. Ben Page is chief executive of Ipsos MORI, where he has carried out hundreds of studies of political and consumer behaviour; the Guardian called him one of the 100 most influential people in the public sector. They met in Tank’s offices to talk about the fetish for “big data”, how people (mis)understand probabilities and why you can’t make a decent living from poker anymore.
BEN PAGE Survey research is mostly about asking people questions, adding the answers up and then trying to interpret what they say. Increasingly, it’s moving into passive measurement as well. And obviously you are the aggregator-in-chief. What do you find most interesting these days? Are you going to try and forecast the US jobs market?
NATE SILVER Probably not! The evidence suggests that economic forecasts are not very good, and have not been getting any better.
BP The meteorologists claim to be better than the economists…
NS They are; the meteorologists are kind of the heroes of the book. There are a few things that help. One is that we have a good physical understanding of the dynamics of the weather system, so they’re actually building a simulation of the atmosphere – it’s not really statistical. It’s been a success story: hurricanes now are forecast 250 per cent more accurately than 25 years ago. Hurricane Sandy was predicted almost exactly five days in advance. The sports gamblers are also very good. Part of it is that if you’re putting money on the line it has a disciplining effect. A lot of people in American politics in particular are just pontificating – kind of like in the essay “On Bullshit”? Where it’s not that you’re lying, so much as you’re entirely indifferent toward whether you’re accurate or not.
BP I read you as somebody who’s big on Bayesian statistics. Tell us about Mr Bayes.
NS So Thomas Bayes was an English reverend and kind of a pioneer in the theory of statistics, and – this was 250 years ago – his basic insight is something called Bayes’ theorem, which is a simple algebraic equation. It says, first of all, you have a probabilistic view on the world – and, by the way, it’s not a metaphysical statement, it’s not saying the world is uncertain, it’s just saying our knowledge of the world is uncertain. The other thing is that it assumes you start out with a prior belief – a presumption or a bias – and you weigh new evidence against that.
BP I’m interested in just how good both ordinary people and decision-makers in business and in government are at dealing with probabilities.
NS Yeah, people don’t have an intuitive grasp of that at all. They think if you say something is 80 per cent likely to occur, it’s tantamount to saying it’s guaranteed to occur, when obviously it’s not. If you woke up every day and you had an 80 per cent probability of not being stabbed, you wouldn’t survive very long. Those 20 per cent outcomes occur quite a bit. If you follow sports as I do, or especially playing poker, where you’re on the losing side of that 20 per cent enough, your intuition for probability is better. There were a couple of years where it was my main source of income, and...
BP You’ve stopped this now? You’ve found the writing is better?
NS Well, the poker’s worse! In 2003 you had a kind of fat accountant slob named Chris Moneymaker, this everyman character – nice guy – who won the World Series of Poker in Las Vegas. He’s an OK player, but basically got very lucky. And ESPN would edit out all the hands that he would lose or play badly. That deceived people as to how easy it was, so for a while there was a lot of dumb money in the poker economy – but now it’s gotten very, very competitive.
BP People talk a lot about big data. To me, some of that is like – there was a great quote by a guy from Google about social media: “Social media is like teen sex: lots of people talk about it, lots of people look forward to it but when it’s finally achieved, most people are disappointed.”
NS If you read the Harvard Business Review, every other ad is about big data. They’re pitching this as this magic serum: that’s always dangerous. The problem with some of this data is that it’s big, but it’s not structured. It’s about understanding, you know, when are you really in a data-rich environment, ’cause it’s not just the number of observations you have. The credit-rating agencies in advance of the crunch had millions of observations on individual mortgages, but all from a period when housing prices were increasing. Part of the issue with social-media metrics, for example, is you’re collecting data that’s only a couple of years old. Google had a product, Flu Trends, which is meant to look in real time…
BP This is people typing in “flu” and they can predict actual influenza cases.
NS They say they can, but they considerably overestimated the flu in the United States last year. You build a model that accurately describes what search terms people were using in, say, 2008 to 2010, but people change their search habits. Now that you have Google auto-complete, that will suggest different queries to people, and when people are typing on mobile phones you tend to have shorter queries, so that will change things. Three years is a lifetime in social media.
BP We know people aren’t rational: Nudge is very popular here, as it is in US policy circles, but what’s your advice to the average human being? Most people’s financial planning, for example, seems to involve them dying at about 70, when actually they should know it’s very unlikely to happen.
NS People apply a lot of attention and a lot of bandwidth to where they’re going to go for dinner – and I’m a foodie, so I don’t begrudge that entirely – but those heuristics don’t work as well for things like career or educational planning, where you have to be more detached. One thing we’re seeing now is that once you’re unemployed for more than about six months it becomes very difficult to find a job again. Employers begin to assume: “A lot of other employers have passed on this person, there must be something wrong with them.”
BP So the advice is if you become unemployed, take any job, don’t wait for the right job? OK. Any other examples for people?
NS There was a column in the Sun the other day – slightly embarrassing! But there’s a dating website called OKCupid that analyses a lot of data, and a couple of years ago we looked at which is the best night of the week to go out to meet someone. They looked at what percentage of people had updated their status to say, “I wanna get laid”, basically, and the percentage peaks on Wednesday night. We think the issue is that early in the week, people go out to drink; then Friday, Saturday everyone goes out with their friends, it’s kind of amateur hour. But Wednesday and Thursday… The headline of the Sun article was “Maths Can Be Fun!”
BP Very good! More seriously, in terms of governments’ use of this, in Britain – censuses are expensive, people are more mobile, there’s higher refusal rates – one argument is that you could just move to using a whole load of databases of credit data, crime, all sorts of other data, link it all together and you’d build a pretty good census of the population. What’s your view?
NS I think you’ll move toward kind of a dual model where you have big, expensive telephone polls occasionally and then polls that are done online at a higher frequency. In theory now, 90 per cent or something of people access the internet regularly, whereas in the US, only 65 per cent have a landline. So you’re actually missing a much larger universe without that.
BP And what criticisms would you make of pollsters, just out of interest, if any?
NS There’s this idea of having Democratic pollsters and Republican pollsters, which is inherently very strange…
BP I went to a global meeting on immigration, and people were saying: “Yeah, I’m a pollster and I do polls in favour of immigration.” How does that work? There was a great piece – I think it was in an Ohio newspaper – “The Death of Facts”. It said: Facts has been ill for some time, but he finally succumbed during the 2012 election campaign: poor old Facts has gone! Does it feel like that to you?
NS That is an American thing. I think Republicans have – it’s not a very conservative attitude in fact, it’s quite radical and avant-garde to say that we can manipulate reality in the way that we want it, and there’s no absolute truth.
BP Well, in the UK, pollsters – certainly the reputable ones – will be trying absolutely not to ask biased questions. The industry has a pretty good record of transparency and self-regulation.
NS I think that’s true in general. In some ways, these are two very opposite functions: opinion measurement vs opinion shaping, or manipulation. §