Why Polls Are Bad For Us
By Nick Cohen
The Observer - UK
A few weeks ago furious protesters pounded on the windows of the head office of the Minneapolis Star and Tribune, a respectable newspaper in the northern US state of Minnesota, whose journalists aren't used to being the targets of rage and derision. Rob Daves, the paper's pollster, saw the crowd gathering on the front steps as he came out for his lunchtime jog. He didn't realise why they were there and went off on his run. By the time he got back, they were gone, which was just as well because much of the rage and derision was directed at him.
'Hey, hey, ho, ho, Rob Daves has got to go!' the Republican pickets chanted. 'Liberals!' they screamed at hacks leaving for a bite to eat. It may not sound much of an insult to you, but in conservative America alleging that journalists are liberals is almost as wounding as alleging that they are communists, which the demonstrators proceeded to do when they switched to the chant of 'Star and Sickle, you're in a pickle'.
The Star and Tribune had a run a poll saying that John Kerry had a 9 per cent lead over George W Bush in Minnesota. Rival polls claimed the race was tied. The paper's editor had endorsed Kerry, and the Republicans believed they had uncovered a liberal plot to convince their supporters that there was no point turning out to vote.
Over in New York,, an alliance of two million Bush loathers who, like the Minnesota Republicans, are unlikely to get through the tension of election night without a nervous collapse, paid for an attack ad in the New York Times. 'Why Does America's Top Pollster Keep Getting it Wrong?' it wanted to know. Gallup's polls, used by CNN and USA Today, had given Bush a 14 point lead but other polls conducted at the same time put it at around 3 per cent. The Kerryites noted that George Gallup Jr, son of the poll's founder and director of its research centre, was a born-again Christian of the nosiest type who had said earlier this year that 'the most profound purpose of polls is to see how people are responding to God'.
'We thought the purpose is to faithfully and factually report public opinion,' retorted with a sniff.
You get the picture. Religious fundamentalists in the polling industry back God and Bush and do nothing to dispel the helpful illusion that the President is invincible.
The polling companies dismissed the protests as overheated allegations from an angry campaign. But the demonstrators were on to something. How can one poll give Kerry a nine point lead in Minnesota and others give him no lead at all? Why do CNN viewers hear that Bush is cruising to a landslide victory, while he's in the tightest race imaginable on other channels? Why for that matter has Labour's poll share in the past month bounced between 28 per cent and 39 per cent according to which newspaper your read? Are any of these figures true or false? And if newspapers and broadcasters can't say they're true, why do they pretend that they are facts on a par with the statement 'Yasser Arafat was admitted to hospital last week'?
The pollsters can't all be right, but they may all be wrong, and a few American journalists have the guts to admit it.
Sceptics in the media have always seen opinion polls as crooked: not as deceitful as horoscopes, but way short of the exacting standards met by the racing tipsters. Where the Minnesota Republicans and the New York liberals get it wrong is to assume the crookedness is the result of political bias. To be sure, pollsters are a part of the process which sets 'the agenda' and determines which political positions are 'appropriate' and which politicians are 'credible', but most are honest by their own lights. It's just that their lights are dimming with each passing year.
Last week the Washington Post 's pollster Richard Morin wrote a long, despairing piece whose headline 'Don't Ask Me' said it all. In Britain, the campaign polls may have underestimated Tory support in the last three elections, but the exit polls, which have far more money behind them, were reasonably close in 1997 and 2001. Morin noted that in the US, even the exit polls were a shambles.
The reason why pollsters fear their racket is 'on the way to extinction' is that the public is fed up with being pestered by hucksters. If you think about it, talking to a polling company is an odd way to behave. Strangers ask you to give them time and personal information for nothing so that they can profit from it. Once the pollsters had few competitors, now everyone's at it. In the vast United States the only way to sample the electorate is by phone.
But the polling companies aren't the only people cold calling at the precise moment when the dinner has boiled over and the baby's nappy exploded. Telemarketers infest the country. To combat them, there are call screening systems and attempts in state legislatures to make cold calling illegal. Those who can't stop the pollsters getting through are fighting back with ingenious technologies.
The Washington Post told the sad story of Rick Klaastad, a market researcher for a polling company in Oregon, who dialled an elderly man on his contact list. 'Apparently he has some electronic device attached to the phone and the next thing I hear was this loud, painful sound. It was as about as bad as it gets - imagine turning an FM stereo up full blast with a really good set of speakers. It bruised my eardrum.'
Huge numbers refuse to talk to pollsters. Morin found that 'in some surveys fewer than one in five calls produces a completed interview - raising doubts whether such polls accurately reflect the views of the public or merely report the opinions of stay-at-home Americans who are too bored, too infirm or too lonely to hang up.'
To put it at its politest, I think it's fair to say that American polls aren't want scientists call random samples.
Nor are they in Britain. Whenever the media is forced to ask whether findings that 57.2 per cent of the population don't trust Tony Blair or 12.8 per cent of twentysomething women have given up one-night stands after a three-in-a-bed romp with a two-timing love rat are true, it always quotes pollsters. What journalists neglect to mention is that the polling industry makes its real money from business, not newspapers or television. If CEOs began to suspect that the beguilingly precise findings on the existence of niche markets were false, tens of millions of pounds would be lost.
If you leave the self-interested defenders of the industry behind and turn to objective outsiders such as Roger Jowell, who runs the British Social Attitudes Survey, you find that they have been saying for years that opinion polls aren't random samples.
He explains the persistent failure to get Tory support right in campaign polls by the fact that a sizeable number of Conservatives think that time is money and don't see why they should give something for nothing. They brush past the market researcher standing in the street with a tight smile. Labour supporters and Liberal Democrats are kinder people who are more likely to help out a stranger.
Whether they will continue to do so is doubtful. Commerce is crowding into private space via junk mail, cold calls, texted adverts and the street confrontations arranged by those lovable charities. Cold calling has prejudiced Americans against telephone polling. Chugging and other forms of aggressive begging are likely to prejudice a fair chunk of the British against the poor woman in the shopping centre trying to scratch a living wage from Mori.
A lot flows from the decline of polling. First, readers who are close to tears because most US polls give Bush a slight lead can relax until election night. Only two of the 11 final national polls were right in 2000 - and there's no reason to believe that that dismal record will be bettered. (Conversely, Bush supporters - a minority among you, I suspect - shouldn't be too cocksure, defeat or a landslide is as likely as a narrow win.)
Second, all the focus-group policies the government uses to please the electorate may not only be wrong in principle but a failure in practice because the public may very well not be pleased in the least.
Finally, if the next election is as messy as a few of us suspect, with three-cornered contests taking place across the country and voters deciding whether to vote tactically against Blair or the Tories, the polls will be all important to people who want to work out what the consequences of their vote will be. They should know in advance that there's a fair chance that the polls will be tosh.
Guardian Unlimited © Guardian Newspapers Limited 2004



This Site Served by TheHostPros