Six Things That Political Pollsters Don’t Want You to Know

Every profession has stuff that it would prefer for people — be it clients, stakeholders or punters — not to know.

Think of the all the hidden cost and profit structures around modern medicine and pharmaceuticals.

Or, you know that you’re only getting glimmers of insight when you listen to ex-captains of the national cricket team doing the TV commentary.

And so it is with the profession that is political polling.

Times, though, have changed due to the debate — which was ironically somewhat inaccurate — about the accuracy of Presidential polling in the 2016 Trump upset win. (Some of the polls — if you knew where and how to look — were in fact pretty close and even more so in 2020.)

What’s good for Yanks isn’t necessarily good for Aussies (says this dual citizen), but it’s probably useful for discourse in Australia for there to be more light shed on polling. And with today’s Newspoll numbers showing the LNP somehow “down” and the ALP “somehow” up, it’s also a timely reminder to not get too excited about what you’re seeing.

My friend, Jim Reed, who has been part of dozens of winning election campaigns in different parts of the world, did an excellent piece on LinkedIn in 2019 on related themes and I’ll never be smarter than him.

But, I’ve had some unique opportunity to look at and use political party polling, and to be part of organisations that produce it. In terms of stuff you might not readily know from the public record, that’s probably the best starting point — and a few more trade not-so-secrets.

Party polling is different from what you see in the papers — and for clear reasons.

In polling as in all other things, you get what you pay for. And you pay for what you ask for. It’s vital to understand that the briefs that party pollsters work to are entirely different from the briefs for polling done for public consumption — even by the same polling businesses. Based in part on budget, private and public briefs differ in a whole bunch of ‘i’s’: intensity (how often, how many people and how local), intention (objectives), and insights and interpretation.

They are both fruit — but as different as apples and tomatoes. For that reason, serious political operators — and that excludes most backbenchers — only note public polling as part of the theatrics that need to be managed, rather than any substantive indication of one’s position or prospects. So, don’t be a mug and tell your partner what the trend is in Newspoll or whatever. You’ll be talking out of your arse and be less attractive to him or her.

When you see a poll, ask yourself who wants you to see it.

If a public poll (or an allegedly leaked private poll) is in the media, it’s because somebody is trying to score a political point or drive home some advantage. Again, what goes public is conducted according to different briefs to achieve different outcomes. For example, if you want to bury some opponent, you only find out their popularity or favourability (which almost always sucks for all politicians as a category) rather than the qualities that the public may respect about that politician.

Alternatively, we sometimes see media organisations commission polls of a sort. Putting aside the ridiculously flawed methodologies of most of such exercises (read: the media company doesn’t want to spend any money and it’s all usually done on the super cheap), this is simply frolic, froth and fodder. It’s main intent — call me a cynic — is to produce some nice graphs on the front page and drive sales. And, given their national or state-wide scope, the reported figures are as reliable as “reading the route numbers off passing buses” as a great colleague once said.

Just because you know the sample size and the margin of error doesn’t mean you know much.

Many people interested in politics have done some stats course during their undergraduate degree and can reasonably comment on whether a certain number in a sample etc is reasonable. But the truth is — for statistically validity — that pumping up overall sample sizes doesn’t really make much difference (except for the polling business’ P&L).

Pollsters know — and journalists don’t seem to care — that what’s more crucial for accurate results and useful insights has more to do with things that are not publicly seen, such as: quotas for different demographic groups within an overall sample; survey question design and length; overall research methodology (including the use of qualitative methods); technology selection, and; the quality and training of the call centre operators.

Every human system — including polling — is subject to unpredictable human emotions and choices.

While polls deal in numbers, issues, and ‘hard data’, the best pollsters know that what’s most important about voter attitudes and choices is actually emotions and values — which again don’t feature in what’s publicly available. It’s a whole other science and art to understand what a voting public is optimistic or anxious about and how that relates to given policies, positions and people.

And it’s Dr Spock 4-level chess stuff to translate that science and art from the national level to the local level of Nagambie, Nambour and Nambucca Heads where local factors can be just as influential. This is especially so due to the modern tendency of voters to see the bigger picture as more problematic than one’s own community.

Moreover, just because the community is nominating say “national security” or “the economy” or “cost of living” or “COVID” as their “most important” issue doesn’t mean that has a direct correlation on their vote, and especially changing it. Indeed, the notion that a single issue determines an election — including the WA wipeout — is fanciful.

This is even more true when political sophistication has reached the point that some participants ‘game’ polls with fake answers, especially when they know the poll will be public.

Powerful polling isn’t about where you’re at — it’s about where you want to go and how to get there.

Public polling is like a screen shot from Google Street View. It shows you where you are at and usually the pictures too old anyway. Private polling done for parties and other interested players is a GPS. It’s there to show you where to go, what the signposts will be, the distance you’ll be travelling and how long to arrive there. Who care if you’re on 30% primary vote right now when you know you need to be 35% and want to see what key turns to make to get there.

Polling is only as good as the people reading the poll.

A dumb ass reading a brilliant poll will come up with dumb ass conclusions. Conversely, the people who consistently win are the diviners of truth who see what’s important in between all the numbers and methodological considerations. And the truth of the matter, those unique and extremely rare operators are known to spend as much time looking at people at the Westfields food court and to read in a range of disciplines as they are to be ‘behind the glass’ at focus groups. They’re great at observing, listening and channelling sentiment — not just using Excel and doing arithmetic.