| |
9 December 2007
Now that the election is done and dusted, it’s possible to look back and compare the actual results with the voting
intentions reported by opinion polls during the course of the year, and with the predictions made by commentators
on the basis of those opinion polls. In summary, the verdict is that the level of Labor support on polling day was
lower than reported by virtually every opinion poll during 2007, and that consequently the great majority of predictions made by commentators of the
election outcome, based on those polls, were wrong.
The two-party vote for the ALP on 24 November was 52.9%. Only two polls in the whole of 2007 put the ALP vote this
low: the final Galaxy and the final Newspoll, which both came out in the last days of the campaign and put the ALP
vote at 52%. Every other poll showed the ALP vote at 53% or more, usually much more. All through July, August,
September and October most polls put the ALP vote in a band between 54% and 58%. Only towards the end of the campaign itself did the polls begin to move back towards the Coalition. As well as the
Galaxy and Newspoll, there was a final Morgan poll putting the ALP at 53.5%, the lowest Labor vote Morgan reported
all year. But some persisted in giving high figures for Labor. The final ACNielsen poll put Labor’s vote at 57%.
The obvious question is: were the polls wrong all along, or was there a genuine shift back to the Coalition in the
last week of the campaign? The short answer is that we will never know, because there is no way of checking in
December whether an opinion poll taken in August was correct or not. The Newspoll at the beginning of September
which had the ALP on 59% is not retrospectively proved wrong by the actual ALP vote of 52.9%.
My view is that the polls were broadly accurate through the course of the year. There was a surge of support for
the ALP when Kevin Rudd was elected leader in December 2006,
and this was sustained through the year, with a
gradual drift downwards from March to July, and then a plateauing from July to November. The much-derided
“narrowing” did indeed come, but only in the last week, when some of the “soft” ALP vote (another much-derided
expression) went back to the Coalition.
Opinion polls are not meant to be predictive. Although Newspoll reported in September that 59% of voters had told
them they would vote Labor, Newspoll itself did not make a prediction about what would actually happen. That role
is left to the election commentators, both professional (in the mainstream media) and amateur (on election blogs).
Most commentators correctly predicted that the ALP would win, despite the best efforts of News Ltd journalists to
talk up the Coalition’s chances. But the great majority over-estimated the size of Labor’s win, both in terms of
votes and seats.
We can see this in the predictions made by regular participants at the psephological weblog
Pollbludger. These
predictions were collected on 28 October. Only ten Pollbludger
commentators thought (or claimed to think) that the
Coalition would win. Another 25 (including me) thought that Labor would win narrowly, that is with 80 seats or
fewer. Forty Pollbludgers got the result approximately right, predicting between 81 and 86 Labor seats. But a
massive 73 Pollbludgers tipped a Labor landslide, with Labor winning more than 87 seats. Thus of 148 Pollbludgers
making a prediction, only 23% underestimated the scale of Labor’s victory, while another 27% got it approximately
right. Fully half the Pollbludgers overestimated Labor’s victory, some wildly so – 17 predicting that Labor would
win 100 seats or more.
In drawing attention to the incorrect predictions of others, I do not claim omniscience for myself. I made plenty
of incorrect predictions during this election, as I have done during others. But the record does show that on 19
October I predicted
that the ALP would win 80 seats, which required a uniform swing of 5.4%. The actual result was
84 seats (assuming Labor’s 7-vote win in
McEwen holds up) and
a swing of 5.6%.
I slightly underestimated the swing
and the seat tally because I didn’t pick up the big swing in regional Queensland, which brought
Dawson,
Leichhardt
and
Flynn to the ALP. But
it wasn’t a bad pick, and it was a lot better than most others.
All these polls can’t be wrong
Looking back over the year’s commentary, it seems to me that there were two reasons why most commentators over-
stated the likely scale of the predicted ALP victory. The first was the tendency to treat opinions polls as infallible and as predictors, rather than as what they are:
very imprecise measures of voter sentiment on a given date, with little or no predictive value. The second was to
assume that the electorate as a whole shared the same scale of values as the commentators themselves, and therefore
thought and felt about issues in the same way.
Now of course, everybody with any knowledge of elections and opinion polls knows that polls are not predictive.
Everybody knows that polls only measure voter sentiment on a given day, and even that not very precisely (otherwise
all the polls would agree, which they don’t).
But a long string of polls all showing the same thing does have a seductive effect, particularly when they accord
with one’s own desires. (And let’s not try to deny that the great majority of election commentators, including me,
wanted Labor to win this election.) Surely, most of us felt, and some of us said, all these polls can’t be wrong.
Surely such a steady and commanding lead won’t evaporate at the last minute. Surely Labor is headed for a massive
win.
The nearer we got to the election, the more alluring the message of the polls became. As the long-awaited Narrowing
failed to happen, commentators became increasingly convinced that it couldn’t and wouldn’t happen. This was partly
a case of the wish being father to the belief, and partly a reaction against the blatantly partisan commentary of
the Murdoch press, which insisted all year that the Narrowing was at hand. If Dennis Shanahan
said
there would be a Narrowing, this was surely good evidence that there would not be.
Just as important as the over-reliance on opinion polls to predict the result was the innate cultural bias of the
commentariat. Election commentators are overwhelmingly university-educated professionals living in the inner
cities. They belong to what I called in an earlier article the “the inner urban cosmopolitan elite.” They share
the common assumptions of this elite, including of course its antipathy to John Howard.
This is not to say that most commentators are overtly or deliberately partisan. No doubt they try not to be. But
their political commentary is inevitably influenced by the worldview of the class to which they belong, and not
all of them are sufficiently aware of this to be able to counter it when making political predictions. The
predictions made by participants at Pollbludger show this clearly. These are for the most part very intelligent
and well-informed people. Their overestimation of Labor’s likely performance did not result from stupidity or
ignorance. It arose from a collective misunderstanding of what the electorate was thinking and about what issues
were influencing the voters.
(Disclosure: I am myself a university-educated professional living in an inner city. How then have I been
inoculated against the biases I criticise my fellow commentators for? I believe the answer, paradoxically, is
my membership of the ALP, and more particularly the ALP Right.
Since my political affiliation is formal and widely
known, I have to guard especially carefully against partisanship. And since I support a faction dominated by
hard-boiled pragmatists and generally hostile to the views of the inner urban cosmopolitan elite, I am better
equipped to avoid the trap of elite group-think into which many others have fallen.)
The case of the doctors’ wives
The problem of the commentariat is not just one of simple partisanship. It is one of shared assumptions
about what is important and about what influences the behaviour of the electorate. We saw this very strikingly
during election year with the case of
Dr Haneef. Virtually all the commentators, both
professional and amateur, even in the Murdoch press, assumed that the government’s bungling of the case was yet
another nail in the Coalition’s electoral coffin. I disagreed at the time, and I still do. My view was and is that
the crucial section of the electorate (the floating voters) did not see this as a case of unjust treatment of an
individual. They saw it as tough action against a suspected Islamist terrorist, and approved of it. I said at the
time that Haneef was a net plus for the Coalition, and that is still my view.
For the inner urban cosmopolitan elite, the most important election issue by a long way was
climate change,
followed by the Iraq War (as a symbol of Australia’s
subservience to the loathed George Bush), opposition to
nuclear power, asylum seekers and Indigenous affairs. I make no judgement about the intrinsic merits of any of
these issues. The question is whether these issues were in fact the decisive ones in the electorate at large.
This is linked to the question of the so-called
“doctors’ wives.”. Throughout election year, there was a widely held
view that the Coalition was fighting on two fronts: that it was losing support among lower-income voters (the
“Howard battlers”) because of the unpopularity of the
WorkChoices legislation, while at the same time losing the
loyalty of its upper-income voters over the social and foreign policy issues mentioned in the previous paragraph.
These voters were dubbed the “doctors’ wives” by some commentator or other, and this became the accepted code for
“upper-income voters shifting their vote because of post-material issues.” Others began to call them “the Rudd
liberals,” as a counterpoint to the “Howard battlers.”
It was on the basis of this assumption – that all well-educated and right-thinking people must share the views of
the inner urban cosmopolitan elite on issues such as climate change, and would cast their votes accordingly – that
some commentators began to predict that the election would see a dramatic shift in the loyalties of traditional
urban Liberal strongholds such as
North Sydney,
Wentworth and
Warringah (NSW),
Kooyong,
Higgins and
Goldstein
(Vic),
Ryan (Qld) and
Boothby and
Sturt (SA).
There was certainly some evidence, in the form of opinion polls, to support these predictions. Published polls did
indeed show close results in Wentworth, North Sydney, Ryan and Sturt. There were reports that private polls
commissioned by the parties showed Coalition candidates in these seats to be in trouble. I certainly did not
discount this evidence at the time. I coined the expression “the
Bangkok dilemma” as a rather
crude allusion to the belief that the Coalition was being “screwed at both ends.” I wrote an
article about the contest for
North Sydney – a seat Labor has never won in 106 years – based on the assumption that the Liberals were under real threat
there.
I can, however, claim that I remained sceptical about such predictions. In my 19 October article predicting that
Labor would win the election with a gain of 20 seats, I wrote: “In recent weeks we have seen reports of ‘internal
polling’ showing some extremely unlikely Liberal losses, including such traditional bastions as Kooyong (9.5),
Goldstein (10%), North Sydney (10.1%), Ryan (10.4%) and
Casey (11.4%). It’s
impossible to know the truth of these
reports. ‘Internal polling’ is always unverifiable, and is usually leaked for political reasons – if not actually
fabricated. These huge swings may be in the offing, driven by upper-middle-class concerns about climate change,
the Iraq war and refugee issues, but then again they may not.” In my seat-by-seat predictions, I stated my view
that the Liberals would retain Wentworth.
Am I a prize prat?
I certainly did not go nearly as far as some of my contemporaries. Most strikingly, at the end of September the
online commentator who uses the name
Possum Comitatus published a
commentary on the aggregated Newspoll results
for the September quarter. These, he said, showed that within the large swing to Labor which Newspoll was
consistently showing, there was an even larger swing in the Coalition’s “safe” seats – which as a result were no
longer safe at all.
Possum thus calculated that although the statewide swing in NSW during the September quarter had been 9.2%, in
“safe” Coalition seats it was 11.5%, and while the statewide swing in Victoria was 11.0%, in “safe” Coalition
seats it was 13.8%. The figures were 9.1% and 10.1% for Queensland, 4.4% and 5.2% for Western Australia, and 9.4%
and 11.6% for South Australia.
Possum then translated these swing figures into a table of seats, which showed Labor gaining a massive 52 seats,
for a total of 112. The projected gains included, as well as all the marginal seats, a group of urban upper-income
seats: Wentworth, North Sydney, Warringah,
Hughes and
Greenway (NSW), Kooyong,
Higgins, Goldstein,
Menzies,
Flinders, Casey and
Aston (Vic), and Sturt and
Boothby (SA). Of these seats,
Labor has never won Wentworth, North Sydney, Warringah,
Kooyong, Goldstein or Menzies, and hasn’t won Boothby since 1946. Possum’s table also included some regional seats
with large Coalition majorities such as
Paterson,
Gilmore, Dawson,
Flynn,
Hinkler,
Gippsland and
Wannon. Labor has never won
Gippsland and hasn’t won Wannon since 1954.
When I saw this exercise in interpretation of aggregated opinion poll data, I was immediately sceptical. I
commented at
Possum’s blog: “I’m not sure what the purpose of this exercise is when it so obviously defies
commonsense and the observations of everyone in the political process. Labor is not going to win Dawson,
Warringah or Higgins. If your calculations say they are, there is something wrong with your calculations.”
There are three things to say about this in retrospect. The first is that Labor did in fact win Dawson, so I was
a little too sweeping in my dismissal of Possum’s analysis. The second is that I was somewhat less polite about
my dissent than I might have been. The third, however, is that I was broadly right. Of all the seats mentioned
above, Dawson was the only Labor gain. Of the urban seats mentioned, Labor won none, and came close only in Sturt.
Wentworth and Kooyong actually swung to the Liberals, while Higgins and Warringah produced swings of less than 2%.
Possum’s response to my criticism was that this was what the polls were showing and that he was only presenting
the poll data as it translated into seats. I’m not a statistician, and I didn’t attempt to engage in a statistical
debate. My comment was: “I can’t argue statistics, but I maintain my view that an analytical process which
produces results that no-one actually believes has something wrong with it. Does Possum really think Labor is
going to win a swag of seats now classed as “safe” for the Coalition?”
Possum replied: “What I think is that unless people are telling lies to Newspoll, the average national swing in
government safe seats is around the 11.6% mark.” I replied: “No no, Possum, I want you to tell me whether you
yourself think Labor is going to win seats of the order of Kooyong, Warringah and Dawson.” Possum then said that
he did not in fact think that Labor would win Kooyong, Warringah, Dawson or Wentworth. (So we were both wrong
about Dawson.)
I then said: “OK, that’s four seats which your calculations say Labor will win, but you agree Labor will not win.
Now, what about Wannon, Menzies, North Sydney, Hughes, Gilmore, Macarthur, Aston, Flynn and Hinkler, all of which
appear as Labor seats on your table?”
Possum replied: “For any given safe government seat, an argument can be made as to why it wont fall. We don’t know
exactly which of the safe government seats are guaranteed to topple, what we do know is that there is an average
11.6% swing against the government in its safe seats. For that to be wrong, thousands of people would have had to
be telling lies to Newspoll over a 9 month period, which I simply do not believe. So for every safe government
seat that swings less than the average amount, others in the same category will swing more, even though an
argument can be made for nearly every single one of them as to why they shouldn’t fall.”
So there we had the nub of the matter. Possum was not just relaying what Newspoll had found. He did actually
believe that there was a massive swing afoot in the “safe” Coalition seats, and that even if a specified seat,
such as Kooyong or Wannon, would not fall, some other seat of the same order of “safeness” would fall in its place.
Others spoke up in support of Possum’s view. Simple Simon said: “There is another possibility for why these numbers
are not providing a result that matches your perceptions - that is, your perceptions are wrong.” When I commented
that “Psephological wisdom requires that statistics be interpreted in the light of political knowledge, not
presented raw,” Greensborough Growler said: “Adam, You have made a prize prat of yourself today. Possum’s numbers
are Possum’s numbers. Presenting information in the light of political insight will lead you to a bollocking of
Pearson dimension.” (This last was a reference to Possum’s very thorough and entirely correct
demolition of
Christopher Pearson’s absurd
pseudo-psephological
meanderings in The Australian.)
I responded: “Of course, as a matter of logic, if there is a contradiction between Possum’s data and my beliefs,
then it is just as possible that my beliefs are wrong as it is that Possum’s data are wrong, or wrongly
interpreted. And no-one would be happier than me if that were the case. But I persist in my view, based on 38 years
of fairly intense observation of Australia elections and practical experience of electioneering, that at least a
dozen seats on Possum’s list will not change hands, and there therefore there is unlikely to be a swing in the
safe Coalition seats of anything like the scale he postulates. Since there is no logical way of resolving this
impasse, I will not pursue the matter further. We will see what we will see on election night. If Labor wins any
one of Warringah, Dawson, Wannon and Kooyong I will be the first to nominate Possum for the Nobel Prize in
Psephology.”
HarryH put me in my place when he said: “Adam’s 38 years of political/ALP experience is stopping him from seeing
what is happening. The old rules of Lib and Lab voters are being thrown out. Howard won a lot of “true” Lab voters
because he gave/promised them what they wanted at the time. Now Rudd is doing what no Lab Leader has done in
history. He has made an audacious and deliberate raid on Lib heartland voters. Howard’s whole frontbench, bar
1 or 2, are totally on the nose to lifelong small L liberals. And in perfect timing for Lab, Rudd is everything
these Lib voters are yearning for. This is the perfect storm that is brewing. Possum has presented the data. It
has been so consistent all year, and seems to be widening, that it has to be believed. There will be shockwaves
rummaging through this extreme right wing coalition on election night, and disbelief on the faces of longtime
political junkies who are not reading the data and the mood correctly.”
The mystery of the Rudd liberals
Let no-one call me a sore loser. Since Labor did win Dawson (sending an astonished
James Bidgood on his first
ever trip to Canberra), I have emailed the
King of Sweden and nominated Possum for the
Nobel Prize in Psephology.
Correspondingly, however, I think Possum might now concede that his table was wrong about all the other seats I
mentioned earlier (in fact there were 28 seats on Possum's list of predicted ALP gains which the Coalition
retained). Greensborough Growler might like to reflect on his allegation that
I was a prize prat for daring to apply some political commonsense to the sacred findings of the psephometricians.
HarryH might be more wary in future about blind reliance on “data” without critical analysis.
Possum’s key defence of his data was that he refused to believe that thousands of people had been “telling lies
to Newspoll over a 9 month period.” So do I think they were telling lies? Yes and no. I don’t dispute that polls
showed larger-than-average swings in safe Liberal seats. ACNielsen had Labor winning Wentworth on 19 November.
A McNair poll on 20 November found North Sydney tied on 50-50. “Informed sources” said that the Liberals were
certain to lose Ryan (which wasn’t even on Possum’s list). My point is that all such polls and reports needed
to be interpreted in the light of knowledge derived from fields other than statistics, such as history, political
science and sociology. My view was always that “Rudd liberals” would get cold feet at the last minute, and that
this would be shown in a last-days narrowing in the opinion polls, which is precisely what happened.
To show that I am not merely being wise after the event in this respect, let me quote another comment I made at
Possum’s blog at the start of October: “Simon is correct in his observation that the Liberal vote is being
eroded at both ends by a combination of WorkChoices and other issues. But I doubt that when it comes to the
crunch those trends will be enough to win the more improbable of the seats on Possum’s list. People in Kooyong
and Warringah may be venting about Howard at the moment, but they have objective class interests which a Labor
government will threaten, however mildly. There will be a major focus in the campaign on tax policy, and this
is always THE basic issue for high income voters, and particularly high-income/high-debt voters. It will drive
quite a lot of them back to the Liberal fold by election day.” And so it came to pass.
What actually happened to the “doctors’ wives” vote on 24 November? If we take the 30 seats with the highest
median weekly family income (all urban seats, 12 of them held by Labor), we see that the average swing to Labor
was 5.3%, only slightly below the national swing of 5.6%. This figure, however, conceals a wide variation of
swing, ranging from 9.1% in Mitchell to minus 1.1% in Wentworth.
There is a noticeable pattern among these high-income seats. Those which produced the biggest swings to Labor
tended to be further out from the city centres: Aston (8.1%),
Cook (6.6%), Greenway (6.9%),
Hughes (6.4%),
Mitchell (9.1%),
Parramatta (7.8%).
These are seats with high levels of people paying off homes: they are
mortgage belt seats, albeit high-income ones. This is particularly true of Mitchell, Hughes, Greenway and Aston.
They are not really “doctors’ wives” seats at all. Women in these seats are working two jobs to pay their kids’
school fees, not discussing climate change over lunch in Double Bay.
The real “doctors’ wives” seats, the high-income seats in the city centres, whether held by Liberal or Labor,
produced the lowest swings of any urban seats:
Brisbane (2.8%),
Canberra (1.8%),
Curtin (0.8%),
Fraser (1.8%),
Higgins (2.0%), Kooyong (minus 0.1%),
Melbourne (1.0%),
Sydney (1.8%) and
Wentworth (minus 1.1%). All of these
have very low levels of people paying mortgages, and also low levels of families with dependent children.
If on the other hand we take the 30 urban seats with the lowest median weekly family income (21 of them held by
Labor before the election), we get an average swing of 5.8%, slightly above the national swing. Again, there is
a pattern within this overall figure, but this time it is a state-based pattern. The biggest swings were mostly
in Queensland (
Forde 15.1%,
Blair 10.2%,
Petrie 9.5%,
Rankin 8.7%,
Oxley 7.0%) or South
Australia (
Makin 8.6%,
Wakefield 7.3%). The
exceptions were
Calwell (11.1%) and
Chifley (8.6%). The lowest
swings in this group were
all in WA (
Hasluck 3.2%,
Perth 2.3%,
Fremantle 1.3%,
Brand 1.0%,
Stirling 0.8%,
Swan minus 0.3%).
If WA is excluded, the average swing in these seats was 6.9%.
It’s worth noting that the only seats Labor won among the 30 seats with the highest median weekly family income
were
Bennelong, Parramatta and
Solomon, all of which are
significantly more multicultural than most high-income
seats. We know that coming from a non-Anglo background is predictive of Labor voting, regardless of income.
Before the election the Coalition held only four of the 30 seats with the highest proportion of voters born in
a non-English-speaking country, and they lost three of them: Bennelong,
Moreton and Parramatta,
retaining only
Menzies. The other relatively high-income seats gained by Labor were
Bonner, Moreton,
Deakin and McEwen, which
are only slightly above average in terms of median weekly family income. Meanwhile, in the 30 urban seats with
the lowest median weekly family income, Labor made a net gain of six seats. The only seats among this 30 now
held by the Liberals are
Dunkley, Stirling and Swan.
A typical class-based election
What these figures tell us that the 2007 election was a typical class-based election, in which most people voted
for the party that seemed to them to represent best their economic interests. Not everyone votes according to
their economic interest, of course. Labor already had a base of loyal support among upper-income people, mainly
in inner-city cosmopolitan seats like Sydney,
Grayndler, Melbourne,
Melbourne Ports, Brisbane,
Adelaide and
Canberra. There was a further cohort of upper-income people who spent most of 2007 flirting with the idea of
voting Labor. They were tired of John Howard, they liked Kevin Rudd, they were concerned about climate change,
they were unhappy about the Iraq War. When it came time to vote, however, class interest prevailed for most of
them.
Why did these voters change back to the Coalition? Howard’s massive tax bribe at the start of the campaign had
no immediately apparent effect, but it may have shifted opinions in the course of the campaign. But his
last-minute giveaway to the parents of private-school students, regardless of wealth, appears to have been a
powerful influence, wildly irresponsible as it seemed to many to be at the time. A private school education is
the key determinant of class privilege and status in Australia, and is thus a central aspiration, second only to
home-ownership, for many middle-class families. Making it easier for upper-income parents to keep their kids in
elite schools probably trumped climate change in the minds of many of these voters.
In the end, however, clawing back support among the upper middle class was not enough to save the Howard
government. John Howard’s peculiar achievement was to drive a large slab of socially-conservative working-class
voters, who had switched to the Coalition in 1996 in response to his anti-elitist rhetoric, back to Labor by
attacking their economic interests with his industrial relations laws. That’s why the Coalition lost seats like
Lindsay,
Dobell,
Page, Parramatta, Deakin,
Bonner, Blair, Dawson, Flynn, Wakefield,
Kingston, Makin, Hasluck,
Bass and
Braddon. The lesson for Labor is that the pursuit of upper-middle class voters is a chimera; they will
usually vote with their wallets or purses in the end. Pandering to the concerns of the elites is not the way for
Labor to gain or retain office. Labor is the party of the middle-to-lower-income majority of Australians, and it
must never forget that.
|
|