Adam Carr's guide to
The 2007 Australian federal election

Polls and prophecies: a post mortem
         

9 December 2007

Now that the election is done and dusted, it’s possible to look back and compare the actual results with the voting intentions reported by opinion polls during the course of the year, and with the predictions made by commentators on the basis of those opinion polls. In summary, the verdict is that the level of Labor support on polling day was lower than reported by virtually every opinion poll during 2007, and that consequently the great majority of predictions made by commentators of the election outcome, based on those polls, were wrong.

The two-party vote for the ALP on 24 November was 52.9%. Only two polls in the whole of 2007 put the ALP vote this low: the final Galaxy and the final Newspoll, which both came out in the last days of the campaign and put the ALP vote at 52%. Every other poll showed the ALP vote at 53% or more, usually much more. All through July, August, September and October most polls put the ALP vote in a band between 54% and 58%. Only towards the end of the campaign itself did the polls begin to move back towards the Coalition. As well as the Galaxy and Newspoll, there was a final Morgan poll putting the ALP at 53.5%, the lowest Labor vote Morgan reported all year. But some persisted in giving high figures for Labor. The final ACNielsen poll put Labor’s vote at 57%.

The obvious question is: were the polls wrong all along, or was there a genuine shift back to the Coalition in the last week of the campaign? The short answer is that we will never know, because there is no way of checking in December whether an opinion poll taken in August was correct or not. The Newspoll at the beginning of September which had the ALP on 59% is not retrospectively proved wrong by the actual ALP vote of 52.9%.

My view is that the polls were broadly accurate through the course of the year. There was a surge of support for the ALP when Kevin Rudd was elected leader in December 2006, and this was sustained through the year, with a gradual drift downwards from March to July, and then a plateauing from July to November. The much-derided “narrowing” did indeed come, but only in the last week, when some of the “soft” ALP vote (another much-derided expression) went back to the Coalition.

Opinion polls are not meant to be predictive. Although Newspoll reported in September that 59% of voters had told them they would vote Labor, Newspoll itself did not make a prediction about what would actually happen. That role is left to the election commentators, both professional (in the mainstream media) and amateur (on election blogs). Most commentators correctly predicted that the ALP would win, despite the best efforts of News Ltd journalists to talk up the Coalition’s chances. But the great majority over-estimated the size of Labor’s win, both in terms of votes and seats.

We can see this in the predictions made by regular participants at the psephological weblog Pollbludger. These predictions were collected on 28 October. Only ten Pollbludger commentators thought (or claimed to think) that the Coalition would win. Another 25 (including me) thought that Labor would win narrowly, that is with 80 seats or fewer. Forty Pollbludgers got the result approximately right, predicting between 81 and 86 Labor seats. But a massive 73 Pollbludgers tipped a Labor landslide, with Labor winning more than 87 seats. Thus of 148 Pollbludgers making a prediction, only 23% underestimated the scale of Labor’s victory, while another 27% got it approximately right. Fully half the Pollbludgers overestimated Labor’s victory, some wildly so – 17 predicting that Labor would win 100 seats or more.

In drawing attention to the incorrect predictions of others, I do not claim omniscience for myself. I made plenty of incorrect predictions during this election, as I have done during others. But the record does show that on 19 October I predicted that the ALP would win 80 seats, which required a uniform swing of 5.4%. The actual result was 84 seats (assuming Labor’s 7-vote win in McEwen holds up) and a swing of 5.6%. I slightly underestimated the swing and the seat tally because I didn’t pick up the big swing in regional Queensland, which brought Dawson, Leichhardt and Flynn to the ALP. But it wasn’t a bad pick, and it was a lot better than most others.

All these polls can’t be wrong

Looking back over the year’s commentary, it seems to me that there were two reasons why most commentators over- stated the likely scale of the predicted ALP victory. The first was the tendency to treat opinions polls as infallible and as predictors, rather than as what they are: very imprecise measures of voter sentiment on a given date, with little or no predictive value. The second was to assume that the electorate as a whole shared the same scale of values as the commentators themselves, and therefore thought and felt about issues in the same way.

Now of course, everybody with any knowledge of elections and opinion polls knows that polls are not predictive. Everybody knows that polls only measure voter sentiment on a given day, and even that not very precisely (otherwise all the polls would agree, which they don’t).

But a long string of polls all showing the same thing does have a seductive effect, particularly when they accord with one’s own desires. (And let’s not try to deny that the great majority of election commentators, including me, wanted Labor to win this election.) Surely, most of us felt, and some of us said, all these polls can’t be wrong. Surely such a steady and commanding lead won’t evaporate at the last minute. Surely Labor is headed for a massive win.

The nearer we got to the election, the more alluring the message of the polls became. As the long-awaited Narrowing failed to happen, commentators became increasingly convinced that it couldn’t and wouldn’t happen. This was partly a case of the wish being father to the belief, and partly a reaction against the blatantly partisan commentary of the Murdoch press, which insisted all year that the Narrowing was at hand. If Dennis Shanahan said there would be a Narrowing, this was surely good evidence that there would not be.

Just as important as the over-reliance on opinion polls to predict the result was the innate cultural bias of the commentariat. Election commentators are overwhelmingly university-educated professionals living in the inner cities. They belong to what I called in an earlier article the “the inner urban cosmopolitan elite.” They share the common assumptions of this elite, including of course its antipathy to John Howard.

This is not to say that most commentators are overtly or deliberately partisan. No doubt they try not to be. But their political commentary is inevitably influenced by the worldview of the class to which they belong, and not all of them are sufficiently aware of this to be able to counter it when making political predictions. The predictions made by participants at Pollbludger show this clearly. These are for the most part very intelligent and well-informed people. Their overestimation of Labor’s likely performance did not result from stupidity or ignorance. It arose from a collective misunderstanding of what the electorate was thinking and about what issues were influencing the voters.

(Disclosure: I am myself a university-educated professional living in an inner city. How then have I been inoculated against the biases I criticise my fellow commentators for? I believe the answer, paradoxically, is my membership of the ALP, and more particularly the ALP Right. Since my political affiliation is formal and widely known, I have to guard especially carefully against partisanship. And since I support a faction dominated by hard-boiled pragmatists and generally hostile to the views of the inner urban cosmopolitan elite, I am better equipped to avoid the trap of elite group-think into which many others have fallen.)

The case of the doctors’ wives

The problem of the commentariat is not just one of simple partisanship. It is one of shared assumptions about what is important and about what influences the behaviour of the electorate. We saw this very strikingly during election year with the case of Dr Haneef. Virtually all the commentators, both professional and amateur, even in the Murdoch press, assumed that the government’s bungling of the case was yet another nail in the Coalition’s electoral coffin. I disagreed at the time, and I still do. My view was and is that the crucial section of the electorate (the floating voters) did not see this as a case of unjust treatment of an individual. They saw it as tough action against a suspected Islamist terrorist, and approved of it. I said at the time that Haneef was a net plus for the Coalition, and that is still my view.

For the inner urban cosmopolitan elite, the most important election issue by a long way was climate change, followed by the Iraq War (as a symbol of Australia’s subservience to the loathed George Bush), opposition to nuclear power, asylum seekers and Indigenous affairs. I make no judgement about the intrinsic merits of any of these issues. The question is whether these issues were in fact the decisive ones in the electorate at large.

This is linked to the question of the so-called “doctors’ wives.”. Throughout election year, there was a widely held view that the Coalition was fighting on two fronts: that it was losing support among lower-income voters (the “Howard battlers”) because of the unpopularity of the WorkChoices legislation, while at the same time losing the loyalty of its upper-income voters over the social and foreign policy issues mentioned in the previous paragraph. These voters were dubbed the “doctors’ wives” by some commentator or other, and this became the accepted code for “upper-income voters shifting their vote because of post-material issues.” Others began to call them “the Rudd liberals,” as a counterpoint to the “Howard battlers.”

It was on the basis of this assumption – that all well-educated and right-thinking people must share the views of the inner urban cosmopolitan elite on issues such as climate change, and would cast their votes accordingly – that some commentators began to predict that the election would see a dramatic shift in the loyalties of traditional urban Liberal strongholds such as North Sydney, Wentworth and Warringah (NSW), Kooyong, Higgins and Goldstein (Vic), Ryan (Qld) and Boothby and Sturt (SA).

There was certainly some evidence, in the form of opinion polls, to support these predictions. Published polls did indeed show close results in Wentworth, North Sydney, Ryan and Sturt. There were reports that private polls commissioned by the parties showed Coalition candidates in these seats to be in trouble. I certainly did not discount this evidence at the time. I coined the expression “the Bangkok dilemma” as a rather crude allusion to the belief that the Coalition was being “screwed at both ends.” I wrote an article about the contest for North Sydney – a seat Labor has never won in 106 years – based on the assumption that the Liberals were under real threat there.

I can, however, claim that I remained sceptical about such predictions. In my 19 October article predicting that Labor would win the election with a gain of 20 seats, I wrote: “In recent weeks we have seen reports of ‘internal polling’ showing some extremely unlikely Liberal losses, including such traditional bastions as Kooyong (9.5), Goldstein (10%), North Sydney (10.1%), Ryan (10.4%) and Casey (11.4%). It’s impossible to know the truth of these reports. ‘Internal polling’ is always unverifiable, and is usually leaked for political reasons – if not actually fabricated. These huge swings may be in the offing, driven by upper-middle-class concerns about climate change, the Iraq war and refugee issues, but then again they may not.” In my seat-by-seat predictions, I stated my view that the Liberals would retain Wentworth.

Am I a prize prat?

I certainly did not go nearly as far as some of my contemporaries. Most strikingly, at the end of September the online commentator who uses the name Possum Comitatus published a commentary on the aggregated Newspoll results for the September quarter. These, he said, showed that within the large swing to Labor which Newspoll was consistently showing, there was an even larger swing in the Coalition’s “safe” seats – which as a result were no longer safe at all.

Possum thus calculated that although the statewide swing in NSW during the September quarter had been 9.2%, in “safe” Coalition seats it was 11.5%, and while the statewide swing in Victoria was 11.0%, in “safe” Coalition seats it was 13.8%. The figures were 9.1% and 10.1% for Queensland, 4.4% and 5.2% for Western Australia, and 9.4% and 11.6% for South Australia.

Possum then translated these swing figures into a table of seats, which showed Labor gaining a massive 52 seats, for a total of 112. The projected gains included, as well as all the marginal seats, a group of urban upper-income seats: Wentworth, North Sydney, Warringah, Hughes and Greenway (NSW), Kooyong, Higgins, Goldstein, Menzies, Flinders, Casey and Aston (Vic), and Sturt and Boothby (SA). Of these seats, Labor has never won Wentworth, North Sydney, Warringah, Kooyong, Goldstein or Menzies, and hasn’t won Boothby since 1946. Possum’s table also included some regional seats with large Coalition majorities such as Paterson, Gilmore, Dawson, Flynn, Hinkler, Gippsland and Wannon. Labor has never won Gippsland and hasn’t won Wannon since 1954.

When I saw this exercise in interpretation of aggregated opinion poll data, I was immediately sceptical. I commented at Possum’s blog: “I’m not sure what the purpose of this exercise is when it so obviously defies commonsense and the observations of everyone in the political process. Labor is not going to win Dawson, Warringah or Higgins. If your calculations say they are, there is something wrong with your calculations.”

There are three things to say about this in retrospect. The first is that Labor did in fact win Dawson, so I was a little too sweeping in my dismissal of Possum’s analysis. The second is that I was somewhat less polite about my dissent than I might have been. The third, however, is that I was broadly right. Of all the seats mentioned above, Dawson was the only Labor gain. Of the urban seats mentioned, Labor won none, and came close only in Sturt. Wentworth and Kooyong actually swung to the Liberals, while Higgins and Warringah produced swings of less than 2%.

Possum’s response to my criticism was that this was what the polls were showing and that he was only presenting the poll data as it translated into seats. I’m not a statistician, and I didn’t attempt to engage in a statistical debate. My comment was: “I can’t argue statistics, but I maintain my view that an analytical process which produces results that no-one actually believes has something wrong with it. Does Possum really think Labor is going to win a swag of seats now classed as “safe” for the Coalition?”

Possum replied: “What I think is that unless people are telling lies to Newspoll, the average national swing in government safe seats is around the 11.6% mark.” I replied: “No no, Possum, I want you to tell me whether you yourself think Labor is going to win seats of the order of Kooyong, Warringah and Dawson.” Possum then said that he did not in fact think that Labor would win Kooyong, Warringah, Dawson or Wentworth. (So we were both wrong about Dawson.)

I then said: “OK, that’s four seats which your calculations say Labor will win, but you agree Labor will not win. Now, what about Wannon, Menzies, North Sydney, Hughes, Gilmore, Macarthur, Aston, Flynn and Hinkler, all of which appear as Labor seats on your table?”

Possum replied: “For any given safe government seat, an argument can be made as to why it wont fall. We don’t know exactly which of the safe government seats are guaranteed to topple, what we do know is that there is an average 11.6% swing against the government in its safe seats. For that to be wrong, thousands of people would have had to be telling lies to Newspoll over a 9 month period, which I simply do not believe. So for every safe government seat that swings less than the average amount, others in the same category will swing more, even though an argument can be made for nearly every single one of them as to why they shouldn’t fall.”

So there we had the nub of the matter. Possum was not just relaying what Newspoll had found. He did actually believe that there was a massive swing afoot in the “safe” Coalition seats, and that even if a specified seat, such as Kooyong or Wannon, would not fall, some other seat of the same order of “safeness” would fall in its place.

Others spoke up in support of Possum’s view. Simple Simon said: “There is another possibility for why these numbers are not providing a result that matches your perceptions - that is, your perceptions are wrong.” When I commented that “Psephological wisdom requires that statistics be interpreted in the light of political knowledge, not presented raw,” Greensborough Growler said: “Adam, You have made a prize prat of yourself today. Possum’s numbers are Possum’s numbers. Presenting information in the light of political insight will lead you to a bollocking of Pearson dimension.” (This last was a reference to Possum’s very thorough and entirely correct demolition of Christopher Pearson’s absurd pseudo-psephological meanderings in The Australian.)

I responded: “Of course, as a matter of logic, if there is a contradiction between Possum’s data and my beliefs, then it is just as possible that my beliefs are wrong as it is that Possum’s data are wrong, or wrongly interpreted. And no-one would be happier than me if that were the case. But I persist in my view, based on 38 years of fairly intense observation of Australia elections and practical experience of electioneering, that at least a dozen seats on Possum’s list will not change hands, and there therefore there is unlikely to be a swing in the safe Coalition seats of anything like the scale he postulates. Since there is no logical way of resolving this impasse, I will not pursue the matter further. We will see what we will see on election night. If Labor wins any one of Warringah, Dawson, Wannon and Kooyong I will be the first to nominate Possum for the Nobel Prize in Psephology.”

HarryH put me in my place when he said: “Adam’s 38 years of political/ALP experience is stopping him from seeing what is happening. The old rules of Lib and Lab voters are being thrown out. Howard won a lot of “true” Lab voters because he gave/promised them what they wanted at the time. Now Rudd is doing what no Lab Leader has done in history. He has made an audacious and deliberate raid on Lib heartland voters. Howard’s whole frontbench, bar 1 or 2, are totally on the nose to lifelong small L liberals. And in perfect timing for Lab, Rudd is everything these Lib voters are yearning for. This is the perfect storm that is brewing. Possum has presented the data. It has been so consistent all year, and seems to be widening, that it has to be believed. There will be shockwaves rummaging through this extreme right wing coalition on election night, and disbelief on the faces of longtime political junkies who are not reading the data and the mood correctly.”

The mystery of the Rudd liberals

Let no-one call me a sore loser. Since Labor did win Dawson (sending an astonished James Bidgood on his first ever trip to Canberra), I have emailed the King of Sweden and nominated Possum for the Nobel Prize in Psephology. Correspondingly, however, I think Possum might now concede that his table was wrong about all the other seats I mentioned earlier (in fact there were 28 seats on Possum's list of predicted ALP gains which the Coalition retained). Greensborough Growler might like to reflect on his allegation that I was a prize prat for daring to apply some political commonsense to the sacred findings of the psephometricians. HarryH might be more wary in future about blind reliance on “data” without critical analysis.

Possum’s key defence of his data was that he refused to believe that thousands of people had been “telling lies to Newspoll over a 9 month period.” So do I think they were telling lies? Yes and no. I don’t dispute that polls showed larger-than-average swings in safe Liberal seats. ACNielsen had Labor winning Wentworth on 19 November. A McNair poll on 20 November found North Sydney tied on 50-50. “Informed sources” said that the Liberals were certain to lose Ryan (which wasn’t even on Possum’s list). My point is that all such polls and reports needed to be interpreted in the light of knowledge derived from fields other than statistics, such as history, political science and sociology. My view was always that “Rudd liberals” would get cold feet at the last minute, and that this would be shown in a last-days narrowing in the opinion polls, which is precisely what happened.

To show that I am not merely being wise after the event in this respect, let me quote another comment I made at Possum’s blog at the start of October: “Simon is correct in his observation that the Liberal vote is being eroded at both ends by a combination of WorkChoices and other issues. But I doubt that when it comes to the crunch those trends will be enough to win the more improbable of the seats on Possum’s list. People in Kooyong and Warringah may be venting about Howard at the moment, but they have objective class interests which a Labor government will threaten, however mildly. There will be a major focus in the campaign on tax policy, and this is always THE basic issue for high income voters, and particularly high-income/high-debt voters. It will drive quite a lot of them back to the Liberal fold by election day.” And so it came to pass.

What actually happened to the “doctors’ wives” vote on 24 November? If we take the 30 seats with the highest median weekly family income (all urban seats, 12 of them held by Labor), we see that the average swing to Labor was 5.3%, only slightly below the national swing of 5.6%. This figure, however, conceals a wide variation of swing, ranging from 9.1% in Mitchell to minus 1.1% in Wentworth.

There is a noticeable pattern among these high-income seats. Those which produced the biggest swings to Labor tended to be further out from the city centres: Aston (8.1%), Cook (6.6%), Greenway (6.9%), Hughes (6.4%), Mitchell (9.1%), Parramatta (7.8%). These are seats with high levels of people paying off homes: they are mortgage belt seats, albeit high-income ones. This is particularly true of Mitchell, Hughes, Greenway and Aston. They are not really “doctors’ wives” seats at all. Women in these seats are working two jobs to pay their kids’ school fees, not discussing climate change over lunch in Double Bay.

The real “doctors’ wives” seats, the high-income seats in the city centres, whether held by Liberal or Labor, produced the lowest swings of any urban seats: Brisbane (2.8%), Canberra (1.8%), Curtin (0.8%), Fraser (1.8%), Higgins (2.0%), Kooyong (minus 0.1%), Melbourne (1.0%), Sydney (1.8%) and Wentworth (minus 1.1%). All of these have very low levels of people paying mortgages, and also low levels of families with dependent children.

If on the other hand we take the 30 urban seats with the lowest median weekly family income (21 of them held by Labor before the election), we get an average swing of 5.8%, slightly above the national swing. Again, there is a pattern within this overall figure, but this time it is a state-based pattern. The biggest swings were mostly in Queensland ( Forde 15.1%, Blair 10.2%, Petrie 9.5%, Rankin 8.7%, Oxley 7.0%) or South Australia ( Makin 8.6%, Wakefield 7.3%). The exceptions were Calwell (11.1%) and Chifley (8.6%). The lowest swings in this group were all in WA ( Hasluck 3.2%, Perth 2.3%, Fremantle 1.3%, Brand 1.0%, Stirling 0.8%, Swan minus 0.3%). If WA is excluded, the average swing in these seats was 6.9%.

It’s worth noting that the only seats Labor won among the 30 seats with the highest median weekly family income were Bennelong, Parramatta and Solomon, all of which are significantly more multicultural than most high-income seats. We know that coming from a non-Anglo background is predictive of Labor voting, regardless of income. Before the election the Coalition held only four of the 30 seats with the highest proportion of voters born in a non-English-speaking country, and they lost three of them: Bennelong, Moreton and Parramatta, retaining only Menzies. The other relatively high-income seats gained by Labor were Bonner, Moreton, Deakin and McEwen, which are only slightly above average in terms of median weekly family income. Meanwhile, in the 30 urban seats with the lowest median weekly family income, Labor made a net gain of six seats. The only seats among this 30 now held by the Liberals are Dunkley, Stirling and Swan.

A typical class-based election

What these figures tell us that the 2007 election was a typical class-based election, in which most people voted for the party that seemed to them to represent best their economic interests. Not everyone votes according to their economic interest, of course. Labor already had a base of loyal support among upper-income people, mainly in inner-city cosmopolitan seats like Sydney, Grayndler, Melbourne, Melbourne Ports, Brisbane, Adelaide and Canberra. There was a further cohort of upper-income people who spent most of 2007 flirting with the idea of voting Labor. They were tired of John Howard, they liked Kevin Rudd, they were concerned about climate change, they were unhappy about the Iraq War. When it came time to vote, however, class interest prevailed for most of them.

Why did these voters change back to the Coalition? Howard’s massive tax bribe at the start of the campaign had no immediately apparent effect, but it may have shifted opinions in the course of the campaign. But his last-minute giveaway to the parents of private-school students, regardless of wealth, appears to have been a powerful influence, wildly irresponsible as it seemed to many to be at the time. A private school education is the key determinant of class privilege and status in Australia, and is thus a central aspiration, second only to home-ownership, for many middle-class families. Making it easier for upper-income parents to keep their kids in elite schools probably trumped climate change in the minds of many of these voters.

In the end, however, clawing back support among the upper middle class was not enough to save the Howard government. John Howard’s peculiar achievement was to drive a large slab of socially-conservative working-class voters, who had switched to the Coalition in 1996 in response to his anti-elitist rhetoric, back to Labor by attacking their economic interests with his industrial relations laws. That’s why the Coalition lost seats like Lindsay, Dobell, Page, Parramatta, Deakin, Bonner, Blair, Dawson, Flynn, Wakefield, Kingston, Makin, Hasluck, Bass and Braddon. The lesson for Labor is that the pursuit of upper-middle class voters is a chimera; they will usually vote with their wallets or purses in the end. Pandering to the concerns of the elites is not the way for Labor to gain or retain office. Labor is the party of the middle-to-lower-income majority of Australians, and it must never forget that.

         

Back to commentary page
Leave a comment
Return to front page