After the Kensington & Chelsea by election in 1999 people tried to project to a general election result . . . and the BBC gets it wrong
The Kensington & Chelsea by-election
By Quentin Langley
Who would win a general election?
Traditionally there are several ways of answering this question, all of
which are open to reasonable questioning. First of all you have to predict how many votes each party will get, and then project that into seats. There is a plain possibility of error at each stage. This is why some commentators – including Peter Kellner of the Standard – were able to predict that the Conservatives would come third or fourth in the Euro-elections while others, using exactly the same data, correctly predicted that they would come first.
The sources of information for the first stage of the analysis are fourfold: opinion polls, European elections, local elections, and by-elections. Opinion polls ask the right question: "How would you vote if there were a general election tomorrow?" but do not necessarily get the right answer. The actual elections plainly give you the accurate answer, but to a slightly different question, so data has to be intelligently interpreted.
Opinion polls suggest that Labour's current level of support is higher than in 1997 and the Conservative Party's is lower. This is not borne out by any elections. I suspect that as in the last two general elections (especially 1992) the polls underestimate Conservative support. On the other hand local elections show far higher levels of support for the Liberal Democrats than national opinion polls (or other elections) and much lower levels of support for Labour. These elections may well favour the LibDems at Labour's expense, though my gut feeling is that they get Conservative levels of support about right – somewhere in the mid thirties, ie up on 1997 but not much.
The Euro-elections, being about Europe, may have exaggerated Conservative support at Labour's expense. But given the same point, and that the electoral system allowed small parties to be taken more seriously, it may have cost the Conservatives votes to both the Pro-Euro Conservatives and to UKIP. The Kensington and Chelsea by-election may have marked the end of the Pro-Euro Conservatives, given both the nature of the constituency and the profile of the Conservative candidate their failure to match their June levels of support (already low) could prove fatal, at least for non-Euro elections. Don't be surprised to see John Stevens joining the LibDems or Labour.
So how representative was the K&C by-election? It is possible that the
Conservative candidate could have cost his party some voters and won over others. Labour certainly believed that they could gain support by focussing on Michael Portillo personally. On the other hand he is plainly a candidate of considerable personal charisma. The presence of minor parties on the ballot paper may also have cost Portillo some votes. The truth is we just don't know whether it accurately reflects movements in support since 1997. So let us play a game, and pretend that it does.
The thing that the BBC in particular has been atrocious at is not predicting votes but in converting them into seats. Most of our media is trapped in the 1950s. They use what is called the "Uniform Swing Hypothesis". It assumes that if the Labour vote falls by 5% of the vote nationally it will do so in each constituency. No matter that in some seats Labour has more than 60% of the vote and in others under 10%. Since all constituencies are supposed to be the same size, it actually means that exactly the same number of people will desert the Labour Party in Rhondda Valley South as in Surrey Heath.
In any election in which the principle movement of votes is between Labour and the Conservatives this will give you a broadly accurate result. Although it will get the results of the Welsh Valleys and the South Downs badly wrong, it will get the result of most marginals, and therefore the overall result, about right. This is why the media were right about 1997, but badly wrong on 1987 and 1992, when most of the movement in votes was from the Alliance or LibDems to Labour, and the Conservative vote was stable. Remember how, even after the votes had been cast, the BBC predicted a Conservative majority of 26 in 1987 and a hung Parliament in 1992?
The more sensible analysis is the Proportional Loss Hypothesis. This assumes that Labour voters, being a self-selecting group, are more likely to behave like Labour voters in another constituency than to behave like their Conservative voting neighbours. Sensible? Not only that, it works. The BBC nationally may tend to get election results wrong, but BBC Radio Bristol, which turns not to a politics professor but to a professor of mathematical engineering (Dr Gordon Reece, no relation to the speech writer of the same name) for its analysis, and gets the results right.
While everyone who applies the USH has assumed that the "swing" in K&C
was insufficient to overturn Labour's majority this is not, in fact,
correct. If voters are leaving the Labour Party for the Conservatives,
Kensington and Chelsea is certain to under-record this trend. After all,
Labour did not have that many votes in the seat to start with. You
would reasonably expect the Conservatives to pick up more disillusioned
ex-Labour voters in the Welsh Valleys and the dockside areas of
Liverpool than in Chelsea or Surrey Heath. This, of course, would not
gain the Conservatives any seats at all. (Though, if the Welsh Assembly
elections are anything to go by, it might gain a few for Plaid Cymru).
What actually happened in the by-election is not that 5% electorate
deserted Labour but that a fifth of Labour's voters did so. This is entirely in line with local elections, but not with national opinion polls. The LibDems
lost a third of their support, which is in line with national opinion polls but not local elections. Of course, not all these voters shifted to the Conservatives, only a third did, with almost twice as many going to the Pro-Euro Conservatives and other minor parties. Nonetheless, in seats where Labour and/or the LibDems had substantial numbers of votes at stake, this result would have seen a massive rise in the Conservative Party's fortunes. Indeed, it would see the gap that Labour opened up at the last election closing completely, with both parties getting around 36% of the national vote and the LibDems falling from 18% to 12%.
Of course, had there been a general election on Thursday, things would
have been different. The smaller parties would have got nowhere near as
much support as they did in the by-election, and it is not at all clear which of the major parties would have got that vote.
But those who say that projecting that "swing" to a general election would have given Blair a majority of 70, are making exactly the same mistake that made the BBC seem so foolish in 1987 and 1992.
Copyright © Quentin Langley 28 November 1999
