The Wisdom of Political Crowds

Faced with conflicting data from a wide variety of sources, PollyVote simply averages all the available numbers. The latest reading from PollyVote is that Obama leads by 4.4 percentage points. The same technique predicted in 2004 that George W. Bush would win 51.5% of all two-party votes; his actual share was 51.2%.

The numbers in this case are grouped into four categories, each weighted equally: polls; the Iowa Electronic Markets, which allows traders to buy and sell contracts tied to the results of the election, and turns the market's collective wisdom into a prediction of vote shares; quantitative models, including one that has correctly predicted the last six presidential elections based on factors such as whether the sitting president has suffered a major scandal; and a survey of political experts conducted by PollyVote.

There are drawbacks to averaging disparate forecasts. Polls and the electronic markets are updated daily, but the quantitative models aren't always as current; the latest expert survey was conducted last month and predicts a 48.9% two-party vote share for McCain, meaning Obama would win with 51.1%. Also, PollyVote's utility is limited by its focus on the national popular vote, whose winner may not carry the Electoral College. The site does also average electoral-vote forecasts in a separate projection (Obama is ahead by 131 electoral votes), but it's working with less data there.

Armstrong maintains that combining forecasts is better than relying on any single metric. "The purpose of the site is to demonstrate the power of combining forecasts," he said. "This is not an intuitive concept and few organizations use it. Most intelligent people think that they can do better, but they cannot." He points to a 2006 paper demonstrating that averaging forecasts is better than choosing one somewhat arbitrarily, so long as not all the forecasts are wrong in the same direction – and that the majority of a group of business students didn't understand this concept.

It's impossible to know before Election Day whether all the forecasts combined in the PollyVote model are erring in the same direction, but it's exceedingly unlikely.

Read more…