The Subversion of Democracy Part 1: Why Pollsters Are Wrong

by David K. Levine, Andrea Mattozzi and Salvatore Modica on May 15, 2017

Special interests subvert elections by lobbying, but do not have the power to win elections, which are mostly decided by large groups. How is it the case? Lobbying is, after all, like voting with dollars. How come small groups win at one, and large groups at the other? Not surely because small groups have more dollars. A larger group has more resources: both votes and money. In the USA farmers - who are very effective at lobbying for subsidies for themselves, in fact so effective that each American works about 11 hours a year to pay the farmers - control less than 2% of total income. It is scarcely the fact that they overwhelm the rest of the population with their riches. 

But then what are the differences between voting and lobbying? In both voting and lobbying groups exert effort - money, time, votes - to win a prize - a piece of legislation, for example. There are some distinctive differences in the nature of the contest however, which have specific consequences on the final outcome. They are highlighted in the table below.

Who wins a political contest?

Think first about lobbying - about bribing a politician. Each side communicates to the politician how much they are willing to pay to win, there is back and forth and bids are raised until someone drops out. An economist would describe this as a first price open auction: bids are raised until only one bidder is left and the winner pays their high bid. A useful concept in analyzing an auction is the willingness to bid: the most that a bidder is willing and able to bid if winning for sure rather than losing for sure. For example, if the prize is a pizza and it is worth $10 to me and I have $5 in my pocket I am willing to bid $5 since the pizza is worth more than that and I cannot bid more than $5. If I have $15 in my pocket my willingness to bid is $10 since I am not willing to pay more than that for the pizza, although I can afford to do so.

The value of winning is not necessary the same to different groups. It may be, as in the case of subsidies or trade restrictions, that the benefit to the special interest is less than the cost to the general public because there is an economic distortion: in the case of some regulations such as environmental regulations, things can go either way depending on whether the cost of the regulation to the special interest exceeds or not the benefit to the general public. Similarly, with civil rights the benefit of receiving rights by the minority may greatly exceed the benefit to the majority of denying such rights.

What happens in lobbying - in a first price open auction? The bid is raised until the group with the lower willingness to bid drops out at which point the group with the higher willingness to bid wins. To a good approximation the group with the highest willingness to bid wins the lobbying contest and pays what it costs to match the willingness to bid of the loser. The difference between the two willingness to bid we refer to as the surplus.

What does this tell us? First, since each group can provide only limited effort and certainly no more that its willingness to bid, it matters whether the size of the prize - the stakes - are high or low. If the stakes are high then the smaller group cannot provide enough effort to match the willingness to bid of the larger group, so the large group is successful - regardless of whether the prize might be worth more to the smaller group. We say in this case that the larger group is advantaged and we see that they earn a positive surplus. If the stakes are low both groups are on an equal footing since both can afford to pay what the prize is worth - in this case the group with the higher value wins - which is after all what we would like. Overall neither the small nor the large group is advantaged - the matter is decided by considerations of economic efficiency.

What about voting? The rules of the game are different in voting. Lobbying is a winner pays auction. Voting is what is known as an all-pay auction: both groups provide effort in the form of getting voters to the polls and that effort is spent regardless of who wins the election. The striking fact is: as far as surplus goes voting and lobbying are identical. In expected value the group with the lower willingness to bid still gets nothing and the group with the higher willingness to bid gets the difference between their own willingness to bid and the cost of matching the willingness to bid of the loser. This is called the tripartite auction: see our explainer. So as far as which group is advantaged and what the groups get in expected value, there is no difference between lobbying and voting just because the game is played differently.

Why Pollsters Are (sometimes) Wrong: The Neumann principle

People vote for lots of reasons: some vote out of a sense of civic duty, others to register their opinion - and some vote to help their side win. Political scientists refer to those who vote to help their side win as strategic voters: whether or not they vote depends upon circumstances. For example in the USA 2012 presidential election voter turnout in the electoral college swing states was 7.4% higher than in other states: strategic voters correctly turned out where their vote was most likely to count. Each party and its strategic voters must decide whether to engage in the costly effort of turning out at the polls.

What does this tell us? There are more Democrats than Republicans in the USA, so if the Democrats turn out their strategic voters they should win. And the same if neither party turns out their strategic voters. But there are plenty of Republicans, so if the Democrats do not turn out and the Republicans do the Republicans will win.

Now suppose that Sam has a giant computer that engages in deep mathematical big data analysis. Sam tells us: everyone is going to turn out so the Democrats are going to win - or else he will eat a bug. Since Sam has such a large computer and backs up his prediction with the promise of a bug everybody should believe him. So naturally the Republicans save the time and effort of turning out their strategic voters: oops, Sam has to eat a bug. But perhaps Sam should tell us that the Democrats are definitely going to turn out and win, and the Republicans are going to stay home. Then the Democrats should save the time and effort of turning out their strategic voters, so  all strategic voters stay home and the Democrats still win: another bug for Sam. Fine: Sam should predict that everyone stays home. Oops - now the Republicans should turn out and win: yet another bug for Sam. Finally, Sam might predict that the Republicans will turn out and the Democrats not - in which case the Democrats will turn out and - yuck! - Sam still has to eat that bug.

The point is: there is nothing Sam can say that if it is believed will be correct. In economics this is known as the Lucas critique and is the foundation of rational expectations - a theory that says only that forecasts should be true when people believe them. It is odd that we accept the Heisenberg uncertainty principle in quantum mechanics where spooky particles seem to anticipate what other are thinking - yet the idea in the social sciences that intelligent human beings might react to forecasts and predictions is controversial. At one time, perhaps, there was an excuse for this - Sir Arthur Conan Doyle made exactly the same mistake as Sam. In Doyle’s 1893 short story “The Final Problem” Holmes outwits Moriarity by getting off the train in Canterbury. Moriarity - despite his great intellect - foolishly believes that Holmes will attempt to go to Dover where he will inevitably be captured by Moriarity. There is some excuse for Doyle since the paradox was only solved by John Von Neumann in 1928: Von Neumann showed that the only solution is uncertainty. Nobody can know for sure which side is going to turn out the most voters. In the language of game theory, the turnout game - like the Moriarity game - has no pure strategy equilibria. Why don’t we call this the Neumann uncertainty principle?

Many reasons have been advanced why the polls sometimes fail to predict the outcome of elections. It might be that people lie to pollsters. This is called the Bradley effect after the California gubernatorial election of 1982 in which a black man - Tom Bradley - lost a narrow election to a white man - George Deukmejian - despite polls showing that Bradley would win. It was widely argued afterwards that the reason the polls failed was because voters were reluctant to admit to pollsters that they would not vote for a black man. Another possible reason could be that  pollsters are wrong because people change their minds at the last minute. However, political scientists have not found support for this hypothesis nor for the Bradley effect. For example Daniel J. Hopkins in the Journal of Politics 2009 studies a broad range of elections and  finds no support for the existence of a Bradley effect. Furthermore even though there are many voters not affiliated with a particular party, they have Democratic or Republican leaning and that is a good predictor of how they will vote.

What polls do a poor job of - and pollsters are well aware of this - is in predicting whether people will vote and we constantly read after the fact analysis of the form: “this year turnout among Hispanic voters was unusually low”  and so forth. But even pundits like Nate who - unlike Sam - made the sensible prediction that Trump had a 27% chance of winning the 2016 Presidential election - does not seem to realize that the reason that polls do not predict turnout is not because of inadequate data or defective models - it is because of the Neumann principle - it is because the polls cannot predict turnout.

There is a second fallacy that can be categorized as overreaction. After the polls are wrong in close elections - as they often are - people start to argue that there is something wrong with the polls and therefore they are inaccurate even in elections that are not close. This is not what theory tell us: it does tell us that the outcome is uncertain when the parties are evenly matched. It also tells us that the outcome is not so uncertain when the parties are not evenly matched. When one party is ahead by 20% in the polls a few weeks before the election the polls are rarely wrong.

The Norwood Fallacy

A corollary of the Neumann principle is what Jonathan Weinstein calls the Norwood fallacy: the idea that whether or not a coin comes up heads or tails carries some deep meaning about the nature of the universe. The example Weinstein gives is of the 1991 Super Bowl. The Giants won the game when Bills kicker Scott Norwood missed a 47-yard field goal. Statistically a professional field goal kickers misses at that distance about 50% of the time: so the Giants won because they had good luck that day. This - of course - did not prevent pundits from writing the next day how the Giants’ old-fashioned style proved superior to the Bills’ new-fangled no-huddle offense. Similarly we have the book by Victor Hanson Davis and published by the respected Hoover Institute entitled Why Trump Won. We eagerly await the sequel: Why the Coin Came Up Tails.

Next: Part 2 - Why Lobbyists Win