Many commentators have recently sought for political or professional purposes to discredit exit polls. In the past few elections exit polls have received unwarranted bad publicity. This is unfortunate because exit polls since the late 1960s have overall been extremely reliable and served us well, providing us with extensive information on how and why Americans vote as they do. No college class on electoral behavior could be taught without the valuable information provided by exit poll data.
In 2000, exit polls were attacked because results indicated that Gore had defeated Bush in Florida by more than seven percentage points. Yet, though Bush won the state, even the president’s supporters generally acknowledge that a plurality of Florida’s voters intended to vote for Gore. “Anomalies” such as flawed ballots and counting processes disproportionately disqualified Gore votes.
In 2002, exit poll results were never reported or released at all because the pollsters “lost all confidence in the polls,” perhaps due in part to discrepancies with official counts in a slew of surprising Republican victories that enabled the GOP to gain control of the Senate.
In 2004 exit polls were especially attacked because the Presidential election exit poll results differed from “actual” results like literally never before. In Ohio, a state that would have given Kerry the White House, exit poll results deviated from official results by eleven percentage points, that is to say, exit poll data indicates that Bush did not win by 120,000 votes, but rather lost by 500,000.
And even while the polls are being disparaged here, they are used throughout the world to verify the integrity of the count. In democracies with hand-counted paper ballots such as Germany and the United Kingdom, exit polls predict the outcome of national elections with extreme accuracy. Around the world, exit polls have been used to verify the integrity of elections, and discrepancies between exit polls and the official vote count have been used to successfully overturn election results in Peru, Serbia, Georgia, and Ukraine. At the same time that Republican National Committee Chairman Ed Gillespie was saying, "I would encourage the media to abandon exit surveys on Election Day,” the administration was paying for exit polls in the Ukraine to help ensure that any fraud committed would be exposed.
Foreign nations aside, pollsters and political scientists who have spent their career analyzing election results are highly resistant to any suggestion that the official tally is questionable. The official count is their North Star, without which their analyses are lost. In fact, when they are “wrong,” they try to “correct” their data and analysis so as to conform to the official numbers. Moreover, in today’s political climate, any suggestion of the possibility of a corrupted count will be attacked as a partisan bias, and if exit pollsters are perceived as partisan, they will never see another media contract.
Article comments
1 - Louis
Excellent, excellent article. I remember reading about his report after the election, but hadn't heard anything since. Anyone read the book?
2 - Elizabeth Liddle
Professor Freeman is of course right when he says that in the context of the evidence of voter suppression, magnitude of the exit poll discrepancy in 2004 (large enough to reverse the estimate of the final result) was, as Conyers' put it, "one indicia" of possible outright vote theft. And I myself read Professor Freeman's paper "The Unexplained Exit Poll discrepancy" with interest.
However, a number of explanations have since emerged that fit the data rather well, and, additionally, suggest that wholescale vote theft does not. Moreover, several of Professor Freeman's inferences do not stand up to close examination.
Firstly, Professor Freeman alleges, with no evidence to support his assertion, that:
"Many commentators have recently sought for political or professional purposes to discredit exit polls."
And adds:
"In the past few elections exit polls have received unwarranted bad publicity. "
I wouldn't want to comment on whether the polls received bad publicity, but it is clear from the pollster's own evaluation document (January 2005) that in every year since at least 1988, the raw polling data was "bluer" than the count, and in 1992, the discrepancy was almost as great as in 2004. There may be methodological reasons for this; there may be fraud reasons for this; but to assert that the reputation for accuracy that Professor Freeman now complains has been subject to "unwarranted" bad publicity would seem to fly in the face of evidence that the polls do not have the history of "accuracy" the Professor Freeman claims for them.
Secondly, Professor Freemans asserts that:
"And even while the polls are being disparaged here, they are used throughout the world to verify the integrity of the count." While one would not expect Professor Freeman to cite his source in an interview, I have yet to find any source that supports this assertion. As a UK citizen and election junkie I can tell you that exit polls certainly do not predict the results of UK elections with "extreme accuracy", and I have the scars (1992) to prove it. And I do not know of any country in which exit polls are, or have been, used to "verify the integrity of the count".
Thirdly, and most importantly, Professor Freeman states that:
"Those who defend the election say that Bush voters must have participated in the polls at a lower rate than Kerry voters did. "
Quite apart from the implication that those who are skeptical that the exit poll discrepancy was caused by fraud are coterminous with those who "defend the election", Professor Freeman's next statement:
"This presumption has never been substantiated by any data or even coherent theory" is simply false. In the Edison-Mitofsky evaluation document, data are presented that demonstrate that the discepancy correlated with factors likely to be associated with greater likelihood of departures from strict random sampling protocol. Moreover, previous studies, using actual experimental manipulation of methodological factors have demonstrated that they are associated with increased bias in the poll. It is not necessary to know why one group of voters are more likely to participate in a poll than another group: the fact that they have been observed, in experimental studies, to do so, tells us that this the case.
Professor Freeman's next point that:
"In fact, the data that has been made available by the pollsters not only fail to validate the presumption, they undermine it entirely" is also simply false.
He states:
"All indicators on poll participation suggest not lower, but slightly higher response rates among Bush voters " which is true, according to the Edison Mitofsky evaluation, although they also tell us that the difference was not statistically significant. But Professor Freeman then proceeds to say that:
"For example, if Bush voters were responding at lower rates, then response rates should be lower in precincts where Bush voters predominated as compared to Kerry precincts", which is not. There is absolutely no reason to expect this. Professor Freeman indulges in the ecological fallacy. What may have caused bias in the poll is differential response rates, not absolute response rates. Many factors are likely to affect overall response rates, including factors that are collinear with party support. But for any given response rate, bias will only occur if supporters for one candidate respond at a higher rate than the other, whether the two rates are 18% and 20% or 72% and 80%.
However, Professor Freeman appears to have entirely missed the point about participation bias. Non-response bias is only one possible cause of bias in polls, and not the one best supported by Edison-Mitofsky's data. Their data suggests not so much that Kerry supporters responded at a higher rate than Kerry supporters, but that Kerry supporters were selected at a higher rate than Bush supporters. This could arise from a number of factors involving the interaction between the interviewers and the voters; what is clear, however, from the data, that where there was greater opportunity for the selection process to be non-random, the discrepancy was greater. In other words: while non-random selection will not necessarily produce bias, if there is an underlying tendency for one group of voters to be more enthusiastic about participating than another (and we know, from experimental data, that this is true) it will tend to be translated into bias where opportunities for non-random selection are greatest. Which is what happened.
But finally, it is worth noting, that in a presentation Warren Mitofsky actually gave with Professor Freeman, he presented data that showed that there is absolutely no correlation, at precinct level, between the extent of the "redshift" in the poll, and benefit to Bush as measured by "swing" towards him relative to his voteshare in 2000. As the overall redshift in 2000 was (unusually) small, this is a major problem for the argument that the discrepancy was caused by fraud. Unless the fraud was uniform (which Professor Freeman himself argues is not the case - indeed it is a major part of his case that it was not) it is hard to square this finding with widespread fraud. What kind of fraud could produce a marked "redshift" in the exit poll, and yet no associated improvement with Bush's vote? Or, to put it another way, if Bush's improvement is uncorrelated with redshift - which it is - how does one argue both were due to fraud?
None of which is to defend an indefensibly undemocratic election. But I see no point in using the exit poll data to upscale the all-too-real evidence of abuse and corruption to allegations of hyper-fraud, when the data simply do not warrant it, and indeed, if anything contra-indicate such allegations. In fact, I think that making such unsubstantiated allegation runs the risk of diverting attention to the wrong set of problem, and, indeed, of discouraging Democrats from voting at all. There are so many real problems with your democracy that need urgent attention, including the fact that so much of your voting technology is opaque, insecure, unreliable and unauditiable, and the fact that so many of those very US citizens who would have most to gain from a Democratic administration are disenfranchised.
Chasing after hyper-fraud, and the hope that perhaps a Kerry won by millions of votes seems to me a recipe for avoiding the problems, rather than facing up to them.
Size matters.
3 - Nancy
I fully expect the Republicans to win the november elections, not through legitimate voting, but through massive voting fraud via electronic voting machines which have been preprogrammed to do so. Indeed, Karl Rove has openly stated his intention to keep the GOP in power via fair means or foul, including voting fraud, and I take him at his word. He should certainly know.
We need to invite the UN to monitor our elections, as much as any 3rd-world country.