Pre-Election State Polls and Exit Polls

Claim: TruthIsAll writes, "In 2004, the final pre-election state and 18 national polls closely MATCHED the corresponding 12:22am exit polls [which showed Kerry ahead by 3 points nationwide]." (If you don't know who TruthIsAll is, never mind: it doesn't matter to the argument.)

Facially, most national pre-election polls showed Bush ahead (see pollingreport.com). Most pre-election polls in Ohio showed Bush ahead there, too (see RealClearPolitics or the Ohio portion of the comma-delimited data file at electoral-vote.com [better saved locally than viewed in a browser]). But for present purposes, let's assume that, as TruthIsAll (TIA) argues, the pre-election polls properly understood pointed to a Kerry victory. By favoring "registered voter" rather than "likely voter" results, and by aggressively allocating undecided voters in Kerry's favor, TIA concludes that Kerry had a decisive lead in the pre-election polls.

If the pre-election polls and the exit polls both evince fraud that varies in magnitude from state to state, then both can be used to calculate crude measures of the fraud. Using TIA's poll numbers and assumptions, I computed surprise: the difference between Kerry's margin in the official returns versus the pre-election polls (as interpreted by TIA). Positive surprise means that Kerry did better in the official returns than TIA expected him to; negative surprise means that Kerry did worse. Given TIA's assumptions, most values of surprise are negative, presumably indicating that votes were stolen from Kerry. To measure red (or blue) shift -- the discrepancy between the exit poll results and official returns -- I use Edison/Mitofsky's measure of "Call 3 Best Geo" error, which incorporates their best exit poll estimates based on interview data. Negative Best Geo error (shift), like negative surprise, means that Kerry did worse than "expected" (in this case based on the exit polls).

The basic intuition is simple. If negative surprise (vs. pre-election polls) and negative shift (vs. exit polls) both point to fraud favoring Bush -- and if fraud is larger in some states than in others -- then surprise and shift are likely to be positively correlated. They might not "closely MATCH[]," since other sources of error and variation will influence both shift and surprise. But certainly one would expect that states with large exit poll red shifts would often also be states where Bush did surprisingly well vis-a-vis pre-election polls.

They aren't. If anything, states with large exit poll red shifts tend to be states where Kerry did relatively well compared with pre-election polls. In the scatterplot above, surprise is plotted on the horizontal axis: the states where Kerry did worst compared with (TIA's interpretation of) pre-election polls are at left, and the few states where Kerry did better than even TIA expected are at right. Shift is plotted vertically: the states with the largest "red shift" are at the bottom, and the relatively few states with "blue shift" (where Kerry did better in the official returns than in the exit poll projections) are at the top. Thus, the states with the largest fraud favoring Bush are expected to be at lower left, while states with little fraud (or fraud favoring Kerry) are likely to appear at upper right. A best-fit line would slope upward from lower left to upper right. In reality, an unweighted best-fit line (blue) slopes rather sharply downward; a best-fit line that weights the states by number of voters (black) is practically flat.

Whatever else one says, one can hardly say that the pre-election polls closely match the exit polls. The pre-election polls (as interpreted by TIA) suggest that fraud favoring Bush was greatest in such states as Massachusetts, Idaho, and South Dakota (states at the left of the scatterplot). The exit polls suggest that fraud favoring Bush was greatest in such states as Delaware, New York, and Vermont (states at the bottom).