Tracking the Trackers, how AOL compares to the real world
Published September 17, 2004
Both Instapundit and Vodkapundit have made note of the odd potpourri of polling data that has been released in the last 24 hours. One poll, conducted by Pew, indicates Bush has a negligible 1% lead at 47%-46%. A Gallup poll released today shows Bush with a 13%-14% lead among voters nationally.
Clearly, something's not adding up. Steve Green opts to ignore both in favor of tracking polls, since, while said daily polls may not be as accurate, they are perhaps better at indicating trends. A post I wrote a few weeks back on the AOL Straw poll, a tracking poll which Bush is now dominating, both popularly and electorally, has been generating energetic discussion at Blogcritics.com.
Some suggest that the straw poll is meaningless, because it is not scientific. Sure, the AOL poll has its flaws, and I don't consider the raw numbers themselves to mean that much, but I think it does show some trending, particularly because over 300,000 people "voted" in it over a one week period beginning September 1. Since Bush now holds a commanding 65%-34% lead over Kerry in the AOL popular vote, and merciless 535-3 lead electorAOLly, many conservatives have been eager to trumpet this straw poll out to the masses. I'm not sure that's a fair, or for that matter wise thing to do (as I write for the second time in a few weeks about it).
How does the AOL Straw Poll compare to other trackers? For your visual enjoyment, I have created an overlay of the AOLSP and the electoral graph from www.electoral-vote.com . Here's what it looks like (KERRY in Blue, Bush in Red):

There are a lot of problems with this overlay. First of all, AOLSP tracks popular vote, which can shift in small increments. The EVT tracks electoral votes, which means that bigger shifts are likely. Additionally, Bush has never trailed Kerry in the AOLSP since about the 25th of June, so we're only looking for relative correlations.
From the end of June to the beginning of September, these trackers indicate the relative rise and fall of public opinion quite similarly. Take this overlay of Red on Red

Not perfect, but pretty close. Here's the Blue on Blue.

Again, not perfect, but pretty similar.
What do the polls mean? Less and less, and nothing when the chips fall in November, but that doesn't mean we'll stop taking them, or stop trying to explain them...
Take Kaus's explanation of the disparity of polling data we've seen just this week:
One obvious possible way to half-reconcile the divergent polls, suggested by
Rassmussen's robo-survey: Wednesday 9/15, when the probable Danron forgery began to sink in, was a gruesome day for Kerry. The Harris poll (Kerry up 1) stopped Monday. The Pew poll (Kerry down 1) stopped on Tuesday. Gallup (Kerry down 14)
includes Kerry's bleak Wednesday.
I'm not buying this explanation, particularly given all the talk of a resolute electorate. I'm with Green on this one...let's watch the trackers, at least they're prettier to look at. What do you think?
love it, hate it, there's more of it at pacetown.
- Tracking the Trackers, how AOL compares to the real world
- Published: September 17, 2004
- Type:
- Section: Culture
- Writer: Jeremy Chrysler
- Jeremy Chrysler's BC Writer page
- Jeremy Chrysler's personal site
- Spread the Word
- Like this article?
- Email this
Save to del.icio.us
Comments
Pollsters are in an interesting position, because unlike direct marketers or tele-marketers, they don't actually have to deliver. They just make all that shit up based on spurious data and methodologies, get paid, and skip off to fleece the next sucker, which usually is the previous sucker.
They do have the circle jerk going with the media conglomerates and PR firms, so it's an evergreen business. It's a great deal when you don't have to be accountable for results, but get paid up front.
Let's just go on and face the most basic fact. Our correspondent is relying on a fallacy:
. . .I think it does show some trending, particularly because over 300,000 people "voted" in it over a one week period beginning September 1.
We don't know that at all. The lack of controls means 300,000 votes may have been generated for the period, but we don't know why or how.
Bob has addressed the other analytical problems.
Bottom line: Give it up! The AOL straw poll is entertainment. The Right has latched onto it as anything but what it is. Going to the trouble of making up fanciful graphs based on phony data is worthless. Surely, one has something better to do with one's time. With a nod to Jim, masturbation would be a better pastime.
Wake up, everybody.
I think it's all pretty interesting - thanks Jeremy!
I am convinced that Bush will win every electoral college vote other than DC, which always goes Democratic because that's where the politicians live and everyone knows all politicians vote Democratic even if they are Republican, just like the journalists.
And immediately following that most vertiginous of landslides, I anticpate the pope and Queen Elizabeth will mud wrestle for a pack of condoms and a Milky Way bar.





Thanks for the detail, but why not just throw out a survey like AOL's whose findings would be considered statistical outliers compared to any other legitimate survey? If the "trending" is by some fortunate coincidence similar, the AOL poll becomes unnecessary. I doubt, for example, that a poll (like AOL's) that has Kerry losing his home state of Massachusetts -- an absolute impossibility since he leads there by 20-30% -- would reflect a similar trending upward for Kerry as we've seen in the Harris and Pew polls this week.
I've read that any online poll generally skews conservative regardless of the issue. I'm not smart enough in my sociology to venture why this might be, but I think that the discussion about how conservatives dominate short-form, quick-view media like talk radio or "political news" like Fox might apply to the Internet as well. Blogs are a good example of this: people can present stark viewpoints without editorial review with a modicum of research accompanied by flaming rhetoric. Certainly, it seems that conservatives (for whatever reason) are more motivated to vote in online polls -- I also think AOL's sample pool would skew significantly more conservative than the country as a whole.
You give this nonsense poll too much credit, especially for someone who's linking articles disputing poll methodology. AOL's is clearly the least scientific and least representative poll you've cited -- hence the 30%+ right-ward skew. That's why you'll never hear any legitimate conservative politician or analyst cite the AOL poll -- besides you, who are these conservatives you mention (bloggers don't count) that are touting the AOL electoral map? It might as well be drawn in crayon.
That is all.