A closer look at the polls, response bias?

Monday, September 8, 2008

Response Bias and the Shy Tory Factor

I have to confess that I'm having a hell of a time figuring out what to make of the post-convention polling.

Undoubtedly, the Republican convention has produced a bounce in John McCain's direction. This is nothing especially unusual. Almost all conventions do so, and in most (though not all) cases, the bounce evaporates at some point in the following weeks.

But some of the internal numbers in the polls really throw one for a loop. For example, take a look at these numbers from the new USA Today / Gallup poll. John McCain has moved into a 4-point lead in their registered voter numbers, and a 10-point lead in their likely voter numbers. I do not find the latter number especially credible, as the Gallup likely voter model is infamous for overstating the effects of short-term shifts in enthusiasm. Nevertheless, even sticking with the registered voter numbers, John McCain has received a pretty decent-sized bounce.

And yet, when this same poll asked voters whether the Republican convention had made them more or less likely to vote for John McCain, the results were fairly tepid. Just 43 percent responded with "more likely" as compared to 38 percent who said "less likely". That +5 score would make the GOP convention one of the least successful conventions in recent memory, trailing only the 2004 Republican convention which scored at a +3.

Could it be the Sarah Palin factor -- irrespective of the events of the convention itself? Certainly, this seems like it must be part of the story. But, as I've documented here, while Palin receives an unusually high number of strongly favorable ratings, she also received an unusually high number of unfavorable ratings. Gallup shows her at a +8 in terms of her effect of likelihood of voting for GOP ticket, not much different than Joe Biden's +7 (although with much higher numbers expressing an opinion on either side, as opposed to people who say her selection makes no difference).

My horse sense is that the numbers are affected to some degree to response bias. Republicans, especially evangelical conservatives, are pumped now, after having been indifferent toward John McCain for most of the election cycle. They may be picking up the phone when a pollster calls when they had been screening out the call before, perhaps to the extent that they are biasing the sample. In a recent SurveyUSA poll that asked people who had seen both McCain and Obama's convention speeches to rate the candidates on various issues, the partisan identification of the sample actually tilted Republican by a couple of points. Perhaps that is a consequence of the Republican and Democratic conventions having received roughly equal television ratings, but perhaps it is also a reflection of response bias.

more