Reliability of Zogby Interactive polling in 2006

http://online.wsj.com/public/article/SB116360961928023945-NgMgbTwNTEbcTx_C47luM8eH8lM_20071115.html?mod=blogs

<edit>

But the performance of Zogby Interactive, the unit that conducts surveys online, demonstrates the dubious value of judging polls only by whether they pick winners correctly. As Zogby noted in a press release, its online polls identified 18 of 19 Senate winners correctly. But its predictions missed by an average of 8.6 percentage points in those polls -- at least twice the average miss of four other polling operations I examined. Zogby predicted a nine-point win for Democrat Herb Kohl in Wisconsin; he won by 37 points. Democrat Maria Cantwell was expected to win by four points in Washington; she won by 17. (Zogby cooperated with WSJ.com on an online polling project that tracked some Senate and gubernatorial races.)

The picture was similar in the gubernatorial races (where Zogby polled only online, not by phone). Mason-Dixon's average error was under 3.4 points in 14 races. Rasmussen missed by an average of 3.8 points in 30 races; SurveyUSA was off by 4.4 points, on average, in 18 races. But Zogby's online poll missed by an average of 8.3 points, erring on six races by more than 15 points.

Zogby's online polls "just blew it" in Colorado and Arkansas governor races, Chief Executive John Zogby told me. (See Zogby's scorecard.) In other races, such as the two Senate races I mentioned, "we had the right direction but a closer race than the final." One explanation, he said, may be that Zogby's final online polls collected responses one to two weeks before the election, whereas other polling firms were active until the final week. "We have more work to do" to improve online polling, Mr. Zogby said, but he added, "we believe it's not only the wave of the future, but the future is very close to now."

more...

the odds were the DEMOCRAT would win, except in the mouth-breathing states.

I was right...

given the diminishing number of land line telephones. But it needs more work.

To me the fatal flaw is choosing to participate, rather than being chosen in a telephone poll and deciding to participate. There is more effort involved in an online poll. By telephone, you just sit there and answer questions. You're more motivated to do the online poll. You may just have answered the phone and started answering when the phone rang.

JMHO.

His server was always overloaded. Too frustrating.

Not yet.

Zogby does have a good record of predicting winners; but its hard to judge why that is. Could just be sunspotting, for example (sunspottings a polling technical term for not being able to explain cause and effect, but nonetheless making an accurate prediction from the data you have). Sort of like predicting winter comes every year , but never getting the average winter temperature right, and not being able to explain why.

nt