How big a sample do these polls have? Big enough.
Last edited Tue Aug 28, 2012, 06:29 PM - Edit history (1)
It may be hard to believe that a national poll that samples only 1,000 people is a reliable picture of a nation of 300 million.
But, like may counter-intuitive things in math and science, a sample of 1,000 is plenty, and enlarging the sample would not make the poll dramatically more accurate.
The key is the quality of the sample, not the size. A low quality sample will not yield accurate results in any scenario. It is very difficult to have a reliable sample.
But a good sample will, as a matter of strightforward mathematical fact, yield reliable results with a surprisingly small sample... much smaller than intuition suggests would be nessecary.
Start flipping a coin and recording the results. You will quickly get the idea that the proportion of heads and tails is roughly 50-50. As you add more flips to the sample, each flip is less useful—one flip moves the percentages less and less as the sample of flips expands.
After you have coin flips down to around 50-50 (say, 49%-51%) there is little point to more flips. You will hone in closer and closer to 50%-50%, but 49.5%-50.5% is not a big improvement on 49%-51%. (And even if you flip 1,000,000 times it is quite unlikely that you will get exactly 500,000 heads.)
However many flips are in your sample, you express the results with a margin of error. Say your coin flip result is heads 51%-49% with a margin of error of plus or minus 2%. That does not mean that heads 53% or 49% are as likely as 51%. It means that the likeliest number is heads 51%, and that there is a 95% chance that the correct heads number is within 49%-53%. Both 49% and 53% are a good bit less likely than 51%.
Since our coin flip sample wasn't dead-on 50-50 the poll is not perfectly accurate, but it is accurate in what it says. It is very, very likely that heads = 49%-53%, which is correct.
And adding another 1,000 flips will not improve that result much. Say we get to 50.3%-49.7% with a margin of error of 1%. So it is 95% certain that the correct probability of heads is between 49.3% and 51.3%, and the likeliest result is 49.8%.
(I m making those MOEs up as examples. I don't know the MOE95% on 1,000 coin flips, but there is some number of flips with a MOE of 2% and a larger number of flips with an MOE of 1%.)
If our poll of flipped coins said that heads=50.3%, the most we can improve that headline number is 0.3%, even with an infinite number of flips.
One the other hand...the quality of the sample is almost everything. If you start out with a lop-sided coin that comes up 60% heads you will never get to 50-50. The sample is unrepresentative of coins.
Put 900,000 blue marbles and 100,000 red marbles in a big jar and stir them up. You won't have to pull out many marbles to get the idea that 90% of them are blue. But if they are not stirred up properly then you would have to count almost the whole million.
Unstirred marbles are a bad sample.
If a pollster assumes that black voters will be 7% of the electorate then his poll will be useless no matter how large the sample. If a pollster called only people who are home at 4 PM the result could never be a reliable snapshot of everybody, even if a million people were called at 4 PM.
If another pollster has a perfect sample of the electorate a national poll of only 500 people would usually be quite close to reality. But he would still want 800-1200 to have a low margin if error.
The mathematical parts of the science of polling are quite dependable. All of these poll samples are large enough to say what they say.
It is the composition of the sample that matters. And that varies a great deal from poll to poll. And there is little point in picking a favorite set of assumptions... in picking one pollster to trust... because no pollster always has the right assumptions.
And that is why averaging polls works better. It isn't because the sample is larger, but because the things that affect the quality of the samples are averaged out.
There will usually be at least one poll that is closer to the result than the average of polls. But we do not know which one that will be, and the average of polls will be more reliable than the typical individual poll.
= new reply since forum marked as read
Thank you for this 
