WTF is Nate Silver's experience polling elections

Sure he got everything right in 2008 - the easiest election to predict - a washout against ReTHUGS.

I see you're still hedging your bets. Fuck you Nate. Go Dems.

Love your straight-to-the-point posts, my friend. :thumbsup:

http://www.youtube.com/watch?v=fHVBF93mkeI

Keep on chanting down babylon :D

need is a mango daiquiri and my toes in the sand! :D

but Silver predicted a lot more than that (the winner of every Senate race, the winner of the Presidency in 49/50 states, the closeness of NC/MO, etc.)

The really sad part is, to you, evidence is false if it goes against your preconceptions. No exceptions. There is no such thing as a piece of evidence that simultaneously goes against your preconceptions and is valid to you. That is a signal to not take you that seriously.

Democratic Polling Improves in Key Senate Races, Lengthening G.O.P. Takeover Odds

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=433x475946

everyone else's models. This isn't because Democrats are in good shape. Rather, it is because we have a very favorable set of Senate seats up for re-election, and more importantly, we have a very unfavorable set of Senate seats NOT up for re-election. If we had lots of seats in the mid-west up for re-election (as opposed to the coasts), the GOP would easily take the Senate.

It is the House where the GOP stands to gain huge numbers of seats.

Mr. Silver hasn't been exactly optimistic...

G.O.P. Stays on Upswing in Senate Forecast

Republican chances of taking over the Senate have improved again in this week’s forecast. They are now 22 percent — up from 18 percent last week and 15 percent two weeks ago. Republican chances are now approaching the point where they stood prior to the Delaware primary, when they had peaked at 26 percent before Christine O’Donnell’s victory.

There are six states currently held by Democrats where Republicans have at least a 75 percent chance of winning, according to the model. These are North Dakota, Arkansas, Indiana, Pennsylvania, Wisconsin and Colorado. Although some of these races — particularly Wisconsin and Colorado — could tighten, the G.O.P. has a fairly strong chance to win all six if they are having a good night overall. (They will probably win North Dakota, Arkansas and Indiana even if they are having a disappointing night.)

Although earlier in the cycle, it had looked like some Republican-held seats were in play, that is less of a concern for them now. Instead, some races that had once seemed competitive — like Ohio, North Carolina, New Hampshire and Florida — have seen the Republican candidate gain ground in recent weeks.

One partial exception is Kentucky, where some polls show the Democrat, Jack Conway, moving into a somewhat stronger position. It is not that Republicans couldn’t lose Kentucky — Mr. Conway has been running some strong ads lately — but it is unlikely to be the state that prevents them from gaining a majority.

Instead, it is a set of seven states that are likely to determine Republican chances to control the Senate. These are Nevada, Illinois, West Virginia, Washington, California, Connecticut, and the New York special election. Republicans would need to win four of these seven races to take claim of the Senate.

more...

http://fivethirtyeight.blogs.nytimes.com/2010/09/30/g-o-p-stays-on-upswing-in-senate-forecast/#more-1637

******************************


http://fivethirtyeight.blogs.nytimes.com/2010/10/07/gop-senate-odds-rise-third-consecutive/#more-1927

snip//

These Eastern states, however — ordinarily solidly Democratic — were always something of the rainbow over the horizon for Republicans: states that might have been winnable if Democrats were to suffer losses of catastrophic proportions. But those states were not the most essential to their chances to taking over the Senate. Closer to the frontier, instead, are a set of six states — West Virginia, Nevada, Illinois, Washington, Colorado, and California — where Republican odds are notably stronger. Were Republicans to win five of the six states, they would claim control of the Senate, unless Democrats were able to engineer a rebound in a state like Wisconsin, where they now appear to be underdogs.

He is not a pollster. He aggregates outside polling data together in a model. Last week, the Republicans had a 24% chance of taking the Senate, and this week they have an 18% chance. Is that really so stunning? The tipping point is clearly WV/WA/NV, and in 2 of those states, new more-favorable polling came in for Democrats. I would be more surprised if he DIDN'T lower the Republican chances. 24% and 18% are not high probabilities.

"The GOP taking over the Senate was always a longshot" is what you wrote, re: Silver. I was correcting you.

I realize his prognostications will change, hopefully in our directions. Appears to be going that way so I remain hopeful.

"always a longshot" wasn't a great choice of words -- "always unlikely" would have been better. But I still have no idea what your complaint about Nate Silver is.

according to the model. ;)

Dems are much more likely than repugs to have cut the land line and pollsters don't call cell phones. Just like Dewey when only rich repugs had telephones, the pollsters will be wrong big time.
Every body fusses about the enthusiasm gap. A repug that is enthusiastic gets one vote (sometimes) and I, while holding my nose unenthusiastically, will also get one vote.

It will not be nearly as bad as "predicted". Why would you vote repug? Do independents really think the shit that the wingnuts let slip will actually be good for them. People will remember what is was like under the repugs and reconsider.

"....evidence is false if it goes against your preconceptions."

And yes, it is really sad but echoes precisely the same head in the sand posture of our "progressive" President and his entire administration.

and decide the policy based upon data. For example, the individual mandate might not be popular, but that doesn't mean it isn't equally necessary. Looking at the data, all the states that got rid of pre-existing without a mandate saw either gigantic premium increases (relative to normal) or a complete collapse of the insurance market.

The same is true with the stimulus, FinReg, auto bailout, etc.

Rather than ignore data, they embrace it.

that causes bloggers to fail at determining what promises Obama ran on in '08. Want to discuss it?

So the ridiculous accuracy of all of his predictions and analysis and numbers was all just a lucky guess.

I'll bookmark this and check back with you after election day.

or some other excuse du jour.

after one election.

The fact that they all happened on the same day doesn't mean he didn't accurately predict 49/50 state Presidential election winners and 35/35 Senate election winners.

In statistics it doesn't really matter if he made 80 predictions in one election cycle or one prediction in 80 election cycles. The likelihood that he "guessed" right 80+ times is virtually impossible.

Try to predict 80 coin flips. Would it be easier to predict 80 coin flips in one day or 80 coin flips in 80 days or 80 years?

Me too.

He does the best job I have ever seen of laying out his statistical methods for everyone to see, be critical of, etc. If there is an underlying problem in the poll data that cannot be corrected for what do you want him to do about it? There is a place for a less statistical approach to poll analysis as well (i.e. based on intuition, political insight), I think we need both.

who parlayed his techniques into a name doing baseball predictions.

His weakness is that his technique relies on polls conducted by others. If there are not sufficient well-conducted polls with large sample sizes, then he produces models with incredibly wide margins of error--which makes his findings seem rather useless, if not ridiculous.

Of course he does. Each poll for a given candidate has a weight, which depends on age, sample size, pollster quality, etc. Some polls (such as old partisan polls with low sample sizes) have almost no weight in his model.

Nate. Good God. Why is it when those of us who work in statistics speak accurately about the limitations of the methods, some get so defensive?

He is dependent on the available polls to produce his models. He appropriately uses statistics and mathematatical modeling techniques, including weighting to attempt to address the inherent weaknesses of his "inputs". But, regardless, he is going to produce wide margins of errors because in many races, polling is infrequent and he is forced to use the available polls, no matter how small or flawed, methodologically to produce his models. THus, the wide margins of error.

I've seen models that provide ridiculously narrow "margins of error" with very little relationship to reality. It would be cool if Silver could magically obliterate uncertainty, but short of that, his honest efforts to measure uncertainty are very useful.

It is the reality that when you input the relatively few available polls polls (all that have very wide MOEs) to produce your model, you will get wide MOEs. It, frankly, is the "garbage in," garbage out principal of statistics. I don't fault Silver. That is the nature of the beast.

But, his honesty about his MOEs is expected. Have we really gotten so cynical that we expect others to lie about that aspect of statistics? :shrug:

However, numbers such as the trend estimates in TPMPollTracker don't include MoEs, so it isn't immediately obvious which ones are fairly robust and which ones Not So Much.

Also, as you know, uncertainty isn't just a function of sample size(s). An extreme example: Sam Wang and TruthIsAll were both way off in 2004 because they assumed that undecideds would break heavily to Kerry, and assigned zero uncertainty to the allocation of undecideds. Oops.

Hence, the entire premise of the original rant and those that support it in this thread is shown to be tragically ignorant - if not just abysmally inaccurate.

Nate Silver, finds his bones in the world of sports statistics - an arena where the only thing placing second to statistics in importance are jockstraps - is eminently prepared and qualified to do the work he now does in politics.

Nate is a very competent and good man who does not practice that which has become a profession amongst some: whistling past the graveyard of this "progressive" Presidency.

Mathematical modeling is a very useful tool for any number of disciplines. Unlike its use in baseball, where the input data are "hard data," he is dependent on the polling of others for his inputs. That doesn't make his methods any less appropriate, but it does effect his precision.

he thinks they are, he still has to rely on polling data from the organized crime syndicate. You know the majority of polling firms are owned by RepubliCONS.

But, has he considered this new report in his numbers?

"This year, according to today's report, the Pew Center finds that sampling only landline phones creates an even bigger bias -- "differences of four to six points on the margin" - in favor of the Republicans."

http://www.huffingtonpost.com/2010/10/13/pew-research-cell-phone-p_n_761760.html

I've also noticed his House numbers don't seem to be as up to date as his Senate numbers.

cell phone bias. He has talked at length about this before.

"You know the majority of polling firms are owned by RepubliCONS."

Bullshit.

"I've also noticed his House numbers don't seem to be as up to date as his Senate numbers."

He updates the House, Senate, and Governors once per week (staggered across the week).

Yes, Nate has talked about cell phone bias but the margin of error when not sampling cell phones has increased this year as opposed to last year. He needs to look at it again.

Your claim that most Polling corporations are NOT owned by RepubliCONS is so chuck full of facts. I mean who could argue with you when you so convincingly lay out all the evidence. Really a one word answer which happens to be a swear word is just so convincing -- NOT.

Currently the House numbers were updated 8 October, the Senate numbers updated 13 October. Every time I've visited his site the House numbers were older.

then as far as I'm concerned, there's nothing I can do to help them. So if you want to continue to believe that the polling industry is a Republican corporate conspiracy, be my guest. I would similarly not try to provide evidence of evolution to a creationist. YMMV.

He weights them based on HOW ACCURATE they are (how much error they had between prediction and results in previous elections).

Thus higher weighted polls ARE more accurate.

The only thing his model (or any model) can't account for is a change in accuracy.

Say you have a firm w/ really good accuracy/sampling/bias. Then new owners buys them and he starts biasing towards Republicans. The model is looking at historical data and sees the new poll as still "high value". The converse is also true.

The good thing is that since there as so many polls each individual poll isn't very "important" and as the accuracy of the pollset changes it will have more or less weight in the future.

If a large number of pollsters suddenly changed accuracy in the same election cycle it "could" radically throw off the results. Nate has even admitted that. This isn't a knock against his model. All models have to have assumptions and no model can account for all variables (like sudden shift in poll accuracy).

but Silver's claim to fame is based on one election - 2008. No doubt his accuracy was unbelievable but it was still one election.

One election is a meaningless metric.

80 predictions in one elections vs 1 prediction an election for 80 elections. It doesn't really matter.

The point is he used a statistically valid model (and even shared the formulas for his model, reasoning, and mathematical analysis) to make over 80 independent predictions.

The likelyhood of his level of accuracy base only on "luck" is somewhere in the ballpark of winner the lottery ... twice ... back to back and only purchasing 2 tickets in your lifetime.

We all acknowledged his accuracy in 2008, but remember my question here is about his experience.
We'll know shortly.

Having 59405845034 years as a pollster would be utterly irrelevant to the purpose of building a mathematical model to analyze multiple pollsters (and compensate for varying historical biases and accuracy).

It would be like determining how good your accountant is by how many years of experience he has watering plants.

In simple terms Nate is taking an "average" of what Pollsters predict. However just taking a straight average is inaccurate. It treats all pollsters the same. The good ones the bad ones would be treated equal. His model looks at historical data (past predictions vs past results) to hedge each prediction by historical bias then weighted according to historical accuracy (difference between expected and actual outcome) along with sample size, MOE, and other factors. That data is used for a simulation (imagine the election occurring in a million parallel universes) and the outcome of the independent simulations tallied to show the expected "outcome".

There is no mathematical flaw to his model (except on thing I posted up-thread). If his model is flawed they show proof it is flawed. Attacking the results is merely a childish rant and gets you no credibility from anyone who does this for a living.

I mean IF (and it isn't) he model is horribly flawed then analyze the model and point out the flaws.

Pointing out outcomes you don't like is well childish at best. No peer review is ever done based on the outcome. The outcome is merely the outcome.

Weaknesses must be pointing out in the methodology in determining that outcome ... to have any validity (which isn't really a requirement on DU lately).

Predict the outcome and marginal for all open Senate races this year.

356 didn't just say "Obama will win". He showed the projected win & margin for all 50 states (and was right on 49 of 50). He showed that many of the so called battleground states were nothing of the sort (and they were posting huge margins for Obama).

So easy right? So do it for all the Senate races (and if you feel lucky the battle ground house races too). when you get 90%+ accuracy in both predicting the winner and margin for >50 races well then I will agree it must be easy.

Dumb

and lots of lingo to surround what is basically a 'predict the future' sideshow. He's got the props and the phrases, and he makes some guesses and makes some money and gets some attention. Like all the prognosticators of history, when he's wrong, he mutters some nonsense about great forces and points at the handful of correct guesses, then cuts directly to the next round of Kreskin like predictions. It is an entertainment, which is proven by the fact that he takes this act to mass media every chance he gets. It is not about the 'numbers' it is about talking about the numbers, for a fee. First you have to make the numbers. They you talk about them. First you reveal the crystal ball, speak of your mysterious background and special gifts, then you use lots of language, and deliver the predictions. Never for say, today, always predictions off in the future, so that by the time they fail to come to pass, we are already talking about the new numbers, for the next show, I mean election.
Nate gets paid, and that is the entire point of such an act. He could do it with ping pong balls, a deck of cards, entrails, you name it, it is the same old dog and pony.

I feel stupider after reading that.

though it might be nice to think so, there is absolutely no reason to assume he's wrong now.

Every time I hear a prediction or opinion from Bill Crystal, I think to myself:
"This guy's NEVER been right about ANYTHING! Why should I pay attention to him NOW?", and I immediately dismiss his drivel.

On the other hand, when I hear a prediction or opinion from Nate Silver, I think to myself:
"This guy's been right A LOT. Maybe I should pay him some heed.", and I carefully consider what he's offered.

... on November 3rd.

Gpod day.

You don't normally do uninformed rants, ever.