Exit polls 2004 discrepancies made quick & EASY for the math impaired !

http://www.oregonvrc.org/files/Timeline%20of%20Results_B&W.pdf

This is a timeline of exit poll data put together by the Oregon Voter Rights Coalition. I finally got it when I looked at this...
Most of the exit poll posts lose me, but this was easy to get and I finally saw why this is so incriminating.

Here for more, or for the PDF in color:
http://www.oregonvrc.org/2005/05/timeline_of_exit_poll_machine_tally_data_in_pres_race_election_day_2004

As Febble and I have pointed out repeatedly on this board, the folks who conducted the exit poll said in advance that they would weight the exit poll results to match the official returns, as they had also done in past elections. It is quite true that the exit poll results vary widely from the official returns. That leaves the question of why, which has received extensive attention here over the last year-plus, but isn't actually addressed in the timeline.

As for the vote totals being altered to more nearly match the exit polls -- migosh, I never noticed that claim before. Gee, I think that is just plain goofy. (I am pretty sure the graph is wrong, by the way: it depicts votes counted as of 5:00 PM Eastern, which makes little sense. I think that was supposed to be around 8 or 9 PM, which would make for even more dramatic slopes.) We're talking about a small fraction of the national vote, tallied from unrepresentative states; why should the percentage match the final national totals? Is there any state where the figures actually went backwards? That might be interesting.

The main difference between 13,660 and 12,219 is that the CNN release confusingly counts the absentee telephone respondents four times instead of one, inflating the count by 1500.

I'm happy to answer any other questions about this -- I think we've been through it all before.

"Did you know that exit polls have always been remarkably accurate, usually to within 1/2 of 1 percent (0.005%)?"

Yikes. First of all, surely it is obvious that 0.5% is not the same thing as 0.005%? Second, this simply isn't true.

"Did you know that those promoting the concept that exit polls are not accurate are the corporate-owned mainstream media and certain political leaders and pundits?"

WTF? Hey, Dad, I'm a pundit!

"Did you know that research and polling experts believe whole- heartedly in the accuracy of scientifically designed exit polls?"

Hmm. Well, if "irreverent" says so, it must be so! And so does Michael Keefer, Ph.D.! (His Ph.D. is in English, but whatever.)

Now, here's the problem with the graph (well, another problem with the graph) -- it strangely doubles all the differences. For instance, it starts with a "machine-tallied count" for Bush of +11 (meaning that Bush is 11 points ahead of Kerry) and for Kerry of -11 (meaning... right, Kerry is 11 points behind Bush). Which puts Kerry 22 points below Bush in the "Comparison of Candidates' Standing." What's with that?

Sorry to grump, but this is not very well done.

Yeah, how'd they do that? First I've ever heard on that matter. I'm guessing the machines were being tapped by who knows (Rove, the NSA, Diebold?) and somehow that info has been leaked. Certainly, anyone as connected as Mitofshy, et al, would have been privy to such info. It'd be nice to see a link to that.

The graph depicting the large move, in the later hours, shows quite clearly the massive alteration Mishofki embedded upon their previously unaltered numbers.

Interesting that all along Missofski said they would weight the exit-polls. But why? Why would they weight the polls after all the polls were closed? There would be no news at that time. The news orgs. would have already established a winner by then using official returns, rendering exit-polls basically useless at that time. Of course the Mistoffki supporters all along have said the exit-polls don't mean squat. So why did they get paid ten mill?

And what is Misofski hiding by not releasing the raw numbers...still, a year and a half after the fact? What are they hiding?

As far as I can tell, whoever prepared the graph just entered the info under the wrong time. But hey, if anyone has any screen shots of vote totals being reported on CNN at 5 pm ET, that would certainly be entertaining.

You already know the answers to your questions (as you know how to spell Mitofsky) -- you just don't care. If anyone else does, let me know.

Answer the questions. There are a lot of new people who would like to hear a lucid answer to those questions, and you are the expert, eh?

Otherwise folks just might presume you are hiding things. You wouldn't want folks to think that, would you?

Besides, there was a new twist that we never discussed: What good would weighting the exit-polls be to the people who hired yall to give results before the official numbers came in from around the country by 11pm? IOW, Why weight the raw numbers afterwards when the official results were already available?

I don't know what you think I was doing on Election Night, but I was home getting election returns off the TV and Internet, like a lot of other folks here. I wasn't hired to do anything.

Well, gee, why does the Washington Post print the results of the exit polls? By your reasoning, they should be useless at that point, since everyone already knows who won. Why do newspapers actually subscribe to exit polls when they aren't even on the air election night?

Umm, because there is other information in the exit polls. But you knew this.

So, why do news outlets want weighted numbers? Well, what is the point of a table that shows Kerry winning if Bush actually won? If you open your mind to the possibility that polls can be wrong, that might actually make sense. I will not hold my breath.

E/M has released many, many thousands of raw numbers. But you knew this, too.

The exit-poll numbers you refer to were the weighted numbers. You still haven't answered the question of why the numbers were weighted. But then you say: Well, what is the point of a table that shows Kerry winning if Bush actually won?

The point is that the $10 million dollar collection of numbers is useless if it was not accurate, and you do realize they weren't? So, the $10 million was a waste, you seem to be saying, but not if they matched the official results, only then would it be worth something? You want it both ways, I'd say.

Oh, I believe polls could be wrong. But if they were as wrong as you claim, then they were worthless, and any further support thrown that way is tantamount to utter deception.

Still the unanswered question is: Why weight the numbers AFTER official results were available? Or is your answer: They had to alter the numbers so that they could get another contract in four years? Or is it that they would have been laughed off the air by showing such wrong numbers?

I know you are not saying the raw numbers were right, I know. Heck, even you haven't seen the raw numbers, right? How could you know, if you'd never seen them? And tell our audience again, please, just why the raw numbers are not public, OK?

No, dude, you are not thinking like a reporter, or a scientist, or like any form of inquirer into reality that I can think of. One doesn't simply categorize data as Accurate or Useless; one assesses them.

Do you really think that as far as we know from the exit poll data, Bush probably won the youth vote by 80-20 -- or Kerry won the youth vote by 80-20 -- because the data are not Accurate and therefore are Useless?

"Still the unanswered question is: Why weight the numbers AFTER official results were available?"

No, I answered the question, but my additional answer in the other thread may be helpful: if the official results are accurate, then weighting to the official results is likely to get closer to the actual cross-tabs. (It may seem to you that pollsters should always assume, when confronted with a discrepancy between their results and the official returns, that the official returns are wrong. For better or for worse, pollsters are not that megalomaniacal.)

I've seen the same raw numbers you have, if you have downloaded them from the ICPSR FTP site (which I admit that I doubt you have the gumption actually to do). Some 70,000 individual interviews. What I haven't seen is precinct totals linked to precinct identifiers. That information (1) is proprietary whether you (or I) like it or not, (2) could allow individual respondents to be identified and their votes to be revealed, and (3) wouldn't prove much anyway. (If you have a strong argument that it would, don't waste the argument on me -- try it out in court.)

Obviously not.

One doesn't simply categorize data as Accurate or Useless; one assesses them. You said the numbers were useless to establishing an audit. You've said it would be pointless to use numbers that contradict the official results, not me. And the best we get from you is: "Don't believe the numbers until we changed them after an assessment of two hours." Man, that's some assessment. That's why Mussorgsky gets the big bucks, eh?

It may seem to you that pollsters should always assume, when confronted with a discrepancy between their results and the official returns, that the official returns are wrong. Even faced with the number altering tabulators, and the corruptible DRE's, you seem to think the pollsters should just throw up their hands and say: "Our numbers must have been wrong, oh well, we'll just adjust them to the official numbers and everything will be ok."

Do you realize the hole you are digging? Here, let me hand you a shovel! LOL

Your first paragraph here is basically made up, although in fairness, you might actually not know enough to realize it. Although surely you realize that your quotation is entirely made up.

I don't assume that either the polls or the official returns are correct. There is not much point in debating what "the pollsters" assume -- it's you and me. You have no basis for assuming that the polls are correct, which is perhaps why you are blowing smoke.

The polls are then useless if I have no basis for assuming the polls are correct. That is what you are saying. It is deceptive for any one to say such a thing and then try to protect the pollers or the official returns. That's plain idiotic, and we know you are not an idiot, so why do you continue to protect the pollers? And why should I not assume either one was correct or right?

So you don't "...assume that the polls or the official returns are correct." Yes, lets's not assume they are doing their jobs?

Just cause you are cornered, you say it isn't my field. Up against the barb wire and all you can say is that it isn't my field? How utterly republican of you.

I've got you cornered that really IS all you can say.

What you've got is an unfounded conviction that I am here to trick you, or something like that.

"The polls are then useless if I have no basis for assuming the polls are correct."

No, it isn't. Return to the message where I reject the dichotomy between Accurate and Useless, read it several times slowly, and...

well, I guess we're done. If you ever are interested in serious discussion, I am not hard to find. If you just want to shake your fist at me, hey, go wild. I don't blame you that it isn't your field, but I admit that your willingness to be dogmatic about it anyway depresses me.

I do convict the cooking of the exit poll numbers after the election was over. The graph presented in the OP shows the cooking.

I do assume that the exit-pollers did a good job of compyling the numbers, heck, that's their profession, their job; it's why they get paid the big bucks.

I do not think they had anything to do with the fraud, only that they tried to hide the evidence of fraud.

Now, if you really want to discuss this, and stop your flaming, personal attacks, I'm all ears.

Oh, I see you ducked behind the curtain of prior posts, once again.

Here is the situation:
1) E/M has a public duty to make an honest attempt at clearing up what happened.
2) E/M is hiding the data which could help clear this up.
3) Obviously, E/M has hired shills to help them keep the data hidden.
4) Therefore, one has to conclude E/M is not being honest.

But what the heck, it's not like anyone has died, yet, or anything.

Here is the actual situation:

You think I am wrong (and, apparently, a shill).
I think you are wrong (and I know that I am not a shill).
E/M does not care what either one of us thinks.

It sort of hurts, doesn't it, to admit that it is that simple. Given the time we have spent on this, it would be nice if it mattered.

Has E/M paid you for your efforts?

Oh, it matters. You may say it doesn't, but it does.

One way that it matters is that E/M will never again be trusted, and trust is BIG.

But at least no one has died, yet.

On edit: Is it true you are the author of a book on how to manipulate public opinion - or something like that?

http://www.perseusbooksgroup.com/westview/author_detail.jsp?id=1000001270

:shrug:

that I think you have just broken a DU rule, I completely fail to see the relevance of your link to anything in this thread.

Did you have a point you wanted to make?

I guess my point would be those who look for the truth have no reason to be deceitful.
Thanks for asking.

And no, I haven't broken any rules.
A) He has posted his own website with his name and place of employment numerous times.
B) Authoring a book is not personal information
C) I found that last year when I was doing research, I did not 'dig-it-up'.

"Is it true you are the author of a book on how to manipulate public opinion - or something like that?"

His answer was 'no'.

This is the publishers description of the book...
"Public Opinion is a comprehensive and interdisciplinary examination of public opinion formation and change."
http://www.perseusbooksgroup.com/westview/book_detail.jsp?isbn=0813341728

Now....whats your point?

The book surveys the entire field of public opinion -- how it is formed, and how it changes (including in the course of political campaigns). It is not "a book on how to manipulate public opinion." Or, if any book that discusses public opinion change is "a book on how to manipulate public opinion," then that pretty well covers the profession. But somehow I don't think that was the, umm, point BeFree intended to make.

Which amazes me that you chose to answer with a definitive no.

Never mind the fact that Febble chose to support your answer given it
was obviously inaccurate.

I guess that 'stand up for your colleague' mentality runs deep.

of standing up for someone who writes an apparently good book on public opinion research when it is snidely referred to as a book on "how to manipulate public opinion", colleague or not.

I don't think his answer was inaccurate in the slightest.

If anything, a book that explores "the nature of political and social attitudes in the United States and how these attitudes are shaped by various institutions, with an emphasis on mass media" is likely to be important reading for those who want to be on guard against having their opinions manipulated.

"No" was an entirely appropriate answer to BeFree's unpleasant question.


(edited for yet another mistype. I must be cross or something)

yours, or rather BeFree's original point, which you seem to endorse, was.

It isn't a book on "how to manipulate public opinion". Or even "something like that", any more than a forum about election fraud is a forum about how to steal elections.

It seems to be an undergraduate text (do US colleges run undergraduate courses in "how to manipulate public opinion"?) that "is simply the best book published on public opinion today. No other authors do a better job in covering how public opinion is formed, measured, and used in democratic society. Scholars and students of public opinion will love reading and learning from this superb book."

I might buy it.

(edited for clarification as to who was making which point, which I still, frankly, don't get).

"The authors explore the nature of political and social attitudes in the United States and how these attitudes are shaped by various institutions, with an emphasis on mass media."

I can see it's 'nothing like that',...absolutely nothing even remotely close.

If you have a good reason to pile on in an attack on my integrity, could you kindly find a way of letting me know what it is?

The book is not a book about how to manipulate public opinion. I think you know that.

BeFree posted a very snarky post. You endorsed it. I defended OTOH. Now you seem to be disowning BeFree.

And no, the book is "nothing like that", as should be obvious from your own quotation from the blurb. See my other post. A book about how opinions are shaped is not a book about "how to manipulate public opinion". Rather the reverse, I would expect.

If you want a book about "how to manipulate public opinion" I suggest you try some text book on advertising techniques, if there is such a thing. I expect there isn't. I don't expect its stuff we mere mortals are supposed to know about. Which is why the book you link to is probably rather useful.

(edited for mistype)

besides the one where, say, a general history of the 20th century is not a book about how to commit genocide.

I am junior author among five, and the parts I primarily authored are probably among the least helpful for figuring out how to manipulate public opinion.

(nor did I defend someone elses characterization of it)
I called his answer what it is....innaccurate.

I will say definitively, yet again: the book is no more about how to manipulate public opinion than a general history of the 20th century would be about how to commit genocide. Or, in words of one syllable, no. Is this hard? No.

Bye.

I did not characterize your book.
If you don't like what your publisher wrote about your book...talk to them.

Simple enough? I think so.

this isn't like you. BeFree characterised the book as "how to manipulate public opinion". Which was not only nasty, it isn't a characterization of the book by any stretch of the imagination, and is not how it is characterized in the publisher's blurb.

If you don't want to characterize it like that, fine. OTOH is happy to acknowledge his authorship on a book about public opinion research. But BeFree didn't ask him whether he'd authored a book about public opinion research, he asked him whether he'd authored a book on "how to manipulate public opinion", and OTOH was quite entitled to answer "no" to BeFree's question. A book about public opinion is not even "like" a book about how to manipulate it.

Message removed by moderator. Click here to review the message board rules.

I have made no attempt to "discredit any and all election problems and discrepancies" -- you made that up. Check your facts and try again?

I think linking to progressiveindependent is against board rules, but whatever.

TruthIsAll is not a statistician; he is a self-described "Excel expert." You could trace my credentials if you care, but since you are just ranting about "fancy statistics jargon," I assume not.

I've read that talking point before about who is promoting the accuracy of exit polls. I don't know who made that one up. TIA himself has posted that one of the presidential exit polls in 1988 put Dukakis ahead; does that make him a political pundit, a politician, or a troll? (Ironically, that appears to be a "raw" result and not the actual projection of the exit poll.)

Who are the professional researchers and statisticians you refer to, and why do (e.g.) Mark Blumenthal, Walter Mebane, and Fritz Scheuren not count? Have you ever tried sampling any, oh, twenty conference papers about the 2004 election to see how many of them argue that Kerry actually won? Do you actually have any basis for believing that professional researchers and statisticians generally accept your views?

What makes Sheldon Drobny a reliable reference? He appears to be a CPA by training, which is admirable, but doesn't make him an exit poll expert. Zogby links to articles that mention him. If you think Zogby endorses this view, why don't you ask Zogby to endorse it? That would be interesting.

Anyway, why is it impressive to say that the reliability "can be" one half or one tenth of a percent? The real question is whether it was.

If you have honest questions, or even credible sources of information, I am happy to work through them with you. If you are all hat and no cattle, that's your prerogative, I guess.

The numbers were weighted because:

a) one purpose of the poll is to predict the counted result reliably, so that a state can be "called" even if not all the votes are in
b) another purpose of the poll is to enable people to figure out who voted for whom and why, and as pollsters do not share your faith in their polls (but do, possibly mistakenly, have faith in the counted result) they weight the poll to the count, not the other way round.

The "collection of numbers" is not useless: it fulfils both its stated purposes. It correctly predicts the counted result (before the full result is in); and even if the count is wrong and the poll is right, the weighting adjustment does not hugely change the values in the crosstabs. In any case, the weights are provided in the raw data, which is indeed available, and you can ignore the weighting if you want to.

What the exercise does not do, of course, is give you an estimate of the result derived from the poll that is uncontaminated by vote count returns (although on this occasion, E-M did actually release these estimates in their January evaluation). So they don't do the job you would like them to do - instead they do the job they were commissioned to do.

However, even though they were designed to do a different job from the one you want them for (to provide a check on the count) it is arguable that they can be reverse-engineered to do this. OTOH and I have both worked pretty damn hard to find ways in which this might be done. For all your sneers about statistical jargon, it's not as easy as it looks.

As for your question: "Why weight the numbers AFTER official results were available?" - well, part of my reason is given above. If we assume the count is correct then we will get better estimates from the crosstabs. So that is why. If you don't believe the count was correct, then obviously that won't suit your purpose. But it is perfectly rational. But the other reason is that it was a dynamic processes - as the vote-returns come in, the projections are weighted to the count. The exit polls had Bush winning long before all the counts were in. Sure, that might because there was massive fraud.

Here's an analogy: at the beginning of a race, the bookies predict that KerryMeLad is going to win; and BushyBoy is going to come second. Half way through the race, KerryMeLad slows up. The bookies shorten the odds on BushyBoy. Eventually KerryMeLad is so far behind that BushyBoy is a dead cert. So they close the book. Even though the race isn't actually finished.

Afterwards, everyone suspects that KerryMeLad was nobbled. So that throws the whole question as to who really won wide open. But it doesn't render the bookies' actions irrational.

As you say, the polls could be wrong. So, most of us around here believe, could the count. We want to know which. Frankly I don't find that circular firing squads help much.


Oh - and yes, both OTOH and I have seen the "raw numbers". So can you:

ftp://ftp.icpsr.umich.edu/pub/FastTrack/General_Election_Exit_Polls2004/

The cookery recipe (marked "weight") is also provided. Salad or ragout, it's your choice.

"The "collection of numbers" is not useless: it fulfils both its stated purposes. It correctly predicts the counted result"

Yes. We can, and should, assume that the polls correctly predict the counted result. Thank you. Someone else told me I shouldn't assume any correctness from the polls. Oh, all bets are off once the race has started.

although I am not clear quite where they are crossed.

What I am saying is that the entire poll exercise, including the reweighting, fulfilled its stated purpose, whatever you think its purpose should have been, which was

a) to predict the official counted result (with, if necessary, the help of the incoming vote returns), and thus enable the networks to call the states in advance of the full count.
b) to provide information as to the characteristics of who voted for whom.

I was NOT saying that the UN-REWEIGHTED polls correctly predicted the uncorrupted vote (if that is what you thought I was saying, I'm not sure). They may have done. But they were not particularly expected to (which was why the reweighting component was included), because participation bias is an ever-present hazard with polls, and some reweighting is normally done (a considerable amount in 1992, for example).

Our task, as I see it, is to figure out whether the unususally large re-weighting required in 2004 was required because non-participation bias was unusually large, or the count was unusually corrupt. My understanding is that that is what we are debating. I don't happen to think the answer is obvious.

see, you hang around us long enough with an open mind and you can make progress, eh? Good on you. We've a ways to go yet, but your getting there.

Where, out of interest, do you think I started, if you think my present position represents progress?

This is not (mainly) snark - I really want to know. I get a bit fed up of being painted as the bad guy. The entire reason I'm here on DU posting about exit polls is that I couldn't believe that Bush could actually have won.

I think you've progressed, don't you? Where do you think you were when you started? It really isn't my place to get anymore personal.

I have learned a lot on DU: all about exit-polls, the DREs, Opscans, and HCPB. Most of it was constructive and valuable information. The bad news is the the Opscans and DREs are kin to rattlers coiled up in my sleeping bag, and the exit-polls stink like a pile of something on the trail. How 'bout you?

but where I was when I started was convinced that Bush couldn't really have won. I'm afraid I now think it is likely that he won the popular vote, and I'm on the fence over Ohio.

But, yes, I've learned a huge amount on DU, and I am much MORE convinced than I was at the outset that the election was both corrupt and unjust, and that even if it wasn't, it is criminally vulnerable to corruption. As a Brit, I still find it hard to accept that hand-counted paper ballots aren't a realistic option - it's so cheap, so simple, so easily verifiable.

I don't know if you call that progress. But I'm with you with the need for reform. I suppose at bottom, the main difference is that I'm not as gloomy - I think the Republicans did all they could to cheat, including simply telling lies. But I don't think they actually managed to switch the result. Which, if I'm right, is good news. It means you can still win. The other bit of good news is that they know you guys are on to them. It is going to be a heck of a lot harder for them next time.

Peace.

"It is going to be a heck of a lot harder the next time"

No, its in the bag. The DRE's will steal it again. The software will be programmed to switch the vote again, and it will be easier because know-it-alls who could have called them on stealing the last one work with them to let it happen.

I have no recourse but to blame those who work in the business of counting the votes, whether it be election officials, vendors, or those who made money by helping to hide the fraud. It is on their shoulders, this corruption of democracy. A democracy, I will remind you, that many a soldier has shed blood for.

The simple crux is that the motive and opportunity was there and coupled with the history of the criminals equals an indictment of the whole bunch of professional electioneers, and no amount of fancy wording and claims of expertise from any of those involved will whitewash the stolen election.

Try this link:

http://www.exit-poll.net/faq.html#a10

Amaryllis - no offense friend, but I'm really surprised you would start this thread. Over here in another thread you went out of your way to agree with my reply to Sperk's request for talking points in preparation for his town hall forum. I wrote:



and Amaryllis replied:



Threads like the one we're in (as opposed to the older one I just quoted) are some election forum equivalent of flame bait. It doesn't mean nasty hate-filled flames, exactly, but an analogous form of useless he-said/she-said contradictions. This doesn't get us anywhere except to confuse matters worse than before and validate my point that we can never know the true outcome of an "election" held under current conditions.

OTOH and Febble - you guys bring your own logic and analysis which I do not wish to censor. I also do not wish to prompt or provoke more of the same from either of you as it does nothing to further our collective ability to resist this fascist tyranny. Simply put, I ignore you, will continue to do so, and suggest a moratorium on threads like these that tend to inspire your participation.

It is your right to ignore threads like this. But Amaryllis is not wrong to post this, or anything Amaryllis wants. We are not afraid of a debate, are we? This is a discussion board, and that is what we are doing. You see, those posters are supporters of the exit-pollers and we need to hear from them if we are to have a discussion about the exit-polls on this discussion board.

Sticking fingers in our ears is no solution. But comments are appreciated.

for appreciating my comments. I'm not saying Amaryllis is wrong or not entitled to start threads like this. I'm simply advising against it, and reminding Amaryllis that she *seemed* to share this view in another recent thread.

Open discussion is the point of this site, definitely. But some topics have been played to death, to the point where they are counterproductive. This is one of them. No one should feel prohibited from starting another thread like this, but put in context, perhaps it will seem to more people to be inadvisable.

I get your point. Where have those two been? Just waiting in the wings for another chance to expose themselves, I guess. And this thread did leave the door open for them.

As for counterproductive.... it is their fault for the counterness, but once they expose themselves they have to be beaten back so that newbies will not be misled. I only hope that I have argued our case sufficiently.

Whilst well aware that such discussions can tend to turn some folks off, and since I truly want folks to tune in and turn on, I hope no one has been turned off. If so, I apologize.

as well.

in exit poll discussions might find it interesting; it made the exit poll issue understandable for me when it was all Greek to me before, and there are valid reasons to post threads about info that we would never consider using as talking points, and I didn't start this one with the intention that the info would be used as talking points. The majority of threads in this forum are informational rather than intended to be talking points. I don't see how this in any way contradicts or conflicts with the opinion I expressed in agreement with you in the other thread.

If you will notice, I have ignored getting into the discussion on this thread because I consider it a waste of time (wiht the exception of your post to which I am now responding), and certainly would not start another thread on this topic after seeing what happened to it. I didn't know it would be flame bait since I normally don't read the exit poll threads becasue they lose me so fast! Argument for the sake of argument bores me in seconds and I don't get involved in it. Waste of time, also

And I completely agree wtih your statement: "I also do not wish to prompt or provoke more of the same from either of you as it does nothing to further our collective ability to resist this fascist tyranny. Simply put, I ignore you, will continue to do so, and suggest a moratorium on threads like these that tend to inspire your participation."

Yes, the graph does explain better then words what happened with the exit-polls, and I thank you for bringing it to our attention. Glad to hear I did not turn off at least one person. LOL.

Don't be shy about telling it like you see it. Never, ever keep quite because you think it might get someone's hackles up, at least not on DU.

We need discussion on the exit-polls because it is the chief bit of evidence linking the secret vote counting and Kerry not getting in the white house. Of course, there is other evidence out and about, but the polls decisively enumerate the discrepancies.

Thanks for bringing the graph to our attention. It was bookmarked.

:) I just see the discussion on this thread RE exit polls a waste of time.

I helped the poor dude dig himself ever deeper into the hole he belongs, hopefully all the way to china, with the intent of destroying whatever credibility he had, and you say i was wasting my time? Sorry you feel that way. Oh well.

I do not believe that posting misinformation does anything to counter fascist tyranny, either. I will not apologize for pointing out that it was misinformation. There is absolutely no evidence that the vote counts were jiggered to more closely match the exit poll results!

It seems very clear to me who wanted a flame war here, and it was neither you, nor me, nor Febble.

I will, I'm afraid, participate in any thread that appears to offer misleading information, as I consider the information linked to in the OP is.

Sure, I'd rather misleading information didn't appear, and sure I'd rather talk about something else, like figuring what needs fixing about your democracy (a lot), and how to fix it, although I've probably done what I can on that score, not being in America. But I also happen to think that bad (and demonstrably bad) information about the exit-poll discrepancy actually hampers "our collective ability" to do anything constructive about fixing it, especially if it encourages people to think it is not worth attempting to vote the Republicans out.

BeFree: there are rules in this forum about not being a supporter of Republicans. There are no rules, as far as I can see, about being supporters of "pollsters". It's a profession, not a political ideology. I'm not a pollster, but I am a social scientist, so I have a fair bit of understanding of the limitations of survey data. I have also acquired a fair bit of understanding about the methodology of the E-M exit polls.

Does that disqualify me from debating the interpretation of the exitpoll-vote-count discrepancy?

And to be clear: I am not, in any case, a "supporter" of "pollsters". I am a supporter of analysing data objectively. It's called science.

I'm a social scientist, but not a pollster. I disagree with your assertion here. First, availability of all the data-including unit analysis that tracks issues is clearly part of data. It's not "scientific" that some level of the data is not released. As a "privacy" issue, this is easily fixed.

More importantly is the convergent validity of variables associated with the data: Were there "predictors" in the surveys (demographics) that appeared unlikely? Were strategic states/counties more in error than less important ones? It's the missing level of data that prevents these associated analyses.

I live in Florida and witnessed a "voting machine" error where a voter complainted loudly that the machine wouldn't count her vote correctly. At the same time, I saw "impossible" results, where Democratic issues on taxes and local races were completely unrelated to a single race - for President - and there was no explanation. Science would really need to see the poll data for those groups to coordinate the annecdotal with the empirical.

I don't think we can be a scientist on a subset of an issue. If I were a consultant to E-M, and I've consulted on many measurement projects, I'd have to say that this is likely a solvable issue if we really wanted to find sources of error in the discrepancy. It is a question to me why pollsters don't do that unless they already know the answers...

Folks here have gone round and round on the privacy issue -- many survey researchers believe that it is not that "easily" fixed. I'm sure you know that many survey data sets are not fully released due to privacy concerns. It is also true that E/M do not treat their data in all respects as academic researchers do.

But if you wonder why there is no evident campaign by political scientists to press for release of the data, well, it may be because error-riddled data (and really, there is no plausible way to assimilate all those results to unbiased sampling error plus vote miscount) from some 30 to 50 precincts per state (Ohio has over 11,000 precincts) doesn't give much leverage on the questions you suggest. Strategic counties??

If you have tinkered with the ICPSR datasets, you probably already have some intimation of these limitations. This is, essentially, the point that the SSRC commission on elections and voting made here (numbered pages 2-6; note what disclosure proposals were and were not made) and here.

I think many of us feel in our guts that if we could do the poke-and-soak with the data, somehow The Truth Would Out. But what I have observed over the last year is that very few people actually propose testable hypotheses, and when they do, the results are generally null -- and we know more about the limitations of the exit polls than ever. The actual forensic progress AFAICS has come from analyzing other data sets that are much more detailed. Once or twice a week I really want to have all the data, but I have no reason to believe that it would transform my understanding. And it blows my mind that some folks who talk a lot about wanting access to the exit poll data don't seem especially interested in election data. Think of everything we know about Florida 2000; how much of it came from exit poll analysis? Weird, just weird.

Don't get me wrong: if you can propose some testable hypotheses, please do! I think that fewer abstract demands for data release, and something more like a research agenda, would be more likely to encourage interest and some form of cooperation from the various parties at the NEP.

Without revealing who it is...take the polls to a local election (did you vote for Castor or Martinez; did you vote for the local school tax referendum) for about a congressional district. Test if the polls are accurate on races that would be unlikely or impossible to manipulate with "tabulator hacking" etc. Null is that local votes are the same accuracy as the Presidential races.

Also, test if reported demograpics predict (black, white, hispanic, rich, poor, young, old) predict with the same accuracy for (again) different races. This is testing one regression line (or SEM) as "significantly different" than another.

Null: Error at the poll level is equal to the district is equal to the state is equal to the nation. MANOVA or Discriminant - all possible races with and without Presidential.

Null: Reported poll errors ("my machine didn't work") has no Spacial or Geographical pattern where error differs from demographics (like Jews voting for Buchannon in Palm Beach) that don't make sense...

Time series of "changes" in Presidential polls compared to other recorded votes and races....

etc...etc...

I don't have access to the data and haven't planned any of these specifically, but unless the pollsters are idiots, didn't train anyone, didn't have a form of specific questions - all these are possible. In my work, even locals marketing pizza or asking questions for the school board would have all the above variables. Small samples would not be the only issue of power in the designs - what's important is the confidence/error attributed to a given unit of analysis moving from the individual to the whole data set...would you think that 12 local votes would be accurate within a certain margin but all those people lied only one one questions? Would demographics predict within a certain value except for the presidential races? Would error only occur where Diebold tabulators are used, or in "key" districts, or at "certain times of day"?

TIA likes to use normal distributions to test for probable differences, but error on the "same technique" should change (decrease) systematically as the sample increases...does it? I'm sure "pollsters" have their methods for a reason, but in this case I've seen very little analysis except the debate between USCountVotes and E-M. There's pretty good evidence that poll error seems unexplainable at the state level for Ohio, Florida, and Pennsylvania. That looks convincing to me. I don't see why, except for lack of precinct identiy - which would allow all the other comparisons - that this is not an analysis that's doable.

One good start would be to look at the questionnaires -- well, in fact, one good start would be to read the evaluation report. It would not take that long (although it is the sort of document that one comes back to), and it will help you.

For instance, upon reflection, it will probably not surprise you that the 2004 exit polls did not ask questions about "local school tax referend(a)" that could be used to calibrate their accuracy. (Even if such a question were on the survey, it would be rare to have more than one precinct per jurisdiction. Even the sample in any one House district is not going to give much statistical power.) But we can do this with Senate races. However, your null is probably not warranted. For instance, if the Senate race is much less competitive than the presidential race, then an equal degree of non-response bias seems likely to evoke a smaller percentage error in the Senate race, unless the bias is heavily concentrated among the straight-ticket Republican voters. Another problem with the null is that it assumes that fraud is concentrated in the presidential rather than the "local" races, but why should it be?

Overall, the average error in Senate races actually was substantial but smaller than in presidential races in the same states (mean -3.6 points on the margin as opposed to -5.0 points). If we took the 1.4-point difference as a measure of fraud in the presidential race (which seems like a big stretch), I suppose we would conclude that Bush won the popular vote. So, folks committed to the prior that Kerry won the popular vote might prefer not to go down this line at all; again, there is always the legitimate point that Senate races (and even local races) aren't immune to fraud. You can examine the state-by-state results provided in the evaluation report and see what further questions emerge.

The EIRS dataset is so spotty that spatial analysis vis-a-vis the exit polls is practically impossible. Freeman tried it at the state level and thinks he got some positives; I looked fairly closely at both the EIRS and the MY-VOTE1 datasets (confusingly as part of a working paper about TIA's pre-election stuff, called "Surprise and Confusion") and got no satisfaction. It may well be possible to compare the EIRS dataset with precinct-level results in, say, Cuyahoga County, although I think 'Time for change' looked at this and concluded that it would be a waste of time because the geographic details are too sketchy. If it doesn't work with electoral results, I can't imagine how it could ever work with the much, much sparser exit poll data.

I can't think of any a priori reason to assume that a demographic vector should have the same predictive power in one race as another.

The evaluation report makes clear that errors aren't limited to one voting technology -- but I have never seen an authoritative DB of tabulators, so it may be possible to make progress there. A fairly representative allegation is that 'Blackwell had a computer on his desk that let him change the results in any county.' OK, if that is generalizable (i.e., if the tabulator problem is one that typically applies statewide), then we already have at the state level all the data we need to analyze variance, if there is any.

It is of course not possible to determine whether error only occurred at certain times of day.

There are swing-state comparisons, which have been discussed elsewhere.

I'm not sure what you mean by a time series of changes in one race compared to another. Any comparison of races over time would seem, again, to be better served by analysis of election data (see, for instance, Mebane and Mebane & Herron in the DNC report on Ohio, including race vs. race within 2004 and between 2002 and 2004) than of exit poll data. Election Science Institute of course did an analysis of Ohio exit poll data vis-a-vis change in presidential vote between 2000 and 2004, which Mitofsky has extended to the entire national dataset in a few slides I've posted via my home page. The word from the National Election Data Archive is that one should not examine archived election data in this manner, because the results are meaningless bunk that undermine democracy. Well, then.

So far I haven't really seen a question here that made me itch to get at the original data, but there might be some more juice in the presidential-vs.-Senate issue -- I will think about that some more. (And I think some noodling-around could be done with demographics, although I am really skeptical that a crucial test is possible.)

"error on the 'same technique' should change (decrease) systematically as the sample increases...does it?" Umm, sampling error should decrease; bias should not. TIA seems not to understand that bias is a real, variegated and ubiquitous problem in survey research -- as opposed to something that Mitofsky invented in a moment of panic -- and therefore belabors sample size while ignoring the actual issues, IMHO. Not to single him out in this respect. At any rate, I am not sure where you are headed with this point.

I don't really know what it means to say that "poll error seems unexplainable at the state level for Ohio, Florida, and Pennsylvania." We seem to have at least two explanations -- vote miscount and non-response bias (or more generally participation bias). In my experience, the people who strongly favor the vote miscount hypothesis generally do not know much about non-response bias and are not much interested in looking at infirming evidence, which at best has delayed the progress of forensic research.

The evaluation report is at http://www.exit-poll.net/election-night/EvaluationJan192005.pdf ; the questionnaires are at http://www.exit-poll.net/election-night/Final_State_Questionnaires.pdf and http://www.exit-poll.net/election-night/Nat_Final.pdf. You may find several things at my homepage http://inside.bard.edu/~lindeman/ useful. I won't link to the ESI and DNC reports right now, but you should have no trouble finding them. Happy hunting!

"Another problem with the null is that it assumes that fraud is concentrated in the presidential rather than the "local" races, but why should it be?"

No...I don't think that a person would pick out a particular reason to lie on one question as opposed to others...

Meanwhile, there WERE poll local questions here - at least there was in Florida - so data could be compared from any local polls to national elections IF the local data were available from E-M which it isn't.

Yes, it is possible to look at error by time of day, unless the pollsters were idiots - I assume they know that time is important in elections and have times stamps on all data as it came in...

"I can't think of any a priori reason to assume that a demographic vector should have the same predictive power in one race as another. " No, but might reveal evidence of manipulation...I assume you are aware of correlations in highly diverse populations - hmmmm, like Florida.

Spacial is possible in a number of ways...I'm sure the locations of the pollsters were known...and they didn't get one response and move to the next block..

I'm sure that hypotheses like these could support or refute some of the speculation - but there's no real reason for raw data, collection times and places, and similar things to be secret. It's also impossible for me to believe that someone at E-M hasn't investigated...already.

a few comments:

One problem is that people assume the data isn't available. It is. You can download it yourself. What is NOT available are precinct identifiers, so you can't figure out precinct level disparities, and what is also not available are any weights that were applied solely for observed non-response bias by visible features of non-responders (in other words you have the choice of cooked or raw - not half-cooked).

The datasets are here:

ftp://ftp.icpsr.umich.edu/pub/FastTrack/General_Election_Exit_Polls2004/

Precinct-level analysis obviously useful for precinct-level variables, which is what you can't do unless you know which precincts are which, which you don't. But precinct-level analysis within-state is, as OTOH says, pretty underpowered. There are only a few tens of precincts per state - so you certainly can't use county-level variables, even if you knew the counties. But you DO know the state results, and the states are identified (each state has its own file) so state-level analysis can certainly be done on the data that is publicly available.

And of course, the reason precinct level vote return data is not available is that vote return numbers would enable you to identify the precinct, and as many precincts are small, and a large amount of demographic detail is given on each responders, it would certainly make some respondents identifiable. Not many perhaps, but it would certainly violate the ethical regulations of any institute that I have worked in.

Time of day: there were three calls. Call numbers are marked in the data set for each respondent.

"It's also impossible for me to believe that someone at E-M hasn't investigated...already."

Well, E-M issued their big evaluation which reflected a pretty exhaustive internal study, although the report itself is short of analytical detail (but rich in aggregate datasets). A couple of other analyses have been presented by Mitofsky in public, including one hosted on OTOH's page:

http://inside.bard.edu/~lindeman/slides.html

And perhaps you'd better check my sig.

I still am sure that all our local data on local races and demographics are organized by precinct. It appears that just about everyone except E-M doesn't mind reporting that way. As you relate: individual data doesn't allow interesting comparison and the most likely error analysis indicating manipulation; while state-wide data hints at a problem (as TIA continues to express) but no way to track down the error! Catch-22!

It would be most interesting to test at precinct, tabulator, zipcode or any level of analysis with power to detect or dispel the question. As you know, If I'm looking for a bacteria and have only an electron microscope and a hand magnifying glass, I'll never see one cell! I have to have power at the level of the issue. I've seen a few E-M reports, and though I'm not as excited about it as some critics, I agree that there are some graphs and charts short of information at strategic places.

The easiest thing to do would be for E-M to take the detailed information from the polls and compare with other voters and elections at the appropriate level besides the President to get an idea of "Bush-related lying" or "Bad memory from last election effects", etc. Respondent effects (and rater effects which I analyze every day) can be estimated with a number of techniques (FACETS, etc.) and either corrected or a decision to toss the data is obvious. Why not invite some neutral statisticians to use the data to investigate and agree not to reveal anyone's name?

Certainly, any systematic effects can be distinguished from outliers.

I think that IF the precinct level data were available, a really good question (given the accusations in Florida) would be to compare the subset of precincts that fed each tabulator or aggregation point. Systematic error at that level would be evidence of an issue that needs investigation!

Thanks for the link. I'll download the data and I promise NOT to waste time data mining...if I see anything striking I'll post it.

(EDIT: sorry, missed Febble's response -- I haven't tried to edit out the redundant parts here)

"No...I don't think that a person would pick out a particular reason to lie on one question as opposed to others..."

AFAIK no one is assuming that respondents are "lying" about any of the elections. That may have been an issue in the 1989 Virginia gubernatorial exit poll that went bad (and, yes, I do think that some voters in 2004 at least misremembered how they voted four years ago). Some folks may deliberately lie on exit polls, as a newspaper columnist once suggested they should. But that isn't integral to any account of the result.

Here is where I am really not following you: what do you mean by "there WERE poll local questions here... so data could be compared from any local polls to national elections..."? Sorry, I'm lost.

It is possible to look at the interview results by call or wave (the interviewers phone in the interview results three times during the day), but since we don't have an independent source of info about the vote by time of day, I don't see how it adds much bite to fraud analysis. Do you know that in Ohio, the interviewers started at different times because Blackwell had sought to bar them from conducting interviews within 100 feet?

"No, but might reveal evidence of manipulation...I assume you are aware of correlations in highly diverse populations" -- Which correlations? "No, but might reveal evidence" sounds a lot like "Hey, let me poke around and see what I come up with," which has not proven a very compelling argument so far.

"Spacial is possible in a number of ways..." -- talk is cheap, but have you actually looked at the EIRS data? And do you have a reason why the spatial analysis would work better with exit poll data than with election data from 200+ times as many precincts?

"there's no real reason for raw data... to be secret." If you mean that the precinct identifiers should be released so that we can match every interview to a precinct, then, well, you are entitled to your opinion. But the prevailing views in the profession, AFAICT, are that the privacy concerns are at least potentially real, and the benefits are minimal.

I suspect you've never really done a human subjects review for major research for a medical experiment...AFAICT is a profession of crapheads if they can't have neutral people look at data without revealing identity. Take my freshman course!

Meanwhile, "feeble" sent me a link to the data, but it's clear already that anyone who doesn't think a geographic interest is important is also out of touch with reality. After all, haven't you seen the voting district case in front of the SC now? Don't know about the electoral college? You're defending a silly assertion. This is a case of power to detect if there was or was not evidence of manipulation - and by all accounts manipulation would have occured between the precinct or tabulator level...what's so hard about that? The ASA (I've presented and published with them - if you know what that is..) clearly has a large number of specialists for spacial/geographical analyses....E-M avoids that issue.

As a reviewer of journal articles for 20 years, I often see that what's "not addressed" is the issue that needs focus. Why, except for a lame reasoning that has easy solutions, doesn't E-M release data - even to the critics in "private" - for debate?

Finally, it's the ONLY level of data that's useful which E-M won't release. Meanwhile, I'm not a politician, but it's been fairly easy to see local and state polls (usually by precinct) of any number of local elections - even if you use observed instead of expected results - for example a hypothetical:

IF a given local precinct in Florida:
Was 40% Hispanic, 30% Black, and ....
Voted 80% for the democratic senate candidate...
Voted 75% for the teacher pay raise...
Was 80% BUSH for president after....
14 people reported that the machines were not counting right...
Then that poll was HACKED....finding the level of manipulation; IF it exists is critical to the issue. I'm surprised that is not obvious to you.

We already have "weird" results by several reports at the state level. So why not provide more evidence of the TRUTH from E-M? The ONLY explanation you offer is "privacy" - of a vote that people have already agreed to be polled AND is 2 years ago? You can't think of a way to deal with that?....why attack the obvious questions or the multiple ways to look for an answer. You asked for a NULL and I have given you a few. If you want to make a dissertation out of it, get all the data (including all identifiers) and come to Florida. We'd be glad to approve the human subjects design officially and give you a Ph.D. if you can produce a QUALITY analysis. I don't care if the evidence goes one way or the other. I do care that you seem to offer critique without "appearing" to see the obvious conclusions: E-M should figure out a way to answer the questions if the data is there.

BTW, "lying" is a simple term for beginners...I'd prefer to look for "respondent effects".

I am happy to stipulate that as a political scientist, I haven't done human subjects review for medical experiments. And this would matter why? Heck, my opinions on this don't even matter, but you would be well advised to pay a bit more attention to the details.

Neutral folks have already looked at the Ohio data, if Fritz Scheuren counts as neutral. It probably would be possible to build on that precedent. (The people who are screaming the loudest for access don't exactly come across as neutral. Here is Steve Freeman on Warren Mitofsky: "For him (to) say that the exit polls did not indicate a victory for John Kerry, is stunning. He might as well stand before us and say 'blue is red.'" Oh-kay.) If folks want more disclosure, they will have to make a better case for it. That is reality. If you want to crap on the pollsters for not practicing their research like you practice yours, indulge yourself, but it won't do a bit of good. That, too, is reality. The critics need to do their homework, and too often they haven't.

"Don't know about the electoral college?" Hello? hello? the state-level results are all in the evaluation report. Do you make a practice of lumping all spatial units of analysis together as "geographical" and using them interchangeably, or are you just having a bad day?


If a precinct voted 80% for the dem senate candidate and 80% for Bush, we don't need exit poll data to tag it for suspicion, now, do we? You really need to stop and think. (And learn something about the machine reports while you are at it. foo_bar is the guy to ask, if you can treat him with respect.)

SOP in social science is not to release datasets with identifying data, period. Often some provision can be made to use the data without release, and that may well be possible in this case, but it's not an open-ended ethical obligation on E/M and NEP's part, in most folks' opinion. Sputter at me all you want -- or, better, come up with some much better specific questions to investigate.

I am perfectly happy with my first Ph.D., thanks, but that is not the point. The only thing in your example that I've quoted above that even might have exit poll content is the demographics, which we could probably get or estimate another way (partly depending on the state). So what questions, specifically, do you want E-M to figure out a way to answer?

You can look for respondent effects, fine, but you shouldn't neglect to look for non-response and sampling bias.

"I am perfectly happy with my first Ph.D., thanks, but that is not the point. The only thing in your example that I've quoted above that even might have exit poll content is the demographics, which we could probably get or estimate another way (partly depending on the state). So what questions, specifically, do you want E-M to figure out a way to answer?"

Here's what E-M or someone needs to do: Simply analyze the data at the level of appropriate power to look at the question of potential manipulation evidence. That has not been reported and the data is hidden.

There are MANY ways to skin the cat. What is E-M worried about?

"It is possible to look at the interview results by call or wave (the interviewers phone in the interview results three times during the day), but since we don't have an independent source of info about the vote by time of day, I don't see how it adds much bite to fraud analysis."

Let's see...three times a day by 4 times zones....hmmm...that's enough for time series! Meanwhile, after the election partial results reported as precincts are turned in are often faxed with time stamps...
----------------------
"Here is where I am really not following you: what do you mean by "there WERE poll local questions here... so data could be compared from any local polls to national elections..."?"

It's easy enough to toss out all other poll data except the variable so interest....but that still doesn' t mean representative data from a given location and sample can't be compared with actual results at that location, demographic by demographic with error by "high stakes" elections (President) compared with other probable results.
-----------------------
"No, but might reveal evidence of manipulation...I assume you are aware of correlations in highly diverse populations" -- Which correlations?

Pre-election polls clearly predict that black, hispanic, poor, etc...have certain expected voting results based on polls; previous results, etc....do those correlations stand up in all cases; or differ in selected precincts, zip codes, polling stations, etc...

"But the prevailing views in the profession, AFAICT, are that the privacy concerns are at least potentially real, and the benefits are minimal."
From Chapter 9, "How to do an Election Survey" - Page 21 - "A bias in a sample will be both invisible and harmless so long as the bias is not correlated with anything that you are trying to measure." www.unc.edu~/pmeyer/book/chapter9.doc

-----------------------
"Spacial is possible in a number of ways..." -- talk is cheap:
Please read a few years of...International Journal of Geographic Information Science; Journal of Geographical Information Systems; Journal of the American Statistical Association.......
Just as an example...Second-Order Neighborhood Analysis of Mapped Point Patterns (Geis & Franklin, 1987) starts, "In any study where spatial data or patter analyses are required, the appropriate scale for analysis must be chosen." P. 473.
-------------------------------
But the prevailing views in the profession, AFAICT, are that the privacy concerns are at least potentially real, and the benefits are minimal.

Council of American Survey Research Organizations Code of Standards and Ethics for Survey Research:

"a minimal amount of....information will be disclosed to permit...VALIDATE INTERVIEWS...DETERMINE AN ADDITIONAL FACT OF ANALYTICAL IMPORTANCE TO THE STUDY..."

Reading the standards easily allow precinct data with names redacted, etc....and always behind closed doors with qualifed neutral statisticians...

------------------------------------

...that benefit from use of the exit poll data. Sure, three waves of data are enough to do time series analysis, but so what? What time series analysis?

I have no idea why you think that Phil Meyer quotation is on point. In the exit poll context, many of the biases quite possibly are correlated with things we are trying to measure.

And the way you quoted CASRO is shocking. The part you carefully edited out, twice, is "the Client." Oops. As I've said, you really need to slow down and think.

(You seem to be confusing yourself by painting me as a hard-liner on confidentiality, which I'm not. But a lot of folks on DU pay no regard to it at all, and you do not help by articulating the issues so sloppily.)


The assumption is very tenuous (why would we assume that, e.g., Hispanics would vote for Kerry in the same proportions everywhere? or in the alternative, specifically how do pre-election polls and previous results shed light on any variations?), and you still haven't offered a testable hypothesis that makes sense to a political scientist (this one, anyway) with respect to assessing vote miscount. Once you do, you can develop a rationale why analysis of self-reported demographics among an average of 40-odd voters in each of about 50 Ohio precincts would be more revealing than the work that has already been done on election data -- which, from all appearances, you haven't read. As always, it is very difficult (barring sheer dumb luck) to build upon work you haven't read.

Once again, I am not arguing against investigating your hypotheses, I am pointing out that you so far haven't offered any useful ones tailored to the data actually available. If you did offer testable hypotheses, they could (potentially) in fact be tested; that would be much more useful than misquoting CASRO on DU. "Please read a few years of..." is obviously more cheap talk, and I won't waste time reciprocating. As in other contexts, if you are serious about contributing to the research agenda, you need actually to contribute to it.

All our official data in Florida is reported by precinct. All the local news pre-election and polls are reported by precinct. The accusations down here of manipulation of tabulators, software, etc..are testable by precinct. IF E-M produced data by precinct, the simple hypotheses would be to test if

1.) Demographics made sense compared to observed results (compared to pre-election and poll data), or there was no weird results
2.) If there were differences in precincts which could have been hacked vs. those which couldn't due to systems used or not
3.) If there were geographic or spacial error differences that may be explained by targeted manipulation of results or not
4.) There were no time bumps in the results that indicate a manipulation of votes (our voter tapes and machines can give number of votes by various time segments
etc...etc...look at the menu of observed data available below.

The issue continues....state-wide data nor individual data without knowing locations and times is appropriate for analysis if the level of manipulation is either by precinct, poll station, or a subset of precincts. Hypotheses have to look at the question of interest. Statewide data masks the possible manipulation) which appears a larger than expected WPE, etc.) Also, weighting creates confusion. Expected vs. observed results at the appropriate level of data is the question.

You can argue all around the issue. E-M has the data. Maintaining confidentiality is no more than a typical exercise that anyone could deal with...I haven't seen any reports except that there is some weird and unexpected error in the data (admitted by E-M) that seems hard to explain.

If you want hypotheses, the above all have the same issue: they require data a the level appropriate to detect variance to the issue at hand. Most of our political scientists, sociologists, and everyone else here wants to either confirm or refute the possibility of interference with elections before the next election. BTW - it was reported that our local elections YESTERDAY had "computer problems" with votes counted that had never been cast! Here is a typical report menu that all of us can get on any election by precinct since 1999 from the local election supervisors:

----------------pasted from a typical Florida website of election supervisors-----------------

General Election - November 2, 2004

Statement of Vote
Pinellas County, Florida
2004_General_Election_Nov_2

November 18, 2004 3:40 PM

Click on a contest below to see results

PRESIDENT AND VICE PRESIDENT
UNITED STATES SENATOR
REPRESENTATIVE IN CONGRESS DIST. 9
REPRESENTATIVE IN CONGRESS DIST. 10
REPRESENTATIVE IN CONGRESS DIST. 11
STATE SENATOR DISTRICT 11
STATE REPRESENTATIVE 45TH HOUSE DISTRICT
STATE REPRESENTATIVE 51ST HOUSE DISTRICT
STATE REPRESENTATIVE 52ND HOUSE DISTRICT
CLERK OF THE CIRCUIT COURT
SHERIFF
BOARD OF COUNTY COMMISSIONER DISTRICT 1 AT LARGE
BOARD OF COUNTY COMMISSIONER DISTRICT 3 AT LARGE
BOARD OF COUNTY COMMISSIONER DISTRICT 5
BOARD OF COUNTY COMMISSIONER DISTRICT 7
JUSTICE KENNETH B. BELL RETENTION
JUSTICE RAOUL G. CANTERO, III RETENTION
JUDGE CHARLES T. CANADY RETENTION
JUDGE PATRICIA J. KELLY RETENTION
JUDGE STEVAN T. NORTHCUTT RETENTION
JUDGE CRAIG C. VILLANTI RETENTION
JUDGE DOUGLAS A. WALLACE RETENTION
CIRCUIT JUDGE 6TH CIRCUIT - GROUP 22
EAST LAKE TARPON FIRE CONTROL DIST- COMMISSIONER 3
EAST LAKE TARPON FIRE CONTROL DIST- COMMISSIONER 5
LEALMAN FIRE CONTROL DISTRICT-COMMISSIONER SEAT 1
LEALMAN FIRE CONTROL DISTRICT-COMMISSIONER SEAT 3
LEALMAN FIRE CONTROL DISTRICT-COMMISSIONER SEAT 5
PINELLAS SUNCOAST FIRE DIST.-COMMISSIONER SEAT 2
SOUTH PASADENA COMMISSIONER
Constitutional Amendment No. 1
Constitutional Amendment No. 2
Constitutional Amendment No. 3
Constitutional Amendment No. 4
Constitutional Amendment No. 5
Constitutional Amendment No. 6
Constitutional Amendment No. 7
Constitutional Amendment No. 8
County Charter Amendment No. 1
County Charter Amendment No. 2
County Charter Amendment No. 3
County Charter Amendment No. 4
County Charter Amendment No. 5
School Board Referendum
Pinellas Suncoast Fire & Rescue Referendum
Indian Rocks Beach Amendment No. 1
Indian Rocks Beach Amendment No. 2
Indian Rocks Beach Amendment No. 3
Indian Rocks Beach Amendment No. 4
Indian Rocks Beach Amendment No. 5
Indian Rocks Beach Amendment No. 6
Indian Rocks Beach Amendment No. 7
Indian Rocks Beach Amendment No. 8
Indian Rocks Beach Amendment No. 9
Indian Rocks Beach Amendment No. 10
Indian Rocks Beach Amendment No. 11
Indian Rocks Beach Amendment No. 12
Indian Rocks Beach Amendment No. 14
Indian Rocks Beach Amendment No. 15
Indian Rocks Beach Amendment No. 16
Indian Rocks Beach Amendment No. 17
Largo Amended Charter
Oldsmar Charter Amendment 2004-18
Oldsmar Charter Amendment 2004-19
Oldsmar Charter Amendment 2004-20
Oldsmar Charter Amendment 2004-21
St. Petersburg Question 1
St. Petersburg Question 2
St. Petersburg Question 3
St. Petersburg Question 4
St. Petersburg Question 5
Tarpon Springs Referendum 1
Treasure Island Question

Those aren't "simple hypotheses" at all, as surely you realize. Those are a fishing expedition. 'We can look for all sorts of anomalies that may or may not indicate vote miscount.' Well, yes.

One last time, to the extent that I am able to infer from your laundry lists what analyses you would actually like to run, most of them do not hinge on exit poll data at all. Indeed, you made that very point yourself, didn't you?

If you are serious about this enterprise, it would behoove you to see how far you can take the work based on data already available. Yes, indeed, you have potential access to a bounty of election and demographic information at a very low level of analysis. What have you done with it so far?

There is no way to "refute the possibility of interference with elections before the next election" -- that is not a good problem definition. We already know that it is possible to interfere with elections. If "our political scientists, sociologists, and everyone else here" -- or anyone else there -- has some concrete ideas about this, could you please ask one or more of them to post them on AAPORNET or something? What specific questions have they pursued using other available data, that now require access to the exit poll data to advance the investigation? Why are they not trying to convince other political scientists and sociologists that they have specific, interesting, and urgent research questions?

Insanity is doing the same thing repeatedly and expecting a different result. Bitching on DU is never, ever going to yield any additional insight. Given the professional background you describe, you should have a much better political as well as intellectual analysis of how to get something useful done, if indeed there is something useful to be done. It has nothing to do with my arguing around some issue; I am not an obstacle to what you claim to want. If you or folks you know have something serious to contribute, do it.

Those are testable hypotheses...there's no requirement to use a particular technique such as SEM or HLM or anything but basic statistics...less complexity than already published from the links you gave me. I've tried to avoid "statistical language" on bulletin boards. The statisticians reading along already realize that...Here's the point again:

If E-M provides the precinct level identifiers to the data, within a short time there would either be evidence of hanky-panky, no evidence of problems, or else evidence that analyses are not possible due to quality of data, sample size, etc.

What you're unwilling to do because of rhetoric is admit that research data which may be useful is being withheld without a reasonable explanation. You protest too much. What do you know or fear from an investigation? If someone wastes time chasing ghosts - so what?

In short scanning some of the links and reports on this topic (some links you and feeble sent), it's clear that others have already stated that E-M needs to release data and that some of the issues we've been arguing have been made by others after performing some analyses.

I'm willing to see what the data offer no matter what the outcome. You're defending a completely indefensible position if you say, "we don't need to go looking." Science, as we all know, is a combination of planned hypotheses and serendipitous discoveries. I can't believe that you'd even suggest that all our hypotheses would be a priori or that complexity has anything to do with the issue of the H: Once you form an H: ; there can be multiple ways, from a range of simple to complex, to look for evidence to reject a null in almost any H: Frankly, inferential tests are only one part of this statistical world, and it's pretty simplistic to restrict everything to null hypotheses with a preset alpha.

We have some questions, data is available, we can test some things and also see if new trends emerge. If we already predicted everything as you seem to be doing - we'd never get anywhere....somehow I don't think you're going to change your mind though

it's febble, not feeble, so if it's a mistake, fix it, or if you are being rude, don't be. I'm not.

You are asserting that there is not a reasonable explanation for withholding precinct identifiers.

There is. Respondent confidentiality. That apart, there is an extraordinary amount of data in the public domain.

You then say, well, independent analysts could analyse data with some safety measure for maintaining respondent ID.

They have. ESI commissioned "blurred" data for Ohio and analysed it. Their brief report is here:
http://www.electionscience.org/Members/stevenhertzberg/report.2005-07-19.7420722886/view?searchterm=None
and I understand a full report is currently undergoing peer-review. I also understand further blurred datasets are being commissioned.

You also, I think, accuse E-M of not analysing the data. They did, and their report is here:

http://www.exit-poll.net/election-night/EvaluationJan192005.pdf

You may not like these analyses; and you may think that more need to be done, and more data needs to be released. Fair enough. I didn't much like the E-M evaluation myself, although the quantity of precinct-level data they collected was prodigious, but it wasn't couched as an academic report.

If you want a hypothesis tested, I suggest you state it. Mitofsky apparently offered to test any hypothesis Steve Freeman proposed. I've thought of a few myself, and a few more have been suggested on this forum, but remarkably few, and,frankly, remarkably little analysis of the enormously rich data that is in that link I posted. Spend some time on it, and if a specific fraud hypothesis occurs to you - well, Mitofsky may test it for you, you never know.

I will look at the data. If there are "blurred" data, that is something that is possible to do. It is a common solution. I would prefer all the data, or data in Florida; but the solution is fine.

A good analysis would work to support or dispel the issue of election manipulation - or else conclude the data aren't adequate. I deal with "confidential" human subjects data every day - children, medical, etc. We have extensive reviews and codes of ethics. There are so many ways to deal with adult, voluntarily given, post hoc, barely dangerous data that a survey is a red herring (as we say). An excuse that simply doesn't work. You've already mentioned one solution. Sorry if I entered this discussion at a point that I have to catch up.

I don't know Mitofsky, but I'm sure that all of us would think of simlar things to investigate. In my area, I use some different techniques, but not so different as to not be parallel.

I can't stop and spend hours on a data set today, but will get to it eventually. Meanwhile, I'm sure that there have been "anecdotal" discrepancies and "questionable" results in Florida. I'd be glad to run something here if it appears feasible with the data.

I did not accuse E-M of failing to analyze the data! I suspect they not only analyzed it, but carefully avoided reporting or releasing anything that may be an issue for their clients and their profitable contract. Interestingly enough, I often tell consulting contracts up front that I will report the results to them, but not hide anything. If that's not acceptable, I don't work with them. I assume you'd do the same. You think election polls are tough clients - try a new drug or $100,000,00 program that affects all the kids in the state! They don't like bad news!

I still think that whatever the issue, having data available at the level of analysis is the only way to detect the variability of interest.

Thanks for the links - I not sure why you'd think otherwise - but so far I haven't had any beef with you. I'm used to "academic" debate at work, so this is a little hardheaded and emotional for me. Even though I'm not happy with our current president, I'm a registered Republican! I have looked at DU occasionally due to dissatisfaction with the current directions and some curiosity. About once or twice a year, I see something interesting and post a note.

This issue was of interest because I saw TIA and others (yours) "statistical" posts...

I appreciate your academic approach, which I'd like to think is also mine, usually. Apologies.

A couple of points:

RE the confidentiality issue: I too deal daily with confidential human subjects data, and our ethics committee is extraordinarily strict about anything that might tie data to an identifiable subject, however remote the chance. I take your point that this isn't "dangerous" data - but nor is the data I work with. It is just private - and so is your vote. If a pollster asks an interviewee to divulge the content of his/her secret ballot, on a questionnaire with "confidential" written in large letters on it, then it would be quite unethical to do anything that would allow that data to be matched to the participant. It includes an extraordinary amount of personal detail in addition to the vote, and these details are in the public domain (I've given you the link). This has two implications: firstly that in a small precinct, those details could easily identify certain voters; secondly, that if a voter was thus identified, that personal information enters the public domain . I know I am in an uncomfortable minority here, and I know that people not only scoff at the confidentiality issue, but infer from it that something "else" is being hidden, but from where I'm standing, the pollsters cannot divulge either the precinct ID or the vote count (from which the ID could easily be traced). So blurring, is, I think, the only answer, but it is intricate and labour intensive if the statistical properties of the data are to be preserved. It has, however, been done for Ohio, and may be done for other states. I would imagine Florida would be on the list.

Regarding what can and can't be gleaned from the data. I said above that at state level, the precinct-level predictors are thin. I think people imagine (maybe not you) that because the poll is so huge in terms of participants, it has huge statistical power. Well, it has, for some things. But at precinct level, there is not much statistical power; the samples are small (mean size 80) and the precinct Ns are small (40-50). Not a lot of degrees of freedom for multiple regression. So between-precinct analysis is not a lot of help for figuring out what went wrong in a state. Across the country, of course there is much more statistical power (1250 precincts were considered good enough for analysis - the remainder mostly had large numbers of absentee votes rolled into the total), which is why the lack of a correlation between swing from 2000 and redshift is a fairly powerful null finding.

Finally, what I have been working with, together with OTOH, is developing a decent dependent variable to represent the magnitude of redshift. It's not as easy as it sounds, but is rather important.

Like it or not, your felt need to make "serendipitous discoveries" has not evoked a groundswell of professional pressure upon E/M to be more open. Febble and I are making specific suggestions that could actually work. Stop shooting the messenger. Fish or cut bait.

I downloaded Febble's suggestions, and will carefully consider them objectively.

You remind me of the HS debate squad. I have offered several hypotheses, questions, and testable relationships. I ALSO stated that science depends on discovery beyond the predicted hypothesis, and if you don't get that then you're not a scientist nor have ever worked in real science.

IF you want to test the questions suggested, then I'll hold your hand and help you explore...you may want to start with the first ones...

Here's the last time:
------------------------------simple one for you------------------------------------
At the unit of analysis of Precinct data: was the error significantly different from precinct to precinct in Florida.

IF you test that...I'll do the post hoc to see which units are "significantly" different from others. Set the alpha at .05 or what ever you want. Most of us today simple report the p generated by the analysis.
-----------------------------better one for the hot shots-----------------------------
Produce an LDF (discriminate function) from the survey variables on the dichotomous results (Bush, Kerry). Do likewise for senate races, and other reported parallels. Test what demographics are significant predictors. Run the LCF (classification function) on demographics (predictors) on suspect precincts or "tabulator" subsets. Test centroids/Lambda's from state to state...compare classification accuracies. H: Lambda(a)=Lambda(b)=.... If anyone wants to argue regression - fine - you know the difference.
---------------------------for the truely informed--------------------------------
Treat all survey items as scores. Initially look at Rasch (interval logistic transformation) and FIT statistics. Consider three facets: state/precinct, Busch/Kerry, survey items. Look for "rater effect" in facet one (state/precinct).
H: Facet One: MeanSqr(1)=MeanSqu(2),=....
---------------------------------------------------------------------

Do you want references? For Discriminant analysis - search for Karl Huberty, get the book. For FACETS - search for Mike Linacre, get the book.

For a start, even your first one isn't "simple". How do you want to measure the error?

You need a way to discriminate between vote miscount and non-response bias (and other things that might bias the poll).

Testing for significant differences in error across Florida precincts doesn't cut it. Neither does analyzing the hell out of demographics. I know that when one dives into datasets, often Good Things Happen -- but if you are trying to sell the case that this is important, I think a stronger case will help.

And, finally, please don't hold my hand. Thanks.

Expected vs. observed (remember your weights that we don't have)...if that's a problem, it's possible to simply test variance against each other by precinct...take outliers (significantly different within logical categories would make sense - like voting method; demographics, etc.). A glorified version of http://www.socialreseachmethods.net/kb/stat_t.htm (I don't have a textbook in front of me.)

The idea is to identify outliers by simple variance...I didn't do anything different than it appears some have already been doing except suggested a little more power and took the precinct as unit of analysis...tabulator units would make sense also.

I haven't spent any time with the data or survey, yet. Sorry, I have some paying projects right now...but I'll eventually be able to identify variables or see sample sizes within units, etc...what logical subset of surveys above individual maximize differences in demographics or any other recorded variable? Likely it's pollster locations (poll number), precincts, tabulator units...but could be time of day polled or something like that...possibly maximizing demographics may prove useful if they are correlated or preditive from pre-election, previous election, or post-election evidence.

you seem very statistically informed, which is great. But you also need to know a bit more about the data.

First: upthread you rightly surmised that pollsters don't just pick voter from a precinct then go on to another one. But if you are only surmising this you need a few actual facts. Start with the E-M evaluation. The precincts themselves are sampled, carefully, weighted by size to ensure that every voter in the state has an equal chance of participation. Then a target of sample of around 100 voters per precinct is aimed at, by allocating an "interviewing" interval to each precinct, based on past turnout (and therefore expected number of voters). Interviewing interval turns out to be an important predictor of discrepancy.

Although voters are treated as the unit of analysis for crosstabs and projections (with an allowance made for the clustered sampling method) precincts can be treated as a unit of analysis for diagnostics after-the-event, and precinct variables were painstakingly collected by E-M after the election in an attempt to find out what factors were predictive of within-precinct discrepancies between poll and count. However, the way these discrepancies are measured is in itself an issue, one that has been explored by OTOH and myself in some depth, although there is still a way to go. You might be interested in our paper, presented to the ASA last August, page-proof of the proceedings paper here:http://www.geocities.com/lizzielid/ASApaper.pdf

The E-M investigation into within-precinct error largely used precinct-counts, collected at the precinct on election night, not tabulator counts, although I understand that where precinct counts were not available, county tabulations were used. Time-Of-Day is no use for precinct level analysis as obviously there is no time-stamp on the vote count data.

There are also a number of other issues you will need to know about if you want to get into this. And yes, many of the variables you mention have been considered, and are reported on (if inadequately) in the E-M report.

But I have to say - although I appreciate all these text books you are recommending, this isn't, actually, text-book stuff. Or at least there are important issues not dealt with in text books. And I could do without the jibes about "real" science. Sure, post hoc, data-driven findings are interesting especially if they lead to modified hypotheses that can be tested on new data. But we don't have new data. So the more specific your a priori, the more statistical power you are going to have. No?

I assumed a stratifed sample intended to be representative of the "state". As Wainer says, it is the "non-ignorable non-respondent" that we have to worry about...I recognize that issue, but I'd like to move away from heroic attempts to do the RA Fisher thing and predict why some didn't respond as expected and work with the variables that we do have and see what power there is...including data outside the E-M data that is public...and find evidence for or against a systematic bias. I have no idea what your background is...but I'll put in a quick link to illustrate why I'll move to a more sensitive analysis.

I just looked very quickly at the data. Here's a powerful way to do a bias analysis which may work here:

http://www.winsteps.com/winman/example15.htm This was used in 2002 to find a biased ice skating judge in the Olympics with skimmpy, unconnected data. I'm sure that IF I had the likely "unit of manipulation" data (if manipulation exists), it would be manipulation at the precinct or tabulator level...based on the computer hacking reports.

I think that I'll start with using the questionnaire as performance items which "might" measure voter intent (first I'll put all the dichotomous and likert things on an interval scale with irt), get a measure for each person, and then see if there is a bias treating the "state" as a judge.

It would be better to treat a precinct or tabulator as a judge. That way, we can detect more subtle "rater effects" or bias (doesn't matter if we call it towards or away from Bush). Different forms of the questionnaire won't matter if there's any connectivity, but I haven't looked yet. We can also then determine which questionnaire items "misfit" OR which persons "misfit". A large number of misfitting person's in the same state, precinct, etc. would be an indication of manipulation - and the type of fit (infit or outfit) would tell us if the misfit is systematic or random. Using precinct as an item would really be cool. If it looks like it's working, I'd ask E-M to give me precinct data with random id labels. If I could pick out the biased precincts and they all turned up in logical place...what would E-M say? I don't know the power in the data at this time, so I'll just try it...

By the same token, misfit in the item "Who did you vote for as President?" would be an issue if it occured with in a particular location or demographic. Given the data which I downloaded, that's a good start.

At least you're (Febble) more reasonable than some others on the issue. I'm perfectly willing to accept correction and discuss reasonably possible issues; philosophical debates about science notwithstanding.

I don't have time to set up the control files tonight...might be fun this spring while hiding from hurricanes to play with this...

You describe the problem of unpredictible WPE - good - I can't say much about your suggested formula fix or how well it works...
but..knowing nothing about the pollster techniques per say...my first thoughts were:

1.) What would pollsters say about comparison's of WPE on different questions on the poll?
Does WPE (Presidential race) = WPE (Senate Race)= WPE (other races)...With multiple races, that would be a good way to identify a confounded percent of response for a correction?

2.) Given the description of the process, determining error that is WPE vs BPE is intuitively difficult or impossible without connectivity.

3.) I want to consider your section on log transformation again...I'm still thinking about that...

4.) Are samples of "100 selected and 55 chose to answer" typical of the samples sizes per pollster that you expect?

5.) I also recognized the difficulty of power to detect bias and problems meeting assumptions of multivariate analysis...your article reaches the same conclusion that I was concerned with...thanks, at least I'm learning what pollsters do...in general we'd agree on your description of the issue.

6.) In my field, we'd look for "pollster error" due to a particular person as the cause of attracting or distracting a type of person who biased the targeted sample. I'm sure there is training, and also sure that still may be the cause of a missample. IF White Male Professionals represent a typical "Republican" and are stopping to be polled by cute blond coed pollsters, the best looking guy doesn't stand a chance - and 5% response confound is easily possible...so making "equally attractive" poll circumstances could be a big an issue as any. I wouldn't expect to control it all the time, just measure it when it happened to correct it...and we do lots of rater, observer, performance measures...so I've have a profile on every pollster as part of the analysis...

Later...

Due to the limited nature of the data, and the secrecy of the counts, we start out with a difficult morass of (mostly non-)data. Then we add some totals and a paucity of other data, and everybody gets to take their Rohrshach pass at what that data means. Progress can be made, but it's not terribly easy going.

Let's just say for now that they experts are divided. (this conclusion may satisfy no one on either side of the exit poll debate, but it will have the salutary effect from the perspective of LandShark's evil twin of making a lot of dogged reporters give up on the story).

These divided experts are each doing their own take on the election Rohrschach. If an outline of some sort is dimly made out in the deliberately created election fog, one side can credibly argue that nothing is really "proved" (Febble, OTOH) while the other side can point to lots of "definition" in the data constituting the outline made out dimly in the fog (the rest). Election science is not quite as advanced as, say, reconstituting grainy photographs.

But I think there's common ground in the idea that the data is excessively limited, and true outcome of the election at least uncertain.

And THAT'S THE PROBLEM in the end. We SHOULD be able to know for sure who won, but WE DON'T KNOW WHO WON. There's far more uncertainty than there should be. Whether one can, through exit polling, "PROVE" that Kerry won is what's debated, IMHO. I Think **everyone** agrees the state of our election data and elections generally is deplorable.

But since On the Other Hand is aptly named, he'll probably take issue with this too.

for the good old fashion and COMMON SENSE discussion. :thumbsup:

19,20 21,22,25,23,28,27,24.........Thanks

After that, I was pretty sure Kerry would win. Either that or much of the American public has severe brain damage. I really wish instead of arguing who won in 2004 (pst....it was Kerry), we should perhaps be making sure 2006 doesn't get jacked.....most important mid-term....EVER! We get a Dem majority and sh#% really hits the fan for the Repubs. I can't wait!

For future reference, ignoring works better than responding.
:hide:

better than "kicking and recommending," imo.