In graduate school I prepared a bunch of computer simulated data with a small, but deliberate bias in the outcomes. I gave copies of the identical data to two groups of math and physics professors asking them if the data was truly random, or if it showed any signs of systematic bias.

Group 1 was told the data represented the output from a hardware random number generator I was designing.
Group 2 was told the data from an ESP experiment.

Everyone in group 1 agreed unanimously that the data showed a definite systematic bias and that the random number generator was flawed because it was not behaving in a truly random manner.

Everyone in group 2 agreed unanimously that the data was completely random and showed no bias of any kind, and that the outcome was due entirely to chance.

So the same data will draw different conclusion depending on

1. The preconceptions of the person looking at the data, and
2. What they think the data they are looking at represents

This taught me to be skeptical of the skeptics. They believe that they know far more than they actually do know. For them, the data, regardless of what it objectively contains, will be seen to confirm their own beliefs.

Tests for randomness are usually done through deterministic mathematical tests. But, of course, you can use different hypotheses to run the data against; and you can agree to accept different levels of error.

For Group 1, were the random numbers supposedly from a uniform(0,1) distribution? I think that's where most random number generators begin. There may be a standard suite of tests to run against this type of data.

For Group 2, how did data that could be generated by a random number generator represent data from an ESP experiment? Testing this type of data may be more ad hoc than tests of random number generators.

The experiment that you describe is interesting. I think that the statistical tests of data are accurate within the given error bounds of the tests. But, how people decide what tests to run, and what error level to accept could have a psychological bias. Did you make that type of determination?

at the time the conclusion I came to was that whether a deviation was considered significant or not depended on the subjective judgement of the probability of the hypothesis claimed to explain the deviation. It is very believable that a grad student could design a faulty piece of hardware that failed to generate truly random numbers. It is very difficult to believe that something happened which you think contradicts the basic laws of nature.

So you have a balancing act:

Case 1: probability of student error VS probability that the deviation is random
Case 2: Probability of something you believe to be impossible VS probability that the deviation is random

Or an even more extreme case: Suppose you were told that the deviations were caused by Leprechauns dancing on the computer when the trials were run. What will you believe:

Case 3: Probability that dancing Leprechauns caused the deviation VS probability that the deviation is random.

Given those two extreme alternatives, any sane person would conclude that the deviation was random before admitting that dancing Leprechauns had anything to do with it.

The problem is that events considered impossible under today's paradigm might not actually be impossible. The French Academy rejected claims of meteors falling from space because under their paradigm such a thing was impossible. So suppose, just suppose, that some small, but measurable ESP effect did exist, and that some theoretical framework were to be found to explain it without "violating the laws of physics", then given plausible theoretical underpinnings the balancing act between ESP VS random deviation might shift and the same data could then be seen to support the claim of real ESP. However, such a change of attitude can never happen without a paradigm shift, and so the reigning paradigm rejects statistically significant data because it considers accepting the data as significant to be anathema within the paradigm.

And so surgeons could not be convinced to wash their hands between an autopsy and a surgery, and respected astronomers could not be persuaded to believe that rocks were floating around in outer space, and scientists (except physicists, of course. But they're always ahead of their time) cannot be persuaded to accept the possibility that information transfer between biological computers is possible without violating any laws of physics.

Arthur Clarke put it best: "When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong."

Or as Bertrand Russel said: "What we need is not the will to believe but the will to find out." and those who see only impossibility in the data lack the will to find out. Or, to quote my own words in an essay on the subject: "If they say 'I can't imagine how this could be true.' they have not proven it false, they have simply confessed to a failure of imagination."

the desire to want your thoughts and ideas to be right or supported by "data" of some kind is just part of being a human being. It's nice to feel right, but the greater discussion is how to we KNOW what's right and wrong and what can we not know. Be skeptical, criticla thinking is the name of the game, and yes be skeptical of the skeptics, but forgive me if I'm skeptical of you. I've learned to be skeptical of those that are skepitcal of the skeptics.

was coined by Truzzi, cofounder of CSICOP. He's been called skeptic of skeptics, and his criticism can be taken by as pyrrhonian skeptical criticism of scientific skepticism that likes to believe in at least in something, at least in the individual and cultural prejudices of the scientific skeptic in question.

based on the QM theory or interpretation of physical "genuine" randomness of events like radioactive decay.

But even there - randomness of radioactive decay - there is strange regularity called shnoll effect, first observed in biomatter. Some info here: http://noosphere.princeton.edu/shnoll2.html

Do the right people die?