Tuesday, December 13, 2005

Secret Sins

Here's a compelling read:

http://www.prickly-paradigm.com/paradigm4.pdf

Bombastically written, but worthwhile. I've committed these sins myself.

This reminds me of another thought I've had. This has probably been thought of by other people before, if you know who, please tell me. The idea is: If a field publishes papers on statistical results, and the accepted threshold for statistical significance is 1% (meaning that there is a 1% chance that the results were due to randomly picking a perverse sample), then 1% of those papers' conclusions will be bogus!

The question then, is: Which ones?

2 Comments:

At 7:55 PM, Blogger gary said...

No, it doesn't necessarily mean that 1% of their findings are bogus, unless all their results are exactly 1% significant. Many of their results will be much much more significant than that.

 
At 9:56 AM, Blogger emtel said...

ok - that's true. but i still think that for most fields, if you were to calculate the expected number of findings that were wrong by random chance, it would not be vanishingly small.

 

Post a Comment

<< Home