Print Friendly
Write an Article
A recent Wall Street Journal write-up discussed the findings of one Dr. John Ioannidis, who has posited that most of the thousands of peer-reviewed research papers published every year are full of flawed findings and analysis. The vast majority of mistakes, he says, aren’t purposeful, but stem from miscalculation, poor study design or self-serving data analysis. The summary to his widely-cited essay states, "Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias."
The WSJ article claims that "To root out mistakes, scientists rely on each other to be vigilant. Even so, findings too rarely are checked by others or independently replicated. Retractions, while more common, are still relatively infrequent. Findings that have been refuted can linger in the scientific literature for years to be cited unwittingly by other researchers, compounding the errors."
An ironic question to ask: Is Dr. Ioannidis study subject to the same flaws he ascribes to the rest of the scientific community? If his findings are true, what does this mean for hot-button topics such as Global Warming?
RSS
His arguments don't apply to himself by scottb :: NR9 :: Show
The question asked at the end, about whether Dr. Ioannidis’ study is subject to its own conclusion, is fairly easy to answer.
The paper is really only relevant to studies that perform statistical analysis on a dataset, compute a "statistical significance" (p) for the dataset, and then asserts a claim based on the statistical significance. Ioannidis’ paper doesn’t do that.
Strictly speaking, it’s not a "scientific" paper at all. The argument he makes is entirely mathematical. He’s talking about the application of Probability Theory.
Anyway, I’ve been trying to digest his argument (and since the essay is actually a couple of years old, the counter-arguments) before I jump in too far, here. But I thought I’d at least throw this out there to see if anyone else is thinking about it.