Truth and Peer-Reviewed Science...

Reader Doug W. forwarded this fascinating Atlantic article (by David Freedman), about a scientist who is systematically studying the errors in science as it is practiced today.  It's longer than most of what I link to, but if you're at all interested in understanding the process of science, it's well worth your time.

Though the article focuses on medical science, Doug points out that what it says is obviously applicable far more broadly – such as to climate research.  I read the article with that thought in mind, and this passage jumped out at me:

Though scientists and science journalists are constantly talking up the value of the peer-review process, researchers admit among themselves that biased, erroneous, and even blatantly fraudulent studies easily slip through it. Nature, the grande dame of science journals, stated in a 2006 editorial, “Scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.” What’s more, the peer-review process often pressures researchers to shy away from striking out in genuinely new directions, and instead to build on the findings of their colleagues (that is, their potential reviewers) in ways that only seem like breakthroughs—as with the exciting-sounding gene linkages (autism genes identified!) and nutritional findings (olive oil lowers blood pressure!) that are really just dubious and conflicting variations on a theme.

Most journal editors don’t even claim to protect against the problems that plague these studies. University and government research overseers rarely step in to directly enforce research quality, and when they do, the science community goes ballistic over the outside interference. The ultimate protection against research error and bias is supposed to come from the way scientists constantly retest each other’s results—except they don’t. Only the most prominent findings are likely to be put to the test, because there’s likely to be publication payoff in firming up the proof, or contradicting it.

These are exactly the points that many AGW skeptics make about the IPCC reports and much of the “settled science” of climatology, and the reaction (both by the scientists and the press) is exactly as described.

The overall conclusion is that science is a messy, error-filled process that nonetheless drifts in the general direction of increasing truth and knowledge.  I'd argue that skepticism is an essential part of the scientific process, in particular helping with the direction of the drift...