An Email and My Response...

I received an email this morning that raises an interesting question.  The email (edited to preserve the anonymity of the writer):

I have enjoyed reading your posts on climategate, and am quite sympathetic to your position.  I am very skeptical about the science underlying the various global warming claims.  I am also sympathetic with your claim that publicly funded researchers should release their software codes to the public.

Nonetheless, the counter of these scientists is that other scientists can produce their own code and see what results they obtain.  I was wondering whether you knew if such attempts at reproduction have occurred, and if not, why not.  Is it that it is very expensive to undertake these studies and the ordinary funding sources do not want to fund such efforts?  Or do you think something else is goind on.  I would love to know the answer.  Thanks.

In any event, keep up the good work.
M.

Here's my response (slightly edited from the original to remove personal information):

Hi, M.!

Not only do they claim that other scientists could reproduce their results, they claim that two groups (one in Russia, and one in Australia) have already done so.  But I'm very skeptical of these results, as the two groups are hardly independent of the CRU crowd -- they're "insiders" in every sense.

It's an interesting question you raise, though, and one I've pondered a bit as I read through those CRU emails.  Reproducing their results has several possible meanings.  In the purest scientific sense, "reproduction" would mean that the same results were obtained with independently acquired and analyzed data.  In this sense, the CRU's results have NOT been reproduced, and they do not claim that they have been.  I believe reproduction in that sense would be quite challenging, as the CRU's claim is that virtually all available proxy temperature data was assessed in their efforts.  One step down on the reproduction scale would be for independent groups to take the same data but independently analyze it.  I can't be certain, as I don't have access to the actual studies, but from the emails it sounds like this approach is also NOT the kind of reproduction that the two groups did.  So one more step down on the reproduction scale would be for independent groups to take the same data and use the same general analytical approach.  I believe that this is what Mann et al are referring to when they claim their results have been reproduced, and could be reproduced by any qualified scientist.  Essentially that means someone checked their math, not exactly what I would call a validation of the approach.

What Mann et al have consistently refused to provide are detailed descriptions of exactly how (and why) they selected and adjusted their data sets.  Other researchers (like Steve McIntyre) with access to the raw data have been unable to reproduce the CRU's results with anything that resembles unbiased data selection or adjustment.  In fact, in some cases they come to very different results that would lead one to very different conclusions.  Hence the debate on Climate Audit, which has become increasingly informed over the past couple of years as McIntyre and his cohorts get more and more educated.

Speaking as a concerned citizen whose pocketbook is highly at risk here, I believe that what is urgently needed is for a truly independent and skeptical group of scientists to attempt to reproduce the CRU's results in the first, purest sense I described above.  We should not be considering sweeping and expensive mitigation efforts until we have done our very best to confirm the accuracy of the underlying science.