A cautionary tale

Don't lose your lab notebook, or this may happen to you.

Pissing in the meme pool, Climategate edition: "They keep their sources and methods secret!"

"Climategate" could be described largely as a combination of poor science reporting--think of what might have been were that credulous Washington Post reporter to have had even an infinitesimal epsilon of perspective!--and a "shotgun slander": spin the content of the e-mails, the codes, etc. in the worst way possible, see what you can get away with, and never, ever, retract a falsehood. That isn't to say that some of the criticism hasn't been judicious or constructive--'bloggers Will Wilkinson, Megan McArdle, Tyler Cowen, and Robin Hanson, among others, had very judicious things to say--but that's been drowned out by anti-science types who are unable to or refuse to distinguish good argument from bad.

I suppose it's been improving. Last week "hide the decline" was the rage, followed by complaints that (gasp!) submitters of papers being allowed to suggest referees must be an evil conspiracy, and the Bizarro-world narrative that had concern for the integrity of the peer review process (e.g. the DeFreitas affair at Climate Research) was subversion of the peer review process. Now the complaint has settled on possible planned evasion of FOI requests and the supposed deletion of raw data from CRU servers.

The gripe about FOI request evasion is possibly legitimate; that of deletion of other people's data from CRU servers, less so. If I had to identify a party that could be harmed by it, I'd identify the CRU itself.

Suppose that the would-be "skeptics"--and I hate that term because it implies that everyone else is a bad scientist--decided to do something constructive (instead of producing the usual flood of obviously-wrong, long-debunked specious arguments) and attempted to duplicate one of the CRU temperature results. They have no reason to expect help from CRU--should scientists help people who incessantly libel them and look for cute-but-false arguments that their life's work is a stupid mistake?--but suppose that they find that following the published information in good faith, they can't duplicate the set. It's incumbent on CRU scientists, then, to publish clarifications or supplemental information that enables duplication. But deleting those files may be like throwing out lab notebooks: CRU might find itself lacking information necessary to duplicate their own results! It would seem that the only option would be retraction! That would be a coup for the "skeptics", one that IMO they would not deserve.

But wait, you say: the "skeptics" couldn't possibly do this, because the CRU has kept both the raw data sources and methodology secret. Think about it for a minute: would the rest of the scientific community use the CRU results if this were so?

The release of each CRU derived data set was accompanied by a peer-reviewed paper explaining the methodology and listing the sources. The CRU TS 2.1 temperature data, for example, were explained by an International Journal of Climatology article by Timothy Mitchell and Philip Jones.

From this article, someone who gets his hands on the same raw data should be able to duplicate the result. If the paper doesn't contain sufficient information to do this, it should not have passed peer review--explaining methods well enough to permit duplication is one of the universal standards for review, and complaints about this are common in referee reports.

The existence of this paper and others like it doesn't keep the credulous "skeptics" from saying that everything is secret. Libertarian 'blogger "CLS" is a prime example. To quote his comments section (linked):

They won't list a source for the data so you can't find it easily. And then you need to know how they massaged the figures, what assumptions they used, etc., in order to replicate their results or to find errors in the process, all of which is necessary before this can rightfully call itself science.

As far as I am concerned these men took all their work and flushed it down the toilet. By deleting the original data and refusing to discuss how they massage the original data to get their final results, they remove their claims from "scientific theory" entirely. Science requires replication and scrutiny and they made both impossible. By doing so they themselves took their "findings" and moved them outside the realm of science entirely. If I were a warmer I'd be furious with them.

Clearly they are listing sources, yet CLS says "They won't list a source." Clearly their papers discuss methodology, yet this CLS says they "[refuse] to discuss how the 'massage' the original data".

If nothing else, this illustrates that denialist, faux-sceptic, anti-science, or whatever you call it methodology hasn't changed: Tell the truth about the state of the science when it is convenient, lie about the state of science when it is not. Make false accusations about others at your leisure. The primary technique shall be Just Making Stuff Up. And when called on it, never explain, retract, or apologize.

"Climategate" has revealed interesting things about science politics and the need for a few things in climate science if not in science as a whole to change. But it has also made clear that the "skeptics"'s moral bankruptcy is on par with their scientific illiteracy. Their near-universal participation in the last week's lies--and not a peep from any of them speaking out against their fellows!--shows that this is not limited to a few bad eggs like Ian Plimer.

If you don't know something, Just Making Stuff Up, especially about the work of others, should never be an option.

I couldn't make that up if I tried!

On Helium.com, an article heading:

"Why is global warming damaging the ozone layer?"

Excepting that some anthropogenic greenhouse gases (think Freon...) are also ozone depleting, the two matters are unrelated. There's certainly no simple causal link between AGW and ozone depletion. It is possible, as a pure hypothetical I haven't spotted in the literature, that the climate change associated with AGW will affect stratospheric air currents (at the poles or elsewhere) in such a way as to enhance or diminish ozone depletion, but that sort of thinking is probably not what is at work here.

Over a decade ago, while still in high school, I wrote in to the editors of a several-book introduction to economics recommending that they remove a phrase "and greenhouse gases are still depleting the ozone layer" from a discussion of externalities. I didn't understand how anyone could confuse these two very different problems--the mistake is inconceivable, not one of those for which one can follow the thought process that led to the wrong conclusion.

Strangely enough, Helium.com attracted three writers to this heading. One treats us to the denialist Gish Gallop, one notes that some greenhouse gases break down into free radicals in the upper atmosphere, and one merely states both problems. To their credit, neither made the connection. But someone on the Helium.com staff did. I'd like to put myself in his shoes mentally, but I find it impossible. That has depressing implications.

An ocean acidification reference

The seemingly thorough lack of concern of Steven Levitt and Steven Dubner for ocean acidification in the geoengineering chapter of their recent Superfreakonomics has brought this once-obscure environmental problem to the mainstream consciousness.

Of course,this means that denialist hoke is starting. ("They can't call it 'acidification' if the ocean is not turning to ACID. It's ALARMISM! RELIGION!!!!!1111!!ONE")

Those who want to bring themselves up-to-speed on the science of global warming and the associated climate changes have things easy: the IPCC Working Group 1 summarized the literature up to 2007. The IPCC doesn't cover ocean acidification at all, as it isn't climate change. The Royal Society, however, has written a very fair review, accessible to the lay reader.

Goodness of fit.

From someone working in a bio-inorganic chemistry lab of fairly strong reputation, I heard of quite the outlandish abuse of statistics.

Apparently, they were using reduced chi-square (the usual one, that assumes a gaussian distribution for the noise or of the uncertainty at each point) to fit curves to the data, and also for model selection, comparing models with different numbers of parameters. The number of parameters in the system is supposed to be physically relevant, in a way I don't recall. I don't want to identify the lab, anyway.

Using reduced chi-square to do model selection is bad enough as it is: there's no reason to take a model with a higher number of parameters as being better just because it has a lower reduced chi-square. How much lower it should be is not clear.

But the real trouble is: the reduced chi-square method comes, at least, with a sort of "warning light" for overfitting, that is, fitting models with too many free parameters. If the reduced chi-square is greater than one, one may or may not be overfitting. But if the reduced chi-square is less than one, and the noise in the data is independently distributed, it is an almost sure sign that one is overfitting, and filling in the noise.

Have a look at the formula for the reduced chi-square statistic. For a perfect fit, what value does it converge to in probability? The answer jumps right out, doesn't it.

Supposedly, at least one dissertation in this lab drew physical conclusions from fits for which the reduced chi-square was less than one. Several papers, as well, may have been published using this method.

The social scientists and the climatologists are very good about their statistics; I'd like to think we're at least getting better about statistics in biophysics. But most of the physical sciences have a long way to come. The curriculum is already somewhat overloaded, but perhaps a course in practical statistics should be required of all doctoral students in the natural sciences, just as in the social sciences; such silly mistakes should be inexcusable. The mistake was not over petty matters such as the size of the error bar (a la the never-ending trouble some people have understanding the difference between noise standard deviation and standard error of the mean.) It directly affected categorical, conceptual conclusions.

Reading phylogenetic trees.

As cladistics, facilitated by genetic comparisons and molecular-clock modeling, has become the dominant taxonomic method, the phylogenetic tree diagram has become nearly ubiquitous, but little explanation is given for its meaning. Where the branch points occur along one axis has meaning; placement on the other axis is a matter of taste.

In a 11 November 2005 short article in Science, entitled The Tree Thinking Challenge, David A. Baum and Susan DeWitt Smith of the University of Wisconsin, and Samuel S. S. Donovan at the University of Pittsburg present a simple phylogeny and ask the reader to, based on that information, determine which of two critters is more closely related to humans. Take their quiz and see how you fare. If you get it wrong, the article--a very quick read--will set you straight and establish a good habit.

A recommended climate lecturer.

Paul Kushner of the University of Toronto gives a truly outstanding hour-long introduction to climate modeling for the scientifically literate audience, managing also to slip in some of his current work, for illustrative purposes, near the end.

I got a chance to sit in on it earlier today, while visiting Stanford University. I recommend him highly as a speaker, if you get the chance to invite him.