In the Line of Fire - The Hockey Stick and the Climate Wars - page 72

about the earlier medieval portion of their reconstruction because of the relatively small sample size
of their tree ring dataset during that time period—which could be especially problematic with their
particular method. 15
Cook and colleagues conceded some of these points in a follow-up paper. 16 When one accounted
for the differing nature of the Esper et al. reconstruction and the hockey stick with regard to seasonal
and spatial representativeness and simple issues such as the scaling of the records, many of the
apparent discrepancies were resolved—though not all. For example, the thirteenth century (which
falls within the medieval warm period of most reconstructions) remains quite a bit colder in the
Esper et al. reconstruction than in other published reconstructions. 17
The exchanges over the Esper et al. paper represent scientific give-and-take at its best: good
faith challenges and responses between colleagues that took place both formally in the peer reviewed
literature and informally in private correspondence among authors. We differed in our interpretations
of the data, but we were respectful of one another's work. Our aim was the same: a better scientific
understanding.
In 2002, I introduced the notion of “pseudoproxy” data as a means of testing climate
reconstruction methods. 18 The idea was fairly simple: Introduce contaminating noise to actual
temperature records (or model temperature output) to create synthetic proxy datasets that approximate
the imperfect and noisy character of real-world proxy records. One can then test a climate
reconstruction method using these synthetic proxy records and compare against the actual
temperatures, which are known, to evaluate the performance of the method. In 2004, Hans Von Storch
of Germany and his collaborators also used pseudoproxy tests of this sort, 19 employing a brand-new
climate model simulation of the past thousand years—a potentially richer test bed for investigating
climate reconstruction methods than those my colleagues and I had used in our past work. Based on
these tests, Von Storch et al. argued that reconstruction methods substantially underestimated the
amount of long-term variability in past temperatures—in other words, they underestimated the amount
of wiggle in the “handle” of the hockey stick, and other related climate reconstructions.
Their principal finding—that so-called “regression” methods underestimate the amplitude of
variation—was already familiar. We had discussed the issue in 2002 in our earliest work with
pseudoproxy networks, 20 and it was accounted for in the very wide error bars shown on the
millennial MBH99 hockey stick. Though such criticisms applied equally to virtually all statistical
climate reconstruction approaches, Von Storch et al. focused their attention on the hockey stick,
perhaps because it was the most prominent of the published climate reconstructions. The critique
would have been perfectly fair and appropriate were it not for a number of significant complications
to the story.
In a press release put out by his institution, Von Storch referred to the hockey stick as quatsch —
the German equivalent of nonsens e or garbage . This was shocking, arguably unprofessional language
coming from a fellow scientist. But that was a matter of style. Perhaps more important, there were
some problems of substance. There was a significant methodological error in the Von Storch study
that would largely undermine its conclusions. While it was well known that regression methods tend
to underestimate the amplitude of variability, the real issue is by how much in the particular situation,
and whether that underestimation is accounted for in the size of the error bars. The level of
underestimation suggested in the Von Storch et al. analysis seemed implausibly large. After repeated
inquiries from other researchers, it eventually came to light that there was a key undisclosed