North's testimony under oath re. the Mann Hockey Stick

The hockey-stick graph, used prominently in the IPCC 2001 report as evidence of unprecedented global warming, became in 2005-2006 the subject of investigations by two high-level scientific panels commissioned by the US National Academy of Sciences and the U.S. House of Representatives.

The former, chaired by Gerald North, upheld the conclusions which the controversial graph and its creator, scientist Michael Mann supported, that unprecedented warming had occurred recently. The latter, chaired by Edward Wegman, unequivocally determined the hockey stick to be based on shoddy science, and thus failed to provide any evidence to uphold beliefs that recent global warming is "unprecedented".

When testifying before Congress under oath, North and his statistician agreed with the findings of the Wegman panel. But the press and the public came away with the impression that Mann's hockey stick had been vindicated. What happened? The following narrative details the key events and exact words.

Hockey Stick controversy generates two committees to investigate it

Because Mann had refused to provide other researchers with the computer code necessary to verify his work, because the credibility of the science surrounding the hockey stick had become a cause célèbre, and because federal government funds had financed the hockey stick study, the Energy and Commerce committee had decided in 2005 to hold hearings into the matter. The committee commissioned a study by Edward Wegman, arguably America’s top statistician (Wegman was the National Academy of Sciences’s own Chair of its Committee on Applied and Theoretical Statistics), and arranged for him to testify in July, 2006.

But meanwhile, the NAS decided to produce a competing study into the Mann controversy, with North as its chair. The Energy and Commerce committee then decided to have Wegman and North both testify before it. Now of all the scientists who have come to Michael Mann’s defence, none have more impressive credentials than those of Gerald North, a former Head of the Department of Atmospheric Sciences at Texas A&M University. North, a physicist, has not only spent decades addressing the dangers of climate change, he has done so through his work in climate models and his knowledge of statistics, a suite of qualifications that make him particularly well qualified to comment on Michael Mann’s statistics-based work. Because of his background, and because Mann’s hockey-stick graph had become a source of great controversy, the National Academy of Sciences (NAS) asked North to chair a panel to investigate the statistical validity of the hockey stick graph. The NAS, like most national academies, backs the man-made global warming thesis.

North report is widely understood to vindicate Mann's Hockey Stick

The North report for the NAS was published in June 2006, a month before the scheduled US Congress hearing, with a press conference and press release that said: "There is sufficient evidence from tree rings, retreating glaciers, and other "proxies" to say with confidence that the last few decades of the 20th century were warmer than any comparable period in the last 400 years, according to a new National Research Council report..."

North said "I have no cause to think that there was anything inappropriate, professionally."

and Peter Bloomfield (North's statistician) said "Mann’s methods were all quite reasonable choices. I think in some cases a lot of work by others in following up on that have showed that some of those choices could have been made better, but they were quite plausible at the time. I would not have been embarrassed by the work at the time, had I been involved in it and I certainly saw nothing that spoke to me of any manipulation or anything other than an honest attempt at constructing a data analysis procedure."

The worldwide press understandably reported that Mann had been vindicated.

A very different picture emerges when North testifies under oath

When North and other panel members, less than one month later, were required to testify under oath, they reveal something seriously different. The setting was now not a press conference but formal hearings before the Energy and Commerce Committee of the U.S. House of Representatives.Wegman’s testimony powerfully demonstrated that Mann’s work had no validity. But then North testified in Mann’s defence, also powerfully, by showing that Mann’s conclusions were valid.

From the US Congress hearing (Energy and Commerce Subcommittee, July 2006) in which NORTH, WEGMAN, MCINTYRE and others testified:

DR. NORTH. It is difficult to see how [the social networking data] has any bearing on the peer-review process, the need to include statisticians on every team that engages in climate research (which in my view is a particularly unrealistic and unnecessary recommendation), or *any of the other findings and recommendations in Dr. Wegman's report.* I was also somewhat taken aback by the tone of the Wegman Report, which seems overly accusatory towards Dr. Mann and his colleagues, rather than being a neutral, impartial assessment of the techniques used in his research. In my opinion, while the techniques used in the original Mann et al papers may have been slightly flawed, the work was the first of its kind and deserves considerable credit for moving the field of paleoclimate research forward. It is also important to note that the main conclusions of the Mann et al studies have been supported by subsequent research. Finally, while our committee would agree with Dr. Wegman that access to research data could and should be improved, as discussed on page 23 of the prepublication version of our report, we also acknowledge the complicated nature of such mandates, especially in areas such as computer code where intellectual property rights need to be considered.

This is often quoted in defence of the North report vindicating Mann.

**************************************

But at the July 19 hearing, Barton asked North very precisely whether he disagreed with any Wegman's findings and North (under oath) said no as follows:

CHAIRMAN BARTON. I understand that. It looks like my time is expired, so I want to ask one more question. Dr. North, do you dispute the conclusions or the methodology of Dr. Wegman's report?

DR. NORTH. No, we don't. We don't disagree with their criticism. In fact, pretty much the same thing is said in our report. But again, just because the claims are made, doesn't mean they are false.

CHAIRMAN BARTON. I understand that you can have the right conclusion and that it not be–

DR. NORTH. It happens all the time in science.

CHAIRMAN BARTON. Yes, and not be substantiated by what you purport to be the facts but have we established–we know that Dr. Wegman has said that Dr. Mann's methodology is incorrect. Do you agree with that? I mean, it doesn't mean Dr. Mann's conclusions are wrong, but we can stipulate now that we have–and if you want to ask your statistician expert from North Carolina that Dr. Mann's methodology cannot be documented and cannot be verified by independent review.

DR. NORTH. Do you mind if he speaks?

CHAIRMAN BARTON. Yes, if he would like to come to the microphone.

MR. BLOOMFIELD. Thank you. Yes, Peter Bloomfield. Our committee reviewed the methodology used by Dr. Mann and his coworkers and we felt that some of the choices they made were inappropriate. We had much the same misgivings about his work that was documented at much greater length by Dr. Wegman.

********************************

What was going on? North’s NAS panel confirmed, without stating so clearly, that Mann’s science was shoddy, and that Mann’s conclusions, on their own, could not be trusted. But that didn’t mean that Mann’s answer was wrong — North’s panel believed that man-made global warming exists and they had lots of evidence, by other scientists, to support their belief. Therefore, the NAS panel concluded, Mann was right in his ultimate conclusion that man causes global warming, even if Mann’s study provided no basis for that conclusion.

********************************

THE NORTH COMMITTEE was composed of 12 experts in various aspects of climate research and statistics.

From the North report:

"The basic conclusion of Mann et al. was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years. This conclusion has subsequently been supported by an array of evidence that includes both additional large-scale surface temperature reconstructions and pronounced changes in a variety of local proxy indicators, such as melting on ice caps and the retreat of glaciers around the world, which in many cases appear to be unprecedented during at least the last 2,000 years. Not all individual proxy records indicate that the recent warmth is unprecedented, although a larger fraction of geographically diverse sites experienced exceptional warmth during the late 20th century than during any other extended period from A.D. 900 onward."

"The committee finds it plausible that the northern hemisphere was warmer during the last few decades of the 20th century than during any comparable period over the preceding millennium."

"In practice, [Mann's PCA] method, though not recommended, does not appear to unduly influence reconstructions of hemispheric mean temperature; reconstructions performed without using principal component analysis are qualitatively similar to the original curves presented by Mann et al."

"The panel did not support Mann's conclusions about the 1990s and 1998 because that is too specific a claim to make given the data. The evidence does support that recent temperatures are the warmest in a millennium as the above quotes make clear."

*****************************

WEGMAN'S "BLUE CHIP PANEL" was Wegman himself, fellow statistician David Scott, and grad student Yasmin Said. It had nothing to do with the National Academy of Sciences and had no formal peer review. The NAS Panel had 12 members including skeptic John Christy, plus 13 anonymous reviewers, and two anonymous monitors to make sure that all reviewer comments were addressed. The NAS panel did not agree that the problem was significant or that any deception was involved.

From the Wegman report:

The debate over Dr. Mann's principal components methodology has been going on for nearly three years. When we got involved, there was no evidence that a single issue was resolved or even nearing resolution. Dr. Mann's RealClimate.org website said that all of the Mr. McIntyre and Dr. McKitrick claims had been 'discredited'. UCAR had issued a news release saying that all their claims were 'unfounded'. Mr. McIntyre replied on the ClimateAudit.org website. The climate science community seemed unable to either refute McIntyre's claims or accept them. The situation was ripe for a third-party review of the types that we and Dr. North's NRC panel have done.

While the work of Michael Mann and colleagues presents what appears to be compelling evidence of global temperature change, the criticisms of McIntyre and McKitrick, as well as those of other authors mentioned are indeed valid.

Where we have commonality, I believe our report and the [NAS] panel essentially agree. We believe that our discussion together with the discussion from the NRC report should take the 'centering' issue off the table. [Mann's] decentred methodology is simply incorrect mathematics …. I am baffled by the claim that the incorrect method doesn't matter because the answer is correct anyway.

Method Wrong + Answer Correct = Bad Science.

The papers of Mann et al. in themselves are written in a confusing manner, making it difficult for the reader to discern the actual methodology and what uncertainty is actually associated with these reconstructions.

It is not clear that Dr. Mann and his associates even realized that their methodology was faulty at the time of writing the [Mann] paper.

We found MBH98 and MBH99 to be somewhat obscure and incomplete, and the criticisms of MM03/05a/05b to be valid and compelling.

Overall, our committee believes that Mann's assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis.

[The] fact that their paper fit some policy agendas has greatly enhanced their paper's visibility… The 'hockey stick' reconstruction of temperature graphic dramatically illustrated the global warming issue and was adopted by the IPCC and many governments as the poster graphic. The graphics' prominence together with the fact that it is based on incorrect use of [principal components analysis] puts Dr. Mann and his co-authors in a difficult face-saving position.

We have been to Michael Mann's University of Virginia website and downloaded the materials there. Unfortunately, we did not find adequate material to reproduce the MBH98 materials. We have been able to reproduce the results of McIntyre and McKitrick

Generally speaking, the paleoclimatology community has not recognized the validity of the [McIntyre and McKitrick] papers and has tended to dismiss their results as being developed by biased amateurs. The paleoclimatology community seems to be tightly coupled as indicated by our social network analysis, has rallied around the [Mann] position, and has issued an extensive series of alternative assessments, most of which appear to support the conclusions of MBH98/99… Our findings from this analysis suggest that authors in the area of paleoclimate studies are closely connected and thus 'independent studies' may not be as independent as they might appear on the surface.

It is important to note the isolation of the paleoclimate community; even though they rely heavily on statistical methods, they do not seem to be interacting with the statistical community. Additionally, we judge that the sharing of research materials, data and results was haphazardly and grudgingly done. In this case we judge that there was too much reliance on peer review, which was not necessarily independent.

Based on the literature we have reviewed, there is no overarching consensus on [Mann's work]. As analyzed in our social network, there is a tightly knit group of individuals who passionately believe in their thesis. However, our perception is that this group has a self-reinforcing feedback mechanism and, moreover, the work has been sufficiently politicized that they can hardly reassess their public positions without losing credibility.

It is clear that many of the proxies are re-used in most of the papers. It is not surprising that the papers would obtain similar results and so cannot really claim to be independent verifications.

Especially when massive amounts of public monies and human lives are at stake, academic work should have a more intense level of scrutiny and review. It is especially the case that authors of policy-related documents like the IPCC report, Climate Change 2001: The Scientific Basis, should not be the same people as those that constructed the academic papers.

*****************************

Question to Gerald North in a live discussion September 2006

Even the National Academy of Sciences could not support Mann's use of statistics. But they praised Mann's efforts and clearly still believed in unprecedented global warming, although the Hockey Stick's evidence for this had been declared invalid. The press can be forgiven for believing that the NAS panel chaired by North had vindicated the evidence that depended on the science behind the hockey stick graph, when it had actually not done so.

The NAS cited other evidence they believed upheld the "warming" thesis. At that time, there was no clear challenge to this other evidence - but challenge has since emerged in abundance, mainly through Steve McIntyre's continuing efforts to expose shoddy statistics where they uphold unproven climate beliefs of major import, and to press for transparency and public availability of data to check results, as would be considered standard scientific practice elsewhere.