North's testimony under oath re. the Mann
Hockey Stick
The hockey-stick graph, used prominently in
the IPCC 2001 report as evidence of unprecedented global
warming, became in 2005-2006 the subject of investigations
by two high-level scientific panels commissioned by the
US National Academy of Sciences and the U.S. House of Representatives.
The former, chaired by Gerald North, upheld
the conclusions which the controversial graph and its creator,
scientist Michael Mann supported, that unprecedented warming
had occurred recently. The latter, chaired by Edward Wegman,
unequivocally determined the hockey stick to be based on
shoddy science, and thus failed to provide any evidence
to uphold beliefs that recent global warming is "unprecedented".
When testifying before Congress
under oath, North and his statistician agreed with the findings
of the Wegman panel. But the press and the public came away
with the impression that Mann's hockey stick had been vindicated.
What happened? The following narrative details the key events
and exact words.
Hockey Stick controversy generates
two committees to investigate it
Because Mann had refused to provide other
researchers with the computer code necessary to verify his
work, because the credibility of the science surrounding
the hockey stick had become a cause célèbre,
and because federal government funds had financed the hockey
stick study, the Energy and Commerce committee had decided
in 2005 to hold hearings into the matter. The committee
commissioned a study by Edward Wegman, arguably America’s
top statistician (Wegman was the National Academy of Sciences’s
own Chair of its Committee on Applied and Theoretical Statistics),
and arranged for him to testify in July, 2006.
But meanwhile, the NAS decided to produce
a competing study into the Mann controversy, with North
as its chair. The Energy and Commerce committee then decided
to have Wegman and North both testify before it. Now of
all the scientists who have come to Michael Mann’s
defence, none have more impressive credentials than those
of Gerald North, a former Head of the Department of Atmospheric
Sciences at Texas A&M University. North, a physicist,
has not only spent decades addressing the dangers of climate
change, he has done so through his work in climate models
and his knowledge of statistics, a suite of qualifications
that make him particularly well qualified to comment on
Michael Mann’s statistics-based work. Because of his
background, and because Mann’s hockey-stick graph
had become a source of great controversy, the National Academy
of Sciences (NAS) asked North to chair a panel to investigate
the statistical validity of the hockey stick graph. The
NAS, like most national academies, backs the man-made global
warming thesis.
North report is widely understood
to vindicate Mann's Hockey Stick
The North report for the NAS was published
in June 2006, a month before the scheduled US Congress hearing,
with a press conference and press release that said:
"There is sufficient evidence from tree rings,
retreating glaciers, and other "proxies" to say
with confidence that the last few decades of the 20th century
were warmer than any comparable period in the last 400 years,
according to a new National Research Council report..."
North said "I have no cause to think
that there was anything inappropriate, professionally."
and Peter Bloomfield (North's statistician)
said "Mann’s methods were all quite
reasonable choices. I think in some cases a lot of work
by others in following up on that have showed that some
of those choices could have been made better, but they were
quite plausible at the time. I would not have been embarrassed
by the work at the time, had I been involved in it and I
certainly saw nothing that spoke to me of any manipulation
or anything other than an honest attempt at constructing
a data analysis procedure."
The worldwide press understandably reported
that Mann had been vindicated.
A very different picture emerges
when North testifies under oath
When North and other panel members, less than
one month later, were required to testify under oath, they
reveal something seriously different. The setting was now
not a press conference but formal hearings before the Energy
and Commerce Committee of the U.S. House of Representatives.Wegman’s
testimony powerfully demonstrated that Mann’s work
had no validity. But then North testified in Mann’s
defence, also powerfully, by showing that Mann’s conclusions
were valid.
From the US
Congress hearing (Energy and Commerce Subcommittee,
July 2006) in which NORTH, WEGMAN, MCINTYRE and others testified:
DR. NORTH. It is difficult to see how
[the social networking data] has any bearing on the peer-review
process, the need to include statisticians on every team
that engages in climate research (which in my view is a
particularly unrealistic and unnecessary recommendation),
or *any of the other findings and recommendations in Dr.
Wegman's report.* I was also somewhat taken aback by the
tone of the Wegman Report, which seems overly accusatory
towards Dr. Mann and his colleagues, rather than being a
neutral, impartial assessment of the techniques used in
his research. In my opinion, while the techniques used in
the original Mann et al papers may have been slightly flawed,
the work was the first of its kind and deserves considerable
credit for moving the field of paleoclimate research forward.
It is also important to note that the main conclusions of
the Mann et al studies have been supported by subsequent
research. Finally, while our committee would agree with
Dr. Wegman that access to research data could and should
be improved, as discussed on page 23 of the prepublication
version of our report, we also acknowledge the complicated
nature of such mandates, especially in areas such as computer
code where intellectual property rights need to be considered.
This is often quoted in defence of the North
report vindicating Mann.
**************************************
But at the July 19 hearing, Barton asked North
very precisely whether he disagreed with any Wegman's findings
and North (under oath) said no as follows:
CHAIRMAN BARTON. I understand that. It
looks like my time is expired, so I want to ask one more
question. Dr. North, do you dispute the conclusions or the
methodology of Dr. Wegman's report?
DR. NORTH. No, we don't. We don't
disagree with their criticism. In fact, pretty much the
same thing is said in our report. But again, just
because the claims are made, doesn't mean they are false.
CHAIRMAN BARTON. I understand that you
can have the right conclusion and that it not be–
DR. NORTH. It happens all the time in
science.
CHAIRMAN BARTON. Yes, and not be substantiated
by what you purport to be the facts but have we established–we
know that Dr. Wegman has said that Dr. Mann's methodology
is incorrect. Do you agree with that? I mean, it doesn't
mean Dr. Mann's conclusions are wrong, but we can stipulate
now that we have–and if you want to ask your statistician
expert from North Carolina that Dr. Mann's methodology cannot
be documented and cannot be verified by independent review.
DR. NORTH. Do you mind if he speaks?
CHAIRMAN BARTON. Yes, if he would like
to come to the microphone.
MR. BLOOMFIELD. Thank you. Yes, Peter
Bloomfield. Our committee reviewed the methodology used
by Dr. Mann and his coworkers and we felt that some of the
choices they made were inappropriate. We had much
the same misgivings about his work that was documented at
much greater length by Dr. Wegman.
********************************
What was going on? North’s NAS panel
confirmed, without stating so clearly, that Mann’s
science was shoddy, and that Mann’s conclusions, on
their own, could not be trusted. But that didn’t mean
that Mann’s answer was wrong — North’s
panel believed that man-made global warming exists and they
had lots of evidence, by other scientists, to support their
belief. Therefore, the NAS panel concluded, Mann was right
in his ultimate conclusion that man causes global warming,
even if Mann’s study provided no basis for that conclusion.
********************************
THE NORTH COMMITTEE was composed of 12 experts
in various aspects of climate research and statistics.
From the North report:
"The basic conclusion of Mann et al.
was that the late 20th century warmth in the Northern Hemisphere
was unprecedented during at least the last 1,000 years.
This conclusion has subsequently been supported by an array
of evidence that includes both additional large-scale surface
temperature reconstructions and pronounced changes in a
variety of local proxy indicators, such as melting on ice
caps and the retreat of glaciers around the world, which
in many cases appear to be unprecedented during at least
the last 2,000 years. Not all individual proxy records indicate
that the recent warmth is unprecedented, although a larger
fraction of geographically diverse sites experienced exceptional
warmth during the late 20th century than during any other
extended period from A.D. 900 onward."
"The committee finds it plausible that
the northern hemisphere was warmer during the last few decades
of the 20th century than during any comparable period over
the preceding millennium."
"In practice, [Mann's PCA] method, though
not recommended, does not appear to unduly influence reconstructions
of hemispheric mean temperature; reconstructions performed
without using principal component analysis are qualitatively
similar to the original curves presented by Mann et al."
"The panel did not support Mann's conclusions
about the 1990s and 1998 because that is too specific a
claim to make given the data. The evidence does support
that recent temperatures are the warmest in a millennium
as the above quotes make clear."
*****************************
WEGMAN'S "BLUE CHIP PANEL" was Wegman
himself, fellow statistician David Scott, and grad student
Yasmin Said. It had nothing to do with the National Academy
of Sciences and had no formal peer review. The NAS Panel
had 12 members including skeptic John Christy, plus 13 anonymous
reviewers, and two anonymous monitors to make sure that
all reviewer comments were addressed. The NAS panel did
not agree that the problem was significant or that any deception
was involved.
From the Wegman report:
The debate over Dr. Mann's principal components
methodology has been going on for nearly three years. When
we got involved, there was no evidence that a single issue
was resolved or even nearing resolution. Dr. Mann's RealClimate.org
website said that all of the Mr. McIntyre and Dr. McKitrick
claims had been 'discredited'. UCAR had issued a news release
saying that all their claims were 'unfounded'. Mr. McIntyre
replied on the ClimateAudit.org website. The climate science
community seemed unable to either refute McIntyre's claims
or accept them. The situation was ripe for a third-party
review of the types that we and Dr. North's NRC panel have
done.
While the work of Michael Mann and colleagues
presents what appears to be compelling evidence of global
temperature change, the criticisms of McIntyre and McKitrick,
as well as those of other authors mentioned are indeed valid.
Where we have commonality, I believe our report
and the [NAS] panel essentially agree. We believe that our
discussion together with the discussion from the NRC report
should take the 'centering' issue off the table. [Mann's]
decentred methodology is simply incorrect mathematics ….
I am baffled by the claim that the incorrect method doesn't
matter because the answer is correct anyway.
Method Wrong + Answer Correct = Bad Science.
The papers of Mann et al. in themselves are
written in a confusing manner, making it difficult for the
reader to discern the actual methodology and what uncertainty
is actually associated with these reconstructions.
It is not clear that Dr. Mann and his associates
even realized that their methodology was faulty at the time
of writing the [Mann] paper.
We found MBH98 and MBH99 to be somewhat obscure
and incomplete, and the criticisms of MM03/05a/05b to be
valid and compelling.
Overall, our committee believes that Mann's
assessments that the decade of the 1990s was the hottest
decade of the millennium and that 1998 was the hottest year
of the millennium cannot be supported by his analysis.
[The] fact that their paper fit some policy
agendas has greatly enhanced their paper's visibility…
The 'hockey stick' reconstruction of temperature graphic
dramatically illustrated the global warming issue and was
adopted by the IPCC and many governments as the poster graphic.
The graphics' prominence together with the fact that it
is based on incorrect use of [principal components analysis]
puts Dr. Mann and his co-authors in a difficult face-saving
position.
We have been to Michael Mann's University
of Virginia website and downloaded the materials there.
Unfortunately, we did not find adequate material to reproduce
the MBH98 materials. We have been able to reproduce the
results of McIntyre and McKitrick
Generally speaking, the paleoclimatology community
has not recognized the validity of the [McIntyre and McKitrick]
papers and has tended to dismiss their results as being
developed by biased amateurs. The paleoclimatology community
seems to be tightly coupled as indicated by our social network
analysis, has rallied around the [Mann] position, and has
issued an extensive series of alternative assessments, most
of which appear to support the conclusions of MBH98/99…
Our findings from this analysis suggest that authors in
the area of paleoclimate studies are closely connected and
thus 'independent studies' may not be as independent as
they might appear on the surface.
It is important to note the isolation of the
paleoclimate community; even though they rely heavily on
statistical methods, they do not seem to be interacting
with the statistical community. Additionally, we judge that
the sharing of research materials, data and results was
haphazardly and grudgingly done. In this case we judge that
there was too much reliance on peer review, which was not
necessarily independent.
Based on the literature we have reviewed,
there is no overarching consensus on [Mann's work]. As analyzed
in our social network, there is a tightly knit group of
individuals who passionately believe in their thesis. However,
our perception is that this group has a self-reinforcing
feedback mechanism and, moreover, the work has been sufficiently
politicized that they can hardly reassess their public positions
without losing credibility.
It is clear that many of the proxies are re-used
in most of the papers. It is not surprising that the papers
would obtain similar results and so cannot really claim
to be independent verifications.
Especially when massive amounts of public
monies and human lives are at stake, academic work should
have a more intense level of scrutiny and review. It is
especially the case that authors of policy-related documents
like the IPCC report, Climate Change 2001: The Scientific
Basis, should not be the same people as those that constructed
the academic papers.
*****************************
Question to Gerald North in a live
discussion September 2006
Even the National Academy of Sciences
could not support Mann's use of statistics. But they praised
Mann's efforts and clearly still believed in unprecedented
global warming, although the Hockey Stick's evidence for
this had been declared invalid. The press can be forgiven
for believing that the NAS panel chaired by North had
vindicated the evidence that depended on the science behind
the hockey stick graph, when it had actually not
done so.
The NAS cited other evidence they
believed upheld the "warming" thesis. At that
time, there was no clear challenge to this other evidence
- but challenge has since emerged in abundance, mainly through
Steve McIntyre's continuing efforts to expose shoddy statistics
where they uphold unproven climate beliefs of major import,
and to press for transparency and public availability of
data to check results, as would be considered standard scientific
practice elsewhere.