In March 2007, Rajendra Pachauri, the chairman of the Intergovernmental Panel on Climate Change (IPCC) was interviewed by a reporter from the Bloomberg business news service. The discussion centered on the soon-to-be-released second installment of the IPCC's newly updated climate bible.
Tangentially, Pachauri was asked about the Stern Review, a report written by economists employed by the British government. Pachauri told Bloomberg the IPCC was aware of the 700-page report but that his organization's ability to make use of it was limited because it was not peer-reviewed.
Imagine my surprise therefore, when an audit of IPCC references I organized recently revealed that the IPCC had cited the Stern Review all over the place. Not once or twice. And not in a chapter or two. I'm talking at least 25 times across 12 chapters.
Given that Pachauri told a reporter that relying on this report would be improper why would the IPCC cite it on this page, this page, this page - and on two separate occasions on this page?
Why would the Stern Review be used as the sole supporting evidence for an IPCC claim regarding how many people in India and China depend on glaciers for their water supply? Why would it be cited on this page, this page, this page, twice on this page, on this page, this page, this page, in an executive summary here, and five times on this page?
Why would it be mentioned here, here, here, two more times here, and here as well? I mean, how many more times could it possibly have been cited had it been a full-fledged, peer-reviewed document?
But that isn't the only irregularity. The IPPC likes to brag about its allegedly rigorous internal review process. After the first draft is written, expert reviewers are invited to offer their feedback. A second draft is then prepared and, once again, reviewers are invited to submit comments.
The IPPC is at perfect liberty to ignore these comments, but that is a discussion for another time. What's important here is that, in order to be eligible for inclusion in the IPCC report, documents had to be published prior to a hard deadline. As chairman Pachauri explained in an essay four days ago, the 2007 IPPC report:
...was based on scientific studies completed before January 2006, and did not include later studies...
The big reason for this deadline is that all material discussed in the IPCC report needed to be available for inspection by the expert reviewers.
Moreover, there were cutoff dates after which comments from said reviewers were no longer accepted. For the two sections of the IPCC report where references to the Stern Review abound those dates were July 21 and September 15, 2006.
So guess when the Stern Review was released? Not until October 30th - ten full months after the publication date deadline, and well after the expert reviewers were out of the picture. Which begs the question: why bother with an elaborate internal review process if, after all the reviewers go home, you're going to insert new material into 12 different chapters?
Not one of the "2,500 expert reviewers" Pachauri boasted about as recently as four days ago was given any opportunity to read the 700-page Stern Review - never mind advise the IPCC as to whether that report disregards evidence from bona fide peer-reviewed studies.
The conclusion here isn't pretty: by citing the Stern Review, the IPPC broke not one, not two, but three of its own rules. First, it had to deliberately overlook the fact that this document is not peer-reviewed. (See examples here of Pachauri claiming that the IPCC bases its report solely on peer-reviewed literature and that non-peer-reviewed material belongs "in the dustbin".)
Second, it had to violate the published-before-January-2006 rule about which Pachauri recently reminded us.
Third, it had to subvert its own requirement that text in the IPCC report be subject to two rounds of expert review.
Are we impressed yet?
..
April 25th UPDATE: The orginal post implied, in the first line of paragraph six at the first link, that the Stern Review was cited twice on this page. Actually, it's mentioned twice by name within the same paragraph, but there's only one citation. I've therefore removed the "twice" that originally appeared in that clause. Having now quadruple-checked the numbers, it seems the 2007 IPCC report cited the Stern Review on 26 separate occasions. I arrived at my findings this way (clicking on the links in paragraphs 4, 5 and 6 above will take you to each of these pages):
Working Group 2
-page 4.7.2 cites the Stern Review once
-page 6.6.5 cites it once
-page 7.5 cites it once
-page 9.1 cites it twice (5 so far)
-page 10.4.2 cites it once
-page 16.5.2 cites it once
-page 16.5.4 cites it once
-page 16.6 cites it once
-page 18.4.2 cites it twice (11 so far)
-page 18.5 cites it once
-page 19.3.2 cites it once
-page 19.3.7 cites it once
-page 20 cites it once
-page 20.6.1 cites it 5 times
TOTAL: 20 citations, 9 chapters
Working Group 3
-page 1.2.2 cites it once
-page 1.2.3 cites it once
-page 1.2.3.2 cites it once
-page 9.1 cites it twice
-page 11.5.1 cites it once
TOTAL: 6 citations, 3 chapters
..
>> Climate bible gets 21 'F's on report card
>> The great peer-review fairy tale
>> What's left if we disregard non-peer-reviewed claims?
>> Dr. Pachauri, call your office
>> The water cannon of the climate debate
