Consensus on consensus: a synthesis of consensus estimates on human-caused global warming

Climate scientists overwhelmingly agree that humans are causing recent global warming. The consensus position is articulated by the Intergovernmental Panel on Climate Change (IPCC) statement that 'human influence has been the dominant cause of the observed warming since the mid-20th century' (Qin et al 2014, p 17). The National Academies of Science from 80 countries have issued statements endorsing the consensus position (table S2). Nevertheless, the existence of the consensus continues to be questioned. Here we summarize studies that quantify expert views and examine common flaws in criticisms of consensus estimates. In particular, we are responding to a comment by Tol (2016) on Cook et al (2013, referred to as C13). We show that contrary to Tol's claim that the results of C13 differ from earlier studies, the consensus of experts is robust across all the studies conducted by coauthors of this correspondence.

Tol's erroneous conclusions stem from conflating the opinions of non-experts with experts and assuming that lack of affirmation equals dissent. A detailed technical response to Tol is provided in (S1) where we specifically address quibbles about abstract ID numbers, timing of ratings, inter-rater communication and agreement, and access to ratings. None of those points raised by Tol affect the calculated consensus. Most importantly, the 97% consensus derived from abstract ratings is validated by the authors of the papers studied who responded to our survey (N = 2142 papers) and also reported a 97% consensus in papers taking a position. The remainder of this paper shows that a high level of scientific consensus, in agreement with our results, is a robust finding in the scientific literature. This is used to illustrate and address the issues raised by Tol that are relevant to our main conclusion.

Efforts to measure scientific consensus need to identify a relevant and representative population of experts, assess their professional opinion in an appropriate manner, and avoid distortions from ambiguous elements in the sample. Approaches that have been employed to assess expert views on anthropogenic global warming (AGW) include analysing peer-reviewed climate papers (Oreskes 2004; C13), surveying members of the relevant scientific community (Bray and von Storch 2007, Doran and Zimmerman 2009, Bray 2010, Rosenberg et al 2010, Farnsworth and Lichter 2012, Verheggen et al 2014, Stenhouse et al 2014, Carlton et al 2015), compiling public statements by scientists (Anderegg et al 2010), and mathematical analyses of citation patterns (Shwed and Bearman 2010). We define domain experts as scientists who have published peer-reviewed research in that domain, in this case, climate science. Consensus estimates for these experts are listed in table 1, with the range of estimates resulting primarily from differences in selection of the expert pool, the definition of what entails the consensus position, and differences in treatment of no position responses/papers.

The studies in table 1 have taken various approaches to selecting and querying pools of experts. Oreskes (2004) identified expressions of views on AGW in the form of peer-reviewed papers on 'global climate change'. This analysis found no papers rejecting AGW in a sample of 928 papers published from 1993 to 2003, that is, 100% consensus among papers stating a position on AGW.

Following a similar methodology, C13 analysed the abstracts of 11 944 peer-reviewed papers published between 1991 and 2011 that matched the search terms 'global climate change' or 'global warming' in the ISI Web of Science search engine. Among the 4014 abstracts stating a position on human-caused global warming, 97.1% were judged as having implicitly or explicitly endorsed the consensus. In addition, the study authors were invited to rate their own papers, based on the contents of the full paper, not just the abstract. Amongst 1381 papers self-rated by their authors as stating a position on human-caused global warming, 97.2% endorsed the consensus.

Shwed and Bearman (2010) employed citation analysis of 9432 papers on global warming and climate published from 1975 to 2008. Unlike surveys or classifications of abstracts, this method was entirely mathematical and blind to the content of the literature being examined. By determining the modularity of citation networks, they concluded, 'Our results reject the claim of inconclusive science on climate change and identify the emergence of consensus earlier than previously thought' (p. 831). Although this method does not produce a numerical consensus value, it independently demonstrates the same level of scientific consensus on AGW as exists for the fact that smoking causes cancer.

Anderegg et al (2010) identified climate experts as those who had authored at least 20 climate-related publications and chose their sample from those who had signed public statements regarding climate change. By combining published scientific papers and public statements, Anderegg et al determined that 97%–98% of the 200 most-published climate scientists endorsed the IPCC conclusions on AGW.

Other studies have directly queried scientists, typically choosing a sample of scientists and identifying subsamples of those who self-identify as climate scientists or actively publish in the field. Doran and Zimmerman (2009) surveyed 3146 Earth scientists, asking whether 'human activity is a significant contributing factor in changing mean global temperatures,' and subsampled those who were actively publishing climate scientists. Overall, they found that 82% of Earth scientists indicated agreement, while among the subset with greatest expertise in climate science, the agreement was 97.4%.

Bray and von Storch (2007) and Bray (2010) repeatedly surveyed different populations of climate scientists in 1996, 2003 and 2008. The questions did not specify a time period for climate change (indeed, in 2008, 36% of the participants defined the term 'climate change' to refer to 'changes in climate at any time for whatever reason'). Therefore, the reported consensus estimates of 40% (1996) and 53% (2003) (which included participants not stating a view on AGW) suffered from both poor control of expert selection and ambiguous questions. Their 2008 study, finding 83% agreement, had a more robust sample selection and a more specific definition of the consensus position on attribution.

Verheggen et al (2014) surveyed 1868 scientists, drawn in part from a public repository of climate scientists (the same source as was used by Anderegg et al), and from scientists listed in C13, supplemented by authors of recent climate-related articles and with particular effort expended to include signatories of public statements critical of mainstream climate science. 85% of all respondents (which included a likely overrepresentation of contrarian non-scientists) who stated a position agreed that anthropogenic greenhouse gases (GHGs) are the dominant driver of recent global warming. Among respondents who reported having authored more than 10 peer-reviewed climate-related publications, approximately 90% agreed that greenhouse gas emissions are the primary cause of global warming.

Stenhouse et al (2014) collected responses from 1854 members of the American Meteorological Society (AMS). Among members whose area of expertise was climate science, with a publication focus on climate, 78% agreed that the cause of global warming over the past 150 years was mostly human, with an additional 10% (for a total of 88%) indicating the warming was caused equally by human activities and natural causes. An additional 6% answered 'I do not believe we know enough to determine the degree of human causation.' To make a more precise comparison with the Doran and Zimmerman findings, these respondents were emailed one additional survey question to ascertain if they thought human activity had contributed to the global warming that has occurred over the past 150 years; among the 6% who received this question, 5% indicated there had been some human contribution to the warming. Thus, Stenhouse et al (2014) concluded that '93% of actively publishing climate scientists indicated they are convinced that humans have contributed to global warming.'

Carlton et al (2015) adapted questions from Doran and Zimmerman (2009) to survey 698 biophysical scientists across various disciplines, finding that 91.9% of them agreed that (1) mean global temperatures have generally risen compared with pre-1800s levels and that (2) human activity is a significant contributing factor in changing mean global temperatures. Among the 306 who indicated that 'the majority of my research concerns climate change or the impacts of climate change', there was 96.7% consensus on the existence of AGW.

The Pew Research Center (2015) conducted a detailed survey of 3748 members of the American Association for the Advancement of Science (AAAS) to assess views on several key science topics. Across this group, 87% agreed that 'Earth is warming due mostly to human activity.' Among a subset of working PhD Earth scientists, 93% agreed with this statement.

Despite the diversity of sampling techniques and approaches, a consistent picture of an overwhelming consensus among experts on anthropogenic climate change has emerged from these studies. Another recurring finding is that higher scientific agreement is associated with higher levels of expertise in climate science (Oreskes 2004, Doran and Zimmerman 2009, Anderegg 2010, Verheggen et al 2014).

How can vastly different interpretations of consensus arise? A significant contributor to variation in consensus estimates is the conflation of general scientific opinion with expert scientific opinion. Figure 1 demonstrates that consensus estimates are highly sensitive to the expertise of the sampled group. An accurate estimate of scientific consensus reflects the level of agreement among experts in climate science; that is, scientists publishing peer-reviewed research on climate change. As shown in table 1, low estimates of consensus arise from samples that include non-experts such as scientists (or non-scientists) who are not actively publishing climate research, while samples of experts are consistent in showing overwhelming consensus.

Zoom In Zoom Out Reset image size

Tol (2016) reports consensus estimates ranging from 7% to 100% from the same studies described above. His broad range is due to sub-groupings of scientists with different levels of expertise. For example, the sub-sample with 7% agreement was selected from those expressing an 'unconvinced' position on AGW (Verheggen et al 2014). This selection criterion does not provide a valid estimate of consensus for two reasons: first, this subsample was selected based on opinion on climate change, predetermining the level of estimated consensus. Second, this does not constitute a sample of experts, as non-experts were included. Anderegg (2010) found that nearly one-third of the unconvinced group lacked a PhD, and only a tiny fraction had a PhD in a climate-relevant discipline. Eliminating less published scientists from both these samples resulted in consensus values of 90% and 97%–98% for Verheggen et al (2014) and Anderegg et al (2010), respectively. Tol's (2016) conflation of unrepresentative non-expert sub-samples and samples of climate experts is a misrepresentation of the results of previous studies, including those published by a number of coauthors of this paper.

In addition to varying with expertise, consensus estimates may differ based on their approach to studies or survey responses that do not state an explicit position on AGW. Taking a conservative approach, C13 omitted abstracts that did not state a position on AGW to derive its consensus estimate of 97%; a value shown to be robust when compared with the estimate derived from author responses. In contrast, in one analysis, Tol (2016) effectively treats no-position abstracts as rejecting AGW, thereby deriving consensus values less than 35%. Equating no-position papers with rejection or an uncertain position on AGW is inconsistent with the expectation of decreasing reference to a consensual position as that consensus strengthens (Oreskes 2007, Shwed and Bearman 2010). Powell (2015) shows that applying Tol's method to the established paradigm of plate tectonics would lead Tol to reject the scientific consensus in that field because nearly all current papers would be classified as taking 'no position'.

We have shown that the scientific consensus on AGW is robust, with a range of 90%–100% depending on the exact question, timing and sampling methodology. This is supported by multiple independent studies despite variations in the study timing, definition of consensus, or differences in methodology including surveys of scientists, analyses of literature or of citation networks. Tol (2016) obtains lower consensus estimates through a flawed methodology, for example by conflating non-expert and expert views, and/or making unsupported assumptions about sources that do not specifically state a position about the consensus view.

An accurate understanding of scientific consensus, and the ability to recognize attempts to undermine it, are important for public climate literacy. Public perception of the scientific consensus has been found to be a gateway belief, affecting other climate beliefs and attitudes including policy support (Ding et al 2011, McCright et al 2013, van der Linden et al 2015). However, many in the public, particularly in the US, still believe scientists disagree to a large extent about AGW (Leiserowitz et al 2015), and many political leaders, again particularly in the US, insist that this is so. Leiserowitz et al (2015) found that only 12% of the US public accurately estimate the consensus at 91%–100%. Further, Plutzer et al 2016 found that only 30% of middle-school and 45% of high-school science teachers were aware that the scientific consensus is above 80%, with 31% of teachers who teach climate change presenting contradictory messages that emphasize both the consensus and the minority position.

Misinformation about climate change has been observed to reduce climate literacy levels (McCright et al 2016, Ranney and Clark 2016), and manufacturing doubt about the scientific consensus on climate change is one of the most effective means of reducing acceptance of climate change and support for mitigation policies (Oreskes 2010, van der Linden et al 2016). Therefore, it should come as no surprise that the most common argument used in contrarian op-eds about climate change from 2007 to 2010 was that there is no scientific consensus on human-caused global warming (Elsasser and Dunlap 2012, Oreskes and Conway 2011). The generation of climate misinformation persists, with arguments against climate science increasing relative to policy arguments in publications by conservative organisations (Boussalis and Coan 2016).

Consequently, it is important that scientists communicate the overwhelming expert consensus on AGW to the public (Maibach et al 2014, Cook and Jacobs 2014). Explaining the 97% consensus has been observed to increase acceptance of climate change (Lewandowsky et al 2013, Cook and Lewandowsky 2016) with the greatest change among conservatives (Kotcher et al 2014).

From a broader perspective, it doesn't matter if the consensus number is 90% or 100%. The level of scientific agreement on AGW is overwhelmingly high because the supporting evidence is overwhelmingly strong.

We thank Richard Tol for his comment on C13. Thanks to Neal J King and Robert Way for helpful comments on this note, and to Collin Maessen for his initial efforts contacting authors of previous consensus studies.