John CookpersName, Dana NuccitellipersName, Sarah A. GreenpersName (2013date). “Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature.” Environmental Research Letterstitle·j, BrisbaneplaceName.
Object of study — The positionterm·key taken on anthropogenic global warmingterm·dom by 11,944 peer-reviewed climate abstractsterm·var published between 1991date and 2011date.
Research question — The public believes climateterm·key scientists are split - what proportion of the literature that takes a positionterm·key on the cause of warmingterm·key actually endorses the human one?
Contribution — 97.1% of abstractsterm·key stating a positionterm·key endorse anthropogenicterm·key warmingterm·key, confirmed at 97.2% by an independent author self-ratingterm·mth arm.
phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | scientific consensus on anthropogenic global warming — “We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics ‘global climate change’ or ‘global warming’.” | |
| claim-type | existence — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.” | |
| method — evidence | — “We classified each abstract according to the type of research (category) and degree of endorsement.” | |
| object of study — evidence | — “Through analysis of climate-related papers published from 1991 to 2011, this study provides the most comprehensive analysis of its kind to date in order to quantify and evaluate the level and evolution of consensus over the last two decades.” | |
| research question — evidence | — “Despite numerous indicators of a consensus, there is wide public perception that climate scientists disagree over the fundamental cause of global warming (GW; Leiserowitz et al 2012 , Pew 2012 ).” | |
| contribution — evidence | — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.” | |
| transfer | from social science (content analysis / systematic coding): a content-analysis coding methodology (independent raters categorizing texts) applied to quantify the scientific consensus on anthropogenic global warming — “Each abstract was categorized by two independent, anonymized raters” | transfers[0] |
| mechanism | two independent blind raters per abstract, with identifying information stripped, so a rating cannot be anchored on the author or the journal — “Abstracts were randomly distributed via a web-based system to raters with only the title and abstract visible.” | mechanisms[0] |
| mechanism | disagreement is not hidden but reconciled in stages - raters see each other's ratings and revise, then a third party resolves the residue — “Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity.” | mechanisms[1] |
| mechanism | a second, independent measurement of the same quantity: authors rate their own full papers against the same rubric, which tests whether abstract-level coding under-reads a paper's position — “Authors were emailed an invitation to participate in a survey in which they rated their own published papers (the entire content of the article, not just the abstract) with the same criteria as used by the independent rating team.” | mechanisms[2] |
| mechanism | 'no position' is separated from 'uncertain' by re-reading a random 1,000 of the former, because silence and doubt are different states that a coarse scheme would merge — “Upon completion of the final ratings, a random sample of 1000 ‘No Position’ category abstracts were re-examined to differentiate those that did not express an opinion from those that take the position that the cause of GW is uncertain.” | mechanisms[3] |
| mechanism | THE SAME PAPERS, TWO RATERS, TWO ANSWERS - AND THE SECOND RATER IS THE AUTHOR: `In a second phase of this study, we invited authors to rate their own papers. Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%)` against 66.4% by abstract rating. WHETHER A PAPER TAKES A POSITION DEPENDS ON WHO IS READING IT, and this paper measures the gap rather than choosing a side. — “In a second phase of this study, we invited authors to rate their own papers. Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%)” | mechanisms[4] |
| mechanism | AND `ENDORSEMENT` IS OPERATIONALISED WITH AN INFERENCE STEP BUILT INTO THE CATEGORY: level `(3) IMPLICIT ENDORSEMENT - IMPLIES humans are causing global warming. E.g., RESEARCH ASSUMES greenhouse gas emissions cause warming without explicitly stating humans are the cause.` A paper counts as endorsing when the rater judges that it assumes the conclusion, so the construct deliberately includes what an abstract does not say. — “(3) Implicit endorsement Implies humans are causing global warming. E.g., research assumes greenhouse gas emissions cause warming without explicitly stating humans are the cause” | mechanisms[5] |
| extends | Oreskes (2004), analysis of peer-reviewed climate papers — “In the most comprehensive analysis performed to date, we have extended the analysis of peer-reviewed climate papers in Oreskes ( 2004 ).” | claim_relations[0] |
| replicates | Oreskes (2004), the finding that no papers in the 1993-2003 sample reject the consensus — “Of the remaining 894, none rejected the consensus, consistent with Oreskes’ result.” | claim_relations[1] |
| contradicts | WITHDRAWN AND REPLACED (RETRACTED AND RE-ADDED WITH THE QUANTITY CORRECTED) · Schulte (2008), which reported 6% rejection among 2004-2007 abstracts — “Schulte also found a significantly greater number of rejection papers, including 6 explicit rejections compared to our 0 explicit rejections.” | claim_relations[2] |
| replicates | Bray (2010), the finding of a strengthening consensus over time — “For both self-ratings and our abstract ratings, the percentage of endorsements among papers expressing a position on AGW marginally increased over time, consistent with Bray ( 2010 ) in finding a strengthening consensus.” | claim_relations[3] |
| extends | Oreskes (2004), which found no disagreeing papers among 928 abstracts — “An analysis of abstracts published from 1993–2003 matching the search ‘global climate change’ found that none of 928 papers disagreed with the consensus position on AGW (Oreskes 2004 ).” | claim_relations[4] |
| replicates | direct surveys of climate scientists reporting 97–98% agreement — “Surveys of climate scientists have found strong agreement (97–98%) regarding AGW amongst publishing climate experts (Doran and Zimmerman 2009 , Anderegg et al 2010 ).” | claim_relations[5] |
| contradicts | the public perception that climate scientists disagree about the cause of warming — “Despite numerous indicators of a consensus, there is wide public perception that climate scientists disagree over the fundamental cause of global warming (GW; Leiserowitz et al 2012 , Pew 2012 ).” | claim_relations[6] |
| contradicts | Schulte (2008), which reported 6 explicit rejections among 2004-2007 abstracts — “Schulte also found a significantly greater number of rejection papers, including 6 explicit rejections compared to our 0 explicit rejections.” | claim_relations[7] |
| sense | consensus - as measured in the literature: the share of papers taking a position that endorse one explanation — “The peer-reviewed scientific literature provides a ground-level assessment of the degree of consensus among publishing scientists.” | senses[0] |
| sense | consensus - as measured by asking scientists directly, which earlier surveys put at 97–98% — “Surveys of climate scientists have found strong agreement (97–98%) regarding AGW amongst publishing climate experts (Doran and Zimmerman 2009 , Anderegg et al 2010 ).” | senses[1] |
| sense | consensus - as perceived by the public, which is that scientists disagree, and which the paper treats as the thing to be corrected — “Despite numerous indicators of a consensus, there is wide public perception that climate scientists disagree over the fundamental cause of global warming (GW; Leiserowitz et al 2012 , Pew 2012 ).” | senses[2] |
| sense | `the 97% consensus` - A NUMBER MADE BY A DENOMINATOR CHOICE, AND THE PAPER PRINTS BOTH: `66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW ... Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position.` The famous figure is computed on the third of the sample that took a position; two thirds are excluded BY DEFINITION, not by data. — “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming. Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming” | senses[3] |
| sense | Degree of endorsement - seven-level code ranging from explicit quantified endorsement through no position to explicit quantified rejection — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | senses[4] |
| sense | Explicit quantified endorsement - statement that humans are the primary cause of recent global warming — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | senses[5] |
| sense | Explicit unquantified endorsement - statement of human causation without a percentage contribution — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | senses[6] |
| sense | Implicit endorsement - assumed human causation even when the abstract does not state it explicitly — “Implicit endorsement Implies humans are causing global warming. E.g., research assumes greenhouse gas emissions cause warming without explicitly stating humans are the cause” | senses[7] |
| sense | Implicit rejection - natural-main-cause implication that minimizes human impact — “Implicit rejection Implies humans have had a minimal impact on global warming without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warming ‘…anywhere from a major portion to all of the warming of the 20th century could plausibly result from natural causes according to these results’ (6) Explicit rejection without quantification Explicitly minimizes or rejects that humans are causing global warming ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’ (7) Explicit rejection with quantification Explicitly states that humans are causing less than half of global warming” | senses[8] |
| sense | Explicit unquantified rejection - explicit minimization or rejection of human causation — “Implicit rejection Implies humans have had a minimal impact on global warming without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warming ‘…anywhere from a major portion to all of the warming of the 20th century could plausibly result from natural causes according to these results’ (6) Explicit rejection without quantification Explicitly minimizes or rejects that humans are causing global warming ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’ (7) Explicit rejection with quantification Explicitly states that humans are causing less than half of global warming” | senses[9] |
| sense | Explicit quantified rejection - claim that humans caused less than half of global warming — “Implicit rejection Implies humans have had a minimal impact on global warming without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warming ‘…anywhere from a major portion to all of the warming of the 20th century could plausibly result from natural causes according to these results’ (6) Explicit rejection without quantification Explicitly minimizes or rejects that humans are causing global warming ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’ (7) Explicit rejection with quantification Explicitly states that humans are causing less than half of global warming” | senses[10] |
| sense | Blind abstract rating - web-distributed title and abstract with author, affiliation, journal, and date hidden — “Abstracts were randomly distributed via a web-based system to raters with only the title and abstract visible. All other information such as author names and affiliations, journal and publishing date were hidden. Each abstract was categorized by two independent, anonymized raters” | senses[11] |
| sense | Rating reconciliation - anonymous comparison and revision followed by third-party adjudication — “Initially, 27% of category ratings and 33% of endorsement ratings disagreed. Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party” | senses[12] |
| sense | Four endorsement metrics - shares among all papers, positioned papers, all scientists, and positioned scientists — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | senses[13] |
| sense | Abstract-only categorization - method that underestimates how many full papers take an AGW position — “Another potential area of uncertainty involved the text of the abstracts themselves. In some cases, ambiguous language made it difficult to ascertain the intended meaning of the authors. Naturally, a short abstract could not be expected to communicate all the details of the full paper. The implementation of the author self-rating process allowed us to look beyond the abstract. A comparison between self-ratings and abstract ratings revealed that categorization based on the abstract alone underestimates the percentage of papers taking a position on AGW” | senses[14] |
| sense | Scientific reticence / least-drama bias - possible pressure toward a no-position abstract classification — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | senses[15] |
| sense | Consensus gap - public under-perception of overwhelming scientific agreement on human-caused warming — “Contributing to this ‘consensus gap’ are campaigns designed to confuse the public about the level of agreement among climate scientists. In 1991, Western Fuels Association conducted a $510 000 campaign whose primary goal was to ‘ reposition global warming as theory (not fact) ’. A key strategy involved constructing the impression of active scientific debate using dissenting scientists as spokesmen” | senses[16] |
| sense | Equal-attention media norm - opposing-side balance that amplifies a vocal minority — “The situation is exacerbated by media treatment of the climate issue, where the normative practice of providing opposing sides with equal attention has allowed a vocal minority to have their views amplified” | senses[17] |
| sense | Systematic literature review - quantitative counter-evidence to claims that scientific rejection is growing — “A systematic, comprehensive review of the literature provides quantitative evidence countering this assertion. The number of papers rejecting AGW is a miniscule proportion of the published research, with the percentage slightly decreasing over time” | senses[18] |
| control | a second phase in which the authors of the papers were invited to rate their own abstracts, alongside the volunteer raters — held — rules out the consensus figure being a property of how the volunteers read the abstracts - the authors know what their own papers claimed, and their ratings are an independent check no coding protocol can supply — “In a second phase of this study, we invited authors to rate their own papers.” | controls[0] |
| control | written criteria supplied to the raters for both the category and the level of endorsement of anthropogenic global warming — held — rules out each rater applying a private standard - the criteria are fixed and written before the rating rather than settled case by case — “Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ).” | controls[1] |
| control | the article type restricted to 'article', excluding books, discussions, proceedings papers and other document types — held — rules out a consensus measured over documents that are not research reports at all — “Article type was restricted to ‘article’, excluding books, discussions, proceedings papers and other document types.” | controls[2] |
| retracted | WITHDRAWN AND REPLACED (RETRACTED AND RE-ADDED WITH THE QUANTITY CORRECTED) · Schulte (2008), which reported 6% rejection among 2004-2007 abstracts — RETRACTED AND RE-ADDED WITH THE QUANTITY CORRECTED. The target said Schulte 'REPORTED 6% REJECTION'. The span says 'Schulte also found a significantly greater number of rejection papers, including 6 EXPLICIT REJECTIONS compared to our 0 explicit rejections.' A COUNT IS NOT A RATE. Recorded in the paper that is already this corpus's example of a denominator that does not travel - the neighbouring field in the same record carried the error. — “Schulte also found a significantly greater number of rejection papers, including 6 explicit rejections compared to our 0 explicit rejections.” — “6 explicit rejections compared to our 0 explicit rejections” | claim_relations[2] |
| differs in | Schulte (2008), which reported 6 explicit rejections among 2004-2007 abstracts — THE AXIS IS THE COUNT OF EXPLICIT REJECTIONS, AND BOTH COUNTS ARE GIVEN. 'Schulte also found a significantly greater number of rejection papers, including 6 explicit rejections compared to our 0 explicit rejections.' Six against zero, over overlapping literatures - so the disagreement is about what counts as an explicit rejection, not about how many papers exist. This row's target was repaired earlier today: it had said '6% rejection' where the paper says SIX PAPERS. — “Schulte also found a significantly greater number of rejection papers, including 6 explicit rejections compared to our 0 explicit rejections.” — “Schulte also found a significantly greater number of rejection papers, including 6 explicit rejections compared to our 0 explicit rejections” | claim_relations[7] |
| relation | No Position —distinct-from→ Uncertain — Uncertain states the cause is not yet determined; No Position states nothing at all — “An ‘Uncertain’ abstract explicitly states that the cause of global warming is not yet determined (e.g., ‘…the extent of human-induced global warming is inconclusive…’) while a ‘No Position’ abstract makes no statement on AGW.” — “while a ‘No Position’ abstract makes no statement on AGW” | concept_edges[0] |
| audit | EVIDENCE CORRECTED (RE-GROUNDED) · self-rating —contrasted-with→ raters — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but abstract ratings were compared with author self-ratings; contrasted-with is loose. — “These sources of bias were partially addressed by the use of multiple independent raters and by comparing abstract rating results to author self-ratings.” | concept_edges[2] |
| retracted | EVIDENCE CORRECTED (RE-GROUNDED) · self-rating —contrasted-with→ raters — RE-GROUNDED. The old span named neither 'self-rating' nor 'raters'. This sentence names both and states the comparison the edge records - and states it as a partial answer to bias, which is the paper's own framing. — “These sources of bias were partially addressed by the use of multiple independent raters and by comparing abstract rating results to author self-ratings.” — “Authors were emailed an invitation to participate in a” | concept_edges[2] |
| effect direction | Communicating the scientific consensus —causes→ people’s acceptance — increases — “An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al 2011 ). Communicating the scientific consensus also increases people’s acceptance that climate change (CC) is happening” — “Communicating the scientific consensus also increases people’s acceptance” | concept_edges[6] |
| note | Communicating the scientific consensus —causes→ people’s acceptance — The paper states that communicating consensus increases acceptance. — “An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al 2011 ). Communicating the scientific consensus also increases people’s acceptance that climate change (CC) is happening” | concept_edges[6] |
| audit | Implicit endorsement —part-of→ endorsements — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span writes 'endorsements (including implicit and explicit ...)', not 'implicit endorsement'. — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concept_edges[7] |
| note | Implicit endorsement —part-of→ endorsements — The consolidation explicitly includes implicit and explicit categories within endorsements. — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concept_edges[7] |
| audit | Implicit rejection —part-of→ rejections — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span writes 'rejections (including implicit and explicit ...)', not 'implicit rejection'. — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concept_edges[8] |
| note | Implicit rejection —part-of→ rejections — The consolidation explicitly includes implicit and explicit categories within rejections. — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concept_edges[8] |
| audit | opposing sides with equal attention —causes→ vocal minority — Audit 2026-09-29: DOUBTFUL - read against its full span and kept, but the span says equal attention allowed a vocal minority to have their VIEWS amplified; 'vocal minority' is the wrong object. — “The situation is exacerbated by media treatment of the climate issue, where the normative practice of providing opposing sides with equal attention has allowed a vocal minority to have their views amplified” | concept_edges[9] |
| effect direction | opposing sides with equal attention —causes→ vocal minority — increases — “The situation is exacerbated by media treatment of the climate issue, where the normative practice of providing opposing sides with equal attention has allowed a vocal minority to have their views amplified” — “has allowed a vocal minority to have their views amplified” | concept_edges[9] |
| note | opposing sides with equal attention —causes→ vocal minority — The media norm is said to have allowed amplification of the minority’s views. — “The situation is exacerbated by media treatment of the climate issue, where the normative practice of providing opposing sides with equal attention has allowed a vocal minority to have their views amplified” | concept_edges[9] |
| outcome (stated by the tagger, no quotation) | control — held | controls[0] |
| outcome (stated by the tagger, no quotation) | control — held | controls[1] |
| outcome (stated by the tagger, no quotation) | control — held | controls[2] |
| for which use | limitation — the 97% figure: it is computed among abstracts that took a position, and most ABSTRACTS EXPRESSED NO POSITION ON AGW — “of abstracts expressed no position on AGW” | limitations[0] |
| for which use | limitation — the consensus estimate: THE PROCESS CONTAINS SEVERAL SOURCES OF UNCERTAINTY, INCLUDING THE REPRESENTATIVENESS OF THE SAMPLE, LACK OF CLARITY IN THE ABSTRACTS AND SUBJECTIVITY — “The process of determining the level of consensus in the peer-reviewed literature contains several sources of uncertainty, including the representativeness of the sample, lack of clarity in the abstracts and subjectivity in rating the abstracts.” | limitations[1] |
| for which use | limitation — the coverage claim: 11 944 PAPERS IS ONLY A FRACTION OF THE CLIMATE LITERATURE — “Nevertheless, 11 944 papers is only a fraction of the climate literature.” | limitations[2] |
| for which use | limitation — every rating in the study: SOME SUBJECTIVITY IS INHERENT IN THE ABSTRACT RATING PROCESS — “Lastly, some subjectivity is inherent in the abstract rating process.” | limitations[3] |
| for which use | limitation — the abstract ratings: SCIENTIFIC RETICENCE AND ESLD REMAIN POSSIBLE BIASES IN THE ABSTRACT RATINGS PROCESS, in the direction of 'no position' — “This mitigated concerns about rater subjectivity, but suggests that scientific reticence and ESLD remain possible biases in the abstract ratings process.” | limitations[4] |
| for which use | limitation — any count of papers taking a position: CATEGORIZATION BASED ON THE ABSTRACT ALONE UNDERESTIMATES THE PERCENTAGE OF PAPERS TAKING A POSITION ON AGW — “A comparison between self-ratings and abstract ratings revealed that categorization based on the abstract alone underestimates the percentage of papers taking a position on AGW.” | limitations[5] |
| for which use | limitation — the reliability of the ratings the result rests on: INITIALLY, 27% OF CATEGORY RATINGS AND 33% OF ENDORSEMENT RATINGS DISAGREED — “Initially, 27% of category ratings and 33% of endorsement ratings disagreed.” | limitations[6] |
| for which use | limitation — the denominator of the headline percentage: 66.4% OF ABSTRACTS EXPRESSED NO POSITION ON AGW, 32.6% ENDORSED AGW, 0.7% REJECTED AGW AND 0.3% WERE UNCERTAIN - so the consensus figure is among the abstracts that took a position, as the paper states — “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming.” | limitations[7] |
| for which use | limitation — the same scope as limitations[5], with both numbers: A SMALLER PERCENTAGE OF SELF-RATED PAPERS EXPRESSED NO POSITION ON AGW (35.5%) than the 66.4% found by abstract rating — “Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%).” | limitations[8] |
| for which use | limitation — the self-rating arm specifically: EMAIL ADDRESSES FOR 8547 AUTHORS WERE COLLECTED, so the responding sample is partial and self-selected — “To complement the abstract analysis, email addresses for 8547 authors were collected, typically from the corresponding author and/or first author.” | limitations[9] |
| weight | raters themselves endorsed the scientific consensus on AGW → rating bias — 0.5 — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | unhoused_relations[3] |
| weight | scientific reticence → ‘no position’ classification — 0.5 — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | unhoused_relations[4] |
| replication | untested | |
| no type for this | accurate perception of the degree of scientific consensus → public support for climate policy — Accurate consensus perception is presented as prerequisite for public support; prerequisite-for is not in the eight-type registry. — “An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al 2011 ). Communicating the scientific consensus also increases people’s acceptance that climate change (CC) is happening” | unhoused_relations[0] |
| no type for this | four metrics → degree of endorsement — The four metrics quantify endorsement level; quantifies is not in the eight-type registry. — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | unhoused_relations[1] |
| no type for this | ESLD → erring on the side of least drama — ESLD expands to erring on the side of least drama; acronym-expansion is not in the eight-type registry. — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | unhoused_relations[2] |
| no type for this | raters themselves endorsed the scientific consensus on AGW → rating bias — Raters’ own endorsement may bias classification toward shared endorsement; biases-toward is not in the eight-type registry. — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | unhoused_relations[3] |
| no type for this | scientific reticence → ‘no position’ classification — Scientific reticence may bias ratings toward no position; biases-toward is not in the eight-type registry. — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | unhoused_relations[4] |
| no type for this | campaigns designed to confuse the public → consensus gap — Public-confusion campaigns contribute to the consensus gap; contributes-to is not in the eight-type registry. — “Contributing to this ‘consensus gap’ are campaigns designed to confuse the public about the level of agreement among climate scientists. In 1991, Western Fuels Association conducted a $510 000 campaign whose primary goal was to ‘ reposition global warming as theory (not fact) ’. A key strategy involved constructing the impression of active scientific debate using dissenting scientists as spokesmen” | unhoused_relations[5] |
| limitation | most abstracts expressed no position; the 97% figure is computed among those that did — “of abstracts expressed no position on AGW” | limitations[0] |
| limitation | Three named sources of uncertainty: sample representativeness, ambiguous abstract wording, and rater subjectivity. — “The process of determining the level of consensus in the peer-reviewed literature contains several sources of uncertainty, including the representativeness of the sample, lack of clarity in the abstracts and subjectivity in rating the abstracts.” | limitations[1] |
| limitation | The sample, though the largest of its kind, covers only part of the climate literature. — “Nevertheless, 11 944 papers is only a fraction of the climate literature.” | limitations[2] |
| limitation | Rating abstracts is inherently partly subjective. — “Lastly, some subjectivity is inherent in the abstract rating process.” | limitations[3] |
| limitation | Scientific reticence and 'erring on the side of least drama' may still bias abstract ratings towards 'no position'. — “This mitigated concerns about rater subjectivity, but suggests that scientific reticence and ESLD remain possible biases in the abstract ratings process.” | limitations[4] |
| limitation | Rating abstracts alone undercounts papers that actually take a position. — “A comparison between self-ratings and abstract ratings revealed that categorization based on the abstract alone underestimates the percentage of papers taking a position on AGW.” | limitations[5] |
| limitation | Initial inter-rater disagreement was substantial - 27% on category and 33% on endorsement - so the headline figure depends on the reconciliation procedure, not on raw agreement. — “Initially, 27% of category ratings and 33% of endorsement ratings disagreed.” | limitations[6] |
| limitation | Two thirds of abstracts take no position at all, so the 97.1% is a proportion of the third that do. — “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming.” | limitations[7] |
| limitation | The abstract is a proxy for the paper: the self-rating arm found fewer papers taking no position (35.5% against 66.4%), which is the size of what abstract-level coding misses. — “Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%).” | limitations[8] |
| limitation | Response is partial and self-selected: email addresses were obtainable for at least 60% of papers per year, and authors chose whether to reply. — “To complement the abstract analysis, email addresses for 8547 authors were collected, typically from the corresponding author and/or first author.” | limitations[9] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Among abstracts expressing a position on AGW, 97.1% endorsed the consensus that humans are causing global warming. — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.” | content_tags.findings[0] |
| finding | For both abstract ratings and author self-ratings, the share of endorsements among papers expressing a position marginally increased over time. — “For both abstract ratings and authors’ self-ratings, the percentage of endorsements among papers expressing a position on AGW marginally increased over time” | content_tags.findings[1] |
| evidence status note | finding — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.findings[1] |
| finding | Of abstracts taking any position, 97.1% endorse human-caused warming. — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.” | content_tags.findings[2] |
| finding | Authors rating their own full papers reached essentially the same figure, 97.2%, by a different route. — “Among self-rated papers expressing a position on AGW, 97.2% endorsed the consensus.” | content_tags.findings[3] |
| finding | The dominant category is silence: two thirds of abstracts state no position on the cause at all. — “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming.” | content_tags.findings[4] |
| finding | A smaller share of self-rated papers (35.5%) than of abstract ratings expressed no position on AGW. — “Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%).” | content_tags.findings[5] |
| finding | The finding agrees with the earlier Oreskes analysis, which found no disagreeing papers among 928. — “An analysis of abstracts published from 1993–2003 matching the search ‘global climate change’ found that none of 928 papers disagreed with the consensus position on AGW (Oreskes 2004 ).” [ATTRIBUTED TO THE CITED WORK, NAMED IN THE TRAILING PARENTHESIS] the paper’s own words: “8 papers disagreed with the consensus position on AGW (Oreskes 2004 ).” | content_tags.findings[6] |
| quantity | 97.1% — share of position-taking abstracts endorsing that humans cause global warming — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming” | content_tags.quantities[0] |
| quantity | 11,944 abstracts — peer-reviewed climate abstracts examined (1991-2011) — “examining 11 944 climate abstracts from 1991–2011” | content_tags.quantities[1] |
| evidence status note | quantity — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.quantities[1] |
| quantity | 66.4% — share of abstracts expressing NO position on AGW — “We find that 66.4% of abstracts expressed no position on AGW” | content_tags.quantities[2] |
| quantity | 32.6% / 0.7% / 0.3% — abstracts endorsing / rejecting / uncertain on AGW — “32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming” | content_tags.quantities[3] |
| quantity | 11 944 abstracts, 1991–2011 — size and span of the rated corpus — “We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics ‘global climate change’ or ‘global warming’.” | content_tags.quantities[4] |
| quantity | 97.1% / 97.2% — endorsement among abstracts, and among self-rated papers, that take a position — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming. In a second phase of this study, we invited authors to rate their own papers. Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%). Among self-rated papers expressing a position on AGW, 97.2% endorsed the consensus.” | content_tags.quantities[5] |
| quantity | 66.4% / 32.6% / 0.7% / 0.3% — no position / endorse / reject / uncertain, across all abstracts — “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming.” | content_tags.quantities[6] |
| quantity | 27% and 33% initial disagreement; 11% and 16% after reconciliation — inter-rater disagreement on category and on endorsement, before and after raters compared ratings — “Initially, 27% of category ratings and 33% of endorsement ratings disagreed. Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party.” | content_tags.quantities[7] |
| quantity | 23 061 ratings by 12 people — 97.4% of all ratings came from a team of twelve — “A team of 12 individuals completed 97.4% (23 061) of the ratings; an additional 12 contributed the remaining 2.6% (607).” | content_tags.quantities[8] |
| quantity | 97–98% — agreement on AGW found by earlier direct surveys of publishing climate experts — “Surveys of climate scientists have found strong agreement (97–98%) regarding AGW amongst publishing climate experts (Doran and Zimmerman 2009 , Anderegg et al 2010 ).” [ATTRIBUTED TO THE CITED WORK, NAMED IN THE TRAILING PARENTHESIS] the paper’s own words: “ing climate experts (Doran and Zimmerman 2009 , Anderegg et al 2010 ).” | content_tags.quantities[9] |
| variable | climate abstracts (independent) — “examining 11 944 climate abstracts from 1991–2011” | content_tags.variables[0] |
| evidence status note | variable — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.variables[0] |
| definition | AGW — “consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature” | content_tags.definitions[0] |
| definition | consensus position — “the consensus position that humans are causing global warming” | content_tags.definitions[1] |
| definition | explicit endorsement — “Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations).” | content_tags.definitions[3] |
| key-terms | consensus, abstracts, AGW, warming, climate, endorsed, papers, position, anthropogenic, scientific consensus, anthropogenic global warming, endorsement, No Position, Uncertain, abstract, self-rating, citizen science, raters, peer-reviewed scientific literature |
| concept | scientific consensus — “We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics ‘global climate change’ or ‘global warming’.” | concepts[0] |
| concept | anthropogenic global warming — “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.” | concepts[1] |
| concept | endorsement — not defined in this paper's own words; the seven-point coded position of an abstract, from explicit quantified endorsement to explicit quantified rejection — “Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations).” | concepts[4] |
| concept | No Position — not defined in this paper's own words; an abstract that makes no statement on the cause at all - silence, not doubt and not denial — “An ‘Uncertain’ abstract explicitly states that the cause of global warming is not yet determined (e.g., ‘…the extent of human-induced global warming is inconclusive…’) while a ‘No Position’ abstract makes no statement on AGW.” | concepts[5] |
| concept | Uncertain — not defined in this paper's own words; an abstract that explicitly states the cause is not yet determined - a stated position of doubt, distinct from silence — “An ‘Uncertain’ abstract explicitly states that the cause of global warming is not yet determined (e.g., ‘…the extent of human-induced global warming is inconclusive…’) while a ‘No Position’ abstract makes no statement on AGW.” | concepts[6] |
| concept | self-rating — not defined in this paper's own words; authors coding their own full papers against the same rubric - the second, independent measurement of the same quantity — “Authors were emailed an invitation to participate in a survey in which they rated their own published papers (the entire content of the article, not just the abstract) with the same criteria as used by the independent rating team.” | concepts[7] |
| concept | raters — not defined in this paper's own words; the coders, working blind to author, journal and date, two per abstract — “Each abstract was categorized by two independent, anonymized raters.” | concepts[8] |
| concept | citizen science — not defined in this paper's own words; the organisational form of the study: volunteers from a public website doing the coding — “This letter was conceived as a ‘citizen science’ project by volunteers contributing to the Skeptical Science website” | concepts[9] |
| concept | Communicating the scientific consensus — not defined in this paper's own words; communication intervention reported to increase acceptance that climate change is happening — “An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al 2011 ). Communicating the scientific consensus also increases people’s acceptance that climate change (CC) is happening” | concepts[10] |
| concept | people’s acceptance — not defined in this paper's own words; public acceptance that climate change is happening — “An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al 2011 ). Communicating the scientific consensus also increases people’s acceptance that climate change (CC) is happening” | concepts[11] |
| concept | public support for climate policy — not defined in this paper's own words; policy-support outcome for which accurate consensus perception is described as essential — “An accurate perception of the degree of scientific consensus is an essential element to public support for climate policy (Ding et al 2011 ). Communicating the scientific consensus also increases people’s acceptance that climate change (CC) is happening” | concepts[12] |
| concept | ISI Web of Science — not defined in this paper's own words; bibliographic database searched for the 1991–2011 literature sample — “In March 2012, we searched the ISI Web of Science for papers published from 1991–2011 using topic searches for ‘global warming’ or ‘global climate change’. Article type was restricted to ‘article’, excluding books, discussions, proceedings papers and other document types” | concepts[13] |
| concept | topic searches — not defined in this paper's own words; global-warming and global-climate-change queries defining the candidate paper set — “In March 2012, we searched the ISI Web of Science for papers published from 1991–2011 using topic searches for ‘global warming’ or ‘global climate change’. Article type was restricted to ‘article’, excluding books, discussions, proceedings papers and other document types” | concepts[14] |
| concept | research category — not defined in this paper's own words; one axis on which each abstract was classified — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | concepts[15] |
| concept | degree of endorsement — not defined in this paper's own words; second classification axis coding the strength and direction of an abstract’s AGW position — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | concepts[16] |
| concept | Explicit endorsement with quantification — not defined in this paper's own words; level stating that humans are the primary cause of recent warming — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | concepts[17] |
| concept | Explicit endorsement without quantification — not defined in this paper's own words; level stating human causation without quantifying the contribution — “We classified each abstract according to the type of research (category) and degree of endorsement. Written criteria were provided to raters for category (table 1 ) and level of endorsement of AGW (table 2 ). Explicit endorsements were divided into non-quantified (e.g., humans are contributing to global warming without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warming, consistent with the 2007 IPCC statement that most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations)” | concepts[18] |
| concept | Implicit endorsement — not defined in this paper's own words; level inferring endorsement when research assumes human-caused warming without explicitly stating it — “Implicit endorsement Implies humans are causing global warming. E.g., research assumes greenhouse gas emissions cause warming without explicitly stating humans are the cause” | concepts[19] |
| concept | Implicit rejection — not defined in this paper's own words; level implying minimal human impact by proposing a natural main cause — “Implicit rejection Implies humans have had a minimal impact on global warming without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warming ‘…anywhere from a major portion to all of the warming of the 20th century could plausibly result from natural causes according to these results’ (6) Explicit rejection without quantification Explicitly minimizes or rejects that humans are causing global warming ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’ (7) Explicit rejection with quantification Explicitly states that humans are causing less than half of global warming” | concepts[20] |
| concept | Explicit rejection without quantification — not defined in this paper's own words; level explicitly minimizing or rejecting human causation without a numeric share — “Implicit rejection Implies humans have had a minimal impact on global warming without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warming ‘…anywhere from a major portion to all of the warming of the 20th century could plausibly result from natural causes according to these results’ (6) Explicit rejection without quantification Explicitly minimizes or rejects that humans are causing global warming ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’ (7) Explicit rejection with quantification Explicitly states that humans are causing less than half of global warming” | concepts[21] |
| concept | Explicit rejection with quantification — not defined in this paper's own words; level stating that humans caused less than half of warming — “Implicit rejection Implies humans have had a minimal impact on global warming without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warming ‘…anywhere from a major portion to all of the warming of the 20th century could plausibly result from natural causes according to these results’ (6) Explicit rejection without quantification Explicitly minimizes or rejects that humans are causing global warming ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’ (7) Explicit rejection with quantification Explicitly states that humans are causing less than half of global warming” | concepts[22] |
| concept | web-based system — not defined in this paper's own words; blind-distribution interface showing raters only titles and abstracts — “Abstracts were randomly distributed via a web-based system to raters with only the title and abstract visible. All other information such as author names and affiliations, journal and publishing date were hidden. Each abstract was categorized by two independent, anonymized raters” | concepts[23] |
| concept | rating disagreements — not defined in this paper's own words; initial and post-discussion differences between independent category and endorsement ratings — “Initially, 27% of category ratings and 33% of endorsement ratings disagreed. Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party” | concepts[24] |
| concept | third party — not defined in this paper's own words; adjudicator resolving disagreements left after anonymous comparison and revision — “Initially, 27% of category ratings and 33% of endorsement ratings disagreed. Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party” | concepts[25] |
| concept | four metrics — not defined in this paper's own words; paper-, positioned-paper-, scientist-, and positioned-scientist denominators used to quantify endorsement — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concepts[26] |
| concept | endorsements — not defined in this paper's own words; consolidated group containing implicit and explicit endorsement categories — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concepts[27] |
| concept | rejections — not defined in this paper's own words; consolidated group containing implicit and explicit rejection categories — “To simplify the analysis, ratings were consolidated into three groups: endorsements (including implicit and explicit; categories 1–3 in table 2 ), no position (category 4) and rejections (including implicit and explicit; categories 5–7). We examined four metrics to quantify the level of endorsement: (1) The percentage of endorsements/rejections/undecideds among all abstracts. (2) The percentage of endorsements/rejections/undecideds among only those abstracts expressing a position on AGW. (3) The percentage of scientists authoring endorsement/ rejection abstracts among all scientists. (4) The same percentage among only those scientists who expressed a position on AGW” | concepts[28] |
| concept | abstract rating — not defined in this paper's own words; independent team’s classification based on title and abstract rather than the full paper — “A direct comparison of abstract rating versus self-rating endorsement levels for the 2142 papers that received a self-rating is shown in table 5 . More than half of the abstracts that we rated as ‘No Position’ or ‘Undecided’ were rated ‘Endorse AGW’ by the paper’s authors” | concepts[29] |
| concept | sources of uncertainty — not defined in this paper's own words; representativeness, abstract clarity, and rating subjectivity in literature-consensus measurement — “The process of determining the level of consensus in the peer-reviewed literature contains several sources of uncertainty, including the representativeness of the sample, lack of clarity in the abstracts and subjectivity in rating the abstracts” | concepts[30] |
| concept | representativeness of the sample — not defined in this paper's own words; whether the selected literature sample represents the wider climate literature — “The process of determining the level of consensus in the peer-reviewed literature contains several sources of uncertainty, including the representativeness of the sample, lack of clarity in the abstracts and subjectivity in rating the abstracts” | concepts[31] |
| concept | lack of clarity in the abstracts — not defined in this paper's own words; ambiguity limitation of abstract-level evidence — “The process of determining the level of consensus in the peer-reviewed literature contains several sources of uncertainty, including the representativeness of the sample, lack of clarity in the abstracts and subjectivity in rating the abstracts” | concepts[32] |
| concept | subjectivity in rating the abstracts — not defined in this paper's own words; judgment-dependent uncertainty in classifying abstracts — “The process of determining the level of consensus in the peer-reviewed literature contains several sources of uncertainty, including the representativeness of the sample, lack of clarity in the abstracts and subjectivity in rating the abstracts” | concepts[33] |
| concept | categorization based on the abstract alone — not defined in this paper's own words; classification restricted to an abstract rather than the full paper — “Another potential area of uncertainty involved the text of the abstracts themselves. In some cases, ambiguous language made it difficult to ascertain the intended meaning of the authors. Naturally, a short abstract could not be expected to communicate all the details of the full paper. The implementation of the author self-rating process allowed us to look beyond the abstract. A comparison between self-ratings and abstract ratings revealed that categorization based on the abstract alone underestimates the percentage of papers taking a position on AGW” | concepts[34] |
| concept | percentage of papers taking a position on AGW — not defined in this paper's own words; share of papers judged to state a position on AGW — “Another potential area of uncertainty involved the text of the abstracts themselves. In some cases, ambiguous language made it difficult to ascertain the intended meaning of the authors. Naturally, a short abstract could not be expected to communicate all the details of the full paper. The implementation of the author self-rating process allowed us to look beyond the abstract. A comparison between self-ratings and abstract ratings revealed that categorization based on the abstract alone underestimates the percentage of papers taking a position on AGW” | concepts[35] |
| concept | rating bias — not defined in this paper's own words; possible directional classification effects from rater commitments or scientific reticence — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | concepts[36] |
| concept | raters themselves endorsed the scientific consensus on AGW — not defined in this paper's own words; possible pro-endorsement influence on classification acknowledged by the paper — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | concepts[37] |
| concept | scientific reticence — not defined in this paper's own words; possible tendency biasing raters toward a no-position classification — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | concepts[38] |
| concept | erring on the side of least drama — not defined in this paper's own words; least-drama reporting tendency discussed alongside scientific reticence — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | concepts[39] |
| concept | ‘no position’ classification — not defined in this paper's own words; rating outcome potentially favored by reticence and least-drama language — “Two sources of rating bias can be cited: first, given that the raters themselves endorsed the scientific consensus on AGW, they may have been more likely to classify papers as sharing that endorsement. Second, scientific reticence (Hansen 2007 ) or ‘erring on the side of least drama’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing raters towards a ‘no position’ classification” | concepts[40] |
| concept | consensus gap — not defined in this paper's own words; gap between public perception and scientific agreement on anthropogenic warming — “Contributing to this ‘consensus gap’ are campaigns designed to confuse the public about the level of agreement among climate scientists. In 1991, Western Fuels Association conducted a $510 000 campaign whose primary goal was to ‘ reposition global warming as theory (not fact) ’. A key strategy involved constructing the impression of active scientific debate using dissenting scientists as spokesmen” | concepts[41] |
| concept | campaigns designed to confuse the public — not defined in this paper's own words; organized efforts presented as contributing to public misperception of scientific agreement — “Contributing to this ‘consensus gap’ are campaigns designed to confuse the public about the level of agreement among climate scientists. In 1991, Western Fuels Association conducted a $510 000 campaign whose primary goal was to ‘ reposition global warming as theory (not fact) ’. A key strategy involved constructing the impression of active scientific debate using dissenting scientists as spokesmen” | concepts[42] |
| concept | opposing sides with equal attention — not defined in this paper's own words; media norm amplifying a dissenting minority’s views — “The situation is exacerbated by media treatment of the climate issue, where the normative practice of providing opposing sides with equal attention has allowed a vocal minority to have their views amplified” | concepts[43] |
| concept | vocal minority — not defined in this paper's own words; dissenting group whose views receive amplification under equal-attention coverage — “The situation is exacerbated by media treatment of the climate issue, where the normative practice of providing opposing sides with equal attention has allowed a vocal minority to have their views amplified” | concepts[44] |
| concept | systematic, comprehensive review of the literature — not defined in this paper's own words; literature-wide review used to counter claims of a collapsing consensus — “A systematic, comprehensive review of the literature provides quantitative evidence countering this assertion. The number of papers rejecting AGW is a miniscule proportion of the published research, with the percentage slightly decreasing over time” | concepts[45] |
| concept | quantitative evidence — not defined in this paper's own words; numeric literature evidence countering the claim that dissent is growing — “A systematic, comprehensive review of the literature provides quantitative evidence countering this assertion. The number of papers rejecting AGW is a miniscule proportion of the published research, with the percentage slightly decreasing over time” | concepts[46] |
| confidence-interval CI | as written: 95% CI — “Over time, the no position percentage has increased (simple linear regression trend 0.87% ± 0.28% yr −1 , 95% CI, R 2 = 0.66, p < 0.001) and the percentage of papers taking a position on AGW has equally decreased” | content_tags.statistics[0] |
| evidence status note | confidence-interval — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.statistics[0] |
| p-value p | as written: p < 0.001 — “Over time, the no position percentage has increased (simple linear regression trend 0.87% ± 0.28% yr −1 , 95% CI, R 2 = 0.66, p < 0.001) and the percentage of papers taking a position on AGW has equally decreased” | content_tags.statistics[1] |
| evidence status note | p-value — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.statistics[1] |
| regression β | as written: linear regression — “Over time, the no position percentage has increased (simple linear regression trend 0.87% ± 0.28% yr −1 , 95% CI, R 2 = 0.66, p < 0.001) and the percentage of papers taking a position on AGW has equally decreased” | content_tags.statistics[2] |
| evidence status note | regression — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.statistics[2] |
| sample-size n/N | as written: sample sizes — “The year-to-year variability is larger in the self-ratings than in the abstract ratings due to the smaller sample sizes in the early 1990s.” | content_tags.statistics[3] |
| inter-rater-reliability disagreement rate | — “Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party.” | content_tags.statistics[4] |
| proportion % | — “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming.” | content_tags.statistics[5] |
| random-sampling random sample | — “Upon completion of the final ratings, a random sample of 1000 ‘No Position’ category abstracts were re-examined to differentiate those that did not express an opinion from those that take the position that the cause of GW is uncertain.” | content_tags.statistics[6] |
| claim type | existence — a second reading reaching the same value; recorded: existence — “Our analysis indicates that the number of papers rejecting the consensus on AGW is a vanishingly small proportion of the published research.” | alternatives.claim_type |
| method | corpus-analysis — a second reading DIFFERING from the recorded value; recorded: corpus-analysis — “We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics ‘global climate change’ or ‘global warming’.” | alternatives.method |
| phenomenon | the gap between the level of scientific consensus on anthropogenic global warming and the public perception of it — a second reading DIFFERING from the recorded value; recorded: scientific consensus on anthropogenic global warming — “From 1997 to 2007, public opinion polls have indicated around 60% of the US public believes there is significant disagreement among scientists about whether GW was happening (Nisbet and Myers 2007 ).” | alternatives.phenomenon |
Full text — tagged
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
Quantifying the consensusterm·key on anthropogenic global warmingterm·dom in the scientific literature John CookpersName , Dana NuccitellipersName , Sarah A GreenpersName , Mark Richardson , Bärbel Winkler , Rob Painting , Robert Way , Peter Jacobs and Andrew Skuce Published 15 May 2013 • © 2013 IOP Publishing Ltd Environmental Research Letters , Volume 8 , Number 2 Citation John CookpersName et al 2013 Environ. Res. Lett. 8 024024 DOI 10.1088/1748-9326/8/2/024024 PDF Opens in a new tab.
Authors John CookpersName AFFILIATIONS Global Change Institute, University of Queensland, Australia Skeptical Science, Brisbane, Queensland, Australia School of Psychology, University of Western Australia, Australia EMAIL j.cook3@uq.edu.au Dana NuccitellipersName AFFILIATIONS Skeptical Science, Brisbane, Queensland, Australia Tetra Tech, Incorporated, McClellan, CA, USA Sarah A GreenpersName AFFILIATIONS Department of Chemistry, Michigan Technological University, USA Mark Richardson AFFILIATIONS Department of Meteorology, University of Reading, UK Bärbel Winkler AFFILIATIONS Skeptical Science, Brisbane, Queensland, Australia Rob Painting AFFILIATIONS Skeptical Science, Brisbane, Queensland, Australia Robert Way AFFILIATIONS Department of Geography, Memorial University of Newfoundland, Canada Peter Jacobs AFFILIATIONS Department of Environmental Science and Policy, George Mason University, USA Andrew Skuce AFFILIATIONS Skeptical Science, Brisbane, Queensland, Australia Salt Spring Consulting Ltd, Salt Spring Island, BC, Canada Article metrics 1519786 Total downloads 0 Video abstractterm·key views Submit Submit to this Journal Share this article Article information Dates Received 18 January 2013 Accepted 22 April 2013 Published 15 May 2013 Buy this article in print Journal RSS Sign up for new issue notifications 1748-9326/8/2/024024 Abstractterm·key We analyze the evolution of the scientific consensusterm·dom on anthropogenic global warmingterm·dom (AGWterm·key) in the peer-reviewed scientific literatureterm·key, examining 11 944 climate abstractsterm·var from 1991date–2011date matching the topics ‘global climateterm·key change’ or ‘global warmingterm·key’. We find that 66.4% of abstractsterm·key expressed no positionterm·mth on AGWterm·key, 32.6% endorsedterm·key AGWterm·key, 0.7% rejected AGWterm·key and 0.3% were uncertainterm·mth about the cause of global warmingterm·key. Among abstractsterm·key expressing a positionterm·key on AGWterm·key, 97.1% endorsedterm·key the consensus positionterm·def that humans are causing global warmingterm·key. In a second phase of this study, we invited authors to rate their own papersterm·key. Compared to abstract ratingsterm, a smaller percentage of self-rated papersterm·key expressed no positionterm·mth on AGWterm·key (35.5%). Among self-rated papersterm·key expressing a positionterm·key on AGWterm·key, 97.2% endorsedterm·key the consensusterm·key. For both abstract ratingsterm and authors’ self-ratingsterm·mth, the percentage of endorsementsterm among papersterm·key expressing a positionterm·key on AGWterm·key marginally increased over time. Our analysis indicates that the number of papersterm·key rejecting the consensusterm·key on AGWterm·key is a vanishingly small proportion of the published research.
Export citation and abstractterm·key BibTeX RIS Previous article in issue Next article in issue Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence . Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Supplementary data 1. Introduction An accurate perception of the degree of scientific consensusterm·dom is an essential element to public support for climate policyterm (Ding et al 2011date ). Communicating the scientific consensusterm also increases people’s acceptanceterm that climateterm·key change (CC) is happening (Lewandowsky et al 2012 ). Despite numerous indicators of a consensusterm·key, there is wide public perception that climateterm·key scientists disagree over the fundamental cause of global warmingterm·key (GW; Leiserowitz et al 2012 , Pew 2012 ). In the most comprehensive analysis performed to date, we have extended the analysis of peer-reviewed climateterm·key papersterm·key in Oreskes ( 2004 ). We examined a large sample of the scientific literature on global CC, published over a 21 year period, in order to determine the level of scientific consensusterm·dom that human activity is very likely causing most of the current GW (anthropogenic global warmingterm·dom, or AGWterm·key).
Surveys of climateterm·key scientists have found strong agreement (97–98%) regarding AGWterm·key amongst publishing climateterm·key experts (Doran and Zimmerman 2009 , Anderegg et al 2010 ). Repeated surveys of scientists found that scientific agreement about AGWterm·key steadily increased from 1996 to 2009 (Bray 2010 ). This is reflected in the increasingly definitive statements issued by the Intergovernmental Panel on Climateterm·key Change on the attribution of recent GW (Houghton et al 1996 , 2001 , Solomon et al 2007 ).
The peer-reviewed scientific literatureterm·key provides a ground-level assessment of the degree of consensusterm·key among publishing scientists. An analysis of abstractsterm·key published from 1993–2003 matching the search ‘global climateterm·key change’ found that none of 928 papersterm·key disagreed with the consensus positionterm·def on AGWterm·key (Oreskes 2004 ). This is consistent with an analysis of citation networks that found a consensusterm·key on AGWterm·key forming in the early 1990s (Shwed and Bearman 2010 ).
Despite these independent indicators of a scientific consensusterm·dom, the perception of the US public is that the scientific community still disagrees over the fundamental cause of GW. From 1997 to 2007, public opinion polls have indicated around 60% of the US public believes there is significant disagreement among scientists about whether GW was happening (Nisbet and Myers 2007 ). Similarly, 57% of the US public either disagreed or were unaware that scientists agree that the earth is very likely warmingterm·key due to human activity (Pew 2012 ).
Through analysis of climate-related papersterm·key published from 1991date to 2011date, this study provides the most comprehensive analysis of its kind to date in order to quantify and evaluate the level and evolution of consensusterm·key over the last two decades.
2. Methodology This letter was conceived as a ‘citizen scienceterm·mth’ project by volunteers contributing to the Skeptical Science website ( www.skepticalscience.com ). In March 2012, we searched the ISI Web of Scienceterm for papersterm·key published from 1991date–2011date using topic searchesterm for ‘global warmingterm·key’ or ‘global climateterm·key change’. Article type was restricted to ‘article’, excluding books, discussions, proceedings papersterm·key and other document types. The search was updated in May 2012 with papersterm·key added to the Web of Science up to that date.
We classified each abstractterm·key according to the type of research (category) and degree of endorsementterm. Written criteria were provided to ratersterm·mth for category (table 1 ) and level of endorsementterm·mth of AGWterm·key (table 2 ). Explicit endorsementsterm·def were divided into non-quantified (e.g., humans are contributing to global warmingterm·key without quantifying the contribution) and quantified (e.g., humans are contributing more than 50% of global warmingterm·key, consistent with the 2007 IPCC statement that most of the global warmingterm·key since the mid-20th century is very likely due to the observed increase in anthropogenicterm·key greenhouse gas concentrations).
Table 1. Definitions of each type of research categoryterm.
Category Description Example (1) Impacts Effects and impacts of climateterm·key change on the environment, ecosystems or humanity ‘…global climateterm·key change together with increasing direct impacts of human activities, such as fisheries, are affecting the population dynamics of marine top predators’
(2) Methods Focus on measurements and modeling methods, or basic climateterm·key science not included in the other categories ‘This paperterm·key focuses on automating the task of estimating Polar ice thickness from airborne radar data…’
(3) Mitigation Research into lowering CO 2 emissions or atmospheric CO 2 levels ‘This paperterm·key presents a new approach for a nationally appropriate mitigation actions framework that can unlock the huge potential for greenhouse gas mitigation in dispersed energy end-use sectors in developing countries’
(4) Not climate-related Social science, education, research about people’s views on climateterm·key ‘This paperterm·key discusses the use of multimedia techniques and augmented reality tools to bring across the risks of global climateterm·key change’
(5) Opinion Not peer-reviewed articles ‘While the world argues about reducing global warmingterm·key, chemical engineers are getting on with the technology. Charles Butcher has been finding out how to remove carbon dioxide from flue gas’
(6) Paleoclimate Examining climateterm·key during pre-industrial times ‘Here, we present a pollen-based quantitative temperature reconstruction from the midlatitudes of Australia that spans the last 135 000 years…’
Table 2. Definitions of each level of endorsementterm·mth of AGWterm·key.
Level of endorsementterm·mth Description Example (1) Explicit endorsement with quantificationterm Explicitly states that humans are the primary cause of recent global warmingterm·key ‘The global warmingterm·key during the 20th century is caused mainly by increasing greenhouse gas concentration especially since the late 1980s’
(2) Explicit endorsement without quantificationterm Explicitly states humans are causing global warmingterm·key or refers to anthropogenic global warmingterm·dom/climateterm·key change as a known fact ‘Emissions of a broad range of greenhouse gases of varying lifetimes contribute to global climateterm·key change’
(3) Implicit endorsementterm Implies humans are causing global warmingterm·key. E.g., research assumes greenhouse gas emissions cause warmingterm·key without explicitly stating humans are the cause ‘…carbon sequestration in soil is important for mitigating global climateterm·key change’
(4a) No positionterm·mth Does not address or mention the cause of global warmingterm·key (4b) Uncertainterm·mth Expresses positionterm·key that human’s role on recent global warmingterm·key is uncertainterm·mth/undefined ‘While the extent of human-induced global warmingterm·key is inconclusive…’
(5) Implicit rejectionterm Implies humans have had a minimal impact on global warmingterm·key without saying so explicitly E.g., proposing a natural mechanism is the main cause of global warmingterm·key ‘…anywhere from a major portion to all of the warmingterm·key of the 20th century could plausibly result from natural causes according to these results’
(6) Explicit rejection without quantificationterm Explicitly minimizes or rejects that humans are causing global warmingterm·key ‘…the global temperature record provides little support for the catastrophic view of the greenhouse effect’
(7) Explicit rejection with quantificationterm Explicitly states that humans are causing less than half of global warmingterm·key ‘The human contribution to the CO 2 content in the atmosphere and the increase in temperature is negligible in comparison with other sources of carbon dioxide emission’
Abstractsterm·key were randomly distributed via a web-based systemterm to ratersterm·mth with only the title and abstractterm·key visible. All other information such as author names and affiliations, journal and publishing date were hidden. Each abstractterm·key was categorized by two independent, anonymized ratersterm·mth. A team of 12 individuals completed 97.4% (23 061) of the ratings; an additional 12 contributed the remaining 2.6% (607). Initially, 27% of category ratings and 33% of endorsementterm·mth ratings disagreed. Ratersterm·mth were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsementterm·mth ratings disagreed; these were then resolved by a third partyterm.
Upon completion of the final ratings, a random sample of 1000 ‘No Positionterm·mth’ category abstractsterm·key were re-examined to differentiate those that did not express an opinion from those that take the positionterm·key that the cause of GW is uncertainterm·mth. An ‘Uncertainterm·mth’ abstractterm·key explicitly states that the cause of global warmingterm·key is not yet determined (e.g., ‘…the extent of human-induced global warmingterm·key is inconclusive…’) while a ‘No Positionterm·mth’ abstractterm·key makes no statement on AGWterm·key.
To complement the abstractterm·key analysis, email addresses for 8547 authors were collected, typically from the corresponding author and/or first author. For each year, email addresses were obtained for at least 60% of papersterm·key. Authors were emailed an invitation to participate in a survey in which they rated their own published papersterm·key (the entire content of the article, not just the abstractterm·key) with the same criteria as used by the independent rating team. Details of the survey text are provided in the supplementary information (available at stacks.iop.org/ERL/8/024024/mmedia ).
3. Results The ISI search generated 12 465 papersterm·key. Eliminating papersterm·key that were not peer-reviewed (186), not climate-related (288) or without an abstractterm·key (47) reduced the analysis to 11 944 papersterm·key written by 29 083 authors and published in 1980 journals. To simplify the analysis, ratings were consolidated into three groups: endorsementsterm (including implicit and explicit; categories 1–3 in table 2 ), no positionterm·mth (category 4) and rejectionsterm (including implicit and explicit; categories 5–7).
We examined four metricsterm to quantify the level of endorsementterm·mth: (1) The percentage of endorsementsterm/rejectionsterm/undecideds among all abstractsterm·key.
(2) The percentage of endorsementsterm/rejectionsterm/undecideds among only those abstractsterm·key expressing a positionterm·key on AGWterm·key.
(3) The percentage of scientists authoring endorsementterm·mth/ rejectionterm abstractsterm·key among all scientists.
(4) The same percentage among only those scientists who expressed a positionterm·key on AGWterm·key (table 3 ).
Table 3. Abstract ratingsterm for each level of endorsementterm·mth, shown as percentage and total number of papersterm·key.
Positionterm·key % of all abstractsterm·key % among abstractsterm·key with AGWterm·key positionterm·key (%) % of all authors % among authors with AGWterm·key positionterm·key (%) Endorse AGWterm·key 32.6% (3896) 97.1 34.8% (10 188) 98.4 No AGWterm·key positionterm·key 66.4% (7930) — 64.6% (18 930) — Reject AGWterm·key 0.7% (78) 1.9 0.4% (124) 1.2 Uncertainterm·mth on AGWterm·key 0.3% (40) 1.0 0.2% (44) 0.4 3.1. Endorsementterm·mth percentages from abstract ratingsterm Among abstractsterm·key that expressed a positionterm·key on AGWterm·key, 97.1% endorsedterm·key the scientific consensusterm·dom. Among scientists who expressed a positionterm·key on AGWterm·key in their abstractterm·key, 98.4% endorsedterm·key the consensusterm·key.
The time series of each level of endorsementterm·mth of the consensusterm·key on AGWterm·key was analyzed in terms of the number of abstractsterm·key (figure 1 (a)) and the percentage of abstractsterm·key (figure 1 (b)). Over time, the no positionterm·mth percentage has increased (simple linear regressionterm·stat trend 0.87% ± 0.28% yr −1 , 95% CI, R 2 = 0.66, p < 0.001) and the percentage of papers taking a position on AGWterm has equally decreased.
Zoom In Zoom Out Reset image size Figure 1. (a) Total number of abstractsterm·key categorized into endorsementterm·mth, rejectionterm and no positionterm·mth. (b) Percentage of endorsementterm·mth, rejectionterm and no positionterm·mth/undecided abstractsterm·key. Uncertainterm·mth comprise 0.5% of no positionterm·mth abstractsterm·key.
Download figure: Standard image High-resolution image The average numbers of authors per endorsementterm·mth abstractterm·key (3.4) and per no positionterm·mth abstractterm·key (3.6) are both significantly larger than the average number of authors per rejectionterm abstractterm·key (2.0). The scientists originated from 91 countries (identified by email address) with the highest representation from the USA ( N = 2548) followed by the United Kingdom ( N = 546), Germany ( N = 404) and Japan ( N = 379) (see supplementary table S1 for full list, available at stacks.iop.org/ERL/8/024024/mmedia ).
3.2. Endorsementterm·mth percentages from self-ratingsterm·mth We emailed 8547 authors an invitation to rate their own papersterm·key and received 1200 responses (a 14% response rate). After excluding papersterm·key that were not peer-reviewed, not climate-related or had no abstractterm·key, 2142 papersterm·key received self-ratingsterm·mth from 1189 authors. The self-rated levels of endorsementterm·mth are shown in table 4 . Among self-rated papersterm·key that stated a positionterm·key on AGWterm·key, 97.2% endorsedterm·key the consensusterm·key. Among self-rated papersterm·key not expressing a positionterm·key on AGWterm·key in the abstractterm·key, 53.8% were self-rated as endorsing the consensusterm·key. Among respondents who authored a paperterm·key expressing a view on AGWterm·key, 96.4% endorsedterm·key the consensusterm·key.
Table 4. Self-ratingsterm·mth for each level of endorsementterm·mth, shown as percentage and total number of papersterm·key.
Positionterm·key % of all papersterm·key % among papersterm·key with AGWterm·key positionterm·key (%) % of respondents % among respondents with AGWterm·key positionterm·key (%) Endorse AGWterm·key a 62.7% (1342) 97.2 62.7% (746) 96.4 No AGWterm·key positionterm·key b 35.5% (761) — 34.9% (415) — Reject AGWterm·key c 1.8% (39) 2.8 2.4% (28) 3.6 a Self-rated papersterm·key that endorse AGWterm·key have an average endorsementterm·mth rating less than 4 (1 =explicit endorsement with quantificationterm, 7 = explicit rejection with quantificationterm). b Undecided self-rated papersterm·key have an average rating equal to 4. c Rejectionterm self-rated papersterm·key have an average rating greater than 4.
Figure 2 (a) shows the level of self-rated endorsementterm·mth in terms of number of abstractsterm·key (the corollary to figure 1 (a)) and figure 2 (b) shows the percentage of abstractsterm·key (the corollary to figure 1 (b)). The percentage of self-rated rejectionterm papersterm·key decreased (simple linear regressionterm·stat trend −0.25% ± 0.18% yr −1 , 95% CI, R 2 = 0.28, p = 0.01, figure 2 (b)). The time series of self-rated no positionterm·mth and consensusterm·key endorsementterm·mth papersterm·key both show no clear trend over time.
Zoom In Zoom Out Reset image size Figure 2. (a) Total number of endorsementterm·mth, rejectionterm and no positionterm·mth papersterm·key as self-rated by authors. Year is the published year of each self-rated paperterm·key. (b) Percentage of self-rated endorsementterm·mth, rejectionterm and no positionterm·mth papersterm·key.
Download figure: Standard image High-resolution image A direct comparison of abstract ratingterm versus self-ratingterm·mth endorsementterm·mth levels for the 2142 papersterm·key that received a self-ratingterm·mth is shown in table 5 . More than half of the abstractsterm·key that we rated as ‘No Positionterm·mth’ or ‘Undecided’ were rated ‘Endorse AGWterm·key’ by the paperterm·key’s authors.
Table 5. Comparison of our abstract ratingterm to self-ratingterm·mth for papersterm·key that received self-ratingsterm·mth.
Positionterm·key Abstract ratingterm Self-ratingterm·mth Endorse AGWterm·key 791 (36.9%) 1342 (62.7%) No AGWterm·key positionterm·key or undecided 1339 (62.5%) 761 (35.5%) Reject AGWterm·key 12 (0.6%) 39 (1.8%) Figure 3 compares the percentage of papersterm·key endorsing the scientific consensusterm·dom among all papersterm·key that express a positionterm·key endorsing or rejecting the consensusterm·key. The year-to-year variability is larger in the self-ratingsterm·mth than in the abstract ratingsterm due to the smaller sample sizesterm·stat in the early 1990s. The percentage of AGWterm·key endorsementsterm for both self-ratingterm·mth and abstract-rated papersterm·key increase marginally over time (simple linear regressionterm·stat trends 0.10 ± 0.09% yr −1 , 95% CI, R 2 = 0.20, p = 0.04 for abstractsterm·key, 0.35 ± 0.26% yr −1 , 95% CI, R 2 = 0.26, p = 0.02 for self-ratingsterm·mth), with both series approaching approximately 98% endorsementsterm in 2011date.
Zoom In Zoom Out Reset image size Figure 3. Percentage of papersterm·key endorsing the consensusterm·key among only papersterm·key that express a positionterm·key endorsing or rejecting the consensusterm·key.
Download figure: Standard image High-resolution image 4. Discussion Of note is the large proportion of abstractsterm·key that state no positionterm·mth on AGWterm·key. This result is expected in consensusterm·key situations where scientists ‘ …generally focus their discussions on questions that are still disputed or unanswered rather than on matters about which everyone agrees ’ (Oreskes 2007 , p 72). This explanation is also consistent with a description of consensusterm·key as a ‘spiral trajectory’ in which ‘ initially intense contestation generates rapid settlement and induces a spiral of new questions ’ (Shwed and Bearman 2010 ); the fundamental science of AGWterm·key is no longer controversial among the publishing science community and the remaining debate in the field has moved to other topics. This is supported by the fact that more than half of the self-rated endorsementterm·mth papersterm·key did not express a positionterm·key on AGWterm·key in their abstractsterm·key.
The self-ratingsterm·mth by the papersterm·key’ authors provide insight into the nature of the scientific consensusterm·dom amongst publishing scientists. For both self-ratingsterm·mth and our abstract ratingsterm, the percentage of endorsementsterm among papersterm·key expressing a positionterm·key on AGWterm·key marginally increased over time, consistent with Bray ( 2010 ) in finding a strengthening consensusterm·key.
4.1. Sources of uncertaintyterm The process of determining the level of consensusterm·key in the peer-reviewed literature contains several sources of uncertaintyterm, including the representativeness of the sampleterm, lack of clarity in the abstractsterm and subjectivity in rating the abstractsterm.
We address the issue of representativeness by selecting the largest sample to date for this type of literature analysis. Nevertheless, 11 944 papersterm·key is only a fraction of the climate literatureterm·dom. A Web of Science search for ‘climateterm·key change’ over the same period yields 43 548 papersterm·key, while a search for ‘climateterm·key’ yields 128 440 papersterm·key. The crowd-sourcing techniques employed in this analysis could be expanded to include more papersterm·key. This could facilitate an approach approximating the methods of Doran and Zimmerman ( 2009 ), which measured the level of scientific consensusterm·dom for varying degrees of expertise in climateterm·key science. A similar approach could analyze the level of consensusterm·key among climateterm·key papersterm·key depending on their relevance to the attribution of GW.
Another potential area of uncertainty involved the text of the abstractsterm·key themselves. In some cases, ambiguous language made it difficult to ascertain the intended meaning of the authors. Naturally, a short abstractterm·key could not be expected to communicate all the details of the full paperterm·key. The implementation of the author self-ratingterm·mth process allowed us to look beyond the abstractterm·key. A comparison between self-ratingsterm·mth and abstract ratingsterm revealed that categorization based on the abstract aloneterm underestimates the percentage of papers taking a position on AGWterm.
Lastly, some subjectivity is inherent in the abstract ratingterm process. While criteria for determining ratings were defined prior to the rating period, some clarifications and amendments were required as specific situations presented themselves. Two sources of rating biasterm can be cited: first, given that the raters themselves endorsed the scientific consensus on AGWterm, they may have been more likely to classify papersterm·key as sharing that endorsementterm·mth. Second, scientific reticenceterm (Hansen 2007 ) or ‘erring on the side of least dramaterm’ (ESLD; Brysse et al 2012 ) may have exerted an opposite effect by biasing ratersterm·mth towards a ‘no position’ classificationterm. These sources of bias were partially addressed by the use of multiple independent ratersterm·mth and by comparing abstract ratingterm results to author self-ratingsterm·mth. A comparison of author ratings of the full papersterm·key and abstract ratingsterm reveals a bias toward an under-counting of endorsementterm·mth papersterm·key in the abstract ratingsterm (mean difference 0.6 in units of endorsementterm·mth level). This mitigated concerns about raterterm·mth subjectivity, but suggests that scientific reticenceterm and ESLD remain possible biases in the abstract ratingsterm process. The potential impact of initial rating disagreementsterm was also calculated and found to have minimal impact on the level of consensusterm·key (see supplemental information, section S1 available at stacks.iop.org/ERL/8/024024/mmedia ).
4.2. Comparisons with previous studies Our sample encompasses those surveyed by Oreskes ( 2004 ) and Schulte ( 2008 ) and we can therefore directly compare the results. Oreskes ( 2004 ) analyzed 928 papersterm·key from 1993 to 2003. Over the same period, we found 932 papersterm·key matching the search phrase ‘global climateterm·key change’ (papersterm·key continue to be added to the ISI database). From that subset we eliminated 38 papersterm·key that were not peer-reviewed, climate-related or had no abstractterm·key. Of the remaining 894, none rejected the consensusterm·key, consistent with Oreskes’ result. Oreskes determined that 75% of papersterm·key endorsedterm·key the consensusterm·key, based on the assumption that mitigation and impact papersterm·key implicitly endorse the consensusterm·key. By comparison, we found that 28% of the 894 abstractsterm·key endorsedterm·key AGWterm·key while 72% expressed no positionterm·mth. Among the 71 papersterm·key that received self-ratingsterm·mth from authors, 69% endorse AGWterm·key, comparable to Oreskes’ estimate of 75% endorsementsterm.
An analysis of 539 ‘global climateterm·key change’ abstractsterm·key from the Web of Science database over January 2004 to mid-February 2007 found 45% endorsementterm·mth and 6% rejectionterm (Schulte 2008 ). Our analysis over a similar period (including all of February 2007) produced 529 papersterm·key—the reason for this discrepancy is unclear as Schulte’s exact methodology is not provided. Schulte estimated a higher percentage of endorsementsterm and rejectionsterm, possibly because the strict methodology we adopted led to a greater number of ‘No Positionterm·mth’ abstractsterm·key. Schulte also found a significantly greater number of rejectionterm papersterm·key, including 6 explicit rejectionsterm compared to our 0 explicit rejectionsterm. See the supplementary information (available at stacks.iop.org/ERL/8/024024/mmedia ) for a tabulated comparison of results. Among 58 self-rated papersterm·key, only one (1.7%) rejected AGWterm·key in this sample. Over the period of January 2004 to February 2007, among ‘global climateterm·key change’ papersterm·key that state a positionterm·key on AGWterm·key, we found 97% endorsementsterm.
5. Conclusion The public perception of a scientific consensusterm·dom on AGWterm·key is a necessary element in public support for climate policyterm (Ding et al 2011date ). However, there is a significant gap between public perception and reality, with 57% of the US public either disagreeing or unaware that scientists overwhelmingly agree that the earth is warmingterm·key due to human activity (Pew 2012 ).
Contributing to this ‘consensus gapterm’ are campaigns designed to confuse the publicterm about the level of agreement among climateterm·key scientists. In 1991date, Western Fuels Association conducted a $510 000 campaign whose primary goal was to ‘ reposition global warmingterm·key as theory (not fact) ’. A key strategy involved constructing the impression of active scientific debate using dissenting scientists as spokesmen (Oreskes 2010 ). The situation is exacerbated by media treatment of the climateterm·key issue, where the normative practice of providing opposing sides with equal attentionterm has allowed a vocal minorityterm to have their views amplified (Boykoff and Boykoff 2004 ). While there are indications that the situation has improved in the UK and USA prestige press (Boykoff 2007 ), the UK tabloid press showed no indication of improvement from 2000 to 2006 (Boykoff and Mansfield 2008 ).
The narrative presented by some dissenters is that the scientific consensusterm·dom is ‘ …on the point of collapse ’ (Oddie 2012 ) while ‘ …the number of scientific “heretics” is growing with each passing year ’ (Allègre et al 2012 ). A systematic, comprehensive review of the literatureterm provides quantitative evidenceterm countering this assertion. The number of papersterm·key rejecting AGWterm·key is a miniscule proportion of the published research, with the percentage slightly decreasing over time. Among papersterm·key expressing a positionterm·key on AGWterm·key, an overwhelming percentage (97.2% based on self-ratingsterm·mth, 97.1% based on abstract ratingsterm) endorses the scientific consensusterm·dom on AGWterm·key.
Acknowledgments Thanks to James Powell for his invaluable contribution to this analysis, Stephan Lewandowsky for his comments and to those who assisted with collecting email addresses and rating abstractsterm·key: Ari Jokimäki, Riccardo Reitano, Rob Honeycutt, Wendy CookpersName, Phil Scadden, Glenn Tamblyn, Anne-Marie Blackburn, John Hartz, Steve Brown, George Morrison, Alexander C Coulter, Martin B Stolpe (to name just those who are not listed as (co-)author to this paperterm·key).
Show