Expert judgments about transient climate response to alternative future trajectories of radiative forcing

Abstract

There is uncertainty about the response of the climate system to future trajectories of radiative forcing. To quantify this uncertainty we conducted face-to-face interviews with 14 leading climate scientists, using formal methods of expert elicitation. We structured the interviews around three scenarios of radiative forcing stabilizing at different levels. All experts ranked “cloud radiative feedbacks” as contributing most to their uncertainty about future global mean temperature change, irrespective of the specified level of radiative forcing. The experts disagreed about the relative contribution of other physical processes to their uncertainty about future temperature change. For a forcing trajectory that stabilized at 7 Wm^-2 in 2200, 13 of the 14 experts judged the probability that the climate system would undergo, or be irrevocably committed to, a “basic state change” as ≥0.5. The width and median values of the probability distributions elicited from the different experts for future global mean temperature change under the specified forcing trajectories vary considerably. Even for a moderate increase in forcing by the year 2050, the medians of the elicited distributions of temperature change relative to 2000 range from 0.8–1.8 °C, and some of the interquartile ranges do not overlap. Ten of the 14 experts estimated that the probability that equilibrium climate sensitivity exceeds 4.5 °C is > 0.17, our interpretation of the upper limit of the “likely” range given by the Intergovernmental Panel on Climate Change. Finally, most experts anticipated that over the next 20 years research will be able to achieve only modest reductions in their degree of uncertainty.

Uncertainty about the response of the climate system to future changes in radiative forcing arises from incomplete forcing and climate response data, incomplete understanding of climate system processes, and the limitations of climate models. A number of studies using models of different complexity and different statistical methods have produced probabilistic estimates of equilibrium climate sensitivity (see refs. 1 and 2 for an overview) and projections over the twenty-first century (2, 3). Such modeling studies offer considerable insight about future climate change, its likely impacts, and associated uncertainties.

However, experts working in climate science possess knowledge that is not captured in models. To better explore this knowledge, we have previously employed methods of formal expert elicitation (4–9) to gain additional insight about the likely value of climate sensitivity (10), the likely impact of climate change on tropical and boreal forest ecosystems (11), the likely impacts of climate change on the Atlantic Meridional Overturning Circulation (12) and the likely values of direct and indirect radiative forcing from anthropogenic aerosols (13).

Here we report results from a series of detailed formal face-to-face elicitations conducted with leading climate scientists in North America and Europe (Table 1) on the time-dependent response of the climate system to scenarios of radiative forcing. In a previous expert elicitation conducted with climate scientists (10) we focused on uncertainty in the value of equilibrium climate sensitivity (the equilibrium global mean temperature change resulting from a doubling of the preindustrial atmospheric CO₂ concentration). Whereas that quantity has been widely discussed in the scientific and policy literatures, it is far less relevant to policy making and to the assessment of likely impacts over the coming centuries, than the time-dependent response of the climate system.

Table 1.

Experts interviewed in this study

Name	Affiliation
Allen, M.	University of Oxford, Oxford, UK
Collins, M.	Hadley Centre for Climate Prediction and Research, Exeter, UK
Flato, G.	Canadian Centre for Climate Modelling and Analysis, Victoria, BC, Canada
Forest, C.	Massachusetts Institute of Technology, Cambridge, MA, USA
Karl, T.	National Climatic Data Center, Asheville, NC, USA
Knutti, R.	National Center for Atmospheric Research, Boulder, CO, USA
Rahmstorf, S.	Potsdam Institute for Climate Impact Research, Potsdam, Germany
Schlesinger, M.	University of Illinois, Urbana-Champaign, IL, USA
Schneider, S.H.	Stanford University, Palo Alto, CA, USA
Senior, C.	Hadley Centre for Climate Prediction and Research, Exeter, UK
Stainforth, D.	University of Oxford, Oxford, UK
Stone, P.	Massachusetts Institute of Technology, Cambridge, MA, USA
Weaver, A.J.	University of Victoria, Victoria, BC, Canada
Wigley, T.	National Center for Atmospheric Research, Boulder, CO, USA

The numbers that identify experts in the text and figures were randomly assigned and do not correspond to the order they are listed in this table.

To focus attention on the transient climate response, we structured the elicitation around scenarios that reflect a range of plausible future radiative forcing trajectories. Whereas the scenarios we employed were developed prior to the publication of the representative concentration pathways (RCPs) proposed for the climate model simulations in support of the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) (14), they are similar to the RCPs (Fig. S1).

We constructed three scenarios of net radiative forcing at the top of the atmosphere from anthropogenic sources through the year 2200 (Fig. 1). In a “high” scenario radiative forcing stabilized at 7 Wm^-2 in 2200, in a “medium” scenario it stabilized at 4 Wm^-2, and in a low “overshoot” scenario forcing peaked at about 3 Wm^-2 in 2070 and then declined to near zero by 2200. We asked experts to assume that forcing from non-CO₂ greenhouse gases and aerosols remained constant at year 2000 levels. Since the year 2000 forcings of these agents nearly compensate each other, the total forcing is very similar to that of CO₂ alone. To improve the match between the experts’ knowledge and the question domain, we chose deliberately to specify scenarios of radiative forcing (as opposed to emissions) so as to limit discussion to the uncertainty in the physical rather than biogeochemical processes that determine the response of the climate system to forcing. Most of our respondents have limited expert ecological and other biogeochemical knowledge. Specifying total radiative forcing instead of emissions obviated the need to ask about carbon cycle feedbacks, although two experts did explicitly discuss such effects.

Fig. 1.

Scenarios of radiative forcing used in this study. The right hand vertical scale denotes the equivalent CO₂ concentration.

Results and Discussion

Key Factors Influencing Uncertainty in Transient Temperature Response.

Before the face-to-face interviews, experts completed an e-mail survey to identify the factors they believed would most contribute uncertainty to their judgments about the change in global mean temperature, ΔT(t), for each of the three forcing trajectories. From those responses, we compiled the list of factors shown in the left-most column of Table S1.

We began the face-to-face interviews with a briefing on judgment under uncertainty (8, 15, 16) and then introduced and discussed the three forcing scenarios (Steps 1 and 2 in Table 2). In Step 3, experts were given a set of cards that listed each of the factors that had been identified in the e-mail survey and were asked to “rank these cards in terms of their relative importance in influencing uncertainty about the time trajectory of average global temperature.” We also asked the experts whether their ranking would be different for different levels of forcing. If the answer was positive, we asked them to sort the factors for each of the three scenarios displayed in Fig. 1. Some experts chose to give the same ranking for all three forcing trajectories. Others gave different rankings for the medium and high scenarios, but did not distinguish between the low and medium trajectories. Experts 5 and 6 chose to include climate-carbon cycle feedbacks among the factors most contributing to their uncertainty about transient temperature response.

Table 2.

Structure of the face-to-face elicitation protocol used in this study

1.	Briefing on issues involved in making judgments about uncertainty, focusing particularly on the large body of evidence on systematic overconfidence among lay and expert respondents.
2.	Introduction and discussion of the three hypothetical trajectories of future radiative forcing (Fig. 1).
3.	Discussion of the relative importance of factors that influence the expert’s judgments about uncertainty in the transient climate response.
4.	Exploration of whether, and at what level of forcing, the climate system might undergo a state change in the climate system given varying levels of forcing in 2200.
5.	Probabilistic judgments about the amount of warming resulting from alternative plausible future levels of forcing.
6.	Discussion of whether and how the expert’s uncertainty about transient response might change as a function of future research.
7.	Probabilistic judgments about the value of classic climate sensitivity.
8.	Although not reported in this paper, the elicitation closed with a series of questions about the feasibility of downscaling climate variables for 11 specific geographical regions.

The full interview protocol is reproduced in the SI Appendix.

Under all three forcing scenarios, “cloud radiative feedbacks” is the factor that experts unanimously believe most contributes to uncertainty in the transient global mean temperature response. Experts’ rankings of other factors are scenario dependent, and display large disagreements (see individual expert responses in Table S1, Table S2, Table S3, and Table S4).

We computed mean (averaged over experts) ranks using different procedures (see SI Text). We find that except for the top ranked process (cloud radiative feedbacks) the ordering of mean ranks is not entirely robust with respect to the procedure used. However, applying different ordering procedures allowed us to identify distinct sets of processes, whose relative ordering is independent on the ordering procedure applied. The ordering of these sets is reported in Table 3 for the three forcing scenarios.

Table 3.

Experts’ average ranking of physical processes influencing uncertainty in the global mean surface air temperature response to the three forcing scenarios displayed in Fig. 1

Rank	Medium scenario	High scenario	Low scenario
1	Cloud radiative feedback	Cloud radiative feedback	Cloud radiative feedback
2	Sea-ice albedo feedback	Land-ice and snow albedo feedback	Sea-ice albedo feedback
	Land-ice and snow albedo feedback	Water vapor feedback	Land-ice and snow albedo feedback
3	Water vapor feedback	Vertical/diapycnal ocean mixing	Deep water formation
	Deep water formation	Vegetation albedo feedback	Ocean circulation (wind-driven and thermohaline)
	Ocean circulation (wind-driven and thermohaline)	Ocean circulation (wind-driven and thermohaline)	Water vapor feedback
	Vegetation albedo feedback	Deep water formation	Vegetation albedo feedback
	Lapse rate feedback	Sea-ice albedo feedback	Lapse rate feedback
	Vertical/diapycnal ocean mixing	Lapse rate feedback	Vertical/diapycnal ocean mixing
4	Atmospheric convection and precipitation	Soil moisture	Atmospheric convection and precipitation
	Horizontal/isopycnal ocean mixing	Large-scale atmospheric circulation	Soil moisture
	Soil moisture	Horizontal/isopycnal ocean mixing	Large-scale atmospheric circulation
	Ocean eddies	Atmospheric convection and precipitation	Horizontal/isopycnal ocean mixing
	Large-scale atmospheric circulation	Ocean eddies	Ocean eddies

Although the mean ranking of individual processes is sensitive to the ordering procedure used (details in SI Text), four sets of processes could be identified for each scenario whose ordering is independent on the specific procedure. Within each set, the processes are ranked according to the first ordering procedure described in SI Text.

For the medium scenario, four processes or sets of processes can be identified: a first set of cloud radiative feedbacks; a second set including the sea-ice and land-ice albedo feedbacks; a third set including the water vapor feedback, processes affecting ocean heat uptake (deep water formation, wind-driven and thermohaline ocean circulation, vertical/diapycnal mixing), the vegetation albedo feedback, and the lapse rate feedback; and lastly a set including atmospheric convection and precipitation, large-scale atmospheric circulation, soil moisture, horizontal/isopycnal ocean mixing, and ocean eddies.

The ranking for the low forcing scenario is very similar to that for the medium scenario. For the high forcing scenario, the second ranked set includes the water vapor feedback, along with the snow- and land-ice albedo feedback. The sea-ice albedo feedback was on average ranked lower than under the medium forcing scenario, because sea-ice cover is expected to have largely disappeared under such a strong forcing, implying a low influence on the uncertainty in the temperature response.

Probability of State Change in the Climate System.

In Step 4 of the interview, we asked experts to make judgments about the probability that different levels of radiative forcing could trigger some “basic” state change in the climate system. Our written protocol defined basic as a state change “with global consequences persisting for several decades.” When pressed by the experts with candidate examples, we verbally elaborated that the changes would need to have effects that were at least hemispheric in scale, and that several decades was a lower bound on their persistence. Such events have been referred to as “tipping points” in the most recent literature (17). We elicited the experts’ probability that “the climate system will have undergone, or been irrevocably committed to a basic state change by the year 2200” on a scale from zero to one (0 = no chance; 1 = definite chance) for each of the three forcing trajectories displayed in Fig. 1.

The experts’ responses are summarized in Fig. 2. The elicited probabilities that the climate system will undergo a transition into a different state increases with the severity of the forcing scenario. For the low forcing scenario, the probabilities range from 0.01–0.5, with the majority of experts giving a probability between 0.1 and 0.2. For the medium forcing scenario the elicited probabilities lie between 0.2 and 0.9 with eight experts (out of 14) assigning a probability > 0.5. For the high forcing scenario, 13 experts gave probabilities ≥0.5, and 9 experts gave probabilities ≥0.9. Phenomena experts discussed when assigning probabilities include loss of a large portion of the Greenland and West Antarctic ice sheets, a substantial change in large-scale oscillatory patterns such as El Nino Southern Oscillation or the North Atlantic Oscillation, a shutdown of the Atlantic meridional overturning circulation, or a transition from sink to source in the terrestrial carbon cycle.

Fig. 2.

Elicited probabilities that the climate system will undergo, or would be irrevocably committed to a fundamental state change (i.e., a state change with global consequences persisting for several decades) by 2200 in response to the forcing trajectories displayed in Fig. 1.

These results are qualitatively similar to those of Kriegler et al. (18), who elicited subjective probability intervals for the occurrence of a variety of tipping points in the climate system. Many of the events our experts noted are similar to those discussed by Kriegler et al. That study reported conservative intervals for the probability of triggering at least one tipping point of 0.16–1.00 for medium (2–4 °C) and 0.56–1.00 for high (4–8 °C) global mean temperature change relative to 2000. Given the similarity between their medium and high scenarios and the temperature estimated by our experts, we can compare the two results. The probability intervals reported in Kriegler et al. encompass 100% of the experts’ probabilities of triggering a basic state change elicited in this study for the medium, and 71% (10 out of 14) for the high scenario. The discrepancy for the high scenario could be due to different assumptions regarding the warming at 2200. To explore this hypothesis, we calculated the correlation coefficient between the experts’ estimates of probability of state change, and their estimates of warming at 2200, for the three scenarios. Interestingly, within scenarios, we find no relationship between the probability of state change estimated by our experts and their assessed level of global mean temperature change (Fig. S2).

Probabilistic Judgments about Temperature Response to Forcing.

In Step 5 of the interview, experts were asked to quantify the global mean temperature change that might arise at specific times in response to a specific level of forcing. To assist the experts with this task, we provided a simple heuristic aid in the form of a simple energy balance model implemented in Mathematica® (SI Text). Experts could adjust the equilibrium climate sensitivity and the ocean time-scale with slider bars and view the global mean temperature response to each of the three forcing trajectories. Whereas several experts explored this model, most made no serious use of it in providing their judgments.

Fig. 3 reports box plots summarizing the full probability distributions that were elicited from each expert for the three forcing scenarios at 2050 and 2200 (points marked with dots in Fig. 1). Note the very considerable variation in the amount of uncertainty the different experts assessed. Even for the year 2050 (where the forcing for all three scenarios is very similar), the 90% confidence intervals of global mean surface air temperature change relative to 2000 range from 0.1–3.8 °C, and in some cases the interquartile ranges (i.e., the central boxes that contain half the probability) do not overlap. The median estimates for 2050 range from 0.8–1.8 °C.

Fig. 3.

Box plots of elicited probability distributions of global mean surface air temperature change (ΔT) relative to 2000 for the four points marked in Fig. 1 (Upper Left) 2050; (Upper Right) 2200, low scenario; (Lower Left) 2200, high scenario; (Lower Right) 2200, medium scenario. Vertical lines denote the range from minimum to maximum assessed possible value (in most cases, corresponding to the 1 and 99 percentiles). Horizontal tick marks encompass the 90% confidence interval, the box spans the 50% confidence interval and the dot marks the median. Note that expert 6 included climate-carbon cycle feedbacks in his probability estimates.

For the medium scenario at 2200, the medians range from 1.2–3.6 °C. Three experts (3, 12, and 14) include temperature increases > 6° C in their 90% confidence intervals. For the high forcing scenario at 2200, the medians range from 2.7–6.7 °C. The 90% confidence intervals include values from 0.8–12.5 °C. The upper bound of the probability distribution of expert 3 extends to 18 °C.

For the low forcing scenario at 2200, the medians range from -0.2–1.1 °C. The 90% confidence intervals encompass temperature changes from -1.2–2.6 °C. Note that some experts gave probability distributions that include values < -0.7 °C, that is net cooling relative to preindustrial global mean temperature levels, although the forcing is slightly positive. One potential mechanism experts invoked is a shutdown of the Atlantic meridional overturning circulation, which would lead to a cooler Northern Hemisphere and larger sea-ice extent.

In Step 5, experts were also asked to sketch how they believed their median estimate of ΔT(t) would evolve over time under the high and the low forcing scenarios. Fig. 4 reports these trajectories. The range of experts’ median estimates of the mean rate of warming between 2000 and 2050 falls between 0.16 °C/decade and 0.36 °C/decade. All experts expect the most probable temperature response for the low forcing scenario to peak sometime between 2100 and 2150 and decline thereafter, following the declining forcing. Peak temperature changes range between 1 and 2.5 °C. For the high forcing scenario, most experts’ estimates display a roughly linear temperature increase, with the rate of temperature change in some cases decreasing over the course of the century as the radiative forcing levels off.

Fig. 4.

Experts’ median estimates of the transient response of globally averaged temperature change (relative to 2000) for the high (Upper) and low (Lower) forcing trajectories.

Uncertainty Reduction Through Future Research.

Step 6 was designed to explore the extent to which experts believed that research could reduce their uncertainty about the future response of ΔT(t) to specific forcings. We asked experts to “suppose that we could turn to an oracle who could tell you everything you need to know to completely eliminate the uncertainty in your understanding about the influence of the top ranked factors on the resulting ΔT. Of course, even if the oracle could do that … you would likely still have uncertainty about the value of ΔT because there are a number of other factors that also contribute to your uncertainty.” We constructed box plots based on the probability distributions for ΔT that the experts had provided for the year 2050 and for the low and high forcing scenarios in 2200. In each case, we asked the experts to indicate how much (in percent) the width of their probability distribution would shrink if uncertainty about the top three factors (individually or collectively) could be completely eliminated.

In addition, we posed two more realistic questions, in which we specified that data collection and model development would continue at their current pace and funding levels for 20 more years (i.e., to 2027) or that future data collection and model development would be “optimally allocated” and funded at three times the current funding levels for the next 20 years.

Responses are summarized in Table 4. The potential for uncertainty reduction is on average assessed to be similar for the two time periods and for both the high and low forcing trajectories under the scenario that an “oracle” can completely eliminate uncertainty about the three factors that most contribute to the expert’s judgment about ΔT.

Table 4.

Percent decrease in the width of experts’ probability distributions of global mean temperature change in 2050 and 2200, if uncertainty in the top three physical processes could be completely eliminated (by an hypothetical oracle) or reduced through research at different funding levels

	2050			2200 low forcing			2200 high forcing
	Oracle	3x funding	Current funding	Oracle	3x funding	Current funding	Oracle	3x funding	Current funding
Max	90	90	80	90	50	30	80	40	30
Min	25	13	5	15	8	5	20	6	5
Mean	57	39	29	49	25	15	59	25	15
σ	20	19	20	23	15	8	23	13	8

The rows list the maximum, minimum, mean, and standard deviation (σ) of the values assigned by the experts.

Under scenarios in which observation and funding continues as today, or in which optimally allocated funding levels increased by a factor of three, the potential for uncertainty reduction is on average assessed to be largest for the year 2050. Several experts noted that, independent of improvements in process-knowledge, their uncertainty should reduce because there would be 20 more years of observational data.

For the year 2200 assessments, the values of percentage reduction in uncertainty reported in Table 4 are very similar for the low and high forcing scenarios in relative terms. However, note that the absolute uncertainty reduction is largest for the high forcing, because the probability distributions are much wider in this case (compare the box plots in the top right panel of Fig. 3 with the box plots in the bottom left panel of the same figure). Note that even under the scenario that an oracle can completely eliminate uncertainty about the three factors contributing the most to the expert’s uncertainty, on average experts believe that the width of their probability distribution would decrease only by 50–60%. This does not mean that all experts believe that a large part of their uncertainty is irreducible. Rather, as several experts pointed out, it may indicate that processes other than the top three contribute significantly to uncertainty about the value of ΔT. Tripling the research budget is judged to reduce future uncertainty, but not in direct proportion to the increased level of funding.

These results are consistent with findings from a study conducted by Morgan and Keith (10) in which 16 climate scientists were asked to design an optimal research program and then make judgments about whether and how much that research could reduce uncertainty about equilibrium climate sensitivity. In that study, conducted 15 years ago, experts’ judgments “strongly suggest(ed) that our ability to predict the gross character of climate change will improve slowly, even with well designed research programs.”

Equilibrium Climate Sensitivity.

In Step 7, we elicited probability distributions of “classic” climate sensitivity, which we defined, following the IPCC, as the equilibrium global mean surface air temperature resulting from a doubling of the preindustrial atmospheric CO₂ concentration. We again began by asking the participants to rank the same set of factors introduced earlier in terms of the relative importance in influencing uncertainty about equilibrium climate sensitivity. The resulting average ranking (Table S4) is similar to that for the medium forcing scenario (Table 3), except that oceanic processes (deep-water formation, ocean circulation, etc.) have lower ranks. In terms of their contribution to the global mean temperature response, these processes mainly determine the rate of ocean heat uptake, which is important for the transient response but which settles down as the system approaches equilibrium. However, as expert 12 noted, oceanic processes can contribute to uncertainty about equilibrium climate sensitivity because observational evidence on climate feedbacks is influenced by uncertainty in oceanic heat uptake.

We elicited probability distributions of the value of equilibrium climate sensitivity (Fig. 5). Experts’ median values lie in the range 2.8–4.2 °C, with 11 out of 14 experts’ medians lying between 3 and 3.5 °C. The experts 90% confidence intervals include values from 1–8 °C. There is broad agreement between the experts’ judgments and the several model-based cumulative distribution functions (CDFs) of climate sensitivity reported in Box 10.2 of the IPCC Fourth Assessment Working Group I (WGI) report (2). The ranges of the 50%, and in particular the 95% confidence levels of the model-based climate sensitivity distributions are wider than those elicited from our experts. The IPCC report (2) assesses that the “equilibrium climate sensitivity is likely to lie in the range 2–4.5 °C, with a most likely value of about 3 °C.” IPCC defines likely as a 0.66–0.90 probability, which in Chapter 19, Working Group II (WGII) (19) is interpreted as a 0.05–0.17 probability that climate sensitivity is > 4.5 °C. Examining the elicited distributions obtained from our experts, we find that 10 of the 14 experts placed > 0.17 of their probability above 4.5 °C.

Fig. 5.

Box plots of elicited probability distributions of equilibrium climate sensitivity. The vertical bars at the right span the values reported in Fig. 2, Box 10.2 of IPCC WGI (2). Tick marks on the line marked L report the lower 5% confidence levels of the CDFs in that figure, those on the line marked H report the 95% confidence levels. Median values for the distributions reported in Box 10.2 are given by solid dots on the line marked M. The values in brackets below each expert number indicate the probability that each expert allocated to values > 4.5 °C. We read IPCC (2) as assessing this value as 0.17. The gray box plots denote the distributions that were obtained from experts 2, 4, 6, and 8 in an earlier elicitation conducted by two of the authors (10).

For four of the participating experts it is possible to compare the probability distributions to those elicited 12 years previously from the same experts (10) (Fig. 5). In all four cases, the probability distributions are shifted toward higher value of climate sensitivity. Except for expert 4, the widths of the distributions are approximately unchanged, despite 12 years of additional research. This confirms the expectation of most participants in the Morgan and Keith study (10), who anticipated a modest reduction in uncertainty about climate sensitivity over the following two decades.

A comparison of the elicited probability distributions in Figs. 3 and 5 provides insight into the experts’ beliefs about the relationship between transient and equilibrium warming. For instance, comparing the experts’ estimates of warming in 2200 for the medium scenario with those of equilibrium climate sensitivity indicates that most experts estimate the warming commitment after 2200 due to ocean thermal inertia to be very small. Linear regression of the median warming estimates in 2050 and the median estimates of equilibrium climate sensitivity (Fig. S3) suggests that the experts’ short-term transient warming estimates are independent of climate sensitivity. This is consistent with model-based studies, which indicate that for time-scales well short of equilibrium, the model’s transient climate response is independent of the model’s climate sensitivity (2). The positive correlation between transient warming and equilibrium climate sensitivity is higher in the year 2200, particularly for the medium and high forcing scenarios (Fig. S3).

Materials and Methods

We developed the written interview protocol over a period of approximately eight months (November 2005 to June 2006). It was refined following pilot interviews with scientists at Woods Hole Oceanographic Institution and the National Center for Atmospheric Research in July 2007. While it was similar in basic structure to previous elicitation studies we have conducted (10–13), this protocol included several methodological innovations including the preinterview mail survey that was used to better structure the issues to be addressed in the face-to-face sessions, the use of a simple Mathematica® model that experts could use to explore the space of possible outcomes, and a new approach to assessing the contribution that future research might likely make to reducing uncertainties. The full interview protocol is available in SI Text.

The elicitations were conducted between February and August 2007 in face-to-face interviews that took place at the experts’ home institutions and lasted 3–4 hours. All the interviews were conducted by Morgan, assisted by Zickfeld in the interviews with United States- and Canada-based experts, and by Frame in the interviews with Europe-based scientists. Experts annotated their specific responses in written form in an interview workbook. Audio recordings were made of all interviews and the interviewers took extensive notes. Experts were encouraged to consult literature, simulation results, notes, and other materials during the interview to obtain the experts’ carefully considered opinion.

After completing all interviews, we examined the experts’ judgments for consistency. For example, because the medium scenario stabilizes at a level of radiative forcing corresponding to approximately twice the preindustrial CO₂ concentration, one would expect the transient warming at 2200 to be equal to or smaller than the equilibrium climate sensitivity. When inconsistencies were found, we brought them to the attention of the respective expert, and asked for revision. We also sent all experts a summary of the elicited data, giving them an opportunity to compare their judgments with those of others, and potentially reconsider.

Details on how the experts were chosen, the strategies used in eliciting probability distributions, the approaches used to compute mean rankings of factors, and the simple Mathematica® model, are described in SI Text.