Commitment-Failures Are Unlikely to Undermine Public Support for the Paris Agreement

Abstract

Success of the 2015 Paris Agreement, which is founded on Nationally Determined Contributions (NDCs), hinges on whether domestic support for international environmental agreements would be undermined if countries that are crucial to the global effort fail to reduce their emissions. Here we find that citizens in China (n = 3000) and the United States (n = 3007) have strong preferences over the design of international climate agreements, and contributions of other countries to the global effort. However, contrary to what standard accounts of international politics would predict, a survey-embedded experiment in which respondents were randomly exposed to different information on other countries’ behavior showed that information on other countries failing to reduce their emissions does not undermine support for how international agreements are designed. While other factors still make large emission cuts challenging, these results suggest that the Paris approach per se is not posing a problem.

Many of the world's environmental problems can only be solved through concerted international efforts. This has led to hundreds of multilateral environmental treaties, which vary tremendously in their ambition levels and effectiveness (1;2). One key challenge with such efforts is that any given country's contribution to a collective effort is likely to depend, to varying degrees, on how much other countries contribute (3;4;5). This makes such efforts potentially vulnerable to non-cooperation by others, notably when countries responsible for a large part of a highly salient environmental problem fail to cooperate or disengage from existing cooperation (6;7;8;9).

Non-cooperation by countries that are important to international environmental problem solving can affect collective efforts in various ways. Most obviously, such non-cooperation usually means that problem solving is incomplete, in the sense that the share of the problem the non-cooperator is responsible for is not taken care of. Equally important, however: non-cooperation by others can also enhance the political leverage of non-green parts of electorates and interest groups in any given country whose government otherwise would want to cooperate, particularly when cooperation is costly and economic level-playing field arguments can be invoked (10;11;12).

While the former pathway to vulnerability of international environmental agreements is certainly crucial and rather well-researched, the latter pathway, operating primarily through public opinion, has received much less attention. Climate change mitigation, on which we focus here, is highly interesting in this respect. The transition from an explicit burden-sharing and formal targets-based approach under the Kyoto Protocol to the pledge and review approach under the Paris Agreement has come with less clarity about how strongly key emitters are really committed to reducing their emissions. Moreover, the largest historical emitter country, the United States, has explicitly disengaged from the Paris process, while some other important countries are sending mixed signals (13).

As already noted, one important mechanism through which uncertainty over the problem solving contribution by large emitter countries may weaken the political resolve of other countries operates through public opinion. Many studies show that policy choices by governments tend to follow some kind of thermostatic process (14;15). That is, they are associated with prevailing public demand for government (in-)action in a given area (16), though well-organized interest groups may at times lead to deviations from prevailing public opinion (median voter policy preferences) (17;18;19) and public opinion may oftentimes affect policy-choices not directly but via elite perception of public opinion (20). In climate policy, where mitigation measures have strong and direct effects on people's everyday lives, public support for such measures is thus particularly relevant (21).

To begin with, the NDC approach, relative to the previous approach of explicit, legally binding emission targets, may from the viewpoint of the public in any given country create more uncertainty, a priori, about whether other countries will stick to their pledges. Such uncertainty arises at two levels. First, various studies explaining widespread compliance with international legal commitments point to domestic audience costs as a key factor (22;23). They argue that government accountability to citizens implicates that when governments accept an international legal commitment and then fail to comply this is likely to reduce political support levels from citizens. Second, non-compliance with international legal commitments usually attracts criticism from other parties of the respective agreement and may even result in punitive action by others. The shift to NDCs increases uncertainty about how the domestic public and other countries would respond to “non-compliance” with less formal commitments that did not result from an explicit international bargain following a legally binding “give-and-take” logic.

These two potential sources of uncertainty have not prevented the large majority of countries from engaging in the Paris process and putting forth NDCs. Nonetheless, they could make governments and their publics more “edgy” with respect to the behavior of other countries, compared to the Kyoto approach. Following the logic of “tacit bargaining” (24), in the absence of a formal international deal, one should assume that governments and their publics are likely to adopt a wait-and-see attitude and adjust their support levels for their own country’s emission cuts in response to observing what other countries do. We should thus expect that public climate policy support levels in any given country decrease in response to other countries’ failure to cut their emissions, and vice-versa. This should be particularly so when commitment-failure occurs in a large emitter country.

To assess this possibility, and thus the risk posed by the NDC approach, we examined how public opinion in the world's two largest emitter countries, China and the United States, is affected by the mitigation behavior of other countries. Our approach can be regarded as a stress-test on the Paris Agreement from a public opinion perspective (25). To this end, we carried out survey experiments in China (n = 3000) and the United States (n = 3007), using quota sampling to ensure representative samples in terms of age, employment status, gender, income, and region (see Methods). In a conjoint experiment, participants were asked five times to choose between two randomly assigned policy proposals whose characteristics differ along a set of attributes (cost per household, legal strength of the agreement, decision-making process, inclusion of large emitters, number of countries included, and compensation to affected areas; see Supplementary Table 1). This resulted in a final number of observations (N) of 30000 and 30070 for China and the USA respectively. The attributes tested here are particularly relevant, as they capture citizens’ willingness to accept certain cost implications, domestic compensation for policy-losers, as well as important international policy design features that could safeguard against free-riding or commitment failure by others. This conjoint experiment was combined with an information provision experiment, where respondents were randomly assigned information on other countries' coal consumption compared to the home country's consumption (see Supplementary Figure 1). While conjoint experiments have previously been used in a few studies to examine public support for international climate agreements and also unilateral climate policy (26;27), the combination with an information provision experiment, the treatment designs, and the statistical approach to analyze the data are novel. This study design allows us to estimate the causal effect of information about other countries' behavior upon preferences over the design of global climate policy as well as national effort levels, as well as the causal effect of specific contents of policy proposals upon support (28).

To estimate the treatment effects of interest, we use a recently developed Bayesian variable selection method, LASSOplus, which enables us to simultaneously select and estimate relevant heterogenous treatment effects (29). See Methods for details.

Public support for climate agreement design features

The unconditional average marginal component effects (AMCEs, Figure 1) for the China and US samples, which result from the analysis of data from the conjoint experiment, give us a sense of the general patterns of public support for particular features of international climate agreement design. Notably, the results are broadly similar for both countries and in the direction of previous research in this area, though most of this research focuses only on the US and a few other Western countries (26;27). International climate agreements become less popular when costs increase, fewer of the major emitters are included, and when agreements are weaker in terms of legal status and enforcement. They become more popular with a larger number of participating countries, when details of the agreement are jointly decided by countries, and when some form of compensation mechanism is included.

Figure 1. Public support for features of international climate agreements.

The points indicate how the probability of support for an international environmental agreement changes when respondents receive an attribute value, compared to the appropriate baseline value, for respondents in China (N = 30000; circles) and the United States (N = 30070; triangles). Varied attributes related to a) Cost to household per year; b) Legal strength of the agreement; c) decision-making process; d) inclusion of large emitters; e) number of countries included; f) compensation to affected areas. The lines indicate 95% credible intervals.

We also observe some substantive differences in the salience of agreement design features when comparing Chinese and US publics. Chinese respondents tend to place more importance on the legal status of the international agreement. US respondents, however, tend to be particularly concerned about China's inclusion in the agreement, with an international agreement including China receiving higher levels of support. US study participants are more sensitive to the individual cost of an agreement. Nevertheless, preferences over the design of an international climate agreement are quite similar, suggesting commonalities in what citizens in different countries consider important features of such an agreement.

These results demonstrate that the public does care a lot about what an international climate agreement should look like, and also about contributions of other countries to the global effort. In fact, the most popular agreement proposals in both the China and US samples have a probability of acceptance in the order of around 80% and 90%, compared to 25% and 15% for the least popular agreement proposals, respectively.

Effects of information on choice of climate agreement

Combining the conjoint experiment with the information provision treatments, which implicates a much more complex study design than in previous research, allows us to address whether and how much non-cooperation by other countries could undermine public support. Specifically, it reveals whether non-cooperation by other countries induces less public support for climate policy, and whether it induces changes in the relative saliency of design features of an international climate agreement that reflect reciprocity concerns.

The first four treatment conditions (see Supplementary Figure 1) compare the fossil fuel, specifically coal, consumption behavior of the respondent’s home country to that of another country (USA/China, depending on sample, or the UK) or generic “other countries”. These information treatments capture the possible combinations of whether a specific country is mentioned or not, and whether the other country(ies) increased or decreased fossil fuel consumption. Treatment 5 gives respondents information only on their own country, while the control group receives no information about either their own country and other countries’ coal consumption. By allowing AMCEs from the conjoint experiment to vary according to which information treatment a study participant receives in the information provision experiment, we can estimate whether and how much preferences are dependent upon other countries' fossil fuel consumption and thus emissions behavior.

Figure 2 first displays how support for the most popular international climate agreement in the control group changes when individuals are given information about their own and other countries’ behaviour. The results show that information about other countries’ increasing emissions can lead to a decline in support for the respective climate agreement, relative to other possible agreements. For example, the probability of acceptance for this climate agreement decreases by 0.12 when study participants are given information about the UK increasing coal consumption. However, this impact is not particularly significant, as even with this change the probability of acceptance for the agreement remains higher than 0.75, suggesting that support remains high.

Figure 2. Effect of information provision on support for the most popular climate agreement.

Probability of support for the most popular agreements in China (N = 30000) and the United States (N = 30070) as a function of information provided about other countries. Lines indicate 83.4% confidence intervals, to visually assess whether the estimates are significantly different from one another. The most popular agreements are located in Supplementary Table 23.

Effects of information upon support for agreement features

To further understand our results, Figure 3 displays how preferences for the specific features of an agreement are affected by information about the behavior of other countries, compared to respondents’ respective home country. As a general result, we see that Chinese respondents are remarkably unresponsive to other countries’ behavior, when compared to the US sample. The only heterogenous effect selected is that Chinese respondents who receive information only about China’s coal consumption are less sensitive to paying 100RMB compared to 50RMB. Therefore, for the remaining discussion of these results we focus on the US sample.

Figure 3. Effect of information provision on support for international climate agreement features.

Change in support for features of an agreement in China (N = 30000; circles) and the United States (N = 30070; triangles) based on exposure to information about other countries' behavior. Points indicate the change in probability of support for an agreement when respondents receive this value, compared to the baseline, within a treatment condition. Lines indicate 95% credible intervals. a) Cost to household per year; b) legal strength of agreement; c) decision-making process; d) inclusion of large emitters; e) number of countries included; f) compensation to affected areas.

Cost

Other countries increasing or decreasing their coal consumption does not have a consistent impact. The only pattern that emerges is that if other countries are increasing emissions US respondents are less sensitive to cost, compared to if others are decreasing emissions. This suggests that inaction by other countries may actually lead to US respondents being more willing to pay for climate action.

Legal strength

We find some evidence that other countries increasing their emissions leads to a decrease in support for legally less binding agreements. This is matched by an increase in support for agreements with a formal legal basis, notably in the case of US respondents receiving information about China’s increase in coal consumption or only information on US coal consumption.

Decision-Making

We find some evidence that other countries increasing their emissions motivates US respondents to support agreements that involve a joint decision-making process. This effect is also mirrored in the control group.

Inclusion of other countries

US respondents do not wish to exclude countries that have increased their emissions. For example, when given information about China’s increased emissions, they are more supportive of including China in the agreement. If anything, US respondents are less likely to demand that “friendly” countries be a part of the agreement if the information suggests that other countries are also reducing coal consumption. This is shown by the decrease in support for including the EU in agreement when individuals receive information about other countries decreasing coal consumption.

Number of countries

There is no consistent pattern with respect to this policy attribute. While increased emissions by the UK lead to a desire for involving more countries, information on other countries’ and China’s emissions behavior does not induce such a response.

Compensation

The results show that information about the US’ decrease in coal consumption leads to increased support for compensation measures, independent of information about other countries’ coal consumption. This is shown by the control group, which receives no information about US or others’ coal consumption, being significantly less supportive of compensation measures. This suggests that a lack of support for compensation for the coal industry is driven by a belief that the industry has not faced a significant decline in previous years.

Discussion

We find that both Chinese and US publics have strong preferences over the design of international climate policy, in terms of what they prefer most and least. Nevertheless, failure of other countries to act does not, per se, reduce the public’s appetite for taking action against climate change. This is most obviously the case in China, where the behavior of other countries has effectively no impact whatsoever upon preferences over climate policy design. While we do find some effects of information on other countries’ behavior on preferences towards particular attributes of climate policy design, this does not lead to a substantive decrease in overall support for the most popular international climate policy.

One limitation of our research might be that our information provision treatments focus on coal. While this is meant to make those treatments easy to grasp and avoid false negatives due to “ordinary citizens’” inability to properly understand this information, it might have stacked the deck in favor of non-reciprocal policy preferences. The reason is that reducing coal consumption, in addition to mitigating global climate change, can also have local public health benefits. In view of other research suggesting that the mass public supports even unilateral reductions in fossil fuel consumption generally (not just coal) (9;27;30;31), we do not think that focusing on coal has, per se, biased our findings. Moreover, our results are broadly similar for China, which faces severe local air pollution problems linked to burning of coal, and the United States, whose air pollution levels are much lower. We are thus quite confident that our results are not an artefact of focusing on coal. However, it would be useful to replicate our study focusing on fossil fuels and emissions more generally. Moreover, further research could re-assess the audience cost argument noted above to obtain more nuanced insights into the relevance of international reciprocity in global climate policy. Specifically, it would be interesting to explore whether and how political support of citizens for their respective government (rather than for climate policy more narrowly), defined in broad terms, changes as they learn that their government sticks to its existing NDC, or weakens its commitment to the latter, in response to other countries weakening their commitment or disengaging from mitigation policy.

A further limitation of our study might be that a majority of Chinese respondents did not correctly understand our treatments shown that Chinese coal consumption increased in the time period covered by treatment information (see Supplementary Figure 3). We suspect that this may be a result of the current political climate in China. Another possibility is that respondents may have focused on a small decrease in 2011-2012, rather than the general increase in the overall time frame. Analyses excluding those who failed the respective comprehension checks does not significantly change the results for Chinese respondents. This, along with the fact that the main focus of our study is on the effect of information about other countries’ behavior, leads us to conclude that this issue does not significantly impact the inferences we wish to make in this paper. Nevertheless, future research could benefit from more closely considering how the political climate in countries may impact attempts to measure public opinion.

Our findings build on and add to existing research that has used simpler study designs to investigate public support for unilateral climate policy (9;27;30;31). They suggest that, at least for the time being, public support for current and future commitments countries have expressed as part of the Paris process is not vulnerable to commitment failures or even disengagement by other countries, for example the United States, even though citizens do pay attention to what other countries do and prefer other countries to contribute. Rather, it seems that citizens' preferences towards the make-up of international climate policy are quite solid, in the sense of being stable also in view of information on non-cooperation by other countries. As a result, climate policy has probably become less prone to traditional concerns over reciprocity and, related, burden-sharing and free-riding that are important in other areas of international cooperation, such as trade and security. Rather, we think that our findings may be reflective of an emerging global norm, to which the IPCC, civil society, and other actors have been contributing, that mitigating climate change is the “right thing to do”, even in view of commitment failures by others.

Methods

Survey procedure and respondents

The survey was fielded with Ipsos online panels in November and December 2016. Fielding the survey directly with Ipsos means that they engaged in a number of quality control steps, for instance excluding speeders or preventing duplicate responders, details of which can be found at: https://ems.ipsos-mori.com/Assets/Docs/Techniques/ESOMAR-28-Questions.pdf

Ipsos used quota sampling in order to be representative of the general population. For the USA quotas were based upon age, employment status, gender, income, and region. Participants from China were recruited online from tier I and II cities and quota sampling was used to ensure representativeness for age, employment status, gender, income, and region.

In Supplementary Figures 2 and 3 we show that these observable characteristics of individuals are balanced across treatment conditions. See Supplementary Tables 2-22 for regression tables of these estimates.

Participants provided demographic information, then received the information treatment followed by a comprehension check, prior to the conjoint experiment.

Information Provision Treatments

To identify the causal effects of information on other countries' behavior on preferences over specific climate policy attributes, we combined this conjoint experiment with an experiment where we randomly assigned participants to one of six information treatment arms where information was provided on other countries' coal consumption compared to the home country's consumption in the past few years (see Supplementary Table 2 for the distribution of treatment assignment in the sample). By using real information, we made these treatments as plausible as possible while still adhering to the non-deception principle. Supplementary Figure 1 summarises the treatment conditions, in terms of whether a country or set of countries increased or decreased coal consumption. Study participants received such information both in verbal and graphical form (see Section 7 of the Supplementary Information). The first four treatment conditions compare the consumption behavior of the respondent’s home country to that of another country or countries. These information treatments capture the possible combinations of whether a specific country is mentioned or not (T1 and T2 vs. T3 and T4), and whether the other country(ies) increased or decreased coal consumption. Treatment 5 gives respondents only their own country’s information, while the control group receives no information about both their own country and other countries’ coal consumption. These treatment conditions and their combinations allow us to assess and compare, based on realistic information, the effect of learning about one’s own and also other countries’ emissions.

Respondents were asked comprehension check questions (see Supplementary Figure 3), depending on the information treatment they received.

Conjoint Experiment

For our conjoint experiment, participants were asked to choose between randomly assigned policy proposals whose characteristics differ along a set of attributes: Cost to household per year, legal strength of agreement, decision-making process, inclusion of large emitters, number of countries included, and compensation to affected areas. Study participants were shown sets of two proposed climate agreements, side by side, where the values on specific policy attributes are manipulated and randomly assigned. Participants then choose which policy they prefer. They chose between two randomly generated climate agreements five times in total. The 3007 participants in the USA and 3000 in China thus generate information on their support levels for a total of 30000 and 30700 hypothetical climate agreements respectively (five rounds times two proposals, times the number of study participants).

Supplementary Table 1 displays the policy (climate agreement) attributes and their possible values (expressions), as used in the conjoint experiment. Section 1 of the Supplementary Information also displays the text that introduced the conjoint experiment. The climate policy attributes in our experiment reflect features that could change in salience from the viewpoint of citizens, depending on the fossil fuel consumption, and thus emissions behavior of other countries. These features can be grouped into three broad categories: financing and expenditures; country participation; and the legal nature and strength of the international agreement. To make the experiment as simple and straightforward as possible, which is essential for “ordinary citizens” being able to understand and process the information they receive, we decided to focus on coal consumption.

For financing and expenditures, we vary the cost individuals would pay in the form of increased utility bills. We also vary whether an agreement includes provisions for (domestic) government expenditures that would be targeted to regions dependent upon the coal industry. We expect that citizens will be more sensitive to cost considerations and demand higher financial support if other countries are seen to be failing to reduce emissions. For country participation, we vary the number of countries participating in the agreement, as well as whether specific "large emitter" countries are participating. We expect that providing information that other countries have increased their fossil fuel consumption could alter whether individuals think it is necessary to include more countries, that could potentially also be laggards, into the agreement. Finally, we also vary the legal and decision-making features of the agreement. We vary the extent to which the agreement is legally binding and envisages punishment for non-compliance. We also vary the decision-making process, ranging from full autonomy in deciding the level of a carbon (coal) tax to having all countries involved in the decision. Faced with information that other countries have increased fossil fuel consumption, citizens may demand more stringent legal provisions in the agreement and give greater autonomy to their home country in making the decision.

Conjoint designs are analogous to factorial designs. However as shown by (28), full randomisation overcomes the practical issue that the number of combinations in a complicated factorial can easily exceed the number of observations. Specifically, (28) show within the potential outcomes framework that with full randomisation of the values, the average treatment effect of each value is non-parametrically identified, which they label as the Average Marginal Component Effect (AMCE). For further details see (28).

Analysis

The traditional way to estimate how the information treatments affect respondents’ choice in the conjoint would be to include interaction effects between dummy variables for the attribute values and the treatment conditions. This, however, results in substantial statistical inefficiency as it introduces a number of strongly collinear variables to estimate parameters for, of which many of these effects may be zero.

To avoid this issue for identifying the treatment effects, we use a recently developed estimator, LASSOplus (29). This estimator both selects and estimates the relevant heterogenous effects to estimate, in a way that also accounts for the fact that respondents evaluate multiple conjoint profiles. By simultaneously selecting and estimating the heterogenous effects it avoids the bias that can occur by conducting several separate sub-group analyses, and also provides appropriate measures of uncertainty. To estimate the most popular proposal we revert to using a logistic regression. The analysis is conducted using R version 3.4.3, and version 1.2 of the sparsereg library, using the default settings of 200 saved posterior samples, with a burn-in of 200 samples using thinning (retaining every 10^th sample).

Comprehension Check

Results of the comprehension check are shown in Supplementary Figure 3. US respondents performed very well in terms of treatment comprehension. However we do find that Chinese respondents performed poorly when answering the question of whether China increased or decreased emissions in the time period. Specifically a majority answer that China did not increase emissions in this time period. However when examining the information about other country(ies) behavior, pass rates are significantly higher. Therefore for the important part of our treatment, we are confident that most respondents correctly understood it. Nevertheless we re-estimated the statistical models where we exclude individuals who failed the comprehension check. In this case the results do not substantively change for either country.

Supplementary Figures 4 and 5 display how the main effects vary comparing the whole sample (Intention-to-Treat effects) to the sample that only includes those who passed the comprehension check (Exclude Failures) for both USA and China. As is clear from the two figures, the preferences toward international agreements remain consistent. Supplementary Figures 6 and 7 show the estimated heterogenous effects from this approach and leads us to much the same conclusion. While excluding failures leads to two additional effects for those in the China only information group, we believe this is not a large enough change to lead us to believe that our original results are severely biased

Ethics Statement.

The surveys that generated the data for this study were a part of the ERC Advanced Grant project no. 295456 (Sources of Legitimacy in Global Environmental Governance). Ethical approval for the project as a whole was obtained at the beginning of the project (ETH Ethics Committee approval, 28 Sept. 2012, EK-2012-N-41, extended to March 2018). The Ethics Committee of ETH Zürich and the ERC Ethics Monitoring unit regularly monitored the project, with the final survey items provided on a continuous basis. The surveys for this study were fielded by Ipsos, and respondents were first informed about the nature of the study before being asked to consent. The study also followed the no deception principle, whereby only factual information was provided to respondents.