Field interventions for climate change mitigation behaviors: A second-order meta-analysis

Significance

Behavior change, such as sustainable transportation, resource conservation, and circular consumption, has the capacity to mitigate climate change. In this work, we seek to advance current knowledge by conducting a synthesis of interventions aimed to promote climate change mitigation behaviors in field-settings. Results from a second-order meta-analysis, including 10 meta-analyses and a total of 430 primary studies, show that pro-environmental behaviors increased by 2 to 12 percentage points compared to what would have been expected without treatment. Social comparison and financial approaches were the most effective tools, while information and feedback were the least effective. These results provide a comprehensive state-of-the-art summary of climate change mitigation interventions, guiding both future research and practice.

Abstract

Behavioral change is essential to mitigate climate change. To advance current knowledge, we synthesize research on interventions aiming to promote climate change mitigation behaviors in field settings. In a preregistered second-order meta-analysis, we assess the overall effect of 10 meta-analyses, incorporating a total of 430 primary studies. In addition, we assess subgroup analyses for six types of interventions, five behaviors, and three publication bias adjustments. Results showed that climate change mitigation interventions were generally effective (d_unadjusted = 0.31, 95% CI [0.30, 0.32]). A follow-up analysis using only unique primary studies, adjusted for publication bias, provides a more conservative overall estimate (d = 0.18, 95% CI [0.13, 0.24]). This translates into a mean treatment effect of 7 percentage points. Furthermore, in a subsample of adequately powered large-scale interventions (n > 9,000, k = 32), the effect was adjusted downward to approximately 2 percentage points. This discrepancy might be because large-scale interventions often target nonvoluntary participants by less direct techniques (e.g., “home energy reports”) while small-scale interventions often target voluntary participants by more direct techniques (e.g., face-to-face interactions). Subgroup analyses showed that interventions based on social comparisons or financial incentives were the most effective, while education or feedback was the least effective. These results provide a comprehensive state-of-the-art summary of climate change mitigation interventions, guiding both future research and practice.

Climate change is worsening extreme weather events, causing loss of biodiversity, and threatening human health. Without anthropogenic causes, these events are extremely unlikely (1). Human behavioral change is essential to mitigate climate change. Research on behavioral change interventions has identified various tools for mitigating climate change (e.g., refs. 2 and 3). Yet, an overarching quantitative synthesis is lacking. To assess the effectiveness of climate change mitigation interventions, we conducted a preregistered second-order meta-analysis summarizing ten meta-analyses.

Changing behaviors to mitigate climate change has attracted research across the social sciences (e.g., res. 2, 4, and 5). Past research has assessed interventions targeting behaviors such as resource conservation, food consumption, and sustainable transportation, using tools such as education, feedback, social norms, and financial incentives (e.g., refs. 6–8).This line of research has resulted in a vast number of studies summarized in meta-analyses. Past meta-analyses are, however, often restricted to specific behaviors or interventions. This second-order meta-analysis exceeds these boundaries and provides the most extensive meta-analytic summary to date, by synthesizing ten meta-analyses in a preregistered synthesis including a total of 430 primary studies.

Interventions aiming to mitigate climate change can be defined as tools designed and applied in field settings to promote voluntary changes in behaviors intended to mitigate climate change. Two aspects of this definition are worth noting. First, it is focused on “voluntary” actions, hence we excluded traditional nonvoluntary policy tools (e.g., targeting infrastructure and procedural barriers to change, see (9) and nonvoluntary systems (e.g., the school system), while including voluntary change measures such as information, education, prompts, feedback, financial incentives, social influence approaches, and nudging techniques applied in real-world settings (3). Second, the definition emphasizes “intended,” hence, interventions should aim to mitigate climate change. Therefore, interventions that did not explicitly aim to mitigate climate change were excluded from the second-order meta-analysis. Importantly, when assessing climate change mitigation, descriptions of the targeted behaviors were done on the meta-analytic level. This means that we excluded meta-analyses described as targeting for example prohealth behaviors, even if such outcomes could potentially mitigate climate change (e.g., promoting a healthy diet by reducing red meat consumption).

Climate change mitigation behavior, or proenvironmental behavior, has been defined as the “commission of acts that benefit the natural environment (e.g., recycling) and the omission of acts that harm it (e.g., avoid air travel)” (10, p. 92). Based on this definition, we included studies in the second-order meta-analysis that reported observed and self-reported behaviors explicitly defined as mitigating climate change, while excluding studies reporting attitudes, intentions, willingness to pay, and fictitious choices (Method). Included studies were summarized in the following categories: conservation (e.g., saving water or electricity), consumption (e.g., buying organic products), recycling (e.g., recycling paper or plastics), transportation (e.g., sustainable transportation alternatives), and littering. Littering is linked to emissions of greenhouse gases in at least three respects. 1) Littering constitutes a missed opportunity to reduce greenhouse emissions by recycling (11). 2) Frequently littered items, such as cigarette butts and packages, contain plastics (12–14), and these plastics release greenhouse gases when decomposed in both terrestrial and marine environments (15). 3) Marine (micro)plastics are related to greenhouse gases as these plastics have multiple negative effects on phytoplankton communities which leads to a destabilized marine system (11, see also refs. 16 and 17)

The first quantitative synthesis of climate change mitigation interventions was published in 2012, demonstrating that interventions are effective in general (g = 0.45; ref. 18). Since then, meta-analyses on specific interventions have been published, estimating the effect of, for example, feedback (8), financial incentives (7), and social norms (6). It is worth noting that these meta-analyses report noticeable variability of the overall effects, ranging from d = 0.09 (19) to g = 0.45 (18). Past meta-analyses are thus inconclusive in terms of both the overall effectiveness of interventions, and the conditional effects of specific interventions and specific proenvironmental behaviors (e.g., refs. 2, 8, and 18).

For this research, we aim to advance current knowledge on interventions for promoting mitigation interventions by conducting a second-order meta-analysis (20). The main goal was to advance the evidence from first-order meta-analyses, which often assess one specific intervention tool or one specific proenvironmental behavior, by providing an integrative synthesis of the effectiveness of mitigation interventions across different interventions and proenvironmental behaviors.

For these reasons, the current research was guided by and addressed the following three research questions:

1. How effective are climate change mitigation interventions overall in promoting pro-environmental behaviors in field settings?

Past meta-analyses have often focused on one specific intervention (e.g., financial incentives; ref. 21) or a specific proenvironmental behavior (e.g., recycling; ref. 22). Although these meta-analyses have their own merit, knowledge on the generalizability across interventions and outcome types is lacking. Therefore, the first objective of this meta-analysis was to synthesize previous meta-analytic results to provide an estimate of the overall effectiveness of climate change mitigation interventions.

2. What is the most effective mitigation intervention for changing proenvironmental behaviors in field settings?

It remains inconclusive whether certain interventions are more effective than others, because meta-analyses sometimes reached different conclusions about the effectiveness of specific interventions, such as financial incentives (21, 22) or social norms (2, 6). Moreover, some meta-analyses examined one specific type of intervention (e.g., ref. 7) or one category of behavior (e.g., ref. 23). Consequently, the second objective of this synthesis was to compare and evaluate the effectiveness of different climate change mitigation interventions.

3. Which proenvironmental behaviors are most susceptible to mitigation interventions?

Past meta-analyses have often focused on one type of behavior, such as energy consumption (23) or transportation behavior (24). When assessing different behaviors, significant variability was found, with some behaviors appearing to be more susceptible to change than others (19). Thus, the third goal of this meta-analysis was to identify, integrate, and compare all previously investigated proenvironmental behaviors.

In summary, we conducted a second-order meta-analysis to 1) assess the overall effectiveness of climate change mitigation interventions, 2) compare interventions in their effectiveness, and 3) examine their impact on different categories of proenvironmental behaviors. This second-order meta-analysis provides a comprehensive summary of climate change mitigation interventions, guiding both future research and practice.

Results

We included 10 meta-analyses from 8,881 identified studies (SI Appendix, Appendix A). These meta-analyses comprised 430 unique primary studies including 36 subgroup effects (i.e., effect sizes for specific interventions or proenvironmental behaviors).

Overall Effect.

Results from the second-order meta-analysis showed a positive overall effect (d = 0.31, 95% CI [0.30, 0.32]). Each of the 10 included meta-analyses found positive overall effect sizes for interventions to promote climate change mitigation behavior in field settings (Fig. 1). The positive direction of the overall effect indicated that, on average, interventions promoted behavioral change. To put this result into perspective, an effect size of d = 0.31 indicates that approximately 62% of the treatment group is above the mean of the control group. In other words, climate change mitigation behavior was approximately 12 percentage points higher than would have been expected without treatment (25).

Fig. 1.

Forest plot of the mean overall unadjusted effect sizes of the included meta-analyses. Note. k = number of studies, N = number of participants, dotted line = overall effect, Estimate (95% CI) = estimate and 95% CIs. *The effect size from Nisa et al. (19) is based on the restricted maximum likelihood estimator, as reported by van der Linden and Goldberg (26).

The proportion of the observed variance accounted for by the second-order sampling error was approaching zero (proportion of true variance (ProVar) < 0.001), meaning that differences in effect sizes across meta-analyses cannot be explained by sampling error alone, but are likely driven by moderating factors. An analysis of the subgroup effects yielded similar results (d = 0.30, 95% CI [0.29, 0.31], ProVar = 0.0002, see Table 1).

Table 1.

Second-order meta-analytical results

	Meta-analyses m	Meta-analytic effect sizes n	Primary studies k	Overall grand mean d	95% CI lower limit	95% CI upper limit
Overall
Averaged over meta-analyses	10	10	1,178	0.310	0.302	0.320
Averaged over subgroup effect size	10	38	1,041	0.303	0.292	0.313
Unique effect size	10	10	663	0.310	0.280	0.340
Interventions
Appeals	1	1	10	0.279	0.279	0.279
Commitment	3	3	67	0.272	0.261	0.284
Education	5	5	121	0.087	0.076	0.099
Feedback	4	4	120	0.159	0.149	0.169
Financial incentives	4	6	73	0.317	0.296	0.338
Social comparison	5	9	199	0.370	0.351	0.389
Outcomes
Conservation	6	13	404	0.254	0.239	0.270
Consumption	2	3	18	0.197	0.178	0.217
Littering	1	1	22	0.519	0.519	0.519
Recycling	4	9	103	0.273	0.236	0.309
Transportation	4	5	57	0.079	0.064	0.093

Note. All statistics in Table 1 are second-order meta-analytic estimates without publication bias adjustments. The number of unique included primary studies is lower than reported here due to study overlap, see main text for more details. 1) number of meta-analyses; 2) number of meta-analytical effect sizes; 3) number of primary studies; 4) second-order, grand mean standardized difference estimate; 5) lower limit of the 95% confidence interval (CI); 6) upper limit of the 95% CI. See Appendix for 7) observed variance across first-order mean standardized difference estimates; 8) expected second-order sampling error variance; 9) estimated true variance across first-order mean standardized difference estimates (expected sampling error variance removed).

To verify these results and assess the potential influence of overlapping meta-analyses, we extracted all accessible unique effect sizes from the meta-analyses^*. The final data included 663 unique effect sizes. Results of a random-effects meta-analysis corroborated results from the second-order meta-analysis (d = 0.31, 95% CI [0.28, 0.34], SE = 0.02, t² = 0.12, I² = 99.31, 95% Prediction interval (PI) [−0.35, 0.76]). The overall effect was further assessed with publication bias adjustment methods. First, a p-curve analysis showed a right-skewed distribution, which indicates little evidence of selective reporting. Yet, the range of P-values between 0.040 and 0.049 deviated from the expected distribution, pointing toward a few cases of selective reporting (27: www.p-curve.com, See SI Appendix, Appendix D). Second, both PET-PEESE and selection models indicated publication bias and downwardly adjusted the overall effect size (d = 0.18, 95% CI [0.13, 0.24]; 7 percentage points). Third, we ran a WAAP analysis assessing a subset of 32 studies with adequate statistical power (1−β > 0.80, n > 9,000). Results showed a drastically reduced, yet positive, overall effect (d = 0.04, 95% CI [0.03, 0.05]; 2 percentage points; see Method for details). Taken together, publication bias methods downwardly adjusted the overall effect. Still, these adjustments showed a statistically significant positive overall effect of interventions aiming to promote climate change mitigation behaviors. We encourage readers to evaluate each of these publication bias adjustments based on their respective assumptions, and from their own specific theoretical or applied interest.

Next, we tested the subgroup effects for a) type of intervention and b) category of proenvironmental behaviors, as an approach to explaining the systematic variance.

Type of intervention.

In a first subgroups analysis, we investigated the effect of different types of interventions on proenvironmental behavior, as provided by the authors of the included meta-analyses. We identified six types of interventions: appeals, commitment, feedback, education, financial incentives, and social comparison. A description of each intervention type is presented in Table 2.

Table 2.

Typology and description of mitigation interventions

Type of intervention	Description
Appeals	Appeals demand and urge people to act more sustainably by targeting their values or responsibilities. Appeals could, for example, remind people to save electricity.
Commitment	Commitment interventions are trying to motivate people to commit to sustainable behaviors. Examples are goal setting, public commitments, or implementation intentions.
Education	Education interventions aim to increase knowledge about sustainable behaviors by educating people with factual information. Examples are informational flyers or videos, statistics, practical tips, or energy labels.
Feedback	Feedback provides individuals or households with information about their own past behaviors. Interventions could, for example, provide feedback about the water or electricity consumption, or recycling behavior of a specific household.
Financial incentives	Financial incentives are financial rewards to people for acting in a sustainable way. Financial incentives include, for example, financial rewards, reimbursements, or unit pricing programs.
Social comparison	Social comparisons highlight other people’s proenvironmental behaviors or attitudes as a means to increase proenvironmental behaviors. These include modeling and social norms.

Results showed that the interventions varied in their effectiveness in changing proenvironmental behaviors. The largest effects were found for interventions using social comparison (d = 0.37, [0.35, 0.39]) and financial incentives (d = 0.32, [0.30, 0.34]). Appeals (d = 0.28^†) and commitment interventions (d = 0.27, [0.26, 0.28]) showed somewhat smaller effects. The smallest effects were found for interventions using feedback (d = 0.16, [0.15, 0.17]) or education (d = 0.09, [0.08, 0.10]). Publication bias adjustments altered some of these findings substantially. Importantly, these adjustments are confounded with, for example, sample size and number of studies (see SI Appendix, Appendix E for details). The proportion of true variance ranged from 0.003 to 0.10 across subgroups indicating a large amount of remaining systematic variance within each type of intervention.

Categories of Proenvironmental Behaviors.

We investigated the general effect of interventions on different types of proenvironmental behaviors. We classified the coded data into five broader categories: conservation, consumption, littering, recycling, and transportation behavior. Conservation included behaviors such as saving water or electricity, while consumption entailed, for example, meat consumption or food waste. Within the transportation category, we included, for example, the use of a car, public transport, walking, and biking. The last two categories, recycling and littering, were concerned with the reuse of products and the disposal of waste in the environment, respectively. Interventions targeting littering showed by far the strongest effects (d = 0.52²). Interventions to promote recycling (d = 0.27, [0.24, 0.31]), conservation (d = 0.25, [0.24, 0.27]), and consumption behaviors (d = 0.20, [0.18, 0.22]) were less effective but still statistically significant. Compared with the other types of proenvironmental behaviors, interventions targeting transportation behaviors showed the smallest effects (d = 0.08, [0.06, 0.09]).

Across subgroups, the proportion of true variance ranged from 0.002 to 0.015 for conservation, recycling, and transportation, indicating a large amount of remaining systematic variance within some but not all subgroups. However, observed variances are again small.

Discussion

We present a comprehensive synthesis of climate change mitigation interventions. Based on 10 meta-analyses, we conducted a second-order meta-analysis including 430 unique primary studies. Analyses yielded three main results: 1) overall, interventions resulted in behavioral change. Importantly, publication bias methods adjusted the overall effect downward, but did not nullify the main effect of interventions. 2) Interventions were found to be most effective in changing behaviors when based on social comparisons or financial incentives, but least effective when based on education or feedback alone. 3) We found that interventions are most effective when targeting littering behavior, and least effective when targeting sustainable transportation. We will discuss each of these main results in turn.

The second-order meta-analysis found that climate change mitigation interventions had a positive and significant effect on sustainable behaviors. After being exposed to an intervention, proenvironmental behaviors were about 12 percentage points higher than what would be expected without any intervention. These results are similar to interventions targeting health behaviors (e.g., refs. 28–30). After adjusting for publication bias, the overall effect was reduced to 7 percentage points. Still, this suggests that “soft interventions,” as examined in this second-order meta-analysis, are indeed a useful tool for mitigating climate change. We would like to emphasize that these results are based on the overall effectiveness of various interventions on a diverse set of behaviors. The effectiveness of a specific intervention is likely dependent on the extent to which an intervention matches key determinants and characteristics of the targeted behavior (e.g., refs. 3 and 31). The overall effect could thus serve as an estimated effect when knowledge about key determinants and behavioral characteristics is scarce or uncertain.

The overall effect was drastically reduced to approximately 2 percentage points when assessing a subsample of sufficiently powered studies (n > 9,000, k = 32). Although this correction was based on a relatively small subsample, these studies are highly important as they implement interventions to mitigate climate change on a large-scale. Scalability does, however, often come with the cost of weaker manipulations. Compared with small-scale interventions such as the “EcoTeam Program,” which included voluntary participation in a face-to-face interaction (32), large-scale interventions, such as the “home energy report” tend to be based on nonvoluntary participation where participants are informed about others’ energy usage, rather than engage in social interaction (e.g., ref. 4)—which will likely weaken the effect of the intervention (e.g., refs. 2 and 6). Moreover, small versus large-scale interventions based on social norms might differ in the persuasive processes or motives. Large-scale interventions are unlikely to include direct social interaction, whereas many small-scale interventions do. We suggest that motives such as mimicry and maintenance of social relationships are primarily applicable in small-scale interventions, while large-scale interventions are more likely to be based on information. Yet another difference between small- and large-scale interventions is that the latter communicates the social normative message via a third party, which might reduce the influence because people perceive that someone is trying to influence them (see refs. 33–35 for reviews). Taken together, although the WAAP-analysis suggests that large-scale interventions tend to have small average effects, these are indeed important tools as, by definition, large-scale interventions have the capacity for broader reach.

It is important to note that results of the ten meta-analyses might be exaggerated due to publication bias. If the first-order meta-analyses suffer from publication bias, it will also affect the estimate of the second-order meta-analysis. We performed several publication bias assessments for the overall effect and concluded that—although the effect size is reduced when measurable bias is taken into account—the results appear nonetheless directionally robust.

When assessing the practical implications of these findings, we focused on the effect in terms of percentage points of behavioral change; however, the climate change mitigation consequences of a specific intervention also depend on their durability and reach. That is, to what extent does the intervention induce long-term effects, and how many people are or could the intervention potentially target (36)? There might, as discussed above, be a tradeoff between effectiveness and reach, as small-scale interventions tend to use stronger interventions (e.g., ref. 2). In line with these findings, DellaVigna and Lindos (37) reported that nudge interventions were substantially less effective when implemented on a large-scale by nudging units, compared with when reported in academic journals.

The effectiveness of climate change mitigation interventions differed across subgroups. Interventions using social influence approaches or financial incentives had the strongest effects on sustainable behavior. Interventions relying on appeals or commitment approaches showed weaker yet promising effects in changing proenvironmental behavior, while feedback and education interventions demonstrated the smallest effects. We assessed publication bias on the subgroup level. In sum, the p-curve analysis indicated selective reporting for interventions based on appeals and feedback (see also the p-curve for education in SI Appendix, Appendix E). After adjusting for publication bias using selection models and WAAP, we found the following respective corrected effect sizes for appeal (d = N/A, d = 0.39), commitment (d = 0.14, d = 0.13), education (d = 0.43, d = 0.06), feedback (d = 0.14, d = 0.06), financial incentives (d = 0.34, d = 0.14), and social comparisons (d = 0.12, d = 0.06). Publication bias assessments on the subgroup level should, however, be interpreted with caution as these interventions diverge in terms of sample size. Additionally, some interventions have a higher prevalence of large-scale interventions, while others often rely on small samples, which makes comparisons more challenging. Furthermore, the subgroup analyses are limited due to low statistical power (i.e., total subgroup analysis is based on samples as small as k = 8, p-curve analysis based on samples as small as k = 6, and WAAP analysis based on samples as small as k = 5). Therefore, we encourage readers to critically evaluate the presented subgroup effects that are based on the publication bias assessments (SI Appendix, Appendix E).

The influence of social comparisons has broad scientific consensus, demonstrating that people are affected by other people’s behaviors and opinions. Research has shown that individuals conform to social norms to gain the approval of others and/or to behave appropriately (e.g., refs. 38–40). Importantly, social norms-based interventions are plausibly more influential when communicated implicitly rather than explicitly (6), when referring to a proximal reference in-group (41–43), and when it is communicated in an adversarial information environment (44). Such interventions are less influential when the message elicits psychological reactance (45). If these conditions are not considered when designing interventions, social influence interventions can be ineffective (46), or have undesirable consequences, such as boomerang effects (e.g., ref. 40) or moral licensing effects (47), (see refs. 33–35, and 48 for review).

Similar limitations apply to feedback interventions, which can be an effective tool under certain conditions. It has been indicated that interventions using real-time, direct, and frequent feedback can be effective in changing proenvironmental behavior (8, 49, 50). Feedback might be an effective tool for situations where barriers to performing the target behavior are low and the benefits of the behavior are high (3).

Financial incentives, such as cash payments, coupons, or reimbursements, showed a consistently positive effect across all 73 primary studies. For example, Khanna et al. (23) noted that financial incentives were the most effective strategy to promote energy conservation. Still, the effect seemed to depend on the size of the monetary incentive (which has also been shown in experimental studies; e.g., ref. 51) and if financial incentives match the values of participants (e.g., refs. 52 and 53). Moreover, one meta-analysis reports that financial incentives can undermine intrinsic motivation, making behavioral change instrumental (54). Thus, it may thus be detrimental to only reward people for sustainable behaviors without strengthening intrinsic motivation by, for example, commitment approaches (e.g., ref. 55). Similarly, interventions based on financial incentives might undermine potentially positive spillover effects across proenvironmental behaviors. It is important to note, however, that the overall evidence for such effects favors the conclusion that spillover is rare for behavioral outcomes (e.g., ref. 56). Finally, although all interventions come with a cost (e.g., developing and distribution feedback or information materials), financial incentives come with the additional cost of the financial rewards. We encourage future research to assess the extent to which such an approach is more or less cost-effective than, for example, a social norms-based or a commitment-based intervention. In sum, we encourage researchers and practitioners to closely attend to potential conditional effects when implementing financial incentives to promote proenvironmental behavioral change (e.g., ref. 53).

Commitment approaches, such as goal setting or implementation intentions, have been shown to reduce energy use by about 10% on average, which is consistent with our overall results (57). Commitment approaches seem to be most effective when the goals are realistic, self-set, and publicly announced (49, 57 see ref. 58 for review). According to Schultz (3), commitment approaches are suitable when barriers to sustainable action cannot be removed or reduced, but when the motivation to behave sustainably is high, since motivated people are more likely to adhere to their commitment.

Interventions seeking to solely educate people demonstrated the smallest effects. These interventions aim to improve people’s knowledge about environmental problems and how to reduce them. Knowledge, such as basic problem awareness, is likely a necessary but insufficient condition for behavioral change. Therefore, using an educative information-based approach alone is most likely an insufficient approach to change proenvironmental behaviors, especially when barriers are high, and motivation is low (3). However, for other interventions to be effective, education might be needed. For example, combining education about energy conservation behaviors with social comparisons can be an effective approach (23).

The effectiveness of climate change mitigation interventions depended on the targeted proenvironmental behavior. Littering behavior was most likely to be changed by the interventions, followed by recycling, conservation, and consumption behaviors. Transportation behaviors were less likely to change as a result of the measures tested. These results should, however, be interpreted with caution as meta-analyses are an observational method. Synthesis-generating evidence does not imply causal inferences. Although different barriers and habits are likely to make some proenvironmental behaviors more or less likely to change (59, 60), differences between behaviors might be due to the effectiveness of interventions developed and implemented to target that specific behavior. For example, the home energy report, targeting resource conservation (e.g., ref. 4), is plausibly based on a weaker manipulation than “implicit descriptive norms” targeting littering (e.g., ref. 39). Other possible confounders are the time period when the study was conducted and its sample size, which might influence effect sizes (e.g., refs. 39 and 61).

It has been argued that climate change mitigation interventions should focus more on high-impact behaviors and less on frequent low-impact behaviors (9, see also ref. 33). Consequently, researchers have identified transportation behavior and food consumption as the household behaviors with the greatest mitigation potential (62). Above all, fossil fuel car-free living, the use of electric vehicles, and fewer flights could significantly reduce greenhouse gas emissions in the transportation sector (62). Regarding food consumption, the reduction in animal products and the use of improved cooking equipment promise the greatest CO2 savings (62). Although transportation and consumption behaviors were the behaviors least likely to be changed by the included mitigation interventions in our second-order meta-analysis, focusing merely on effectiveness does not adequately reflect the climate change impact of the targeted behavior. For example, in terms of climate change mitigation, an increase of 7 percentage points in recycling is not equivalent to an increase of 7 percentage points in sustainable food consumption. Thus, the total effect of a given intervention comprises the increase in the proenvironmental behavior as well as the impact of the behavior on carbon emissions. With this lens, it becomes clear that even behaviors that are difficult to change might nonetheless have a large impact because even small changes in the behavior can have large effects on the outcome of interest (e.g., carbon emissions; refs. 36 and 62). We therefore encourage future research to develop and evaluate interventions and policies targeting behaviors with maximal climate change mitigation impact (63). This is important given that there is currently strong evidence against the proenvironmental spillover hypothesis (e.g., ref. 64). This implies that less impactful proenvironmental behaviors are unlikely to spread into more impactful behaviors, especially when interventions are based on financial- rather than social- and autonomy-supportive motives (56).

The quality of our second-order meta-analysis is limited by the quality of the first-order meta-analyses and their included primary studies. Thus, limitations from first-order meta-analyses will translate to this work. We did, however, set up eligibility criteria ensuring that all included meta-analyses used a systematic search, targeted behaviors explicitly defined as proenvironmental, used (quasi-) experimental designs, were conducted in field settings, included a control group, and were not restricted to a specific population. Following these criteria, all included meta-analyses qualified as holding sufficient standards. It should however be mentioned that we extracted categories of interventions and behaviors on the meta-analytic level. This could have weakened the internal validity because primary studies often test combined interventions or interventions that could be arbitrarily categorized on the meta-analytic level (see also ref. 31).

The majority of the first-order meta-analyses had some of their primary studies in common, resulting in a large overlap between some meta-analyses. Therefore, some primary studies were included in the second-order meta-analysis more than once. We attempted to minimize the overlap by calculating two overlap measures to be able to exclude meta-analyses with high overlap. In addition, we performed a robustness check with a more conservative cutoff (i.e., a smaller overlap), which yielded similar results to the original analysis. Finally, we extracted underlying effect sizes from all included meta-analyses, excluded all duplicates, and ran a random-effects first-order meta-analysis with only unique effect sizes. Results were highly consistent with those obtained from the second-order meta-analysis. Taken together, these results are robust both across different overlap cutoffs and compared with the results from unique effect sizes.

In order to advance the interpretation and limitations of future meta-analyses, we would like to provide the following recommendations. 1) Exhaustive statistics. To be able to assess the validity of any meta-analysis, we suggest that the following statistics should be reported for each of the included primary studies: effect size (define which), SE, number of observations for each condition, tau-squared, prediction intervals, and P-values. 2) Transparent effect size calculations. We encourage future meta-analyses to transparently report how each of the included effect sizes was calculated and, when applicable, describe which variables were used from each of the primary studies. 3) Open/raw data. Providing raw and/or open data for primary studies would enable readers and future meta-analysts to comprehend, evaluate, and reuse data. 4) Preregistrations or registered reports. When assessing the included studies (from 2012 when the first preregistration was uploaded on Open Science Framework), we found no registered reports nor any preregistrations. Only three primary studies (65–67) made their materials or/and data available. We encourage future meta-analyses to preregister their work or/and to use registered reports. 5) Pooling the data. Including the same control condition more than once risks distorting the overall meta-analytic effect due to an inflated total sample. We, therefore, encourage future meta-analyses to avoid “double-counting” control conditions. 6) Subgroup analyses. Running subgroup analysis might detect important moderators for future research, yet as these analyses are at risk of being low-powered and plausibly confounded, we encourage future researchers to interpret any such findings with caution. 7) Publication bias. Finally, we encourage future meta-analyses to use modern statistics and bias adjustments such as the p-curve analyses, selection models, and WAAP–WLS analyses.

For future research on interventions aimed to promote climate change mitigation, we provide the following recommendations. First, future research should explore the effects of interventions combining different approaches. These combined interventions might promise larger effects on proenvironmental behavior and may complement each other (e.g., providing education and financial incentives following receiving feedback). Further, this can help estimate the upper limits of the effects of these behavioral interventions by estimating whether there are larger effects with each additional factor added or diminishing returns beyond one intervention type. Second, to achieve the goals of the Paris Agreement, more research on infrequent but high-impact household behaviors is urgently needed (9). We recognize that studying these behaviors can be a challenging task because they occur infrequently, such as choosing a place to live or buying solar panels, and therefore collecting sufficient data can be a lengthy, difficult, and expensive process. Importantly, many infrequent behaviors are characterized as “efficiencies” (e.g., installing an energy efficient lightbulb), which are expected to have a longer duration than more frequent “curtailments” (e.g., turning off the lights when leaving a room) (68). Finally, we still observed a large amount of systematic variance within each subgroup of intervention or outcome types. This suggests the presence of moderating variables between the included meta-analytic effect sizes within a subgroup. Future research should investigate these moderating factors and examine how effective mitigation interventions are under different conditions, for example, analyzing barriers, key determinants, or characteristics of the outcome behavior that might moderate the effect of a specific intervention (3, 31, 59)

Conclusion

The main goal of our second-order meta-analysis was to provide a comprehensive synthesis of climate change mitigation interventions. Results showed that climate change mitigation interventions can promote sustainable behaviors. Interventions using social comparison approaches or financial incentives promised the largest effects on climate change mitigation. Results provide extensive evidence for researchers and practitioners to develop interventions targeting high-impact behaviors and implement the most effective interventions on a large scale.

Method

Preregistration and Accessibility.

This work was preregistrated on the Open Science Framework. All data and the code for the second-order meta-analysis are publicly available at https://osf.io/6dyq9/ (69).

Eligibility Criteria.

We used the following eligibility criteria for assessing the inclusion or exclusion on the meta-analytic level.n example, to determine if studies were conducted in a field setting, we assessed the eligibility criteria stating, for example, that studies were “…conducted as a naturalistic field study…” (8), “…describe field trial(s)…” (22), or “…observed behavior in a real-world setting…” (18). When sufficient information was lacking on the meta-analytic level, which was the case when assessing eligibility criterion 3 (7, 19), we assessed eligibility on the primary study level.

1. We included meta-analyses based on a systematic search. Hence, we excluded meta-evaluations/program evaluations (e.g., ref. 4 and ref. 70) and internal meta-analyses.

2. We included meta-analyses in which the dependent variables were proenvironmental behaviors as defined above (10, p. 92). We included observed or self-reported behaviors explicitly defined as proenvironmental on the meta-analytic level.

3. We included meta-analyses described as examining experiments or quasi-experiments conducted in a field setting. We excluded meta-analyses of laboratory experiments, case studies, survey data, and qualitative studies.

4. We included meta-analyses that incorporated studies with a control group or a proxy for a control group. We included meta-analyses incorporating both within- and between-subject designs and multiple measurement points.

5. We only included meta-analyses targeting the general population rather than specific subgroups (e.g., specific occupational groups).

6. We included meta-analyses that provided (either in the article, through open access sources, or in correspondence with the authors) adequate statistics for us to calculate both an effect size measure and a measure of dispersion.

Search Strategies.

We used five different search strategies: 1) database search, 2) searching reference lists of reviews, 3) mailing lists, 4) scanning conference programs, and 5) searching within scientific journals.

First, in collaboration with librarians from the University of Gothenburg, we chose to search the databases Scopus, ProQuest social science, GreenFile, and EconLit since we wanted to include as many disciplines as possible, as well as gray literature. We limited the search to titles, abstracts, and keywords. In Scopus, we restricted the search to the following research areas: Agricultural and Biological Sciences, Environmental Science, Social Sciences, Psychology, Energy, and Economics and multidisciplinary Science. The search terms were developed to match eligibility criteria in terms of methods and proenvironmental mitigation behaviors. We conducted our search on the 15th of February 2022 using the following search string:

(“Meta-analysis” OR “Research synthesis” OR “Quantitative review” OR “Meta-analytic structural equation modelling” OR “Meta-analytic SEM” OR “MASEM” OR “Meta-analytic path analysis” OR “Meta-regression” OR “Cumulative meta-analysis” OR “Mega-analysis” OR “Bayesian meta analysis” OR “Second order meta-analysis” OR “Secondary use of meta analytic data”) AND (“pro-environmental“ OR “environmental friendly“ OR “pro-environmental behavio*” OR “PEB” OR “environmental behavio*” OR “ecological behavio*” OR “sustainable behavio*” OR “green behavio*” OR “consumer behavio*” OR “climate mitigation” OR “environment* conser* change” OR “organic” OR “ecological” OR “eco label” OR “sustainable consumption” OR “sustainable transportation” OR “public transportation” OR “conserve*” OR “conservation” OR “recycling”)

Second, the electronic database search was supplemented by scanning the reference lists and citations of already-identified and eligible meta-analyses including: Abrahamse and Steg (2), Bergquist et al. (6), Nisa et al. (19), Osbaldiston and Schott (18), Mi et al. (21), Maki et al. (7), Varotto and Spagnolli (22), Lokhorst et al. (58), Karlin et al. (8), and Delmas et al. (71). Third, we searched for unpublished studies by means of a call via the Environmental Psychology mailing list (February 2022). Fourth, we reviewed the most recent proceedings of the following conferences: International Conference on Environmental Psychology, Nordic Environmental Social Science Conference, Conference on Behaviour and Energy Efficiency, Sustainability Psychology Preconference convention and the convention of American Psychological Association (APA) division 34. Fifth, we searched (February 2022) in the following journals using the term “meta-analy*”:

Journal of Environmental Psychology, Environment and Behavior, Sustainability, Frontiers in Psychology, Appetite, Journal of Environmental Education, Environmental Education Research, Transportation, Transportation Research, Transportation Policy, Energy Policy, Journal of Consumer Policy, Journal of Social Issues, Journal of Applied Social Psychology, Global Environmental Change, Psychological Bulletin, Journal of Applied Communications, Nature Sustainability, Renewable and Sustainable Energy Reviews, Nature communications, Nature Climate Change, Nature Energy, Frontiers in Communication

Data Screening and Assessing Study Overlap.

First, we excluded all hits that did not include either of the “methods” search terms (e.g., meta-analy*) in the title, abstract, or keywords. Second, all remaining hits were blindly assessed for eligibility by two researchers using criteria 2 to 5. The interrater reliability showed strong agreement (Cohen’s Kappa = 0.996). Six conflicting cases were resolved by a discussion based on the eligibility criteria. Third, for all studies not providing adequate data, the first author was contacted and asked for the missing data and/or raw data. In five cases, we could not access either of these, and therefore these studies were excluded. Fourth, we followed the recommendations by Hennessy and Johnson (72) for addressing study overlap. Initially, we calculated the corrected covered area (CCA) for the entire matrix of included primary studies (73). The CCA represents the degree of study overlap between included meta-analyses and can range from 0 to 100%. The initial CCA was 4%, which can be interpreted as a slight overlap (for the formula, see SI Appendix, Appendix B). However, the overall CCA may conceal a large overlap between two individual meta-analyses that examined, for example, the same outcome variable. Therefore, we calculated the CCA between each pair of meta-analyses sharing more than one primary article. We used the recommendations by Pieper et al. (73) and excluded studies with an overlap of over 15%. This led to the exclusion of three articles.

Importantly, the CCA has some methodological limitations, as it does not take the added value of a meta-analysis into account and is highly dependent on the number of included studies. We, therefore, computed a complementary measure of overlap, assessing the unique contribution of each meta-analysis. It represents the percentage of unique primary studies per included meta-analysis compared with all other meta-analyses. Sequentially, we excluded meta-analyses starting with the lowest percentage of uniqueness or highest degree of overlap and recalculated the uniqueness of the remaining meta-analyses until each meta-analysis contributed more than 25% of unique primary studies. This procedure led to the exclusion of another four meta-analyses. In addition, we examined how robust our results are to study overlap by limiting overlap to less than 50%. An overview of the exclusion process is given in Fig. 2.

Fig. 2.

Flow chart. Note. One meta-analysis was excluded due to “educational setting” (Zelezny, 1999 (74)), five meta-analyses were excluded due to “insufficient data” (Buckley, 2020 (75); Delmas et al., 2013 (71); Green et al., 2019 (76); Nemati and Penn, 2020 (77); and Sanguinetti et al., 2020 (78)), and seven meta-analyses were excluded due to “CCA”/”uniqueness” (Abrahamse and Steg, 2013 (2); Arnott et al., 2014 (79); Lokhorst et al., 2013 (58); Osbaldiston 2004 (80); Nisa et al., 2019 (19); Poškus et al., 2016 (81); and Scheibehenne et al., 2016 (82)).

Data Extraction.

For each meta-analysis included in this second-order meta-analysis, we coded: name of the authors, year of publication, the general intervention effect, the effects of different intervention types, the effects on different outcome types, the number of estimates per meta-analysis, the overall sample size of a meta-analysis, the type of effect size, the effect size and its CIs or SD/SE, how the outcome variables were measured (observed vs. self-reported), the included types of study designs (between-subjects vs. within-subjects), and the meta-analytic estimator used.

We extracted four different types of effect sizes from the included meta-analyses: Cohen’s d, Hedge’s g, Fisher’s z, and Pearson’s r. In a first step, we converted z and r to d allowing us to compare effects between meta-analyses (see SI Appendix, Appendix B for formulas). In a second step, we calculated the variances of the included effect sizes based on reported 95% CI. For the z-values, we first calculated the CI for r based on CI of z (SI Appendix, Appendix B). The results of Nisa et al. (19) were recalculated using the restricted maximum likelihood (REML) estimator following the criticism of the DerSimonian–Laird procedure by van der Linden and Goldberg (26). As d and g are conceptually similar and sample sizes were large, the numerical differences between both effect size measures should be minimal. Thus, we included both d and g in our main analysis.

Data Synthesis.

First-order meta-analyses reduce sampling error by integrating effect sizes across different primary studies. However, the amount of included primary studies is limited and thus, the sampling error will never reach zero (83). The goal of a second-order meta-analysis is to calculate this remaining sampling error, the so-called second-order sampling error. A second-order meta-analysis synthesizes the effect sizes of conceptually similar first-order meta-analyses instead of primary studies. As second-order meta-analyses compute how much of the variance between effect sizes of first-order meta-analyses is due to sampling error or systematic variance, they can reveal the existence of possible moderators between meta-analyses (20).

We calculated a second-order meta-analysis of bare-bones first-order meta-analyses following the approach by Schmidt and Oh (20). In bare-bones meta-analyses, the sampling error is the only artifact that is corrected for within the first-order meta-analyses (83). Thus, the estimate of the population variance of the uncorrected mean differences equals the weighted variance of the mean differences minus the expected second-order sampling error across first-order meta-analyses (20).

$${\widehat{\sigma }}_{{\overline{\rho }}_d}^2=S_{\widehat{\overline{d}}}^2-E\left(S_{e_{{\widehat{\overline{d}}}_i}}^2\right).$$

Our first aim was to compute the grand mean of all included first-order meta-analyses:

$$\begin{array}{c}\widehat{\overline{\overline{d}}}={\displaystyle \sum _1^m{w}_i{\widehat{\overline{d}}}_i/\sum _1^m{w}_i \mathrm{with}} w_i={\left({\displaystyle \frac{S_{d_i}^2}{k_i}}\right)}^{-1},\hfill \end{array}$$

where $w_i$ is the weight and ${\widehat{\overline{d}}}_i$ is the mean difference of meta-analysis $i$ (20).

Our second aim was to model the variance between the included first-order meta-analyses that is attributable to second-order sampling error, which can be calculated as:

$$E\left(S_{e_{\widehat{\overline{d}}}{}_i}^2\right)=\frac{m}{{{\displaystyle \sum }}_1^m{w}_i}.$$

The second-order sampling error can be estimated by dividing the number of meta-analyses by the sum of the weight applied to each meta-analysis. Using these estimates, the proportion of the observed variance accounted for by the second-order sampling error was calculated by dividing the expected second-order sampling error by the weighted variance of the mean differences (20):

$$ProportionVar=\frac{E\left(S_{e_{\widehat{\overline{d}}}{}_i}^2\right)}{S_{\widehat{\overline{d}}}^2}.$$

If ProportionVar is relatively small or close to zero, it implies that there are underlying mechanisms or moderating factors explaining the differences between the results of the meta-analyses (20). If this value is close to 1, it indicates that almost all of the variance between meta-analyses is due to second-order sampling error. Our third aim was to investigate the moderator effects of mitigation intervention type and outcome type if ProportionVar indicated remaining “true” variance. For this aim, we categorized intervention and outcome type into subgroups based on the included meta-analysis and our definition of proenvironmental behavior. For all analyses we used the R Studio package “psychmeta” and its function “ma_d_order2” (84). We used the “metafor” (85) package for creating the funnel and forest plots.

Publication Bias and Robustness.

An important limitation of meta-analyses is publication bias, indicating an association between the publication status of a manuscript and the significance or magnitude of the effect found (86). In a second-order meta-analysis, publication bias might occur at two different levels given that both primary studies (first level) and meta-analyses (second level) reporting null effects are less likely to be published. To detect and reduce publication bias, we used a triangulation approach (86). First, we tried to minimize publication bias on the second-order level by searching for gray literature (i.e., theses and dissertations) in ProQuest Dissertations and Theses Global: Social Sciences. In addition, we sent out a call on the “Environmental Psychology Mailing List” searching for unpublished meta-analysis on interventions to promote proenvironmental behavior comparing one or more treatment groups with a control group. Second, we assessed and summarized the reported publication bias analyses within each first-order meta-analysis addressing first-order publication bias (overview in SI Appendix, Appendix C). Eight meta-analyses assessed small-study bias. Out of these, six meta-analyses found indications of at least moderate publication bias. Third, we used various strategies to statistically assess the potential influence of publication bias. We used a funnel plot and Egger’s test including all 38 subgroup effects (consisting of 36 subgroup effects and two mean effect sizes from meta-analyses not reporting subgroup effects) to assess asymmetry and small-study bias. The funnel plot showed only weak indications of asymmetry (Egger’s test, z = 1.64, p = 0.10, See Fig. 3). The observation with value d = 1.4 as can be seen in Table 2, shows the social modeling subgroups from Varrotto and Spagnolli (22). In controlling for the potential influence of this observation, we ran two separate subgroup analyses for the social comparison subgroups. Results showed that removing this observation leads to a minor adjustment (d = −0.003). We ran two separate second-order meta-analyses using nonadjusted and selection models adjusted unique overall effect sizes from eight meta-analyses^‡. Results showed that the overall effect decreased from d = 0.33, 95% CI [0.32, 0.34] to d = 0.24, 95% CI [0.23, 0.25] when adjusting for publication bias. We assessed publication bias for all accessible unique effect sizes reported in the ten meta-analyses. When summarizing these unique effect sizes, we excluded studies with insufficient information to calculate effect sizes or SEs. Furthermore, to include only unique studies, we excluded duplicates based on the following criteria: 1) exclude studies coding the control condition multiple times to decrease the inflation of n and 2) include studies with the smallest of multiple effect sizes to result in a conservative estimate of the overall effect. First, a p-curve analysis indicated little evidence of selective reporting (27: www.p-curve.com; See SI Appendix, Appendix D). Second, results for both the precision-effect test and precision-effect estimate with standard errors (PET-PEESE) [t (663) = 15.70, p < 0.001] and selection models [χ² (2) = 76.05, p < 0.001] indicated significant publication bias. The publication bias adjustment using selection models in Jeffreys's Amazing Statistics Program (JASP) (P-value cutoffs: 0.05, 0.10, expecting a positively directed effect size using a one-sided selection) substantially decreased the overall effect size to d = 0.18, 95% CI [0.13, 0.24]. Third, we also assessed publication bias by analyzing the adequately powered (>0.80) studies using weighted average of the adequate powered weighted least squares (WAAP-WLS) models in JASP. Based on 32 adequately powered studies, the WAAP model provided a severely downward adjusted, but still statistically significant, overall effect (d = 0.04, 95% CI [0.03, 0.05]). Taken together, we concluded that there are clear indications of publication bias in the literature. Each of our attempts to correct for publication bias resulted in downward adjustments, still statistically significant overall effects: d = 0.04, 0.18, and 0.24. We encourage readers to evaluate each of these publication bias adjustments from their own specific theoretical or applied interest.

Fig. 3.

Funnel plot of subgroup effects.

To examine study overlap, we conducted two second-order meta-analyses, one with 25% overlap cutoff, and another with 50%. This implies that, meta-analyses had to contribute more than 25% or 50% of unique studies, or allowing 75% or 50% overlap with other meta-analyses. The conservative overlap resulted in the exclusion of three additional meta-analyses, leaving a total of seven meta-analyses with 357 unique primary studies. Results showed that the second-order meta-analysis was robust across the more and less restricted cutoffs, respectively (d = 0.32, [0.31, 0.33] versus d = 0.31, 95% CI [0.30, 0.32]). We chose the less restricted cutoff for our main analysis as it included more unique primary studies.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Pre-print, code data have been deposited in Open Science Framework (https://osf.io/6dyq9/) (69).