Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon

Significance

There are efforts to integrate the reduced carbon emissions from avoided deforestation claimed by voluntary REDD+ projects into national greenhouse gas emission inventories. This requires careful consideration of whether and how much of the reduced carbon emissions can be attributed to projects. However, credible evidence on the effectiveness of such voluntary activities is limited. We adopted the quasi-experimental synthetic control method to examine the causal effects of 12 voluntary REDD+ projects in the Brazilian Amazon. We compared these ex-post estimates of impacts with the reductions in forest loss claimed by those projects based on ex-ante baselines. Results suggest that the accepted methodologies for quantifying carbon credits overstate impacts on avoided deforestation and climate change mitigation.

Abstract

Reducing emissions from deforestation and forest degradation (REDD+) has gained international attention over the past decade, as manifested in both United Nations policy discussions and hundreds of voluntary projects launched to earn carbon-offset credits. There are ongoing discussions about whether and how projects should be integrated into national climate change mitigation efforts under the Paris Agreement. One consideration is whether these projects have generated additional impacts over and above national policies and other measures. To help inform these discussions, we compare the crediting baselines established ex-ante by voluntary REDD+ projects in the Brazilian Amazon to counterfactuals constructed ex-post based on the quasi-experimental synthetic control method. We find that the crediting baselines assume consistently higher deforestation than counterfactual forest loss in synthetic control sites. This gap is partially due to decreased deforestation in the Brazilian Amazon during the early implementation phase of the REDD+ projects considered here. This suggests that forest carbon finance must strike a balance between controlling conservation investment risk and ensuring the environmental integrity of carbon emission offsets. Relatedly, our results point to the need to better align project- and national-level carbon accounting.

Concerns over global warming have led both the public and private sectors to promote climate change mitigation through the reduction of carbon (CO₂) emissions from deforestation and forest degradation in tropical countries—a concept known as REDD+ (1). This strategy gained international attention after 2005 as a voluntary, performance-based payment mechanism for reduced carbon emissions (2). While the regulations and capacity for national REDD+ programs are still under development in many countries, hundreds of voluntary, subnational REDD+ projects are operational worldwide (3). These projects intend to preserve forests through a variety of activities, e.g., improved monitoring and control, promotion of sustainable land uses, and engagement of local communities (4), either as proof of concept or to profit from the commercialization of “carbon-offset credits” (i.e., Mg CO₂ removed from or not emitted to the atmosphere) in a variety of markets. While these markets do not provide the level of funding originally envisioned for national REDD+ programs, they are substantial: In 2018 alone, the volume of carbon offsets traded totaled 98.4 million Mg CO₂, with a market value of US$295.7 million; a third of those credits (30.5 million Mg CO₂) were generated by REDD+ projects (5). The Paris Agreement has raised thorny questions about how the carbon emission reductions claimed by these projects relate to nationally determined contributions (NDCs) and national greenhouse gas (GHG) emission inventories reported to the United Nations Framework Convention on Climate Change (6–8).

Carbon credits from REDD+ [at both the project and national levels (1)] are issued based on performance, as defined by the comparison of realized forest cover to a baseline scenario constructed by projecting the forest cover expected in the absence of REDD+ (9). These baseline scenarios typically assume a continuation of historical deforestation trends (10), and thus eventually become unrealistic counterfactuals as the regional economic and political context change. Notably, these types of changes were observed in the Brazilian Amazon during 2004–2012, a period of sharply declining rates of forest loss (11), and also during 2019, when deforestation soared again (12) (Fig. 1). Consequently, credits for reduced deforestation (or lack thereof) claimed by voluntary REDD+ projects in the Brazilian Amazon may have been artifacts of external factors rather than REDD+ activities. Furthermore, critics of voluntary REDD+ projects have raised concerns that deforestation baselines might be intentionally inflated by profiteers seeking to financially benefit from the commercialization of superfluous credits, or “hot air” (13–15). In addition to the direct cost of not effectively offsetting GHG emissions, the excess credits generated by these projects impose an indirect cost on legitimate climate change mitigation efforts by undercutting the price of their credits.

Fig. 1.

Annual deforestation in the Brazilian Amazon from PRODES data (bars). The blue bars indicate voluntary REDD+ project start dates. The red lines illustrate 10-y deforestation averages prior to project implementation, commonly adopted as projects’ deforestation baselines.

Early efforts to address these concerns included the establishment of standards and registries for voluntary carbon-offset projects. These standards were designed to ensure the environmental integrity of carbon offsets by requiring projects to use approved carbon-accounting methodologies for establishing deforestation baselines, monitoring, and reporting, all subject to third-party audits. Among those, the verified carbon standard (VCS) (16) has certified the greatest number of voluntary REDD+ projects worldwide (5).

Despite the growing literature on local REDD+ interventions, there have only been a few evaluations of their impacts on carbon emissions using rigorous, counterfactual-based methods (17–19). This study systematically compares deforestation baselines established ex-ante with counterfactual estimates of deforestation constructed ex-post. We employ the synthetic control method to construct deforestation counterfactuals and assess the reductions in forest loss that can be attributed to voluntary REDD+ projects (20–22). We apply this method to all VCS-certified REDD+ projects for unplanned deforestation implemented in the Brazilian Amazon in the last decade (2008–2017; Fig. 2 and SI Appendix, Table S1). We focus on this region for several reasons: its global relevance for conservation and REDD+; the ongoing discussions in Brazil about “nesting” voluntary projects into a national REDD+ program (6–8); and the recent availability of a spatially explicit cadastral database (23) that allows us to define a pool of rural properties similar to the REDD+ project areas. We construct synthetic controls from donor pools of properties based on weighted combinations of accessibility and biophysical characteristics that result in the best matches of historical deforestation trends. Unlike the typical approach to crediting baselines, we then construct counterfactual deforestation scenarios based on the actual deforestation observed in those synthetic controls during the period when the REDD+ projects were operational. We evaluate whether the REDD+ projects caused additional reductions in deforestation compared to the counterfactual deforestation as represented by the synthetic controls (i.e., REDD+ additionality) and assess the robustness of our results with placebo tests (22). We also examine trends in forest loss in buffer zones around the REDD+ project areas after project implementation to assess the plausibility that any apparent reductions in deforestation may have been displaced instead (24). Finally, we contrast our counterfactuals to the crediting baselines adopted by the voluntary projects.

Fig. 2.

VCS-certified REDD+ projects established during 2008–2017 in the Brazilian Amazon forest biome.

Results

Before assessing the impacts of the REDD+ projects, we explored whether the synthetic controls can accurately replicate deforestation trends in the project areas without REDD+. This “proof of concept” was implemented by dividing the pretreatment period (i.e., before project implementation) into “training” and “testing” periods. We found that the synthetic control method was able to replicate pretreatment deforestation trends reasonably well in 10 of the 12 synthetic controls (SI Appendix, Fig. S2). Our findings for the other two projects (i.e., Jari/Amapá and Suruí) must be interpreted with particular caution.

Deforestation in the REDD+ Areas.

Overall, we find no significant evidence that voluntary REDD+ projects in the Brazilian Amazon have mitigated forest loss. Deforestation is consistently lower in the REDD+ project site than in the synthetic control in only four of the projects (Fig. 3 and SI Appendix, Fig. S3), and this difference is only outside the confidence interval around zero established by the placebo tests in one project (Maísa; Fig. 4 and SI Appendix, Fig. S4). The only two REDD+ projects from our sample that were implemented in protected areas, i.e., Suruí and Rio Preto-Jacundá, experienced among the largest cumulative losses of forest cover after REDD+ implementation, along with Jari/Amapá (Fig. 3). This is partly a function of their large project areas and the widespread forest fires that occurred in those protected areas in 2010–2011 and 2015, respectively (see SI Appendix for details). For Rio Preto-Jacundá, we find much higher deforestation than in its synthetic control (which is the same order of magnitude in size); specifically, the differences between deforestation (both cumulative and annual) in the Rio Preto-Jacundá area and its synthetic controls were substantially greater than the differences between deforestation in the placebos and their synthetic controls (Fig. 4 and SI Appendix, Fig. S4).

Fig. 3.

Cumulative post-2000 deforestation in Amazonian areas with REDD+ projects (red) versus synthetic controls (blue). The dashed black lines are the project start dates.

Fig. 4.

Placebo tests: cumulative deforestation in REDD+ project areas minus deforestation in their respective synthetic controls (red), and placebos minus their respective synthetic controls (blue dots). The dashed black lines are the project start dates (assumed the same for placebos). The shaded blue areas represent 99% confidence intervals around the mean of the placebos. The number of placebos varies by project based on whether synthetic controls with low MSPE could be constructed for the placebo tests.

Across all projects, we find substantial differences between the deforestation baseline scenarios adopted ex-ante by the REDD+ projects and the observed forest loss (ex-post) in the synthetic controls (Fig. 5 and SI Appendix, Fig. S5). The Suruí project, implemented in an indigenous territory, is the only case where the synthetic control deforestation exceeded the baseline deforestation adopted by the project proponents. This may reflect the fact that the baseline for Suruí was developed based on a participatory, system dynamics model (25), as opposed to the assumptions based on historical deforestation trends adopted by all other projects (see SI Appendix for details).

Fig. 5.

Cumulative deforestation from the baseline scenarios adopted by the REDD+ projects (orange) versus observed cumulative deforestation in the synthetic controls (blue). The dashed black lines are the project start dates.

Carbon Offset Implications.

Credits from the voluntary REDD+ projects are generally issued after a third-party audit (i.e., verification) every 1 to 5 y. These credits are based on the estimated carbon-emission reductions from the avoided deforestation brought about by the projects, calculated as the difference between the carbon emissions under the baseline scenario minus the observed emissions from the project area and leakage.

According to the projects’ ex-ante estimates, up to 24.8 million carbon offsets could potentially have been generated by the REDD+ interventions by 2017 (Fig. 5 and SI Appendix, Table S1). According to the VCS database, only 5.4 million tradable credits from these projects have been certified and made available to offset GHG emissions from private and public sources by that year (SI Appendix, Table S1) (26). Using the synthetic control method to estimate REDD+ counterfactuals, we find no systematic evidence that the certified carbon offsets claimed by the voluntary projects in our sample (with the exception of Maísa) are associated with additional reductions in deforestation in the REDD+ areas above and beyond the background reduction in deforestation achieved in the Brazilian Amazon over the same period (11). Even for the Maísa case, our results suggest that nearly 40% of the 50,000 tradable carbon offsets issued by the project by 2017 (SI Appendix, Table S1) may not be genuinely additional (Fig. 5).

Leakage.

If REDD+ implementation mitigates forest loss in project areas by effectively excluding deforestation agents, it could displace, and hence increase, deforestation next to the project areas. Shifts in deforestation after project start in 10-km buffer zones surrounding the REDD+ projects suggest that such leakage effects could have occurred in three cases (i.e., Maísa, Florestal Santa Maria, and Manoa; SI Appendix, Fig. S6). Furthermore, leakage presupposes a direct conservation impact, and all three of the projects exhibited lower deforestation than their synthetic controls, although this estimated effect of REDD+ is only larger than the placebo tests in the Maísa project (Fig. 4 and SI Appendix, Fig. S4). It is also worth noting that while deforestation in the buffer zones of these three projects rose between the project start dates and 2017, postintervention rates were still lower on average than in the pre-REDD+ period.

Discussion

Our findings partially support early skepticism about the contribution of voluntary REDD+ projects to climate change mitigation (15, 27). In particular, they raise questions about the environmental integrity of offsets calculated using deforestation counterfactuals based on the continuation of historical trends (e.g., Fig. 1). In all projects that established crediting baselines using historical trends, we find that the crediting baselines significantly overstate deforestation in comparison to the counterfactual estimates based on synthetic controls. This pattern reflects the confounding effect created by Brazil’s post-2004 efforts to control Amazonian deforestation that were uniquely successful (11, 28, 29). If carbon credits are expected to reflect changes in emissions caused by REDD+, then using historical baselines leads to excess carbon credits for projects when deforestation at the regional level drops below the historical baseline. The opposite happens when unanticipated forest threats, such as fires, emerge at the regional scale.

In contrast, the synthetic control methodology uses historical trends to identify appropriate weighted combinations of comparison areas but then constructs the counterfactual based on the observed deforestation in those areas. These counterfactuals thus incorporate the effects of contemporaneous drivers of deforestation, including agricultural commodity prices, currency exchange rates, and environmental regulations (28–30). As such, the synthetic control method is less prone to incorrectly attribute changes in deforestation to REDD+.

We note some caveats on our analysis. First, we base our evaluation on the project boundaries defined by the polygons available from the VCS project database, which are somewhat larger than the areas officially reported by project proponents (SI Appendix, Table S2). Most of those polygons correspond to Amazonian rural properties registered in the Brazilian Rural Environmental Registry (CAR), whose owners are legally entitled to clear up to 20% of their forest area. Second, our synthetic controls do not perfectly match the REDD+ project areas in terms of size, accessibility, and biophysical characteristics. In particular, the synthetic control for Agrocortex is only 61% the size of the project area (SI Appendix, Tables A1 and A2). While historical deforestation is similar in the synthetic controls and project areas, clearly there is future potential for more deforestation in the larger project areas than in their smaller synthetic controls. Third, the construction of our synthetic controls may not have included all relevant structural determinants of deforestation. Last, the period of analysis may not have been long enough to observe significant REDD+ impacts in some cases.

Despite these caveats, the weight of the evidence suggests that these projects caused less reduction in deforestation than claimed (Fig. 5 and SI Appendix, Fig. S5) and that few projects actually achieved emission reductions. Suspicion about the environmental integrity of carbon offsets is not restricted to REDD+ or voluntary interventions. A series of reports on other market-based initiatives for climate change mitigation, i.e., the Joint Implementation (JI) and the Clean Development Mechanism (CDM) of the Kyoto Protocol, also raised concerns about the true climatic contributions from certified carbon offsets. These reports suggest that about three-quarters of JI credits are unlikely to represent additional emission reductions (31) and that 73% of the potential 2013–2020 CDM credits have a low likelihood of environmental integrity (in contrast to 7% with high likelihood) (32).

The projects that we evaluated may have had little additional impact because they did not adopt the most effective actions to achieve their REDD+ objectives, perhaps because of uncertainties about the future availability of funds or concerns about unfairly raising local expectations of carbon payments. Hence, our results do not imply that voluntary REDD+ projects cannot achieve their objectives if designed and implemented effectively. There is both quasi-experimental and experimental evidence that conditional payments for environmental services (PES) can effectively reduce deforestation (3, 33), and recent literature suggests that REDD+ implemented through well-designed conditional PES can deliver positive conservation outcomes (34–36).

Another possible explanation for the lack of impact is difficulty with the on-the-ground implementation and execution of activities envisioned by project proponents (37, 38). One example is the Suruí project, which attracted international attention as one of the first voluntary REDD+ interventions implemented in an indigenous territory (4). The project aimed to use the financial revenues from carbon sales to promote sustainable land-use practices in the Suruí territory but was not able to prevent the illegal invasion of loggers and miners.

A third possible explanation for underperformance relates to challenges with the commercialization of carbon offsets and correspondingly limited revenues available to implement project activities (39). One way that voluntary REDD+ projects overcome that challenge is by claiming “retroactive credits” (40). Often, projects that are certified in a given year claim to have started much earlier (SI Appendix, Table S1). As a result, those projects are eligible to issue large amounts of carbon offsets at the time of certification, retroactively corresponding to the period between the certification and the project start date. This can help to fund project start-ups, but it also implies that projects have not actually had access to carbon revenues during their early years of operation. Carbon crediting rules may thus partially explain why we find limited evidence for avoided deforestation.

Our results emphasize the need to reassess approaches to measuring project additionality. While ex-post counterfactual methods such as illustrated here would ensure a high level of environmental integrity, they would introduce substantial uncertainty about the credits that can be obtained from a given reduction in deforestation in project areas. An alternative approach often suggested in the literature is to require projects to adopt national or subnational (jurisdictional) baselines that are predefined, and periodically updated, by the government (6, 7, 41), as well as default carbon-stock values or a common carbon-density map (42). Imposing one common baseline would have the benefits of facilitating the inclusion of carbon emission reductions claimed by decentralized initiatives into national GHG emission inventories, ensuring consistency in the treatment of leakages, and avoiding double-counting reductions (6, 8, 43), while still offering relative certainty about carbon credits conditional on project performance. However, national and subnational baselines are typically based on historical data and thus are not any more likely to capture contemporaneous deforestation drivers and their dynamism [although it is also possible to apply the synthetic control method to nations (44)]. Thus, they do not address the main problem identified by our analysis: the limitations of historical data for baseline development.

Periodic baseline updates based on recent deforestation trends could help mitigate the influence of factors external to voluntary REDD+ projects on the carbon credits that they claim. In fact, current VCS rules already require projects to revise their baselines every 10 y (16). Our results suggest that this interval should be shorter. Baseline updates could be based on control areas that share similar characteristics as the REDD+ projects, as demonstrated in this study with the construction of the synthetic controls. In addition, coupled human–natural system models, such as was used in the Suruí case, can be used to explore alternative baseline scenarios and quantify the potential downside risks involved in conservation investments under dynamic patterns of land-use change, although at increased project development costs (25). These models could also shed light on the potential impacts of REDD+ on local livelihoods and biodiversity (45, 46), which we do not consider here but recognize as fundamentally important.

We do provide empirical evidence for a phenomenon that was anticipated in the early policy debate over REDD+ (47), i.e., de facto additionality of REDD+ projects depends on both project implementation and national circumstances. Carbon finance and crediting systems must safeguard against both hot air from overstated claims of carbon additionality and excessive risks to private conservation investments associated with desirable government action to combat deforestation, as observed in Brazil from 2005 to 2012.

Materials and Methods

We examined the impacts of 12 voluntary REDD+ projects implemented in the Brazilian Amazon since 2008 and certified under the VCS before May 2019 to curb local unplanned deforestation (Fig. 2 and SI Appendix, Tables S1 and S2). Project areas were defined by the geospatial polygons reported by the project proponents and available from the VCS project database. Ten of the 12 projects were implemented in privately owned properties, whereas the other two, Suruí and Rio Preto-Jacundá , were implemented in an indigenous territory and a sustainable-use reserve, respectively. Following VCS-approved carbon-accounting methodologies, historical deforestation rates were the basis of all project deforestation baselines with the exception of the Suruí project (e.g., Fig. 1). In the latter, baseline deforestation rates were informed by a participatory, and community-specific, system dynamics model (25).

Rigorous impact evaluations rely on the establishment of credible counterfactuals for what would have happened in the absence of an intervention (48, 49), which are unobservable. We construct “synthetic controls” to serve as counterfactuals for the REDD+ project areas (20, 50). We adopted this approach, as opposed to more traditional methods from the impact evaluation literature (e.g., difference-in-differences estimator), because of our small number of treated units and likely heterogeneity of the treatment across them (49, 51, 52). Synthetic controls were constructed as a weighted average of selected donor units through a nested optimization procedure that minimizes the differences in pretreatment characteristics between the project and the control, with characteristics weighted such that the resulting weighted average outcome of the selected donor units most closely matches the pretreatment outcome in the treated unit (21, 22). Specifically, the iterative procedure minimizes the mean squared prediction error (MSPE) of the outcome, or the sum of squared residuals between the treated unit and the synthetic control, over the pretreatment period (50).

Two sets of synthetic controls were constructed as a weighted combination of areas selected from “donor pools” (20, 50) composed of Amazonian properties registered in the CAR database (23) that do not overlap with project areas and that had ≥90% forest cover in the first year of the analysis. In the first set, we used cumulative deforestation as the optimization outcome, whereas the second set was based on annual deforestation. We note that the optimization algorithm selected different donors for the synthetic controls for each outcome, which allows us to use the second set as a robustness check. Donor pools were preferably based on properties from the same state as the REDD+ project and within ±25% the size of the project area. Whenever the resulting synthetic controls had substantially different land areas or pretreatment annual and cumulative deforestation (i.e., before project implementation), the donor pools were expanded to all properties in the Amazon biome (see SI Appendix for details). Last, for the cases of persistent unbalanced synthetic controls, donor pools were expanded to properties with ±50% the size of the project area. Synthetic controls for the REDD+ projects implemented in a sustainable-use reserve (i.e., Rio Preto-Jacundá) and an indigenous territory (i.e., Suruí) were constructed based on donor pools composed of other sustainable-use reserves and indigenous territories, respectively.

The spatial covariates structurally related to deforestation (30) used for the construction of the synthetic controls were obtained from official maps produced by government agencies in Brazil (SI Appendix, Fig. S7 and Table S4). The covariates represent 1) property size, 2) initial forest cover, 3) slope, 4) soil quality, and distances from 5) state capitals, 6) towns, 7) federal highways, and 8) local roads, as well as the proportion of 9) primary and 10) secondary forest, 11) pastureland, 12) agriculture, and 13) urban areas in 2000, 2004, 2008, and 2012 (for projects implemented after 2012) within 10-km buffer zones of the project and potential donor areas. In accordance with the previous literature (21, 50), we also used the pretreatment annual and cumulative deforestation rates to inform the construction of the two sets of synthetic controls. Temporal land-use information in the buffer zones was obtained from the TerraClass dataset produced by Brazil’s National Institute for Space Research. Annual deforestation data for the 2001–2017 period were processed from the MapBiomas land-use/cover dataset, version 3.1, for the Brazilian Amazon biome (Fig. 2 and SI Appendix, Fig. S1).

While the construction of our synthetic controls was based on all information available from 2001 to the project start year (i.e., pretreatment period), we conducted a separate analysis in which a different set of synthetic controls were constructed based on data constrained to the first half of the pretreatment period (i.e., training period), so they could be tested against the second half (i.e., testing period; SI Appendix, Fig. S2). We evaluated the outcome of this analysis both visually and by comparing training and testing MSPEs (SI Appendix, Table S3). This proof of concept differs from standard model-validation practices because the donors selected as synthetic controls based on the first half of the pretreatment periods do not necessarily match the final set of donors when the full pretreatment period is used.

We examined the robustness of our findings with a series of placebo tests, in which we create synthetic controls for all CAR polygons in the donor pool (i.e., not subject to REDD+ activities) and compute the difference in both annual and cumulative deforestation between each placebo and its synthetic control (Fig. 4 and SI Appendix, Fig. S4). Because placebo areas are not exposed to REDD+, any differences in forest loss between placebos and their synthetic controls are statistical “noise.” In order to increase the number of placebo tests, we use the expanded placebo donor pools of all Amazonian properties within ±50% the project size. In accordance with the previous literature (22), we discarded placebo tests with pretreatment MSPE five times higher than the pretreatment MSPE of the REDD+ polygon. We used the gaps in deforestation between the placebos and their respective synthetic controls to create 99% confidence intervals around the mean placebo effect estimate, which is approximately zero in all cases. Analyses were conducted with the Synth package (version 1.1) available for R software (version 3.6.0) (50). Last, we computed the annual deforestation in 10-km buffer zones surrounding the project areas as an indicator of possible leakage effects (24), i.e., because increasing deforestation could reflect the displacement of deforestation due to the REDD+ activities.

Data Availability.

All study data are included in the article and SI Appendix.