Revisiting the benefits of active approaches for restoring damaged ecosystems. A Comment on Jones HP et al. 2018 Restoration and repair of Earth's damaged ecosystems

Jones et al. [1] (hereafter JEA) performed a meta-analysis to assess contributions of ecological restoration to recovery of biodiversity and ecosystem processes, a potentially valuable contribution to restoration science. Further, by publishing their data [2] and code (https://github.com/DocHPJones/Restoration_Database, accessed 7 May 2018), they enabled others to leverage their efforts. Briefly, JEA reported that restored ecosystems were ‘progressing towards recovery’ but ‘rarely recover completely’. They additionally tested whether actively restoring through ongoing management yielded better results than passively restoring by ending an underlying disturbance; they found no added benefits of active restoration and concluded that ‘passive recovery should be considered as a first option’. Examining their work more closely, we identified issues that call into question these findings regarding the rarity of complete recovery and lack of added benefits of active restoration.

Here, we illustrate a fundamental flaw in one of JEA's two measures of restoration progress (recovery completeness) and strong sensitivity to outlier removal in their second measure (recovery rate). Reanalysing their data, restorations often were measured to have exceeded goals and active restoration was consistently associated with faster recovery. However, our reanalysis and the original paper should be interpreted in the light of further issues with JEA's classification of passive versus active restoration, which we followed, and limitations of the underlying data, which come primarily from observational studies wherein restoration approach is likely to be confounded with other factors [3]. We close by discussing the promise and perils of meta-analysis and ways to facilitate restoration becoming more efficient and effective.

JEA reported that ‘ecosystems rarely recovered to reference conditions' based on recovery completeness (ln[End/Goal]), which was intended to indicate whether an attribute was ‘completely recovered’, ‘below complete recovery’ or ‘more than fully recovered’ (0, less than 0 and greater than 0, respectively; [1], electronic supplementary material, methods). As JEA note, this metric is complicated by the fact that restoration can seek to increase or decrease an attribute. Thus, they sought to reverse sign when responses ‘increased under disturbance’. Logically, this should be based on the relationship between disturbed and reference conditions (i.e. Start and Goal). However, JEA used the relationship between End and Goal (H. P. Jones 2018, personal communication). The consequences can be seen in paired examples from JEA's dataset with shared Ends and Goals but different Starts (figure 1a): highly divergent outcomes cannot be distinguished by recovery completeness. These examples reflect a broader structural problem. By definition, progress cannot be determined without knowing the starting condition and desired direction of change (figure 1b; electronic supplementary material, supporting code 1.1–1.3). For cases with complete Start/End/Goal data, 25% exceeded goals, but 99% of these were measured as incomplete recovery due to sign reversal. Cases JEA did identify as exceeding goals were predominantly misclassifications—artefacts of recovery completeness being positive when End and Goal were both negative (figure 1a). In addition, recovery completeness lumps restoration setbacks—no change (4% of cases) or deterioration (23%)—with improvements short of full recovery (figure 1b). In sum, recovery completeness confounded the diverse outcomes of restoration, making it unreliable as an effect-size metric [4].

Figure 1.

(a) Recovery completeness's inability to distinguish different restoration outcomes is illustrated with paired examples from JEA's data with equal Ends and Goals but different Starts (i) and positive values arising as artefacts of negative End and Goal values (ii). (b) Counts of restoration outcomes interpreted using Start (appropriate baseline) versus End. (c) Density plot showing distribution of recovery rates. Grey-shaded areas outside dotted lines were excluded from JEA's analysis due to errant outlier removal; dashed lines indicate intended thresholds. (Online version in colour.)

JEA also reported that ‘active restoration … was not associated with more complete recovery or faster recovery than passively recovering systems' based on recovery completeness and rate, respectively. We did not re-evaluate this conclusion for recovery completeness, given its limitations, but did for recovery rate, examining the sensitivity of JEA's results to outlier removal and statistical approach. Here, we focus on strong sensitivity to outlier removal because, although we disagree with aspects of their model implementation, sensitivity analysis showed results to be robust to alternative modelling decisions (electronic supplementary material, supporting code 4.0). While it can be appropriate to remove outliers in meta-analysis, it is critical to assess sensitivity to outlier removal [5], which was not reported in this case. Furthermore, JEA made an error when removing outliers (H. P. Jones 2018, personal communication), resulting in the loss of more data than they intended (electronic supplementary material, supporting code 2.3; figure 1c).

We reanalysed recovery rate using JEA's first and third model structures, with recovery rate as the dependent variable and restoration type as the sole predictor (third) or with disturbance or habitat types as additional predictors (first). Models were tested using three approaches differing in treatment of variance: generalized estimating equations, linear mixed-effects models and the meta-analytic multi-level model used by JEA (rma.mv) (electronic supplementary material, supporting code 4.0). All models were re-run using full data and data reduced using the IQR-based rule as JEA intended. We also reran rma.mv models following JEA's errant outlier removal to confirm we could re-create their null findings for restoration type. Our results demonstrate consistently higher recovery rates for active than passive approaches, with the significance of this effect attenuated by JEA's intended outlier removal and eliminated by their implemented outlier removal (electronic supplementary material, table S1 and figure S2).

Ultimately, we would like to know if active restoration yields better results than passive restoration in a particular context. Unfortunately, inferences from retrospective analyses using primarily observational studies are problematic, since confounding factors will often have influenced decisions to use active versus passive approaches [3]. For example, as JEA point out, managers may be correctly identifying less easily restorable sites as requiring active interventions, undermining passive–active comparisons [1]. In addition, categorizing approaches as passive or active is not straightforward and, while we agree with JEA that the distinction is ‘blurry’, we find their classification inconsistent. They purported to differentiate actions aiming solely to end a disturbance (passive) from actions aiming to further boost recovery (active), but applied this distinction inconsistently. For example, alum application (reducing disturbance from excess phosphorus) and remediation of mine-polluted waters (reducing disturbance from pollutants) were defined as active. A common construct is to consider passive restoration as manipulating abiotic attributes causing impairment and active restoration as manipulating biotic attributes [6]. Following this definition, most actions JEA characterized as active should be considered passive. In other contexts, passive restoration is erroneously conflated with lower effort or resource requirements [7]. While this was not the framework JEA used, one of their main conclusions seems to make this link: ‘Letting ecosystems repair themselves in many cases may be the most effective restoration strategy—a counterintuitive yet critical finding that could help society allocate restoration funds more efficiently in the future.’ This conclusion is undercut by restoration costs not aligning with JEA's passive/active classification. To use stream restoration as an example, removing a dam (passive) is far costlier than inserting woody debris (active). Furthermore, JEA's suggestion to phase-in active restoration after first trying passive restoration would increase the cost of many projects and, as they acknowledge, is incompatible with practical constraints common in restoration.

An activity we were surprised to see excluded from JEA's analysis was invasive species management, a core global restoration activity [8,9] with a concomitantly large literature. Invasive species control was excluded because JEA applied a strict requirement that the invader be fully removed (H. P. Jones 2018, personal communication) and therefore had too few cases to include in their analysis. With some exceptions (e.g. oceanic islands), full eradication of invasive species is rarely feasible [10]. However, reducing invasive species abundance and impacts is a common aim that can contribute to meeting restoration goals [8]. Holding invasive species management to a standard of complete removal eliminated a huge area of global restoration effort from consideration, and is inconsistent with JEA's treatment of other actions that mitigate but do not eliminate disturbances (e.g. applying oil dispersants).

Meta-analysis is increasingly being applied to restoration studies. The promise is tremendous as inferences can be strengthened by the accumulated weight of multiple studies. But with that power comes peril. There are many ways that meta-analysis can go astray, potentially leading to faulty conclusions that nonetheless become influential because they appear to distil an entire body of research [4]. And while meta-analysis can characterize broad patterns, by definition, it operates at a coarse resolution. By contrast, restoration is highly context-dependent and practitioners make informed decisions based on project-specific factors. For example, restoring native vegetation in human-dominated landscapes degraded by invasive plants requires active approaches like invasive species control, seeding, and prescribed fire [11]. By contrast, where there is high capacity for passive recovery, as in large forested landscapes following logging, such actions may be unnecessary [12]. Regardless of differences revealed by broad retrospective analyses, on-the-ground decisions should be driven by managers' system-specific expertise and research on the attributes being targeted. As JEA discuss, restoration strategies should be tailored to overcome specific barriers; in our experience, practitioners generally do so.

We agree with JEA that identifying more efficient, economical approaches to restoration is critical. Unfortunately, limited funding and lax performance standards have too often tipped the scales in favour of lower-effort approaches that fail to offset the losses restoration is intended to reverse [13]. JEA's work highlights the need to better account for factors limiting restoration progress. Restoration research typically focuses on ecological components more than social components, such as organizational capacity or long-term commitment—factors that can be difficult to assess and are thus addressed rarely relative to their importance. More fully examining the human dimensions of restoration represents an opportunity to better target and overcome barriers to progress.

Data accessibility

Reanalysis output (.HTML), code (.R) and data (.RData) are available at https://larkin-et-al-procbcomment.netlify.com/. These resources draw upon data available from the Dryad Digital Repository: https://doi.org/10.5061/dryad.rj849k6 [2] and code (GitHub: https://github.com/DocHPJones/Restoration_Database) published by Jones et al.

Authors' contributions

D.J.L. analysed the data and drafted the manuscript; R.J.B. analysed the data and revised the manuscript; J.F. analysed the data and revised the manuscript; S.M.G. revised the manuscript.

Competing interests

We declare we have no competing interests.

Funding

We received no funding for this study.