Does stakeholder participation improve environmental governance? Evidence from a meta-analysis of 305 case studies

Highlights

• •We examine how key design features of participation affect environmental outcomes.
• •In general, participation has a positive impact on environmental governance outcomes.
• •Power delegation to participants is the most reliable predictor of outcomes.
• •Communication intensity and participants’ environmental stance are also influential.
• •Effects vary depending on the context and government agencies’ environmental goals.

Abstract

Participation and collaboration of citizens and organized stakeholders in public decision-making is widely believed to improve environmental governance outputs. However, empirical evidence on the benefits of participatory governance is largely scattered across small-N case studies. To synthesize the available case-based evidence, we conducted a broad case-based meta-analysis across 22 Western democracies, including 305 individual cases of public environmental decision-making. We asked: How do ‘more’ participatory decision-making processes compare against ‘less’ participatory ones in fostering – or hindering – strong environmental governance outputs, (i.e. environmental provisions in plans, agreements or permits)? Which design features make a difference? What role does the decision-making context play? How do results change if we control for the intentions of the leading governmental agency? To capture the central design features of decision-making processes, we distinguish three dimensions of participation: the intensity of communication among participants and process organizers; the extent to which participants can shape decisions (“power delegation”); and the extent to which different stakeholder groups are represented. Our regression analysis yields robust evidence that these three design features of participation impact upon the environmental standard of governance outputs, even when controlling for the goals of governmental agencies. Power delegation is shown to be the most stable predictor of strong environmental outputs. However, communication intensity only predicts the conservation-related standard of outputs, but not the environmental health-related standard of outputs. Participants’ environmental stance was another strong predictor, with considerable variation across different contexts. While our results remain broadly stable across a wide range of contexts, certain contextual conditions stood out in shaping the relation between participation and environmental outputs. Overall, our findings can inform the design of participatory processes that deliver governance outputs of a high environmental standard.

1. Introduction

The participation of citizens and organized stakeholders in public decision-making has become a fundamental principle of governance for sustainability worldwide (Dietz and Stern, 2008, Sachs et al., 2019, UN Environment, 2019). While stakeholder participation has long been assumed to empower local actors (Arnstein, 1969, Mancilla García and Bodin, 2019) and enhance the democratic legitimacy of decision-making (Fischer, 1993), participation is now promoted as a key instrument to improve environmental decision-making and environmental outcomes (Renn et al., 1995, Beierle and Cayford, 2002, Reed, 2008, Scott, 2015, Newig et al., 2018, Eckerd and Heidelberg, 2020). It is assumed that participation, compared to traditional ‘top-down’, governmental decision-making, incorporates diverse environmental values, integrates stakeholder knowledge (Brody, 2003, Smith, 2003, Armitage et al., 2011, Sterling et al., 2017), and produces – through dialogue and collaboration – more creative solutions, serving the common good rather than particular interests (Renn et al., 1995, Smith, 2003, Koontz and Thomas, 2006, Newig et al., 2018). However, the effectiveness of participation has also been questioned. Participants may lack environmental preferences (Dupke et al., 2019) or capacities (Laird, 1993, Geissel, 2009, Batory and Svensson, 2019, Wamsler et al., 2020), and consensus-based decision-making may inhibit transformative change towards environmental sustainability (Brody, 2003).

Many of these assertions about environmental outcomes¹, however, rely on scattered evidence, or lack empirical support altogether, and the field is dominated by individual or small-N case studies that do not permit robust generalizations. Few comparative studies have assessed participatory public environmental decision-making. An early meta-study on participation in environmental governance (Beierle and Cayford, 2002) does not assess environmental outcomes, as is typical of most research in the field (Ansell and Gash, 2008, Newig and Fritsch, 2009b). Most studies lack non-participatory control-groups (Gerlak et al., 2013), and do not systematically examine contextual factors shaping success of participatory approaches (Bodin, 2017). Some studies find (limited) positive influence of participation on environmental outcomes (Baldwin, 2020, Newig and Fritsch, 2009b, de Vente et al., 2016, Cattino and Reckien, 2021). Other studies find little to no substantive impact of participation in conservation planning (Young et al., 2013), biodiversity conservation (Sterling et al., 2017) or impact assessment (Ulibarri et al., 2019). One early study found governmental agencies to be more decisive than participatory process (Chess and Purcell, 1999), yet the role of government agencies has received little attention in subsequent studies.

There is an urgent need to scrutinize and consolidate knowledge of what works, and what does not work, when engaging citizens, non-governmental organizations (NGOs) and private businesses in environmental governance. In a scientific field as fragmented and heterogeneous as environmental governance research, the systematic cumulation of (existing) knowledge is essential but also particularly challenging (Newig and Rose, 2020). In order to derive robust generalizations, we must examine how participatory processes play out across contexts in comparison to less or non-participatory processes. Given the responsibility of government agencies in decision-making, we also need to understand the impact of agency interest in participatory processes.

To better understand which principal design features of participation actually affect environmental outputs, we took a meta-analytical approach, synthesizing published accounts of 305 cases of public environmental decision-making from 22 different Western democracies, spanning several decades (Newig et al., 2021). We looked for general patterns across cases with different process formats and contexts, and addressing different environmental issues. This ‘case survey’ method enables translation of empirical findings from methodologically diverse studies into a common analytical language, thereby allowing systematic analysis of the numerous case studies available (Jensen and Rodgers, 2001). We conducted regression analysis on the links between multiple aspects of participation and different areas of environmental outputs. We also controlled for organizational factors related to the leading public agency, and studied the impact of contextual conditions. Case selection was intentionally not limited to participatory and collaborative processes, but covered a broad spectrum of governance modes, including traditional agency decision-making, which functions as a ‘control group’ against which to measure the effects of participation.

2. Concepts and definitions: What is participation in public environmental governance?

Public environmental governance, while serving various ends and taking various forms, typically aims to produce collectively binding decisions (e.g. agreements, plans, licenses) on environmental matters of public concern. We focus on sub-national governance processes because of their high potential for and frequent use of intensive participation. Such processes – such as the designation of a nature protection area, the approval of a power plant, or the drafting of an air quality plan – are generally initiated by government agencies, but stakeholder-driven initiatives also exist. Within legal limits, process initiators normally have some discretion as to how participatory the process will be, making participation a design choice (Newig et al., 2018, Uittenbroek et al., 2019).

To identify those features of participation that (likely) affect environmental governance outputs, we distinguish three dimensions in which participation can be more or less ‘intensive’ (Fung, 2006, Newig et al., 2018, Newig et al., 2019, Jager et al., 2020, Jager et al., 2022). This is to acknowledge the fact that there is no single measure of participation ‘intensity’; rather, participation can be more ‘intensive’ in one dimension and less ‘intensive’ in another one. These dimensions (Fig. 1, left-hand side) reflect the principal design choices organizers of participatory processes face, which may impact governance outputs:

• ▪Stakeholder representation: To what degree are different stakeholder groups represented in the process? While some authors maintain that participation of ordinary citizens matters most (Fischer, 1993, Renn et al., 1995, Smith, 2003), others highlight the role of organized interest groups, NGOs and private businesses (Meadowcroft, 2004, Young et al., 2013). Yet others argue that environmental outcomes depend on the environmental values, concerns and interests of participants (Fung, 2006, Newig and Fritsch, 2009b, Baudoin and Gittins, 2021).
• ▪Communication: How intensive is communication and interaction among participants and process organizers? It is typically assumed that more intensive communication produces higher quality outcomes (Beierle and Cayford, 2002, Rowe and Frewer, 2005). For instance, face-to-face dialogue, and deliberation would lead to stronger environmental outcomes than a public referendum or non-interactive consultation.
• ▪Power delegation: What degree of decision-making power is assigned to participants? Literature on participation and collaboration (Arnstein, 1969, Beierle and Cayford, 2002, Feist et al., 2020) has maintained that without the opportunity for participants to actually shape decisions, participation remains symbolic and ineffective.

Fig. 1.

Conceptual model showing how the ‘intensity’ of participation is considered to impact the environmental standard of decision-making outputs. We measure the ‘intensity’ of participation in three dimensions: extent of stakeholder representation; intensity of communication; and extent of power delegation to participants. As control variables, we also consider factors related to the agency leading the decision-making process. Contextual conditions potentially affect the relation between participation and environmental outputs.

This conceptual representation of the essential design choices of participation enables comparative analysis of actual decision-making processes as they play out in practice – as opposed to using “formats” (e.g. citizen jury, public referendum, round table, townhall meeting, etc.) as analytical units. Because a given “format” may be conducted in very different ways (in particular, power delegation can vary considerably across instances of the same format such as a citizen jury), assessing the three dimensions is a more meaningful approach.

We tested the extent to which the mode of participation, or non-participation – measured through the ‘intensity’ of participation in each dimension – impacts the environmental standard of decision-making outputs of public governance processes (see Fig. 1). Just how these dimensions influence environmental governance outputs likely depends on the context of the decision-making process (Beierle and Cayford, 2002, Bodin, 2017, Baker and Chapin, 2018). We consider a range of external contextual conditions (Fig. 1, top middle box) that are likely to moderate the way in which aspects of participation play out in shaping environmental outcomes (Ansell and Gash, 2008, Newig et al., 2018, Fischer, 1993, Renn et al., 1995). For example, local-level participation may produce stronger environmental outcomes than participation on higher levels because stakeholders can contribute local knowledge and self-organize to address environmental problems (Leach et al., 2002).

We define ‘environmental (governance) outputs’ as the content of the decision produced through a decision-making process – the plan, agreement, license, etc. The extent to which it is likely to drive improvement (or deterioration) of environmental conditions concerning the issue at hand we term ‘environmental standard’ of the governance output (Newig et al., 2019, Jager et al., 2020). The expectation is that participation makes a difference for the degree to which governance outputs benefit the environment. Within the content of governance outputs, we distinguish two main areas (Newig et al., 2013): First, conservation-related outputs refer to nature conservation and the protection of natural resources – for example, an agreement to mandate riparian buffers in an intensive farming area to enhance biodiversity and mitigate nitrate contamination of water bodies. Second, environmental health-related outputs relate to environmental issues with implications for human health – for example, a traffic management plan that bans combustion vehicles from the town center to reduce airborne particulate matter. Both categories are not mutually exclusive, so a given governance output can be assessed in terms of both conservation and environmental health.

With few exceptions, public governance processes involve government agencies, either in a leading or contributing role (Chess and Purcell, 1999, Ryan, 2001). While it stands to reason that government agencies substantially determine governance modes and outcomes, this has never been thoroughly tested. We therefore introduce two variables to control for the influence of government agencies: One variable measures the environmental aspirations of the responsible agency (if present in the process) in terms of conservation or environmental health, respectively. Another measures the degree to which the mode of participation (or non-participation) was explicitly adopted to achieve environmental benefits, rather than other goals, such as societal acceptance or cost efficiency.

3. Methods

To test the relationship between ‘intensity’ of participation in its three dimensions and the environmental standard of governance outputs, we conducted a quantitative analysis of 305 case of public environmental decision-making. Using the case survey method (Yin and Heald, 1975, Larsson, 1993, Newig and Fritsch, 2009a) allowed us to gather and synthesize evidence from the many qualitative case studies that dominate the research field. All cases were coded independently by three researchers.

Each ‘case’ constitutes a public environmental governance process (such as policy-making, permitting procedures, or conservation planning) that is oriented towards a collectively binding decision (policy, permit, plan, etc.). In order to allow for sufficient variation in our independent variables, we included cases with little or no participation (e.g. classical political-administrative decision-making processes) as well as cases with pronounced participation and collaboration with non-state actors.

For each case, we assessed the ‘intensity’ of participation in multiple dimensions (independent variables), and the environmental standard of the governance outputs in two dimensions (dependent variables), as well as a number of control variables related to governmental agencies leading the governance process. We used multi-level modelling to test the links between participation and environmental outcomes.

3.1. Case study identification and selection

The process of case identification and selection is summarized in Fig. 2. Aiming for a comprehensive coverage of public environmental decision-making processes in developed Western democracies, we searched several online scientific databases and library catalogues (e.g. Web of Science, Scopus, Google Scholar, BASE, SSRN) for studies published between 1968 and 2014 in English, German, French, or Spanish language on environmental governance processes. We did not apply any restrictions to the degree or nature of stakeholder participation, so long as decision-making processes could in principle have allowed for some degree of participation (hence excluding most national policy-making processes that are characterized by highly formal legislative procedures). Decision-making could be non-state actor-initiated or agency-initiated. In order to keep the political and cultural context broadly similar, we restricted our study to cases from Europe, North America, and Australia and New Zealand. In order to include case studies from different disciplinary perspectives, we used combinations of a variety of different search terms, including environment-related terms (e.g. ecosystem-based; wetlands; waste-siting, watershed), governance-related terms (e.g. collaboration, participatory, decision making, deliberation, stakeholder involvement, controversy, planning) and terms for particular process forms (e.g. citizen jury, public hearing, town meeting, taskforce, consensus conference).

Fig. 2.

Flowchart of case identification and selection, following PRISMA guidelines (Moher et al., 2009).

Adapted from Jager et al. (2022).

In order to minimize publication bias through over-representation of ‘successful’ cases (Mahood et al., 2014, Banks et al., 2015), we included peer-reviewed journal articles, books and book chapters as well as ‘grey literature’ (working papers, conference papers or reports) that was publicly available. As a robustness check in this respect, we repeated our statistical analyses (as detailed below) excluding all cases that relied solely on grey literature. Results remained stable.

Our search – conducted up to the point of saturation where we discovered no further cases with additional search efforts – yielded more than 2,000 cases of governance processes, described in more than 3,300 individual published texts. In subsequent screening we eliminated all cases with insufficient information on the context, process and outcomes of decision-making, retaining 639 ‘codable’ cases. While we coded this whole dataset for basic information (see below), we took a random sample of 305 cases for full coding. These cases are described in around 500 different publications, with 107 cases drawing on two or more publications. The cases cover 22 Western democracies, encompassing a wide range of environmental issues, including land use, biodiversity and water resources, but also particular topics such as waste facility siting, transport infrastructure, and energy planning. Further details and descriptive statistics on the database of 639 cases (‘extended dataset’), as well as the 305 cases of this sample (‘core dataset’), can be found in Suppl. Figs. 1–5. A complete and sortable list of all cases (with filtering options), is available at https://partscout.org/en/cases.

3.2. Coding scheme development

On the basis of our conceptualization of participatory decision-making processes and theory-derived relations between process attributes, environmental outcomes, and relevant contextual variables we developed a comprehensive coding scheme (Newig et al., 2013), comprising 259 quantitative variables, and additional qualitative variables – each with an accompanying measurement scale and detailed coding instructions. Most variables were coded on a five-point quantitative Likert-type scale (from 0 to 4).

3.3. Specification of variables

For detailed definitions of all variables, along with descriptive statistics, please refer to Suppl. Table 1.

3.3.1. Independent variables

Independent variables refer to those features of participation that likely affect the environmental standard of governance outputs. We distinguish three dimensions in which participation can be more or less ‘intensive’:

• ▪Stakeholder representation was measured using multiple variables as the representation of civil society actors, private business actors and individual citizens, and also as the representation of pro-environmental and pro-economic development interests in a given case (each measured on a scale from 0 to 4).
• ▪Communication was measured as the average intensity of information flow from and to participants, the intensity of dialogue among organizers and participants, and the extent to which communication occurs in a face-to-face setting. All variables were measured on a 0 to 4 scale, and aggregated by arithmetic means to a communication index (Cronbach’s Alpha = 0.89).
• ▪Power delegation to participants was measured by the degree to which the process design provided the possibility for participants to develop and determine the decision (Newig et al., 2013, p. 37), also measured on a 0 to 4 scale.

3.3.2. Dependent variables

Our study seeks to explain the environmental standard of governance outputs through participation-related variables. To this end, we coded the outputs produced by a public decision-making process, typically set down in writing in the form of a management plan, a permit, a law, etc. In case of multiple decisions, the ‘final decision’ discussed in the case material is identified as the most legally binding decision described, excluding subsequent changes through litigation. In 286 of the 305 cases, a decision was produced and governance outputs could be coded.

In order to compare the environmental standard of governance outputs across a variety of processes and contexts, we were inspired by the notion of ‘regime effectiveness’ as conceptualized by Underdal (Underdal, 2002), who proposes to evaluate regime effectiveness against a hypothetical collective optimum, “one that accomplishes […] all that can be accomplished – given the state of knowledge at the time” (p. 8). Accordingly, we defined the environmental standard of the governance output as the degree to which the decision required an improvement (or tolerated a deterioration) of environmental conditions. This was assessed moving from the ‘business as usual’ scenario (projected trend) towards a hypothetical ‘optimal’ (or ‘worst case’) condition. (Newig et al., 2013). This variable was defined on a −4 to 4 scale, with 0 referring to a hypothetical business-as-usual scenario, −4 referring to a governance output corresponding to a ‘worst-case’ scenario, and + 4 to a hypothetical optimum.

Since the meaning of ‘environment’ in “environmental output” can vary considerably from case to case, we distinguish the protection of the natural environment, including natural resources, from the protection of the environment with a view to human health.

• ▪The first dimension is a composite of two variables, capturing both an eco-centric perspective on conservation and a more anthropocentric perspective of natural resource protection. The former relates to preserving, protecting or restoring the natural environment and ecosystems largely independently of their instrumental value to humans (mean = 0.61). The latter relates to protecting, preserving, enhancing or restoring stocks and flows of natural resources that are of instrumental value to humans, and providing for their sustainable use (mean = 0.02) (Newig et al., 2013). As both dimensions were highly correlated (r = 0.76, p <.001), we combined them into a single scale (alpha = 0.91). We term this variable Conservation-related output; it has an observed range from −4 to + 4, with a mean value of 0.82 (SD = 1.44).
• ▪The second dimension, protection of environmental health, relates to protecting the quality of (human) life through enhancing environmental factors beneficial to human health, and/or mitigating environmental impacts and remediating environmental problems detrimental to human health, e.g. noise or pollution of air and water (Newig et al., 2013). It also has a range from −4 to 4, and a mean value of 0.6 (SD = 1.26).

3.3.3. Case coding

Each case was independently read and coded by three trained raters. Three raters were deemed sufficient to achieve high data quality (Libby and Blashfield, 1978). For each variable, coders also assessed the reliability of the information underpinning their coding decision, using a scale from 1 (indicating sufficient information for an informed estimate) to 3 (indicating explicit, detailed and reliable information), with missing data coded with a reliability of 0 (Newig et al., 2013). After initial coding, raters met to discuss technical errors and explore divergent interpretations to potentially correct individual codes, but not to aim for consensus. Hence, our method accommodates different interpretations of the published material by individual raters (Kumar et al., 1993).

3.4. Reliability checks

We measured interrater reliability through $G\left(q,k\right)$ (Putka et al., 2008), and through interrater agreement ($r_{\mathit{WG}}$) (James et al., 1984). These scores were 0.78 and 0.71, respectively. We explored our data for the influence of distorting factors, such as the influence of rater drift or publication type (Jager et al., 2022), and did not detect any undue distorting effects.

3.5. Model development

To explain the conservation-related and the environmental health-related standard of governance outputs, we applied a multi-level modelling approach, based on the triple-coded data. In our model, level 2 is constituted by the cases as such, while level 1 comprises the rater-specific case instances, three per case. Due to missing data, 277 cases were included in the analysis with an overall sample size of 734 observations.

3.5.1. Explaining the conservation-related standard of governance outputs

In our analysis, we followed the guidelines provided by Zuur et al., 2009, Field et al., 2012. All analyses and graphs were done in R, using the packages lme4 (Bates et al., 2015), car (Fox and Weisberg, 2019), psych (Revelle, 2019), ggbeeswarm (Clarke and Sherrill-Mix, 2017) and ggplot 2 (Wickham, 2016).

In a first step, we established a linear regression model that contains all potential explanatory variables (beyond optimum), however without random intercepts or slopes:

$${\mathit{OutputConservation}}_i=\alpha +b_1{\mathit{Communication}}_i+b_2{\mathit{PowerDelegation}}_i+b_3{\mathit{ReprPrivate}}_i+b_4{\mathit{ReprCivil}}_i+b_5{\mathit{ReprCitizen}}_i+b_6{\mathit{ReprConservation}}_i+b_7{\mathit{ReprEconDevelop}}_i+b_8{\mathit{AgencyEnvRationale}}_i+b_9{\mathit{AgencyConservationAspiration}}_i+{\epsilon }_i$$ (4.1)

where OutputConservation is the dependent variable across all observations i, i.e. the onservation-related standard of governance outputs. The residual ${\epsilon }_i$ is assumed to be normally distributed with mean 0 and variance σ².

Following Zuur et al., (2009), we then built the same model but with random intercepts:

$${\mathit{OutputConservation}}_{\mathit{ij}}=\alpha +b_1{\mathit{Communication}}_{\mathit{ij}}+b_2{\mathit{PowerDelegation}}_{\mathit{ij}}+b_3{\mathit{ReprPrivate}}_{\mathit{ij}}+b_4{\mathit{ReprCivil}}_{\mathit{ij}}+b_5{\mathit{ReprCitizen}}_{\mathit{ij}}+b_6{\mathit{ReprConservation}}_{\mathit{ij}}+b_7{\mathit{ReprEconDevelop}}_{\mathit{ij}}+b_{8}Agency{\mathit{EnvRationale}}_{\mathit{ij}}+b_9{\mathit{AgencyConservationAspiration}}_{\mathit{ij}}+a_i+{\epsilon }_{\mathit{ij}}$$ (4.2)

The term a_i is a random intercept and is assumed to be normally distributed with mean 0 and variance d². The residual ${\epsilon }_{\mathit{ij}}$ is also assumed to be normally distributed with mean 0 and variance σ².

We fitted the models using the restricted log-likelihood (REML), allowing for applying the likelihood ratio test to assess the need for the random intercept. The likelihood ratio test indicates that the model with random intercept is considerably better (L = 202.69, df = 1, p <.001), supported by a strong decrease in the AIC from model (4.1) (AIC = 2369.14) to model (4.2) (AIC = 2168.45).

In the next step, we fit the optimal fixed structure. Inspecting the regression parameters, we observe that neither representation variable is significant. To that end, we fit two concurring models, one including representation variables by actor type (private, civic, citizen (4.3)), one by interest (nature conservation, economic development (4.4)):

$${\mathit{OutputConservation}}_{\mathit{ij}}=\alpha +b_1{\mathit{Communication}}_{\mathit{ij}}+b_2{\mathit{PowerDelegation}}_{\mathit{ij}}+b_3{\mathit{ReprPrivate}}_{\mathit{ij}}+b_4{\mathit{ReprCivil}}_{\mathit{ij}}+b_5{\mathit{ReprCitizen}}_{\mathit{ij}}+{b_{6}AgencyEnvRationale}_{\mathit{ij}}+b_7{\mathit{AgencyConservationAspiration}}_{\mathit{ij}}+a_i+{\epsilon }_{\mathit{ij}}$$ (4.3)

$${\mathit{OutputConservation}}_{\mathit{ij}}=\alpha +b_1{\mathit{Communication}}_{\mathit{ij}}+b_2{\mathit{PowerDelegation}}_{\mathit{ij}}+b_3{\mathit{ReprConservation}}_{\mathit{ij}}+b_4{\mathit{ReprEconDevelop}}_{\mathit{ij}}+b_{5}Agency{\mathit{EnvRationale}}_{\mathit{ij}}+b_6{\mathit{AgencyConservationAspiration}}_{\mathit{ij}}+a_i+{\epsilon }_{\mathit{ij}}$$ (4.4)

Again, we use a likelihood ratio test for model comparison of nested models and the AIC, but now fitted by maximizing the log-likelihood (ML). Results indicate that the complete model (4.2) is considerably better than the model limiting representation variables to those by actor type (4.3), L = 6.41 (df = 2, p =.04), whereas it is not better than the model containing only representation variables by interest (4.4), L = 3.71 (df = 3, p =.29). This finding is supported by comparing AIC, which is the smallest for the more parsimonious model (4.4) AIC = 2125.8, vis-à-vis model (4.2), AIC = 2128.09.

Finally, we refit the model with REML and validated the results. Our model has shown a significant variance in intercepts across cases, SD = 0.9 (95% CI: 0.81, 1.01). Regression coefficients are presented in Suppl. Table 4. Assessing underlying assumptions of multicollinearity (Suppl. Table 4), normal distribution of residuals and homogeneity of variance (Suppl. Figure 6) did not show reason for concern.

3.5.2. Explaining the environmental health-related standard of governance outputs

The same procedure as in S.4.1. was followed to determine the best model for the environmental health-related output.

These models were fit:

$${\mathit{OutputHealth}}_i=\alpha +b_1{\mathit{Communication}}_i+b_2{\mathit{PowerDelegation}}_i+b_3{\mathit{ReprPrivate}}_i+b_4{\mathit{ReprCivil}}_i+b_5{\mathit{ReprCitizen}}_i+b_6{\mathit{ReprHealth}}_i+b_7{\mathit{ReprEconDevelop}}_i+b_8{\mathit{AgencyEnvRationale}}_i+b_9{\mathit{AgencyHealthAspiration}}_i+{\epsilon }_i$$ (4.5)

$${\mathit{Outputealth}}_{\mathit{ij}}=\alpha +b_1{\mathit{Communication}}_{\mathit{ij}}+b_2{\mathit{PowerDelegation}}_{\mathit{ij}}+b_3{\mathit{ReprPrivate}}_{\mathit{ij}}+b_4{\mathit{ReprCivil}}_{\mathit{ij}}+b_5{\mathit{ReprCitizen}}_{\mathit{ij}}+b_6{\mathit{ReprHealth}}_{\mathit{ij}}+b_7{\mathit{ReprEconDevelop}}_{\mathit{ij}}+b_8{\mathit{AgencyEnvRationale}}_{\mathit{ij}}+b_9{\mathit{AgencyHealthAspiration}}_{\mathit{ij}}+a_i+{\epsilon }_{\mathit{ij}}$$ (4.6)

$${\mathit{OutputHealth}}_{\mathit{ij}}=\alpha +b_1{\mathit{Communication}}_{\mathit{ij}}+b_2{\mathit{PowerDelegation}}_{\mathit{ij}}+b_3{\mathit{ReprPrivate}}_{\mathit{ij}}+b_4{\mathit{ReprCivil}}_{\mathit{ij}}+b_5{\mathit{ReprCitizen}}_{\mathit{ij}}+b_6{\mathit{AgencyEnvRationale}}_{\mathit{ij}}+b_7{\mathit{AgencyHealthAspiration}}_{\mathit{ij}}+a_i+{\epsilon }_{\mathit{ij}}$$ (4.7)

$${\mathit{OutputHealth}}_{\mathit{ij}}=\alpha +b_1{\mathit{Communication}}_{\mathit{ij}}+b_2{\mathit{PowerDelegation}}_{\mathit{ij}}+b_3{\mathit{ReprHealth}}_{\mathit{ij}}+b_4{\mathit{ReprDevelop}}_{\mathit{ij}}+b_{5}Agency{\mathit{EnvRationale}}_{\mathit{ij}}+b_6{\mathit{AgencyHealthAspiratio}}_{\mathit{nij}}+a_i+{\epsilon }_{\mathit{ij}}$$ (4.8)

As before, model (4.5) proved considerably better than model (4.6), L = 179.87 (df = 1, p <.0001), and model (4.8) in turn displayed the best model fit (likelihood ratio test (4.6) vs. (4.8), L = 1.15, df = 3, p =.76).

In the final model (4.8), intercepts varied randomly with a SD of 0.82 (95% CI: 0.67, 0.92). Regression coefficients are presented in Suppl. Table 6.

3.6. Modeling the impact of context

To test the moderating impact of the decision-making context on participation-output relationships, we considered 20 different contextual variables that were included in the coding scheme (Newig et al., 2013). Contextual factors were either categorical variables (e.g. continent, or existence of a NIMBY situation) or coded on a five-point (0 to 4) or nine-point (-4 to 4) scale. For every context variable, the dataset was partitioned into two subsets: one for the presence (or high values) of a factor, one for the absence (or low values) of a factor, resulting in a total of 40 subsets of cases. Five-point-scale variables were partitioned into ‘high’ (>= 2) and ‘low’ (<2) values, nine-point-scale variables into positive (>= 0) and negative (<0) values. For all 40 subsets, we ran the above-specified regression models for both environmental health- and conservation outcomes.

4. Results and discussion

Of the 305 studied cases of public environmental decision-making, 61% are from North America, 32% from Europe, and 7% from Australia and New Zealand. Decision-processes had a duration of 46 months on average (median: 35 months) and ranged from strictly agency-based decision-making and standard forms of public hearings, to intensive forms of participatory governance, such as citizen juries, stakeholder workshops and advisory committees, engaging a wide range of citizens and stakeholder groups. The great majority of cases (95%) were located on a sub-national level of governance, ranging from municipal to cross-provincial decision-making processes.

4.1. Explaining the environmental conservation-related standard of governance outputs

Our results suggest that all three dimensions of participation matter to varying degrees for explaining the conservation-related standard of governance outputs (see Fig. 3, right-hand side; Suppl. Tables 3 and 4). In general, although effect sizes are modest, we are able to identify some conclusive patterns.

Fig. 3.

Explaining the environmental health-related (left) and conservation-related (right) standard of public environmental decision-making. We show a graphical depiction of multilevel linear model analysis with random intercepts. Different types of stakeholder representation (environmental interests vs. societal sectors) were assessed in separate models (for details, see Suppl. Tables 3 and 4). The middle box shows the ‘intensity’ of participation in three dimensions (stakeholder representation, communication, power delegation). Blue arrows denote positive effects, red arrows negative effects. Reported results are fixed effects. Line width is proportional to effect size. Dashed lines indicate effects not statistically significant at p <.05. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Of the three dimensions of participation, the strongest effect is attributed to ‘power delegation’ (b = 0.12; p =.009. The more decision-making power is delegated to participating stakeholders, the higher the conservation-related standard of the governance output. This confirms the widely held assumption that genuine and meaningful participation that actually gives stakeholders a say, in fact improves environmental outcomes (see, e.g. Cattino and Reckien, 2021).

Communication intensity shows a similar positive effect (b = 0.12; p =.047) on conservation outcomes. The more intensive and dialogical the communication among participants, the stronger the conservation-related standard of the governance output. This generally supports the assumptions made that the exchange of knowledge, values, and ideas, and the production of shared perspectives and innovative solutions in participatory settings benefit strong environmental provisions.

To assess stakeholder representation, we grouped stakeholders first by their environment-oriented interest, and second by societal sector. Comparing the respective models, we find that environmental interests show more explanatory power and significant effects: The representation of conservation interests is positively related (b = 0.16; p =.009) to strong conservation-related standards, and the representation of economic development interests is inversely related, yet with a less strong effect (b = -0.10; p =.058). Examining the representation of stakeholders from different societal sectors, only civil society shows a noteworthy effect (b = 0.10; p =.064). Remarkably, the representation of citizens shows virtually no effect.

The results furthermore highlight the importance of agency-related factors (see Galais et al., 2021 for a similar attempt). Both the conservation-related aspirations of the responsible government agency, and the environmental rationale for choosing a particular process design, are strong predictors for conservation outcomes. Controlling for these two variables allows assessment of the actual effects of participation, given the aspirations and motivations of the leading agency. Without these agency variables, participation-related variables show stronger effects on the conservation-related standard of the governance output (Suppl. Table 3, model 2). The two agency variables are strongly correlated not only with conservation outcomes, but also with most participation-variables (Suppl. Table 2). This suggests that agencies with strong environmental aspirations tend to design decision-making processes that are more participatory and that involve pro-environmental stakeholders (and vice versa for agencies with less pronounced environmental aspirations). Including agency-related controls (as few available quantitative studies do) thus adds explanatory power and nuance to our results and puts the effects of participation on conservation outcomes into perspective.

4.2. Explaining the environmental health-related standard of governance outputs

Next, we studied how the different dimensions of participation impact governance outputs related to environmental health (addressing, for example, pollution in drinking water, ambient air pollution, or flood risks) (Fig. 3, left-hand side; Suppl. Tables 5 and 6). Among the dimensions of participation, the representation of environmental health-related interests is the strongest positive predictor for the environmental health-related standard of governance output (b = 0.22; p <.001). Similar to the results on conservation outputs, we find that power delegation to stakeholders shows a significant positive effect (b = 0.09; p =.045). However, in contrast to the conservation model, communication intensity shows but a small, though still positive, effect on environmental health outputs (b = 0.04; p =.45). This suggests that for issues of immediate concern to people (health), the assertion of one’s own interests trumps dialogue; while for conservation concerns – which less immediately determine people’s well-being – discursive interaction shows a higher potential to strongly anchor environmental concerns in the governance output. Similar to the above models on conservation, agency control variables score high in explaining the environmental health-related standard of governance outputs.

4.3. The role of the decision-making context

As argued above, the cultural and situational context of decision-making can play an important role in determining whether and how participation impacts the environmental standard of governance outputs. Therefore, we tested the stability of our findings on the participation-outcome link for 20 contextual factors by running the same regression models for context-specific subsets of the whole dataset (see Fig. 4; Suppl. Table 7 & 8).

Fig. 4.

Regression results for 20 different contextual factors, explaining the environmental health-related (left) and conservation-related (right) standard of governance outputs. Each dot in this ‘violin graph’ represents the effect (regression coefficient) of the variable named on the horizontal axis for a given context. The further away a dot (and a related ‘violin’) is from the zero axis, the greater the size of the effect of the respective variable on the environmental standard of governance outputs. Moreover, the more compact a ‘violin’ is vertically, the less pronounced the effect of context. For example, the effect of power delegation on conservation-related outputs is relatively consistent across contexts, whereas the effect of communication varies considerably across contexts, ranging from slightly negative to strongly positive. Circled dots depict coefficients for the whole dataset. Pwr. del. = power delegation; Communic. = Communication intensity; Repr. health = Representation of environmental health-oriented interests; Repr. cons = Representation of conservation-oriented interests; Repr. econ = Representation of economic development interests; Agcy. health = Agency environmental health-related aspiration; Agcy cons. = Agency conservation aspiration; Agcy. rat. = Agency environmental rationale.

This analysis broadly confirms that power delegation and the representation of environmental and economic development interests are the main predictors of environmental outcomes, followed by communication intensity (for conservation-related outcomes). Many of the considered contextual factors have little bearing on the relationship between participation and outcomes.

However, we find that a number of situational contextual conditions play a role in shaping the relation between participation and environmental standard of governance outputs.

NIMBY situations pose particular challenges to environmental decision-making. Here, people living close to the site of a planned project (such as a rail corridor) tend to perceive negative impacts and oppose the project, whereas wider society would typically stand to benefit (Kraft and Clary, 1991, Jenkins-Smith et al., 2009). Participation is considered one means to mitigate these challenges and to arrive at viable solutions, e.g. through mobilization of well-informed citizens or grassroots initiatives (Jenkins-Smith et al., 2009). We find two pathways by which NIMBY settings affect the participation-outcome relationship. The first implies a shift in the role of actor groups: While in NIMBY situations citizen participation has a positive (though not statistically significant) effect for the environmental standard of governance outputs (unlike in almost all other tested contexts), the representation of pro-environment and pro-development interests has less of an effect. This suggests that in these mostly place-based contexts, direct affectedness and mobilization of the local population tend to play a larger role than the representation of organized interests. Second, in NIMBY situations, “genuine” participation, as expressed through power delegation to participants, shows a particularly strong positive effect, supporting the assumed value of participant input in NIMBY situations.

Another notable contextual factor is whether or not a previously unsuccessful attempt at deciding the issue at stake preceded the decision-making process (see also Ansell and Gash, 2008). In such cases, communication intensity shows considerably higher effects on the environmental standard of governance outputs than in the remaining cases. This suggests that in such contentious contexts with a prehistory of failure, there might be increased potential for communicative interaction to facilitate decision-making and generate benefits for the environment.

Decision-making processes are, to different degrees, embedded in a preexisting culture of collaboration and participation. We find that in contexts of strong collaborative traditions, the representation of environment-related interests (pro-environment and pro-development) makes virtually no difference for the environmental standard of governance outputs. Apparently, the ‘interest game’ becomes less relevant where decision-making is embedded in a collaborative context vis-à-vis communication and power delegation.

Relatedly, environmental interest representation plays out differently depending on the nature and level of preexisting conflicts. We distinguish between conflicts of distribution (who gets what) and conflicts of values (ethical, social, cultural, or ideological). We find that where distributional conflicts are high, environment-related interests are strong predictors for environmental outcomes, but with a reversed relationship for value conflicts. This suggests that interest representation matters for environmental outcomes specifically where material interests are at stake.

Finally, we find that the jurisdictional levels of governance matter: If the decision-making process was situated at a relatively high level (i.e. state/provincial, national, or higher), communication intensity, environmental interest representation, and civil society representation (for conservation outcomes) had noticeable positive effects on the environmental standard of governance outputs, whereas these effects were much smaller for more local processes. A similar effect was observed for the jurisdictional level of the lead agency. These findings may be explained by earlier observations that competencies and capacities of environmental groups tend to be higher at more aggregate levels of decision-making (Rockloff and Moore, 2006), making their inputs into decision-making more effective. Alternatively, more in line with classical literature on policy implementation following Pressman and Wildavsky, 1984 [1973], we might speculate that decision-making at more aggregate levels is further removed from the messiness of real-world implementation, therefore less contentious and hence more likely to yield environmentally “beneficial” decisions, albeit with more uncertain impacts on the ground.

4.4. Reflections on the methodology

This study drew on a dataset that is unique in several ways. As far as we are aware, our study presents the largest meta-analysis on environmental decision-making processes available to-date. Arguably, it is also the most rigorously conducted study of this kind, drawing on an extensive coding scheme, with every case read and coded independently by three coders. To our knowledge, no other larger comparative study on participation has yet included non-participatory processes (as a control group). What is more, no other larger comparative study on participation has systematically analyzed (and quantified) the key dimensions of participation (power delegation to participants; communication intensity; and the kind of participants). Finally, our data present the first larger comparative study on participation that systematically accounts for the decision-making context, and for the role of governmental agencies.

The dataset, of course, also has its limitations. Our data is limited to developed Western democratic countries. This has been a deliberate choice to constrain the societal context under which the effects of participation on governance outputs are observed. Even as our results are broadly stable across regions within our sample, we would be hesitant to extrapolate findings to other world regions. Moreover, our sample raises potential issues of bias. The distribution of cases across countries is obviously skewed towards North America. Arguably, this cannot be explained by the large number of (participatory) environmental decision-making processes in the US and Canada alone, but is likely also due to language and publication activity. These latter two factors likely distort our sample, such that we cannot assume our sample to be geographically representative of the universe of actual environmental decision-making processes. However, we find geography to be of limited importance in terms of how participation impacts governance outputs – especially compared to situational factors such as NIMBY situations, pre-existing conflicts, and governance levels.

Additional distortions could result from ‘publication bias’, meaning the expectation that ‘successful’ studies are more likely to be published than ‘unsuccessful’ ones. However, given the great variety of cases within our sample, and widely varying notions of what may constitute ‘success’ (more inclusive, more just, more effective, more lasting collaborations, etc.), we feel relatively confident that the study has avoided a manifest publication bias (see also the discussion in Beierle and Cayford, 2002), all the more as we deliberately diversified our case sources (to include grey literature) and also conducted some extensive tests for publication bias (Jager et al., 2022).

However, the heterogeneity of our case material also posed challenges. To enable meaningful comparison across very different cases, we had to rely on rather abstract categories that were coded based on the expert judgement of three independent, trained raters, mostly on Likert-type scales. Naturally, such a procedure involves interpretations by raters, and not the elicitation of unambiguous ‘facts’ – all the more as the assessed case material was, of course, not specifically written for our coding scheme. Therefore, information density varied by variable and case. Nonetheless, we were able to code almost all variables for all cases (only 9 cases were excluded due to missing data; a further 19 cases were excluded because they did not lead to a governance decision). This is because, first, comprehensive coverage of decision-making process, context and outcomes was a selection criterion for case studies to be included in the first place (case material averaged more than 15,000 words, with a median of 8,000 words, not counting references). Second, for every variable per case, we coded the reliability of the presented information in a given case, ranging from 0 (no assessment possible) to 1 (informed guess possible), 2 (reasonable evaluation) and 3 (explicit, detailed and reliable information). This allowed us to include codes with less than ‘full’ information. Third, working with three independent raters with subsequent interaction allowed us to capture a high level of detail explicitly or implicitly mentioned in the case material.

5. Conclusions

The participation of citizens and organized stakeholders in public policy and administration has become a key feature in democracies worldwide, promising empowerment and emancipation of marginalized groups, enhanced democratic legitimacy, and more environmentally effective decision-making (Gerlak et al., 2013, Newig and Kvarda, 2012). Many additional benefits, such as social learning or integration of local knowledge, also contribute to the goal of improved effectiveness (Newig et al., 2018). In an era of unprecedented global environmental change, participation has entered center stage as a tool to deliver environmentally sustainable public decisions. However, the effectiveness of participatory approaches has not been comprehensively tested. In a research field dominated by individual case studies, no larger comparative study is available that assesses environmental outcomes, involves non-participatory cases as a control-group, includes context variables, and controls for the role governmental agencies. At the same time, participation remains a hotly debated topic, with some countries seeking to increase efficiency through re-centralization, while others are experimenting with novel forms of participation (Boswell et al., 2023). In this context, it is more important than ever to have sound evidence on what has worked previously in participatory governance.

In this case-based meta-analysis, we drew on a unique evidence base of 305 coded cases of environmental decision-making to test whether and how different dimensions of stakeholder participation improve (or diminish) environmental governance outcomes. Overall, we find clearly positive effects of participation-related dimensions on the environmental standard of governance outputs – one exception being the representation of economic development interests, which shows a negative effect. Adding control variables related to the governmental agency reduced the strength of virtually all effects, but most effects remained clear and significant across a variety of contextual settings.

Our results demonstrate the importance of distinguishing different dimensions of participation: representation of stakeholders, the intensity of communication among participants and with governmental agencies, and the delegation of decision-making power to participants. Results for each dimension are highly specific. Power delegation to participants shows the most positive effect, which explains both conservation-related and environmental health-related standards of governance outputs. This effect is remarkably stable across a wide range of contexts. Therefore, to maximize environmental outcomes, it appears beneficial to grant stakeholders a fair degree of (co–) decision-making competence, which is in line with ongoing calls for truly empowering participatory processes (Cook and Melo Zurita, 2019). Second, we found communication intensity to predict conservation outcomes, but not environmental health-related outcomes, with some variation across different contexts. Third, we found the environmental stance of participating stakeholders to be a considerably strong predictor. This rather sobering finding reminds us that stakeholder interests may also prevail in participatory or collaborative settings. Where represented stakeholders are more or less interested in the environment, this will be evident in the decisions made – even though this effect proved to be highly dependent on the context. Fourth, citizen participation, although often promoted in environmental governance literature (Renn et al., 1995, Smith, 2003), was not found to significantly improve environmental outcomes. Instead, organized civil society participation showed some effect with respect to conservation outcomes. This is consistent with the expectation that organized rather than unorganized publics can contribute meaningfully to creative problem solving and long-term commitment to continued environmental protection (Meadowcroft, 2004), notwithstanding other positive effects citizen participation may have. However, we should keep in mind that our findings relate to the average effect of citizen representation; insofar as citizens participate as advocates of environmental interests, their representation does have an effect on environmental outcomes. Moreover, the decision-making processes examined here do not yet reflect the potential of emerging formats such as deliberative ‘mini-publics’, wherein randomly selected citizens prepare public decisions in highly structured processes, which are argued to have great promise for advancing environmentally sustainable solutions (Willis et al., 2022).

Regarding the environmental issues at stake, our findings support the distinction drawn between outcomes on environmental conservation and those on environmental health issues, showing clear differences in the way participation-related variables can explain these kinds of outcomes. Comparing both areas of environmental outcomes, our findings suggest that to achieve a strong conservation-related standard of governance outputs, communication among stakeholders is as important as the representation of environment-related interests. On the other hand, to achieve strong environmental health-related standard of governance outputs, the representation of pro-environmental health interests is more important than intensive communication. A possible explanation could be that when human health is at stake, stakeholders’ immediate (self-) interest (expressed by the representation of pro-environmental health interests) outplays other factors. Conversely, conservation issues may be more detached from stakeholders’ immediate interests, such that reasoned debate (as expressed by communication intensity among involved actors) becomes a more important factor for achieving strong outcomes. The distinction between the two areas of environmental outcomes and its impact on relevant explanatory variables is an important insight that future studies should further examine.

Previously understudied in the context of participatory governance, our findings show that the interest of governmental agencies is as important as the features of participation in explaining both conservation and environmental health-related outcomes. So, participation alone is not necessarily a game-changer for the environment – it must be examined within the broader context of the governance system and its political-administrative processes. In other words, government and civil society are intertwined. Even in participatory governance, public officials are accountable for environmental outcomes, given their impact, as shown in this study.

Finally, we found the observed relations between participation and environmental governance outcomes to be fairly stable across a wide range of different contexts. Nonetheless, we identified several contextual factors that affect how and to what extent the different dimensions of participation impact the environmental standard of governance outputs: NIMBY situations; previously unsuccessful attempts at decision-making; the culture of collaboration and participation; pre-existing conflicts; and the jurisdictional governance level. Across different contexts, power delegation is the most stable predictor of environmental standards of governance outputs, as compared to the other dimensions of participation. Communication intensity tends to have stronger effects in settings with a previously unsuccessful attempt, a high collaborative culture, and at higher governance levels. While on average, the presence of environment-related interests fosters strong environmental governance outcomes, this varies widely across contexts.

Our findings have important implications for the design of participatory processes. Four recommendations emerge for those aiming to fashion decision-making processes that foster good environmental outcomes: (1) Grant real co-decision-making power to stakeholders in order to harness their knowledge and concerns; (2) Be aware that while intensive communication is important for many reasons, it may not necessarily lead to better environmental outcomes; (3) It matters who participates, in particular as regards stakeholders’ stance towards the environment; (4) Take into account the specific circumstances surrounding the decision-making process to make informed choices in designing and running a (participatory) environmental decision-making process.

Whether or not there is a leading agency or other body that can determine the design of a (participatory) decision-making process (Uittenbroek et al., 2019), our conclusions underscore the importance of conscious, reflective design decisions. We hope, of course, that the kind of evidence provided here will be able to inform such conscious decision-making on participatory design².

Further research should more closely look into the causal mechanisms at work, to understand, for example, how precisely the delegation of decision-power to participants fosters good environmental decisions. We furthermore encourage researchers to take inspiration from our work, extending this into other settings, including in the global South. This would likely require an adapted conceptual basis and coding scheme, accounting for potentially different mechanisms at work in countries with partly undemocratic regimes, with different traditions of participation, and distinct sustainability issues not experienced in most of the Western democracies studied in this research.

Participation matters for environmentally beneficial governance solutions. It has become an established part of the toolbox of governance instruments for addressing pressing environmental problems worldwide. Our results suggest, however, that we should move from regarding participation as a panacea to more realistically considering the specific benefits it may have across different contexts, allowing for tailor-made rather than generic process designs.

CRediT authorship contribution statement

Jens Newig: Conceptualization, Methodology, Investigation, Data curation, Writing – original draft, Writing – review & editing, Project administration, Supervision, Funding acquisition. Nicolas W. Jager: Methodology, Validation, Formal analysis, Investigation, Data curation, Writing – review & editing, Visualization. Edward Challies: Investigation, Writing – review & editing, Supervision, Project administration. Elisa Kochskämper: Investigation, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Data availability

Original data will be made available on request. The full set of ggregated and annotated data is available at https://doi.org/10.7802/2134.