Characterizing public perceptions of social and cultural impacts in policy decisions

Significance

Large infrastructure projects such as highways, pipelines, and ports undergo formal impact evaluations to assess the likely impacts to social and cultural priorities, the economy, public health, and the environment. Several scientific and citizen groups have expressed concern that social and cultural impacts are not given sufficient consideration as compared to more tangible impacts such as economic concerns. In two large surveys, we find that the public does differentiate impacts, using qualities such as tangibility and scope, and we provide evidence supporting the widespread public perception that many policy decisions fail to reflect concerns about a range of social and cultural impacts. Knowing more about public perceptions of impacts could help aid the development of more equitable impact assessment procedures.

Abstract

Social scientists and community advocates have expressed concerns that many social and cultural impacts important to citizens are given insufficient weight by decision makers in public policy decision-making. In two large cross-sectional surveys, we examined public perceptions of a range of social, cultural, health, economic, and environmental impacts. Findings suggest that valued impacts are perceived through an initial lens that highlights both tangibility (how difficult it is to understand, observe, and make changes to an impact) and scope (how broadly an impact applies). Valued impacts thought to be less tangible and narrower in scope were perceived to have less support by both decision makers and the public. Nearly every valued impact was perceived to have more support from the public than from decision makers, with the exception of three economic considerations (revenues, profits, and costs). The results also demonstrate that many valued impacts do not fit neatly into the single-category distinctions typically used as part of impact assessments and cost–benefit analyses. We provide recommendations for practitioners and suggest ways that these results can foster improvements to the quality and defensibility of risk and impact assessments.

Potential social and cultural impacts, along with economic, health, and environmental concerns, are critical inputs to formal project impact evaluations and risk or cost–benefit analyses. The goal of these analyses is to identify and evaluate the likely effects of a broad range of public policy initiatives. These include large infrastructure projects such as highways or ports, controversial initiatives such as pipelines, and intensive resource developments such as mines or hydroelectric dams.

Elected officials and other stakeholders involved in decisions about the merits of such projects are often faced with the challenging task of making value trade-offs among a diverse set of impacts. For example, a proposed highway project might be expected to generate jobs and decrease traffic congestion but could also affect the physical and mental health of nearby residents or impact culturally significant lands. Comprehensive evaluations of potential consequences across economic, social, and environmental benefits and costs can place decision makers in the unenviable position of needing to balance these wide-ranging and not readily comparable impacts.

In recent decades, these concerns have moved from the technical domain of risk and impact assessments to the center of highly visible political and societal controversies. Media attention given to the Standing Rock protests or to the Keystone and TransMountain pipeline expansions in the United States and Canada (1), for example, have focused on the perceptions of many members of the public that important scientific, social, and cultural considerations are not fully taken into account as part of required project assessment procedures such as environmental impact assessments (EIAs). The quality of the science and its perceived legitimacy can be improved by addressing substantive and procedural local knowledge needs (2). However, two additional reasons for controversy are also often cited: that social and cultural impacts are devalued by officials relative to other (e.g., economic) concerns and that changes in social and cultural impacts are more difficult to articulate, measure, and incorporate as part of conventional assessments (3, 4). This has led to disagreements among professionals concerning the identification of appropriate assessment procedures for use by industry and government and, in many jurisdictions across North America, to citizen protests and lengthy court proceedings that frustrate both supporters and those opposed to a proposed project.

In this paper, we explore public perceptions of the impacts commonly considered as part of cost–benefit and risk, social, or EIAs. We seek to understand how people perceive the various qualities or attributes of different categories of impacts including environmental, cultural, societal, health, and economic consequences. Knowing more about the existence of any systematic differences in public perceptions of various impacts could provide theoretical support for more equitable impact assessment procedures. These results could also significantly aid both professionals responsible for conducting project assessments and the decision makers who are ultimately responsible for approving, turning down, or redesigning proposed initiatives.

Background and Theory

Several factors might explain how different impacts (e.g., cost or physical health) are identified and evaluated within a particular public policy decision context. These include the concerns of stakeholders involved in the decision-making process, the range of alternatives under consideration, how the impacts are likely to change under each alternative, and the types of values-based trade-offs that are needed. Research also has emphasized the constructive nature of peoples’ beliefs and preferences, in the sense that people may lack well-formed perceptions and beliefs until confronted with a specific decision context (5). All of these factors can drive whether impacts are included in decisions and the importance assigned to each concern. In addition to these context-specific considerations, various individual-level factors (e.g., relating to peoples’ values, knowledge, and political worldviews) can affect how people typically value different actions and their perceived consequences (6). In this section, we first review current knowledge about the nature of these individual-level factors from a range of theoretical perspectives. We then explore the idea that common perceptions of the fundamental nature of different impacts could also act as an initial lens through which people view the likely effects of a proposed action.

Individual-Level Factors Affecting the Perceived Value of Different Impacts.

There is a rich and long-standing literature on values, beliefs, and attitudes. These concepts have been defined and used in different ways depending on the discipline; for example, the term “value” can mean something quite different in economics as compared to social psychology. Foundational work by Rokeach suggested that people account for their personal perspective as part of criteria “for guiding action (and) for developing and maintaining attitudes toward relevant objects and situations” (7). More recent research has advanced the perspective that an individual’s values are constructed rather than merely revealed as part of an elicitation or assessment procedure (5, 8). This is particularly the case when topics or technologies in question are novel; in such cases, responses to a mix of impact types can be highly labile and influenced by the ways in which information is presented, with the result that “what people want can appear unstable or inconsistent” (9).

Several parallel lines of research highlight what constitutes something of “value” and the connections between individual factors and the reasons why some impacts matter (or are “valued”) more than others. First, there is an extensive literature focused on value classifications in assessment contexts (10). The premise in such classifications is that nature as a physical world produces services which both directly and indirectly create value that can be represented and measured as part of formal evaluations, including risk analyses and EIAs. Most often these considerations are loosely cast as the “valued components” or “valued impacts” specific to any project and are typically glossed as the environmental, economic, and social effects that are valued by citizens. As a result, changes in their quantity, quality, or accessibility could lead to changes in experienced utility or well-being (11, 12).

A second body of work looks at values as norms or ethical principles, emphasizing how individuals typically form connections between their beliefs and the perceived impacts of actions that could be undertaken by society (13). Research by Stern et al. (14), for example, explored ways in which our fundamental value orientations—distinguished as ego-centric, altruistic, or eco-centric—serve to guide beliefs and encourage or discourage certain types of behaviors. Others have examined the relative power of values, beliefs, and/or norms as they effect specific decisions and behaviors (15, 16). What people value is also driven by factors such as political orientation, cultural meanings, and in-group norms. Political worldviews, for example, typically describe people by their preference for social organizations and/or political structures that are more or less individualistic, communitarian, egalitarian, or hierarchical (17, 18). Worldviews have been shown to predict peoples’ perceptions of the importance of a range of contentious societal issues, from climate change to gun control (19).

This extensive body of earlier research suggests a number of factors that may contribute to differences between how citizens and public policy decision makers typically weight social and cultural impacts, including specifics of the decision context and information framing as well as foundational principles and an individual’s values and political worldviews. This paper considers another possible consideration: an individual’s perceptions of the fundamental nature or quality of the different impacts associated with an action. As with worldview orientations, perceptions can strongly influence how members of the public and decision makers understand and value particular impacts as part of an evaluative structure. Knowing more about the qualities people perceive as fundamental to decisions may help to understand differences across segments of the population and aid government, industry, and non-governmental organizations in defining, assessing, and communicating the anticipated benefits, costs, and risks of large-scale, often controversial public policy decisions.

Perceptions About the Underlying Qualities of Different Valued Impacts.

The economic, physical health, and environmental impacts associated with an initiative are often described as tangible, because in many cases, they can easily be seen or experienced (4). These tangible impacts generally are included as part of industry and government assessments and formal calculations of project net benefits. Conversely, the social, cultural, and nonphysical health impacts likely to be associated with an action are often described as intangible, because they are perceived to be difficult to describe and, in many instances, lack standard measures. Examples of prominent social and cultural impacts include changes to a community’s quality of life, increased conflict among residents related to a proposed initiative, and effects on long-established food gathering or hunting practices. In recent decades, a heightened understanding of the social and cultural values associated with changes to Indigenous lands and practices has enhanced their role as part of risk and environmental debates, particularly in the context of controversial land-use decisions, but their inclusion as part of formal assessments remains limited (20–22).

An important consequence of this presumption of intangibility is that many risk and impact assessments include comprehensive descriptions of the full range of potential impacts but fail to include either qualitative or quantitative analyses of the more intangible effects. Examples of quality dimensions that might drive how different impacts are valued include the perceived ease with which they can be understood, their geographic scope (applies to a small area versus a large area), or the anticipated scale of impact (applies to a small group of people versus a larger group). Impacts perceived to be more difficult to understand or less general might be perceived to be of less value and, in some cases, omitted from the formal assessments presented to decision makers (23).

Our assumption is that people may use an explicit set of qualities to categorize an impact and thereby infer its relative importance in a decision or project development context. By considering an explicit list of impacts commonly used to characterize the pros and cons of large infrastructure projects, it might become possible to ascertain the qualities people typically attribute to these and thus provide a basis for how people are likely to interpret and give meaning to the different impacts. In order to examine perceptions of social and cultural values in particular, we require a decision context that includes a broad range of the possible impacts typically attributed to large-scale initiatives (e.g., economic, environmental, and health impacts) as well as their social and cultural effects. A familiar example might involve possible impacts of a project characterization that include predicted changes in government revenues (economic) and water quality (environmental) along with access to sacred sites by tribal members (cultural) and community well-being (social).

To the best of our knowledge, there have been few empirical studies examining people’s perceptions of the fundamental nature of different economic, environmental, social, and cultural impacts or the range of plausible dimensions on which these impacts may vary. Our use of psychometric methods adds to the quantitative literature examining perceptions of the discrepancy between impacts that concern the lay public and what decision makers typically emphasize as part of risk and policy analyses (24). By drawing from a variety of environmental and social impact assessments, our intent is to deepen understanding of how people perceive the nature or fundamental attributes of impacts across a broad range of projects or initiatives. We suspect that each category of impact, expressed either as part of a larger class (such as environmental impact) or as a subset (such as impacts on water quality), might also have its own discrete perceptual signature; that is, the bundle of qualities perceived as characterizing one set of impacts is predictably different from that perceived as characterizing another. In particular, we are interested in whether people’s judgments of the perceived qualities or attributes of social and cultural values are consistently different from those typically associated with economic, environmental, and health-related values. If so, these differences might help to explain the observation that social and cultural values often are given lower weights as part of risk analyses and EIAs.

Specific Research Aims and Hypotheses.

The primary aim of this research is to examine whether the lay public, on average, has coherent global perceptions that define or characterize different types of impacts. If these general, intuitive perceptions do not exist, then we would expect no discernible patterns of ratings across valued impacts: each person would react to each valued impact in more or less idiosyncratic ways that would average each other out. If, however, there are consistent perceptions about the qualities of these valued impacts, then we would expect discernible patterns to emerge. Three hypotheses outline these expectations:

H1: Valued impacts will vary as a function of their tangibility, for example, the difficulty of observing or making changes to them.

H2: Valued impacts will vary as a function of scope, for example, whether an impact is local versus global.

H3: Social and cultural impacts will be perceived to be less tangible and have a narrower scope as compared to economic and environmental impacts.

In addition, we are also interested in exploring differences between public perceptions of the importance placed on different impacts by decision makers and the lay public.

H4: In general, social and cultural impacts will be perceived as less important to decision makers as compared to the public at large.

Methods

To address these four hypotheses, we conducted two large surveys of the US public. We used nearly identical methods in both studies, with Study 2 acting as a replication in a nationally representative sample of the US public. We therefore discuss the methods and results of these studies together. All study materials and procedures were approved by the Decision Research Institutional Review Board. All participants provided informed consent before participating in this research. The data for Studies 1 and 2 are publicly available on the Open Science Framework at https://osf.io/pgq62/.

Design Sources for Valued Impacts and Qualities.

Our selection of the valued impacts used in these studies was based on several sources. The first was formal legislation governing impact assessment requirements such as regulations under the US National Environmental Protection Act (25), the Canadian Environmental Assessment Act (26), and the United Nations Declaration on the Rights of Indigenous People (27). We also drew from studies by economists of different categories of use and nonuse values (28) as well as guidance documents for assessing and managing the social and cultural impacts of projects (29). A third source was a recent review of social and cultural impact assessments for several large projects in the United States and Canada (21). We identified five primary categories of valued impacts from these sources, including changes that are typically classified as affecting environmental, economic, social, health, and cultural “valued components” of projects. Table 1 shows the final list of 22 valued impacts, with minor language clarifications made in Study 2.

Table 1.

Values/impacts shown to respondents (without category headings)

Values (Environmental)	Values (Cultural)
Water quality: the quality of water resources, affecting humans or nature	Local practices: long-established food gathering, hunting, and access to nature
Climate change: greenhouse-gas emissions and amounts or timing of climate change effects	Identity: how people view themselves and the roles that make them feel part of the larger community
Biodiversity: the variety and mix of plants and animals that make up a region’s ecosystem	Reputation: how people think about your community, its image or associations
Recreational opportunities: the quality or quantity of available recreational experiences	Morals and ethics: cultural, political, or spiritual principles that guide how people think and act in life
Connection to nature: established linkages between nature and humans	Native culture: the knowledge and rights held by Native American tribes
Values (Economic)	Values (Health)
Government revenues: flows of money for public needs and expenditures	Physical health: the number of illnesses or deaths
Jobs: the number and quality of employment opportunities	Safety and security: crime and related safety risks
Financial costs: the total amount of money to be spent	Emotional health: how people feel including joy or dread, calmness or worry
Profits: financial gains to the private sector	Aesthetics: visible changes to the natural or built landscape
Values (Social)
Community well-being: the typical quality of life for people living in your area
Interpersonal conflict: disagreements that arise among affected groups of people
Equity: effects on vulnerable people (elderly, children, the poor) and opportunities for minorities
Participation: the ability of citizens to influence what takes place in society

Note: Jobs included in Study 1 only.

Our selection of impact qualities was drawn from the discussion and hypotheses addressed above and informed by the authors’ experience in the conduct and review of risk analyses and impact assessments (30, 31). Members of our study team also conducted interviews with local officials involved in public policy decision-making. In particular, we generated qualities to reflect the nature of valued impacts and their perceived importance in reference to the tangibility and scope dimensions discussed above. Qualities to describe the extent to which a valued impact is perceived as tangible include 1) Understanding (whether the value is easy or difficult for people to understand), 2) Measurability (how easy or difficult it is to measure), 3) Stability (whether it can readily be changed), and 4) Observability (how easy or difficult it is to observe changes if they occur). Qualities to describe the extent to which a valued impact is perceived as broadly applicable include 5) Context (whether it applies more to individuals or to society-at-large) and 6) Geographic extent (whether it generally applies to a small versus large geographic area). These same six qualities were rated in both studies, with minor language clarifications made in Study 2.

Participants.

For Study 1, 308 participants were randomly recruited from the Dynata, Inc. online opt-in panel between April 5 to 9, 2019. Respondents were 59.4% female, with a mean age of 61 y (range 22 to 90). A total of 17% had a high school degree or less, 30% had some college or a 2-y degree, and 52% had a college degree or more. Some 87% identified as White, 3.3% as Black, 2.6% Hispanic, 4.3% Asian, 0.7% as Native American, and 2% selected “other” among these identifications. Politically, the sample was nearly evenly split among Democrats (32.1%), Republicans (30.1%), and Independents (28.5%).

For Study 2, 430 participants were recruited from YouGov online panels. YouGov uses “Matched Sampling,” first developing a target sample that is representative of the US population, stratified on several characteristics from the 2012 American Community Survey including, age, gender, social status, marital status, employment, and education. A sample of participants matched to these characteristics were then invited from their opt-in pool to participate. Lastly, sample weights were generated using a propensity scoring function. The unweighted sample was 51.4% female, with a mean age of 47.13 y [range 19 to 86], 38.7% with a high school degree or less, 31.3% with some college or a 2-y degree, and 30% with a college degree or more. A total of 64% identified as non-Hispanic White, 12.3% as Black, 2.8% Asian, and 19% identifying with another group. A total of 20% of the sample identified as Hispanic/Latino. Politically, 30.5% identified as Democrat, 25.6% Republican, 30.3% Independent, 7.0% undeclared, and 6.6% another party.

Study Design and Measures.

Both studies used a psychometric rating paradigm to explore differences among peoples’ perceptions of the qualities assigned to different valued impacts. The psychometric paradigm has been widely used in numerous studies focused on public perceptions of hazardous technologies (32). This includes hazardous objects or practices as varied as commercial aviation, pesticides, nuclear power, wildfires, and hydroelectric dams (33). Given the number of valued impacts and qualities, we sought to alleviate participant burden by using a planned missing survey design (34). Each participant was randomly assigned to see either seven or eight valued impacts (from the full set of 22 or 21; the jobs value was not included in Study 2 due to oversight) and rated these values on all qualities and outcomes (elaborated below). This resulted in a data structure in which each valued impact was rated by a roughly equal-sized, overlapping subset of respondents.

Study participants first read a general introduction to the survey and then separate paragraphs before each section (SI Appendix). In addition to the six rated qualities, we asked participants to rate three outcomes for each valued impact: 1) Importance to decision makers: How central is the value to how decisions are made? 2) Frequency of Inclusion: Is the value commonly included in decisions? and 3) Support: How strong do you think public support for this value is? We also included several individual difference measures and demographic questions; responses will be discussed as part of a separate paper. Study 2 used the same design and measures, with the exception that after answering questions relating qualities to valued impacts, all subjects were asked to place each of the valued impacts within one of the following five broad categories of effects: economic, ecosystem/environmental, social/family, cultural, and health. Participants were asked to identify “the single category of effect that best describes your understanding of the value in question”; a prompt allowed individuals to refresh their understanding of the definition of each value.

Analytic Approach.

There were ∼100 and ∼130 participants who responded to each valued impact in Study 1 and Study 2, respectively. Since each valued impact was rated by a subset of participants, we calculated the mean quality and outcome rating for each impact across all responses. Thus, the mean rating on each valued impact was the primary unit of analysis resulting in n = 22 and n = 21 in Study 1 and 2, respectively. We used principal component analysis (PCA) with orthogonal rotation to examine the dimensionality of the mean quality ratings across valued impacts. K-means cluster analysis was then used to explore whether there were any coherent subgroups of valued impacts based on the rated qualities. We also examined the relation between the rated qualities and the outcomes of perceived importance to decision makers and the public. Finally, we examined differences between perceptions of decision maker and public support for each valued impact using a standardized distance metric (Cohen’s d). For the replication Study 2, the analytic approach was identical to Study 1 with the exception of including person-level sampling weights when generating the mean quality and outcome ratings for each valued impact. The inclusion of these person-level sampling weights allowed more precise estimation of population means.

Results and Discussion

In Study 1, we used a diverse general population sample to explore public perceptions of the various types of valued impacts often associated with public policy decisions. In Study 2, we replicated the findings from Study 1 in a nationally representative sample of the US public. In addition, we asked participants to place each of the valued impacts within its single most-dominant category among the five major divisions (environment, economy, society, culture, and health) used in most EIAs and risk analyses.

PCA.

PCA resulted in a two-factor solution for both Study 1 (eigenvalues were 2.50 and 1.42 accounting for 78.4% of the variance) and Study 2 (eigenvalues were 2.27 and 1.62 accounting for 77.9% of the variance). Table 2 shows factor loadings for each item on each extracted factor after varimax (orthogonal) rotation. The quality “Measurable” showed substantial cross loading on the components in both studies and was subsequently removed. This cross loading remained across a range of alternative PCA and factor analyses using alternative estimators (e.g., maximum likelihood) and rotations (e.g., Oblimin rotation). Principal Component (PC) 1 included three qualities relating to tangibility: the difficulty of understanding the value, the difficulty of observing changes in the value, and the difficulty of making changes to the value. PC 2 included two qualities relating to scope: the geographic area it refers to and whether it is typically a valued impact that applies to many people or to an individual.

Table 2.

PCA: Study 1 (Study 2)

Quality	PC 1: Tangibility	PC 2: Scope
1. Understanding: How difficult is it to understand the value?	0.92 (0.81)	−0.06 (−0.06)
2. Observable: How difficult is it to observe changes in the value when they occur?	0.90 (0.89)	−0.18 (0.11)
3. Stability: How difficult is it to make changes in the value?	0.92 (0.94)	−0.01 (−0.13)
4. Context: Does this value apply more to what I want as an individual or what I think is best for society?	−0.10 (0.03)	0.82 (0.89)
5. Geographic extent: Does the value apply to only a small area or to a large geographic area?	−0.02 (−0.10)	0.84 (0.86)

Standardized factor loadings are shown for Study 1 (Study 2).

Cluster Analysis.

K-means cluster analysis was used to cluster the valued impacts by these five qualities. Average Silhouette width suggested that two clusters were optimal for both Studies 1 and 2. Fig. 1 A and B shows the results of the K-means clustering (highlighted sections indicate observed clusters) superimposed on top of a scatterplot of each valued impact by the two PCs of tangibility and scope.

Fig. 1.

K-means cluster results are plotted with respect to the two PCs of perceived scope (y-axis) and tangibility (x-axis). Valued impacts clustering together are included in the shaded region. A and B show the results for Study 1 and Study 2, respectively.

With respect to H1 and H2, both studies showed that valued impacts varied with respect to the two attribute-based PCs. However, the identifiable clusters were more strongly distinguished by tangibility than for scope (reference separation of clusters along x-axis in Fig. 1 A and B). Hypothesis 3 was generally confirmed in that most, but not all, of the valued impacts we classified as social/cultural were in the harder to understand, observe, and change (less-tangible) cluster. In Study 1, jobs and safety/security were perceived as the most tangible whereas the environmental value climate change was rated the least tangible. A notable exception in the more tangible cluster was participation; government revenues and financial costs were included in the least tangible cluster. In Study 2, valued impacts we classified as social/cultural were again rated as less tangible and with a narrower scope. Notable exceptions included participation, community well-being, and reputation. Unlike Study 1, financial costs were in the more tangible cluster. Water quality, recreational opportunities, and aesthetics were perceived as the most tangible; government revenue, emotional health, and climate change were perceived as the least tangible.

Most of what are typically classified as social/cultural valued impacts were included within the harder to understand (less-tangible) cluster, but several other valued impacts were included in this cluster as well. For example, many environmental impacts and emotional health also were in this same cluster. Thus, there do seem to be some general distinctions made by people with respect to the qualities of different valued impacts, but the distinctions are relatively broad, and there is substantial between-impact variability within clusters. Overall, the results of this exploratory analysis support two classes or groupings of valued impacts: a first broad social/cultural/environmental class and a second economic/physical health/safety–security class.

Perceived Importance to Decision Makers and Public Support.

The ratings for perceived importance of a valued impact to decision makers and its frequency of inclusion in decisions were highly correlated in both Study 1 (r = 0.92) and Study 2 (r = 0.90). These two ratings were averaged to create a single composite score for each valued impact. Fig. 2 shows each of the PCs plotted against each of the outcomes of perceived importance to decision makers and public support. These results are displayed only for the nationally representative sample from Study 2, although results from Study 1were substantively identical. The top two panels show the relation between perceived tangibility and perceived decision maker and public support, respectively. Valued impacts considered to be less tangible were perceived to receive less public and decision maker support. This relation appears to be stronger for public support as evidenced by the relatively steeper slope and tighter 95% CI. The bottom two panels of Fig. 2 show the same relations but with perceptions of the scope of each valued impact instead of tangibility on the x-axis. There is a negative trend for both outcomes such that valued impacts perceived to be more societal/broad (i.e., larger scope) are perceived to receive more support from both the public and decision makers.

Fig. 2.

(Top) The relation between perceived tangibility and perceived importance to decision makers and perceived public support. (Bottom) The relation between perceived scope and perceived importance to decision makers and perceived public support. All valued impacts included in Study 2 are displayed.

In Hypothesis 4, we predicted a difference in how members of the public rate the importance of social and cultural valued impacts as compared to how they believe decision makers will rate them. The three impacts perceived to be most highly valued by decision makers were profits, financial costs, and government revenues, whereas for public support, the top three valued impacts were water quality, physical health, and safety and security. Table 3 shows each valued impact ranked by the standardized magnitude of the perceived difference between decision maker and public support. Positive (negative) numbers mean that perceived public (decision maker) support is higher. In both studies, nearly every valued impact was perceived to have more public support than decision maker support except for three of the economic impacts: government revenues, profits, and financial costs.

Table 3.

Standardized differences between average perceived public and decision maker support for each valued impact

Value	Study 1	Study 2
Emotional health	0.49	0.37
Physical health	0.50	0.59
Connection to nature	0.47	0.60
Safety/security	0.42	0.45
Recreational opportunities	0.42	0.61
Water quality	0.39	0.89
Aesthetics	0.36	0.34
Government revenue	−0.32	−0.51
Community well-being	0.34	0.43
Profits	−0.30	−0.81
Moral/ethics	0.26	0.49
Identity	0.26	0.26
Jobs	0.25	–
Participation	0.21	0.22
Equity	0.16	0.27
Reputation	0.14	0.07
Local practices	0.10	0.34
Financial costs	−0.09	−0.39
Native culture	0.07	0.16
Interpersonal conflict	0.06	0.15
Climate change	0.05	0.26
Biodiversity	0.01	0.30

The top 10 largest standardized differences are bolded. Positive (negative) differences indicate more (less) public support than decision maker support.

In Study 1, the top five valued impacts with respect to the divergence between perceived decision maker and public support (standardized difference values > 0.40) were related to health (emotional and physical), safety/security, connection to nature, and recreational activities. In Study 2, the magnitude of the divergence was even larger overall with nine valued impacts showing standardized differences larger than 0.40. In general, the order of the valued impacts divergence remained similar across studies. Overall, our prediction in Hypothesis 4 was supported although the divergence between perceived public and decision maker support was strongest for several of the health and environmental valued impacts. There was general agreement among participants that the impacts classified as social and cultural were either at the middle or lower end of decision maker and public support.

How the Lay Public Categorizes Valued Impacts.

These survey results are somewhat predicated on the a priori classification scheme that we applied to the valued impacts included in the study. Participants were never presented with these categories in the survey, but they did influence our thinking and interpretation of the results. A natural question that might arise is whether participants would generally share these classifications about the valued impacts they saw. To this end, in Study 2 we asked participants to place each of the individual valued impacts within one of the five identified categories, based on their understanding of which single category best describes their understanding of the valued impact.

As summarized in Table 4, results suggest that despite the sharp distinctions typically provided by impact assessment guidelines, many valued impacts do not fit neatly within a single category. All valued impacts were rated by at least some participants to fit best within each of the five major categories; this result speaks to the interconnected nature of the range of impacts under consideration. If a cutoff reflecting at least 15% of responses is adapted to identify the more important primary category, then fully 15 of the 21 valued impacts were identified as strongly belonging to more than one category. The exceptions (those valued impacts mapping onto only one category) are profits, government revenues, financial costs (economic), climate change and biodiversity (environment), and physical health (health). In addition, three of the valued impacts we thought would be viewed as primarily cultural were instead described as either primarily or equally social (identity, reputation, and morals and ethics); one that we thought would be viewed as primarily social was instead described as primarily economic (equity). Further, one valued impact we thought would be viewed as primarily environmental was instead described as primarily social (recreation), and two we thought would be viewed as primarily relating to health instead were described as primarily social (safety and security) or social and environmental (aesthetics).

Table 4.

Participants’ categorization of impacts (%), Study 2

	Economic	Ecosystem/environmental	Social/family	Cultural	Health
Community well-being	17.1	9.3	45.5	13.3	14.7
Interpersonal conflict	9.0	14.7	48.8	22.9	4.6
Safety and security	10.8	14.2	39.5	14.3	21.1
Local practices	13.2	23.5	30.1	30.8	2.4
Equity	37.7	8.1	32.8	18.9	2.6
Identity	6.3	8.3	49.1	32.3	4.0
Government revenues	75.4	7.3	5.8	6.8	4.7
Climate change	9.1	76.0	4.6	4.2	6.1
Recreational opportunities	6.4	28.2	42.3	7.2	16.0
Water quality	8.7	47.4	6.4	3.0	34.5
Reputation	13.8	10.0	42.4	31.5	2.3
Biodiversity	8.7	64.2	10.2	13.3	3.6
Financial costs	73.4	8.5	11.5	3.9	2.7
Physical health	3.0	8.2	11.1	3.3	74.4
Morals and ethics	3.3	4.6	45.5	45.9	0.8
Emotional health	2.1	4.1	30.9	7.5	55.4
Native culture	3.0	8.7	17.8	67.2	3.3
Profits	80.9	4.2	9.7	3.2	2.1
Aesthetics	14.9	30.3	11.1	32.4	11.3
Connection to nature	3.9	65.6	15.9	11.7	2.9
Participation	13.6	3.6	50.2	28.7	3.9

These categorization results are particularly striking when comparing social and cultural valued impacts to economic impacts. All five of the primarily cultural and all four of the primarily social valued impacts were identified as being important to more than one category. In contrast, all three economic valued impacts were identified as important to only the one category.

General Discussion

For some time, community and environmental advocates, along with academic researchers, have called on elected officials to give increased attention to the social and cultural effects of large-scale development projects. Over two decades ago, the influential report “Understanding Risk” stated that “Risk characterizations typically do not address social effects, perhaps they are considered outside the purview of formal risk analysis. Yet they are legitimate objects for risk characterization because participants in decisions need to understand them to make informed decisions” (23). Over the past quarter century, several studies by the US National Research Council have emphasized the point that social impact assessments should identify what matters to potentially affected parties, even if these impacts are difficult to quantify (9, 35). To the best of our knowledge, however, there has been little basis for understanding the nature of these concerns and few empirical studies examining people’s perceptions of the fundamental nature of different valued impacts or the range of plausible dimensions on which these values may vary.

Our results shed light on how the lay public on average perceive the various qualities or attributes of different valued impacts commonly considered in risk and impact assessments, including effects on the environment, the economy, health, and important social and cultural phenomena. The use of a psychometric rating approach showed that public perceptions of a range of valued impacts did vary by perceived tangibility and scope, with two broad clusters primarily distinguished by tangibility. As expected, most of the social/cultural valued impacts were located within the less-tangible cluster; several environmental and health impacts were perceived to be in the same cluster. Overall, consistent distinctions were made with respect to value qualities, but the distinctions are relatively broad, and there is substantial variability within clusters.

At the level of individual valued impacts, tangibility and scope ratings were related to perceived support in expected ways: impacts thought to be less tangible and more narrow in scope were perceived to have less support by both decision makers and the public. Examples that were very low in perceived decision maker support include identity, native communities, emotional health, connection to nature, and biodiversity. From the standpoint of public perceptions, native cultural concerns were rated as the lowest in perceived support followed by biodiversity, identity, government revenue, and connection to nature.

With reference to our primary research question, “Are social/cultural valued impacts viewed as fundamentally different with respect to tangibility and scope and could this act as an initial lens through which people perceive these valued impacts?” The answer appears to be yes, although these valued impacts are not entirely unique in that several other environmental and even economic (e.g., equity) valued impacts share the same space. These results thus provide evidence supporting the widespread public perception that decisions made by public officials concerning many large projects fail to reflect concerns about a range of impacts including social, cultural, psychological health, and environmental considerations. This general conclusion remains when considering the categories that participants assigned to the valued impacts. However, perceptions of tangibility and scope cannot entirely explain the observed underweighting of social and cultural valued impacts in evaluations, because if so, other valued impacts that occupy the same space (e.g., biodiversity) should also be perceived as underweighted. What the results do imply is that valued impacts judged to be less tangible and of more narrow scope, regardless of how they are categorized, are perceived to be given less support by both decision makers and the public.

A further implication of these findings pertains to the legitimacy of deliberations with stakeholder groups. The need to conduct high-quality deliberations and improve the integration of public concerns into formal analysis and assessment is a long-standing requirement underlying sound practice (9, 36, 37). This shift requires proper representation of public concerns in decisions, yet that goal remains compromised—particularly with respect to analyses capable of reflecting less-tangible impacts or those more narrow or local in scope. Our results emphasize that paying particular attention to the nature and presentation of the full range of valued impacts under consideration remains paramount and support the need for clarity and capacity-building on the part of analysts, facilitators, and decision makers so that those engaged in public deliberations can be helped to understand the role of less-tangible impacts and agree on a transparent approach to including them fully as part of project analyses (38). This is increasingly the case, for example, when valuing cultural ecosystem services in reference to aesthetic and spiritual value (39).

Additional context-specific work may be needed to describe how an impact such as degradation of key cultural sites or protection of biodiversity should be operationalized and measured in light of stakeholders’ initial beliefs. This is particularly important due to the labile nature of tangibility as an underlying factor, because considerations such as the vividness or salience of a value can be influenced by the actions of a group facilitator or designer of a survey as well as by recent media coverage or current events (23). A similar cautionary message regarding presentation and communication applies to valued impacts that may commonly but erroneously be assumed to be more tangible and easily expressed, as has been found in the wide-ranging definitions given to values such as safety and national security (40). In such cases, additional work examining disagreements in the understanding of impacts among decision makers can prove both insightful and essential.

These descriptive results also point to areas of possible disconnect between decision makers and the public at large regarding what potential impacts should be most highly valued when making important public policy decisions. Future research should examine the extent of these normative differences and their implications for public policy decisions.

The interdependent nature of many valued impacts is also an important practical consideration. For example, our results suggest that an adverse effect on emotional health could also result in negative impacts on social and family considerations, or a decline in community well-being (primarily a social value) could also negatively affect economic values. Similar observations are now appearing in the work of other researchers concerned about impacts on the well-being and quality of life; for example, recent empirical work by the economists Case and Deaton (41) confirms strong linkages between physical and mental health and both social and economic values. In addition, our results suggest that many values generally classified as belonging to a single category may be nearly as important for other reasons; recreational opportunities, for example, contribute to social impacts but also affect economic and cultural values, whereas community well-being is perceived to contribute important social, economic, health, and cultural benefits. Greater attention to the cross-category nature of valued impacts is needed to help ensure that important dimensions of value are not neglected as part of project or program evaluations.

The policy relevance of these exploratory results is limited to the extent that they may reflect the survey descriptions, the language used to characterize results, and the specific valued impacts and qualities that were selected. The research design also assumes that people are able to answer a number of questions about topics, and using language, with which they may not be familiar. For example, the lack of consensus in categorizing impacts (Table 4) might partially be explained by random responses due to some participants not being familiar with categorizing impacts in this way. In addition, we only surveyed nonexpert members of the public in a general-impact setting, and a different pattern of results could have emerged if we had focused on expert decision makers, members of highly vulnerable communities, or examined specific decision contexts. Finally, the psychometric approach we employed focuses on average responses across participants only. Future research should examine the nature and magnitude of individual differences in the perceptions of important valued impacts.

With these limitations in mind, future research should focus on further documenting public and expert perceptions of various valued impacts, which could provide theoretical support for more accurate impact assessment procedures. Equitable treatment of values across impacts will only be possible when there exists a better understanding of the views of minority and other communities that might experience social and cultural impacts differently from the general population (42). Future work should also seek to develop methods that can be used by analysts and decision makers to improve how the impacts of activities are characterized and communicated so as to encourage more informed public debates and more comprehensive assessments of the effects of public policy initiatives on citizens and society.

Data Availability

Survey data have been deposited in Open Science Framework (https://osf.io/PGQ62) (43).

Change History

June 15, 2021: Table 1 has been updated.