Testing Map Features Designed to Convey the Uncertainty of Cancer Risk: Insights Gained From Assessing Judgments of Information Adequacy and Communication Goals

Abstract

Barriers to communicating the uncertainty of environmental health risks include preferences for certain information and low numeracy. Map features designed to communicate the magnitude and uncertainty of estimated cancer risk from air pollution were tested among 826 participants to assess how map features influenced judgments of adequacy and the intended communication goals. An uncertain versus certain visual feature was judged as less adequate but met both communication goals and addressed numeracy barriers. Expressing relative risk using words communicated uncertainty and addressed numeracy barriers but was judged as highly inadequate. Risk communication and visual cognition concepts were applied to explain findings.

Mathematical models are used to estimate health risks from environmental hazards and, increasingly, maps are used to communicate such estimated risk information to the public. For example, the U. S. Environmental Protection Agency (EPA) National Air Toxic Assessment program uses models to estimate cancer risk from air pollution and then illustrates the estimates on maps (U.S. EPA, 2011a). The intended use of the maps is to identify geographic areas that warrant further investigation—not to assess personal cancer risk (U.S. EPA, 2011b). The uncertainty inherent in modeled risk estimates (Marks, Coleman, & Matthew, 2003) presents a communication challenge because people have difficulty comprehending uncertainty and typically prefer certain to uncertain information (Han et al., 2009; Han, Moser, & Klein, 2006; Johnson & Slovic, 1995; Longman, Turner, King, & McCaffery, 2012). To address this challenge cartographers advise incorporating uncertainty into the map (Bostrom, Anselin, & Farris, 2008; MacEachren, 1992; MacEachren et al., 2005).

Previous findings showed that several map features led to more perceived ambiguity (Severtson & Myers, 2013). However, the analysis did not examine participants’ subjective evaluations of the information as recommended by risk communication researchers (Bostrom et al., 2008; Lipkus, 2007). The purpose of the current study was to address this gap by testing how the certainty of map features influenced participants’ subjective evaluations of information adequacy (hereafter, adequacy judgments) and then by assessing how these judgments related to risk beliefs and perceived ambiguity. This study used previously unanalyzed data on adequacy judgments that was collected as part of the Severtson and Myers (2013) study.

Theoretical Concepts

Uncertainty

Scientific uncertainty is inherent in the risk assessment process used to generate estimates of health risk from environmental hazards (National Research Council, 1994). Scientific uncertainty generally pertains to unpredictability or inadequacy of knowledge (Palmer, 2011). Some scholars study ambiguity, described as the lack of reliability, credibility, or adequacy of information (Ellsberg, 1961; Han, Klein, & Arora, 2011). Both terms (uncertainty, ambiguity) are used in this paper in keeping with the literature.

A body of evidence shows people are averse to ambiguity which is typically manifested in negative emotions, increased risk perception, and the avoidance of decision making (Ellsberg, 1961; Han et al., 2011). Typically, communicating the uncertainty of estimated health risk reduces both comprehension and the perceived credibility of the message (Han et al., 2009; Longman et al., 2012). Images may enhance comprehension of uncertainty. In one experiment, Johnson and Slovic (1995) found that an image appropriately increased the amount of uncertainty recognized by participants compared to the alphanumeric format, although generally decreased trust in the information. Evidence from the previous study suggests that uncertain map features generate ambiguity in beliefs about the risk (Severtson & Myers, 2013).

Visual cognition

Concepts from the field of visual cognition explain how images can support comprehension and shape beliefs about health risks (Severtson, 2013; Severtson & Vatovec, 2012). Concepts that pertain to this study are top-down and bottom-up information processing, iconicity, unit of perception, visual complexity, and visual discrimination.

Information influences cognition via top-down and bottom-up processes. Top-down processing is consciously driven by the person—for example, purposively using a map to answer a question. Bottom-up processing of images is unconscious because stimuli detected by the retina are directly linked to cognitive processing areas in the brain (Pinker, 1990). This physiological linkage explains how seeing an image can influence derived meaning without conscious thought.

Images are comprised of perceptual units that are visually represented by features such as shape and color. Seeing perceptual units leads to bottom-up comprehension (Pinker, 1990). Severtson and Vatovec (2012) describe the unit at which information is displayed on the map as the unit of perception. For example, the unit of perception in Figure 1a is a map contour and in Figure 1b is a fuzzy blur of shaded color. Evidence suggests the unit of perception can substantially influence the meaning derived from a map (Severtson & Vatovec, 2012).

Figure 1.

Shows two map portions and two legends to illustrate the three map features that were tested (see 1a – 1d). Each map depicted only one of the four assigned locations. All maps used incrementally darker color shades to depict incremental risk levels. Risk levels depicted here (black text) were not provided on study maps. For the print version of the paper, lower to higher levels of estimated cancer risk appear as lighter to darker shades of gray. For the electronic version, Figure 1 is in color and available via e-mail from the author.

1a. Focused contour map with three incrementally darker colors (yellow, light green, darker green, dark blue).

1b. Unfocused contour map with one incrementally darker color (shades of blue).

1c. Legend with numeric risk expression for map with three colors and unfocused contours.

1d. Legend with verbal-relative risk expression for map with one color and focused contours.

Concepts from the field of semiotics, the study of signs, also explain how images can convey meaning. A sign refers to something other than itself. Iconic signs (iconicity) support comprehension by resembling familiar objects or ideas (Chandler, 2007), such as depictions of lakes and rivers that resemble their shape, spatial location, and idealized blue color. Robinson and Petchenick (1976) propose that map symbols fall on a continuum of iconicity with arbitrary symbols on the non-iconic end and pictorial icons that resemble real-world objects on the highly-iconic end. Resemblance supports comprehension because the meaning is inherent in the iconic symbol. This decreases the need to consult the legend in order to understand the symbol’s meaning, which facilitates comprehension (Robinson & Petchenik, 1976). Features proposed to foster bottom-up comprehension of magnitude (Cleveland & McGill, 1985) iconically resemble magnitude to some extent, such as incremental shading or incremental length to convey incremental magnitude (Severtson & Myers, 2013). Cartographers recommend using incrementally darker color shades on maps to represent incremental magnitudes of a phenomenon (Brewer, 2006).

To accurately understand magnitude for a given map location, people must be able to match color shades on the map to the color shades defined in the legend (Breslow, Trafton, & Ratwani, 2009). People can accurately discriminate about 5–7 shades of a single color. Using additional colors increases visual differences among color shades (visual discrimination), which facilitates matching map and legend color shades to accurately comprehend magnitude (Breslow et al., 2009; Brewer, 2006).

Visualizing map information as “out of focus” is proposed as ideal for representing uncertainty (MacEachren, 1992). The appearance of map contours as focused or unfocused can be produced by choosing how the map data is classified into categorical ranges. Classifying map data into many categorical ranges (many data classes) or not classifying the data at all (unclassed map) generates maps with soft seamless gradations of shaded color with no discernable contour boundaries that appear unfocused (as in Figure 1b with 32 classes). Classifying map data into few categorical ranges with one color shade per contour generates a map with well-defined contour boundaries that appears focused (Figure 1a). Using many data classes represents incremental magnitude more precisely than fewer classes, but requires more color shades which decreases visual discrimination and the ease of matching map shades to the legend (Breslow et al., 2009; Harrower, 2007).

Given that map contours appear focused and unfocused without the need for conscious thought, it can be argued that this feature is processed bottom-up. Given that focused and unfocused contours have some resemblance to certainty and uncertainty, contour focus is proposed as an iconic method for representing uncertainty on maps (Severtson & Myers, 2013).

Extrinsic features are added to convey additional information, for example adding a symbol to convey uncertainty in addition to a symbol that conveys the magnitude of risk levels. On the other hand, intrinsic features are integrated into the map (Howard & MacEachren, 1996). Using many data classes builds uncertainty into the map and therefore generates a unit of perception (unfocused shaded contours) that intrinsically conveys uncertainty concurrently with incremental magnitude. This decreases the visual complexity of the map. Less visual complexity supports ease of comprehension (Cutter, 2008; Florence & Geiselman, 1986).

Risk expression

In a map legend, estimated risk can be expressed as (a) absolute risk or relative risk and (b) by using words or numbers. Absolute risk is the risk of a consequence over a time period and is optimally expressed as a simple frequency over a common denominator (Trevena et al., 2013) as in Figure 1c. Relative risk, a comparative risk expression, conveys the idea of “more” or “less” risk. Providing a baseline value, a threshold such as a safety standard, or an evaluative label with the risk expression improves meaningful comprehension (Lipkus, 2007; Trevena et al., 2013; Weinstein, Sandman, & Miller, 1991). Generally, people prefer numerical over verbal risk information (Wallsten, Budescu, Zwick, & Kemp, 1993), and a numerical risk expression is interpreted more accurately than a verbal risk expressions (Stacey et al., 2014). Relative risk tends to heighten perceived risk, and when provided without a baseline, threshold, or evaluative label as in Figure 1d, it is highly ambiguous (Koehler, 1996).

Adequacy judgments

Risk information provided to public audiences should be “credible, accurate, useful, relevant, comprehensive, trustful, clear, and easy to understand” (Lipkus, 2007, p. 700). Lipkus (2007) recommended assessing several of these characteristics to assess the degree to which adequacy judgments explain the influence of risk information on outcomes. Measures of adequacy judgments vary across studies.

Ease of understanding is typically assessed using objective measures of comprehension. Interestingly, participants sometimes prefer a format that results in poorer comprehension compared to less preferred formats (Elting, Martin, Cantor, & Rubenstein, 1999; Gerteis, Gerteis, Newman, & Koepke, 2007; Hildon, Allwood, & Black, 2011). No published studies were found that assessed perceived ease of understanding, a notable gap because people may avoid using information perceived as difficult to comprehend (Fortin, Hirota, Bond, O’Connor, & Col, 2001).

Perceiving information as credible, trustworthy, and/or accurate is expected to foster message adoption and resulting changes in perceived risk and intentions to engage in protective behavior (Lipkus et al., 1999; Schapira, Nattinger, & McAuliffe, 2006). However, evidence of these proposed influences is mixed. While Lipkus et al. (1999) found no association between risk beliefs and these adequacy judgments, Dieckmann, Slovic, and Peters (2009) found that participants who rated risk information as more credible tended to have beliefs of greater risk.

Numeracy

Numeracy is the ability to understand probability and mathematical concepts and influences the comprehension of risk information (Nelson, Reyna, Fagerlin, Lipkus, & Peters, 2008). Objective numeracy is assessed using math problems. Subjective numeracy is assessed using perceived numerical ability and preferences for numerical information. Generally, greater objective or subjective numeracy relates to more appropriate risk beliefs (Nelson et al., 2008). Numeracy also influences comprehension and adequacy judgments. In one study, images improved comprehension for participants with low subjective numeracy, but did not for participants with high subjective numeracy (Hawley et al., 2008). In another study of hypothetical risk information provided by a physician, objective numeracy related positively to trust. Generally, participants trusted numerical risk estimates more than verbal risk estimates. However, the few participants with the lowest objective numeracy scores judged verbal information as more trustworthy (Gurmankin, Baron, & Armstrong, 2004).

Risk beliefs

Beliefs about a risk reflect the gist of how people comprehend the meaning of risk information (Reyna, Nelson, Han, & Dieckmann, 2009). Common dimensions of risk beliefs in health behavior theories include perceptions of (a) susceptibility to a health threat, (b) severity of health consequences, and (c) whether the threat is a serious problem (Weinstein, 1993).

Weinstein and Sandman (1993) propose that effective risk communication should foster risk beliefs that are consistent with the dose or level of the hazard. Effectiveness is indicated when incrementally higher risk levels foster incrementally greater risk beliefs.

Specific Aims

Specific aims were to assess (1) how map features influence adequacy judgments; (2) how adequacy judgments relate to risk beliefs; (3) how adequacy judgments relate to perceived ambiguity; (4) how subjective numeracy relates to adequacy judgments and whether subjective numeracy moderates the influence of map features on adequacy judgments; and (5) whether map features moderate the consistency between risk levels and risk beliefs. Hypotheses (H) and research questions (RQ) are outlined using numbers (H1, H2… RQ5) aligned with the five numbered study aims.

• H1. Given people’s preference for certain information: Certain features will result in judgments of greater adequacy than uncertain features.

• H2. Given the assumption that judgments of greater adequacy will foster more consistency between risk levels and risk beliefs than judgments of lessor adequacy: Judgments of adequacy will relate positively with risk beliefs at higher risk levels, and negatively with risk beliefs at lower risk levels.

• H3. Given that lower adequacy judgments are expected to foster more perceived ambiguity: Judgments of adequacy will relate negatively with perceived ambiguity.

• H4a. Given that maps are conveying risk estimates which is a numerical concept: Subjective numeracy will relate positively with perceived understanding (map, legend, overall).

• RQ4a. How does subjective numeracy relate to perceived accuracy and trust in validity?

• H4b. Given that the numerical risk expression used numbers and the verbal-relative risk expression did not: Subjective numeracy will moderate the influence of risk expression on understanding (map, legend, overall). Subjective numeracy will relate positively with understanding for the numerical risk expression but not the verbal-relative risk expression.

• RQ4b. Does subjective numeracy moderate the influence of risk expression on perceived accuracy and trust in validity?

• H4c. Given that subjective numeracy is not expected to relate to the perceived ease of understanding non-numerical visual information: Subjective numeracy will not moderate the influence of the visual map features (color or contour focus) on understanding.

• RQ4c. Does subjective numeracy moderate the influence of map features on perceived accuracy and trust in validity?

• RQ5. Do map features moderate the consistency between risk levels and risk beliefs?

Methods

Study Design and Experimental Maps

Previously unanalyzed data on adequacy judgments, collected as part of the Severtson and Myers (2013) study, were used to address study aims. The full factorial 2 × 2 × 2 × 4 study (resulting in 32 study maps) was designed to test how three dichotomous map features across four cancer risk levels influenced dependent variables. Map features (uncertain vs. certain) were (a) appearance of map contours (unfocused vs. focused); (b) number of colors (one vs. three); and (c) how risk was expressed in the legend (verbal and relative without evaluative labels vs. a numerical simple frequency with evaluative labels). Certainty was not referred to on study maps, nor on the maps of estimated cancer risk provided by the U. S. EPA (2011a). All maps used incrementally darker color shades to represent incremental magnitude. Figure 1 illustrates map variables.

Each study map depicted one assigned “You live here” location within one of the four risk level areas depicted in Figure 1. The 32 study maps, organized into eight blocks with four maps in each, were provided in reverse order to control for ordering effects, for a total of16 blocks. Four map blocks are available at http://tinyurl.com/mkjgmq9.

Study Sample and Procedures

Over two semesters, about 3,300 undergraduate students at the University of Wisconsin-Madison (UW-Madison) were invited to participate in this online survey study via a course website in the psychology department or by verbal invitation to students in three nursing classes. Students could choose when and where to complete the online study and received extra credit for participation. Students entered their unique campus login and were assigned to one of the 16 map blocks in rotating order to achieve random block assignment. Participants answered map-related survey items as they viewed the four maps in their assigned block. Since UW-Madison is at the center of the map, most participants likely resided in the highest cancer risk area.

Survey Measures

Risk beliefs and perceived ambiguity

Risk beliefs were assessed with ten survey items and emotion with one item. Participants were instructed to answer items based on their assigned map location. Unless otherwise noted, items used a 6-point scale from strongly disagree-strongly agree. Three risk beliefs about air pollution (is a serious problem, exposed to unsafe amounts, good chance of having related health problems) were assessed at a neighborhood level and also assessed at a personal level for a total of six survey items. A seventh item assessed the perceived severity of air pollution-related health effects. The last three risk beliefs were assessed at a personal level and included perceived: safety of outdoor air (6-point scale: very safe-very unsafe), the chance of developing air pollution-related health problems (11-point scale: 0%, 10%, 20%, … 100%), and the amount of air pollution-related risk from no risk to very high risk (11-point scale: 0–10). Perceived ambiguity (10 items) was assessed by inserting the phrase “It is hard to know” at the beginning of each risk belief item (Han et al., 2006), for example “It is hard to know whether air pollution is a serious problem for me.” (6-point scale strongly disagree-strongly agree). Survey items are in the Appendix of Severtson and Myers (2013).

Adequacy judgments

Adequacy judgments reflect two domains: perceived ease of understanding, and facets of uncertainty emphasized by scholars and the EPA (accuracy, validity, trustworthiness) (Han et al., 2011; National Research Council, 1994). Three survey items assessed perceived ease of understanding for the (a) map portion, (b) legend, and (c) information overall. Two items assessed (d) perceived accuracy of the risk estimates and (e) trust in the validity of the information. Hereafter, these adequacy judgments are referred to as map understanding, legend understanding, overall understanding, perceived accuracy, and trust in validity. In addition, perceived adequacy of the size of the text in the legend and ease of understanding the map title were assessed. All items used a 6-point scale (strongly disagree-strongly agree). For each block of four maps, adequacy judgments were assessed for only the first two maps in order to minimize testing effects. Per the block design, participants first assessed an unfocused map and then a focused map. As maps were in reverse order, adequacy judgments were assessed for all 32 maps.

Prior beliefs and personal characteristics

To control for prior beliefs about air pollution, four survey items assessed beliefs based on participants’ current residential location: the safety, appearance, and smell of outdoor air (three items); and skepticism about air pollution-related health effects. An example is “In your opinion, what is the overall safety of outdoor air where you currently live as a UW-Madison student?” using a 6-point scale (very safe-very unsafe). Prior perceived ambiguity was assessed with the item “It is hard to know the overall safety of outdoor air where I currently live as a UW-Madison student” using a 6-point scale (strongly disagree-strongly agree). Numeracy was assessed using the eight-item subjective numeracy scale (Fagerlin et al., 2007) that strongly correlates with objective measures of numeracy, but is more user-friendly (Zikmund-Fisher, Smith, Ubel, & Fagerlin, 2007). The scale consists of four items that assess an individual’s perceived ability to work with fractions and percentages and four items that assess preferences for numerical information. Other covariates were gender, cancer in the family, and area of academic study.

Data Analysis

PAWS Version 18 (SPSS Inc, 2009) was used to conduct an exploratory factor analysis of variables followed by a confirmatory factor analysis using M-plus Version 5.1 (Muthen & Muthen, 2011). The analyses produced latent measures of prior beliefs (air safety, smell, appearance), numerical preference (four items), perceived numerical ability (four items), risk beliefs (all ten risk belief items and the single emotion item), and perceived ambiguity (all ten ambiguity items). Single-item variables included prior skepticism and prior perceived ambiguity.

Structural equation modeling (SEM) using M-plus was used to address hypotheses and research questions. Independent variables were usually the four manipulated map variables (contour focus, color, risk expression, risk level), and numerical preference. SEM outcomes were risk beliefs and perceived ambiguity. Each SEM included one measure of adequacy judgment. Most SEMs used numerical preference because it related more to adequacy judgments than perceived numerical ability. However, perceived numerical ability was used to address aim four. Covariates were gender, prior beliefs, and map direction because family cancer history and students’ area of study did not relate to adequacy judgments. Participants were nested because each participant answered survey items for two maps. Significance was set at p < .05. The SEMs for aims two and three were analyzed for each risk level (see Figure 2).

Figure 2.

Illustrates the model with map understanding for risk level four. Paths illustrate standardized SEM coefficients at +p< .10, *p< .05, **p< .01, ***p< .001 using shading and outline symbols to facilitate matching coefficients to independent variables. Rectangles with square and rounded corners respectively represent single indicator variables and latent CISE variables. Covariates of gender, prior risk beliefs, and map direction were included in the SEM but not illustrated here. Model fit indices were: SRMR = 0.042, RMSEA = 0.079 [0.057; 0.103], and CFI = 0.881 (N = 389 observations).

To increase statistical power for subgroup analyses, a composite indicator structural equation (CISE) approach was used to create a single composite latent variable for each of the latent variables (McDonald, Behson, & Seifert, 2005). The CISE variables were the mean of the items contributing to the latent variable with an assigned estimate of measurement error based on Cronbach’s alpha for the contributing variables using the formula, (1−α) ÷ the variance of the composite mean. Testing indicates this approach addresses Type I and Type II error conditions (McDonald et al., 2005).

Interactions between risk level and map features on risk beliefs were assessed to determine whether map features moderated the consistency between risk levels and risk beliefs. The SEMs for this analysis did not include adequacy judgments; thus data for all four study maps were used. When standard errors are similar, non-overlapping 84% confidence intervals (CIs) approximate a p < .05 test of significance (Payton, Greenstone, & Schenker, 2003). As such, 90% CIs support statements about differences between SEM coefficients.

Regarding the general magnitude of reported effect sizes, Kline (1998) cited Cohen (1988) to conclude that standardized SEM or path coefficients indicate the magnitude of effects with small effects less than or around 0.10, medium effects around 0.30, and large effects over 0.50.

Results

Of approximately 3,300 students invited to participate in the study over two semesters, 989 students (30%) logged into the website, 863 students (26%) entered data for at least one of the first two study maps, and 826 (25%) entered at least 85% of the data for one of first two study maps. The low response may have occurred because the psychology students (about 2,800) could choose from numerous research study options, the study was closed mid-semester with another study offered in its place, and a few students commented that answering survey questions for four maps was ‘tedious.’ Sample characteristics (n=826) were: 66.3% female, 32.9% reported a family cancer diagnosis, and the following majors: 1% environmental studies, 44% a health-related major, and 55% in another major or undeclared. Mean values with standard deviations indicated (6-point scale) a prior belief that air pollution-related health risk was “slightly” not exaggerated [2.92(1.054)]; prior beliefs of safe air [2.52(0.705)]; mid-level ambiguity about air safety beliefs [3.50(1.087)]; a preference for numbers [4.51(0.812)], and perceptions of moderate numerical ability [4.34(1.115)].

Non-overlapping 90% CIs suggest mean ease of understanding was less for the map than the legend or overall (Table 1). Mean trust in validity was greater than mean perceived accuracy, and both were less than means for the understanding variables.

Table 1.

SEM Coefficients for the Influence of Map Features, Subjective Numeracy, and Interactions (Numeracy by Map Feature) on Adequacy Judgments; with Explained Variance and Mean Values for Adequacy Judgments

		SEM Coefficients(SE) on Adequacy Judgments (standardized for main effects and unstandardized for interaction effects)
Independent Variables		Map Undb	Legend Und	Overall Und	Perceived Accuracy	Trust in Validity	Und Title
Perceived numerical abilitya	γ (SE)	.116*** (.036)	.148*** (.038)	.109** (.038)	.009 (.037)	.043 (.038)	.110*** (.031)
Numerical preferencea	γ (SE)	.196*** (.041)	.216*** (.040)	.242*** (.040)	.032 (.042)	.073 (.044)	.134*** (.037)
Color	γ (SE)	−.117+ (.062)	.013 (.062)	−.014 (.063)	.037 (.063)	0 (.066)	−.040 (.053)
Contour focus	γ (SE)	.305*** (.033)	.278*** (.031)	.224*** (.029)	.218*** (.030)	.141*** (.025)	.049 (.046)
Risk expression	γ (SE)	.045 (.061)	.058 (.061)	.121* (.061)	.362*** (.060)	.303*** (.064)	−.001 (.053)
Risk level	γ (SE)	−.009 (.021)	−.048* (.019)	−.027 (.020)	−.029 (.020)	−.020 (.019)	−.013 (.025)
Legend text too small	γ (SE)	−.078* (.031)	−.104*** (.031)	−.115*** (.032)	−.006 (.032)	−.013 (.033)	−.022 (.028)
Direction (covariate)	γ (SE)	.063 (.062)	.096 (.062)	.192** (.062)	.071 (.063)	.015 (.065)	.317*** (.040)
Interaction Effects
NumAble* Colorc	γ (SE)	.045 (.062)	0 (.062)	−.004 (.060)	.107+ (.064)	.124* (.062)	.037 (.069)
NumAble * Contour focusc	γ (SE)	.058 (.063)	.017 (.063)	.033 (.061)	.072 (.064)	−.003 (.062)	.076 (.072)
NumAble * Risk expressionc	γ (SE)	.206*** (.062)	.268*** (.061)	.171** (.060)	.253*** (.064)	.128* (.062)	.060 (.069)
NumPref* Risk expressionc	γ (SE)	.131 (.116)	.183+ (.112)	.111 (.109)	−.043 (.098)	−.017 (.098)	.054 (.136)
Adequacy Judgments		Model R2 and Mean with 90% CI for Adequacy Judgments
Model with perceived numerical ability	R2	.049*** (.012)	.064*** (.015)	.055*** (.013)	.059*** (.014)	.043*** (.013)	.016* (.008)
Model with numerical preference	R2	.074*** (.017)	.087*** (.018)	.100*** (.021)	.060*** (.014)	.046*** (.013)	.022* (.010)
Mean Values and 90% CI	M	4.33 [4.28,4.37] N=1608	4.59 [4.54,4.63] N=1608	4.50 [4.45,4.54] N=1605	3.74 [3.69,3.79] N=1613	3.91 [3.87,3.96] N=1615	4.74 [4.69,4.79] N=1615

^aOnly one “numeracy” variable was in the model.

^bFit indices for the model with numerical preference and understand maps were: CFI = .898, RMSEA = .059(ns), SRMR = .036. Fit indices for other models were the same or similar (available from the author).

^*p < .05,

^**p < .01,

^***p < .001

^cNumAble = perceived numerical ability, NumPref = numerical preference

Partial correlations were large (> .50): (a) among adequacy judgments of understanding (map, legend, overall) and (b) between perceived accuracy and trust in validity (Table 2). Overall understanding was correlated more with understanding the legend than the map. Among understanding variables, overall understanding was most correlated with perceived accuracy and trust in validity. Perceived accuracy versus trust in validity correlated more strongly with understanding variables.

Table 2.

Partial Correlations Among Adequacy Judgments, Numerical Preference, and Perceived Numerical Ability Controlling for Gender, Prior Beliefs, and Map Direction

	MU	LU	OU	Acc	TV	UT	NA
Legend Understanding (LU)	.525
Overall Understanding (UO)	.662	.730
Perceived accuracy (Acc)	.350	.437	.487
Trust in validity (TV)	.318	.391	.441	.698
Understand title (UT)	.296	.323	.335	.149	.182
Perceived numerical ability (NA)	.104	.136	.096	−.009ns	.026ns	.098
Numerical preference	.152	.170	.186	.011ns	.046ns	.104	.415

MU is map understanding. All correlations were significant at p < .001 (two-tailed) unless otherwise noted (ns = non-significant). Degrees of freedom ranged from 1540–1555.

Aim 1. How do map features influence adequacy judgments?

The hypothesis (H1, Table 1) that more certain features would result in greater perceived understanding was fully supported for contour focus, partially supported for risk expression, and not supported for color. Focused versus unfocused contours had small to medium positive effects on each of the five adequacy judgments. The numerical versus the verbal-relative risk expression had medium positive effects on perceived accuracy and trust in validity and a small positive effect on overall understanding. Number of colors had a small negative effect on map understanding, opposite to the hypothesized influence. Judging the legend text as too small had small negative effects on perceived ease of understanding (map, legend, overall). Map features did not influence perceived understanding of the map title. The influence of interactions between map features on adequacy judgments was assessed, but all were non-significant (results available from author). Of the five adequacy judgments, explained variance was greatest for overall understanding and least for trust in validity for SEMs with numerical preference.

Aim 2. How do adequacy judgments relate to risk beliefs?

Disregarding statistical significance, greater ease of understanding was related to beliefs of more risk at higher risk levels and beliefs of less risk at lower risk levels (Figure 3). Understanding (map, legend, overall) had medium effects on risk beliefs at risk level one (RL1) and RL4, but coefficients at RL2 and RL3 were small and usually non-significant, thus H2 was only supported at RL1 and RL4. Perceived accuracy and trust in validity had medium positive effects on risk beliefs at RL3 and RL4, but coefficients at RL1 and RL2 were small, positive, and non-significant. Thus, H2 was supported at higher but not lower risk levels.

Figure 3.

Shows standardized SEM coefficients for the influence of user ratings on risk beliefs at risk levels 1 – 4. “Und” denotes Understanding. For Und Map, the three largest coefficients were significant at p< .05. For the other adequacy judgments, the largest two were significant.

Aim 3. How do adequacy judgments relate to perceived ambiguity?

Less understanding, perceived accuracy, and trust in validity related to more perceived ambiguity at each risk level, fully supporting H3 (Figure 4). For combined risk levels, standardized coefficients for the effects of understanding on perceived ambiguity (low to high) were: map understanding (−0.173, 90% CI [−0.222, −0.124]); legend understanding (−0.224 [−0.274, −0.173]); and overall understanding (−0.294, [−0.342, −0.246]), 90% CIs indicated larger effects for overall understanding than map understanding. Perceived accuracy (−0.292, [−0.340, −0.245]) related more to perceived ambiguity than did trust in validity (−0.183, [−0.236, −0.129]).

Figure 4.

Shows standardized SEM coefficients for the influence of adequacy judgmentson perceived ambiguity at risk levels 1 – 4. All coefficients were significant at p< .05 and most were significant at p< .001. “Und” denotes Understanding.

Aim 4. How does subjective numeracy relate to adequacy judgments?

Recall from the Data Analysis subsection that two latent variables for subjective numeracy were used in the analysis. Numerical preference and perceived numerical ability related positively to understanding (map, legend, overall), fully supporting H4a (Table 1)—and to understanding the map title. Numerical preference related more to adequacy judgments than did perceived numerical ability. Neither numerical preference nor perceived numerical ability related to perceived accuracy or trust in validity (RQ4a).

Subjective numeracy will moderate the influence of risk expression (H4b)

Interactions between numerical preference and risk expression only related to overall understanding (Table 1). However, interactions between perceived numerical ability and risk expression related to all understanding variables; thus, perceived numerical ability was used to address H4b, H4c, RQ4b, and RQ4c. Subgroup analyses supported the hypothesis (H4b) that subjective numeracy would positively relate to understanding for the numerical risk expression, but not the verbal-relative risk expression. For the numerical risk expression, perceived numerical ability positively related to: map understanding (standardized coefficient = .212, p < .001); legend understanding (.306, p < .001); and overall understanding (.218, p < .001). For the verbal-relative risk expression, perceived numerical ability did not relate to: map understanding (.021, p =.692); legend understanding (.019, p = .713); and overall understanding (.032, p = .560).

Does subjective numeracy moderate the influence of risk expression on perceived accuracy and trust in validity? (RQ4b)

Interactions between perceived numerical ability and perceived accuracy or trust in validity related to risk beliefs (Table 1). For the numerical risk expression, perceived numerical ability related positively to perceived accuracy (standardized coefficient = 0.140, p =.006) and trust in validity (.109, p = .034). For the verbal-relative risk expression, perceived numerical ability related negatively to perceived accuracy (−.108, p = .047) and but not trust in validity (−.027, p = .627).

Subjective numeracy will not moderate the influence of visual map features on understanding (H4c)

Interactions between perceived numerical ability and visual variables (contour focus, color) were not related to the understanding variables, supporting H4c (Table 1).

Does subjective numeracy moderate the influence of visual map features on perceived accuracy and trust in validity? (RQ4c)

Interactions between perceived numerical ability and contour focus had no influences on perceived accuracy or trust in validity (Table 1). The interaction between perceived numerical ability and number of colors did not significantly influence perceived accuracy, but had a small effect on trust in validity. There was a small positive relationship between perceived numerical ability and trust in validity for three-color maps (standardized coefficient =.109, p = .028), but not one-color maps (−.020, p =.229).

Aim 5. Do map features moderate the consistency between risk levels and risk beliefs? RQ5

Interactions between risk level and each of the three map features on risk beliefs were significant for two map features: risk level by contour focus (unstandardized coefficient =.268, p < .001); and risk level by risk expression (092, p = .029), but not risk level by color (.075, p = .078). Non-overlapping 90% CIs for standardized coefficients indicate risk level had a larger effect on risk beliefs for focused contours (.710, p < .001, 90% CI [.687, .733]) than unfocused contours (.585, p < .001 [.557, .612]), although large effects for both. Despite small interaction effects, differences were not suggested by CIs across risk expressions (numerical= .662, p < .001 [.631, .692]; verbal-relative= .638, p < .001 [.605, .671]).

Discussion

Recall that (a) the unit of perception (map contours) shapes what people see and understand, (b) bottom-up processing fosters seemingly effortless comprehension for some visual features, (c) iconic features support comprehension by resembling what they are intended to represent, and (d) visual discrimination allows viewers to match color shades between the map and legend. These and other concepts summarized in the introduction are applied to discuss study results.

Influences of Map Features on Adequacy judgments

Contour focus

Of the map features, only contour focus consistently generated judgments of greater adequacy for the certain version and lesser adequacy for the uncertain version as hypothesized. Four visual cognition concepts support these findings: unit of perception, bottom-up processing, iconicity, and visual discrimination.

The many color shades on maps with unfocused contours had no discernable contour boundaries and imperceptible visual discrimination from one shade to the next, even for maps with three colors. Less visual discrimination hinders matching map and legend colors (Breslow et al., 2009; Harrower, 2007) and decreases the accuracy of comprehending a map (Mersey, 1990), likely decreasing adequacy judgments. For focused contours, visual discrimination was sufficient because the four color shades were less than the recommended upper limit of seven shades of a single color (Brewer, 2006) and the contours had well defined boundaries. These characteristics support accurately comprehending risk levels for given map locations, likely increasing adequacy judgments. Participants’ written comments illustrate these tendencies “the gradual shading made it more difficult to decide where exactly each spot fell on the spectrum of cancer risk” (unfocused contours), and “the lines are more clear-cut so you can easily tell what areas have which levels of air pollution” (focused contours). Seeing unfocused and focused contours (the unit of perception) may convey uncertainty and certainty via bottom-up processing and by resembling (iconic) what they are intended to represent—potentially strengthening the influence of visual discrimination on adequacy judgments.

Recall that more data classes represent incremental magnitude more precisely than fewer data classes. When judging the accuracy of the information, participants appeared to focus more on the ease of accurately matching map to legend colors and less on the accurate portrayal of risk estimates across the map. Both ideas are mentioned in this participant’s comment: “The map is more clear when the colors are more clearly demarcated, rather than the probably more accurate but fuzzier, graded version.”

Risk expression

The unexpected null effect of risk expression on perceived ease of understanding the legend may have occurred because each of the risk expressions posed challenges for interpretation. The simple “less – more” message of the verbal-relative risk expression was likely easy to comprehend, but does not support understanding the health risk as indicated by this comment: “Because it [verbal-relative risk expression] could have been from 0% – 100% or 1% – 3%; whether I would have felt unsafe if I lived in a region with ‘more’ risk would have varied greatly depending on whether ‘more’ meant 100% or 3%.” Ratios, decimals, numbers less than one, and large ratio denominators pose barriers to comprehension (Cuite, Weinstein, Emmons, & Colditz, 2008; Garcia-Retamero & Galesic, 2011; Lipkus, 2007; Reyna & Brainerd, 2007). Using these in the numerical risk expression may decrease perceived ease of understanding, as illustrated by this comment: “The thing that is confusing about this map is the ratios they give in the key.” For the numerical risk expression, the health risk meaning of the “acceptable” and “unacceptable” evaluative labels must be inferred—another potential barrier to ease of understanding.

The lack of risk expression effects on perceived understanding of the legend and map portion may be further explained by the use of features that support comprehending incremental risk. Incrementally darker shades in both the map portion and legend may foster comprehension of incremental risk via both bottom-up processing and iconic resemblance that supports perceived ease of understanding regardless of the risk expression. The small effect of the numerical compared to the verbal-relative risk expression on overall understanding may have occurred because the numerical ratios and evaluative labels added meaning beyond the incremental risk meaning of the verbal-relative risk expression and shaded map colors.

Tendencies to (a) prefer numbers (Wallsten et al., 1993); (b) want evaluative labels (Trevena et al., 2013); and (c) perceive numerical versus verbal expressions as more scientifically credible (Lipkus, 2007), accurate (Stacey et al., 2014), and trustworthy (Gurmankin et al., 2004) suggest why the numerical risk expression was judged as more accurate, valid, and trustworthy. The verbal-relative risk expression lacks accuracy in two ways—the meaning of less and more is ambiguous, and the relative difference between less and more with no evaluative labels or baseline is highly ambiguous (Koehler, 1996). Ambiguity aversion fosters less perceived credibility (Han et al., 2009; Longman et al., 2012) which further supports these findings.

Number of colors

Additional colors may increase visual complexity which can decrease comprehension (Cutter, 2008; Florence & Geiselman, 1986), a potential explanation for the greater perceived difficulty of understanding the three-color map compared to the one-color map. More colors are not expected to substantially influence visual discrimination for focused maps with only four color shades, as this is less than the recommended upper limit of seven shades (Brewer, 2006). However, the additional colors did not appear to facilitate visual discrimination for unfocused contours. For the many shaded contours, three colors may be too few to provide the visual discrimination needed to support comprehension via matching map to legend colors. Furthermore, the text size of the risk labels in the legend was large enough to span multiple color shades—potentially impeding the accuracy of matching map colors to labeled risk levels in the legend for unfocused contours. In addition, the influence of number of colors on adequacy judgments may have been mitigated by the effectiveness of incremental shading to support accurate bottom-up comprehension of incremental magnitude (Cleveland & McGill, 1985; Severtson & Myers, 2013), and by the ordered progression of lower to higher risk from rural to urban areas that matches beliefs of less air pollution in rural than urban areas (Brody, Peck, & Highfield, 2004).

Summary

When viewers are asked to assess location-based risk for maps, perceived ease of understanding may be driven primarily by the ease of matching color shades between the map and legend. Perceived accuracy and trust in validity may be driven primarily (a) by beliefs that a numerical risk expression is more accurate, trustworthy, valid, and scientific than a verbal-relative risk expression and (b) by deep dissatisfaction with relative risk information that lacks evaluative labels or other interpretive information.

Influences of Adequacy Judgments on Risk Beliefs and Perceived Ambiguity

General support for the hypothesis that perceived ease of understanding relates to beliefs of more risk at higher risk levels and beliefs of less risk at lower risk levels indicates that perceived understanding relates broadly to consistency between risk beliefs and risk levels. Analyzing potential moderators of the relationship between perceived understanding and risk beliefs for risk-level/map-feature subgroups may suggest why this influence was substantially attenuated at middle risk levels.

The hypothesis that more perceived accuracy and trust in validity would relate to more consistency between risk levels and risk beliefs was only supported for higher risk levels. Dieckmann, Slovic, and Peters (2009) found that perceived credibility relates positively to perceived risk, but did not examine these effects across lower and higher risk levels. At lower risk levels, evidence suggests null effects may be due in part to more variability in how low risk levels are interpreted (MacGregor, Slovic, & Malmfors, 1999).

Support for the hypothesis that judgments of lower adequacy relate to more perceived ambiguity is consistent with claims that people are averse to ambiguity and prefer certain information (Han et al., 2009; Han et al., 2006; Longman et al., 2012). The larger negative influence of perceived accuracy on perceived ambiguity suggests accuracy could be more important to viewers than trust in validity. The larger negative relationship between perceived ambiguity and understanding the information overall compared to understanding the map portion may reflect the importance of comprehending how the components of map information work together to support comprehension. Overall understanding may encompass a more holistic assessment of the available information as indicated by the larger correlations between overall understanding and understanding the other map components than correlations among the other map components.

Influence of Subjective Numeracy on Adequacy judgments

Given people’s preference for information perceived as easy to understand (Fortin et al., 2001), coherence among preferences may explain why numerical preference (vs. perceived numerical ability) was more related to self-rated ease of understanding. The positive relationships between understanding the non-numerical map title and both numerical preference and numerical ability suggest these subjective measures of numeracy relate somewhat to literacy.

The main effects of perceived numerical ability on adequacy judgments are not discussed because these influences differed substantially between the numerical and the verbal-relative risk expressions. Lack of numbers and the simplicity of the “less – more” labels may explain why perceived numerical ability had no influence on ease of understanding for the verbal-relative risk expression. Barriers to comprehension posed by ratios, decimal values, numbers less than one, and a large denominator (Garcia-Retamero & Galesic, 2011; Lipkus, 2007; Reyna & Brainerd, 2007) may explain why perceived numerical ability related positively to understanding the numerical risk expression. The role of numerical ability in understanding the numerical risk expression in the legend, and the importance of understanding the legend in order to understand the meaning of a given map location, explains why perceived numerical ability influenced perceived understanding for the map portion and overall information in addition to understanding the legend.

Perceiving better numerical ability may relate to beliefs that numerical information is more accurate than verbal information. These beliefs could explain why perceived numerical ability had a positive influence on perceived accuracy for the numerical risk expression and a negative influence for the verbal-relative risk expression.

Using the appearance of map contours to convey uncertainty appears to address numeracy barriers because perceived numerical ability did not moderate the influence of contour focus (a) on adequacy judgments in this study and (b) on risk beliefs and perceived ambiguity in the previous study (Severtson & Myers, 2013).

Perceived numerical ability may have had similar influences on adequacy judgments across focused and unfocused contours as hypothesized because (a) no numerical information was provided with map contours and (b) incrementally darker color shades foster comprehension via bottom-up processing and by iconically resembling incremental risk (Cleveland & McGill, 1985; Severtson & Myers, 2013).

Galesic, Garcia-Retamero and Gigerenzer (2009) claim that iconic arrays (grids of dots or symbols illustrating numerators and denominators for a simple numerical risk expression) address numeracy barriers. Iconic arrays visualize the size of the grid area for the numerator relative to the size of the grid area for the denominator. Visualizing area is proposed to support bottom-up comprehension (Cleveland & McGill, 1985). This illustrates the generalizability of visual cognition concepts across different studies to explain why visual features that are iconic and support bottom-up processing may be especially effective for addressing numeracy barriers.

Consistency Between Risk Levels and Risk Beliefs and Meeting Communication Goals

The large effect of risk level on risk beliefs for unfocused contours suggests that incrementally shaded unfocused contours fostered consistency between risk levels and risk beliefs while concurrently fostering perceived ambiguity across a range of subjective numeracy levels. Using iconic features that support bottom-up processing, and decreasing visual complexity by intrinsically conveying uncertainty within the unit of perception appear to address three important risk communication goals: (a) consistency between risk levels and risk beliefs), (b) comprehension of uncertainty, and (c) mitigation of numeracy barriers. The larger influence of risk level on risk beliefs for focused contours than unfocused contours may suggest risk estimates were interpreted more literally than warranted given the uncertainty of the risk estimates. When risk information is highly uncertain, too much consistency between risk levels and risk beliefs may not indicate effective risk communication.

Map features judged as more adequate (focused contours and numerical risk expression) were less effective for supporting communication goals than their lower rated counterparts. This is somewhat consistent with findings that, sometimes, preferred formats are less effective for supporting accurate comprehension than non-preferred formats (Elting et al., 1999; Gerteis et al., 2007; Hildon et al., 2011). However, the outcomes (risk beliefs versus accurate comprehension of facts) are an important difference between these studies. Risk beliefs are an appropriate indicator of meeting risk communication goals because they are more predictive of decisions and behavior than is factual knowledge (National Cancer Institute, 2005).

Limitations, Implications, and Conclusions

Limitations

The measure of trust in validity included two concepts; thus difficult to know the degree to which participants’ answers reflected trust or validity. The younger and likely more numerate student sample compared to the general public, in addition to less personal relevance for the fictitious assigned map locations and risk estimates based on 70 years of exposure, decreases external validity. The individual effects of risk expression components (verbal, numeric, relative, absolute, evaluative labels) could not be assessed due to the study design. Awareness of the difference between focused and unfocused contour was potentially enhanced because each participant viewed maps with both, but only viewed maps with one of the two risk expressions or color schemes. Map appearance varies across computer monitors, although this may enhance external validity because maps are typically online.

Implications for research

The external validity of future research would be enhanced by using a more representative target audience and personally relevant tasks in which participants respond to the information for actual rather than assigned map locations. Research is needed to test versions of the verbal-relative risk expression that would be more acceptable to public audiences, for example including baseline values or evaluative labels. Measures of adequacy judgments should distinguish between the concepts of validity and trust, use a familiar synonym for validity (e.g. truth), and assess whether the information is perceived as “scientific.” Assessing eye-tracking as participants answer survey questions would provide additional insights into how map features influence visual attention and how visual attention influences adequacy judgments and the moderating role of numeracy. A brief explanation of why EPA’s cancer risk estimates are uncertain and the intended uses for the mapped estimates is provided on EPA’s webpage that provides links to these maps (U. S. EPA, 2011a). Such information is likely to influence judgments of adequacy, risk beliefs and perceived ambiguity. Research is needed to develop and test user-centered versions of this explanatory information.

Implications for practice

Unfocused contours with incremental color shading effectively fostered the gist of uncertainty and consistency between risk levels and risk beliefs, while also addressing numeracy barriers. However, unfocused contours were judged as more difficult to understand than focused contours. At issue is whether perceiving information as more difficult to understand would hinder people’s use of maps with unfocused contours. Using features proposed to support bottom-up comprehension should mitigate aversion to using such maps. Providing a user-friendly explanation of why the information is uncertain and how it is intended to be used is cautiously advised. Such messages should be developed based on feedback from individuals who represent the literacy spectrum of the target audience. Given the uncertainty of the risk estimates, the even greater consistency between risk levels and risk beliefs for focused contours may suggest the risk levels were interpreted too literally. In conclusion, unfocused contours appear to be an effective method for representing uncertainty on maps.

Although the verbal-relative risk expression was effective for conveying the gist of uncertainty and addressed numeracy barriers, participants were very dissatisfied with this risk expression due to its highly ambiguous nature (Severtson & Myers, 2013). Providing some context—for example providing safety benchmarks, evaluative labels, or other normative indicators—might address some of this dissatisfaction while still conveying uncertainty.

Conclusions

Visual features that foster comprehension by supporting bottom-up processing and by resembling what they are intended to represent (are iconic) may be especially effective for meeting risk communication goals. Such features (contour focus and shaded color) fostered the accurate gist of risk magnitude and uncertainty, while also addressing numeracy barriers and perhaps other literacy barriers. These goals were addressed even though unfocused contours were judged as more difficult to understand than focused contours.

Consistency between risk levels and risk beliefs was similar across risk expressions, but the verbal-relative risk expression conveyed uncertainty and addressed numeracy barriers, while the numerical risk expression did not. Although the numerical risk expression was judged as more accurate, trustworthy and credible than the verbal-relative expression, both risk expressions were judged as difficult to understand.

Interpreting adequacy judgments requires balancing people’s tendency to perceive uncertainty as difficult to understand with goals of communicating the magnitude and uncertainty of estimated risk. When selecting risk expressions and visual features for conveying estimated health risks on maps, it is important to (a) use evidence-based methods that support meaningful comprehension and (b) balance people’s information preferences with communication goals.