Representing randomness in the communication of individualized cancer risk estimates: effects on cancer risk perceptions, worry, and subjective uncertainty about risk

Abstract

Objective

To test the effect of novel representations of randomness on risk perceptions, worry, and subjective uncertainty about individualized colorectal cancer risk estimates.

Methods

A Web-based factorial experiment was conducted, in which 225 adults aged 40 years and older were provided with hypothetical individualized colorectal cancer risk estimates, using 5 different textual and visual representations varying in expressed randomness. Outcome measures were perceived cancer risk, cancer worry, and subjective uncertainty about cancer risk; the moderating effect of dispositional optimism was also examined.

Results

Representational format was significantly associated with subjective uncertainty about cancer risk, but not with perceived cancer risk or worry. A format using software-based animation to express randomness dynamically led to the highest subjective uncertainty, although a static visual non-random format also increased uncertainty. Dispositional optimism moderated this effect; between-format differences in uncertainty were significant only for participants with low optimism.

Conclusion

Representing randomness in individualized estimates of cancer risk increases subjective uncertainty about risk. A novel dynamic visual format produces the greatest effect, which is moderated by individual differences in optimism.

Practice Implications

Novel representations of randomness may be effective in improving people’s understanding of the essential uncertainty pertaining to individualized cancer risk estimates.

1. Introduction

The communication of individuals’ risk of diseases such as cancer has become an increasingly common endeavor in health care. Over the past several years a growing number of disease risk prediction models have been developed, and their use has rapidly expanded beyond research to the domains of clinical care and public health communication (1, 2). Numerous models are now available to clinicians and patients in health care settings and to laypersons on the Internet (3, 4).

Individualized disease risk prediction models are a promising educational tool for patients and laypersons; however, they also raise important concerns given the significant potential for risk information to be misunderstood by model users. Deficits in numeracy—the ability to understand and use numbers—are widespread even among highly-educated persons (5), and pose a particular challenge for the comprehension of risk information, which requires higher-order reasoning and computational skills including the ability to understand and manipulate ratios and proportions (6).

Difficult conceptual problems present further challenges for the interpretation of individualized disease risk information. Among the most problematic is the meaning of objective risk estimates at the individual level. These estimates are based on the aggregated outcomes observed in a population of individuals, and expressed in terms of the expected number of affected individuals or lives. However, their meaning for a single individual, who has only one life to live, is not straightforward and has engendered major controversy in the philosophy of statistics. Indeed, several theorists have argued that the very idea of individual or “single event” probability—as in an individual’s risk of disease—is logically incoherent, since probabilities can only be assigned to events that are repeated in the long run or aggregated across a population of individuals (7–9). Furthermore, the applicability of population-based risk estimates to individuals is limited by the existence of unmeasured risk factors that distinguish individuals from members of the larger reference population. Risk estimates are therefore “individualized” only in a figurative sense; individuals’ true risks are nonquantifiable, their future outcomes unknowable.

Comprehension of this type of uncertainty is essential to understanding individualized risk information; however, it is difficult to grasp (10, 11), and has seldom been addressed in disease risk communication efforts. Past efforts have mainly focused on uncertainty that arises from model misspecification and limitations in external validity, and is manifest by imprecision of risk estimates; this type of uncertainty is typically expressed using confidence intervals (12, 13). Only recently have investigators begun exploring visual methods to convey the distinct uncertainty arising from the fundamental inability to predict future outcomes of individuals, and manifest by randomness. For example, some investigators have used icon arrays displaying affected individuals in a scattered—rather than a conventional clustered or sequential—arrangement across the array (14–16). The assumption here is that scattered arrays appear to represent higher levels of randomness, as shown in previous work on human perception of random sequences (17). Building on these efforts, Ancker and colleagues recently developed a game-like web-based interactive graphical display of disease risk that attempts to convey both the randomness and magnitude of risk estimates by requiring users to click on an icon array to search for affected individuals within a population (18).

These efforts are promising, but their effectiveness in promoting users’ understanding of uncertainty has not been fully evaluated. A qualitative study by Schapira and colleagues suggested that use of a visual icon array displaying affected individuals in a scattered arrangement was effective in conveying randomness; however, this finding has not been replicated. Ancker and colleagues conducted an experimental evaluation of their interactive graphical displays and found no significant effect on users’ disease risk estimates, risk feelings (perceived susceptibility or vulnerability), or intentions to take preventive action. However, this study did not ascertain users’ perceived uncertainty regarding risk estimates. Since the intended effect of communicating randomness is to promote users’ recognition of this uncertainty, the lack of evaluation of this outcome is an important research gap.

In the study we report now, we attempted to address these issues regarding the communication of randomness in individualized estimates of cancer risk. We conducted an experiment to test how alternative novel methods of communicating randomness regarding individualized colorectal cancer risk estimates affect risk perceptions, worry, and, most germane to the above discussion, subjective uncertainty about risk levels. We compared the effects of visual displays varying in expressed randomness, including non-random (clustered) and random (scattered) icon arrays, and a newly developed random icon array using software-based animation to enhance the representation of randomness. We hypothesized that the scattered arrays would convey the notion of randomness more effectively and thereby increase users’ subjective uncertainty about their cancer risk, and that this effect would be greatest for the dynamic array given its more vivid depiction of randomness. We also hypothesized that communicating randomness would increase perceived cancer risk and cancer worry, given people’s demonstrated aversion to unknown vs. known risks—a phenomenon known as “ambiguity aversion,” which is manifest by heightened perceptions of risk and avoidance of decision making. Related to these potential effects, we explored the moderating influence of an individual personality variable, dispositional optimism, on responses to the communication of randomness. In a previous study of which the current experiment was a part, we found that high levels of optimism reduced ambiguity aversion, blunting the predicted increase in cancer risk perceptions and worry elicited by the communication of ambiguity arising from imprecision and represented using confidence intervals (19, 20). We hypothesized that dispositional optimism might similarly moderate any increase in risk perceptions, worry, and subjective uncertainty resulting from the communication of randomness.

2. Methods

2.1. Study population and participants

The study consisted of an interactive web-based survey experiment conducted in March 2009. Participants were drawn from a professionally-managed nationwide web survey panel maintained by Keynote Systems, Inc., and had experience and interest in completing online surveys both related and unrelated to health. Participants were recruited by email invitation, and received compensation in the form of “points” from the panel administrator, which could be accumulated and used for online purchases with select vendors. Eligibility criteria for the current study included age ≥40 years and no prior history of colon cancer. We specified a target sociodemographic mix of equal men and women and varying race/ethnicity (≥15% non-white/Caucasian) and education level (≥30% high school diploma or less). Recruitment continued until target sample sizes and sociodemographic composition were reached.

2.2. Development of representational formats

Alternative Web-based representational formats for communicating colorectal cancer risk estimates were developed by our team, which included a human factors engineer with substantial experience in website development and usability testing. We developed 5 different formats for evaluation (Appendix 1), each of which depicted a “9%” risk of colorectal cancer in different ways: 1) text-only non-random: “Your chances of developing colorectal cancer over your lifetime are 9%” (control condition); 2) text-only random, which included a textual representation of randomness: “Based on the information provided, the estimated chance for developing colorectal cancer over the lifetime is 9%. We can’t predict the future of any one person. Risk estimates only tell us how many people in a population are likely to get colon cancer; they can’t tell us who will get the disease or not”; 3) visual non-random (clustered) icon array; 4) visual random (scattered) icon array; and 5) visual dynamic random (scattered) icon array. We designed the new dynamic random representation to display 9 individuals scattered randomly through an icon array of 100 persons, employing web animation to randomly change the placement of these 9 individuals every 2 seconds. All visual formats (static and dynamic) also included the enhanced text used in the second representation to reinforce the concept of randomness (Appendix 1). Cognitive interviewing was conducted in 9 volunteer participants to test the understandability and usability of the representational formats, and to guide their refinement.

2.3. Study design and procedure

The experiment utilized a 5-group factorial design in which respondents were randomly assigned to one of the representational formats. Panel members were invited to participate via email, and those who agreed and met eligibility criteria were emailed a link to a web-based survey corresponding to their assigned study condition. After accessing the survey website, participants were provided with a brief overview of the National Cancer Institute (NCI) Colorectal Cancer Risk Assessment Tool (http://www.cancer.gov/colorectalcancerrisk/), describing how the tool is simple to use and calculates a person’s lifetime risk of colorectal cancer based on questions about lifestyle and medical and family history. Participants were then asked to imagine they used the tool and received the hypothetical results that were then displayed, corresponding to their assigned experimental group. They then completed the set of measures described below. We did not utilize subjects’ actual colorectal risk cancer estimates, given the lack of knowledge of the effects of doing so, which motivated the current study.

2.4. Measures

Perceived cancer risk was measured using 2 items (α=.784). The first item was adapted from the Health Information National Trends Survey (HINTS) administered by the NCI, and read, “Based on these results from the computer program, how would you describe your chances of developing colon cancer in your lifetime? Would you say your chances of getting colon cancer are…” A 5-point Likert response scale was used, with the endpoints labeled “Very Low” and “Very High.” The second item was adapted from Cameron et al (21) and read, “If I received these results, I would feel that I’m going to get colon cancer.” A 5-point Likert response scale was used, with the end-points labeled “Strongly Disagree” and “Strongly Agree.” Responses to these 2 items (each scored 1–5) were averaged.

Cancer-related worry was measured using a single item adapted from the HINTS, “If you received these results, to what extent would you feel worried about developing colon cancer?” A 5-point Likert response scale (scored 1–5) was used, with the end-points labeled “Not at all” and “Extremely.”

Subjective uncertainty about cancer risk was measured by a single item developed by our team: “How certain do you feel about the opinions you just offered regarding your chances of getting colon cancer?” A 5-point Likert response scale (scored 1–5) was used, with the end-points labeled “Not at all certain” and “Very certain.” Scores were reverse-coded to signify uncertainty.

Dispositional optimism was measured using the 3-item Optimism subscale (α =.866) of the Life Orientation Test (22). An example item is: “In uncertain times, I usually expect the best”; respondents indicate their level of agreement using a 5-point Likert scale (scored 1–5). Responses to these 3 items were averaged.

2.5. Statistical analysis

Analysis of variance (ANOVA) was used to test the effect of representational format (text-only nonrandom control, enhanced text-only, visual non-random, visual random, and visual dynamic-random) on each of the 3 outcome variables: perceived cancer risk, subjective certainty about cancer risk, and cancer worry. Dispositional optimism was included in each model as a cofactor in order to assess its main effects and potential interactions with representational format; optimism scores were dichotomized by median split (Mdn=12, range 3–15) because they were highly skewed, and to aid in interpretability of the results.

3. Results

3.1. Sample characteristics

A total of 225 participants drawn from all geographic regions of the U.S. completed the study (Table 1). The mean age was 53 years (range 40–72), and males outnumbered females (54% vs. 46%). Approximately 8% identified themselves as non-white/Caucasian, and 25% reported an educational level of high school graduate or less. There were no statistically significant differences in sociodemographic characteristics between experimental groups.

Table 1.

Sociodemographic characteristics of study population

	Text-only non-random		Text-only random		Visual non-random		Visual random static		Visual random dynamic
	N	%	N	%	N	%	N	%	N	%
Sex
Female	21	46.7	18	40	22	48.8	20	44.4	22	48.8
Male	24	53.3	27	60	23	51.1	25	55.6	23	51.1
Age
40–49	18	40	19	42.2	19	42.2	13	28.9	17	37.8
50–59	18	40	15	33.3	11	24.4	19	42.2	17	37.8
≥60	9	20	11	24.4	15	33.3	13	28.9	11	24.4
Education
≤HS graduate	11	24.4	10	22.2	13	28.9	11	24.4	12	26.7
Trade school/some college	16	35.6	14	31.1	16	35.6	17	37.8	16	35.6
College graduate	18	40	21	46.7	16	35.6	17	37.8	17	37.8
Race
White	40	88.9	42	93.3	41	91.1	41	91.1	44	97.8
Black	2	4.4	1	2.2	3	6.7	3	6.7	1	2.2
Other	3	6.7	2	4.4	1	2.2	1	2.2	0	0

3.2. Effects of representational format

There was no significant effect of representational format on perceived cancer risk or worry. However, representational format was significantly associated with subjective uncertainty about cancer risk (F(4, 210)=2.98, p=.02, partial η²=.054). Post-hoc analyses, corrected for multiple comparisons using Tukey’s test, showed a significant difference (p=.02) between the visual dynamic random and the text-only random formats (M=2.7 vs. 2.1), and a borderline significant difference (p=.06) between the visual non-random and the text-only random format (M=2.5 vs. 2.1) (Figure 1). Participants exposed to the visual dynamic random and the visual non-random formats demonstrated greater uncertainty than those exposed to the text-only random format. There were no other significant pairwise differences.

Figure 1.

Level of perceived uncertainty for different representations of randomness, and interaction with dispositional optimism.

Dispositional optimism was also significantly associated with subjective uncertainty (F(1, 206)=5.80, p=.02, partial η²=.027); higher optimism was associated with lower uncertainty. There was also a significant interaction between representational format and dispositional optimism (F(4, 210)=3.51, p=.01, partial η²=.063) (Figure 1). Notably, participants with low optimism exhibited responses generally consistent with predictions: low uncertainty with the text-only formats, and high uncertainty with the dynamic random format. These relationships were confirmed using stratified ANOVAs including only participants with low (n=156) or high (n=64) optimism. For participants with low optimism, post-hoc pairwise tests showed that the dynamic random group had significantly higher subjective uncertainty levels (p<.05) than all other groups except for the visual non-random group. In contrast, for participants with high optimism, there was no significant difference between any of the representational formats, and a counterintuitive trend towards lower rather than higher uncertainty with the dynamic and other visual formats compared to the textual formats.

4. Discussion and conclusion

4.1. Discussion

In this study we explored laypersons’ responses to the communication of randomness regarding individualized estimates of colorectal cancer risk. Using a real world disease problem and risk application prototype, we examined the effects of alternative representational formats designed to convey randomness, including a newly-developed dynamic representation utilizing software-based animation. We ascertained the extent to which these alternative representations increase cancer risk perceptions and worry, as well as subjective uncertainty pertaining to risk. We obtained findings that have several implications for risk communication research and for the development and use of disease risk prediction models for patient education and counseling.

First, the data demonstrate complexity in the effects of different formats for representing randomness. The relationships between the type of format and participants’ perceived uncertainty about cancer risk were not straightforward, and the data only partially supported our hypothesis that visual representations per se would increase subjective uncertainty compared to textual ones. Mean uncertainty scores were higher in general for participants exposed to visual vs. text-only representations of randomness; however, only the visual dynamic format group showed significantly higher scores compared to the text-only group—consistent with our a priori hypothesis that the dynamic format would convey randomness most vividly. However, contrary to expectations, uncertainty scores for the static visual random group were among the lowest of all groups and were not significantly different from those of the text-only random group. Meanwhile, the text-only non-random control group—which did not receive any randomness information—had the highest mean uncertainty score of all groups. Although this latter difference was non-significant statistically, its direction and magnitude and the overall pattern of relationships were unexpected, and argue against a straightforward “dose-response” relationship in the communication of randomness.

These findings are exploratory and need to be confirmed, given that the effects of communicating randomness on subjective uncertainty have not been previously evaluated. However, it is useful to speculate on potential reasons for our unexpected findings. The relatively high uncertainty of participants exposed to the textual non-random (control) format may reflect the influence of implicitly-conveyed sources of uncertainty other than expressed randomness. For example, simply presenting a textual statement that provides a point estimate of cancer risk (e.g., “9%”)—without further explanation—may be sufficiently uninformative to leave people feeling uncertain. This possibility seems likely given the known difficulty people have in interpreting the meaning of individualized risk estimates (10).

The other unexpected finding was the relatively low subjective uncertainty of participants exposed to the static visual random format, and its apparent lack of effect compared to the textual format. This finding raises questions about whether scattered icon displays convey randomness effectively, as previous studies have suggested (14–16). In our study, this seemed to be the case only when the scattered display was presented in a dynamic rather than static manner. Paradoxically, static fixed displays of scattered icons might increase people’s certainty about risk, by implying a capacity of the model to isolate which patients in a population will be affected. This explanation is speculative and calls for further research, but in the meantime our data call for caution in using such icon arrays as a means of communicating uncertainty.

Additional insight on the mechanisms underlying people’s responses to the communication of randomness is provided by our previously hypothesized finding of a significant moderating effect of dispositional optimism, which appears to influence both the strength and direction of the effect of alternative representations of randomness on subjective uncertainty. People with low optimism show more predictable, substantial effects and a heightened sensitivity to representational format compared to people with high optimism. To our knowledge, this finding has not been previously reported and needs to be replicated, particularly given that our study’s relatively small sample size may have obscured the extent of the observed interaction. The small proportion of high-optimism (n=64) vs. low-optimism (n=156) participants could explain the lack of significance of the contrasting trend towards lower rather than higher uncertainty with visual vs. textual formats in high-optimism individuals.

Nevertheless, all these findings may account for the relatively weak and unexpected main effects of different representational methods, and has important implications for both research and practice in health communication. It suggests that individuals with low optimism have the greatest need for—and may derive the greatest benefit from—efforts to communicate randomness. It also raises questions about the mechanisms by which optimism might promote indifference to the communication of randomness. It could be that optimism predisposes people to already view the future as undetermined or random, thereby reducing any added effect of representing this concept. Further research is needed to test this and alternative explanations.

The other notable finding of our study was the lack of effect of representational format on perceived cancer risk or worry, which was contrary to predictions based on findings from our prior study demonstrating aversion to ambiguity arising from imprecision and represented using confidence intervals. The negative finding of the current study—corroborated by data from Ancker and colleagues (18)—suggests that the uncertainty conveyed by representations of randomness does not necessarily elicit a psychological response akin to ambiguity aversion, and that randomness may represent a type of uncertainty distinct from ambiguity. These cumulative data also provide reassurance that communicating randomness does not adversely affect people’s understanding of the magnitude of risk estimates.

Of course, one might reasonably question whether the subjective uncertainty induced by the communication of randomness is a favorable outcome and valid marker of people’s understanding of uncertainty. After all, it could be argued that the ultimate goal of communicating individualized cancer risk information is to reduce people’s uncertainty about their risk. Nonetheless, one must acknowledge the impossibility, from a normative standpoint, to be certain about any individual’s risk of cancer or any other disease. For such individual events, there is no single “true” objective probability; “individualized” risk estimates are inherently uncertain expressions of subjective belief, rather than literal accounts of some objective reality (7, 8, 23). A thorough understanding of the meaning of any given risk estimate must thus entail some degree of subjective uncertainty about the estimate itself.

The challenge of risk communication, therefore, is to both decrease and increase people’s subjective uncertainty about their risk—to provide the best available scientific knowledge to inform people’s subjective beliefs about their own futures, while promoting an understanding of the fundamental limitations of this knowledge so that people can make truly informed medical decisions. It remains for future research to explore how to accomplish this latter task, and how much subjective uncertainty is justified. We need to know whether there is a point at which subjective uncertainty about risk estimates is counter-productive, causing confusion or otherwise undermining people’s self-efficacy and ability to make good decisions.

Our study had several limitations. It used hypothetical risk estimates and ascertained participants’ anticipated—rather than “real”—perceptions in response to these estimates. This was an intentional aspect of our study design, given the unknown potential harms of communicating uncertainty. However, it raises questions about the generalizability of our findings, as does the use of a study sample comprised of web survey panel members who, by virtue of their motivation to participate, may not be representative of the general population. Notably, this limitation also applies to the substantial body of health risk communication research being conducted using similar samples.

The representational formats and measures used in the study were also limited. We did not examine the effects of communicating randomness at lower or higher levels of risk. We also used single-item measures for cancer worry and for subjective uncertainty. Although single-item measures of constructs such as cancer worry have been shown to predict behavioral outcomes (24, 25), more work is needed to assess the performance of these measures. Furthermore, our subjective uncertainty measure was new and unvalidated, and more research is needed to develop and test alternative methods of ascertaining subjective uncertainty as well as the related outcome of people’s understanding of the notion of randomness. Finally, the sample sizes for our experimental groups were relatively small, introducing the possibility of both Type I and II errors. As noted previously, our study may have been particularly underpowered to fully characterize the observed interaction between dispositional optimism and representational format. Further research is needed to replicate our findings using larger and more diverse samples.

Despite these limitations, our study provides seminal data on a promising novel dynamic visual method for representing randomness, and new insights about its effects and mechanisms. More work is needed to evaluate this and other innovative risk communication methods, and to determine optimal strategies to improve people’s understanding of the various uncertainties implicit in individualized risk information in health care and other domains.

4.2. Conclusion

A strong rationale exists for representing randomness in efforts to communicate individuals’ risks of cancer and other diseases. This study is the first to experimentally test the effects of different representations on both risk perceptions and individuals’ subjective uncertainty about risk. The data suggest that representing randomness in individualized estimates of cancer risk increases subjective uncertainty while having no significant effect on risk perceptions or worry, and a novel dynamic visual format produces the greatest effect. However, the effect of representational format is complex, and non-random, ordered visual representations may also promote subjective uncertainty. Furthermore, these format effects are moderated by individual differences in optimism, and are significant in individuals with low optimism.

4.3. Practice implications

New methods of representing randomness may be effective in improving people’s understanding of the essential uncertainty pertaining to individualized cancer risk estimates. These methods may be particularly effective in individuals with low optimism.

Appendix 1. Textual and visual representations used in experiment

1. Text-only, non-random

   

2. Text-only, random

   

3. Visual non-random

   

4. Visual random, static

   

5. Visual random, dynamic: animation was employed to randomly change the pattern of shaded icons every 2 seconds