Conceptual problems in laypersons’ understanding of individualized cancer risk: a qualitative study

Abstract

Objective  To explore laypersons’ understanding of individualized cancer risk estimates, and to identify conceptual problems that may limit this understanding.

Background  Risk prediction models are increasingly used to provide people with information about their individual risk of cancer and other diseases. However, laypersons may have difficulty understanding individualized risk information, because of conceptual as well as computational problems.

Design  A qualitative study was conducted using focus groups. Semi‐structured interviews explored participants’ understandings of the concept of risk, and their interpretations of a hypothetical individualized colorectal cancer risk estimate.

Setting and participants  Eight focus groups were conducted with 48 adults aged 50–74 years residing in two major US metropolitan areas. Participants had high school or greater education, some familiarity with information technology, and no personal or family history of cancer.

Results  Several important conceptual problems were identified. Most participants thought of risk not as a neutral statistical concept, but as signifying danger and emotional threat, and viewed cancer risk in terms of concrete risk factors rather than mathematical probabilities. Participants had difficulty acknowledging uncertainty implicit to the concept of risk, and judging the numerical significance of individualized risk estimates. The most challenging conceptual problems related to conflict between subjective and objective understandings of risk, and difficulties translating aggregate‐level objective risk estimates to the individual level.

Conclusions  Several conceptual problems limit laypersons’ understanding of individualized cancer risk information. These problems have implications for future research on health numeracy, and for the application of risk prediction models in clinical and public health settings.

Introduction

The assessment of an individual’s risk of cancer is a matter of growing interest for researchers, clinicians and the general public. Since the development by Gail and colleagues in 1989 of a statistical model to predict a woman’s absolute risk of breast cancer, ¹ research in the field has burgeoned, resulting in the refinement of models to predict the risk of several other malignancies. ² As these models have emerged, their application has expanded beyond the research domain. In recent years, cancer risk prediction models have been used not only to guide recruitment for clinical trials, ³ , ⁴ but to counsel average‐ and high‐risk patients in clinical care settings, ⁵ , ⁶ and to communicate cancer risk information to the general public through Internet websites. ²

The broadening application of cancer risk prediction models has been driven by several healthcare trends. A heightened emphasis on evidence‐based medicine and informed patient choice has focused attention on improving patients’ knowledge of risks, and bringing this knowledge to bear in decision making. With respect to these aims, a growing body of research does suggest that perceptions of risk have important effects, ⁷ , ⁸ and that the provision of individualized risk information – i.e. information utilizing an ‘individual’s own risk factors to calculate personal risk based on epidemiologic data’ ⁹ – promotes outcomes such as more accurate risk perceptions, ¹⁰ , ¹¹ , ¹² and greater uptake of screening tests. ¹³ , ¹⁴

Regardless of these outcomes, however, the extent to which patients truly understand individualized risk information remains unclear. ¹⁴ , ¹⁵ This understanding is difficult to evaluate, ¹⁵ , ¹⁶ , ¹⁷ but there is ample reason to believe it is often inadequate. Several studies have documented significant limitations in patients’ numeracy (the ability to understand and use numbers) as measured in terms of basic mathematical knowledge and computational skills. ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ Low numeracy has been demonstrated in highly educated individuals, ¹⁸ , ²³ and raises doubts about people’s capacity to understand individualized risk information.

Past studies, furthermore, likely underestimate the scope of the problem, as most existing numeracy measures do not ascertain elements of understanding other than basic mathematical knowledge or skill. But understanding risk information is a conceptual as well as a computational task, involving the adoption of particular ways of thinking. ²⁴ Conceptual problems have received less attention, although they may confound people’s understanding. For example, laypersons do not always think about risk in the numerical terms used by experts, and may view risk estimates as expressions of concern rather than representations of mathematical probability. ²⁵ , ²⁶

Other conceptual issues relate to more fundamental uncertainties about the normative meaning of risk estimates derived from statistical models. These models use aggregate‐level epidemiologic data to generate predictions about groups of people with similar characteristics. Applying these models to make predictions about individuals rather than groups, however, raises difficult interpretive problems. ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ Individual risk estimates may be interpreted in at least two different ways: as subjective expressions of people’s belief or confidence about a future event or as objective statements of the event’s frequency. ³² , ³³ Subjective (belief‐type) and objective (frequency‐type) understandings of probability represent competing interpretations that have long been debated among statisticians and philosophers, ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³² , ³³ , ³⁴ , ³⁵ and may engender miscommunication between patients and clinicians. ³⁴ , ³⁶

Little is known about the extent to which these or other conceptual problems figure in laypersons’ understanding of individualized risk information. In the present study, we examined this issue using qualitative methods. Our specific aims were to explore laypersons’ understanding of the concept of risk and the meaning of individualized cancer risk information, and to identify important conceptual problems that may limit this understanding.

Methods

Study design, participants and data collection

The study employed semi‐structured focus group interviews to elicit the range of laypersons’ understandings of risk, and to explore these understandings in greater depth. The open‐ended interactive nature of this methodology ³⁷ , ³⁸ , ³⁹ , ⁴⁰ suited the current study, given its exploratory aim and abstract subject matter.

The study was initiated to inform the development by National Cancer Institute investigators of a new colon cancer risk prediction model, ⁴¹ similar to models developed for other malignancies and made publicly accessible through computer‐based programs and Internet websites. The study, therefore, targeted individuals likely to access such models through these means, and focused on colon cancer risk.

In June 2007, eight focus groups were conducted with 48 adults (three to eight participants per group) in two US metropolitan areas – Washington, DC and Chicago, IL (four groups per location). Participants were recruited over the telephone by a professional recruitment service, using eligibility criteria listed in Table 1. A purposive recruiting strategy was employed to select participants age‐eligible for most cancer screening interventions, and with average levels of exposure to health information, yet no extraordinary concern or expertise regarding cancer risk. Individuals with low education or poor familiarity with information technology were excluded, given our focus on likely users of the risk prediction model.

Table 1.

 Eligibility criteria

• English‐speaking

• Age 50–74 years

• Minimum education level of high school diploma and maximum level of Master’s degree

• Personal computer use of at least once per month

• Not employed in health, computer programming, mathematical or statistical fields

• Responsible for making one’s own health decisions

• No personal history of cancer

• No history of cancer in household members

In order to achieve sufficient within‐group homogeneity to encourage open discussion among members, ⁴⁰ the groups were stratified by a 2 × 2 × 2 design (Fig. 1) according to three factors potentially relevant to people’s understanding of risk information: sex, perceived colon cancer risk and subjective health numeracy (self‐rated ability to understand health‐related numerical information). ¹⁹ , ⁴² , ⁴³ Perceived cancer risk was categorized as ‘high’ or ‘low,’ using an item from the Health Information National Trends Survey. ⁴⁴ , ^a Subjective health numeracy was also categorized as ‘high’ or ‘low,’ using a single item from the STAT‐confidence scale developed by Woloshin and Schwartz to measure people’s confidence in their ability to understand medical statistics. ⁴⁵ , ^b

Figure 1.

 Focus group composition.

The groups were held at focus group facilities in Rockville, MD and downtown Chicago. Participants received $50 compensation. Each session lasted 2 h and was audiotaped and transcribed verbatim by a professional transcription service. Investigators observed all sessions behind a one‐way mirror; participants gave prior consent for audiotaping and observation.

Interview content

All groups were led by the same experienced professional focus group moderator who was not one of the research investigators and was naïve to the subject matter prior to being oriented to the study. The moderator used an interview guide consisting of questions based on review of the risk and decision science literature, and written by research team members with experience in focus group methods.

The interview guide began with open‐ended questions regarding the meaning of risk and of cancer risk. Participants were then informed about a new risk prediction model being developed at the National Cancer Institute. ⁴¹ They were told the model could calculate a person’s lifetime risk of colon cancer, using information about nine risk factors, which were listed on a chart. Next, participants were told to imagine that a friend, ‘Mr. or Mrs. Jones,’ had used the model and received a risk estimate of ‘9%.’ They were asked to write down how they would explain this estimate, and the facilitator then led a group discussion of responses. Finally, participants were asked their interpretations of three risk expressions with varying representations of uncertainty: a numeric point estimate (9%), a numeric range (5–13%) and a verbal comparison of the person’s risk relative to the population average (higher than average).

More detailed probes explored conceptual issues identified from existing literature – e.g. the distinction between subjective and objective risk, uncertainties involved in interpreting individualized cancer risk estimates. During the course of the study, minor revisions were made in the interview guide to clarify emergent themes.

Data analysis

Data presented in this paper related to participants’ understandings of risk and individualized cancer risk estimates. Findings regarding participants’ responses to different representations of uncertainty are reported separately.

Two investigators (PKJ Han and TC Lehman) performed in‐depth analysis and line‐by‐line software‐assisted coding of all interview transcripts using the program NVivo^® (Version 7; QSR International, Melbourne, Australia). Participants’ verbatim statements were categorized according to thematic content, and emergent themes were organized hierarchically within an overall conceptual schema. The interpretive approach was both deductive and inductive. We began analysis with prior knowledge of specific conceptual problems identified in the literature, which sensitized us to deduce their presence or absence in the interview text. At the same time, we remained open to new concepts and interpretations emerging from the data, consistent with an inductive ‘grounded theory’ approach. ⁴⁶

One investigator (PKJ Han) conducted initial analysis of all transcripts and generated a preliminary conceptual schema and codebook, which was iteratively reviewed and revised by the research team. Two investigators (PKJ Han and TC Lehman) then reapplied the revised codebook to the interview text. Coding decisions were compared, new themes were identified, and areas of disagreement were resolved through further discussions among research team members.

Results

A total of 48 respondents participated (Table 2). Primary interview domains consisted of the meaning of risk as a general concept and of individualized estimates of cancer risk. Within these domains, several major themes and concepts emerged.

Table 2.

 Characteristics of focus group participants

Participant characteristics	Total eight groups (n = 48)
	n	%
Age (years)
50–59	29	60
60–69	16	33
70–74	3	6
Sex
Female	25	52
Male	23	48
Race/ethnicity
White/Caucasian	27	56
African American	19	40
Hispanic	2	4
Other	0	0
Education level
High school graduate	1	2
Some college	22	46
College graduate	16	33
Post graduate	9	19
Perceived colon cancer risk
Low	29	60
High	19	40
Subjective numeracy
Low	22	46
High	26	54
Location
Washington, DC	22	46
Chicago, IL	26	54

The meaning of risk: multiple interpretations

Three main interpretations emerged in discussions of the meaning of risk as a general concept: (1) risk as danger, (2) risk as risk factors and (3) risk as uncertainty.

Risk as danger

Most participants equated the concept of risk with the notion of danger. Although some participants interpreted risk in neutral technical terms – e.g. as ‘chance,’‘probability’ or ‘general occurrence in the population’– the majority used terms such as ‘danger,’‘potentiality for danger’ and ‘vulnerability.’ This negative connotation of risk, stemming from the word’s etymology ⁴⁷ and previously described in terms of ‘risk as danger’ ⁴⁸ and the psychometric dimension of ‘dread risk’, ⁴⁹ received broad agreement in group discussions. Participants described the term risk as evoking negative emotions, making ‘alarm bells going off in my head.’

Risk as risk factors

When participants were further asked to describe the meaning of cancer risk, the dominant response was to promptly discuss various possible risk factors for cancer. Participants enumerated several known factors, including diet, exercise, genetics and family history, and speculated about many others including emotional stress and environmental toxins. The tendency to equate cancer risk with cancer risk factors has been described previously, and may reflect the public’s heightened awareness of disease risk factors, ⁵⁰ the abstract nature of the idea of risk – with respect to which concrete risk factors may be easier to conceptualize – or people’s underlying motivation to perceive risk as controllable. ⁴⁷

Risk as uncertainty

A final prominent theme was the interpretation of risk as uncertainty about future outcomes. Several participants acknowledged this uncertainty, noting the lack of correlation between risk factors and future outcomes:

Participant 1: How about nature? You can … have good diet, you don’t smoke, your family never had cancer, you take care of yourself, your workplace is great, but you still get cancer.

Participant 2: But there is so much out there that now they think it causes cancer. You can cut it out, then if it doesn’t cause cancer you can add it back. And even if you cut it out, you might get it anyway … And then my mother‐in‐law … She’s been smoking for more than 60 years, and her lungs are clear … unbelievable.
[Female, High Perceived Risk, High Subjective Numeracy Group]

Importantly, however, this view was not universally held; other participants equated risk with certainty about an outcome’s occurrence: ‘At risk for cancer says that your probability factor just went right out the window.’ Group discussions reflected a tension between these contradictory views:

Participant 1: So what you’re saying is that if your brother has it, that’s an indication that you’re going to have it because you’ve done the same thing.

Participant 2: No, I don’t think it’s an indication that you’re going to. I think the risk is higher.

Participant 1: Okay, who will … I’m stuck again on that word risk.

Participant 2: Well, I think risk is that amount that we’re talking about. I think there is a risk, there is a chance. I don’t think it says that you are going to.
[Male, High Perceived Risk, Low Subjective Numeracy Group]

This tension has been described in other qualitative studies, ³⁵ , ⁵¹ , ⁵² and long recognized by philosophers of statistics as reflecting a duality of meanings inherent to the risk concept. ²⁴ , ³² , ³³ Risk signifies that future outcomes are at once random and determined, unpredictable and predictable through known causal or statistical laws. The primacy of determinism and predictability in people’s understandings of risk was evidenced by participants’ reactions to a normative interpretation of the 9% risk estimate:

Facilitator: It means the risk. Nine percent risk of getting cancer means that out of 100 people who are like Mrs. Jones, 9 of them will develop cancer in their lifetime.

Participant 1: Will, yeah.

Facilitator: So what did that mean? Why did you question that?

Participant 1: Because Mrs. Jones had a 9% chance of developing … and now it’s telling me that 9 people out of 100 like Mrs. Jones will have it. They’re not just at risk. They will get it. I mean, that’s a different interpretation than what I had.

Participant 2: But that’s how they establish risk, is by how many people get it.

Participant 1: Wow!
[Female, Low Perceived Risk, High Subjective Numeracy Group]

The determinism implied by risk estimates may be sufficiently compelling to obscure the fact that such estimates provide only pseudo‐certainty – what Tanenbaum has described as a ‘certainty of what is probable.’ ⁵³

The meaning of individualized risk estimates: interpretive problems

Additional interpretive problems emerged in discussions about the meaning of the hypothetical 9% cancer risk estimate, and related primarily to: (1) the numerical significance of the risk estimate, (2) the distinction between objective and subjective risk and (3) the translation of objective risk estimates to the individual domain.

Numerical significance

The numerical significance of the 9% lifetime colon cancer risk estimate was an important theme in all groups. In written responses prior to group discussion, 33/48 participants (69%) explained the 9% risk estimate not in mathematical terms, but in terms of qualitative judgements of its numerical significance – e.g. ‘You are at low risk of developing colon cancer in your life.’ The remainder used mathematical terms – e.g. ‘nine out of 100’– with or without a judgement of significance.

At least some participants, however, expressed uncertainty about the numerical significance of a 9% risk. This uncertainty emerged vividly in group discussions, where computational errors and judgemental biases became evident. Some participants had difficulty converting percentages to frequencies – e.g. calculating that 9% of 10 people equaled nine people. Others mistakenly believed that the 9% number was on a 10‐point maximum scale, thus signifying high risk. Participants also exhibited well‐known judgemental biases such as the ‘ratio bias’ ⁵⁴ in comparing probabilities – the tendency to base comparisons on the magnitude of the numerator instead of the ratio:

Participant: Everybody is assuming that we’re talking about out of 100. Suppose somebody walked in and said, “You’ve got a 9% chance of catching cancer amongst the people in here.” That changes the whole meaning of it … that right there says very low.
[Male, Low Perceived Risk, High Subjective Numeracy Group]

Consensus was also lacking about whether 9% represented a low, average or high risk. In groups three to eight, we probed participants’ beliefs regarding the average lifetime colon cancer risk. All participants overestimated this risk; most responses were 50% or higher, a small number ranged from 10 to 50% and none approximated the actual 6% risk for US adults aged 50 years and older. ⁵⁵

This tendency to overestimate cancer risks has been well described, ⁵⁶ , ⁵⁷ as has the tendency to equate 50% with one’s own risk, which may be attributable to respondents’‘epistemic uncertainty’ regarding their own knowledge. ⁵⁸ However, participants’ responses also suggested a propensity to use 50% to indicate at least some chance of cancer for that individual, with points other than 50% indicating a higher or lower chance:

Facilitator: But so, getting back to colon cancer, a lot of you are saying 50% or over 50%… What do you think it is, B?

Participant: I kind of think about 50/50.

Facilitator: Fifty percent of the people die from colon cancer?

Participant: Well, not for the risk … I mean … I think all of us have a chance …
[Female, High Perceived Risk, Low Subjective Numeracy Group]

For this participant, ‘50/50’ signified not a point along a continuum of probabilities, but a dividing line for a conceptual dichotomy: the presence or absence of any possibility of cancer, irrespective of its probability. This all‐or‐none type of thinking may manifest an ‘insensitivity to probability’ for potential outcomes having strong affective meaning. ⁵⁹ , ⁶⁰

For other participants, the equation of 50% with the average risk reflected confusion about whether ‘average’ risk meant the mean, the median or the mode:

Participant: If we talk about average statistically being 50%, but if you say “I am average,” does that mean that I am on the 50% and there is 50% higher and 50% lower? Or does it mean that the majority of the people in that community are like me?
[Male, High Perceived Risk, Low Subjective Numeracy Group]

Similarly, some participants misinterpreted the 9% risk estimate as a percentile rank:

Participant: I still say that’s low because I’m looking at it like, hey, those other 91 people, their chances are higher than mine … As I say, there is 8% that is lower than me … or 8 other people, rather. But that 91 are higher than I am.
[Male, Low Perceived Risk, Low Subjective Numeracy Group]

Computational errors and biases such as these confounded participants’ interpretations of the 9% risk estimate. The remainder of the interview explored the extent to which interpretations manifested more fundamental conceptual problems.

Objective vs. subjective risk

An important problem related to the distinction between subjective and objective risk. After eliciting participants’ written interpretations of the hypothetical 9% risk estimate, we probed perceptions about which interpretation was correct – the objective, frequency‐type (‘nine out of 100 people like Mr. Jones will develop colon cancer’) vs. the subjective, belief‐type (‘scientists are 9% certain or confident that Mr. Jones will develop colon cancer’) interpretation. Most participants endorsed the objective interpretation. However, consistent with data from Weinfurtet al., ³⁶ a minority gave credence to the subjective view, or believed the interpretations ‘were all saying the same thing.’

Furthermore, even when participants explicitly endorsed the objective interpretation, additional probing revealed a second, subjective interpretation:

Facilitator: Let’s go back to this 9%…Why do you think that’s low, N?

Participant: It’s low; I don’t know how to explain it. But 9% of 100 would be 9 people out of 100; so you would be one of the 9 possibly. But I don’t know the answer … when you think in terms of 100% it doesn’t seem very high to me, but it would probably be a good idea to talk to your doctor.
[Female, Low Perceived Risk, Low Subjective Numeracy Group]

Irrespective of the magnitude of objective risk (e.g. nine out of 100), the remaining uncertainty is the individual’s probability of being in one risk group or the other – i.e. the unlucky nine vs. the lucky 91. Any given person has some unknown chance of being in either group; this chance represents a second‐order subjective probability – a probability of a probability – for which there is no normative answer. ³³ , ³⁴ , ³⁶ , ⁶¹ This independence of subjective and objective probabilities may explain how individuals can believe their personal cancer risk to be 50%, or arrive at seemingly idiosyncratic judgements about the significance of a risk estimate:

Participant: 9% is not a great chance but chances are he will get colon cancer… 9% means to me that you still have a good chance of getting it … He’s got a real good chance … somebody else might tell him well, your chances is slim.
[Male, High Perceived Risk, Low Subjective Numeracy Group]

Kahneman and Tversky noted that because of the logical independence of subjective and objective probabilities, different degrees of confidence can be assigned to the same probability judgement. ⁶² Consequently, people’s subjective risk estimates may turn out to be poorly calibrated – i.e. not matching the objective empirical frequency of an event – a problem termed the ‘overconfidence effect’. ⁶³ , ⁶⁴

It is problematic, however, to ascribe overconfidence to people’s subjective probabilities for single events such as cancer in an individual. Objective probabilities pertain strictly to frequencies of events in the long run or among a population of individuals; thus, they cannot be used to calibrate subjective probabilities for a single event. ⁶⁵ This issue manifests deeper conceptual difficulties involved in individualizing objective risk, which we probed further.

The individualization of objective risk

The most difficult conceptual problems that emerged in group discussions related to the translation of risk estimates to the individual domain. Objective cancer risk is calculated in terms of the number of individuals, in a given population, expected to develop cancer. But what does that mean with respect to the risk of a cancer for a single individual with one life to live? The paradox involved in individualizing objective risk emerged in participants’ attempts to understand the relationship of statistical ‘people’ to an individual’s ‘chances’:

Participant 1: I don’t know. I’m still confused about the 9%.

Participant 2: I think it’s 9 out of 100.

Participant 3: There’s 9 out of 100 chances.

Participant 1: Does it necessarily mean 9 out of 100 people or is it just ….

Facilitator: What’s the difference between the people and the chances?

Participant 1: My first inclination was that it was a 9% out of 100 that she could get it. Now, I’m thinking maybe it’s 9% out of 100 chances.

Facilitator: Okay. So that 100 base could not necessarily be people but chances. Is that what you were thinking?

Participant 1: I think that’s what I was thinking because I was not necessarily equating that to like 9 out of this many.
[Female, Low Perceived Risk, Low Subjective Numeracy Group]

The confusion reflects the non‐intuitive and incoherent nature of the conceptual leap from the number of affected people in a population to an individual’s chances of being affected. The idea of individual probability has been the subject of a long‐standing debate among philosophers of statistics, and many have criticized the concept. ³² , ³³ , ³⁴ As Hacking has asserted: ‘It does not make sense to speak of the ‘frequency’ of a single event. A patient either has, or has not, got strep throat.’ ³³ Exactly what the frequency of an event in a group of people means for an individual contemplating his/her own chances is not obvious.

The problem here, however, lies not only in the concept of single event probability, but in the derivation of individualized risk estimates. Because these estimates are based on empirical observations in a particular reference population, their applicability to a given individual’s case depends on the extent to which the characteristics of the reference population match those of the individual. But it is practically impossible to account for all risk‐relevant characteristics; there will always be missing, unmeasured or confounding factors that introduce uncertainty in applying objective risk estimates to individuals. ³² , ³³ , ⁶⁶ Several participants acknowledged this concern in comments that the risk prediction model was ‘not inclusive enough,’ that ‘more information on the person’ was needed; similar perceptions have been found in other studies. ¹⁰ , ⁶⁷

Even deeper logical contradictions emerged, however, regarding the concept of the reference class. On the one hand, reference classes of increasing number and specificity are necessary for objective risk estimates to discriminate sufficiently at the individual level; this is the ‘principle of the narrowest reference class’ ³² that participants were able to discern:

Participant: It’s not just demographics, it’s the number of demographics. Just make it narrow so that you make sure I’m in this group, and don’t include a bunch of people who have nothing to do with my life.
[Male, Low Perceived Risk, High Subjective Numeracy Group]

On the other hand, participants recognized a logical paradox in this endeavour: as an individual’s reference class becomes more narrowly defined, it may become less meaningful psychologically. The concept of the reference class was already exceedingly abstract for many participants, who expressed difficulty imagining an idealized sample of ‘totally similar people’ or ‘clones’ of oneself. But the idea of a homogeneous reference class was not only hard to imagine, but meaningless to some participants:

Participant 1: I walk out this door and I randomly go through this building and pick out 91 people, not that I look at this room here and I see we’re all male … roughly the same age … I don’t want 9% of us. I want 9% of the whole population.

Facilitator: Why?

Participant 1: Because the lifetime statistic for me is against … everybody in my life. Against the total, not against a select group. If you want to say, well, I’ve got a 9% chance out of 50‐to‐70‐year‐old males that live in northern Virginia, that’s a different statistic than 9% against all Americans … against everybody in the Western Hemisphere. That’s different than the whole world.

Facilitator: So you want the whole world?

Participant 1: I want a figure that I can go out and I don’t have to go, “Okay, let’s see what my chances are against you, you, you, you, and you,” which would leave you out because you’re a female and the statistic right now is just us guys.
[Male, Low Perceived Risk, High Subjective Numeracy Group]

Knight pointed out the conceptual paradox that ‘if we had absolutely homogeneous groups we should have uniformity and not probability … men exactly alike and identically circumstanced would all die at once … and the idea of probability becomes meaningless.’ ⁶⁶ Reference class homogeneity, furthermore, may be not only logically, but psychologically problematic. Participants’ comments suggest that the narrowest reference class possible may not represent the most meaningful basis of comparison for understanding risk.

Discussion

In this qualitative study, we identified several important conceptual problems in laypersons’ understanding of individualized cancer risk information. These problems have important implications for both research on health‐related numeracy and the application of risk prediction models – for cancer as well as other diseases – in clinical and public health settings.

Firstly, our data corroborate past research showing that the concept of risk has multiple meanings beyond the strictly mathematical and normative. For most participants, risk was not a neutral statistical concept, but instead signified danger and emotional threat. Most participants thought of cancer risk in terms of concrete risk factors rather than abstract probabilities. Furthermore, while participants acknowledged the uncertainty central to the concept of risk, many also equated risk with certainty about future outcomes.

These findings shed light on the ways in which laypersons’ mental representations of risk diverge from experts’ normative views. ²⁶ , ⁴⁹ , ⁵⁴ , ⁶⁸ Our data add to accumulating evidence that laypersons’ everyday thinking about risk is shaped by different conceptual models that are more intuitive than mathematical, more qualitative than quantitative and more deterministic than probabilistic. ²⁶ , ⁶⁹ The findings support Reyna and Brainerd’s distinction between ‘gist’ and ‘verbatim’ representations of risk, ⁷⁰ and suggest the precedence of gist‐based conceptions in laypersons’ understanding of individualized risk information.

The more problematic findings of our study, however, related to particular difficulties involved in understanding individualized risk estimates. Participants in all groups – representing both low and high self‐rated numeracy – had trouble determining the numerical significance of these estimates, because of computational errors and judgemental biases. They miscalculated frequencies, misunderstood proportions, overestimated average risks and confused different measures of central tendency. These basic numeracy deficits show that laypersons’ understanding of risk information cannot be taken for granted by risk model developers, and suggest specific areas that might be researched further and intervened upon in future risk communication efforts.

The interpretive difficulties posing the greatest challenge for participants were conceptual rather than computational, arising from logical problems inherent to the very idea of individualized risk. Participants implicitly switched back and forth between subjective and objective understandings of this risk. Although Hacking has argued that most people do not know or care about the incompatibility of these understandings, ³³ our study illustrates vividly that laypersons do struggle with it. Furthermore, the conflict between subjective and objective understandings of risk remains a critical problem, because the way it is resolved determines the precise meaning of individualized risk estimates and the validity of using predictive models in different settings.

The problem acknowledged implicitly by some participants is that objective estimates do not literally represent an individual’s true cancer risk. One can be presented with a nine out of 100 risk but the abiding uncertainty is the individual’s second‐order probability of being one of the nine or not. This uncertainty will always remain for several reasons, including unmeasured risk factors and difficulties assigning individuals to sufficiently narrow reference classes, which participants also recognized. The individual discriminatory power of objective cancer risk estimates is additionally limited by the weakness of the risk factors incorporated within existing predictive models. ²⁷ , ²⁸ , ³¹

The limited predictive significance of objective risk estimates, however, is not merely a pragmatic problem that might be overcome with more empirical evidence or the discovery of stronger risk factors. At a conceptual level, we simply cannot determine – or speak of – an individual’s objective risk for any single event. ³² , ³³ The methods and language of objective probability are confined to the realm of events that are repeatable, whether in time (by a single individual) or space (across a population of individuals). An individual contemplating the risk of cancer in his/her own lifetime, however, is neither a gambler rolling a die continually nor a member of a cohort of identical clones. The endeavour to individualize risk brings to the fore the logical incommensurability of subjective and objective risk: with respect to an individual’s future, we can speak only in subjective terms of the strength of our beliefs.

This does not invalidate the effort to calculate objective disease risk and to use this knowledge to inform individuals. It does, however, call for greater clarity about the meaning of objective risk, and awareness of the uncertainties involved in applying risk prediction models in clinical settings. Objective risk estimates can inform individuals’ subjective risk judgements, ³² and this is their principal value. At the same time, individuals’‘true’ disease risk remains unknown, because of not only the logical difficulties of applying objective risk information to the individual case, but also the many factors that engender uncertainty regarding the accuracy, reliability and credibility of objective risk information – a type of uncertainty that decision theorists have termed ‘ambiguity’. ⁷¹ Although risk model developers and clinicians may appreciate these uncertainties, laypersons – even those with high subjective numeracy – may not.

Comprehension of these conceptual uncertainties is, thus, a critical element of health numeracy that warrants greater attention. By eliciting people’s own accounts of their reasoning, our study was able to explore the breadth and depth of conceptual problems in people’s understanding of individualized risk estimates. Yet, the generalizability of our findings remains unknown; the study samples were limited in size and diversity, and interpersonal dynamics of group discussions may have limited – as well as facilitated – elicitation of important ideas. Further research is needed to address these limitations and to examine the extent to which conceptual understanding can be measured and related to other elements and outcomes of numeracy.

Another potential limitation of our study was its artificial nature; participants were led through a mental exercise of thinking about an abstract concept and a hypothetical risk estimate calculated using an unfamiliar technology. We posed questions that participants may never have contemplated before, and their responses could have been artefacts of the elicitation process. One can ask, however, whether the situation is substantially different in the real world, where abstract risk information abounds on the Internet and other venues – often with little or no supporting explanation. This calls for further research to critically examine both the settings in which disease risk prediction models are used, and the information that is being communicated to model users.

Our study suggests several key conceptual issues that deserve particular emphasis in communicating and interpreting individualized risk information. Laypersons need to recognize the caveats involved with translating objective risk estimates to the individual domain, and to be made aware that these estimates do not literally represent their own risk of disease. Such an awareness can be liberating, ⁶¹ and might be facilitated by interventions that teach statistical reasoning skills, ⁶⁹ , ⁷² or emphasize the qualitative bottom‐line ‘gist’ meaning of risk information. ⁵⁴ At the same time, researchers and clinicians need to be clear that because patients’‘N of 1’ (subjective) risks do not mean the same thing as the aggregate (objective) risks calculated from mathematical models, ⁷³ , ⁷⁴ it is impossible to know the ‘accuracy’ or ‘calibration’ of patients’ risk perceptions. This calls for flexibility and caution in judging patients’ responses to objective risk information and in using individualized risk information for purposes other than informing patients – e.g. encouraging behaviour change. The fundamental uncertainties inherent to the interpretation of individualized risk estimates make such tasks ethically problematic.

Greater understanding of both the value and the limitations of individualized risk information is a need made all the more critical by the growing movement of personalized health care – care that purports to be ‘calibrated to each patient and personally effective for each individual’ and based on individual differences in disease susceptibility and response to therapies. ⁷⁵ This bold vision calls for clarity in our language and expectations, to ensure that people understand what is known, not known and paradoxical about the idea of individual risk.