Translating cancer risk prediction models into personalized cancer risk assessment tools: Stumbling blocks and strategies for success

Abstract

Cancer risk prediction models such as those published in Cancer Epidemiology, Biomarkers, and Prevention are a cornerstone of precision medicine and public health efforts to improve population health outcomes by tailoring preventive strategies and therapeutic treatments to the people who are most likely to benefit. However, there are several barriers to the effective translation, dissemination, and implementation of cancer risk prediction models into clinical and public health practice. In this commentary, we discuss two broad categories of barriers. Specifically, we assert that the successful use of risk-stratified cancer prevention and treatment strategies is particularly unlikely if risk prediction models are translated into risk assessment tools that (1) are difficult for the public to understand or (2) are not structured in a way to engender the public’s confidence that the results are accurate. We explain what aspects of a risk assessment tool’s design and content may impede understanding and acceptance by the public. We also describe strategies for translating a cancer risk prediction model into a cancer risk assessment tool that is accessible, meaningful, and useful for the public and in clinical practice.

Scientists have been developing mathematical models to predict disease risk and clinical outcomes for decades (1–5). These models can be used to increase scientific understanding of disease incidence, prevention, detection, and treatment, and to understand long-term outcomes such as disease progression and survival (5). In recent months, Cancer Epidemiology, Biomarkers & Prevention has published two articles describing the development and use of colorectal cancer risk prediction models.

In “A New Comprehensive Colorectal Cancer Risk Prediction Model Incorporating Family History, Personal Characteristics, and Environmental Factors,” a colorectal cancer incidence model is reported that includes lifestyle behaviors (e.g., diet), personal medical history (e.g., colorectal cancer screening history), and a detailed family history that accounts for the sex, age at diagnosis, and relationship to the affected individual (6). The authors concluded that the model showed promise for use in risk-stratified screening recommendations (rather than uniform guidelines based only on age and colorectal cancer diagnosis in a first-degree relative).

In “Clinical and Economic Impact of Tailoring Screening to Predicted Colorectal Cancer Risk: A Decision Analytic Modeling Study,” the authors compare uniform screening guidelines to tailored risk-based screening guidelines in quality-adjusted life years, cost, and cost-effectiveness in a U.S. population (7). They concluded that tailored risk-based screening could be preferred to uniform screening for the average risk population; however, even “relatively modest” rates of misclassification of individuals into different risk categories or increases in the cost of risk prediction tools could tip the scales in favor of uniform screening.

The common theme in these two papers is that, under certain circumstances, cancer risk prediction models could be used in clinical practice to inform which patients get screened, at what ages, and how often. This assumption echoes the basic premise of precision medicine and public health: providing patients, caregivers, and providers information that fosters informed and shared decision making and leads to improved health outcomes (8, 9). Recognizing this, some researchers have translated risk prediction models into clinician- or patient-facing risk assessment tools that calculate the likelihood that an individual will develop a disease in the future. These personalized risk assessment tools (also called risk calculators) have been developed for a variety of cancers, including those of the colon, breast, lung, skin (melanoma), kidney, and cervix (10, 11). Risk assessment tools have also been developed for chronic conditions such as heart disease, stroke, and diabetes (12). Whereas some tools are intended for use in clinical practice, with the goal targeting primary prevention of disease (13), other tools are disseminated via the internet – including through popular health information portals or advocacy groups – and therefore are widely accessible to the public outside of clinical settings (11, 14, 15).

Benefits and Limitations of Personalized Risk Communication

Developing and disseminating risk assessment tools assumes that general users (i.e., patients or members of the general public) will calibrate their subjective perception of risk with their risk as calculated by the prediction model. Presumably, this leads to behavior change in the form of tailored approaches to screening (e.g., screening higher-risk individuals earlier and more often, and screening average or lower-risk individuals later and less often) or engagement in disease prevention behaviors. However, the evidence supporting their utility for improving uptake of cancer prevention and detection behaviors is equivocal.

Providing personalized (rather than non-personalized) risk information can increase concordance of people’s subjective risk perceptions with the calculated estimates for cancer and other common health conditions (16–23). When included in decision aids, they encourage informed decision making about colorectal and breast cancer screening (24, 25). Although the effect of personalized risk information on actual uptake of cancer screening appears limited (25), such information may influence decisions more for people at high risk (24) or when the personalized information results in a tailored physician recommendation (26). However, few studies have demonstrated that providing people with personalized cancer risk information can increase or decrease their intentions to engage in cancer prevention and detection behaviors (23, 27). A systematic review of systematic reviews concluded that personalized disease risk information does not generally produce actual behavior change for smoking, physical activity, diet, or alcohol consumption (28) (This statement pertains to personalized risk communication based on risk prediction models that do not include genetic testing. For information about the effects of genetic testing on decisions, see (29, 30)).

Many factors have likely contributed to the limited effectiveness of personalized risk assessment tools on health promotion and disease detection behaviors. One broad category of factors pertains to the design and content of the tool. Specifically, the tool must provide information in a way that people can understand and apply. However, many cancer risk assessment tools violate recommended risk communication formats and strategies (14, 31–33) by providing risk estimates only in numbers, not including a visual display or pictorial depiction of risk, using complex numerical formats such as 1 in N or number needed to treat, and having high literacy requirements (34). These characteristics can impede users’ ability to understand the information provided (35–37). Furthermore, lack of transparency about what organization developed the tool (i.e., the information source), the underlying motivations of the developers, whether the tool is based on a validated risk prediction model, who are the intended users (e.g., general public or medical professionals), and the rationale underlying the selection of risk factors may limit users’ ability to evaluate the credibility of the tool and thereby limit its effectiveness (14, 38). Such transparency may be especially important when describing how race is associated with risk (39, 40).¹

Researchers must also recognize that it is unrealistic to expect people to change their behavior based only on being given a probabilistic estimate of developing a disease. Not only are the concepts of risk and risk comprehension far richer than simply recalling a single probability estimate (41–45), but there is also a broad range of psychosocial, interpersonal, and socio-contextual factors that influence behavior and behavior change (46–48). Cancer risk assessment tools that attempt to promote behavior change without recognizing the richness and complexity of people’s lived experiences are unlikely to be effective and may inadvertently reduce motivation to change (49, 50). For example, people who work physically demanding jobs or jobs with limited or no paid sick leave may not be able to take the actions needed to engage in colorectal cancer screening without risking their employment or incomes (51).

A second broad category of factors that may impede the ability of personalized risk assessment tools to affect health decisions and behaviors relates to people’s tendency to either reject risk information, or to reject its implications for them personally. Although participants often accurately recall the information a risk calculator provides, and in many cases report perceptions of risk that are more congruent with the calculated estimates, perceptions rarely become completely congruent (19, 23, 52, 53) (but see (21)). People might disbelieve risk information for a multitude of reasons. Some people may reject the information because it conflicts with their prior beliefs or expectations (52, 54–56). For instance, the Study of Tamoxifen (STAR) Trial reported that women’s personal experiences with breast cancer, such as having a family history or having had a biopsy, led them to expect that they were at high risk (54). When they received a probabilistic estimate that did not appear as high as they anticipated, they rejected the risk information.

Other studies reported that users of personalized risk assessment tools rejected the results because the participants doubted the validity or reliability of the risk prediction model (52, 55, 56). For example, in one study some women disbelieved their breast cancer risk estimates because the risk prediction model excluded aspects of their family history, medical history, or lifestyle behaviors that they felt affected risk (52). Participants in another study reported feeling mistrust for the creators or corporate sponsors of the tool, suggesting concerns about conflicts of interest (55). Still other participants had concerns about the ability to determine an individual’s cancer risk using population-based data (52, 57). This issue is well-known in epidemiology (58), and was highlighted as “the most difficult conceptual problem” discussed in focus groups of people who were asked to grapple with personalized colon cancer risk information (57). It is also possible that some participants may question the generalizability or personal relevance of the model, wondering whether it was created based on data from people from their racial/ethnic group (59).

Another element of risk assessment tools that may exacerbate uncertainty about the validity of the results among the general public is the complexity of the information required. People who use tools that require results of medical tests or diagnoses (e.g., prostate specific antigen levels; type of cancer (60)), detailed family history (61, 62) that may not be available to all individuals (e.g., adoptees, people whose families never discussed cancer), information from many years in the past (e.g., asking a 50-year old person to remember their weight at age 18), and highly detailed information (e.g., number of servings of meat eaten per week) (63) may be uncertain about the information they enter and therefore be justifiably skeptical of the accuracy of the resulting risk estimate. Reducing information complexity by asking for categorical information (e.g., eat more than or less than 5 servings of meat per week) or allowing for missing data may increase the usability of the tools without significantly reducing their calibration or discrimination (64). Having health care providers complete the tool with patients may allow patients an opportunity to ask questions about the meaning of certain elements of the tool or diagnoses, and thereby increase the accuracy of the results. A virtual counselor may be useful for patients who are concerned about privacy (65). Provider-mediated tool use, or making the tools available in clinical settings, may also overcome barriers to access due to limitations in digital literacy and the digital divide (66).

Practical Advice for Facilitating and Optimizing the Translation of Cancer Risk Prediction Models into Cancer Risk Assessment Tools

If a cancer risk assessment tool is placed online and is not access-restricted, it will likely be found and used by members of the public, possibly without a clinician’s involvement. Thus, risk prediction modelers who wish to translate their model into practical applications should consider the following before they begin the process of developing the website, to ensure that their information is communicated in a way that people can comprehend, will accept rather than reject, and will use appropriately when making health decisions. Many of these suggestions will also be beneficial in increasing the tool’s comprehensibility and utility for practicing clinicians, who may desire a patient-friendly way to discuss cancer risk during clinical encounters.

1. Make sure the tool is written at a 6^th grade reading level, define or describe medical terminology in straightforward terms, and include pictures, drawings, or illustrations (67, 68). Health literacy (i.e., the ability to understand and use health information to make medical decisions) and its conceptual cousins numeracy and graph literacy are limited in the U.S., particularly among individuals who are older than 65 years of age, are indigenous or people of color, have limited formal education or low income, or are non-native speakers of English, refugees, or immigrants (69, 70). Clinicians are not immune to having limited numeracy (71).

2. Risk assessment tools should be designed in a way that allows users to easily and correctly enter the information needed to calculate risk estimates. Risk assessment tools often require information that is difficult, sensitive, and potentially stigmatizing for the general public to answer (e.g., requesting detailed family history, biological, medical, or behavioral information that the user does not have access to); this might limit the extent to which users view the tool as personally relevant and useful.

3. Refer to established risk communication guidelines (31, 32) and adopt their recommended risk communication formats and strategies. Hundreds of studies have investigated how to communicate health risk information in ways that the public can understand, find motivating, and use effectively when making health and medical decisions (for reviews, see (31–33, 72, 73)). Although the field has identified some best practices (e.g., use percentage or frequency format, but not 1 in N; bar graphs and icon arrays – sometimes called “pictographs – facilitate understanding, but pie charts do not), other areas are understudied (e.g., how to best convey multiple disease risk estimates simultaneously).

4. Provide the information people need to make a decision, rather than the information scientists would like to see (74). The level of detail required for scientific evaluation is often more than the level of detail that is needed for people to make good decisions. For example, a more precise risk estimate that includes decimals may be recalled with less accuracy and perceived as less credible than an integer-based risk estimate (75). Indeed, there are circumstances in which only qualitative risk categories (e.g., “high risk”) and not specific numerical estimates are needed to convey the intended meaning of the communication (74).

5. Provide information that consumers need to evaluate the validity, generalizability, personal relevance, and credibility of the risk prediction model, including information to help patients understand what role each risk factor plays in their risk estimate (38, 76). Mere recall of a specific risk estimate should not be interpreted as the individual’s understanding of that estimate, nor should it be assumed that they believe that the risk estimate is an accurate reflection of their personal risk (42). This may be particularly relevant for populations who are underrepresented in medical research and, therefore, are also underrepresented in cohorts used to develop risk prediction models. Unfortunately, little is known about how to make risk assessment tools believable to individuals who reject their results; more research is needed.

6. Provide evidence-based “action steps” for people to take to reduce their risk. If the behavior is complex (e.g., engaging in lifestyle-based cancer prevention behaviors or accessing cancer screening), provide links to respected national and local resources to facilitate change. Providing an estimate of how much changing behavior might reduce risk also may be helpful (42, 46). Providing risk information without also providing information about how a person can reduce their risk can prompt people to engage in behaviors that help them avoid thinking about the risk (e.g., by thinking of reasons to reject the information) rather than behaviors that help them avoid the actual health problem (e.g., getting screened) (49, 50).

7. Involve behavioral scientists early in the development process. Helping people change their cancer prevention and detection behaviors is an exceedingly complex and multifaceted endeavor (46–48). Attempting to change people’s behavior without seeking relevant expertise may inadvertently impede, rather than support, behavior change.

8. Implementing novel tools, technologies, and practices into clinical and public settings is also a highly complex endeavor that, if done poorly, could negate its benefits. To maximize the likelihood that the target audience not only uses the tool, but also finds it useful, it is important to design the risk assessment tool with dissemination and implementation in mind (77, 78). To accomplish this, developers should: (a) convene an advisory board comprised of stakeholders from several perspectives at multiple levels of influence (e.g., patients, physicians, clinic staff, behavioral scientists, and information technology specialists) to elicit their insight about the content and format of the tool and the extent to which its use may affect clinical operations, and (b) subject the tool to iterative rounds of usability and acceptability testing to ensure that it meets the needs of its intended users (79, 80), whether they are health care providers, patients, and/or members of the general public. Ideally, by the end of this process, users (whether provider or patient) will not need any specialized training to use the tool or understand the results.

Conclusion

Risk prediction models like those developed by Zheng et al. and Ladabaum et al. hold great promise for personalized medicine and for improving patient health and well-being. Indeed, risk prediction models are already being translated into risk assessment tools and incorporated into patient-facing tools to assist patients and providers in making informed decisions about whether and when to initiate and stop cancer screening, to take medication to prevent breast cancer, to foster discussions about family history, and to engage in cancer prevention behaviors (10, 11, 25, 81, 82). However, there are pitfalls to using these tools, as their utility depends on the extent to which they support patient understanding, acceptance, and, when appropriate, behavior activation or change. We encourage the continued development of risk prediction models and support continued consideration of how to integrate risk assessment tools into routine clinical and public health practice in the service of improving health outcomes.