Factors to Consider in Developing Breast Cancer Risk Models to Implement into Clinical Care

Abstract

Purpose of the review:

This article outlines considerations for individuals interested in developing and implementing breast cancer risk models and has relevance for individuals developing risk-models with the goal of implementing them into health systems.

Recent findings:

There has been increased focus on developing risk models for clinical use—often with less attention model implementation. Epidemiologists developing risk-models must think through model outcomes including stakeholder needs, time horizons, terminology and reference groups and clarity on what actionable steps are for health systems, providers and patients following its implementation.

Summary:

Model performance needs to be evaluated relative to complexity of the model to be implemented–not just from the risk-prediction perspective, but also from the burden on patients, providers and systems for the amount and frequency of required data collection and with clear actionable steps to be taken with the information collected.

Introduction

No cancer screening is perfect. Mammography is a great example of this because the benefit to harm ratio has been well studied and is known to be influenced by several factors.(1, 2) Overall, one in ten women who undergo mammography are recalled for additional testing, which includes additional imaging (ultrasound and/or diagnostic mammography) and/or more invasive work-up which vary in invasiveness.(2) Five in 100 women undergoing additional diagnostic testing will be diagnosed with breast cancer and 8 in 10,000 women undergoing screening will have their tumors missed.(3, 4) These screening harms (diagnostic work-ups and missed cancers) vary in women based on several characteristics. For example, women with dense breasts are more likely to experience greater harms from mammography. Similarly, younger women are more likely to have a false negative mammogram, in part, because they are more likely to have faster growing tumors that are more aggressive that arise during the recommended screening interval. As a result, there has been an increased focus on improving the benefit to harm ratio for breast cancer screening, including risk-based screening through risk-based screening recommendations and tools to help women and providers with shared decision-making.(5–7)

Risk models – risk of what and for whom?

Risk models have been used in clinical practice for decades.(8, 9) However, with the growth of electronically available data, there has been increased focus on using these data to develop risk models with the intention of implementing into practice(10)—often with less attention paid to how implementation will happen. There has also been an increased focus on stakeholder engagement, which has shed an important light on varied needs and desired outputs for risk models. For example, geneticists might desire a risk model to identify individuals who need counseling to identify individuals who would benefit from genetic testing. Primary care providers may want to know which of their patients would benefits from earlier screening initiation, more frequent screening, supplemental screening, chemoprevention and stopping screening earlier. Radiologists might want to know which women or lesions may be most likely to develop into a high-risk or more aggressive cancer vs. ones that are more likely to be slow-growing or indolent. Women may have an interest in different risk models for breast cancer over the course of their lifespan with early identification of who is at risk for a genetic mutation (for prevention strategies) to identification of who would benefit from more-or less-frequent screening including starting and stopping ages. It is impossible that one risk model could be developed to predict outcomes that are of interest to different stakeholder groups.

In addition to modeling different outcomes, models often have varied outputs including absolute vs. relative risk and time horizons (e.g., lifetime vs. 5- or 10-year risk). Absolute vs. relative-risk outputs fuel confusion for women and providers, given challenges with health literacy.(11) Time horizons for risk models also make interpretation challenging for patients and providers, with an important gap in how these time horizons map to specific outcomes of interest for varied stakeholders and specific actions that could be taken to reduce risk. The Breast Cancer Surveillance Consortium (BCSC)(12) has validated breast cancer screening risk models that predict risk of breast cancer in the next 5- and 10-years with the goal of helping to improve benefits and reduce harms of breast cancer screening.(13–17) Specific goals include identifying women who could have longer-screening intervals for women at lower risk. Additionally, the BCSC has been evaluating whether supplemental imaging could improve outcomes for women at the highest risk for breast cancer in the next 5-years.(18) The implementation of these BCSC models rely on information collected at the time of screening, since it incorporates breast density (one of the strongest breast cancer risk factors) and therefore is a model that relies on an initial screen to develop risk-prediction. This type of a model is distinctly different from those designed to identify women at high-risk for familial breast cancer; the evaluation of risk should begin at a much earlier age than screening is recommended.(19)

Terminology and reference groups matters

As with many risk factors, reference groups matter. For example, using almost entirely fatty breasts (Breast Imaging Reporting and Data System (BIRADS)(20) breast density category a) as the referent group yields a 3–7-fold elevated breast cancer risk in women with extremely dense breasts.(21) Comparison categories matter for absolute and relative risk estimates and need to be evaluated in terms of the overall distribution of risk in the population.(22) Mammographic breast density is a perfect example with roughly 10% of the population of screening age women falling into fatty and extremely dense breast and 40% into scattered and heterogeneously dense categories. Using fatty as a reference group puts 90% of women of screening age into high-risk categories. For example, when scattered fibroglandular is used as a referent group rather than almost entirely fatty, hazards ratios for women with extremely dense breasts decrease to ~2.0.(16) There is a need to continue to standardize language around risk for patients and providers and consider integrating the concept of “lower-than-average” into models and vernacular.

What will be done with model outputs?

Additionally, there needs to be clear actionable steps at all phases of risk-assessment with clarity on clinical pathways for individuals identified across risk levels. For breast cancer, this includes considering identifying women much earlier than recommended screening ages to appropriately triage the highest risk women to genetic counselors, consideration for chemoprevention and appropriate MRI screening. Among high-risk women before and after age 40 year, systems are needed to support women who will vary in their decisions about what course of action they want to take to address their higher risk. For example, for genetic screening there needs to be steps in place along the lifespan to address prevention strategies, particularly since there are well known barriers to individuals undergoing genetic screening and taking up varied forms of chemoprevention.(23) Implementing these systems are challenging because of the discontinuity of health systems in the US in general, but also because there are so many paths women can choose to address their risk factors. Additionally, the cost savings of patient-management may be so far in the future, that health systems may not be incentivized to invest in systems to track individual preferences with built-in systems for follow-up (and currently no way to pass this information between health systems as individuals shift insurance coverage). Furthermore, evidence continues to evolve around surveillance and prevention strategies that challenge health systems and women to follow these evolving guidelines.

Additional considerations for model developers

Epidemiologists should also be aware of health-system questions that will arise when these models are available to be adopted. For example, basic questions that arise in health plan guideline development and implementation include: How well does a population-based model predict an outcome for an individual woman or for a population like ours? Has the model been validated? Has it been in validated in populations like ours? Is the model transparent to patients and providers or is it a “black box” that includes proprietary calculations that cannot be re-created? Does the model have “good enough” performance to implement and to be used for medical decision making (note: “good enough” definitions differ across diseases and health systems)? Are there clear clinical actionable outcomes that can and should happen for individuals identified at high, medium and low risk? Are the recommended cut-offs from researchers appropriate in our setting?

Conclusions

Epidemiologists developing risk-models must think through model outcomes including stakeholder needs, time horizons, terminology and reference groups and clarity on what actionable steps are for health systems, providers and patients following its implementation. This can be particularly challenging in the United States, where population-based health management programs are the exception rather than the norm, where there are no centralized electronic medical records and where there is significant movement between health plans/systems over the life span. Too often models are published and recommended because of increase statistical improvements, rather than focusing on clinically meaningful improvements. It could be easily argued that any model improvement has the potential to positively impact outcomes at the population-level; however, model performance needs to be evaluated relative to complexity of the model to be implemented – not just from the risk-prediction perspective, but also from the burden on patients, providers and systems for the amount and frequency of required data collection and with clear actionable steps to be taken with the information collected. Continued work is needed to ensure stakeholder engagement starts earlier in the process of model development and validation and with clarity around items outlined in this review.