Patient and physician-identified considerations for clinical implementation of a new risk-based prediction tool to guide surveillance mammography in breast cancer survivors

Abstract

Purpose:

Patient, tumor, and treatment factors can help predict the chance that a woman with a history of breast cancer diagnosis will be diagnosed with a second breast cancer within a year of a negative mammogram. This qualitative study elucidates breast cancer survivor and multi-specialty physician perspectives on barriers/facilitators to clinical implementation of a risk-prediction tool to support surveillance decisions.

Methods:

We enrolled women who completed primary breast cancer treatment and physicians from November 2023 – April 2024. Participants were recruited through Breast Cancer Surveillance Consortium’s registries; patients participated in one of four focus groups and physicians participated in individual semi-structured interviews. Participants were presented with information about an interval cancer risk prediction tool and were prompted to share perspectives on facilitators and barriers to using such a tool. To identify salient themes, thematic analysis was undertaken by three research team members.

Results:

Participants included 40 physicians and 23 patients. Three themes emerged: 1) evidence needed for tool acceptance, 2) tool features to facilitate usage, and 3) barriers to tool adoption. Both cancer survivor and physician groups were accepting of risk prediction tool use for surveillance imaging when tool development information was available; they perceived the tool would fit within workflows; and data integrity could be verified. Both groups anticipated structural (time) and technological barriers (MRI availability) could impede adoption.

Conclusion:

Qualitative findings from focus groups and interviews analyzed thematically suggest implementing a risk prediction tool for surveillance imaging requires evidence transparency, health record integration, data integrity protection, and system supports to promote ease of use in clinical settings while mitigating unintended consequences. All are important to consider during tool development and implementation planning.

CONTEXT SUMMARY:

Key Objective:

What do physicians and breast cancer survivors need to use risk prediction tools to guide tailored breast imaging plans in survivorship?

Knowledge Generated:

Patient and physician acceptance depends on balancing risk tool transparency and usability within the constraints of system resources like access to technology and insurance.

Relevance:

This study identified key considerations for co-developing equitable, user-centered implementation strategies for risk-based breast cancer surveillance.

Introduction

Risk prediction is increasingly used to personalize care after primary breast cancer treatment. However, tools that identify high-risk populations are generally not readily available in clinical practice, inhibiting their uptake.¹ Research has shown that patients’ and physicians’ acceptance of risk prediction tools for breast cancer screening has grown alongside the development of several calculators now available to estimate the future risk of breast cancer recurrence to guide adjuvant treatment.^1–3 Fewer tools have been used for breast cancer surveillance imaging after treatment completion. Yet, breast cancer survivors have heterogeneous risks associated with interval breast cancers,^4,5 defined as those cancers identified within one year of a negative mammogram. Emerging tools to guide tailored surveillance imaging by risk (e.g., addition of annual MRI for women at high risk for interval cancers) hold promise for improving outcomes for breast cancer survivors.^4,6 Willingness to adopt these tools by end users is critical as these tools prepare for clinical implementation.

Identifying and addressing health system, clinician, and patient barriers and facilitators are imperative to ensuring effective integration. Research on barriers and facilitators for the use of risk prediction in breast cancer has demonstrated that physicians are concerned with the accuracy of risk prediction, as well as applicability to specific patient subgroups.^7–10 For clinical implementation, minimizing disruption to existing clinical workflows is a primary concern, along with the time it takes to use the tool, and interpretability of outputs.⁷ While studies show that patients generally want more support in making surveillance imaging decisions,¹¹ some studies have shown limited changes in patient deliberation when decision supports are used.¹² Effective use of risk prediction tools for clinical decision support requires early usability input from end users to promote successful uptake in clinical settings.¹³

In parallel with the development of a risk prediction tool, we conducted this qualitative study that aimed to explore multi-specialty physician and breast cancer survivor perspectives on barriers and facilitators to the acceptance of a risk prediction tool, and the preferred features of a tool required for clinical implementation.

Materials and Methods

Through the Breast Cancer Surveillance Consortium (BCSC), a network of six breast imaging registries that seeks to assess and improve the delivery and quality of breast cancer screening and surveillance in the United States,¹⁴ a risk model to predict cumulative absolute 5-year risk of an interval second breast cancer diagnosis was developed. The risk prediction tool combines risk factors for second breast cancers with additional parameters influencing cancer detection by mammography. This qualitative study aimed to understand physicians’ and patients’ perspectives on barriers and facilitators to using a risk-based prediction tool to support decisions about surveillance imaging. The project team conducted focus groups with breast cancer survivors and semi-structured interviews with physicians from November 2023 through April 2024.

Recruitment

Patients and physicians were recruited from BCSC registries using purposive sampling to reflect diversity in physician specialty and patient and physician geographic location. Each BCSC registry site mailed referral letters to women who: (1) had a history of ductal carcinoma in situ or American Joint Commission on Cancer stage 1–3 invasive cancer,¹⁵ (2) completed primary curative treatment for breast cancer (adjuvant therapies were excepted), (3) had breast cancer imaging within the last four years at a participating BCSC site, (4) could participate in a virtual focus group by video or phone, and (5) communicated in the English language. Physicians were recruited by BCSC registry sites if they: (1) currently worked directly with patients with breast cancer histories, and (2) specialized in the fields of primary care, medical oncology, surgical oncology, or radiology. Recruitment continued until thematic saturation was reached. Full recruitment details have been previously reported.¹⁶ Participants received a $100 gift card as compensation.

The study was determined to meet criteria for federal exemption by the Institutional Review Boards of Dartmouth College and participating registries. All elements of consent were reviewed with participants; written documentation of informed consent was waived due to the minimal risk nature of this research. All participants verbally agreed to the recording of focus groups and interviews.

Data collection

Four, 75-minute patient focus groups were conducted and audio-recorded by two trained study team members (NB, SS) by Zoom using a semi-structured focus group guide. The guide was developed by the study team, including physicians and researchers with expertise in oncology, primary care, radiology, decision-making, and qualitative methods. Prior to first use, the facilitation guide was piloted with four breast cancer survivors and refined for clarity. All questions were open-ended with follow-up probes and covered patients’ experiences with breast cancer surveillance, barriers and facilitators to surveillance decision-making and compliance. Both team members attended all focus groups to maintain consistency in facilitation. During the focus group, the facilitators presented a risk prediction tool description (Figure 1).

Figure 1:

Description of a Risk Prediction Tool to Support Surveillance Imaging Decisions Presented to Participants

Following focus group completion, the study team (NB, SS) conducted 30–45-minute semi-structured physician interviews audio-recorded via Zoom. Interviewers co-trained on use of the interview guide and listened to audio recordings to ensure consistency. Interviews covered physicians’ experience with and barriers and facilitators to surveillance decision-making and compliance, and their perceptions of the risk prediction tool. As with patients, physicians viewed the description of the risk prediction tool (Figure 1) and shared their initial reaction to the tool, challenges to integrating a tool in their practice, important attributes and features, perceived drawbacks and opportunities. We piloted the guide with a medical oncologist and made minor modifications to improve the clarity of questions.

Data analysis

The data were professionally transcribed and imported into a qualitative data management program in preparation for analysis (Dedoose, SocioCultural Research Consultants, LLC; version 9.2.012). The codebook was developed based on the principles of theoretical thematic analysis,^16,17 driven by the study team’s interest in understanding preferences for risk prediction tool use in clinical settings. A subset of transcripts was independently coded by two coders (NB, SS), who met with a third study team member (CG) to review the application of codes and their definitions during codebook development and refinement. Subsequent transcripts were coded by the two coders independently with a finalized codebook and repeated review of coding by the first author. Through coding and discussion, analysis moved towards deriving themes from the data using memos.¹⁸ Focus groups and interviews were first coded and underwent thematic development separately, then triangulated seeking confirming and disconfirming evidence. The study team reviewed the findings, and any discrepancies were resolved through collaborative discussion.

Results

Twenty-three patients participated in the four focus groups and forty physicians completed interviews. The sample consisted of mostly White, non-Hispanic women (91%) with at least a 4-year college degree (78%). Patients resided in Vermont (48%), New Hampshire (30%), Washington (9%), North Carolina (9%), and New York (4%). Most physicians worked in academic clinical settings (70%). The sample demographic characteristics are provided in Table 1.

Table 1:

Participant Characteristics

Patients (n = 23)
Age: Mean (range), years	65.9 (55 – 81)
Years since diagnosis: Mean (range)	9.0 (3 – 20)
Active treatment, n (%)
Not currently undergoing breast cancer treatment	19 (83%)
Active or adjuvant treatment	4 (17%)
Education, n (%)
High school graduate	1 (4%)
Associates/Some college	4 (17%)
4-year college degree	6 (26%)
Graduate degree	12 (52%)
Race/ethnic identity, n (%)
Non-Hispanic White	21 (91%)
Other	2 (9%)
Physicians (n = 40)
Years of clinical experience: Median (range)	18.5 (5–35)
Practice Type, n (%)
Academic	29 (73%)
Community	11 (28%)
Specialty, n (%)
Primary care	13 (33%)
Medical oncology	9 (23%)
Surgical oncology	7 (18%)
Radiology	11 (28%)
Gender, n (%)
Man	12 (30%)
Woman	25 (63%)
No response	3 (8%)

Three themes on clinical implementation of the tool were developed: the evidence needed to accept the tool, important user features to facilitate tool usage, and potential barriers to adoption of the tool.

Theme 1: Willingness to use a risk prediction tool was predicated on transparent evidence, high prediction accuracy, demonstrated impact on care, and performance equity.

Most patient and physicians’ willingness to use the risk prediction tool hinged on specific background information they required.

Evidence and Transparency

Both physicians and patients almost universally discussed the need for strong, recent evidence of the benefit of the tool, including data inputs, research on its effectiveness, and associated impact on outcomes before they would use it.

“There’s so much research that’s going on all the time in relation to cancer, I would want to know that it was something that…continued to reflect the latest medical knowledge about cancer, and prevalence, and risk.”

(Patient)

“I would assume the tool incorporates all sorts of evidence, and I guess I’d just like to see that in advance. Or at least have the people who manage our breast care program look through it in depth and look at all the research, and make sure they agree with the thought process behind it. And then we’d apply it blanket-wide as an organization.”

(Radiologist)

Prediction Accuracy and Clinical Relevance

For a tool to usefully guide care, physicians mostly sought high prediction accuracy, often described as high sensitivity and specificity, along with demonstrated benefits on outcomes. As noted by a primary care physician, “I’d just have to be really convinced that it was accurate and that it would have a demonstrable impact in terms of helping diagnose recurrences at an early enough stage that would make a difference in the outcome for the patient.” Another primary care physician wanted information around precision. “I guess I’d want to know something about precision in addition to accuracy. So how wide is the confidence around the prediction? People can sometimes get very focused on single numbers when the range around the estimate is pretty wide.” While less commonly mentioned by patients, some discussed the need to have conclusive, lasting clinical relevance: “Does it actually provide a better outcome for the patient over five years, 10 years? … You don’t want to make a mistake today and find out five years from now that the tool led you in the wrong direction.”

Performance Equity

Most physicians and patients also sought a tool that would be widely useful across patient populations, and were thus concerned about equity in performance across a variety of age groups, races, ethnicities, and cancer features:

“I would like to make sure that it’s been validated across a variety of age groups and across all different ethnicities.”

(Surgical Oncologist)

“What research is it based on? Because if it’s all middle-class, white women, then it’s going to have less accuracy for people who aren’t middle-class, white women. I do think that it’s important to have an inclusive research base there. So often, that’s not really the case. The other thing is age. What is the impact of age?”

(Patient)

Theme 2: Ease of use was the principal facilitator for risk prediction tool adoption discussed by physicians, although they varied in suggested implementation strategies to promote a positive user experience.

Figure 2 summarizes the features discussed by physicians that would promote risk prediction tool use for surveillance in clinical practice, highlighting their key role as primary end-users of such a tool. They described variation in ideal levels of integration with clinical systems, considerations of data integrity, and desired features related to tool accessibility. Physicians suggested a range of integration options that would facilitate use, from a web-based application that could be accessed by phone or another device, to partial integration with the EMR (e.g., output is available in patient records), to a fully integrated tool (e.g., automation of data inputs and presentation of recommendations). At each level, physicians acknowledged potential challenges associated with data integrity, including the data accuracy and availability, completeness, and ease of finding data necessary for tool use. Finally, physicians discussed the need to consider tool accessibility, which included minimizing the number of ‘clicks’ required to access the tool and having ready access to outputs.

Figure 2:

Suggested levels of risk prediction tool integration and associated implementation considerations

Theme 3: Barriers to risk prediction tool use predominantly focused on structural or external influences that served to inhibit potential implementation.

Physicians and patients discussed the potential downsides and barriers to using a risk-prediction tool for surveillance decisions. Physician-identified barriers included limited time available in clinical encounters, difficulty of adopting another tool in a rapidly changing technology space and added resources for implementation within constrained institutional resources. Patient- and physician identified challenges included concerns about insurance and access to recommended testing. Barriers by type are summarized below, with example quotations provided in Table 2.

Table 2:

Quotations by Categorizations of Barriers to Use of Risk Prediction Tools for Surveillance

Type of Barrier	Patient Quotations	Clinician Quotations
Time	“I’m just going to speak from my experience and my primary care doctor is so busy. She only finds out what’s going on is if I tell her… I would not trust her to know the best way to interpret and to use those tools… I’m not saying she’s not able mentally or as a doctor, I just think she has so many patients she can’t handle it.”	“The primary care often is dealing with multiple issues during a visit…requiring more of our time and the patient’s time. And just being sure that additional imaging surveillance is actually going to make a difference in outcomes because there are risks to imaging.” (Primary care physician)
Changing Evidence	“Even now, if you think about whatever studies and data, by the time it actually gets back to the patient affecting the patient’s care, how long is that already?…Because you’ll see a study that tells you this and you can find up a study that tells you the exact opposite thing…’Well, which one am I supposed to believe?’”	“Recommendations are changing, and women are doing different things over a pretty short period of time just because things have changed. I think having a tool, it could be hard to have it be adaptable to those changing recommendations, and you’re not always comparing apples-to-apples.” (Surgical Oncologist)
System Resources	“I mean there would have to be a training of the physicians who would be using this tool in its use and understanding what information it is providing.”	“From an IT standpoint … I’m sure there’s just projects beyond projects in their queue for all throughout the health system, things that all need to be updated or reworked or added. So just I’m sure time is an issue…yeah, we can build it. It’s just like join the queue.” (Radiologist)
Access and Insurance	“If you’re in a small hospital in the middle of Kansas, do you have the ability to get PET scans and MRIs and other things that you wouldn’t have to go to a big hospital. … accessibility is a huge hurdle for some places and people.” “If there is additional testing recommended, I guess a big piece would be whether this study was enough to then get it approved for insurance coverage. Because if you’re recommending something, and then it’s not covered… [there’s]anxiety with that.”	“I think it might be helpful if a certain threshold supports more insurance coverage. But conversely… [if] I can’t prove that someone has a high enough risk in the non-breast cancer population, it can lead to insurance denials, too. So you just don’t want it to be used against you.” (Surgical oncologist) “I think if you’re trying to order non-standard imaging or something different maybe than radiology has recommended, then if you had this tool to back you up and say, ‘This is what this evidence-based tool has recommended,’ I think that could help with coverage.” (Primary care physician)

Time.

Limited time in clinical encounters to use and discuss risk prediction results emerged as a challenge to tool use for many physicians: One noted that the tool ‘should not take10 minutes to use in a 20-minute visit’. Others saw the potential downside of extensive time needed to discuss findings and answer questions, though it was recognized that the time investment would be worthwhile if the tool “catches breast cancer earlier or saves time down the road” or “saves imaging money.” Others cautioned that the tool would need to be “easily incorporated” into the visit.

Changing Evidence.

Physicians highlighted that surveillance recommendations and technologies are continually changing, and that the tool would need to efficiently adapt to these changes.

System Resources.

As an extension of the concerns about evolving evidence, guidelines and technologies, several physicians worried that integrating a risk prediction tool might add burden to constrained institutional resources:

Access and Insurance.

The final challenges to risk prediction tool use were concerns about the relationship between risk results, recommendations, local technology availability, and/or insurance coverage. Both physicians and patients indicated worry about denial of coverage, especially if insurance companies have not “bought into” the tool. Physicians, however, also suggested a potential benefit of the tool would be supporting expanding coverage for supplemental testing.

In summary, both physicians and patients, even if enthusiastic about the potential of risk prediction to improve care and outcomes, foresee the need to ensure clinic structures, technology, and insurance policies are aligned with the use of risk prediction tools.

Conclusion

Collectively, the themes developed in this qualitative study of breast cancer survivors and multi-disciplinary physicians illustrate that patient and physician acceptance depends on balancing risk tool transparency and usability within the constraints of system resources. Below we describe how these findings integrate with prior research on personal and clinic-based factors associated with adoption of risk-based approaches in several settings.

First, individual acceptance by patients and physicians conditioned upon some level of evidence for improved patient outcomes is a common theme across studies of using risk prediction tools,^7,8,19 with transparency being a growing issue as proprietary and artificial intelligence-based tools become more common in clinical practice.²⁰ As described by physicians in the sample, high-quality risk prediction tools should have good predictive accuracy to minimize misclassification of patient risk.²¹ A tool’s discriminatory ability is a key function, meaning that the tool must be able to distinguish those at high risk for interval cancers from those at low risk to support appropriate surveillance imaging recommendations. In addition, both adaptability to new evidence and equity in performance across subpopulations were noted as primary factors in whether someone would use a risk prediction tool. Despite debate over how factors such as race and ethnicity should be incorporated into risk prediction,²² ensuring these tools ‘work’ for everyone was noted to drive willingness to use a risk prediction tool by both patients and physicians. This speaks to the need to externally validate risk prediction tools across a variety of populations and settings to ensure performance is generalizable.

With little comparative effectiveness research on risk prediction tools, tool selection is an ongoing challenge for health systems and clinicians who seek to implement risk-stratified care. Indeed, physicians in this study mentioned the need to have a vetting process for use of clinical risk prediction tools. To support the selection of risk prediction tools, a framework for Grading and Assessment of Predictive Tools (GRASP) has been developed and tested.^23,24 A randomized study of physicians found that GRASP assessment use was associated with higher quality risk prediction tool selection.²⁵ Such evaluation tools offer practical and systematic guidance for those seeking to evaluate these interventions for adoption. Nonetheless, care must be taken to assess whether evidence aligns with the setting in which tools are implemented.

While feasibility of clinical use is often an afterthought in the development of risk prediction tools,²¹ it is important to consider in early planning for implementation to increase uptake.¹³ Indeed, physicians in this study called out several facets of clinical feasibility, including integration with the electronic health record, availability and quality of data for inputs, and tool accessibility in clinical settings. This is concordant with other literature: usability is a driving factor for willingness to adopt tools that support risk stratification.²⁵ Our findings highlight the intersection between level of integration within clinical systems and associated data integrity issues. While some physicians sought online risk tools separate from the medical record, others deemed complete electronic health record (EHR) integration to be essential. Yet with each approach, physicians sought some control over data validation, whether that was ensuring accurate documentation in the medical record, or having the opportunity to review automated inputs for accuracy. These concerns are well-founded: several studies have documented the ongoing need to address the quality and completeness of electronic health record data in the context of risk prediction.^26,27 Generating acceptable implementation strategies that balance preferences for integration across variable EHR system capabilities while maintaining data quality remains a barrier to widespread adoption of risk prediction tools across settings.²⁷

In terms of cost and insurance implications, a review of 22 studies suggested that patients were not concerned about variation in treatment costs in using risk prediction and stratification in cancer screening among cancer-naïve populations.²⁸ However, insurance coverage of risk-stratified care was a common consideration in this sample of breast cancer survivors. This could be a distinction between those who have been exposed to the downstream costs of cancer screening and treatment, and perhaps experiencing the now well-documented phenomena of ‘financial toxicity’ associated with cancer care.²⁹ We previously reported that patients and physicians were concerned about a risk prediction tool recommending less intensive surveillance, especially if not aligned with patient preferences for more intensive imaging schedules, and the potential impact the tool could have on insurance coverage.²⁸ Contextual information regarding financial hardship and potential unintended consequences of risk-stratified care for cancer survivors warrants further investigation..

Limitations of this study include its relatively racially, geographically, linguistically, and educationally homogeneous patient sample. We thus may not have identified all relevant clinical implementation considerations of patients and studies that include more diverse patient experiences are needed. In recruiting through an existing network of breast cancer screening registries, physicians in this sample may have more favorable attitudes toward risk prediction tools than in general practice. Finally, because we sought early input to guide development of our risk prediction tool, we were limited in using a general description, rather than being able to user-test a validated tool. Thus, our findings are limited by the description provided to participants. As the tool is developed for implementation, we will conduct continued studies to ensure it is both efficacious and usable across diverse populations and include real-world pilot testing.

Given the evidence that patients may benefit from supplemental imaging,³⁰ tools that improve identification of those at risk for interval cancers are needed to support evidence-based care. While the accuracy of risk prediction tools is important, this study highlights that it is but one factor in determining acceptance by patients and physicians. For clinical implementation, balancing transparency and usability within the constraints of differing health record systems poses a challenge. A critical issue is the need for readily available, accurate data for tool inputs to ensure ease of use and relevance to clinical practice. While risk prediction tools hold promise for personalizing care, addressing multi-faceted barriers to clinical implementation is vital to fully achieving real world impact on cancer outcomes.