Patient and Caregiver Perceptions of an Interface Design to Communicate Artificial Intelligence–Based Prognosis for Patients With Advanced Solid Tumors

Abstract

PURPOSE

Use of artificial intelligence (AI) in cancer care is increasing. What remains unclear is how best to design patient-facing systems that communicate AI output. With oncologist input, we designed an interface that presents patient-specific, machine learning–based 6-month survival prognosis information designed to aid oncology providers in preparing for and discussing prognosis with patients with advanced solid tumors and their caregivers. The primary purpose of this study was to assess patient and caregiver perceptions and identify enhancements of the interface for communicating 6-month survival and other prognosis information when making treatment decisions concerning anticancer and supportive therapy.

METHODS

This qualitative study included interviews and focus groups conducted between November and December 2022. Purposive sampling was used to recruit former patients with cancer and/or former caregivers of patients with cancer who had participated in cancer treatment decisions from Utah or elsewhere in the United States. Categories and themes related to perceptions of the interface were identified.

RESULTS

We received feedback from 20 participants during eight individual interviews and two focus groups, including four cancer survivors, 13 caregivers, and three representing both. Overall, most participants expressed positive perceptions about the tool and identified its value for supporting decision making, feeling less alone, and supporting communication among oncologists, patients, and their caregivers. Participants identified areas for improvement and implementation considerations, particularly that oncologists should share the tool and guide discussions about prognosis with patients who want to receive the information.

CONCLUSION

This study revealed important patient and caregiver perceptions of and enhancements for the proposed interface. Originally designed with input from oncology providers, patient and caregiver participants identified additional interface design recommendations and implementation considerations to support communication about prognosis.

Cancer patient and caregiver perceptions affect interface design for sharing AI-based prognosis.

INTRODUCTION

The integration of artificial intelligence (AI) into health care, particularly oncology care, is growing with capabilities including image analysis and using electronic health record (EHR) data to screen, diagnose, and predict patient outcomes.^1-6 Machine learning (ML), a common AI method, is useful for making predictions and can improve upon expert human performance for tasks such as predicting patient survival.^7,8 Given the surge and breadth of AI use,⁹ there has been increased attention on the need for trust,^10,11 oversight to ensure patient safety,¹² and AI that supports, not replaces, communication and decision making among health care teams and patients.^1,13-15 Prognosis-related AI applications are often clinician-facing or run in the background to identify high-risk populations.^9,16,17

CONTEXT

• Key Objective

• We assessed patient and caregiver perceptions and identified design changes to an interface that conveys artificial intelligence (AI)–based 6-month prognostic information to support provider-patient communication at treatment decision points for patients with advanced solid tumors.

• Knowledge Generated

• Overall, patients and caregivers perceived the interface had value in supporting decision making, reducing isolation, and facilitating communication among patients, their caregivers, and oncology providers. Participants identified novel challenges, information gaps, and implementation considerations not recognized earlier in the design process that involved oncologists, thus demonstrating the value of soliciting feedback from patients and caregivers.

• Relevance (F.P.-Y. Lin)

• This study underscores the importance of incorporating patient and caregiver feedback in refining AI tools for conveying prognostic information, leading to a more effective tool for facilitating shared decision-making between patients and their oncology providers.*

• *Relevance section written by JCO Clinical Cancer Informatics Associate Editor Frank P.-Y. Lin, PhD, MBChB, FRACP, FAIDH.

Understanding prognosis allows patients and providers to make decisions that align with patient goals. Studies show that patients are less likely to choose aggressive anticancer care at end of life and clinicians refer patients for palliative care earlier when there is an accurate understanding of a patient's prognosis.^18-20 Although patients with cancer are open to receiving prognostic information, including AI-based prognostication,²¹ it is often not shared with or understood by them.^22,23 AI-based prognostication combined with clinician-facing behavioral nudges has improved engagement in serious illness conversations (SICs) for oncology patients,^16,17 although documented SIC rates remain low.

To support treatment decision making, we developed a ML model to predict 6-month survival using EHR data^24,25 in patients with advanced solid tumors, specifically those with malignant brain or nervous system cancer or other solid tumor with metastases. The development and validation of the ML model, using 45 commonly available features, are described elsewhere.^24,25 The model demonstrates satisfactory performance metrics for predicting mortality with an AUC value of 0.80 or greater, positive predictive value of 0.64, negative predictive value of 0.82, and sensitivity and specificity of 0.25 and 0.96, respectively.²⁵ Features predictive of low or likely 6-month survival included albumin (most predictive), selected laboratory values, pain score, palliative care history, time since index date, time since next treatment, and age; cancer type had low importance.²⁴

To communicate ML-based prognosis, we designed an interface intended for use by oncology providers to address key information needs when preparing for and discussing prognosis in the context of treatment decisions with patients with advanced solid tumors and their caregivers.²⁶ The resulting design prioritized interpretability²⁷ over explainability²⁸: clinicians sought to communicate context-specific interpretation of model output to guide decision making over model performance.²⁶ Additionally, clinicians noted the interface may help facilitate SICs and should be shared with patients and their caregivers.²⁶ Thus, we applied user-centered design processes to solicit input from patients and caregivers.

In this study, we again focused on interpretability over explainability and sought to (1) assess patient and caregiver perceptions regarding how prognosis information is conveyed by the interface and (2) identify needed changes and considerations for implementing the interface within clinical workflow. This formative evaluation will guide further interface design that will require development and real-world, summative evaluation²⁹ before implementation.

METHODS

Design, Setting, and Participants

For this qualitative study, we recruited participants through purposive sampling from Huntsman Cancer Institute (HCI), UT, and across the United States. Initial recruitment methods included outreach to HCI-affiliated social workers, advocacy groups, and oncologists, and posting on a study locator dashboard. We partnered with the Community Collaboration & Engagement Team (CCET) from the University of Utah Clinical & Translational Science Institute who expanded recruitment methods to include social media and outreach through community partnerships.

Interested participants were sent a Research Electronic Data Capture screening questionnaire to determine eligibility. Eligible participants were age 18 years or older and self-identified as former patients with cancer (who had undergone treatment, though with no current evidence of disease) and/or former caregivers of living or deceased patients with cancer who participated in treatment decisions. Exclusion criteria included lack of fluency in spoken or written English and lack of access to Zoom web-conferencing technology. Eligible participants were contacted to schedule participation, with the goal of balancing patients with cancer and caregivers across interviews and focus groups. If participants were not available for the focus group, they were scheduled for individual interviews. Participants received a gift card for a 1-hour interview ($50) or a 2-hour focus group ($75).

Individual Interviews and Focus Groups

The combination of individual interviews and focus groups allowed for more thorough exploration of research questions.³⁰ We referred to the interface as the visual information tool, shortened to tool.

Semistructured, individual interviews were conducted via Zoom (E.A.S.) and lasted 45-60 minutes. The interview guide, developed by two researchers (E.A.S. and C.J.S.), was informed by user-centered metrics from the planning and development phases of information technology life cycle depicted by the ELICIT framework³¹ and pilot-tested both among and with researchers external to the research team (Data Supplement, Methods). Individual interview participants responded to open-ended questions and then indicated agreement or disagreement to statements using a Likert scale, which prompted additional open-ended feedback about the tool.

Using focus groups, we sought to facilitate discussion among participants where both cancer survivors and caregivers could agree or disagree with others' comments, allowing for different perspectives to emerge. Two focus groups lasting approximately 120 and 100 minutes were conducted via Zoom (CCET personnel). The sessions used open-ended questions derived from the individual interview guide and followed a semistructured guide to elicit participant feedback (Data Supplement, Methods).

Development and Presentation of the Visual Tool

Previous iterative design with oncologists resulted in an interface that displays patient-specific EHR data and conveys 6-month chance of survival, change in prognosis over time, survival of similar patients (“Patients Like Me”), and recommended next steps including supportive care options.²⁶ For this study, the tool was presented to study participants in a pre-recorded video to (1) ensure standardization of content and presentation, (2) remove dependence on researchers explaining the tool in real time, and (3) given the sensitive nature of prognosis, we wanted the study participant to be an observer and not feel as though the conversation was directed toward them. The video depicted a fictional conversation between an oncologist and patient at a third treatment decision point and focused on the oncologist's explanation of the tool. In the video, the oncologist shared the tool at earlier treatment decision points when 6-month survival was likely, and at the current time point when 6-month survival was low. Figure 1 shows time point 2 of the interface (likely survival); time point 3 (low survival) is published elsewhere.²⁶ The video was unchanged for interviews 1-4 and focus group 1.

FIG 1.

Time point 2 of the interface, where 6-month survival was likely. This version of the interface was designed on the basis of user-centered design methods with input from oncologists.²⁶

Midway through data collection, we reviewed preliminary findings, applied user-centered design principles,³² and identified three changes to the interface (Fig 2; Data Supplement, Changes to Interface). We re-recorded relevant sections of the video for interviews 5-8 and focus group 2 using the same fictional scenario with minimal changes to the explanation of the tool. Saturation was continuously assessed over the final four interviews. Although we did not achieve content-level saturation and new information was shared about specific changes to the tool, categories related to perceptions and use remained unchanged.

FIG 2.

Description of changes to the interface and content presented during the latter half of data collection. Key: (1) Revised banner heading to include text that mentions “machine learning.” (2) Modified the process for revealing content within the interface: (a) Each component is hidden with text explaining the question that will be answered by the content when it is revealed. (b) When the Recommended Actions are revealed, the prognosis information in the other three quadrants is hidden, and the Recommended Actions are centered on the screen. (3) Added an additional resource to guide patient to articulate values relevant for decision making.

Data Analysis

Audio recordings of interviews and focus groups were transcribed verbatim and transcripts were deidentified. Analysis of transcripts was conducted using NVivo (QSR International) software and included all open-ended responses and participant explanations for Likert scale responses but excluded responses about information needs reported elsewhere.³³ To develop a codebook for analysis, three researchers (E.A.S., C.J.S., and J.W.G.) used open coding and independently coded the same transcript. After resolving disagreements and revising the codebook to reflect consensus, researchers recoded the same transcript. Interrater reliability was calculated in NVivo, on the basis of Cohen's kappa coefficient. Once a strong level of agreement between all three coders (k ≥ 0.80)³⁴ was achieved, the remaining transcripts were independently coded by two researchers (E.A.S. and C.J.S.). No codebook changes were needed when coding the remaining transcripts. Data were analyzed using thematic analysis, a widely used method for analyzing qualitative data that involves identifying patterns and themes.³⁵ Because only four participants provided Likert responses for each iteration of the tool, quantitative findings are not reported.

After coding all transcripts, researchers organized initial codes into broader categories. We reviewed and refined themes, and produced a final analysis report that included a description of themes and participant quotes. For the category Implementation Considerations, themes were determined deductively and mapped to the five rights of clinical decision support: Channel, Format, Information, Person, and Workflow.^36,37 We used the Consolidated Criteria for Reporting Qualitative Research (COREQ) Checklist³⁸ to ensure complete and comprehensive reporting of our findings (Data Supplement, COREQ checklist).

Ethical Considerations

This study was approved by the institutional review board at the University of Utah. Participants were informed about study purpose, data confidentiality, and their right to withdraw. Verbal consent was obtained from all participants before study procedures.

RESULTS

Twenty participants provided feedback on the tool across eight individual interviews and two focus groups, each involving six participants. Participants represented perspectives of cancer survivors (n = 4), caregivers (n = 13), or both (n = 3), and are described in Table 1. We identified four categories: (1) positive perceptions and benefits, (2) negative perceptions and wants, (3) areas requiring clarification, and (4) implementation considerations.

TABLE 1.

Participant Demographics

Characteristic	Individual Interview (n = 8)	Focus Group 1 (n = 6)	Focus Group 2 (n = 6)	All Participants (N = 20)
Age, years, mean	53.0	50.0	53.7	52.3
Age, years, range	29-70	33-78	28-71	29-78
Self-identified experience, No. (%)
Cancer survivor	3 (37.5)	1 (16.7)	0	4 (20)
Caregiver	3 (37.5)	5 (83.3)	5 (83.3)	13 (65)
Both	2 (25)	0	1 (16.7)	3 (15)
Sex, No. (%)
Female	7 (87.5)	4 (66.7)	4 (66.7)	15 (75)
Male	1 (12.5)	2 (33.3)	2 (33.3)	5 (25)
Race, No. (%)
Asian or Asian American	1 (12.5)	1 (16.7)	0	2 (10)
Black or African American	1 (12.5)	3 (50)	0	4 (20)
Middle Eastern or Northern African	0	1 (16.7)	0	1 (5)
White or European	6 (75)	1 (16.7)	4 (66.7)	11 (55)
Other: mixed	0	0	1 (16.7)	1 (5)
Unknown/prefer not to answer	0	0	1 (16.7)	1 (5)
Total annual household income, No. (%)
<$10,000	0	1 (16.7)	0	1 (5)
$25,000-$39,999	2 (25)	0	2 (33.3)	4 (20)
$40,000-$49,999	0	2 (33.3)	0	2 (10)
$50,000-$74,999	2 (25)	3 (50)	2 (33.3)	7 (35)
$75,000 or more	2 (25)	0	2 (33.3)	4 (20)
Unknown/prefer not to answer	2 (25)	0	0	2 (10)
Religious affiliation, No. (%)
Agnostic	1 (12.5)	0	0	1 (5)
Atheist	0	0	1 (16.7)	1 (5)
Christian	3 (37.5)	3 (50)	2 (33.3)	8 (40)
Muslim	0	2 (33.3)	0	2 (10)
None	0	1 (16.7)	1 (16.7)	2 (10)
Unknown/prefer not to answer	4 (5)	0	2 (33.3)	6 (30)
Highest level of education, No. (%)
High school	0	1 (16.7)	0	1 (5)
Some college	0	0	2 (33.3)	2 (10)
Associate's degree	0	1 (16.7)	1 (16.7)	2 (10)
Bachelor's degree or higher	5 (62.5)	4 (16.7)	3 (50)	12 (60)
Unknown/prefer not to answer	3 (37.5)	0	0	3 (15)
State of residence, No. (%)
CA	0	1 (16.7)	0	1 (5)
ID	0	1 (16.7)	1 (16.7)	2 (10)
IL	0	1 (16.7)	0	1 (5)
UT	7 (87.5)	3 (5)	5 (83.3)	15 (75)
WI	1 (12.5)	0	0	1 (5)
No. of people living in household, No. (%)
1	2 (25)	2 (33.3)	2 (33.3)	6 (30)
2	1 (12.5)	0	2 (33.3)	3 (15)
3	1 (12.5)	0	2 (33.3)	3 (15)
4	2 (25)	1 (16.7)	0	3 (15)
5 or more	0	1 (16.7)	0	1 (5)
Unknown/prefer not to answer	2 (25)	0	0	2 (10)

Positive Perceptions and Benefits

Participants shared positive perceptions of the tool. In most sessions, at least one participant expressed strong, positive sentiment that the tool was useful (eg, “it's a handy tool”—Cancer Survivor/Focus Group 1, “the tool can help”—Cancer Survivor/Interview 2). Several participants reported liking the specific numeric and graphical elements displayed (eg, “the combination of the actual number and the graphics is really good, because that gets both sides of the brain”—Both/Interview 1).

Positive perceptions and benefits encompassed themes important when discussing prognosis (Table 2). Related to the tool's value for decision making, participants expressed the benefit of considering quality of life when making treatment decisions. Participants also expressed value of the tool in helping the patient to feel less alone when looking at population data for “Patients Like Me.” Another theme included value of the tool in supporting communication about prognosis between patients and oncologists, as well as supporting communication between patients and caregivers.

TABLE 2.

Themes Related to Positive and Negative Perceptions

Theme	Example Quotes	Relevant Tool Component
Positive perceptions and benefits
Value of the tool for decision making	“I know several friends who've gone through end-of-life things, and it's a really confusing place to be… I think this tool would allow them to have a more concrete view of what was really possible, so that they can make better decisions about what they actually wanted to do with time…” (Individual Interview Participant 2—Cancer Survivor) “…seeing this information like this perhaps provides the patient with some sense of agency… especially in our case where the treatment was destroying the quality of life…” (Focus Group 1 Participant—Cancer Survivor)	General comment “Today's 6-month chance of survival”
Value of feeling less alone	“It can feel like you're alone when you're on that journey, and it's kind of nice to know that people have walked that path before you.” (Individual Interview Participant 6—Caregiver)	“Survival of patients like me”
Value of the tool for supporting communication	“With my father-in-law, he has children, and they have spouses, and grandchildren… Instead of bombarding him and my mother-in-law with question after question, if they would've been able to just send that [tool] to us, and have us be able to look at it, and discern the information that we needed to get from it, and then ask questions if we needed to, we wouldn't have felt like we were overwhelming them with the same questions over and over again.” (Individual Interview Participant 6—Caregiver)	General comment
Negative perceptions and wants
Challenging to process	“Honestly, I think I couldn't—I was probably still stuck on “Low.” Low is what stood out and stuck with me.” (Individual Interview Participant 2—Cancer Survivor)	“Today's 6-month chance of survival”
Challenging to understand	“The Kaplan-Meier curve I can understand, someone else who is not a scientist cannot understand…” (Individual Interview Participant 3—Caregiver)	“Observed 6-month population survival”"
Lack of information about what to expect (related to treatment or no treatment, side effects, quality of life, cost of treatment)	“Does survival mean I'm alive medically, or does survival mean I'm playing ball with my kids? So, does survival literally mean I'm medically alive?… I don't know if this is medically possible, but are… there …numbers about people who chose no treatment?” (Individual Interview Participant 1—Cancer Survivor and Caregiver)	“Today's 6-month chance of survival”
Desire for information about the prognosis model	“I would think that you would want to have something… so that they could say, ‘Oh, that number means this,’ or, ‘The reason why that number's on this information is because of this.’” (Individual Interview Participant 6—Caregiver)	Patient information (displayed in left column)
Need for help navigating information gaps	“The medical team is being proactive, knowing what I need. I don't even know what questions to ask. That's one of the terrors of going through all this. Is being afraid of, if you have any questions, ‘Well, I don't know enough to have questions. Walk me through this. You've been through this a gazillion times. I'm going through it once. Walk me through it.’” (Individual Interview Participant 1—Cancer Survivor and Caregiver)	“Recommended Actions”
Contributes to decision fatigue	“…that entire phrase [Choosing Wisely] is something that's just stressing me out. I mean, choosing itself is stressful, but ‘choose wisely’ makes you think that there is a wrong answer and that, like choosing by itself is stressful, that's just super stressful.” (Individual Interview Participant 3—Caregiver)	“Recommended Actions”

Negative Perceptions and Wants

Participants expressed several themes related to negative perceptions and wants (Table 2). It was difficult to separate negative perceptions from wants, as participants expressed negative sentiment while also suggesting improvements for the tool.

Initially, participants expressed that information on estimated prognosis was challenging to process once a low chance of survival was displayed. On the basis of this feedback, during interviews 5-8 and focus group 2, we (1) used a slow reveal approach by covering sections of the tool until the appropriate time during the conversation when sections were revealed incrementally and (2) separated the display of Recommended Actions from prognosis information (Fig 2). After this change, participants did not express challenges with processing information when low chance of survival was displayed and, when probed, viewed the slow reveal as a positive change.

Participants shared comments about how specific components of the tool were challenging to understand, particularly, graphical depictions of prognosis, including Kaplan-Meier curves and confidence intervals. Another theme that emerged was the lack of, or need for, additional information about what to expect. This refers to lack of information provided by the tool related to treatments (other treatments, experimental drugs, and clinical trials), side effects of treatment, expected quality of life, or cost of treatment. There was also desire for information about prognosis if the patient decided not to start another line of therapy.

Three additional themes emerged related to negatives perceptions and wants. First, participants expressed a desire for more detailed information to be provided about the ML model; for example, information on variables such as clinical inputs and the population used to make a prediction. Additionally, participants noted the need for guidance on what and how to obtain information related to their disease course (navigating information gaps). Finally, participants disliked components of the tool that place pressure on the patient to make the right decision, supporting the theme of contributing to decision fatigue.

Areas Requiring Clarification

We identified three themes related to clarification needs. First, several participants demonstrated confusion about the tool's context of use, when deciding whether or not to start a new anticancer treatment. Additionally, further clarification is needed related to the mechanics of the predictive model, and specifically, model inputs (eg, “…one other piece that I think was confusing was some of the numbers that were plugged in. Like… were those updated each time for like the current status of things… at that point in time?”—Both/Interview 4). In fact, participants noted that the tool did not identify that prognosis information was ML-derived; thus, we revised text in the banner to explicitly communicate that ML methods were used. Finally, the concept of supportive care is poorly understood (eg, “Supportive care means like somebody that can answer your questions, can kind of give you an idea of like what is my next step…”—Cancer Survivor/Interview 5).

Implementation Considerations

Participants shared perspectives related to the tool's implementation. Table 3 displays the implementation themes, operational definitions, related participant quotes, and key implications. Notably, participants indicated that prognosis information should be shared early in the course of diagnosis and treatment using the tool with interested patients. During the first focus group, for example, a cancer survivor participant expressed that the tool provided the patient with agency; this was followed by agreement from several caregiver participants who had previously said the tool may not be helpful to patients.

TABLE 3.

Implementation Considerations Expressed by Study Participants, by Theme

Theme	Example Quotes	Key Implications
Channel: processes for communicating the information	“I think the tool can help, I just think you wouldn't want the tool to be the one sharing the message, if that makes sense? I'd still want to have the doctor be the one sharing the message.” (Individual Interview Participant 2—Cancer Survivor) “The medical team is being proactive, knowing what I need. I don't even know what questions to ask. That's one of the terrors of going through all this. Is being afraid of, if you have any questions, ‘Well, I don't know enough to have questions. Walk me through this. You've been through this a gazillion times. I'm going through it once. Walk me through it. Get me to supportive care. I mean, do I need to get my paperwork in order? How do I manage blah-blah-blah?’ So, yeah, everything from a nutritionist to a lawyer, you know? You know what I need, I don't. So, be proactive. That would be the message I would love in that recommended action box.” (Individual Interview Participant 2—Cancer Survivor)	It is important that the patient's oncologist be the one who shares the tool and guides discussion about prognosis
Format: format for communicating the information	“And he was very helpful with, it was all in discussion, but a sheet of paper would have been wonderful.” (Focus Group 2 Participant—Caregiver)	A screenshot of the tool should be printable or saved to the EHR and patient portal, so a summary can be viewed after the appointment
Information: information that's communicated when using the tool	“I don't know that there's ever enough information, but… it prompts discussion then to gather more information…” (Focus Group 1 Participant—Caregiver)	The tool springboards information seeking, which reinforces the need to present the tool in the context of a conversation with the patient's oncologist
Person: who should receive communication about prognosis information using the tool	“… It depends on the patient. Like, finding out whether a patient wants to know their prognosis—I've met people who even at that point would be like ‘I do not want to know any of that information. I just want to try all the things possible.’” (Individual Interview Participant 4—Cancer Survivor and Caregiver)	Receptiveness to the tool is dependent upon the individual
Workflow: when the tool may be used in the process of delivering care	“I wouldn't want to wait until things didn't work. I'd want that tool from the very, very onset. And with that tool, I would want to know, from the very beginning, I would want to know what it would look like if I didn't do any treatment at all, no matter what my prognosis was, right?” (Individual Interview Participant 6—Caregiver)	Prognosis information should be shared early in the course of diagnosis and treatment using the tool with interested patients who meet the criteria for advanced solid tumors rather than waiting until the 6-month chance of survival is low

Abbreviation: EHR, electronic health record.

DISCUSSION

In our assessment of perceptions of an interface designed to communicate prognosis, patients and caregivers provided valuable insight, identified crucial enhancements, and shared considerations for future implementation. Our findings support the usefulness of the tool to convey prognosis to patients with advanced solid tumors who want prognosis information when making treatment decisions. Its value was recognized not only for supporting communication between patient and provider, but also for supporting communication between patients and caregivers. Importantly, participants agreed that prognosis information should only be shared if the patient wants to receive it.

Regarding negative perceptions or wants, most participant feedback concerned information that was missing, namely, providing information on treatment options and side effects, or prognosis if they decided not to continue treatment. These comments highlight limitations of the tool: it was not designed to summarize a patient's health status or be a freestanding patient decision aid that presents all alternatives.^39,40 For these reasons, participants underscored the importance of thoughtful implementation. Agreeing with oncologists,²⁶ participants thought the tool should be used in context of a conversation with their oncologist; however, participants further suggested presenting Recommended Actions separate from prognosis information. In the future, the tool's content could be incorporated into a more comprehensive interface with individualized data about specific treatments and side effects, especially if the tool is integrated into the EHR and patient portal, and population outcomes can be described for alternative treatment options.

An unanticipated critique of the tool related to terminology perceived to be judgmental. Participants identified terms such as “consider” or “Choosing Wisely” as problematic, as they placed burden on the patient to make the right decision. Both cancer survivors and caregivers expressed this sentiment—that a value judgment was inferred, and they could make the wrong decision. Choosing Wisely, an initiative launched in 2012, promotes conversations between clinicians and patients.⁴¹ Although Choosing Wisely provides clinician- and patient-facing resources, we are not aware of any user testing with patients. Participants suggested removing judgmental language in patient-facing materials.

Our findings highlight the importance of explaining the tool to patients and caregivers when used in routine care. Clinicians,²⁶ patients, and caregivers all wanted information about the factors that influenced AI output. Furthermore, we demonstrated continued poor understanding of what supportive care entails. Better explanation that palliative and supportive care includes pain and symptom management by medical experts, caregiver support, emotional and spiritual counseling, and end-of-life care⁴² must be provided.

A key takeaway from this work is that patients and caregivers provided valuable and actionable feedback for continued iterative design of a tool that supports provider-patient communication when making treatment decisions. Future directions include continuing to refine the tool to meet clinician, patient, and caregiver information needs and identifying implementation requirements such as creating a process to differentiate patients who want to receive information from those who do not. Building from this low-cost design study, next steps include resource-intensive pragmatic evaluation of the tool implemented in real-world settings with current patients with cancer. Impacts on clinical workflow, including time spent with patients and their caregivers while having SICs, must be considered. In response to participant requests for more information about how the ML model works, an assessment of how to promote explainability²⁸ and describe implications of model performance for patients and caregivers is needed.

Our study has limitations. It is likely we did not capture all patient and caregiver perspectives and perceptions of this tool, given our sample included individuals predominantly White and English-speaking, with reported higher income and education levels relative to the general population; 75% were residents of Utah, although the remainder were geographically diverse. We did not obtain feedback from patients who, themselves, had an advanced solid tumor (ie, a target user); however, over half of the participants were caregivers who represented the tool's target user by reporting they had cared for someone with advanced solid tumors and participated in prognosis conversations, thus providing key insight. Our sample size of 20 was small, and we stopped enrollment just as we were reaching saturation, although most usability issues are identified with as few as 10 users.⁴³ Finally, researcher bias in qualitative work is unavoidable. We sought to reduce researcher bias and increase dependability of coding by using standardized, semistructured interview guides and using multiple coders to interpret transcripts.⁴⁴

In conclusion, assessing patient and caregiver perceptions of an interface that displays ML prognosis information for patients with advanced solid tumors resulted in valuable, generalizable feedback beyond design decisions on the basis of provider input. Participants shared that AI-based prognosis information should be communicated early with patients who want to receive it and provided insight about avoiding judgmental terminology in communication. Finally, we demonstrate the importance and describe methods for including patients and caregivers early in the design of AI-based decision support tools.

AUTHOR CONTRIBUTIONS

Conception and design: Elizabeth A. Sloss, Jordan P. McPherson, Anna C. Beck, Carolyn H. Scheese, Naomi R. Flake, George Chalkidis, Catherine J. Staes

Financial support: George Chalkidis

Administrative support: Elizabeth A. Sloss, Catherine J. Staes

Provision of study materials or patients: Elizabeth A. Sloss

Collection and assembly of data: Elizabeth A. Sloss, Jordan P. McPherson, Anna C. Beck, Jia-Wen Guo, Naomi R. Flake, Catherine J. Staes

Data analysis and interpretation: Elizabeth A. Sloss, Jordan P. McPherson, Anna C. Beck, Jia-Wen Guo, Catherine J. Staes

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

No potential conflicts of interest were reported.