Perceptions of Implementing Real-Time Electronic Patient-Reported Outcomes and Digital Analytics in a Majority-Minority Cancer Center

Abstract

PURPOSE

Electronic patient-reported outcome (ePRO) tools are increasingly used to provide first-hand information on patient's symptoms and quality of life. This study explored how patients and health care providers (HCPs) perceive the use of a digital real-time ePRO tool, coupled with digital analytics at a cancer center located in a majority-minority county. Furthermore, we described the implementation barriers and facilitators identified from the participants' perspectives.

METHODS

We conducted a qualitative substudy as part of a larger implementation study conducted at University of California Irvine Chao Family Comprehensive Cancer Center. Patients and HCPs completed semistructured interviews and a focus group discussion. Thematic analysis was used to identify key themes regarding perceived impact of the intervention on patient's care and implementation factors.

RESULTS

A total of 31 participants, comprising 15 patients (67% English-speaking, 33% Spanish-speaking) and 16 HCPs (43.8% pharmacists, 37.5% physicians, 18.8% nurses), were interviewed. The utilization of real-time ePRO was perceived to beneficially affect patient care, improve patient-provider communication, and increase symptom awareness. Implementation facilitators included ease of comprehension and completion within the infusion center. Barriers included the need to incorporate results in electronic medical records and create real-time referral pathways to address patient's needs.

CONCLUSION

The use of real-time ePRO in a majority-minority population was perceived to enhance patient-centered oncology care, yet implementation barriers must be addressed for successful integration in clinical settings. The findings from this study may inform implementation strategies to reduce health disparities.

INTRODUCTION

Patient-reported outcomes (PROs) have gained recognition because of their ability to provide valuable insights into patients' experiences, symptoms, functional status, and quality of life directly from the patient's perspective. Since their development, PRO measures have been used for symptom monitoring in various populations, including patients with cancer.^1-5 Efforts have been made to make PRO tools applicable for routine clinical use, such as identifying clinically meaningful cut points and changes in scores,⁴ establishing real-time reporting of scores,^6-8 evaluating PRO effectiveness on clinical outcomes,⁹ and analyzing implementation factors.^10,11 Relatedly, with benefits of electronic data capture including fewer missing data, reduction of administrative burden, and automatic scoring, PRO tools are increasingly administered through electronic devices and platforms (ePRO).¹²

CONTEXT

• Key Objective

• How do patients and health care providers perceive the use of a digital real-time electronic patient-reported outcome (ePRO) tool in a cancer center located in a majority-minority county, and what are the perceived implementation barriers and facilitators?

• Knowledge Generated

• The use of real-time ePRO was perceived to improve patient care, communication, and symptom awareness. However, successful implementation requires addressing barriers such as integrating results into electronic medical records and establishing real-time referral pathways.

• Relevance

• ePRO tools are increasingly important for capturing patients' real-time symptoms and quality of life, providing first-hand information that can enhance clinical decision making. This study highlights how patients and health care providers at a cancer center in a majority-minority county perceive the use of a digital real-time PRO tool coupled with digital analytics, identifying implementation barriers and facilitators that are crucial for optimizing patient care and integrating ePRO tools into routine practice.

Past qualitative studies have evaluated implementation strategies and perceived clinical utility of PRO in patients with chronic diseases such as systemic lupus erythematosus,¹³ diabetes,¹⁴ substance use treatment,¹⁵ and heart failure.¹⁶ Moreover, studies have explored health care providers' (HCPs) experiences using PRO in oncology clinical settings,^17,18 reporting varied perspectives on barriers toward PRO implementation from administrative and nonadministrative stakeholders. However, to our knowledge, there is a lack of qualitative data from both patients and HCPs in oncology settings regarding the use of PRO to monitor for toxicities, symptoms, and wellness. Given that different stakeholders (ie, patients, HCPs, administrators) may hold differing or incompatible opinions, qualitative research can uncover nuances and shed light on details that quantitative studies might miss. Importantly, considering perspectives from patients belonging to racial/ethnic minority (REM) populations is particularly relevant because of the widely documented health disparities affecting cancer care among these groups,^19-21 as well as reported inequities related specifically to the use of PRO.^22-24 Similarly, HCPs serving mostly REM patients may possess unique perspectives which are critical to addressing health disparities.

The Patient-Reported Outcomes Measurement Information System (PROMIS), a repository of valid and reliable PRO measures developed by the National Institutes of Health, evaluates physical, mental, and social well-being, encompassing symptoms and functional abilities.^25,26 PROMIS has been used in oncology in both research and clinical settings to evaluate patient's psychosocial and physical health, showing potential for enhancing the quality of care and support for patients with cancer.²⁷ The purpose of this study was to describe perceptions regarding how the use of a digital real-time ePRO tool, coupled with digital analytics, can guide clinical interventions. Furthermore, we aim to explore the implementation facilitators and barriers of our intervention. These findings can inform further use of ePRO to enhance patient care within oncology settings.

METHODS

Study Design

This qualitative substudy is part of a larger prospective implementation study (N = 250) conducted at University of California Irvine (UCI) Chao Family Comprehensive Cancer Center (CFCCC), whose catchment area predominantly consists of REM patients, from July 2021 to June 2023. The parent study¹¹ evaluated the use of the PROMIS tool to monitor and intervene on symptom burden by oncology pharmacists. This substudy uses inductive thematic analysis to explore and analyze patients' and HCPs' perspectives on the use of PROMIS to inform providers' care within an oncology setting.

Participants

Inclusion criteria for participants in the parent study included being at least 18 years old, newly diagnosed with cancer, and receiving intravenous anticancer treatment at CFCCC. Participants of this substudy had completed the PROMIS tool and received pharmacist's counseling at least twice. To avoid sampling bias, we recruited patients with diverse language preferences (survey completion in English or Spanish) and differing opinions regarding satisfaction and acceptability toward the program, which were assessed in the parent study after each tool completion.¹¹ Patients responded to one Likert scale item assessing satisfaction (How satisfied are you with the counseling provided by your pharmacist?) and two Likert scale items assessing acceptability (How do you find the length of the electronic survey tool, and what do you think if this electronic survey tool is offered to you every visit to the infusion center?).¹¹ HCPs who worked at CFCCC during the time of data collection (June 2021-July 2023) and who may or may not have been involved with the parent study were invited to participate via email. Both patients and HCPs were compensated for their time. All procedures were approved by the UCI institutional review board (IRB#2021-6431). Written informed consent was obtained from all participants.

Intervention

The evaluated intervention (Fig 1) used an electronically administered PROMIS tool that assessed seven health domains at various time points throughout treatment (beginning with first chemotherapy visit). The domains included nausea and vomiting, physical impairment, anxiety, depression, fatigue, cognitive impairment, and pain interference. Patients completed the tool, administered through REDCap, in a designated iPad during their infusion unit visits. Scores were calculated in real time in the same iPad and displayed as degrees of severity (normal, mild, moderate, or severe).²⁸ Scores were evaluated by the oncology pharmacists immediately after completion, facilitating necessary education and interventions. The tool was available in English and Spanish.

FIG 1.

Overview of trialed intervention. ID, patient ID; PROMIS, Patient-Reported Outcomes Measurement Information System.

Procedures

Patients

Patients completed a one-on-one semistructured interview developed for this study (Data Supplement, Table S1) in-person at their private infusion chair. Interview questions were collaboratively generated by the parent study's principal investigator's research team, considering details of implementation and perception of utility and usability. Interviews were conducted by a bilingual researcher (D.A.) in English or Spanish, depending on patients' preferred language, and lasted approximately 15 minutes.

HCPs

HCPs completed a one-on-one semistructured interview developed for this study or participated in a focus group discussion (Data Supplement, Table S1). Individual interviews lasted approximately 15 minutes and were conducted via Zoom by a researcher (D.A.). The focus group was conducted in-person at a pharmacy room within CFCCC by the same researcher (D.A.) and lasted approximately 25 minutes. All interviews and the focus group discussion were conducted in English.

Analysis

Interviews and the focus group discussion were audio-recorded and transcribed verbatim, with identifiable information removed. Spanish transcriptions were translated to English by a bilingual researcher (D.A.) and then translated back to Spanish by a second bilingual researcher who had not read the original transcripts. The back translation was compared and analyzed with the original transcript to detect errors before finalizing the translated version.²⁹ Data were analyzed by the same coding team comprising four researchers (C.N., D.A., M.D., and R.N.). Analyses were performed on the English (original or translated version) transcripts using Braun & Clarke six-step approach.³⁰ The team met to reach a final consensus on the assigned codes, and as such, there was 100% agreement. Analyses were completed using the Dedoose Version 9.0.17 software.³¹ For a detailed description of the thematic analyses processes, see the Data Supplement (Table S2).

RESULTS

Participating patients (N = 15)—of whom 66.7% completed the interview in English and 33.3% in Spanish—included more than half male (60%) and majority Hispanic/Latino (46.7%) or White (33.3%; Table 1). For HCPs (N = 16), there was an equal distribution of male (50%) and female (50%) participants, which included pharmacists (43.8%), physicians (37.5%), and nurses (18.8%). HCPs in the one-on-one interviews (n = 11) had not been involved with the parent study, whereas those in the focus group discussion (n = 5) had been. Three themes emerged from the patients' interviews: (1) positive impact on patients' experience, (2) suggested improvements, and (3) elements that function. Similarly, we uncover three themes from the HCPs data: (1) PROMIS poses advantages to the quality of care, (2) improvements needed for optimal function, and (3) time and resources are necessary. Distribution of verbatims for each theme/code and verbatim examples are described in the Data Supplement, Tables S3 and S4, respectively.

TABLE 1.

Demographic Characteristics

Patients (N = 15)			Health Care Providers (N = 16)
	Mean	Range		N	%
Age, years	49	27-75	Sex
	N	%	Male	8	50
Sex			Female	8	50
Male	9	60	Profession
Female	6	40	Pharmacists	7	43.8
Race/ethnicity			Physicians	6	37.5
Hispanic/Latino	7	46.7	Nurses	3	18.8
White	5	33.3	Interview language
Asian	2	13.3	English	16	100
Other	1	6.7
Interview language
English	10	66.7
Spanish	5	33.3
Cancer type
Cervical	2	13.3
Colon	2	13.3
Testicle	1	6.7
Bladder	1	6.7
Endometrial	1	6.7
Lymphoma	1	6.7
Parotid gland	1	6.7
Pancreas	1	6.7
Lung	1	6.7
Ovary	1	6.7
Rectal	1	6.7
Breast	1	6.7
Melanoma	1	6.7

Impact on Patients' Care

The use of the PROMIS tool and the corresponding pharmacist counseling was perceived to be largely beneficial to the quality of care by both patients and HCPs. Specifically, patients expressed a sense of being well-cared for and of facilitated expression and communication (eg, “gives you a feeling of being cared for”—female patient/48 years, “[the tool] allows us to express what we feel”—male patient/48 years). This seemed to be particularly salient with Spanish-speaking patients, some of whom referred to existing language barriers regarding their communication with HCPs (eg, “we can’t always say what [we] feel”—male patient/48 years). Relatedly, HCPs recognized the potential of PROMIS in easing communication with non–English-speaking patients should the tool be available in multiple languages, which was noted within the discussion of language barriers negatively affecting patients' well-being (eg, “we see so many different languages so trying to expand the languages…would be awesome”—nurse, “about half [of the patients] speak English…it’s a very different population and very hard to communicate with”—physician, “when…[the patients] are not very capable with English or… they speak a language that we are not able to…it’s a little bit more challenging”—physician).

Patients reported increased awareness of their symptoms (eg, “made me think a little bit more about, ‘Oh, did I feel that?’”—female patient/48 years) and obtaining a benchmark that allowed them to compare their experiences with other patients. HCPs recognized how PROMIS could track symptom changes, ensure relevant information is not overlooked, and potentially aid in decision-making processes. Although most codes in this theme described a beneficial effect, HCPs shared concerns of burdening patients (eg, “some patients are…under a lot of stress…adding all those questions…it [would be] additional things that they have to do”—physician).

Implementation

Facilitators

Among elements perceived to function properly in its trialed form, the tool was reported to be easily understood and, generally, of an acceptable length. In addition, patients clearly preferred completing the tool during their infusion center visits (eg, “I wouldn’t want to have to go online and do it… [here, they] walk up and hand me that tablet. It’s just so easy.”—male patient/58 years, “in my house, well, there are children…here, I have the time”—female patient/49 years), in agreement with the procedure followed in the trial.

Barriers

Patients and HCPs mentioned aspects in which the intervention could be improved. Among these, patients perceived the tool to be redundant (eg, “like ‘oh, the same of question again’”—male patient/51 years). Similarly, patients suggested lengthening the tool's timespan to cover a wider timeframe (eg, “[it] was asking me if in the past 7 days I have felt nausea or anything, but since it’s been about 2 weeks since my last 5-day cycle, I haven’t really felt the symptoms that I felt during that [time]”—female patient/27 years). Relatedly, HCPs' suggested that a fixed frequency, rather than one dependent on treatment cycle lengths, could allow for better standardization (eg, “time periods are probably a little bit easier to do…than cycles”—physician).

Some patients expressed that the PROMIS tool had missed relevant symptoms (eg, “you should have hand-feet syndrome, tingling, some symptom[s] related to neuropathy and swelling”—male patient/59 years). In agreement, HCPs suggested modifying the questions to fit specific cancer or treatment types, while acknowledging the difficulties of having a survey that aimed to fit all (eg, “symptoms will depend on the type of tumor…there might be a challenge or a limitation in trying to capture all patients with the same survey”—pharmacist). In addition, patients expressed an interest in having enduring access to survey results (eg, “getting results over time through MyChart would be helpful”—male patient/53 years), something not currently in place during the parent study.

Importantly, both patients and HCPs acknowledged the necessity of referral pathways to address needs based on the tool's results (eg, “there should be some rows put into the system on the next step, which is providing some support on that area”—male patient/59 years). Occasionally, this was mentioned alongside HCPs' concerns of litigation (eg, “it’s risky…if you're measuring something and…you don't have programs to specifically act on positive results”—physician). Furthermore, some patients perceived the survey was more beneficial to the researchers or the institution and less to the patient, which affected willingness to pay for a service comprised of the tool's use to guide providers' care (eg, “would I pay for it? No…it’s benefiting more you guys’ research”—female patient/48 years). Nonetheless, opinions about payment were varied, with mention of financial situation or insurance coverage as factors that would influence ability or willingness to pay.

Finally, HCPs suggested patients complete the tool at home or before their visits to the medical center and stressed the need of incorporating the PROMIS results in the electronic medical records (EMRs)—something not available in the trialed version. The benefits of having the tool completed before appointments and the results incorporated in the EMRs included increased efficiency (eg, “we can incorporate these questions…earlier or more efficiently, rather than waiting until patients sit down”—physician), facilitated use of the results for clinical decision making, and reduction of administrative burden (eg, “our workflow is complex and tedious, and anything we can do to reduce additional administrative burden is extremely helpful”—physician). Relatedly, HCPs acknowledged issues with patients' use of current electronic platforms that could affect the implementation of PROMIS (eg, “at present…we don’t even have close to 80% of our patients who have MyChart”—physician), while sharing concerns about whether the use of the tool could disrupt workflow or represent additional work (eg, “there’s so much information that’s [already] expected to be put into the EMR”—nurse).

DISCUSSION

In our exploration of perspectives on an ePRO intervention with real-time reporting driving oncology pharmacist counseling, patients and HCPs offered insights regarding its impact in quality of care, highlighted facilitators, and suggested improvements for future applications. The findings add to evidence^13,32-34 indicating that the use of PRO is perceived to have a beneficial impact in quality of care, improving provider-patient communication, providing patients with a sense of being well-cared for, and increasing symptom awareness. The potential benefits were emphasized in relation to having the tool available in multiple languages and, as such, reducing language barriers—stressing the value of multilanguage ePRO in a majority-minority setting. Importantly, with a significant representation of patients belonging to REM backgrounds in our sample, this study offers insights into implementation aspects that should be considered when addressing health disparities in PRO utilization and cancer care.

Regarding limitations of the trialed intervention, some patients viewed the tool as repetitive; an interesting finding given that we used the PROMIS computer adaptive tests versions (in six of seven domains¹¹), which are designed to be more efficient by tailoring items to participant responses, thus reducing response burden.³⁵ Relatedly, results from our parent study showed that compared with White patients, REM patients spent additional time (1.7-2.7 minutes depending on race/ethnic group) completing the tool.¹¹ Furthermore, both patients and HCPs noticed relevant symptoms to be missing. Nonetheless, providers recognized the limitations of attempting to have one tool that would be applicable to all patients, acknowledging the wide range of symptoms and how these differ depending on cancer and treatment type.

The latter supports previous findings reporting clinicians perceive the need to tailor PRO to the individual³⁶ and prompts the necessity of devising methods that use ePRO in a way that achieves both comprehensiveness and succinctness. For example, future research should allow clinicians to select symptom domains based on the patient's specific cancer or treatment type, provide opportunities for patients to choose relevant domains, or offer open-text options to capture missed symptoms. Despite potential limitations, assessment approaches that aim to improve conciseness while maintaining thoroughness warrant further consideration. Moreover, participants highlighted the need to incorporate referral pathways to address psychosocial symptoms screened for in the PROMIS tool. Among HCPs, this related to concerns of litigation if symptoms are documented but not addressed, in agreement with concerns previously expressed by administrative stakeholders.¹⁸

In contrast to the procedure followed in the trialed intervention, providers often suggested patients complete the tool at home before appointments. This recommendation was supported by time and resource concerns, such as needing to reduce the time patients spend at infusion chairs because of high rotation rates. Similarly, HCPs emphasized the efficiency of having scores available before visits in the EMR, in support of previously documented workload apprehensions if PROs are not incorporated appropriately.^37-39 Importantly, however, patients appreciated completing the tool at the medical center, referring to convenience, time availability, absence of competing tasks, and lack of connectivity issues. These differing perspectives stress the need to consider HCPs' logistical concerns while acknowledging and accommodating patients' preferences to devise ePRO interventions that can be optimally used in clinical practice.

Patient's preference for completing the tool within the cancer center can be linked to digital disparity concerns. A previous study showed that a shift from PRO completion in a tablet in clinic (which had equitable completion rates across race/ethnicities) to an online portal led to profound inequities in data collection—with Black and Hispanic patients completing PRO at significantly lower rates compared with White patients.²² Although our study did not explicitly seek to compare perspectives from White versus non-White patients nor from English- versus Spanish-speaking patients, the significant representation of REM and Spanish-speaking patients in our sample hints toward the relevance of these sentiments. Location of PRO completion should be carefully evaluated to avoid perpetuating health disparities impacting REM.

Our study is subject to certain limitations. Although our purposeful sampling aimed to recruit a varied group of patients in terms of demographic characteristics, satisfaction, and acceptability toward the intervention, as well as include both English-speaking and non–English-speaking patients, there might be novel perspectives that were not captured. Similarly, although attempts were made to include both HCPs with and without prior exposure to the parent study, it is not possible to know whether our sample had particularly positive or negative attitudes about ePRO before participating. Relatedly, we did not obtain HCPs' age nor race/ethnicity to preserve anonymity of participants in our study. Finally, although we acknowledge the presence of research bias in qualitative research is inevitable, semistructured interview guides and a team of multiple coders were used to mitigate potential biases.

In conclusion, this study significantly contributes to the understanding of patients' and HCPs' perceptions on a real-time, multilanguage ePRO-driven pharmacist counseling intervention in an oncology setting within a majority-minority setting. Our findings corroborate existing evidence suggesting that ePRO may enhance care delivery, while underscoring the need for careful evaluation of workflow integration that considers both HCPs' and patients' interests. Findings from this study may inform ePRO implementation strategies within oncology clinical settings, as well as support efforts to address health disparity issues.

AUTHOR CONTRIBUTIONS

Conception and design: Daniela Arcos, Mary Dagsi, Alexandre Chan

Financial support: Alexandre Chan

Collection and assembly of data: Daniela Arcos, Mary Dagsi, Reem Nasr, Carolyn Nguyen, Alexandre Chan

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).