Real-World Use of Electronic Patient-Reported Outcome (ePRO) Tools Integrated in the Electronic Medical Record During Radiation Therapy for Head and Neck Cancer: Feasibility Study

Abstract

Purpose

Use of electronic patient-reported outcome (ePRO) tools in routine oncology practice can be challenging despite evidence showing they can improve survival, improve patient and practitioner satisfaction, and reduce medical resource utilization. Head and neck cancer (HNC) patients receiving radiation therapy (RT) may be a group that would particularly benefit from interventions focused on early symptom management.

Methods

Patients undergoing definitive RT for HNC were enrolled in a feasibility study and received ePRO surveys integrated within the electronic medical record (EMR) on a weekly basis during RT. After completion of each ePRO survey, a radiation oncology registered nurse documented the findings and subsequent interventions within the EMR.

Results

Thirty-four patients with HNC who received curative RT at a single center were enrolled. The total number of surveys completed was 194 with a median of 7 surveys per patient (range 1–8). There was a total of 887 individual abnormal findings reported on the ePROs, and the authors found that all 887 had a corresponding documented intervention. Post-treatment practitioner questionnaires highlighted that ePROs were felt to be helpful for the care team in providing care to HNC patients.

Conclusion

For patients with HNC receiving RT, ePROs can be effectively utilized to address patient symptoms within an integrated health care system. Creating an infrastructure for the use of ePROs integrated within the EMR in routine care requires an approach that accounts for local workflows and buy-in from patients and the entire care team.

Introduction

Electronic patient-reported outcomes (ePROs) use is associated with improved clinical outcomes and reduced resource utilization. ePROs have become an indispensable part of clinical trials in which they provide key information about participants’ health-related quality of life. ^1,2 The use of ePROs has been shown to improve overall survival in patients with cancer ^3,4 reduce hospitalizations, contribute to improved patient satisfaction and patient–practitioner communication. ^4–19 However, the routine use of ePROs in oncology care outside of clinical trials remains challenging. ^1,17,20–24

For patients with HNC, where intensive treatments such as surgery, chemotherapy, and radiation therapy (RT) frequently impact both short-term and long-term quality of life, prompt symptom management is vital. Based on data showing the utility of ePROs in reporting patient symptoms in HNC, ^25–27 in 2014 a National Cancer Institute working group identified a standard core set of patient-reported symptoms and health-related quality-of-life domains to be assessed in HNC clinical trials. ²⁸ One challenge to successful integration of ePROs in routine oncology care is patient adherence. Response rates for patient-reported outcome (PRO) surveys in the literature have varied widely, with online and paper-survey response rates typically between 64% and 80%. ^16,29–31 Some groups have reported higher response rates, for example, by adding electronic reminders and real-time monitoring. ⁶ PRO response rates as high as 89% were reported in oncology patients in the ProtecT study. ⁶ Specifically, in the HNC patient population, one study used in-person PRO surveys to achieve a response rate of 90%, ⁵ and the authors previously reported a PRO completion rate of 97.9% among a cohort of 290 patients using a dedicated medical assistant. ³²

In this feasibility study, the aim was to evaluate the process of incorporating ePRO surveys into the routine care of patients with HNC undergoing treatment with RT. The ePRO surveys were directed to patients using their electronic health care portal, and responses were collected and stored within the electronic medical record (EMR). This permitted ePRO data to be readily available to health care practitioners at patient visits. Abnormal findings on ePROs were reviewed by radiation oncology registered nurses (RNs), and interventions were documented using a standardized documentation process. The primary aim was to assess the feasibility of ePROs integrated within the EMR during RT for HNC patients within a large health care delivery system. Secondary endpoints included resource utilization and practitioner satisfaction.

Methods

A PRO tool, consisting of the Functional Assessment of Cancer Therapy—General—7 Item Version, and Functional Assessment of Cancer Therapy—Head and Neck, with a total of 19 questions, was integrated into the EMR using the flow sheet function within Epic (2020 Epic Systems Corporation). Patients were consecutively enrolled in this feasibility study between January 1, 2021, and December 31, 2021. Eligible participants included members of Kaiser Permanente in Northern California, age 18 years and older, with newly diagnosed HNC who were undergoing RT for HNC at the Kaiser Permanente Northern California South San Francisco campus.

Eligible patients were assigned ePRO questionnaires in English through the EMR at diagnosis (baseline) and weekly thereafter during RT treatment (Figure 1). Patients with access to the secure practitioner–patient messaging system within Epic received a secure email that linked to their individual ePRO survey with results stored in the individual’s EMR in Epic. For patients without portal access or those who were unable to complete due to language barriers, surveys were administered by the RN in the radiation oncology clinic by phone or in-person at the time of their RT clinical visit. For patients seeking assistance with electronic administration, RNs in the radiation oncology clinic provided additional guidance on how to access their health portal. Additionally, language interpreters were available for patients who were non-English speakers. Results, viewable through the flow sheet function in Epic, were reviewed by an RN, who then coordinated care with radiation oncologists, medical oncologists, primary care practitioners, palliative care specialists, dietitians, social workers, and other team members as needed and consistent with routine care symptom management pathways.

Figure 1:

Consort diagram. RN = registered nurse; ePRO = electronic patient-reported outcome; EMR = electronic medical record.

Patient clinical and demographic characteristics (such as age, sex, race and ethnicity, preferred language, tobacco use, cancer stage, and treatment) were extracted from the EMR. A modified version of the Charlson Comorbidity Index was calculated by identifying comorbidity diagnosis codes in the year before the administration of each patient’s baseline survey, excluding the contribution of cancer diagnoses to the overall score. ^33,34

The radiation oncology care team, led by RNs, organized a tracking system for patients who agreed to participate in the study (Table 1). Data in the tracking system required manual data entry to maintain a list of patients who required assignment of weekly ePRO surveys and to track their participation at each interval. Patients would then receive ePRO surveys in their online portal where they could complete the surveys at a convenient time and place. The data were subsequently imported into the EMR using RN patient care notes. Abnormal symptom reports from ePROs were easily identified because they appeared in a predesigned distinguishable alert format (all caps). Using a standardized format (Table 2), interventions based on the ePRO findings were documented in the EMR. To assist the RN team members who were intervening in the ePRO findings, a symptom management pathway was created in collaboration with other members of the care team.

Table 1:

Example of patient tracking tool designed by radiation oncology registered nurses

Patient name	Radiation oncologist	Consult date	Radiation simulation date	Dental evaluation?	Patient active on patient portal?	Requires interpreter? (Language)	Start of treatment date	End of treatment	Baseline ePRO?	Next ePRO date
Patient 1	Dr X	1/10/2021	1/12/2021	Yes	Yes	Yes (Cantonese)	1/22/2021	4/5/2021	Done	2/3/2021

Table 2:

Example of standardized format for documentation of electronic patient-reported outcome findings within the electronic medical record

Sample ePRO abnormal finding	Sample RN action plan
Lack of energy	Check vitals, encourage rest, evaluate hydration status, evaluate nutrition status
Pain	Radiation oncologist to discuss changes in pain regimen and/or input from pain team
Nausea	Check vitals, consider antiemetics, check with oncology pharmacy if patient is receiving chemotherapy
Worry	Consider social work referral
Lack of sleep	Consider melatonin, next step is MD evaluation for prescription medications

ePRO, electronic patient-reported outcome; MD, medical doctor; RN, registered nurse.

To promote stakeholder engagement, an expansive team of HNC care practitioners was identified to lead this effort. Team members included RT RNs, a dietitian, RN department manager, radiation oncologists, and medical oncologists. There were regular team meetings that included workflow and symptom management optimization prior to the start of the study and during the study period. Monitoring of the electronic processes and receptivity from patients was also discussed among the team during the study. A “kick-off” event was held on the first day of the study period to promote practitioner engagement. At the end of the study, satisfaction questionnaires and interviews were performed with team members on information ranging from ease of use to perceived benefit in the clinic setting, and to receive any feedback clinicians received from patients regarding the use of ePRO surveys.

This was an observational study. The Kaiser Permanente Northern California institutional review board confirmed that no ethics approval is required.

Results

The study population included 34 patients, with a median age of 65 years, and was predominately male (79%) and White (59%; Table 3). The patients had a median modified Charlson index score of 1. Ninety-seven percent of the overall cohort during the study period had access to the online secure portal. Approximately 18% of the cohort preferred a language other than English, with the majority of these being Cantonese speakers. The most common primary cancer sites were oropharynx (32%), nasopharynx (24%), larynx (18%), and oral cavity (9%). Sixty-five percent of patients received concurrent chemotherapy with RT, and 35% received RT alone.

Table 3:

Demographic data

Factor	Overall (N = 34)
	n	%
Age, y
< 60	8	23.5
60–69	15	44.1
70–79	9	26.5
80 +	2	5.9
Sex
Male	27	79.4
Female	7	20.6
Race
White	20	58.8
Black	2	5.9
Asian/PI	8	23.5
Other	4	11.8
Ethnicity
Hispanic	2	5.9
Non-Hispanic	32	94.1
Modified Charlson Comorbidity Index
0	13	38.2
1	9	26.5
2	5	14.7
3 +	7	20.6
Tumor site
Oral cavity	3	9
Oropharynx	11	32
Larynx	6	18
Other	14	41
Treatment
RT only	12	35
RT with chemotherapy	22	65
Treatment intent
Curative	34	100
kp.org access a
Yes	33	97.1
No	1	2.9
Preferred language
English	28	82.4
Other	6	17.6
No. of FACT/QoL
Baseline survey	34	100

^aKp.org access = having a registered secure health care portal account with Kaiser Permanente.

FACT/QoL, Functional Assessment of Cancer Therapy/quality of life; PI, Pacific Islander; RT, radiation therapy.

During the study period a total of 194 ePRO surveys were completed. The median number of surveys per patient was 7 (range 1–8). The overall completion rate for ePROs by patients was 82% during the study period (Figure 1). RNs documented a total of 887 individual abnormal findings within the EMR and also documented a plan of action for each abnormal finding (RN documentation rate of 100%). Over the course of RT for each patient the mean number of abnormal ePRO findings increased, corresponding to increased symptom burden as treatment progressed. The identified symptoms increased from 2.26 at baseline (n = 34) to 6.27 at week 7 (n = 24), p < 0.0001 (Figure 2).

Figure 2:

Change in mean number of abnormal electronic patient-reported outcome symptoms. ePRO = electronic patient-reported outcome.

End-of-study satisfaction questionnaires revealed that 67% (4/6) of practitioners felt that ePROs were helpful in discussions with patients, and they helped to provide more personalized care. Respondents were evenly split on the question of whether ePRO was a time-saving tool in the clinical setting. The most frequently identified barriers to the routine use of ePROs identified by practitioners were language (surveys in English only) and lack of patient access to a computer or the internet.

Resource utilization data for patients enrolled in the study, including hospitalizations and emergency department visits during the study and up to 30 days from last day of RT, identified 2 hospitalizations and 8 emergency department visits among this cohort. One hospitalization was related to a chronic obstructive pulmonary disease exacerbation and one for COVID-19. The most frequent cause of emergency department referral was dehydration.

Discussion

Our findings are similar to findings from other organizations that have incorporated ePROs into the management of patients with cancer or other chronic conditions, ^12,30,35,36 including HNC patients. ^36,37 Regarding the use of ePROs with RT, one group showed that ePROs could replace paper assessments and provide reliable pain assessments, ²⁹ and another group even designed a practical guide for navigating ePROs during RT. ^37,38 However, to the best of the authors’ knowledge, this study is the first systematic evaluation of ePRO surveys given weekly for HNC patients during RT outside of a clinical trial and within a large integrated health care system.

Conclusions

Studying the feasibility for ePROs during radiation therapy among a cohort of patients with HNC provided important insights into the challenges as well as potential benefits for streamlined patient symptom management. The authors learned that ePROs could successfully be deployed within a large integrated health care system after the necessary infrastructure was in place to support the care team. Key learning points are outlined in Table 4.

Table 4:

Procedural steps for electronic patient-reported outcome implementation within a large integrated care system

Timing	Process Elements
Prior to start of study	Team meetings to explain the benefits of ePROs Incorporating desired PRO into EPIC Creating a patient tracking tool Plan for shared use of tracking tool Creating standard symptom management protocols Creating standard documentation process for responding to ePROs Kick-off event to spread the word about the study
During ePRO study period	Verifying tracking tool is identifying all relevant patients Modifications/adjustments to standardized symptom management protocols Modifications to documentation process Periodic team check-ins
After the study	Satisfaction assessment from patients and practitioners Assessment of outcomes

ePRO, electronic patient-reported outcome; PRO, patient-reported outcome.

Although this study was not designed to evaluate outcomes from ePROs in this patient population, it is likely that benefits demonstrated previously for the use of ePROs including improved overall survival, improved communication between patients and practitioners, and decreased resource utilization, should benefit patients like those in this cohort and within this organization as well. The authors hope that these findings will promote ePRO use more broadly within organizations related to cancer and possibly for disorders outside of oncology.

The authors identified several limitations of this feasability study. First, ePRO surveys were available only in English language. Approximately 18% of the patients in the study’s cohort preferred a language other than English, with the majority being Cantonese speakers. As a result, there was a strong reliance on interpreters for assisting with survey administration, making the use of ePRO more cumbersome for those patients. This highlighted the need for ePRO surveys in patients’ native languages. Second, the authors’ small cohort of 34 patients, without a control arm and limited to patients who consented to enroll, limits the ability to evaluate clinical outcomes and resource utilization. Third, satisfaction questionnaires by health care practitioners were limited by small sample size (n = 6) but generally pointed toward support for ePROs, although practitioners were split on whether they saved time. Fourth, patient-satisfaction questionnaires were not provided in this study. Patient-satisfaction questionnaires can be informative in future studies. Despite these limitations, the authors were able to demonstrate the feasibility of ePROs in routine care of HNC patients during RT. These limitations will help in designing future studies and quality improvement initiatives.

In sum, the findings from this feasibility study demonstrate that with proper planning and a dedicated care team, ePROs can be effectively deployed for HNC patients during RT. The population of patients with HNC who face intensive treatment with RT with or without chemotherapy are a group who require close symptom monitoring and warrant future study on the use of tools that may aid in symptom management, such as ePRO. These findings support the feasibility of ePRO evaluations and follow-up interventions in large, diverse, community-based settings and can inform future studies of ePROs.