Key Points
Question
In patients undergoing cancer treatment, does electronic symptom monitoring improve quality-of-life (QOL) outcomes?
Findings
In this cluster randomized trial that included 52 oncology practices and 1191 patients, an intervention consisting of patient-reported surveys completed electronically each week to monitor symptoms, compared with usual care, resulted in statistically significant improvements in QOL outcomes at 3 months (measured as difference in change from baseline using the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire; range, 0-100 points) for physical function (mean difference, 2.47 points; minimum clinically important difference [MCID], 2-7), symptom control (mean difference, 2.56 points; no established MCID), and health-related QOL (mean difference, 2.43; no established MCID).
Meaning
Among adults receiving treatment for metastatic cancer, weekly electronic symptom monitoring resulted in statistically significant mean improvements of approximately 2.5 points on a 0- to 100-point scale in QOL outcomes at 3 months; the findings should be interpreted provisionally, pending results of the study’s primary outcome of survival.
Abstract
Importance
Electronic systems that facilitate patient-reported outcome (PRO) surveys for patients with cancer may detect symptoms early and prompt clinicians to intervene.
Objective
To evaluate whether electronic symptom monitoring during cancer treatment confers benefits on quality-of-life outcomes.
Design, Setting, and Participants
Report of secondary outcomes from the PRO-TECT (Alliance AFT-39) cluster randomized trial in 52 US community oncology practices randomized to electronic symptom monitoring with PRO surveys or usual care. Between October 2017 and March 2020, 1191 adults being treated for metastatic cancer were enrolled, with last follow-up on May 17, 2021.
Interventions
In the PRO group, participants (n = 593) were asked to complete weekly surveys via an internet-based or automated telephone system for up to 1 year. Severe or worsening symptoms triggered care team alerts. The control group (n = 598) received usual care.
Main Outcomes and Measures
The 3 prespecified secondary outcomes were physical function, symptom control, and health-related quality of life (HRQOL) at 3 months, measured by the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire (QLQ-C30; range, 0-100 points; minimum clinically important difference [MCID], 2-7 for physical function; no MCID defined for symptom control or HRQOL). Results on the primary outcome, overall survival, are not yet available.
Results
Among 52 practices, 1191 patients were included (mean age, 62.2 years; 694 [58.3%] women); 1066 (89.5%) completed 3-month follow-up. Compared with usual care, mean changes on the QLQ-C30 from baseline to 3 months were significantly improved in the PRO group for physical function (PRO, from 74.27 to 75.81 points; control, from 73.54 to 72.61 points; mean difference, 2.47 [95% CI, 0.41-4.53]; P = .02), symptom control (PRO, from 77.67 to 80.03 points; control, from 76.75 to 76.55 points; mean difference, 2.56 [95% CI, 0.95-4.17]; P = .002), and HRQOL (PRO, from 78.11 to 80.03 points; control, from 77.00 to 76.50 points; mean difference, 2.43 [95% CI, 0.90-3.96]; P = .002). Patients in the PRO group had significantly greater odds of experiencing clinically meaningful benefits vs usual care for physical function (7.7% more with improvements of ≥5 points and 6.1% fewer with worsening of ≥5 points; odds ratio [OR], 1.35 [95% CI, 1.08-1.70]; P = .009), symptom control (8.6% and 7.5%, respectively; OR, 1.50 [95% CI, 1.15-1.95]; P = .003), and HRQOL (8.5% and 4.9%, respectively; OR, 1.41 [95% CI, 1.10-1.81]; P = .006).
Conclusions and Relevance
In this report of secondary outcomes from a randomized clinical trial of adults receiving cancer treatment, use of weekly electronic PRO surveys to monitor symptoms, compared with usual care, resulted in statistically significant improvements in physical function, symptom control, and HRQOL at 3 months, with mean improvements of approximately 2.5 points on a 0- to 100-point scale. These findings should be interpreted provisionally pending results of the primary outcome of overall survival.
Trial Registration
ClinicalTrials.gov Identifier: NCT03249090
This randomized clinical trial assesses the effect of using weekly electronic patient-reported outcome surveys to monitor symptoms, compared with usual care, on physical function, symptom control, and health-related quality of life among patients receiving treatment for metastatic cancer.
Introduction
Symptom management is an essential component of treating patients with cancer. Patients with cancer frequently experience symptoms related to disease and treatment,1,2 which can contribute to distress and complications.3,4 However, symptoms during treatment of cancer are frequently undetected and untreated.5,6,7 Interventions that detect and treat symptoms may improve outcomes in people undergoing treatment for cancer.
Symptom monitoring using electronic systems that facilitate patient-reported outcome (PRO) surveys via the internet, mobile applications, or automated telephone interfaces have been shown to be feasible for identifying symptoms that can be treated by clinicians.8 Prior studies have reported improved outcomes for physical function, symptom control, health-related quality of life (HRQOL), hospitalizations, and survival when such electronic systems are used by patients receiving cancer treatment.8,9,10,11,12,13,14 Integrating symptom monitoring into routine treatment of cancer may improve outcomes with an opportunity for reimbursement by payers.15
Based on findings from a prior single-center study,10 a multicenter randomized clinical trial, the PRO-TECT trial (Alliance AFT-39), was conducted across 52 US-based community oncology practices to determine whether electronic monitoring of symptoms by patients undergoing treatment for metastatic cancer improves survival, QOL outcomes, and other outcomes compared with usual care. In this report, the QOL secondary outcomes and selected exploratory outcomes are presented.10 Analysis of the trial’s primary outcome, overall survival, requires national administrative data that are not yet available.
Methods
Patients
This was a multicenter cluster randomized trial evaluating electronic symptom monitoring with PRO surveys compared with a usual care control group. All participants signed informed consent. The trial protocol and consent form were approved by central and local institutional review boards. The statistical analysis plan and trial protocol were finalized prior to data analysis and are available in Supplement 1.
Community oncology practices in the US national network of the Alliance for Clinical Trials in Oncology (http://www.allianceforclinicaltrialsinoncology.org) were invited to participate. Practices were asked to consecutively approach and enroll up to 50 adults with metastatic cancer of any type receiving treatment with chemotherapy, targeted oral therapy, and/or immunotherapy if they understood English, Spanish, or Mandarin. Patients with indolent lymphoma or acute leukemia or who were receiving hormonal monotherapy were excluded. Full eligibility is listed in the trial protocol (Supplement 1). Race and ethnicity information was obtained from patients using fixed categories to determine whether effects of the intervention differed by race or ethnicity.16
Randomization
Participating practices were stratified by rural/urban designation using US Census criteria and randomized using permuted blocks with block sizes of 2 or 4. Randomization lists were computer generated based on random numbers and concealed until sites were randomized. A cluster randomized design was used to avoid influencing symptom management procedures in people randomized to usual care.
Intervention
Practices received online access to standardized educational materials for managing symptoms, including patient-level and clinician-level versions (eFigures 1 and 2 in Supplement 2). Practices randomized to PROs received access to the electronic survey system based at the University of North Carolina’s PRO-Core facility.17
The electronic survey system was developed based on prior publications.18,19,20 The system included a survey with questions from the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) validated item library,21,22 selected based on prevalence across different types of advanced cancers,2 asking about symptoms of pain, nausea, vomiting, constipation, diarrhea, dyspnea, insomnia, and depression, as well as questions about oral intake (eating/drinking), performance status (patient-reported Eastern Cooperative Oncology Group [ECOG] criteria),23 falls, and financial challenges24 (eTable 1 in Supplement 2). Surveys were available in English, Spanish, or Mandarin. Patients could complete surveys via the internet or an automated telephone system.
Patients at practices randomized to the PRO intervention were asked to complete surveys weekly for 1 year or until they discontinued all cancer treatment. On a day of the week and time of day selected by each patient, an email or automated call prompted survey completion. If a patient did not complete the survey within 24 hours, they received a reminder prompt. If they did not complete the survey after 72 hours, they received a “backup” call from a clinic staff member reminding them to self-report or offering to administer the survey orally. Family members, caregivers, or staff were permitted to assist patients completing surveys.
Whenever a PRO-CTCAE score reached a prespecified level of magnitude or worsening compared with the prior survey (eTable 1 in Supplement 2), the patient received an email with a link to patient-level educational materials about their self-management of that symptom (eFigure 1 in Supplement 2). In addition, an email-based alert notification was sent to a designated administrative staff member at the patient’s oncology practice who routed it to a clinical nurse responsible for that patient; notifications included PRO scores and clinician-level educational materials for managing the alert symptom(s) (eFigure 2 in Supplement 2). Alerts were also sent for ECOG scores greater than 2 (range, 0-5; 4 = worst function; 5 = death), ECOG scores that worsened by 2 or more points, falls (“yes”), and financial distress scores greater than 2 (“quite a bit” or “very much”; range, “not at all” to “very much”). Reports showing the trajectory of PROs could be visualized on screen or printed for clinicians (eFigure 3 in Supplement 2). Responses to the alerts by nurses or oncologists were not dictated by the study, were left to the discretion of the clinicians, and were documented on a standardized form. The electronic system was not integrated into electronic medical record systems.
Outcomes
The initial study protocol, approved on July 27, 2017, included co–primary outcomes of physical function at 3 months and overall survival. On December 5, 2020, prior to data analysis, an amendment to the protocol changed the primary outcome to overall survival as the single primary outcome in order to increase statistical power for this outcome and because it was determined that there was more than adequate power for the physical function outcome. At this time, physical function was changed to a key secondary end point. Additional secondary outcomes included symptom control and HRQOL. Emergency department visits and duration of chemotherapy were included in a list of secondary outcomes in the initial protocol but were clarified as exploratory outcomes on April 1, 2021, prior to data analysis. The statistical analysis plan specified month 3 as the principal analysis time point for secondary outcomes, with additional outcomes collected at months 1, 6, 9, and 12.
Physical function, symptom control, and HRQOL outcomes were obtained from patients during clinic visits at baseline and months 1, 3, 6, 9, and 12 of participation via the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire (QLQ-C30).25 The QLQ-C30 is a widely used 30-item questionnaire with established measurement properties.26 Physical function was assessed using 5 items, which generated a single score (symptom burden/control was assessed as a composite of 8 QLQ-C30 symptom scale scores27; and overall HRQOL was assessed by combining function and symptom scale scores.26,28 Each outcome was evaluated using a 0- to 100-point scale (higher scores are better). For the physical function outcome, a small minimum clinically important difference (MCID) has been defined as 2 to 7, but no MCID has been defined for symptom control or HRQOL as scored in this trial.29
Prespecified exploratory outcomes included emergency department visits and duration of chemotherapy administration, which are not reported in this article, and individual symptom items from the QLQ-C30. In the PRO group only, feedback was solicited from participants and nurses about acceptability of and satisfaction with the PRO system, after 3 and 6 months of participation, respectively.10 A post hoc analysis examined the frequency of and reasons for alert notifications as well as clinicians’ responses to them.
Patients in both groups received $75 for their time and effort related to completing a research demographics form and the QLQ-C30 at enrollment and 3 months. Data for overall survival will be obtained from administrative databases and were not available at the time this report was prepared.
Sample Size Calculation
The initial protocol included a sample size of 1000, which was increased to 1200 in an institutional review board amendment on January 17, 2019, when additional funding became available to increase the number of participants who were Black or African American, yielding 90% power for the primary outcome of overall survival, when physical function was still a co–primary outcome, for a hazard ratio of 0.76 (based on prior research10) using a 2-sided α = .05 log-rank test with 576 observed events, with an intracluster correlation coefficient of 0.001 (based on prior trials), assuming dropout of 150 patients in the first 2.5 years. When physical function became a key secondary end point on December 5, 2020, 1200 patients at 50 to 55 sites provided greater than 90% power to detect a 0.37-SD difference between randomization groups based on prior research,10 assuming an intracluster correlation coefficient of 0.055 and assuming that 85% of patients provided data at 3 months.
Statistical Analysis
Mean change from baseline in physical function, symptom control, and HRQOL were compared between groups at each visit using a linear combination of parameters from a general linear mixed model. Each model included all available data from all time points from all patients according to their randomization group. Fixed effects included study group, time point, cancer type, and group-by-visit interaction. A random practice intercept term was included to account for clustering by practice. Repeated observations by patient were modeled using compound symmetric correlation structure over time. General linear mixed modeling produces unbiased estimates under an assumption of missing-at-random data including handling patient dropout due to death or other causes.30
In a responder analysis, patients who completed the QLQ-C30 at baseline and each follow-up time point were categorized as improved on each outcome if their score increased by at least 5 points from baseline; worse if their score decreased by at least 5 points; and otherwise as stable. A 5-point change was selected based on prior research.31 In a sensitivity analysis, results were repeated using a 10-point change to define improvement or worsening. The proportion of patients with improvement, stability, or worsening was compared between groups at each time point using a cumulative logistic regression model with fixed effects for group and cancer type and a random practice intercept term to account for clustering by practice. Minimum clinically important differences were not established for the differences in response rates between groups for the measured outcomes. The primary time point for mean comparisons and responder analyses was prespecified as month 3, with additional outcomes collected at months 1, 6, 9, and 12. Preplanned subgroup analysis of physical function used the same mixed-model approach described above to compare mean changes from baseline at month 3 between groups and within subgroups. An additional mixed model was performed for interaction testing (eTable 2 in Supplement 2). Patient and nurse feedback items were analyzed descriptively. Completion of weekly PRO surveys was defined as the proportion of patients who completed an expected survey at each weekly time point.
Statistical testing was 2-sided, with P < .05 considered statistically significant, and was carried out in SAS version 9.4 (SAS Institute Inc). Because of the potential for type I error, findings for the secondary end points presented in this article should be considered exploratory.
Results
On March 23, 2020, the study chair made the decision to discontinue enrollment, with 1191 (99.3%) of 1200 planned enrollees, because practice site personnel were no longer able to recruit patients during the COVID-19 pandemic.
This trial included 52 community oncology practice sites across 25 states, randomized 1:1 to PRO or control, with 14 (26.9%) designated as rural (7 per group) (Figure 1; eTable 3 in Supplement 2). The post hoc intracluster correlation coefficient for physical function was 0.020 and varied between less than 0.001 and 0.039 for other outcomes (eTable 4 in Supplement 2). Each practice site was discrete. No staff worked at more than 1 site. Between October 31, 2017, and March 23, 2020, 1444 patients were approached and 1191 were enrolled (593 to the PRO group and 598 to the control group). Participants had a median age of 63 years (range, 28-93 years); 694 (58.3%) were female; 925 (79.5%) were White; 317 (26.6%) were recruited from a rural practice location; 468 (39.4%) had a high school education or less; and 201 (16.9%) had limited internet experience (Table). At 3 months, 1098 (97.3%) of 1191 participants remained in the study and 1066 (97.1%) of those 1098 participants had completed the QLQ-C30 (Figure 1).
Figure 1. Participant Flow in the PRO-TECT Trial.
PRO indicates patient-reported outcome; QLQ-C30, European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire. Patient cluster size: intervention group, mean, 22.8 (SD, 12.5) [range, 2-50]; control group, mean, 23.0 (SD, 8.6) [range, 3-42]. Rural/urban practice location (based on 2010 US Census data, confirmed with practice self-designation): intervention group, 7 rural (26.9%); 19 urban (73.1%); control group, 7 rural (26.9%); 19 urban (73.1%).
Table. Participant Characteristics.
| Characteristics | No./total (%) | |
|---|---|---|
| Patient-reported outcomes intervention (n = 593) | Usual care (n = 598) | |
| Age, median (range), y | 64 (29-89) | 62 (28-93) |
| Sex | ||
| Female | 359 (60.5) | 335/597 (56.1) |
| Male | 234 (39.5) | 262/597 (43.9) |
| Race (regardless of ethnicity) | ||
| American Indian or Alaska Native | 11/588 (1.9) | 13/576 (2.3) |
| Asian | 2/588 (0.3) | 16/576 (2.8) |
| Black or African American | 99/588 (16.8) | 94/576 (16.3) |
| Native Hawaiian or Pacific Islander | 2/588 (0.3) | 1/576 (0.2) |
| White | 473/588 (80.4) | 452/576 (78.5) |
| Multiple races reported | 1/588 (0.2) | 0/576 (0.0) |
| Hispanic ethnicity (regardless of race) | 14/591 (2.4) | 39/596 (6.5) |
| Weekly patient-reported outcome survey mode of administration (intervention group only) | ||
| Internet | 378 (63.7) | |
| Automated telephone | 215 (36.3) | |
| Education | ||
| First-eighth grade | 10/592 (1.7) | 14/596 (2.3) |
| Ninth-eleventh grade | 35/592 (5.9) | 49/596 (8.2) |
| High school graduate/GED | 173/592 (29.2) | 187/596 (31.4) |
| Some college, associate’s degree, or other certification | 218/592 (36.8) | 203/596 (34.1) |
| College degree | 91/592 (15.4) | 93/596 (15.6) |
| Advanced degree | 65/592 (11.0) | 50/596 (8.4) |
| Employment status | ||
| Full time (≥40 h/wk) | 94/592 (15.9) | 89/596 (14.9) |
| Part time | 72/592 (12.2) | 48/596 (8.1) |
| Not currently working | 426/592 (72.0) | 459/596 (77.0) |
| Rural practice locationa | 154 (26.0) | 163 (27.3) |
| Marital status | ||
| Married/partnered | 385 (64.9) | 349/597 (58.5) |
| Single, never married | 58 (9.8) | 75/597 (12.6) |
| Separated/divorced | 82 (13.8) | 110/597 (18.4) |
| Widowed | 68 (11.5) | 63/597 (10.6) |
| Technology use | ||
| Never use a computer, tablet, or smartphone | 62 (10.5) | 81/597 (13.6) |
| Never use the internet | 87 (14.7) | 114/597 (19.1) |
| Never use email | 114 (19.2) | 158/597 (26.5) |
| Difficulty paying monthly bills | ||
| Not at all | 260/592 (43.9) | 224/596 (37.6) |
| Not very | 106/592 (17.9) | 127/596 (21.3) |
| Somewhat | 161/592 (27.2) | 184/596 (30.9) |
| Very/extremely | 65/592 (11.0) | 61/596 (10.2) |
| Cancer type | ||
| Colorectal, anal | 100 (16.9) | 132 (22.1) |
| Thoracic (lung, thyroid, thymus) | 118 (19.9) | 110 (18.4) |
| Breast | 97 (16.4) | 80 (13.4) |
| Gynecologic (ovarian, cervix, uterine, vaginal) | 64 (10.8) | 53 (8.9) |
| Pancreas, hepatobiliary | 48 (8.1) | 49 (8.2) |
| Gastroesophageal, small bowel | 25 (4.2) | 38 (6.4) |
| Genitourinary nonprostate (bladder, kidney, testicular, penile) | 36 (6.1) | 26 (4.3) |
| Myeloma, lymphoma | 31 (5.2) | 31 (5.2) |
| Prostate | 33 (5.6) | 18 (3.0) |
| Melanoma, skin | 11 (1.9) | 21 (3.5) |
| Other (brain, sarcoma, other soft tissue, head/neck, unknown primary) | 30 (5.1) | 40 (6.7) |
GED, General Educational Development certificate.
Rural/urban practice location based on US Census data (County Rurality Census Table), confirmed with practice self-designation.
Secondary Quality-of-Life Outcomes
At 3 months, the mean change from baseline on the QLQ-C30 was significantly better for the PRO group than for the control group for physical function (PRO group, from 74.27 to 75.81 points; control group, from 73.54 to 72.61 points; mean difference, 2.47 [95% CI, 0.41-4.53] points; P = .02), symptom control (PRO group, from 77.67 to 80.03 points; control group, from 76.75 to 76.55 points; mean difference, 2.56 [95% CI, 0.95-4.17] points; P = .002), and HRQOL (PRO group, from 78.11 to 80.03 points; control group, from 77.00 to 76.50 points; mean difference, 2.43 [95% CI, 0.90-3.96] points; P = .002) (Figure 2; eTable 5 in Supplement 2).
Figure 2. Score Distribution and Model-Based Mean Change From Baseline at Each Assessment Time Point for Physical Function, Symptom Control, and Health-Related Quality of Life.
Scores on the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire. PRO indicates patient-reported outcome. Positive values represent improvement. In panels A-C, P values for model-based mean change from baseline at 1-, 3-, 6-, 9-, and 12-month assessments, respectively, are as follows: panel A (physical function), P = .21, P = .02, P = .003, P = .02, and P = .68; panel B (symptom control), P = .003, P = .002, P = .02, P = .045, and P = .32; and panel C (health-related quality of life), P = .003, P = .002, P = .006, P = .03, and P = .24. In panels D-F, circles indicate means; horizontal bars, medians; box tops and bottoms, IQRs; and whiskers, 1.5× the IQRs. See eTable 5 in Supplement 2 for point estimates and confidence intervals.
In the responder analysis, the proportion of patients with clinically meaningful (≥5-point) changes in outcomes at 3-month follow-up was significantly greater with PRO, compared with usual care, for the outcome of physical function, with 13.8% more patients in the PRO group experiencing physical function benefits (either a higher rate of improvement or a lower rate of decline) (odds ratio [OR], 1.35 [95% CI, 1.08-1.70]; P = .009), including 7.7% more with improvements (227/522 [43.5%] in the PRO group vs 195/544 [35.8%] in the control group), and 6.1% fewer with worsening (162/522 [31.0%] vs 202/544 [37.1%], respectively). There were 16.1% more patients in the PRO group with symptom control benefits compared with the control group (OR, 1.50 [95% CI, 1.15-1.95]; P = .003), including 8.6% more with improvements (195/517 [37.7%] vs 158/543 [29.1%], respectively) and 7.5% fewer with worsening (123/517 [23.8%] vs 170/543 [31.3%], respectively). There were 13.4% more patients in the PRO group with HRQOL benefits compared with the control group (OR, 1.41 [95% CI, 1.10-1.81]; P = .006), including 8.5% more patients with improvements (191/517 [36.9%] vs 154/543 [28.4%], respectively) and 4.9% fewer with worsening (127/517 [24.6%] vs 160/543 [29.5%], respectively) (eTable 6 and eFigure 4 in Supplement 2).
Analyses at Other Time Points and Subgroup Analyses
Mean differences remained statistically significant at months 6 and 9 but not at month 12 (Figure 2; eTable 5 in Supplement 2). Similarly, significant benefits in the proportion of patients with clinically meaningful changes persisted through month 9, but there was no statistically significant difference by month 12. At 12-month follow-up, 508 (42.7%) of 1191 patients were no longer participating in study follow-up (Figure 1; eTable 6 in Supplement 2). In prespecified subgroup analyses, no interaction tests were statistically significant (eTable 7 and eFigure 5 in Supplement 2).
Sensitivity Analysis
Results were statistically significant in a responder analysis of the proportion of patients with 10-point changes in outcomes at 3-month follow-up in the PRO group compared with usual care. There were 12.2% more PRO group patients who experienced physical function benefits compared with the control group (either a higher rate of improvement or a lower rate of decline) (OR, 1.43 [95% CI, 1.10-1.84]; P = .007), including 5.9% more patients with improvements (135/522 [25.9%] in the PRO group vs 109/544 [20.0%] in the control group) and 6.3% fewer patients with worsening (93/522 [17.8%] vs 131/544 [24.1%], respectively). There were 11.4% more patients who experienced symptom control benefits in the PRO group than in the control group (OR, 1.51 [95% CI, 1.14-2.00]; P = .004), including 4.2% more with improvements (118/517 [22.8%] vs 101/543 [18.6%], respectively) and 7.2% fewer with worsening (68/517 [13.2%] vs 111/543 [20.4%], respectively), and 13.1% more patients who experienced HRQOL benefits in the PRO group relative to the control group (OR, 1.69 [95% CI, 1.24-2.31]; P = .004), including 5.4% more with improvements (103/517 [19.9%] vs 79/543 [14.5%], respectively) and 7.7% fewer with worsening (62/517 [12.0%] vs 107/543 [19.7%], respectively) (eTable 6 and eFigure 6 in Supplement 2).
Exploratory Outcomes
Symptom Trajectories
In an analysis of individual symptom trajectories, significant mean differences in favor of the PRO group were observed at all time points for fatigue and at multiple time points for nausea, insomnia, appetite loss, and diarrhea but not for pain, dyspnea, or constipation (eTable 8 and eFigure 7 in Supplement 2).
Intervention Adherence and Feedback
Among PRO group patients, 20 565 (91.5%) of 22 486 expected weekly PRO surveys were completed, without substantial reduction in PRO completion rates over time (Figure 3 and eTable 9 in Supplement 2). “Backup” reminder calls by staff for patients who did not complete the electronic surveys when prompted were required for 3120 (13.9%) of 22 486 PRO surveys. A total of 18 973 (84.4%) surveys were completed by patients themselves, 846 (3.8%) by caregivers, and 606 (2.7%) by staff. A post hoc analysis found that among 20 565 completed surveys, there were 6979 (33.9%) alert notifications to nurses, most commonly for pain, diminished performance status, and diarrhea (eTable 10 in Supplement 2). Among these notifications, 4122 (59.1%) of 6979 resulted in immediate nursing actions, including telephone discussion with the patient or caregiver (n = 3097 [75.1%]), advice for self-management at home (n = 944 [22.9%]), and supportive medications prescribed or modified (n = 868 [21.1%]).
Figure 3. Proportions of Patients Completing Expected Patient-Reported Outcome Surveys, Patients Remaining in the Study, and Completed Surveys at Each Week of Participation.
The number of patients shown below the x-axis is the number of patients remaining in the study in that week.
In feedback from patients in the PRO group at 3-month follow-up, 387 (76.8%) of 504 noted that electronic symptom monitoring made them feel in greater control of their care, 365 (72.4%) of 504 that it improved discussions with the care team, 359 (91.6%) of 392 that it was relevant to their care, and 452 (89.5%) of 505 that they would recommend it to other patients. Among nurses after 6 months of participation, 49 (84.5%) of 58 indicated that symptom monitoring information improved the quality of discussions with patients, 49 (84.5%) of 58 that it increased efficiency of discussions, and 46 (79.3%) of 58 that it was helpful for electronic medical record documentation. While 44 (75.9%) of 58 nurses found alerts helpful, 28 (49.1%) of 57 felt that there were too many alerts.
Dropout rates were greater in the PRO group (41/593 [6.9%]) vs control (7/598 [1.2%]). The higher dropout rates in the PRO group were attributed to survey fatigue by 9 (1.5%) of 593 in the PRO group and 3 (0.5%) of 598 in the control group.
Discussion
Among patients receiving treatment at US community oncology practices, electronic symptom monitoring with PROs improved patient-reported physical function, symptom control, and HRQOL at 3 months based on small but statistically significant mean differences between groups and based on statistically significant differences in the proportions of patients meeting the criteria for clinically meaningful benefits. Although effect sizes were smaller than in a prior single-center trial, the statistically significant findings are consistent with the prior single-center trial10,11 and are consistent with the findings of other prior studies.8,9,10,11,12,13,14
Symptoms are common and debilitating for patients with cancer1,2 and are often unmeasured and unrecognized between visits.5,6,7 Identifying symptoms early via PROs and alerting clinicians to their presence facilitated interventions to prevent subsequent symptom worsening or complications.3 This trial also demonstrated that high rates of patient survey completion can be attained in routine clinical practice—even when patients are ill and have limited prior technology experience. The intervention in this trial used best practices for implementation and focused on usability, clinical follow-up, and integration of the intervention into existing clinical practice.18,19,32 In a prior analysis, there was no significant correlation between completion of PRO surveys and prior computer experience.18
Despite prior evidence of the benefits and feasibility of electronic symptom monitoring with PROs, there has not been widespread adoption in treatment for cancer.31 Implementation requires technology, patient engagement, staff effort, and modification of information flow. A prior analysis of user feedback from this trial found high levels of enthusiasm from patients and clinicians.18 However, nurses reported that alerts could be burdensome unless there is dedicated time to address the alerts. Despite cost savings associated with PRO monitoring for insurance companies,13,33,34 currently funding is not available to sustain additional costs to clinics of technology and staffing in the US. The Centers for Medicare & Medicaid Services has suggested including PROs in a proposed oncology payment model.35 Expansion of current billing codes for remote monitoring to include PROs in oncology would facilitate greater use of the intervention. Future work could delineate the relationship of PRO monitoring with patient navigation.36 Work is needed to refine algorithms for communicating alerts to assess whether the number of notifications can be safely reduced. Among patients randomized to PRO, 215 (36.3%) of 593 chose automated telephone rather than internet for completing surveys, a choice previously associated with older age and lower education.37 This suggests that noninternet interfaces may increase participation and equity.
This trial demonstrated the feasibility, clinical utility, and clinical benefits of administering questions from the National Cancer Institute’s PRO-CTCAE item library in routine clinical practice. The PRO-CTCAE library was initially validated in a routine care setting21 yet has largely been used in clinical research. This trial demonstrated that administering the PRO-CTCAE questions for symptom monitoring in routine therapy was beneficial and feasible for people with cancer.
However, the following should also be considered. First, the effect size was smaller than demonstrated in a prior single-center trial.10 Second, the effect size diminished over time, which may have reflected reduced benefits after a year of participation. Third, dropout rates were higher in the intervention group (6.9% vs 1.2%), suggesting that some participants may have found the intervention burdensome.
Limitations
This study has several limitations. First, the primary outcome of overall survival was not available at the time of this report. Second, benefits were not attained for all patients or all symptoms. This is likely related to heterogeneity of included patients. In the tests for interactions, some subgroups had small sample sizes, limiting statistical power to identify subgroups more likely to benefit from the intervention. Future trials can further identify populations most likely to benefit from the intervention. Third, surveys were only available in English, Spanish, and Mandarin. Fourth, there was no attention control, and it is possible that the additional attention received by the intervention group, rather than the intervention components, explained the benefits reported here. Fifth, MCIDs are not established for the differences in response rates between groups. Sixth, the trial was not blinded and the outcomes reported herein were patient reported. It is possible that awareness of group assignment contributed to differences between the 2 groups.
Conclusions
In this report of secondary outcomes from a randomized clinical trial of adults with metastatic cancer, use of weekly electronic PRO surveys to monitor symptoms, compared with usual care, resulted in statistically significant improvements in physical function, symptom control, and HRQOL at 3 months, with mean improvements of approximately 2.5 points on a 0- to 100-point scale. These findings should be interpreted provisionally pending results of the primary outcome of overall survival.
Trial Protocol and Statistical Analysis Plan
eTable 1. Patient-Reported Outcome (PRO) Weekly Survey Items and Alert Notification Criteria
eTable 2. Subgroup Analysis Variables
eTable 3. Roster of Participating Practices, Locations, Allocations, and Patient Accrual
eTable 4. Intracluster Correlation Coefficients for Each Patient-Reported Outcome Endpoint
eTable 5. Physical Function, Symptom Control, and Health-Related Quality of Life Mean Score Estimates and Differences in Mean Changes From Baseline Between Arms at Each Visit
eTable 6. Proportion of Patients in the Patient-Reported Outcome (PRO) Intervention Arm and Control Arm With 5-Point and 10-Point Changes in Physical Function, Symptom Control, and Health-Related Quality of Life
eTable 7. Subgroup Analysis of Physical Function
eTable 8. Individual Symptom Scale Mean Score Estimates and Differences Between Groups in Mean Change From Baseline at Each Measured Timepoint, From the EORTC QLQ-C30 Questionnaire
eTable 9. Completion Rates by Study Week
eTable 10. Alerts Triggered by the Weekly Survey System to the Care Team, by Symptom
eFigure 1. Example Patient-Level Educational Materials for Home Symptom Self-management
eFigure 2. Example Clinician-Level Educational Materials for Symptom Management
eFigure 3. Example Actual Clinician Report Showing Longitudinal Trajectory of Patient-Reported Outcomes (for Visualizing/Printing at Clinic Visit)
eFigure 4. Responder Analysis
eFigure 5. Preplanned Subgroup Analysis of Physical Function, as Measured by the EORTC QLQ-C30
eFigure 6. Responder Sensitivity Analysis
eFigure 7. Mean Changes From Baseline at Each Visit for Symptom Scales
Data Sharing Statement
References
- 1.Cleeland CS, Zhao F, Chang VT, et al. The symptom burden of cancer: evidence for a core set of cancer-related and treatment-related symptoms from the Eastern Cooperative Oncology Group Symptom Outcomes and Practice Patterns study. Cancer. 2013;119(24):4333-4340. doi: 10.1002/cncr.28376 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Reilly CM, Bruner DW, Mitchell SA, et al. A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment. Support Care Cancer. 2013;21(6):1525-1550. doi: 10.1007/s00520-012-1688-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Panattoni L, Fedorenko C, Greenwood-Hickman MA, et al. Characterizing potentially preventable cancer- and chronic disease-related emergency department use in the year after treatment initiation: a regional study. J Oncol Pract. 2018;14(3):e176-e185. doi: 10.1200/JOP.2017.028191 [DOI] [PubMed] [Google Scholar]
- 4.Henry DH, Viswanathan HN, Elkin EP, Traina S, Wade S, Cella D. Symptoms and treatment burden associated with cancer treatment: results from a cross-sectional national survey in the US. Support Care Cancer. 2008;16(7):791-801. doi: 10.1007/s00520-007-0380-2 [DOI] [PubMed] [Google Scholar]
- 5.Di Maio M, Gallo C, Leighl NB, et al. Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. J Clin Oncol. 2015;33(8):910-915. doi: 10.1200/JCO.2014.57.9334 [DOI] [PubMed] [Google Scholar]
- 6.Laugsand EA, Sprangers MAG, Bjordal K, Skorpen F, Kaasa S, Klepstad P. Health care providers underestimate symptom intensities of cancer patients: a multicenter European study. Health Qual Life Outcomes. 2010;8:104. doi: 10.1186/1477-7525-8-104 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Basch E, Iasonos A, McDonough T, et al. Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. Lancet Oncol. 2006;7(11):903-909. doi: 10.1016/S1470-2045(06)70910-X [DOI] [PubMed] [Google Scholar]
- 8.Warrington L, Absolom K, Conner M, et al. Electronic systems for patients to report and manage side effects of cancer treatment: systematic review. J Med Internet Res. 2019;21(1):e10875. doi: 10.2196/10875 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kotronoulas G, Kearney N, Maguire R, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? a systematic review of controlled trials. J Clin Oncol. 2014;32(14):1480-1501. doi: 10.1200/JCO.2013.53.5948 [DOI] [PubMed] [Google Scholar]
- 10.Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34(6):557-565. doi: 10.1200/JCO.2015.63.0830 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017;318(2):197-198. doi: 10.1001/jama.2017.7156 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Denis F, Basch E, Septans AL, et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA. 2019;321(3):306-307. doi: 10.1001/jama.2018.18085 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Barbera L, Sutradhar R, Seow H, et al. Impact of standardized Edmonton Symptom Assessment System use on emergency department visits and hospitalization: results of a population-based retrospective matched cohort analysis. JCO Oncol Pract. 2020;16(9):e958-e965. doi: 10.1200/JOP.19.00660 [DOI] [PubMed] [Google Scholar]
- 14.Mir O, Ferrua M, Fourcade A, et al. Digital remote monitoring plus usual care versus usual care in patients treated with oral anticancer agents: the randomized phase 3 CAPRI trial. Nat Med. Published online April 25, 2022. doi: 10.1038/s41591-022-01788-1 [DOI] [PubMed] [Google Scholar]
- 15.Schmidt T, Valuck T, Perkins B, et al. Improving patient-reported measures in oncology: a payer call to action. J Manag Care Spec Pharm. 2021;27(1):118-126. doi: 10.18553/jmcp.2020.20313 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Samuel CA, Schaal J, Robertson L, et al. Racial differences in symptom management experiences during breast cancer treatment. Support Care Cancer. 2018;26(5):1425-1435. doi: 10.1007/s00520-017-3965-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Patient-Reported Outcomes Core of the University of North Carolina. Accessed May 27, 2022. https://pro.unc.edu/
- 18.Basch E, Stover AM, Schrag D, et al. Clinical utility and user perceptions of a digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial. JCO Clin Cancer Inform. 2020;4:947-957. doi: 10.1200/CCI.20.00081 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Schoen MW, Basch E, Hudson LL, et al. Software for administering the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: usability study. JMIR Hum Factors. 2018;5(3):e10070. doi: 10.2196/10070 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Basch E, Barbera L, Kerrigan CL, Velikova G. Implementation of patient-reported outcomes in routine medical care. Am Soc Clin Oncol Educ Book. 2018;38:122-134. doi: 10.1200/EDBK_200383 [DOI] [PubMed] [Google Scholar]
- 21.National Cancer Institute . Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events. Accessed May 27, 2022. https://healthcaredelivery.cancer.gov/pro-ctcae
- 22.Dueck AC, Mendoza TR, Mitchell SA, et al. ; National Cancer Institute PRO-CTCAE Study Group . Validity and reliability of the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 2015;1(8):1051-1059. doi: 10.1001/jamaoncol.2015.2639 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Al-Rashdan A, Sutradhar R, Nazeri-Rad N, Yao C, Barbera L. Comparing the ability of physician-reported versus patient-reported performance status to predict survival in a population-based cohort of newly diagnosed cancer patients. 2021;33(7):476-482. Clin Oncol (R Coll Radiol). doi: 10.1016/j.clon.2021.01.008 [DOI] [PubMed] [Google Scholar]
- 24.de Souza JA, Yap BJ, Wroblewski K, et al. Measuring financial toxicity as a clinically relevant patient-reported outcome: the validation of the Comprehensive Score for Financial Toxicity (COST). Cancer. 2017;123(3):476-484. doi: 10.1002/cncr.30369 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365-376. doi: 10.1093/jnci/85.5.365 [DOI] [PubMed] [Google Scholar]
- 26.Fayers P, Bottomley A; EORTC Quality of Life Group; Quality of Life Unit; European Organisation for Research and Treatment of Cancer . Quality of life research within the EORTC—the EORTC QLQ-C30. Eur J Cancer. 2002;38(suppl 4):S125-S133. doi: 10.1016/S0959-8049(01)00448-8 [DOI] [PubMed] [Google Scholar]
- 27.Gundy CM, Fayers PM, Groenvold M, et al. Comparing higher order models for the EORTC QLQ-C30. Qual Life Res. 2012;21(9):1607-1617. doi: 10.1007/s11136-011-0082-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Fayers PM, Aaronson NK, Bjordal K, Groenvold M, Curran D, Bottomley A; EORTC Quality of Life Group . The EORTC QLQ-C30 Scoring Manual (3rd Edition). European Organisation for Research and Treatment of Cancer; 2001. [Google Scholar]
- 29.Cocks K, King MT, Velikova G, et al. Evidence-based guidelines for interpreting change scores for the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30. Eur J Cancer. 2012;48(11):1713-1721. doi: 10.1016/j.ejca.2012.02.059 [DOI] [PubMed] [Google Scholar]
- 30.Coens C, Pe M, Dueck AC, et al. ; Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints Data Consortium . International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: recommendations of the SISAQOL Consortium. Lancet Oncol. 2020;21(2):e83-e96. doi: 10.1016/S1470-2045(19)30790-9 [DOI] [PubMed] [Google Scholar]
- 31.Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol. 1998;16(1):139-144. doi: 10.1200/JCO.1998.16.1.139 [DOI] [PubMed] [Google Scholar]
- 32.Basch E, Abernethy AP. Encouraging clinicians to incorporate longitudinal patient-reported symptoms in routine clinical practice. J Oncol Pract. 2011;7(1):23-25. doi: 10.1200/JOP.2010.000186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lizée T, Basch E, Trémolières P, et al. Cost-effectiveness of web-based patient-reported outcome surveillance in patients with lung cancer. J Thorac Oncol. 2019;14(6):1012-1020. doi: 10.1016/j.jtho.2019.02.005 [DOI] [PubMed] [Google Scholar]
- 34.Carlotto A, Hogsett VL, Maiorini EM, Razulis JG, Sonis ST. The economic burden of toxicities associated with cancer treatment: review of the literature and analysis of nausea and vomiting, diarrhoea, oral mucositis and fatigue. Pharmacoeconomics. 2013;31(9):753-766. doi: 10.1007/s40273-013-0081-2 [DOI] [PubMed] [Google Scholar]
- 35.Centers for Medicare & Medicaid Services . Informal request for information on proposed Oncology Care First model. Accessed May 27, 2022. https://innovation.cms.gov/Files/x/ocf-informalrfi.pdf
- 36.National Academies of Sciences, Engineering, and Medicine . Establishing Effective Patient Navigation Programs in Oncology: Proceedings of a Workshop. National Academies Press; 2018. [PubMed]
- 37.Stover AM, Henson S, Jansen J, et al. Demographic and symptom differences in PRO-TECT trial (AFT-39) cancer patients electing to complete weekly home patient-reported outcome measures (PROMs) via an automated phone call vs. email: implications for implementing PROs into routine care. Qual Life Res. 2019;28(suppl 1):S1. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Trial Protocol and Statistical Analysis Plan
eTable 1. Patient-Reported Outcome (PRO) Weekly Survey Items and Alert Notification Criteria
eTable 2. Subgroup Analysis Variables
eTable 3. Roster of Participating Practices, Locations, Allocations, and Patient Accrual
eTable 4. Intracluster Correlation Coefficients for Each Patient-Reported Outcome Endpoint
eTable 5. Physical Function, Symptom Control, and Health-Related Quality of Life Mean Score Estimates and Differences in Mean Changes From Baseline Between Arms at Each Visit
eTable 6. Proportion of Patients in the Patient-Reported Outcome (PRO) Intervention Arm and Control Arm With 5-Point and 10-Point Changes in Physical Function, Symptom Control, and Health-Related Quality of Life
eTable 7. Subgroup Analysis of Physical Function
eTable 8. Individual Symptom Scale Mean Score Estimates and Differences Between Groups in Mean Change From Baseline at Each Measured Timepoint, From the EORTC QLQ-C30 Questionnaire
eTable 9. Completion Rates by Study Week
eTable 10. Alerts Triggered by the Weekly Survey System to the Care Team, by Symptom
eFigure 1. Example Patient-Level Educational Materials for Home Symptom Self-management
eFigure 2. Example Clinician-Level Educational Materials for Symptom Management
eFigure 3. Example Actual Clinician Report Showing Longitudinal Trajectory of Patient-Reported Outcomes (for Visualizing/Printing at Clinic Visit)
eFigure 4. Responder Analysis
eFigure 5. Preplanned Subgroup Analysis of Physical Function, as Measured by the EORTC QLQ-C30
eFigure 6. Responder Sensitivity Analysis
eFigure 7. Mean Changes From Baseline at Each Visit for Symptom Scales
Data Sharing Statement