Quality of life during electronic patient-reported outcome (ePRO) monitoring in thoracic surgery patients: a pilot randomized controlled trial

Abstract

Purpose

Patients undergoing thoracic surgery experience high symptom burden and negative impacts on postoperative health-related quality of life (HRQOL). Remote symptom monitoring using electronic patient-reported outcomes (ePROs) in other patient populations improves HRQOL but may increase workload for surgical providers. We evaluated the feasibility of ePRO use by thoracic surgery patients and providers as well as its impact on HRQOL.

Methods

In this single center randomized controlled trial, thoracic surgery patients were assigned to either ePROs with alerts to their providers for severe symptoms or ePROs for measurement only, for 90 days after hospital discharge. Primary outcomes were feasibility (measured by patient survey completion) and change in postoperative HRQOL, assessed using the EORTC QLQ-C30 and LC-13 instruments. We also assessed symptom burden, health care utilization during ePRO use, and provider management of alerts.

Results

A total of 113 patients planned for thoracic surgery were randomized to ePROs with alerts (n = 56) vs. ePRO measurement (n = 57) only. Of these, 99 participants were discharged from the hospital using postoperative ePROs. More surveys were completed in the alerting arm (61.9% vs. 52.8%, p<0.001, h=0.18). HRQOL at 2 months (social and role function, summary score) was improved in the ePRO with alerts arm, and emotional function at 12 months was improved in the ePRO measurement only arm.

Conclusion

Remote symptom monitoring using ePROs with alerts is feasible for thoracic surgery patients and providers. Short-term HRQOL may be improved with ePROs with alerts. The downstream impact of ePRO use on long-term HRQOL and other outcomes requires further study.

Trial registration

https://Clinicaltrials.gov NCT04342260.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-026-04215-x.

Plain English Summary

1. Why is this study needed?

   Thoracic surgery is needed to treat common lung conditions but causes pain, cough, and difficulty breathing. These symptoms may impact how well patients feel and function during their recovery.

2. What is the key problem/issue/question this manuscript addresses?

   Surveys about common symptoms after thoracic surgery may be able to detect high levels of pain and other concerning symptoms. However, completing and reviewing the surveys requires effort from patients and their doctors and nurses. The benefits of participating in these surveys are unknown for thoracic surgery patients.

3. What is the main point of your study?

   The main point of the study was to find out how often patients completed the symptom surveys and what providers did after reviewing the responses. Another point of this study was to evaluate how reviewing the response improved how patients felt and functioned during their recovery.

4. What are your main results and what do they mean?

   The symptom surveys were easy for thoracic surgery patients to complete and providers to review. Preliminary results suggest that provider review of alerts may improve short term patient quality of life.

Introduction

Health-related quality of life (HRQOL) and symptom burden after thoracic surgery are important postoperative outcomes, alongside traditional clinical outcomes, for both patients and surgeons [1, 2]. Patients undergoing thoracic surgery commonly have baseline symptoms due to chronic cardiopulmonary comorbidities [3]. In addition, the invasive nature of these procedures, results in acute postoperative pain, fatigue, and dyspnea, amongst other symptoms, and is associated with frequent postoperative complications, such as pneumonia [4, 5]. This symptom burden may persist for up to one year after surgery, leading to long-term deficits in HRQOL following thoracic surgery [6, 7].

Electronic patient-reported outcomes (ePROs) are digital systems that periodically collect symptom surveys from patients at home and deliver results to providers. ePROs may be used for patient-level remote symptom monitoring (RSM), assessing functional outcomes of treatment, and/or clinic-level benchmarking [8]. ePROs for RSM have been shown to improve HRQOL, readmissions, and survival in medical oncology populations [9–11], including in real-world settings [12, 13]. ePROs have also been trialed in surgical oncology patients with improvement in symptom burden and HRQOL [14–18] and avoidable emergency department (ED) visits [19]. However, outside of oncology, ePROs for RSM in surgical patient populations are not yet routine, due to the burden of completing repeated surveys for patients having recently undergone invasive procedures and a lack of integration with clinical workflow, amongst other barriers [20–22].

Therefore, we conducted a trial to examine the feasibility of ePROs for RSM among general thoracic surgery patients and providers. This trial additionally explored the impact of alerting providers for ePRO responses compared to measurement-only of ePROs on HRQOL and acute care utilization to inform ongoing work in ePRO implementation.

Methods

Study design and patients

This was a single-center, prospective, randomized, controlled trial evaluating the feasibility of ePROs for RSM in thoracic surgery (NCT04342260). The impact of ePROs for RSM on outcomes was also explored. Participants were recruited from the University of North Carolina (UNC) Multidisciplinary Thoracic Oncology Program (MTOP) outpatient clinic. All new patients presenting to the UNC MTOP surgical clinic were identified from clinic schedules and screened for surgical procedure, primary language (only English speaking), age (18 years or older), and ability to complete a web-based symptom survey. Patients were excluded if they were planned for an outpatient surgery or foregut surgery, had a cognitive disability prohibiting informed consent or ePRO participation, were pregnant, or incarcerated. Eligible patients were enrolled by a clinical research assistant in the clinic or remotely after the visit and provided informed consent before the conduct of any study activities. Patients who enrolled and were later deemed ineligible (e.g., did not undergo surgery as planned, failure to discharge home) were removed from the study and did not continue any study activities; these were considered screen failures, and any enrollment data collected were excluded from analyses.

Following enrollment, participants were randomly allocated in a 1:1 ratio to either the intervention (i.e., alerting) or control arm (i.e., measurement-only) using block randomization to ensure equal distribution of subjects across both arms over time. This trial did not include masking; therefore, to reduce predictability, we selected a larger block size of 20, in accordance with the overall sample size and anticipated accrual rate. Block size was not disclosed to the (CRA) responsible for enrollment and randomization procedures. We employed a third-party using a random number generator to create the sequence, print on cards, and place in sealed envelopes. The CRA opened the next available envelope immediately upon obtaining informed consent to learn the participant’s assignment.

All participants received standard post-discharge care in addition to completing ePRO-based symptom reporting. The web-based ePROs for RSM tool was developed based on prior studies [10, 23]. The symptom survey utilized items from the NCI Common Terminology Criteria for Adverse Events (PRO-CTCAE^®), designed to evaluate symptomatic toxicity in patients on cancer clinical trials [24]. PRO-CTCAE items (n = 24) representing 11 symptomatic adverse events were empirically selected based on clinical experience and literature review [25, 26] These included: frequency, interference, or severity of anxiety, chills, constipation, cough, dyspnea, limb edema, fatigue, insomnia, numbness, pain, and palpitations. Additionally, three questions were generated by the clinical team addressing fever, redness, and drainage at incision sites as these issues are specifically relevant to surgery and not addressed by PRO-CTCAE item bank. All items were reported using a 5-point Likert scale (0–4) with higher values indicating greater symptoms. Additionally, the Patient-Reported Outcomes Measurement Information System (PROMIS^®) Physical Function Short Form-4a (PF SF-4a) [27] was used to monitor physical function over time.

Enrollment and survey distribution procedures

After enrollment, the CRA provided participants with a brief orientation on symptom survey completion, which included guided completion of a baseline symptom survey. The CRA also collected self-reported participant characteristics, including demographics, frequency of technology use, and experience with financial difficulty. After the participant’s surgery, but prior to discharge from the hospital, the CRA re-approached and re-oriented the participant to the planned ePRO symptom surveys. On the day of discharge from the hospital (day 0), symptom surveys were initiated. Participants received daily surveys during the early postoperative phase (day 0 to 14), followed by weekly surveys during the late postoperative phase (week 3 to 13; Fig. 1). Participants received each survey invitation via automated email. No reminders were sent to participants during the daily monitoring period (aside from the daily survey invitation). During the weekly monitoring period, participants received an automated survey completion reminder via a second email after three days if the survey had not yet been not completed. The CRA then contacted participants by phone if two consecutive surveys were missed.

Fig. 1.

ePROs for RSM integration with the standard care perioperative pathway

No incentives were provided for participation in this study. Surveys were completed within a web-based environment hosted by UNC PRO Core [28]. In some cases, when reminder calls were made, participants may have completed symptom surveys by phone with the CRA.

The alerting arm

In the alerting arm, ePRO-based automated alerts were sent to the care team for concerning symptoms. Alerts were triggered when a participant responded to an item with a value of 3 or 4 (severe/very severe), or an increase of 2 or more points on the scale from the participant’s discharge day score. If a discharge day survey was not completed, scores of zero were used for the purposes of providing a reference point for “worsening” symptom alert generation. Alerts were delivered to the CRA automatically by PRO Core and then forwarded to providers via email by the study team and contained text informing the provider of which symptom(s) triggered the alert and the severity level. An example graphical display of symptom reports contained within alerts is provided in Fig. 2. For symptom report graphs, composite scoring of the symptom item attributes (frequency, severity, interference) was calculated automatically and displayed according to previously described methods [29]. Providers were instructed to manage and document alerts per their clinical protocols into the EMR. Responses were abstracted from the EMR onto a standardized form.

Fig. 2.

Example of graphical display of patient-reported cough severity over time provided in alerts to providers. Diamond corresponds to post-operative day 0, circles represent completed surveys, absence of a circle corresponding to a date represents a missed survey.

Measurement-only arm

In the measurement-only arm, no information was sent to the provider regarding symptom survey responses. Providers were informed of the option to request ePROs for RMS data for any patients on study. No requests were made during the study period.

Study endpoints

The primary endpoint of this study was the feasibility of ePRO survey completion. Feasibility was assessed by the proportion of completed ePRO symptom surveys out of the total delivered during the 3-month intervention period in both arms. An a priori analysis of individuals with 100% survey completion was planned. However, given the limited number of patients meeting this criteria we elected to utilize a survey completion of > 80% of delivered surveys to determine high compliance, 50–79% as mid compliance, and < 50% as low compliance [30]. Completion rates during early and late monitoring periods were examined by arm.

The impact on the change in HRQOL of alerting providers for concerning symptom survey results was explored. HRQOL was assessed at baseline (enrollment preoperatively), week 2, and months 2, 4, 6, and 12 using the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 [31] and QLQ-LC-13 [32, 33] questionnaire. The QLQ-C30 is a validated, cancer-specific instrument that includes 30 items covering five functional scales (physical, role, emotional, cognitive, and social), three symptom scales (fatigue, pain, nausea/vomiting), a global health status/QOL scale, a total/summary score [34], and several single-item symptom measures. Scale scores range from 0 to 100, with higher scores indicating better functioning for the functional and global health scales, and higher scores indicating worse symptoms for the symptom scales. The EORTC QLQ-LC13 is a lung cancer-specific module that complements the QLQ-C30 by measuring lung cancer-related symptoms such as cough, hemoptysis, dyspnea, and pain. The EORTC minimally clinical important difference (MCID) for lung cancer is 5 to 10 points [35]. Similarly, its scoring follows the same 0-100 scale, where higher scores on symptom items represent greater symptom burden. A survey completion window of 1 month for week 2, and 60 days for months 2 and 4 surveys were allowed. A completion window of 6 months for the 6 month and 12-month HRQOL endpoints were allowed.

Statistical analysis

The analysis plan included a modified intention-to-treat sample consisting of all eligible randomized participants who underwent surgery and initiated ePRO monitoring. The original design anticipated 140 patients (excluding screen failures) over approximately one year. Sample size calculations indicated that with 70 patients per group, the study would have 80% power to detect a clinically meaningful difference of 10 points in the EORTC QLQ-C30 summary score (common standard deviation 22.9), corresponding to an effect size of 0.437, using a two-group comparison at a 0.05 significance level.

This study was planned in 2019 and opened to accrual on April 30, 2020. Due to slower accrual related to the COVID-19 pandemic, the study period was extended, and the target sample size was revised to 100 participants (per IRB-approved protocol version dated December 3, 2021). The final patient enrolled in November 2022. With 100 patients, 50 in each group, the study retained adequate power to detect an effect size of 0.5 (difference in means of approximately 11.5 points, common standard deviation 22.9) for a two-group comparison of the EORTC QLQ scores. The primary analysis plan included linear mixed-effects modeling to leverage repeated measures within participants and accommodate missing data. Power calculations were performed using SAS version 9.4 (SAS Institute, Cary, NC) with PROC POWER.

Patient-level descriptive statistics were used to characterize demographic and clinical characteristics by ePRO compliance level, summarize the proportion of completed surveys at each time point, and describe symptom burden within each arm. As the ePRO was used only for post-discharge symptom monitoring, details of inpatient management (e.g. number of chest tubes, complication rates) were not adjusted in analysis as related symptoms were managed prior to discharge. Comparisons of between-arm survey completion rates were performed using three Fisher’s exact tests for different time intervals (i.e., first two weeks post discharge, week 3 to 13 post discharge, and all time points). For each HRQOL outcome over time, a linear mixed-effect model with random intercepts (clustered by each patient) and interaction between study arm and time (treated as discrete) were fit to the longitudinal data to investigate between-arm differences in mean HRQOL score changes from baseline. The overall statistical significance of the interaction between study arm and time was assessed using a nested-models approach, comparing the full model to a reduced model without study arm. The statistical significance of the between-arm difference at each assessment (Baseline, Week 2, Month 2, Month 4, Month 6, and Year 1) was also tested using pairwise contrasts within the overall model. Given the many outcome measures, and exploratory nature of this study, unadjusted p-values are reported.

Alert frequency by symptom type and over time are presented utilizing descriptive statistics. A simple comparison of means between alert-level symptom frequency and duration was conducted between arms using the Wilcoxon rank-sum test. An exploratory analysis of risk of alerting was conducted combining both arms and using a post hoc definition of frequent alerting. Frequent alerting was defined as 4 or more alert level symptom surveys in the early phase and 2 or more alert level surveys in the late phase. This was chosen as it represents the highest tertile of alert level symptoms. Chi-squared tests were performed to determine demographic features associated with frequent alert level symptom reports. Frequency of specific types of response to alerts in the alerting arm based on entries into the standardized form are presented. Healthcare utilization and survival outcomes were assessed using the Kaplan-Meier method.

Study procedures were approved by the University of North Carolina (UNC) IRB (19-2585).

Results

Demographics and enrollment

A total of 247 patients were identified as potentially eligible based on EMR screening and approached to assess for full eligibility. Of these, 113 met inclusion criteria and agreed to participate. There was no difference in age, gender, or race/ethnicity of patients who declined compared to patients who enrolled (Supplemental Table 1). After randomization, there were 14 screen failures, leaving 99 participants who initiated ePROs for RSM after discharge from the hospital (see CONSORT diagram, Fig. 3).

Fig. 3.

Consort diagram

Participant characteristics were generally balanced between arms (Table 1). Overall, the majority of participants were female (63%) and White (71%), and had a current or former smoking history (64%). The most common diagnosis was lung cancer (38%). Surgical procedures included lung resection (69%), biopsy (11%), chest wall repair (6%), diaphragm repair (6%), thymectomy (5%), or other procedures (3%). Most patients underwent minimally invasive surgery (76%). Most participants had at least a high school education (94%) and were married/partnered (65%). Only 30% of participants were employed, and 9% reported difficulty paying bills. Only 4% of participants reported never using a computer or email.

Table 1.

Participant characteristics at baseline by study arm

Characteristics	Alerting Arm (n = 50)a	Measurement-only Arm (n = 49)a	Total (N = 99)a
Female gender, n (%)	29 (58.0)	33 (67.3)	62 (62.6)
Age, mean (SD)	56.9 (14.2)	62.5 (13.3)	59.6 (14.0)
Race, n (%)
Black	11 (22.0)	9 (18.4)	20 (20.2)
White	34 (68.0)	36 (73.5)	70 (70.7)
Other	5 (10.0)	4 (8.2)	9 (9.1)
Hispanic, n (%)	2 (4.3)	0 (0.0)	2 (2.1)
Smoker, n (%)	29 (58.0)	34 (69.4)	63 (63.6)
BMI, mean (SD)	30.1 (7.3)	28.8 (6.7)	29.5 (7.0)
FEV1, mean (SD)	88.4 (20.4)	79.4 (23.8)	83.8 (22.5)
DLCO, mean (SD)	82.9 (21.6)	72.8 (21.7)	77.8 (22.1)
Married/Partnered, n/N (%)a	23/39 (59.0)	30/42 (71.4)	53/81 (65.4)
Education, n/N (%)a
College or more	17/38 (44.7)	13/40 (32.5)	30/78 (38.5)
Some college or Associate’s degree	11/38 (28.9)	14/40 (35.0)	25/78 (32.1)
High school	6/38 (15.8)	12/40 (30.0)	18/78 (23.1)
Less than high school	4/38 (10.5)	1/40 (2.5)	5/78 (6.4)
Employed, n/N (%)a	15/40 (37.5)	9/40 (22.5)	24/80 (30.0)
Difficulty paying bills, n/N (%)a	3/36 (8.3)	4/39 (10.3)	7/75 (9.3)
Frequency of technology use, n/N (%)a
Never use email	1/38 (2.6)	2/40 (5.0)	3/78 (3.8)
Never use computer	0/38 (0.0)	3/40 (7.5)	3/78 (3.8)
Never use internet	0/38 (0.0)	2/40 (5.0)	2/78 (2.6)
Lung cancer, n (%)	13 (26.5)	24 (49.0)	37 (37.8)
Other malignancy, n (%)	10 (20.0)	11 (22.4)	21 (21.2)
Other diagnosesb, n (%)	26 (52.0)	14 (28.6)	40 (40.4)
Procedure type, n (%)
Biopsy	7 (14.0)	4 (8.2)	11 (11.1)
Chest wall repair	3 (6.0)	3 (6.1)	6 (6.1)
Diaphragm repair	4 (8.0)	2 (4.1)	6 (6.1)
Lung resection	33 (66.0)	35 (71.4)	68 (68.7)
Thymectomy	2 (4.0)	3 (6.1)	5 (5.1)
Other	1 (2.0)	2 (4.1)	3 (3.0)
Procedure approach, n (%)
Minimally invasive surgery	40 (80.0)	35 (71.4)	75 (75.8)
Open surgery	10 (20.0)	14 (28.6)	24 (24.2)
Baseline Measures, mean (SD)a
EORTC QLQ- C30
Summary score	79.82 (16.73)	81.41 (15.04)	80.62 (15.82)
Global health status	68.54 (22.61)	67.28 (23.08)	67.9 (22.71)
Physical function	82.26 (24.67)	81.31 (20.96)	81.78 (22.71)
Role function	75.83 (32.68)	78.17 (32.4)	77.03 (32.36)
Social function	81.25 (24.8)	88.49 (18.95)	84.96 (22.16)
Fatigue	30.28 (26.63)	32.01 (23.94)	31.17 (25.15)
Pain	27.5 (27.88)	21.83 (31.56)	24.59 (29.77)
Dyspnea	29.17 (33.92)	28.57 (28.1)	28.86 (30.88)
Insomnia	34.17 (36.58)	30.08 (34)	32.1 (35.14)
EORTC QLQ – LC13
Pain in chest	16.67 (27.22)	7.14 (17.32)	11.79 (23.05)
Pain in arm or shoulder	16.67 (25.04)	15.87 (25.75)	16.26 (25.25)
Pain in other parts	29.17 (29.42)	26.19 (29.94)	27.64 (29.54)
Peripheral neuropathy	20 (28.04)	11.9 (23.07)	15.85 (25.78)
Dyspnea	21.94 (23.26)	23.68 (21.64)	22.83 (22.32)
Cough	28.33 (30.71)	29.37 (24.64)	28.86 (27.6)
PROMIS PF-SF4a	45.75 (9.52)	44.66 (10.71)	45.19 (10.1)

^aFor self-reported data, percentages are calculated using the number of participants with available data; denominators are indicated when they differ from the total sample size due to missing self-reported baseline information.

^bOther diagnoses = rib fracture, pectus excavatum, benign lung masses, pleural effusion, pneumothorax, pulmonary fibrosis, diaphragm paralysis, diaphragm hernia, thymic lesion, other conditions.

ePRO completion

Overall, 57.4% of ePRO surveys were completed, with more surveys completed in the alerting arm (61.9% vs. 52.8%, p < 0.001, Cohen’s h = 0.18). 89 patients completed at least 1 ePRO. There was no difference between arms in the proportion of patients with no surveys completed, with only 10 patients completing no surveys throughout the entire measurement period.

More than one-third (n = 37) of participants met our criteria for high ePRO survey completion rate compliance (i.e., > 80% completion) with no difference between arms. Smokers were less likely to have high compliance (p = 0.009), and married/partnered individuals were more likely to have high compliance (p = 0.006). Demographics by survey compliance are provided in Supplemental Table 2.

The median time participating in ePROs was 13 weeks. During the early phase of monitoring, which involved daily reporting, the median survey completion rate was 53% (IQR, 11.4%). During the late phase of monitoring with weekly reporting, the median completion rate increased to 65% (IQR, 5.7%). In the early phase, participants completed a median of 9 (of 14) surveys (IQR, 9.5), and in the late phase, a median of 9 (of 11) surveys completed were completed (IQR, 9). Survey completion was higher in the alerting arm across early (56.9% vs. 46.2%, P < 0.001, Cohen’s h = 0.21) and late (68.8% vs. 62.2%, p = 0.026, Cohen’s h = 0.14) postoperative monitoring periods (Fig. 4).

Fig. 4.

ePRO survey completion rates over time during the early postoperative period (days 0 to 14) and late postoperative period (weeks 3 to 13), by study arm

Quality of life endpoints

Total EORTC-QLQ-C30 summary scores were similar in both arms and did not differ in the model based means analysis. The mean summary scores were 79.82 (sd 16.73) in the alerting arm and 81.31 (15.04) in the measurement-only arm at baseline. Median summary scores were 83.5 (IQR 67.3,95.7) in the alerting arm and 83.6 (IQR 77, 90.9) in the measurement-only group at baseline. Over the course of the study, the changes in the mean EORTC summary score showed a clinically important decrease at week 2 in both arms (-4.68 alerting arm; -6.40 measurement-only arm) (Figs. 5). See supplemental table 3a and 4a for all EORTC QLQ C-30 and LC-13 differences in means for all time points. Supplemental tables 3b and 4b describe the medians and IQRs for the EORTC QLQ C-30 and LC-13 scores at all time points.

Fig. 5.

Model-based mean change from baseline in EORTC functional domains^a at each assessment time point. ^a For EORTC HRQOL Functional Domains, higher scores equate to better function. * Denotes domain with significant difference in at least one time point

Statistically significant between-arm differences were observed in several EORTC functional domains and symptom scales Fig. 6 when comparing QOL score changes from baseline using mixed models with random intercepts. Follow-up pairwise comparisons with Tukey’s adjustments were also performed using the ‘emmeans’ package in R. At week 2, mean change from baseline in the Social function domain was significantly better in the alerting arm compared to measurement-only (-12.09 vs. -21.85, mean difference 9.76, 95% CI 0.56, 18.97). At month 2, the QLQ-C30 Summary Score (4.46 vs. − 1.88, mean difference 6.33, 95% CI 0.81, 11.86), Role Function (5.74 vs. − 7.68, mean difference 13.42, 95% CI 1.09, 25.76), Appetite Loss (− 11.2 vs. 8.6, mean difference 19.8, 95% CI 9, 30.6; negative value indicates improvement), and Nausea and Vomiting (− 2.17 vs. 4.09, mean difference 6.27, 95% CI 0.85, 11.69) were significantly better in the alerting arm. At month 12, Emotional function was better in the measurement-only arm (11.54 vs. 2.41, mean difference 9.13, 95% CI 0.38, 17.89). For Dysphagia, better change scores were found in the alerting arm at both Week 2 (-3.5 vs. 3.7, mean difference 7.2, 95% CI  1.1, 13.3; negative value indicates improvement) and Month 12 (0.2 vs. 7.6, mean difference 7.5, 95% CI 0.9, 14.1).

Fig. 6.

Model-based mean change from baseline at each assessment time point in selected symptom scales. For EORTC Symptom scales and items, higher scores equate to worse symptoms. *Denotes domain with significant difference in at least one time point.

Alerts and management

Alerts were examined in the alerting arm participants. Pain was the most frequent symptom generating alerts in the alerting arm during the early and late phase of ePRO measurement, with 87% of participants having at least one alert for pain in the early phase and 43% generating an alert for pain in the late phase. Pain also had the longest duration of persistent alert level reports, with an average of 2.7 days during the early phase and 1.19 weeks in the late phase. Overall, 93.6% (n = 44) of participants who completed ePROs generated at least one alert during their monitoring period.

The highest percentage of alerts generated occurred on day 2, with 85% of surveys completed generating an alert (Fig. 7). Of note, all day 2 alerts were related to severe or very severe symptoms (i.e., “3” or “4”). Six patients who alerted on day 2 were missing the discharge day score, but because there were no alerts for worsening symptoms (i.e., increase from “0”/missing on discharge day to “2” on day 2), these did not contribute at all to the 85% rate of alerts on post-discharge day 2. Alerts then steeply declined and stabilized to approximately 30–55% of surveys generating alerts over the subsequent 12 days of the early measurement period. In the late phase, alerts were highest at week 3 with over 50% of surveys generating an alert. This declined steadily to approximately 30–35% of surveys generating alerts over the remaining 10 weeks.

Fig. 7.

Frequency of alerts generated for alerting arm participants

Nursing responses to alerts triggered in the alerting arm were captured in 202 surveys. One or more clinical action was taken for 62.8% of alerting surveys (n = 127/202). The most common response to an alert was prescription of a medication (n = 46). Next, over-the-counter medication was recommended in n = 21 surveys. No clinical action was taken in 75 of the alerting surveys. Nurses documented reasons for no action, including not being able to reach the patient and stable or repeated symptoms. A response of other indicated a variety of nursing actions including left voicemails and provided assistance with various medical supplies (Table 2).

Table 2.

Nursing actions in response to alerts from alerting arm (n = 202)

No action taken	75
Action taken a	127
Prescribed a medication	46
Recommended an over-the-counter medication	21
Offered reassurance for patient self-management	16
N/A: the symptom had resolved by time of contact	14
N/A: another clinician was already managing the symptoms	13
Recommended/facilitated making an MTOP clinic appointment	9
Escalation to acute care (e.g., referred to ED, hospital admission)	2
Coordinated care (e.g., ordered labs, specialist referral)	2
Other nursing action	46

^aMore than one response may have been selected for each individual alert response, but the same type of nurse response is counted only once per survey, regardless of how many times it is ordered for different symptom alerts within the same survey

The alert level symptoms were also examined in the measurement-only arm. There were no differences between arms in frequency or duration of alert level symptoms during the study period (supplemental Tables 7–10). Individuals with frequent alert level symptoms (i.e., more than 4 alerting days or surveys ) in the early postoperative period had a lower baseline PF SF-4a (42.7 vs. 49.1, p = 0.005). This relationship did not persist in the late phase of measurement. (supplemental Tables 11 and 12).

Clinical outcomes

No statistically significant differences in healthcare utilization outcomes could be detected between arms (ED visit, logrank p = 0.19; avoidable ED visit, logrank p = 0.27; Readmission logrank p = 0.62) (Table 3). Survival was not different between study arms (logrank p = 0.99) with 6 deaths over the 1-year period of which only 1 occurred during the measurement period (i.e., first 90 days). Notably, however, emergency department visits were, while not significantly different, consistently lower in the alerting arm than in the measurement-only arm (Fig. 8).

Table 3.

Healthcare utilization outcomes during study period by study arm

Category	Alerting arm (N = 50)	Measurement-only arm (N = 49)
ED visit
< = 30 days	4.0%	14.8%
< = 90 days	8.1%	19.3%
Readmission
< = 30 days	4.0%	10.7%
< = 90 days	12.2%	15.1%
Avoidable ED visit
< = 30 days	2.0%	10.6%
< = 90 days	6.1%	15.1%

Fig. 8.

Cumulative incidence plots of emergency department visits and readmissions post-discharge

Discussion

This single-center, prospective, randomized controlled trial demonstrates that implementing ePRO measurement in a thoracic surgery clinic at an academic public hospital is feasible. Overall, 37% of participants met our criteria for high compliance, completing more than 80% of their assigned surveys. Notably, provider alerts were associated with higher survey completion rates in the alerting arm compared to the measurement-only arm (62% vs. 53%), suggesting that real-time clinical engagement may enhance patient adherence. Future multicenter studies with larger sample sizes and prespecified time-by-treatment hypotheses will be informed by our results and be needed to confirm whether these observed differences in HRQOL represent sustained effects.

Survey completion was lower during the early monitoring period—when symptoms are typically most problematic—with participants completing only 60% of daily surveys (9 of 15 delivered) in the first two weeks after discharge. In contrast, completion rates improved to 82% during the later follow-up period (9 of 11 weekly surveys delivered over the subsequent 11 weeks). Our qualitative work has shown that the high frequency of surveys in the early period after surgery are difficult for patients to complete in part because of the burden of the symptoms themselves [21]. In response, we reduced the frequency of symptom surveys in the early postoperative period in subsequent trials, which has been associated with improved survey completion rates. These findings suggest that both the timing and frequency of ePRO survey delivery, as well as integration with provider alert systems, may significantly impact uptake and sustained use of ePROs in routine surgical care. The effect of these schedule modifications on clinical outcomes is currently under investigation.

We also examined the impact of alert generation on patient outcomes, as phone calls for alert management can be a burden to both providers and patients [21]. Several previously published studies have demonstrated that measurement only of ePROs may improve outcomes [36–38]. However, best practices would suggest that patients expect providers to act on information they report [39]. This is further supported by studies that have shown the mechanism of ePRO impact appears to be through increased connection between patients and providers [36, 38, 40]. In this trial, we found that alert generation in response to ePROs improved HRQOL in our participants in the first 2 months after surgery. Notably, the number of times medications were prescribed in the alerting arm were relatively low in the tracked nursing alert responses. These findings suggest that ePROs with alerts improve patient HRQOL and symptom burden, though the mechanism is still unknown.

HRQOL trajectories over 1 year did not differ overall between arms, however, this was a fairly small study and was only powered to detect large differences. A larger sample size is needed to detect small differences between arms. This may also be attributed to the relatively early return to baseline; as the majority of patients in both arms had returned to baseline by month 2. Future studies are designed to evaluate a shorter post-discharge timeline. Further, the limited difference in arms in our study over time may have resulted from comparison of alert generation to measurement without a usual care arm. It is possible that measurement-only of ePROs has an influence on patient-centered outcomes through mechanisms like increased awareness or self-efficacy. In keeping with this, previous work by Cleeland et al. evaluated the effect of alert generation in response to ePRO monitoring compared to measurement-only. Their study showed that individuals generating alerts experienced a higher magnitude and rate of decline of alert level symptoms in the first 4 weeks after surgery [15]. Similarly, Dai et al. showed lower symptom threshold events in ePRO alerting arm compared to measurement-only arm patients at 4 weeks. Other studies [41] demonstrating prolonged benefits of early ePRO monitoring [10] have been conducted in medical oncology patients who have undergone significantly different medical interventions (e.g., chemotherapy infusion cycles) from surgical patients and represent a distinctly different population. The absence of differences in HRQOL after 2 months in our patients is likely related to a shorter duration of symptom burden in surgical patients. Interestingly, long term follow up in Dai’s study showed persistent improved symptom threshold events at 1 year in alerting patients (though notably the number of symptom threshold events in both arms was every low) [41], suggesting there is a role for ePROs with alerts in supportive care for patients throughout their care trajectory.

Pain was the most common reason for alert level symptoms in our study. The majority of patients reported alert level pain during the early postoperative period in both arms. Patients in both arms experienced alert level pain for approximately 3 surveys in a row on average during the early post discharge period. Interestingly, patients in the alerting arm had no differences at any time point in pain scores as measured on either the QLQ-C30 or LC13. This may be attributed to other mechanisms through which the patient could have their pain, and other symptoms addressed such as calling or messaging the clinic.

While no significant difference in clinical utilization outcomes was detected in our small sample, other studies have suggested reduced avoidable ED visits in the first 90 days when utilizing ePROs [42]. In our study, while the difference was not statistically significant, at 90 days only 6.1% of the alerting arm had experienced an avoidable ED visit, while 15.1% of the measurement-only arm had been readmitted after a visit to the ED (Table 3). This suggests that ePRO measurement with alerts may help reduce healthcare costs in the postoperative period given the high expense of ED care.

This study is not without limitations. The primary limitation was the emergence of the COVID-19 pandemic at the onset of enrollment. Changes in postoperative care pathways, increased reliance on remote communication, and heightened patient concern regarding in-person visits may have affected engagement with ePRO surveys and responsiveness to provider alerts. These contextual factors should be considered when interpreting adherence patterns and may limit generalizability to post-pandemic settings. The COVID-19 pandemic significantly impaired recruitment, resulting in a reduced sample size. Although feasibility remained a primary objective, the study was therefore not fully powered to detect between-group differences in outcomes. Nevertheless, the generated data preliminary estimates of effect sizes (e.g. anticipated difference in post-baseline scores) and variability that can inform appropriate power calculations for future studies. The absence of a usual care arm also limits conclusions regarding the influence of ePRO measurement-only on patient outcomes.

Conclusion

The implementation of ePROs for remote symptom monitoring in postoperative thoracic surgery patients is feasible and well tolerated. Additionally, the generation of real-time alerts in response to symptom surveys is associated with improved HRQOL in the first two months following surgery and may contribute to a reduction in unnecessary healthcare utilization. Further work is needed to optimize implementation strategies to enhance the reach, adherence, and overall impact of ePRO-based interventions in surgical populations.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Author contributions

GNM, AB, and EB contributed to the study conception and design. Material preparation, data collection and analysis were performed by SS, GNM, AM, MW. The first draft of the manuscript was written by BW and CEC and all authors commented on versions of the manuscript. All authors read and approved the final manuscript.

Funding

G.N.M. is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL157765, the Thoracic Surgery Foundation research award, and the UNC surgery. Chase E. Cox is supported by the National Cancer Institute’s National Research Service Award sponsored by the Lineberger Comprehensive Cancer Center at the University of North Carolina (T32 CA116339).

Declarations

Competing interests

None.

Ethical approval

This study was conducted in accordance with the ethical standards of the University of North Carolina at Chapel Hill Institutional Review Board and with the principles of the Declaration of Helsinki. Ethical approval for the study protocol LCCC1945.

Consent to participate

Informed consent was obtained from all individual participants included in the study. Participants were provided with information about the purpose, procedures, potential risks, and benefits of the study, and their participation was voluntary. Participants had the right to withdraw from the study at any time without penalty.