Real-world impact of patient-reported outcome measurement on overall survival, healthcare use and treatment discontinuation in cancer patients

Abstract

Aim:

The purpose of this retrospective, population-based, observational cohort analysis was to assess whether routine patient-reported outcomes (PRO) monitoring alone has an impact on real-world overall survival (OS) and hospitalizations among individuals diagnosed with lung, breast or colorectal cancer. The importance of follow-up care in post-PRO data collection was also discussed.

Patients & methods:

Administrative databases covering 17 cancer centers from Alberta, Canada were queried and individuals ≥18 years old and diagnosed with lung, breast or colorectal cancer from 1 January 2016 to 31 December 2019 were included and followed until 31 December 2020. Patients were stratified by whether they received routine PRO monitoring initiated within 120 days of diagnosis and matched 1:1 with use of propensity scores based on baseline characteristics. OS was assessed from the index date to death, and the respective Kaplan–Meier curves were estimated along with hazard ratios from Cox Proportional Hazard Models. Linear and logistic regression models were used to estimate mean differences and odds ratios (OR) respectively for healthcare resource utilization events including cancer physician visits, emergency department visits and outpatient ambulatory care encounters.

Results:

4800 patients were included in each matched cohort. There was no statistically significant difference between PRO monitoring and non-monitoring cohorts in OS (HR = 1.01; 95% CI: 0.93–1.09; p = 0.836) and treatment discontinuation (OR = 0.98; 95% CI: 0.85–1.12; p = 0.75). Median OS was 51.5 months for unmonitored cohort (95% CI: 47.5–NA) versus 50.6 months for monitored cohort (95% CI: 47.6–55.7). Compared with PRO-monitored patients, unmonitored patients were associated with lower hospitalization risks (OR = 1.12; 95% CI: 1.03–1.22; p = 0.01). However, PRO-monitored patients experienced significantly fewer physician visits in comparison to unmonitored patients (MD = -1.036; 95% CI: -1.288 to -0.784, p < 0.001).

Conclusion:

Our results show that capturing patient-reported symptoms alone reduced the number of physician visits but neither reduced hospitalizations nor improved OS in this real-world cancer population. To drive more meaningful clinical impact, PRO monitoring programs must be met with rigorous follow-up response to the identified symptoms.

Plain language summary

What is this article about?

Despite compelling evidence supporting their use, patient-reported outcomes (PROs) are not widely integrated into routine oncology care. This study aimed to assess whether routine PRO monitoring alone has an impact on real-world overall survival (OS) and hospitalizations among individuals diagnosed with lung, breast, or colorectal cancer.

What were the results?

Our results show that capturing patient-reported symptoms alone reduced the number of physician visits but neither reduced hospitalizations nor improved OS in this real-world cancer population.

What do the results mean?

Patient-reported outcome measures can supplement clinical outcomes and indicators for provincial and worldwide reporting, allowing healthcare systems to become more patient-centered and value-based, improving the quality of life of patients. Our results suggest that capturing patient-reported symptoms alone reduced the number of physician visits but neither reduced hospitalizations nor improved OS in this real-world cancer population. To drive more meaningful clinical impact, PRO monitoring programs must be connected closely to care in response to identified symptoms.

Tweetable abstract

Capturing patient-reported cancer symptoms alone reduced physician visits but neither reduced hospitalizations nor improved overall survival. To drive more meaningful clinical impact, PRO monitoring programs must be connected closely to care in response to identified symptoms.

The patient perspective has traditionally been viewed through the lens of the healthcare system and healthcare providers. However, in recent years, there has been an elevated commitment to placing patients at the center of clinical care and health research [1]. A patient-centered approach that captures first-hand patient experiences, needs and priorities is necessary to assess the effectiveness of the healthcare system in delivering high-quality care [2].

In oncology, integrating electronic patient-reported outcomes (PROs) into routine practice has been increasingly recognized as digital therapeutics to improve cancer outcomes in recent years [3]. Evidence from the current studies highlights the benefits of capturing PROs in clinical practice, particularly on improving overall survival (OS) in patients with cancer [4,5]. For example, a randomized controlled trial conducted by Basch et al. found that integrating PROs into the routine care of 776 patients with metastatic cancer yielded better survival outcomes compared with usual care. Median OS was 31.2 months (95% CI, 24.50–39.60) in the PRO group and 26.0 months (95% CI, 22.10–30.90) in the usual care group [6]. Similarly, using real-world data, Barbera et al. showed that monitoring symptoms among 128,893 ambulatory patients with cancer via the Edmonton Symptom Assessment System (ESAS) were significantly associated with decreased mortality (hazard ratio [HR]: 0.48; 95% CI, 0.47–0.49) and lower rates of both emergency department visits (relative rate [RR]: 0.92; 95% CI, 0.91–0.93) and hospitalizations (RR: 0.86; 95% CI, 0.85–0.87) [7,8].

Real-world evidence, however, is limited and largely from studies using the Canadian Ontario data. Moreover, several limitations could be highlighted in previously conducted studies that have reported a meaningful improvement in OS with PRO monitoring [6–8]. For example, these studies did not adequately account for immortal-time bias (i.e., error due to misclassification or exclusion of time intervals) [9]; included a significant number of patients that had missing or incomplete data; did not address the impact of treatment, among other limitations.

To address these limitations, we conducted a retrospective analysis of population-based data to investigate whether routine PRO monitoring alone has an impact on real-world OS, healthcare resource utilization, and treatment discontinuation among individuals diagnosed with lung, breast, or colorectal cancer of any stage in Alberta, Canada.

Additionally, the process of incorporating PRO monitoring into routine clinical practice is complicated [10] and varies by jurisdiction in Canada. While some province regions and health organizations have established systems for collecting and reporting PRO data, there is a scarcity of information and lack of guidelines on how follow-up care is handled between different provinces and how it should be managed post-collection. In this study, we further investigated the possible differences in the process of implementing PRO between Ontario and Alberta, Canada and discussed the importance of follow-up care in post-PRO data collection.

Patients & methods

Data sources & study design

We conducted a retrospective cohort study of adults ≥18 years old diagnosed with lung, breast (female only), or colorectal cancer between 2016 and 2019 in Alberta, Canada using linked administrative data. The details of this database have been previously described in Cuthbert et al. [11]. Briefly, individuals diagnosed with cancer were identified using the provincial cancer registry. Demographic and clinical characteristics as well as the date of death or last known contact with the healthcare system were abstracted from the registry. Treatment information was obtained from electronic medical records. Healthcare resource utilization and comorbidity was ascertained using information from the discharge abstract database, the national ambulatory care reporting system, and the practitioner claims database. Information on initiation of PRO measurement was obtained from the Putting Patients First (PPF) Survey database. The PPF survey is a paper-based questionnaire collected from ambulatory patients with cancer in the waiting room, data is then reviewed and documented by clinicians in the Electronic Medical Record (EMR) system. The PPF includes two standardized measures: the ESASr and the Canadian Problem Checklist (CPC). These measures collect information on symptoms and experiences of patients with cancer. The PPF questionnaire is completed in the waiting room, and PRO measures are then manually entered into the patient's EMR by the clinician [11]. Details pertaining to the construct, questionnaire, and validity of the PPF survey have been previously described in Cuthbert et al. [11].

Index date & follow-up period

The index date was defined at 120 days after the initial diagnosis. Individuals were followed until death, last known contact with the healthcare system, or 31 December 2020, whichever occurred first.

Exposure & outcome definition

Exposed patients were classified as those who completed at least one PRO assessment within 120 days of the cancer diagnosis (120-day landmark time), whereas unexposed patients did not complete a PRO questionnaire prior to the 120-day landmark time. We selected the 120-day cut-off point as this was judged to be a plausible time point at which a patient would begin PRO monitoring in real-world clinical practice. To account for immortal-time bias, a landmark analysis was conducted whereby individuals were excluded if they had died or were lost to follow-up prior to 120 days post-diagnosis [12].

The following outcomes were examined: OS defined as the time from the index date until death from any cause; time to next treatment (TTNT) defined as the time from the index date until initiation of a subsequent line of systemic therapy or death, whichever occurred first; healthcare resource utilization (HCRU) defined as the total number of events within the first 12-months of the index date with respect to hospitalizations, ambulatory care encounters, and physician visits; hospitalization within 12-months of the index date (yes/no); emergency room (ER) visit within 12-months of the index date (yes/no); outpatient ambulatory care encounters within 12-months of the index date (total number); cancer physician visits within 12-months of the index date (total number); and treatment discontinuation defined as completion of less than six-months of systemic therapy (yes/no). The TTNT and time to treatment discontinuation analyses were restricted to individuals who initiated systemic therapy prior to the landmark time. Individuals who initiated a subsequent line of therapy or who discontinued therapy prior to the index were also excluded from the TTNT and time to treatment discontinuation analyses, respectively, per the landmark analysis.

Covariates

The following covariates were controlled for in the analyses: age at initial diagnosis (years), sex (male/female), year of diagnosis (2016/2017/2018/2019), number of Charlson comorbidities assessed within the year prior to diagnosis (0/1/2+; assessed using the algorithm described in Quan et al. [13], location of residence at diagnosis (urban/rural), neighborhood household income (dollars), proportion of residents in neighborhoods who achieved a high school degree or higher levels of educations (%), cancer type (breast/colorectal/lung), and metastatic disease status at initial diagnosis (yes/no). We also adjusted for whether or not the patient had received systemic therapy, radiation, or surgery (yes/no) as well as the number of hospitalizations or ER visits (0/1/2/3+) between the time of diagnosis and the index date. We further adjusted for cancer subtypes specific to breast (HER2-/HER2+/triple-negative), colorectal (left colon/right colon/rectal), and lung (squamous/non-squamous) diagnoses. Individuals missing information on one or more confounders were excluded from the analyses.

Propensity score matching & statistical analysis

We estimated the intention-to-treat effect of initiating PRO monitoring within 120 days of diagnosis compared with not initiating PRO monitoring within 120 days. Patients who initiated PRO monitoring were propensity-score matched 1:1 using a nearest-neighbor algorithm. This approach estimated the effect among the exposed who were successfully matched. Matching was conducted using the MatchIt package in R (4.1.3). In accordance with best practice guidelines, a caliper width equal to 0.2 standard deviations of the logit of the propensity score was used to match individuals [14]. The propensity scores were estimated using a logistic regression model. Linearity was assumed with respect to the continuous covariates. To account for non-overlap of the sex, cancer type and cancer subtype covariates, a single categorical variable was generated representing all possible combinations of these three variables.

Baseline characteristics of matched exposed and unexposed patients were compared using absolute standardized differences (ASD) before and after matching. The ASD was estimated using the CreateTableOne package in R (4.1.3). ASDs larger than 0.1 were judged to represent a meaningful imbalance in the given characteristic between the exposed and unexposed groups [15]. The propensity score matching algorithm was judged to be successful if all ASDs were less than 0.1. Cox proportional hazards models were used to estimate the HR for OS and TTNT outcomes. Linear regression was used to estimate the mean difference (MD) in the number of HCRU events, cancer physician visits, and outpatient ambulatory care encounters. Logistic regression was used to estimate the odds ratio (OR) for the hospitalization, ER visit and treatment discontinuation outcomes. To account for matching, robust variance estimation was used to construct the 95% confidence intervals [16].

Additional analyses

Analyses were repeated, including the propensity score matching algorithm and assessment of ASD pre- and post-matching, within strata defined by cancer type (breast/colorectal/lung) and among those with metastatic disease. In addition, the analyses were repeated using a landmark time of 365 days instead of the original 120 days.

Results

An initial 19,509 individuals were included in the analysis, of which 5950 (30%) initiated a PRO measure within 120 days. Prior to matching, there were notable imbalances with respect to the year of diagnosis (ASD: 0.72), metastatic disease at initial diagnosis (ASD: 0.22), cancer type/subtype/sex (ASD: 0.25), as well as initiation of systemic therapy (ASD: 0.67), radiation therapy (ASD: 0.31) and surgery (ASD: 0.20) (Supplementary Table 1). A total of 4800 out of 5950 (81%) individuals who initiated PRO collection were successfully matched versus 4800 out of 13,559 (35%) individuals who did not initiate PRO collection (total successfully matched: 9600/19,509 (49%). After matching, the ASD for all baseline characteristics was <0.1 (Supplementary Table 2). In the matched cohort, the mean age at diagnosis was 63 years, 24% were male, and 22% had metastatic disease at initial diagnosis. In total, 54% had breast cancer, 22% had colorectal cancer, and 24% had lung cancer (Table 1).

Table 1. . Baseline characteristics of matched patients (n = 9600).

Strata	n (%)
Age at diagnosis, years
Mean (SD)	63 (13)
Sex
Male	2,304 (24%)
Female	7,296 (76%)
Primary cancer site
Breast	5,184 (54%)
Lung	2,304 (24%)
Colorectal	2,112 (22%)
Metastatic disease status
Yes	2,112 (22%)
No	7,488 (78%)
Initiated therapy prior to the index date
Systemic therapy	6,336 (66%)
Surgery	6,432 (67%)

SD: Standard deviation.

The majority of individuals initiated systemic therapy (66%) or surgery (67%) prior to the index date. After propensity score matching, PRO monitoring was not associated with OS (HR = 1.01; 95% CI: 0.93–1.09; p = 0.83), TTNT (HR = 1.03; 95% CI: 0.94–1.11; p = 0.56), HCRU (MD = 0.29 encounters; 95% CI: -0.20 to 0.78; p = 0.25), or treatment discontinuation (OR = 0.98; 95% CI: 0.85–1.12; p = 0.75) (Figure 1, Table 2). PRO monitoring was significantly associated with fewer cancer physician visits (MD = -1.04 visits; 95% CI: -1.29 to -0.78; p < 0.01) but greater outpatient ambulatory care encounters (MD = 1.12; 95% CI: 0.77–1.46; p < 0.01) and higher odds of being hospitalized (OR = 1.12; 95% CI: 1.03–1.22; p = 0.01) or having an ER visit (OR: 1.10; 95% CI: 1.01–1.19; p = 0.02).

Figure 1. . Kaplan–Meier survival curve for PRO-monitored and -unmonitored patients.

PRO: Patient-reported outcome.

Table 2. . Examining the association between PRO collection and outcomes of interest among matched patients.

Strata	Outcome	Measure of effect	Estimate	Lower 95% CI	Upper 95% CI	p-value
Overall	OS	HR	1.01	0.93	1.09	0.836
	TTNT	HR	1.03	0.94	1.11	0.56
	Total HCRU	MD	0.29	-0.20	0.78	0.248
	Treatment discontinuation	OR	0.98	0.85	1.12	0.75
	Cancer physician visits	MD	-1.04	-1.29	-0.78	<0.001
	Outpatient ambulatory care encounters	MD	1.12	0.77	1.46	<0.001
	Hospitalization	OR	1.12	1.03	1.22	0.01
	ER Visits	OR	1.10	1.01	1.19	0.024

ER: Emergency room; HCRU: Healthcare resource utilization; HR: Hazard ratio; MD: Mean difference; OR: Odds ratio; OS: Overall survival; TTNT: Time to next treatment.

Additional analysis results

In sensitivity analysis, the outcome results were generally similar when stratifying by cancer type or when restricting to individuals with metastatic disease (Supplementary Table 3). Exceptions to this rule included a lack of association with respect to cancer physician visits (p = 0.35) but an increased mean number of total HCRU events with PRO monitoring (MD = 1.18 events; 95% CI: 0.04 to 2.32; p = 0.04) among colorectal patients with cancer; a lack of association with respect to ER visits for patients with breast cancer (p = 0.82); and a lack of association with respect to hospitalization (p = 0.64) or outpatient ambulatory care encounters (p = 0.19) for patients with lung cancer (Supplementary Table 2). Similarly, changing the landmark time to 365 days did not meaningfully alter the findings (data not shown).

Discussion

We conducted a retrospective cohort study to investigate the associations between patient-reported symptoms measured within 120 days of diagnosis and health outcomes, including OS, HCRU and treatment discontinuation among adult patients diagnosed with breast, colorectal and lung cancer in a real-world Canadian setting. We found that capturing patient symptoms alone was not associated with improved survival of patients with breast, lung and colorectal cancer. While PRO-monitored patients experienced significantly fewer cancer physician visits, they had higher odds of being hospitalized or admitted to the ER within the first year of follow-up. Despite the statistical significance of these findings, the magnitude of the effect was small and, therefore, may not be clinically significant.

There are several potential reasons why the findings from this investigation contrast with those from prior studies demonstrating a meaningful improvement in OS with PRO monitoring in both clinical trials and real-world settings [6–8]. The primary reason may be a lack of or delayed clinical care in response to collected patient-reported symptoms. In prior studies, patient symptom monitoring was reported to the care team and clinical response was promptly triggered when severe or worsening symptoms were noted. This earlier intervention presumably helped avoid a worse disease state [17]. In the clinical trial study conducted by Basch et al. [3], when the PRO group participants reported a severe or worsening symptom, an email alert was triggered to a clinical nurse responsible for the care of that patient. Nurses responded to symptom alerts 77% of the time and provided clinical interventions, such as symptom management counseling via phone calls, medication modifications and referrals. Similarly, patients enrolled in the real-world Ontario PRO study [7,8], used the Revised ESAS and Patient Reported Functional Status questionnaires to report their symptoms via an onsite computer touch screen or a web-based application. Before receiving care, patients receive a printed copy of their score to present to their healthcare provider. Further, an onsite computer-generated alert will notify providers if a low score is detected, this enables healthcare providers to respond to symptom distress in a timely manner. The early identification of the symptoms and the subsequent follow-up management from the clinical team may have explained the survival benefits associated with ESAS exposure.

Integration of PROs into routine clinical practice is complex [10]. In Alberta, the PRO program was introduced in a stepwise fashion in 2016 so it is still in its infancy of development. Further, clinicians must manually enter reported symptoms and log into a separate dashboard in order to access current and historical PRO data measures which may be too cumbersome and time-consuming for a clinician in a busy real-world clinical practice. More importantly, the process of how follow-up care is planned and executed after the collection of PRO measures remains unclear. No detailed practice or management guidelines were developed with the system and implementation studies were not completed prior to widespread implementation. Our results highlight the challenges in scaling up a PRO alert system in the real-world while integrating with routine clinical processes. In contrast, patients in Ontario have their PRO data printed off and physically presented to the clinician at each visit [7,8]. Therefore, differences in the implementation of PRO monitoring with respect to the clinicians' ability to access PRO information may have impacted the timely clinical response to the identified symptoms and accounted for the disparity in these results [18,19]. As Alberta prepares for the new EMR system, new data collection methods are set to improve the functionality and usability of collected PRO measures. For example, patients will be able to log their PRO measures into their own EMR directly without any clinician involvement. Patients will also be able to report their symptoms a few days prior to their appointment [19]. We speculate that these changes will deliver symptom management frameworks that will set to improve patient-centered care across the province.

Additionally, our study differed from the prior studies in definitions employed to measure PRO exposure and the study population of interest. In our study, we examined the effect of initiating PRO monitoring within 120 days of diagnosis. In contrast, Barbera et al. defined exposed patients as those completing at least one ESAS assessment at any time point post-diagnosis. Their investigation reported a mean time to initiation of PRO monitoring of 1.1 years since diagnosis (SD: 1.6 years). Within the trial conducted by Basch et al., the mean time to enrollment was 46 days since initiation of chemotherapy (range: 0 to 1025 days) [6]. Differences with respect to the timing of when PRO monitoring is initiated and in which specific disease/treatment state (e.g., at initial diagnosis vs recurrence; before vs after initiating treatment) may further explain the differences in the study results. By accounting for immortal-time bias we were able to measure the impact of initiating PRO monitoring more accurately on OS within 120 days of diagnosis. By excluding individuals who died or were lost to follow-up prior to this landmark point, we minimized the potential for bias that could have resulted in an overestimation of the effect of PRO monitoring on OS.

Concerning the study population, we focused our investigation on individuals diagnosed with breast, colorectal and lung cancer of any stage, regardless of treatment. While Barbera et al., similarly, included all stages and made no restrictions on the receipt of therapy, they included over 14 different types of cancer. In contrast, Basch et al. were restricted to individuals with metastatic genitourinary, gynecologic, breast, or lung cancer and who planned to receive chemotherapy. While we did not observe any meaningful differences in our analyses when stratifying by cancer type or restricting to those with metastatic disease, significant differences may exist in response to PRO monitoring across cancer type, stage, and treatment setting, which may further help to explain the result differences.

Our study has limitations. First, by using the 120-day post-index landmark time, we excluded sicker patients who died within 120 days of cancer diagnosis. Second, a considerable proportion of individuals were not successfully matched in our analysis. This makes the estimate difficult to interpret and further limits generalizability since the estimates would correspond to the effect in the PRO measured individuals who were successfully matched rather than the effect on the entire population. Third, the lack of detailed information on PRO-related follow-up care in routine clinical practice in Alberta limited our ability to fully scrutinize the root causes behind the discrepancy in our results. Without a clear understanding of the model of care behind the use of PROs, it is difficult to assess the effectiveness of PRO monitoring in improving patient outcomes. Lastly, we conducted a propensity score-matched analysis with inherent limitations regarding its ability to address certain biases. We chose to use propensity score matching because it was similarly used in Barbara et al.; however, since the initiation of PRO monitoring within 120 days is a time-varying exposure, there exists time-varying confounding which can only be controlled for using more sophisticated g-methods and not the more traditional adjustment methods such as propensity score matching [20,21]. In addition, our reliance on the propensity-score matching method would have led to the misclassification of person-time at risk between exposure groups since individuals could have data consistent with both exposure categories prior to the initiation of PRO monitoring at 120 days post-diagnosis. More sophisticated methods are also required to properly account for this misclassification of person-time at risk [22,23].

With respect to the null findings, future research could help identify the extent to which clinicians in Alberta are leveraging the PRO monitoring system and identify potential barriers to utilizing such data in the clinic. In addition, researchers could explore the estimation of the per-protocol effect of adhering to different PRO monitoring strategies, including the assessment of whether or not there was an appropriate referral once an issue has been reported or the specific frequency and duration of PRO monitoring needed to observe a meaningful benefit [20].

In summary, while we believe in PRO monitoring as an important aspect of patient-centered care and shared decision making, its effectiveness in improving clinical outcomes is highly dependent on the quality of follow-up care that is implemented. This includes prompt clinical response to PRO measures and proactive interventions to address the underlying causes of those symptoms. As such, healthcare providers must prioritize the development and implementation of evidence-based care protocols that integrate PRO follow-up care guidelines into routine clinical practice.

Conclusion

Patient-reported outcome measures can supplement clinical outcomes and indicators for provincial and worldwide reporting, allowing healthcare systems to become more patient-centered and value-based, improving the quality of life of patients. Our results suggest that capturing patient-reported symptoms alone reduced the number of physician visits but neither reduced hospitalizations nor improved OS in this real-world cancer population. To drive more meaningful clinical impact, PRO monitoring programs must be connected closely to care in response to identified symptoms. Future studies should investigate the challenges of implementing PRO programs in the real-world setting.

Summary points

• This study used real-world population-level, retrospective administrative data from Alberta, Canada to assess the impact that patient-reported outcome measures have on overall survival (OS) and other clinical outcomes among patients with lung, breast and colorectal cancer.

• Our results suggest that capturing patient-reported symptoms alone reduced the number of physician visits but neither reduced hospitalizations nor improved OS in this real-world cancer population.

• To drive more meaningful clinical impact, patient-reported outcome monitoring programs must be connected closely to care in response to identified symptoms.