Testing Symptom Severity Thresholds and Potential Alerts for Clinical Intervention in Patients With Cancer Undergoing Chemotherapy

Abstract

PURPOSE:

Symptom monitoring is attracting attention as a way to improve adherence to cancer therapy, reduce treatment-related toxicities, and possibly improve overall survival. How reporting thresholds affect symptom alert generation and clinical outcomes is poorly understood.

PATIENTS AND METHODS:

We analyzed data from 38 US health care institutions collected for the prospective Eastern Cooperative Oncology Group–American College of Radiology Imaging Network E2Z02 Symptom Outcomes and Practice Patterns study. Participants were outpatients receiving chemotherapy for breast (n = 642), colorectal (n = 486), or lung cancer (n = 340) who rated symptom severity using the MD Anderson Symptom Inventory at 2 assessment points 1 month apart. Percentages of patients with pain, dyspnea, fatigue, or distress at different thresholds (score of 4-7 on a 0-10 scale) were compared. The percentage of patients whose performance status had worsened at follow-up was used to estimate risk for missing clinically important symptom data by using higher severity thresholds

RESULTS:

At the guideline-recommended threshold of ≥ 4, suprathreshold rates were 60% for any of the 4 symptoms at the initial survey; performance status worsened at follow-up for 27% of all patients with any symptom rated ≥ 4 at the initiate survey. When the threshold was increased to ≥ 7, approximately half of patients (51%) with worsened performance status were not identified.

CONCLUSION:

The burden to clinicians from an alert threshold of ≥ 4 (per many current guidelines) would be substantial. However, setting higher alert thresholds may miss a large percentage of patients who need clinical intervention. These results may inform resource planning when implementing electronic symptom screening at an institutional or practice level.

INTRODUCTION

As patient-reported outcomes (PROs) become more integrated into clinical care for cancer, symptom screening is showing beneficial effects in terms of improved symptom management (and therefore improved treatment tolerability) and, in some cases, longer survival.¹ Many studies of effectiveness and practice change have been done in clinical trials,^2,3 but the clinical burden and practice impact of using severity thresholds to generate alerts have not been reported in the real world of patients with active cancer therapy.

Several symptom-management guidelines have been developed based on symptom ratings and typically include severity thresholds that can be used to generate alerts for clinical action at specified levels. Investigations have used anchor-based methods to describe grades of symptom severity and their interaction with other PROs (eg, functioning or quality of life) as the basis for establishing cut points between mild, moderate, and severe levels for various symptoms.^4,5 Findings from this kind of research have been used to establish clinical decision action steps based on severity.^6-9 Numerical rating scales are used in treatment guidelines from ASCO and the National Comprehensive Cancer Network, suggesting that symptoms such as pain,^6,8 fatigue,^7,9 and distress¹⁰ need clinical attention when they are rated ≥ 4 on a 0-10 scale.

Using 5 on a 0-10 scale as the moderate-severe threshold for symptoms, the Eastern Cooperative Oncology Group (ECOG)–American College of Radiology Imaging Network (ACRIN) E2Z02 Symptom Outcomes and Practice Patterns (SOAPP) study conducted between 2006 and 2008 found that the prevalence of moderate to severe fatigue was 37.4% for patients receiving cancer therapy.¹¹ A more recent (2007-2014) Canadian study reported similar prevalence for multiple symptoms in patients in the first year after cancer diagnosis.¹² A meta-analysis of studies published in 2006-2016 examining the prevalence of cancer pain in adults found that 32.4% (95% CI, 26.6% to 38.2%) of patients receiving anticancer treatment reported having moderate to severe pain (rated ≥ 5 on the 0-10 scale).¹³ High prevalence rates of symptoms requiring clinical attention suggest that PRO assessment in practice might create a burden for health care providers, especially when alerts are generated for further action, using guideline-recommended thresholds and intervention strategies. Addressing this alert burden will require defining symptom alert thresholds that balance feasibility with sensitivity. As thresholds are increased, opportunities to provide interventions will decrease, leading to missed opportunities to address meaningful symptom care. This may be particularly relevant for patients receiving systemic cancer treatment. Consequently, understanding to what extent increasing severity of alert thresholds would release any of the burden and would decrease opportunities to provide symptom interventions is essential to consider in planning for action steps dependent on symptom severity.

In the current analysis, we examined a large national database to investigate the percentages of alerts that would be generated for selected symptoms at various severity thresholds. We examined a new end point, deterioration in performance status at 1-month follow-up, and the effect of different thresholds on identifying patients at risk for deterioration in performance status (PS). Using multicenter data reported by outpatients with cancer in this study, we aimed to provide evidence for planning and implementing symptom monitoring in real-world oncology care.

PATIENTS AND METHODS

Patients

Participants were drawn from the ECOG-ACRIN E2Z02 SOAPP study, a large, multicenter, prospective study of patients with breast, colorectal, prostate, or lung cancer treated at 38 US health care institutions.¹⁴ Eligible patients were ≥ 18 years of age, willing to complete the follow-up survey, and judged by the study screener to have adequate cognitive function to complete study surveys and were recruited at any point of their oncologic care. Conducted from March 3, 2006, to May 19, 2008, the SOAPP study described the prevalence of, severity of, and functional interference caused by physical and psychological symptoms experienced over a 1-month period. Participants filled out the study questionnaires, MD Anderson Symptom Inventory (MDASI), and Brief Pain Inventory at outpatient visits. Local institutional review boards approved the protocol. Written informed consent was required from each participant before registration onto the study. For the current study, we selected the subset of SOAPP patients who were on active chemotherapy and had completed the initial symptom assessment.

Measures

The SOAPP study used the MDASI¹⁵ as its outcome measure of symptom severity. The MDASI asks patients to report their worst level of symptoms in the past 24 hours on a 0-10 scale (0 = no symptom, 10 = as bad as you can imagine). The MDASI was administered at the initial assessment and again at follow-up 1 month later. For the current study, we focused a priori on the MDASI items of pain, fatigue, and distress, which are the symptoms most frequently reported by patients with cancer^12,16,17 and for which management guidelines are available.^6,7,10 We also included dyspnea, which is often associated with emergency visits or rehospitalization.¹⁸

PS was measured at the initial assessment and the 1-month follow-up survey using the clinician-rated ECOG PS score. Before the clinician’s visit, patients’ ECOG PS was rated by research staff. If a patient’s ECOG PS score increased by 1 or greater from the initial assessment to follow-up, that patient was defined as having worsened ECOG PS. Demographic variables (age, sex, and race) and disease-related variables (cancer type, time to diagnosis, and disease stage) were also used in this analysis.

Statistical Analysis

Percentages of suprathreshold alerts that would be generated for targeted symptoms (pain, dyspnea, fatigue, and distress) at various test thresholds (4, 5, 6, or 7 on a 0-10 scale) are described. We calculated 95% Wilson CIs and used χ² tests to examine differences in percentages by cancer type. Using a logistic regression model, we estimated the association between reporting suprathreshold symptom at the initial survey and having worsened ECOG PS at the 1-month follow-up, with adjustment for age, sex, race (non-Hispanic white v other), cancer site, time to diagnosis (within 1 year v > 1 year), and disease status (metastasis v no metastasis). For patients who rated any of the 4 targeted symptoms as ≥ 4, ≥ 5, ≥ 6, or ≥ 7 at the initial survey, we examined the percentages and 95% CIs of those with worsened ECOG PS at the second MDASI assessment as a way of estimating the risk for missing patients with clinically important symptom data as the threshold increased.

All analyses were performed using SAS 9.4 (SAS Institute, Cary, NC). The 2-sided significance level was set at P = .05.

RESULTS

From the 3,106 patients in the SOAPP data set, we identified 1,587 who were being treated with chemotherapy and had completed an MDASI assessment at the first SOAPP survey (661 patients with breast cancer, 505 with colorectal cancer, 346 with lung cancer, and 75 with prostate cancer). Because the prostate cancer sample was small, we removed these patients from analysis, yielding a final sample of 1,512 patients. The missing data for individual items on the first MDASI assessment ranged from 0.6% to 2.2%. The follow-up MDASI survey was completed by 1,386 (91.7%) of 1,512 patients. The missing data for individual items at the second survey ranged from 0.0% to 1.4%. Fatigue was the most missed item at both the initial assessment (30 of 1,512 surveys; 2.0%) and the follow-up (19 of 1,386 surveys; 1.4%). For further analysis on the percentages of suprathreshold alerts, we only included patients with no missing ratings for pain, fatigue, dyspnea, and distress (1,468 patients for the first assessment and 1,356 patients for the follow-up). For analysis on the relationship between reporting suprathreshold symptom and ECOG PS worsening, patients with complete data for the MDASI and ECOG PS at both assessments were included (n = 1,151; Appendix Fig A1, online only).

Table 1 lists selected patient characteristics. Overall, 84.6% of patients (1,242 of 1,468 patients) were non-Hispanic white, 59.6% (875 of 1,468 patients) were in their first year after diagnosis, and 52.1% (765 of 1,468 patients) had metastases; 1,176 patients had ECOG PS ratings from both assessments. At the follow-up, ECOG PS scores had worsened for 23.8% of patients (280 of 1,176 patients).

TABLE 1.

Selected Characteristics

At the first assessment, 59.7% of patients rated ≥ 1 of the 4 targeted symptoms as ≥ 4 on the severity scale; 30.5% rated ≥ 1 symptoms as ≥ 7 (Table 2). Similar percentages were found at the follow-up (61.3% for threshold ≥ 4 and 30.5% for ≥ 7). Among all patients, more patients with lung cancer rated at least one symptom ≥ 4 (244 [71.2%] of 340 patients) than did patients with breast cancer (373 [58.1%] of 642) or colorectal cancer (261 [53.7%] of 486) at the first survey (Table 2). The percentages of patients by diagnosis who had 1, 2, 3, or 4 symptoms above the threshold of ≥ 4 and the percentages of patients reporting each targeted symptom at various severity thresholds are provided in the Data Supplement.

TABLE 2.

Percentages of Patients Reporting ≥ 1 Targeted Symptom at Various Severity Thresholds (potential alerts)

The existence of ECOG PS data did not differ between patients with or without any symptom rated ≥ 4 (77.7% v 83.6%, respectively; P = .36). Among 1,151 patients with ECOG PS data at both the initial assessment and the follow-up, 275 patients (23.9%) had ECOG PS worsening. A higher symptom level of targeted symptoms (mean of pain, fatigue, dyspnea, and distress) at baseline was related to an increased risk of ECOG PS deterioration (Data Supplement). More patients with any of the 4 target symptoms rated as ≥ 4 at the first survey experienced worsened ECOG PS, compared with those who did not report any target symptom ≥ 4 (26.7% v 19.8%, respectively; P = .007). The logistic regression model found that the odds ratio was 3.68 (95% CI, 2.10 to 6.45), with a C-statistic of 0.65 (95% CI, 0.57 to 0.72).

Among patients who rated any of the 4 targeted symptoms as ≥ 4 (n = 681), 26.7% (182 of 681 patients) had worsened ECOG PS (Fig 1). The rates of worsened ECOG PS were 22.9% for a threshold ≥ 5, 15.9% for ≥ 6, and 13.7% for ≥ 7. If the threshold were increased from the moderate level (≥ 4) to the severe level (≥ 7), 48.9% of patients (89 of 182 patients) with worsened ECOG PS would not be identified because an alert might not be generated at the first assessment. By cancer site, 48.4% of patients (44 of 91 patients) with breast cancer, 50.0% of patients (23 of 46 patients) with colorectal cancer, and 48.9% of patients (22 of 45 patients) with lung cancer would not have been identified at the first assessment.

Fig 1.

(A) Percentages of patients reporting any suprathreshold event at initial survey and worsened Eastern Cooperative Oncology Group (ECOG) performance status (PS) at 1-month follow-up assessment. (B) Numbers and percentages of patients with worsened ECOG PS by threshold.

DISCUSSION

Using a large, multicenter sample of patients with breast, colorectal, or lung cancer being treated with chemotherapy, we examined the percentages of suprathreshold alerts to clinicians that might be generated at various severity thresholds on a 0-10 scale for 4 symptoms (pain, fatigue, distress, and dyspnea). We determined the performance of these various thresholds based on symptom ratings made at an initial assessment. We also examined patients whose PS declined over a 1-month period. More than half of patients reported at least 1 symptom as ≥ 4, and approximately one third rated any symptom as ≥ 7. More than one fourth of patients rating any of the 4 symptoms as ≥ 4 experienced worsened physical functioning, measured by ECOG PS, during the 1-month investigation. Half of these patients would not have produced alerts if the alert threshold were to be increased from 4 (moderate or severe) to 7 (severe only).

Both disease-related and treatment-related symptoms are common among patients with cancer and can result in distress, decrements in physical function, unplanned hospitalizations, and treatment interruptions. Symptoms go undetected as frequently as half of the time as a result of typically unstructured discussions at clinic visits and lack of contact between visits.¹⁹ As a result, clinicians may miss opportunities to intervene and alleviate suffering of patients on active cancer treatment. It has been suggested that the optimal strategies for symptom control are those that are embedded within clinician-patient communications and therapeutic relationships, that empower patients to self-manage and coordinate their care, and that are routinely integrated into standard cancer care.²⁰

Routine symptom monitoring combined with alerting is attracting increasing attention as a way to manage treatment-related toxicities, improve cancer treatment adherence, and improve overall survival.^1,3 However, unlike symptom monitoring activities defined in a clinical trial, implementing this approach in routine oncology care will require more consideration and effort to change the workflow within the health care system.²¹ One of many reasons for reluctance to adopt such systems is concern that implementation will significantly increase the burden of clinical care. Our results showed that in patients receiving chemotherapy, alerts for any of the 4 targeted symptoms would have been generated by approximately half of all patients when the alert threshold was set at 4 on the 0-10 scale, as recommended in multiple symptom management guidelines.^6-10 This prevalence rate is consistent with that generated from large-scale data analyses using 4 on a 0-10 scale as the cut point¹² and trials with automated symptom alerts.^2,22 Results from an automated symptom monitoring trial showed that responding to those alerts via direct telephone required 2 hours per patient on average.²³ In the real world, this time-consuming process has understandably engendered concerns from the providers about changing routines and added work²⁴ and about system barriers such as lack of time and inadequate reimbursement for follow‐up.²²

Increasing the alert threshold for targeted symptoms would be a simple way to reduce the alert burden. In this analysis, we showed that when 7 (severe) instead of 4 (moderate to severe) is used as the alert threshold, the number of alerts that would be generated for our targeted symptoms decreased by one half. However, our data also show that reducing the number of alerts in this way risks missing half of the patients who will deteriorate (experience worsened ECOG PS) within a short period of time. Although patients with lung cancer generated more alert than those with breast or colorectal cancer, the probabilities of missing alerts were similar across all 3 cancer types when the threshold was increased. ECOG PS is commonly used in oncology practice to measure a patient’s functional capacity, with an emphasis on physical dimensions.²⁵ Because of its prognostic value for survival,²⁶ ECOG PS is often a key factor in determining whether patients are healthy enough to tolerate cancer therapy.²⁷ Thus, simply increasing the alert threshold would fail to identify patients who might be clinically deteriorating or who might be less tolerant to treatment (one of the ultimate goals of automated symptom monitoring) and would overlook the opportunity to manage symptoms before serious adverse events emerge, which could improve adherence to treatment.

Because higher thresholds would not capture a significant amount of important symptomatic information, the better alternative may be to introduce carefully thought out implementation plans and practical response options to ease the burden of symptom monitoring. Some solutions have been applied in clinical trials and have proven to be feasible and reliable. For instance, in the clinical trial by Basch et al,³ participants were informed that alerts would not generate an immediate response but to expect a response within 24 hours or during working hours; patients were encouraged to call the clinic during off hours whenever they were concerned about their symptoms. This approach provided flexibility for both clinicians and patients. Another option is to have a filtering mechanism in the symptom monitoring system with additional questions to triage alerts that do not require immediate action. In a study by Mooney et al,² symptom self-management instructions were provided to patients whenever an alert was generated, while a study-based research nurse group managed alerts by telephoning patients according to protocol guidance. However, symptom improvement obtained in those trials still largely relied on timely responses to alerts; further, part of the clinician burden was shared by study-based research nurses who are not available for real-world symptom monitoring. Without timely and proper clinical follow‐up, remote symptom monitoring may not improve patient outcomes.

For the purpose of triggering actions to prevent severe clinical consequences (eg, treatment interruption),¹ target monitoring symptoms are supposed to be able to inform clinical outcomes, and the recommended lists always include at least 10 items.^28-31 In the current study, we focused on 4 symptoms because they are commonly reported in patients with cancer¹⁶ and are included in guidelines with recommended thresholds for clinical intervention. However, it is probably more reasonable or practical to generate alerts for a subset of symptoms that need more immediate attention or that are actionable, such as rapidly increasing pain, diarrhea, vomiting, or dyspnea. Future studies should focus on minimizing the components of monitoring and establishing interventions directed at reducing major clinical concerns (eg, adverse events and emergency department visits). Furthermore, a triage algorithm on the association of each symptom to clinical outcomes would clarify the need for immediate attention.

This study had several limitations. First, the SOAPP data were collected between 2006 and 2008, when cancer treatment started changing rapidly. However, the percentages of patients reporting suprathreshold symptoms were consistent with those generated from data collected more recently,¹² suggesting that symptom burden experienced by patients during cancer treatment has not improved along with the increased cancer survival as a result of new therapies, such as targeted therapy and immunotherapy. Therefore, the current results are still interpretable for care of patients with cancer today. Second, patients in the SOAPP study completed MDASI assessments during a clinic visit. To inform symptom monitoring between clinical visits, analysis on data collected at home via remote assessment techniques is needed in the future. Third, we used declined functional status measured by ECOG PS as a surrogate for negative clinical consequences, which may not be able to adequately support the clinical action triggered by PRO alerting, when only approximately 25% of those with actionable symptom scores actually had worsened ECOG PS. Outcomes directly relating to symptom levels, such as treatment change or adverse events, need to be considered for the validation of current analysis. Furthermore, using less symptom burden and better quality of life as outcomes, we may be able to interpret the current results into a broader scope of patient care. Fourth, the SOAPP study was designed to investigate the prevalence of moderate to severe symptoms in patients with cancer. Thus, we could only define alert burden based on how many alerts would have been generated according to symptom ratings, without information on responses to those alerts and without addressing concerns from health care professionals who would be involved in routine PRO monitoring.

Although increasing the threshold for symptom intervention alerts would reduce the number of alerts generated and lessen clinical burden, our data suggest that doing so may miss clinically actionable information that is important for patient care. More research is required to support the cost-effective use of PRO monitoring in routine practice, including determining clinically meaningful alert thresholds, evaluating clinician and system burden, and implementing strategies to identify severity levels that need immediate action.

Appendix

Fig A1.

Diagram of patient selection (total patient set). ECOG PS, Eastern Cooperative Oncology Group performance status; MDASI, MD Anderson Symptom Inventory.

AUTHOR CONTRIBUTIONS

Conception and design: Qiuling Shi, Xin Shelley Wang, Charles S. Cleeland

Financial support: Qiuling Shi, Xin Shelley Wang

Administrative support: Qiuling Shi

Provision of study materials or patients: Qiuling Shi, Ju-Whei Lee

Collection and assembly of data: Qiuling Shi, Ju-Whei Lee

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Testing Symptom Severity Thresholds and Potential Alerts for Clinical Intervention in Patients With Cancer Undergoing Chemotherapy

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/op/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

Qiuling Shi

Research Funding: AstraZeneca (Inst), Merck (Inst)

Xin Shelley Wang

Consulting or Advisory Role: Phytogine

Patents, Royalties, Other Intellectual Property: Symptom Assessment Systems

Michael J. Fisch

Employment: AIM Specialty Health

Stock and Other Ownership Interests: Anthem

Consulting or Advisory Role: WellPoint

Patents, Royalties, Other Intellectual Property: Healthcore

Victor T. Chang

Research Funding: Incyte (Inst)

Lynne Wagner

Consulting or Advisory Role: Celgene

Travel, Accommodations, Expenses: Celgene

Charles S. Cleeland

Research Funding: Bayer (Inst), Genentech (Inst)

Patents, Royalties, Other Intellectual Property: Bayer License agreement to use the MD Anderson Symptom Inventory (Inst)

No other potential conflicts of interest were reported.