Do High Symptom Scores Trigger Clinical Actions? An Audit After Implementing Electronic Symptom Screening

The authors examine whether patient visits with higher symptom scores are associated with higher rates of symptom documentation in the chart and symptom-specific actions being taken.

Abstract

Purpose:

Standardized, electronic, symptom assessment is purported to help identify symptom needs. However, little research examines clinical processes related to symptom management, such as whether patients with worsening symptoms receive clinical actions more often. This study examined whether patient visits with higher symptom scores are associated with higher rates of symptom documentation in the chart and symptom-specific actions being taken.

Methods:

Retrospective chart reviews on cancer patient visits at a regional cancer center. An electronic Edmonton Symptom Assessment Scale (ESAS), a validated tool to measure symptoms, was implemented center-wide to standardize symptom screening at every patient visit. The independent variable was ESAS scores for pain and shortness of breath, categorized by severity: 0 (none), 1-3, 4-6, 7-10 (severe). Outcomes included symptom documentation in the chart on the visit date and symptom-related action(s) taken within 1 week.

Results:

Nine hundred twelve visits were identified. Pain and shortness of breath were documented in 51.8% and 29.7% of charts, and a related-action occurred in 16.9% and 3.9% of charts, respectively. As ESAS severity score category increased from none to severe, the proportion of visits with pain documented increased significantly (36.9%, 49.2%, 55.2%, and 71.4%; P < .001). Likewise, as ESAS score severity increased, the proportion of visits with a pain-related action increased significantly (4.2%, 10.6%, 21.3%, and 37.0%; P < .001). Trends were similar for shortness of breath.

Conclusion:

Results show a positive association between higher symptom scores and higher rates of documentation and clinical actions taken. However, symptom-related actions were documented in a minority of visits in which symptoms were noted as severe.

Introduction

Standardizing symptom screening is purported to help oncology providers more effectively identify symptom needs and manage significant symptom issues. To help standardize symptom assessment into routine clinical care, several large oncology settings have invested in electronic systems to capture patient-reported symptom data.¹ Electronic symptom assessment has been shown to be feasible and efficient at helping to standardize screening, inform symptom management, and monitor adverse events and quality^1–4; however, it also represents a major investment in time and resources by the oncology system.

Several randomized trials using electronic symptom reporting have suggested that routinely providing oncology providers with patient-reported symptom outcomes is beneficial, though the evidence is nuanced, with limitations to specific symptoms, particular tools or interventions, or nonsignificant results.^4–9 As a result of these nuanced results, the clinical processes related to symptom management practices are important to investigate, particularly the association between symptom screening and symptom-related clinical actions taken, which are the means to improve patient outcomes, for example reduce symptom burden. However, little empirical research exists describing whether and how implementing electronic, standardized, symptom screening affects clinical processes related to symptom management, such as whether patients reporting worse symptoms receive clinical actions more often.

In this study, we investigate the process links between standardized symptom screening and clinical actions to manage symptoms. Specifically, we examined patient-reported symptom scores during patient visits in a regional cancer center in Ontario, Canada. We audited visits by reviewing patient charts for symptom-related clinical notes and actions taken. Symptom screening utilized an electronic version of the Edmonton Symptom Assessment System (ESAS), a patient-reported, validated tool developed for rapid assessment of symptom needs in routine practice.^10–12 At the time of data collection, no guidelines existed on how clinical teams ought to incorporate ESAS scores into practice. Thus, symptom management practices were at the discretion of the clinical team. We conjectured that if a patient reported high ESAS scores for particular symptoms, the oncologist-nurse care team would focus on the identified issue and develop a care plan with the patient when appropriate. The steps in the care plan were assumed to be documented in the chart, and included clinical actions such as referrals, treatments, or prescriptions, where appropriate. Thus we hypothesized that the proportion of visits where (1) the symptom was documented in the chart, and (2) a symptom-related action was taken would be positively associated with increasing ESAS symptom scores. Our study focused on two cancer types, breast and lung cancer, and two particular symptoms, pain and shortness of breath.

Methods

Design and Setting

Retrospective chart reviews (paper) were conducted on visits to a regional cancer center in Hamilton, Ontario, Canada by ambulatory patients with lung and breast cancer. Lung and breast cancer were chosen because they represent two major cancer types with large numbers of patient visits, allowing for cancer-specific comparisons. The study focused on pain and shortness of breath because they are prevalent in patients with lung and breast cancer and have been well studied using ESAS in other research.^13–17 Of note, in Ontario, unlike in the United States, cancer pain is not measured as a fifth vital sign or as part of routine care.¹⁸ Cancer diagnoses were taken from the cancer center's electronic administrative database. Visits in which patients did not complete ESAS scores for pain and shortness of breath were excluded.

The cancer center serves a population of > 2 million individuals, with > 5,000 new patients and 200,000 patient encounters per year. Since March 2009, approximately 3,500 to 4,000 ESAS reports were being completed each month at the Juravinski Cancer Centre, with ESAS completed at approximately 50% of patient visits.¹⁹

Implementation of ESAS

Since 2007, all 14 cancer centers in Ontario have implemented an electronic version of the ESAS in virtually all clinics for patients to complete at every visit, thus effectively standardizing cancer symptom screening across the oncology system.²⁰ The instrument measures the severity (scale of 0-10; 0 = none, 10 = worst) of nine common cancer physical and psychological symptoms, specifically, pain, shortness of breath, nausea, anxiety, depression, tiredness, drowsiness, appetite, and well-being. It has been used in oncology settings in the United Kingdom, United States, and elsewhere internationally.^21–23 The process of using ESAS involves the patient visiting the cancer center and voluntarily self-reporting their ESAS symptom burden using an electronic touch-screen kiosk. A printed summary of the symptom scores, including those from previous visits, is attached to the patient chart for review by the oncologist-nurse care team before meeting the patient.

Sampling Strategy and Chart Reviews

Cancer visits were sampled between September 1, 2009 and December 31, 2009, after the ESAS had been fully implemented for several months. Previous research showed that ESAS scores in the Ontario cancer population were heavily skewed toward 0, with approximately half of ESAS assessments reporting 0 scores for pain and shortness of breath, respectively, and approximately 10% reporting scores of 7-10.¹⁷ Following methods used in prior research, the ESAS symptoms were categorized into four categories of severity: none (0 score), mild (1 to 3 score), moderate (4 to 6 score) and severe (7-10 score), where scores of > 4 indicate clinically significant symptom issues.^24,25 To ensure adequate sample size in each of the ESAS categories by cancer type and symptom, we chose a stratified sampling method. A priori, we aimed for approximately 110 lung and breast cancer visits, respectively, within each of the ESAS score categories (ie, scores = 0, 1-3, 4-6, and 7-10); within each ESAS score category, we aimed for half of the scores to represent pain and half shortness of breath. Among those visits eligible within each ESAS score category by disease site, visits were chosen randomly. Visits selected contributed both the pain and the shortness of breath score reported for that visit. Note that random selection of visits meant that some patients contributed multiple visits to our sample, which was intentional to allow for subanalysis of clinical actions over time, but was not pursued because of the small numbers of actions observed.

Once visits were selected, the corresponding patient chart was retrieved and reviewed for that visit date. Chart reviews were conducted by 4th-year nursing students under the training and supervision of a registered nurse practitioner (L.M.R.) with expertise in symptom management for patients with cancer. Reviewers completed a standardized chart review form described below and were blinded to ESAS scores for that date. During the initial 100 chart reviews, all charts were reviewed by two student reviewers for quality assurance, where any discrepancies were discussed with the lead reviewer (L.M.R.) and resolved as a group (chart review form and instructions shown in Appendix, online only).

Outcomes and Independent Variable (ESAS score categories)

The main independent variable was patient-reported ESAS symptom scores for pain and shortness of breath, categorized by symptom severity (ie, scores = 0, 1-3, 4-6, 7-10). The outcomes were clinical actions related to the two symptoms. The standardized chart review form assessed whether pain and shortness of breath, respectively, were mentioned in the patient chart (eg, notes) on the selected visit date, and if symptom-related actions were taken, as documented in the chart within 7 days after the selected visit date. Seven days was deemed sufficient time to document symptom actions in the chart by clinical coauthors (J.S. and L.M.R.) and based on other research.²⁶ Symptom-related actions included relevant drugs being prescribed or modified (eg, dosage change), or a related test, treatment, or referral being made. Specific drugs, tests, treatments, and referrals deemed related to a particular symptom were identified a priori by a group of nurse and physician researchers and compiled as categorical actions to search for in the chart review form (Table 1 footnote). Other symptom-related actions were captured in an “Other” category.

Table 1.

Summary of Patient Visits

Variable	All		Breast		Lung
	No.	%	No.	%	No.	%
No. of patients	912	100	459	50.3	453	49.7
Sex
Male	232	25.4	5	1.1	227	50.1
Female	680	74.6	454	98.9	226	49.9
Pain score
0	263	28.8	115	25.1	148	32.7
1-3	236	25.9	119	25.9	117	25.8
4-6	221	24.2	110	24.0	111	24.5
7-10	192	21.1	115	25.1	77	17.0
Pain documented in chart
Yes	472	51.8	227	49.5	245	54.1
Any action related to pain
Yes	154	16.9	69	15.0	85	18.8
Specific action
Prescribed medicinea	64	7.0	29	6.3	35	7.7
Modifiedb	40	4.4	14	3.1	26	5.7
Related testc	51	5.6	23	5.0	28	6.2
Therapyd	15	1.6	8	1.7	7	1.5
Referrale	19	2.1	7	1.5	12	2.6
Shortness of breath score
0	242	26.5	129	28.1	113	24.9
1-3	228	25.0	114	24.8	114	25.2
4-6	226	24.8	113	24.6	113	24.9
7-10	216	23.7	103	22.4	113	24.9
Shortness of breath documented in chart
Yes	271	29.7	75	16.3	196	43.3
Any action related to shortness of breath
Yes	31	3.4	3	0.7	28	6.2
Specific actions
Prescribed medicinef	3	0.3	1	0.2	2	0.4
Modifiedg	1	0.1	0	0.0	1	0.2
Related testh	17	1.9	1	0.2	16	3.5
Therapyi	8	0.9	0	0.0	8	1.8
Referralj	6	0.6	1	0.2	5	1.1

^aOpioid, nonsteroidal anti-inflammatory drugs (NSAIDS), acetaminophen, tricyclic antidepressant (TCA), anticonvulsant, corticosteroids.

^bOpioid, NSAIDS, acetaminophen, TCA, corticosteroids.

^cx-ray, spiral computed axial tomography (CT scan), bone scan, magnetic resonance imaging, ultrasound, laboratory analysis, other (mammogram).

^dChemotherapy, radiation therapy, other (acupuncture therapy, pamidronate, lymphedema management).

^ePain and symptom management team, home care, other (thrombosis, surgical oncologist, supportive care services, rapid response bone metastasis clinic, radiation oncologist, medical oncologist).

^fInhalers, corticosteroids.

^gCorticosteroids.

^hx-ray, spiral CT scan, other (spirometry, electrocardiography, CT scan of thorax, bronchoscopy).

ⁱChemotherapy, radiation therapy, chest drain, other (thoracentesis, bronchoscopy, brachytherapy, packed red blood cell units).

^jBreathing clinic, cardiology, respirology, radiology, other (home care, family doctor follow-up).

Statistical Analyses

Descriptive statistics were used to summarize characteristics by visit. A Cochran-Armitage test was used to examine whether a trend in outcomes was observed across ESAS score categories, and Fisher's exact test was used to investigate whether the frequency of actions taken differed by whether the symptom was documented in the chart. Outcomes were examined across both disease sites combined and by lung and breast cancer sites separately. The unit of analysis was the patient visit, although it is acknowledged that each visit was not necessarily independent (eg, one patient might have multiple visits). As a result, a sensitivity analysis was conducted that included only the first visit for each patient, and which showed almost identical trends by symptom and cancer type. Methods that account for correlation between dependent data points, such as generalized estimating equations, were not performed because of the small numbers of actions. The study was approved by the ethics review board of McMaster University.

Results

During our study period, 2,096 breast cancer and 742 lung cancer visits occurred that had ESAS scores for pain and shortness of breath. From that pool, our stratified sampling criteria identified a sample of 912 visits from 648 unique patients (Table 1). Of the 648 patients, 481 (74.2%) were sampled only once, 110 (17.0%) were sample twice, 32 (4.9%) were sampled three times, and 25 (3.9%) were sampled four or more times. The mean age of breast and lung cancer patients was 61.3 years (standard deviation [SD] = 12.7 years) and 68.3 years (SD = 10.1 years), respectively, and overall average age was 64.3 years (SD = 12.2 years). Among all visits, as per our a priori sampling criteria, half were breast and lung cancer visits, respectively; within each disease site, approximately a quarter of visits were from each of the four categories of ESAS scores.

The proportion of all visits for which pain was documented in the chart and a pain-related action documented increased significantly as ESAS symptom score increased by category (ie, 0, 1-3, 4-6, and 7-10) (Figure 1). When reported as moderate to severe (4-10 score), pain was documented in the chart in 63% of visits (n = 259 of 413), and a pain-related action was documented in 29% of visits (n = 118 of 413) (Table 2). Furthermore, 48% of visits (n = 66 of 137) had patient-reported severe pain and documentation in the chart, but no actions documented. Of the 154 pain-related actions documented, the most common was having a new drug prescribed (more than half of which were opioids), followed by having a test ordered (eg, bone scans, x-rays, or computed tomography scans). A referral to a pain and symptom management team occurred in only six visits.

Figure 1.

Pain and shortness of breath outcomes for all patient visits. ESAS, Edmonton Symptom Assessment Scale. (*) Sample size by ESAS score category: pain: 0 (n = 263), 1-3 (n = 236), 4-6 (n = 221), 7-10 (n = 192); shortness of breath: 0 (n = 242); 1-3 (n = 228); 4-6 (n = 226); 7-10 (n = 216).

Table 2.

Summary of Symptom Outcomes Overall and by Cancer Site

Cancer Site and Outcome	ESAS Score Category
	0		1-3		4-6		7-10		P
	No.	%	No.	%	No.	%	No.	%
Overall
Pain	263		236		221		192
Documented in chart	97	36.9	116	49.2	122	55.2	137	71.4	< .001
Symptom-related action taken	11	4.2	25	10.6	47	21.3	71	37.0	< .001
Shortness of breath	242		228		226		216
Documented in chart	54	22.3	51	22.4	71	31.4	95	44.0	< .001
Symptom-related action taken	1	0.4	2	0.9	8	3.5	20	9.3	< .001
Breast
Pain	115		119		110		115
Documented in chart	41	35.7	54	45.4	55	50.0	77	67.0	< .001
Symptom-related action taken	7	6.1	16	13.5	16	14.6	30	26.1	< .001
Shortness of breath	129		114		113		103
Documented in chart	19	14.7	10	8.8	20	17.7	26	25.2	.013
Symptom-related action taken	0	0.0	0	0.0	1	0.9	2	1.9	—
Lung
Pain	148		117		111		77
Documented in chart	56	37.8	62	53.0	67	60.4	60	77.9	< .001
Symptom-related action taken	4	2.7	9	7.7	31	27.9	41	53.3	< .001
Shortness of breath	113		114		113		113
Documented in chart	35	31.0	41	36.0	51	45.1	69	61.1	< .001
Symptom-related action taken	1	0.9	2	1.8	7	6.2	18	15.9	< .001

Similarly, the proportion of all visits with shortness of breath documented in the chart and a symptom-related action increased from lowest to highest ESAS score category. When shortness of breath was reported as moderate to severe, 38% of visits had shortness of breath recorded in the chart (n = 166 of 442), and 6% had a symptom related-action reported (n = 28 of 442). Furthermore, 79% (n = 75 of 95) had patient-reported severe shortness of breath and documentation in the chart, but no symptom-related actions documented. The most frequent action was having a test ordered, more than half of which were x-rays.

When examining symptoms for breast and lung cancer separately, a similar trend of increasing symptom documentation and symptom-related actions occurred with increased ESAS scores. Similar proportions of breast and lung cancer visits had pain documented in the chart across the ESAS score categories. In contrast, higher proportions of lung cancer visits had shortness of breath documented in the chart and related actions taken across ESAS score categories. When it was rated as severe, shortness of breath was documented at the visit of a patient with lung cancer more than 60% of the time, compared with a 25% of visits for patients with breast cancer. With few exceptions, a symptom-related action occurred only when pain or shortness of breath were documented in the chart, regardless of ESAS score category or cancer type.

Discussion

Our chart review of more than 900 patient visits confirmed both our hypotheses: visits in which patients reported a higher ESAS score category for pain or shortness of breath were significantly associated with higher rates of (1) having that symptom documented in the chart, and (2) having a symptom-specific action taken. This trend was evident when comparing symptoms for breast and lung cancer separately.

Our finding of a positive association between symptom scores and documentation and actions supports the notion that standardized, electronic screening can help clinicians to better manage severe symptom issues. One hypothesized pathway is that high ESAS scores trigger a discussion by the physician and patient about symptom management. On the other hand, the associations between higher ESAS scores and higher rates of documentation and clinical actions do not imply causality (ie, that ESAS was the direct cause of the increased chart documentation or symptom-specific actions taken). Another pathway is that patients with severe symptoms may discuss these with their physician anyway, regardless of ESAS score. However, if the completion of ESAS helps to prompt or empower the patient to discuss their symptoms with their physician, this may be a positive outcome in and of itself.

Perhaps the most striking result is the low proportion of resultant actions taken even when moderate-to-severe symptoms were documented in the chart. When symptoms were reported as moderate to severe (ie, score = 4-10) for pain and shortness of breath, clinical actions were documented in only 29% and 6% of visits, respectively. Other research has found similarly high rates of inaction for moderate-to-severe pain.²⁶ A lack of documented action when a patient reports a high symptom score does not necessarily imply poor patient care. Providers may inquire about high scores but discover a misinterpretation of the scale or a symptom unrelated to cancer care, such as shortness of breath caused by climbing stairs too quickly. A discussion or action plan (eg, monitoring) may occur that does not get recorded in the chart. Further, a provider and patient may rationally decide to not pursue treatment (eg, because of possible adverse effects), or a provider may offer treatment but the patient refuses.

There are noted barriers to using symptom screening information to influence clinical practice²⁷; nonetheless, other explanations for a lack of documented actions in response to moderate-to-severe symptoms may indicate areas for improved care. In the most extreme case, a provider may never refer to the ESAS scores or may not inquire about symptoms at all during the visit. In other cases, effective identification and management of symptom issues might be cancer specific, which might explain why shortness of breath was more commonly documented and treated in patients with lung cancer patients. Also, despite established guidelines for pain^28–30 and dyspnea,^31–33 physicians may lack the knowledge or experience to manage worsening or complex symptomatology, particularly when patients report a zero score for the majority of symptoms.¹⁷ The ease of treatment options (eg, prescriptions) might explain why pain-related actions occurred more commonly than actions for shortness of breath.

Our results may illuminate areas to improve care. First, ESAS assessment was introduced in the Ontario cancer centers without a clear clinical pathway for dealing with moderate-to-severe symptom scores. Standardized symptom assessment alone may be insufficient to demonstrably improve symptom management, but instead, in conjunction, requires that automated symptom alerts and/or management care plans are implemented and provided to clinicians when symptoms reach predetermined thresholds.^1,9,34 Second, the rate of symptom screening has been adopted as an indicator of quality of care in some settings.^{19,28,32,35–37} However, because our results demonstrate that high screening rates do not necessarily translate into high rates of symptom-related actions being taken, our results also suggest caution in interpreting screening rates alone as indicators of high-quality care.³⁸

Our study adds to the literature on the implementation of patient-reported outcome assessment in clinical practice. Other studies, typically pilot studies, have demonstrated the feasibility of implementing electronic assessment systems to improve cancer quality^1,39; this study builds on past research by examining clinical actions taken by providers to determine whether and how electronic patient-reported outcomes influence routine clinical practice.²⁷ To date, the research evidence on the influence of electronic symptom assessment is mixed. Two reviews found limited evidence that symptom screening influenced clinical practice to improve patient health status.^40,41 However, several randomized trials have suggested that routinely providing oncology providers with symptom outcomes has several patient benefits, such as reduced symptom prevalence and severity and improved well-being.^{4,5,7–9,42,43} In contrast, our study was not randomized. Moreover, unlike the trials with a defined intervention after a predetermined screening threshold, our cancer centers implemented screening without clear care pathways or designated trained staff to handle problem scores, which is more indicative of real-world settings. Finally our study focused not on reduced symptom burden, but on the clinical processes of care related to symptom management, such as actions taken to address severe symptoms.

This study has several limitations. Chart reviews have inherent challenges, such as missing documentation and the inability to assess the appropriateness of actions to a particular symptom. We did not differentiate whether documentation in the chart indicated the symptom's presence or absence, potentially explaining why 0 scores also had documentation of symptoms (ie, their absence). We did not assess the potential effect of prior symptom scores or care plans on audited symptom-related actions. The study cannot make causal inferences about ESAS reporting on clinical processes related to symptom management; symptom-related actions may have occurred without reference to ESAS scores. Moreover, our results are limited to patient visits and clinical actions where an ESAS was completed, which for voluntarily reported outcomes do not include every patient encounter. These results may not be true for other cancer symptoms, assessment tools, cancer centers, or cancer types. Future research might prospectively examine the intermediate process steps between screening and patient outcomes using audio- or video-recorded research methods.

In conclusion, our results showed that higher symptom scores for pain and shortness of breath were associated with a higher likelihood of documenting those symptoms in the patient chart and with taking clinical action. However, opportunities to improve symptom management remain. Despite high rates of symptom documentation, symptom-specific clinical actions occurred at much lower rates, even when symptoms were documented as severe. More research is needed to fully understand the impact of electronic, standardized symptom screening on clinical processes related to symptom management and on interventions that reduce symptom burden and improve other patient outcomes.

Authors' Disclosures of Potential Conflicts of Interest

The author(s) indicated no potential conflicts of interest.

Author Contributions

Conception and design: Hsien Seow, Jonathan Sussman, Lorraine Martelli-Reid, Greg Pond

Financial support: Hsien Seow

Administrative support: Hsien Seow, Jonathan Sussman, Daryl Bainbridge

Provision of study materials or patients: Lorraine Martelli-Reid, Daryl Bainbridge

Collection and assembly of data: Hsien Seow, Lorraine Martelli-Reid, Greg Pond, Daryl Bainbridge

Data analysis and interpretation: Hsien Seow, Jonathan Sussman, Lorraine Martelli-Reid, Greg Pond, Daryl Bainbridge

Manuscript writing: All authors

Final approval of manuscript: All authors