Abstract
Introduction
To compare the time duration of self‐completion (SC) of the Edmonton Symptom Assessment Scale (ESAS) by patients with advanced cancer (ACPs) versus assisted completion (AC) with a health care professional.
Materials and Methods
In this randomized comparison of ACPs seen in initial consultation at the outpatient Supportive Care Center at MD Anderson, ACPs who have never completed the ESAS at MD Anderson were allocated (1:1) to either SC of the ESAS form versus AC by a nurse. Time of completion was measured by the nurse using a stopwatch. Patients completed the Rapid Estimate of Adult Literacy in Medicine (REALM) test prior to administration of the ESAS. In the SC group, the nurse reviewed the responses to verify that the reported ESAS scores were correct.
Results
A total of 126 ACPs were enrolled (69 patients to AC and 57 to SC). Seventy‐one patients were female, median age was 60 years, and median REALM score was 65. Median (interquartile range) time (in seconds) of SC was significantly less than AC (73 [42.9–89.1] vs. 109 [79.5–136.7], p < .0001). With nurse review time included, median time of SC increased to 117 seconds, which was not significantly different from AC (p = .28). Lower literacy (REALM) score and shortness of breath were significantly associated with increased completion time (p = .007).
Conclusion
Regular use of ESAS will have minimal impact on clinical time, as it can be completed in about 1 minute and provides a concise yet comprehensive and multidimensional perspective of symptoms that affect quality of life of patients with cancer.
Implications for Practice
Because the Edmonton Symptom Assessment Scale can be completed in less than 2 minutes, hopefully the routine use of this simple yet comprehensive and multidimensional symptom assessment tool will be used at all medical visits in all patients with cancer so that the timely management of symptoms affecting patients' lives and treatment courses can occur, further enhancing personalized cancer care.
Keywords: Symptom assessment • Palliative care • Supportive care
Short abstract
The Edmonton Symptom Assessment Scale (ESAS) is a multidimensional assessment tool for self‐reporting of symptom intensity and developed for daily symptom assessment of palliative care patients. This study compared time requirements for self‐completed reports versus reports completed with assistance.
Introduction
The Edmonton Symptom Assessment Scale (ESAS) is a multidimensional assessment tool for self‐reporting of symptom intensity that was developed in the Palliative Care Unit of the Edmonton General Hospital (Canada) for use in daily symptom assessment of palliative care patients [1]. The ESAS has been implemented and validated [2, 3, 4, 5] in diverse palliative and cancer programs and countries. It consists of numerical rating scales for 10 common symptoms of advanced cancer (pain, tiredness, nausea, depression, anxiety, drowsiness, appetite, well‐being, shortness of breath, and sleep) and uses a score range of 0 to 10 to measure distress associated with both physiological and psychological symptoms, with 0 meaning no symptom at all and 10 meaning the worst possible symptom. A new version of the ESAS has been validated [11, 12, 13], with similar characteristics as the original.
ESAS has been widely adopted worldwide to guide daily interdisciplinary clinical care, patient referral to higher complexity programs, and monitor quality of care [14, 15]. Cancer Care Ontario launched a quality improvement project to implement ESAS in palliative cancer care clinics in late 2006. The team performed a patient satisfaction survey regarding ESAS in 2009. Of the 5,844 respondents, 89% felt ESAS was important to complete, 79% agreed that their health care team used the ESAS symptom scores in decision making, and 78% agreed that their pain and other symptoms had been controlled to a comfortable level, a significant improvement as compared with a previous 2007 survey [16].
A recent bibliometric analysis of the ESAS highlighted a rapid and multinational uptake of the tool [17], where it has been employed for clinical, research, and administrative purposes [11, 18]. Nurses' perceptions of the ESAS have been generally favorable in one survey [19], and 63% of frontline palliative care health professionals in another survey reported that the assessment tools which included the ESAS added value to their practice [20]. Yet, at the present time, the vast majority of patients with advanced cancer in the U.S., including at MD Anderson Cancer Center, do not undergo regular assessment of symptoms. One of the main reasons why symptom assessment is not routinely implemented is the perception that it may be time‐consuming [21].
Self‐reporting of symptoms by patients themselves is often regarded as the standard for assessment in palliative care in instances such as pain assessments because observers often have different perceptions about the symptoms experienced by patients [22, 23, 24, 25]. Studies comparing palliative care patient self‐assessment with ratings by physicians or nurses have shown significant differences in symptoms' rating [26, 27, 28, 29, 30], hence the importance of ESAS self‐completion.
In this randomized comparison, we sought to compare the time requirement of self‐completion as compared with assisted completion of the ESAS in patients with advanced cancer. We hypothesized that the presence of a health care assistant may shorten time to complete the ESAS form, as patients would have access to help or clarification provided by the health care assistant.
Materials and Methods
This is an institutional review board–approved randomized comparison done in the outpatient Supportive Care Center at MD Anderson. Patients who had initial consultations to the supportive care team were reviewed by research staff for eligibility and included only if they had a diagnosis of advanced cancer (defined as recurrent, locally advanced, or metastatic cancer) and had never completed an ESAS at MD Anderson prior to enrollment. Eligible patients were approached by our research staff after they were roomed and invited to participate in the study. They were not told that time of completion would be assessed as the major outcome to prevent any potential bias. Other inclusion criteria included knowledge of cancer diagnosis; 18 years of age or older; ability to read, write, and understand English; and ability to sign informed consent and independently complete the study. Exclusion criteria included refusal to participate, completion of previous ESAS documented in the electronic medical record at MD Anderson, cognitive impairment, and/or severe physical or emotional distress as assessed by the research staff. A total of 127 patients were enrolled; one was excluded because the patient did not have an active cancer diagnosis (Table 1).
Table 1.
Patient characteristics
| Characteristics | Assisted | Self | p value a |
|---|---|---|---|
| n | 69 | 57 | |
| Age, mean | 58 | 57 | .81 |
| Female | 37 | 34 | |
| Primary cancer site | |||
| Breast | 6 | 7 | .57 |
| GI | 15 | 14 | |
| GU | 8 | 3 | |
| Gyn | 9 | 9 | |
| Head and neck | 10 | 6 | |
| Hematologic | 0 | 3 | |
| Other | 6 | 4 | |
| Sarcoma | 3 | 4 | |
| Thoracic | 12 | 7 | |
| Median ECOG (Q1,Q3) | 3 (2,3) | 3 (2,3) | .42 |
| Median ESAS (Q1,Q3) | |||
| Pain | 5 (2,7) | 4 (3,7) | .78 |
| Fatigue | 5 (3,7) | 5 (3,7) | .85 |
| Nausea | 1 (0,3) | 1 (0,4) | .96 |
| Depression | 1 (0,3) | 1 (0,4) | .75 |
| Anxiety | 2 (0,4) | 2 (1,4) | .90 |
| Drowsiness | 3 (1,5) | 3 (1,5) | .86 |
| Shortness of breath | 3 (1,5) | 3 (1,5) | .67 |
| Appetite | 2 (0,5) | 1 (0,5) | .53 |
| Sleep | 4 (2,7) | 5 (2,7) | .66 |
| Feeling of wellbeing | 4 (2,6) | 4 (2,6) | .61 |
| Median literacy score (Q1, Q3) | 65 (63,65) | 65 (64,65) | .49 |
| Health literacy level | |||
| Adequate | 59 | 47 | .22 |
| Low | 0 | 3 | |
| Marginal | 9 | 7 | |
| Missing | 1 | 0 | |
| Highest level education | |||
| Advanced degree | 14 | 4 | .20 |
| College | 15 | 17 | |
| Some college/junior college | 22 | 20 | |
| High school/technical school | 13 | 14 | |
| Less than high school | 5 | 2 | |
| Marital status | |||
| Divorced | 7 | 5 | .99 |
| Married | 52 | 44 | |
| Single | 6 | 5 | |
| Widowed | 4 | 3 | |
| Ethnicity | |||
| White | 52 | 44 | .51b |
| Asian | 3 | 0 | |
| Black | 5 | 5 | |
| Hispanic | 8 | 8 | |
| Other | 1 | 0 | |
| Religion | |||
| Buddhist | 1 | 0 | .19 c |
| Catholic | 11 | 17 | |
| Christian/protestant | 39 | 26 | |
| Hindu | 2 | 0 | |
| Jewish | 1 | 2 | |
| Muslim | 1 | 1 | |
| Other | 14 | 11 |
Wilcoxon rank sum.
Comparing ethnicity excluding other.
Comparing religion excluding other.
Abbreviations: ECOG, Eastern Cooperative Oncology Group; ESAS, Edmonton Symptom Assessment Scale; GI, gastrointestinal; GU, genitourinary; Gyn, gynecologic; Q, quartile
After signing informed consent, patients underwent health literacy screening using the Rapid Estimate of Adult Literacy in Medicine (REALM) tool. The REALM is a well‐validated, practical reading test of 66 words that patients commonly encounter in medical settings [31, 32]. Participants read the words aloud, and the interviewer records whether each word is pronounced correctly or not. Participants are not expected to know the definitions of the words. After completion of the REALM, patients were then randomized using the Clinical Oncology Research Web site 1:1 to self‐completion versus assisted completion by the research assistant. A total of 126 patients were randomly allocated to either self‐completion (n = 57) or assisted completion (n = 69) (Fig. 1). Both groups completed a paper and pencil form of the ESAS.
Figure 1.
CONSORT diagram.
In the self‐completion group, patients must complete the ESAS alone, and the time recorded started at the time the ESAS was given until the patient finished the tool. Responses with scores close to zero (0–3) or close to 10 (7–10) were then counterchecked by the research assistant or nurse (designated as “nurse review”) for accuracy, and the time spent reviewing responses was timed separately. The assisted completion group completed the ESAS with the help of a health care assistant (in this case a research assistant), where the research assistant named each symptom and explained to the patient that they can score each symptom from 0 to 10, with 0 being the lowest and 10 being the highest. Patients were able to make comments regarding each symptom if they desired, and this was included in the total time spent completing the ESAS.
Descriptive statistics were summarized for patients' characteristics age, gender, race, education level, religion, marital status, cancer diagnosis, and ESAS score. Correlation was assessed for continuous covariates using Spearman rank correlation. Wilcoxon rank sum test was used to examine the difference of continuous covariates between assisted versus self‐completion. Kruskal‐Wallis test was used to examine the difference in time to complete ESAS among three health literacy levels. The χ2 test or Fisher's exact test, where appropriate, was used to examine the difference in type of ESAS completion between or among patients' categorical characteristics. General linear model was applied to assess the effect of important covariates, such as patients' demographic and clinical characteristics on the duration of completing ESAS.
All computations were carried out in SAS 9.3 (SAS Institute Inc., Cary, NC).
Results
All 126 patients were evaluable. All patients who were assigned to the self‐completion group were able to complete the ESAS questionnaire in its entirety, with the median time of self‐completion being 73 seconds (interquartile range [IQR], 43–89). The median time for self‐completion in addition to nurse review was 117 seconds (IQR, 81–165), which was not significantly different from that of health care professional (HCP)–assisted completion (109 seconds; IQR, 80–136.7; p = .28; Table 2). Additionally, there was no statistical difference in completion time among age, ECOG, pain, nausea, depression, anxiety, drowsiness, appetite, sleep, wellbeing, marital status, or religion (Table 1).
Table 2.
Completion time in seconds
| Completion type | n | Median (Q1, Q3) | p value a |
|---|---|---|---|
| Completion time | <.0001 | ||
| Assisted | 68 | 109 (79.5, 136.7) | |
| Self | 57 | 73 (42.9, 89.1) | |
| Review time: self | 57 | 37.2 (23, 59) | |
| Total time for self‐completion (completion + review): self | 57 | 117 (81, 165) | .28 b |
Wilcoxon rank sum
vs. AC group.
Abbreviation: Q, quartile.
Association Between Health Literacy and Time to Complete the ESAS in Patients with Advanced Cancer
Patients who had the low health literacy level spent significantly more time (230.9 ± 155.9) to complete the ESAS compared with the adequate health literacy level (112.3 ± 53.9) or marginal health literacy level (146 ± 62.8; p = .03; Table 3).
Table 3.
Completion time by health literacy level
| Health literacy level | n | Mean ± SD | Median (Q1, Q3) | p value a |
|---|---|---|---|---|
| Adequate | 105 | 112.3 ± 53.9 | 104 (78, 137.1) | .03 |
| Low | 3 | 230.9 ± 155.9 | 242 (69.8, 381) | |
| Marginal | 16 | 146 ± 62.8 | 133.5 (113.7, 188.3) |
Values are in total completion time in seconds.
Kruskal‐Wallis.
Abbreviation: Q, quartile.
Factors Associated with Self‐Completion Time
On univariate regression analysis, shortness of breath, male gender, nonwhite ethnicity, lower literacy score, lower education, and lower literacy level were significantly associated with a longer self‐completion time (Table 4).
Table 4.
Univariate regression analysis of the time to self‐complete Edmonton Symptom Assessment Scale
| Variable | Estimate (95% CI) | p value |
|---|---|---|
| Age | 0.91 (−0.46 to 2.28) | .19 |
| ECOG | −6.43 (−26.92 to 14.09) | .53 |
| Pain | 3.01 (−3.34 to 9.38) | .35 |
| Fatigue | 5.68 (−0.76 to 12.14) | .08 |
| Nausea | 3.49 (−3.18 to 10.18) | .30 |
| Depression | −0.28 (−7.94 to 7.37) | .94 |
| Anxiety | 4.29 (−4.14 to 12.73) | .31 |
| Drowsiness | 2.87 (−3.55 to 9.31) | .37 |
| Shortness of breath | 10.58 (4.03 to 17.12) | .001 |
| Appetite | 1.57 (−5.23 to 8.38) | .64 |
| Sleep | 4.68 (−2.26 to 11.64) | .18 |
| Feeling of wellbeing | 4.56 (−1.93 to 11.05) | .16 |
| Literacy score | −4.48 (−7.01 to −1.94) | <.001 |
| Gender: female vs. male | −40.05 (−76.19 to −3.92) | .03 |
| Ethnicity | ||
| Black vs. white | 90.93 (30.48 to 151.37) | .008 |
| Hispanic vs. white | 40.11 (−9.11 to 89.34) | |
| Highest level education | ||
| Advanced degree vs. some college/junior college | 20.96 (−48.15 to 90.07) | .02 |
| College vs. some college/junior college | 9.31 (−32.31 to 50.94) | |
| High school/ tech school vs. some college/junior college | 60.14 (16.17 to 104.11) | |
| Less than high school vs. some college/junior college | 116.42 (22.83 to 210.00) | |
| Marital status | ||
| Divorced vs. married | −8.62 (−73.36 to 56.12) | .60 |
| Single vs. married | −31.08 (−95.82 to 33.65) | |
| Widowed vs. married | 36.32 (−45.52 to 118.18) | |
| Religion | ||
| Catholic | 5.36 (−37.66 to 48.38) | .87 |
| Jewish | −46.58 (−147.80 to 54.63) | |
| Muslim | 16.63 (−123.92 to 157.20) | |
| Other | −10.00 (−59.61 to 39.60) | |
| Health literacy level | ||
| Low vs. adequate | 116.01 (42.92 to 189.10) | <.001 |
| Marginal vs. adequate | 70.65 (20.93 to 120.38) |
Abbreviations: CI, confidence interval; ECOG, Eastern Cooperative Oncology Group.
Table 5 shows the multivariate regression analysis of factors associated with time to self‐completion of ESAS. Shortness of breath, gender, and health literacy level were selected as predictors of time to self‐complete ESAS using Akaike's Information Criteria (AIC) and adjusted R2 (lowest AIC and highest adjusted R2). Shortness of breath (p = .0016) and lower literacy score as measured by REALM (p = .01) were significantly associated with longer self‐completion time of ESAS.
Table 5.
Multivariate regression analysis on the time to self‐complete Edmonton Symptom Assessment Scale
| Level | Estimate (95% CI) | p value |
|---|---|---|
| Shortness of breath | 9.87 (3.92 to 15.82) | .0016 |
| Gender: female vs. male | −28.88 (−60.91 to 3.15) | .07 |
| Health literacy level | ||
| Low vs. adequate | 3.49 (1.55 to 5.44) | .0004 |
| Marginal vs. adequate | 42.70 (−4.93 to 90.33) | .07 |
Abbreviation: CI, confidence interval.
Discussion
For patients with advanced cancer who have never completed an ESAS form, median completion time overall was less than 2 minutes, with median self‐completion time being 73 seconds, and median assisted completion time being 109 seconds. Marginal to low health literacy level and shortness of breath were associated with a longer completion time.
One of the main reasons why symptom assessment is not implemented is the perception that it may be time‐consuming. We found that ESAS can be completed with minimal time commitment, much shorter than reported by others. Few studies specify the time required for completing an ESAS. In the ESAS‐Spanish version validation, 171 patients took an average of 5.5 min (range, 3–20) to complete the ESAS [6]. In a study validating ESAS, Chang et al. found that 233 patients required approximately 5 minutes for each patient to complete the ESAS, but they did not report other statistical values [2]. The screening Ontario Program Web site only mentioned that the patients completed the ESAS in 3 minutes at the kiosk. Watanabe et al., in a study about palliative care nurses' perceptions of the ESAS, found that 63.8% of the 48 nurses opined that the ESAS does not take a lot of time and effort to complete, and only 25.5% of them thought it takes a lot of time and effort for patients to complete the ESAS. The median of 48 nurses estimated time required to administer and chart the ESAS was 6.7 minutes (range, 1.5–25) [25]. However, our study showed that a well‐trained health care provider can conduct an ESAS in less than 2 minutes, and a patient with advanced cancer can complete an ESAS in 1 to 2 minutes alone without help from an HCP.
One reason time was shorter in our study could be that the very symptomatic patients were excluded so as to not delay seeing the physician with the added time and burden of completing consents and the REALM test. Highly symptomatic patients may require more time to complete ESAS. Moderate dyspnea could be a marker for patients with worse symptom status requiring longer completion time. For average patients, the median time may be much shorter, but, for very symptomatic patients, the median time will need to be examined in future studies.
Another potential contributor to completion time in addition to education level and health literacy could be English as a second language as we see patients of vastly diverse backgrounds at our institution. This factor should be considered in future studies.
One possible reason for the increased completion time in previous studies is that the HCP may have engaged in a prolonged discussion regarding symptoms listed in ESAS, rather than simply screening for symptoms. As shown in our study, if the nurse only verifies the accuracy of the responses, completion time was not much different from completing the assessment alone.
The verification process took an extra 36 seconds. Although the number of corrections was not recorded, one could conclude that a minimal number of mistakes occurred given the minimal amount of time needed to review the accuracy. Future studies should record error rate to better determine the tradeoff of having an HCP help complete the form.
The time for completion is an extremely important because it would have a major impact on the cost of implementation and administration of this tool. Having patients complete the ESAS alone may seem like a savings of only seconds in one patient encounter. However, seconds translate to a savings of hundreds of hours when looking at the entire fiscal year.
Our palliative care program alone provides 35,000 encounters per year. If each ESAS takes about 1.2 minutes to complete alone, that is about 29,400 minutes per year, or 490 hours per year. With assistance from a nurse, time of completion increases by 36 seconds. In our clinical program, this equates to 1.26 million seconds (about 350 additional hours) of clinical assessment per year.
One proposal may be that the nurse may help patients complete the ESAS at the initial visit for teaching and accuracy, and the patient can later complete the ESAS alone at future visits. This could save hundreds of hours of clinical work over a year in this current fast‐paced, high‐demand health care environment requiring high‐efficiency care.
Having an HCP present for the administration of ESAS may be helpful in certain cases. In our study, marginal to low health literacy level was significantly associated with a longer completion time as compared with adequate health literacy as measured by REALM. Patients with marginal to low health literacy may benefit from assistance from a HCP in completing the ESAS, as recent studies have found health literacy to be associated with the health care outcomes [33, 34, 35] in variable conditions and the time it takes an individual to complete a health survey.
In a previous ESAS study [6], almost all patients (97%) perceived the ESAS as a tool that was easy to complete because of clarity of instruction and order of question, but 87.5% commented that having a health care professional ask them the questions helped them to complete and understand the instrument, especially when completing it for the first time. In our study, 22% preferred the help of a nurse, whereas 58% of patients did not mind completing ESAS alone or with the help of a nurse, and 21% preferred completing the ESAS alone. However, having a nurse assist with completing the ESAS may save overall time, as the nurse plays a significant role in flagging symptom concerns for the clinician responsible for future assessment; therefore, this should be the focus of future research.
Conclusion
In this fast‐paced period in the medical field in which reducing time constraints and increasing efficiency are valued, a simple 10‐ to 12‐item symptom assessment tool such as the ESAS can quickly help clinicians identify distressing symptoms that are affecting quality of life and possibly patients' ability to continue cancer treatments. Our study found that the ESAS can provide a multidimensional perspective of patients' perceived symptom burden in less than 2 minutes. Routine symptom assessments should occur in all patients with cancer.
Author Contributions
Conception/design: Eduardo Bruera, Supakarn Tayjasanant, Alfredo Rodriguez‐Nunez, Angelique Wong
Provision of study material or patients: Supakarn Tayjasanant, Alfredo Rodriguez‐Nunez, Minjeong Park, Diane Liu, Kresnier Perez Zapata, Julio Allo, Susan Frisbee‐Hume, Janet Williams
Collection and/or assembly of data: Angelique Wong, Minjeong Park, Diane Liu, Janet Williams, Kresnier Perez Zapata
Data analysis and interpretation: Angelique Wong, Minjeong Park, Diane Liu, Janet Williams, Eduardo Bruera
Manuscript writing: Angelique Wong, Eduardo Bruera, Supakarn Tayjasanant, Alfredo Rodriguez‐Nunez
Final approval of manuscript: Angelique Wong, Supakarn Tayjasanant, Alfredo Rodriguez‐Nunez, Minjeong Park, Diane Liu, Kresnier Perez Zapata, Julio Allo, Susan Frisbee‐Hume, Janet Williams, Eduardo Bruera
Disclosures
Eduardo Bruera: Helsinn Health (RF). The other authors indicated no financial relationships.
(C/A) Consulting/advisory relationship; (RF) Research funding; (E) Employment; (ET) Expert testimony; (H) Honoraria received; (OI) Ownership interests; (IP) Intellectual property rights/inventor/patent holder; (SAB) Scientific advisory board
Acknowledgments
Previously presented at the ASCO Palliative and Supportive Care in Oncology Symposium.
This article has not been previously published and is not under review or consideration for publication elsewhere.
Eduardo Bruera is supported in part by the National Institutes of Health grants RO1NR010162–01A1, RO1CA122292‐01, and RO1CA124481‐01.
Disclosures of potential conflicts of interest may be found at the end of this article.
No part of this article may be reproduced, stored, or transmitted in any form or for any means without the prior permission in writing from the copyright holder. For information on purchasing reprints contact commercialreprints@wiley.com. For permission information contact permissions@wiley.com.
References
- 1. Bruera E, Kuehn N, Miller MJ et al. The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. J Palliat Care 1991;7:6–9. [PubMed] [Google Scholar]
- 2. Chang VT, Hwang SS, Feuerman M. Validation of the Edmonton Symptom Assessment Scale. Cancer 2000;88:2164–2171. [DOI] [PubMed] [Google Scholar]
- 3. Nekolaichuk C, Watanabe S, Beaumont C. The Edmonton Symptom Assessment System: A 15‐year retrospective review of validation studies (1991‐2006). Palliat Med 2008;22:111–122. [DOI] [PubMed] [Google Scholar]
- 4. Richardson LA, Jones GW. A review of the reliability and validity of the Edmonton Symptom Assessment System. Curr Oncol 2009;16:55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Philip J, Smith WB, Craft P et al. Concurrent validity of the modified Edmonton Symptom Assessment System with the rotterdam symptom checklist and the brief pain inventory. Support Care Cancer 1998;6:539–541. [DOI] [PubMed] [Google Scholar]
- 6. Carvajal A, Centeno C, Watson R et al. A comprehensive study of psychometric properties of the Edmonton Symptom Assessment System (ESAS) in Spanish advanced cancer patients. Eur J Cancer 2011;47:1863–1872. [DOI] [PubMed] [Google Scholar]
- 7. Kwon JH, Nam SH, Koh S et al. Validation of the Edmonton Symptom Assessment System in Korean patients with cancer. J Pain Symptom Manage 2013;46:947–956. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Moro C, Brunelli C, Miccinesi G et al. Edmonton Symptom Assessment Scale: Italian validation in two palliative care settings. Support Care Cancer 2006;14:30–37. [DOI] [PubMed] [Google Scholar]
- 9. Schulman‐Green D, Cherlin EJ, McCorkle R et al. Benefits and challenges in use of a standardized symptom assessment instrument in hospice. J Palliat Med 2010;13:155–159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pautex S, Berger A, Chatelain C et al. Symptom assessment in elderly cancer patients receiving palliative care. Crit Rev Oncol Hematol 2003;47:281–286. [DOI] [PubMed] [Google Scholar]
- 11. Watanabe SM, Nekolaichuk CL, Beaumont C. The Edmonton Symptom Assessment System, a proposed tool for distress screening in cancer patients: Development and refinement. Psychooncology 2012;21:977–985. [DOI] [PubMed] [Google Scholar]
- 12. Watanabe SM, Nekolaichuk C, Beaumont C et al. A multicenter study comparing two numerical versions of the Edmonton Symptom Assessment System in palliative care patients. J Pain Symptom Manage 2011;41:456–468. [DOI] [PubMed] [Google Scholar]
- 13. Carvajal A, Hribernik N, Duarte E et al. The spanish version of the Edmonton Symptom Assessment System‐revised (ESAS‐r): First psychometric analysis involving patients with advanced cancer. J Pain Symptom Manage 2013;45:129–136. [DOI] [PubMed] [Google Scholar]
- 14. Hui D, Bruera E. The Edmonton Symptom Assessment System 25 years later: Past, present, and future developments. J Pain Symptom Manage 2017;53:630–643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Pereira J, Green E, Molloy S et al. Population‐based standardized symptom screening: Cancer care Ontario's Edmonton Symptom Assessment System and performance status initiatives. J Oncol Pract 2014;10:212–214. [DOI] [PubMed] [Google Scholar]
- 16. Dudgeon D, King S, Howell D et al. Cancer Care Ontario's experience with implementation of routine physical and psychological symptom distress screening. Psychooncology 2012;21:357–364. [DOI] [PubMed] [Google Scholar]
- 17. Cummings G, Biondo PD, Campbell D et al. Can the global uptake of palliative care innovations be improved? Insights from a bibliometric analysis of the Edmonton Symptom Assessment System. Palliat Med 2011;25:71–82. [DOI] [PubMed] [Google Scholar]
- 18. Dudgeon DJ, Harlos M, Clinch JJ. The Edmonton Symptom Assessment Scale (ESAS) as an audit tool. J Palliat Care 1999;15:14–19. [PubMed] [Google Scholar]
- 19. Watanabe S, McKinnon S, Macmillan K et al. Palliative care nurses' perceptions of the Edmonton Symptom Assessment Scale: A pilot survey. Int J Palliat Nurs 2006;12:111–114. [DOI] [PubMed] [Google Scholar]
- 20. Dudgeon DJ, Knott C, Chapman C et al. Development, implementation, and process evaluation of a regional palliative care quality improvement project. J Pain Symptom Manage 2009;38:483–495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Bainbridge D, Seow H, Sussman J et al. Multidisciplinary health care professionals' perceptions of the use and utility of a symptom assessment system for oncology patients. J Oncol Pract 2011;7:19–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Tang ST, McCorkle R. Use of family proxies in quality of life research for cancer patients at the end of life: A literature review. Cancer Invest 2002;20:1086–1104. [DOI] [PubMed] [Google Scholar]
- 23. Sneeuw KC, Sprangers MA, Aaronson NK. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease. J Clin Epidemiol 2002;55:1130–1143. [DOI] [PubMed] [Google Scholar]
- 24. Sprangers MA and Aaronson NK. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: A review. J Clin Epidemiol 1992;45:743–760. [DOI] [PubMed] [Google Scholar]
- 25. Strömgren AS, Groenvold M, Pedersen L et al. Symptomatology of cancer patients in palliative care: Content validation of self‐assessment questionnaires against medical records. Eur J Cancer 2002;38:788–794. [DOI] [PubMed] [Google Scholar]
- 26. Nekolaichuk CL, Bruera E, Spachynski K et al. A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat Med 1999;13:311–323. [DOI] [PubMed] [Google Scholar]
- 27. Nekolaichuk CL, Maguire TO , Suarez‐Almazor M et al. Assessing the reliability of patient, nurse, and family caregiver symptom ratings in hospitalized advanced cancer patients. J Clin Oncol 1999;17:3621–3630. [DOI] [PubMed] [Google Scholar]
- 28. Rhondali W, Hui D, Kim SH et al. Association between patient‐reported symptoms and nurses' clinical impressions in cancer patients admitted to an acute palliative care unit. J Palliat Med 2012;15:301–307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Strömgren AS, Groenvold M, Sorensen A et al. Symptom recognition in advanced cancer. A comparison of nursing records against patient self‐rating. Acta Anaesthesiol Scand 2001;45:1080–1085. [DOI] [PubMed] [Google Scholar]
- 30. Garyali A, Palmer JL, Yennurajalingam S et al. Errors in symptom intensity self‐assessment by patients receiving outpatient palliative care. J Palliat Med 2006;9:1059–1065. [DOI] [PubMed] [Google Scholar]
- 31. Davis TC, Crouch MA, Long SW et al. Rapid assessment of literacy levels of adult primary care patients. Fam Med 1991;23:433–435. [PubMed] [Google Scholar]
- 32. Davis TC, Long SW, Jackson RH et al. Rapid estimate of adult literacy in medicine: A shortened screening instrument. Fam Med 1993;25:391–395. [PubMed] [Google Scholar]
- 33. Han HR, Huh B, Kim MT et al. Development and validation of the assessment of health literacy in breast and cervical cancer screening. J Health Commun 2014;19(Suppl 2):267–284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Morris NS, Field TS, Wagner JL et al. The association between health literacy and cancer‐related attitudes, behaviors, and knowledge. J Health Commun 2013;18(Suppl 1):223–241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Williams KP, Templin TN, Hines RD. Answering the call: A tool that measures functional breast cancer literacy. J Health COmmun 2013;18:1310–1325. [DOI] [PubMed] [Google Scholar]