Abstract
Medical literature does not have clear consensus on inter-rater reliability of PS assessment by different oncology health care professionals (HCPs) although it plays an important role in treatment decision and prognosis for oncology patients. Eastern Cooperative Oncology Group (ECOG) and Karnofsky performance status (KPS) scores are commonly used for this purpose by oncology HCPs around the world. This study was conducted to find variability or similarities in assessment of PS among the different oncology HCPs. A survey based on four hypothetical clinical scenarios was devised and sent to 50 oncology HCPs to assess the PS using ECOG PS tool. No significant variations in PS assessment by oncology HCPs was noted in our study sample.
Keywords: Performance status, Eastern Cooperative Oncology Group (ECOG), Karnofsky performance status (KPS)
Introduction
Performance status (PS) of a patient is the assessment of level of function and capability of self-care. It plays a key role in treatment decisions and an independent prognostic indicator for patients with advanced malignancy. It is crucial to thoroughly assess and accurately document PS for each patient as many clinical decisions are mainly based on this. Oncology healthcare professionals (HCPs) use different tools to assess PS of cancer patients before making decision on systemic anticancer therapy. Eastern Cooperative Oncology Group (ECOG) and Karnofsky performance status (KPS) scores are commonly used for selection of appropriate patients in clinical trials or for standard cancer treatments. KPS score was first introduced by Karnofsky et al. in 1948 in patients receiving chemotherapy for lung cancer [1]. It uses linear scores of 0 to 100 depending upon patient's ability to perform daily activities and the assistance they require. On the other hand, ECOG uses 5 points score to assess PS and is considered simple tool to use in daily clinical practice (Table 1). It is also called world health organization (WHO) PS and was first developed in 1960 [2].
Table 1.
Assessment of ECOG PS by oncology healthcare professionals
| Clinical Scenario Number | ECOG PS 0 | ECOG PS 1 | ECOG PS 2 | ECOG PS 3 | ECOG PS 4 | Unclear |
|---|---|---|---|---|---|---|
| 1 | 4 (8%) | 18 (36%) | 25 (50%) | 1 (2%) | 0 | 2 (4%) |
| 2 | 0 | 2 (4%) | 28 (56%) | 8 (16%) | 0 | 12 (24%) |
| 3 | 0 | 1 (2%) | 12 (24%) | 33 (66%) | 1 (2%) | 3 (6%) |
| 4 | 0 | 1 (2%) | 7 (14%) | 19 (38%) | 14 (28%) | 9 (18%) |
Assessment of PS before systemic therapy is also necessary as patients with poor PS have higher risk of chemotherapy toxicity and poor outcome [3, 4]. Inter-observer variability in the assessment of PS is not uncommon but uniformity in recording of PS can help in accurate assessment of prognosis, fitness for treatment and response to treatment. We conducted this study to find variability or similarities in assessment of PS among the different oncology HCPs.
Methods and Materials
A survey based on clinical scenarios was devised and sent to oncology healthcare professionals (HCPs). This survey questionnaire was created with four hypothetical clinical scenarios to assess the PS of patients with advanced lung cancer, metastatic melanoma, metastatic ovarian cancer and advanced germ cell tumor. Information about patient's history, comorbidities, and description of functional status was detailed in each scenario (Table 2).
Table 2.
Eastern Cooperative Oncology Group (ECOG) performance status [18]
| Grade | ECOG performance status |
|---|---|
| 0 | Fully active, able to carry on all pre-disease performance without restriction |
| 1 | Restricted in physically strenuous activity but ambulatory and able to carry out work of a light or sedentary nature, e.g., light housework, office work |
| 2 | Ambulatory and capable of all selfcare but unable to carry out any work activities; Up and about more than 50% of waking hours |
| 3 | Capable of only limited selfcare, confined to bed or chair more than 50% of waking hours |
| 4 | Completely disabled; Cannot carry out any selfcare; Totally confined to bed or chair |
| 5 | Dead |
Survey questionnaire was sent to consultant oncologists, specialist trainee registrars and oncology specialist nurses. They were asked to assess the PS of each case. ECOG scale of PS was used for the purpose of this survey, as it is considered simple and the most commonly used PS assessment tool used by HCPs. An option of unclear was also added to ECOG scale of 0, 1, 2, 3, 4. Responses were received from 50 oncology HCPs and were analysed for individual clinical scenario.
Data Analysis
Data were analysed using SPSS (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY, USA: IBM Corp.). A Shapiro-Wilk test score (p value > 0.05) and visual inspection for Histogram, Normal Q-Q Plot and boxplot showed approximately normal distribution of all scenarios. We compared the number of responses for each scenario for all responder groups. We used analysis of variance (ANOVA) in which the dependent variable was one (scenario) and the independent variables were groups (consultant oncologists, specialist trainee registrars and oncology specialist nurses). Two-tailed p values of less than 0.05 were considered to be statistically significant.
Results
A total of 50 Oncology HCPs were randomly invited to participate in the survey and all of them responded (100%). Oncology HCPs included oncology consultants (n = 18, 36%), oncology specialist registrars (n = 22, 44%) and oncology specialist nurses (n = 10, 20%). Responses were analysed for each clinical scenario separately.
The first clinical scenario of a 78 year old gentleman with a diagnosis of non-small cell lung cancer was considered to have ECOG PS of 2 by majority (n = 25, 50%) of responders and ECOG PS 1 (n = 18, 36%) followed by ECOG PS 0 (n = 4, 8%), ECOG PS 3 (n = 1, 2%) and unclear (n = 2, 4%). Eleven of 18 consultants rated ECOG PS of 2 and other 7 rated ECOG PS as 1. Majority of registrars (n = 10) reported ECOG PS of 2 followed by ECOG PS 1 (n = 7) and ECOG PS 0 (n = 3). Two registrars were unclear about ECOG PS in this case. ECOG PS 0, 1, 2 and 3 were reported by 1, 4, 4 and 1 specialist nurses respectively. Statistically, no significant variations in PS assessment among the responders (p = 0.885) were found (Fig. 1, 2).
Fig. 1.
Numbers of HCPs responses for the first scenario.
Fig. 2.
Questionnaire with clinical scenarios used for the study.
The second clinical scenario was about a 66 year old lady with long standing history of rheumatoid arthritis and a recent diagnosis of metastatic melanoma. She was on regular methotrexate for rheumatoid disease and lived alone with little outdoor activities. She had a cleaner once a week and one of her daughter shops for her. The respondents were asked to comment on her ECOG scale of PS. Majority of respondents rated this scenario as ECOG PS 2 (n = 28, 56%) and unclear (n = 12, 24%) followed by ECOG PS 3 (n = 8, 16%) and ECOG PS 1 (n = 2, 4%). Eleven of 18 consultants rated ECOG PS of 2 and 2 rated ECOG PS as 3. Five were unclear of the ECOG PS in this case. Majority of registrars (n = 10) reported ECOG PS of 2 followed by ECOG PS 3 (n = 4) and ECOG PS 1 (n = 2). Six registrars were unclear about ECOG PS in this case. ECOG PS 2 and 3 was reported by 7 and 2 specialist nurses respectively and 1 unclear about the ECOG PS. These variations were not statistically significant (p = 0.621) (Fig. 2, 3).
Fig. 3.
Numbers of HCPs responses for the second scenario.
Oncology HCPs were then asked to comment on ECOG PS of a third clinical scenario involving a 46 year old lady with learning difficulties and new diagnosis of metastatic ovarian carcinoma. Her other past medical history included depression and inflammatory bowel disease. She lived in a residential care home and spent most of her daytime in a chair while watching television. She needed encouragement to walk. Majority of respondents rated this scenario as ECOG PS 3 (n = 33, 66%) and ECOG PS 2 (n = 12, 24%) followed by ECOG PS 1 (n = 1, 2%), ECOG PS 4 (n = 1, 2%) and unclear (n = 3, 6%). Fourteen of 18 consultants rated ECOG PS of 3 and 3 rated ECOG PS as 2. One consultant was unclear of the ECOG PS in this case. Majority of registrars (n = 12) reported ECOG PS of 3 followed by ECOG PS 2 (n = 7) and ECOG PS 4 (n = 1). Two registrars were unclear about ECOG PS in this case. ECOG PS 3, 2 and 1 was reported by 7, 2 and 1 specialist nurses respectively. Variations in assessment of ECOG PS for this scenario were also not statistically significant (p = 0.496) (Fig. 2, 4).
Fig. 4.
Numbers of HCPs responses for the third scenario.
The fourth clinical scenario was about a 52 year old gentleman with recent diagnosis of germ cell tumor. His past medical history included bilateral above knee leg amputations secondary to a road traffic accident. He was wheel chair bound for the last 3 years and spent most of his daytime in bed or chair. He was unable to transfer himself from chair to bed. The oncology HCPs were asked to comment on ECOG PS for this case. Majority of respondents rated this scenario as ECOG PS 3 (n = 19, 38%) and ECOG PS 4 (n = 14, 28%) followed by ECOG PS 2 (n = 7, 14%), ECOG PS 1 (n = 1, 2%) and unclear (n = 9, 18%). Eight of 18 consultants rated ECOG PS of 3 and 3 rated ECOG PS as 4 followed by ECOG PS 1 and 2 by one consultant each. Five consultants were unclear of the ECOG PS in this case. Majority of registrars (n = 8) reported ECOG PS of 3 followed by ECOG PS 4 (n = 6) and ECOG PS 2 (n = 4). Four registrars were unclear about ECOG PS in this case. ECOG PS of 4, 3 and 2 were reported by 5, 4 and 1 specialist nurses respectively. There were no statistically significant variations in assessment of PS among the responders (p = 0.878) for this scenario (Fig. 2, 5).
Fig. 5.
Numbers of HCPs responses for the fourth scenario.
Discussion
Performance status is an important tool used by oncology healthcare professionals to assess fitness of patients for systemic anticancer therapy and to predict prognosis in advanced malignancy. Variations in PS assessment can have significant impact on management decisions. In the context of advanced metastatic malignancy, fit patients with good PS are usually considered for systemic anti-cancer therapy. In patients with poor PS, the risks of treatment sometimes outweigh the benefits and best supportive is usually the most appropriate management option. WHO ECOG PS is considered shorter and easy to remember by the oncology HCPs and so fewer variations in the assessment among different oncology HCPs are expected. Our study demonstrated no significant variations in PS assessment by oncology HCPs, when using ECOG criteria. This was consistent across all four clinical scenarios. We found good correlation in PS rating assessment among oncologists, specialist trainee registrars and oncology specialist nurses. There were no significant intergroup variations among HCPs. Similar to our study, similarities in PS assessment among healthcare professionals have been reported in other studies. In a study reported by Taylor et al., 100 oncology patients were assessed for PS using KPS and ECOG PS scoring scale. The study concluded strong agreement in PS assessment among assessors. Like our study, there were fewer variations in the assessment with WHO ECOG PS scale as compared to KPS [5]. Inter-rater reliability of palliative performance scale (PPS) was assessed by Campos et al. in a study of 102 consecutive oncology patients. Strong correlation was observed between oncologists and research assistants (r = 0.86) and good correlation was observed between oncologists and radiation therapists (r = 0.69) [6]. Blagden et al. reported a moderate agreement in PS assessment between patients and oncologists in a study of 101 suspected lung cancer patients. Eight of 10 participating oncologist in this study were specialist registrars [7]. The results of our study correspond with the findings of the above studies.
Contrary to the studies reporting similarities or less variations in PS assessment, there are number of studies reporting significant variations in PS assessment among healthcare professionals. Kim et al. reported variations in ECOG PS assessment among medical oncologists, nurses and palliative care specialists, in a study of 278 patients. Palliative care specialists and nurses rated a worse PS as compared to medical oncologists but no significant difference in PS assessment between palliative care specialists and nurses were noted [8]. In a retrospective study of 223 lung cancer patients, May et al found significant difference in PS rating between multidisciplinary team (MDT) and oncologists with PS concordance of 39%. Results of this study might be confounded as MDT PS and oncologist PS were recorded at different times with mean time difference of 17.5 days [9]. Variations in the assessment of PS can occur due to number of factors including assessors background knowledge, training, level of awareness about patient's socioeconomic status and impact of assessment on future treatment decision. Oncologist may sometimes be biased in rating PS in situations where they believe a certain treatment can help. On the other hand, oncology specialist nurses who usually are more aware of patient's social situation may be more inclined for symptomatic care and poorer PS rating [10, 11]. A thorough understanding of patient's social status, level of activities of daily living, degree of dependence, quality of life and comorbidities is of paramount importance in rating PS as this can have significant impact on the outcome.
Medical literature does not have clear consensus on inter-rater reliability of PS assessment by different HCPs. Two studies reported better reliability for healthier PS scores [12, 13] while the other two reported better reliability for poorer PS scores [11, 14]. A systematic review of fifteen articles, published between 1946 and 2015, reported both good and bad inter-rater reliability of PS scores. It could not conclude, any specific HCPs PS assessments as more accurate. Three studies reported a lack of agreement (kappa = 0.19–0.26; Krippendorff's alpha = 0.61–0.63), four reported moderate inter-rater reliability (kappa = 0.31–0.72), two reported mixed reliability, and six reported strong reliability (kappa = 0.91–0.92; Spearman rank correlation = 0.6–1.0; Kendall's correlation = 0.75–0.82) [15]. Regarding the PS tools, four studies reported KPS as having better inter-rater reliability as opposed to ECOG PS and PPS [10, 13, 16, 17] while only one study reported the ECOG PS has a better agreement than KPS [5].
Our study found an ECOG PS scoring system as an effective tool to assess functional status and no significant variations in PS assessment by different healthcare professionals using this tool were noted. The small sample size of our study might be the limiting factor but our results indicated reliability and consistency in PS assessment using ECOG PS scoring system.
Conclusion
Our study shows no significant variations in PS assessment by oncology healthcare professionals. Eastern Cooperative Oncology Group scoring system is an effective and reliable tool to assess PS in oncology patients.
Statement of Ethics
The study protocol was approved by the institute's committee on human research.
Informed consent is not applicable in this study given it's a survey conducted by health care professionals for 4 hypothetical clinical scenarios.
Disclosure Statement
We declare that authors have no conflict of interest.
Funding Sources
We declare that authors received no funds for this research.
References
- 1.Karnofsky David A, Walter H, Abelmann, Lloyd F, Burchenal Craver Joseph H, The use of the nitrogen mustards in the palliative treatment of carcinoma With particular reference to bronchogenic carcinoma. Cancer. 1948 Nov;1((4)):634–56. [Google Scholar]
- 2.Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982 Dec;5((6)):649–55. [PubMed] [Google Scholar]
- 3.Anderson F, Downing GM, Hill J, Casorso L, Lerch N. Palliative performance scale (PPS): a new tool. J Palliat Care. 1996;12((1)):5–11. [PubMed] [Google Scholar]
- 4.Ciara M. Kelly, Armin Shahrokni, Moving beyond Karnofsky and ECOG Performance Status Assessments with New Technologies. J Oncol. 2016 Mar;2016((3)):1–13. doi: 10.1155/2016/6186543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Taylor AE, Olver IN, Sivanthan T, Chi M, Purnell C. Observer error in grading performance status in cancer patients. Support Care Cancer. 1999 Sep;7((5)):332–5. doi: 10.1007/s005200050271. [DOI] [PubMed] [Google Scholar]
- 6.Campos S, Zhang L, Sinclair E, Tsao M, Barnes EA, Danjoux C, et al. The palliative performance scale: examining its inter-rater reliability in an outpatient palliative radiation oncology clinic. Support Care Cancer. 2009 Jun;17((6)):685–90. doi: 10.1007/s00520-008-0524-z. [DOI] [PubMed] [Google Scholar]
- 7.Blagden SP, Charman SC, Sharples LD, Magee LR, Gilligan D. Performance status score: do patients and their oncologists agree? Br J Cancer. 2003 Sep;89((6)):1022–7. doi: 10.1038/sj.bjc.6601231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kim YJ, Hui D, Zhang Y. Difference in performance status assessments between palliative care specialists, nurses and oncologists. J Pain Symptom Manage, 2015;49:1050–1058. doi: 10.1016/j.jpainsymman.2014.10.015. e2. 26. [DOI] [PubMed] [Google Scholar]
- 9.May CH, Lester JF, Lee S. PS score discordance and why it matters. Lung Cancer. 2012;75:S18. [Google Scholar]
- 10.Zimmermann C, Burman D, Bandukwala S, Seccareccia D, Kaya E, Bryson J, et al. Nurse and physician inter-rater agreement of three performance status measures in palliative care outpatients. Support Care Cancer. 2010 May;18((5)):609–16. doi: 10.1007/s00520-009-0700-9. [DOI] [PubMed] [Google Scholar]
- 11.Campos S, Zhang L, Sinclair E, Tsao M, Barnes EA, Danjoux C, et al. The palliative performance scale: examining its inter-rater reliability in an outpatient palliative radiation oncology clinic. Support Care Cancer. 2009 Jun;17((6)):685–90. doi: 10.1007/s00520-008-0524-z. [DOI] [PubMed] [Google Scholar]
- 12.Sørensen JB, Klee M, Palshof T, Hansen HH. Performance status assessment in cancer patients. An inter-observer variability study. Br J Cancer. 1993 Apr;67((4)):773–5. doi: 10.1038/bjc.1993.140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Roila F, Lupattelli M, Sassi M, Basurto C, Bracarda S, Picciafuoco M, et al. Intra and interobserver variability in cancer patients' performance status assessed according to Karnofsky and ECOG scales. Ann Oncol. 1991 Jun;2((6)):437–9. doi: 10.1093/oxfordjournals.annonc.a057981. [DOI] [PubMed] [Google Scholar]
- 14.Liem BJ, Holland JM, Kang MY, Hoffelt SC, Marquez CM. Karnofsky Performance Status assessment: resident versus attending. J Cancer Educ. 2002;17((3)):138–41. doi: 10.1080/08858190209528821. [DOI] [PubMed] [Google Scholar]
- 15.Chow R, Chiu N, Bruera E, Krishnan M, Chiu L, Lam H, et al. Inter-rater reliability in performance status assessment among health care professionals: a systematic review. Ann Palliat Med. 2016 Apr;5((2)):83–92. doi: 10.21037/apm.2016.03.02. [DOI] [PubMed] [Google Scholar]
- 16.de Borja MT, Chow E, Bovett G, Davis L, Gillies C. The correlation among patients and health care professionals in assessing functional status using the karnofsky and eastern cooperative oncology group performance status scales. Support Cancer Ther. 2004 Oct;2((1)):59–63. doi: 10.3816/SCT.2004.n.024. [DOI] [PubMed] [Google Scholar]
- 17.Conill C, Verger E, Salamero M. Performance status assessment in cancer patients. Cancer. 1990 Apr;65((8)):1864–6. doi: 10.1002/1097-0142(19900415)65:8<1864::aid-cncr2820650832>3.0.co;2-u. [DOI] [PubMed] [Google Scholar]