Abstract
Importance
Persons living with serious illness often need skilled symptom management, communication, and spiritual support. Palliative care addresses these needs and may be delivered by either specialists or clinicians trained in other fields. It is important to understand core elements of palliative care to best provide patient-centered care.
Objective
To describe frequency, predictors, and timing of core elements of palliative care during the last 6 months of life.
Design
Retrospective chart review.
Setting
Inpatient academic medical center.
Participants
Decedents with cancer, dementia, or chronic kidney disease (CKD) admitted during the 6 months preceding death.
Exposures
We identified receipt and timing of core elements of palliative care: pain and symptom management, goals of care, spiritual care; and specialty palliative care utilization; hospital encounters; demographics; and comorbid diagnoses. We ran Poisson regression models to assess whether diagnosis or hospital encounters were associated with core elements of palliative care.
Results
Among 402 decedents, the mean (SD) number of appropriately screened and treated symptoms was 2.9 (1.7)/10. Among 76.1% with documented goals of care, 58.0% had a primary goal of comfort; 55.0% had documented spiritual care. In multivariable models, compared with decedents with cancer, those with dementia or CKD were less likely to have pain and symptom management (respectively, 31% (incidence rate ratio [IRR], 0.69; 95% CI, 0.56–0.85) and 17% (IRR, 0.83; CI, 0.71–0.97)). There was a median of 3 days (IQR, 0–173) between transition to a goal of comfort and death, and a median of 12 days (IQR, 5–47) between hospice referral and death.
Conclusions and Relevance
Although a high proportion of patients received elements of palliative care, transitions to a goal of comfort or hospice happened very near death. Palliative care delivery can be improved by systematizing existing mechanisms, including prompts for earlier goals-of-care discussion, symptom screening, and spiritual care, and by building collaboration between primary and specialty palliative care services.
INTRODUCTION
Palliative care is healthcare provided with a goal to reduce suffering and improve quality of life for patients with serious illness and their families.1 The National Academy of Medicine endorses palliative care for all patients with serious illness, who often experience significant symptom burden, receive intensive treatments that often do not align with their values, and have high healthcare costs.2 From a public health perspective, palliative care may simultaneously honor patient values and preferences while reducing costs.3
A national shortage and maldistribution of board-certified, palliative care specialists limits access to specialty palliative care services.4 Robust delivery of the elements of palliative care by non-specialists—termed primary palliative care—may address needs and improve quality of life.3 Much remains unknown about access to elements of palliative care—regardless of delivery clinician—for persons with serious illness, including which elements of palliative care they receive and when relative to the time of death.2, 5–7
In this study, we examine how often core elements of palliative care are provided to patients with one or more of three serious illnesses. Among decedents with late-stage cancer, dementia, or CKD hospitalized in the last 6 months of life, we describe the frequency, predictors, and timing of core elements of palliative care (pain and symptom management, goals of care, spiritual care) during the last 6 months of life.
METHODS
Study Sample and Setting
This retrospective chart review study was conducted at the University of North Carolina (UNC) Medical Center, a 929-bed teaching hospital complex with a specialty palliative care consult service of an attending physician, two advance practice providers, one social worker, and one chaplain. We identified three cohorts of decedents based on a diagnosis of either late-stage cancer, dementia, or CKD recorded in the electronic health record (EHR). We selected these conditions based on their high palliative care need.8–13 Starting with a date of death on 12/31/2017 back through 2017, we included consecutive patients until each group had 134 decedents.
Decedents were eligible if they had: (1) late-stage disease defined as stage 4 solid tumor cancer, stages 5, 6, or 7 dementia using the Global Deterioration Scale (GDS; respectively, moderate, moderately severe, and severe dementia), or stage 4–5 CKD;14 decedents with more than one of these three diagnoses were included in the group of their most life-limiting diagnosis based on admission diagnosis and cause of death; (2) date of death between 1/12/2017 and 12/31/2017; and (3) an index admission consisting of the first acute, non-planned hospitalization lasting > 24 h during the 6 months preceding death. They were excluded if they were prisoners at admission, were < 18 years of age, or had been at an outside hospital for > 24 h before transfer to UNC Medical Center. All study procedures were approved by the UNC Institutional Review Board.
Selection of Serious Illness Cohorts
We selected cancer, dementia, and CKD. Patients with cancer may experience uncontrolled symptoms and receive treatments that do not necessarily match their goals, despite integration of palliative care and cancer care.8–11 Patients with late-stage dementia often experience fragmented care and frequent care transitions resulting in much of their interaction with physicians occurring during hospitalizations. Patients with late-stage CKD face two critical decision points (whether to start dialysis, whether/when to withdraw dialysis), in addition to complex symptom management.
Identifying Cohorts
We identified decedents by applying EHR phenotypes—structured algorithms consisting of International Classification of Disease 9th and 10th edition codes (ICD-9 and ICD-10), hospitalization dates, laboratory values, and vital status—to the patient data warehouse based on the UNC inpatient and outpatient EHR. Vital status was based on death certificate data for the state of North Carolina. The EHR phenotypes were validated for each of our three disease group populations.15
Data Collection: Chart Reviews
We conducted structured, systematic chart reviews to abstract data from the UNC inpatient and outpatient EHR for each decedent in the 6 months preceding their index admission through death.16 We abstracted demographic and clinical characteristics, encounter details, utilization information, elements of palliative care, and timing relative to death.
Two experienced and trained chart abstractors (NCE and KLW) abstracted data from the same set of 10–15 decedents (across all three disease states) and compared results, adjudicating any discrepancies. Decisions were logged in an operation guide to support consistency over time. They had high inter-rater reliability (kappa = 0.84). One reviewer (NCE) conducted the remaining chart reviews with frequent discussion.
Other Measures
Demographic and Clinical Characteristics
We collected date of death, age at death, gender, race, ethnicity, insurance status, length of index admission, days between index admission and death, and discharge diagnosis. We collected patient communication barriers that might limit decision-making. We assessed comorbidities pre-dating the index admission using the Charlson Comorbidity Index (CCI; range 0–37; higher scores indicate higher disease burden) and adjusted for primary disease group.17 We recorded cancer type (e.g., lung, breast); dementia stage (GDS 5, 6, or 7); CKD stage (4 or 5); and treatments (dialysis, prior transplant, or waiting for transplant).
Hospital Encounters
We measured the total number of emergency department (ED) and unplanned hospital admissions in the 180 days preceding the index hospitalization. Using the Happy Together module in Epic’s Care Everywhere, we could view notes for encounters at other hospitals that use Epic (including other large, local systems and hospitals across the state) when those records had been requested by a clinician at UNC Medical Center for clinical care.
Specialty Palliative Care Consults
We captured presence of specialty palliative care consults between index admission and death, and date of the first consult.
Outcome Measures: Elements of Palliative Care
We described the frequency, predictors, and timing of three nationally endorsed elements of high-quality palliative care: pain and symptom management, goals of care, and spiritual care using standard measures to capture their delivery.18
Pain and Symptom Management
Within 2 days of the index admission date, we reviewed physician and advance practice provider (APP) notes and the medication record to extract whether each of ten symptoms (pain, dyspnea, constipation, nausea/vomiting, appetite, fatigue, depression, anxiety, hypoactive delirium/decreased level of consciousness, and hyperactive delirium/agitation) 19, 20 was (1) screened, (2) identified as an active problem, and (3) treated. We defined appropriate screening and treatment as either (1) screened and absent or (2) screened and treated, if present. We calculated the composite Pain and Symptom Management Index, a 10-point score where one point was given for each symptom that met this definition of appropriate screening and treatment.21
Goals of Care
Presence of goals of care was operationalized as documented evidence of discussion about the primary goal of care in the index admission History and Physical (baseline) and physician or APP notes (subsequent), including the primary goal of care (i.e., prolong life, support function, or improve comfort); major disease-directed treatment decisions (cancer: chemotherapy/radiation; CKD: dialysis/continuous renal replacement therapy); and hospice. We also captured the time between death and (1) the first documented goals-of-care discussion, (2) transition to a primary goal of comfort, and (3) hospice referral.
We recorded documented discussions of prognosis; code status; presence or absence of an advance care planning (ACP) note; and surrogate decision-maker name and contact information.
Spiritual Care
Presence of a spiritual care treatment plan was operationalized by documentation of screening for spiritual needs during or after the index admission (e.g., desire to see hospital chaplaincy or community spiritual care provider) and addressing spiritual care needs when appropriate. We searched for chaplaincy notes and key terms in notes from all provider types between the index admission date and date of death to determine if spiritual care was screened or addressed.
Analysis
We conducted univariate analyses of all variables, stratified by disease group, and bivariate analyses to assess for association of each element of palliative care and decedent demographics, primary diagnosis, frequency of hospitalization, and receipt of a specialty palliative care consult. All statistical tests were two-tailed with a 0.05 significance level.
For multivariable models, we hypothesized three factors would predict receipt of elements of palliative care near the end of life.22 First is primary diagnosis, as palliative care principles may be differentially integrated across diagnoses, and some diagnoses may have differential demand for specific elements of palliative care. Cancer is the reference group because patients with cancer often have more access to palliative care than other disease groups. Second, prior hospital encounters may reflect uncontrolled symptoms, lack of advance care planning, and/or disease progression, which may drive delivery of palliative care; for analytic purposes, we defined hospital encounters as ED visits or unplanned hospital admissions. Third, we included patients treated by palliative care specialists, which increases the likelihood of patients receiving elements of palliative care.
We used Poisson regression to estimate the association of primary diagnosis, the number of hospital encounters in the 6 months prior to the index hospitalization, and specialty palliative care with the Pain and Symptom Management Index. Second, we used modified Poisson regression models for the presence or absence of goals-of-care documentation and, secondarily, an ACP note or surrogate decision-maker information. Finally, we used modified Poisson regression to predict the presence or absence of spiritual care assessment, again with the same predictors. Results are interpreted as incidence rate ratios (IRR).23, 24 For all outcomes, we ran exploratory models stratified by disease group. All analyses controlled for age, race, gender, insurance status, unadjusted CCI, and the number of days between the index admission and date of death. Disease-specific exploratory results also controlled for admitting clinical service, disease stage for dementia and CKD (where earlier stage was the reference group), and cancer type in the cancer-specific models. We ran separate analyses for the three primary outcomes and two secondary outcomes.
Power
Sample size for the full cohort was calculated for 0.80 power using baseline data for the primary outcomes: pain and symptom management, goals-of-care discussion, and spiritual needs assessment, with effect sizes of 21%, 23%, and 8%, respectively.7, 25, 26 After adjusting the sample size for the number of variables in the model, the endpoint requiring the largest sample was spiritual needs assessment (N = 392).27
RESULTS
Our sample consisted of 402 eligible decedents with late-stage cancer, dementia, or CKD (134/group). Seven decedents were excluded due to transfer after > 24 h at an outside hospital. Decedents were 53% female, tended to be White (61.2%), and had a mean age of 72 years. The median time from index admission to death was 55 days. Decedents with CKD tended to have a higher comorbidity burden on average (adjusted CCI 3.7) than those with dementia (adjusted CCI 2.6) or cancer (adjusted CCI 1.2). Of the full sample, 53.0% of decedent had a communication barrier, including all decedents with dementia, 17.2% in cancer, and 52.3% in CKD (Table 1).
Table 1.
Decedent Characteristics
| Variable, n (%) | Cancer | Dementia | CKD | Total |
|---|---|---|---|---|
| N = 134 | N = 134 | N = 134 | N = 402 | |
| Age, mean (SD; range) | 64.4 (12.5; 29–88) | 83.7 (10.5; 23–102) | 66.4 (14.6; 27–91) | 71.5 (15.3; 23–102) |
| Gender, female | 76 (56.7) | 78 (58.2) | 61 (45.5) | 215 (53.8) |
| Race | ||||
| White | 85 (63.4) | 91 (67.9) | 70 (52.2) | 246 (61.2) |
| Black | 42 (31.3) | 36 (26.9) | 58 (43.3) | 136 (33.8) |
| Other | 7 (5.2) | 7 (5.2) | 6 (4.5) | 20 (49.8) |
| Ethnicity, Hispanic/Latino | 3 (2.2) | 0 | 4 (3.0) | 7 (1.7) |
| Insurance status (some pts > 1) | ||||
| Private | 58 (43.3) | 25 (18.7) | 34 (25.4) | 117 (29.1) |
| Medicare | 76 (56.7) | 130 (97.0) | 116 (86.6) | 322 (80.1) |
| Medicaid | 25 (18.7) | 21 (15.7) | 33 (24.6) | 79 (19.7) |
| Tricare | 6 (4.5) | 7 (5.2) | 6 (4.5) | 19 (4.7) |
| Uninsured | 9 (6.7) | 0 | 3 (2.2) | 12 (3.0) |
| Index length of stay, median days (range) | 5 (1–33) | 5 (1–88) | 7 (1–80) | 5 (1–88) |
| Time from index admit to death, median days (range) | 58.5 (1–175) | 50 (2–179) | 55 (1–180) | 55 (1–180) |
| Primary reason for index admission | ||||
| Acute medical problem | 46 (34.3) | 56 (41.8) | 64 (47.8) | 166 (41.3) |
| Infection | 17 (12.7) | 45 (33.6) | 37 (27.6) | 99 (24.6) |
| Uncontrolled symptoms | 57 (42.5) | 13 (9.7) | 9 (6.7) | 79 (19.7) |
| AMS/confusion/delirium | 4 (3.0) | 11 (8.1) | 8 (6.0) | 23 (5.7) |
| Exacerbation of chronic illness | 5 (3.7) | 3 (2.2) | 14 (10.5) | 22 (5.5) |
| Failure to thrive/nutritional insufficiency | 5 (3.7) | 6 (4.5) | 2 (1.5) | 13 (3.2) |
| Patient communication barriers | 23 (17.2) | 134 (100) | 56 (41.8) | 213 (53.0) |
| None | 111 (82.8) | 0 | 78 (58.2) | 189 (47.0) |
| Dementia | 1 (0.8) | 134 (100) | 9 (6.7) | 144 (35.8) |
| Confusion/AMS (> ½ of index) | 13 (9.7) | 26 (19.4) | 29 (21.6) | 68 (16.9) |
| Sedation/intubation | 6 (4.5) | 9 (6.7) | 19 (14.2) | 34 (8.5) |
| Hearing impairment | 0 | 3 (2.2) | 2 (1.5) | 5 (1.2) |
| Language discordance | 2 (1.5) | 0 | 1 (0.8) | 3 (0.8) |
| Charlson Comorbidity Index (CCI), mean (SD) | 7.2 (1.5) | 3.6 (2.4) | 5.71 (2.0) | 5.51 (2.5) |
| Stage 4 cancer | 134 (100) | 0 | 1 (0.8) | 135 (33.6) |
| Dementia | 2 (1.5) | 134 (100) | 12 (3.0) | 148 (36.8) |
| Moderate/severe CKD | 8 (6.0) | 30 (22.4) | 134 (100) | 172 (42.8) |
Almost half (43.0%) of decedents had at least one hospital encounter (ED visit or hospitalization) in the 6 months before their index admission; the mean number of encounters was 2.20 (range 1–15). Decedents with CKD (50.8%) had more hospital encounters than those with cancer (45.5%) or dementia (32.8%). Thirty-five percent of decedents received a specialty palliative care consult during their inpatient stay. Decedents received their first palliative care consult a median 30 days before death.
Predictors of Receiving Each Element of Palliative Care
Pain and Symptom Management
As measured by the Pain and Symptom Management Index, the total cohort had a mean 2.9 (standard deviation [SD] 1.7) symptoms appropriately treated (either screened negative or screened positive and treated): 3.6 (SD 1.9) in cancer, 2.4 (SD 1.5) in dementia, and 2.8 (SD 1.4) in CKD. Pain was the most commonly screened and appropriately treated symptom (81.1%), followed by nausea (57.2%), dyspnea (47.3%), and constipation (39.8%) (appropriate screening and treatment in Figure 1).
Figure 1.
Pain and symptom management displayed as the mean rates of appropriate pain and symptom management stratified by disease groups based on a definition of either (1) screened negative or (2) screened positive and treated.
Goals of Care
By the time of death, 76.1% of decedents had documented goals of care; among those, 76.1% indicated a primary goal of improving comfort—compared with supporting function (17.3%) or prolonging life (2.9%). There was a median 3 days (interquartile range [IQR] 0–173) between transition to a primary goal of comfort and death. Among the 233 decedents with a primary goal of comfort, 86 (36.9%) transitioned to a primary goal of comfort in the last day of life. Among those who discussed disease-directed treatment, 37.6% of decedents with cancer decided not to receive chemotherapy or radiation; disease-directed treatments were not options for 31.2% of decedents with cancer by the time of death. For CKD, 37.4% of decedents who discussed disease-specific treatment decided not to use dialysis or continuous renal replacement therapy. Among all decedents, 41.8% were referred to hospice (84.9% who had documentation of a hospice discussion). The median number of hospice days before death was 12 (IQR 5–47). Among those who were referred to hospice (n = 168), 32 (16.2%) enrolled in the last 3 days of life (Table 2).
Table 2.
Goals of Care and Decision-making (N= 402). Tabs Indicate the Denominator Reflects the Superseding Variable
| Variable, n (%) | Initial (N = 402) | Final (N = 402) |
|---|---|---|
| Overall goal of care | ||
| Documentation | 45 (11.19) | 306 (76.12) |
| Discussion | 34 (75.56) | 303 (99.02) |
| Specify | ||
| Prolong life | 5 (11.11) | 9 (2.94) |
| Limit/support function | 20 (44.44) | 53 (17.32) |
| Improve comfort | 12 (26.67) | 233 (76.14) |
| If comfort, days before death, median (IQR) | 3 (1–14) | |
| Transition to comfort in last day of life | 86 (36.91) | |
| No decision | 8 (17.78) | 11 (3.59) |
| Disease-directed treatment options (n = 134 patients with cancer) | ||
| Documentation: Chemotherapy/radiation | 78 (58.21) | 125 (93.28) |
| Discussion | 13 (16.67) | 101 (80.80) |
| Specify | ||
| No decision | 17 (21.79) | 11 (8.80) |
| Use disease-directed therapy | 50 (64.10) | 28 (22.40) |
| Do not use disease-directed therapy | 7 (8.97) | 47 (37.60) |
| Not a treatment option | 4 (5.13) | 39 (31.20) |
| Disease-directed treatment options (n = 134 patients with CKD) | ||
| Documentation: Dialysis/continuous renal replacement therapy | 83 (61.94) | 115 (85.82) |
| Discussion | 11 (13.25) | 61 (53.04) |
| Specify | ||
| No decision | 7 (8.43) | 5 (4.35) |
| Use disease-directed therapy | 68 (81.93) | 61 (53.04) |
| Do not use disease-directed therapy | 5 (6.02) | 43 (37.39) |
| Not a treatment option | 3 (3.61) | 6 (5.22) |
| Hospice | ||
| Documentation | 28 (6.97) | 257 (63.93) |
| Discussion | 14 (50.00) | 244 (94.94) |
| Specify | ||
| Acceptable | 12 (42.86) | 216 (84.05) |
| Undecided | 13 (46.43) | 28 (10.89) |
| Do not use | 3 (10.71) | 13 (5.06) |
| Enrolled at baseline | 5 (17.86) | |
| Referral to hospice | 168 (84.85) | |
| If yes, days before death, mean/median (IQR) | 12 (5–47) | |
| Referral to hospice in last 3 days of life | 32 (16.16) | |
| Prognosis | ||
| Documentation | 20 (4.98) | 263 (65.42) |
| Discussion | 14 (70.00) | 257 (97.72) |
| Code status | ||
| Documentation | 389 (96.77) | 402 (100.00) |
| Discussion | 124 (31.88) | 316 (78.61) |
| Specify | ||
| Full code | 266 (68.38) | 101 (25.12) |
| Limited code | 11 (2.83) | 10 (2.49) |
| DNR only | 5 (1.29) | 6 (1.49) |
| DNR/DNI | 107 (27.51) | 285 (70.90) |
Code status was either DNR or DNR/DNI for 27.9% of decedents at the time of index admission; by the time of death, that number increased to 72.39% (Table 2). ACP notes were on file for 46.3% of decedents by the time of death. Goals-of-care discussions occurred for 83.1% of all decedents. Surrogate decision-maker name and contact information was documented for 83.8% of all decedents (Table 3).
Table 3.
Advance Care Planning, Surrogate Decision-making, and Utilization
| Variable, n (%) | Cancer | Dementia | CKD | Total |
|---|---|---|---|---|
| N = 134 | N = 134 | N = 134 | N = 402 | |
| Any discussion of advance care planning | 125 (93.28) | 106 (79.10) | 103 (76.87) | 334 (83.08) |
| If any discussion, days before death, median (IQR) | 24 (7–59) | 16.5 (4–64) | 6 (1–23) | 15 (4–50) |
| ACP note | 70 (52.24) | 63 (47.01) | 53 (39.55) | 186 (46.27) |
| Living will | 16 (11.94) | 12 (8.96) | 11 (8.21) | 39 (9.70) |
| MOST/POLST | 4 (2.99) | 14 (10.45) | 4 (2.99) | 22 (5.47) |
| No discussion | 8 (5.97) | 25 (18.66) | 29 (21.64) | 62 (15.42) |
| Family refused | 0 (0.00) | 0 (0.00) | 1 (0.75) | 1 (0.25) |
| Surrogate name and contact information | 106 (79.10) | 120 (89.55) | 111 (82.84) | 337 (83.83) |
| Documented relationship to patient* | 107 (79.85) | 122 (91.04) | 113 (84.33) | 342 (85.07) |
| Spouse/life partner | 47 (35.07) | 32 (23.88) | 44 (32.84) | 123 (30.60) |
| Adult daughter | 26 (19.40) | 51 (38.06) | 30 (22.39) | 107 (26.62) |
| Adult son | 7 (5.22) | 32 (23.88) | 18 (13.43) | 57 (14.18) |
| Parent | 11 (8.21) | 1 (0.75) | 10 (7.46) | 22 (5.47) |
| Adult grandchild | 0 (0.00) | 1 (0.75) | 3 (2.24) | 4 (1.00) |
| Sibling | 16 (11.94) | 5 (3.73) | 6 (4.48) | 27 (6.72) |
| Other | 7 (5.22) | 6 (4.48) | 8 (5.97) | 21 (5.22) |
| Documented no surrogate | 16 (11.94) | 5 (3.73) | 6 (4.48) | 27 (6.72) |
| Spiritual care needs | ||||
| Screened | 82 (61.2) | 56 (41.8) | 83 (61.9) | 221 (55.0) |
| Desired (among screened) | 73 (89.0) | 52 (92.9) | 77 (92.8) | 202 (91.4) |
| Addressed (among screened and desired) | 72 (98.6) | 49 (94.2) | 75 (97.4) | 202 (97.0) |
| Saw a hospital spiritual care provider | 69 (51.5) | 46 (34.33) | 76 (56.72) | 191 (47.5) |
| Utilization | ||||
| Specialty palliative care consult | 71 (52.99) | 32 (23.88) | 42 (31.34) | 145 (36.07) |
| If consult, no. of visits, mean (range) | 4.4 (1–21) | 2.69 (1–7) | 2.21 (1–9) | 3.62 (1–21) |
| If consult, mean/median days before death (IQR) | 32 (12–73) | 39 (8.5–74.5) | 39 (4–51) | 30 (8–72) |
| Other palliative care services | ||||
| Outpatient oncology palliative care | 25 (18.66) | 0 (0.00) | 0 (0.00) | 25 (6.22) |
| Home-based serious illness care | 0 (0.00) | 7 (5.22) | 4 (2.99) | 11 (2.74) |
| Inpatient specialty palliative care, outside hospital | 2 (1.49) | 0 (0.00) | 0 (0.00) | 2 (0.50) |
| ED | ||||
| Any ED visits, 180 days before index until index | 34 (25.37) | 39 (29.10) | 36 (26.87) | 109 (27.11) |
| If yes, mean (range) | 1.74 (1–14) | 1.95 (1–9) | 1.89 (1–8) | 1.86 (1–14) |
| Hospitalization | ||||
| Any hospitalization, 180 days before index until index | 41 (30.60) | 14 (10.45) | 51 (38.06) | 106 (26.37) |
| If yes, mean (range) | 1.68 (1–9) | 1.28 (1–3) | 0.67 (0–11) | 1.76 (1–11) |
| Encounters (ED + hospitalization) | ||||
| Any encounter, 180 days before index until index | 61 (45.52) | 44 (32.84) | 68 (50.75) | 173 (43.03) |
| If yes, mean (range) | 2.10 (1–14) | 2.14 (1–9) | 2.32 (1–15) | 2.20 (1–15) |
*Sums to > 100%. Some patients had > 1 documented primary surrogate
Spiritual Care
Half (55.0%) of decedents were screened for spiritual care engagement, 91.4% of whom desired spiritual engagement. Nearly half (47.5%) of all decedents saw a hospital spiritual care provider.
Specialty palliative care was positively associated with receipt of pain and symptom management (IRR, 1.35; CI, 1.19–1.52), goals of care (IRR, 1.53; CI, 1.38–1.69), and spiritual care (IRR, 1.58; CI, 1.32–1.88). Decedents with cancer were more likely to have pain and symptom management than those with dementia (IRR, 0.69; CI, 0.56–0.85) or CKD (IRR, 0.83; CI, 0.71–0.97). Having more hospital encounters was associated with a higher likelihood of receiving pain and symptom management (IRR, 1.04; CI, 1.01–1.07) (Table 4).
Table 4.
Predictors of Elements of Palliative Care, Incidence Rate Ratio (IRR)
| Predictor | Unadjusted, bivariate IRR (CI) | Adjusted, full model IRR (CI) |
|---|---|---|
| Primary outcome: Pain and symptom management, full sample | ||
| Disease state (ref: cancer) | ||
| Dementia | 0.65 (0.57, 0.75)* | 0.69 (0.56, 0.85)* |
| CKD | 0.77 (0.68, 0.88)* | 0.83 (0.71, 0.97)* |
| Encounters | 1.05 (1.02, 1.08)* | 1.04 (1.01, 1.07)* |
| Palliative care consult | 1.47 (1.31, 1.64)* | 1.35 (1.19, 1.52)* |
| Primary outcome: Pain and symptom management, within cancer | ||
| Palliative care consult | 1.51 (1.25, 1.81)* | 1.39 (1.09, 1.77)* |
| Primary outcome: Pain and symptom management, within dementia | ||
| Disease stage (ref: GDS 5) | ||
| GDS 6 | 1.01 (0.80, 1.28) | 0.97 (0.75, 1.26) |
| GDS 7 | 0.75 (0.51, 1.10) | 0.73 (0.48, 1.11) |
| Palliative care consult | 1.43 (1.13, 1.82)* | 1.45 (1.11, 1.89)* |
| Primary outcome: Pain and symptom management, within CKD | ||
| CKD stage 5 (ref: stage 4) | 1.02 (0.82, 1.27) | 0.98 (0.75, 1.27) |
| Palliative care consult | 1.15 (0.93, 1.43) | 1.11 (0.88, 1.41) |
| Primary outcome: Goals of care, full sample | ||
| Disease state (ref: cancer) | ||
| Dementia | 0.94 (0.83, 1.06) | 1.03 (0.85, 1.25) |
| CKD | 0.87 (0.76, 1.00)* | 1.01 (0.88, 1.16) |
| Encounters | 0.99 (0.95, 1.03) | 1.00 (0.97, 1.03) |
| Palliative care consult | 1.49 (1.36, 1.64)* | 1.53 (1.38, 1.69)* |
| Primary outcome: Goals of care, within cancer | ||
| Palliative care consult | 1.42 (1.18, 1.70)* | 1.38 (1.12, 1.70)* |
| Primary outcome: Goals of care, within dementia | ||
| Disease stage (ref: GDS 5) | ||
| GDS 6 | 0.97 (0.78, 1.23) | 0.94 (0.75, 1.19) |
| GDS 7 | 1.27 (1.04, 1.55)* | 1.25 (0.97, 1.61) |
| Palliative care consult | 1.39 (1.21, 1.61)* | 1.48 (1.23, 1.78)* |
| Primary outcome: Goals of care, within CKD | ||
| CKD stage 5 (ref: stage 4) | 0.80 (0.65, 0.98)* | 0.91 (0.74, 1.12) |
| Palliative care consult | 1.74 (1.46, 2.07)* | 1.63 (1.33, 1.99)* |
| Primary outcome: Spiritual care, full sample | ||
| Disease state (ref: cancer) | ||
| Dementia | 0.68 (0.53, 0.87)* | 1.02 (0.75, 1.40) |
| CKD | 1.01 (0.84, 1.22) | 1.26 (1.02, 1.56)* |
| Encounters | 1.01 (0.96, 1.05) | 1.00 (0.95, 1.04) |
| Palliative care consult | 1.63 (1.38, 1.93)* | 1.58 (1.32, 1.88)* |
| Primary outcome: Spiritual care, within cancer | ||
| Palliative care consult | 1.81 (1.32, 2.47)* | 2.00 (1.41, 2.83)* |
| Primary outcome: Spiritual care, within dementia | ||
| Disease stage (ref: GDS 5) | ||
| GDS 6 | 0.74 (0.49, 1.12) | 0.77 (0.51, 1.17) |
| GDS 7 | 0.65 (0.32, 1.33) | 0.56 (0.29, 1.10) |
| Palliative care consult | 1.51 (1.01, 2.25)* | 1.76 (1.14, 2.71)* |
| Primary outcome: Spiritual care, within CKD | ||
| CKD stage 5 (ref: stage 4) | 0.96 (0.73, 1.27) | 0.87 (0.64, 1.19) |
| Palliative care consult | 1.44 (1.13, 1.85)* | 1.37 (1.01, 1.85)* |
| Secondary outcome: Advance care planning, full sample | ||
| Disease state (ref: cancer) | ||
| Dementia | 0.90 (0.71, 1.15) | 1.57 (1.13, 2.19)* |
| CKD | 0.76 (0.58, 0.99)* | 1.07 (0.82, 1.39) |
| Encounters | 1.01 (0.96, 1.07) | 1.01 (0.95, 1.07) |
| Palliative care consult | 2.62 (2.12, 3.24)* | 2.64 (2.12, 3.28)* |
| Secondary outcome: Advance care planning, within cancer | ||
| Palliative care consult | 3.25 (2.05, 5.16)* | 2.95 (1.79, 4.87)* |
| Secondary outcome: Advance care planning, within dementia | ||
| Disease stage (ref: GDS 5) | ||
| GDS 6 | 0.71 (0.47, 1.06) | 0.65 (0.45, 0.96)* |
| GDS 7 | 1.25 (0.82, 1.92) | 1.05 (0.62, 1.79) |
| Palliative care consult | 2.39 (1.77, 3.24)* | 2.52(1.80, 3.54)* |
| Secondary outcome: Advance care planning, within CKD | ||
| CKD stage 5 (ref: stage 4) | 0.74 (0.49, 1.13) | 0.82 (0.54, 1.25) |
| Palliative care consult | 2.65 (1.77, 3.95)* | 2.64 (1.64, 4.23)* |
| Secondary outcome: Surrogate decision-maker name and contact information, full sample | ||
| Disease state (ref: cancer) | ||
| Dementia | 1.13 (1.02, 1.26)* | 1.23 (1.07, 1.43)* |
| CKD | 1.05 (0.93, 1.18) | 1.15 (1.02, 1.30)* |
| Encounters | 1.01 (0.98, 1.03) | 1.00 (0.97, 1.03) |
| Palliative care consult | 1.26 (1.17, 1.36)* | 1.32 (1.22, 1.42)* |
| Secondary outcome: Surrogate decision-maker name and contact information, within cancer | ||
| Palliative care consult | 1.46 (1.20, 1.79)* | 1.41 (1.16, 1.73)* |
| Secondary outcome: Surrogate decision-maker name and contact information, within dementia | ||
| Disease stage (ref: GDS 5) | ||
| GDS 6 | 1.02 (0.90, 1.16) | 0.97 (0.85, 1.10) |
| GDS 7 | 0.93 (0.74, 1.17) | 0.92 (0.72, 1.16) |
| Palliative care consult | 1.16 (1.07, 1.25)* | 1.19 (1.07, 1.32)* |
| Secondary outcome: Surrogate decision-maker name and contact information, within CKD | ||
| CKD stage 5 (ref: stage 4) | 0.80 (0.70, 0.92)* | 0.89 (0.77, 1.04) |
| Palliative care consult | 1.33 (1.18, 1.50)* | 1.32 (1.14, 1.53)* |
We controlled for age, race, gender, insurance status, unadjusted Charlson Comorbidity Index (CCI), and time from index admission date until date of death. For disease-specific models, we controlled for admitting clinical service and cancer type in the cancer-specific models
*statistically significant
DISCUSSION
This study highlighted three important findings about palliative care for patients with serious illness. First, although elements of palliative care are being delivered, delivery is inconsistent. Second, goals of care, transitions to comfort, and referral to hospice generally happened late in the disease trajectory. Third, this study highlights gaps where systematic delivery of palliative care could be improved, including earlier goals-of-care discussions and systematic symptom screening.
Pain and symptom management, goals-of-care discussions, and spiritual care were each delivered at higher rates than the frequency of specialty palliative care consults. This suggests that non-specialty palliative care clinicians are recognizing palliative care need and addressing those needs, often consulting specialty services. Comfort and hospice transitions are happening very near death, suggesting that recognizing the need for palliative care occurs later than is ideal with guidelines typically recommending these services up to 3 months before death.28–30
Goals-of-care discussions, transitions to comfort, and hospice referral generally occurred near the end of life which is problematic because patients may not have enough time to fully benefit from hospice before they die.31 Starting conversations earlier in the disease trajectory may introduce and improve patient-centered care and decisions that align with patient preferences for several reasons.32 Early engagement can provide a picture of potential trajectories and help with long-term planning, help patients make decisions consistent with the goals over time, and gradually introduce decision-making as a disease progresses.33
One common reason for admission was uncontrolled symptoms. This may result from under-reporting and inadequate screening and management.34–36 Systematically screening for and treating common symptoms is likely a reasonable expectation for inpatient services treating patients with serious illness.37 Some services address this problem in part by using PRN (pro re nata; as needed) orders for medications to manage symptoms. Beyond symptom screening, EHRs also aid systemization by, for example, triggering spiritual care upon transitions to comfort care.38
Both specialty and primary palliative care move outcomes, though head-to-head trials have not been conducted to evaluate their relative effectiveness. The continued development and testing of different types and innovative models of palliative care delivery is essential to meet the needs of patients with serious illness.3 Collaborative models of palliative care combine specialty and primary palliative care services to meet needs while considering the limited pool of specialists.39 Such models can incorporate training of discrete skills. For example, ACP was often documented outside of ACP notes, making it difficult for clinicians to find and use when making decisions. Facilitating and teaching use of ACP note templates or flags is one mechanism to document work that is already being done in a readily available format. Champions for palliative care or specific elements thereof (e.g., symptom management, ACP, goals-of-care discussions) offer additional promising approaches to increasing access.40
There are several limitations of this study. First, although we were adequately powered for our primary and secondary outcomes, we were underpowered to examine each disease group. Second, this study was conducted at one public, academic medical center. Although the decedent population is diverse, generalizability is limited. Third, this study was limited to information in the UNC EHR and via Care Everywhere, which may be incomplete, and services may differentially document. We did not have access to data from other regional sources. EHR documentation is the primary way clinicians communicate with each other over time; therefore, assessing goals of care and decision-making via the EHR is a reasonable approach to estimate the reality of inter-clinician communication. Incomplete records are more pronounced for decedents who frequently sought care at other institutions. Lack of documentation in physician and APP notes was likely pronounced for symptoms that were screened and not present; nursing notes would likely have more comprehensive symptom assessment. Under-reporting of symptoms may occur most frequently among subjects with dementia, which may lead to undertreatment of symptoms.41
We found that elements of palliative care are being delivered for decedents with serious illness, but goals-of-care discussions, transitions to comfort care, and referral to hospice all happen very near death. Palliative care delivery can be improved by systematizing existing mechanisms, including prompts for earlier goals-of-care discussion, symptom screening, and spiritual care, and by building strong collaboration between clinicians primarily caring for patients with late-stage serious illness and specialty palliative care. Providing elements of palliative care may support patient-centered care and improve quality of life for patients with serious illness and their families.
Compliance with Ethical Standards
All study procedures were approved by the UNC Institutional Review Board.
Footnotes
Key Points
Question: What are the frequency, predictors, and timing of core elements of palliative care (pain and symptom management, goals of care, spiritual care) during the last 6 months of life?
Findings: Palliative care is occurring, though inconsistently, for patients in the last 6 months of life. Transitions to a primary goal of comfort and hospice referral occur very late in disease trajectory.
Meaning: Palliative care delivery can be improved by systematizing existing mechanisms, including prompts for earlier goals-of-care discussion, symptom screening, and spiritual care, and by building collaboration between primary and specialty palliative care services.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Morrison RS, Meier DE. Palliative Care. N Engl J Med. 2004;350(25):2582–2590. doi: 10.1056/NEJMcp035232. [DOI] [PubMed] [Google Scholar]
- 2.Institute of Medicine (US). Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life : Health and Medicine Division; 2015. http://www.nationalacademies.org/hmd/Reports/2014/Dying-In-America-Improving-Quality-and-Honoring-Individual-Preferences-Near-the-End-of-Life.aspx. Accessed 26 November 2017.
- 3.Kavalieratos D, Corbelli J, Zhang D, et al. Association between palliative care and patient and caregiver outcomes: A systematic review and meta-analysis. JAMA J Am Med Assoc. 2016;316:20. doi: 10.1001/jama.2016.16840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lupu D. Estimate of Current Hospice and Palliative Medicine Physician Workforce Shortage. J Pain Symptom Manag. 2010;40(6):899–911. doi: 10.1016/J.JPAINSYMMAN.2010.07.004. [DOI] [PubMed] [Google Scholar]
- 5.Brooks GA, Abrams TA, Meyerhardt JA, et al. Identification of potentially avoidable hospitalizations in patients with GI cancer. J Clin Oncol. 2014;32(6):496–503. doi: 10.1200/JCO.2013.52.4330. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Manzano JGM, Luo R, Elting LS, George M, Suarez-Almazor ME. Patterns and predictors of unplanned hospitalization in a population-based cohort of elderly patients with GI cancer. J Clin Oncol. 2014;32(31):3527–3533. doi: 10.1200/JCO.2014.55.3131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ernecoff Natalie C., Curlin Farr A., Buddadhumaruk Praewpannarai, White Douglas B. Health Care Professionals’ Responses to Religious or Spiritual Statements by Surrogate Decision Makers During Goals-of-Care Discussions. JAMA Internal Medicine. 2015;175(10):1662. doi: 10.1001/jamainternmed.2015.4124. [DOI] [PubMed] [Google Scholar]
- 8.Wright AA, Keating NL, Ayanian JZ, et al. Family Perspectives on Aggressive Cancer Care Near the End of Life. JAMA. 2016;315(3):284. doi: 10.1001/jama.2015.18604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Solano JP, Gomes B, Higginson IJ. A comparison of symptom prevalence in far advanced cancer, AIDS, heart disease, chronic obstructive pulmonary disease and renal disease. J Pain Symptom Manag. 2006;31(1):58–69. doi: 10.1016/j.jpainsymman.2005.06.007. [DOI] [PubMed] [Google Scholar]
- 10.van den Beuken-van Everdingen M, de Rijke J, Kessels A, Schouten H, van Kleef M, Patijn J. Prevalence of pain in patients with cancer: a systematic review of the past 40 years. Ann Oncol. 2007;18(9):1437–1449. doi: 10.1093/annonc/mdm056. [DOI] [PubMed] [Google Scholar]
- 11.Singer AE, Meeker D, Teno JM, Lynn J, Lunney JR, Lorenz KA. Symptom trends in the last year of life from 1998 to 2010: a cohort study. Ann Intern Med. 2015;162(3):175–183. doi: 10.7326/M13-1609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kupeli N, Sampson EL, Harrington J, et al. Why is integrated care not working in end of life care for those with advanced dementia? From the health care professional perspective. BMJ Support Palliat Care. 2015;5(1):113.1–113. doi: 10.1136/bmjspcare-2014-000838.29. [DOI] [Google Scholar]
- 13.Murphy E, Germain MJ, Murtagh F. Palliative Nephrology: Time for New Insights. Am J Kidney Dis. 2017;70(5):593–595. doi: 10.1053/J.AJKD.2017.07.008. [DOI] [PubMed] [Google Scholar]
- 14.Reisberg B, Ferris SH, De Leon MJ, Crook T. The global deterioration scale for assessment of primary degenerative dementia. Am J Psychiatry. 1982;139(9):1136–1139. doi: 10.1176/ajp.139.9.1136. [DOI] [PubMed] [Google Scholar]
- 15.Ernecoff Natalie C., Wessell Kathryn L., Gabriel Stacey, Carey Timothy S., Hanson Laura C. A Novel Screening Method to Identify Late-Stage Dementia Patients for Palliative Care Research and Practice. Journal of Pain and Symptom Management. 2018;55(4):1152-1158.e1. doi: 10.1016/j.jpainsymman.2017.12.480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hanson LC, Zimmerman S, Song MK, et al. Effect of the Goals of Care Intervention for Advanced Dementia: A Randomized Clinical Trial. JAMA Intern Med. 2017;177(1):24–31. doi: 10.1001/jamainternmed.2016.7031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Charlson M, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol. 1994;47(11):1245–1251. doi: 10.1016/0895-4356(94)90129-5. [DOI] [PubMed] [Google Scholar]
- 18.Ferrell B, Connor SR, Cordes A, et al. The National Agenda for Quality Palliative Care: The National Consensus Project and the National Quality Forum. J Pain Symptom Manag. 2007;33:737–744. doi: 10.1016/j.jpainsymman.2007.02.024. [DOI] [PubMed] [Google Scholar]
- 19.McCorkle R, Young K. Development of a symptom distress scale. Cancer Nurs. 1978;1(5):373–378. doi: 10.1097/00002820-197810000-00003. [DOI] [PubMed] [Google Scholar]
- 20.McCorkle R, Quint-Benoliel J. Symptom distress, current concerns and mood disturbance after diagnosis of life-threatening disease. Soc Sci Med. 1983;17(7):431–438. doi: 10.1016/0277-9536(83)90348-9. [DOI] [PubMed] [Google Scholar]
- 21.Hanson LC, Gabriel S, Ernecoff NC, et al. Triggered Palliative Care in Advanced Dementia: A Pilot Study; 2017.
- 22.Stewart AL, Teno J, Patrick DL, Lynn J. The Concept of Quality of Life of Dying Persons in the Context of Health Care. J Pain Symptom Manag. 1999;17(2):93–108. doi: 10.1016/S0885-3924(98)00131-6. [DOI] [PubMed] [Google Scholar]
- 23.Zou G. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol. 2004;159(7):702–706. doi: 10.1093/aje/kwh090. [DOI] [PubMed] [Google Scholar]
- 24.Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol. 2005;162:199–200. doi: 10.1093/aje/kwi188. [DOI] [PubMed] [Google Scholar]
- 25.Hanson Laura C., Rowe Charlotte, Wessell Kathryn, Caprio Anthony, Winzelberg Gary, Beyea Annette, Bernard Stephen A. Measuring Palliative Care Quality for Seriously Ill Hospitalized Patients. Journal of Palliative Medicine. 2012;15(7):798–804. doi: 10.1089/jpm.2011.0471. [DOI] [PubMed] [Google Scholar]
- 26.Hanson Laura C., Collichio Frances, Bernard Stephen A., Wood William A., Milowsky Matt, Burgess Erin, Creedle Crista J., Cheek Summer, Chang Lydia, Chera Bhisham, Fox Alexandra, Lin Feng-Chang. Integrating Palliative and Oncology Care for Patients with Advanced Cancer: A Quality Improvement Intervention. Journal of Palliative Medicine. 2017;20(12):1366–1371. doi: 10.1089/jpm.2017.0100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Vittinghoff E, McCulloch CE. Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression. Am J Epidemiol. 2007;165(6):710–718. doi: 10.1093/aje/kwk052. [DOI] [PubMed] [Google Scholar]
- 28.Schockett ER, Teno JM, Miller SC, Stuart B. Late Referral to Hospice and Bereaved Family Member Perception of Quality of End-of-Life Care. J Pain Symptom Manag. 2005;30(5):400–407. doi: 10.1016/J.JPAINSYMMAN.2005.04.013. [DOI] [PubMed] [Google Scholar]
- 29.Scibetta C, Kerr K, Mcguire J, Rabow MW. The Costs of Waiting: Implications of the Timing of Palliative Care Consultation among a Cohort of Decedents at a Comprehensive Cancer Center. J Palliat Med. 2016;19(1):69–75. doi: 10.1089/jpm.2015.0119. [DOI] [PubMed] [Google Scholar]
- 30.Bickel KE, McNiff K, Buss MK, et al. Defining High-Quality Palliative Care in Oncology Practice: An American Society of Clinical Oncology/American Academy of Hospice and Palliative Medicine Guidance Statement. J Oncol Pract. 2016;12(9):e828–e838. doi: 10.1200/JOP.2016.010686. [DOI] [PubMed] [Google Scholar]
- 31.Earle CC, Landrum MB, Souza JM, Neville BA, Weeks JC, Ayanian JZ. Aggressiveness of cancer care near the end of life: is it a quality-of-care issue? J Clin Oncol. 2008;26(23):3860–3866. doi: 10.1200/JCO.2007.15.8253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Grubbs V, Moss AH, Cohen LM, et al. A Palliative Approach to Dialysis Care: A Patient-Centered Transition to the End of Life. Clin J Am Soc Nephrol. 2014;9:2203–2209. doi: 10.2215/CJN.00650114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Bernacki RE, Block SD. Communication About Serious Illness Care Goals. JAMA Intern Med. 2014;174(12):1994. doi: 10.1001/jamainternmed.2014.5271. [DOI] [PubMed] [Google Scholar]
- 34.Basch E, Kris MG, Scher HI, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. 2015;34:557–565. doi: 10.1200/JCO.2015.63.0830. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Hanson LC, Eckert JK, Dobbs D, et al. Symptom Experience of Dying Long-Term Care Residents. J Am Geriatr Soc. 2008;56(1):91–98. doi: 10.1111/j.1532-5415.2007.01388.x. [DOI] [PubMed] [Google Scholar]
- 36.Abdel-Kader K, Unruh ML, Weisbord SD. Symptom Burden, Depression, and Quality of Life in Chronic and End-Stage Kidney Disease. Clin J Am Soc Nephrol. 2009;4:1057–1064. doi: 10.2215/CJN.00430109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Pereira JL, Chasen MR, Molloy S, et al. Cancer Care Professionals’ Attitudes Toward Systematic Standardized Symptom Assessment and the Edmonton Symptom Assessment System After Large-Scale Population-Based Implementation in Ontario, Canada. J Pain Symptom Manag. 2016;51(4):662–672.e8. doi: 10.1016/J.JPAINSYMMAN.2015.11.023. [DOI] [PubMed] [Google Scholar]
- 38.Dexter PR, Perkins SM, Maharry KS, Jones K, McDonald CJ. Inpatient Computer-Based Standing Orders vs Physician Reminders to Increase Influenza and Pneumococcal Vaccination Rates. JAMA. 2004;292(19):2366. doi: 10.1001/jama.292.19.2366. [DOI] [PubMed] [Google Scholar]
- 39.University of Washington. Collaborative Care Principles | University of Washington AIMS Center. http://aims.uw.edu/collaborative-care/principles-collaborative-care. Published 2018. Accessed 9 November 2018.
- 40.Witkamp Frederika Erica, van Zuylen Lia, van der Rijt Carin CD, van der Heide Agnes. Effect of palliative care nurse champions on the quality of dying in the hospital according to bereaved relatives: A controlled before-and-after study. Palliative Medicine. 2015;30(2):180–188. doi: 10.1177/0269216315588008. [DOI] [PubMed] [Google Scholar]
- 41.Trotti A, Colevas AD, Basch E, Colevas AD, Setser A. Patient-Reported Outcomes and the Evolution of Adverse Event Reporting in Oncology. Artic J Clin Oncol. 2007;25:5121–5127. doi: 10.1200/JCO.2007.12.4784. [DOI] [PubMed] [Google Scholar]