Predictors of documented goals-of-care discussion for hospitalized patients with chronic illness

Abstract

Context:

Goals-of-care discussions are important for patient-centered care among hospitalized patients with serious illness. However, there are little data on the occurrence, predictors, and timing of these discussions.

Objectives:

To examine the occurrence, predictors, and timing of electronic health record (EHR)-documented goals-of-care discussions for hospitalized patients.

Methods:

This retrospective cohort study used natural language processing (NLP) to examine EHR-documented goals-of-care discussions for adults with chronic life-limiting illness or age ≥80 hospitalized 2015–2019. The primary outcome was NLP-identified documentation of a goals-of-care discussion during the index hospitalization. We used multivariable logistic regression to evaluate associations with baseline characteristics.

Results:

Of 16,262 consecutive, eligible patients without missing data, 5,918 (36.4%) had a documented goals-of-care discussion during hospitalization; approximately 57% of these discussions occurred within 24 hours of admission. In multivariable analysis, documented goals-of-care discussions were more common for women (OR=1.26, 95%CI 1.18–1.36), older patients (OR=1.04 per year, 95%CI 1.03–1.04), and patients with more comorbidities (OR=1.11 per Deyo-Charlson point, 95%CI 1.10–1.13), cancer (OR=1.88, 95%CI 1.72–2.06), dementia (OR=2.60, 95%CI 2.29–2.94), higher acute illness severity (OR=1.12 per National Early Warning Score point, 95%CI 1.11–1.14), or prior advance care planning documents (OR=1.18, 95%CI 1.08–1.30). Documentation of these discussions was less common for racially or ethnically minoritized patients (OR=0.823, 95%CI 0.75–0.90).

Conclusion:

Among hospitalized patients with serious illness, documented goals-of-care discussions identified by NLP were more common among patients with older age and increased burden of acute or chronic illness, and less common among racially or ethnically minoritized patients. This suggests important disparities in goals-of-care discussions.

INTRODUCTION:

Aligning medical decision-making with an individual’s goals and values is important for patients with serious illness.^1,2 Goals-of-care discussions explore a patient’s overarching aims for medical care within the context of these values and goals, and often involve examination of one’s understanding of his or her illness and specific treatment preferences.^3,4 These discussions help guide clinical decision making, aid in the provision of goal-concordant care^1,5 and are associated with improvements in patient-centered outcomes.^6,7 Despite this, goals-of-care discussions often do not occur until late in serious illness.^8–12 Moreover, there is frequently discordance between patient and provider understanding about the occurrence and conclusions of these discussions.^13,14 Barriers to high quality goals-of-care discussions include differences in expectations around prognosis^15,16 and life-sustaining treatments,¹⁷ along with clinician discomfort, time constraints, and inadequate training.^9,18–22 These issues highlight a need to improve the frequency and quality of goals-of-care communication between clinicians, patients with serious illness, and surrogate decision-makers.^21,23

The electronic health record (EHR) serves as a large and comprehensive data source that may enable robust analysis of several palliative care metrics, including documentation of goals-of-care discussions for patients with serious illness.^24–26 Documentation of these discussions promotes goal-concordant care⁷ and facilitates communication among providers and across care settings.^27–29 In recent years, our research group and others have developed several natural language processing (NLP) approaches to identifying goals-of-care discussions within unstructured, free-text clinician documentation in the EHR.^30–33 One such study that used NLP to measure goals-of-care discussions for patients admitted to the ICU found that patients who had such conversations were more likely to be older, female, and with a non-elective admission to the ICU.³⁴ However, there is a need for additional investigation into the occurrence, predictors, and timing of inpatient goals-of-care discussions, particularly given the complexity of shared decision-making during acute illness^35,36 and the potential for both goal-discordant care and death during hospitalization.^37–39 This investigation would help guide future research and quality improvement efforts for patients with serious illness.

We conducted a retrospective cohort study to examine the occurrence, predictors, and timing of NLP-identified documented goals-of-care discussions for adult patients with chronic life-limiting illness or age ≥80 hospitalized on a medical service. We hypothesized that EHR documentation of a goals-of-care discussion would be more likely to occur for patients with one or more of the following characteristics: older age, life-limiting cancer, dementia, higher burden of chronic illness, higher acute severity of illness, and a previously completed advance directive in the EHR. We hypothesized that these discussions would be less likely to occur for racially or ethnically minoritized patients, given well-documented disparities in physician-patient communication ⁴⁰ and palliative care. ⁴¹

METHODS:

Study Design and Data Source

We conducted a retrospective cohort study using EHR data collected from patients hospitalized at two large teaching hospitals within an academic healthcare system. We utilized NLP to measure the primary outcome of occurrence of a documented goals-of-care discussion in any EHR note written by a physician or advance practice provider (APP), including admission notes, progress notes, consult notes, and discharge summaries.^30,42 The University of Washington Institutional Review Board approved this study (STUDY00011002).

Study population

We included adult patients 18 years or older with at least one chronic life-limiting illness hospitalized on a medical service at one of two study hospitals between January 1, 2015 and December 31, 2019. Chronic life-limiting illness was defined using International Classification of Diseases 10^th Revision (ICD-10) diagnosis codes (Appendix I) available in the EHR during the 24 months prior to the index admission for any of the chronic conditions used by the Dartmouth Atlas Project to study end-of-life care in the United States.⁴³ These included cancers with poor prognosis (i.e. primary malignancies with poor prognoses or metastatic disease), chronic lung disease, coronary artery disease, congestive heart failure, peripheral vascular disease, moderate-to-severe chronic kidney disease, severe chronic liver disease, diabetes with end-organ damage, and dementia.^44–46 These conditions are associated with 90% of deaths among Medicare patients.⁴⁴ Similar definitions have been utilized in other recent studies of patients with serious illness and are intended to capture a patient population likely to benefit from a goals-of-care discussion.^47–49 We also included all patients with age greater than or equal to 80 years, given the burden of illness and importance of palliative care in this age group.⁵⁰ For patients with multiple hospitalizations within the timeframe of interest, we included only the first medical hospitalization in analyses to achieve independence of observations. We excluded patients with a restricted status in the EHR, such as prisoners or victims of violence.

Natural Language Processing and Machine Learning

Our research group has developed an NLP modeling approach to identify documented goals-of-care discussions in EHR notes.³⁰ Goals-of-care discussions were defined as those exploring one or more of the following: a patient’s overarching values and goals, understanding of his or her illness, and specific treatment preferences in the context of this illness.^3,28,51 The NLP model used in this study employs a hybrid approach combining machine-derived features, which classify text based on the frequency distribution of words, and expert-defined textual search patterns indicative of goals-of-care documentation, such as the term “goals of care” or authorship by the specialty palliative care service. This approach and its performance have been described in a previous report.⁴² For this study, we selected a discrimination threshold that corresponded to patient-level sensitivity of 81.3%, specificity of 87.3%, positive predictive (PPV) of 63.4%, and negative predictive value (NPV) of 94.5% in leave-one-group-out cross-validation with the training data from the same hospitals and using the same eligibility criteria.⁴² Preliminary findings from a different, external dataset of 160 adults hospitalized with serious illness and meeting the same eligibility criteria demonstrated satisfactory generalizability, with patient-level sensitivity of 74.8%, specificity of 89.1%, PPV of 80.5%, and NPV of 85.5% at the same threshold (unpublished data, Lee RY, personal communication, October 2021).

Predictor and Outcome Variables

The primary outcome variable in our study was the presence or absence of a documented goals-of-care discussion during hospital admission, as identified by NLP. We also describe the timing of the goals-of-care discussion, measured in days from hospital admission to the first documented goals-of-care discussion. Based on prior findings in the literature, we examined the association between the following patient characteristics and the occurrence of a documented goals-of-care discussion: study site, patient age, patient gender, racial or ethnic minority status (defined as Black, Asian or Pacific Islander, Indigenous, or Hispanic ethnicity), health insurance status (adequately insured was defined as private/commercial, Medicare, military, or other insurance; under-insured was defined as self-insured, uninsured, and Medicaid-insured at the time of admission), English-speaking proficiency, life-limiting cancer diagnosis or dementia (using diagnosis codes prior to the admission), burden of chronic illness, acute severity of illness, presence of an advance care planning document (defined as a living will, durable power of attorney for health care, or Physician Orders for Life-Sustaining Treatment form) uploaded in the EHR at least 24 hours prior to admission, and hospital admission date to identify temporal trends.^{34,41,47–49,52} Burden of chronic illness was measured at the time of admission using the Deyo-Charlson Comorbidity Index, which incorporates 17 chronic comorbidities and is validated as a predictor of long-term survival.^53,54 Acute severity of illness was assessed by two criteria: admission to an intensive care unit within the first 24 hours in the hospital and the National Early Warning Score (NEWS) at the time of admission.^55,56 Admission to the ICU within the first 24 hours may have occurred after the goals-of-care discussion, but is included as a marker of early acute severity of illness. The NEWS score was developed to standardize assessment of acute illness severity and has demonstrated superiority in discriminating risk of clinical deterioration in hospitalized patients compared with other early warning scores.^55,56

Statistical Analysis

Occurrence of an NLP-identified documented goals-of-care discussion during hospital admission was analyzed as a dichotomous variable using logistic regression analysis. This was a complete case analysis of all patients with data for the outcome and predictors. We excluded 520 patients (3.1%) for incomplete data, including patients lacking information regarding racial or ethnic minority status (n=302), limited English proficiency (n=48), or National Early Warning Score components (n=194). The following characteristics were included in the model: age, gender, racial or ethnic minority status, health insurance status, English-speaking proficiency, Deyo-Charlson Comorbidity Index, NEWS, early ICU admission, life-limiting cancer, dementia, advance directive in the EHR at least 24 hours before admission, hospital site, and time elapsed between January 1, 2015 and index hospital admission.⁵⁷ P-values <0.05 were considered evidence of statistical significance.

RESULTS:

Table 1 presents the demographic information of the 16,262 patients who met inclusion criteria for our study. Of these patients, 5,918 (36.4%) had a goals-of-care discussion documented in the EHR during the index hospitalization as identified by NLP. The median age of patients was 63 years old and 41.7% were female.

Table 1.

Characteristics of the Sample^a

Characteristic	No Discussions	Discussions	TOTAL
Sample size, n	10,344	5,918	16,262
Characteristics at Index Admission
Hospital #2, n (%)	4,955 (47.9)	2,851 (48.2)	7,806 (48.0)
Age in years, median (IQR)	60 (20)	68 (21)	63 (21)
Female, n (%)	4,132 (39.9)	2,652 (44.8)	6,784 (41.7)
Race, n (%)
White	7,449 (72.0)	4,447 (75.1)	11,896 (73.2)
Black	1,346 (13.0)	566 (9.6)	1,912 (11.8)
Asian/Pacific islander	1,168 (11.3)	709 (12.0)	1,877 (11.5)
Indigenous	320 (3.1)	157 (2.7)	477 (2.9)
Race not specified, but Hispanic	61 (0.6)	39 (0.7)	100 (0.6)
Ethnicity, n (%)
Non-Hispanic	9,469 (91.5)	5,565 (94.0)	15,034 (92.4)
Hispanic	866 (8.4)	339 (5.7)	1,205 (7.4)
Ethnicity not specified, but Asian	1 (0.0)	4 (0.1)	5 (0.0)
Ethnicity not specified, but Black	6 (0.1)	(0.1)	11 (0.1)
Ethnicity not specified, but Indigenous	2 (0.0)	5 (0.1)	7 (0.0)
Limited Spoken English Proficiency, n (%)
No; English preference	9,094 (87.9)	5,174 (87.4)	14,268 (87.7)
Yes; other spoken language preference	1,226 (11.9)	739 (12.5)	1,965 (12.1)
Non-spoken language userb	24 (0.2)	5 (0.1)	29 (0.2)
Cancer, n (%)c	2,898 (28.0)	2,133 (36.0)	5,031 (30.9)
Dementia, n (%)d	541 (5.2)	941 (15.9)	1,482 (9.1)
Adequately Insured, n (%)e	5,674 (54.9)	3,777 (63.8)	9,451 (58.1)
Deyo-Charlson score, median (IQR)	4 (3)	5 (3)	4 (4)
National early warning score (NEWS), median (IQR)	2 (3)	3 (4)	2 (3)
ICU admission within 24 hours after index admission, n (%)	2,201 (21.3)	2,138 (36.1)	4,339 (26.7)
ACP documentation 24 hours before index admission, n (%)f	1,743 (16.9)	1,368 (23.1)	3,111 (19.1)
Days from 1/1/2015 to index admission, median (IQR)	980 (802)	996.5 (777)	986.5 (791)
Characteristics of Index Hospitalization
Died during index hospitalization, n (%)	31 (0.3)	1,130 (19.1)	1,161 (7.1)
Length of index hospital stay in days, median (IQR)	4.5 (5.1)	7.8 (12.1)	5.1 (7.2)

^aThe sample included all patients with complete data for the predictors used in the analyses. Exclusions: 520 patients lacking information regarding racial-ethnic minority status (n=302), limited English proficiency (n=48), or National Early Warning Score components (n=194).

^bSign language or relay interpretation.

^cICD code for advanced cancer in medical record at any time during the 24 months prior to index admission.

^dICD code for dementia in medical record at any time during the 24 months prior to index admission or dementia identified through manual review of medical record.

^eCovered by private/commercial, Medicare, military, or other insurance. (Under-insured reference group: self-insured or Medicaid-insured).

^fACP documentation included power-of-attorney designation, living will, healthcare directive, or POLST form.

The association between our predictors of interest and the occurrence of an NLP-identified documented goals-of-care discussion during hospital admission based on the multivariate model is summarized in Table 2. Associations based on models without adjustment for other covariates were similar except that discussions were significantly more likely for patients with adequate health insurance. In multivariable analyses, older age, female gender, higher Deyo-Charlson score, life-limiting cancer, dementia, higher NEWS score, early ICU admission, and a pre-existing advance directive in the EHR were all significantly associated with increased likelihood of a documented goals-of-care discussion occurring during the hospitalization of interest. Minoritized racial or ethnic identity (OR 0.823, 95%CI 0.753–0.899), was significantly associated with decreased likelihood of a documented goals-of-care discussion occurring during the hospitalization of interest. When racial subgroups were analyzed separately, the findings supported this effect for patients who identify as Black or Asian/Pacific Islander. When Hispanic ethnicity was analyzed separately, the point estimate supported a similar effect, though it did not reach statistical significance. Supplementary Table S1 outlines the findings of this sensitivity analysis. Hospital site, English-speaking proficiency, and health insurance status were not associated with likelihood of a documented goals-of-care discussion. Time elapsed from the start of the study period was also not associated with likelihood of a documented goals-of-care discussion.

Table 2.

Associations of Baseline Characteristics with Occurrence of Goals-of-Care Discussion during Index Hospitalization^a

Predictors	n	b	p	OR	95% CI
Age	16,262	0.035	<0.001	1.036	1.033, 1.039
Gender
Male	9,478	0.000		1.000
Female	6,784	0.235	<0.001	1.265	1.178, 1.358
Race and Ethnicity			<0.001
White non-Hispanic	10,940	0.000		1.000
Minoritized race and/or Hispanic ethnicity	5,322	−0.193		0.824	0.755, 0.900
Limited Spoken English Proficiency			0.174
No; English preference	14,268	0.000		1.000
Yes; other spoken language preference	1,965	0.056		1.057	0.932, 1.199
Non-spoken language userb	29	−0.821		0.440	0.165, 1.176
Cancerc
No	11,231	0.000		1.000
Yes	5,031	0.633	<0.001	1.884	1.721, 2.062
Dementiad
No	14,780	0.000		1.000
Yes	1,482	0.953	<0.001	2.594	2.290, 2.939
Adequately Insurede
No	6,811	0.000		1.000
Yes	9,451	0.011	0.802	1.011	0.931, 1.097
Deyo-Charlson score	16,262	0.103	<0.001	1.108	1.092, 1.125
National early warning score (NEWS)	16,262	0.118	<0.001	1.125	1.110, 1.141
ICU admission within 24 hours
No	11,923	0.000		1.000
Yes	4,339	0.688	<0.001	1.990	1.819, 2.177
ACP documentationf
No	13,151	0.000		1.000
Yes	3,111	0.136	0.003	1.146	1.047, 1.254
Elapsed years from 1/1/2015 to hospital admission	16,262	0.000	0.409	1.011	0.985, 1.039
Hospital
#1	8,456	0.000		1.000
#2	7,806	0.056	0.204	1.058	0.970, 1.153

^aEstimates were based on a logistic regression model that included all predictors listed in the rows, based on data from 16,262 patients with complete data, and estimated with maximum likelihood with robust standard errors.

^bSign language or relay interpretation.

^cICD code for advanced cancer in medical record at any time during the 24 months prior to index admission.

^dICD code for dementia in medical record at any time during the 24 months prior to index admission or dementia identified through manual review of medical record.

^eCovered by private/commercial, Medicare, military, or other insurance. (Under-insured reference group: self-insured or Medicaid-insured).

^fACP documentation included power-of-attorney designation, living will, healthcare directive, or POLST form.

When NLP-identified goals-of-care discussions occurred, they tended to occur early during the hospital admission. Figure 1 shows the distribution of NLP-identified documented goals-of-care discussion occurrence from time of hospital admission; about 57% of these discussions occurred within 24 hours of admission and 75% occurred within 3 days of admission.

Figure 1. Timing of Goals-of-Care Discussions during Index Hospitalization^a.

a Graph shows for each day, the volume of discussions that day as a percentage of discussions that occurred during the first 21 days of hospitalization. Discussions occurring during this 3-week period represented 97.3% of all discussions that occurred the hospitalization. An additional 1.8% of the discussions had occurred by Day 35, with 0.8% occurring after day 35 – the latest taking place on Day 216. Of the 5,918 patients who had goals-of-care discussions during the index hospitalization, 3,390 (57.3%) had their initial discussion on Day 1 (i.e., fewer than 24 hours after admission). IQR = 3.

DISCUSSION:

The results of this study support our hypothesis that NLP-identified EHR documentation of a goals-of-care discussion are more likely to occur for hospitalized patients with older age, life-limiting cancer diagnosis, dementia, higher burden of chronic illness, higher acute severity of illness, early ICU admission, or a previously completed advance directive in the EHR. A diagnosis of dementia was associated with a 2.6 higher odds of a documented discussion among patients with the same values of the other covariates. This finding may reflect the growing awareness surrounding the importance of providing goal-concordant end-of-life care for patients with dementia,^52,58 as this patient population has been shown to receive less aggressive interventions in the hospital.⁵⁹ It is also possible that clinicians are prompted to conduct and document these discussions for patients who are unable to readily communicate their values, goals, and care preferences due to severity of acute illness or underlying dementia.

Our results suggest that NLP-identified inpatient goals-of-care discussions occurred less often for racially or ethnically minoritized patients. This finding reaffirms known racial and ethnic disparities in palliative and end-of-life care: as these patient populations are also less likely to receive hospice services,^60,61 more likely to receive higher intensity care near the end of life,^62,63 and more likely to report concerns regarding quality of care and provider communication.⁶⁴ These disparities highlight a need for race-conscious research efforts ⁶⁵ and interventions that seek to improve the provision of fair and equitable care for minoritized patient groups. We also found that presence of an advance care planning document in the EHR was associated with a greater likelihood of an NLP-identified documented goals-of-care discussion. These documents may provide important context and serve as a launching point for providers to delve deeper into a patient’s goals-of-care during hospitalization.

We did not find evidence of a significant change in the occurrence of NLP-identified documented goals-of-care discussions within our study population between 2015 and 2019. It is possible that the growing national focus on primary and specialty palliative care for patients with serious illness^66–68 has not translated into a noticeable increase in documented goals-of-care discussions for hospitalized patients with chronic life-limiting illness over this time period. This supports the importance of ongoing research to further elucidate the barriers to inpatient goals-of-care discussions for this patient population, along with efforts to improve clinician education and training in the realm of goals-of-care communication.

Finally, we found that most NLP-identified goals-of-care discussions occur early during the hospitalization and only a quarter occurred after day 3. Prior research has demonstrated the importance of conducting goals-of-care discussions early in a patient’s hospital stay to minimize unwarranted, life-sustaining interventions.^37,69 Our findings suggest that interventions should focus on prompting goals-of-care discussion during the hospitalization, but that there may be less value in focusing on having these discussions earlier than they already occur for most hospitalized patients.

This study has several important limitations. First, our findings of association cannot be assumed to be causal in this observational study.⁷⁰ We performed multivariable analyses to adjust for possible confounders, but there may be unmeasured confounders that we could not assess and there may not have been sufficient overlap between covariates to sufficiently adjust for confounding among those that were considered. Second, this study utilized data derived from electronic health records and our findings are thus susceptible to measurement error in exposures and misclassification of outcomes.⁷¹ We mitigated this risk by assessing predictor variables that are reliably documented in the EHR (demographic information, comorbidities) and employing composite variables (Deyo-Charlson score, NEWS score) to measure health status.⁷² Third, while NLP represents a promising approach for measuring EHR-documented goals-of-care discussions, our analysis does not address outcome misclassification by NLP. It is possible that unknown differential performance of NLP between predictor groups could explain some of the observed associations. Future studies should continue to characterize the performance of NLP models for identifying goals-of-care discussions in the EHR, especially for minoritized patient groups. Fourth, the construct of a goals-of-care discussion represents a spectrum of important discussions that are multifaceted and vary over dimensions of context, timing, depth, content, and execution.⁴² Therefore, it is important to operationalize the definition of this construct. We are continuing to refine the criteria by which goals-of-care discussions are defined and measured.⁷³ Fifth, the duration of hospitalization varies across participants and may affect conclusions. Choosing a fixed time period is problematic because some participants have very short hospital stays, and modeling time to discussion is further complicated by competing risks (e.g. death). Sixth, we combined multiple racial and ethnic groups into a single category for the primary analyses because of sample size limitations. When racial subgroups and Hispanic ethnicity were examined separately, the finding of decreased occurrence of goals-of-care discussions was maintained for patients who identify as Black or Asian/Pacific Islander. The point estimate for Hispanic ethnicity suggests a similar effect, though was not statistically significant, likely due to inadequate power. Future studies should further explore individual racial and ethnic groups. Seventh, we chose to include the first hospitalization during the study period to achieve independence of observations, but may have consequently missed features of repeat hospitalizations. Lastly, our sample of notes is derived from patients hospitalized at two hospitals within a single academic healthcare system. Because the prevalence and content of goals-of-care documentation differs across patient populations and health systems, our study findings may not generalize to other patient populations or health systems.

CONCLUSION:

Through this retrospective cohort study, we found important patterns related to the occurrence, predictors, and timing of NLP-identified goals-of-care discussions for hospitalized patients with serious illness. Documentation of these discussions was more likely to occur for patients of female gender and older age, as well as for those with cancer or dementia, higher burden of chronic illness, higher acute severity of illness, and for those with a prior advance care planning document in the EHR. Importantly, documentation of these discussions was less likely to occur for racially or ethnically minoritized patients. Our study supports the utility of the EHR as a tool to analyze important palliative care metrics. Our findings also highlight a need for future efforts to further elucidate and overcome barriers to inpatient goals-of-care discussions for patients with serious illness, particularly for younger patients with chronic illness and for those who are racially and ethnically minoritized.