Nudging Clinicians to Promote Serious Illness Communication for Critically Ill Patients: A Pragmatic Cluster Randomized Trial

Katherine R Courtright; Jaspal Singh; Erich M Dress; Brian Bayes; Michael O Harhay; Marzana Chowdhury; Yingying Lu; Kenneth M Lee; Dylan S Small; Casey Whitman; Jenny Tian; Vanessa Madden; Timothy Hetherington; Lindsay Placket; D Matthew Sullivan; Henry L Burke; Michael B Green; Scott D Halpern

doi:10.1001/jamainternmed.2025.0090

. 2025 Mar 17;185(5):510–520. doi: 10.1001/jamainternmed.2025.0090

Nudging Clinicians to Promote Serious Illness Communication for Critically Ill Patients

A Pragmatic Cluster Randomized Trial

Katherine R Courtright ^1,^2,^3,^4,^✉, Jaspal Singh ⁵, Erich M Dress ¹, Brian Bayes ¹, Michael O Harhay ^1,^4,⁶, Marzana Chowdhury ¹, Yingying Lu ¹, Kenneth M Lee ^1,⁶, Dylan S Small ⁷, Casey Whitman ¹, Jenny Tian ¹, Vanessa Madden ¹, Timothy Hetherington ⁸, Lindsay Placket ⁹, D Matthew Sullivan ⁹, Henry L Burke ¹⁰, Michael B Green ⁵, Scott D Halpern ^1,^2,^4,^6,¹¹

¹Palliative and Advanced Illness Research (PAIR) Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia

²Pulmonary, Allergy, and Critical Care Division, Perelman School of Medicine, University of Pennsylvania, Philadelphia

³Palliative Care Program, Penn Medicine, Philadelphia, Pennsylvania

⁴Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia

⁵Critical Care Network, Adult Acute Division, Department of Medicine, Atrium Health, Charlotte, North Carolina

⁶Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia

⁷Department of Statistics, Wharton School, University of Pennsylvania, Philadelphia

⁸Center for Outcomes Research and Evaluation, Atrium Health, Charlotte, North Carolina

⁹Information and Analytics Services, Atrium Health System, Charlotte, North Carolina

¹⁰Division of Palliative Care, Adult Acute Care Services, Department of Medicine, Atrium Health, Charlotte, North Carolina

¹¹Department of Medical Ethics and Healthy Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia

Accepted for Publication: January 7, 2025.

Published Online: March 17, 2025. doi:10.1001/jamainternmed.2025.0090

^✉

Corresponding Author: Katherine R. Courtright, MD, MS, University of Pennsylvania, 423 Guardian Dr, 303 Blockley Hall, Philadelphia, PA 19104 (katherine.courtright@pennmedicine.upenn.edu).

Author Contributions: Dr Courtright had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Courtright, Singh, Madden, Plickert, Sullivan, Burke, Green, Halpern.

Acquisition, analysis, or interpretation of data: Courtright, Singh, Dress, Bayes, Harhay, Chowdhury, Lu, Lee, Small, Whitman, Tian, Madden, Hetherington, Sullivan, Green, Halpern.

Drafting of the manuscript: Courtright, Singh, Dress, Harhay, Lu, Lee, Whitman, Green, Halpern.

Critical review of the manuscript for important intellectual content: Courtright, Singh, Bayes, Harhay, Chowdhury, Lu, Lee, Small, Tian, Madden, Hetherington, Plickert, Sullivan, Burke, Green, Halpern.

Statistical analysis: Courtright, Bayes, Harhay, Chowdhury, Lu, Lee, Small, Whitman, Tian, Halpern.

Obtained funding: Courtright, Singh, Madden, Halpern.

Administrative, technical, or material support: Courtright, Singh, Dress, Bayes, Madden, Hetherington, Plickert, Sullivan, Green, Halpern.

Supervision: Courtright, Singh, Dress, Madden, Sullivan, Burke.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work was supported by a grant from the Donaghue Foundation.

Role of the Funder/Sponsor: The Donaghue Foundation had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We thank the chair of the data and safety monitoring board, Michael Elliott, PhD (University of Michigan). We also thank Toan Huyhn, MD, Jessica Kearney-Bryan, RN, BSN, Whitney Rossman, MS, and Erica L. Frechman, PhD, NP, at Atrium Health for their contributions during trial preparation and data collection. Finally, we thank Jason N. Doctor, PhD (University of Southern California) and Michael E. Detsky, MD, MSHP (University of Toronto) for serving as paid scientific advisors.

^✉

Corresponding author.

PMCID: PMC11915113 PMID: 40094649

Key Points

Question

Does nudging intensive care unit (ICU) clinicians to adhere to communication guidelines improve clinical outcomes?

Findings

This pragmatic trial involving 3500 encounters among adults with a chronic serious illness and at least 48 hours of mechanical ventilation in 17 ICUs (February 2018-October 2020) found that nudging clinicians to document prognosis, whether a comfort-focused treatment alternative was offered or not, did not significantly reduce hospital length of stay. The comfort-focused treatment alternative nudge led to a significant increase in discharge to hospice (10.9% vs 7.3%) and earlier comfort-care orders (4.5 days vs 3.6 days), without significantly affecting hospital or long-term mortality.

Meaning

Nudging ICU clinicians to adhere to communication guidelines did not reduce length of stay, but the treatment alternative nudge improved certain secondary end-of-life care processes among critically ill patients with limited prognoses.

Abstract

Importance

Guidelines recommend that intensive care unit (ICU) clinicians consider prognosis and offer a comfort-focused treatment alternative to patients with limited prognoses to promote preference-sensitive treatment decisions.

Objective

To determine whether nudging ICU clinicians to adhere to communication guidelines improves outcomes among critically ill patients at high risk of death or severe functional impairment.

Design, Setting, and Participants

This 4-arm pragmatic, stepped-wedge, cluster randomized trial (conducted February 1, 2018-October 31, 2020, follow-up through April 29, 2021, and analyses December 2023-January 2024) involved 3500 encounters of adults with chronic serious illness receiving mechanical ventilation for at least 48 hours at 10 hospitals comprising 17 medical, surgical, specialty, or mixed ICUs in community, rural, and urban settings.

Interventions

Two clinician-directed electronic health record nudge interventions were each compared with usual care alone and combined: document of 6-month functional prognosis and whether a comfort-focused treatment alternative was offered or a reason why not.

Main Outcomes and Measures

The primary outcome was hospital length of stay, with death coded at the 99th percentile. Secondary end points included 22 measures of acute care utilization, end-of-life care processes, and mortality.

Results

Of 3500 patient encounters among 3250 patients (mean [SD] age, 63.2 [13.5] years; 46.1% female), 3384 encounters (96.7%) had complete baseline data and were included in risk-adjusted analyses. The overall intervention document completion rate for all patients was 75.0% (n = 1714) and similar across groups. Among the 3500 encounters, observed hospital mortality was 35.7% (n = 1249), and the median observed length of stay was 8.93 days (IQR, 4.64-16.23). The median length of stay with deaths coded as the 99th percentile did not differ between any intervention and usual care groups (for length of stay, all adjusted median difference 95% CIs include 0; for hospital mortality, all adjusted risk difference [RD] 95% CIs include 0). Results were similar in sensitivity analyses with death coded as low at the fifth percentile and without ranking deaths. Compared with usual care, a higher percentage of patients were discharged to hospice in the treatment alternative group (10.9% vs 7.3%; adjusted RD, 6% [95% CI, 1%-10%]) and the combined group (8.9% vs 7.3%; adjusted RD, 6% [95% CI, 0%-12%]). The treatment alternative intervention led to earlier comfort-care orders (3.6 vs 4.5 days; adjusted hazard ratio, 1.42 [95% CI, 1.06-1.92]). The 20 other secondary end points were unaffected by the interventions.

Conclusions and Relevance

This cluster randomized clinical trial found that electronically nudging ICU clinicians to adhere to communication guidelines was feasible but did not reduce hospital length of stay.

Trial Registration

ClinicalTrials.gov Identifier: NCT03139838

This cluster randomized trial assess whether including 2 nudge messages in the electronic health record asking clinicians to document 6-month functional prognosis of critically ill patients and whether a comfort care alternative was offered reduced length of stay compared with usual care.

Introduction

Poor communication with critically ill patients and their surrogates contributes to goal-discordant and low-value care near the end of life.^1,2,3 Guidelines recommend that intensive care unit (ICU) clinicians routinely elicit patients’ care goals and preferences within the context of their prognosis and offer treatment alternatives to facilitate preference-sensitive treatment decisions,^4,5 including the option of comfort-focused care for patients at high risk of death or severe functional impairment.⁶ However, due in part to variable adherence to these guidelines, many ICU patients receive aggressive care near the end of life that can cause adverse outcomes for themselves and family.^7,8,9

Prior efforts to address these challenges, including trials of supportive or communication interventions in the ICU, have yielded mixed results.^{10,11,12,13,14,15} The shortage of and variability in access to palliative care specialists limits approaches that rely solely on increasing consultation.¹⁶ Surrogate decision-making interventions have been similarly resource intensive and have not consistently improved patient or surrogate outcomes.^10,17 In contrast, low-cost, clinician-directed behavioral interventions (nudges) have been shown to be scalable and effective in many settings^18,19 and hypothesized to improve serious illness communication in the ICU.²⁰

This pragmatic randomized clinical trial²¹ used 2 nudges to promote recommended communication for critically ill patients at high risk of death or severe functional impairment. Both interventions tested in this trial were designed based on behavioral economic principles and preliminary studies^19,22 and were embedded in the electronic health record (EHR). The study’s hypothesis was that nudging ICU clinicians to document prognosis, provide a reason if they had not offered a comfort-focused treatment alternative, or both would reduce length of stay (LOS) and improve end-of-life care processes compared with usual care.

Methods

The Prognosticating Outcomes and Nudging Decisions in the Electronic Health Record in the ICU (PONDER-ICU) trial was designed as highly pragmatic, according to Pragmatic Explanatory Continuum Indicator Summary 2 (PRECIS-2) criteria.²³ Details of the trial design and the underlying conceptual model were previously published.²¹ The institutional review boards at the University of Pennsylvania and Atrium Health approved the study’s conduct with a waiver of informed consent because the interventions were designed to promote standards of care (the trial Protocol is available in Supplement 1). The trial was overseen by a data safety and monitoring board (DSMB). This study followed the Consolidated Standards of Reporting Trials Extension (CONSORT Extension) for the stepped-wedge cluster randomized trial reporting guideline.

Study Design and Setting

This trial was conducted in 10 hospitals containing 17 medical, surgical, specialty, or mixed ICUs (bed size range, 4-35) within Atrium Health in North Carolina (eTable 1 in Supplement 2). All hospitals used a Cerner Corp EHR. Seven hospitals, including 4 in rural settings, had a single medical-surgical ICU. Three ICUs had a closed (high-intensity) staffing model, and 14 ICUs had an open (low-intensity) staffing model in which intensivist consultation was required for patients receiving mechanical ventilation. All ICUs had 24-hour critical care telemedicine support available.

The stepped-wedge, cluster randomized design increased power relative to a parallel cluster design and minimized contamination that would have occurred with randomization at the ICU level (clinicians routinely rotate in different ICUs within a hospital), at the clinician level (patients exposed to multiple clinicians during ICU encounters), or at the patient level (clinicians care for multiple patients simultaneously). All hospitals started in usual care for at least 5 months (eFigure 1 in Supplement 2). Hospitals were randomly assigned into pairs that transitioned to the intervention phase in a random sequence every 3 months. Within each hospital pair, 1 was randomly assigned to adopt either the prognosis or treatment alternative intervention. After 12 months, the other intervention was added for at least 4 months. Because this design resulted in different numbers of hospital-months in each phase of the trial, we expected enrollment to differ across the 4 study groups. We engaged ICU, informatics, and palliative care leaders at Atrium throughout the pretrial phase to implement the interventions within usual ICU clinician workflows and local EHR norms. All ICU clinicians received information about the interventions prior to the trial and again before their hospital adopted the first intervention.

Participants

Enrollment occurred from February 1, 2018, through October 31, 2020, with follow-up through April 29, 2021. Race and ethnicity were collected from the EHR. We built and validated a rule-based EHR algorithm²¹ that identified critically ill patients at high risk of near-term mortality or severe functional impairment using the following eligibility criteria: age at least 18 years, a preexisting serious illness, and continuous mechanical ventilation for at least 48 hours.²⁴ A preexisting serious illness was defined by at least 1 International Classification of Disease, Tenth Revision, Clinical Modification code in the medical record on admission or within the prior 365 days for amyotrophic lateral sclerosis, chronic obstructive pulmonary disease (COPD), interstitial lung disease, cirrhosis, heart failure (HF), dementia, kidney failure, leukemia, lymphoma, or solid organ malignant neoplasm (eTable 2 in Supplement 2). The algorithm systematically screened and enrolled newly eligible patients every 6 hours (eFigure 2 in Supplement 2) to promote timely intervention without overburdening a large health system’s EHR.

Interventions

The interventions targeted both the attending physician (ie, intensivist or hospitalist) primarily responsible for the patient’s ICU care at enrollment and, when applicable, the bedside advanced practice provider (APP). Each intervention comprised 2 brief queries in a document that was automatically created in the patient’s chart at enrollment (eFigure 3 in Supplement 2). The attending and APP each had an intervention document that they could complete once through ICU discharge or cessation of mechanical ventilation. Completed documents became part of the medical record viewable by all acute care clinicians.

Asking intensivists to estimate a short-term prognosis for hypothetical critically ill patients makes them more inclined to discuss the option of withdrawal of life support in a family meeting,²² a phenomenon known as a focusing effect.²⁵ Hypothesizing that this effect would translate to real patient encounters, we prompted clinicians to document in the EHR a patient’s 6-month prognosis (“Do you think this patient will be alive 6 months from now?”), and if yes, to estimate their functional status along a spectrum from “no noticeable limitations in physical and/or cognitive function” to “bedbound and almost entirely dependent on others.”

Clinicians are trained to make decisions backed by reason and are accustomed to being held professionally accountable for their choices. Thus, simply requiring clinicians to provide a justification for their decisions and having those remain visible in the EHR can substantially reduce low-value practices.^19,26 Accordingly, we required clinicians to document whether they had offered comfort-focused care as an alternative to continued intensive care⁶ and if not, to provide a brief justification. If no justification was provided, the phrase “No justification given” was automatically entered to promote accountability.²⁷

Clinicians had the opportunity to complete the intervention document(s) without receiving an EHR alert through the first 2 pm following the patient’s enrollment time. Thereafter, we promoted adherence with a reminder alert (eFigure 4 in Supplement 2) whenever the attending physician or APP placed an order for the patient. Additionally, if an APP completed the intervention first, they received an alert to encourage the attending’s adherence (eFigure 4 in Supplement 2). Telemedicine ICU nurses also received a daily report of incomplete interventions and sent email reminders to nonresponding clinicians.

Outcomes

All patient characteristics and most outcomes were obtained from the EHR data warehouse. The EHR was the primary source of death data. We also matched records to the Social Security Death Index to capture deaths occurring outside of Atrium.

The primary outcome was postenrollment hospital LOS with death coded at the 99th percentile of the distribution.²⁸ Minimizing LOS is important to patients with serious illness and their families who prefer to spend more time at home,^29,30 and prior ICU communication interventions have been shown to reduce ICU and hospital days.^10,12 Hospital LOS is also a critical operational and financial metric for hospitals. Having first chosen LOS as the primary outcome, the second choice was which of several approaches to analyzing LOS to use. Whereas the approaches most commonly deployed in prior ICU-based trials are either biased or rely on statistical assumptions that are unlikely to be met, assigning a fixed value of LOS to decedents avoids these problems.³¹ The third choice was which value of LOS to assign to decedents. We chose to rank death at the 99th percentile of the LOS distribution in primary analyses based on prior evidence showing that although patients and families prefer shorter hospital stays overall, they also prefer long stays that end in survival to shorter stays that end in death.^30,32

We selected 18 secondary outcomes based on existing evidence, clinical hypotheses, and relevance for clinical decision-making aligned with the goals of pragmatic trials. These included ICU LOS; ICU mortality; ICU readmission; mechanical ventilation duration; receipt of cardiopulmonary resuscitation (CPR); and hospital readmission and mortality at 30, 90, and 180 days. End-of-life care processes included change in code status, time to and presence of a comfort-care order, time to and presence of a palliative care consult order, and discharge to hospice. Within 48 hours of in-hospital death, the primary bedside nurse was invited via email to complete the Quality of Dying and Death 1-item survey: “Overall, how would you rate the quality of the patient’s dying?” (0, terrible experience to 10, almost perfect experience).³³ Four additional secondary outcomes of hospital mortality and hospital-free days at 30, 90, and 180 days were added during interim analytic discussions with the DSMB.

Blinding

Analysts remained blinded to treatment group throughout the trial. Data managers were unblinded to prepare reports for the DSMB. Patients were unaware of the trial, and clinicians were not blinded given the nature of the interventions.

Statistical Analysis

Due to the use of a partial factorial variant stepped-wedge design,³⁴ some clusters were unexposed to the prognosis or treatment alternative interventions alone. However, we included the full control sample in all analyses because inclusion of unexposed clusters promotes precision in estimating the control effect in a given period, which in turn augments the precision of the treatment effect estimates in that period.³⁵ P values were 2 sided.

For primary analyses, we used simultaneous quantile regression (sqreg package in STATA version 18, StataCorp) to compare hospital LOS at the 20th, 30th, 40th, 50th, 60th, and 70th percentiles. This approach provides a joint test of the overall significance of the coefficients for each intervention tested at each specified quantile without assuming any outcome distribution, an important feature given that coding deaths at the 99th percentile of the study cohort produced a right-skewed distribution with a spike at the end. We constructed 95% CIs using cluster-period bootstrapping. We adjusted for prespecified baseline patient characteristics (Table 1) and included fixed effects for time and hospital.^36,37 Because patients could meet eligibility criteria at different times in their hospital course, we adjusted for the time in hours from hospital admission to enrollment. We adjusted for multiple comparisons on the primary outcome using the Holm method.³⁸ No adjustment was made for secondary outcomes.³⁹ Finally, we tested interaction terms between each intervention and selected baseline patient characteristics.

Table 1. Characteristics of Participants in the Intention-to-treat Sample (N = 3500).

Characteristic^a	No. (%) of patients
Characteristic^a	Usual care (n = 1214)	Prognosis (n = 845)	Treatment alternative (n = 479)	Combined (n = 962)
Age, y
Mean (SD)	62.6 (13.6)	62.7 (13.9)	64.5 (12.2)	63.7 (13.5)
Median (IQR)	63.0 (54.0-72.0)	64.0 (55.0-73.0)	65.0 (57.0-73.0)	65.0 (56.0-73.0)
Race
American Indian or Alaska Native	13 (1.1)	8 (0.9)	3 (0.6)	6 (0.6)
Asian	12 (1.0)	13 (1.5)	1 (0.2)	11 (1.1)
Black	362 (29.8)	273 (32.3)	132 (27.6)	328 (34.1)
White	782 (64.4)	520 (61.5)	336 (70.1)	587 (61.0)
Other^b	28 (2.3)	4 (0.5)	2 (0.4)	3 (0.3)
Missing	17 (1.4)	27 (3.2)	5 (1.0)	27 (2.8)
Ethnicity
Hispanic	28 (2.3)	31 (3.7)	4 (0.8)	40 (4.2)
Not Hispanic	1062 (87.5)	705 (83.4)	396 (82.7)	878 (91.3)
Missing	124 (10.2)	109 (12.9)	76 (16.5)	44 (4.6)
Sex
Male	652 (53.7)	474 (56.1)	247 (51.6)	513 (53.3)
Female	561 (46.2)	371 (43.9)	232 (48.4)	449 (46.7)
Missing	1 (0.1)	0	0	0
Marital status
Single	318 (26.2)	220 (26.0)	103 (21.5)	267 (27.8)
Married	530 (43.7)	358 (42.4)	211 (44.1)	396 (41.2)
Separated	45 (3.7)	27 (3.2)	20 (4.2)	36 (3.7)
Divorced	167 (13.8)	106 (12.5)	69 (14.4)	127 (13.2)
Widowed	137 (11.3)	128 (15.1)	73 (15.2)	120 (12.5)
Missing	17 (1.4)	6 (0.7)	3 (0.6)	16 (1.7)
ICU admission source
Emergency department	550 (45.3)	372 (44.0)	264 (55.1)	526 (54.7)
Operating room	65 (5.4)	43 (5.1)	6 (1.3)	16 (1.7)
Transfer from other hospital	164 (13.5)	106 (12.5)	41 (8.6)	91 (9.5)
Ward	435 (35.8)	323 (38.2)	166 (34.7)	323 (33.6)
Missing	0	1 (0.1)	2 (0.4)	6 (0.6)
Preexisting serious illness^c
Heart failure	633 (52.1)	441 (52.2)	262 (54.7)	488 (50.7)
Chronic obstructive pulmonary disease	523 (43.1)	316 (37.4)	261 (54.5)	380 (39.5)
Solid organ malignant neoplasm	195 (16.1)	158 (18.7)	62 (12.9)	160 (16.6)
Cirrhosis	135 (11.1)	123 (14.6)	49 (10.2)	98 (10.2)
Kidney failure	106 (8.7)	74 (8.8)	36 (7.5)	101 (10.5)
Dementia	92 (7.6)	76 (9.0)	45 (9.4)	93 (9.7)
Interstitial lung disease	59 (4.9)	36 (4.3)	31 (6.5)	45 (4.7)
Leukemia and lymphoma	54 (4.4)	45 (5.3)	17 (3.5)	29 (3.0)
Amyotrophic lateral sclerosis	10 (0.8)	12 (1.4)	8 (1.7)	4 (0.4)
SOFA score, mean (SD)^d	8.82 (3.33)	8.84 (3.27)	8.15 (3.16)	8.87 (3.11)
Elixhauser Comorbidity Index, mean (SD)^e	18.9 (12.4)	19.2 (12.6)	18.3 (12.8)	19.8 (12.2)
SARS-CoV-2 status^f
Positive	0	0	0	69 (7.2)
Undetected	1214 (100)	845 (100)	479 (100)	893 (92.8)

Open in a new tab

Abbreviations: ICU, intensive care unit; SOFA, Sequential Organ Failure Assessment.

^{^a}

Data presented as No. (%) except where otherwise noted. Percentages may not total to 100 because of rounding. All standardized mean differences are <.21.

^{^b}

Other race included: Iranian, Middle Eastern or North African, Native Hawaiian or Pacific Islander, or West Indian.

^{^c}

Preexisting serious illness included if present in the medical record at admission or within the prior 365 days. Categories were not mutually exclusive.

^{^d}

The SOFA score was determined from the subscores calculated closest to enrollment between 48 hours before and 12 hours after enrollment. The renal subscore was calculated using creatinine levels alone given the unreliability of urine output documentation.

^{^e}

Elixhauser Comorbidity Index was calculated as a weighted sum of each of 29 binary comorbidity variables identified in the electronic health record by International Statistical Classification of Diseases, Tenth Revision, Clinical Modification codes present on admission. Higher scores indicate a greater comorbidity burden and severity of illness.

^{^f}

COVID-19 pandemic overlapped with the final 7 months of enrollment after 8 of 10 hospitals had transitioned to the combined intervention in this stepped-wedge trial (eFigure 1 in the Supplement 2).

Secondary outcomes were analyzed using mixed-effects logistic regression for binary outcomes, linear mixed-effects regression for continuous outcomes, and a 0-inflated negative binomial regression for hospital-free days. The time to comfort-care order was evaluated among hospital decedents using a mixed-effects Cox model, and time to palliative care consult was analyzed using a Fine-Gray model, where death was the competing event and alive discharges were censored. All secondary outcome models were adjusted for time, hospital, and baseline patient characteristics. We secondarily analyzed a modified intention-to-treat (ITT) sample inclusive of all patients who remained eligible through 2 pm following enrollment when ICU clinicians began receiving reminder alerts.

A prespecified sensitivity analysis alternatively ranked deaths at the 5th, 10th, 15th, 85th, 90th, and 95th percentiles of LOS. As described in the statistical analysis plan (Supplement 1), we added 3 sensitivity analyses of the primary outcome after the trial started: (1) an analytic sample that restricted to patients with SARS-CoV-2 negative status because the final 7 months of enrollment occurred during the study region’s initial surge of the pandemic; (2) a linear mixed-effects regression with time as a fixed effect and hospital as a random intercept; and (3) a clustered competing-risks model in which death was the competing event rather than ranked. Finally, we fit the linear mixed-effects model with hospital-period random interactions to estimate the within-period and between-period intraclass correlations (ICCs).³⁷

Statistical Power and Sample Size

A detailed sample size calculation was previously reported (see the Protocol in Supplement 1).²¹ Briefly, data from study hospitals during the year preceding the trial suggested that we would enroll at least 3500 patient encounters with a mean (SD) LOS of 16.2 (11.8) days, median LOS of 12.63 days, and 30% hospital mortality. Placing deaths at the longest LOS, and assuming ICCs of up to 0.10 within clinicians and 0.05 within ICUs, we estimated an 80% or higher power to detect a difference of 3 days in LOS at the median between either the prognosis or treatment alternative intervention and usual care, and at least 80% power to detect an additive effect of the combined interventions vs usual care.

Results

Participants

Among 72 984 ICU admissions for patients aged 18 years or older, 3500 encounters (4.8%) among 3250 unique patients met eligibility criteria (Figure 1). Of these, 1214 (34.7%) were enrolled in usual care, 845 (24.1%) in prognosis, 479 (13.7%) in treatment alternative, and 962 (27.5%) in combined intervention groups. The modified ITT sample included 3015 encounters (86.1%) (eTables 3 and 4 in Supplement 2). The 10 hospitals contributed a median of 209 encounters (IQR, 104-324). Standardized mean differences between baseline patient characteristics across the 4 study groups were generally small (<0.2), although absolute differences in the ICU admission source were potentially meaningful (Table 1). Patients’ mean (SD) age was 63.2 (13.5) years, 1095 (31.3%) were Black, and 1613 (46.1%) were female. Of those who had the most common preexisting illnesses, 52.1% (n = 1824) had HF, 42.3% (n = 1480) had COPD, and 16.4% (n = 575) had solid organ malignant neoplasm.

Intervention Implementation Processes

Overall intervention adherence ranged from 73.7% to 76.3% across intervention groups, with a median of 1 (IQR, 1-2) reminder alert per provider per patient among 239 attending physicians and 89 APPs (eTable 5 in the Supplement 2). Among patients who remained eligible for at least 1 day after reminder alerts began, intervention adherence ranged from 92.7% to 97.1%. Clinicians reported offering comfort-focused care to 247 patients (70.0%) and 529 patients (72.1%) with completed documents in the treatment alternative and combined intervention groups, respectively. Clinicians provided no justification for not offering comfort-focused care for only 23 encounters (6.5%) in the treatment alternative group and 66 (9.0%) in the combined group.

Primary Outcome

The observed median hospital LOS for each study group was usual care 9.2 days (IQR, 4.9-16.6), prognosis 9.2 days (IQR, 4.8-18.0), treatment alternative 7.7 days (IQR, 4.5-13.6), and combined 9.0 days (IQR, 4.3-15.7). Overall, 1249 patients (35.7%) died in the hospital, with no differences detected between any intervention (unadjusted mortality range, 27.8%-41.7%) and usual care (unadjusted mortality, 31.5%) in adjusted analyses (Table 2).

Table 2. Differences in Hospital Length of Stay Between Each Intervention and Usual Care.

	No. of patients	Prognosis		Treatment alternative		Combined
	No. of patients	Estimate (95% CI)^a	P value	Estimate (95% CI)^a	P value	Estimate (95% CI)^a	P value
Primary quantile regression (20, 30, 40, 50, 60, 70 percentiles)^b^,^c^,^d	3384	0.35 (−6.87 to 7.56)	F_6,3348 = .74	1.55 (−4.58 to 7.68)	F_6,3348 = .94	2.39 (−6.65 to 11.43)	F_6,3348 = .86
SARS-CoV-2 negative (20, 30, 40, 50, 60, 70 percentiles)	3319	0.52 (−6.37 to 7.41)	F_6,3284 = .56	1.00 (−5.02 to 7.00)	F_6,3284 = .91	2.22 (−7.37 to 11.81)	F_6,3284 = .74
Linear mixed-effects model
Hospital random effect^e	3384	% Change, −1.86 (−18.82 to 19.00)	.85	% Change, −5.02 (−22.31 to 16.09)	.63	% Change, 1.82 (−22.86 to 34.79)	.90
Hospital and period random effects^e^,^f	3384	% Change, −2.18 (−31.40 to 39.50)	.90	% Change, −5.57 (−39.40 to 47.14)	.80	% Change, 1.86 (−31.39 to 51.22)	.93
Clustered competing risk model (discharge alive as event of interest vs death as competing event)	3384	aSHR, 0.91 (0.73 to 1.14)	.42	aSHR, 1.22 (0.90 to 1.66)	.19	aSHR, 1.06 (0.82 to 1.38)	.66

Open in a new tab

Abbreviation: aSHR, adjusted subdistribution hazard ratio.

^{^a}

All results were adjusted for time, hospital, and patient characteristics at enrollment: age, sex, race (Black, White, other [See Table 1 legend for included race and ethnicity]), marital status (married vs other), preexisting cancer (solid organ malignancy or leukemia and lymphoma), ICU admission source (emergency department vs other), Elixhauser Comorbidity Index, Sequential Organ Failure Assessment (SOFA) score, SARS-CoV-2 status, and time between hospital admission and enrollment.

^{^b}

Observed median hospital length of stay for each study group: usual care 9.2 days (IQR, 4.9-16.6), prognosis 9.2 days (IQR, 4.8-18.0), treatment alternative 7.7 days (IQR, 4.5-13.6), and combined 9.0 days (IQR, 4.3-15.7). Quantile regression estimates are presented as the median difference (95% CI). CIs were generated using cluster-period bootstrap. Complete quantile results are presented in eTable 6 in Supplement 2.

^{^c}

In the intention-to-treat sample, 116 patients (3.3%) had covariate missingness.

^{^d}

Sequential Holm adjustment to control the family-wise type I error rate was preplanned (see the statistical analysis plan in Supplement 1) but is not shown because the use of more restrictive α levels would not impact trial interpretation based on unadjusted P values shown in the Table. Specifically, the sequential Holm adjustment orders unadjusted P values from each contrast from smallest to largest, and then these are adjusted by multiplying each P value by the number of remaining contrasts. For example, for the P value of .62, which is the lowest of the 3, the adjusted level of significance was .0167 (ie, .05 ÷ n remaining contrasts; and in this first contrast .05 ÷ 3 = .0167). Because P = .62 is greater than .0167, there is no numerical rationale for further adjustment.

^{^e}

Length of stay with deaths coded at the 99th percentile was log-transformed for the linear mixed-effects models.

^{^f}

Standard errors were estimated using the bias-corrected robust variance estimator. The between-period intraclass correlation was 0.068 and within-period intraclass correlation was 0.069, with a cluster-autocorrelation of 0.976.

For the primary analysis with in-hospital deaths coded at the 99th percentile (76.2 days) of the overall LOS distribution (eFigure 5 in Supplement 2), there were no statistically significant differences in median LOS between any intervention and usual care: for prognosis, 0.35 days (95% CI, −6.87 to 7.56; joint F_6,3348; P = .74); for treatment alternative, 1.55 days (95% CI, −4.58 to 7.68; joint F_6,3348; P = .94); for combined, 2.39 days (95% CI, −6.65 to 11.43; joint F_6,3348; P = .86) (Table 2 and eTable 6 in Supplement 2). Similar results were found in all sensitivity analyses (Table 2), including alternative coding of deaths as low as the fifth percentile of LOS (eTable 7 in Supplement 2), and among the modified ITT sample (eTable 8 in Supplement 2). The prognosis, treatment alternative, or combined intervention effects were not modified by patient characteristics (eTable 9 in Supplement 2). The between-period ICC was 0.068 and within-period ICC was 0.069, with a cluster-autocorrelation of 0.976 indicating negligible between-period, within-cluster heterogeneity (Table 2).

Secondary Outcomes

Compared with usual care, discharge to hospice was significantly increased in the treatment alternative group (10.9% vs 7.3%; adjusted risk difference (RD), 6% [95% CI, 1% to 10%]) and in the combined group(8.9% vs 7.3%; adjusted RD, 6% [95% CI, 0% to 12%]) but not in the prognosis group (7.3% vs 7.7%; adjusted RD, 3% [−1% to 6%]). Comfort-care orders were placed earlier in the treatment alternative group (adjusted HR, 1.42; 95% CI, 1.06 to 1.92) compared with the usual care group but not significantly so in the prognosis group (adjusted HR, 1.09; 95% CI, 0.83 to 1.44) or the combined group (adjusted HR, 1.36; 95% CI, 0.93 to 2.00) (Figure 2). The reduction in time to palliative care consultation seen in the treatment alternative group was not statistically significant (adjusted subdistribution HR [SHR], 1.42; 95% CI, 0.99-2.04), and the prognosis and combined interventions similarly did not significantly affect this outcome (Table 3).

Figure 2. — A, Displays a model-based estimate of the duration (in days) from enrollment until a new comfort-care order was placed among patients in the intention-to-treat sample who died in the hospital (n = 1249). The estimated values were derived using a mixed-effects Cox proportional hazards model with shared frailty for hospital, adjusted for time and patient characteristics at enrollment: age, binary gender, race (Black, White, other [See Table 1 footnote for race and ethnicity]), marital status (married vs other), preexisting cancer (solid organ malignant neoplasm or leukemia and lymphoma), intensive care unit admission source (emergency department vs other), Elixhauser Comorbidity Index, Sequential Organ Failure Assessment score, SARS-CoV-2 status, and time between hospital admission and enrollment. B, Highlights days 0 to 15 from Panel A where separation between groups began.

Table 3. Secondary Outcomes in the Intention-to-Treat Sample.

Outcome	Usual care patients, unadjusted (n = 1214)	Prognosis (n = 845)		Treatment alternative (n = 479)		Combined (n = 962)
Outcome	Usual care patients, unadjusted (n = 1214)	Unadjusted	Adjusted effect estimate (95% CI)^a	Unadjusted	Adjusted effect estimate (95% CI)^a	Unadjusted	Adjusted effect estimate (95% CI)^a
Code status change, No. (%)	467 (38.4)	361 (42.7)	RD, 0.03 (−0.05 to 0.10)	176 (36.7)	RD, 0.01 (−0.07 to 0.10)	449 (46.6)	RD, 0.01 (−0.10 to 0.11)
Comfort care
Order, No. (%)	341 (28.1)	278 (32.9)	RD, 0.04 (−0.03 to 0.10)	131 (27.3)	RD, −0.004 (−0.07 to 0.06)	357 (37.1)	RD, 0.08 (−0.01 to 0.17)
Time to order, median (IQR), [n = 1193]^b	4.5 (1.6 to 8.9)	4.8 (1.7 to 10.8)	aHR, 1.09 (0.83 to 1.44)	3.6 (1.6 to 8.3)	aHR, 1.42 (1.06 to 1.92)^c	4.8 (1.9 to 10.2)	aHR, 1.36 (0.93 to 2.00)
Palliative care
Consult, No. (%) [n = 3331]	262 (22.8)	189 (23.0)	RD, 0.04 (−0.03 to 0.11)	115 (26.0)	RD, 0.05 (−0.03 to 0.13)	220 (24.0)	RD, 0.02 (−0.08 to 0.12)
Time to consult, median (IQR), d [n = 3218]^d	7.5 (3.2 to 14.2)	7.4 (3.6 to 14.3)	aSHR, 1.24 (0.80 to 1.94)	5.9 (3.0 to 10.7)	aSHR, 1.42 (0.99 to 2.04)	7.0 (3.1 to 13.3)	aSHR, 1.25 (0.84 to 1.87)
Discharge to hospice, No. (%)	89 (7.3)	65 (7.7)	RD, 0.03 (−0.01 to 0.06)	52 (10.9)	RD, 0.06 (0.01 to 0.10)^c	86 (8.9)	RD, 0.06 (0.00 to 0.12)^c
ICU length of stay, median (IQR)^e	128.7 (66.6 to 228.9)	131.8 (69.3 to 247.7)	% Median difference, 1.03 (−16.46 to 22.38)	125.8 (67.5 to 235.5)	% Median difference, 0.31 (−17.95 to 22.61)	128.7 (61.4 to 238.8)	% Median difference, 7.38 (−18.68 to 41.95)
ICU readmission, No. (%)	109 (9.0)	74 (8.8)	RD, −0.01 (−0.06 to 0.04)	24 (5.0)	RD, −0.02 (−0.07 to 0.04)	71 (7.4)	RD, −0.03 (−0.09 to 0.03)
Duration of mechanical ventilation, % (median change), h [n = 3373]	61.3 (22.0 to 142.0)	70.4 (26.8 to 160.0)	% Median difference, 15.01 (−13.66 to 52.31)	62.3 (22.8 to 137.5)	% Median difference, −14.82 (−36.84 to 15.12)	75.0 (27.3 to 176.7)	% Median difference, 8.45 (−27.93 to 62.32)
QODD score, mean (SD) [n = 601]^f	7.4 (2.1)	7.2 (2.4)	Mean difference, −0.21 (−1.14 to 0.75)	7.7 (2.2)	Mean difference, −0.16 (−1.16 to 0.93)	7.2 (2.6)	Mean difference, −0.52 (−1.81 to 0.87)
Receipt of CPR, No. (%)	57 (4.7)	60 (7.1)	RD, 0.03 (−0.01 to 0.06)	15 (3.1)	RD, 0.00 (−0.03 to 0.03)	74 (7.7)	RD, 0.06 (−0.02 to 0.13)
Hospital-free days, median (IQR), d [n = 3048]^g
30 d	4.5 (0.0 to 18.0)	0.0 (0.0 to 16.0)	cond, 0.98 (0.85 to 1.12) zi, 1.39 (0.92 to 2.11)	7.0 (0.0 to 20.0)	cond, 0.97 (0.84 to 1.11) zi, 0.87 (0.56 to 1.34)	0.0 (0.0 to 16.0)	cond, 0.97 (0.80 to 1.17) zi, 1.28 (0.76 to 2.15)
90 d	40.5 (0.0 to 75.0)	22.0 (0.0 to 72.0)	cond, 0.97 (0.84 to 1.12) zi, 1.29 (0.85 to 1.95	49.0 (0 to 77.0)	cond, 0.99 (0.86 to 1.14) zi, 1.01 (0.65 to 1.57)	6.0 (0.0 to 71.0)	cond, 0.94 (0.79 to 1.13) zi, 1.49 (0.85 to 2.60)
180 d	80.5 (0.0 to 161.0)	38.0 (0.0 to 157.0)	cond, 0.97 (0.82 to 1.15) zi, 1.25 (0.82 to 1.91)	99.0 (0 to 164.0)	cond, 0.96 (0.81 to 1.14) zi, 0.95 (0.61 to 1.47)	8.0 (0.0 to 156.0)	cond, 0.92 (0.75 to 1.15) zi, 1.40 (0.80 to 2.45)
Mortality, No. (%)
Hospital	394 (32.4)	321 (38.0)	RD, 0.04 (−0.03 to 0.11)	133 (27.8)	RD, −0.03 (−0.10 to 0.04)	401 (41.7)	RD, 0.02 (−0.08 to 0.11)
ICU	325 (26.8)	263 (31.2)	RD, 0.02 (−0.05 to 0.09)	119 (24.9)	RD, 0.00 (−0.07 to 0.07)	339 (35.5)	RD, 0.05 (−0.06 to 0.15)
30 d	464 (38.2)	373 (44.1)	RD, 0.07 (0.003 to 0.14)^c	189 (39.5)	RD, 0.05 (−0.02 to 1.12)	467 (48.5)	RD, 0.10 (0.004 to 0.19)^c
90 d	559 (46.0)	420 (49.7)	RD, 0.06 (−0.01 to 0.13)	219 (45.7)	RD, 0.04 (−0.04 to 0.11)	539 (56.0)	RD, 0.08 (−0.01 to 0.18)
180 d	613 (50.5)	454 (53.7)	RD, 0.05 (−0.02 to 0.13)	245 (51.1)	RD, 0.04 (−0.03 to 0.11)	581 (60.4)	RD, 0.08 (−0.01 to 0.18)
Readmission, No. (%)
30 d	162 (13.3)	97 (11.5)	RD, 0.01 (−0.002 to 0.02)	54 (11.3)	RD, −0.01 (−0.06 to 0.04)	100 (10.4)	RD, −0.0001 (−0.06 to 0.06)
90 d	267 (22.0)	182 (21.5)	RD, 0.01 (−0.05 to 0.07)	115 (24.0)	RD, 0.02 (−0.04 to 0.08)	173 (18.0)	RD, −0.01 (−0.09 to 0.07)
180 d	341 (28.1)	237 (28.0)	RD, 0.01 (−0.06 to 0.07)	147 (30.7)	RD, 0.01 (−0.06 to 0.08)	218 (22.7)	RD, −0.01 (−0.10 to 0.07)

Open in a new tab

Abbreviations: CPR, cardiopulmonary resuscitation; HR, hazard ratio; ICU, intensive care unit; QODD, Quality of Dying and Death, RD, risk difference; aSHR, adjusted subdistribution hazard ratio.

^{^a}

In the intention-to-treat sample, 116 of 3500 patients (3.3%) had covariate missingness; adjusted effect estimates include 3384 patients, except where otherwise noted. Estimates were adjusted for time and hospital and for characteristics listed in the Figure 2 legend.

^{^b}

Days to comfort care were analyzed among in-hospital deaths; days to palliative care consult were analyzed using a competing risk model in which consult was the event of interest and death and no consult were competing events.

^{^c}

P <.05.

^{^d}

Two hospitals did not contribute palliative care consult data (see eTable 1 in Supplement 2).

^{^e}

Deaths ranked at the 99th percentile of hospital length of stay.

^{^f}

Nurses completed the 1-item survey (0, terrible experience to 10, almost perfect experience) for 601 of 1193 decedents (60%).

^{^g}

Analyzed using a 0-inflated negative binomial model, comprising a conditional model (cond) and a 0-inflation model (zi). Estimates are incident rate ratios (95% CI) for the conditional model and odds ratios (95% CI) for the 0-inflation model.

All other utilization and process outcomes—duration of mechanical ventilation, ICU LOS, quality of dying and death, hospital-free days through 180 days, time to palliative care consultation, and the frequencies of comfort-care orders, code status changes, palliative care consultations, ICU readmissions, ICU deaths, receipt of CPR, and hospital readmissions through 180 days—did not differ significantly between any intervention group and usual care (Table 3). None of the interventions affected in-hospital mortality, the prognosis and combined interventions each yielded higher 30-day mortality than usual care (prognosis, 44.1% vs 38.2%; RD, 7% [95% CI, 0.3%-14%] and combined, 48.5% vs 38.2%; RD, 10% [95% CI, 0.4%-19%]), and these differences were no longer present at 90 or 180 days. Similar effects on secondary outcomes were found in the modified ITT sample (eTable 10 and eFigure 6 in Supplement 2), although some had different levels of statistical significance.

Discussion

In this pragmatic trial among patients at high-risk of death or severe functional impairment, nudging ICU clinicians in the EHR to document 6-month functional prognosis or whether they offered the alternative of comfort-focused care did not reduce the primary outcome of hospital LOS. The treatment alternative nudge led to increased hospice use and reduced time to comfort-focused care, although the 20 other secondary end-of-life care, acute care utilization, and mortality outcomes studied were not affected. Variable adherence to guidelines for ICU communication^4,5,6 is influenced by clinician-, hospital-, and system-level factors^40,41 and contributes to delivery of nonbeneficial and unwanted care.⁴² Whereas previous trials to improve ICU communication employed designs, interventions, or settings that limit generalizability,^10,11,12,13 this trial of low-cost, scalable interventions conducted in primarily community hospitals offers several findings applicable in settings similar to where most US residents receive their care.

First, ICU clinicians adhered to these simple behavioral interventions for three-fourths of patients overall. Given the intensity and pace of ICU work and the intervention’s low-touch approach, this finding contributes to growing evidence of the feasibility and acceptability of clinician-targeted, electronic nudges for serious illness communication and palliative care delivery.^18,43,44 However, the fact that only 2 clinicians completed the interventions before receiving an alert in the EHR (eTable 5 in Supplement 2) and that adherence increased to more than 90% within 1 day of the reminder, suggests that future nudges could achieve greater adherence by alerting clinicians closer to when patients are identified.

Second, nudging ICU clinicians to document prognosis or whether they offered comfort-focused care did not affect LOS. This finding was consistent across sensitivity analyses and alternative strategies for analyzing LOS. Although the observed variance in hospital LOS in the study population was larger than expected and the minimum clinically important difference for ICU patients is unknown but may be smaller than 3 days, the small percentage differences in LOS observed in our linear mixed-effects models (Table 2) make it unlikely that we failed to detect a clinically important difference due to lack of power. Although a shorter LOS is preferred by many critically ill patients and families when ICU survival or acceptable functional recovery is unlikely,⁴⁵ powerful social and system factors can influence LOS even when high-quality serious illness care and communication occurs.

Third, the treatment alternative intervention reduced the time to comfort-care orders—causing an estimated 13% more patients to receive such care by day 5 (Figure 2)—and increased the proportion of patients discharged to hospice without affecting mortality. These findings align with those from our prior trial testing default palliative care consultation¹⁸ in suggesting that nudging clinician behavior can improve the timing and quality of end-of-life care for inexorably dying hospitalized patients.⁴⁶ Efforts to develop reliable, pragmatic approaches for measuring goal-concordant care may enable future studies to show whether such care directly promotes patients’ values.⁴⁷

Fourth, the prognosis intervention did not impact any end-of-life processes or outcomes. The SUPPORT study demonstrated that merely providing clinicians with a predicted near-term mortality and functional prognosis for hospitalized patients did not impact communication behavior.¹⁴ In contrast, the intervention prompted clinicians to estimate their patients’ prognoses in light of a proof-of-principle study of ICU clinicians who used hypothetical scenarios.²² However, a follow-up study of simulated family meetings⁴⁸ published after we launched this trial found that this more active form of engaging ICU clinicians in prognostication was ineffective in promoting offers of comfort-focused care. Collectively, these studies strongly suggest that prognostic awareness alone without a nudge is unlikely to change ICU clinician behavior.

Finally, the combined intervention led to an increase in discharges to hospice, likely attributable to the treatment alternative component, but it did not reduce time to comfort-care orders. Although only 2% of the study cohort had SARS-CoV-2 infection, which did not impact the primary outcome, 56 of the 100 total hospital-months with the combined intervention occurred during the COVID-19 pandemic due to the stepped-wedge design. Throughout this time, there were significant challenges in end-of-life care communication with families and increased mistrust that impacted care for all critically ill patients and likely reduced the overall effectiveness of the combined intervention.^49,50,51,52

Strengths and Limitations

Among the strengths of this trial are that more than half of the participating hospitals had a single ICU with an open staffing model, similar to most US community hospitals and in contrast to the settings of most prior ICU communication trials. Evidence that the low-cost and readily implemented treatment alternative intervention successfully engaged bedside clinicians in opportunities to provide preference-sensitive care earlier and more often is essential in community and rural settings, and should also be useful in better-resourced health systems operating with increasingly tight margins. The demographic and clinical diversity of the enrolled patients further enhances the generalizability of the trial's findings.⁵³ Additionally, because the study population was nearly 3 times larger than prior ICU communication trials and used broad, easily implemented eligibility criteria, the results provide practical guidance for hospitals with various patient catchment areas and case mixes.

This study has several limitations. First, we could not assess the frequency or quality of serious illness conversations because clinical note data were not accessible, and reliable collection of patient- or surrogate-reported outcomes was not feasible. Second, we did not provide clinician education in serious illness communication. Such training is increasingly accessible and promoted by health systems,⁵⁴ and if coupled with the interventions in this study could yield greater benefits. Third, intervention adherence was incomplete. Intervention flexibility is paramount in the design of pragmatic trials,²³ but future studies and clinical applications of these and other effective EHR nudges¹⁸ may need to use earlier alerts to maximize effectiveness, while ensuring implementation remains acceptable to clinicians. Finally, the trial was not powered to detect effects of the interventions on the secondary outcomes; thus, these findings warrant confirmation.

Conclusions

This pragmatic randomized clinical trial found that nudging clinicians to focus on prognosis and to offer comfort-focused care as an alternative to continued intensive care did not reduce hospital LOS among critically ill patients with limited prognoses. However, the treatment alternative nudge, alone or in combination with the prognosis nudge, may be a low-cost and effective approach to improve some end-of-life care processes among critically ill patients who ultimately would not have survived.

Supplement 1.

Trial Protocol

jamainternmed-e250090-s001.pdf^{(940.9KB, pdf)}

Supplement 2.

eTable 1. Characteristics of the Participating Hospitals and Intensive Care Units

eFigure 1. Stepped-wedge, Cluster-randomized Study Design

eTable 2. Diagnostic Codes for Eligible Pre-existing Life-limiting Illnesses

eFigure 2. Screening, Enrollment, and Intervention Processes

eFigure 3. Intervention Documents in the Electronic Health Record

eFigure 4. Clinician Alerts in the Electronic Health Record

eTable 3. Characteristics of Participants in the Modified Intention-to-treat Sample

eTable 4. Characteristics of Patients Excluded From the Modified Intention-to-treat Sample

eTable 5. Intervention processes and adherence

eFigure 5. Hospital Length of Stay at Each Quantile of the Distribution

eTable 6. Median Difference in Hospital Length of Stay at Each Quantile in the Primary Analysis

eTable 7. Median Difference of Hospital Length of Stay with Alternative Coding of Deaths

eTable 8. Median Differences in Hospital Length of Stay at Each Quantile in the Modified Intention-to-treat Analysis

eTable 9. Effect Modification of the Interventions on Hospital Length of Stay Among Subgroups

eTable 10. Secondary Outcomes Among the Modified Intention-to-treat Sample

eFigure 6. Time to comfort-care order in the Modified Intention-to-treat Sample

jamainternmed-e250090-s002.pdf^{(2.3MB, pdf)}

Supplement 3.

Data Sharing Statement

jamainternmed-e250090-s003.pdf^{(16.8KB, pdf)}

References

1.Chiarchiaro J, Buddadhumaruk P, Arnold RM, White DB. Quality of communication in the ICU and surrogate’s understanding of prognosis. Crit Care Med. 2015;43(3):542-548. doi: 10.1097/CCM.0000000000000719 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Taylor LJ, Johnson SK, Nabozny MJ, et al. Barriers to goal-concordant care for older patients with acute surgical illness: communication patterns extrinsic to decision aids. Ann Surg. 2018;267(4):677-682. doi: 10.1097/SLA.0000000000002282 [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Scheunemann LP, Ernecoff NC, Buddadhumaruk P, et al. Clinician-family communication about patients’ values and preferences in intensive care units. JAMA Intern Med. 2019;179(5):676-684. doi: 10.1001/jamainternmed.2019.0027 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Kon AA, Davidson JE, Morrison W, Danis M, White DB; American College of Critical Care Medicine; American Thoracic Society . Shared decision making in ICUs: an American College of Critical Care Medicine and American Thoracic Society policy statement. Crit Care Med. 2016;44(1):188-201. doi: 10.1097/CCM.0000000000001396 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Sullivan DR, Iyer AS, Enguidanos S, et al. Palliative care early in the care continuum among patients with serious respiratory illness: an official ATS/AAHPM/HPNA/SWHPN policy statement. Am J Respir Crit Care Med. 2022;206(6):e44-e69. doi: 10.1164/rccm.202207-1262ST [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Halpern SD, Becker D, Curtis JR, et al. ; Choosing Wisely Taskforce; American Thoracic Society; American Association of Critical-Care Nurses; Society of Critical Care Medicine . An OFFICIAL American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: the Choosing Wisely® Top 5 List in Critical Care Medicine. Am J Respir Crit Care Med. 2014;190(7):818-826. doi: 10.1164/rccm.201407-1317ST [DOI] [PubMed] [Google Scholar]
7.Field MJ, Cassel CK. Approaching Death, Improving Care at the End of Life. National Academy Press; 1997. [PubMed] [Google Scholar]
8.Wright AA, Zhang B, Ray A, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665-1673. doi: 10.1001/jama.300.14.1665 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Azoulay E, Pochard F, Kentish-Barnes N, et al. ; FAMIREA Study Group . Risk of post-traumatic stress symptoms in family members of intensive care unit patients. Am J Respir Crit Care Med. 2005;171(9):987-994. doi: 10.1164/rccm.200409-1295OC [DOI] [PubMed] [Google Scholar]
10.White DB, Angus DC, Shields AM, et al. ; PARTNER Investigators . A randomized trial of a family-support intervention in intensive care units. N Engl J Med. 2018;378(25):2365-2375. doi: 10.1056/NEJMoa1802637 [DOI] [PubMed] [Google Scholar]
11.Curtis JR, Nielsen EL, Treece PD, et al. Effect of a quality-improvement intervention on end-of-life care in the intensive care unit: a randomized trial. Am J Respir Crit Care Med. 2011;183(3):348-355. doi: 10.1164/rccm.201006-1004OC [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Curtis JR, Treece PD, Nielsen EL, et al. Randomized trial of communication facilitators to reduce family distress and intensity of end-of-life care. Am J Respir Crit Care Med. 2016;193(2):154-162. doi: 10.1164/rccm.201505-0900OC [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Carson SS, Cox CE, Wallenstein S, et al. Effect of palliative care-led meetings for families of patients with chronic critical illness: a randomized clinical trial. JAMA. 2016;316(1):51-62. doi: 10.1001/jama.2016.8474 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.The SUPPORT Principal Investigators . A controlled trial to improve care for seriously ill hospitalized patients: the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). JAMA. 1995;274(20):1591-1598. doi: 10.1001/jama.1995.03530200027032 [DOI] [PubMed] [Google Scholar]
15.Curtis JR, Back AL, Ford DW, et al. Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness: a randomized trial. JAMA. 2013;310(21):2271-2281. doi: 10.1001/jama.2013.282081 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Courtright KR, Cassel JB, Halpern SD. A research agenda for high-value palliative care. Ann Intern Med. 2018;168(1):71-72. doi: 10.7326/M17-2164 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Seaman JB, Arnold RM, Buddadhumaruk P, et al. Protocol and fidelity monitoring plan for four supports: a multicenter trial of an intervention to support surrogate decision makers in intensive care units. Ann Am Thorac Soc. 2018;15(9):1083-1091. doi: 10.1513/AnnalsATS.201803-157SD [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Courtright KR, Madden V, Bayes B, et al. Default palliative care consultation for seriously ill hospitalized patients: a pragmatic cluster randomized trial. JAMA. 2024;331(3):224-232. doi: 10.1001/jama.2023.25092 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Meeker D, Linder JA, Fox CR, et al. Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA. 2016;315(6):562-570. doi: 10.1001/jama.2016.0275 [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Halpern SD. Using default options and other nudges to improve critical care. Crit Care Med. 2018;46(3):460-464. doi: 10.1097/CCM.0000000000002898 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Courtright KR, Dress EM, Singh J, et al. ; PONDER-ICU Investigative Team . Prognosticating outcomes and nudging decisions with electronic records in the intensive care unit trial protocol. Ann Am Thorac Soc. 2021;18(2):336-346. doi: 10.1513/AnnalsATS.202002-088SD [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Turnbull AE, Krall JR, Ruhl AP, et al. A scenario-based, randomized trial of patient values and functional prognosis on intensivist intent to discuss withdrawing life support. Crit Care Med. 2014;42(6):1455-1462. doi: 10.1097/CCM.0000000000000227 [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147. doi: 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]
24.Detsky ME, Harhay MO, Bayard DF, et al. Six-month morbidity and mortality among intensive care unit patients receiving life-sustaining therapy: a prospective cohort study. Ann Am Thorac Soc. 2017;14(10):1562-1570. doi: 10.1513/AnnalsATS.201611-875OC [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Kahneman D, Krueger AB, Schkade D, Schwarz N, Stone AA. Would you be happier if you were richer? a focusing illusion. Science. 2006;312(5782):1908-1910. doi: 10.1126/science.1129688 [DOI] [PubMed] [Google Scholar]
26.O’Connor SD, Sodickson AD, Ip IK, et al. Journal club: requiring clinical justification to override repeat imaging decision support: impact on CT use. AJR Am J Roentgenol. 2014;203(5):W482-90. doi: 10.2214/AJR.14.13017 [DOI] [PMC free article] [PubMed] [Google Scholar]
27.De Cremer D, Barker M. Accountability and cooperation in social dilemmas: the influence of others’ reputational concerns. Curr Psychol. 2003;22:155-163. doi: 10.1007/s12144-003-1006-6 [DOI] [Google Scholar]
28.Lin W, Halpern SD, Prasad Kerlin M, Small DSA. A “placement of death” approach for studies of treatment effects on ICU length of stay. Stat Methods Med Res. 2017;26(1):292-311. doi: 10.1177/0962280214545121 [DOI] [PubMed] [Google Scholar]
29.Barnato AE, Herndon MB, Anthony DL, et al. Are regional variations in end-of-life care intensity explained by patient preferences?: a study of the US Medicare population. Med Care. 2007;45(5):386-393. doi: 10.1097/01.mlr.0000255248.79308.41 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Auriemma CL, Harhay MO, Haines KJ, Barg FK, Halpern SD, Lyon SM. What matters to patients and their families during and after critical illness: a qualitative study. Am J Crit Care. 2021;30(1):11-20. doi: 10.4037/ajcc2021398 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Harhay MO, Ratcliffe SJ, Small DS, Suttner LH, Crowther MJ, Halpern SD. Measuring and analyzing length of stay in critical care trials. Med Care. 2019;57(9):e53-e59. doi: 10.1097/MLR.0000000000001059 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Auriemma CL, Lyon SM, Strelec LE, Kent S, Barg FK, Halpern SD. Defining the medical intensive care unit in the words of patients and their family members: a freelisting analysis. Am J Crit Care. 2015;24(4):e47-e55. doi: 10.4037/ajcc2015717 [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Glavan BJ, Engelberg RA, Downey L, Curtis JR. Using the medical record to evaluate the quality of end-of-life care in the intensive care unit. Crit Care Med. 2008;36(4):1138-1146. doi: 10.1097/CCM.0b013e318168f301 [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Lyons VH, Li L, Hughes JP, Rowhani-Rahbar A. Proposed variations of the stepped-wedge design can be used to accommodate multiple interventions. J Clin Epidemiol. 2017;86:160-167. doi: 10.1016/j.jclinepi.2017.04.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Lee KM, Ma X, Yang GM, Cheung YB. Inclusion of unexposed clusters improves the precision of fixed effects analysis of stepped-wedge cluster randomized trials. Stat Med. 2022;41(15):2923-2938. doi: 10.1002/sim.9394 [DOI] [PubMed] [Google Scholar]
36.Li F, Hughes JP, Hemming K, Taljaard M, Melnick ER, Heagerty PJ. Mixed-effects models for the design and analysis of stepped wedge cluster randomized trials: an overview. Stat Methods Med Res. 2021;30(2):612-639. doi: 10.1177/0962280220932962 [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Lee KM, Cheung YB. The fixed-effects model for robust analysis of stepped-wedge cluster trials with a small number of clusters and continuous outcomes: a simulation study. Trials. 2024;25(1):718. doi: 10.1186/s13063-024-08572-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat Theory Appl. 1979;6(2):65-70. [Google Scholar]
39.Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology. 1990;1(1):43-46. doi: 10.1097/00001648-199001000-00010 [DOI] [PubMed] [Google Scholar]
40.Schenker Y, Tiver GA, Hong SY, White DB. Association between physicians’ beliefs and the option of comfort care for critically ill patients. Intensive Care Med. 2012;38(10):1607-1615. doi: 10.1007/s00134-012-2671-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Garland A, Connors AF. Physicians’ influence over decisions to forego life support. J Palliat Med. 2007;10(6):1298-1305. doi: 10.1089/jpm.2007.0061 [DOI] [PubMed] [Google Scholar]
42.Kruser JM, Cox CE, Schwarze ML. Clinical momentum in the intensive care unit: a latent contributor to unwanted care. Ann Am Thorac Soc. 2017;14(3):426-431. doi: 10.1513/AnnalsATS.201611-931OI [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Manz CR, Parikh RB, Small DS, et al. Effect of integrating machine learning mortality estimates with behavioral nudges to clinicians on serious illness conversations among patients with cancer: a stepped-wedge cluster randomized clinical trial. JAMA Oncol. 2020;6(12):e204759. doi: 10.1001/jamaoncol.2020.4759 [DOI] [PMC free article] [PubMed] [Google Scholar]
44.Curtis JR, Lee RY, Brumback LC, et al. Intervention to promote communication about goals of care for hospitalized patients with serious illness: a randomized clinical trial. JAMA. 2023;329(23):2028-2037. doi: 10.1001/jama.2023.8812 [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Auriemma CL, Harhay MO, Haines KJ, Barg FK, Halpern SD, Lyon SM. What matters to patients and their families during and after critical illness: a qualitative study. Am J Crit Care. 2021;30(1):11-20. doi: 10.4037/ajcc2021398 [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Halpern SD, Temel JS, Courtright KR. Dealing with death as an outcome in supportive care clinical trials. JAMA Intern Med. 2021;181(7):895-896. doi: 10.1001/jamainternmed.2021.1816 [DOI] [PubMed] [Google Scholar]
47.Halpern SD. Goal-concordant care—searching for the Holy Grail. N Engl J Med. 2019;381(17):1603-1606. doi: 10.1056/NEJMp1908153 [DOI] [PubMed] [Google Scholar]
48.Turnbull AE, Hayes MM, Brower RG, et al. Effect of documenting prognosis on the information provided to ICU proxies: a randomized trial. Crit Care Med. 2019;47(6):757-764. doi: 10.1097/CCM.0000000000003731 [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Connolly M, Bell M, Lawler F, Timmins F, Ryder M. Hospital-based palliative and end-of-life care in the COVID-19 pandemic: a scoping review. Am J Hosp Palliat Care. 2022;39(9):1105-1120. doi: 10.1177/10499091211057049 [DOI] [PMC free article] [PubMed] [Google Scholar]
50.Wiener RS, Kim SYI, Fried TR, et al. Early changes in rates of documented goals-of-care conversations in the Veterans Health Administration during the COVID-19 pandemic. J Gen Intern Med. 2021;36(5):1466-1469. doi: 10.1007/s11606-021-06652-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Uppal A, Silvestri DM, Siegler M, et al. Critical care and emergency department response at the epicenter of the COVID-19 pandemic. Health Aff (Millwood). 2020;39(8):1443-1449. doi: 10.1377/hlthaff.2020.00901 [DOI] [PubMed] [Google Scholar]
52.Fiest KM, Krewulak KD, Jaworska N, et al. ; Canadian Critical Care Trials Group . Impact of restricted visitation policies during COVID-19 on critically ill adults, their families, critical care clinicians, and decision-makers: a qualitative interview study. Can J Anaesth. 2022;69(10):1248-1259. doi: 10.1007/s12630-022-02301-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
53.Schwartz AL, Alsan M, Morris AA, Halpern SD. Why diverse clinical trial participation matters. N Engl J Med. 2023;388(14):1252-1254. doi: 10.1056/NEJMp2215609 [DOI] [PubMed] [Google Scholar]
54.Rogers MM, Chambers B, Esch A, Meier DE, Bowman B. Use of an online palliative care clinical curriculum to train US hospital staff: 2015-2019. J Palliat Med. 2021;24(4):488-495. doi: 10.1089/jpm.2020.0514 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials