Prognosticating Outcomes and Nudging Decisions with Electronic Records in the Intensive Care Unit Trial Protocol

Abstract

Expert recommendations to discuss prognosis and offer palliative options for critically ill patients at high risk of death are variably heeded by intensive care unit (ICU) clinicians. How to best promote such communication to avoid potentially unwanted aggressive care is unknown. The PONDER-ICU (Prognosticating Outcomes and Nudging Decisions with Electronic Records in the ICU) study is a 33-month pragmatic, stepped-wedge cluster randomized trial testing the effectiveness of two electronic health record (EHR) interventions designed to increase ICU clinicians’ engagement of critically ill patients at high risk of death and their caregivers in discussions about all treatment options, including care focused on comfort. We hypothesize that the quality of care and patient-centered outcomes can be improved by requiring ICU clinicians to document a functional prognostic estimate (intervention A) and/or to provide justification if they have not offered patients the option of comfort-focused care (intervention B). The trial enrolls all adult patients admitted to 17 ICUs in 10 hospitals in North Carolina with a preexisting life-limiting illness and acute respiratory failure requiring continuous mechanical ventilation for at least 48 hours. Eligibility is determined using a validated algorithm in the EHR. The sequence in which hospitals transition from usual care (control), to intervention A or B and then to combined interventions A + B, is randomly assigned. The primary outcome is hospital length of stay. Secondary outcomes include other clinical outcomes, palliative care process measures, and nurse-assessed quality of dying and death.

Clinical trial registered with clinicaltrials.gov (NCT 03139838).

Many patients with life-limiting illnesses receive aggressive care in their final months of life (1–3). Aggressive care near life’s end may be misaligned with patients’ goals (4, 5) and is associated with reduced quality of life (6, 7). Moreover, family members of patients who die in intensive care units (ICUs) commonly experience pathological bereavement (6–11). For these reasons, guidelines recommend that clinicians discuss prognosis and offer the option of care focused on comfort for critically ill patients at high risk of death or severely impaired functional recovery (12–14). However, ways to promote adherence to these guidelines have yet to be identified.

It is tempting to believe that specialist palliative care consultation or communication interventions will improve the quality of palliative care in the ICU. And indeed, several efficacy studies have shown that such interventions improve outcomes for critically ill patients under controlled circumstances (15–19). However, to date, larger effectiveness trials of similarly resource-intensive palliative interventions have had mixed results (20–22). Relying on palliative care specialists to see all critically ill patients at high risk of death (23) is neither feasible nor sustainable because of their short supply (24–28), and palliative care educational initiatives for ICU clinicians require considerable upfront investment with uncertain benefit (29, 30). Furthermore, the effectiveness of interventions targeting surrogates may be limited by their modest knowledge of patients’ end-of-life care preferences (31–33) and, often, excessive optimism (34).

By contrast, interventions designed to change ICU clinician behavior hold great promise (35) but have rarely been tested (36). Specifically, increasing the frequency and timeliness of ICU clinicians’ engagement of critically ill patients and their surrogates about their goals and all potential treatment options may better align the care delivered with patient preferences and reduce family distress and bereavement (37–41). Thus, with support from the Donaghue Foundation, we are conducting the PONDER-ICU (Prognosticating Outcomes and Nudging Decisions with Electronic Records in the ICU) study, a multicenter trial representing a collaboration between the University of Pennsylvania and Atrium Health, one of the largest health systems with integrated electronic health record (EHR) technology in the United States. This study’s protocol is designed to test our hypotheses that patient-centered outcomes can be improved by simple, scalable EHR-based interventions that nudge ICU clinicians to document patients’ functional prognoses, offer care focused on comfort, or both.

Conceptual Model for Nudging ICU Clinicians to Improve Palliative Care

Many factors unrelated to patients, such as which ICU clinician is responsible for the patient’s care, which ICU the patient is admitted to, and how busy the ICU is that day, strongly influence whether or not hospitalized patients will receive life support or ongoing critical care (42–44). Similarly, physicians largely determine whether or not comfort care will be offered for critically ill patients (45, 48), with the physician’s belief that life support should be withheld or withdrawn being a highly determinative factor (49, 50). Our conceptual model in Figure 1 stems from evidence of clinicians’ failures to reliably identify seriously ill patients appropriate for palliative approaches to care (42, 49–51) and several well-described barriers clinicians face in initiating these discussions (51–55). Past efforts to improve clinicians’ prognostic awareness and goals-of-care conversations for seriously ill hospitalized patients who have relied on information provision alone, communication-skills training, or specially trained interventionalists have been largely ineffective (21, 56, 57). In contrast, behavioral economic insights offer novel, scalable, and potentially powerful approaches to promote goal-concordant care by targeting specific clinician barriers and “nudging” recommended communication behaviors gently without restricting their choice (58–61). Furthermore, EHR-based nudges have been effective at increasing clinicians’ adherence to guidelines in several practice settings (62–65).

Figure 1.

Conceptual model of barriers to intensive-care-unit clinicians’ offering palliative approaches to care.

Methods

Study Objectives

The objectives of the PONDER-ICU trial are to 1) provide experimental evidence of how two behavioral, EHR-based interventions designed to increase ICU clinicians’ engagement of critically ill patients and surrogates in discussions about the option of comfort-focused care impact patient-centered outcomes; 2) assess the extent to which the effectiveness of each intervention or the combination of both varies among clinicians; and 3) identify patient subgroups for whom the interventions are particularly effective.

Study Design and Setting

PONDER-ICU is a pragmatic, multicenter, stepped-wedge cluster randomized trial in which all hospitals begin in a usual-care control phase and adopt one or the other intervention in a randomly assigned sequence. The stepped-wedge study design was chosen because of certain practical and scientific advantages over traditional parallel cluster randomized designs (66–69). Importantly, given the baseline variability in processes of care across hospitals and ICUs (42, 70–72), the stepped-wedge design enables comparisons of outcomes before and after implementation within hospitals as well as at a given point in time among hospitals, which will have been, at that time, randomly assigned to different interventions (73).

We designed this study in close adherence to the Pragmatic-Explanatory Continuum Indicator Summary-2 criteria for pragmatic clinical trials and the Consolidated Standards of Reporting Trials extension for stepped-wedge cluster randomized trials (74–76). The trial is being conducted in 17 ICUs across 10 hospitals within Atrium Health (formerly Carolinas HealthCare System), one of the nation’s largest healthcare organizations, with more than 40 hospitals across North Carolina, South Carolina, and Georgia. Participating hospitals use an EHR produced by Cerner Corporation, which feeds into Atrium’s centralized data warehouse and connects across all care settings within the health system. Three ICUs are closed units (intensivist as primary attending physician), and 14 are open units (hospitalist or surgeon as primary attending physician) with selective intensivist consultation (72), including for all patients receiving mechanical ventilation (Table 1). Some ICUs also have house staff and/or advanced-practice providers (nurse practitioner or physician assistant) as additional bedside clinicians, and all ICUs in this trial have critical care telemedicine available (24-hour nurse and nocturnal-intensivist support).

Table 1.

Geography and characteristics of the participating Atrium Health hospitals and ICUs (n = 17) in North Carolina

Hospital	Setting	ICU Type	No. of Beds	Staffing Model*†	Additional Bedside Clinicians
Cabarrus	Urban, regional referral	Cardiothoracic	4	Closed	APP
		Cardiac	10	Closed	APP
		Medical–surgical	35	Closed	APP
CMC	Urban, academic center	Cardiothoracic	14	Open	APP; house staff‡
		Cardiac	20	Open	APP; house staff
		Medical	29	Open	APP; house staff
		Neurosurgical	29	Open	APP; house staff
		Trauma	29	Open	APP; house staff
CMC-Mercy	Urban, community	Medical–surgical	20	Open	None
Blue Ridge–Morganton	Rural	Medical–surgical	10	Open	APP
Cleveland	Rural	Medical–surgical	18	Open	None
Lincoln	Rural	Medical–surgical	10	Open§	None
Pineville	Urban, community	Cardiac	14	Open	None
		Medical	16	Open	None
Stanly	Rural	Medical–surgical	10	Open	None
Union	Suburban	Medical–surgical	14	Open	None
University City	Urban, community	Medical–surgical	8	Open	None

Definition of abbreviations: APP = advanced-practice provider; CMC = Carolinas Medical Center; ICU = intensive care unit.

^*Closed ICUs have intensivists as the primary attending physicians; open ICUs have hospitalists or surgeons as the primary attending physicians, with required intensivist consultation for patients receiving mechanical ventilation or vasoactive infusions and for those with a life-threatening emergency.

^†All participating ICUs have critical care telemedicine available, including 24-hour nurse and nocturnal-intensivist support.

^‡House staff may include resident physicians and/or fellows.

^§No bedside intensivist available.

The 10 study hospitals were randomly grouped into five clusters of 2 hospitals each. Every 3 months, a new hospital cluster adopts the interventions in a sequence determined by computerized random-number generation, and within each cluster, one hospital is randomly assigned to begin with intervention A and the other is assigned to begin with intervention B (Figure 2). All hospitals contribute a minimum of 5 months of usual-care (control) data before adopting either intervention; by the end of the 33-month study period, all hospitals will have been using both interventions combined for at least 4 months. This design enables comparisons of each intervention alone and both interventions combined with usual care (77).

Figure 2.

Schematic of the PONDER-ICU (Prognosticating Outcomes and Nudging Decisions with Electronic Records in the Intensive Care Unit) pragmatic, stepped-wedge cluster randomized trial. The sequence and timing with which each hospital transitions between phases is randomly assigned. Wedge intervals are 3 months after all hospitals spend a minimum of 5 months in the usual-care phase; the total trial duration is 33 months.

Identification of Eligible Critically Ill Patients

Discussions regarding goals of care, and the option of focusing primarily on comfort in particular, are wrought with complexities and may be distressing to patients and surrogates (8, 10, 11, 78, 79). Such discussions also require substantial time and energy from clinicians and thus must be reserved for the patients most likely to benefit from them. We sought to identify adult patients likely to benefit from a timely discussion of prognosis and palliative care options (26) and to include a diverse population to promote generalizability. Existing evidence suggests that patients with preexisting life-limiting illnesses are at high risk of mortality in critical illness with organ failure (80–85), and among survivors, few return to their baseline functional status (86). Thus, eligible patients include adults (≥18 yr) with a preexisting life-limiting illness (amyotrophic lateral sclerosis, chronic obstructive pulmonary disease cirrhosis, congestive heart failure, dementia, end-stage renal disease, leukemia or lymphoma, pulmonary fibrosis/interstitial lung disease, or solid organ malignancy) as determined by the presence of one or more applicable International Classification of Diseases Clinical Modification code(s) (see Table E1 in the online supplement) within the prior 365 days and acute respiratory failure requiring continuous mechanical ventilation for ≥48 hours (Figure E1).

Eligible patients are identified using an EHR algorithm that includes a real-time calculation of mechanical ventilation duration using the ventilatory data entered by bedside clinicians. To validate the algorithm, we conducted a blinded independent review of 100 medical charts randomly selected from across the 17 participating ICUs. The algorithm identified 51 of these patients as eligible and 49 as ineligible. Using the investigators’ blinded chart review as the gold standard, the EHR eligibility algorithm had a false-positive rate of 0% (i.e., all 51 patients deemed eligible truly met the criteria) and a false-negative rate of 2% (i.e., 1 of the 49 patients deemed ineligible was actually eligible).

Interventions

The interventions each consist of two simple queries that ICU clinicians complete directly in an eligible patient’s EHR, where they are subsequently saved and viewable as a document. For this trial, the ICU clinician is defined as the attending physician (intensivist, hospitalist, or surgeon) and the advanced-practice provider, when applicable, who is primarily responsible for the patient’s medical decision-making at the time of enrollment. In pilot testing among ICU clinicians at two health systems uninvolved in the trial, the time to complete the two queries for each intervention was approximately 2 minutes or less.

Focusing effect (intervention A)

In many contexts, framing choices in ways that focus people on certain elements of the decisions they face can substantially change behavior (87, 88). In a nationwide, scenario-based randomized trial, simply requiring ICU physicians to document a patient’s short-term functional prognosis significantly increased their willingness to discuss the possibility of withdrawal of life support (89). In a follow-up simulation study, requiring ICU physicians to estimate patients’ functional outcomes again changed certain behaviors (90). In this simulation study, although physicians’ likelihood of specifically offering comfort-oriented care was unchanged, they were more likely to disclose a poor prognosis in an initial family meeting. Our trial leverages this focusing effect by prompting clinicians to prognosticate patients’ functional outcomes 6 months later, selecting from options ranging from death to the ability to live at home independently. Although clinicians are required to document this prognostication in the EHR, they are not required as part of the trial to share this information with patients or surrogates.

Accountable justification (intervention B)

Physicians are trained to make decisions backed by reason. Accordingly, prior work has shown that requiring physicians to provide justification for their decisions reduces overuse of resources (91) and inappropriate prescribing practices (64). By requiring physicians to document in the EHR their reasons for not having offered a critically ill patient or their surrogate the option to pursue comfort-oriented care, physicians may reflect more deeply on all of the reasonable care options for the patient and be more likely to follow guidelines to have such discussions. Intervention B prompts clinicians to document whether they have offered the patient or surrogate the option for care focused primarily on comfort and, if not, to explain why not in a free-text box. If clinicians decline to provide a justification, the phrase “No justification given” is automatically populated in the saved electronic document. Visibility of clinicians’ responses in the EHR is essential because lack of public accountability may diminish such interventions’ effects (92–94).

Study Procedures and Data Collection

Enrollment

In all phases of the study, the EHR patient-identification algorithm runs four times per day (00:00, 06:00, 12:00, and 18:00) (Figure 3). This frequency was selected in collaboration with the Atrium Health Information and Analytic Services team to balance the scientific goal of maximizing enrollment opportunities and the operations goal of minimizing computational resource burdens on Atrium’s shared information systems. During the intervention phases of the trial, the appropriate queries are automatically generated and are immediately available for completion in the clinicians’ routine EHR document workflow at the time a patient becomes eligible. Clinicians can complete the queries any time up until an eligible patient either dies in the ICU, is discharged from the ICU, or is no longer receiving mechanical ventilation.

Figure 3.

Patient-identification and intervention-delivery schema. *Eligible life-limiting illnesses: amyotrophic lateral sclerosis, chronic obstructive pulmonary disease, cirrhosis, congestive heart failure, dementia, end-stage renal disease, leukemia or lymphoma, pulmonary fibrosis/interstitial lung disease, or solid organ malignancy. EHR = electronic health record; ICU = intensive care unit; VCC = virtual critical care.

To promote adherence to the interventions, ICU clinicians begin receiving an EHR reminder alert at 14:00 after eligibility determination whenever they attempt to place an order for a patient with incomplete queries until they are completed. This time was chosen to minimize disruptions for newly eligible patients during morning multidisciplinary rounds when ICU clinicians tend to be focused on making acute care plans. Clinicians may bypass the alert and proceed with their intended order if they indicate that there is a clinical emergency (alerting resumes with the subsequent order) or that they are not the primary ICU clinician for the patient. In addition, the telemedicine nursing team is automatically notified of incomplete intervention queries each day. This minimally intrusive monitoring enables the nurses to send e-mail reminders to the appropriate ICU clinician(s).

Trial Regulation and Engagement

The chief medical officers and ICU medical directors at each hospital were notified about the trial 3 months before and again 1 week before initiation of the usual-care phase. ICU clinicians were notified about the trial 1 month before initiation of the usual-care phase and again 2 weeks before their hospital switched to an intervention phase using Atrium’s established communication channels, including e-mail, flyers, and leadership meetings. This trial is being conducted under a waiver of the requirement for informed consent for both patients and clinicians on the basis of the criteria set forth by the “Common Rule.” (95) This waiver, and the entire trial protocol, were approved by the institutional review boards of the University of Pennsylvania and Atrium Health. Additional methods regarding the human research protections and data management plan for this study are described in the online supplement.

Outcome Measures

Primary Outcome

We have chosen hospital length of stay (LOS) as our primary outcome because it can be uniformly captured within the EHR, provides high statistical power, and reflects patient- and family-centered priorities (4, 96–98). Reducing LOS is also a critical operational and financial metric for hospitals (16, 98, 99). Our measure of hospital LOS conservatively ranks deaths as long hospital stays, partially on the basis of our prior work showing that this approach avoids the selection bias or informative censoring associated with other methods of handling death in LOS analyses (100) and that patients’ and families’ considerations of tolerable amounts of time spent in the hospital depend on whether the patient survives (101, 102).

Hospital LOS is defined as starting from the time of patient enrollment. We chose enrollment rather than hospital admission on the basis of our prior work showing biases attributable to “immutable time”—that is, time in the hospital preceding deployment of an intervention that cannot be affected by the intervention (103). In primary analyses, death is ranked at the longest end of the LOS distribution. Sensitivity analyses assess how the results may change with different rankings of death (100)—in this case, by ranking death at the 75th, 85th, and 95th percentiles of the LOS distribution.

Secondary Outcomes

Our secondary process and clinical outcomes include palliative care consults, withdrawal of life support, ICU LOS, and hospital mortality and discharge disposition (Table 2). Most secondary outcomes are collected from the EHR, except for a nursing assessment of the end-of-life experience among enrolled patients who die in the hospital. The widely-used Quality of Dying and Death questionnaire has been validated among a diverse critically ill population and has been adapted and tested for nurse completion (104–107). The primary bedside nurse at the time of a patient’s death is invited via e-mail within 48 hours to complete the single-item version of the Quality of Dying and Death questionnaire (108) using the Research Electronic Data Capture platform (109).

Table 2.

Secondary process and outcome measures

Outcome Measure	Variable Definition and Coding
Change in code status	Categorical: no change, new limitations on life support (e.g., DNAR, DNI), or discontinued limitations on life support
Initiation of additional forms of life support (e.g., acute dialysis, gastrostomy tube)	Binary; yes if CPT code 90935, 90937, 49440, 49441, or 43830
Palliative care consult	Binary; yes if palliative care consult order
Time to palliative care consult	Continuous (h); time starts at enrollment
Palliative withdrawal of mechanical ventilation	Binary; yes if time of comfort-care order* precedes time of hospital death
Receipt of CPR	Binary; yes if CPT code 92950
ICU mortality	Binary; yes if death occurred in the ICU or within 1 d of transfer out of ICU
ICU length of stay	Continuous (h); time starts at enrollment
ICU readmission	Binary; yes if readmitted to any ICU during same hospital encounter
Duration mechanical ventilation	Continuous (h); time starts at enrollment
Time to withdrawal of life support	Continuous (h); time starts at enrollment and ends at time of comfort-care order
Hospital discharge disposition	Categorical
Quality of Dying and Death questionnaire (1 item)†	Continuous; nurse-reported postmortem
30-d hospital readmission‡	Count
30-d all-cause mortality§	Binary; time starts at enrollment
90-d all-cause mortality§	Binary; time starts at enrollment
180-d all-cause mortality§	Binary; time starts at enrollment

Definition of abbreviations: CPR = cardiopulmonary resuscitation; CPT = Current Procedural Terminology; DNAR = do not attempt resuscitation; DNI = do not intubate; ICU = intensive care unit.

^*Sensitivity 85.7% among 112 cases with palliative withdrawal of mechanical ventilation across participating ICUs.

^†“Overall, how would you rate the quality of the patient’s dying?” (score ranges from 0 = terrible experience to 10 = almost perfect experience) (102).

^‡Includes admission to any Atrium Health acute care hospital.

^§Date of death determined from the electronic health record primarily and Social Security Death Index database secondarily (119).

Study Analyses

Primary Analytic Approaches

We will use the intention-to-treat (ITT) approach for all primary and main secondary analyses of the effectiveness of each intervention, such that all eligible patient encounters will be evaluated, regardless of clinician adherence to the EHR queries during the intervention phase. Because of anticipated skewness of the LOS distribution, we anticipate transforming to its log value before analysis. On the basis of data from participating ICUs during 2016, we anticipate the eligible cohort for this trial will have a mortality rate around 30%. With this high mortality rate, ranking death at or near the longest LOS produces a distribution of the primary outcome with a large spike at the end of the right skewed tail that is not mitigated by taking the log value of LOS. Thus, we will analyze the primary outcome using simultaneous quantile regression at the 20th, 30th, 40th, 50th, 60th, and 70th percentiles (110). This quantile-regression approach is robust because it makes no distributional assumptions (111). Testing above the 70th quantile is unnecessary if roughly 30% of patients die. This approach can efficiently identify plausible intervention effects by enabling a joint test of the overall significance across all quantiles, such that the null hypothesis is that there is no effect of treatment at any of the quantiles (112). In addition, this approach provides effect estimates at each individual quantile, about which we will construct 95% confidence intervals using bootstrapping. For the three planned comparisons (intervention A vs. control; intervention B vs. control; interventions A + B vs. control), we will adjust significance levels for multiple comparisons using the Holm method (113).

To augment precision in estimating treatment effects in this stepped-wedge cluster trial, primary analyses will be conducted with adjustment for prespecified patient- and encounter-level factors that exist before randomization (Table E2), so long as there is not substantial missingness in these factors (114, 115). Unadjusted analyses will also be reported. We will include fixed effects for clinician, hospital cluster, and time (69, 116). We will also include “time from enrollment” as a patient-level covariate to account for the possible differences in outcomes related to natural illness progression during a hospitalization that may influence processes of care (e.g., more family meetings), independently of any effect of the interventions or of calendar time. In preparing our final model, we will evaluate whether three different interaction terms change our results. First, we will include an interaction term for cluster and time to account for the possibility that the time effect might not be the same for all clusters. Second, we will investigate the variation in treatment effect across hospital clusters by including an interaction term between the intervention and cluster. Third, because of the “factorial” stepped-wedge design, we will test for the interaction between interventions A and B (117).

Because of the delay (a minimum of 2 h for patients enrolled at 12:00 up to a maximum of 20 h for patients enrolled at 18:00) between eligibility determination (when the intervention queries are available in the EHR for completion) and initiation of the clinician reminder alerts at 14:00 if queries are not completed, it is possible that clinicians will never receive alerts for some enrolled patients (i.e., those who die in the ICU, are no longer receiving mechanical ventilation, or are discharged from the ICU between enrollment and 14:00). Because active-alert reminders could plausibly increase adherence to (and therefore, potential effectiveness of) the interventions, we will perform a modified ITT analysis among patients who remained eligible through 14:00 after enrollment. Because all intervention-phase patient encounters included in this secondary analysis would have been subject to at least one active clinician alert, this analysis examines what the effect of the intervention(s) would be if active alerting were universal.

Analyses Accounting for Nonadherence

Because ICU clinicians may bypass the EHR alerts, there may be differences between the interventions’ effectiveness (i.e., effects among all patients to whom the interventions are targeted) and efficacy (i.e., effects among patients whose clinicians adhere to the interventions). Therefore, we will evaluate efficacy in secondary analyses using complier-average-treatment-effect (CATE) methods (118–120). Unlike per-protocol analyses, CATE analyses are not biased by selection effects, and have been shown to be valid inferential approaches in stepped-wedge trials (121). These analyses use data on all participants in the primary analytic sample and model the randomization arm as an instrumental variable to conduct an instrumental-variable quantile-regression analysis for the effect of the interventions on the median LOS (122). In sensitivity analyses, we will also explore effects on the 75th and 90th percentiles of the primary outcome distribution with fixed effects for cluster and time.

Analyses of Potential Effect Modifiers

We will assess whether prespecified characteristics of patients (age, sex, presence of a cancer diagnosis, race, and ICU admission source) or clinicians (sex, specialty board certification, years in practice) modify the sizes of the effects of each intervention on the primary outcome by conducting stratified analyses. If differences appear, we will formally evaluate for effect modification by testing the significance of statistical interaction terms between the potential effect modifier and the study phase (control, intervention A, intervention B, or combined) on the primary outcome.

Approach to Missing Data

We do not anticipate substantial missing data because the primary outcome and most secondary outcomes will be obtained or calculated from the hospitals’ routinely collected data in the EHR. Nonetheless, we will explore and potentially adjust for missing values using pattern-mixture methods in secondary analyses (123, 124).

Statistical Power Calculations

On the basis of pretrial data in 2016 across all participating ICUs, we estimate that approximately 4,750 patients will be eligible for inclusion during the 33-month trial period. We found that such patients had a mean total hospital LOS of 16.2 days (standard deviation, 11.8 d) and a median hospital LOS of 12.6 days, and we found that 31.5% died in the hospital. Ranking deaths at the longest LOS changes the mean and median to 28.0 days and 22.4 days, respectively. Using the conservative assumptions of intraclass correlations of patient outcomes of up to ρ = 0.10 within clinicians and ρ = 0.05 within ICUs, and conservatively estimating that our actual enrollment may lag our projections by as much 25% (i.e., 3,500–3,600 patients), the minimal detectable difference in the primary outcome between either intervention and control corresponds to a 13.4% decrease in the median hospital LOS (or roughly 3 d). Our design also provides at least 80% power to detect an additive effect of interventions A and B (i.e., that the two interventions combined will produce at least twice the effect of either intervention alone). Because these power calculations did not adjust for patient-level covariates, the true minimal detectable difference may be smaller.

Interim Analysis and Safety Monitoring

We convened a Data and Safety Monitoring Board comprising experts in critical care, palliative care, and biostatistics to help monitor and address anticipated and unforeseen risks arising during the study. The Data and Safety Monitoring Board and study investigators agreed not to establish formal stopping rules, given that most deaths are likely to reflect goal-concordant care, but we are monitoring time to withdrawal of life support and 30-, 90-, and 180-day all-cause mortality rates using the EHR or Social Security Death Index database (125) as safety endpoints to assess for persistent differences between arms over time. There will be one planned, blinded interim analysis of the primary outcome and these safety endpoints after approximately two-thirds of patients have been enrolled to enable sufficient accrual and follow-up in each intervention arm for comparison with the control.

Discussion

Study Limitations

There are several limitations to acknowledge in this highly pragmatic trial with EHR-based interventions. First, a fundamental challenge of pragmatic trials is designing a flexible yet effective intervention-adherence plan to enhance fidelity (126). To address this, we have designed a systematic approach of multifaceted clinician reminders for timely completion of the EHR intervention(s), and we will perform a secondary efficacy (CATE) analysis among patients who would have received the intervention if assigned to it. Second, the interventions used in this trial may be less effective among clinicians who experience distress about or lack skills in engaging families in goals-of-care discussions (127, 128). In keeping with our pragmatic approach, we did not employ resource-intensive clinician education (30, 129) or collect self-reported characteristics. However, such clinician-level effects will be estimated in the primary ITT analysis and secondary effect-modification analyses. Finally, we are not capturing patient- and family-reported outcomes, largely because of infeasibility in a trial conducted with a waiver of informed consent, the large sample size, and the anticipated nonrandom missingness and nonresponse bias.

Potential Outcomes and Conclusions

The two interventions being tested in this trial build from a strong evidence base for changing clinicians’ behaviors and represent the first applications of these approaches for promoting palliative approaches to care for patients in whom such care is recommended. As a randomized trial within one of the nation’s largest integrated health systems, the PONDER-ICU trial is powered to detect differences in outcomes that are important to patients, families, hospitals, and payers. Most importantly, the simple EHR-based interventions being tested are highly scalable, thereby facilitating widespread adoption if they are shown to be effective.