Rationale and Design of the Randomized Evaluation of Default Access to Palliative Services (REDAPS) Trial

Abstract

The substantial nationwide investment in inpatient palliative care services stems from their great promise to improve patient-centered outcomes and reduce costs. However, robust experimental evidence of these benefits is lacking. The Randomized Evaluation of Default Access to Palliative Services (REDAPS) study is a pragmatic, stepped-wedge, cluster randomized trial designed to test the efficacy and costs of specialized palliative care consultative services for hospitalized patients with advanced chronic obstructive pulmonary disease, dementia, or end-stage renal disease, as well as the overall effectiveness of ordering such services by default. Additional aims are to identify the types of services that are most beneficial and the types of patients most likely to benefit, including comparisons between ward and intensive care unit patients. We hypothesize that patient-centered outcomes can be improved without increasing costs by simply changing the default option for palliative care consultation from opt-in to opt-out for patients with life-limiting illnesses. Patients aged 65 years or older are enrolled at 11 hospitals using an integrated electronic health record. As a pragmatic trial designed to enroll between 12,000 and 15,000 patients, eligibility is determined using a validated, electronic health record–based algorithm, and all outcomes are captured via the electronic health record and billing systems data. The time at which each hospital transitions from control, opt-in palliative care consultation to intervention, opt-out consultation is randomly assigned. The primary outcome is a composite measure of in-hospital mortality and length of stay. Secondary outcomes include palliative care process measures and clinical and economic outcomes.

Clinical trial registered with www.clinicaltrials.gov (NCT02505035).

Although most seriously ill patients wish to avoid aggressive and burdensome care (1–3), many receive intensive, surgical, and emergency treatments in the final months of their lives (4–7). This is particularly true for patients with chronic obstructive pulmonary disease (COPD) and other nonmalignant illness (8–13). This mismatch between the care patients want and the care they receive stems, in part, from clinicians’ failure to elicit patients’ values and goals in a timely manner, to adequately manage distressing symptoms, and to provide patient-centered recommendations (14–19). To address this problem, palliative care—that is, specialized medical care for difficult-to-manage symptoms, complex family dynamics, and challenging medical decisions (20–24)—is widely advocated.

The potential for palliative care to improve quality and help control costs has captured the attention of clinicians, hospitals, payers, and policy makers around the world (25). However, large-scale, experimental evidence is lacking regarding the effectiveness and costs of inpatient specialty palliative care services. With support from the National Institute on Aging, the Randomized Evaluation of Default Access to Palliative Services (REDAPS) trial is the largest prospective study ever conducted in palliative care. This multicenter study involves a collaborative effort between the University of Pennsylvania and Ascension, the largest nonprofit health system in the United States.

The purpose of this article is to describe the design and rationale for the REDAPS trial. The protocol was derived from the hypothesis that improved patient-centered outcomes can be achieved without increasing costs by simply changing the default for inpatient palliative care consultation from an opt-in to an opt-out system for patients with advanced COPD and other life-limiting illnesses. This trial is particularly timely, given recent attention to the infrequency with which patients with COPD receive palliative services (26).

Overview of Hospital-based Palliative Care

Team-based palliative care consultation is now available in more than 70% of U.S. hospitals with at least 50 beds (27). Several studies suggest that inpatient palliative care improves some clinical and patient-reported outcomes (28–33) and reduces hospital costs (34–38). However, none of these studies are sufficiently large or diverse to establish the effects of specialized palliative care consultation on clinical and economic outcomes in real-world settings (39) or to answer critical questions regarding optimal deployment of these services.

Structure and Processes of Palliative Care Consultation

Guidelines suggest that optimal palliative care teams are multidisciplinary and address multiple domains of care (40–43). Despite strong conceptual appeal, these recommendations are not backed by evidence regarding which disciplines and services optimize the effectiveness of palliative care. Such uncertainty may contribute to the considerable heterogeneity in the structure and processes of palliative care teams (44, 45).

Moreover, it is estimated that there is presently one palliative care physician for every 1,200 patients living with serious illness in the United States (46) and no possibility of rapidly increasing this supply (47). Thus, it is essential to understand which services can be delivered equally well or better by nonphysician team members.

Patients with Palliative Care Needs

Different types of patients are likely to derive different degrees of benefit from palliative care, yet it is uncertain which hospitalized patients benefit most from palliative care consultation. Patients’ specific underlying illnesses, stages of these illnesses, locations within the hospital, and social situations may all modify the need for and effectiveness of palliative care. However, clinicians often fail to identify unmet needs for seriously ill patients or to initiate appropriate responses in a timely manner (14–17, 48). Triggers and other efforts to surmount these problems are unlikely to be effective or sustainable without evidence to suggest which patients are most likely to benefit from which services.

Penetration of Palliative Care

Patients with cancer and many noncancer diagnoses have similar symptom burdens and prognoses (49–53). However, patients with noncancer diagnoses are less likely to receive palliative care (10, 54, 55). In preparation for this trial, we queried the electronic health records (EHRs) of two Ascension hospitals and found that only 4.6% of patients with oxygen-dependent COPD received a palliative care consult during a hospitalization in 2013. Rates among patients with end-stage renal disease (ESRD) and advanced dementia were slightly higher at 9.4 and 6.4%, respectively (unpublished data). Such penetration is much lower than published rates for patients with cancer (10, 54, 56) and suggests that the current “opt-in” model, whereby primary clinicians must identify patients in need of specialty palliative care and actively order such consults, may not be sufficient (56).

Behavioral economic theory predicts that as interventions to increase palliative care consultation impose greater restrictions on choice, their effectiveness may also increase (Table 1). One such intervention is to change the default option, or the events that will be set into place if no alternative is actively chosen. Manipulating defaults is appealing because doing so guides choice without restricting any options (57). Strategic manipulation of default or “opt-out” options in healthcare has been shown to increase influenza vaccination and organ donation rates (58–60) and influence seriously ill patients’ choices for interventions at the end of life (61). Thus, making palliative care consultation the default for appropriate patients may increase its uptake more than triggering initiatives (62–65), which still require clinicians to actively choose to order a consult.

Table 1.

Hierarchy of behavioral interventions to affect clinicians’ choice regarding palliative care consultation

Minimize Restriction of Choice	Types of Intervention	Example Study	Consult Rate	Maximize Effectiveness
	Eliminate choice: require PCC for all seriously ill patients	Gade et al. 2008 (32): Mandatory PCC for all patients over age 18 years with a life-limiting diagnosis and prognosis of <1 year*	100%
	Guide choice through changing the default: make PCC the default for triggered patients	REDAPS trial: Electronically triggered PCC (with physician opt-out) for hospitalized patients with advanced COPD, ESRD, or dementia	N/A
	Enable choice: physician opt-in for a PCC for triggered patients	Rocque et al. 2015 (62): Physician-initiated PCC for patients with advanced solid malignancy and unplanned admission	60%
	Do nothing: usual care/no systems intervention	Szekendi et al. 2016 (56): Point prevalence of PCC at 33 U.S. hospitals for patients with advanced cancer, CHF, or COPD	39%

Definition of abbreviations: CHF = congestive heart failure; COPD = chronic obstructive pulmonary disease; ESRD = end-stage renal disease; N/A = not available; PCC = palliative care consultation; REDAPS = Randomized Evaluation of Default Access to Palliative Services.

Adapted by permission from Reference 94.

^*Physicians could decline participation in the trial for a patient but could not decline consultation for patients in the trial.

The REDAPS Trial Design and Methods

Study Objectives

The aims of the REDAPS trial are to (1) provide experimental evidence of the real-world effectiveness of inpatient palliative care consultation in improving a number of patient-centered outcomes; (2) compare the specific effectiveness of different palliative care team structures and services offered; (3) identify patient subgroups for whom palliative care consults are particularly effective; and (4) test the incremental effectiveness and costs of a simple, scalable method to increase the penetration of palliative care consultation.

Study Design

REDAPS is a pragmatic, stepped-wedge cluster randomized trial in which hospitals adopt the intervention of electronically triggered, default palliative care consultation orders for eligible patients at randomly assigned times (wedges) (Figure 1). As the intervention entails a system-level modification of clinician behavior, this study design is preferable to alternate options that would randomize individual patients or use parallel or crossover cluster randomized designs because it (1) staggers the time required to prepare each hospital, (2) increases statistical power, (3) ensures that all hospitals ultimately implement the intervention, and (4) maintains at least the usual level of access to palliative care consultation (66). Eight of the 11 participating hospitals will adopt the intervention in 2.7-month intervals. The other three hospitals have linked EHR operations requiring a contracted interval of 1.5 months between them for technical feasibility. The 11 hospitals were chosen because they all use an EHR produced by Cerner Corporation (Kansas City, MO) that feeds into a centralized data warehouse at Ascension central offices.

Figure 1.

Schematic of the stepped-wedge randomized trial. Wedge intervals are 2.4 months, representing the average of eight hospitals having 2.7-month intervals between them and the other three electronic health record–linked hospitals having 1.5-month intervals.

The sequence in which hospitals adopt the intervention was determined by computerized random-number generation, with block randomization of the three hospitals with linked EHRs. All hospitals contribute at least 3.5 months of control data before adopting the intervention. Similarly, the end of the 31-month study period includes a 3.5-month period during which all hospitals are using the intervention. This design enables comparisons of outcomes before and after implementation within hospitals as well as comparisons among hospitals at given time points. Although allocation concealment from hospitals and staff is infeasible, given the intervention is a default palliative care consult order that is freely visible within a patient’s EHR, all analyses will be performed by study personnel blinded to the study phase at each hospital.

Identification of Seriously Ill Patients and Validation of the Method

The eligibility criteria were chosen to include patients (1) with sufficiently complex needs that they are likely to benefit from specialty palliative care, (2) who could be identified relatively easily from the EHR, and (3) who differ from the populations most commonly included in prior or ongoing studies of palliative care. Thus, eligible patients include those aged 65 years or older with a diagnosis of advanced COPD, ESRD, or dementia. “Advanced” stage is defined by chronic home oxygen use or two or more hospitalizations within 12 months (COPD); chronic dialysis therapy (ESRD); and admission from a long-term care facility, a surgical feeding tube in place, or two or more hospitalizations within 12 months (dementia). These diagnoses were all identified as portending a high risk of unmet needs (21). Patients with a hospital length of stay (LOS) less than 72 hours are excluded from intent-to-treat analyses because the timing of the intervention is such that it may not be expected to influence outcomes for these patients.

An EHR-based algorithm to identify eligible patients was constructed based on International Classification of Diseases, 9th and 10th Revisions, Clinical Modification codes (67–69) (see Table E1 in the online supplement) that are present on admission. In addition, nurses complete a five-item electronic checklist during intake to denote the disease-specific eligibility criteria. To validate the algorithm, we reviewed 271 medical charts across the participating hospitals. The algorithm identified 171 of these patients as eligible and 100 as ineligible. Using manual chart review as the gold standard, the algorithm had a false-positive rate of 1% and a false-negative rate of 5%.

Intervention

During the intervention phase of the trial at each hospital, a palliative care consult order is entered by default for all eligible patients. It is initially an inactive “standing order” that is generated with a future start date. Physicians caring for patients who trigger the default order are notified and instructed on how to cancel it if they wish. If it is not cancelled within 24 hours, the standing order becomes active. In keeping with the principles of pragmatic trials testing real-world effectiveness of interventions, palliative care teams retain discretion regarding prioritization of patients, provision of services, and documentation practices within a standardized palliative care consultation form.

One year before trial launch, a clinical governance group was formed with representation from the palliative care, nursing, and informatics disciplines at each study hospital. The clinical governance group will provide in-service conferences and webinars to all palliative care staff 3 months before the intervention phase at each hospital. All other clinical staff will be notified about the study 2 weeks before launch of the intervention using the individual hospitals’ established communication channels, including e-mails, posters, leadership meetings, and weekly communications. These notifications will continue throughout each hospital’s intervention period.

On the basis of preliminary data on current rates of palliative care consultation among eligible patients, and the palliative care staffing at the participating hospitals (reported preintervention and annually thereafter), we anticipate a 16.4% overall increase in consult volume for the palliative care teams during the intervention phase. Each participating hospital has indicated that it can handle this modest increase during the trial period, but we will nonetheless monitor for strain on workforce capacity throughout the trial. If concerns arise, we will work with individual hospitals to implement feasible solutions.

Study Procedures and Data Collection

Enrollment

We define the calendar day of admission as Day 0. The patient-identification algorithm begins to run after 15:00 on hospital Day 1 (i.e., 15 to 39 h after admission) during the control and intervention phases of the trial (Figure 2). During the intervention phase only, the default palliative care order is automatically placed once eligibility is determined. Physicians have up to 24 hours to cancel the order before it becomes active after 15:00 on hospital Day 2 (i.e., 39 to 63 h after time of admission). Physicians are required to provide a reason if they cancel the order. During both phases of the trial and regardless of eligibility, a physician may choose to order a palliative care consult for any patient.

Figure 2.

Consort diagram for the Randomized Evaluation of Default Palliative Care Services (REDAPS) trial. The presence of an eligible diagnosis is determined from electronic health record data mining. For reference, Day 0 equals the calendar day of admission. COPD = chronic obstructive pulmonary disease; EHR = electronic health record; ESRD = end-stage renal disease; ITT = intention to treat; LOS = length of stay; PC = palliative care; PCC = palliative care consult.

Human Research Protections and Ethical Considerations

There is no evidence that palliative care consultative services, as used in standard practice, carry any untoward effects for patients. Thus, the REDAPS trial is being conducted under a waiver of the requirement for informed consent on the basis of the criteria set forth by the “Common Rule” (70). Specifically, the risk to subjects of participating in this study is no more than minimal because: (1) the intervention consists of the default ordering of a palliative care consultation, which is within the standard of care for hospitalized patients with life-limiting illnesses; and (2) the intervention does not restrict the options available to the patient, family, or physician. Furthermore, this study cannot be practicably conducted without a waiver of the requirement for informed consent.

The trial protocol, including all enrollment and data-handling procedures, is compliant with the Health Insurance Portability and Accountability Act and was approved by the University of Pennsylvania Institutional Review Board, Ascension’s ethics department, and the medical executive committees at all 11 hospitals.

Data Handling and Repository

Data routinely collected in the course of providing clinical care will be extracted from the Cerner EHR at each hospital and the Premier database. Premier is a central data warehouse containing Ascension’s patient-level billing and administrative data reported by individual hospitals. The Data Coordinating Center (DCC) at the University of Pennsylvania uses secure data transfer protocols to receive a limited data set, as defined under the Health Insurance Portability and Accountability Act Privacy Rule (71), monthly from the central Clinical Information Systems team at Ascension and the Care Excellence team (Ascension’s centralized data repository responsible for routine analysis of Premier data for the health system). Data transmission to the DCC from Ascension is governed by a data use agreement and maintained securely, with access limited to authorized users.

Outcome Measures

Primary Outcome

We have chosen a primary outcome measure that (1) can be uniformly captured within the EHR (minimizing outcome misclassification and missing data), (2) provides high statistical power (as a continuous variable), and (3) is considered by many to be patient and family centered. Specifically, we will use a combined metric of two traditional EHR-based outcomes, hospital mortality and LOS. In addition to being important to patients and families (2, 72, 73), LOS is also a critical operational and financial metric for hospitals (74–76) and palliative care consultative services (42, 74).

However, several studies suggest that for hospital LOS to be a true patient-centered outcome it must be considered hand-in-hand with hospital survival (77–80). Furthermore, traditional approaches to analyzing LOS data that ignore death, stratify by it, or censor on it violate important statistical principles or lead to ambiguous results (75). Thus, we recently completed methodological work showing the statistical merits of a composite outcome that ranks deaths along the LOS distribution, typically at or near the longest end of this distribution, combined with sensitivity analyses to assess the impact of different rankings of death (81). This approach avoids the survivor selection bias associated with stratifying on death by comparing the entire intervention group with the entire control group, avoids the informative censoring associated with survival analyses that censor on death, and allows the analysis to be adjusted to show how the results vary according to whether death is treated as the worst possible outcome or as preferable to an extremely long LOS.

Secondary Outcomes

Using Ascension’s mature EHR platform, we will measure a robust slate of outcomes that are either directly patient centered or highly correlated with patient- and family-centered outcomes. These include numerous clinical, economic, and palliative care process-related outcomes (Table 2). In addition, Cerner Corporation will use a natural language processing algorithm to capture nondiscrete but essential data within clinician narratives to enhance the identification and accuracy of coding of several palliative care process-related and symptom-based outcomes. The development of the natural language processing algorithm is described further in the online supplement.

Table 2.

Secondary process and outcome measures

Process or Outcome	Variable Coding or Calculation
Process measure
Documentation of goals of care	Binary (coded by NLP algorithm)
Documentation of family meetings	Binary (coded by NLP algorithm)
Documentation of durable power of attorney, surrogate, or proxy	Binary (coded by NLP algorithm)
Documentation of pain assessment	Binary
Palliative care team visits per patient	Restricted to patients receiving consultation
Use of bowel regimen for patients on opioids	Binary; will be coded as present if a contraindication was documented
Clinical outcome
Pain scores (excluding patients with dementia)	Scores are standardized within hospitals
Dyspnea	Binary (coded by NLP algorithm)
Code status (most recent at time of death or discharge)	Categorical; full, do not resuscitate, do not intubate
Hospital mortality	Binary; yes if death occurred in the hospital (excluding ICU) or patient transferred to inpatient hospice and died within 24 h
ICU mortality	Binary; yes if death occurred in the ICU or patient transferred from ICU to inpatient hospice and died within 24 h
Transfer to ICU after randomization	Binary
CPR after randomization	Binary
Days of mechanical ventilation	Ordinal
Hospital discharge disposition	Categorical; home, home with home care, home hospice, inpatient hospice, nursing facility, long-term acute care facility, other
30-d hospital readmissions*	Binary
Economic outcome
Direct cost per hospitalization	Continuous; nonlinear
Direct cost per day	Continuous; nonlinear

Definition of abbreviations: CPR = cardiopulmonary resuscitation; ICU = intensive care unit; NLP = natural language processing.

^*Limited to readmissions at an Ascension hospital.

Study Analyses

Primary Analytic Approaches

The primary and main secondary analyses of the effectiveness of palliative care consultation will use the intention-to-treat approach, in which all patients meeting eligibility criteria during a time period are included as randomized, regardless of adherence to the assigned strategy. Primary analyses will be conducted with adjustment for the following prespecified, patient-level covariates that exist before randomization: age, sex, and the five patient-level factors indicated in Table E2 (82, 83). Primary analyses will not adjust for palliative care service-level covariates (e.g., staffing by a physician), as these could conceivably (though improbably) be affected by the intervention (82, 83). Fully unadjusted analyses will also be presented so as to reveal the potential contributions of patient-level covariates. We will use a mixed effects model with random effects for hospitals and fixed effects for time to account for the stepped-wedge cluster randomized design (84). Sensitivity analyses will include all enrolled patients regardless of LOS and hospital fixed effects to account for unobserved characteristics common to all admissions to a particular hospital. We will transform the primary outcome to its log value to account for skewness in LOS. For analyses of secondary outcomes, we will adjust significance levels for multiple comparisons using the Sidak-Holm method (27).

Analyses Accounting for Nonadherence

There are several reasons that palliative care services may not be delivered even when ordered by default: (1) physicians may opt out any given patient from the intervention by cancelling the order, (2) patients and their families may decline consultation, (3) the patient may be discharged shortly after the default order is placed, or (4) constraints among the palliative care teams may preclude completion of the consult. Each of these forms of nonadherence creates differences between the intervention’s effectiveness and its efficacy. As in a recent randomized trial (85), we will evaluate the efficacy of palliative care services in secondary, explanatory analyses that model the randomization arm as an instrumental variable, thus addressing the fact that analyses restricted to those who receive the intervention do not maintain the virtues of randomization and may be influenced by selection effects (86–88).

Analyses of Potential Hospital-Level and Patient-Level Effect Modifiers

We will explore effect modification by conducting analyses stratified by the different characteristics of the patients (e.g., disease type or location in the ICU versus ward) or palliative care services (e.g., proportion of consults staffed by physicians). All of the prespecified potential effect modifiers are listed in Table E2. If differences appear, we will formally evaluate for effect modification by testing the significance of the coefficients for statistical interaction terms between the potential effect modifier and the study period (intervention or control) on the primary outcome. Importantly, we will evaluate palliative care service-level factors (summarizing characteristics of the services provided across patients) rather than patient-level receipt of specific services, because the latter approach would be subject to confounding by indication.

Economic Analyses

We will adopt a cost-minimization approach, comparing total costs with versus without the intervention. In contrast to cost-effectiveness analyses, this approach does not directly incorporate the benefits obtained per dollar spent. Because hospitals, payers, and policy makers seek interventions that improve care without increasing costs, this cost-counting approach is both more consistent with a pragmatic trial paradigm and responsive to the needs of those in positions to act on the data. Because hospital costs tend to be highly skewed, we will use generalized linear models for cost analyses (89, 90). We will use a modified Park test (91) to identify optimal distributions and link functions for the estimation models.

Approach to Missing Data

The validity of outcomes data capture was ascertained before study launch by comparing the data transferred to the DCC with those verifiable by perusal of physicians’ and nurses’ notes and orders in the EHR. Nonetheless, we will explore and potentially adjust for missing values using pattern-mixture methods in secondary analyses (92, 93).

Statistical Power Calculations

We estimate that between 12,000 and 15,000 eligible patients will be admitted to the 11 hospitals during the enrollment period on the basis of extrapolated admissions data from 2 Ascension hospitals during 2013. Even using the conservative assumption of an intracluster (within-hospital) correlation of patient outcomes of up to ρ = 0.20, this design provides 89% power to detect a difference in the primary outcome between intervention and control of 0.5 days at the median, with α = 0.05. These analyses assume that the composite outcome of hospital LOS and mortality follows a log-normal distribution, with a median of 5.5 days and mean of 8.3 days in the control group (corresponding to an SD of 7.9 d). These analyses also assume that the palliative care consultation rate among eligible patients in the control group will be approximately 10% (based on the foregoing preliminary data) and allow for the rate in the intervention to be as low 30%. If adherence to the intervention is greater than 30% in the intervention group, and if palliative care consultation influences outcomes, then the observed effect size may be larger and the observed power would be greater.

These estimates, using the formula of Hussey and Hughes (84) and confirmed with Monte Carlo simulation, also suggest that we will have 80% power to detect small effects for both patient- and cluster-level effect modifiers. Even if only 10,000 patients are enrolled, the design will provide at least 80% power to detect differences in the interaction terms of fewer than 2 days.

Interim Analysis

The trial will be monitored monthly for issues of data quality and study conduct, including participant enrollment and rates of intervention adherence. We will also conduct two interim analyses of the primary outcome after data collection on roughly one-third and roughly two-thirds of patients exposed to the intervention.

A Data and Safety Monitoring Board (a complete list of members may be found before the beginning of the References) has approved our proposals regarding trial-stopping rules. We will not stop the trial early for evidence of effectiveness of the intervention, because doing so would markedly reduce power to detect which types of palliative care consult services are most useful and which types of patients derive the most benefit. However, the Data and Safety Monitoring Board is empowered to stop the trial for (1) early evidence of harm, defined as a lower bound of the confidence interval for the change in the primary outcome exceeding 2 days; (2) increased patient or family complaints during the intervention phase; or (3) futility, which could manifest in the unlikely event that penetration of palliative care consultations among eligible patients did not differ by at least 10% between the intervention and control phases.

Potential Outcomes and Conclusions

As the largest prospective study ever conducted in palliative care, the REDAPS trial will provide high statistical power to detect differences in outcomes that can be extracted from EHRs and are important to patients, families, hospitals, and payers (Figure 3). In addition, the geographic diversity, size, and community nature of Ascension hospitals are representative of the hospitals in which most inpatient palliative care is delivered in the United States. This study will result in (1) the ethically sound generation of experimental evidence regarding the effectiveness and costs of palliative care consultation, (2) the first-ever comparisons of different elements of such services, and (3) identification of patient subgroups that benefit most.

Figure 3.

Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2). This schematic represents the PRECIS-2 criteria developed to measure how the design of a trial will impact the applicability of the results. It depicts a nine-spoked wheel representing domains of trial design that affect where on the continuum a trial is pragmatic or explanatory. It is scored on a 5-point Likert scale for each domain, 1 being very explanatory and 5 being very pragmatic. The values in this figure represent the consensus scores of four raters (the principal investigator, two National Institutes of Health program officials, and a methodologist from Westat Inc.) who were convened at a National Institutes of Health–sponsored review of pragmatic trials in April 2015.

If effective, the simple, scalable intervention of changing the default for palliative care consultation could be applied broadly among patients at high risk for having unmet needs. Thus, this trial could spur widespread and urgently needed improvement in the quality of care provided to patients with COPD and other serious illnesses. Determining that the intervention is not effective—or, perhaps more likely, that the magnitude of the intervention’s effectiveness depends on characteristics of the palliative care teams and/or patients—would have similarly actionable implications. Indeed, given the scarcity of palliative care clinicians, understanding which patients benefit the most from palliative care services, and which of these services can be delivered as well or better by nonphysician team members, will be essential for optimizing the efficiency and sustainability of palliative care delivery in the future.