Delivery of Lung-protective Ventilation for Acute Respiratory Distress Syndrome: A Hybrid Implementation–Effectiveness Trial

Abstract

Rationale

Lung-protective ventilation (LPV) improves outcomes for patients with acute respiratory distress syndrome (ARDS), but adherence remains inadequate.

Objectives

To measure the process and clinical impacts of implementation of a science-based intervention to improve LPV adherence for patients with ARDS, in part by increased use of clinical decision support (CDS).

Methods

We conducted a type III hybrid implementation/effectiveness pilot trial enrolling adult patients with ARDS admitted to three hospitals before and after the launch of a multimodal implementation intervention to increase the use of mechanical ventilation CDS and improve LPV adherence. The primary outcome was patients’ percentage of time adherent to low tidal volume (⩽6.5 ml/kg predicted body weight) ventilation (LTVV). Secondary outcomes included adherence to prescribed oxygenation settings, the use of the CDS tool’s independent oxygenation and ventilation components, ventilator-free days, and mortality. Analyses employed multivariable regression to compare adjusted pre- versus postintervention outcomes after the exclusion of a postintervention wash-in period. A sensitivity analysis measured process outcomes’ level and trend change postintervention using segmented regression.

Results

The 446 included patients had a mean age of 60 years, and 43% were female. Demographic and clinical characteristics were similar pre- versus postintervention. The adjusted proportion of adherent time increased postintervention for LTVV (9.2%; 95% confidence interval [CI], 3.8–14.5%) and prescribed oxygenation settings (11.9%; 95% CI, 7.2–16.5%), as did the probability patients spent ⩾90% of ventilated time on LTVV (adjusted odds ratio [aOR] 2.58; 95% CI, 1.64–4.10) and use of ventilation CDS (aOR, 41.3%; 95% CI, 35.9–46.7%) and oxygenation CDS (aOR, 54.3%; 95% CI, 50.9–57.7%). Ventilator-free days (aOR, 1.15; 95% CI, 0.81–1.62) and 28-day mortality (aOR, 0.78; 95% CI, 0.50–1.20) did not change significantly after intervention. Segmented regression analysis supported a causal relationship between the intervention and improved CDS usage but suggested trends before intervention rather than the studied intervention could explain increased LPV adherence after the intervention.

Conclusions

In this pilot trial, a multimodal implementation intervention was associated with increased use of ventilator management CDS for patients with ARDS but was not associated with differences in clinical outcomes and may not have independently caused the observed postintervention improvements in LPV adherence.

Clinical trial registered with www.clinicaltrials.gov (NCT 03984175).

Acute respiratory distress syndrome (ARDS) is a common reason for intensive care unit (ICU) admission and invasive mechanical ventilatory support (1). The syndrome, defined by noncardiogenic, acute-onset hypoxemic respiratory failure associated with bilateral pulmonary infiltrates after a known trigger (2), is associated with substantial morbidity, mortality, and healthcare costs (1, 3).

Lung-protective ventilation (LPV) strategies involving low tidal volume ventilation (LTVV) combined with cotitration of the fraction of inspired oxygen (Fi_O2) and positive end-expiratory pressure (PEEP) improve ARDS outcomes (4, 5). Despite the adoption of LPV as the standard of care for ARDS (6), adherence to LPV for ARDS management remains inadequate (1, 7–9). Barriers to appropriate management of ARDS include underdiagnosis of the syndrome, inexperience, lack of written protocols, lack of clear measures of success, clinician perceptions that LPV is discordant with patient needs, and perceptions that LPV is too time intensive (1, 10–13).

Computerized clinical decision support (CDS) systems have been proposed to facilitate increased adherence to LPV (14–16). We recently developed an open-loop computerized CDS tool embedded within our electronic medical record designed to provide decision support for mechanical ventilator management and help clinicians deliver guideline-adherent LPV to patients with ARDS. However, the mere provision of CDS systems does not guarantee that clinicians will implement their recommendations (17, 18), and we found variable use and effectiveness of this tool after its initial deployment. In the present pragmatic pilot trial, we evaluated the effect of a multimodal implementation program on LPV adherence, use of ventilation management CDS, and clinical outcomes.

Methods

Study Design and Setting

This pragmatic, pilot, type III hybrid implementation/effectiveness trial (19) was conducted at three regional referral hospitals in Utah belonging to the Intermountain Healthcare hospital network. Trials of this design focus on evaluating an intervention’s impact on implementation outcomes while secondarily measuring its effect on clinical outcomes (19). All three hospitals are level II trauma centers and have a mixed medical/surgical ICU (Table E1 in the data supplement). Three study periods were identified at each study hospital: pre-intervention analysis, wash-in, and post-implementation analysis (Figure 1). This study was approved by the Intermountain Healthcare Institutional Review Board with a waiver of informed consent and was registered on ClinicalTrials.gov (NCT03984175) before the launch of the intervention at the first site.

Figure 1.

Study timeline and data analysis periods. LPV = lung-protective ventilation.

All study hospitals had access to a novel open-loop CDS tool for ventilator management integrated into the electronic medical record. After being ordered by a physician, the CDS tool uses electronically- and manually-captured data to generate ventilator management instructions for the bedside respiratory therapist, who then either accepts and follows or rejects each instruction. Four independent open-loop ventilator management subsystems are integrated within the CDS tool:

• 1)Ventilation CDS: targets tidal volume of 6 ml/kg predicted body weight (PBW) and a plateau pressure of 30 cm H₂O or less using volume control or pressure-regulated volume control ventilation mode and titration of set respiratory rate and tidal volume on the basis of arterial partial pressure of carbon dioxide (Pa_CO2) and pH according to the ARDSnet ventilation protocol (4).
• 2)Oxygenation CDS: adjusts PEEP and Fi_O2 on the basis of observed arterial partial pressure of oxygen (Pa_O2) or, if not available, oxygen saturation using either the ARDSnet normal PEEP/Fi_O2 ladder or the Prevention and Early Treatment of Acute Lung Injury (PETAL) Network high PEEP/Fi_O2 ladder (4, 20, 21).
• 3)Weaning assessment CDS: Recommends patients for the transition from assisted control ventilation to a spontaneous breathing mode (e.g., pressure support).
• 4)Spontaneous breathing CDS: Titration of pressure support (PS) and continuous positive airway pressure (CPAP) modes during weaning from mechanical ventilation, including the resumption of full support assist control ventilation when indicated.

Subjects

Adult (age 18 yr or older) patients were eligible for inclusion in the analysis if they were first intubated at a study ICU with ARDS during the 180-day baseline analysis period preceding the launch of the site-level implementation program at each hospital or, after its launch, the 180-day postimplementation analysis period that followed a 70-day wash-in period. The enrollment period at each study hospital depended on the hospital’s implementation launch date (Figure 1) and ran from January or February 2019 to March or April 2020. Patients with ARDS for the primary analysis were identified on the basis of receipt of invasive mechanical ventilation, a Pa_O2 to Fi_O2 ratio of less than 255 (standard criteria adjusted for hospital altitude), bilateral infiltrates on chest radiograph or computed tomography scan, and presence of one or more risk factor for ARDS. Chest imaging and ARDS risk factors were adjudicated by consensus of 2 or more experienced clinician investigators (C.K.G., I.D.P., M.L., and/or L.L.). Patients who were pregnant or prisoners were excluded, as were patients who died or transitioned to comfort-focused care within 24 hours of intubation, had a documented pulmonary capillary wedge pressure of greater than 18 mm Hg, were intubated less than 24 hours, or were intubated for more than 7 days before meeting ARDS criteria.

Intervention

We developed a multimodal implementation strategy to address previously identified system-, site-, and clinician-level adherence barriers to the use of LPV and mechanical ventilation CDS (11, 22). The tested intervention was a package of site-level implementation strategies. A crossfunctional centralized implementation team that included quality improvement specialists, biomedical informaticists, and implementation scientists supported key strategies, including:

• 1.Daily key indicator audit: For daily interdisciplinary rounds, the bedside respiratory therapist audited ventilator settings and CDS use for all mechanically ventilated patients and, as needed, discussed with the intensivist and other team members.
• 2.Local process improvement teams: A team of three to five clinicians, including a frontline respiratory therapist (RT) who served as the project champion and one intensivist, led local implementation efforts on the basis of a rapid-cycle PDSA (Plan-Do-Study-Act) approach (23). Each process improvement team met weekly or biweekly to identify LPV adherence barriers and select, design, and assess experiments targeted to achieve a specific improvement in behavior. Performance on a weekly or biweekly improvement goal was tracked visually in an area visible to all unit RTs and was discussed in leader rounds.
• 3.Leader rounding campaign: RT leaders conducted fast, open-ended rounds focused on the importance and expectation of LPV adherence with individual frontline RTs while providing the opportunity for education, idea sharing, and escalation of concerns. Leaders created a rounding schedule for themselves and a team of trained delegates (trusted peers and members of the process improvement team) that reinforced specific expectations over the course of each week.
• 4.Weekly summative reporting and case review: Pilot site leaders received detailed performance data weekly via email, including a summative report on site-level LPV adherence performance with comparison versus peers and detailed data on all compliant and noncompliant activities that included the specific deviation from best practice.

Before site-level intervention deployment, LPV adherence was promulgated in early 2019 by health system clinical leadership as a system-level goal. In addition, the central implementation team conducted outreach activities to facilitate site engagement, interviewed stakeholders to understand barriers to LPV adherence, and solicited input on LPV performance metrics and reporting strategies during the first quarter of 2019. Finally, the creation and training of local implementation teams on rapid-cycle PDSA methods and training of RT leaders and delegates in rounding strategies were conducted by investigators experienced in quality improvement strategies (L.A. and D.W.) from May to June 2019. The site-level implementation intervention was then deployed in a staggered fashion beginning in July 2019, with a 1-month optimization period at the first study hospital, followed by sequential program launches 1 week apart at the other two hospitals (Figure 1). Each hospital’s 180-day postintervention assessment period began 10 weeks after the site launch to allow for implementation wash-in.

Data Sources

Demographic and clinical data were abstracted from the electronic data warehouse maintained by Intermountain Healthcare (24). A preexisting linkage between this data warehouse and Utah State death records provided information on 28-day mortality. ARDS diagnoses by the bedside team were identified during the daily interdisciplinary rounds sites conducted after intervention. Trained research coordinators verified outlying data and obtained missing data on the basis of a manual review of the medical record.

Exposures and Outcomes

The primary exposure was the implementation phase during which the patient was first intubated at the study hospital. The Charlson Comorbidity Index, acute physicology score, and SOFA (Sequential Organ Failure Assessment) score were calculated as previously described (25–28).

The primary outcome was LTVV compliance, defined as the percentage of total time on invasive mechanical ventilation that a patient had a set tidal volume of 6.5 ml/kg PBW or less via volume control or pressure-regulated volume control ventilator mode. Time spent on appropriate spontaneous ventilation (CPAP or PS mode ventilation with Fi_O2 of 0.5 or less, PEEP of 10 cm H₂O or less, and PS 15 cm H₂O or less) was excluded from the denominator when calculating LTVV compliance. Implementation-focused secondary outcomes, which also excluded time on or events during appropriate spontaneous ventilation from the denominator, were:

• 1.PEEP/Fi_O2 compliance: percentage of ventilated time during which a patient was on an appropriate combination of PEEP and Fi_O2;
• 2.Ventilation CDS use: percentage of opportunities (eligible arterial blood gas measurements) for which the ventilation CDS was used within 60 minutes after the arterial blood gas was obtained;
• 3.Oxygenation CDS use: percentage of opportunities (eligible ventilator checks) for which the oxygenation CDS was used within 60 minutes before or after the ventilator check was performed;
• 4.LTVV compliance 90% or more of eligible time;
• 5.PEEP/Fi_O2 compliance 70% or more of eligible time.

Clinical secondary outcomes were 28-day mortality and an adaptation of ventilator-free days (VFDs) through Day 28, with death on or before Day 28 assigned a score of −1 (29, 30). Hospital-free days (HFDs) were included as an exploratory outcome. Post hoc analyses were performed for additional implementation outcomes evaluating 1) the proportion of eligible time on potentially harmful tidal volumes (greater than 8 ml/kg PBW); and 2) receipt of harmful tidal volume greater than 10% of the eligible time. Additional information on outcome definitions, including event and time eligibility definitions and exceptions to the general rules above, is included in the data supplement (eMethods).

Statistical Analysis

Bivariable comparisons employed unpooled t tests, Fisher’s exact tests, or Wilcoxon-Mann-Whitney tests as appropriate. The primary, secondary, and post hoc analyses employed multivariable generalized linear models with a cumulative logit link for VFDs and the exploratory outcome of HFDs, a logit link for binary outcomes, and an identity link for continuous outcomes. A directed acyclic graph was generated on the basis of a literature review and expert opinion by consensus of investigators with expertise in ARDS, critical care, and epidemiologic causal inference (31–33) and used to select a minimum sufficient set of adjustment variables (Figure E1): acute physiology score, age, initial Pa_O2/Fi_O2 ratio, Charlson Comorbidity Index, primary ARDS etiology, sex, and body mass index (categorized as less than 20, 20–24.9, 25–29.9, 30–30.9, or ⩾40 kg/m²). We accounted for clustering by study site by including a fixed effect for the study hospital in the multivariable model.

We performed sensitivity analyses for the primary outcome and continuous secondary implementation outcomes using interrupted time series analysis (34, 35). Each hospital’s observation period was divided into 2-week periods relative to the beginning of the wash-in phase, such that the first 2 weeks of the wash-in phase were designated period 0. For each outcome, the average response in each period across hospitals was indirectly standardized on the basis of a linear model of the adjustment set predictors, and we then fitted a simple segmented regression to the standardized estimates, weighting by the number of observations in the period (35, 36). If a joint test for discontinuities in outcome level (at the first week of the postintervention analysis period) or pre/postintervention trend for the segmented regression was significant, each discontinuity was individually tested. We also performed an exploratory mediation analysis to evaluate the proportion of the intervention’s association with LTVV adherence explained by changes in CDS use (37, 38) and evaluated the unadjusted and adjusted association of ARDS recognition with LTVV compliance during the postintervention period (ARDS recognition data was only collected after intervention). See the data supplement (eMethods) for additional details on the sensitivity and mediation analyses.

All analyses excluded observations from the wash-in phase, though the wash-in phase was summarized for interrupted time series figures. We excluded observations from the analysis if the outcome or exposure was missing. There was no missingness for covariates used in multivariable models. We performed a sensitivity analysis repeating the exploratory analysis of ARDS recognition’s impact on LTVV adherence after multiple imputation of missing ARDS recognition data.

Statistical analyses were performed in R version 4.1.1 (R Foundation). We considered a two-sided P ⩽ 0.05 statistically significant.

Results

A total of 446 patients with ARDS were included in analyses, including 197 patients first intubated during the preintervention analysis period and 249 first intubated during the postintervention analysis period (Figure 2). Seventy patients with ARDS first intubated during the wash-in period were excluded from analyses. Analyzed patients had a mean age of 60 (standard deviation, 17) years, were 43% (n = 192) female, and had 37% (n = 163) 28-day mortality. Baseline demographic and clinical characteristics were similar between the pre- and postintervention periods (Table 1). Bedside clinical teams diagnosed ARDS for 91 of 224 postintervention patients (40.6%) for whom information was available. Patients in whom ARDS was recognized by the clinical team were younger, exhibited less nonpulmonary organ failure and lower illness severity, and had more severe ARDS (Table E2).

Figure 2.

CONSORT-style patient inclusion/exclusion diagram. *Some patients met >1 exclusion criterion. ARDS = acute respiratory distress syndrome; CONSORT = consolidated standards of reporting trials; ICU = intensive care unit; P/F = PaO₂/FiO₂ ratio; PCWP = pulmonary capillary wedge pressure.

Table 1.

Primary analysis cohort demographic and clinical characteristics

Parameter	Preintervention (n = 197)	Postintervention (n = 249)	P Value
Age (yr), mean (SD)	58.7 (15.9)	60.3 (17.2)	0.31
Female sex, n (%)	94 (47.7)	98 (39.4)	0.08
Hispanic/Latino ethnicity or race other than White, n (%)	20 (10.2)	32 (12.9)	0.46
Charlson Comorbidity Index, mean (SD)	4.3 (3.7)	4.4 (3.3)	0.80
Body mass index (kg/m2), mean (SD)	33.2 (11.4)	32.0 (9.7)	0.22
SOFA score (without pulmonary subscore), mean (SD)	5.9 (2.4)	5.7 (2.3)	0.44
Acute physiology score at ARDS diagnosis, mean (SD)	22.6 (6.8)	22.6 (6.5)	0.99
Primary ARDS risk factor, n (%)			0.95
Pneumonia	100 (50.8)	133 (53.4)
Aspiration	37 (18.8)	44 (17.7)
Sepsis	21 (10.7)	22 (8.8)
Trauma	19 (9.6)	23 (9.2)
Other	20 (10.2)	27 (10.8)
Initial PaO2/FiO2 ratio, mean (SD)	135 (55)	133 (58)	0.82
ARDS severity,* n (%)			0.16
Mild (PaO2/FiO2, 171–255 mm Hg)	52 (26.4)	72 (28.9)
Moderate (PaO2/FiO2, 86–170 mm Hg)	101 (51.3)	106 (42.6)
Severe (PaO2/FiO2, ⩽85 mm Hg)	44 (22.3)	71 (28.5)
Ventilator mode at ARDS diagnosis, n (%)			0.004
Volume control or PRVC	167 (84.8)	234 (94.0)
Other control mode	7 (3.6)	2 (0.8)
Spontaneous mode	23 (11.7)	13 (5.2)
Ventilator parameters at the time patient met ARDS criteria†
Set tidal volume (ml/kg PBW), mean (SD)	6.07 (0.59)	5.97 (0.57)	0.12
Actual tidal volume (ml/kg PBW), mean (SD)	6.31 (1.62)	6.29 (1.39)	0.92
FiO2 (%), mean (SD)	65.6 (23.2)	63.7 (23.2)	0.41
Positive end-expiratory pressure (cm H2O), mean (SD)	10.0 (3.8)	9.8 (4.0)	0.67
Plateau pressure (cm H2O), mean (SD)	22.7 (5.6)	21.9 (5.4)	0.16
Set respiratory rate (breaths/min), mean (SD)	23.6 (5.6)	22.2 (6.0)	0.02
Actual respiratory rate (breaths/min), mean (SD)	26.1 (6.7)	25.0 (6.8)	0.10
Hospital mortality, n (%)	71 (36.0)	85 (34.1)	0.69
ICU length of stay among survivors (d), mean (SD)	9.7 (7.6)	10.7 (10.1)	0.36
Hospital length of stay among survivors (d), mean (SD)	13.1 (8.6)	16.0 (13.3)	0.03

Definition of abbreviations: ARDS = acute respiratory distress syndrome; ICU = intensive care unit; Fi_O2 = fraction of inspired oxygen; Pa_O2 = arterial partial pressure of oxygen; PBW = predicted body weight; PRVC = pressure-regulated volume control; SD = standard deviation; SOFA score = Sequential (Sepsis-associated) Organ Failure Assessment score.

^*Pa_O2/Fi_O2 criteria for ARDS severity adjusted for altitude at study hospitals.

^† Missingness for ventilator parameters: set tidal volume, n = 45; actual tidal volume, n = 3; positive end-expiratory pressure, n = 1; plateau pressure, n = 49; set respiratory rate, n = 19.

In unadjusted analyses, mean patient-level adherence for LTVV increased postintervention (80 ± 31% to 90 ± 24%; P < 0.001), as did adherence to prescribed PEEP/Fi_O2 (65 ± 29% to 78 ± 25%; P < 0.001). The use of ventilation and oxygenation CDS also increased postintervention, but there was no difference in clinical outcomes (Table 2). After adjustment, patients’ average percentage of time adherent to LTVV increased by 9.2% (95% confidence interval [CI], 3.8–14.5%; P < 0.001). Adjusted percentage of time adherent to prescribed PEEP/Fi_O2 also increased (11.9%; 95% CI, 7.2–16.5%; P < 0.001), as did the likelihood that patients spent 90% or more of eligibile time adherent for LTVV (adjusted odds ratio [aOR], 2.58; 95% CI, 1.64–4.10) and 70% or more of eligibile time adherent for PEEP/Fi_O2 (aOR, 3.60; 95% CI, 2.27–5.28) (Table 2). Adjusted usage rates increased for ventilation CDS (+41.3%; 95% CI, 35.9–46.7%) and oxygenation CDS (+54.3%; 95% CI, 50.9–57.7%). In a post hoc analysis, the adjusted mean percentage of LTVV-eligible time that each patient spent on tidal volumes greater than 8 ml/kg decreased by 8.5% (95% CI, −12.7% to −4.3%; P < 0.001), whereas patients’ odds of spending greater than 10% of LTVV-eligible time on the ventilator receiving potentially harmful tidal volumes decreased 65% (95% CI, 42–79%; P < 0.001). On the basis of mediation analysis, increased ventilation CDS usage explained 50% (95% CI, 10–132%) of the postintervention increase in LTVV compliance.

Table 2.

Association of outcomes with implementation intervention

Parameter	Preintervention (n = 197)	Postintervention (n = 249)	P Value	Adjusted Change or aOR for Outcome* (95% CI)	P Value
Adherence to LTVV,† mean (SD)	80.3% (30.8%)	89.8% (24.4%)	<0.001	+9.2% (+3.8% to +14.5%)	<0.001
Adherence to prescribed PEEP/FiO2,† mean (SD)	65.3% (28.9%)	78.2% (24.6%)	<0.001	+11.9% (+7.2% to +16.5%)	<0.001
Time receiving harmful tidal volumes,† mean (SD)	14.4% (26.0%)	6.1% (17.4%)	<0.001	−8.5% (−12.7% to −4.3%)	<0.001
Use of ventilation CDS tool, mean (SD)	21.1% (27.7%)	60.6% (30.3%)	<0.001	+41.3% (+35.9% to +46.7%)	<0.001
Use of oxygenation CDS tool,‡ mean (SD)	23.9% (22.2%)	77.3% (19.2%)	<0.001	+54.3% (+50.9% to +57.7%)	<0.001
LTVV adherence ⩾90%,† n (%)	121 (63.0)	201 (81.7)	<0.001	aOR, 2.58 (1.64 to 4.10)	<0.001
PEEP/FiO2 adherence ⩾70%,† n (%)	99 (51.6)	187 (76.0)	<0.001	aOR, 3.60 (2.27 to 5.78)	<0.001
Time receiving harmful tidal volumes ⩾10%,† n (%)	57 (29.7)	32 (13.0)	<0.001	aOR, 0.35 (0.21 to 0.58)	<0.001
Ventilator-free days through day 28, median (IQR)	17 (−1 to 24)	18 (−1 to 24)	0.84	aOR, 1.15 (0.81 to 1.64)	0.42
Hospital-free days through day 28, median (IQR)	5 (−1 to 18)	6 (−1 to 17)	0.76	aOR, 1.06 (0.75 to 1.50)	0.76
28-d mortality, n (%)	75 (38.1)	88 (35.3)	0.55	aOR, 0.78 (0.50 to 1.20)	0.25

Definition of abbreviations: aOR = adjusted odds ratio; CDS = clinical decision support; CI = confidence interval; Fi_O2 = fraction of inspired oxygen; IQR = interquartile range; LTVV = low tidal volume ventilation; Pa_O2 = arterial partial pressure of oxygen; PEEP = positive end-expiratory pressure; SD = standard deviation.

^*Adjusted for patient age, acute physiology score, Pa_O2/Fi_O2 ratio, primary acute respiratory distress syndrome risk factor, sex, hospital, body mass index, and Charlson Comorbidity Index.

^† Eight subjects on appropriate continuous positive airway pressure or pressure support for all ventilator checks were excluded from the analysis.

^‡ One subject with a missing outcome variable was excluded from the analysis.

In sensitivity analyses employing segmented regression (Table E3), LTVV adherence suggested a trend toward improvement before intervention (Figure 3A) with no significant level or trend change after intervention (P = 0.54). Findings were similar for prescribed PEEP/Fi_O2 adherence (Figure 3B), which also had a possible trend toward improvement before intervention and no significant change after intervention (P = 0.07). The intervention was associated with a level change in usage of the ventilation (+21.0%; 95% CI, 6.7–35.2%; P = 0.006) and oxygenation (+29.7%; 95% CI, 17.8–41.6%; P < 0.001) CDS tools (Figures 3C and 3D).

Figure 3.

Results of adjusted interrupted times series analysis for adherence to (A) low tidal volume ventilation and (B) positive end-expiratory pressure/Fi_O2 and use of (C) ventilation and (D) oxygenation clinical decision support tools. Each measurement window represents 2 weeks; window 0 is the 2-week period beginning with the intervention launch at each study site. Observed data is indicated in red, and standardized data is in blue. Gray shading indicates the postintervention wash-in period. Fi_O2 = fraction of inspired oxygen; PEEP = positive end-expiratory pressure.

Mortality at 28 days did not differ between preintervention (n = 75 [38.1%]) and postintervention patients (n = 88 [35.3%]) in unadjusted analyses (P = 0.55) or after covariate adjustment (aOR, 0.78; 95% CI, 0.50–1.20; P = 0.25). There was also no difference in VFDs (aOR, 1.15; 95% CI, 0.81–1.64) or HFDs (aOR, 1.06; 95% CI, 0.75–1.50) between the pre- and postintervention periods.

During the postintervention period, there was no significant difference in mean LTVV adherence when comparing patients for whom ARDS was versus was not recognized by the bedside clinical team in either unadjusted (88 ± 23% vs. 92 ± 23% [P = 0.18], respectively) or adjusted analyses (−2.9%; 95% CI, −10.0% to 4.3%; P = 0.43). Repeating this analysis after imputing ARDS recognition status for 25 patients yielded similar results (data not shown).

Discussion

In this pilot trial, we found that an intervention bundling multimodal implementation strategies designed to facilitate evidence-based ARDS care delivery was associated with increased compliance with LPV strategies and the use of CDS for ventilator management in patients with ARDS. However, although supporting a role for the intervention in improving CDS use, quasi-experimental sensitivity analyses suggested that preintervention trends rather than the studied intervention may have explained postintervention improvements in LPV adherence outcomes. Clinical outcomes, including mortality and VFDs, did not change significantly in the postintervention period, although the trial lacked the power to assess differences in clinical endpoints.

Clinical medicine’s adoption of proven therapies and de-adoption of disproven therapies is often slow and incomplete (1, 39–41). In ARDS specifically, a large, international study conducted 14 years after the publication of the seminal study on the benefits of LPV (4) demonstrated that over 35% of patients with ARDS were receiving tidal volumes larger than 8 ml/kg PBW (1). We and others previously identified barriers to LPV adherence in contemporary practice that included inadequate organizational support for and emphasis on LPV adherence, lack of performance information, physician and RT knowledge and their patterns of responsibilities and interactions, and clinician concerns around autonomy (11–13). The present trial applied implementation science (42) and causal inference methods within the framework of “T3” research to provide evidence to aid adherence to an evidence-based therapy (43).

Our intervention targeted LPV adherence directly but was also designed to work by increasing the use of an available CDS tool for ventilator management. Conceptually, therefore, this study’s LPV adherence outcomes — including the primary outcome of LTVV adherence — were second-order implementation outcomes assumed to occur downstream of the first-order implementation outcome of CDS tool use. On the basis of both the pre/postprimary analysis and segmented regression sensitivity analysis, our intervention was successful for the first-order usage outcomes.

Our results regarding the intervention’s impact on LPV adherence, however, require a more nuanced interpretation. Although the primary pre/postanalysis demonstrated improved LPV adherence after intervention, such analyses may be subject to trends in outcomes unrelated to the studied intervention (34). Quasi-experimental methods, including the segmented regression employed here (44), can support more robust causal inference. Conversely, such methods can protect against potentially mistaken conclusions when pre/postanalyses are influenced by such secular trends. At first glance, this appears to be the case in the present study, in which segmented regression analysis suggested that the pre/postanalysis was potentially confounded by preintervention trends toward improved LPV adherence. However, even these results require careful consideration to avoid misinterpretation in the context of a complex implementation study like ours. We suspect that it is possible and perhaps even likely that the work preparatory to our intervention’s formal launch — including prioritization of LPV adherence as a system performance metric and engagement with and training of local implementation teams — “spilled over” and influenced LPV adherence during the preintervention period. In other words, quasi-experimental analysis in the present case may misrepresent as a secular trend what was, in truth, a beneficial effect when such preparatory tasks are considered part of our implementation intervention. These intricacies highlight the challenges when measuring the effects of complex interventions in a complex environment such as health care (45). Further lessons from this study for the conduct and analysis of hybrid implementation/effectiveness trials include the importance of accounting for potential spillover via thoughtful intervention implementation and analysis design, careful selection of wash-in periods, adequate pre- and postintervention measurement periods, and especially, the inclusion of contemporaneous control data. Such strategies can bolster causal inference and reduce the risk of both false-positive and false-negative conclusions.

Strengths and Limitations

Our study’s strengths include its multihospital design, adjustment for a set of confounders selected a priori with the aid of a directed acyclic graph in accordance with current guidelines (32), and application of sensitivity analyses to inform causal conclusions. However, there are several limitations. Generalizability to other health systems and patient populations, including hospitals without CDS for ventilator management, is uncertain. We studied proportional adherence to best practices consistent with our focus on implementation, but absolute rather than the proportional duration that patients are on suboptimal ventilator setting may be more important for clinical outcomes. Though applying standard implementation principles and strategies, the implementation intervention we tested was complex and may not be replicable in settings without experienced quality improvement personnel. We are also unable to define the relative contribution of our multimodal intervention’s component elements. The first surge of the coronavirus disease (COVID-19) pandemic overlapped the final weeks of our trial in March–April 2020, and although the number of ICU admissions relative to capacity was moderate in our hospitals, it is possible that pandemic-related resource strains or care process modifications influenced our findings. As noted above, baseline LPV adherence was higher than expected, and the application of our implementation intervention may show larger effects in settings where baseline LPV adherence is poorer. This pilot trial enrolled patients at three hospitals within a type III hybrid implementation/effectiveness framework — a study design focused on implementation (or process) outcomes (19) — and was not powered for clinical outcomes. However, although not significantly different between the pre- and postintervention period, point estimates did favor improvement in clinical outcomes. A larger trial employing a cluster-randomized stepped wedge design would have better power for the evaluation of clinical outcomes and would also aid more robust causal inference about the intervention’s effect on implementation outcomes.

Conclusions

We observed improvements in adherence to implementation outcomes but no significant change in clinical outcomes in this pilot type III hybrid implementation/effectiveness trial after a multimodal implementation intervention targeting LPV adherence. However, the intervention’s association with LPV adherence was not confirmed in analyses accounting for temporal trends in outcomes unrelated to the intervention, limiting causal inference about the intervention’s impact.