Failure to Improve the Oxygenation Index is a Useful predictor of Therapy Failure in ARDS Clinical Trials

Abstract

Objective

Acute respiratory distress syndrome (ARDS) trials powered for mortality require significant resources, limiting the number of evaluable therapies. Validation of intermediate endpoints would enhance the feasibility of testing novel ARDS therapies in pilot studies and potentially reduce the frequency of failed large clinical trials. We sought to determine whether a change in the oxygenation index (OI) over the first seven days of ARDS could discriminate between therapies likely or unlikely to show benefit in larger clinical trials.

Design

A derivation cohort from three ARDS studies was used to estimate the seven-day change in OI. Receiver operating characteristic curves were used to calculate optimal thresholds and predictability of the change in OI for 28-day mortality and ventilator-free days (VFD). The thresholds were then validated in two cohorts. Then for each individual ARDS study, the threshold 7-day OI change was tested as an outcome measure and compared to mortality and VFD as reported in the original study.

Setting

Medical Intensive Care Units

Patients

ARDS patients

Interventions

Various

Measurements

Change in OI, 28-day mortality and VFD

Main Results

In the derivation cohort, the mean 7-day OI improved by 4.2 (+/−11.7) in 28-day survivors compared to an increase of 2.4 (+/−11.6) in 28-day non-survivors (p<0.001). The mean 7-day OI decreased by 5.9 (+/−8.4) in patients with >14 VFD, compared to a decrease of 1.9 (+/−12.4) among those with <14 VFD (p=0.001). The optimal 7-day OI threshold for predicting mortality was an increase of 1.71, and for predicting <14 VFD a decrease of 2.34. When used as a surrogate endpoint, the optimal 7-day OI change closely approximated mortality and VFD outcomes in three ARDSNet studies used for the derivation cohort and a distinct study used for validation. The change in OI was a poor predictor of individual patient outcome.

Conclusions

Failure to meet a threshold improvement in the OI over the first 7 days of therapy can be used to identify therapies unlikely to succeed in subsequent trials powered for mortality and VFD. By reducing trial time and costs, use of the 7-day OI change as an intermediate endpoint could increase the number of clinical trials of promising therapies for ARDS and reduce the number of large scale trials of therapies unlikely to be of benefit.

Introduction

The acute respiratory distress syndrome (ARDS) is defined as radiographic evidence of alveolar airspace flooding resulting in hypoxemia refractory to oxygen therapy not attributable to cardiac dysfunction. [1] While reduced compliance is not included in this definition, a recent study suggested that low tidal-volume ventilation strategy reduces ARDS mortality in part by reducing the driving pressure required to ventilate the noncompliant lung.[2, 3] The unacceptably high mortality attributable to ARDS and the lack of specific therapies highlight the need to develop novel therapeutics.[4–11] While several candidate therapies have been suggested, the ability to test them in clinical trials is limited in part by the size and expense of trials powered for mortality and ventilator-free days (VFD). Validated, robust intermediate endpoints would facilitate testing of novel interventions in smaller pilot studies, which could be used to discard therapies unlikely to be beneficial and enrich for promising therapies for validation in larger pivotal trials using mortality and VFD.

The oxygenation index (OI) is calculated from the inspired oxygen concentration, the arterial oxygen partial pressure and the airway pressures, thereby incorporating the severity of the shunt contributing to arterial hypoxemia and the reduced lung compliance seen in patients with alveolar flooding into a single variable. The OI is not useful as a predictor of these outcomes in individual patients, however; because it has been associated with mortality and ventilator free days in populations of patients with ARDS and is calculated using values collected routinely in most ICUs, we reasoned it might be useful as an intermediate outcome to identify therapies unlikely to be promising in larger trials powered to detect differences in VFD or mortality.[12–15] To test this hypothesis, we retrospectively examined data from four large randomized trials of diverse therapies in ARDS patients to determine whether the change in OI from day 1 to day 7 of ARDS correctly predicted the outcome of each trial with respect to 28-day mortality and VFD. We hypothesized that the OI would decrease significantly between day 1 and day 7 only in response to interventions later shown to improve 28-day mortality and VFD. We used data from a randomly selected combined stratified sample of 50% of patients enrolled in the three ARDSNet trials (ARMA, FACTT and ALVEOLI) to determine the optimal threshold change in OI to predict 28-day mortality or VFD. The threshold was then validated in the remaining 50% of the cohorts and in an independent ARDS cohort (ACURASYS).[16]

Materials and Methods

Patient groups

We obtained demographic, physiologic and outcome data from 3 ARDSNet studies, ARMA, FACTT, and ALVEOLI, as well as from the ARDS et Curarisation Systematique (ACURASYS) study. [6, 17–19] (See supplementary materials for additional information).

Derivation Cohort and Validation Cohorts

We combined study subjects from the ARMA, FACTT and ALVEOLI databases and randomly selected 50% of the study-stratified combined sample for the derivation cohort and used the remaining 50% as one validation cohort. In addition we used an independent validation cohort from the ACURASYS study.

Measures and parameters recorded

From each study database we abstracted demographic, physiologic and outcome variables. The oxygenation index (OI) was calculated on each intubated patient for days 1, 3 and 7 using the following formula: mean airway pressure × F_IO₂/P_aO₂. Mean airway pressure was estimated as (peak airway pressure + PEEP)/2. The OI slope was calculated using mixed models for subjects with data for at least two days. (See supplementary materials for additional detail).

Statistical Analysis

Estimated OI slope was compared between 28-day survivors and non-survivors among patients still intubated at 7 days. Using a study stratified split sample, receiver operating characteristic (ROC) curves were used to assess the predictability of OI slopes and optimal thresholds were derived using Youden’s index (jointly maximizing the sensitivity and specificity), as well as an a priori threshold of 0 (no change) over 7 days. These thresholds were assessed in the remaining half of the cohort, as well as in an independent study. For each individual ARDS study, results were examined using 7-day OI change and compared to mortality and VFD as reported in the original study. For this analysis we included patients who were extubated or died before day 7. For the 7-day outcome patients were dichotomized into two groups. Patients who died before day 7 or had an OI slope exceeding Youden’s threshold were assigned a poor outcome and patients who were extubated before day 7 or had an OI slope below Youden’s threshold were assigned a salutary outcome.

Results

In the 3 combined ARDSNet trials, 2430 patients with unique IDs were enrolled (Supplementary figure 1). From this combined cohort, there were data missing for 61 patients precluding the calculation of OI, these patients were excluded from the corresponding analysis. Selected demographics for the derivation and validation cohort are shown in supplementary table 1. In the entire cohort, the combined 28-day mortality was 24.5% (596/2430) and the percentage of patients with fewer than 14 VFD was 63.9% (1549/2424) (data regarding timing of extubation was missing in 6 patients). In the derivation sample (n=1215), the OI slope could be estimated in 1184 patients and in the validation sample the OI slope could be estimated in 1185 patients.

For each patient the OI was determined for days 1, 3 and 7 and the OI change for each patient was calculated. At day 1 the mean OI of all available data in the combined ARDSNet cohort was 12.8 (+/−10.8). During the 1^st week of ARDS, 945 (38.9%) patients were extubated and 248 (10.2%) of patients died. The mean OI for all patients who remained intubated at day 7 was 14.4 (SD=10.9) with a mean OI decrease (i.e. improvement) at day 7 (7-day OI) of 4.2 +/−11.7 in those who survived to day 28 compared to an increase (i.e. worsening) of 2.4 +/−11.6 in those who died by day 28 (p<0.001) (Figure 1A). An ROC analysis showed that the optimal threshold for predicting mortality was an estimated 7-day OI increase of 1.71 (p<0.001, c=0.65) (Supplementary Figure 2A). In patients who did not require prolonged mechanical ventilation (≥14 VFD), the mean 7-day OI decreased by 5.9 +/−8.4 compared to a 7-day OI decrease of only 1.9 +/−12.4 among those who required prolonged mechanical ventilation (<14 VFD) (p=0.001) (Figure 1B); ROC analysis showed a 7-day OI decrease of 2.34 (p=0.013, c=0.60) was the optimal cutoff for the use of the OI to predict the need for prolonged mechanical ventilation (supplementary figure 2B). Using these cutoffs the positive predictive values for mortality and prolonged mechanical ventilation were 42.0% and 87.2%, respectively, and were similar in the two validation cohorts (supplementary table 2). As these thresholds may not be intuitive for use in clinical practice, we also examined the utility of no OI change over 7 days (e.g. 7-day OI=0) in predicting mortality or prolonged ventilation in the entire cohort (supplementary table 2).

Figure 1. Comparing OI change in ARDS patients with 28-day survival/non-survival and more/less than 14 ventilator-free days.

(A) The mean OI change over the first week between ARDS 28-day survivors and non-survivors is shown. (For days 3 and 7, mean change and 95% confidence intervals are shown). (B) The mean OI over the first week between ARDS patients with ≥ or < 14 VFD is shown. (For days 3 and 7, mean change and 95% confidence intervals are shown).

We then assessed the utility of OI change as an ARDS outcome measure by examining the ability of the threshold 7-day OI change to predict the outcome of each of three ARDSNet clinical trials —i.e. predict whether the intervention was beneficial or not. For each trial we estimated the treatment effect of an intervention by generating an odds ratio (OR) using the optimal 7-day OI change (e.g. increase in OI of 1.71) as an outcome measure for each patient enrolled in the trial and compared this OR with that from the measured 28-day mortality and duration of MV from the same data set (see table 1). Patients who were extubated before day 7 were assigned an OI change as below the cutoff and patients who died before day 7 were assigned an OI change above the cutoff. Note these estimates may be different than the published results, as we did not make any adjustments for covariates, nor did we impute any data. In all three studies the OR and confidence intervals for the 7-day OI change for both and mortality and mechanical ventilation approximated very closely the OR for measured 28-day mortality and mechanical ventilation (table 1). We then validated the results in the ACURASYS study, which showed a salutary effect of neuromuscular blockade (NMB) in severe ARDS.[16] In NMB patients, the OR for 7-day OI change 0.75 (95% CI 0.45, 1.23) and 0.62 (95% CI, 0.39, 1.00) for 28-day mortality (see table 1). For prolonged MV the OR for 7-Day OI change was 0.83 (95% CI 0.50, 1.39) and 0.65 (95% CI 0.37, 1.15) for VFD.

Table 1.

Study Odds Ratios by treatment group in entire cohort.

	Mortality
Study	Reported 28-day outcomes	7-day outcomes (Youden’s index)
ARMA (high versus low tidal volume ventilation)	1.65 (1.24–2.21)	1.83 (1.36, 2.47)
ALVEOLI (high PEEP vs. low PEEP)	0.97 (0.64–1.47)	0.84 (0.53, 1.33)
FACTT (liberal vs. conservative fluid strategy)	1.20 (0.89–1.63)	1.55 (1.13, 2.12)
ACURASYS (Cisatracurium v. Placebo)	0.62 (0.39–1.00)	0.75 (0.45, 1.23)
	Prolonged ventilation
ARMA	1.39 (1.07, 1.81)	1.32 (1.01, 1.73)
ALVEOLI	1.05 (0.74, 1.49)	0.98 (0.67, 1.42)
FACTT	1.57 (1.19, 2.08)	1.65 (1.26, 2.16)
ACURASYS	0.65 (0.37, 1.15)	0.83 (0.50, 1.39)

Definitions of abbreviations: ACURASYS= ARDS et Curarisation Systematique, ALVEOLI= Assessment of Low Tidal Volume and elevated End-expiratory volume to Obviate Lung Injury, ARMA= Lower versus Traditional Tidal Volumes for ARDS, FACTT= Fluid and Catheter Treatment Trial, OI= Oxygenation Index

Youden’s Index= The index is defined for all points of an ROC curve (supplementary figure 2), and the maximum value of the index was used as a criterion for selecting the optimum cut-off point.

Data are relative risk ratios and 95% CI

Discussion

In this analysis we demonstrate that 7-day OI change can serve as an intermediate outcome to identify interventions in patients with ARDS that should not be pursued in a subsequent pivotal clinical trial powered for 28-day mortality and VFD. These results support the use of a 7-day change in OI as an intermediate outcome to select therapies from shorter, smaller pilot phase II ARDS studies for evaluation in larger trials. As has been previously suggested, the 7-day OI change has limited utility in predicting mortality in individual ARDS patients.

There are multiple strengths of our analysis. Each included study was a large, randomized multi-center trial with standard entry criteria, explicit treatment protocols and prospectively collected data. Further we assessed our findings in both internal and distinct validation cohorts. In addition the clinical trials in our analysis tested interventions which targeted both local lung and systemic effects.

Several groups have failed to demonstrate that the P_aO₂/F_IO₂ ratio is a useful endpoint in ARDS trials.[8, 20, 21] The OI index is a composite index that reflects both the gas exchange and compliance characteristics of the lung. Our analysis suggests that assessing changes in lung compliance is important in understanding factors that impact ARDS outcome. Further support for the importance lung compliance comes from a recent retrospective analysis which reported that the difference between plateau pressure and PEEP (driving pressure), which is mathematically linked to compliance, was associated with ARDS survival.[3] While those investigators did not report the utility of driving pressure in predicting individual patient outcomes, our analysis suggest that it not likely to be useful. Further, our data should not be extrapolated to target OI change for individual patient decision making, either clinically or in a trial. In order to calculate the OI we used a simplified estimate of mean airway pressure which does not account for the I:E ratio and may slightly overestimate the measured value. While using the actual measured value may impact the derived optimal threshold, it is not likely to impact the predictive value of OI change for mortality and VFD.

In order to expedite clinical translation of basic research findings, an NIH commissioned ARDS workshop endorsed early phase trials using endpoints that suggest efficacy, elucidate mechanistic insights and assess safety. [2] Our analysis suggests that 7-day OI change would serve as a useful efficacy and mechanistic endpoint and is already being used in some trials.[22] We suggest that pairing the change in OI with a more liberal p value for efficacy, which is acceptable in pilot studies, would allow more innovative therapies to be tested by reducing patient enrollment and study duration. This strategy would also reduce the number of interventions unlikely to succeed in large pivotal clinical trials as well as decrease the number patients exposed to ineffective therapies. Furthermore, a 7-day OI change could be incorporated into a Bayesian design of multiple concurrent trials to choose the intervention most promising for further evaluation in a pivotal trial powered for mortality and VFD. As with any intervention it will be important to maintain optimal clinical trial procedures with respect to randomization, blinding and other measures to prevent bias.

Conclusion

In summary we provide an analysis which supports the utility of the 7-day OI change in as an intermediate ARDS outcome measure. Though it is not sufficiently discriminatory to provide useful prognostic information for an individual ARDS patient, it performed very well in discriminating between efficacious and non-efficacious ARDS interventions.