Impact of an ROX index-guided intubation strategy on mortality in patients receiving high-flow nasal cannula: a target trial emulation

Abstract

Background

The ROX index (the ratio of peripheral oxygen saturation to the fraction of inspired oxygen, divided by respiratory rate) has been validated to predict high-flow nasal cannula (HFNC) failure, but whether acting on ROX to guide the timing of intubation improves patient outcomes is unknown. We estimated the per-protocol effect of ROX-guided strategies versus usual care on mortality after HFNC initiation.

Methods

We emulated target trials using Medical Information Mart for Intensive Care IV (MIMIC-IV) electronic health record data. Adults initiated on HFNC within 7 days of intensive care unit (ICU) admission were eligible. Strategies were usual care or intubation within 2 h after ROX first fell below 3.85, 4.88, or time-varying thresholds (2.85 for hours 1–5, 3.47 for hours 6–11, and 3.85 from hour 12). We used clone-censor-weighting with pooled logistic regression, adjusted for baseline and time-varying covariates, to estimate 30-day mortality.

Results

We included 1,651 adults (median age, 66 years; women, 41%). Estimated 30-day mortality under usual care was 26.9% (95% CI 24.7–29.2). ROX < 3.85: 20.5% (95% CI 17.1–24.1; risk difference [RD], −6.5 percentage points [95% CI −9.1 to −3.8]; risk ratio [RR], 0.76 [95% CI 0.66–0.86]). ROX < 4.88: 19.8% (95% CI 16.3–24.0; RD, −7.1 percentage points [95% CI −10.3 to −3.5]; RR, 0.74 [95% CI 0.62–0.87]). Time-varying ROX: 21.3% (95% CI 18.5–24.3; RD, −5.6 percentage points [95% CI −7.9 to −3.4]; RR, 0.79 [95% CI 0.70–0.87]).

Conclusions

ROX-guided intubation strategies were associated with lower 30-day mortality than usual care. These findings are hypothesis-generating and support prospective evaluation of the ROX index applied as a decision policy rather than as a prediction score alone; they should not be used to guide intubation decisions at present.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s40560-026-00927-x.

Background

High-flow nasal cannula (HFNC) is widely used for acute respiratory failure in the intensive care unit (ICU), yet many patients require intubation [1, 2]. Delayed intubation after HFNC failure has been linked to increased mortality, emphasizing the clinical importance of early recognition and timely escalation of respiratory support [3, 4].

The ROX index, defined as the ratio of peripheral oxygen saturation to the fraction of inspired oxygen (SpO₂/FiO₂) divided by respiratory rate, has been validated as a predictor of HFNC failure, and thresholds such as 3.85 and 4.88 have been proposed for risk stratification [1, 5, 6]. Although ROX is used in bedside decision-making, evidence that ROX-guided intubation improves patient outcomes remains limited [1, 6–8].

Prediction performance does not establish clinical utility [9–12]; demonstrating utility requires an impact analysis that compares outcomes when a prediction instrument is used to guide action versus the natural course of care. Prior target trial emulations in this database have examined the timing of intubation in broad terms, comparing early with delayed intubation or comparing oxygenation-ratio thresholds for initiating invasive ventilation [13–15]. None has evaluated a specific, widely used bedside prediction score as a decision policy. The ROX index is distinct in combining oxygenation and respiratory rate in a single value that clinicians already compute at the bedside, which makes it a natural candidate for such an impact analysis. A randomized trial of ROX-guided intubation would be resource intensive, whereas target trial emulation can evaluate prespecified decision strategies using routinely collected ICU data while aligning time zero and reducing design-related biases [16, 17]. In the absence of randomized evidence, this approach can provide interim estimates to inform prospective evaluation.

In this study, we conducted an impact analysis of the ROX index reframed as a treatment policy, using a target trial emulation framework to estimate the per-protocol effect of adherence to ROX-guided intubation strategies, compared with usual care, on 30-day all-cause mortality among adult ICU patients initiated on HFNC.

Methods

This study was designed and reported in accordance with the Transparent Reporting of Observational Studies Emulating a Target Trial (TARGET) guideline [17]. Additional methodological detail is provided in Additional file 1 (Supplementary Methods), and the completed TARGET checklist is provided as Additional file 2.

Data source

We emulated the target trial using routinely collected electronic health record (EHR) data from MIMIC-IV version 3.1, a publicly available critical care database hosted on PhysioNet [18]. MIMIC-IV includes approximately 95,000 ICU admissions at Beth Israel Deaconess Medical Center (Boston, MA, USA) from 2008 to 2022, with demographics, vital signs, laboratory data, medications, and clinical documentation. Timestamped events can be aligned to an hourly scale, enabling 1-h resolution analyses in the ICU [19, 20].

All records in MIMIC-IV are de-identified in accordance with the Health Insurance Portability and Accountability Act (HIPAA) Safe Harbor provisions. This study used only publicly available de-identified data, so additional institutional review board approval was not required. Patients and the public were not involved in the study design or conduct, and the study was not prospectively registered.

Target trial and emulation

We emulated three target trials, each comparing one ROX index-guided intubation strategy (ROX < 3.85, ROX < 4.88, or a time-varying threshold; defined in Treatment strategies) with usual care among ICU patients receiving HFNC. In each target trial, patients would be assigned at HFNC initiation to usual care or to one ROX-guided strategy. Target trial protocols and emulation details are summarized in Table 1 and the Supplementary Methods (Additional file 1, Table S1).

Table 1.

Summary of target trial protocol and observational emulation

Protocol component	Target trial (ideal RCT)	Emulation using observational data
Eligibility criteria	Adults aged 18–85 years; first ICU admission; HFNC initiated within 1 h before or within 7 days after ICU admission; not admitted to a neurological ICU; no prior invasive mechanical ventilation; no treatment limitations (DNR, DNI, or CMO) before HFNC initiation	Same eligibility criteria, operationalized using MIMIC-IV electronic health record data
Treatment strategy	Usual care versus ROX-guided intubation strategy during HFNC treatment*; no transition to noninvasive ventilation under any strategy	Same as the target trial; use of noninvasive ventilation is treated as a deviation from all strategies and leads to censoring
Assignment procedure	Randomization to usual care or an ROX-guided strategy at HFNC initiation	Randomization emulated by cloning each patient and assigning clones to usual care or an ROX-guided strategy
Follow-up period	From HFNC initiation up to 30 days, death, or administrative end of follow-up, whichever occurs first	Same as the target trial. Follow-up is constructed from hourly data; death is ascertained from hospital records and linked death registries, and 30-day mortality is available for all patients
Outcome	Primary: 30-day all-cause mortality; secondary: 7-day and 14-day all-cause mortality	Same as the target trial
Causal contrast of interest	Per-protocol effect of each ROX-guided strategy compared with usual care. Intention-to-treat effect	Per-protocol effects. An intention-to-treat effect is not targeted, because it would be diluted by deviations from the strategies in this observational setting
Identifying assumptions	Exchangeability ensured by randomization; positivity holds as long as each strategy is feasible; consistency holds if strategies are well-defined and fully adhered to	Conditional exchangeability assumed after adjustment for baseline and time-varying covariates in the weight models; positivity assumed for all observed covariate patterns; consistency assumed if the emulated strategies correspond to the defined target trial strategies
Data analysis	Per-protocol analysis using inverse probability weighting to adjust for time-varying confounding and selection bias due to nonadherence	Same per-protocol estimand, implemented using the clone-censor-weight approach with time-varying treatment and censoring weights, truncation of extreme weights, and bootstrap confidence intervals

HFNC high-flow nasal cannula, ICU intensive care unit, DNR do-not-resuscitate, DNI do-not-intubate, CMO comfort-measures-only, MIMIC-IV Medical Information Mart for Intensive Care IV, RCT randomized controlled trial

^*Three separate two-arm comparisons were emulated: usual care versus ROX < 3.85 (emulated target trial 1), usual care versus ROX < 4.88 (emulated target trial 2), and usual care versus a time-varying ROX threshold strategy (emulated target trial 3)

Eligibility criteria

Eligible individuals were adults aged 18–85 years who were admitted to the ICU and initiated HFNC therapy either within 1 h before or within 7 days after ICU admission. Only the first ICU admission for each patient was included. Patients were excluded if they were admitted to a neurological ICU, had received invasive mechanical ventilation before HFNC initiation, or had documented treatment limitations, such as do-not-resuscitate (DNR), do-not-intubate (DNI), or comfort-measures-only (CMO) orders, prior to HFNC initiation. Patients admitted to neurological ICUs were excluded, because their respiratory rate and intubation decisions often reflect impaired consciousness or airway protection rather than hypoxemic respiratory failure. The 7-day window was a pragmatic criterion to capture HFNC started for acute hypoxemic respiratory failure early in the ICU course and to exclude heterogeneous late-course HFNC use; it was not derived from a specific prior study.

Treatment strategies

We emulated three separate two-arm trials at HFNC initiation, each comparing usual care with an ROX-guided intubation strategy. The ROX-guided strategies were a fixed threshold of 3.85, a fixed threshold of 4.88, and a time-varying threshold strategy. Usual care represented the natural course of the observed cohort, that is, clinician-directed practice in which no protocol was imposed and intubation occurred at the clinician’s discretion (see Supplementary Methods). The contrast of interest was, therefore, a prespecified ROX rule versus heterogeneous clinician judgment. In usual care, ROX may have been one component of clinical decision-making.

In ROX-guided strategies, ROX was evaluated hourly and defined as (SpO₂/FiO₂) divided by respiratory rate. Intubation was required within 2 h after the first hour in which ROX fell below the assigned threshold, and intubation was not allowed, while ROX remained at or above the threshold. In emulated target trial 1 (primary comparison), the threshold was ROX < 3.85 [1, 6]. Emulated target trial 2 used ROX < 4.88 [1, 6]. Emulated target trial 3 used time-varying thresholds of ROX < 2.85 during hours 1–5, ROX < 3.47 during hours 6–11, and ROX < 3.85 from hour 12 onward [1].

ROX-guided strategies applied from HFNC initiation until intubation, HFNC discontinuation, or death, whichever occurred first. After HFNC discontinuation, subsequent respiratory management followed usual care. To emulate trials in which escalation from HFNC to noninvasive ventilation (NIV) was not permitted, NIV initiation was treated as protocol deviation and led to censoring.

Follow-up and study outcomes

Follow-up began at HFNC initiation (time zero), when all eligibility criteria were met, and continued until death, initiation of NIV (which was treated as censoring), or 30 days after time zero, whichever occurred first. The primary outcome was all-cause mortality within 30 days of HFNC initiation, and secondary outcomes were all-cause mortality at 7 and 14 days. Mortality status was available for patients through linkage to external death registries [18].

Statistical analysis

We targeted the per-protocol effect of each ROX-guided strategy versus usual care on 30-day mortality and estimated risk differences (RDs) and risk ratios (RRs) at 7, 14, and 30 days. Identification relied on conditional exchangeability, positivity, and consistency as described in the Supplementary Methods. We did not target an intention-to-treat effect, because frequent deviations in routine care would be expected to attenuate differences [21].

We used clone-censor-weighting to emulate each target trial [13, 16, 22, 23]. Eligible patients were cloned into two copies assigned to usual care or the ROX-guided strategy. At hourly intervals, copies were artificially censored when observed treatment no longer matched the assigned strategy. Separately, NIV initiation was defined as a protocol deviation and copies were censored at NIV start in all strategies; worked examples of these censoring rules are given in Additional file 1, Table S2. To address selection induced by censoring and time-varying confounding, we estimated time-varying inverse probability weights using pooled logistic regression models for the hourly probability of intubation and for remaining uncensored. Models included time, since HFNC initiation with linear and quadratic terms, baseline covariates measured before HFNC initiation (time zero), and time-varying covariates updated hourly. Baseline covariates were age, sex, insurance status, primary language, marital status, race or ethnicity, Elixhauser comorbidity index [24], and hours from ICU admission to HFNC initiation. Time-varying covariates were heart rate, mean arterial pressure, temperature, respiratory rate, SpO₂, FiO₂, Glasgow Coma Scale category, arterial pH, partial pressure of arterial oxygen (PaO₂), partial pressure of arterial carbon dioxide (PaCO₂), Sequential Organ Failure Assessment (SOFA) score within the prior 24 h, antibiotics, continuous renal replacement therapy (CRRT), vasopressors, and treatment limitations, all lagged by 1 h. Final weights were the cumulative product of treatment and censoring weights, truncated at the 99th percentile.

After cloning, censoring, and weighting, we fit weighted pooled logistic regression for the discrete-time hazard of death, deriving 7-, 14-, and 30-day risks [25]. Models included indicators for strategy, time, time squared, and strategy-by-time interactions. Predicted discrete-time hazards were converted to survival probabilities by cumulative products, and cumulative incidence was calculated as one minus survival. RDs and RRs were computed at each timepoint. Ninety-five percent confidence intervals (95% CIs) were obtained by nonparametric bootstraps with 500 resamples at the patient level, repeating the full analysis in each resample. Missing baseline covariates were imputed using chained equations, whereas time-varying covariates were handled using last observation carried forward. Due to computational constraints, the primary analysis used a single completed dataset.

We additionally applied the same clone-censor-weight framework and stabilized weights to estimate the weighted, strategy-specific cumulative incidence of intubation under each strategy, treating death before intubation as a competing event; 95% CIs were obtained by percentile bootstrap (R = 500). We also repeated the primary emulation (ROX < 3.85 versus usual care) separately in patients with and without a COVID-19 discharge diagnosis, using the same clone-censor-weight framework, weight models, and bootstrap procedure.

Sensitivity analyses

We conducted sensitivity analyses by requiring intubation within 1 h, rather than 2 h, after ROX first fell below the threshold, excluding patients intubated within 1 h of HFNC initiation, and excluding respiratory rate, SpO₂, and FiO₂ from time-varying confounder models. We also repeated the primary analysis using five imputed datasets, with 200 bootstrap resamples within each dataset, and combined results using Rubin’s rules. We calculated E values to quantify the minimum strength of unmeasured confounding needed to explain the estimates [26]. Analyses used R version 4.5.0 (R Foundation for Statistical Computing, Vienna, Austria), and code is available on GitHub (https://github.com/ryohei-hey/mimiciv-rox-strategy-impact-analysis).

Results

Patient characteristics

Among 94,458 ICU stays in MIMIC-IV, 2,513 patients received HFNC within 7 days of ICU admission and met the inclusion criteria. After excluding 862 patients on the basis of the prespecified exclusion criteria (136 admitted to a neurological ICU, 481 with invasive mechanical ventilation before HFNC initiation, and 245 with treatment-limitation orders before HFNC initiation), 1,651 remained for analysis (Fig. 1). Baseline characteristics at HFNC initiation are shown in Table 2. For clone-censor-weighting, all 1,651 patients were included in each strategy. Median age was 66 (57–74) years, 41% were women, and 76% were White. Median respiratory rate was 24 (20–29) breaths/min, SpO₂ 95% (92–97), FiO₂ 60% (50–70), and ROX 6.6 (4.8–9.1). Arterial blood gas values were missing in about 61% at baseline. Other missingness is summarized in Additional file 1 (Table S3). No patient was receiving extracorporeal membrane oxygenation (ECMO) or invasive ventilation at HFNC initiation, and 11 patients (0.7%) received venovenous or venoarterial ECMO during the hospitalization. Only ICD-9/10 discharge diagnoses are available in MIMIC-IV; most patients had a pulmonary cause of respiratory failure, most commonly pneumonia (Additional file 1, Table S4).

Fig. 1.

Selection of study participants and cloning procedure. This flow diagram shows the selection of the analytic cohort from the MIMIC-IV database and the cloning procedure used in the emulated target trial. For simplicity, this figure illustrates the two strategies used in emulated target trial 1 (usual care and the ROX-guided intubation strategy with a threshold of 3.85). The same cohort and cloning framework were used for the other emulated target trials with alternative ROX-guided strategies. MIMIC-IV Medical Information Mart for Intensive Care IV, HFNC high-flow nasal cannula, ICU intensive care unit, ROX (SpO₂/FiO₂)/respiratory rate, DNR do-not-resuscitate, DNI do-not-intubate, CMO comfort-measures-only, NIV noninvasive ventilation

Table 2.

Baseline patient characteristics

Characteristic	Missing, N (%)	Overall N = 1651a
Age, years	0 (0%)	66.0 (57.0, 74.0)
Female	0 (0%)	670 (41%)
Medicare/Medicaid	15 (0.9%)	1,172 (72%)
Language	1 (< 0.1%)
English		1,499 (91%)
Spanish		59 (3.6%)
Other		92 (5.6%)
Married	177 (11%)	737 (50%)
Race/Ethnicity	212 (13%)
White		1,090 (76%)
Black		174 (12%)
Hispanic/Latino		70 (4.9%)
Asian/Pacific Islander		44 (3.1%)
Other		61 (4.2%)
Heart rate, bpm	15 (0.9%)	94.0 (81.0, 109.0)
Systolic blood pressure, mmHg	24 (1.5%)	122.0 (109.0, 141.0)
Diastolic blood pressure, mmHg	24 (1.5%)	69.0 (59.0, 82.0)
Mean blood pressure, mmHg	23 (1.4%)	84.0 (74.0, 96.0)
Respiratory rate, /min	17 (1.0%)	24.0 (20.0, 29.0)
Temperature, °C	157 (9.5%)	36.9 (36.6, 37.2)
SpO2, %	16 (1.0%)	95.0 (92.0, 97.0)
FiO2, %	103 (6.2%)	60.0 (50.0, 70.0)
ROX index	121 (7.3%)	6.6 (4.8, 9.1)
Glasgow Coma Scale	186 (11%)
13–15		1,380 (94%)
9–12		70 (4.8%)
< 9		15 (1.0%)
pH	1,012 (61%)	7.4 (7.4, 7.5)
PaO2, mmHg	1,012 (61%)	81.0 (67.0, 103.0)
PaCO2, mmHg	1,012 (61%)	39.0 (33.0, 45.0)
SOFA score, first 24 h	0 (0%)	5.0 (3.0, 7.0)
Elixhauser comorbidity score	0 (0%)	12.0 (6.0, 20.0)
Antibiotic use	0 (0%)	606 (37%)
Vasopressor use	0 (0%)	80 (4.8%)
CRRT	0 (0%)	13 (0.8%)
Time from ICU admission to HFNC initiation, hours	0 (0%)	5.0 (0.0, 33.0)

SOFA Sequential Organ Failure Assessment. Values represent maximum scores observed during the first 24 h of HFNC initiation, SpO2 Peripheral oxygen saturation, FiO2 Fraction of inspired oxygen, CRRT Continuous renal replacement therapy

^aMedian (Q1, Q3); n (%)

^*Owing to the cloning step in the clone-censor-weight approach, patients' characteristics are identical at baseline for usual care and ROX-guided intubation strategy groups. For detailed explanation of the clone-censor-weight method, see methods section and supplementary methods

During follow-up, 64 patients (3.9%) initiated NIV and 339 patients (20.5%) were intubated. Among intubated patients, median time to intubation was 14 h (8–35) and median ROX at intubation was 4.23 (3.32–5.80) (Additional file 1, Figure S1 and Table S5). Median ROX at intubation under usual care varied across calendar-year groups (approximately 4.0 in 2008–2016, 4.65 in 2017–2019, and 3.77 in 2020–2022) (Additional file 1, Table S6). In ROX-guided strategies, artificial censoring due to nonadherence occurred in 922 (55.8%) for ROX < 3.85, 1,216 (73.7%) for ROX < 4.88, and 664 (40.2%) for the time-varying strategy, with few censored due to NIV initiation (Additional file 1, Figure S2). Among uncensored intubated patients, time to intubation was shorter under ROX-guided strategies than usual care, and ROX at intubation aligned with the prespecified thresholds (Additional file 1, Table S5). The inverse probability weights were generally moderate (median 0.97, maximum 32.9).

Estimated mortality under usual care and ROX-guided strategies

Figure 2 and Table 3 show cumulative incidence curves and estimated mortality risks under usual care and each ROX-guided strategy at days 7, 14, and 30. Mortality was lower under ROX-guided strategies than usual care across follow-up. In target trial 1 (ROX < 3.85), 30-day mortality risk was 20.5% (95% CI 17.1–24.1) under ROX guidance versus 26.9% (95% CI 24.7–29.2) under usual care, for a RD of −6.5 percentage points (95% CI −9.1 to −3.8) and a RR of 0.76 (95% CI 0.66–0.86). Estimates at days 7 and 14 followed the same pattern, and results for ROX < 4.88 and the time-varying strategy were in the same direction (Fig. 2 and Table 3).

Fig. 2.

Cumulative incidence of mortality under usual care and ROX-guided intubation strategies. Panels A–C show the estimated cumulative incidence of mortality up to 30 days after HFNC initiation under usual care and each ROX-guided strategy. Panel A corresponds to emulated target trial 1 (ROX < 3.85), Panel B to emulated target trial 2 (ROX < 4.88), and Panel C to emulated target trial 3 (time-varying ROX strategy). The primary end point was 30-day mortality, and secondary end points were mortality at days 7 and 14. Estimates were obtained from models adjusted for prespecified baseline covariates and hourly time-varying covariates using the clone-censor-weighting approach; 95% confidence intervals were derived from 500 bootstrap replicates. The usual care strategy was identical across trials and estimated from the same underlying cohort

Table 3.

Estimated risks of mortality under usual care and ROX-guided intubation strategies at prespecified timepoints

Trial	ROX strategy	Timepoint	Usual care†, % (95% CI)‡	ROX strategy, % (95% CI)‡	Risk difference, percentage points (95% CI)‡	Risk ratio (95% CI)‡
Emulated target trial 1	ROX < 3.85	Day 7	11.7 (10.2–13.2)	8.3 (6.3–10.4)	−3.4 (−5.3 to −1.6)	0.71 (0.55–0.86)
		Day 14	19.6 (17.8–21.5)	14.1 (11.2–17.0)	−5.6 (−8.0 to −3.2)	0.72 (0.60–0.84)
		Day 30*	26.9 (24.7–29.2)	20.5 (17.1–24.1)	−6.5 (−9.1 to −3.8)	0.76 (0.66–0.86)
Emulated target trial 2	ROX < 4.88	Day 7	11.7 (10.2–13.2)	8.0 (5.8–10.6)	−3.7 (−5.7 to −1.5)	0.69 (0.52–0.88)
		Day 14	19.6 (17.8–21.5)	13.6 (10.6–17.0)	−6.0 (−8.7 to −2.9)	0.69 (0.57–0.84)
		Day 30*	26.9 (24.7–29.2)	19.8 (16.3–24.0)	−7.1 (−10.3 to −3.5)	0.74 (0.62–0.87)
Emulated target trial 3	Time-varying ROX strategy§	Day 7	11.7 (10.2–13.2)	9.1 (7.2–11.1)	−2.6 (−4.2 to −1.1)	0.78 (0.64–0.90)
		Day 14	19.6 (17.8–21.5)	15.0 (12.3–17.5)	−4.6 (−6.6 to −2.7)	0.76 (0.66–0.86)
		Day 30*	26.9 (24.7–29.2)	21.3 (18.5–24.3)	−5.6 (−7.9 to −3.4)	0.79 (0.70–0.87)

Risk differences were calculated from unrounded estimates and may differ slightly from the difference of the rounded risks shown

^*Primary outcome timepoint

^†The usual care strategy was identical across emulated target trials and estimated from the same underlying cohort

^‡Estimates were obtained from models adjusted for prespecified baseline covariates and hourly time-varying covariates; 95% confidence intervals were obtained using 500 bootstrap replicates

^§Time-varying ROX strategy: thresholds were 2.85 (hours 1–5), 3.47 (hours 6–11), and 3.85 (≥ 12 h)

Cumulative incidence of intubation

The weighted 30-day cumulative incidence of intubation was 21.5% (95% CI 19.5–23.4) under usual care, 22.0% (95% CI 17.3–27.1) under ROX < 3.85 (risk difference [RD], 0.5 percentage points [95% CI −3.5 to 4.6]), 25.8% (95% CI 20.1–32.3) under ROX < 4.88 (RD, 4.2 percentage points [95% CI −0.7 to 9.9]), and 19.6% (95% CI 15.4–23.9) under the time-varying strategy (RD, −1.9 percentage points [95% CI −5.2 to 1.6]). The usual care estimate was close to the crude observed intubation proportion in the eligible cohort (20.5%). Estimates at days 7 and 14 followed the same pattern (Figure S3 and Table S7).

Sensitivity analyses

For the primary endpoint (30-day mortality), results were similar across sensitivity analyses (Additional file 1, Tables S8–S11). Requiring intubation within 1 h after ROX first fell below the threshold, the ROX < 3.85 strategy yielded a RD of −7.3 percentage points (95% CI −9.9 to −4.6) and a RR of 0.73 (95% CI 0.63–0.83). Excluding patients intubated within 1 h of HFNC initiation gave a RD of −6.5 percentage points (95% CI −8.8 to −4.2) and a RR of 0.76 (95% CI 0.67–0.84). Excluding respiratory rate, SpO₂, and FiO₂ from time-varying confounder models gave a RD of −6.2 percentage points (95% CI −8.5 to −3.8) and a RR of 0.77 (95% CI 0.68–0.86). In the multiple imputation sensitivity analysis, the ROX < 3.85 strategy yielded a RD of −6.6 percentage points (95% CI −9.1 to −4.1) and a RR of 0.75 (95% CI 0.67–0.85). In a subgroup emulation, the 30-day mortality risk difference for ROX < 3.85 versus usual care was −8.3 percentage points (95% CI −18.6 to 1.0) among patients with COVID-19 (n = 244; 33.0% under usual care versus 24.7% under ROX guidance) and −6.1 percentage points (95% CI −8.7 to −3.7) among patients without COVID-19 (n = 1,407; 25.8% versus 19.7%) (Additional file 1, Table S12).

For 30-day mortality, E values for risk ratio point estimates were 1.96, 2.06, and 1.84 for ROX < 3.85, ROX < 4.88, and the time-varying strategy. E values for the 95% confidence limit closest to 1.0 were 1.60, 1.58, and 1.55.

Discussion

Summary of findings

Using hourly electronic health record data from MIMIC-IV, we emulated three target trials comparing usual care with prespecified ROX-guided intubation strategies at HFNC initiation and estimated per-protocol effects on 30-day mortality. In each emulation, adherence to ROX-guided strategies was associated with lower mortality than usual care, with similar findings across sensitivity analyses. E values suggested that explaining away the observed benefit would require an unmeasured confounder with at least moderate associations with both adherence and 30-day mortality, beyond the measured covariates. By reframing the ROX index from a prognostic score to a model-guided treatment policy, we moved from prediction toward causal effect estimation, estimating the mortality impact of strategy adherence in adults receiving HFNC.

Context in prior literature and mechanistic plausibility

This study moves beyond prognostic use of the ROX by estimating the effect of ROX-guided intubation strategies on mortality. Prior studies primarily evaluated ROX for predicting HFNC failure and for risk stratification using fixed thresholds at prespecified times after HFNC initiation [7, 8]. Target trial emulations in this database have addressed the timing of intubation in broad terms, comparing early with delayed intubation [13, 14] or comparing oxygenation-ratio thresholds for initiating invasive ventilation [15]. Our work differs in evaluating a specific, named bedside score, the ROX index, which combines oxygenation and respiratory rate, operationalized as a decision policy with literature-derived thresholds and compared with the natural course of care. Prediction performance alone does not determine whether acting on ROX improves patient outcomes in routine care, where treatment selection and confounding influence both intubation decisions and prognosis [27].

ROX-guided decision policies may reduce harmful delay in recognizing HFNC failure and prompt invasive ventilation before deterioration becomes emergent. With HFNC deterioration, increased work of breathing may accompany worsening hypoxemia and ventilatory failure, progressing to fatigue, hemodynamic instability, and emergent intubation [1]. Earlier escalation may also limit exposure to excessive spontaneous respiratory effort linked to lung injury in acute hypoxemic respiratory failure (patient self-inflicted lung injury) [28]. In our data, among patients who were intubated, those under ROX-guided strategies had a shorter time to intubation (Table S5); because this comparison is restricted to intubated patients, it is descriptive and subject to selection bias, and it cannot by itself distinguish earlier escalation from a change in which patients are intubated [3, 4, 13]. A prespecified ROX protocol could intubate some patients who would remain on HFNC under clinician-directed care while allowing others to remain on HFNC who would otherwise be intubated, and our observational data cannot separate this treatment selection from earlier escalation. In the present analysis; however, the weighted 30-day cumulative incidence of intubation was similar under ROX < 3.85 and usual care (22.0% versus 21.5%), indicating a change in the timing rather than the number of intubations.

The estimated reduction of approximately 6–7 percentage points may be larger than would be expected from a change in the timing of intubation alone. A systematic review of early versus late intubation in critically ill patients without COVID-19 [29] reported a similar absolute difference (7.7 percentage points), but from a much higher baseline mortality (53.5% versus 26.9% here), so the same absolute gain there implies a smaller proportional one: risk ratio 0.92 (95% CI 0.87–0.97), against 0.74–0.79 in our data. The two agree only if the true effect lies at the weakest end of our range (risk ratio 0.86–0.87); that review also addresses intubation timing rather than an ROX-guided policy. Part of the difference may nonetheless reflect residual confounding rather than the strategy itself, consistent with the only moderate E values observed.

Because our comparison depends on usual care, we clarify what it represents. Usual care here is the natural course of the observed cohort, that is, the management that actually occurred without any protocol imposed; our objective was not to identify an optimal ROX cutoff but to estimate the effect of a prespecified ROX rule relative to this heterogeneous clinician judgment. In this emulation, usual care followed the observed HFNC management in the ICU and did not involve artificial censoring. Under usual care, 30-day mortality was 26.9% and the intubation rate was 20.5%, broadly consistent with published HFNC cohorts [30, 31], supporting that usual care reflected plausible ICU practice during the study period. Among intubated patients, median time to intubation was 14 h (8–35) and median ROX at intubation was 4.23 (3.32–5.80) under usual care, broadly consistent with prior HFNC cohorts [5, 6, 30]. To assess secular changes in usual care over 2008–2022, we summarized intubation characteristics by calendar period (Table S6); the median ROX at intubation varied across periods without a consistent monotonic trend, suggesting clinician-directed practice rather than a uniform ROX threshold protocol, although we did not estimate era-specific effects (see Limitations). Nonetheless, usual care depends on local resources and practice patterns, so replication in other settings remains important [32].

Observed practice often differed from the prespecified ROX-guided rules, with 40–74% of patients deviating from the strategies as operationalized for clone-censor-weighting (Table S5). Deviation of this magnitude is expected in observational data, because no ROX-based protocol was in place and clinicians were never asked to follow these rules. Because these deviations trigger artificial censoring, they do not by themselves establish low real-world feasibility. Instead, they indicate tension between a prespecified ROX-only protocol and clinician-directed care, reflecting both appropriate over-rides (e.g., low work of breathing) and missed opportunities (e.g., delayed escalation), as well as workflow constraints not fully captured in our data. A strategy that is not being implemented will, by construction, be followed by only a minority of patients, so rates of this magnitude do not in themselves make the strategies unrealistic or the estimates uninformative. Their main consequence is statistical: less observed person-time informs each ROX-guided strategy, so its estimate relies more on the weighting model and is less precise, most notably for ROX < 4.88 (73.7% censored). Accordingly, our per-protocol estimates represent effects under hypothetical adherence supported by the observed data, and feasibility should be assessed prospectively.

Clinical and research implications

Our estimand was per protocol, so estimated benefits reflect outcomes under strategy adherence and may not match effects in routine implementation if adherence is lower or workflow burden is higher. For clinical use, ROX should complement bedside assessment, including respiratory effort, mental status, hemodynamics, and trends in oxygen requirements, with arterial blood gases when indicated. ROX may help recognize HFNC failure earlier and limit delays in escalation to invasive ventilation. These observational results do not replace randomized evidence, but they can inform prospective studies by providing plausible effect sizes and operational thresholds. A pilot randomized trial is ongoing (ROX-1, ClinicalTrials.gov NCT04707729). Because the ROX index was developed and most widely applied in COVID-19, and because our COVID-19 subgroup was too small for a reliable subgroup estimate, patients with COVID-19 are a priority population for prospective evaluation.

Two points deserve emphasis given the nature of the data. First, the value of this analysis is not to mandate a fixed ROX threshold but to provide a standardized decision-support trigger that may reduce delayed recognition of HFNC failure, and to generate the plausible effect sizes and operational thresholds needed to justify and design a prospective trial. Second, the high rate of protocol deviation should be distinguished from unmeasured confounding. Deviation is a feasibility and estimand issue, which the clone-censor-weighting design addresses by targeting a well-defined per-protocol effect; confounding is a separate threat, which we addressed by adjusting for a broad set of baseline and hourly time-varying covariates and quantified with E values. Neither device removes the possibility that determinants of intubation that are documented incompletely, such as work of breathing and goals of care, continue to bias the comparison, and this residual confounding cannot be excluded.

From a research perspective, this work moves beyond prediction by estimating the effect of acting on ROX-guided decision policies. The same framework may be applied to other ICU decision rules and trigger-based policies, including model-derived alerts, to evaluate clinical impact using routinely collected data.

Limitations

First, we studied ICU patients initiated on HFNC rather than all patients with acute respiratory failure. Inferences are, therefore, restricted to patients meeting the eligibility criteria and to settings with similar acuity at HFNC initiation, and do not extend to patients managed primarily with noninvasive ventilation, such as those with acute cardiogenic pulmonary edema due to heart failure. The cohort was of mixed etiology, and both respiratory management and the ROX index differ across etiologies; our estimates, therefore, represent an average over this heterogeneity, and etiology-specific evaluation is needed. Second, usual care was drawn from a single center, so findings may not generalize to ICUs with different case mixes, HFNC practices, staffing, or intubation culture. Third, if ROX was already used informally in usual care, the contrast between strategies may have been reduced and estimated effects may be attenuated. Fourth, residual confounding is possible because determinants of intubation decisions, such as respiratory effort, fluid balance, cardiac function, goals of care, and bedside assessment, were not fully captured; because deviations from the ROX-guided strategies were artificially censored, such unmeasured factors may also render that censoring informative. E values quantify the strength of unmeasured confounding needed to explain the estimates but do not rule it out. Fifth, exposures, covariates, and timing were derived from routine EHR documentation, so measurement error and documentation delays may have misclassified ROX values, threshold crossings, and adherence. Sixth, the strategy-specific intubation estimates rest on an assumption the mortality analysis does not require: no unmeasured common causes of treatment at different timepoints. Such common causes are plausibly more frequent than treatment-outcome ones, so these estimates warrant more caution.

Conclusions

In this target trial emulation using hourly ICU data, adherence to ROX-guided intubation strategies was associated with lower 30-day mortality than usual care among adults initiated on HFNC. These findings are hypothesis-generating rather than confirmatory, and they require prospective and multicenter validation. Used as a decision policy rather than as a prediction score, the ROX index is a promising candidate for such evaluation, although its real-world feasibility remains to be established.

Abbreviations

CMO

Comfort-measures-only

CI

Confidence interval

CRRT

Continuous renal replacement therapy

DNI

Do-not-intubate

DNR

Do-not-resuscitate

ECMO

Extracorporeal membrane oxygenation

EHR

Electronic health record

FiO₂

Fraction of inspired oxygen

HFNC

High-flow nasal cannula

HIPAA

Health Insurance Portability and Accountability Act

ICU

Intensive care unit

MIMIC-IV

Medical Information Mart for Intensive Care IV

NIV

Noninvasive ventilation

PaCO₂

Partial pressure of arterial carbon dioxide

PaO₂

Partial pressure of arterial oxygen

RD

Risk difference

ROX

Ratio of oxygen saturation (SpO₂/FiO₂) to respiratory rate

RR

Risk ratio

SOFA

Sequential Organ Failure Assessment

SpO₂

Peripheral oxygen saturation

TARGET

Transparent Reporting of Observational Studies Emulating a Target Trial

Author contributions

All authors contributed to the study conception and design. RY and NP performed material preparation. RY performed the data analysis and wrote the first draft of the manuscript. TT, AH, NP, KNW, JB, and LAC provided critical revisions of the manuscript. All authors read and approved the final manuscript and agree to be accountable for all aspects of the work.

Funding

This work was supported by a grant from The Naito Foundation (FY2024 Overseas Research Fellowship/Grant). The funder had no role in the study design; data collection, analysis, or interpretation; manuscript writing; or the decision to submit the work for publication.

Data availability

The MIMIC-IV dataset analyzed during the current study is available in the PhysioNet repository, https://physionet.org/content/mimiciv/. The analytic code generated during the current study is available in the GitHub repository, https://github.com/ryohei-hey/mimiciv-rox-strategy-impact-analysis.

Declarations

Ethics approval and consent to participate

The establishment of the MIMIC-IV database was approved by the Institutional Review Board of the Beth Israel Deaconess Medical Center (Boston, MA, USA), which granted a waiver of informed consent for the collection and research use of the de-identified data. MIMIC-IV is de-identified in accordance with the Health Insurance Portability and Accountability Act (HIPAA) Safe Harbor provisions, and access is provided via PhysioNet to credentialed users who complete required training and sign a data use agreement. Because the present study used only publicly available de-identified data and contained no directly identifiable information, it was considered not to constitute human subjects research, and additional ethics approval and individual informed consent were not required.

Consent for publication

Not applicable. This manuscript does not contain any individual person’s identifiable data in any form.

Competing interests

The authors declare that they have no competing interests.