Time-dependent predictive value of the HACOR score for non-invasive ventilation failure in the emergency department: a prospective cohort study

Abstract

Background

Early recognition of non-invasive ventilation failure in patients with acute respiratory failure in the emergency department is essential to prevent delayed intubation and its associated adverse outcomes. The HACOR score is a pragmatic bedside tool developed to predict non-invasive ventilation failure, most prior investigations have been conducted in intensive care settings and have predominantly relied on single time-point measurements or simple change analyses. In this study, we aimed to evaluate predictive value of early change in the HACOR score for non-invasive ventilation failure in emergency department patients with acute respiratory failure and to examine the temporal nature of this association using Generalized Estimating Equations models.

Methods

This prospective observational cohort study included 106 adult patients. HACOR, National Early Warning Score, National Early Warning Score 2, and Modified Early Warning Score were calculated immediately before and at 1 h after non-invasive ventilation initiation. ΔHACOR was defined as the difference between 1-hour and baseline HACOR scores. The primary outcome was non-invasive ventilation failure. Discriminative performance was assessed using ROC analysis. Logistic regression was used to evaluate the independent association between ΔHACOR and non-invasive ventilation failure. Generalized estimating equations models were constructed to analyze the time-dependent relationship between HACOR and non-invasive ventilation failure.

Results

Non-invasive ventilation failure occurred in 50.9% of patients. Baseline clinical variables and scores did not significantly discriminate between success and failure groups. In contrast, 1-hour HACOR (AUC = 0.760, 95% CI 0.667–0.854) and ΔHACOR (AUC = 0.798, 95% CI 0.711–0.885) demonstrated significant predictive performance. Patients with ΔHACOR ≥ 0 exhibited a substantially higher failure rate (74.4%) compared to those with ΔHACOR < 0 (34.9%). Each one-point increase in ΔHACOR was independently associated with failure (OR 1.63, 95% CI 1.31–2.03; p < 0.001). In Generalized Estimating Equations analysis, the HACOR × time interaction remained statistically significant across adjusted models, supporting the prognostic value of serial HACOR assessment.

Conclusions

In emergency department patients receiving non-invasive ventilation, early dynamic changes in HACOR provide superior prognostic information compared with baseline measurements alone. Serial HACOR assessment demonstrated through Generalized Estimating Equations modeling supports potential value of response-guided risk stratification and highlights importance of early reassessment in emergency non-invasive ventilation management.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12873-026-01665-x.

Introduction

Noninvasive ventilation (NIV) is an effective respiratory support modality widely used in patients with acute respiratory failure (ARF) to reduce the work of breathing, improve gas exchange, and decrease the need for endotracheal intubation. In clinical conditions such as acute exacerbations of chronic obstructive pulmonary disease (COPD), acute cardiogenic pulmonary edema, and pneumonia, NIV has been shown—when applied to appropriately selected patients—to reduce intensive care unit (ICU) admission rates and mortality [1, 2]. Despite these benefits, a substantial proportion of patients treated with NIV experience treatment failure, necessitating escalation to invasive mechanical ventilation (IMV) [3, 4]. Marjanovic et al. [5] expanded the definition of NIV failure beyond intubation requirement alone, incorporating a broader concept of early clinical deterioration that includes persistent or worsening respiratory failure, decline in level of consciousness, and hemodynamic instability.

In dynamic clinical environments such as the emergency department (ED), where rapid deterioration is common and close monitoring is essential, early recognition of NIV failure and timely transition to IMV are critical determinants of clinical outcomes. In patients at high risk of NIV failure, early escalation to IMV has been associated with reductions in mortality and complications related to delayed intubation, as well as mitigation of prolonged ICU stay and increased hospital costs [6, 7]. The literature suggests that the predictive performance of any single clinical or laboratory parameter in forecasting NIV failure is limited; in contrast, the combined evaluation of multiple variables improves prognostic accuracy [6, 8]. In response to this need, the HACOR score was developed [9] as a bedside risk tool incorporating readily available physiological parameters, including heart rate (HR), acidosis (pH), level of consciousness (Glasgow Coma Scale [GCS]), oxygenation (arterial partial pressure of oxygen [PaO₂] / fraction of inspired oxygen [FiO₂], and respiratory rate (RR). Studies conducted across diverse patient populations have demonstrated that HACOR has good discriminative performance, particularly in the early phase of NIV, and can be safely applied in ED settings [i10, 11].

In the existing literature, the HACOR score has predominantly been evaluated in ICU settings, typically at a single time point or using simple change analyses. Studies conducted in EDs, particularly those involving heterogeneous patient populations and capable of directly informing early clinical decision-making [3, 5, 6, 10–14] remain limited. Moreover, investigations specifically examining the time-varying effect of HACOR (HACOR × time) using repeated-measures modeling approaches are scarce.

In this study, we aimed to evaluate the predictive value of baseline (0-hour, HACOR₀) and 1-hour HACOR (HACOR₁) scores, as well as early dynamic change (ΔHACOR), for NIV failure in patients with ARF treated with NIV in the ED. Additionally, we sought to characterize the temporal structure of this association using a Generalized Estimating Equations (GEE) model.

Material and methods

Study design and ethical approval

This prospective observational cohort study was conducted between March 1, 2024 and October 31, 2025 in accordance with the Declaration of Helsinki and received approval from the Non-Interventional Research Ethics Committee of Ankara Etlik City Hospital (AEŞH-BADEK-2024-909; February 10, 2024).

Participants

A total of 106 patients were included in the study, while 54 patients were excluded due to missing data, venous blood gas measurements, or failure to meet predefined inclusion criteria (Fig. 1). Eligible participants were adult patients (≥ 18 years) who presented to the ED with type I, type II, or mixed ARF and were initiated on NIV. RF was categorized according to conventional physiological classifications. Type 1 respiratory failure was defined as hypoxemic respiratory failure, type 2 respiratory failure as hypercapnic respiratory failure, and mixed respiratory failure as the coexistence of hypoxemia and hypercapnia on arterial blood gas analysis. Patients were required to have sufficient spontaneous respiratory effort and an effective cough reflex to ensure the potential success of NIV. In addition, only patients with preserved levels of consciousness adequate to cooperate with the procedure were included, allowing safe and effective application of NIV. In line with European Respiratory Society / American Thoracic Society (ERS/ATS) Clinical Practice Guideline and Rezoagli et al. recommendations [15, 16], these parameters were assessed based on clinical judgment, as no specific minimum numerical thresholds are defined for patient selection.

Fig. 1.

Flow diagram of patient selection

Pregnant patients; those presenting with respiratory or cardiac arrest; patients with apneic episodes accompanied by loss of consciousness (LOC); those exhibiting agonal (gasping) breathing patterns; and individuals requiring immediate endotracheal intubation were excluded. Patients who refused NIV, were unable to tolerate NIV due to severe agitation, or had contraindications to NIV—such as bullous lung disease or pneumothorax—were also excluded from the study.

Sample size justification

In this study, the HACOR score was calculated for each patient immediately before NIV initiation (baseline, 0-hour) and at 1 h after NIV initiation. The primary analysis was designed to evaluate whether ΔHACOR differed between the two clinical outcome groups (NIV success and NIV failure). Accordingly, ΔHACOR was defined as HACOR (1-hour) − HACOR (0-hour), and group comparisons were planned based on the mean ΔHACOR values. Given that the reported NIV failure rate in the literature and in clinical observations is approximately 50%, the two groups were assumed to be of roughly equal size.

In a study evaluating the association between early HACOR measurements and NIV failure [17] mean ± standard deviation (SD) values of HACOR scores at baseline and at 1 and 2 h were reported for both success and failure groups. Using the summary statistics provided in that study, the mean change values were estimated as ΔHACOR (success) ≈ − 2.2 and ΔHACOR (failure) ≈ − 0.4, corresponding to an approximate between-group mean difference of 1.8 points. As the correlation coefficient between repeated measurements was not reported, a correlation coefficient of ρ = 0.5 was assumed to estimate the SD of the score changes. Under this assumption, the pooled SD of the score changes was calculated to be approximately 2.97, yielding a standardized effect size (Cohen’s d) of approximately 0.61.

Assuming a two-sided significance level of α = 0.05 and 80% statistical power, the required sample size for comparison of two independent groups was calculated using the formula n = 2 × (Z₁−α/2 + Z₁−β)² / d². Based on this calculation, approximately 43 patients per group (≈ 85 in total) were required. To account for potential missing HACOR₁ measurements and possible incomplete data, a 20% attrition rate was anticipated, resulting in a planned final target sample size of approximately 106–108 patients.

Data collection and risk stratification

Data were collected using a structured case report form (CRF) specifically designed for this study. The form was completed prospectively at the bedside by the ED physician responsible for the patient’s initial evaluation and management. The CRF systematically recorded demographic characteristics (age, sex), relevant comorbidities (e.g., hypertension, diabetes mellitus, asthma, COPD, heart failure), admission diagnosis, and type of ARF (type 1, type 2 and mixed-type).

NIV-related variables were documented in a standardized manner, including ventilation mode (bilevel positive airway pressure [BiPAP] or continuous positive airway pressure [CPAP]), device type, and initial ventilator settings. Comprehensive clinical and physiological parameters were also recorded, including GCS and AVPU score, blood pressure (BP), HR, RR, body temperature, peripheral oxygen saturation (SpO₂). Laboratory data were collected concurrently and included serum glucose, urea, creatinine, electrolytes, liver function tests, bilirubin levels, C-reactive protein (CRP), complete blood count parameters, and arterial blood gas parameters (pH, PaO₂, arterial partial pressure of carbon dioxide [PaCO₂], arterial oxygen saturation [SaO₂], lactate).

Patients’ disposition and mortality outcomes were tracked through the hospital information management system. The primary endpoint—NIV success or failure—was determined by the investigators based on data recorded in CRFs, and the final outcome classification was subsequently documented in the CRF.

Risk stratification was performed using four validated early warning and respiratory failure scoring systems: HACOR, National Early Warning Score (NEWS), National Early Warning Score 2 (NEWS2), and Modified Early Warning Score (MEWS). All scores were calculated according to their original definitions and parameter weighting schemes.

The HACOR score incorporates HR, arterial pH, GCS, PaO₂/FiO₂, and RR, yielding a total score range of 0–27. Consistent with prior validation studies, a threshold of ≥ 5 points was used to classify patients as high-risk for NIV failure [8, 9, 17].

The NEWS and NEWS2 scores assign weighted points based on RR, SpO₂, systolic blood pressure (SBP), HR, level of consciousness (AVPU score), and body temperature, with NEWS2 additionally adjusting the oxygenation component for patients with hypercapnic respiratory failure. MEWS evaluates similar physiological variables but does not include oxygenation parameters such as SpO₂ or FiO₂. In NEWS/NEWS2, scores of 0–4 indicate low risk, ≥ 5 indicate moderate-to-high risk with need for urgent reassessment, and ≥ 7 are associated with high-risk physiology and potential need for advanced respiratory support. MEWS follows a comparable interpretation framework, with scores ≥ 5 reflecting a high risk of early clinical deterioration [18–20].

Overall, in all three scoring systems, increasing score values correlate with a progressively higher likelihood of acute clinical deterioration, and a single elevated score (red score) warrants timely evaluation and potential escalation of care according to institutional early warning protocols.

To evaluate the dynamic physiologic response to treatment, all four scoring systems were computed at two predefined time points: baseline and at 1 h after NIV initiation. Calculations were based on the vital signs, arterial blood gas results, and clinical assessments documented at presentation and during early follow-up. Changes in scores between baseline and the 1-hour assessment were subsequently analyzed in relation to NIV success or failure.

Treatment protocol

NIV was administered in accordance with current international recommendations for the management of ARF [15, 16], using CARESCAPE R860 ventilators (GE Healthcare, Chicago, IL, USA) and Evita XL ventilators (Dräger, Lübeck, Germany). The choice of ventilation mode (BiPAP or CPAP) was determined by the attending emergency physician responsible for the patient’s management, based on the underlying respiratory physiology, arterial blood gas findings, and overall clinical status.

NIV was delivered using oronasal masks covering both the mouth and nose (Flow-Safe II Disposable Bilevel CPAP System; Mercury Medical, Clearwater, FL, USA). To minimize interface-related variability and leak-associated differences, the same mask type was consistently used in all patients throughout the study period. NIV was applied in accordance with the institutional protocol, consisting of 45-minute treatment sessions followed by 15-minute intervals. During application, particular attention was paid to minimizing mask leaks.

Initial BiPAP NIV settings were generally established with an inspiratory positive airway pressure (IPAP) of 10–12 cmH₂O and an expiratory positive airway pressure (EPAP) of 4–6 cmH₂O. Pressure levels were titrated in 2 cmH₂O increments based on reduction in dyspnea, decrease in RR, trends in PaCO₂, and overall patient tolerance. During the application of external positive end-expiratory pressure (PEEP), EPAP levels were generally not increased beyond 8 cmH₂O in patients with obstructive lung disease.

FiO₂ adjustments were made to achieve target oxygen saturation levels of 88–92% in hypercapnic patients and ≥ 94% in non-hypercapnic patients.

For CPAP NIV, initial pressure levels were typically set at 5–8 cmH₂O and subsequently titrated according to oxygenation response, improvement in respiratory distress, and hemodynamic tolerance.

NIV was initiated using predefined initial ventilator settings. Subsequent adjustments were individualized by the treating physician based on the patient’s respiratory physiology, arterial blood gas parameters, and overall clinical status, within predefined clinical targets (e.g., improvement in RR, gas exchange, and patient comfort). As this was an observational study, the investigators did not interfere with patient management, and all diagnostic and treatment decisions were made by the attending physicians.

Definitions of early response and failure to NIV

International guidelines [15, 16] do not define a single numerical threshold for NIV failure; rather, failure is generally characterized by the absence of an adequate early physiological response or the development of clinical deterioration. This approach underscores the need for dynamic and multidimensional assessment strategies.

In a meta-analysis conducted by Abdelfattah et al. [21], several clinical and physiological parameters were identified as predictors of NIV failure. These included severe or persistent acidosis (pH < 7.25–7.30), elevated RR (≥ 30–35 breaths/min), and impaired oxygenation, particularly a PaO₂/FiO₂ ratio below 150–200 mmHg, which has been reported as a strong predictor of NIV failure in previous studies [22, 23]. In addition, lack of reduction or an increase in PaCO₂ levels, deterioration in the level of consciousness, and the presence of hemodynamic instability were also associated with NIV failure. These findings are consistent with earlier studies in the literature [24].

Based on this evidence, early favorable response to NIV in the present study was evaluated using the following criteria: a reduction in RR of ≥ 20% from baseline or a RR < 30 breaths/min [25, 26]; a clinically meaningful increase in pH (> 0.03) [27]; a clinically meaningful decrease in PaCO₂ (> 3 mmHg) [27]; improvement in oxygenation parameters (SpO₂ or PaO₂/FiO₂) under the same or lower FiO₂; and regression of clinical signs of respiratory distress, such as reduced use of accessory muscles and subjective relief of dyspnea.

Patients who did not meet these criteria and/or exhibited pH < 7.25, RR ≥ 35 breaths/min, or PaO₂/FiO₂ <150–200 mmHg were classified as being at high risk for NIV failure, in line with previous literature.

The decision to proceed with endotracheal intubation was made by the attending emergency physician in accordance with international guideline recommendations [15, 16]. As current guidelines do not define a single uniform set of criteria for intubation during NIV, the decision was based on dynamic clinical assessment. Intubation was considered in the presence of clinical deterioration, contraindications to NIV, or lack of early physiological improvement despite optimized ventilatory support. Specifically, indications included worsening gas exchange (persistent or worsening acidosis, increasing PaCO₂, or refractory hypoxemia), deterioration in mental status, hemodynamic instability, inability to tolerate NIV, or absence of an early clinical response. This approach is consistent with guideline recommendations emphasizing individualized and time-dependent clinical evaluation rather than fixed numerical thresholds.

Statistical analysis

All statistical analyses were performed using R software within the RStudio environment (R version 4.4.3; 2025-02-28). The distributional characteristics of continuous variables were assessed using both visual methods (histograms and Q–Q plots) and analytical tests (Kolmogorov–Smirnov and Shapiro–Wilk tests), and were summarized as mean ± SD or median (interquartile range [IQR]), as appropriate.

Between-group comparisons were conducted using the independent samples t-test or the Mann–Whitney U test, as appropriate. Categorical variables were analyzed using the chi-square test or Fisher’s exact test. Within-patient comparisons of 0-hour and 1-hour measurements were performed using the paired t-test or the Wilcoxon signed-rank test.

The predictive performance of HACOR, NEWS, NEWS2, and MEWS for NIV failure was evaluated using receiver operating characteristic (ROC) curve analysis, with area under the curve (AUC) and corresponding 95% confidence intervals (CIs). Optimal cut-off values were determined using the Youden index, and AUCs were compared using the DeLong test.

The association between ΔHACOR and NIV failure was examined using univariable logistic regression, reported as odds ratios (ORs) with 95% CIs. To evaluate the time-dependent relationship in repeated measurements, a GEE model with a binomial distribution and logit link function was constructed. Time, HACOR, and the HACOR × time interaction were included as main effects, and relevant clinical covariates were incorporated into the model. Model fit was compared using the quasi-likelihood under the independence model criterion (QIC) and its corrected version (QICC).

All statistical tests were two-sided, and a p-value < 0.05 was considered statistically significant.

Results

A total of 106 patients were included in the study. The mean age was 71.2 ± 11.3 years, and 51.9% of the cohort were female. The most common comorbidities were hypertension (62.3%) and COPD (59.4%). Among admission diagnoses, acute exacerbation of COPD (52.8%) and pneumonia (30.2%) were the most frequent. Regarding the type of RF, mixed-type failure was the predominant presentation (61.3%). Detailed distributions of sociodemographic characteristics, comorbidities, admission diagnoses, and RF types are presented in Supplementary Table 1.

NIV was successful in 49.1% of patients (n = 52) and failed in 50.9% (n = 54). Following ED management, 55.7% of patients were admitted to the ICU, 30.2% were admitted to hospital wards, and 4.7% were discharged directly from the ED. Endotracheal intubation was required in 23.3% of patients; intubations occurred both in the ED and in the ICU. The 28-day mortality rate was 17.5%. Among the patients in whom NIV failed but who were not intubated (n = 34), 58.8% (n = 20) were admitted to the ICU and 32.4% (n = 11) to hospital wards, all of whom were subsequently discharged with clinical improvement. 5.9% (n = 2) refused further treatment in the ED and were discharged against medical advice, with no mortality observed. 2.9% (n = 1) could not be admitted to the ICU due to bed unavailability and was managed in the ED, from which they were later discharged. Detailed distributions of clinical outcomes are provided in Supplementary Table 2.

When baseline and 1-hour measurements were compared, statistically significant changes were observed in HR, RR, SpO₂, pH, PaCO₂, and lactate levels. In contrast, no significant differences were found in systolic or diastolic BP, body temperature, FiO₂ requirement, or the PaO₂/FiO₂ ratio (all p > 0.05).

Regarding clinical scoring systems, significant changes at 1 h were observed in HACOR, NEWS, NEWS2, and MEWS scores. GCS and AVPU scores also demonstrated statistically significant differences over time. Detailed results are presented in Supplementary Table 3.

In comparisons according to NIV outcome, no significant differences were observed between the success and failure groups in terms of sex distribution, hypertension, diabetes, COPD, or other admission diagnoses (all p > 0.05). However, a history of asthma was more frequent in the NIV success group, whereas pneumonia diagnosis and the distribution of RF types were significantly associated with NIV failure (Table 1).

Table 1.

Baseline demographic characteristics, comorbidities, admission diagnoses, and RF types according to NIV outcome

Variable	Failure (n = 54) n (%)	Success (n = 52) n (%)	p-value
Gender			0.99
Female	28 (51.9)	27 (51.9)
Male	26 (48.1)	25 (48.1)
Comorbidities
Hypertension	34 (63)	32 (61.5)	0.88
Diabetes	17 (31.5)	21 (40.4)	0.34
Asthma	1 (1.9)	6 (11.5)	0.045
COPD	36 (66.7)	27 (51.9)	0.12
Other	27 (50)	35 (67.3)	0.07
Diagnosis
COPD exacerbation	30 (55.6)	26 (50)	0.57
Pneumonia	22 (40.7)	10 (19.2)	0.02
Congestive heart failure	16 (29.6)	17 (32.7)	0.73
Hypertensive pulmonary edema	3 (5.6)	6 (11.5)	0.27
Other	7 (13)	5 (9.6)	0.59
RF type			0.02
Type 1	18 (33.3)	16 (30.8)
Type 2	0 (0.0)	7 (13.5)
Mixed-type	36 (66.7)	29 (55.8)

When baseline clinical variables, vital signs, arterial blood gas parameters, and clinical scores were compared according to NIV outcome, age was the only variable significantly higher in the failure group (p = 0.04). No significant differences were observed between the success and failure groups with respect to systolic and diastolic BP, RR, oxygenation parameters, acid–base status, or HACOR, NEWS, NEWS2, MEWS, GCS, and AVPU scores (all p > 0.05). Detailed comparative results are presented in Supplementary Table 4 while Table 2 provides a summary.

Table 2.

Baseline Vital Signs, Arterial Blood Gas Parameters, and Clinical Scores According to NIV Outcome

Variable	Failure (n = 54) mean ± SD	Success (n = 52) mean ± SD	p-value
Age (years)	73.4 ± 11.4	68.9 ± 10.8	0.04
SpO2 (%)	90.0 ± 6.4	90.0 ± 7.4	0.50
RR (/min)	28.8 ± 7.6	28.3 ± 8.7	0.51
pH	7.32 ± 0.10	7.32 ± 0.10	0.51
PaCO2 (mmHg)	55.9 ± 19.6	56.9 ± 17.8	0.72
PaO2/FiO2	218.1 ± 82.0	199.2 ± 72.2	0.14
	Median (IQR)	Median (IQR)
HACOR score	4 (5)	5 (4)	0.72

Values are presented as mean ± SD or median (IQR) as appropriate

When baseline laboratory parameters were compared according to NIV outcome, direct bilirubin, platelet count, lymphocyte count, and eosinophil count differed significantly between the groups (p = 0.01, 0.04, 0.01, and < 0.01, respectively). In contrast, no significant differences were observed in other biochemical or hematological parameters (all p > 0.05). Detailed comparisons of all laboratory parameters are provided in Supplementary Table 5.

At the 1-hour NIV assessment, oxygenation and ventilation parameters were significantly more favorable in the success group. SpO₂ levels and the PaO₂/FiO₂ ratio were higher in the success group, whereas RR, FiO₂ requirement, and PaCO₂ levels were lower. With respect to acid–base status, pH values were significantly higher in the success group (p < 0.01).

Regarding clinical scoring systems, HACOR₁ and NEWS₁ scores, as well as ΔHACOR, were significantly lower in the success group. Comparisons of 1-hour clinical and vital parameters are presented in Table 3 while Supplementary Table 6 provides detailed information.

Table 3.

1-hour clinical and scoring parameters according to NIV outcome

Variable	Failure (n = 54)	Success (n = 52)	p-value
SpO2 (%)	91.65 ± 5.15	93.35 ± 4.35	0.04
RR (/min)	26.43 ± 8.30	22.56 ± 6.96	0.01
pH	7.34 ± 0.09	7.39 ± 0.07	< 0.01
PaCO2	55.55 ± 17.26	47.33 ± 14.47	< 0.01
FiO2 (%)	41.41 ± 12.92	36.04 ± 8.94	0.01
PaO2/FiO2	191.53 ± 52.42	249.73 ± 109.51	< 0.01
HACOR	4 (2)	2 (4)	< 0.001
NEWS	7 (3)	5 (4)	0.03
ΔHACOR	0 (2)	-2.5 (4)	< 0.001

In the analysis stratified by ΔHACOR, NIV success was significantly more frequent among patients with ΔHACOR < 0. In contrast, the failure rate was markedly higher in patients with ΔHACOR ≥ 0. Specifically, 65.1% of patients with ΔHACOR < 0 achieved NIV success, whereas 74.4% of those with ΔHACOR ≥ 0 experienced NIV failure (Table 4).

Table 4.

Association between HACOR response status and NIV outcome

HACOR response status	NIV success n (%)	NIV failure n (%)	Total n (%)
No response (ΔHACOR ≥ 0)	11 (25.6)	32 (74.4)	43 (40.6)
Response (ΔHACOR < 0)	41 (65.1)	22 (34.9)	63 (59.4)
Total	52 (49.1)	54 (50.9)	106 (100)

In patients requiring intubation, both HACOR₀ and HACOR₁ scores were significantly higher (p = 0.001, p < 0.01 respectively). In contrast, ΔHACOR did not differ significantly between groups (p = 0.55). These results indicate that early HACOR scores may be useful in predicting the need for intubation, whereas changes in HACOR over time do not appear to have additional discriminative value (Supplementary Table 7).

In the ROC analysis, none of the baseline scores demonstrated significant discriminative ability for predicting NIV failure (all p > 0.05). In contrast, the HACOR₁ score (AUC = 0.760, 95% CI 0.667–0.854; p < 0.001) and ΔHACOR (AUC = 0.798, 95% CI 0.711–0.885; p < 0.001) showed significant predictive performance. The 1-hour NEWS score (NEWS₁) demonstrated lower discriminative ability (AUC = 0.627; p = 0.03) (Table 5). The 1-hour NEWS (NEWS2₁) and MEWS (MEWS₁) scores were also evaluated; however, neither demonstrated statistically significant predictive performance for NIV failure.

Table 5.

Predictive performance of clinical scores for NIV failure

Variable	AUC	Standard error (SE)	p-value	95% CI
HACOR1	0.760	0.048	< 0.001	0.667–0.854
NEWS1	0.627	0.056	0.03	0.518–0.736
ΔHACOR	0.798	0.044	< 0.001	0.711–0.885

In paired ROC–AUC comparisons, HACOR₁ provided significantly greater discriminative ability than NEWS₁ (p = 0.048), whereas no significant difference was observed between ΔHACOR and HACOR₁ (p = 0.52). A comparative illustration of the ROC curves is presented in Fig. 2, and detailed ROC and DeLong test results are reported in Supplementary Tables 8 and 9.

Fig. 2.

ROC curves of HACOR₁, NEWS₁ and ΔHACOR for predicting NIV failure

For HACOR₁, the highest Youden index was obtained at a cut-off value of 4 (Youden = 0.433), yielding a sensitivity of 83.7% and a specificity of 59.6%. At a higher cut-off value of 5, which provided greater specificity, sensitivity was 65.3%, specificity was 71.2%, the positive predictive value (PPV) was 68.1%, and the negative predictive value (NPV) was 68.5%.

In logistic regression analysis, ΔHACOR was significantly associated with NIV failure (OR = 1.63; 95% CI 1.31–2.03; p = 0.001). Each one-unit increase in ΔHACOR was associated with an approximately 1.6-fold increase in the odds of NIV failure (Table 6).

Table 6.

Coefficients of the logistic regression model for predicting NIV failure

Variable	Coefficient (β)	SE	z	p-value	OR	95% CI (Lower-upper)
Intercept	0.575	0.266	2.16	0.03	1.78	1.06–2.99
ΔHACOR	0.488	0.113	4.34	< 0.001	1.63	1.31–2.03

To evaluate the time-dependent effect of repeated measurements on NIV failure, a GEE model was constructed. In the base model, which included HACOR score, time, and the HACOR × time interaction, both time and HACOR, as well as their interaction term, were statistically significant (p < 0.05). These findings indicate that the effect of HACOR on NIV failure varies over time. GEE Model 1 (time-dependent GEE model with HACOR × time interaction) included HACOR score, time, and their interaction as the primary predictors. Age, pneumonia, RF type, and asthma were incorporated as covariates. Model 2 (time-dependent GEE model adjusted for laboratory parameters) extended Model 1 by adding direct bilirubin, platelet count, lymphocyte count, and eosinophil count, thereby adjusting for laboratory variables. Model 3 (time-dependent GEE model adjusted for physiological response variables) further expanded Model 1 by incorporating 1-hour SpO₂, RR, pH, PaCO₂, and PaO₂/FiO₂, reflecting early physiological response to NIV. This stepwise modeling approach allowed the time-dependent effect of HACOR to be evaluated across different clinical contexts.

In Model 2, the HACOR × time interaction remained statistically significant, and among the laboratory variables, only lymphocyte count made an independent contribution to the model (p = 0.01). In Model 3, the HACOR × time interaction retained strong statistical significance (p < 0.001). In model comparisons, Model 3—incorporating early physiological response variables—demonstrated the best overall fit, as reflected by the lowest QIC and QICC values (Table 7).

Table 7.

Comparison of GEE models according to QIC and QICC

Model	Model Components	QIC	QICC	Model Fit
Model 1	HACOR score, Time, HACOR × time, Age, Pneumonia, Asthma	280.1	279.0	Reference
Model 2	Model 1 + direct bilirubin, platelet count, lymphocyte count, eosinophil count	272.2	268.3	Improved fit
Model 3	Model 1 + 1-hour SpO2, RR, pH, PaCO2, PaO2/FiO2	272.0	265.0	Best fit

Lower QIC and QICC values are indicative of superior model fit

The graphical representation of the interaction is presented in Fig. 3.

Fig. 3.

Predicted probabilities of NIV failure derived from the GEE model for the HACOR score × time interaction

Detailed coefficient tables for Models 1, 2, and 3 from the GEE analysis—including Wald statistics, ORs, and 95% CI—are presented in Supplementary Tables 10, 11, and 12.

Discussion

In this heterogeneous population of ED patients receiving NIV for ARF, we demonstrated that early changes in the HACOR score during NIV treatment outperformed single time-point static measurements in predicting NIV failure. The principal finding of this study is that HACOR₁, and particularly ΔHACOR, demonstrated greater discriminative ability for NIV failure than baseline HACOR measurements. These findings suggest that the prognostic value of NIV assessment extends beyond baseline measurements alone and underscore the importance of early reassessment, as changes occurring during the first hour of treatment provide clinically relevant information regarding the risk of NIV failure.

Our study population represents a high-risk, real-world ED cohort characterized by advanced age, a slight predominance of female patients, a high burden of comorbidities, and a heterogeneous distribution of clinical diagnoses and RF types. The high rates of NIV failure and IMV requirement observed in this population further support its clinical vulnerability. Similar patient profiles and degrees of clinical heterogeneity have been reported in previous studies. Notably, the demographic and clinical characteristics described by Abulesz et al. [28] demonstrate striking parallels with those observed in our cohort.

The relatively high failure rate observed in this study may be explained by the fact that NIV failure was not defined solely by the need for endotracheal intubation or transition to IMV. In ED practice, NIV is frequently applied in patients for whom intubation may not be appropriate or has been designated as the ceiling of therapy, rendering an outcome definition based exclusively on escalation to IMV insufficient to fully capture treatment effectiveness. In the multicenter prospective study by Marjanovic et al. [5], NIV failure was defined not only by intubation but also by early clinical deterioration and the presence of predefined clinical criteria predictive of intubation, with an early deterioration rate of 41% reported. This broader definition provides a more comprehensive and clinically meaningful framework for evaluating NIV effectiveness in real-world ED settings and is methodologically consistent with our findings.

An important finding of our study was that assessment performed after the first hour of NIV provided greater prognostic discrimination than baseline assessment alone. Consistent with previous literature, baseline vital signs and arterial blood gas parameters have shown limited ability to discriminate NIV failure, whereas early changes observed at 1 h carry meaningful prognostic value [21, 26, 29, 30]. The early divergence in oxygenation and ventilation parameters—and their integration into composite clinical scores such as HACOR—suggests that the initial physiological response to treatment, rather than baseline severity alone, represents the key determinant of prognosis. In the multicenter study by Duan et al. [8], HACOR measurements at 1 and 2 h demonstrated greater discriminative ability compared with baseline values, findings that align closely with our results. Similarly, studies conducted in ED populations [11, 13, 31] have reported stronger predictive performance of early-hour scores relative to initial assessments.

In this context, the potential clinical relevance of ΔHACOR becomes apparent. In our study, the direction of score change provided prognostic information beyond that obtained from a single time-point assessment. Although the discriminative performance of ΔHACOR was numerically higher than that of the absolute HACOR₁ value, the difference was not statistically significant. Similar observations have been reported by Bai et al. [32], who found that change-based scoring approaches may offer additional prognostic information compared with static measurements. These findings suggest that a single assessment represents only a snapshot of physiological status, whereas serial evaluation may better reflect a patient’s response to treatment. Accordingly, the prognostic value of HACOR may derive not only from the absolute score itself but also from its ability to capture early physiological changes occurring during the initial phase of NIV.

An important methodological aspect of our study was the evaluation of the association between HACOR score and NIV failure over repeated measurements using GEE modeling. GEE provides an appropriate framework for analyzing correlated observations obtained from the same individuals over time while accounting for within-subject correlation. From a clinical perspective, this approach suggests that risk assessment during NIV may benefit from serial evaluation, as repeated measurements can provide prognostic information beyond that available from a single assessment alone.

To the best of our knowledge, previous studies evaluating the HACOR score have primarily focused on single or serial measurements without formally examining the interaction between HACOR and time using GEE methodology. In our study, the significant HACOR × time interaction observed across the GEE models suggests that the association between HACOR score and NIV failure may vary during the early phase of NIV treatment. These findings support the concept that early reassessment may provide clinically relevant prognostic information in addition to baseline evaluation.

Across all GEE models, the persistence of a significant HACOR × time interaction suggests that the association between HACOR score and NIV failure may change during the early phase of treatment. These findings support the value of serial assessment and indicate that prognostic information obtained after NIV initiation may complement baseline evaluation.

The ED represents a clinical environment in which patients may deteriorate rapidly or demonstrate marked improvement within short time intervals, making rapid and sequential reassessments critically important. Accordingly, risk stratification in patients receiving NIV should not be based solely on baseline physiological status, but rather on the trajectory of the early treatment response.

The significant HACOR × time interaction observed in our study indicates that identical absolute score values may carry different prognostic implications depending on the time point at which they are measured. A score obtained at baseline, when interpreted without consideration of the early treatment course, may incompletely reflect true clinical risk. These findings underscore the central role of reassessment in ED-based NIV management and support the notion that risk should be interpreted within a temporal context.

From a clinical perspective, our results emphasize the importance of early reassessment during NIV therapy. Recalculation of the HACOR score within the first hour and comparison with the baseline value may facilitate early identification of high-risk patients. Lack of improvement or an increase in the score may indicate insufficient physiological response to treatment and may warrant closer clinical attention. As a bedside-applicable and easily implementable tool, this dynamic evaluation approach may complement clinical decision-making and support risk stratification during the early phase of NIV management.

Limitations

This study has several limitations that warrant consideration. Firstly, it was conducted at a single center, which may limit the generalizability of our findings to other emergency department settings with different patient populations or clinical practices. Secondly, while our sample size calculation indicated statistical sufficiency, a larger, multicenter study would further strengthen the external validity of our results. Thirdly, the study did not evaluate the impact of factors such as the experience of the healthcare provider applying NIV or variations in specific device/mask types, which could potentially influence NIV outcomes. Fourth, our primary outcome focused on early NIV failure and intubation; future research could explore longer-term outcomes such as hospital length of stay, ICU admission rates, and 28-day mortality beyond the immediate post-NIV period. Fifth, only baseline NIV settings were recorded, and subsequent changes in ventilator parameters were not systematically documented, limiting our ability to assess the potential impact of NIV management strategies on outcomes. Finally, a potential limitation of this study is the risk of incorporation bias, as certain components of the HACOR score may overlap with parameters used in defining NIV failure; however, this risk was minimized by adopting a broader clinical definition of failure and by performing additional analyses focusing on endotracheal intubation as a more objective outcome.

Strengths

This study has several important strengths. First, the prospective design allowed data to be collected at predefined time points within a standardized protocol, enabling reliable assessment of early physiological changes and strengthening the internal validity of the study. Second, rather than relying solely on the absolute value of the HACOR score, we evaluated its early dynamic change. By analyzing both the direction and magnitude of score variation, this approach provided a dynamic risk assessment framework that more meaningfully discriminated NIV failure compared with static, single time-point measurements. From a methodological perspective, a key distinguishing feature of this study is the formal assessment of the prognostic value of serial HACOR measurements using GEE modeling.

Conclusion

This study demonstrates that NIV failure is determined not by baseline static measurements, but by the early physiological response to treatment. The superior predictive performance of the HACOR₁ score and ΔHACOR compared with the baseline score, together with the time-dependent effect confirmed through GEE modeling, supports the adoption of a dynamic, time-sensitive risk assessment approach for NIV management in EDs.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

NIV

Non–Invasive Ventilation

ARF

Acute Respiratory Failure

COPD

Chronic Obstructive Pulmonary Disease

ICU

Intensive Care Unit

IMV

Invasive Mechanical Ventilation

ED

Emergency Department

HR

Heart Rate

GCS

Glasgow Coma Scale

PaO₂

Arterial Partial Pressure of Oxygen

FiO₂

Fraction of Inspired Oxygen

RR

Respiratory Rate

HACOR × time

The Time Varying Effect of HACOR

HACOR₀

0–hour HACOR score

HACOR₁

1–hour HACOR score

ΔHACOR

HACOR score (1–hour) − HACOR score (0–hour)

GEE

Generalized Estimating Equations

ERS/ATS

European Respiratory Society / American Thoracic Society

LOC

Loss of consciousness

SD

Standard Deviation

CRF

Case Report Form

BiPAP

Bilevel Positive Airway Pressure

CPAP

Continuous Positive Airway Pressure

BP

Blood Pressure

SpO₂

Peripheral Oxygen Saturation

CRP

C–Reactive Protein

PaCO₂

Partial Pressure of Arterial Carbon Dioxide

SaO₂

Arterial Oxygen Saturation

NEWS

National Early Warning Score

MEWS

Modified Early Warning Score

SBP

Systolic Blood Pressure

IPAP

Inspiratory Positive Airway Pressure

EPAP

Expiratory Positive Airway Pressure

PEEP

Positive End–Expiratory Pressure

IQR

Interquartile Range

ROC

Receiver Operating Characteristic

AUC

Area Under the Curve

CI

Confidence Interval

OR

Odds Ratio

QIC

Quasi–likelihood under the Independence Model Criterion

QICC

Corrected Quasi–likelihood under the Independence Model Criterion

NEWS₁

1–hour NEWS score

NEWS2₁

1–hour NEWS2 score

MEWS₁

1–hour MEWS score

SE

Standard Error

PPV

Positive Predictive Value

NPV

Negative Predictive Value

Author contributions

Concept / Hypothesis: BD Design: BD, ABE Ethics Committee Approval Preparation: BD, BK, EÇ, ABE Data Collection: BD, BK, EÇ, AK Data Entry: BD, AK Data Analysis: MY Manuscript Preparation: BD, MY Manuscript Review and Editing: BD, MY, BK, EÇ, AK, ABE.

Funding

The authors received no financial support for the research, authorship, or publication of this article.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics committee approval

Ethical approval was obtained from the Non-Interventional Clinical Research Ethics Committee of Ankara Etlik City Hospital (AEŞH-BADEK-2024-909, 10/02/2024).

Consent to participate

Written informed consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.