Usefulness of the HACOR score in predicting success of CPAP in COVID-19-related hypoxemia

Abstract

Introduction

In COVID-19 associated hypoxemic acute respiratory failure (ARF) without mandatory indication for urgent endotracheal intubation, a trial of CPAP may be considered. We aimed to evaluate HACOR (heart rate, acidosis, consciousness, oxygenation, respiratory rate) score performance in these patients as predictor of CPAP failure.

Methods

Prospective observational multicentric study (three centers in different countries), including adult patients with SARS-CoV-2 pneumonia admitted to a respiratory intermediate care unit, presenting PaO₂/FiO₂ < 300 and PaCO₂ < 45 mmHg, who received CPAP. One hour after starting CPAP, HACOR was calculated.

Results

We enrolled 128 patients, mean age 61,7 years. Mean HACOR at 1 h after starting CPAP was 3,27 ± 3,84 and mean PaO₂/FiO₂ was 203,30 ± 92,21 mmHg; 35 patients (27,3 %) presented CPAP failure: 29 underwent oro-tracheal intubation and 6 died due to COVID-19 (all having a do-not-intubate order). HACOR accuracy for predicting CPAP failure was 82,03 %, while PaO₂/FiO2 accuracy was 81,25 %.

Conclusion

Although HACOR score had a good diagnostic performance in predicting CPAP failure in COVID-19-related ARF, PaO₂/FiO₂ has also shown to be a good predictor of failure.

1. Introduction

Hypoxemic acute respiratory failure is a life-threatening complication of COVID-19 infection. In this subset of patients, respiratory support [1] and admission to intensive care is frequently required. In the absence of mandatory indication for urgent endotracheal intubation, a cautious trial of 1–2 h of non-invasive continuous positive airway pressure (CPAP) may be acceptable as it could avoid intubation and its associated complications in about two thirds of cases [2]. However, close monitoring to early detect CPAP failure and need for escalation is strongly recommended [3]. Furthermore, CPAP use may be justifiable if medical resources become overloaded and without sufficient ability to provide invasive ventilation [3], as well as in patients with shared decision of limitation of maximized care (i.e do-not-intubate, CPAP as treatment ceiling). This early non-invasive ventilator strategy is of relevance to avoid collapse of ICU availability [4], saving beds for intubated and/or multiorgan dysfunction patients.

Several publications have highlighted the potential role of NIV in approaching hypoxemic acute respiratory failure (ARF) secondary to SARS-CoV-2 pneumonia [5,6]. One of the most used non-invasive respiratory support strategies is CPAP, especially if delivered by helmet [7,8]. CPAP allows increase of functional residual capacity and improving ventilation/perfusion matching (by re-inflating collapsed alveoli) [9]. CPAP application is also easier to be quickly set up in term of nursing time consumption as compared to NIV. Previous reports in the beginning of the pandemic showed high failure rates of non-invasive respiratory support [1,10], and concerns about risk of aerosol dissemination and virus transmission led to early avoidance of its use. However, there is increasing evidence of its feasibility outside the ICU [2,11] and of its safety if appropriately applied with aerosol-containing measures [7,12,13].

Since there is a concern of an increased mortality associated with delayed intubation after CPAP failure, it is crucial to quickly identify adequate timing for when to switch from non-invasive to invasive ventilatory support [14]. Duan et al. [15] have proposed a bedside obtained scale for prediction of non-invasive ventilation (NIV) failure in hypoxemia due to several causes. The scale was named HACOR and comprised Heart rate, Acidosis, Consciousness level, Oxygenation and Respiratory rate. The highest possible score is 25 points. At 1 h of NIV, a cutoff score >5 showed a diagnostic accuracy of 81,8 %.

The aim of this study is to evaluate HACOR score performance in predicting CPAP failure in COVID-19 associated ARF.

2. Methods

This is a prospective observational study conducted in three hospitals: Hospital Prof. Doutor Fernando Fonseca (Portugal), Monaldi Hospital (Italy) and Lenox Hill Hospital (United States of America). Patients were admitted to level II respiratory intermediate care units (RICU), specially dedicated to managing ARF with NIV/CPAP, and presented differentiated medical and nursing teams, with a nurse-to-patient ratio up to 1:4. All teams have a longtime expertise in managing critically ill respiratory patients. Patients on invasive ventilation through endo-tracheal tube were not admitted to the mentioned level II units. The study was approved by the Institutions' Ethical Committees of all participating centers.

The study included all consecutive adult patients admitted in RICU for hypoxemic ARF due to SARS-CoV-2 pneumonia (viral identification by PCR on nasopharyngeal sample or inferior respiratory tract sample), who received CPAP support, between April and June 2020.

The inclusion criteria were: PaO₂/FiO₂ < 300 mmHg (and PaCO₂ < 45 mmHg) on room air/low FiO₂ conventional oxygen therapy (Venturi mask with FiO₂ up to 28 %). Patients were excluded if they presented any of the following: imminent cardio-respiratory arrest, incapability of protecting airways, severe hemodynamical instability (medium arterial pressure < 65 mmHg despite vasopressor support), uncontrolled agitation (Richmond agitation-sedation scale [RASS] score > 2) despite mild analgosedation, multi-organ failure, or any other formal indication for invasive mechanical ventilation. Patients who started CPAP as escalation therapy after failure of high-flow nasal cannula oxygen therapy (HFNC) were also excluded. CPAP was chosen over HFNC as it allowed higher and more controllable PEEP levels. Patients who underwent NIV (mainly due to hypercapnic respiratory failure) were also excluded, as were patients on NIV as escalation after CPAP failure.

Patients meeting the inclusion criteria were admitted to the study and started CPAP therapy. CPAP was initiated at 8 cmH2O and titrated, according to patient comfort, to improve oxygenation and respiratory pattern, FiO₂ titrated maintain SpO₂ >94 %. CPAP was applied continuously according to patient's tolerance. CPAP weaning started according to a strategy of reduction of FiO2 and pressure, until a setting of CPAP = 5 cmH2O with FiO2 < 30 % was achieved; weaning from CPAP was attempted deescalating oxygenation therapy to HFNC or conventional oxygen therapy. The devices were ventilators on CPAP mode (most commonly utilized devices were Monnal T60® by Air Liquide Medical Systems™, followed by V60® by Phillips Respironics™, USA, and, France), usually with an oro-nasal mask. Viral filters were used in all cases.

Prone positioning, if used, was applied with a schedule of 2–3 sessions/day with a length of 2–3 h/session according to patient tolerance. Proning was only applied after the HACOR evaluation.

Medical therapy included prophylactic enoxaparin, unless there was formal indication for anticoagulation dose) and methylprednisolone, the second one being prescribed to patients who continued with significant respiratory failure after the seventh day of symptoms. None of the patients received remdesivir or other antiviral drugs.

2.1. Variables analysed

Sociodemographic, clinical and analytical variables were collected on admission and during hospital stay and analysed, including age, gender, relevant comorbidities (also quantified using the Charlson index), heart rate, respiratory rate, consciousness level (using Glasgow Coma Scale), blood gas analysis. Data relative to CPAP methodology, use of prone positioning and sedative/analgesic use were also recorded.

One hour after starting CPAP, HACOR score was calculated for each patient. Patients were then analysed according to CPAP outcome: success or failure (until discharge). CPAP failure was defined as the need for orotracheal intubation, or death in the case of patients with a previous do-not-intubate order. In cases not responsive to CPAP, patients were intubated straightway.

Criteria for endotracheal intubation were (at least one): sustained hemodynamic instability (medium arterial pressure < 65 mmHg despite vasopressor support), deterioration of consciousness level (GCS < 9), respiratory rate > 40 breaths per minute, signs of respiratory exhaustion, PaO₂/FiO₂ persistently < 150 for more than 48 h under CPAP, agitation (Richmond agitation-sedation scale [RASS] score > 2) or CPAP intolerance despite mild analgosedation and optimized interface. Apart these objective criteria, clinical decision to intubate was at the discretion of the attending team.

2.2. Statistical analysis

Qualitative variables were shown as absolute and relative frequencies. Quantitative variables were expressed as mean ± standard deviation or as median and interquartile range, as appropriate.

In order to assess the role played by HACOR in predicting CPAP failure, two sets of regressions were performed. The first set includes the HACOR (at 1 h after starting CPAP) as the key predictive variable, after controlling for the Charlson index, gender, and prone position. The second one includes PaO₂/FiO₂ 1 h after starting CPAP. After that, we calculated both HACOR and PaO₂/FiO₂ accuracy and performed ROC analysis for both in order to avoid collinearity issues. In this analysis, we considered both HACOR and PaO₂/FiO₂ as categorical variables, using a cut-off of 5 for HACOR and 150 for PaO₂/FiO₂. We used a significance level of 5 %. The analysis was performed using Stata 13® program (StataCorp LLC™, Texas, USA).

3. Results

A total of 184 patients were considered to enter the study protocol; 45 were excluded due to bilevel NIV use instead of CPAP. Also 11 on CPAP were excluded to the impossibility to collect blood gas analysis on time to calculate 1-h HACOR score. Consequently, we enrolled 128 patients from the three different centers. A flowchart is presented in Fig. 1 .

Fig. 1.

Flowchart of patients enrollment.

The main clinical and sociodemographic characteristics are shown in Table 1 . The majority was male, and the mean age was 61,7 ± 12,37 years. More than two thirds had no smoking history. The main identified comorbidities were hypertension, diabetes and obesity. The mean PaO₂/FiO₂ before starting CPAP was 146.89 ± 69.36.

Table 1.

Socio-demograhic, clinical and ventilatory characteristics; COPD: chronic obstructive pulmonary disease; bpm: breaths per minute; hbpm: heart beats per minute. Results are presented as absolute value and (percentage), or as means ± standard deviation.

Characteristics	Value
Gender	99 (77 %)
Male	29 (23 %)
Female
Age	61.73 ± 12.37
Smoking status	88 (69 %)
Never	38 (30 %)
Former	2 (1 %)
Active
Charlson comorbidity index	2.63 ± 1.97
Main comorbidities	72 (56 %)
Arterial hypertension	26 (20 %)
Diabetes	22 (17 %)
Obesity	11 (9 %)
Malignancy	9 (7 %)
COPD
Initial clinical variables	86.31 ± 10.81
Heat rate (hbpm) pH	7.46 ± .05
Glasgow coma scale	14.42 ± 1.42
PaO2/FiO2	146.89 ± 69.36
Respiratory rate (bpm)	26.78 ± 4.62
HACOR (before CPAP starting)	4,67 ± 2,74
CPAP level (cmH2O)	10.08 ± 2.48
FiO2	53.81 ± 23.62
Type of circuit	32 (25 %)
Single branch	96 (75 %)
Double branch
Interface	80 (63 %)
Oro-nasal	16 (12 %)
Total facial mask	31 (24 %)
Helmet	1 (8 %)
PEEP-mask
Prone position	54 (42 %)
Sedative/analgesic medication	13 (10 %)
HACOR (1 h after CPAP starting)	3.27 ± 3.84
Heart rate (hbpm) pH	79.56 ± 12.21
Glasgow coma scale	7.45 ± .05
PaO2/FiO2	14.42 ± 1.48
Respiratory rate (bpm)	203.30 ± 92.21
	23.59 ± 4.16

3.1. CPAP characteristics

CPAP was applied with a mean pressure level of 10,08 ± 2,48 cmH₂O and mean FiO₂ of 53,81 ± 23,62, most commonly with a double-branch circuit (76 %) and using an oro-nasal mask (66 %). Prone position was applied in almost half of the patients and only a minority of patients were under sedative medication to improve CPAP tolerance.

3.2. HACOR at 1 h and clinical outcome

Mean HACOR at 1 h after starting CPAP was 3,27 ± 3,84 and mean PaO₂/FiO₂ was 203,30 ± 92,21 mmHg. In total, 35 patients (27,3 %) presented CPAP failure: 29 (24,2 %) underwent oro-tracheal intubation and 6 (4,7 %) died due to COVID-19 (all having a do-not-intubate order). The most common cause of CPAP failure was inability to correct hypoxemia, followed by progression to respiratory exhaustion. Overall hospital mortality rate was 23 % (29 patients, 23 of them directly due to COVID-19).

3.3. HACOR as a predictor of CPAP failure

When comparing demographic and initial clinical characteristics between patients with CPAP failure and those who presented CPAP success (Table 2 ), patients with CPAP failure were significantly older and had significantly lower PaO₂/FiO₂ levels. On the other hand, 1 h after starting CPAP, there was a significant improvement in heart rate, pH, PaO₂/FiO₂ and respiratory rate (Table 3 ).

Table 2.

Demographic and initial clinical characteristics of patients who presented CPAP success, compared to those who presented CPAP failure; bpm: breaths per minute; hbpm: heart beats per minute. Results are presented and absolute value and (percentage), or as means ± standard deviation.

Variables	CPAP Success	CPAP Failure	p-value
Age	60,31 ± 12,25	65,51 ± 12,05	0,03569**
Gender			0,97343
Male	72 (77 %)	27 (77 %)
Female	21 (23 %)	8 (23 %)
Charlson comorbidity index	2,46 ± 1,98	3,09 ± 1,91	0,05393*
Initial clinical variables	84,99 ± 9,67	89,83 ± 12,87	0,12891
Heat rate (hbpm) pH	7,47 ± 0,03	7,45 ± 0,05	0,06765*
Glasgow coma scale	14,50 ± 1,32	14,20 ± 1,66	0,07557*
PaO2/FiO2	159,09 ± 68,74	114,49 ± 60,77	0,00035***
Respiratory rate (bpm)	26,56 ± 4,26	27,37 ± 5,49	0,26505
HACOR before starting CPAP	4,05 ± 3,43	6,26 ± 3,94	0,00094***

***p < 0.01, **p < 0.05, *p < 0.1.

Table 3.

Comparison between HACOR variables before and hour after starting CPAP. bpm: breaths per minute; hbpm: heart beats per minute. Results are presented as means ± standard deviation.

Variables	Before starting CPAP	1h after starting CPAP	p-value
HACOR	4,67 ± 2,74	3.27 ± 3.84	<0,0001***
Heart rate (hbpm) pH	86.31 ± 10.81	79.56 ± 12.21	<0,0001***
Glasgow coma scale	7.46 ± .05	7.45 ± .05	0,00003***
PaO2/FiO2	14.42 ± 1.42	14.42 ± 1.48	0,64157
Respiratory rate (bpm)	146.89 ± 69.36	203.30 ± 92.21	<0,0001***
	26.78 ± 4.62	23.59 ± 4.16	<0,0001***

***p < 0.01, **p < 0.05, *p < 0.1.

The first regression (Table 4 ), depicted in the second column, includes the HACOR as the key predictive variable, after controlling for the Charlson index, gender, and prone position. HACOR assumes a statistically significant role in determining the probability of CPAP failure. The second regression (depicted in the third column) includes PaO₂/FiO₂, which also plays a significant role in determining CPAP failure.

Table 4.

CPAP outcome logistic regression. First regression including HACOR, Charlson index, sex and prone position. Second regression including PaO₂/FiO₂.

Variables	First regression	Second regression
HACOR	13.011 p = 0.00000***
Charlson index	1.015 p = 0.906
Sex	1.138 p = 0.818
Prone position	0.254 p = 0.013**
PaO2/FiO2		0.984 p = 0.00000***
Constant	0.244 p = 0.011**	6.610 p = 0.0005***

***p < 0.01, **p < 0.05, *p < 0.1.

In this sample, 32 (25 %) patients depicted an HACOR score greater than 5, and 22 of them had CPAP failure, resulting in a positive predictive value of 68,75 %. On the other hand, 96 (75 %) patients depicted an HACOR score below or equal to 5, and 83 of them had CPAP success, resulting in negative predictive value of 86,46 %. Sensitivity and specificity were 62,86 % and 89,25 % respectively. HACOR accuracy was 82,03 %. On the other hand, calculated accuracy for PaO₂/FiO₂ was 81,25 %, with positive predictive value of 65,71 %, negative predictive value of 87,10 %, sensitivity of 65,71 % and specificity of 87,10 %. ROC analysis comparing HACOR (>5) and PaO₂/FiO₂ (<150) is presented in Fig. 2 .

Fig. 2.

ROC analysis comparing HACOR and PaO₂/FiO₂ to predict CPAP failure.

Model 1/Black line: HACOR, considering > 5 as the cut-off

Model 2/Blue line: PaO₂/FiO₂, considering <150 as the cut-off. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

4. Discussion

Our results suggest that HACOR may be a reliable score in predicting CPAP failure in COVID-19 patients. The statistical analysis suggests that PaO₂/FiO₂ level at 1 h after starting CPAP might be a key determinant, almost as important as the HACOR score itself in accessing the probability of CPAP failure. In fact, the ROC analysis demonstrates very similar accuracy levels for both HACOR and PaO₂/FiO₂.

Currently, there are few randomized studies addressing effectiveness of non-invasive respiratory support strategies in hypoxemic ARF related to viral infections, namely SARS-CoV-2 [9]. Nevertheless, non-invasive respiratory support has been used in COVID-19-associated AR, with average success rates above 50 % [4]. In order to avoid the risk of potentially delaying oro-tracheal intubation [4,9], tools to early assess the efficacy of non-invasive respiratory support are needed.

In the study by Duan et al. a HACOR score >5 at 1 h showed a sensitivity of 72,6 %, specificity of 90,2 %, positive predictive value of 87,2 %, negative predictive value of 78,1 % and 81,8 % accuracy [15]. In their cohort, patients with NIV failure and HACOR score >5 at 1 h of NIV who had been intubated with less than 12 h on NIV showed lower hospital mortality than those intubated later [15].

Recently Carrillo et al. [16] conducted a study to analyze the validity of the HACOR score, retrospectively enrolling 2711 patients in 2749 episodes of hypoxemic ARF requiring NIV. They confirmed the accuracy of the HACOR score, particularly in pneumonia and ARDS. Innocenti et al. [17] also demonstrated the usefulness of HACOR in identifying patients with ARF treated with NIV who are at risk of in-hospital mortality.

Comparing with the HACOR study, we found a sensitivity of 62,86 % (vs. 72,6 %), specificity of 89,25 % (vs. 90,2 %), positive predictive value of 68,8 % (vs. 87,2 %), negative predictive value of 86,5 % (vs. 78,1 %) and an accuracy of 82,03 % (vs. 81,8 %). These differences can be explained by the lower weight that PaO₂/FiO₂ has in the HACOR score, in comparison with other variables such as the score on the Glasgow coma scale. It appears that in COVID-19 patients, PaO₂/FiO₂ appears to be of particularly high importance compared to what is seen in multiple other causes of hypoxemic respiratory failure, such as those included in the original HACOR study. In fact, in the original HACOR study, multiple etiologies are included, including bacterial pneumonia, pulmonary cancer, pulmonary embolism and heart failure, with mechanisms of respiratory failure not always overlapping those of SARS-CoV-2 pneumonia.

We hypothesize that, in COVID-19 patients, it would be necessary to assign a higher score to each range of PaO₂/FiO₂ values, in order to obtain an even more realistic assessment of the risk of CPAP failure. If lower values of PaO₂/FiO₂ received higher scores, their preponderance in HACOR would increase, which would be in accordance with the high prevalence of significant hypoxemia in COVID-19 and the importance of its correction with CPAP as a predictor of CPAP success.

As COVID-19 pneumonia has specific peculiarities, with a combined damage induced by direct viral cytopathic effect and by the indirect effects of the cytokine storm [18], it could be argued that HACOR could be adapted to become an even more accurate prognostic tool for CPAP-treated COVID-19 patients, which would require a specific and directed study. Furthermore, HACOR score was not previously tested for hypoxemic patients managed by means of CPAP. However, it is a simple, bed-side method [15], easy to implement and that could add valuable information, in lack of a validated tool.

The results of the present study suggest that patients with rapid worsening of PaO₂/FiO₂ and without significant improvement with CPAP will need orotracheal intubation. This underlines that early assessment of variations in gas exchange is essential to avoid delaying necessary oro-tracheal intubation [4,9].

As mentioned before, CPAP allows increase of functional residual capacity and improving ventilation/perfusion matching [9]. In light of this, the lack of improvement in PaO₂/FiO₂ after 1 h of CPAP seems to indicate that this therapy is not providing adequate physiological effects in order to reverse the respiratory failure, so a more invasive form of respiratory support may be necessary.

The results also suggest that prone position may have a favorable effect in CPAP outcomes, although this is not the main focus of the present study, since proning was only applied after 1 h on CPAP therapy.

The fact that we included patients with PaO₂/FiO₂ < 300 mmHg may have contributed to high CPAP success rate, although the mean PaO₂/FiO₂ before starting CPAP was usually lower than 200 mmHg (146.89 ± 69.36). Part of this high success rate may be explained by early initiation of CPAP support and easy availability of the technique, in which the teams were very experienced. In comparison with Aliberti et al. study [19], patients with CPAP success in our study had a higher initial PaO₂/FiO₂ (159,09 ± 68,74 vs. 136 [95,0–204,8]) and a slightly lower respiratory rate (26,56 ± 4,26 vs. 28 [24–32] breaths per minute). On the other hand, in our study patients with CPAP failure had lower PaO₂/FiO₂ (114,49 ± 60,77 vs. 152 [100–202] and higher respiratory rate (27,37 ± 5,49 vs. 25,5 [21–30] breaths per minute). Furthermore, an Italian observational multicentre study [2] involving a large series of COVID-19 patients with similar degree of hypoxemia (PaO₂/FiO₂ 152) reported a rate of CPAP failure of 25 %, not different from our findings.

Strengths of this study include the fact that it was conducted in three different countries in two different continents, which promotes the reproducibility of the results.

The study presents some limitations. We only used CPAP and not NIV in bilevel mode, meaning that results cannot be fully compared with the results obtained in the original HACOR study. Another important bias is the fact that in our study we only used CPAP applied through ventilators (mostly using oronasal mask), which may be less effective than that applied through high-flow systems using Helmet as interface. Also, the study was performed in a pandemic associated high-stress context, which may affect monitoring capacity of the patients’ clinical status. Another caveat is the lack of information on in-hospital total mortality, as well as 30 and 90 days-mortality, although the main purpose of the present study was to evaluate HACOR score performance in predicting early CPAP support failure. The fact that we did not include the time of intubation after starting CPAP is another caveat, since this limitation does not allow us to differentiate HACOR predictive power in detecting early versus late CPAP failure. Also, as it happens with other published papers on respiratory support in COVID-19, there is a lack of a control arm.

The fact that we compared HACOR and PaO₂/FiO2 considering them as categorical variables may constitute itself a risk of bias, as we are choosing a cut-off for a continuous variable. We would also like to stress out that many of the parameters of the HACOR (like Glasgow Coma Scale, PaO2/FiO2 and respiratory rate) were also criteria for intubation and invasive mechanical ventilation, which constitutes another caveat. In the future we need works to study HACOR accuracy in different ranges of the mentioned variables.

It would be interesting, in the future, to carry out randomized trials, comparing the effectiveness of NIV, CPAP, Helmet-CPAP, HFNC and conventional oxygen therapy (Venturi mask), as well as predictive factors of failure of each respiratory support technique, in order to be able to establish more adequate success/failure predictive scales.

5. Conclusions

In conclusion, we found that although HACOR score had a good diagnostic performance in predicting CPAP failure in COVID-19-related ARF, PaO₂/FiO₂ was also shown to be a good predictor of failure.

Clinical trial registration number

This study is registered in ClinicalTrials.gov: Unique Protocol ID: 08-04-2020 (https://clinicaltrials.gov/).

Financial support

None.

CRediT authorship contribution statement

Miguel Filipe Guia: Conceptualization, Methodology, Investigation, Writing – original draft, Writing – review & editing. José Pedro Boléo-Tomé: Methodology, Investigation, Writing – original draft, Writing – review & editing. Pasquale Imitazione: Investigation, Writing – review & editing. Giorgio Emanuele Polistina: Investigation, Writing – review & editing. Carlos Alves: Investigation, Writing – review & editing. Oki Ishikawa: Investigation, Writing – review & editing. Matthew Ballenberger: Investigation, Writing – review & editing. Bushra Mina: Investigation, Writing – review & editing. Giuseppe Fiorentino: Investigation, Writing – review & editing. Antonio Esquinas: Methodology, Writing – review & editing. Raffaele Scala: Methodology, Writing – original draft, Writing – review & editing.

Declaration of competing interest

None.