Stratifying risk outcomes among adult COVID-19 inpatients with high flow oxygen: The R4 score

Abstract

Background

High flow oxygen therapy (HFO) is a widely used intervention for pulmonary complications. Amid the coronavirus infectious disease 2019 (COVID-19) pandemic, HFO became a popular alternative to conventional oxygen supplementation therapies. Risk stratification tools have been repurposed –and new ones developed– to estimate outcome risks among COVID-19 patients. This study aims to provide a simple risk stratification system to predict invasive mechanical ventilation (IMV) or death among COVID-19 inpatients on HFO.

Methods

Among 529 adult inpatients with COVID-19 pneumonia, we selected unadjusted clinical risk factors for developing the composite endpoint of IMV or death. The risk for the primary outcome by each category was estimated using a Cox proportional hazards model. Bootstrapping was used to validate the results.

Results

Age above 62, eGFR under 60 ml/min, room air SpO2 ≤89 % upon admission, history of hypertension, history of diabetes, and any comorbidity (cancer, cardiovascular disease, COPD/ asthma, hypothyroidism, or autoimmune disease) were considered for the score. Each of the six criteria scored 1 point. The score was further simplified into 4 categories: 1) 0 criteria, 2) 1 criterion, 3) 2-3 criteria, and 4) ≥4 criteria. Taking the first category as the reference, risk estimates for the primary endpoint were HR; 2.94 [1.67 – 5.26], 4.08 [2.63 – 7.05], and 6.63 [3.74 – 11.77], respectively. In ROC analysis, the AUC for the model was 0.72.

Conclusions

Our score uses simple criteria to estimate the risk for IMV or death among COVID-19 inpatients with HFO. Higher category reflects consistent increases in risk for the endpoint.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of the 2019 novel coronavirus disease (COVID-19) has overwhelmed entire health systems across the world.1, 2, 3 An estimated 67% of patients with severe COVID-19 develop acute respiratory distress syndrome and almost 20% require ICU admission.^{4 , 5} In Mexico, at the beginning of the surge, 46% of patients categorized as critically ill did not receive invasive mechanical ventilation (IMV). Similar estimates were seen in the United States, both because of a lack of ICU beds availability and mechanical ventilators.³ High flow oxygen therapy (HFO) has become a safe⁶ and effective⁷ respiratory support alternative in patients with acute respiratory failure. However, its benefit in mortality is controversial.^{8 , 9} In the setting of the current pandemic, HFO has been recommended over the use of non-invasive ventilation NIV,^{10 , 11} despite limited evidence regarding its benefit in improving outcomes. Nonetheless, with a global shortage of mechanical ventilators and access to specialized care, HFO may be a useful alternative. Risk stratification for incident IMV or death among patients on HFO is unclear and may be population dependent. Traditional and novel scoring tools have been utilized to provide risk estimates among COVID-19 patients. While some of them have good prognostic value, others are non-specific or employ many variables. This study aims to provide new scoring criteria to estimate the risk for IMV or death on HFO among COVID-19 inpatients receiving HFO.

Methods

A retrospective analysis was conducted in a tertiary care hospital redesigned to treat COVID-19 patients (Hospital San José– TecSalud) in Monterrey, Mexico from April to October 2020. Demographics, clinical and laboratory information was collected in a deidentified database. Approval from the TecSalud Ethics Committee was obtained (P000353-COVID-19-TecSalud-CS001).

Study population

We included hospitalized patients over 18 years of age, with positive Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) for SARS-CoV-2 and oxygen requirements that demanded HFO. Criteria for initiation of HFO were oxygen saturation lower than 92% with conventional oxygen therapy (COT) (reservoir mask at 15 L/min), and tachypnea (> 30 breaths per minute) and/or self-reported rest dyspnea despite COT. All patients received a protocolized treatment which consisted of dexamethasone (6 mg IV, QD) plus baricitinib (4mg PO, QD) for 10 days and those with interleukin-6 (IL-6) higher than 80 pg/mL at third day of hospital stay and C-reactive protein higher than 7.5 mg/dL without respiratory improvement received tocilizumab (8mg/kg/dose IV, BID).

Variables and score implementation

We analyzed demographic characteristics that included: age, sex, body mass index (BMI), estimated glomerular filtration rate (eGFR), smoking status and comorbidities (type 2 diabetes, hypertension, chronic kidney disease and cardiac disease, COPD/Asthma, cancer, cardiovascular disease (CVD), and autoimmune conditions). eGFR was calculated using serum creatinine, with the CKD-EPI formula. All comorbidities were recorded by self-reported or by a proxy if patients were not able to provide their own history. Clinical characteristics included: number of days with symptoms prior to admission, oxygen saturation at room air upon admission, SAFI (saturation/fraction of inspired oxygen index), the CALL score, a COVID-19-specific score to predict disease progression that includes comorbidity, age, lymphocytes and lactate dehydrogenase (LDH),¹² severity scores such as Sequential Organ Failure Assessment (SOFA),¹³ Pneumonia Severity Index (PSI),¹⁴ CURB-65,¹⁵ National Early Warning Score 2 (NEWS 2)^{16 , 17}; days with HFO, length of hospital stay in days, ICU admission and length of stay in ICU, number of days since admission until HFO initiation. Respiratory parameters such as oxygen saturation, respiratory rate per minute, fraction of inspired oxygen (FiO₂) and the Respiratory rate and Oxygenation (ROX)¹⁸ index were recorded upon admission, at the time of HFO, and 24 hours post HFO initiation. Superimposed bacterial infections were also documented at any time during hospitalization.

Admission laboratory tests that were analyzed included: complete blood count, LDH, C-reactive protein (CRP), procalcitonin, IL-6, D- dimer, brain natriuretic peptide (BNP), ferritin, and highly sensitive (HS) troponin.

Statistical analysis

Stata IC-16 was used to conduct statistical analyses. For categorical variables, frequencies and percentages are shown; for continuous variables, according to normal or non-normal distribution, mean and standard deviation or median and interquartile range (IQR), respectively, are shown. Chi-squared and t-test or U Mann-Whitney were used for comparisons between both groups. No imputation methods were utilized for data missingness. An alpha of 5% was set as threshold for statistical significance. The primary endpoint was IMV, or death while on HFO.

Demographic and clinical covariates associated with the endpoint were considered to construct the score. Cox proportional hazard models were employed to estimate the risk for the primary outcome by each category of the score. The proportional hazards assumption was tested. Kaplan-Meier (KM) estimates were used to display cumulative incidence of the primary endpoint, where time zero is the day the patient was placed on HFO. Results from the models are expressed as hazard ratios (HR) and 95% confidence intervals (CI). Receiver operating characteristic (ROC) curves were utilized to assess the performance of our score compared to other utilized scores. Area under the curve (AUC) for different scores were compared using DeLong's test. A classic 1,000- replication bootstrapping method was used to validate the score. Normal-based CI are reported in our HR estimate.

Sensitivity analysis using IMV as one endpoint and using all-cause mortality as another endpoint was performed. A cox proportional hazards model was employed to assess the average increment in risk for each endpoint by each increasing category in the R4 score.

Results

A total of 1465 patients were hospitalized during the study period; 543 patients were started on HFO during their hospital stay. Fourteen had been intubated prior to the initiation of HFO and were excluded from the analysis (Supplement Fig. 1). The analysis includes 529 patients. Median follow-up time was 8 (3-11) days. The mean age was 55.8 ± 15 years, 25% were female. The median number of days of COVID-19 symptoms before admission was 7 (6-10). The median length of stay was 13.0 (9 – 22) days and a total of 286 (54%) patients required intensive care. IMV was required in 200 (38%) of cases, and 13 (2.4%) died while on HFO. Causes for mortality among those on HFO were acute myocardial infarction (8/13), pancreatitis secondary to metastatic melanoma (1/13), respiratory insufficiency secondary to gastric carcinoma (DNR) (1/13), septic shock (1/13) and cerebrovascular complications (2/13). Nonetheless, no patients required cardiopulmonary resuscitation as a result of scarcity of ventilators. All-cause mortality was 126 (24%). A total of 213 (40%) participants met the primary endpoint. The complete description and comparison of the population is displayed in Table 1 .

Table 1.

Overall baseline characteristics and comparative between both outcome groups. LOS= Length of stay, BMI= Body mass index, CVD= Cardiovascular disease, eGFR= estimated glomerular filtration rate, IMV= invasive mechanical ventilation, IL-6= interleukin 6, HS= highly sensitive, BNP= brain natriuretic peptide, CRP= C-reactive protein.

	Overall	Composite endpoint met
		No	Yes
Variables	n=529	n=316	n=213	p-value
Age (years)	55.8 ± 15.1	52.7 ± 14.5	60.4 ± 14.8	< 0.001
Sex (female)	393 (74.3%)	70 (22.2%)	66 (31.0%)	0.02
LOS (days)	17.2 ± 12.8	12.2 ± 6.3	24.7 ± 16.0	< 0.001
Days of symptoms before admission	8.6 ± 4.8	8.9 ± 4.2	8.3 ± 5.4	0.16
BMI (kg/m2)	31.4 ± 5.5	31.5 ± 5.6	31.2 ± 5.3	0.54
Hypertension	211 (39.9%)	101 (32.0%)	110 (51.6%)	< 0.001
Diabetes	153 (28.9%)	77 (24.4%)	76 (35.7%)	0.01
Current smoker	46 (8.7%)	28 (8.9%)	18 (8.5%)	0.80
CVD	32 (6.0 %)	13 (4.1 %)	19 (8.9 %)	0.02
Nephropathy	20 (3.8 %)	7 (2.2 %)	13 (6.1 %)	0.02
Hypothyroidism	31 (5.9 %)	12 (3.8 %)	19 (8.9 %)	0.01
COPD or Asthma	9 (1.7 %)	4 (1.3 %)	5 (2.3 %)	0.35
Active cancer	17 (3.2 %)	7 (2.2 %)	10 (4.7 %)	0.11
Autoimmune disease	6 (1.1 %)	2 (0.6 %)	4 (1.9 %)	0.18
Pregnancy	4 (0.8 %)	1 (0.3 %)	3 (1.4 %)	0.16
SpO2 at admission (%)	82.8 ± 11.9	84.7 ± 9.9	79.9 ± 13.9	< 0.001
ROX score at 24 hours	5.9 ± 2.2	6.4 ± 2.3	5.1 ± 1.9	< 0.001
SOFA	0.9 ± 1.4	0.7 ± 1.1	1.3 ± 1.6	< 0.001
PSI	65.4 ± 27.5	58.4 ± 22.4	76.2 ± 31.1	< 0.001
CURB-65	0.8 ± 0.9	0.6 ± 0.8	1.2 ± 1.0	< 0.001
CALL	9.8 ± 2.2	9.3 ± 2.1	10.5 ± 2.2	< 0.001
eGFR (mL/min)	0.9 ± 1.4	89.7 ± 23.5	76.6 ± 28.0	< 0.001
SpO2 before HFOT (%)	84.4 ± 26.2	89.9 ± 9.6	86.2 ± 13.2	< 0.001
Days on HFOT	5.7 ± 4.7	6.1 ± 4.0	5.1 ± 5.5	0.021
Days on IMV	14.1 ± 13.3	–	14.1 ± 13.3	–
Labs on admission
Hemoglobin (mg/dL)	14.1 ± 1.9	14.2 ± 1.6	13.8 ± 2.2	0.01
Leucocytes (x 103/μL)	10.4 ± 5.3	10.4 ± 4.8	10.4 ± 6.0	1
Lymphocytes (x 103/μL)	1.0 ± 1.2	0.9 ± 1.0	1.0 ± 1.5	0.32
Platelets (x 103/μL)	241.6 ± 99.1	255.0 ± 97.3	221.8 ± 98.5	0.001
Potassium (mmol/L)	5.5 ± 21.8	4.1 ± 0.5	7.5 ± 34.3	0.08
IL-6 (pg/mL)	132.7 ± 302.5	91.8 ± 113.5	190.7 ± 444.6	0.001
Ferritin (μg/L)	2135.1 ± 2649.0	2130.6 ± 2664.5	2141.8 ± 2632.8	0.96
D-dimer (ng/mL)	1485.3 ± 5456.0	1344.8 ± 5647.5	1693.8 ± 5166.0	0.49
HS troponin (ng/mL)	34.5 ± 125.4	23.7 ± 90.9	50.1 ± 161.9	0.03
BNP (pg/mL)	87.1 ± 297.1	82.6 ± 359.6	93.3 ± 175.5	0.72
CRP (mg/dL)	18.0 ± 16.6	17.9 ± 19.2	18.1 ± 11.5	0.87
Procalcitonin (ng/mL)	0.6 ± 1.5	0.5 ± 1.4	0.7 ± 1.6	0.24
LDH (U/L)	508.0 ± 681.1	449.9 ± 317.0	592.6 ± 991.8	0.02
Admission to ICU	286 (54.1%)	88 (27.8%)	198 (93.0%)	< 0.001
Days on ICU	8.3 ± 12.8	1.5 ± 3.0	18.3 ± 14.9	< 0.001
Bacterial infection during hospitalization	161 (30.4%)	15 (4.7 %)	146 (68.5%)	< 0.001

Development of the “Risk 4” (R4) score

Significant variables from Table 1 were imputed into a univariate model for predicting the primary endpoint. Due to low prevalence of comorbidities (CVD, COPD, cancer, autoimmune diseases, and hypothyroidism) were concatenated into one single variable. Clinical variables associated with the endpoint were selected to construct the score (Table 2 ). Bacterial infection was removed from the score because it is a consequence IMV. Six variables were included in the score. The score was optimized into 4 categories as follows: 0 criteria, 1 criterion, 2-3 criteria, and ≥4 criteria met. Event count and proportions per R4 score category are displayed in Supplement Table 1.

Table 2.

Components for the R4 score. Each criterion met sums 1 point.

R4 Score Components
Age ≥ 63 years
eGFR ≤ 60 ml/min
ROX ≤ 5.2 at 24h of HFO
History of Hypertension
History of Diabetes
Any of the following comorbidities:	CVD
	Autoimmune disease
	COPD
	Cancer
	Hypothyroidism

Primary endpoint by R4 category

Median follow-up time in patients who met the primary endpoint was 3 (1-9) days. Each category increase in the R4 score increased likelihood for meeting the endpoint (HR 1.45 [1.31–1.61], p< 0.001). HR and CI by each category, as well as the bootstrapped sample CI are summarized in Table 3 and the KM estimate is displayed in Fig. 1 . Sensitivity and specificity for each scoring component and for the R4 categories is summarized in Supplement Table 2. When breaking our primary endpoint into separate outcomes (VMI and death), the average HR for VMI and death by each increase in R4 category were 1.4 [1.26–1.56], p <0.001, and 1.62 [1.42–1.85] p<0.001, respectively.

Table 3.

Overall sample size, number of events, and hazard ratios for the primary endpoint by R4 category.

R4 Category (# of criteria)	n=	Events	HR	p-value	95 %CI	Bootstrap 95% CI
1 (0)	58	8 (14%)	–	–	–	–
2 (1)	173	53 (31%)	2.94	< 0.001	1.67–5.19	1.65–5.26
3 (2–3)	215	96 (45%)	4.08	< 0.001	2.41–6.93	2.36–7.05
4 (≥4)	83	56 (67%)	6.63	< 0.001	3.76–11.70	3.74–11.77

Figure 1.

Comparison against other scores

We compared the R4 score against other scoring methods described in the literature using receiver operating characteristic (ROC) curves for our primary endpoint. When comparing the R4 score against other scores, it performed better than CALL (AUC 0.64), SOFA (AUC 0.63) and NEWS2 (AUC 0.57) scores (p= 0.005 and 0.001, respectively). However, it did not perform significantly different than PSI (AUC 0.69), ROX-24 (AUC 0.71) and CURB65 (AUC 0.68) (p= 0.206, 0.647, and 0.113, respectively). When using the traditional cutoff of 4.88 for the ROX score at 24 hours of HFO, the AUC was 0.61 with a sensitivity and specificity of 46 and 78%, respectively. The ROC curves and AUC for each score is provided in Supplement Fig. 2.

Discussion

In our cohort, of 529 patients with COVID-19 pneumonia who required HFO, a high proportion (60%) avoided IMV and death. HFO could be an IMV-sparing therapy, improving survival, reducing hospital length of stay and lowering stress in health-care personnel.¹⁹ We leveraged the high number of participants on HFO to construct a prognostic score. The reason some patients progress to severe complications is poorly understood, although, likely multifactorial.^{5 ,} 20, 21, 22 Due to the myriad of factors that may influence the prognosis, developing highly reliable and widely generalizable scoring methods to predict clinical outcomes is challenging. While several authors have successfully designed tools to predict mortality,^{23 , 24} some may have too many variables to incorporate which may hamper the ability of clinicians to complete these scores in some settings. The R4 score is a simple and easy to use tool that integrates clinical data to predict adverse outcomes among adult inpatients on HFO. In this study, the purpose of creating a scoring method has merely prognostic purposes, and its utility for guiding treatment remains unclear. Further studies evaluating its use as a decision making or triage tool must be conducted before attempting to give it such purpose.

The R4 score is composed of six variables, all of which were predictive of the endpoint in univariate analysis. When putting together the score, very few participants met 5 or 6 criteria (n= 17, and 2, respectively), these low frequencies limited the power to predict the endpoint. For this reason, we opted for consolidating into a single group any patient meeting more than 4 criteria. Moreover, participants meeting 2 or 3 criteria had a very similar risk (HR= 3.84 and 4.39, respectively) for meeting the primary endpoint. Additionally, the CI around those meeting 2 and 3 criteria completely overlapped, so creating a single category for patients meeting 2 or 3 criteria simplified the score.

Comparison with other standardized scores

Traditional scoring tools have been repurposed for use among COVID-19 inpatients.²⁵ In a cohort of 830 participants with COVID-19 pneumonia, the performance of qSOFA, NEWS2 and CURB-65 was evaluated. All tools lacked prognostic utility and underestimated the mortality rate in their population. The ROX score is a tool that predicts IVM among patients with HFO¹⁸ and has been validated in the setting of COVID-19.^{26 , 27} Moreover, a recent study demonstrated the value of the ROX in COVID-19 patients, and reiterates it's role in decision-making for clinicians to proceed to IMV.²⁷ This study also suggests that repurposing the ROX score for COVID-19 patients deemed a reevaluation of cutoff values, and found that ROX at 12 hours best predicted IMV when using a ROX cutoff of 5.99.²⁷ In agreement with this study, our data suggests using a higher cutoff value correctly classifies more patients to undergo IMV. Another cohort showed improved performance of the NEWS2 score,¹⁶ with an AUC of 0.82 for predicting mortality. The SOFA score has also been repurposed by different groups but its value is inconsistent among different cohorts.^{28 , 29} Evidence regarding the use of traditional scores to predict outcomes among COVID-19 inpatients is inconsistent, and likely sensitive to different populations and outcomes. Novel scores developed for COVID-19 disease like the CALL score¹² claim high sensitivity and specificity for disease progression. The R4 score proved to perform similarly to PSI, ROX and CURB-65 among our study cohort, while performing better than the SOFA, NEWS2, and CALL scores to predict our primary endpoint. Despite the latter being designed for COVID-19 patients and claiming an AUC of over 0.9 the results did not hold in our cohort, probably in part because we used it for a different endpoint, and we did not have data on HIV status to incorporate into the score. These positive results suggest the R4 score may be used in lieu of, or as a complement of other traditional and novel scores. However, it is important to note this score was evaluated only in the setting of a developmental cohort and reevaluated among our population using bootstrapping, which limits our ability to compare it with other scoring methods.

Our study is limited in that it is a retrospective analysis, thus a casual pathway cannot be determined. Initiation of IMV could have been altered by clinician judgement, as well as ventilator shortage, and ICU availability. Our study does not consider therapies that participants might have received before admission, which may introduce selection bias into our study. Our score was only assessed in a single cohort; it is unclear how it will perform in other cohorts. Comorbidities were assessed by self- or proxy report, which may raise concern for recall bias, exaggerating the effect size. Moreover, with the emergence of SARS-CoV-2 variants, and the implementation of vaccine programs, it is unclear whether the score will perform similarly in populations infected with other variants, and whether vaccines might influence the effect estimates. However, this limitation is present in any scoring tool, until further studies are conducted to evaluate the impact of variants and vaccines on risk tools. The R4 score was not significantly different to some scoring methods in predicting the primary endpoint, although, this may also be seen as a strength, as it may be used in lieu of other scores if information to impute in other tools is insufficient in the clinical setting. Other strengths in our cohort include a large sample size with relatively high event rates, powering our analysis to develop multiple risk stratification categories. Moreover, reliable clinical data collection was accomplished despite having missingness among inflammatory markers, variables used in the score did not suffer from missingness. Additionally, this study employs survival analysis within each R4 score categories to predict our primary endpoint, giving it additional analytical strength.

Inpatients with COVID-19 pneumonia are complex and their outcomes are hard to predict as much about COVID-19 must still be learned. Different tools exist that may be used with limited reliability in predicting outcomes among these patients. The R4 score has proven to be a potentially useful tool that may be complementary to traditional tools to predict IMV or death among COVID-19 patients. It is imperative that this score be validated among another cohort to further understand the implications and utility of this tool. Therefore, we encourage other working groups to validate this tool among other cohorts to better understand the best ways to predict outcomes among COVID-19 patients with HFO.

Author contributions

(1) Conception and design of the study, or acquisition of data, or analysis and interpretation of data: GMA, DR, MTR, RL, JFM, ESV, CAD, AT, VMS, FC, GR MFM.

(2) Drafting the article or revising it critically for important intellectual content: GMA, DR, GTA, MFM.

(3) Final approval of the version to be submitted: GMA, DR, GTA, MTR, RL, JFM, ESV, CAD, AT, VMS, FC, GR MFM.

Conflicts of interest

None.