Development and validation of the CoV19-OM intensive care unit score: An early ICU identification for laboratory-confirmed SARS-CoV-2 patients from a retrospective cohort study in Oman

Abstract

Objective

To develop and validate a clinical score that will identify potential admittance to an intensive care unit (ICU) for a coronavirus disease 2019 (COVID-19) case.

Materials and methods

The clinical scoring system was developed using a least absolute shrinkage and selection operator logistic regression. The prediction algorithm was constructed and cross-validated using a development cohort of 313 COVID-19 patients, and was validated using an independent retrospective set of 64 COVID-19 patients.

Results

The majority of patients were Omani in nationality (n = 181, 58%). Multivariate logistic regression identified eight independent predictors of ICU admission that were included in the clinical score: hospitalization (OR, 1.079; 95% CI, 1.058–1.100), absolute lymphocyte count (OR, 0.526; 95% CI, 0.379–0.729), C-reactive protein (OR, 1.009; 95% CI, 1.006–1.011), lactate dehydrogenase (OR, 1.0008; 95% CI, 1.0004–1.0012), CURB-65 score (OR, 2.666; 95% CI, 2.212–3.213), chronic kidney disease with an estimated glomerular filtration rate of less than 70 (OR, 0.249; 95% CI, 0.155–0.402), shortness of breath (OR, 3.494; 95% CI, 2.528–6.168), and bilateral infiltrates in chest radiography (OR, 6.335; 95% CI, 3.427–11.713). The mean area under a curve (AUC) for the development cohort was 0.86 (95% CI, 0.85–0.87), and for the validation cohort, 0.85 (95% CI, 0.82–0.88).

Conclusion

This study presents a web application for identifying potential admittance to an ICU for a COVID-19 case, according to a clinical risk score based on eight significant characteristics of the patient (http://3.14.27.202/cov19-icu-score/).

Introduction

In December 2019, Wuhan City in Hubei Province, People’s Republic of China became the epicenter of the novel coronavirus disease caused by severe acute respiratory syndrome corona virus 2 (SARS-CoV-2) (Huang et al., 2020, WHO, 2020). The disease spread worldwide and the positivity rate increased exponentially, such that the World Health Organization declared the disease a pandemic on March 11, 2020. In the Sultanate of Oman, from two confirmed cases on February XX, 2020, the number of cases had reached 107 213, with 1053 deaths, as of October 13, 2020 (Dong et al., 2020, Khamis et al., 2020a, Khamis et al., 2020b). This figure translates to a 0.98% mortality rate, which is way below the global mortality rate of 2.83% (Dong et al., 2020). On the other hand, in a sample of 8624 hospitalized cases across the Sultanate, an 8.8% intensive care unit admission rate was observed (Director General of Information Technology, Oman, 2020). Such a rate of ICU admission creates a pressing problem if left undetected.

Our study focused on early identification of laboratory-confirmed coronavirus disease 2019 cases who were likely to progress and be admitted in an intensive care unit facility. The identification was performed using a retrospective study in which the patients’ demographic characteristics, medical histories, symptoms, laboratory parameters on admission, radiological findings, diagnoses on admission, and ICU outcomes were taken into account to build a model that would probabilistically identify their admittance to the ICU facility. The simple algorithm developed is expected to be of great help to clinical practitioners in managing their patients and medical resources, and could lead to better planning and optimization of limited human resources and physical facilities.

Materials and methods

Data collection, ethical approval, and data pre-processing

The patients enrolled on this study were reverse transcription-polymerase chain reaction (RT-PCR)-confirmed COVID-19 cases from the Royal Hospital (RH), Ministry of Health, Oman. A formal ethical approval for the conduct of the study was granted by the Directorate General of the Royal Hospital, Ministry of Health. The necessity for informed consent was waived because the identity of the patients had been masked and anonymized.

The patients’ data were obtained from RH medical records stored in a structured, flat data file. The data included the patients’ demographic characteristics, medical histories, symptoms of COVID-19, laboratory parameters on admission, radiological findings, diagnoses on admission, and ICU outcomes. A team of RH medical experts cross-checked the veracity of the data. In total, 445 cases from several regions and municipalities in Oman were used as an initial dataset. Data pre-processing and checking for zero-variance variables were carried out, resulting in a final total of 313 cases, with 27 potential predictor variables.

Variable selection, statistical analysis, and performance metrics

The development cohort presented several variables that were potentially associated with admission to an intensive care unit. This was a classical problem, in which multicollinearity and over-fitting occurred in a linear model. To address this issue, a variable selection technique using ${\mathcal{L}}_1$ regularization was executed. More specifically, a least absolute shrinkage and selection operator (LASSO) regression was performed on all predictor variables, with dichotomized ICU admission as the response variable. The regularization was tuned using the minimum value for $\lambda$.

To obtain the best-fitting model, the development cohort was resampled using the k-fold cross validation method. Using this approach, the development cohort was partitioned into ten almost equally sized folds or groups, with one of these ten groups selected as the testing set and the remaining nine merged and used as the training set. This approach was repeated ten times to ensure that each fold was selected once as the testing set. The loss function used to select the best-fitting model was the mean squared error. The model with the smallest mean squared error was selected as the final model, and was applied to the independent subjects for validation.

Descriptive and inferential statistics were used to analyze the sample data and population data, respectively. Categorical variables were reported using frequencies and percentages, while continuous non-normal variables were reported using median and interquartile range (IQR). Mean and standard deviation were used to present the continuous normal variables. Patients’ records were divided into two groups — either ICU or not. This dichotomized variable served as the outcome variable in the study. To test the independence between the categorical predictor and the outcome variable, Pearson’s χ² test and Fisher’s exact test for a 2 × 2 contingency table were used. Tests of independence were not performed for variables violating the expected count. For continuous variables, Kolmogorov–Smirnov tests were implemented to assertain normality in the data. For continuous, non-normal variables, the non-parametric Wilcoxon/Mann–Whitney U-test was used; otherwise, Student’s t-test was performed to assess the significance of differences between groups. Statistical significance was determined using a level of p = 0.05. A tolerance of at most 5% missing cases was enforced and median value imputation was imposed on the affected variable. Data analyses were carried out using the open-source statistical package R, version 3.6.

Results

Demographic and clinical characteristics

A pre-processed development cohort totalling 313 COVID-19 patients from different municipalities across the Sultanate of Oman was gathered from the Royal Hospital between March 9, 2020 and July 20, 2020. Of this development cohort, 141 out of 313 (45%) were admitted in an ICU facility either on admission or as their infection progressed during hospitalization. The majority of the patients belonged to the 41–70 years old age group (n = 181, 58%). The cohort comprised mainly male patients (n = 210, 67%) and most were Omani nationals (n = 201, 64%). The observed mortality rate for the development cohort was 15% (48 out of 313). In total, 225 (72%) were considered as low-risk, based on their CURB-65 score on admission. On the other hand, 81 of these 225 were eventually admitted to an ICU facility, which represented more than half (81/151, 57%) of total ICU cases. Hypertension (n = 145, 46%) and diabetes mellitus (n = 139, 44%) were the top two medical history records observed among patients. Fever (n = 237, 76%) and shortness of breath (n = 221, 71%) were the most common symptoms to be were observed among patients upon admission. The top three diagnoses on admission were pneumonia (n = 210, 67%), severe pneumonia (n = 210, 67%), and acute respiratory distress syndrome (ARDS) (n = 210, 67%), respectively. Some 20 patients (7%) were considered asymptomatic upon admission.

Development of CoV19-OM ICU score

Twenty-seven potential variables gathered upon hospital admission were subjected to a regularization procedure using LASSO to treat potential multicollinearity and over-fitting among the variables associated with ICU admission (Table 1 ). The LASSO procedure produced 11 independent variables using a minimum of λ = 0.02 as the tuning parameter to regularize all variables. These included demographic profile, medical history, symptoms, laboratory parameters on admission, and chest radiographic results. More specifically, age group, hospitalization period, nationality, CURB-65 score, shortness of breath, bilateral infiltrates as chest X-ray abnormality, absolute lymphocyte count, C-reactive protein, D-dimer, chronic kidney disease, and lactate dehydrogenase were identified as significant predictors.

Table 1.

Demographics, laboratory findings, and clinical characteristics of patients with respect to ICU admission.

Variables		In ICU?
n = 313, unless specified otherwise	Total	No	Yes	p-Value
No. of valid cases	313b	172	141
Characteristics
No. (%), unless specified otherwise
Age groupa				0.256
40 and below	96 (31%)	54 (31%)	42 (30%)
41–70	181 (58%)	94 (55%)	87 (62%)
71 and above	36 (11%)	24 (14%)	12 (8%)
Male gender	210 (67%)	108 (63%)	102 (72%)	0.090
Omani nationalitya	201 (64%)	73 (77%)	68 (48%)	<0.001
Hospitalization, median (IQR), daysa	8 (4–14)	5 (3–8)	14 (9–22)	<0.001
CURB-65 scorea				<0.001
0–1	225 (72%)	144 (84%)	81 (57%)
2	62 (20%)	22 (13%)	40 (28%)
3–5	26 (8%)	6 (3%)	20 (14%)
Medical history
No. (%)
Diabetes mellitus	139 (44%)	70 (41%)	69 (49%)	0.170
Hypertension	145 (46%)	82 (48%)	63 (45%)	0.649
Dyslipidemia	64 (20%)	30 (20%)	34 (21%)	0.779
Respiratory diseases	22 (7%)	12 (7%)	10 (7%)	0.968
Heart diseases	48 (15%)	32 (19%)	16 (11%)	0.084
Liver diseases	12 (4%)	9 (5%)	3 (2%)	0.237
CKD (eGFR < 70)a	55 (18%)	39 (23%)	16 (11%)	0.011
Alcohol (n = 142)	90 (63%)	58 (64%)	32 (63%)	0.906
Symptoms
No. (%)
Fever	237 (76%)	126 (73%)	111 (79%)	0.291
Urinary tract infection	180 (58%)	88 (51%)	92 (65%)	0.016
Shortness of breatha	221 (71%)	100 (58%)	121 (86%)	<0.001
GI symptoms	96 (31%)	64 (37%)	32 (23%)	0.007
Chest pain	43 (14%)	26 (15%)	17 (12%)	0.510
Loss of taste or smell (n = 278)	13 (5%)	11 (7%)	2 (2%)	0.048
Stroke/CNS (n = 312)	19 (6%)	16 (9%)	3 (2%)	0.009
Bilateral infiltrates — chest X-ray abnormalitya	234 (75%)	98 (57%)	136 (96%)	<0.001c
Lab parameters on admission
Median (IQR)
ALC, × 109/La	1.00 (0.70–1.40)	1.10 (0.70–1.50)	0.90 (0.6–1.2)	<0.001
CRP, mg/dLa	99 (47–168)	70 (31–132)	145 (81–203)	<0.001
Ferritin, μg/L	727 (284–1528)	553 (128–1205)	971 (464–1758)	<0.001
Corrected Ca, mm/L (n = 311)	2.04 (1.95–2.13)	2.06 (1.96–2.15)	2.02 (1.94–2.10)	0.095
Vitamin D, ng/mL (n = 83)	62 (44–68)	64 (44–91)	59 (46–74)	0.428
Troponin, ng/mL (n = 202)	15 (8–48)	14 (7–44)	16 (9–54)	0.105
D-dimer, μg/La	0.77 (0.37–1.79)	0.51 (0.19–1.04)	1.06 (0.60–3.27)	<0.001
ALT, U/L	36 (18–66)	31 (15–61)	44 (26–75)	0.001
Total bilirubin, μmol/L	9 (6–15)	8 (5–12)	10 (7–16)	0.002
LDH, U/La	396 (279–545)	319 (225–418)	511 (393–660)	<0.001
QTc interval, ms (n = 230)	441 (416–467)	436 (412–465)	449 (424–469)	0.093
Chest X-ray result				<.001
No. (%)
Normal	60 (19%)	57 (33%)	3 (2%)
Bilateral infiltrates	234 (75%)	98 (57%)	136 (96%)	<0.001c
Unilateral infiltrates	16 (5%)	15 (9%)	1 (1%)
Pleural effusion	3 (1%)	2 (1%)	1 (1%)
Diagnosis on admission
No. (%)
Asymptomatic (n = 270)	20 (7%)	18 (12%)	2 (2%)
Pneumonia (n = 293)	167 (57%)	120 (73%)	47 (37%)	<0.001
Severe pneumonia (n = 284)	101 (36%)	10 (7%)	91 (68%)	<0.001
ARDS (n = 281)	70 (25%)	1 (1%)	69 (53%)
Sepsis (n = 277)	21 (8%)	7 (5%)	14 (11%)	0.046
Myocardial infarction (n = 275)	10 (4%)	1 (1%)	9 (7%)

Abbreviations: IQR —interquartile range; CURB-65 — confusion, urea, respiratory rate, and blood pressure at age 65 years or older; CKD — chronic kidney disease; eGFR — estimated glomerular filtration rate; CNS — central nervous system; ALC — absolute lymphocyte count; CRP — C-reactive protein; Ca — calcium; ALT — alanine aminotransferase; LDH — lactate dehydrogenase.

^aIdentified as LASSO significant predictor with minimum of λ = 0.02 as tuning parameter for the full model regularization.

^bMortality rate: 48 out of 313 (15%).

^cChest X-ray result was dichotomized as bilateral infiltrates or not.

These 11 identified variables were inputted in a logistic regression with a backward elimination approach. The logistic regression model building needed to undergo internal cross-validation. Specifically, the best-fitting model was estimated using 10-fold cross-validation, in which the development cohort was grouped into ten almost equally sized folds or groups, with one of these ten groups selected as the testing set and the remaining nine merged and used for parameter estimates. This approach was repeated ten times to ensure that each fold was selected once as the testing set. This procedure produced the best-fitting model, which resulted in eight independently and statistically significant predictors of ICU admission. The clinical score Cov19-OM ICU was determined using the following predictors: hospitalization period (OR = 1.079; 95% CI, 1.058–1.100; p < 0.001), absolute lymphocyte count (OR = 0.526; 95% CI, 0.379–0.729; p = 0.049), C-reactive protein (OR = 1.009; 95% CI, 1.006–1.011; p < 0.001), lactate dehydrogenase (OR = 1.0008; 95% CI, 1.0004–1.0012; p = 0.041), CURB-65 (OR = 2.666; 95% CI, 2.212–3.213; p < 0.001), chronic kidney disease with eGFR < 70 (OR = 0.249; 95% CI, 0.155–0.402; p = 0.004), shortness of breath (OR = 3.949; 95% CI, 2.528–6.168; p = 0.002), and bilateral infiltrates — abnormality in chest X-ray (OR = 6.335; 95% CI, 3.427–11.713; p =  0.003) (Table 2 ).

Table 2.

Multivariate logistic regression results for building the CoV19-OM ICU score model.

Variables	Odds ratio (95% CI)	p-Value
Hospitalization period, days	1.079 (1.058–1.100)	<0.001
Absolute lymphocyte count, × 109/L	0.526 (0.379–0.729)	0.049
C-reactive protein, mg/L	1.009 (1.006–1.011)	<0.001
Lactate dehydrogenase, U/L	1.0008 (1.0004–1.0012)	0.041
CURB-65 score	2.666 (2.212–3.213)	<0.001
CKD (eGFR < 70, yes vs no)	0.249 (0.155–0.402)	0.004
Shortness of breath (yes vs no)	3.949 (2.528–6.168)	0.002
Bilateral infiltrates — abnormality in X-ray (yes vs no)	6.335 (3.427–11.713)	0.003
Constant	0.005

Abbreviations: IQR — interquartile range; CURB-65 — confusion, urea, respiratory rate, and blood pressure at age 65 years or older; CKD — chronic kidney disease; eGFR — estimated glomerular filtration rate.

Performance measure of CoV19-OM ICU score

After resolving the multicollinearity and over-fitting among potential predictor variables using the least absolute shrinkage and selection operator, a multivariate logistic regression was used on the remaining eight variables to arrive at the final coefficients for the CoV19-OM ICU score calculation. During the model building, $\hat{p}$ was defined as the estimate of the probability of admission to an ICU. Following this definition, Eq. (1) provides the full equation for the model:

$$\ln \left(\frac{\hat{p}}{1-\hat{p}}\right)=-5.303+0.076*HOS-0.643*ALC+0.009*CRP+0.001*LDH+0.981*CUR-1.389*CKD+1.373*SOB+1.846*BLI$$ (1)

An online web application was created to fully automate the calculation of CoV19-OM ICU scores. This offers clinical practitioners a simple graphical user interface into which they enter the required value for each variable, while the intricate calculation is handled by the hosting server (http://3.14.27.202/cov19-icu-score/, see Figure 1 a). The web app will calculate three metrics based on the model given by Eq. (1). The CoV19-OM ICU score ranges from 1 to 5, with the risk groupings comprizing low-risk, moderate-risk, and high-risk groups. In addition, the predicted probability of being admitted in an intensive care unit facility is also provided, ranging from 0% to 100%.

Figure 1.

(a) Graphical user interface of the CoV19-OM ICU score web application; (b) Pop-up result; (c) ROC curve for the CoV19-OM ICU score, using the development cohort (0.86).

Area under a curve was the metric used to determine the performance of the clinical score, along with k-fold cross-validation for internal consistency. Using a 10-fold cross-validation procedure, the mean area under a curve from the development cohort was 0.86 (95% CI, 0.85–0.87). In two COVID-19-related studies in the literature, based on clinical scores, risk score was calculated with an area under a curve of 0.88 (95% CI, 0.85–0.91) and 0.75 (95% CI, 0.70–0.80) for the COVID-GRAM score (Liang et al., 2020) and CURB-65 model (Lim, 2003), respectively. The performance of our proposed clinical score is comparable with the COVID-GRAM score, with fewer variables in the final model. On the other hand, our proposed clinical score performs well above the CURB-65 model. In fact, the inclusion of CURB-65 as one of the eight final variables in the model (Eq. (1)) makes our clinical score more succinct, because it implicitly includes all the additional variables used in calculating the CURB-65 alone.

Validation of the CoV19-OM ICU score

The validation dataset included 64 retrospective patients. These patients were independent of the development cohort. The majority of the patients were aged from 41 to 70 years old, and around four out of five patients were male. In this cohort, the percentage of Omani nationals was 42% (n = 27) and the percentage of ICU admissions among laboratory-confirmed COVID-19 cases was 70% (n = 45). The significant variables used in the proposed clinical score calculation are shown in Table 3 . The clinical score’s accuracy in the validation cohort was 0.85 (95% CI, 0.82–0.88).

Table 3.

Characteristics of the validation cohort.

Variables		In ICU?
n = 64	Total	No	Yes
No. of valid cases	64	19	45
Characteristics
No. (%), unless specified otherwise
Age, years
40 and below	18 (28%)	7 (37%)	11 (24%)
41–70	40 (63%)	8 (42%)	32 (71%)
71 and above	6 (9%)	4 (21%)	2 (4%)
Male gender	50 (78%)	14 (74%)	36 (80%)
Omani nationality	27 (42%)	10 (53%)	17 (38%)
Hospitalization, median (IQR), days	9.5 (5–18)	5 (4.5–8.5)	16 (6–21)
Absolute lymphocyte count, median (IQR), × 109/L	1.05 (0.50–1.50)	1.20 (0.80–1.50)	1.00 (0.50–1.40)
C-reactive protein, median (IQR), mg/L	156 (59–244)	49 (26–134)	186 (111–260)
Lactate dehydrogenase, median (IQR), U/L	456 (316–592)	287 (249–414)	514 (407–637)
CURB-65 score, median (IQR)
0–1	39 (61%)	17 (89%)	22 (49%)
2	17 (27%)	22 (11%)	15 (33%)
3–5	8 (13%)	0 (0%)	8 (18%)
CKD (eGFR < 70, yes vs no)	6 (9%)	2 (11%)	4 (9%)
Shortness of breath (yes vs no)	51 (80%)	10 (53%)	41 (91%)
Bilateral infiltrates — abnormality in X-ray (yes vs no)	57 (89%)	12 (63%)	45 (100%)

Abbreviations: IQR — interquartile range; CURB-65 — confusion, urea, respiratory rate, and blood pressure at age 65 years or older; CKD — chronic kidney disease; eGFR — estimated glomerular filtration rate.

Discussion

The primary aim of this study was to investigate the association of demographic and clinical characteristics, medical histories, laboratory parameters on admission, and chest radiography results of laboratory-confirmed COVID-19 cases with their admittance to an intensive care unit (ICU) facility. The majority of the patients were middle-aged, and were mostly male. These age and gender characteristics were similar to those found in a number of studies (Galloway et al., 2020, Khamis et al., 2020a, Khamis et al., 2020b). In contrary to some studies in which age and gender were statistically associated with a severe classification of COVID-19 cases (Galloway et al., 2020, Li et al., 2020, Liang et al., 2020, Zhou et al., 2020), our study find non-significant associations between these two demographic variables and admission to an ICU. This non-significance has also been demonstrated in a number of other studies in which clinical scoring systems were being developed (Ji et al., 2020, Meylan et al., 2020).

The median hospitalization period for all patients was 8 days, while those patients who were admitted to an ICU had lower-quartile and median hospitalization periods of 9 and 14 days, respectively. A statistically significant difference in hospitalization period was found between ICU and non-ICU patients. The majority of patients had a low-risk status upon hospital admission based on their CURB-65 score, although a number of these progressed to a severe classification of COVID-19, and were eventually admitted to an ICU. The difference between the CURB-65 scores with respect to admission to ICU and non-admission was statistically significant. This indicated that, along with hospitalization period, CURB-65 score was also a risk factor for ICU admission. Other studies in the literature have shown similar findings with regards to the importance of clinical scores in determining severity of COVID-19 in patients (Oktariani et al., 2019, Satici et al., 2020).

In our study, the most common medical histories among patients were hypertension and diabetes mellitus. The elevated prevalence of these comorbidities has been widely documented in several papers, which have highlighted the coexistence of these non-communicable diseases with the communicable COVID-19 (Barber, 2020, Shibata et al., 2020). However, the association of these top comorbidities was not established in our study.

This retrospective study covered a wide array of potential variables that could be associated with a patient’s admission to an ICU facility. The study involved a very limited number of cases with data on all variables, and therefore some of the variables were dropped due to the significant number of cases lacking information on them. These dropped variables included smoking and body mass index. It worth noting that some studies in the literature have considered these two variables to be significant contributors in severity progression and ICU admission (Baronio et al., 2020, Caliskan and Saylan, 2020). The absence of these variables was a limitation of the study since they could have offered a useful source of variation that could have delivered a different model from that presented in this paper.

Conclusion

This retrospective study presents a web application for identifying potential admittance to an intensive care unit (ICU) facility for laboratory-confirmed COVID-19 cases, which generates a developed and validated clinical score based on eight significant characteristics of the patient. The use of this web app will enable the health-care provider to apply appropriate supportive care and to maximize the use of available ICU facilities in the hospital. However, it should be noted that a lack of generalizability is a limitation of this study due to the relatively small sample sizes used for the training, testing, and validation cohorts. The dataset used for the proposed clinical score was limited to cases in the Sultanate of Oman; results may prove inconclusive if the model were applied in other territories and states. Validation studies undertaken outside the Sultanate’s territory are encouraged in order to generalize the efficiency of the clinical score.

Ethical approval

Ethical approval was obtained from the Directorate General of the Royal Hospital, Ministry of Health (SRC No. 43/2020).

Funding source

Research project funding was obtained from The Research Council (TRC) of the Sultanate of Oman, under a Commissioned Research Program contract (No. TRC/CRP/HCT/COVID-19/20/06).

Conflicts of interest

All authors declare that they have no conflicts of interest.