Development and validation of a laboratory risk score for the early prediction of COVID-19 severity and in-hospital mortality

Abstract

Background and aims

Coronavirus Disease 2019 is characterized by a spectrum of clinical severity. This study aimed to develop a laboratory score system to identify high-risk individuals, to validate this score in a separate cohort, and to test its accuracy in the prediction of in-hospital mortality.

Methods

In this cohort study, biological data from 330 SARS-CoV-2 infected patients were used to develop a risk score to predict progression toward severity. In a second stage, data from 240 additional COVID-19 patients were used to validate this score. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC).

Results

In the development cohort, a step-wise decrease in the average survival duration was noted with the increment of the risk score (p_ANOVA < 0.0001). A similar trend was confirmed when analyzing this association in the validation cohort (p < 0.0001). The AUC was 0.74 [0.66–0.82] and 0.90 [0.87–0.94], p < 0.0001, respectively for severity and mortality prediction.

Conclusion

This study provides a useful risk score based on biological routine parameters assessed at the time of admission, which has proven its effectiveness in predicting both severity and short-term mortality of COVID-19. Improved predictive scores may be generated by including other clinical and radiological features.

Implications for clinical practice.

• •This study provides a laboratory risk score with a satisfying predictive performance of COVID-19 severity and mortality. The score is based on routine parameters that are easy to measure and time-saving at a very low-cost.
• •This score will enable a better risk stratification of COVID-19 patients at the time of admission.
• •This may contribute to optimizing patient’s medical care and to overcoming the lack of medical and material resources particularly frequent in such emergency conditions.

Introduction

Coronavirus 2019 disease (COVID-19) is an infectious pathology characterised by an unprecedented rate of emergence. Since the first case reported in Wuhan Province in China, almost twelve months ago, the disease has spread swiftly to affect more than 200 countries in all continents around the world (Lu et al., 2015, Young et al., 2020).

In Algeria, the first confirmed case of COVID-19 was declared on February 25, 2020. Soon thereafter, a contagion area was formed in the Blida region, which became then the epicentre of the outbreak. After a steady state observed from the beginning of last September, an alarming upsurge was reported from the end of last October, raising the total contamination count to more than 62,300 cases and the total mortality rate of more than 2000 cases, as reported by the latest public reports (“47-Ministère de la Santé de la Population et de la Réforme Hospitalière avec la contribution du Ministère de la Poste et des Télécommunications,” n.d.).

The clinical manifestation of COVID-19 is known for its very heterogeneous aspect; while in most cases, the disease remains silent or is restricted to a few mild respiratory symptoms, for certain individuals the disease may progress to severe or even critical forms, requiring specific management in intensive care units (Wang et al., 2020, Wu et al., 2020, Yang et al., 2020). The seriousness of severe forms is related to the particularly high rate of short-term mortality. Indeed, according to recent epidemiological investigations, the mortality rate is estimated to be twenty-fold higher in patients who acquire severe forms of the disease (Ge et al., 2020, Gong et al., 2020a).

Facing this pandemic, it is a challenge for clinicians to avoid progression to severe forms. It is therefore crucial to dispose of an early risk stratification system. This system will allow identifying high-risk patients at an early stage, possibly leading to optimized management, and overcoming the shortage of medical and material resources (Gao et al., 2020, Li et al., 2020, Shi et al., 2020, Zhou et al., 2020). At the same time, this system will provide a more adequate screening of patients eligible to be transferred to specialized intensive care units, thus avoiding the overload of care and the medical errors that may arise (Garcia-Alamino, 2020, Janssonet al., 2020, Lucchini et al., 2020).

In a recently published work, we have identified, in addition to advanced age, six biological abnormalities that could serve as early risk markers and these include: elevated blood urea nitrogen, elevated C-reactive protein (CRP), decreased natremia, decreased albuminemia, elevated lactate dehydrogenase (LDH), and elevated neutrophil to lymphocyte ratio (NLR) (Bennouar et al., 2020). The present study was designed with the aim to construct a scoring system that would allow the screening of high-risk subjects at admission, to validate this score in a separate cohort, and also to assess its relevance in predicting short-term mortality.

Materials and methods

Study design and development cohort

This study was conducted in accordance with the Declaration of Helsinki; it was approved by the local ethics committee 441/DG/2020 , however, the requirement for written informed consent was waived given the context of the fast emergence of this infectious disease.

This is a single-center, retrospective, cohort study, including SARS-COV-2 infected subjects who were admitted to the Frantz-Fanon Unit of the University Hospital Center (CHU) of Blida. Algeria. The study population, design, and site were detailed elsewhere (Bennouar et al., 2020). Briefly, upon the onset of the outbreak in the Blida area last March, three isolation wards as well as an intensive care unit were set up in the hospital to handle the confirmed cases of COVID-19. After excluding subjects under the age of 18 years, pregnant women, cases of active cancers, and patients who died within 48 hours after admission, a total of 329 subjects, admitted to the hospital's isolation wards, between March 27 and April 22, 2020, were enrolled and followed for up to 28 days, this group represents the risk-score development cohort in the present study.

All included patients were tested upon admission for the following biological parameters: inflammatory markers including: CRP and a total blood count with the calculation of the NLR ratio, blood glucose and renal function markers including: blood urea nitrogen, serum creatinine, and electrolytes (sodium and potassium), albumin and total protein, hepatic enzymes: LDH, glutamo-oxaloacetic transaminase (GOT) and glutamo-pyruvic transaminase (GPT), gamma-Glutamyl-Transpeptidase (γ-GT) and alkaline phosphatases (PAL).

Severity was defined according to the following standard criteria: severe form by one of the following criteria: 1) shortness of breath: respiratory rate > 30 breaths/min in the resting state; 2) pulse oxygen saturation < 93% or 3) arterial blood oxygen pressure (PaO2)/oxygen concentration (FIO2) < 300 mmHg. Very sever or “critical form” was defined by the presence of one of the following criteria: 1) respiratory failure requiring mechanical ventilation; 2) shock or 3) multi-organ failure, requiring ICU (Liu et al., 2020, Wang et al., 2020).

The final endpoint was the progression to a severe form, as defined above by respiratory symptoms requiring invasive or non-invasive mechanical ventilation, transfers to the ICU or death.

Development of a risk score

In the previous study (Bennouar et al., 2020), baseline biological parameters, as measured at admission were integrated into a proportional Cox regression model in order to identify, among them, those that could serve as early predictors of disease progression. As a result of this study, age above 60 years in addition to six biological conditions, were selected as potential risk markers, independently related to COVID-19 severity. These were elevated blood urea nitrogen (≥8 mmol/l), elevated LDH (≥367 UI/l), elevated NLR (≥7.99) and elevated CRP (≥42 mg/l), as well as decreased albumin (≤33.5 g/l) and decreased natremia (≤133.6 mmol/l). These seven factors were then combined into an ordinal risk score consisting of seven consecutive levels (S0-S7).

Validation of the risk score

The significance of the association between the risk level, as assigned upon admission, and the occurrence of severe forms was first assessed in the development cohort, and then in a separated validation cohort composed of 247 additional confirmed COVID-19 cases, who were admitted to the same hospital between April 25 and June 20, 2020. The same exclusion criteria were also applied when enrolling in the validation cohort. In the second stage, the ability of this score to predict in-hospital mortality was tested in both cohorts separately.

Statistical analysis

The statistical analysis was performed using SPSS software version 25.0. The Shapiro-Wilk test was used to explore continuous variable distribution. The continuous variables are presented as Means ± Standard Deviations or Medians (interquartiles) and are compared using the Student t-test, Mann-Whitney U test or ANOVA depending on the normality of the distribution and the number of subgroups to be compared. Qualitative variables are outlined as percentages and are compared using Pearson's χ2 test for trend.

For the purpose of assessing the predictive performance of the developed score in the early detection of worsening outcomes as well as the prediction of in-hospital mortality, the area under the receiver operating characteristic curve (AUC) and its 95% confidence interval (CI) were calculated. Cutoff values were defined using the Youden index; the maximum value reflects the best balance between sensitivity and specificity.

For all statistical tests, a p value of less than 0.05 was considered as statistically significant.

Results

Baseline characteristics of the study population

A total of 576 patients were enrolled in this study, of which 329 patients formed part of the development cohort and 247 patients of the validation cohort. The baseline characteristics of the study population are presented in Table 1 and Table 2 .

Table 1.

Baseline characteristics and outcomes of the development and the validation cohort.

	Development cohort n = 329	Validation cohort n = 247	p
Age (years)	66.6 ± 8.9	65.1 ± 10.6	0.05*
Age > 60 years n (%)	286 (86.9)	204 (82.6)	0.15
Male sex n (%)	205 (62.3)	167 (67.6)	0.18
Outcomes
Mean duration of worsening (days) [min–max]	8.5 ± 7.2 [3–28]	8.7 ± 7.6 [3–28]	0.25
Severe forms n (%)	143 (43.5)	84 (34)	0.02
Mortality rate n (%)	44 (13.4)	66 (26.7)	<0.0001
Survival duration (Days)	25.0 ± 7.7	22.8 ± 9.4	0.002*
After 28 days of follow-up, patients still: n (%) 0.21
On mechanical ventilation	10 (3.03)	05 (2.02)
No mechanical ventilation	20 (6.07)	13 (5.26)
In isolated wards	69 (20.9)	63 (25.5)
Discharged	186 (56.6)	100 (40.5)

p: Pearson’s χ² test, p*: Student t test.

Table 2.

Baseline laboratory characteristics of the development and the validation cohort.

	Development cohort n = 329	Validation cohort n = 247	p
RBC (106 e/μl)	4.5 ± 0.76	4.38 ± 0.81	0.11
Haemoglobin (g/dl)	12.8 ± 2.1	12.5 ± 2.1	0.1
WBC (103 e/μl)	10.1 ± 5.3	11.4 ± 7.1	0.008
Lymphocyte (%)	15.4 ± 9.1	13.5 ± 8.9	0.014
Lymphocyte (103e/ul)	1.3 ± 0.66	1.3 ± 0.93	0.86
Neutrophils (%)	76 ± 12.4	78.6 ± 10.5	0.009
Neutrophils (103 e/ul)	7.9 ± 5.06	9.3 ± 6.4	0.004
NLR	8.4 ± 8.8	10.1 ± 9.9	0.03
Platelet (103 e/μl)	303 ± 137	300 ± 136	0.84
CRP (mg/l)	40.9 ± 24.3	38.9 ± 33.3	0.43
Glucose (mmol/l)	10.0 ± 5.7	10.4 ± 5.7	0.41
Urea (mmol/l)	10.1 ± 8.0	12.0 ± 9.1	0.01
Creatinine (μmol/l)	152 ± 158	167 ± 144	0.24
Sodium (mmol/l)	134 ± 5.9	134.1 ± 6.17	0.79
Potassium (mmol/l)	4.17 ± 0.73	4.21 ± 0.82	0.54
LDH (UI/l)	457 ± 328	495 ± 358	0.18
GOT (UI/l)	67.8 ± 107	76.5 ± 122	0.37
GPT (UI/l)	49.8 ± 85.4	53.5 ± 96.9	0.63
γ-GT (UI/l)	52.4 ± 48.7	55.3 ± 68.4	0.52
ALP (UI/l)	169 ± 80.7	182 ± 178	0.25
Albumin (g/l)	38 (10)	37 (7)	<0.0001*
Total Proteins (g/l)	72 (7)	69 (11)	<0.0001*

ALP: alkaline phosphatases, CRP: C-reactive protein, GOT: glutamo-oxaloacetic transaminase, GPT: glutamo-pyruvic transaminase, γ-GT: gamma-Glutamyl-Trans-peptidase, LDH: lactate dehydrogenase, NLR: neutrophil/lymphocyte ratio. RBC: red blood cells, WBC: white blood cells.

p: Student test, p*: Mann-Withney test.

Bold values indicate a statically significant association (p<0.05).

A clear predominance of both male sex and age above 60 years was noted in each cohort, without any significant differences. Over the follow-up period, 43.5% of patients in the development cohort had progressed toward a severe form requiring intensive care, in the validation cohort this proportion was 34% (p = 0.02). Moreover, the validation cohort exhibited a higher mortality rate (26.7% vs. 13.4%, p < 0.0001) and a lower average survival duration (22.8 ± 9.4 vs. 25.0 ± 7.7 days, p = 0.002) (Table 1).

From a biological standpoint, the two groups were broadly comparable. However, the validation cohort showed a higher level of blood urea nitrogen, white blood cells, neutrophils, and NLR, as well as lower albumin, protein, and lymphocyte levels (Table 2).

Risk score, occurrence of severe forms, and in-hospital mortality

For each patient, the risk score was calculated by the sum of points assigned to each of the included variables (Table 3 ). Seven risk levels were defined. The incidence of severe forms and the mortality rate by risk level are shown in Fig. 1 and Fig. 2 respectively.

Table 3.

List of variables included in the risk score.

Variables		Score
Age	<60 years	0
	≥60 years	1
Natraemia	<133.6 mmol/l	1
	≥133.6 mmol/l	0
Blood urea	<8.0 mmol/l	0
	≥8.0 mmol/l	1
CRP	<42 mg/l	0
	>42 mg/l	1
NLR	<7.99	0
	≥7.99	1
LDH	<367 UI/l	0
	≥367 UI/l	1
Serum Albumin	<33.5 g/l	1
	>33.5 g/l	0
Total*	–	7

The risk score was calculated by the sum of the seven variables.

CRP: C-reactive protein, LDH: lactate dehydrogenase, NLR: neutrophil/lymphocyte ratio.

Fig. 1.

Distribution of severe forms incidences, according to the risk score, in the development cohort (a), and in the validation cohort (b), p for trend < 0.0001 respectively.

Fig. 2.

a: Distribution of mortality rates, according to the risk score in the development cohort, p _{for trend} < 0.0001. b: Distribution of mortality rates, according to the risk score in the validation cohort, p _{for trend} < 0.0001. c: Average survival duration, according to the risk score in the development cohort, p _ANOVA < 0.0001. d: Average survival duration, according to the risk score in the validation cohort, p _ANOVA < 0.0001.

In the development cohort, the lowest severe forms incidences were observed in both score levels S0 and S1 (2.5% in the S0 level, and 3.3% in the S1 level). Then, this incidence increases gradually and significantly across the score levels. Interestingly, it was observed that all patients classified in both score levels S6 and S7 had acquired a severe form (p _{for trend} < 0.0001) (Fig. 1-a).

A similar trend was noted in the validation cohort (Fig. 1-b); whereas no severe form was reported in patients assigned to level S0, all patients classified at level 7 had progressed to a severe form (p _{for trend} < 0. 0001).

With regard to the mortality rate, according to the risk level, a linear by linear association was also found in both the development and the validation cohorts, with no fatalities reported in the S0 level of the development cohort, nor in the S0 and S1 levels of the validation cohort. The highest mortality rate, meanwhile, was reported among patients who were admitted with a level S7 of the risk score, in both the development cohort (66.7%) and the validation one (100%, p_fortrend < 0.0001) (Fig. 2-a and 2-b). Similarly, in each cohort, the average survival duration tended to decrease gradually with increasing risk score levels (p _ANOVA < 0.0001) (Fig. 2-c and 2-d).

Performance of the risk score in predicting both severity and in-hospital mortality:

The ROC curve was used to explore the effectiveness of the risk score in predicting both severity and mortality. The results are shown in Fig. 3 (a-d).

Fig. 3.

a: Receiver operating characteristic (ROC) curve for COVID-19 severity prediction in the development cohort: the area under the ROC curve (AUC) = 0.95 [0.93–0.97], p < 0.0001. b: ROC curve for mortality prediction in the development cohort: AUC = 0.84 [0.78–0.89], p < 0.0001. c: ROC curve for COVID-19 severity prediction in the validation cohort: AUC = 0.74 [0.66–0.82], p < 0.0001. d: ROC curve for mortality prediction in the validation cohort: AUC = 0.90 [0.87–0.94], p < 0.0001.

In the development cohort, the score showed a very satisfying ability for the early prediction of both severity (AUC = 0.95) and mortality (AUC = 0.84). For a threshold value of 3.0 and higher, the score could predict severity with a sensitivity of 95% and a specificity of 85%. Similarly, for a threshold value of 4.0 and higher, the score could predict mortality with a sensitivity of 86% and a specificity of 71% (Fig. 3-a, and 3-b, respectively).

Likewise, in the validation cohort, the score showed a predictive performance as satisfying as what was observed in the development cohort (AUC = 0.74 and 0.90) for severity and mortality respectively. For the same above-mentioned cut-offs, the score presents a sensitivity of 82% and 91% and a specificity of 71% and 70%, respectively for severity and mortality.

Discussion

In this retrospective study, we developed and validated a risk score, which, with satisfying performance, was able to predict not only progression to the severe form but also short-term mortality caused by COVID-19. This ordinal score, to be calculated as early as the admission stage, will enable ranking patients into seven consecutive risk levels, and could thus contribute to the more targeted management of higher-risk cases, particularly in low-resource areas.

This score is built up, besides age, from six biological factors, widely available in current practice, easy to assess, and quite affordable. These are CRP, NLR, urea, natremia, albuminemia, and LDH. Some of these parameters had already proven their individual effectiveness in predicting COVID-19 severity in numerous previously published studies (Gong et al., 2020b, Henryet al., 2020, Mardani et al., 2020, Tan et al., 2020, Wang, 2020).

In this study, the analysis of the ROC curve led to the identification of two important threshold values: a threshold value of 03 predicts the severity with a sensitivity of 95% and a specificity of 85%. In addition, a threshold value of 04 predicts short-term mortality with a sensitivity of 86% and a specificity of 71%. Among the recently published studies on this topic, the study conducted by Galloway et al (Galloway et al., 2020) led to the establishment of a clinical-biological score based on the following parameters: advanced age, male sex, co-morbidities, respiratory rate, oxygenation, radiographic features, neutrophils, elevated CRPs, and reduced albuminemia. The authors reported that a threshold value of 04 was associated with a higher cumulative incidence of critical forms and 28-day mortality.

In another study conducted by Gong et al (Gong et al., 2020a), a nomogram, based on biological parameters, was constructed to predict, at an early stage, the progression of COVID-19. The authors found a significant correlation between the clinical severity and elevated LDH, urea, CRP, direct bilirubin, red blood cell distribution width-coefficient variation (RDW) as well as a decreased albumin level.

In most available studies, the increase in inflammatory markers was a constant sign related to COVID-19 severity (Liang et al., 2020, Wynants et al., 2020). Besides CRP, NLR is a particularly interesting inflammatory marker for monitoring COVID-19 progression, as it indicates both a decrease in lymphocyte count; a direct consequence of viral aggression (Guan et al., 2020, Liu et al., 2020, Zhang et al., 2020), and an increase in neutrophil count; an indicator of exacerbated inflammatory reaction and bacterial super-infection (Tan et al., 2020).

In addition to the inflammatory parameters, the concomitant decrease in albuminemia and natremia, as well as the elevation of cellular damage-related markers (LDH and urea), are further indicators of the extent of the inflammatory response, and thus of the disease's severity. Their prognostic value in hospital settings was discussed well before the COVID-19 pandemic (Barazzoniet al., 2020, Barbosa-Silva, 2008, Bergeret al., 2019, Liu et al., 2020, Thibault et al., 2020, Wei et al., 2020).

Limitations

This study has limitations. Firstly, this is a retrospective single-center study which may limit the generalizability of the findings. Secondly, the score developed is based solely on biological parameters; the inclusion of other clinical and radiological features could contribute to the development of a new score with improved predictive capability. Finally, and despite a satisfactory follow-up period, most of the patients included were still hospitalised at the end of this study, so their status could ultimately change. Further studies with a longer follow-up period and larger sample size are needed to understand the impact of this score on survival beyond 28 days.

Conclusion

This study provides a useful risk score based on biological routine parameters assessed at the time of admission, which has proven its effectiveness in predicting both severity and short-term mortality associated with COVID-19.

Conflict of interest

The authors have no conflicts of interest to declare.

Financial source

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Ethical Statement

Human and animal rights: The authors declare that the work described has been carried out in accordance with the Declaration of Helsinki of the World Medical Association revised in 2013 for experiments involving humans as well as in accordance with the EU Directive 2010/63/EU for animal experiments.

Informed consent and patient details: The authors declare that this report does not contain any personal information that could lead to the identification of the patient(s) and/or volunteers.

Disclosure of interest

The authors declare that they have no known competing financial or personal relationships that could be viewed as influencing the work reported in this paper.

Author contributions

All authors attest that they meet the current International Committee of Medical Journal Editors (ICMJE) criteria for Authorship.