Abstract
Background
The aim of this study is to develop and validate a scoring system as a tool for predicting the in-hospital mortality in COVID-19 patients in early stage of disease.
Methods
This retrospective cohort study, conducted on 893 COVID-19 patients in Tehran from February 18 to July 20, 2020. Potential factors were chosen via stepwise selection and multivariable logistic regression model. Cross-validation method was employed to assess the predictive performance of the model as well as the scoring system such as discrimination, calibration, and validity indices.
Results
The COVID-19 patients’ median age was 63 yrs (54.98% male) and 233 (26.09%) patients expired during the study. The scoring system was developed based on 8 selected variables: age ≥55 yrs (OR = 5.67, 95% CI: 3.25–9.91), males (OR = 1.51, 95% CI: 1.007–2.29), ICU need (OR = 16.32, 95% CI 10.13–26.28), pulse rate >90 (OR = 1.89, 95% CI: 1.26–2.83), lymphocytes <17% (OR = 2.33, 95%CI: 1.54–3.50), RBC ≤4, 10 6/L (OR = 2.10, 95% CI: 1.35–3.26), LDH >700 U/L (OR = 1.68, 95%CI: 1.13–2.51) and troponin I level >0.03 ng/mL (OR = 1.75, 95%CI: 1.17–2.62). The AUC and the accuracy of scoring system after cross-validation were 79.4% and 79.89%, respectively.
Conclusion
This study showed that developed scoring system has a good performance and can use to help physicians for identifying high-risk patients in early stage of disease.
Keywords: COVID-19, In-hospital mortality, Clinical prediction model, Scoring system
1. Introduction
Novel coronavirus (COVID-19) is the most recent emerging viral disease from the end of 2019.1 Following the global outbreak of the virus, on January.20.2020, the World Health Organization (WHO) issued a statement declaring the new coronavirus to be the sixth leading public health emergency worldwide, posing a threat not only to China but to all countries around the globe.2 In addition, the extent of coronaviruses incidences has been constantly on the rise which is an indication that the virus has been able to cross the continents very quickly and become a major pandemic.3 Similarly, the number of suspected and confirmed COVID-19 cases is also growing in Iran.4 Furthermore, although the virus has the tendency to target the host's respiratory cells and cause a range of respiratory infections in humans, it might also proceed to target various other human cells (cardiovascular system, gastrointestinal tract, and other organs) and consequently initiate undesirable prognosis for the disease. According to the results of previous studies, the severity of the disease symptoms and its prognosis have been linked to comorbidities or the health status of the host.5 To reduce the burden on the health care system, while providing the best possible care for patients, efficient diagnosis and information about the prognosis of the disease are needed.6 Due to the acute and different spectrum of disease severity in patients, it seems that a scoring system based on predictive clinical models, with the ability to detect severity and variables affecting the prognosis of the disease, is very helpful for patient triage in the hospital. Moreover, the indicated scoring system should constitute a predictive clinical model with the ability to detect severity of the disease as well as possessing variables affecting the prognosis of the disease. The indicated scoring system is expected to be able to distinguish patients who may manifest severe symptoms of the disease, and therefore support patients' well-being by prioritizing and identifying high-risk individuals.7 Thus, the one of the main task of clinical or statistical tools is risk stratification.8 Nowadays, exploration of the factors that predispose or protect someone from mortality due to COVID- 19 are valuable and has public health importance. These factors would be knowing with the clinical scoring system. So, scoring systems play an important role in clinical medicine and triage.8 , 9 Considering the high prevalence of this disease as well as the significant percentage of COVID19 related fatalities in Iran, it is crucial to evaluate the epidemiological characteristics of this disease and to further build a predictive model for the intention of classification (triage) of patients. In addition, it is also essential to be aware of the risk factors associated with the various outcomes of this emerging disease, such as premature death. In other words, designing predictive models based on risk factors, not only creates more awareness about the disease, but also can be used as an effective tool in the decision making of the hospitals' staff and physicians in the early stage management of the disease.
2. Material and methods
2.1. Study design and participants
The current retrospective cohort study's data collected from the clinical records of 893 patients hospitalized due to COVID-19 complications with positive PCR test results obtained in Tehran's Imam Hossain Hospital from February 18 to July 20, 2020 in Iran using census method. Totally, in duration of study, 1030 COVID-19 patients referred to current hospital that only 893 patients had inclusion criteria for study. In current study, at least one positive test of real-time Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) of nasopharyngeal swab specimen was the main criteria of COVID-19 disease. In addition, confirmed COVID-19 outpatients and the patients whom had been clinically diagnosed with COVID19 whose diagnosis had not been confirmed by PCR test were excluded from the study. It's also important to note that the present study was approved by the ethics committee of Shahid-Beheshti University of Medical Sciences, Tehran, Iran.
2.2. Data gathering
The study data was obtained using a research made-checklist to extract data from the patients' clinical records. Furthermore, the checklist included information such as demographic, laboratory and clinical information, symptoms at the time of admission, underlying diseases, and data regarding the outcome of the disease (death in hospital and hospitalization in the ICU). Moreover, as regards all the clinical records, results from laboratory tests performed within the patients' initial 24 h' hospital admission were considered. However, in case of the necessity for the performance of recurrent measurements within the first patients’ 24-h admission into the hospital merely the first acquired values were taken into account.
2.3. Outcome
In-hospital death among hospitalized COVID-19 patients was the main consequence of the disease recognized by this study and the follow up period to investigate the occurrence of death was till the last day of patients’ hospitalization.
2.4. Definitions
In this study, fever was considered as the measured body temperature in the axilla at 37.3 °C or higher.10 In the meantime, the Mean Arterial Blood Pressure (MAP) was calculated based on systolic and diastolic blood pressure measured at the time of admission to the hospital according to the formula MAP = [(2 × diastolic) +systolic]/3.11 What was meant as the gastrointestinal symptoms upon patient's arrival at the hospital was the presence of any clinical signs and symptoms such as diarrhea, vomiting, nausea, rectorragia, and abdominal pain that the patient complained of during the initial assessment. Regarding the underlying diseases for instance any patient's history of Coronary Artery Disease (CAD), Myocardial Infarction (MI), Cerebrovascular Accident (CVA), angioplasty or any type of vascular surgery in the group of Cardiovascular diseases (CVDs), End-Stage Renal Disease (ESRD), kidney transplant, nephrotic syndrome, nephropathy in the group of kidney diseases, migraine, Alzheimer's disease, neurosurgery, Multiple Sclerosis (MS), Parkinson's, epilepsy or seizures in the group of Central Nervous System diseases (CNS), and any kind of respiratory diseases including asthma, pneumonia, tuberculosis, and allergies in the group of respiratory diseases were considered.
2.5. Statistical analysis
In this study, the normality of the data was based on the results attained from the Kolmogorov-Smirnov test. Furthermore, this study utilized median and interquartile range (IQR) to describe quantitative data and also frequency and percentage to describe qualitative data. T-test or Mann-Whitney test was also applied to compare the mean of variables being studied by separation of living and deceased patients based on the normality of the quantitative data and the Chi-square test or Fisher's exact test was used to compare the qualitative data between the two groups. The cut off value for the quantitative variables was based on maximum optimal cut point value of sensitivity and specificity. For the variables that were less than 15% missing multiple imputations were performed using the “Amelia” package of R.3.6.2 software. It is also important to note that the variables that were above 15% missing were omitted from the study. Moreover, in order to initiate the modeling process and select the best variables to enter the multivariable model, the stepwise selection method with two backward and forward approaches along with Akaike's Information Criterion (AIC) was done using the “MASS” package. Using AIC has been recommended in prediction models. The use of AIC has been found to provide more power for the selection of predictors with relatively weak effects.12 Univariate and multivariable logistic regression models were also employed to evaluate the variables being studied and to further construct a prediction model.
In addition, to assess the overall performance of model the AIC criterion, Nagelkerke R2, Brier score, and Brier scaled were used and also the C-index, and Area Under Curve (AUC, ROC curve) were utilized to measure discrimination capability. Similarly, validity indices such as Negative Predictive Value (NPV), Positive Predictive Value (PPV), accuracy, sensitivity and specificity were applied to evaluate the validity of the final model. Finally, Hosmer-Lemeshow test and calibration plot were employed for the assessment of model's calibration. In addition, to evaluate the internal validity model, cross validation was used to divide the data into train (80% of data) for the purpose of developing the model and test (20% of data) to evaluate the validity of the model and then all the stated indicators were calculated in it. Finally, nomogram was used to representing the results of scoring system and estimation of in-hospital mortality probability based on final multivariable logistic regression model. Furthermore, all statistical analysis in this study performed using STATA.14 and R.3.6.2 software. Also, all calculations were done at a significance level of less than 0.05 with a 95% confidence interval.
3. Results
3.1. Baseline characteristics of studied COVID-19 patients
In this study, the median age of 893 hospitalized COVID-19 patients was 63 years (IQR = 49–75) of which 491 (54.98%) were males. Hypertension (41.32%, n = 369), cardiovascular disease (33.26%, n = 297) and diabetes mellitus (30.35%, n = 271) possessed the highest prevalence of underlying diseases among these patients. Also, the median time of hospitalization among these specific patients was 6 days (IQR = 4–10). Next, 233 patients (26.09%) died in the hospital, of which 124 (53.22%) manifested severe complications of the disease and consequently they had to be admitted into ICU. According to the results of Table 1 , after comparing demographic and clinical factors, it was observed that the median age of COVID-19 patients who expired in the hospital (median = 73, IQR = (61–83)) was significantly greater than those discharged (Median = 59 (47–71)) (P < 0.001). Moreover, upon the arrival at the hospital the patients who expired in the later stages of the disease had exposed significantly higher pulse rates (median = 90 (80–100)) as well as higher respiratory rates (median = 20 (18-24)) than those discharged. In addition, SPO2 level (median = 88 (82–91)) measured in patients that died later was significantly lower than in those who were ultimately discharged (P < 0.001). This study also determined a significant association between some other factors such as sex, myalgia, headache, history of cardiovascular disease, hypertension, and CNS with mortality (P < 0.001). Also, mortality among hospitalized patients was associated with significant rise in laboratory factors such as white blood cells, neutrophils, C-Reactive Protein (CRP), procalcitonin, Prothrombin Time (PT), Aspartate aminotransferase (AST), Lactate Dehydrogenase (LDH), urea, creatinine, cardiac enzymes and with a significant decrease in the levels of hemoglobin, Red Blood Cell count (RBC) and lymphocyte (P < 0.05) (Table 1).
Table 1.
Comparison of demographic and clinical characteristics of COVID-19 patients between alive and dead groups.
| Variables | All patients (n = 893) | Alive (n = 660) | Dead (n = 233) | P_value |
|---|---|---|---|---|
| Age (yrs) | 63 (49–75) | 59 (47–71) | 73 (61–83) | <0.001 |
| Sex (Men) | 491 (54.98) | 349 (52.88) | 142 (60.94) | 0.033 |
| Body Mass Index (BMI, kg/m2) | 26 (24–29.38) | 26.20 (24.22–29.36) | 25.5 (23.43–29.4) | 0.115 |
| Symptoms in admission (Yes) | ||||
| Headache | 101 (11.31) | 85 (12.88) | 16 (6.87) | 0.013 |
| Chest pain | 85 (9.52) | 60 (9.09) | 25 (10.73) | 0.464 |
| Dyspnea | 558 (62.49) | 408 (61.82) | 150 (64.38) | 0.488 |
| Myalgia | 282 (31.58) | 233 (35.30) | 49 (21.03) | <0.001 |
| Cough | 462 (51.74) | 347 (52.58) | 115 (49.36) | 0.398 |
| Fatigue | 337 (37.74) | 253 (38.33) | 84 (36.05) | 0.537 |
| Fever | 446 (49.94) | 340 (51.52) | 106 (45.49) | 0.114 |
| Gastrointestinal Symptoms | 570 (63.83) | 420 (63.64) | 150 (64.38) | 0.840 |
| Need to ICU/Mechanical ventilation (Yes) | 168 (18.81) | 44 (6.67) | 124 (53.22) | <0.001 |
| Length of hospital stat (days) | 6(4-10) | 6(4-10) | 6(3-10) | 0.573 |
| Medical assessment during hospitalization | ||||
| Pulse Rate (PR, pulse/min) | 87 (80–98) | 86.04 (80–96) | 90 (80–100) | 0.001 |
| Respiratory Rate (RR, per 1/min) | 19 (18 - 23) | 19 (18 -23) | 20 (18 -24) | 0.008 |
| a SPO2 (%) | 90 (86–93) | 90 (87–93) | 88 (82–91) | <0.001 |
| Mean Arterial Pressure (MAP, mmHg) | 0.009 | |||
| <70 | 44 (4.93) | 28 (4.24) | 16 (6.87) | |
| 70-100 | 727 (81.41) | 553 (83.79) | 174 (74.68) | |
| >100 | 122 (13.66) | 79 (11.97) | 43 (18.45) | |
| Underlying diseases (Yes) | ||||
| Hypertension | 369 (41.32) | 246 (37.27) | 123 (52.79) | <0.001 |
| Hyperlipidemia | 53 (5.94) | 34 (5.15) | 19 (8.15) | 0.095 |
| Thyroid | 61 (6.83) | 46 (6.97) | 15 (6.44) | 0.782 |
| Cardiovascular Diseases (CVDs) | 297 (33.26) | 191 (28.94) | 106 (45.49) | <0.001 |
| Diabetes Mellitus | 271 (30.35) | 196 (29.70) | 75 (32.19) | 0.477 |
| Kidney diseases | 95 (10.64) | 65 (9.85) | 30 (12.88) | 0.198 |
| Central Nervous System disorders (CNS) | 99 (11.09) | 59 (8.94) | 40 (17.17) | 0.001 |
| Any type of cancer | 38 (4.26) | 26 (3.94) | 12 (5.15) | 0.431 |
| Respiratory diseases | 78 (8.73) | 53 (8.03) | 25 (10.73) | 0.210 |
| Laboratory values (In admission) | ||||
| Hemoglobin (g/dL) | 12.5 (11.2–13.7) | 12.6 (11.5–13.8) | 12.3 (10.4–13.4) | 0.001 |
| Red Blood Cell count (RBC,106/L) | 4.42 (4–4.84) | 4.46 (4.07–4.86) | 4.29 (3.81–4.75) | 0.0003 |
| White Blood Cell count (WBC, 103/L) | 6.9 (5.1–10) | 6.6 (5–9.2) | 8.7 (5.6–11.8) | <0.001 |
| Lymphocyte (%) | 17.3 (11–24.9) | 19.85 (12.75–26.35) | 12.6 (8.8–18.3) | <0.001 |
| Neutrophil (%) | 75.3 (67.7–82.5) | 73.5 (66.15–80.4) | 80.7 (75–86) | <0.001 |
| C-Reactive Protein (CRP, mg/L) | 48.2 (22.8–75) | 44.45 (20–71.9) | 57 (34.8–85) | <0.001 |
| Procalcitonin (ng/L) | 2.25 (0.51–7.1) | 2.04 (0.42–6.85) | 2.5 (0.85–7.93) | 0.003 |
| PT (second) | 12 (11.1–13.55) | 11.8 (11–13.3) | 12.73 (11.5–14.55) | <0.001 |
| ALT (U/L) | 30.30 (19–60) | 31 (19–59) | 29 (18–62) | 0.684 |
| AST (U/L) | 40.80 (27–78.90) | 39 (26.25–72.43) | 46 (32–94) | <0.001 |
| LDH (U/L) | 638 (439.97–967) | 602 (428.5–896.24) | 755.59 (518–1167) | <0.001 |
| Urea (mg/dL) | 41.3 (28.8–60) | 38.05 (27.15–52.6) | 57.7 (39–82) | <0.001 |
| Creatinine (mg/dL) | 1.2 (1–1.6) | 1.2 (1–1.4) | 1.5 (1.2–2) | <0.001 |
| Creatine Phosphokinase (CPK, IU/L) | 128 (68–327) | 117 (60–262.5) | 190 (91–478) | <0.001 |
| Creatine kinase-MB (CK-MB, U/L) | 18 (3.1–39.6) | 17.04 (2.6–37.16) | 19.54 (3.7–47.69) | 0.026 |
| Troponin I (ng/mL) | 0.03 (0.01–0.92) | 0.03 (0.01–0.92) | 0.05 (0.02–1) | <0.001 |
Values are n(%), median (Q1-Q3).
Oxygen saturation measured by pulse oximetry.
3.2. Predictive variables of in-hospital mortality
The evaluation results of effective factors in predicting in-hospital death among COVID-19 patients by means of univariate and multivariable logistic regression model are reported in Table 2 . Furthermore, in this particular study after selecting the variables using the stepwise approach, it was observed that among the selected variables of univariate analysis, increased age, male sex, history of cardiovascular diseases, and the presence of clinical symptoms at the time of hospitalization, such as fever, drop in blood oxygen percentage, increased pulse rate, in addition to other severe symptoms of COVID-19, required ICU care and hence mechanical ventilation and other laboratory evaluated dimensions at the time of admission such as RBC, lymphocytes, LDH, urea and troponin I levels were known as predictors of in-hospital death among COVID-19 patients (P < 0.05). However, according to the results of multivariable analysis, it was observed that among the mentioned variables, only factors such as age ≥55 yrs (OR = 5.67, 95% CI: 3.25–9.91, P_value < 0.001), male sex (OR = 1.51, 95% CI: 1.007–2.29), Pulse Rate (>90 pulses/min) (OR = 1.89, 95% CI: 1.26–2.83, P_value: 0.002), need to ICU/mechanical ventilation (OR = 16.32, 95% CI 10.13–26.28 P_value < 0.001), RBC (≤4, 106/L) (OR = 2.10, 95% CI: 1.35–3.26, P_value = 0.001), Lymphocyte (<17%) (OR = 2.33, 95% CI: 1.54–3.50, P_value < 0.001), LDH (>700 U/L) (OR = 1.68, 95% CI: 1.13–2.51, P_value = 0.010) And Troponin I (>0.03 ng/mL) (OR = 1.75, 95% CI: 1.17–2.62, P_value = 0.005) were known as the leading predictors of in-hospital death among COVID-19 patients (Table 2).
Table 2.
Results of univariate and multivariable logistic regression for predicting the factors related to in-hospital mortality in COVID-19 patients.
| Variables | Crude ORa, 95% CI | P_value | Adjusted OR, 95% CI | P_value |
|---|---|---|---|---|
| Age (≥55 yrs) | 5.46 (3.61–8.26) | <0.001 * | 5.67 (3.25–9.91) | <0.001* |
| Sex (Men) | 1.39 (1.02–1.88) | 0.034 * | 1.51 (1.007–2.29) | <0.046* |
| Underlying diseases (Yes) | ||||
| Diabetes Mellitus (DM) | 1.12 (0.81–1.54) | 0.477 | 0.65 (0.42–1.01) | 0.056 |
| Hyperlipidemia | 1.63 (0.91–2.92) | 0.098 | 1.84 (0.85–3.95) | 0.116 |
| Cardiovascular Diseases (CVDs) | 2.04 (1.50–2.78) | <0.001 * | 1.43 (0.94–2.19) | 0.094 |
| Medical assessment (In admission) | ||||
| Fever (≥37.3 °C) | 1.39 (1.03–1.82) | 0.030 * | 1.35 (0.90–2.01) | 0.140 |
| b SPO2 (≤93%) | 1.60 (1.07–2.40) | 0.021 * | 1.47 (0.87–2.49) | 0.142 |
| Pulse Rate (>90 pulse/min) | 1.79 (1.32–2.43) | <0.001 * | 1.89 (1.26–2.83) | 0.002* |
| Need to ICU/Mechanical ventilation (Yes) | 15.92 (10.67–23.75) | <0.001 * | 16.32 (10.13–26.28) | <0.001* |
| Laboratory factors | ||||
| Red Blood Cell count (≤4, 106/L) | 1.82 (1.31–2.52) | <0.001 * | 2.10 (1.35–3.26) | 0.001* |
| Lymphocyte (<17%) | 3.68 (2.66–5.10) | <0.001 * | 2.33 (1.54–3.50) | <0.001* |
| LDH (>700 U/L) | 1.92 (1.42–2.59) | <0.001 * | 1.68 (1.13–2.51) | 0.010* |
| Urea (>23.5 mg/dL) | 5.56 (2.66–11.59) | <0.001 * | 2.14 (0.86–5.32) | 0.100 |
| Troponin I (>0.03 ng/mL) | 2.08 (1.53–2.83) | <0.001 * | 1.75 (1.17–2.62) | 0.005* |
* Statistical significance, P_ value < 0.05.
Odds Ratio, 95% Confidence Interval.
Oxygen saturation measured by pulse oximetry.
3.3. Performance and internal validity of scoring model
The assessment results of performance and internal validity of the model are reported in Table 3 .
Table 3.
Description of performance and internal validity of scoring model for predicting in-hospital mortality of COVID-19 patients in full, train and test data.
| Indices | Full data N = 893 |
Development (Train) N = 714 |
Validation (Test) N = 179 |
|---|---|---|---|
| Overall performance | |||
| a AIC criterion | 682.53 | 534.15 | 168.64 |
| R2 (Nagelkerke) (%) | 50.0 | 52.7 | 44.0 |
| Brier score | 0.11 | 0.10 | 0.12 |
| Brier scaled (%) | 41.16 | 43.60 | 34.91 |
| Validity indices (%) | |||
| Optimal cut point | 0.26 | 0.26 | 0.24 |
| Sensitivity | 78.97 | 80.32 | 73.33 |
| Specificity | 81.21 | 82.51 | 73.13 |
| Positive Predictive value (PPV) | 59.74 | 62.14 | 47.83 |
| Negative Predictive Value (NPV) | 91.62 | 92.14 | 89.09 |
| Accuracy | 80.63 | 81.93 | 73.18 |
| Discrimination | |||
| C-Index, 95% CI | 0.87 (0.85–0.90) | 0.89 (0.86–0.91) | 0.85 (0.78–0.91) |
| Calibration | |||
| b H-L tests, X2(P) | 4.67 (0.791) | 6.14 (0.630) | 10.27 (0.246) |
Akaike Information Criterion.
Hosmer–Lemeshow test.
After measuring the overall performance and internal validity of the model, the ability of this model to predict in-hospital death of COVID-19 patients became clear. Also, after computing the internal validity of the model, it was observed that at the cut-off point of 0.24, in the case of 73.18% of the subjects the statistical model classified the outcome correctly (validated accuracy) and at the same cut-off point it also possessed the highest amount of sensitivity (73.33%) as well as the most specificity (73.13%). Next, the ability of the model to discriminate the characteristics of the subjects presented in the general data was 87% and after measuring the internal validity of the model it was reduced to 85% (C-index: 0.85, 95% CI: 0.78–0.91). The results of the Hosmer-Lemeshow test also reveal the existence of a potential agreement between the probability of outcome predicted by the model and the actual probability of in-hospital death occurrences (P > 0.05) (Table 3).
3.4. Scoring system and its performance in prediction of in-hospital mortality of COVID-19 patients
After developing the final model, the significant variables were contemplated based on its adjusted effect and according to the results of the multivariable model, and lastly after making the score, its performance was examined (Table 4 ).
Table 4.
Description of the adjusted ß for each of significant variable and score of each variable based on multivariable logistic regression analysis.
| Variable | Adjusted ß, 95% CI | P_value | Score | Interpretation |
|---|---|---|---|---|
| Age (≥55 yrs) | 1.73 (1.17–2.29) | <0.001 | 2 | Low risk ≤2 Moderate 3–5 High risk ≥6 |
| Sex (Men) | 0.41 (0.007–0.83) | 0.046 | 1 | |
| Need to ICU/Mechanical ventilation (Yes) | 2.79 (2.31–3.26) | <0.001 | 3 | |
| Pulse Rate (>90 pulse/min) | 0.63 (0.23–1.04) | 0.002 | 1 | |
| Lymphocyte (<17%) | 0.86 (0.43–1.25) | <0.001 | 1 | |
| Red Blood Cell count (≤4, 106/L) | 0.74 (0.30–1.18) | 0.001 | 1 | |
| Troponin I (>0.03 ng/mL) | 0.56 (0.16–0.94) | 0.005 | 1 | |
| LDH (>700 U/L) | 0.52 (0.12–0.92) | 0.010 | 1 |
Subsequent to measuring the internal validity of the score, it was also determined that at the cut-off point of 0.50, this scoring system classified 79.89% of the subjects correctly (accuracy = 79.89%). Also, cross-validation method was applied to illustrate the values of AUC validated, calibration, and overall performance of the scoring system (Fig. 1, Fig. 2 ).
Fig. 1.
Receiver-operating characteristic curve (ROC curve) of scoring system in train (A) and test data (B) using cross-validation. The AUC of scoring system in train data was 82.6% and sensitivity, specificity and accuracy (optimal cut point: 0.65) were 72.34%, 86.12% and 82.49%, respectively. After performing the internal validation, observed that AUC was 79.4% and validity indices such as sensitivity, specificity and accuracy in optimal cut point of 0.50 were 64.44%, 85.07% and 79.89%, respectively.
Fig. 2.
Calibration plot of scoring system in train (A) and test data (B) using cross-validation. Calibration plot showed agreement of the predicted probability using model with the observed rate of mortality among hospitalized COVID-19 patients (P_value > 0.05).
Furthermore, once developing and validating the scoring model in addition to identifying the variables that determine in hospital-mortality were achieved, a nomogram was designated to estimate the probability of in-hospital death for each individual hospitalized COVID-19 patient. The indicated nomogram comprised of 100 points and based on each factor a specific point was assigned to it. Finally, the total points given to each patient represented the specific odds of mortality for them (Fig. 3 ).
Fig. 3.
Nomogram for the estimation of probability of in-hospital mortality among hospitalized COVID-19 patients based on significant predictors of prediction scoring model. Each predictor with a given value can be found to the points axis. The sum of these points can be referred to in the total Points axis. Total point for each patient corresponds to a predicted probability (probability axis).
4. Discussion
This study intended to design and validate a scoring system according to the available data of hospitalized COVID-19 patients. Moreover, this was to accomplish the task of classifying high-risk individuals for the purpose of disease prognosis. The results of this study concluded that this proposed scoring system was adequate ability in comparison with other suggested scores in the areas of discrimination, calibration, overall performance, and validity indices in predicting the outcome for hospitalized COVID-19 patients (validated AUC = 79.4%). Also, since the beginning of the pandemic around the world, various scores have been designed to classify and predict COVID-19 disease related outcomes. In addition, the sensitivity and specificity of these previous proposed scores have been reported anywhere between 70 and 100%.13, 14, 15 However, after the internal validation of the previous scores, the most effective has been reported to be in the study of shang et al. with 93.8% validated AUC.16 On the other hand, the scoring system in the current study can be used as a prognostic tool due to the adequacy of its validity indicators in addition to the compatibility of its variables with other scores which were established after measuring its external validity. Finally, it can be stated that the scores built around the world have varying purposes and they can measure different parameters. Furthermore, depending on the sample size, different populations, and modeling types, the above-mentioned scores also have various capabilities that can complement one another.
According to the results of the current study, age beyond 55 years has been recognized as a predictor factor of in-hospital mortality. Also, increased age consort with underlying diseases and consequently a decrease in immune function are known factors in the prognosis of patients with COVID-19. As pointed to in earlier studies as well, aging, especially above 65 years of age, has been acknowledged as a potential factor in exacerbating the disease process and thus resulting in poor prognosis of these specific patients.17 , 18 In this study, sex has been proven to play an essential role as one of the prognosis factors of COVID-19 patients. According to other previously published studies, women naturally secrete more interferon type 1, while the female hormone estradiol also provides females with extra protection against infection. In contrast, testosterone hormone unlike estradiol can limit the immune response in males. Hence, it is safe to assume that men are more susceptible to be affected by viruses than women biologically and by the way of their lifestyles. Also, women have a stronger immune response against viruses than men, although the rationality behind it is a subject of debate.19
In this study, another predictor of in-hospital death was the patient referrals who exhibited severe manifestations of the disease and therefore required admittance into ICU or mechanical ventilation assistance. Moreover, this group of patients was at higher risk than others. Furthermore, severe cases often displayed symptoms such as severe hypoxia and severe pulmonary involvement at the time of their admission into the hospital and consequently their admittance into ICU might have even posed a greater risk of co-infection to them. Thus, the aforementioned factors lead to poor prognosis for this group of patients.20
Based on the prognostic factors of the scoring system in this study, patients who complained of heart palpitations had a worse prognosis than others. In addition, it is essential to note that conditions such as fever, pain, lack of appetite, and insufficient fluid intake can similarly cause an increased pulse rate which might also be detected among COVID-19 patients21 and therefore it must be carefully taken into account by physicians. Nevertheless, people with severe pulmonary involvement may directly or indirectly develop myocarditis and heart failure, which ultimately may become the grounds for dangerous tachycardia or arrhythmia.22 Thus, it is crucial to monitor the heart rhythm accurately and on a regular basis in order to avoid the potential serious consequences among COVID-19 cases as much as possible.
In this study, the drop in red blood cell count to below normal standards (≤4, 106/L) was identified as a prognostic biomarker for COVID-19 disease. Also, red blood cells are responsible for delivering oxygen to various tissues in the body, and their deficiency causes hypoxia and a wide range of other difficulties. Anemia may also be due to a patient's specific clinical condition, such as endocrine disease, or it might be age and/or sex related. Therefore, anemia due to any reason should be treated in COVID-19 patients because this disorder will lead to weakened immune system and subsequently poor prognosis for the patients.23 On the other hand, this study considered the drop in the percentage of lymphocytes to less than 17% as an important biomarker in predicting death among COVID-19 patients. Moreover, the percentage of lymphocytes in this type of patients points to the severity of the disease and can further be attributed to the direct result of lung infection as well as increased levels of cytokines during the inflammatory process. As emphasized in other studies, lymphocyte percentage is an important indicator in predicting mortality amongst COVID-19 patients,16 , 24 and thus this factor should be considered in the clinical evaluation of patients. Also, the intracellular LDH enzyme is widely found in the body tissues, and an abnormal surge in its level indicates multi-organ failure in the tissues of the kidney, liver, heart, lungs, and other organs. Consistent other studies, in the present study, elevated LDH levels was identified as a prognostic factor for death among COVID-19 patients.24 , 25 Hence, monitoring of this factor along with other clinical and laboratory aspects is a key element in diagnosing lung failure and tissue damage. On the other hand, according to the scoring system of the current study, in agreement with other studies, an abnormal rise in troponin I levels was associated with poor prognosis in COVID-19 patients.26 , 27 Due to the fact that this virus attacks various tissues and the cardiovascular system being one of the core tissues, the upsurge of troponin I levels in these patients can serve as a warning sign to need special care. Furthermore, this must be particularly accomplished in the case of patients with the prior history of cardiovascular issues.
5. Limitation
There are always a number of limitations to the design and implementation of studies. Single center characteristic of the study added to the impossibility of examining the external validity of the constructed score, prevents the results of this study from being linked to all COVID-19 cases. At last, due to the fact that a scoring system has never been designed exclusively for COVID-19 disease within the Iranian society, the added value of the prediction scoring system is not comparable to other indicators in the country. However, the results of this study can be used as basic information to further investigate the COVID-19 in similar communities.
6. Conclusion
In this study, a scoring system according to laboratory and clinical data was designed for the purpose of early screening of COVID-19 patients. Finally, according to the recognized factors of the scoring system in the present study, routine monitoring of hospitalized patients such as hemodynamic evaluation along with blood assessment factors can be used as an effective tool in rapid rating of patients upon their admission to hospital. As a result, due to the fact that the emerging COVID-19 virus affects various patients differently in addition to its potential impact on different tissues, its early assessment based on the known factors can play a significant role for the care and triage of patients.
Ethical approval
Ethical approval has been granted by Deputy for Research Affairs, Shahid-Beheshti University of Medical Sciences (IR.SBMU.RETECH.REC.1399.678).
Funding
The study was supported via the deputy of research and technology, Shahid Beheshti University of Medical Sciences (SBMU). No specific grants from funding agencies, commercial, and non-profit sectors were received.
Authors’ contributions
M.HA and M.S reviewed the study and revised subsequent drafts. A.P was statistical consultant and reviewed study. Z.A participated in design of checklist and data gathering, N.T designed the study, analyzed and wrote the first version of manuscript. All authors read and approved the final manuscript.
Declaration of competing interest
The authors declare that they have no competing interests.
Acknowledgement
The authors would like to thank the ICU and CCU medical and nursing personnel of the Imam Hossein Hospital, Shahid Beheshti University of Medical Sciences; Dr.Ainaz Samadi, Dr.Amir Heydari, Dr.Mahboubeh Ghazanfarabadi and Dr.Moazzameh Aghamohammadi, Effat Taheri, Faezeh Nesaei, Faezeh Fakour, Ghazaleh Aman-Abadi, Ghodsi Najjari, Golnoush Mortezaei, Maedeh Sayyad and Vida Torabi who participated in data extraction and helped us in performing this project.
References
- 1.Origin of SARS-CoV-2.2020 https://www.who.int/publications/i/item/origin-337 of-sars-cov-2 Available at. Access Jan, 2021.
- 2.Lam T.T., Jia N., Zhang Y.W., et al. Identifying SARS-CoV-2-related coronaviruses in Malayan pangolins. Nature. 2020 Jul;583(7815):282–285. doi: 10.1038/s41586-020-2169-0. Epub 2020 Mar 26. PMID: 32218527. [DOI] [PubMed] [Google Scholar]
- 3.Lai C.C., Shih T.P., Ko W.C., Tang H.J., Hsueh P.R. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): the epidemic and the challenges. Int J Antimicrob Agents. 2020 Mar;55(3):105924. doi: 10.1016/j.ijantimicag.2020.105924. Epub 2020 Feb 17. PMID: 32081636; PMCID: PMC7127800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Daily reporting of COVID-19.2021. https://www.who.int/countries/irn/ Available at. Access Jan,2021.
- 5.Gu T., Chu Q., Yu Z., et al. History of coronary heart disease increased the mortality rate of patients with COVID-19: a nested case-control study. BMJ Open. 2020 Sep 17;10(9) doi: 10.1136/bmjopen-2020-038976. PMID: 32948572; PMCID: PMC7499679. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gebru A.A., Birhanu T., Wendimu E., et al. Global burden of COVID-19: situational analyis and review. Hum Antibodies. 2021 Jan 1;29(2):139–148. doi: 10.3233/HAB-200420. [DOI] [PubMed] [Google Scholar]
- 7.Wynants L., Van Calster B., Collins G.S., et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020 Apr 7:369. doi: 10.1136/bmj.m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zhang Z., Zhang H., Khanal M.K. Development of scoring system for risk stratification in clinical medicine: a step-by-step tutorial. Ann Transl Med. 2017 Nov;5(21) doi: 10.21037/atm.2017.08.22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Mishra P., Singh R.K., Nath A., et al. Transactions of the Royal Society of Tropical Medicine and Hygiene; 2021 Aug 13. A novel epidemiological scoring system for the prediction of mortality in COVID-19 patients. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Shang Y., Liu T., Wei Y., et al. Scoring systems for predicting mortality for severe patients with COVID-19. EClinicalMedicine. 2020;24 doi: 10.1016/j.eclinm.2020.100426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.DeMers D., Wachs D. StatPearls Publishing; 2020. Physiology, Mean Arterial Pressure.https://www.ncbi.nlm.nih.gov/books/NBK538226/ [Updated 2020 Aug 22]. In: StatPearls [Internet]. Treasure Island (FL) Available from: [PubMed] [Google Scholar]
- 12.Steyerberg E.W. Springer International Publishing; Cham: 2019. Clinical Prediction Models; pp. 211–213. [Google Scholar]
- 13.Zhang C., Qin L., Li K., et al. A novel scoring system for prediction of disease severity in COVID-19. Front Cell Infect Microbiol. 2020 Jun 5;10:318. doi: 10.3389/fcimb.2020.00318. PMID: 32582575; PMCID: PMC7292148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Qin L., Yang Y., Cao Q., Cheng Z., Wang X., Sun Q. A predictive model and scoring system combining clinical and CT characteristics for the diagnosis of COVID-19. Eur Radiol. 2020 Dec;30(12):6797–6807. doi: 10.1007/s00330-020-07022-1. Epub 2020 Jul 1. PMID: 32607634; PMCID: PMC7326621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Liang M., He M., Tang J., et al. Novel risk scoring system for predicting acute respiratory distress syndrome among hospitalized patients with coronavirus disease 2019 in Wuhan, China. BMC Infect Dis. 2020;20(1):960. doi: 10.1186/s12879-020-05561-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Shang Y., Liu T., Wei Y., et al. Scoring systems for predicting mortality for severe patients with COVID-19. EClinicalMedicine. 2020;24 doi: 10.1016/j.eclinm.2020.100426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Karimi A., Mansour Ghanaie R., Masjedi M.R., et al. Development of a novel scoring system for comprehensive COVID-19 patient evaluation: an evidence-based systematic review. Arch Pediatr Infect Dis. 2020;9(1) [Google Scholar]
- 18.Zhu N., Zhang D., Wang W., et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. 2020 Feb 20;382(8):727–733. doi: 10.1056/NEJMoa2001017. Epub 2020 Jan 24. PMID: 31978945; PMCID: PMC7092803. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Falahi S., Kenarkoohi A. Sex and gender differences in the outcome of patients with COVID-19. J Med Virol. 2021;93(1):151–152. doi: 10.1002/jmv.26243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Thomson R.J., Hunter J., Dutton J., et al. Clinical characteristics and outcomes of critically ill patients with COVID-19 admitted to an intensive care unit in London: a prospective observational cohort study. PLoS One. 2020 Dec 15;15(12) doi: 10.1371/journal.pone.0243710. PMID: 33320904; PMCID: PMC7737956. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Davies P., Maconochie I. The relationship between body temperature, heart rate and respiratory rate in children. Emerg Med J. 2009 Sep;26(9):641–643. doi: 10.1136/emj.2008.061598. PMID: 19700579. [DOI] [PubMed] [Google Scholar]
- 22.Babapoor-Farrokhran S., Rasekhi R.T., Gill D., Babapoor S., Amanullah A. Arrhythmia in COVID-19. SN Compr Clin Med. 2020 Aug 14:1–6. doi: 10.1007/s42399-020-00454-2. Epub ahead of print. PMID: 32838188; PMCID: PMC7426193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Taneri P.E., Gómez-Ochoa S.A., Llanaj E., et al. Anemia and iron metabolism in COVID-19: a systematic review and meta-analysis. Eur J Epidemiol. 2020;35(8):763–773. doi: 10.1007/s10654-020-00678-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Xiao L-s, Zhang W.-F., Gong M-c, et al. Development and validation of the HNC-LL score for predicting the severity of coronavirus disease 2019. EBioMedicine. 2020;57 doi: 10.1016/j.ebiom.2020.102880. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Henry B.M., Aggarwal G., Wong J., et al. Lactate dehydrogenase levels predict coronavirus disease 2019 (COVID-19) severity and mortality: a pooled analysis. Am J Emerg Med. 2020 Sep;38(9):1722–1726. doi: 10.1016/j.ajem.2020.05.073. Epub 2020 May 27. PMID: 32738466; PMCID: PMC7251362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Al Abbasi B., Torres P., Ramos-Tuarez F., Dewaswala N., Abdallah A., Chen K. Cardiac troponin-I and COVID-19: a prognostic tool for in-hospital mortality. Cardiol Res. 2020 Dec;11(6):398–404. doi: 10.14740/cr1159. Epub 2020 Oct 23. PMID: 33224386; PMCID: PMC7666590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Shah P., Doshi R., Chenna A., et al. Prognostic value of elevated cardiac troponin I in hospitalized covid-19 patients. Am J Cardiol. 2020;135:150–153. doi: 10.1016/j.amjcard.2020.08.041. [DOI] [PMC free article] [PubMed] [Google Scholar]