Abstract
The prediction of disease outcomes in COVID-19 patients in the ICU is of critical importance, and the examination of host gene expressions is a promising tool. The 29-host mRNA Inflam-matix-Severity-3b (IMX-SEV-3b) classifier has been reported to predict mortality in emergency department COVID-19 patients and surgical ICU patients. The accuracy of the IMX-SEV-3b in predicting mortality in COVID-19 patients admitted to the ICU is yet unknown. Our aim was to investigate the accuracy of the IMX-SEV-3b in predicting the ICU mortality of COVID-19 patients. In addition, we assessed the predictive performance of routinely measured biomarkers and the Sequential Organ Failure Assessment (SOFA) score as well. This was a prospective observational study enrolling COVID-19 patients who received mechanical ventilation on the ICU of the Erasmus MC, the Netherlands. The IMX-SEV-3b scores were generated by amplifying 29 host response genes from blood collected in PAXgene® Blood RNA tubes. A severity score was provided, ranging from 0 to 1 for increasing disease severity. The primary outcome was the accuracy of the IMX-SEV-3b in predicting ICU mortality, and we calculated the AUROC of the IMX-SEV-3b score, the biomarkers C-reactive protein (CRP), D-dimer, ferritin, leukocyte count, interleukin-6 (IL-6), lactate dehydrogenase (LDH), neutrophil-to-lymphocyte ratio (NLR), procalcitonin (PCT) and the SOFA score. A total of 53 patients were included between 1 March and 30 April 2020, with 47 of them being included within 72 h of their admission to the ICU. Of these, 18 (34%) patients died during their ICU stay, and the IMX-SEV-3b scores were significantly higher in non-survivors compared to survivors (0.65 versus 0.57, p = 0.05). The Area Under the Receiver Operating Characteristic Curve (AUROC) for prediction of ICU mortality by the IMX-SEV-3b was 0.65 (0.48–0.82). The AUROCs of the biomarkers ranged from 0.52 to 0.66, and the SOFA score had an AUROC of 0.81 (0.69–0.93). The AUROC of the pooled biomarkers CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT for prediction of ICU mortality was 0.81 (IQR 0.69–0.93). Further validation in a larger interventional trial of a point-of-care version of the IMX-SEV-3b classifier is warranted to determine its value for patient management.
Keywords: COVID-19, infectious diseases, virology, severity, transcriptome, classifier, prediction
1. Introduction
The emergence of the SARS-CoV-2 virus caused the global Coronavirus disease 2019 (COVID-19) pandemic and had a major impact on healthcare systems worldwide [1,2]. However, substantial variation exists in the clinical course of the disease between patients, ranging from asymptomatic infection to life-threatening Acute Respiratory Distress Syndrome (ARDS) [3,4]. It is presumed that a disparity in the level of inflammation between patients contributes to this heterogeneous clinical course [5,6,7]. Patients experiencing progressive respiratory failure frequently fulfill the Berlin criteria for moderate and severe ARDS and require admission to the intensive care unit (ICU) [8,9]. These patients often undergo prolonged ICU stays and have high mortality rates [10,11].
The identification of patients at risk of a severe outcome is of great importance, as clinicians may be able to act on signs of clinical deterioration at an earlier stage with intensified monitoring and targeted therapies. Furthermore, it enables clinicians to provide more precise information to patients and their relatives about the prognosis of the disease. Previous research has mainly focused on protein-based biomarkers for COVID-19 prognostication [12,13,14]. Several studies have found that elevated levels of biomarkers of inflammation and coagulation, including procalcitonin (PCT), C-reactive protein (CRP), Interleukin-6 (IL-6), D-dimer and lactate dehydrogenase (LDH) are associated with a severe outcome [15,16,17,18,19]. Also, several risk scores, mainly based on vital signs and laboratory parameters, have been studied to predict outcomes in COVID-19, such as the Sequential Organ Failure Assessment (SOFA) score, the Acute Physiology and Chronic Health Evaluation IV (APACHE-IV) score and the 4C (Coronavirus Clinical Characterisation Consortium) Mortality Score [20,21,22,23,24]. However, the potential for timely intervention may be lost when vital signs have already worsened and protein-based biomarkers are elevated. Furthermore, the prognostic accuracy of single protein-based serum biomarkers in predicting mortality is often insufficient [14,25], therefore supporting the need for the exploration of alternative tools involving underlying pathways of the immune system.
A relatively novel approach for predicting disease severity is the examination of host gene expression by transcriptome analyses. During infection, various genes are upregulated and downregulated in response to the presence of pathogens and immune system activation [26]. The measurement of gene expression levels provides valuable insights into the complex molecular mechanisms underlying the immune response to infection. Gene expression profiles associated with poor outcomes have been identified in various contexts and diseases, including patients with sepsis [27,28]. The Inflammatix Severity-3b (IMX-SEV-3b) classifier quantifies 29-mRNA host response genes from peripheral blood, which encompasses genes associated with diverse aspects of the immune response, such as the interferon and antigen presentation pathways [29]. For the initial selection of the 29 mRNAs databases, heterogeneous cohorts from diverse geographies, age groups, ethnicities, diagnoses, and settings of care were used, including inpatient wards, emergency departments, outpatient departments, and intensive care units. This classifier has proven to be a reliable predictor of severe outcomes in COVID-19 patients in the emergency department [30]. Furthermore, the IMX-SEV-3b successfully predicted 30-day mortality in surgical ICU patients with sepsis or suspected sepsis [31]. Whether IMX-SEV-3b will be able to predict mortality in COVID-19 ARDS patients in the ICU has not yet been studied. Therefore, the goal of this study is to investigate the accuracy of the IMX-SEV-3b classifier in predicting mortality in COVID-19 patients admitted to the ICU.
2. Materials and Methods
2.1. Study Design
This study was a prospective single-center cohort study conducted in the Netherlands. We included patients admitted to the ICU of Erasmus University Medical Center in Rotterdam with a confirmed COVID-19 infection between 1 March 2020 and 30 April 2020. The study was approved by the local Medical Ethics Review Committee under protocol number MEC-2017-417 and conducted according to the principles of the World Medical Association Declaration of Helsinki.
Eligibility for inclusion in this study was defined as follows:
2.2. Inclusion and Exclusion Criteria
Patients who were at least 18 years of age.
Patients who were admitted to the ICU with a Polymerase Chain Reaction (PCR)-confirmed COVID-19 infection.
Patients or their legal representatives gave informed consent for inclusion and PAXgene Blood RNA whole blood draw, as well as analysis and shipment of their sample to Inflammatix Inc., Sunnyvale, CA, USA.
There were no exclusion criteria.
2.3. Data Collection
Patient data, including demographics, body mass index and comorbidities, were collected at baseline. Biomarker levels, including CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT, were evaluated and collected on the day of inclusion as close to 6 AM as feasible. The APACHE-IV score was recorded from the electronic health record on the day of ICU admission and the SOFA score was recorded daily. Patients were monitored until their discharge from the ICU or until their death in the ICU.
2.4. Primary Outcome
The primary outcome of this study was the accuracy of the IMX-SEV-3b classifier in predicting mortality during the ICU stay, expressed as Area Under the Receiver Operating Characteristic Curve (AUROC). We compared this to the accuracy of the biomarkers CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT, and the SOFA score, which were all evaluated on the day of inclusion, in predicting ICU mortality.
2.5. PAXgene Collection and Amplification of Target Genes
Whole blood (2.5 mL) was collected in PAXgene Blood RNA tubes (PreAnalytics, Hombrechtikon, Switzerland) on the day of inclusion, and subsequently aliquoted and stored at −80 °C until analysis. To perform target gene amplification and generate IMX-SEV-3b classifier results, the samples were shipped on dry ice to the Inflammatix laboratory (Burlingame, CA, USA) in batches, following the procedure previously described [29]. RNA was extracted from PAXgene tubes using the RNeasy® Plus Micro Kit (QIAGEN, Hilden, Germany) on the QiaCube® Connect instrument. The NanoString nCounter® SPRINT profiler (NanoString, Seattle, WA, USA) was then used to quantify the 29 host target mRNAs, as well as four housekeeping genes for normalization (CDIPT, KPNA6, RREB1 and YWHAB) from 150 ng of isolated RNA. RNA concentrations were determined for 29 genes of various aspects of the immune response and included CEACAM1, ZDHHC19, C9orf95, GNA15, BATF, C3AR1, KIAA1370, TGFBI, MTCH1, RPGRIP1, HLA-DPB1, HK3, TNIP1, GPAA1, CTSB, IFI27, JUP, LAX1, DEFA4, CD163, RGS1, PER1, HIF1A, SEPP1, C11orf74, CIT, LY86, TST and KCNJ2 [29]. Operators at Inflammatix were blinded to all other study results, and the NanoString data were subjected to direct application of the SEV-3b classifier, blinded to clinical outcomes.
2.6. The IMX-SEV-3b Classifier
The machine learning algorithm IMX-SEV-3b reads and interprets the results of gene amplification. The results of the IMX-SEV-3b classifier were provided as absolute numerical scores between 0 and 1 for increasing severity prediction. To aid clinical usability, predefined thresholds categorized severity scores into five interpretation bands labeled ‘Very Low’, ’Low’, ’Moderate’, ‘High’ and ‘Very High’ illness severity. For each interpretation band, specificity and sensitivity were calculated to show accuracy.
2.7. Statistical Analysis
Continuous variables that were normally distributed are reported as a mean with standard deviation (SD), and non-normally distributed variables as a median with interquartile range (IQR). Categorical variables are shown as proportions. We compared differences between groups with a Chi-Square test and Student’s t-test or Mann–Whitney U test as appropriate. p-values of 0.05 or less were yielded statistically significant. For the primary analysis, we calculated the AUROC of the IMX-SEV-3b score and the biomarkers CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT on the day of inclusion for ICU mortality. To test the performance of an important clinical score used in the ICU, we also calculated the AUROC of the SOFA score on the day of inclusion for ICU mortality. We used a multivariate logistic regression model combining the biomarkers in a composite biomarker model and calculated the AUROC of this logistic regression model. Finally, we calculated the sensitivity, specificity, and likelihood ratios for the 5 interpretation bands generated by the IMX-SEV-3b classifier. We used R (version 4.0.1) for statistical analysis.
3. Results
Between 1 March 2020 and 30 April 2020, a total of 152 patients were admitted to the ICU with a PCR-confirmed SARS-CoV-2 infection. Of these patients, 53 provided informed consent for the inclusion and collection of a PAXgene Blood RNA tube, and they all met the criteria outlined in the Berlin definition of ARDS [8]. A total of 80% of the patients were initially admitted to another hospital in the Netherlands and transferred to the Erasmus MC. The median time from the first day of hospital admission to ICU admission at Erasmus MC was 5 days. The majority of patients (27 out of 53) were included in our study within 24 h after ICU admission, and 47 out of 53 patients were included within 72 h after ICU admission. The median age was 66 years (IQR 60.0–72.0); 83% of patients were male, and the median BMI was 27.4 kg/m2 (IQR 25.3–30.2) (Table 1).
Table 1.
Patient Characteristics | All Patients | Survivors | Non-Survivors | p-Value |
---|---|---|---|---|
n = 53 | n = 35 | n = 18 | ||
Age, median [IQR] | 66.0 (60.0–72.0) | 65.0 (58.0–71.0) | 69.0 (64.3–72.8) | 0.14 |
Sex, male | 44 (83.0%) | 29 (82.9%) | 15 (83.3%) | 1 |
BMI, median [IQR] | 27.4 (25.3–30.2) | 27.5 (25.5–30.1) | 26.3 (25.1–29.9) | 0.98 |
Missing | 1 | 1 | 0 | |
APACHE-IV (%) * | 18.20 (11.10–30.20) | 12.00 (9.00–19.10) | 32.0 (24.40–50.80) | <0.001 |
Comorbidities Cardiovascular disease |
6 (11.3%) | 3 (8.6%) | 3 (16.7%) | 0.67 |
Pulmonary disease | 6 (11.3%) | 4 (11.4%) | 2 (11.1%) | 1 |
Neurological disease | 2 (3.8%) | 1 (2.9%) | 1 (5.6%) | 1 |
Renal disease | 0 (100%) | 0 (100%) | 0 (100%) | 1 |
Diabetes mellitus | 11 (20.8%) | 5 (14.3%) | 6 (33.3%) | 0.21 |
Immunodeficiency | 4 (7.5%) | 3 (8.6%) | 1 (5.6%) | 1 |
Autoimmune disease | 4 (7.5%) | 3 (8.6%) | 1 (5.6%) | 1 |
BMI: body mass index; APACHE: acute physiology, age, and chronic health evaluation. * Estimated Mortality Rate APACHE-IV Score.
A total of 29 (54.7%) patients were diagnosed with a pulmonary embolism during their hospital stay, and the incidence was similar between non-survivors (61.1%) and survivors (51.4%) (p = 0.704). None of the patients had been subjected to corticosteroid therapy for COVID-19 indications prior to their admission to intensive care. In total, 18 patients (34%) died during their ICU stay. There were no significant differences in age, sex, BMI and comorbidity between survivors and non-survivors (Table 1). Similarly, the results of laboratory tests on the day of inclusion in the study, CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT did not differ significantly between survivors and non-survivors (Table 2).
Table 2.
Characteristics | All Patients | Survivors | Non-Survivors | p-Value |
---|---|---|---|---|
n = 53 | n = 35 | n = 18 | ||
CRP (mg/L) | 277 (178–345) | 304 (174–341) | 210 (178–347) | 0.74 |
D-dimer (mg/L) | 2.48 (1.38–5.22) | 2.42 (1.18–3.96) | 3.86 (1.81–8.47) | 0.17 |
Ferritin (mg/L) | 1580 (974–2680) | 1660 (1130–2730) | 1300 (989–2240) | 0.33 |
Leukocyte count (×109/L) | 8.74 (6.53–10.7) | 8.31 (6.47–9.53) | 10.30 (7.57–12.30) | 0.06 |
IL-6 (pg/mL) | 161 (88–307) | 143 (91.5–246) | 197 (87.3–395) | 0.36 |
LDH (U/L) | 322 (271–394) | 320 (264–360) | 341 (293–456) | 0.06 |
NLR | 7.45 (4.78–10.7) | 6.72 (4.69–9.92) | 8.85 (5.03–12.40) | 0.22 |
PCT (ng/mL) | 0.97 (0.470–2.70) | 0.77 (0.36–2.04) | 1.44 (0.90–3.20) | 0.28 |
SOFA score | 7.00 (6.00–10.00) | 7.00 (6.00–8.00) | 11.00 (9.00–11.00) | <0.001 |
IMX-SEV-3b score * | 0.582 ± 0.13 | 0.575 ± 0.11 | 0.660 ± 0.15 | 0.050 |
CRP: C-reactive protein. LDH: lactate dehydrogenase. IL-6: interleukin-6. NLR: neutrophil-to-lymphocyte ratio. PCT: procalcitonin. SOFA: Sequential Organ Failure Assessment. * Mean ± SD; all others are reported as median (IQR).
The mean IMX-SEV-3b was significantly lower (0.58) in survivors versus non-survivors (0.66) (p = 0.050). The AUROC of the IMX-SEV-3b for the prediction of ICU mortality was 0.65 (IQR 0.48–0.82). The calculated AUROCs of the biomarkers are shown in Table 3. The AUROC of the pooled biomarkers CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT for the prediction of ICU mortality was 0.81 (IQR 0.69–0.93). The SOFA score had an AUROC of 0.81 (IQR 0.69–0.93) for the prediction of ICU mortality, and the score was significantly lower (6.00) in survivors compared to non-survivors (9.50).
Table 3.
Test/Biomarker/Clinical Score | AUROC (95% CI) |
---|---|
CRP (mg/L) | 0.52 (0.34–0.70) |
D-dimer (mg/L) | 0.62 (0.45–0.79) |
Ferritin (mg/L) | 0.58 (0.42–0.75) |
Leukocyte count (×109/L) | 0.66 (0.48–0.83) |
IL-6 (pg/mL) | 0.58 (0.40–0.76) |
LDH (U/L) | 0.56 (0.37–0.74) |
NLR | 0.60 (0.43–0.78) |
PCT (ng/mL) | 0.59 (0.43–0.76) |
IMX-SEV-3b | 0.65 (0.48–0.82) |
SOFA score | 0.81 (0.69–0.93) |
Pooled biomarker model | 0.81 (0.69–0.93) |
CRP: c-reactive protein. IL-6: interleukin-6. LDH: lactate dehydrogenase. NLR: neutrophil-to-lymphocyte ratio. PCT: procalcitonin. SOFA: Sequential Organ Failure Assessment. The pooled biomarker model included the biomarkers CRP, D-dimer, ferritin, leukocyte count, IL-6, LDH, NLR and PCT.
When stratifying patients into the IMX-SEV-3b illness severity interpretation bands, 1 patient (2%) fell into the Very Low, 31 (58%) into the Low, 15 (28%) into the Moderate, 2 (4%) into the High and 4 (8%) into the Very High severity category (Table 4). All patients in the Very High illness severity band died in the ICU. The specificity of the ‘rule-in’ Very High, High and Moderate interpretation bands was, respectively, 100%, 94% and 74%, and the sensitivity of the ‘rule-out’ Very Low and Low interpretation bands was 100% and 56% (Table 4).
Table 4.
IMX-SEV-3b Severity Score | Survival Status | IMX-SEV-3b Performance per Band | |||||
---|---|---|---|---|---|---|---|
Survivor | Non-Survivor | % Patients in Band |
Sensitivity | Specificity | Likelihood Ratio | ||
IMX-SEV-3b category | Very High | 0 | 4 | 8% | 22% | 100% | Inf. |
High | 2 | 0 | 4% | 0% | 94% | 0.00 | |
Moderate | 9 | 6 | 28% | 33% | 74% | 1.30 | |
Low | 23 | 8 | 58% | 56% | 66% | 0.68 | |
Very Low | 1 | 0 | 2% | 100% | 3% | 0.00 |
Inf: infinite.
We performed a sub-analysis comparing AUROCs in the 21 patients who were enrolled within 5 days of hospital admission vs. those enrolled after 5 days of hospital admission (Table 5). Within the group of patients enrolled within 5 days the mortality rate was 28.6%, and within the group of patients enrolled after 5 days the mortality rate was 37.5%. The AUROCs of the IMX-SEV-3b and that of the biomarkers, CRP, D-dimer, IL-6, ferritin, leukocyte count and LDH are shown in Table 5.
Table 5.
Test/Biomarker/Clinical Score | AUROC (95% CI) Patients Admitted within 5 Days (n = 21) |
AUROC (95% Cl) Patients Admitted after 5 Days (n = 32) |
---|---|---|
CRP (mg/L) | 0.59 (0.26–0.92) | 0.60 (0.39–0.81) |
D-dimer (mg/L) | 0.72 (0.39–1.00) | 0.56 (0.35–0.77) |
Ferritin (mg/L) | 0.60 (0.28–0.92) | 0.59 (0.38–0.80) |
Leukocyte count (×109/L) | 0.71 (0.44–0.97) | 0.62 (0.38–0.86) |
IL-6 (pg/mL) | 0.67 (0.33–1.00) | 0.55 (0.33–0.77) |
LDH (U/L) | 0.79 (0.56–1.00) | 0.58 (0.37–0.80) |
NLR | 0.38 (0.10–0.66) | 0.62 (0.40–0.84) |
PCT (ng/mL) | 0.56 (0.29–0.84) | 0.63 (0.41–0.84) |
IMX-SEV-3b | 0.72 (0.43–1.00) | 0.60 (0.37–0.83) |
SOFA score | 0.74 (0.49–1.00) | 0.86 (0.73–0.99) |
4. Discussion
The clinical circumstances experienced during the first wave of the pandemic posed unique challenges due to the presence of the unfamiliar SARS-CoV-2 virus and the resulting capacity constraints. The high mortality rate highlighted the pressing need for improved tools that are able to predict which patients are at the highest risk of mortality in the ICU [32,33]. In our study, we investigated the accuracy of the host response classifier, IMX-SEV-3b, in predicting the ICU mortality of COVID-19 patients. Subsequently, we conducted a similar evaluation for several routinely measured biomarkers and the SOFA score. By achieving a non-significant AUROC for mortality prediction in this cohort, the IMX-SEV-3b classifier did perform below our initial expectations.
In relation to the existing literature on the IMX-SEV-3b classifier, the results of our study demonstrated lower performance than previously published. In comparison, Galtung et al. observed an AUROC of 0.84 for predicting in-hospital mortality in their cohort of patients presenting to the emergency department with suspected acute infections [30], and Brakenridge et al. observed an AUROC of 0.81 for predicting 30-day mortality in a surgical intensive care unit [31]. Given the fact that the IMX-SEV-3b has been scarcely tested within other ICU populations, it could be contemplated that the classifier’s performance might be more pronounced during an earlier stage of infection, such as when patients initially present at the emergency department.
However, since the performance of the other biomarkers in predicting mortality was also poor, one of the explanations might be related to the characteristics of the patient cohort in our study. Mainly due to capacity constraints in other hospitals, 80% of the patients were transferred from another hospital to the Erasmus MC, resulting in a median duration of 5 days from their initial hospital admission to study inclusion. Additionally, 51% of the patients were included within 24 h after ICU admission, with the day of inclusion varying among patients. It is presumed that the timing of ICU admission typically aligns with the point at which patients experience clinical deterioration necessitating invasive mechanical ventilation. A state of hyperinflammation present within both the pulmonary and circulatory systems plays a contributing role. Since the host immune response reduces over time, as illustrated by a decrease in cytokine levels and a quantitative reduction in specific mRNA levels, this may suggest that the optimal time window for the determination might have been missed for some of our patients. This could potentially have resulted in a lower predictive value of immune response-based tools, like the IMX-SEV-3b classifier and the biomarkers of inflammation. In this regard, we performed a sub-analysis comparing the AUROCs of patients who were enrolled within 5 days of hospital admission vs. those enrolled after 5 days of hospital admission. Although there seems to be a trend of improvement in the group of patients enrolled within 5 days supporting this hypothesis, it is noteworthy that the AUROC of the IMX-SEV-3b remains statistically insignificant within this subgroup. Likewise, the AUROCs of the biomarkers fail to attain statistical significance in this group, with the exception of LDH.
Corticosteroids, which are nowadays the cornerstone in the treatment of COVID-19, are also known to have an effect on inflammatory profiles. However, at the time of our study, guidelines had not yet incorporated corticosteroid treatment for COVID-19 management, so potential influence on the inflammatory profile is presumed to be absent. As corticosteroids were not included in treatment, this could raise the question of whether the results of our study can be extrapolated to COVID-19 patients treated with corticosteroids. Taking into account the fact that the cohorts used for the selection of the 29 mRNAs and the development of the IMX-SEV-3b classifier were diverse and included patients on immunosuppressive therapy, including corticosteroids, we expect little concern regarding extrapolation.
Many prior studies have indicated that integrating multiple biomarkers into a model enhances the predictive accuracy of ICU outcomes. The analysis of the composite biomarker model showed a substantial improvement, with a statistically significant AUROC. However, this model is highly at risk of overfitting due to the large number of predictors and a cohort with only 53 patients, and therefore, caution must be applied when interpreting this. The SOFA score showed a statistically significant AUROC for mortality prediction in this cohort. One possible reason might be that the SOFA score relies on parameters of organ failure rather than markers of inflammation, which could be less time sensitive in its predictive value. Although the SOFA score is the most well-known score in ICU practice, it is developed to sequentially evaluate the patient’s condition and is not able to provide instant individual predictions about mortality. In contrast, the illness severity interpretation bands provided by the IMX-SEV-3b classifier may allow for earlier clinical decision-making; the most actionable outer bands showed high sensitivity and specificity.
This study has several limitations. Firstly, the sample size is limited to 53 patients from a single center, and therefore, caution must be applied to the generalizability of the results. In light of the elevated risk of overfitting associated with the small sample size, we refrained from conducting statistical comparisons between the performance of the IMX-SEV-3b, biomarkers and SOFA score. Secondly, the inclusion of patients did not uniformly occur on the same day after admission, and their clinical trajectory before ICU admission varied highly. This contributed to the heterogeneity of the cohort. However, it is important to acknowledge that this was the prevailing clinical situation during the first wave of the COVID-19 pandemic that we encountered. Thirdly, the IMX-SEV-3b classifier is not feasible for clinical implementation yet, because the results are currently analyzed by the test manufacturer on a reference platform (NanoString). However, the upcoming availability of the point-of-care test directly from blood with a rapid turnaround time is expected to provide clinicians with immediate usable results.
The outcomes of our study underline the challenges of outcome prediction within a heterogeneous cohort of critically ill ICU patients. Moreover, these findings warrant further studies on the predictive value of the IMX-SEV-3b in the ICU. We acknowledge that the number of COVID-19 patients admitted to ICUs has significantly decreased. As the IMX-SEV-3b was validated using a 30-day all-cause in-hospital mortality endpoint [29], the classifier may be applied to other cohorts of critically ill patients with infectious diseases in the ICU. By incorporating repeated measurements of transcriptomic analyses in future studies, valuable insights into the prognostic value of transcriptome analyses over time may be provided, and critical windows for accurate prediction in the ICU may be identified [34].
5. Conclusions
In this observational pilot study, the mean IMX-SEV-3b score of the 29-gene host response classifier was significantly lower (0.58) in survivors compared to non-survivors (0.66). The AUROCs of the IMX-SEV-3b and the assessed biomarkers failed to achieve statistical significance in predicting mortality within this COVID-19 patient cohort, in contrast to the SOFA score, which did exhibit statistical significance. Further prospective studies are required to test the IMX-SEV-3b classifier in the ICU.
Author Contributions
K.D. and K.T.-M. were involved in the conception or design of the manuscript, performed the analysis and interpretation of the data and drafted the manuscript. K.D., K.T.-M., S.C.M.S., O.L., J.A.H., V.A.S.H.D., D.G., E.C.M.v.G. and H.E. were involved in the critical revision of the manuscript and final approval of the manuscript. All authors have read and agreed to the published version of the manuscript.
Institutional Review Board Statement
This study was approved by the local Medical Ethics Re-view Committee under protocol number MEC-2017-417 and conducted according to the principles of the World Medical Association Declaration of Helsinki.
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Conflicts of Interest
K.D., K.T.-M., S.C.M.S., J.A.H., V.A.S.H.D., D.G., E.C.M.v.G. and H.E. report no conflict of interest. O.L. is an employee of Inflammatix Inc.
Funding Statement
This research received no external funding. Inflammatix Inc., Sunnyvale, CA, USA provided Paxgene ® Blood RNA tubes and performed the IMX-SEV-3b test.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
References
- 1.WHO Coronavirus (COVID-19) Dashboard World Health Organization. [(accessed on 16 March 2023)]. Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019.
- 2.Wang H., Paulson K.R., Pease S.A., Watson S., Comfort H., Zheng P., Aravkin A.Y., Bisignano C., Barber R.M., Alam T., et al. Estimating excess mortality due to the COVID-19 pandemic: A systematic analysis of COVID-19-related mortality, 2020–2021. Lancet. 2022;399:1513–1536. doi: 10.1016/S0140-6736(21)02796-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Zhou F., Yu T., Du R., Fan G., Liu Y., Liu Z., Xiang J., Wang Y., Song B., Gu X., et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet. 2020;395:1054–1062. doi: 10.1016/S0140-6736(20)30566-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Network C-IGobotR, the C-ICUI Clinical characteristics and day-90 outcomes of 4244 critically ill adults with COVID-19: A prospective cohort study. Intensive Care Med. 2021;47:60–73. doi: 10.1007/s00134-020-06294-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Siddiqi H.K., Mehra M.R. COVID-19 illness in native and immunosuppressed states: A clinical-therapeutic staging proposal. J. Heart Lung Transplant. 2020;39:405–407. doi: 10.1016/j.healun.2020.03.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Shi Y., Wang Y., Shao C., Huang J., Gan J., Huang X., Bucci E., Piacentini M., Ippolito G., Melino G. COVID-19 infection: The perspectives on immune responses. Cell Death Differ. 2020;27:1451–1454. doi: 10.1038/s41418-020-0530-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Torres Acosta M.A., Singer B.D. Pathogenesis of COVID-19-induced ARDS: Implications for an ageing population. Eur. Respir. J. 2020;56:2002049. doi: 10.1183/13993003.02049-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Force A.D.T., Ranieri V.M., Rubenfeld G.D., Thompson B.T., Ferguson N.D., Caldwell E., Fan E., Camporota L., Slutsky A.S. Acute respiratory distress syndrome: The Berlin Definition. JAMA. 2012;307:2526–2533. doi: 10.1001/jama.2012.5669. [DOI] [PubMed] [Google Scholar]
- 9.Grasselli G., Cattaneo E., Florio G., Ippolito M., Zanella A., Cortegiani A., Huang J., Pesenti A., Einav S. Mechanical ventilation parameters in critically ill COVID-19 patients: A scoping review. Crit. Care. 2021;25:115. doi: 10.1186/s13054-021-03536-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chang R., Elhusseiny K.M., Yeh Y.-C., Sun W.-Z. COVID-19 ICU and mechanical ventilation patient characteristics and outcomes—A systematic review and meta-analysis. PLoS ONE. 2021;16:e0246318. doi: 10.1371/journal.pone.0246318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Grasselli G., Zangrillo A., Zanella A., Antonelli M., Cabrini L., Castelli A., Cereda D., Coluccello A., Foti G., Fumagalli R., et al. Baseline Characteristics and Outcomes of 1591 Patients Infected with SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy. JAMA. 2020;323:1574–1581. doi: 10.1001/jama.2020.5394. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Karimi A., Shobeiri P., Kulasinghe A., Rezaei N. Novel Systemic Inflammation Markers to Predict COVID-19 Prognosis. Front. Immunol. 2021;12:741061. doi: 10.3389/fimmu.2021.741061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Gibot S., Lafon T., Jacquin L., Lefevre B., Kimmoun A., Guillaumot A., Losser M.-R., Douplat M., Argaud L., De Ciancio G., et al. Soluble TREM-1 plasma concentration predicts poor outcome in COVID-19 patients. Intensive Care Med. Exp. 2023;11:51. doi: 10.1186/s40635-023-00532-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Narvel H., Sayed A., Narvel N., Yakkali S., Katchi T. Do Certain Biomarkers Predict Adverse Out-Comes in Coronavirus Disease 2019? J. Clin. Med. Res. 2021;13:195–203. doi: 10.14740/jocmr4254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Battaglini D., Lopes-Pacheco M., Castro-Faria-Neto H.C., Pelosi P., Rocco P.R.M. Laboratory Biomarkers for Diagnosis and Prognosis in COVID-19. Front. Immunol. 2022;13:857573. doi: 10.3389/fimmu.2022.857573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Malik P., Patel U., Mehta D., Patel N., Kelkar R., Akrmah M., Gabrilove J.L., Sacks H. Biomarkers and outcomes of COVID-19 hospitalisations: Systematic review and meta-analysis. BMJ Evid.-Based Med. 2021;26:107–108. doi: 10.1136/bmjebm-2020-111536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Feng T., James A., Doumlele K., White S., Twardzik W., Zahid K., Sattar Z., Ukponmwan O., Nakeshbandi M., Chow L., et al. Procalcitonin Levels in COVID-19 Patients Are Strongly Associated with Mortality and ICU Acceptance in an Underserved, Inner City Population. Medicina. 2021;57:1070. doi: 10.3390/medicina57101070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Zhang W., Sang L., Shi J., Zhong M., Jiang L., Song B., Kang L., Zhang Y., Zhang D., Yu Y., et al. Association of D-dimer elevation with inflammation and organ dysfunction in ICU patients with COVID-19 in Wuhan, China: A retrospective observational study. Aging. 2021;13:4794–4810. doi: 10.18632/aging.202496. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gorham J., Moreau A., Corazza F., Peluso L., Ponthieux F., Talamonti M., Izzi A., Nagant C., Djangang N.N., Garufi A., et al. Interleukine-6 in critically ill COVID-19 patients: A retrospective analysis. PLoS ONE. 2020;15:e0244628. doi: 10.1371/journal.pone.0244628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Knight S.R., Ho A., Pius R., Buchan I., Carson G., Drake T.M., Dunning J., Fairfield C.J., Gamble C., Green C.A., et al. Risk stratification of patients admitted to hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: Development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339. doi: 10.1136/bmj.m3339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Wynants L., Van Calster B., Collins G.S., Riley R.D., Heinze G., Schuit E., Bonten M.M.J., Dahly D.L., Damen J.A., Debray T.P.A., et al. Prediction models for diagnosis and prognosis of COVID-19: Systematic review and critical appraisal. BMJ. 2020;369:m1328. doi: 10.1136/bmj.m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Doganay F., Ak R. Performance of the CURB-65, ISARIC-4C and COVID-GRAM scores in terms of severity for COVID-19 patients. Int. J. Clin. Pract. 2021;75:e14759. doi: 10.1111/ijcp.14759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Vandenbrande J., Verbrugge L., Bruckers L., Geebelen L., Geerts E., Callebaut I., Gruyters I., Heremans L., Dubois J., Stessel B. Validation of the Acute Physiology and Chronic Health Evaluation (APACHE) II and IV Score in COVID-19 Patients. Crit. Care Res. Pract. 2021;2021:5443083. doi: 10.1155/2021/5443083. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Beigmohammadi M.T., Amoozadeh L., Motlagh F.R., Rahimi M., Maghsoudloo M., Jafarnejad B., Eslami B., Salehi M.R., Zendehdel K. Mortality Predictive Value of APACHE II and SOFA Scores in COVID-19 Patients in the Intensive Care Unit. Can. Respir. J. 2022;2022:5129314. doi: 10.1155/2022/5129314. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kukoč A., Mihelčić A., Miko I., Romić A., Pražetina M., Tipura D., Drmić Ž., Čučković M., Ćurčić M., Blagaj V., et al. Clinical and laboratory predictors at ICU admission affecting course of illness and mortality rates in a tertiary COVID-19 center. Heart Lung. 2022;53:1–10. doi: 10.1016/j.hrtlng.2022.01.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Scicluna B.P., van Vught L.A., Zwinderman A.H., Wiewel M.A., Davenport E.E., Burnham K.L., Nurnberg P., Schultz M.J., Horn J., Cremer O.L., et al. Classification of patients with sepsis according to blood genomic endotype: A prospective cohort study. Lancet Respir. Med. 2017;5:816–826. doi: 10.1016/S2213-2600(17)30294-1. [DOI] [PubMed] [Google Scholar]
- 27.Davenport E.E., Burnham K.L., Radhakrishnan J., Humburg P., Hutton P., Mills T.C., Rautanen A., Gordon A.C., Garrard C., Hill A.V.S., et al. Genomic landscape of the individual host response and outcomes in sepsis: A prospective cohort study. Lancet Respir. Med. 2016;4:259–271. doi: 10.1016/S2213-2600(16)00046-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Chen I.C., Chen H.H., Jiang Y.H., Hsiao T.H., Ko T.M., Chao W.C. Whole transcriptome analysis to explore the impaired immunological features in critically ill elderly patients with sepsis. J. Transl. Med. 2023;21:141. doi: 10.1186/s12967-023-04002-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Mayhew M.B., Buturovic L., Luethy R., Midic U., Moore A.R., Roque J.A., Shaller B.D., Asuni T., Rawling D., Remmel M., et al. A generalizable 29-mRNA neural-network classifier for acute bacterial and viral infections. Nat. Commun. 2020;11:1177. doi: 10.1038/s41467-020-14975-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Galtung N., Diehl-Wiesenecker E., Lehmann D., Markmann N., Bergström W.H., Wacker J., Liesenfeld O., Mayhew M., Buturovic L., Luethy R., et al. Prospective validation of a transcriptomic severity classifier among patients with suspected acute infection and sepsis in the emergency department. Eur. J. Emerg. Med. 2022;29:357–365. doi: 10.1097/MEJ.0000000000000931. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Brakenridge S.C., Chen U.I., Loftus T., Ungaro R., Dirain M., Kerr A., Zhong L., Bacher R., Starostik P., Ghita G., et al. Evaluation of a Multivalent Transcriptomic Metric for Diagnosing Surgical Sepsis and Estimating Mortality among Critically Ill Patients. JAMA Netw. Open. 2022;5:e2221520. doi: 10.1001/jamanetworkopen.2022.21520. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Meijs D.A., van Kuijk S.M., Wynants L., Stessel B., Mehagnoul-Schipper J., Hana A., Scheeren C.I., Bergmans D.C., Bickenbach J., Laenen M.V., et al. Predicting COVID-19 prognosis in the ICU remained challenging: External validation in a multinational regional cohort. J. Clin. Epidemiol. 2022;152:257–268. doi: 10.1016/j.jclinepi.2022.10.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Leoni M.L.G., Lombardelli L., Colombi D., Bignami E.G., Pergolotti B., Repetti F., Villani M., Bellini V., Rossi T., Halasz G., et al. Prediction of 28-day mortality in critically ill patients with COVID-19: Development and internal validation of a clinical prediction model. PLoS ONE. 2021;16:e0254550. doi: 10.1371/journal.pone.0254550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Brakenridge S.C., Starostik P., Ghita G., Midic U., Darden D., Fenner B., Wacker J., Efron P.A., Liesenfeld O., Sweeney T.E., et al. A Transcriptomic Severity Metric That Predicts Clinical Outcomes in Critically Ill Surgical Sepsis Patients. Crit. Care Explor. 2021;3:e0554. doi: 10.1097/cce.0000000000000554. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.