Abstract
COVID-19 patients are oftentimes over- or under-treated due to a deficit in predictive management tools. This study reports derivation of an algorithm that integrates the host levels of TRAIL, IP-10, and CRP into a single numeric score that is an early indicator of severe outcome for COVID-19 patients and can identify patients at-risk to deteriorate.
394 COVID-19 patients were eligible; 29% meeting a severe outcome (intensive care unit admission/non-invasive or invasive ventilation/death). The score’s area under the receiver operating characteristic curve (AUC) was 0.86, superior to IL-6 (AUC 0.77; p = 0.033) and CRP (AUC 0.78; p < 0.001). Likelihood of severe outcome increased significantly (p < 0.001) with higher scores. The score differentiated severe patients who further deteriorated from those who improved (p = 0.004) and projected 14-day survival probabilities (p < 0.001).
The score accurately predicted COVID-19 patients at-risk for severe outcome, and therefore has potential to facilitate timely care escalation and de-escalation and appropriate resource allocation.
Keywords: COVID-19 disease severity prediction, Host-immune protein score, Emergency Department, SARS-CoV-2
1. Introduction
Globally as of January 2022, there have been over 400 million confirmed cases of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and 5.8 million deaths attributed to the resulting coronavirus disease 2019 (COVID-19) (https://coronavirus.jhu.edu/map.html). With vaccination campaigns taking time to implement, hospitals world-wide are still dealing with COVID-19 patients. Moreover, global herd immunity seems improbable given vaccination inequity and hesitancy combined with the emergence of variants [1], [2]. Accordingly, it is predicted that SARS-CoV-2 will become endemic [3].
An unusual feature of SARS-CoV-2 is that it causes a broad spectrum of disease severity, ranging from asymptomatic infection to critical illness. While the vast majority of patients with SARS-CoV-2 infection develop mild to moderate disease (https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html), without requiring supplemental oxygen therapy or hospitalization, up to 20% of patients experience life-threatening disease [4]. The advanced acute phase of SARS-CoV-2 infection includes a dysregulated immune response that can rapidly progress to severe symptoms and critical complications including respiratory failure and death [5], [6], [7], [8], [9], [10], [11], [12], [13]. Despite major advances in COVID-19 patient management, predicting which patients have increased risk for severe outcome remains challenging. Early prediction in COVID-19 is of particular importance because high-risk patients may benefit from earlier escalation of care (i.e., intensive care unit (ICU) placement, immunomodulation treatment) [3], whereas low-risk patients may benefit from avoiding unnecessary treatments or hospital admission. Accurate risk stratification is also helpful when assessing response to treatment and for discharge decisions.
Current clinical scores that incorporate patient demographics (e.g., age, sex), vital signs, comorbidities and radiographic images, are either indicative of organ damage that has already occurred (i.e., not predictive), fail to capture the complexity of COVID-19 disease (i.e., not accurate) or are difficult to integrate in routine decision making (i.e., not practical) [14]. Multiple studies indicate that individual markers, including Interleukin-6 (IL-6), procalcitonin (PCT), ferritin, D-dimer, creatinine, CD3 and CD4 T-cell counts are associated with severe COVID-19 disease outcome [13], [15], [16]. However, the utility of individual biomarkers is typically constrained by uncertainty regarding result interpretation or poor potential to identify severe disease at early stages [17]. Further, combining these markers is either impractical (difficult to measure, time consuming or requires multiple technologies) or has not proven to significantly increase predictive performance as the composite parameters provide overlapping information and are not viral-specific [14]. Given the complexity of the immunopathogenesis of COVID-19, a score combining several markers from distinct biological pathways is more likely to serve as a broadly applicable and accurate tool [15], [16].
Recently, a platform was developed that measures in fifteen minutes two viral-induced biomarkers - tumor necrosis factor-related apoptosis inducing ligand (TRAIL) and interferon gamma inducible protein-10 (IP-10; also known as CXCL10) – as well as the inflammatory biomarker C-reactive protein (CRP). TRAIL regulates immune responses via apoptosis, with lower levels associated with severe infection, including sepsis [18], [19], [20], [21]. TRAIL levels increase in viral infection [22], [23], [24] and are reduced in severe bacterial and viral infections, making it a particularly useful biomarker for severe viral infection. In COVID-19 disease, low TRAIL levels are associated with inability to clear the virus and disease severity [25], [26]. IP-10 expression is also induced by viral infection [22], [23], [24]. A regulator of inflammatory and endothelial cells, IP-10 has been implicated as a player in the lung injury during dysregulated responses to severe viral infection [27]. Accumulating evidence identifies IP-10 as an early marker of COVID-19 progression, with maintained high levels associated with mortality [15], [25], [26], [28], [29], [30]. Induced expression of CRP, a regulator of inflammation [31]. is typically observed in response to bacterial infection [32], [33] but also found to be associated with severe SARS-CoV-2 infection [34], [35], [36]. As each of these biomarkers captures a different dimension of disease progression to severe outcome, their computational integration has the potential to produce a high performing severity score.
This study reports derivation of an algorithm that computationally integrates the levels of TRAIL, IP-10, and CRP into a single numeric score that is an early indicator of severe outcome for COVID-19 patients (‘severity score’). Such a score can identify patients likely to deteriorate or not and in this way, help with objective decision making as to who may benefit from escalation or de-escalation of care as well as targeted (personalized) therapies [37] (https://www.covid19treatmentguidelines.nih.gov/critical-care/oxygenation-and-ventilation/). Taken together, the severity score along with the rapid measurement platform, have potential to serve as an accurate and actionable test for early indication of COVID-19 disease progression.
2. Materials and methods
2.1. Study design
This was an observational, convenience sampling study. Study cohort was composed of SARS-CoV-2-positive adult patients recruited retrospectively at 6 medical centers in the emergency department (ED), wards, and intensive care unit (ICU) between March and November 2020.
Inclusion criteria were: aged 18 years or older; SARS-CoV-2-positive reverse transcription polymerase chain reaction (RT-PCR) result; and a serum blood sample drawn within a week from ED arrival.
Patient data including past medical history, physical examination, laboratory data and medical treatment during hospitalization course were extracted from electronic medical records. Follow-up phone calls and verification of hospital re-admission status were performed to complete the 14-day observation period for patients discharged earlier.
Alignment with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement is indicated by responses given in the TRIPOD checklist (Table S1) [38].
2.2. Definition of clinical outcomes
Severe outcome was defined as mortality or respiratory failure requiring ICU admission, non-invasive ventilation (high flow nasal cannula, continuous positive airway pressure or bi-level airway pressure) or intubation with mechanical ventilation before or within 14 days from blood draw.
Levels of severe outcomes:
-
•
level 1, ICU admission or non-invasive ventilation
-
•
level 2, intubation with mechanical ventilation
-
•
level 3, mortality
Among patients who met a severe outcome prior to blood draw, clinical deterioration was defined as transition from level 1 to level 2 or 3, and clinical recovery was defined as successful de-escalation of care (i.e., treatment termination and/or hospital discharge).
2.3. Sample processing and measurement
See Supplementary Methods.
2.4. Missing data
See Supplementary Methods.
2.5. Multivariable model construction and score bin definition
The score is based on a model trained using L2-regularized logistic regression with balanced class weights. An additional transformation was applied to elevate the model-predicted probability as a non-linear function of CRP concentration. The resulting probability is rounded to an integer between 0 and 100.
Cross-validation was performed using logistic regression with the same configuration used to train the model, in a “leave-site-out” fashion: in each iteration all patients from one of the medical centers were left out as a test set, and a model was trained based on patients from the remaining five sites and used to predict the probability of meeting a severe outcome for patients in the test set. Predicted probabilities from the six test sets were then combined into a single list for AUC calculation. Leave-site-out cross-validation evaluates the performance of a model in medical centers it was not trained on.
To generate a clinically intuitive tool for risk stratification, four score bins were defined. Each patient was assigned to a specific bin based on their score, and within the bin according to their severity outcome (Table S3).
The score is called MeMed COVID-19 Severity™.
2.6. Statistics
See Supplementary Methods.
2.7. Study approval
Ethical approval was obtained from each medical center with informed consent covered by the original protocol.
3. Results
3.1. Study population
Between March and November 2020, 518 Israeli, German, and USA SARS-CoV-2 positive patients were enrolled across six participating medical centers (Table S4). This period encompassed the first and second COVID-19 waves in Israel, the first wave in Germany and the first and second waves in the US (Fig. S1). After exclusion of 124 patients that failed to meet eligibility criteria, the resulting derivation cohort comprised 394 patients (Fig. S2). A composite severity outcome was defined based on mortality or respiratory failure requiring ICU level of care. 113 (29%) patients met the composite severity outcome, of whom 30 patients (27%) died.
Age ranged from 19 to 98 years, with 59.1% male (Table 1 ). Patients who met severe outcomes were older, more likely to be male, and exhibited abnormal vital signs and elevated inflammatory markers on the day of blood draw as compared to non-severe patients. 18.8% of cohort patients (74/394) were hospitalized for a short duration (1–2 days) and did not meet a severe outcome.
Table 1.
Statistics | Study population (n = 394) | Severe, S (n = 113) | Non-Severe, NS (n = 281) | S vs NS p-value |
|
---|---|---|---|---|---|
Age (years) | median (IQR) | 61.5 (25.8) | 65.0 (21.0) | 60.0 (27.0) | 0.009 |
Male sex | n (%) | 233 (59.1%) | 80 (70.8%) | 153 (54.4%) | 0.003 |
Time from ED arrival to blood draw (days) | median (IQR) | 1.0 (2.0) | 2.0 (2.0) | 1.0 (2.0) | <0.001 |
Length of hospital stay (days)A | median (IQR) | 6.5 (10.0) | 11.0 (13.8) | 5.0 (8.0) | <0.001 |
Short hospital stay (1–2 days) A | n (%) | 76 (19.9%) | 2 (1.9%) | 74 (26.8%) | <0.001 |
OutcomeB | |||||
ICU admission | n (%) | 51 (12.9%) | 51 (45.1%) | 0 (0.0%) | N/A |
Non-invasive ventilation | n (%) | 65 (16.5%) | 65 (57.5%) | 0 (0.0%) | N/A |
Intubation with mechanical ventilation | n (%) | 49 (12.4%) | 49 (43.4%) | 0 (0.0%) | N/A |
Mortality | n (%) | 30 (7.6%) | 30 (26.5%) | 0 (0.0%) | N/A |
Comorbidities | |||||
Hypertension | n (%) | 163 (41.4%) | 54 (47.8%) | 109 (38.8%) | 0.114 |
Diabetes mellitus | n (%) | 114 (28.9%) | 39 (34.5%) | 75 (26.7%) | 0.140 |
Chronic heart failure | n (%) | 18 (4.6%) | 7 (6.2%) | 11 (3.9%) | 0.423 |
Malignancy | n (%) | 19 (4.8%) | 10 (8.8%) | 9 (3.2%) | 0.034 |
Vital signsA,C | |||||
Heart rate >100 beats per minute | n (%) | 51 (17.3%) | 16 (18.4%) | 35 (16.9%) | 0.739 |
Temperature ≥37.8 °C | n (%) | 45 (18.1%) | 19 (28.4%) | 26 (14.4%) | 0.015 |
Systolic blood pressure <90 mmHg | n (%) | 8 (3.0%) | 6 (7.5%) | 2 (1.1%) | 0.010 |
Respiratory rate >20 breath/minute | n (%) | 32 (29.6%) | 20 (44.4%) | 12 (19.0%) | 0.006 |
Oxygen saturation on room air (SpO2) ≤93%D | n (%) | 91 (36.3%) | 32 (54.2%) | 59 (30.7%) | 0.002 |
BiomarkersA,C | |||||
WBC (K/ml3) | median (IQR) | 6.4 (3.3) | 7.8 (3.2) | 5.8 (3.0) | <0.001 |
Ferritin (ng/ml) | median (IQR) | 438.4 (1281.4) | 888.8 (1237.7) | 392.2 (864.2) | 0.026 |
CRP (mg/L) | median (IQR) | 66.0 (139.8) | 166.8 (178.0) | 42.4 (83.7) | <0.001 |
PCT (ng/ml) | median (IQR) | 0.1 (0.2) | 0.3 (0.5) | 0.1 (0.1) | <0.001 |
IL-6 (pg/ml) | median (IQR) | 19.2 (44.3) | 50.9 (113.6) | 14.5 (34.6) | <0.001 |
AMissing data for a subset of cohort patients (detailed in Methods and Supplementary Table 2).
BOutcomes met before or within 14 days of blood draw.
CMeasured on day of blood draw.
IQR, interquartile range. ICU, intensive care unit. WBC, white blood cells. CRP, C-reactive protein. PCT, procalcitonin. IL-6, interleukin-6.
3.2. A score that integrates TRAIL, IP-10 and CRP accurately stratifies COVID-19 patients by disease severity
The host proteins TRAIL, IP-10 and CRP were differentially expressed in sera of severe versus non-severe COVID-19 patients (Fig. 1 A, Table S5). Patients meeting severe outcomes exhibited significantly higher levels of CRP (median 167 vs. 42 mg/L; p < 0.001) and IP-10 (median 1632 vs. 420 pg/ml; p < 0.001) and lower levels of TRAIL (median 31 vs. 61 pg/ml; p < 0.001).
A score integrating the serum concentrations of the three proteins was developed using the entire derivation cohort (Methods). The score ranges from 0 to 100, with higher levels reflecting a higher likelihood for severe outcome. Scores were significantly higher in severe as compared to non-severe patients (median 78 vs. 26 score units; p < 0.001; Fig. 1B). The score performance as indicated by area under the receiver operating characteristic curve (AUC) was 0.86 (95%CI: 0.81–0.91).
To assess the score’s potential for generalization its performance was compared to that of cross-validation based on the same cohort of patients (Methods) [38]. Cross-validation AUC was 0.86 (95%CI: 0.81–0.90), indistinguishable from that of the score, indicating the model was not overfitted and is generalizable.
To generate a clinically intuitive tool for severity stratification, four score bins were defined. Each patient was assigned to a bin based on their score, and within the bin according to their severity outcome. In this framework, the score’s performance was demonstrated by a significant increase of the likelihood of COVID-19 severe outcome across the four bins (Cochran-Armitage, CA p < 0.001; Table 2 ). Only 3 severe patients (3% of all severe patients) were assigned to the lowest bin; clinical details for these patients are given in Table S6. The proportion of patients intubated with mechanical ventilation or died increased across the four bins (Table S7).
Table 2.
LR, likelihood ratio. PPV, positive predictive value. NPV, negative predictive value. CI, confidence interval.
4. Score maintained high performance across different sub-populations
To test whether the relationship between score and severe outcome is confounded by known risk factors, performance was inspected across sub-populations. Potential confounders evaluated included sex, age, and number of comorbidities. The time from emergency department (ED) arrival to blood draw was also evaluated as a surrogate for time from symptom onset. A significant increase in the likelihood of severe outcome with increasing score was observed in each of the sub-populations tested (Table S8; p < 0.001 for each of the eight sub-populations). Furthermore, adding each of these risk factors to the host proteins did not improve the performance (Table S9), indicating that the information contained in sex, age, number of co-morbidities and time from ED arrival is already captured by the status of the immune response.
4.1. The score outperforms known risk factors and candidate severity tools
Several parameters (e.g., age, oxygen saturation) [40] and biomarkers (e.g., IL-6) [34] have been proposed as candidate tools to aid the clinician in COVID-19 severity stratification. The score AUC compared favorably with various biomarkers and parameters (Fig. 2 ). The score AUC was higher also when the comparison analysis was restricted to the subset of patients with available measurements for each comparator (Table S10), and when the definition of severe outcome was limited to intubation with mechanical ventilation or death (Table S11).
4.2. Prognostic value of the severity score
For a severity stratification tool to be predictive, its output must not only correlate with observable disease severity, but also identify those individuals who appear stable but have a high likelihood to deteriorate. To test the score’s potential to predict future severe outcomes, the sub-group of patients who met a severe outcome for the first time after the day of blood draw (n = 29) was identified. The time from blood draw to first outcome was 1–13 days with a median of 4 days and the likelihood of future severe outcome increased significantly (p < 0.001) across the score bins (Table S12). This trend was significant also with the definition of future severe outcome restricted to intubation with mechanical ventilation (Table S13).
To further evaluate the score’s predictive value, patients were stratified according to the time between blood draw and meeting the first severe outcome. Three groups of severe patients were defined: those meeting a first severe outcome on the day of blood draw (n = 29), 1–3 days after (n = 14), or >3 days after blood draw (n = 15). Deterioration occurring in proximity to blood draw was associated with a higher score; median scores were 87, 72, and 60 for the same-day, 1–3 days after, and >3 days after groups, respectively (Fig. 3 ). Notably, the scores of each of the three severe patient groups, including the group meeting a first outcome over 3 days after blood draw, were significantly (p < 0.001) higher than the scores of patients who did not meet a severe outcome (n = 281).
Finally, it was examined if the score provides added value beyond one of the parameters measured to assess the respiratory status of COVID-19 patients, namely SpO2 values. Six patients were identified with SpO2 > 93% on the day of blood draw who subsequently met a severe outcome, all of whom were assigned a score falling within the upper two bins (score range 60–85; Table S14). This finding supports the score’s potential to detect early-on patients who are at risk of future deterioration albeit with normal saturation (SpO2 > 93%) at time of blood draw.
4.3. Score’s potential to predict further deterioration and mortality
To evaluate the score’s potential to predict further deterioration of severe patients, we identified the sub-group of severe patients who were admitted to the ICU and/or required non-invasive ventilation before blood draw (n = 19). For this sub-group, deterioration was defined as requiring intubation with mechanical ventilation and/or death following the time of blood draw. Six out of the nineteen patients deteriorated, the time from blood draw to deterioration ranging between 0 and 13 days, with a median of 6.5 days. The remaining thirteen patients recovered, based on treatment termination and hospital discharge. The patients who deteriorated exhibited higher scores compared to those who recovered, with a median score of 94.5 vs. 65 (Fig. S3; p = 0.004). Lastly, the predictive value of the score was assessed by examining survival prognosis. Thirty patients (8% of the cohort) died within 14 days of blood draw. 14-day survival probability was calculated using Kaplan-Meier estimator across the score bins. Survival distribution was significantly different comparing score bins 1 + 2 (score < 40) versus bins 3 + 4 (score ≥ 40) (Fig. S4; p < 0.001).
5. Conclusions
The present study describes derivation of a host-protein score comprising TRAIL, IP-10 and CRP for accurately stratifying the severity of COVID-19 disease based on a multinational cohort recruited across multiple COVID-19 waves. The score was shown to accurately predict clinical deterioration among patients throughout the acute disease course. Patients with a score falling in the upper bins had higher likelihood to meet the composite severity outcome.
The severity score demonstrated higher performance than other candidate severity tools examined herein and performed irrespective of known risk factors and potential confounders, including age, sex, time from ED arrival, and comorbidities. Importantly, high performance was attained in the sub-cohort of patients meeting the severity outcome after the sampling day, supporting the score’s prognostic value. Furthermore, the score predicted deterioration even for patients that met severe outcomes more than three days after blood draw, supporting capability to detect severe outcomes early-on. Patients that deteriorated in proximity to the day of blood draw exhibited higher scores than patients who deteriorated later. The trend in score distribution across time suggests that higher scores may not only indicate the probability of severe outcomes but also give timing information. The tool’s prognostic value is further corroborated by two additional findings. First, among patients admitted to the ICU or subjected to non-invasive ventilation before the day of blood draw, those that deteriorated further exhibited a significantly higher score. Second, in the sub-group of patients with normal saturation levels (SpO2 > 93%), all patients who deteriorated following the day of blood testing were assigned a high score. The latter underscores the added clinical value of this tool beyond today’s standard of care in stratifying the severity of patients that present with similar vital signs.
In addition to detecting early-on who is likely to deteriorate, a severity tool can help reduce unnecessary treatments or hospital admission for low-risk patients. It is notable that almost 20% of the cohort was hospitalized for 1–2 days and did not meet any of the severe outcomes, potentially representing patients that may not have required admission.
It has been recognized since the discovery of CRP as an inflammation marker [31], that host responses to infection have potential to serve as clinical decision-making tools. Advancements in host-response profiling and machine learning have enabled development of a new generation of algorithm-based scores, including one based on TRAIL, IP-10 and CRP for differentiating bacterial from viral infection [23], [24], [41]. Multiple studies have shown that these immune proteins change expression in response to infection etiology [22], [23], [24], [32], [33] and in response to the severity of infection [18], [19], [20], [27]. The fact that these three proteins specifically change expression in the progression of COVID-19 disease [15], [25], [28], [29], [30], [34], [35], [36], [42], prompted the present study to leverage a platform for their rapid measurement to derive a COVID-19-specific severity stratification score. It appears that immune dysregulation is pivotal to disease progression in SARS-CoV-2 infections and that IP-10 may be important in the development of lung damage, possibly when its expression is dissociated from interferon gamma levels [13], [25], [26]. Notably, IP-10 and CRP have been shown to exhibit severity-related expression changes in SARS-CoV and middle east respiratory syndrome (MERS) infections, supporting that their response is not strain-specific but likely associated with viruses causing lung pathology [30], [43], [44]. Generally, the window into the immune response to SARS-CoV-2 infection provided specifically by TRAIL, IP-10 and CRP, as individual biomarkers and as an integrative score, has potential to provide clinicians with insight into disease course and patient management.
COVID-19 patients with severe or critical disease typically suffer respiratory failure and frequently require non-invasive or invasive respiratory support (https://www.covid19treatmentguidelines.nih.gov/). Accordingly, the rationale for the composite severity endpoint which guided score development was to identify patients likely to exhibit respiratory insufficiency and require intense respiratory support. Treatment protocols have been dynamic during the pandemic and therefore multiple respiratory support methods were included [45], [46]. Given that the multiple respiratory methods included in the composite endpoint represent varying levels of respiratory insufficiency, patients already meeting one of the severe outcomes before sampling were included in the derivation cohort, as these cases may deteriorate further and in a real-world setting may benefit from a risk stratification tool. This assumption was supported by our finding that severe patients who deteriorated further exhibited higher scores compared to those who recovered. While the purpose of the proposed score is prognostic, inclusion of such patients in the derivation data is useful on the basis that an immune response observed in patients who are already at a severe state may be detectable also in patients who are to meet a severe outcome in the next several days following sampling.
A major strength of the study is that the population was recruited across multiple international settings and multiple waves of the pandemic, supporting the generalizability of the findings. Broad applicability is also supported by similarity of the score performance to cross-validation performance. Typically, studies that evaluate risk stratification tools for COVID-19 patient management focus on two severity outcomes, intubation with mechanical ventilation and/or mortality. Another strength of this study is the composite severity outcome, as prediction of requirement for non-invasive ventilation and ICU admission gives healthcare providers additional information that may not be obvious at patient evaluation, enabling better patient management.
Future studies are required to validate the performance of this novel severity score and to establish its utility in the evolving workflow of COVID-19 patient management, specifically the prognostic capability in deterioration prediction within 14 days and accordingly in supporting decisions on escalation and de-escalation of care. A notable benefit of this immune-based tool, as compared to severity scores already in use, is that it predicts immune dysregulation associated with lung damage [25], [26], [27]. Furthermore, unlike other severity scores, the three constituent proteins and score can be easily and rapidly measured at the point-of-need using one platform.
An inherent limitation of this study is its retrospective design allowing only routinely measured data parameters to be evaluated. As a result, score performance could not be compared to other commonly used severity scores or biomarkers and certain data points were not available or missing (e.g., time from symptom onset, viral load, etc.). Additionally, the occurrence of co-infections in the study cohort may have confounded the levels of the biomarkers and impacted the severity score. Nonetheless, the occurrence of co-infections in COVID patients during the study period was estimated to be between 1.5% and 11% [47], [48], [49]. Also, the study period did not encompass multiple genetic variants of SARS-CoV-2. The score was derived based on a population treated under protocols applied in the first and second waves of the pandemic, and its performance has not yet been evaluated in populations treated with newer approaches or in populations that have been vaccinated. These limitations may result in constrained generalizability and the extent of impact can be evaluated in future independent score validation studies.
The derived severity score together with the rapid measurement platform have potential to serve as a practical predictive tool for early indication of COVID-19 patients at-risk for severe outcome, facilitating improved patient management and outcomes.
Author contributions statement
NSM, AA, OS, TMG, EE: Conceived and designed the study, analyzed data (including score derivation) and wrote the manuscript. TIB, ER: Designed the study, analyzed data and reviewed the manuscript. RN, PF: Analyzed data (including score derivation) and reviewed the manuscript. ES, MR, YI, MH, NA: Conducted laboratory measurements, analyzed data and reviewed the manuscript. KO: Conceived the study and reviewed the manuscript. BT, PS, IK, SL, DD, AJ, RK, EBC, GD, SAT, CP, SM, MSh, MSt, AK: Enrolled patients and reviewed the manuscript. All authors approved the final version of the manuscript.
Declaration of Competing Interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: AK, BT, MSt and MSh have indicated that they have no potential conflicts of interest to disclose. AA, EE, ER, ES, KO, MH, MR, NA, NSM, OS, RN, TIB, TMG and YI are/were employees of MeMed and hold stock and/or options; PF holds MeMed stock options; SL declares to have received lecturing fees from MeMed; EE, ES, KO, MH, NSM, OS, RN, SL, TIB, TMG are inventors of pending or issued patent applications relating to the TRAIL/IP-10/CRP technology; AJ, EBC, IK, DD, SL, PS, RK, SM, SAT, GD and CP declare that their medical center received funding from MeMed.
Acknowledgments
Acknowledgements
We thank Oren Zarchin, Moran Barak and Efrat Hartog-David for their professional insights. We thank Yaly Orr, Idan Lancry, Dvir Ilan, Noa Kremer, Ruth Yudalevich, Adi Cohen, Michal Largman and Dennis Kuhn for their dedicated data management and Noa Serruya for laboratory support. We thank Einat Moscoviz for leading the clinical coordination.
Funding sources
The study was funded in part by a grant awarded to MeMed from the European Commission, Executive Agency for Small and Medium-sized Enterprises H2020-EIC-SMEInst-2018-2020-2 [grant number 88124] and in part by MeMed. MeMed led study design, analysis and interpretation of data, drafting and decision to submit the manuscript.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.cyto.2023.156246.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
Data availability
Deidentified individual participant data will be made available upon reasonable request. Requests should be submitted to Dr. Tanya Gottlieb, tanya.gottlieb@me-med.com.
References
- 1.Meyer S., Papan C., Last K. A global health perspective on SARS-CoV-2-hazards, disaster and hope. Wien. Med. Wochenschr. 2020;1946(170):357–358. doi: 10.1007/s10354-020-00769-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Papan C., Last K., Meyer S. COVID-19: fighting the foe with Virchow. Infection. 2021 doi: 10.1007/s15010-021-01628-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hu B., Guo H., Zhou P., Shi Z.-L. Characteristics of SARS-CoV-2 and COVID-19. Nat. Rev. Microbiol. 2020:1–14. doi: 10.1038/s41579-020-00459-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wu Z., McGoogan J.M. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323:1239. doi: 10.1001/jama.2020.2648. [DOI] [PubMed] [Google Scholar]
- 5.Gandhi R.T., Lynch J.B., del Rio C. Mild or moderate Covid-19. N. Engl. J. Med. 2020;383:1757–1766. doi: 10.1056/NEJMcp2009249. [DOI] [PubMed] [Google Scholar]
- 6.Jamilloux Y., Henry T., Belot A., Viel S., Fauter M., El Jammal T., Walzer T., François B., Sève P. Should we stimulate or suppress immune responses in COVID-19? Cytokine and anti-cytokine interventions. Autoimmun. Rev. 2020;19 doi: 10.1016/j.autrev.2020.102567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Siddiqi H.K., Mehra M.R. COVID-19 illness in native and immunosuppressed states: a clinical-therapeutic staging proposal. J. Heart Lung Transplant. Off. Publ. Int. Soc. Heart Transplant. 2020;39:405–407. doi: 10.1016/j.healun.2020.03.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Remy K.E., Mazer M., Striker D.A., Ellebedy A.H., Walton A.H., Unsinger J., Blood T.M., Mudd P.A., Yi D.J., Mannion D.A., Osborne D.F., Martin R.S., Anand N.J., Bosanquet J.P., Blood J., Drewry A.M., Caldwell C.C., Turnbull I.R., Brakenridge S.C., Moldwawer L.L., Hotchkiss R.S. Severe immunosuppression and not a cytokine storm characterizes COVID-19 infections. JCI Insight. 2021;5 doi: 10.1172/jci.insight.140329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ye Q., Wang B., Mao J. The pathogenesis and treatment of the ‘Cytokine Storm’ in COVID-19. J. Infect. 2020;80:607–613. doi: 10.1016/j.jinf.2020.03.037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Mehta P., McAuley D.F., Brown M., Sanchez E., Tattersall R.S., Manson J.J. COVID-19: consider cytokine storm syndromes and immunosuppression. The Lancet. 2020;395:1033–1034. doi: 10.1016/S0140-6736(20)30628-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.McGonagle D., Sharif K., O’Regan A., Bridgewood C. The role of cytokines including Interleukin-6 in COVID-19 induced pneumonia and macrophage activation syndrome-like disease. Autoimmun. Rev. 2020 doi: 10.1016/j.autrev.2020.102537. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.van de Veerdonk F.L., Netea M.G. Blocking IL-1 to prevent respiratory failure in COVID-19. Crit. Care. 2020;24:445. doi: 10.1186/s13054-020-03166-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Buszko M., Nita-Lazar A., Park J.-H., Schwartzberg P.L., Verthelyi D., Young H.A., Rosenberg A.S. Lessons learned: new insights on the role of cytokines in COVID-19. Nat. Immunol. 2021;22:404–411. doi: 10.1038/s41590-021-00901-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wynants L., Van Calster B., Bonten M.M.J., Collins G.S., Debray T.P.A., De Vos M., Haller M.C., Heinze G., Moons K.G.M., Riley R.D., Schuit E., Smits L.J.M., Snell K.I.E., Steyerberg E.W., Wallisch C., van Smeden M. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369 doi: 10.1136/bmj.m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Laing A.G., Lorenc A., del Molino I., del Barrio A., Das M., Fish L., Monin M., Muñoz-Ruiz D.R., McKenzie T.S., Hayday I., Francos-Quijorna S., Kamdar M., Joseph D., Davies R., Davis A., Jennings I., Zlatareva P., Vantourout Y., Wu V., Sofra F., Cano M., Greco E., Theodoridis J.D., Freedman S., Gee J.N.E., Chan S., Ryan E., Bugallo-Blanco P., Peterson K., Kisand L., Haljasmägi L., Chadli P., Moingeon L., Martinez B., Merrick K., Bisnauthsing K., Brooks M.A.A., Ibrahim J., Mason F.L., Gomez K., Babalola S., Abdul-Jawad J., Cason C., Mant J., Seow C., Graham K.J., Doores F.D., Rosa J., Edgeworth M., Shankar-Hari A.C.H. A dynamic COVID-19 immune signature includes associations with poor prognosis. Nat. Med. 2020;26:1623–1635. doi: 10.1038/s41591-020-1038-6. [DOI] [PubMed] [Google Scholar]
- 16.Del Valle D.M., Kim-Schulze S., Huang H.-H., Beckmann N.D., Nirenberg S., Wang B., Lavin Y., Swartz T.H., Madduri D., Stock A., Marron T.U., Xie H., Patel M., Tuballes K., Van Oekelen O., Rahman A., Kovatch P., Aberg J.A., Schadt E., Jagannath S., Mazumdar M., Charney A.W., Firpo-Betancourt A., Mendu D.R., Jhang J., Reich D., Sigel K., Cordon-Cardo C., Feldmann M., Parekh S., Merad M., Gnjatic S. An inflammatory cytokine signature predicts COVID-19 severity and survival. Nat. Med. 2020;26:1636–1643. doi: 10.1038/s41591-020-1051-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Liu B.M., Martins T.B., Peterson L.K., Hill H.R. Clinical significance of measuring serum cytokine levels as inflammatory biomarkers in adult and pediatric COVID-19 cases: a review. Cytokine. 2021;142 doi: 10.1016/j.cyto.2021.155478. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tian Y., Tao T., Zhu J., Zou Y., Wang J., Li J., Bo L., Deng X. Soluble tumor necrosis factor related apoptosis inducing ligand level as a predictor of severity of sepsis and the risk of mortality in septic patients. PloS One. 2013;8:e82204. doi: 10.1371/journal.pone.0082204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Schenck E.J., Ma K.C., Price D.R., Nicholson T., Oromendia C., Gentzler E.R., Sanchez E., Baron R.M., Fredenburgh L.E., Huh J.-W., Siempos I.I., Choi A.M. Circulating cell death biomarker TRAIL is associated with increased organ dysfunction in sepsis. JCI Insight. 2019;4:e127143. doi: 10.1172/jci.insight.127143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Oved K., Cohen A., Boico O., Navon R., Friedman T., Etshtein L., Bamberger E., Pri-or E., Gottlieb T., Paz M., Srugo I., Chistyakov I., Klein A., Potasman I., Eden E. Tumor necrosis factor-related apoptosis-inducing ligand protein as a marker for disease severity in patients with acute infection. Open Forum Infect. Dis. 2016;3 doi: 10.1093/ofid/ofw172.103. [DOI] [Google Scholar]
- 21.Peteranderl C., Herold S. The impact of the interferon/TNF-related apoptosis-inducing ligand signaling axis on disease progression in respiratory viral infection and beyond. Front. Immunol. 2017;8:313. doi: 10.3389/fimmu.2017.00313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.van der Does Y., Tjikhoeri A., Ramakers C., Rood P.P.M., van Gorp E.C.M., Limper M. TRAIL and IP-10 as biomarkers of viral infections in the emergency department. J. Infect. 2016;72:761–763. doi: 10.1016/j.jinf.2016.03.004. [DOI] [PubMed] [Google Scholar]
- 23.Oved K., Cohen A., Boico O., Navon R., Friedman T., Etshtein L., Kriger O., Bamberger E., Fonar Y., Yacobov R., Wolchinsky R., Denkberg G., Dotan Y., Hochberg A., Reiter Y., Grupper M., Srugo I., Feigin P., Gorfine M., Chistyakov I., Dagan R., Klein A., Potasman I., Eden E. A novel host-proteome signature for distinguishing between acute bacterial and viral infections. PLoS ONE. 2015;10:e0120012. doi: 10.1371/journal.pone.0120012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.van Houten C.B., de Groot J.A.H., Klein A., Srugo I., Chistyakov I., de Waal W., Meijssen C.B., Avis W., Wolfs T.F.W., Shachor-Meyouhas Y., Stein M., Sanders E.A.M., Bont L.J. A host-protein based assay to differentiate between bacterial and viral infections in preschool children (OPPORTUNITY): a double-blind, multicentre, validation study. Lancet Infect. Dis. 2017;17:431–440. doi: 10.1016/S1473-3099(16)30519-9. [DOI] [PubMed] [Google Scholar]
- 25.Lev S., Gottesman T., Levin G.S., Lederfein D., Berkov E., Diker D., Zaidman A., Nutman A., Ber T.I., Angel A., Kellerman L., Barash E., Navon R., Boico O., Israeli Y., Rosenberg M., Gelman A., Kalfon R., Simon E., Avni N., Hainrichson M., Zarchin O., Gottlieb T.M., Oved K., Eden E., Tadmor B. Observational cohort study of IP-10’s potential as a biomarker to aid in inflammation regulation within a clinical decision support protocol for patients with severe COVID-19. PLOS ONE. 2021;16:e0245296. doi: 10.1371/journal.pone.0245296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Blot M., Jacquier M., Aho Glele L.-S., Beltramo G., Nguyen M., Bonniaud P., Prin S., Andreu P., Bouhemad B., Bour J.-B., Binquet C., Piroth L., Pais de Barros J.-P., Masson D., Quenot J.-P., Charles P.-E., Aptel F., Dargent A., Georges M., Labruyère M., Lagrost L., Large A., Monier S., Roudaut J.-B., Thomas C. Pneumochondrie study group, CXCL10 could drive longer duration of mechanical ventilation during COVID-19 ARDS. Crit. Care. 2020;24:632. doi: 10.1186/s13054-020-03328-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Ichikawa A., Kuba K., Morita M., Chida S., Tezuka H., Hara H., Sasaki T., Ohteki T., Ranieri V.M., dos Santos C.C., Kawaoka Y., Akira S., Luster A.D., Lu B., Penninger J.M., Uhlig S., Slutsky A.S., Imai Y. CXCL10-CXCR3 enhances the development of neutrophil-mediated fulminant lung injury of viral and nonviral origin. Am. J. Respir. Crit. Care Med. 2013;187:65–77. doi: 10.1164/rccm.201203-0508OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Yang Y., Shen C., Li J., Yuan J., Wei J., Huang F., Wang F., Li G., Li Y., Xing L., Peng L., Yang M., Cao M., Zheng H., Wu W., Zou R., Li D., Xu Z., Wang H., Zhang M., Zhang Z., Gao G.F., Jiang C., Liu L., Liu Y. Plasma IP-10 and MCP-3 levels are highly associated with disease severity and predict the progression of COVID-19. J. Allergy Clin. Immunol. 2020 doi: 10.1016/j.jaci.2020.04.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hue S., Beldi-Ferchiou A., Bendib I., Surenaud M., Fourati S., Frapard T., Rivoal S., Razazi K., Carteaux G., Delfau-Larue M.-H., Mekontso-Dessap A., Audureau E., de Prost N. Uncontrolled Innate and impaired adaptive immune responses in patients with COVID-19 acute respiratory distress syndrome. Am. J. Respir. Crit. Care Med. 2020;202:1509–1519. doi: 10.1164/rccm.202005-1885OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Samaras C., Kyriazopoulou E., Poulakou G., Reiner E., Kosmidou M., Karanika I., Petrakis V., Adamis G., Gatselis N.K., Fragkou A., Rapti A., Taddei E., Kalomenidis I., Chrysos G., Bertoli G., Kainis I., Alexiou Z., Castelli F., Saverio Serino F., Bakakos P., Nicastri E., Tzavara V., Kostis E., Dagna L., Koukidou S., Tzatzagou G., Chini M., Bassetti M., Trakatelli C., Tsoukalas G., Selmi C., Samarkos M., Pyrpasopoulou A., Masgala A., Antonakis E., Argyraki A., Akinosoglou K., Sympardi S., Panagopoulos P., Milionis H., Metallidis S., Syrigos K.N., Angel A., Dalekos G.N., Netea M.G., Giamarellos-Bourboulis E.J. Interferon gamma-induced protein 10 (IP-10) for the early prognosis of the risk for severe respiratory failure and death in COVID-19 pneumonia. Cytokine. 2023;162 doi: 10.1016/j.cyto.2022.156111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Tillett W.S., Francis T. Serological reactions in pneumonia with a non-protein somatic fraction of Pneumococcus. J. Exp. Med. 1930;52:561–571. doi: 10.1084/jem.52.4.561. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Aabenhus R., Jensen J.-U.S., Jørgensen K.J., Hróbjartsson A., Bjerrum L. Biomarkers as point-of-care tests to guide prescription of antibiotics in patients with acute respiratory infections in primary care. Cochrane Database Syst. Rev. 2014;11:CD010130. doi: 10.1002/14651858.CD010130.pub2. [DOI] [PubMed] [Google Scholar]
- 33.Eccles S., Pincus C., Higgins B., Woodhead M. Guideline development group, diagnosis and management of community and hospital acquired pneumonia in adults: summary of NICE guidance. BMJ. 2014;349 doi: 10.1136/bmj.g6722. [DOI] [PubMed] [Google Scholar]
- 34.Herold T., Jurinovic V., Arnreich C., Lipworth B.J., Hellmuth J.C., von Bergwelt-Baildon M., Klein M., Weinberger T. Elevated levels of IL-6 and CRP predict the need for mechanical ventilation in COVID-19. J. Allergy Clin. Immunol. 2020;146:128–136.e4. doi: 10.1016/j.jaci.2020.05.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Liu F., Li L., Xu M., Wu J., Luo D., Zhu Y., Li B., Song X., Zhou X. Prognostic value of interleukin-6, C-reactive protein, and procalcitonin in patients with COVID-19. J. Clin. Virol. Off. Publ. Pan Am. Soc. Clin. Virol. 2020;127 doi: 10.1016/j.jcv.2020.104370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Leisman D.E., Ronner L., Pinotti R., Taylor M.D., Sinha P., Calfee C.S., Hirayama A.V., Mastroiani F., Turtle C.J., Harhay M.O., Legrand M., Deutschman C.S. Cytokine elevation in severe and critical COVID-19: a rapid systematic review, meta-analysis, and comparison with other inflammatory syndromes. Lancet Respir. Med. 2020;8:1233–1244. doi: 10.1016/S2213-2600(20)30404-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Nicholson T.W., Talbot N.P., Nickol A., Chadwick A.J., Lawton O. Respiratory failure and non-invasive respiratory support during the covid-19 pandemic: an update for re-deployed hospital doctors and primary care physicians. BMJ. 2020;369 doi: 10.1136/bmj.m2446. [DOI] [PubMed] [Google Scholar]
- 38.Collins G.S., Reitsma J.B., Altman D.G., Moons K.G.M. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann. Intern. Med. 2015;162:55–63. doi: 10.7326/M14-0697. [DOI] [PubMed] [Google Scholar]
- 39.Coronavirus Disease 2019 (COVID-19) Treatment Guidelines (n.d.) 274. [PubMed]
- 40.Guan W., Ni Z., Hu Y., Liang W., Ou C., He J., Liu L., Shan H., Lei C., Hui D.S.C., Du B., Li L., Zeng G., Yuen K.-Y., Chen R., Tang C., Wang T., Chen P., Xiang J., Li S., Wang J., Liang Z., Peng Y., Wei L., Liu Y., Hu Y., Peng P., Wang J., Liu J., Chen Z., Li G., Zheng Z., Qiu S., Luo J., Ye C., Zhu S., Zhong N. Clinical characteristics of coronavirus disease 2019 in China. N. Engl. J. Med. 2019;382(2020):1708–1720. doi: 10.1056/NEJMoa2002032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Srugo I., Klein A., Stein M., Golan-Shany O., Kerem N., Chistyakov I., Genizi J., Glazer O., Yaniv L., German A., Miron D., Shachor-Meyouhas Y., Bamberger E., Oved K., Gottlieb T.M., Navon R., Paz M., Etshtein L., Boico O., Kronenfeld G., Eden E., Cohen R., Chappuy H., Angoulvant F., Lacroix L., Gervaix A. Validation of a novel assay to distinguish bacterial and viral infections. Pediatrics. 2017 doi: 10.1542/peds.2016-3453. [DOI] [PubMed] [Google Scholar]
- 42.Chi Y., Ge Y., Wu B., Zhang W., Wu T., Wen T., Liu J., Guo X., Huang C., Jiao Y., Zhu F., Zhu B., Cui L. Serum cytokine and chemokine profile in relation to the severity of coronavirus disease 2019 in China. J. Infect. Dis. 2019;222(2020):746–754. doi: 10.1093/infdis/jiaa363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Huang K., Su I., Theron M., Wu Y., Lai S., Liu C., Lei H. An interferon-γ-related cytokine storm in SARS patients. J. Med. Virol. 2005;75:185–194. doi: 10.1002/jmv.20255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Zhou J., Chu H., Li C., Wong B.-H.-Y., Cheng Z.-S., Poon V.-K.-M., Sun T., Lau C.-C.-Y., Wong K.-K.-Y., Chan J.-Y.-W., Chan J.-F.-W., To K.-K.-W., Chan K.-H., Zheng B.-J., Yuen K.-Y. Active replication of middle east respiratory syndrome coronavirus and aberrant induction of inflammatory cytokines and chemokines in human macrophages: implications for pathogenesis. J. Infect. Dis. 2014;209:1331–1342. doi: 10.1093/infdis/jit504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Odor P.M., Neun M., Bampoe S., Clark S., Heaton D., Hoogenboom E.M., Patel A., Brown M., Kamming D. Anaesthesia and COVID-19: infection control. Br. J. Anaesth. 2020;125:16–24. doi: 10.1016/j.bja.2020.03.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Alhazzani W., Møller M.H., Arabi Y.M., Loeb M., Gong M.N., Fan E., Oczkowski S., Levy M.M., Derde L., Dzierba A., Du B., Aboodi M., Wunsch H., Cecconi M., Koh Y., Chertow D.S., Maitland K., Alshamsi F., Belley-Cote E., Greco M., Laundy M., Morgan J.S., Kesecioglu J., McGeer A., Mermel L., Mammen M.J., Alexander P.E., Arrington A., Centofanti J.E., Citerio G., Baw B., Memish Z.A., Hammond N., Hayden F.G., Evans L., Rhodes A. Surviving Sepsis Campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19) Intensive Care Med. 2019;46(2020):854–887. doi: 10.1007/s00134-020-06022-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Alshaikh F.S., Godman B., Sindi O.N., Seaton R.A., Kurdi A. Prevalence of bacterial coinfection and patterns of antibiotics prescribing in patients with COVID-19: a systematic review and meta-analysis. PloS One. 2022;17:e0272375. doi: 10.1371/journal.pone.0272375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Karaba S.M., Jones G., Helsel T., Smith L.L., Avery R., Dzintars K., Salinas A.B., Keller S.C., Townsend J.L., Klein E., Amoah J., Garibaldi B.T., Cosgrove S.E., Fabre V. Prevalence of co-infection at the time of hospital admission in COVID-19 patients, a multicenter study. Open Forum Infect. Dis. 2021;8:ofaa578. doi: 10.1093/ofid/ofaa578. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Calderon M., Gysin G., Gujjar A., McMaster A., King L., Comandé D., Hunter E., Payne B. Bacterial co-infection and antibiotic stewardship in patients with COVID-19: a systematic review and meta-analysis. BMC Infect. Dis. 2023;23:14. doi: 10.1186/s12879-022-07942-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Deidentified individual participant data will be made available upon reasonable request. Requests should be submitted to Dr. Tanya Gottlieb, tanya.gottlieb@me-med.com.