Abstract
Purpose
COVID-19 is also referred to as a typical viral septic pulmonary infection by 2019-nCoV. However, little is known regarding its characteristics in terms of systemic inflammation and organ injury, especially compared with classical bacterial sepsis. This article aims to investigate the clinical characteristics and prognosis between COVID-19-associated sepsis and classic bacterial-induced sepsis.
Methods
In this retrospective cohort study, septic patients with COVID-19 in the intensive care unit (ICU) of a government-designed therapy center in Shenzhen, China between January 14, 2020 and March 10, 2020, and septic patients induced by carbapenem-resistant klebsiella pneumonia (CrKP) admitted to the ICU of the Second People's Hospital of Shenzhen, China between January 1, 2014 and October 30, 2019 were enrolled. Demographic and clinical parameters including comorbidities, critical illness scores, treatment, and laboratory data, as well as prognosis were compared between the two groups. Risk factors for mortality and survival rate were analyzed using multivariable logistic regression and survival curve, respectively.
Results
A total of 107 patients with COVID-19 and 63 patients with CrKP were enrolled. A direct comparison between the two groups demonstrated more serious degrees of primary lung injury following 2019-nCoV infection (indicated by lower PaO2/FiO2), but milder systemic inflammatory response, lower sequential organ failure assessment score and better functions of the organs like heart, liver, kidney, coagulation, and circulation. However, the acquired immunosuppression presented in COVID-19 patients was more severe, which presented as lower lymphocyte counts (0.8×109/L vs. 0.9×109/L). Moreover, the proportion of COVID-19 patients treated with corticosteroid therapy and extracorporeal membrane oxygenation was larger compared with CrKP patients (78.5% vs. 38.1% and 6.5% vs. 0, respectively) who required less invasive mechanical ventilation (31.6% vs. 54.0%). The incidence of hospitalized mortality and length of ICU stay and total hospital stay were also lower or shorter in viral sepsis (12.1% vs. 39.7%, 6.5 days vs. 23.0 days and 21.0 days vs. 33.0 days, respectively) (all p < 0.001). Similar results were obtained after being adjusted by age, gender, comorbidity and PaO2/FiO2. Lymphocytopenia and high acute physiology and chronic health evaluation II scores were common risk factors for in-hospital death. While the death cases of COVID-19 sepsis mostly occurred at the later stages of patients’ hospital stay.
Conclusion
Critical COVID-19 shares clinical characteristics with classical bacterial sepsis, but the degree of systemic inflammatory response, secondary organ damage and mortality rate are less severe. However, following 2019-nCoV infection, the level of immunosuppression may be increased and thus induce in more death at the later stage of patients’ hospitalstay.
Keywords: Clinical characteristics, Immunosuppression, Mortality, COVID-19, Carbapenem-resistant klebsiella pneumonia (CrKP)
Introduction
Sepsis is a severe clinical syndrome, defined as the host body's dysregulated response to infection, which leads to life-threatening organ dysfunction. Sepsis is classically understood to be the result of bacterial infections, but the pathogenic microorganisms may also be viral, fungal, or other pathogens.1,2 However, sepsis caused by a virus or fungus is rarely reported, and there is a lack of clinical comparative data regarding the clinical characteristics and prognosis between sepsis caused by viral and classical bacterial infections.2,3
COVID-19 is an infectious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2, 2019-nCoV).4 Between December 2019 and October 28, 2021, there were more than 125,763 people infected with 2019-nCoV, with 5696 deaths in China. Globally, 4,984,415 people died, including a large number of health workers, which made COVID-19 one of the most serious problems for all health staff and researchers.5,6 COVID-19 has also been suggested to be a typical viral septic pulmonary infection with 2019-nCoV, which causes systemic inflammation and immune response dysregulation, potentially leading to multi-organ dysfunction and death.7
The pathophysiology of sepsis is the dysregulation of inflammation and immune response after an infection, while classic bacterial sepsis is primarily characterized by excessive inflammation and acquired immunosuppression. Excessive inflammation includes the activation of a large number of innate immune cells and the release of a variety of inflammatory mediators, causing an inflammatory “storm” that impacts organs and may lead to multiple organ dysfunction syndrome (MODS). While immunosuppression is mostly attributed to the apoptosis and depletion of lymphocytes, including CD4+, CD8+ and B lymphocytes.1,8 A similar process of inflammation and immune changes has been suggested for viral sepsis, which can also lead to a delay in virus clearance, further aggravating viral infection and even the progression of sepsis by secondary bacterial infection.9,10
It has been reported that COVID-19 can also cause an inflammatory storm characterized by an increase in a variety of inflammatory mediators, including IL-6, and immunosuppression, which is characterized by decreased lymphocytes. Collectively this can develop into acute respiratory distress syndrome (ARDS) and MODS.11,12 There are many similarities with sepsis caused by classic pulmonary bacterial infections, but the preliminary clinical outcomes of patients with COVID-19 suggest unique characteristics, such as more severe lung injury and a rapid decrease in lymphocytes, while the systemic inflammatory response and organ damage are not as serious, e.g. moderately elevated IL-6 concentration and milder liver, kidney and other organ injuries at early stages of COVID-19.13,14 However, no comparative study has been reported to evaluate the clinical characteristics and prognosis between COVID-19-associated sepsis and classical bacterial sepsis; thus the understanding of COVID-19 remains empirical.
According to the diagnostic criteria for sepsis 3.0 (more than 2 points of sequential organ failure assessment [SOFA] scores caused by infection), the current study enrolled patients admitted to ICU with viral sepsis caused by 2019-nCoV pulmonary infection (critical COVID-19) and patients with classical bacterial sepsis caused by carbapenem-resistant klebsiella pneumonia (CrKP) pulmonary infection in the ICU. CrKP is also a refractory and severe pulmonary infection, and easily progresses to ARDS and sepsis, similar to the clinical manifestations of critical COVID-19. By comparing the clinical characteristics and prognosis of the two diseases, this study may be able to provide insights into a better understanding of COVID-19 and possible treatment options.
Methods
Study design, setting, and participants
This retrospective cohort study was designed by the investigators and was in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology Guidelines. The data of critical COVID-19-associated sepsis were collected from patients in the ICU of a government-designed therapy center in Shenzhen, China, between January 14, 2020 and March 10, 2020. The data cutoff was March 25, 2020. Data for bacterial-induced sepsis was collected from patients with CrKP infection in the ICU of the Second People's Hospital of Shenzhen, China, between January 1, 2014 and October 30, 2019.
Data collection and analysis were approved by the Human Research Protection Office at our institution with waiver of informed consent. The study analyzed de-identified data from the hospital's healthcare informatics group, which was supervised by the current study's investigator. The study protocol was approved by the Second People's Hospital of Shenzhen (institutional review board number 202003009004).
The patients enrolled met the following criteria: based on 2019-nCoV or CrKP pulmonary infection, and SOFA score >2 points. COVID-19 sepsis: only pulmonary 2019-nCoV infection without other viral or bacterial infection and met the sepsis diagnostic criteria in the ICU within the study period; CrKP sepsis: only pulmonary CrKP infection without viral or bacterial infection in other parts and met the sepsis diagnostic criteria in the ICU within the study period. Exclusion criteria were <18 years of age, pregnant, or diagnosed with other acute diseases that could affect prognosis.
Data collection and definitions
Data regarding patient’ demographics, comorbidities, biochemical parameters, and outcomes were extracted from electronic health records. Acute physiology and chronic health evaluation II (APACHE II) score was calculated within the first 24 h of hospitalization. To ascertain the epidemiological and symptom data, which were not available from electronic medical records, investigators communicated directly with patients or their relatives. If data were missing from the records or clarification was needed, data were obtained via direct communication with attending doctors. All data were confirmed by three physicians.
A confirmed case of COVID-19 was defined as a positive result using a real-time reverse transcriptase polymerase chain reaction (RT-PCR) assay of pharyngeal swab specimens. COVID-19 was determined to be severe or critical as defined by the 7th Chinese national guidelines for COVID-19.15 Discharges criteria included patients being negative two times at a 24-h interval based on results from an RT-PCR assay of a pharyngeal swab specimen. A confirmed case of CrKP pneumonia was defined as a CrKP positive result on microbial cultivation from alveolar lavage or sputum culture with symptoms and signs of lung that cannot be explained by other causes.
Main outcomes
The primary outcome was the in-hospital mortality (28-day and 60-day) and the secondary outcomes including clinical features and interventions.
Statistical analysis
The categorical data were summarized as numbers and percentages, and inter-group comparisons were performed using χ2 test or Fisher's exact test. Continuous variables were expressed as the arithmetic mean and standard deviation (SD) or as the median and interquartile range, depending on the presence of a gaussian distribution. Continuous data with Gaussian distribution were compared using the Student's t-test or one-way ANOVA; for those with a non-Gaussian distribution, the Wilcoxon rank-sum test was used. Univariate analysis and multivariable logistic regression with a forced entry method was used to identify risk factors with a crude model and fully adjusted model: odds ratio (OR) and 95% confidence interval levels (95% CI). The 28-day and 60-day survival analysis were performed using Kaplan-Meier (K-M) analysis and log-rank tests. Statistical analysis was performed using the STATA (version 15.1). p values (two-tailed) below 0.05 were considered statistically significant.
Results
Patient demographic and clinical characteristics
The detailed demographic and clinical characteristics of all septic patients with CrKP pneumonia or COVID-19 at baseline are summarized in Table 1. A total of 107 patients with COVID-19 and 63 patients with CrKP pneumonia were enrolled. Compared with COVID-19-associated sepsis, bacterial sepsis (CrKP group) had more comorbidities (73% vs. 43%, p < 0.001), including hypertension, coronary heart disease, cerebrovascular disease, chronic obstructive pulmonary disease, and chronic renal insufficiency. There were no significant differences in age, gender, and body temperature, however, PaO2 and PaO2/FiO2 was lower in the COVID-19 group.
Table 1.
Baseline characteristics of CRKP sepsis and COVID-19 sepsis.
| Characteristics | CrKP (n = 63) | COVID-19 (n = 107) | χ2/t/Z value | p value |
|---|---|---|---|---|
| Age (years) | 59.9 ± 17.7 | 58.1 ± 15.1 | 0.673 | 0.517 |
| ≤60 | 32 (50.8) | 54 (50.5) | 0.002 | 0.967 |
| >60 | 31 (49.2) | 53 (49.5) | 0.002 | 0.967 |
| Male | 46 (73.0) | 68 (63.6) | 1.608 | 0.205 |
| Temperature (°C) | 37.3 ± 1.4 | 37.3 ± 0.9 | −0.221 | 0.626 |
| <37.3 | 34 (54.0) | 62 (57.9) | 0.255 | 0.634 |
| 37.3–38.0 | 17 (27.0) | 23 (21.5) | 0.664 | 0.456 |
| 38.1–39.0 | 5 (7.9) | 15 (14.0) | 1.413 | 0.325 |
| >39.0 | 7 (11.1) | 7 (6.5) | 1.095 | 0.384 |
| Comorbidities | 46 (73.0) | 46 (43.0) | 14.397 | <0.001 |
| Hypertension | 21 (54.0) | 32 (29.9) | 9.667 | 0.002 |
| Diabetes | 16 (25.4) | 15 (14.02) | 3.443 | 0.064 |
| Heart disease | 18 (28.6) | 8 (7.5) | 13.620 | <0.001 |
| Cerebrovascular disease | 14 (22.2) | 4 (3.7) | 14.310 | <0.001 |
| Chronic obstructive pulmonary disease | 10 (15.9) | 3 (2.8) | 9.590 | <0.001 |
| Malignancy | 6 (9.5) | 4 (3.7) | 2.397 | 0.176 |
| Chronic renal insufficiency | 16 (25.4) | 2 (1.9) | 23.185 | <0.001 |
| Blood gas analysis | ||||
| pH | 7.42 ± 0.09 | 7.42 ± 0.06 | −0.024 | 1.000 |
| PO2 (mmHg) | 128.3 ± 35.4 | 89.7 ± 34.1 | 6.618 | <0.001 |
| PCO2 (mmHg) | 39.9 ± 9.8 | 35.9 ± 5.44 | 3.147 | 0.038 |
| PaO2/FiO2 | 302.0 ± 106.2 | 242.3 ± 106.1 | 2.737 | 0.007 |
Data are expressed as n (%) or mean ± SD.
CrKP: carbapenem-resistant klebsiella pneumonia.
Clinical features and treatments
Patients with COVID-19 had milder systemic inflammatory response, primarily manifested as lower procalcitonin and C-reactive protein concentration, white blood cell count, neutrophil cell count, monocyte count, neutrophil lymphocyte ratio and platelets lymphocyte ratio compared with CrKP patients (all p < 0.05). There was a significant difference in immune and organ function in patients with COVID-19, presented as lower lymphocyte count (0.8 × 109/L vs. 0.9 × 109/L), total bilirubin (μmol/L, 12.5 vs. 15.9), alanine aminotransferase (U/L, 28 vs. 47), blood urea nitrogen (mmol/L, 5.5 vs. 12.5), creatine kinase isomer-MB (ng/mL, 1.04 vs. 2.00), N-terminal pro-B natriuretic peptide (pg/mL, 117.0 vs. 1121.0), lactic acid (mmol/L, 2.1 vs. 2.3), international standard ratio (0.99 vs. 1.11), D-dimer (mg/L, 0.82 vs. 3.33), but higher albumin concentration (g/L, 34.7 vs. 31.1) and mean artery pressure (mmHg, 94.7 vs. 88.1) (all p < 0.05, Table 2). Patients with COVID-19 had a lower SOFA score (3.0 vs. 6.0), APACHE II score (7.0 vs. 20.0), more corticosteroid (78.5% vs. 38.1%), and extracorporeal membrane oxygenation (ECMO) therapy (6.5% vs. 0), however, less continuous renal replacement therapy (CRRT) (15.4% vs. 36.5%) and invasive ventilation (31.6% vs. 54.0%) was observed (all p < 0.05, Table 2).
Table 2.
Clinical features and treatments of patients in CRKP sepsis and COVID-19 sepsis.
| Variables | CrKP (n = 63) | COVID-19 (n = 107) | χ2/t/Z value | p value |
|---|---|---|---|---|
| Inflammatory parameters | ||||
| PCT (ng/mL) | 0.8 (0.3–2.0) | 0.1 (0.0–0.2) | 8.367 | <0.001 |
| CRP (mg/L) | 91.2 (36.0–133.7) | 30.8 (14.0–68.0) | 5.635 | <0.001 |
| Blood routine tests | ||||
| WBC (1 × 109/L) | 11.1 (8.3–15.3) | 6.3 (4.8–8.8) | 6.066 | <0.001 |
| <4 | 3 (4.8) | 18 (16.8) | 5.327 | 0.046 |
| 4-10 | 22 (34.9) | 72 (67.3) | 16.807 | <0.001 |
| >10 | 38 (60.3) | 17 (15.9) | 35.765 | <0.001 |
| Neutrophils (1 × 109/L) | 9.1 (6.5–12.3) | 4.7 (3.3–7.6) | 5.372 | <0.001 |
| <3.9 | 10 (15.9) | 37 (34.6) | 9.512 | 0.012 |
| 3.9–6.3 | 6 (9.5) | 29 (27.1) | 7.495 | 0.006 |
| >6.3 | 47 (74.6) | 37 (34.6) | 25.412 | <0.001 |
| Lymphocyte (1 × 109/L) | 0.9 (0.8–1.7) | 0.8 (0.5–1.1) | 2.465 | 0.014 |
| ≤0.05 | 7 (11.1) | 22 (20.6) | 2.503 | 0.141 |
| 0.6–1.0 | 28 (44.4) | 51 (47.7) | 0.165 | 0.751 |
| ≥1.1 | 28 (44.4) | 34 (31.8) | 2.747 | 0.103 |
| Monocytes (1 × 109/L) | 0.7 (0.4–1.0) | 0.4 (0.3–0.5) | 4.147 | <0.001 |
| ≤0.6 | 30 (47.62) | 87 (81.31) | 20.975 | <0.001 |
| >0.6 | 33 (52.38) | 20 (18.69) | 20.974 | <0.001 |
| HGB (g/L) | 94.9 ± 16.1 | 130.3 ± 18.7 | −12.527 | <0.001 |
| Platelets (1 × 109/L) | 200.4 ± 111.3 | 188.8 ± 58.6 | 0.888 | 0.949 |
| <100 | 10 (15.9) | 6 (5.6) | 4.901 | 0.032 |
| 100-150 | 11 (17.5) | 19 (17.8) | 0.002 | 1.000 |
| >150 | 42 (66.7) | 82 (76.6) | 1.997 | 0.211 |
| PLR | 174.7 (86.1–311.2) | 217.9 (147.8–342.1) | −2.020 | 0.043 |
| NLR | 8.3 (4.4–12.8) | 5.4 (3.3–11.5) | 1.979 | 0.048 |
| Biochemical parameters | ||||
| Albumin (g/L) | 31.1 ± 4.7 | 34.7 ± 3.5 | −4.024 | <0.001 |
| TBiL (μmol/L) | 15.9 (11.5–25.4) | 12.5 (8.8–17.5) | 2.779 | 0.005 |
| ALT (U/L) | 47.0 (24.0–80.0) | 28.0 (18.1–38.5) | 4.192 | <0.001 |
| Creatinine (μmol/L) | 83.0 (47.3–148.1) | 65.3 (54.0–86.0) | 1.753 | 0.080 |
| BUN (mmol/L) | 12.5 (7.5–19.9) | 5.5 (4.0–7.2) | 6.498 | <0.001 |
| Coagulation parameters | ||||
| PT (s) | 12.9 ± 2.4 | 13.0 ± 1.1 | −0.177 | 0.860 |
| APTT (s) | 32.9 ± 7.6 | 34.4 ± 6.1 | −1.050 | 0.131 |
| INR | 1.1 ± 0.2 | 0.99 ± 0.1 | 3.200 | 0.003 |
| D-dimer (mg/L) | 3.3 (1.9–8.1) | 0.8 (0.6–2.2) | 4.729 | <0.001 |
| Cardiac function | ||||
| cTnI (ug/L) | 0.01 (0.01–0.02) | 0.01 (0.01–0.02) | 1.170 | 0.298 |
| CKMB (ng/mL) | 2.00 (2.00–3.12) | 1.04 (0.50–1.77) | 4.610 | <0.001 |
| ProBNP (pg/mL) | 1121.0 (399.5–4395.0) | 117.0 (41.0–510.0) | 6.130 | <0.001 |
| Lact (mmol/L) | 2.3 (1.6–3.2) | 2.1 (1.5–2.6) | 1.199 | 0.230 |
| MAP (mmHg) | 88.1 ± 14.4 | 94.7 ± 13.2 | −3.040 | 0.005 |
| SOFA score | 6.0 (4.0–10.0) | 3.0 (2.0–4.0) | 6.226 | <0.001 |
| APACHII score | 20.0 (13.5–24.5) | 7.0 (5.0–9.8) | 7.552 | <0.001 |
| Corticosteroid therapy | 24 (38.1) | 84 (78.5) | 27.946 | <0.001 |
| ECMO | 0 (0) | 7 (6.5) | 4.299 | 0.038 |
| CRRT | 23 (36.5) | 6 (15.4) | 5.282 | 0.022 |
| Invasive ventilation | 34 (54.0) | 24 (31.6) | 7.101 | 0.008 |
Data are expressed as median (IQR), n (%) or mean ± SD.
CrKP: carbapenem-resistant klebsiella pneumonia; PCT: procalcitonin; CRP: C-reaction protein; WBC: white blood cell; HGB: haemoglobin; PLR: platelets lymphocyte ratio; NLR: neutrophil lymphocyte ratio; TBiL: total bilirubin; ALT: alanine aminotransferase; BUN: blood urea nitrogen; PT: prothrombin time; APTT: activated partial thromboplastin time; INR: international standard ratio; CK-MB: creatine kinase isomer-MB; ProBNP: N-terminal pro-B natriuretic peptide; Lact: lactic acid; MAP: mean artery pressure; SOFA: sequential organ failure assessment; APACHE II: acute physiology and chronic health evaluation II; ECMO: extracorporeal membrane oxygenation; CRRT: continuous renal replacement therapy.
Patient outcomes
Compared with CrKP patients, COVID-19 patients had lower in-hospital mortality (12.1% vs. 39.7%, p < 0.001), shorter ICU length of stay (6.5 days vs. 23.0 days, p < 0.001) and total length of hospital stay (21.0 days vs. 33.0 days, p < 0.001). Considering the probable effects of the differences in age, gender, and underlying disease on prognosis, an adjusted analysis for age, gender, comorbidity, and PaO2/FiO2 were performed. A significant difference regarding in-hospital mortality (p = 0.002), ICU length of stay (p < 0.001), and total length of hospital stay (p < 0.001) remained (Table 3). Further analysis of the survival curve within 28 days and 60 days also showed that death from the CrKP often occurred in early stage, while death from the COVID-19 were mostly in the later stage (Fig. 1, Fig. 2).
Table 3.
Outcomes of patients with CRKP sepsis and COVID-19 sepsis.
| Outcomes | CrKP (n = 63) | COVID-19 (n = 107) | p value | Adjusted p valuea |
|---|---|---|---|---|
| In-hospital mortality | 25 (39.7) | 13 (12.1) | <0.001 | 0.002 |
| ICU length of stay (days) | 23.0 (12.5, 94.6) | 6.5 (3.0, 11.8) | <0.001 | 0.005 |
| Total length of hospital stay (days) | 33.0 (19.5, 89.5) | 21.0 (16.0, 29.3) | <0.001 | 0.011 |
Data are expressed as n (%) or median (IQR).
CrKP: carbapenem-resistant klebsiella pneumonia; ICU: intensive care unit.
Adjusted for age, gender, comorbidity and PaO2/FiO2.
Fig. 1.
The 28-day survival curves between CrKP and COVID-19. CrKP: carbapenem-resistant klebsiella pneumonia.
Fig. 2.
The 60-day survival curves between CrKP and COVID-19. CrKP: carbapenem-resistant klebsiella pneumonia.
Risk factors for in-hospital mortality
In order to confirm the clinical features and interventions associated with in-hospital mortality between patients with COVID-19 sepsis and those with CrKP sepsis, a multivariable logistic regression analysis was performed. Identified risk factors for in-hospital mortality associated with COVID-19 sepsis included lymphocyte (OR 0.12, p = 0.045), neutrophil lymphocyte ratio (NLR) (OR 1.06, p = 0.028), APACHE II score (OR 1.34, p < 0.001), SOFA score (OR 2.11, p = 0.002) and ECMO (OR 33.76 p < 0.001). There were no significant differences in comorbidities (OR 1.41, p = 0.12), corticosteroid therapy (OR 1.04, p = 0.96), and invasive ventilation (OR 2.29, p = 0.241) (Table 4).
Table 4.
Clinical features and interventions associated with in-hospital mortality in COVID-19 sepsis with univariable and multivariable analysis.
| Variables | Univariable |
Multivariable Fully adjusted model |
||
|---|---|---|---|---|
| OR (95% CI) | p value | OR (95% CI) | p value | |
| Age | 1.06 (1.01, 1.11) | 0.021 | NA | |
| Male | 3.57 (0.75, 17.03) | 0.110 | NA | |
| Comorbidities | 0.81 (0.25, 2.65) | 0.725 | 0.34 (0.09, 1.32) | 0.120 |
| Lymphocyte | 0.09 (0.01, 0.66) | 0.018 | 0.12 (0.01, 0.95) | 0.045 |
| SOFA score | 1.81 (1.34, 2.44) | <0.001 | 2.11 (1.31, 3.40) | 0.002 |
| APACHE II score | 1.32 (1.14, 1.52) | <0.001 | 1.34 (1.14, 1.58) | <0.001 |
| NLR | 1.05 (1.01, 1.10) | 0.024 | 1.06 (1.01, 1.11) | 0.028 |
| Creatinine | 1.01 (1.00, 1.02) | 0.037 | 1.01 (1.00, 1.02) | 0.074 |
| PaO2 | 0.96 (0.93, 1.00) | 0.045 | 0.97 (0.93, 1.01) | 0.122 |
| Comorbidities | NA | 1.41 (0.49, 4.06) | 0.120 | |
| Corticosteroid therapy | NA | 1.04 (0.24, 4.47) | 0.960 | |
| ECMO | NA | 33.76 (4.43, 257.16) | <0.001 | |
| CRRT | NA | inf. (0.00, Inf) | 0.995 | |
| Invasive ventilation | NA | 2.29 (0.57, 9.18) | 0.241 | |
SOFA: sequential organ failure assessment; APACHE II: acute physiology and chronic health evaluation II; NLR: neutrophil lymphocyte ratio; ECMO: extracorporeal membrane oxygenation; CRRT: continuous renal replacement therapy, NA: not application.
Risk factors for in-hospital mortality and associated with CrKP sepsis included lower lymphocyte (OR 0.43, p = 0.04), platelets (OR 0.99, p = 0.046), and higher lactic acid (OR 2.75, p = 0.002), activated partial thromboplastin time (OR 1.13, p = 0.005), APACHE II score (OR 1.30, p < 0.001), corticosteroid therapy (OR 0.08, p = 0.002), and CRRT (OR 3.71, p = 0.022). There were no significant differences in comorbidities (OR 0.97, p = 0.957) and invasive ventilation (OR 1.41, p = 0.524) (Table 5).
Table 5.
Clinical features and interventions associated with in-hospital mortality in CrKP sepsis with univariable and multivariable analysis.
| Variables | Univariable |
Multivariable fully adjusted model |
||
|---|---|---|---|---|
| OR (95% CI) | p value | OR (95% CI) | p value | |
| Age | 0.99 (0.96, 1.02) | 0.376 | NA | |
| Male | 0.34 (0.11, 1.07) | 0.064 | NA | |
| Comorbidities | 0.66 (0.21, 2.03) | 0.468 | 0.97 (0.27, 3.43) | 0.957 |
| Lymphocyte | 0.57 (0.28, 1.17) | 0.127 | 0.43 (0.19,0.96) | 0.040 |
| SOFA score | 1.81 (1.34, 2.44) | <0.001 | 1.15 (1.00, 1.33) | 0.053 |
| APACHE II score | 1.32 (1.14, 1.52) | <0.001 | 1.30 (1.13, 1.50) | <0.001 |
| PCT | 1.06 (0.97, 1.15) | 0.192 | 1.05 (0.97, 1.14) | 0.225 |
| Platelets | 0.99 (0.99, 1.00) | 0.026 | 0.99 (0.99, 1.00) | 0.046 |
| Total bilirubin | 1.03 (0.99, 1.06) | 0.150 | 1.02 (0.99, 1.06) | 0.156 |
| Creatinine | 1.00 (1.00, 1.01) | 0.107 | 1.00 (1.00, 1.01) | 0.072 |
| Lact | 2.65 (1.47, 4.78) | 0.001 | 2.75 (1.47, 5.12) | 0.002 |
| PCO2 | 0.95 (0.90, 1.01) | 0.120 | 0.96 (0.90, 1.02) | 0.176 |
| APTT | 1.16 (0.96, 1.40) | 0.129 | 1.13 (1.04, 1.23) | 0.005 |
| Comorbidities | NA | 0.97 (0.27, 3.43) | 0.957 | |
| Corticosteroid therapy | NA | 0.08 (0.02, 0.41) | 0.002 | |
| CRRT | NA | 3.71 (1.20, 11.43) | 0.022 | |
| Invasive ventilation | NA | 1.41 (0.49, 4.06) | 0.524 | |
SOFA: sequential organ failure assessment; APACHE II: acute physiology and chronic health evaluation II.
PCT: procalcitonin; Lact: lactic acid; APTT: activated partial thromboplastin time; CRRT: continuous renal replacement therapy; NA: not application, CrKP: carbapenem-resistant klebsiella pneumonia.
Subgroup analysis
In the survivors group, compared to patients with CrKP, the patients with COVID-19 manifested as lower lymphocytes (0.89 × 109/L vs. 0.99 × 109/L, p = 0.025), diminished neutrophils (4.54 × 109/L vs. 8.62 × 109/L, p < 0.001), and reduced oxygenation index (245.12 vs. 311.30, p = 0.007), but higher corticosteroid therapy (78.72% vs. 50.00%, p = 0.001). In addition, among the non-survivor group, lymphocytopenia was also more identified in patients with COVID-19 (0.88 × 109/L vs. 0.52 × 109/L, p = 0.007), like the more required corticosteroid therapy (Table 6).
Table 6.
Comparisons between survivor and non-survivor in CrKP sepsis and COVID-19 sepsis.
| Variables | Survivors |
Non-survivors |
||||
|---|---|---|---|---|---|---|
| CrKP (n = 38) | COVID-19 (n = 94) | p value | CrKP (n = 25) | COVID-19 (n = 13) | p value | |
| Age (years) | 61.5 (15.9) | 56.9 (14.5) | 0.170 | 57.4 (20.4) | 67.5 (16.7) | 0.161 |
| ≤60 | 18 (47.4) | 51 (54.3) | 0.565 | 14 (56.0) | 3 (23.1) | 0.086 |
| >60 | 20 (52.6) | 43 (45.7) | 0.565 | 11 (44.0) | 10 (76.9) | 0.086 |
| Comorbidities | 29 (76.3) | 41 (43.6) | <0.001 | 6 (24.0) | 5 (38.5) | 0.080 |
| Lymphocyte (1 × 109/L) | 1.0 (0.8–2.0) | 0.9 (0.6–1.2) | 0.025 | 0.9 (0.6–1.5) | 0.5 (0.4–0.9) | 0.014 |
| ≤0.5 | 3 (7.9) | 16 (17.0) | 0.274 | 4 (16.0) | 6 (46.2) | 0.062 |
| 0.6–1.0 | 17 (44.7) | 45 (47.9) | 0.284 | 11 (44.0) | 6 (46.2) | 1000 |
| ≥1.1 | 18 (47.4) | 33 (35.1) | 0.237 | 10 (40.0) | 1 (7.7) | 0.057 |
| Neutrophils (1 × 109/L) | 8.6 (6.8–11.0) | 4.5 (3.2–7.1) | <0.001 | (5.8–14.9) | 7.7 (3.7–8.8) | 0.202 |
| <3.9 | 5 (13.2) | 37 (39.4) | 0.004 | 5 (20.0) | 4 (30.8) | 0.689 |
| 3.9–6.3 | 3 (7.9) | 28 (29.8) | 0.011 | 3 (12.0) | 1 (7.7) | 1.000 |
| >6.3 | 30 (79.0) | 29 (30.9) | <0.001 | 17 (68.0) | 8 (61.5) | 0.730 |
| Platelets (1 × 109/L) | 226.8 (111.1) | 190.7 (59.6) | <0.001 | 160.3 (101.0) | 175.2 (51.2) | 0.303 |
| >150 | 2 (5.3) | 5 (5.3) | 0.999 | 8 (32.0) | 1 (7.7) | 0.126 |
| 100-150 | 7 (18.4) | 17 (18.1) | 0.999 | 4 (16.0) | 2 (15.4) | 1.000 |
| <100 | 29 (76.3) | 72 (76.6) | 0.999 | 13 (52.0) | 10 (76.9) | 0.176 |
| PaO2/FiO2 | 311.30 (86.8) | 245.1 (109.1) | 0.007 | 288.0 (131.1) | 209.9 (65.3) | 0.281 |
| >300 | 22 (57.9) | 14 (38.9) | 0.112 | 12 (48.0) | 0 (0.0) | 0.238 |
| 200-300 | 12 (31.6) | 4 (11.1) | 0.048 | 8 (32.0) | 2 (66.7) | 0.284 |
| <200 | 4 (10.5) | 18 (50.0) | <0.001 | 5 (20.0) | 1 (33.3) | 0.530 |
| Corticosteroid therapy | 19 (50.0) | 74 (78.7) | 0.001 | 5 (20.0) | 10 (76.9) | 0.001 |
Data are expressed as n (%) or median (IQR).
Discussion
This study compared the clinical features and prognosis between COVID-19-associated sepsis and classical bacterial sepsis. Results demonstrated that although the degree of primary lung injury caused by 2019-nCoV infection was more serious (indicated by lower PaO2/FiO2) than of patients with CrKP, the systemic inflammatory response was milder, SOFA score was lower, and organ function damage, such as the heart, liver, kidney, coagulation, and circulation were less severe. Meanwhile, the acquired immunosuppression was more severe, indicated by lower lymphocyte counts.16 Moreover, the proportion of corticosteroid therapy and ECMO support in patients with COVID-19 was even higher, while the use of CRRT and invasive mechanical ventilation was lower, compared with CrKP patients. The hospitalized mortality, ICU and total length of hospital stay were also lower/shorter than that in patients with viral sepsis, and similar results were obtained after being adjusted for age, gender, and PaO2/FiO2. Lymphocytopenia and high APACH II scores were common risk factors for the in-hospital mortality, and the death due to COVID-19 sepsis occurred mostly at the later stages of hospitalization. These results provided a reference for improved understanding of 2019-nCoV infection and possible treatment measures.
Since CrKP is similar to COVID-19 with characteristics that may lead to systemic sepsis, we compared the characteristics of viral sepsis caused by 2019-nCoV with simple pulmonary CrKP infection. In order to eliminate selection bias, this study included all patients who were admitted to the ICU within the study time period. Firstly, the two study groups were similar from the primary infection, and pulmonary function in patients with COVID-19 is worse than that of CrKP, which is similar to the severe pulmonary damage caused by 2019-nCoV in the postmortem examination.17,18 2019-nCoV can directly invade type II alveolar epithelial cells, causing an inflammatory storm characterized by the increase of inflammatory mediators such as IL-6. Inflammatory cell infiltration by cells such as mononuclear phagocyte system, neutrophils, and lymphocytes, may lead to ARDS characterized by massive necrosis, exudation, and hyaline membrane formation of alveolar cells.19,20 This is similar to bacterial pulmonary infection, but there are also differences. Gram-negative bacteria also infect local bronchial or alveolar epithelium, and then release a large number of inflammatory mediators such as IL-6 under the action of bacterial endotoxin lipopolysaccharide. This is accompanied by the infiltration of neutrophils and macrophages, resulting in cell necrosis, exudation, and hyaline membrane degeneration, which may then develop to ARDS.21,22 However, the difference is that the bacterial infection is characterized by massive exudation (showing a large amount of purulent sputum and high lung water value), intensive inflammation and tissue consolidation as local manifestations, and then the infection will gradually spread to the whole lung. In the virus sepsis the evolution of inflammation was slower, the inflammatory exudation was less, but the cellulose-like exudation and cell necrosis were more apparent.19,20 This report is similar to the findings of the current study in that pulmonary function is more severely impaired in patients with COVID-19 sepsis.23,24
In patients with bacterial or viral sepsis, the progression of local infection leads to systemic inflammation and a dysregulated immune response, which in turn leads to secondary organ damage. Many studies explored classical bacterial sepsis, however, there is a paucity of data comparing bacterial and viral sepsis to gain further in-depth understanding of their clinical characteristics, especially for COVID-19. In the current study we found that even in cases with more severe primary infections, the systemic inflammatory response and secondary organ damage caused by viral sepsis were still milder than those in patients with bacterial sepsis. This may be related to the mechanism how they arouse systemic inflammatory response, i.e. the viral sepsis is mainly caused by the secondary spread of local alveolar inflammatory storm and the entry of virus into the blood.1,25,26 While bacteria can not only induce inflammation and injury of systemic and distant organs through bacteria entering the blood, but also through the release of a large number of pathogen-associated molecular patterns mediators, such as lipopolysaccharide, and so on. In addition, bacterial sepsis can further lead to the release of a large number of secondary damage-associated molecular patterns mediators, such as high mobility group box 1, which causes a larger and continuous inflammatory effect and finally leads to serious damage to distant organs, which is more likely to result in MODS and higher mortality.27,28 We also demonstrated more severe hypolymphocytosis in patients with COVID-19, suggesting that there may be more serious acquired immunosuppression. This phenomenon was in line with our current clinical observation that the course of COVID-19 is longer. The existence of secondary infection and lack of immune support at the later stages bring a high risk of rapid deterioration of the disease and possible death. Even in the case of more active breathing or ECMO support, the progression of the condition cannot be reversed, which is also consistent with the current survival results.13,14
We found that the proportion of corticosteroid therapy and ECMO support therapy were higher in the COVID-19 sepsis group, while the need of CRRT and invasive mechanical ventilation was less. The reason may be that the prominent manifestation in viral sepsis is more severe lung injury caused by local inflammation, while other organs are less affected. In order to control local inflammation and support severe respiratory failure, the use of corticosteroids and ECMO increased.29, 30, 31 Bacterial sepsis is more likely to lead to a systemic inflammatory reaction and secondary organ damage, including renal dysfunction, which requires systemic blood purification therapy to alleviate inflammation and improve organ damage, including kidney damage. At the same time, as secondary organ damage progresses, there are additional concerns about the use of ECMO, which is primarily represented by cardiopulmonary support.32 Invasive respiratory support acts as a treatment for severe respiratory infectious disease, but the establishment of artificial airways for invasive mechanical ventilation is more likely to lead to the spread of the virus, which limits the use of ventilators. The emergence of a large number of patients with COVID-19 in a short term may also be one of the reasons for the lack of medical resources, including ventilators.
Several limitations to the current study should be mentioned. Firstly, the project is a retrospective study. Secondly, using CrKP pulmonary infection to represent classical bacterial sepsis may cause certain bias. CrKP is a conditional pathogen infection and the immune function in patients may be weaker. In addition, the drugs and therapeutic methods available in clinics may be limited, which may lead to more serious conditions and a worse prognosis. Ultimately, the number of CrKP cases included was small. Thirdly, the two groups were not infected at the same time, and were used solely for longitudinal comparison. Treatment of CrKP was performed in a single critical care center over the 5 years of data collection, while data collected for patients with COVID-19 used the government designated critical care center for 3 months, so there may be differences in treatment approach and morbidity. Moreover, it is true that the detail mechanisms are still in progress. Presently the interpretation of clinical phenomena in this study mainly depends on the reasoning of literature.
In conclusion, based on existing clinical data, this study directly compared differences between sepsis caused by 2019-nCoV infection and classical bacterial infection. The results showed that critical COVID-19 has the clinical characteristics of typical sepsis and is comparable to classical bacterial sepsis; however, the degree of systemic inflammatory response, secondary organ damage and mortality are lower, while the immunosuppression may be more serious, with more death in later stages of hospital stay. These observations provide additional information to better understand the disease process of COVID-19 and other similar conditions, as well as the possible treatment methods.
Funding
This work was supported by grants from the PLA Logistics Research Project of China [18CXZ030, CWH17L020, 17CXZ008], Sanming Project of Medicine in Shenzhen (SZSM20162011) and Shenzhen Second People's Hospital Clinical Research Fund of Guangdong Province High-level Hospital Construction Project (Grant No 20173357201815, No 20193357003, No 20203357014).
Ethical statement
The study were approved by the Research Ethics Commission of General Hospital of Southern Theater Command of PLA(HE-2020-08) and by the Research Ethics Committee of the Shenzhen Second People's Hospital (20200422008), and the requirement for informed consent was waived by the Ethics Commission. All authors reviewed the manuscript and approved the publication. The data sets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Declaration of competing interest
The authors declare that they have no competing interests.
Footnotes
Peer review under responsibility of Chinese Medical Association.
Author contributions
Ming Wu and Zhi-Feng Liu were responsible for study concept and design. Ming Wu, Zhi-Ye Zou, Yan-Hong Chen, Cong-Lin Wang and Yong-Wen Feng were responsible for collecting data. Zhi-Ye Zou, Yan-Hong Chen and Yong-Wen Feng were responsible for statistical analysis. Zhi-Feng Liu, Ming Wu, Cong-Lin Wang and Yong-Wen Feng were responsible for drafting the manuscript.
References
- 1.Singer M., Deutschman C.S., Seymour C.W., et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3) J Am Med Assoc. 2016;315:801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lin G.L., McGinley J.P., Drysdale S.B., et al. Epidemiology and immune pathogenesis of viral sepsis. Front Immunol. 2018;9:2147. doi: 10.3389/fimmu.2018.02147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Gupta N., Richter R., Robert S., et al. Viral sepsis in children. Front Pediatr. 2018;6:252. doi: 10.3389/fped.2018.00252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Zhu N., Zhang D., Wang W., et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. 2020;382:727–733. doi: 10.1056/NEJMoa2001017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Real-time updates: national outbreak map of new coronary virus pneumonia. https://ncov.dxy.cn/ncovh5/view/pneumonia?from=dxmmmp
- 6.Ran L., Chen X., Wang Y., et al. Risk factors of healthcare workers with corona virus disease 2019: a retrospective cohort study in a designated hospital of Wuhan in China. Clin Infect Dis. 2020;71:2218–2221. doi: 10.1093/cid/ciaa287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Li H., Liu L., Zhang D.Y., et al. SARS-CoV-2 and viral sepsis: observations and hypotheses. Lancet. 2020;395:1517–1520. doi: 10.1016/S0140-6736(20)30920-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Grondman I., Pirvu A., Riza A., et al. Biomarkers of inflammation and the etiology of sepsis. Biochem Soc Trans. 2020;48:1–14. doi: 10.1042/BST20190029. [DOI] [PubMed] [Google Scholar]
- 9.Huang C., Wang Y., Li X., et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395:497–506. doi: 10.1016/S0140-6736(20)30183-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lin L., Lu L., Cao W., et al. Hypothesis for potential pathogenesis of SARS-CoV-2 infection--a review of immune changes in patients with viral pneumonia. Emerg Microb Infect. 2020;9:727–732. doi: 10.1080/22221751.2020.1746199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mehta P., McAuley D.F., Brown M., et al. COVID-19: consider cytokine storm syndromes and immunosuppression. Lancet. 2020;395:1033–1034. doi: 10.1016/S0140-6736(20)30628-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Liu J., Li S., Liu J., et al. Longitudinal characteristics of lymphocyte responses and cytokine profiles in the peripheral blood of SARS-CoV-2 infected patients. EBioMedicine. 2020;55:102763. doi: 10.1016/j.ebiom.2020.102763. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Zhou F., Yu T., Du R., et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395:1054–1062. doi: 10.1016/S0140-6736(20)30566-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Guan W.J., Ni Z.Y., Hu Y., et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. 2020;382:1708–1720. doi: 10.1056/NEJMoa2002032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.National Health Commission & State Administration of Traditional Chinese Medicine Diagnosis and treatment protocol for novel coronavirus pneumonia (trial version 7) Chin Med J. 2020;133:1087–1095. doi: 10.1097/CM9.0000000000000819. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Zou Z., Ren D., Chen R., et al. Persistent lymphopenia after diagnosis of COVID-19 predicts acute respiratory distress syndrome: a retrospective cohort study. Eur J Inflamm. 2021;19:1–7. doi: 10.1177/20587392211036825. [DOI] [Google Scholar]
- 17.Xu Z., Shi L., Wang Y., et al. Pathological findings of COVID-19 associated with acute respiratory distress syndrome. Lancet Respir Med. 2020;8:420–422. doi: 10.1016/S2213-2600(20)30076-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tian S., Hu W., Niu L., et al. Pulmonary pathology of early phase 2019 novel coronavirus (COVID-19) pneumonia in two patients with lung cancer. J Thorac Oncol. 2020;15:700–704. doi: 10.1016/j.jtho.2020.02.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Qin C., Zhou L., Hu Z., et al. Dysregulation of immune response in patients with COVID-19 in Wuhan, China. Clin Infect Dis. 2020;71:762–768. doi: 10.1093/cid/ciaa248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lu R., Zhao X., Li J., et al. Genomic characterization and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. Lancet. 2020;395:565–574. doi: 10.1016/S0140-6736(20)30251-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Leligdowicz A., Chun L.F., Jauregui A., et al. Human pulmonary endothelial cell permeability after exposure to LPS-stimulated leukocyte supernatants derived from patients with early sepsis. Am J Physiol Lung Cell Mol Physiol. 2018;315:L638–L644. doi: 10.1152/ajplung.00286.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Amit S., Mishali H., Kotlovsky T., et al. Bloodstream infections among carriers of carbapenem-resistant Klebsiella pneumoniae: etiology, incidence and predictors. Clin Microbiol Infect. 2015;21:30–34. doi: 10.1016/j.cmi.2014.08.001. [DOI] [PubMed] [Google Scholar]
- 23.Bain W., Yang H.P., Shah F.A., et al. COVID-19 versus non–COVID-19 acute respiratory distress syndrome: comparison of demographics, physiologic parameters, inflammatory biomarkers, and clinical outcomes. Ann Am Thorac Soc. 2021;8:1202–1210. doi: 10.1513/AnnalsATS.202008-1026OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Zhang J.G., Huang X., Ding D.Y., et al. Comparative study of acute lung injury in COVID-19 and non-COVID-19 patients. Front Med. 2021;8:666629. doi: 10.3389/fmed.2021.666629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Gu J., Gong E., Zhang B., et al. Multiple organ infection and the pathogenesis of SARS. J Exp Med. 2005;202:415–424. doi: 10.1084/jem.20050828. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Cillóniz C., Dominedò C., Magdaleno D., et al. Pure viral sepsis secondary to community-acquired pneumonia in adults: risk and prognostic factors. J Infect Dis. 2019;220:1166–1171. doi: 10.1093/infdis/jiz257. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wang H., Ward M.F., Sama A.E. Targeting HMGB1 in the treatment of sepsis. Expert Opin Ther Targets. 2014;18:257–268. doi: 10.1517/14728222.2014.863876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Cristina M.L., Alicino C., Sartini M., et al. Genoan Klebsiella pneumoniae research group. Epidemiology, management, and outcome of carbapenem-resistant Klebsiella pneumoniae bloodstream infections in hospitals within the same endemic metropolitan area. J Infect Public Health. 2018;11:171–177. doi: 10.1016/j.jiph.2017.06.003. [DOI] [PubMed] [Google Scholar]
- 29.Shang L., Zhao J., Hu Y., et al. On the use of corticosteroids for 2019-nCoV pneumonia. Lancet. 2020;395:683–684. doi: 10.1016/S0140-6736(20)30361-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Tang N., Li D., Wang X., et al. Abnormal coagulation parameters are associated with poor prognosis in patients with novel coronavirus pneumonia. J Thromb Haemostasis. 2020;18:844–847. doi: 10.1111/jth.14768. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Zhang C., Shi L., Wang F.S. Liver injury in COVID-19: management and challenges. Lancet Gastroenterol Hepatol. 2020;5:428–430. doi: 10.1016/S2468-1253(20)30057-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Falk L., Hultman J., Broman L.M. Extracorporeal membrane oxygenation for septic shock. Crit Care Med. 2019;47:1097–1105. doi: 10.1097/CCM.0000000000003819. [DOI] [PubMed] [Google Scholar]