Abstract
OBJECTIVES:
The Sepsis-3 definition states the clinical criteria for sepsis but lacks clear definitions of the underlying infection. To address the lack of applicable definitions of infection for sepsis research, we propose new criteria, termed the Linder-Mellhammar criteria of infection (LMCI). The aim of this study was to validate these new infection criteria.
DESIGN:
A multicenter cohort study of patients with suspected infection who were admitted to emergency departments or ICUs. Data were collected from medical records and from study investigators.
SETTING:
Four academic hospitals in Sweden, Switzerland, the Netherlands, and Germany.
PATIENTS:
A total of 934 adult patients with suspected infection or suspected sepsis.
INTERVENTIONS:
None.
MEASUREMENTS AND MAIN RESULTS:
Agreement of infection site classification was measured using the LMCI with Cohen κ coefficient, compared with the Calandra and Cohen definitions of infection and diagnosis on hospital discharge as references. In one of the cohorts, comparisons were also made to adjudications by an expert panel. A subset of patients was assessed for interobserver agreement.
MEASUREMENTS AND MAIN RESULTS:
The precision of the LMCI varied according to the applied reference. LMCI performed better than the Calandra and Cohen definitions (κ = 0.62 [95% CI, 0.59–0.65] vs κ = 0.43 [95% CI, 0.39–0.47], respectively) and the diagnosis on hospital discharge (κ = 0.57 [95% CI, 0.53–0.61] vs κ = 0.43 [95% CI, 0.39–0.47], respectively). The interobserver agreement for the LMCI was evaluated in 91 patients, with agreement in 77%, κ = 0.72 (95% CI, 0.60–0.85). When tested with adjudication as the gold standard, the LMCI still outperformed the Calandra and Cohen definitions (κ = 0.65 [95% CI, 0.60–0.70] vs κ = 0.29 [95% CI, 0.24–0.33], respectively).
CONCLUSIONS:
The LMCI is useful criterion of infection that is intended for sepsis research, in and outside of the ICU. Useful criteria for infection have the potential to facilitate more comparable sepsis research and exclude sepsis mimics from clinical studies, thus improving and simplifying sepsis research.
Keywords: definitions, diagnosis, infectious disease medicine, sepsis
Sepsis is defined as acute organ dysfunction that is caused by a dysregulated host response to infection (1). Although sepsis was described over 2,500 years ago, creating a precise definition of sepsis remains a struggle (1–4). Even among critical care physicians who are experienced in the care of patients with sepsis, there is considerable variation in its diagnosis (5). Whereas organ dysfunction has clear criteria that are defined by the Sequential Organ Failure Assessment (SOFA) score, the definition of underlying infections is not addressed in the Sepsis-3 definition (1).
A common approach in sepsis studies is to include all patients with suspected sepsis. As a result, at least one-fifth of included patients do not suffer from sepsis at all (6, 7). Consequently, the inclusion of such a large proportion of patients without sepsis in sepsis studies might prevent them from achieving significant results if this phenomenon is not considered in a priori power calculations. For example, two studies that included patients with suspected sepsis, in spite of indicative results, were unable to demonstrate any benefits of antibiotic administration in the very first hours after admission (8, 9). Perhaps the timing of antibiotic therapy is less important in hemodynamically stable patients, but the lack of benefit of the early administration of antibiotics in these studies might be attributed to some patients not having an infection.
Similarly, another study that included patients with suspected septic shock was unable to demonstrate benefits of adjunctive glucocorticoid therapy (11), unlike a separate report on corticosteroids in septic shock by Annane et al (10), who examined patients with septic shock with an infection that was diagnosed as the presence of a clinically or microbiologically documented infection.
Other definitions of the infectious component in sepsis research include the International Classification of Disease codes, expert judgment, and algorithms that take into account blood culture sampling and initiation of antibiotic therapy (1, 11, 12–15). These assessment methods demonstrate poor and variable precision in clinical sepsis, rendering comparisons between studies difficult (5, 16, 17). More detailed definitions of infection have been developed by the International Sepsis Forum and the Center for Disease Control (CDC) and the National Healthcare Safety Network (NHSN) (18, 19). The International Sepsis Forum definitions of infection in the ICU by Calandra and Cohen (18) contain comprehensive criteria, but they have not been evaluated. In addition, they define only six foci of infection and may be difficult to apply outside of the ICU, where most sepsis patients are treated (18, 20, 21).
Sepsis research and care need to be conducted with joint efforts both inside and outside of ICUs and involving different specialties, which is why definitions must be useful in these various settings. In addition, the Calandra and Cohen definitions rely heavily on microbiological findings, but not all patients with sepsis have positive microbiological samples. Culture sampling only identifies the causative organism in blood culture isolates (i.e., bacteremia) in 15–30% of sepsis patients. Additionally, 20–30% of sepsis patients have a pathogen that is isolated from other locations. Thus, at least 40% of sepsis patients are culture-negative, and culture-negative sepsis is a substantial cause of morbidity that must be examined further in sepsis studies (20, 22–26). The CDC and NHSN have established a surveillance definition for healthcare-associated infections and criteria for specific types of infections in the acute care setting (19). These definitions showed excellent interobserver agreement in a critical care study but have been validated only for specific infections and are intended primarily for healthcare-associated infections (27–29).
Clear and uniform definitions of infections, although imperfect, would facilitate comparative sepsis research and exclude sepsis mimics from clinical studies, improving the quality of sepsis research. To address the lack of universally applicable definitions of infection for sepsis research inside and outside of the ICU, we propose new criteria that evaluate the presence of an infection by considering 13 potential foci. The aim of this study was to evaluate the performance of these new criteria in diagnosing infections.
MATERIALS AND METHODS
Methods
This study was an analysis of randomly selected patients who were extracted from five cohort studies (Supplementary Appendix I, http://links.lww.com/CCX/A993). The first cohort was derived from a prospective study at Skåne University Hospital, Lund, Sweden (a tertiary hospital), that included febrile adult patients with the highest triage level according to the Rapid Emergency Triage and Treatment System (RETTS) and blood that was sampled for culture in the emergency department (ED). RETTS is presented in Supplementary Table XVII (http://links.lww.com/CCX/A993). The second cohort was derived from the Swedish Intensive Care Registry and included ICU patients with suspected sepsis. Systematic random sampling was employed to select cases. The third cohort included consecutively enrolled patients who were initially treated in the ED of University Hospital Bern, Bern, Switzerland, with blood that was sampled for culture or with suspected sepsis. The fourth cohort was derived from a prospective cohort of consecutively included patients in the “Acutelines” biobank in the ED of University Medical Center, Groningen, the Netherlands, consisting of patients with the three highest triage levels according to the Emergency Severity Index and suspicion of sepsis (30, 31). The fifth cohort was derived from a prospective cohort study that included consecutive patients with at least two Quick Sequential Organ Failure Assessment points and who were admitted to the ED of University Hospital, Jena, Germany. In the Jena cohort, radiological findings could not be graded as more or less likely to demonstrate infectious findings, and there were no data on auscultatory signs. Therefore, the analyses were repeated as a sensitivity analysis, with the Jena cohort excluded. For a detailed description of the separate cohorts, see Supplementary Appendix I (http://links.lww.com/CCX/A993).
Medical charts were reviewed by medically trained researchers for diagnosing infection.
Presence of infection and suspected focus was classified according to the hospital discharge note, our new infection criteria, and the definitions by Calandra and Cohen. In the cohort from Groningen, patients were adjudicated by an expert panel that consisted of two experts who were experienced in acute care and/or sepsis care, and a third expert made the final decision in case of disagreement.
Ethical approval was obtained from the relevant regional ethics boards (Lund file numbers 2015/285 and 2016/39; Bern file number 2019-01149; Acutelines/Groningen file number 2019/589; Jena file number 4912-08/16).
Criteria of Infection
The new criteria of infection, herein called “the Linder-Mellhammar criteria of infection (LMCI),” are intended for sepsis research and not for clinical practice. The LMCI are also intended for use only when an infection is suspected, not if another diagnosis is set (Supplementary Tables I–XIII, http://links.lww.com/CCX/A993). The LMCI were developed by simplifying the Calandra and Cohen definitions into a single table for each organ system, appending tables for organ systems that are not included in the Calandra and Cohen definitions and adding the possibility of infection without microbiological evidence when laboratory, clinical, and radiological evidences of infection exist. Modification of the Calandra and Cohen definitions was based on the clinical experience of two experienced physicians specialized in infectious diseases (Linder and Mellhammar).
The 13 foci comprise upper respiratory tract infections: lower respiratory tract infections; abdominal infections; gastrointestinal infections; urinary tract infections (UTIs); skin and soft-tissue infections; bone and joint infections; CNS infections; primary bloodstream infections; catheter-related bloodstream infections; ear, nose, and throat infections; fever in neutropenia; and reproductive tract infections. For each focus, 1–4 points are given for the following entities: symptoms and signs, radiological findings, laboratory findings, and microbiological findings in relation to the suspected infection. In certain foci, some of these entities are not applicable and are, therefore, not scored.
These entities are grouped as rows in a table for each infection focus (Supplementary Tables I–XII, http://links.lww.com/CCX/A993). The highest scores from all rows are added to yield a total score for each focus. The level of evidence of infection is graded by score: 0–1 points indicate no infection, 2 points indicate a possible infection, 3 points indicate a probable infection, and 4 or more points indicate a proven infection. A score for the level of evidence of infection is necessary because there is a zone of uncertainty between those with a proven infection and those without an infection. The focus with the highest score is considered the primary focus of the infection. For infective endocarditis, the modified Duke criteria by Li et al (32) have already been validated; thus, we did not include a score for endocarditis but recommend the Duke criteria for this use.
Statistics
The diagnostic accuracy of the LMCI was assessed with regard to sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for likelihood of infection, including their 95% CIs.
Because there is no gold standard definition for infection, the LMCI were compared with the diagnosis at discharge, the Calandra and Cohen definitions of infection, and, in the Groningen cohorts, adjudications by an expert panel. Each infection focus in the LMCI was dichotomized by placing all patients with a proven infection into one category (infection) and all patients with probable, possible, or no infection into another category (no infection).
The Calandra and Cohen definitions classify plausibility of infection for some foci on a 4-point scale (none, possible, probable, and definite or confirmed), whereas other foci are classified on a 2-point scale (no infection or infection). The foci in the Calandra and Cohen definition that used a 4-point scale were dichotomized by placing all patients with a possible, probably, definite, or confirmed infection into one category (infection) and all patients with no infection into another category. The “possible infection” category was chosen as the cutoff when dichotomizing the Calandra and Cohen definition to be relevant outside of the ICU. Foci in the Calandra and Cohen definitions that are classified as “present” or “absent” in the Calandra and Cohen definitions were regarded as definite infections when they were scored as “present” and as no infection when they were scored as “absent.” To facilitate comparison between the Calandra and Cohen definitions and the LMCI, estimates were based only on the foci that are included in the Calandra and Cohen definitions.
Agreement on foci of infection between the LMCI and the comparators was assessed by calculating the percentage and Cohen κ coefficient. In the first two cohorts, infections were classified in a randomly selected subset of patients using the LMCI by two assessors to evaluate interobserver agreement. In the Jena cohort, radiological findings were not graded as being more or less likely to demonstrate infection, and there were no data on auscultatory signs; thus, the analyses were repeated as a sensitivity analysis, with the Jena cohort excluded. As a sensitivity analysis, other cutoffs were used for dichotomizations of LMCI and of Calandra and Cohen definitions than that in the primary analysis.
Analyses were performed using Statistical Package for the Social Services, Version 24.0 (IBM, Armonk, NY).
RESULTS
In total, 934 patients were included in this analysis; see Figure 1 for a flow chart of inclusion and exclusion. The clinical characteristics of each cohort are presented in Table 1. A total of 334 patients were admitted to the ICU, and 600 were admitted to wards. The cohorts differed in terms of age, gender, comorbidities, and severity of disease. The patients in the Jena and Lund ED cohorts were older than the other cohorts. All cohorts other than the Bern cohort had an overrepresentation of males. The severity of disease, as measured by SOFA score, was highest in the Lund ICU cohort, in whom bacteremia was also more common. There were no differences in Charlson Comorbidity Index scores between the cohorts.
Figure 1.
Flowchart of inclusion. ED = emergency department.
TABLE 1.
Characteristics of Included Cohorts
| CharacteristicsName of Cohort | Cohorts | ||||
|---|---|---|---|---|---|
| Lund Emergency Department | Lund ICU | Bern | Groningen | Jena | |
| Years of inclusion | 2017–2018 | 2013–2014 | 2017–2019 | 2020–2021 | |
| Setting | Emergency department | ICU | Emergency department | Emergency department | Emergency department |
| Data source and study design | Retrospective study of EHRs | Prospective cohort study | Retrospective study of EHRs | Prospective cohort study | Prospective cohort study |
| Main inclusion criteria | Suspected sepsis | Suspected sepsis | Blood sample for culture or suspected sepsis | Suspected sepsis | Sepsis or septic shock |
| Patients, n | 251 | 197 | 207 | 163 | 116 |
| Female sex, n (%) | 102 (41) | 84 (43) | 121 (50) | 57 (35) | |
| Age, median (IQR) | 74 (65–81) | 69 (59–75) | 69 (55–78) | 69 (59–75) | 77 (66–82) |
| Length of stay, hospital, median (IQR) | 5.0 (3–7) | 4.9 (3.1–9.7) | 5.7 (2.1–12.7) | 11 (7–18) | |
| Length of stay, ICU, median (IQR) | 2.2 (1.1–5.8) | ||||
| Charlson Comorbidity Index, median (IQR) | 5 (3–7) | 4 (3–6) | 5 (3–8) | 4 (3–6) | 5 (3–6) |
| Comorbidities, n (%) | |||||
| Chronic pulmonary disease | 57 (23) | 18 (9) | 32 (16) | 42 (26) | 15 (13) |
| Cancer | 59 (24) | 64 (32) | 44 (21) | 60 (37) | 18 (16) |
| Cardiovascular disease | 134 (53) | 41 (21) | 61 (30) | 46 (28) | 29 (25) |
| Diabetes mellitus | 68 (27) | 42 (21) | 42 (20) | 49 (30) | 36 (31) |
| Liver disease | 3 (1) | 8 (4) | 14 (7) | 9 (6) | 5 (4) |
| Renal disease | 32 (13) | 20 (10) | 58 (28) | 15 (9) | 13 (11) |
| SOFA score at admission, median (IQR) | 0 (0–1) | 9 (6–11) | 2 (0–4) | 2 (1–3) | |
| SOFA score max < 72 hr, median (IQR) | 3 (2–4) | ||||
| ICU admission, n (%) | 15 (6) | 197 (100) | 53 (26) | 15 (9) | 54 (47) |
| Bacteremia, n (%) | 52 (21) | 96 (49) | 43 (21) | ||
| 28-d mortality, n (%) | 22 (9) | 22 (11) | 29 (18) | 21 (18) | |
EHR = electronic health records, IQR = interquartile range, SOFA = Sequential Organ failure Assessment.
Depending on the reference (Calandra and Cohen definitions or hospital discharge diagnosis), the diagnostic accuracy of the LMCI varied (Table 2). The LMCI had higher AUC and κ-values than the Calandra and Cohen definition (AUC, 0.82 [0.77–0.86] vs 0.68 [0.64–0.73] and κ = 0.62 [95% CI, 0.59–0.65] vs κ = 0.43 [95% CI, 0.39–0.47], respectively) when used to predict the hospital discharge diagnosis. Similarly, the LMCI had a higher diagnostic accuracy for the Calandra and Cohen definition compared with the hospital discharge diagnosis (AUC, 0.74 [0.70–0.77] vs 0.50 [0.47–0.54] and κ = 0.57 [95% CI, 0.53–0.61] vs κ = 0.43 [95% CI, 0.39–0.47], respectively). In the subgroup of patients who were admitted to the ICU, the Calandra and Cohen definition had a higher AUC value with regard to the probability of infection but was not significantly different and had worse κ agreement compared with the LMCI (Table 3).
TABLE 2.
Outcomes of Infection Classifications by the Linder-Mellhammar Criteria of Infection and by the Calandra and Cohen Definitions, With Hospital Discharge Diagnosis as the Reference, and for Linder-Mellhammar Criteria of Infection and Hospital Discharge Diagnosis, With the Calandra and Cohen Definitions as the Reference
| Variable/Test | Hospital Discharge Diagnosis as Reference | Calandra and Cohen Definitions as Reference | ||
|---|---|---|---|---|
| Linder-Mellhammar Criteria of Infection | Calandra and Cohen Definitions | Linder-Mellhammar Criteria of Infection | Hospital Discharge Diagnosis | |
| n | 934 | 934 | 934 | 934 |
| True positive | 498 | 379 | 378 | 379 |
| True negative | 179 | 85 | 211 | 85 |
| False positive | 65 | 159 | 185 | 311 |
| False negative | 192 | 311 | 160 | 159 |
| Agreement % | 72 | 50 | 63 | 50 |
| k (95% CI) | 0.62 (0.59–0.65) | 0.43 (0.39–0.47) | 0.57 (0.53–0.61) | 0.43 (0.39–0.47) |
| Sensitivity % | 72 (69–75) | 55 (51–59) | 70 (66–74) | 70 (66–74) |
| Specificity % | 73 (67–79) | 35 (29–41) | 53 (48–58) | 21 (18–26) |
| Area under curve | 0.82 (0.77–0.86) | 0.68 (0.64–0.73) | 0.74 (0.70–0.77) | 0.50 (0.47–0.54) |
TABLE 3.
Outcomes of Infection Classifications for ICU Patients Only by the Linder-Mellhammar Criteria of Infection and by the Calandra and Cohen Definitions, With Hospital Discharge Diagnosis as the Reference, and for Linder-Mellhammar Criteria of Infection and Hospital Discharge Diagnosis With the Calandra and Cohen Definitions as Reference
| Variable/Test | Hospital Discharge Diagnosis as Reference | Calandra and Cohen Definitions as Reference | ||
|---|---|---|---|---|
| Linder-Mellhammar | Calandra and Cohen Definitions | Linder-Mellhammar | Discharge Diagnosis | |
| n | 332 | 332 | 332 | 332 |
| True positive | 195 | 167 | 174 | 167 |
| True negative | 58 | 24 | 44 | 24 |
| False positive | 33 | 67 | 54 | 74 |
| False negative | 46 | 74 | 60 | 67 |
| Agreement % | 76 | 58 | 66 | 58 |
| k (95% CI) | 0.68 (0.63–0.74) | 0.52 (0.46–0.58) | 0.65 (0.59–0.71) | 0.52 (0.46–0.58) |
| Sensitivity % | 81 (75–86) | 69 (63–75) | 74 (69–80) | 71 (65–77) |
| Specificity % | 64 (53–74) | 26 (18–37) | 45 (35–55) | 25 (16–34) |
| Area under curve | 0.82 (0.74–0.90) | 0.88 (0.74–1.0) | 0.76 (0.69–0.83) | 0.69 (0.61–0.76) |
When tested with adjudication as the reference, the new criteria still performed better (κ = 0.65 [0.60–0.70]) than the Calandra and Cohen definitions (κ = 0.29 [0.24–0.33]) (Table 4).
TABLE 4.
Outcomes of Infection Classifications by the Linder-Mellhammar Criteria of Infection and by the Calandra and Cohen Definitions, With Expert Adjudication as the Reference
| Variable/Test | Adjudication Diagnosis | |
|---|---|---|
| Linder-Mellhammar | Calandra and Cohen Definitions | |
| n | 163 | 163 |
| True positive | 111 | 54 |
| True negative | 27 | 33 |
| False positive | 15 | 9 |
| False negative | 10 | 67 |
| Agreement % | 75 | 44 |
| k (95% CI) | 0.65 (0.60–0.70) | 0.29 (0.24–0.33) |
| Sensitivity % | 92 (85–96) | 45 (36–54) |
| Specificity % | 64 (48–78) | 79 (63–90) |
| Area under curve | 0.82 (0.77–0.86) | 0.68 (0.64–0.73) |
The sensitivity analyses indicated that excluding the cohort from Jena did not alter the results (Supplemental Table XVI, http://links.lww.com/CCX/A993). The performance of the three definitions and criteria of infection varied between foci (Supplemental Table XIV, http://links.lww.com/CCX/A993). The LMCI had higher sensitivities for lower respiratory tract and skin and soft-tissue infections but lower sensitivities and higher specificities for UTIs and abdominal infections. The interobserver agreement was tested in a subset of 91 randomly selected patients from the two cohorts from Lund, consisting of ED and ICU patients. The agreement was 77% with a κ value of 0.72 (95% CI, 0.60–0.85) (Supplemental Table XV, http://links.lww.com/CCX/A993). When using other cutoffs, that is, possible for LMCI, the κ value was 0.72 (95% CI, 0.69–0.75) for hospital discharge diagnosis, and when using definite or confirmed as cut offs for Calandra and Cohen definitions, κ value was 0.38 (95% CI, 0.34–0.42) for hospital discharge diagnosis.
DISCUSSION
A widely usable definition of infection would enable more comparable sepsis research and help exclude sepsis mimics from clinical studies, thereby improving the quality of sepsis research. For this purpose, we have developed the LMCI. Because sepsis is a condition that is treated by various medical specialists, including intensivists, acute care physicians, surgeons, and internal medicine specialists, useful criteria need to be applicable in and outside of the intensive care setting. The LMCI were established by infectious disease physicians by modifying the Calandra and Cohen definitions to also consider clinical information that is available outside of the ICU. In this multicenter analysis of 934 ED and ICU admissions, the LMCI demonstrated higher accuracy for infection diagnoses when compared with discharge diagnoses, the Calandra and Cohen definitions, and adjudication as the gold standard.
The Sepsis-3 definitions have improved sepsis research by providing clear and user-friendly definitions of sepsis. However, a definition of infection, which should be a critical component of the definition of sepsis, has not been included in Sepsis-3 (1). One problem in sepsis research is the heterogeneity of sepsis. Calandra and Cohen (18) have attempted to address the need for common definitions of infections, but their definitions are focused on the critical care setting and on microbiological results. Patients who are treated in the ICU receive a high level of diagnostic measures, with more microbiological sampling than patients treated outside of the ICU. However, most sepsis patients are treated outside of the ICU, and the Calandra and Cohen definitions of infection are not applicable in these cases (21). The LMCI fill this gap because they can be used both in and outside of the ICU with better performance and include culture-negative sepsis and additional foci of infection.
The CDC/NHSN surveillance definition of healthcare-associated infection and criteria for specific types of infections in acute care settings has been evaluated for certain infections, with widely varying results (28, 29, 33). A study of ICU patients demonstrated excellent interobserver agreement for the CDC/NHSN surveillance definition, but no comparison was made with a reference definition. Therefore, important patient groups may have been excluded, whereas other etiologies might have been included, resulting in bias in the studies (27). The need for more user-friendly definitions, such as the LMCI, is perhaps best reflected in the wide variety of definitions that are used in high-impact research (11, 12–14).
Our study is the largest multicenter validation of infection definitions and criteria but has several limitations. Its major limitation is the lack of a gold standard definition, which complicates the interpretation of the performance of the LMCI. Despite the lack of a gold standard, there is a need for validated definitions of infection. Therefore, we have included the three references that are most often used as gold standards in the field: the Calandra and Cohen definitions, the diagnosis at hospital discharge, and adjudications by an expert panel (15, 27, 34). All three standards have advantages and disadvantages. For example, the diagnosis at hospital discharge is not indicative of the likelihood of infection and may be considered to be too sensitive. As discussed above, the Calandra and Cohen definitions are intended for the ICU and rely on microbiological results. An adjudicated diagnosis might be subjective and variable. Thus, all three references were included as gold standards, and comparisons of the LMCI with all references were satisfactory.
A larger study cohort would have strengthened our conclusions, and there were insufficient cases to reliably analyze the performance of the LMCI in less common sites of infection. This study is the first validation of the LMCI, but for these criteria to be validated, they must be applied in several studies. It is possible that the criteria will be modified according to the validation and the increasing availability of rapid microbiological testing and biomarkers.
Further, the analyses of the κ coefficients were based on the correct classification of the infection foci. Various definitions and criteria sometimes categorize infectious foci differently, rather than incorrectly. For example, an infection may be classified as pneumonia by one definition and as a bloodstream infection in another. Therefore, we have also presented the AUCs values for the likelihood of infection.
The LMCI and Calandra and Cohen definitions differ in their assessment of infections, with the latter placing higher demands on the examination of patients with pneumonia and abdominal infections compared with the LMCI. These disparate criteria resulted in more patients being assigned as having a lower respiratory tract or abdominal infection when scored using the LMCI versus the Calandra and Cohen definition. These high standards of investigation are not practiced outside of the ICU, and definitions for infection must be adapted for outside of the ICU, as well. In addition, the LMCI have stricter requirements for the diagnosis of UTIs than the Calandra and Cohen definitions. This difference resulted in many patients not being interpreted as having a UTI when assessed using the LMCI but being classified as false negatives compared with the Calandra and Cohen definitions. The LMCI for UTIs was based on the predictive value of bacteriuria and performed well when compared with the hospital discharge diagnosis (35). The false negatives for primary bloodstream infections when patients were scored using the LMCI versus the Calandra and Cohen definition were due to patients having other sites of infection with a higher score (lower respiratory tract infections or abdominal infections). Thus, in these patients, bloodstream infections were considered secondary to the main focus of infection when scored per the LMCI. There were insufficient cases to reliably analyze the performance of the LMCI in less common sites of infection.
We argue that the LMCI are useful criteria for defining infections for sepsis research, demonstrating sufficient reliability and construct and criterion validity (36).
CONCLUSIONS
The LMCI are useful criteria of infection that are intended for sepsis research, both in and outside of the ICU, with the potential to improve and simplify sepsis research, given that current definitions of sepsis include no definition of infection.
ACKNOWLEDGMENTS
We thank Jane Fisher of AdvanSci Research Consulting and Sean Kim of Blue Pencil Science for editing this article. E.M. is funded by Wenner-Gren Foundations (FT2020-0003).
Supplementary Material
Footnotes
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccejournal).
The authors have disclosed that they do not have any potential conflicts of interest.
REFERENCES
- 1.Singer M, Deutschman CS, Seymour CW, et al. : The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315:801–810 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Geroulanos S, Douka ET: Historical perspective of the word “sepsis”. Intensive Care Med 2006; 32:2077. [DOI] [PubMed] [Google Scholar]
- 3.Bone RC, Balk RA, Cerra FB, et al. : Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992; 101:1644–1655 [DOI] [PubMed] [Google Scholar]
- 4.Levy MM, Fink MP, Marshall JC, et al. ; International Sepsis Definitions Conference: 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Intensive Care Med 2003; 29:530–538 [DOI] [PubMed] [Google Scholar]
- 5.Rhee C, Kadri SS, Danner RL, et al. : Diagnosing sepsis is subjective and highly variable: A survey of intensivists using case vignettes. Crit Care 2016; 20:89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Contou D, Roux D, Jochmans S, et al. : Septic shock with no diagnosis at 24 hours: A pragmatic multicenter prospective cohort study. Crit Care 2016; 20:360. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Heffner AC, Horton JM, Marchick MR, et al. : Etiology of illness in patients with severe sepsis admitted to the hospital from the emergency department. Clin Infect Dis 2010; 50:814–820 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Puskarich MA, Trzeciak S, Shapiro NI, et al. ; Emergency Medicine Shock Research Network (EMSHOCKNET): Association between timing of antibiotic administration and mortality from septic shock in patients treated with a quantitative resuscitation protocol. Crit Care Med 2011; 39:2066–2071 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ferrer R, Martin-Loeches I, Phillips G, et al. : Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: Results from a guideline-based performance improvement program. Crit Care Med 2014; 42:1749–1755 [DOI] [PubMed] [Google Scholar]
- 10.Annane D, Renault A, Brun-Buisson C, et al. ; CRICS-TRIGGERSEP Network: Hydrocortisone plus fludrocortisone for adults with septic shock. N Engl J Med 2018; 378:809–818 [DOI] [PubMed] [Google Scholar]
- 11.Venkatesh B, Finfer S, Cohen J, et al. ; ADRENAL Trial Investigators and the Australian–New Zealand Intensive Care Society Clinical Trials Group: Adjunctive glucocorticoid therapy in patients with septic shock. N Engl J Med 2018; 378:797–808 [DOI] [PubMed] [Google Scholar]
- 12.Seymour CW, Kahn JM, Martin-Gill C, et al. : Delays from first medical contact to antibiotic administration for sepsis. Crit Care Med 2017; 45:759–765 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Rhee C, Dantes R, Epstein L, et al. ; CDC Prevention Epicenter Program: Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014. JAMA 2017; 318:1241–1249 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bergh C, Fall K, Udumyan R, et al. : Severe infections and subsequent delayed cardiovascular disease. Eur J Prev Cardiol 2017; 24:1958–1966 [DOI] [PubMed] [Google Scholar]
- 15.Johnstone J, Meade M, Lauzier F, et al. ; Prevention of Severe Pneumonia and Endotracheal Colonization Trial (PROSPECT) Investigators and the Canadian Critical Care Trials Group: Effect of probiotics on incident ventilator-associated pneumonia in critically ill patients: A randomized clinical trial. JAMA 2021; 326:1024–1033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gaieski DF, Edwards JM, Kallan MJ, et al. : Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med 2013; 41:1167–1174 [DOI] [PubMed] [Google Scholar]
- 17.Wilhelms SB, Huss FR, Granath G, et al. : Assessment of incidence of severe sepsis in Sweden using different ways of abstracting international classification of diseases codes: Difficulties with methods and interpretation of results. Crit Care Med 2010; 38:1442–1449 [DOI] [PubMed] [Google Scholar]
- 18.Calandra T, Cohen J; International Sepsis Forum Definition of Infection in the ICU Consensus Conference: The international sepsis forum consensus conference on definitions of infection in the intensive care unit. Crit Care Med 2005; 33:1538–1548 [DOI] [PubMed] [Google Scholar]
- 19.Horan TC, Andrus M, Dudeck MA: CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control 2008; 36:309–332 [DOI] [PubMed] [Google Scholar]
- 20.Mellhammar L, Wullt S, Lindberg Å, et al. : Sepsis incidence: A population-based study. Open Forum Infect Dis 2016; 3:ofw207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Fleischmann-Struzek C, Mellhammar L, Rose N, et al. : Incidence and mortality of hospital- and ICU-treated sepsis: Results from an updated and expanded systematic review and meta-analysis. Intensive Care Med 2020; 46:1552–1562 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Gupta S, Sakhuja A, Kumar G, et al. : Culture-negative severe sepsis: Nationwide trends and outcomes. Chest 2016; 150:1251–1259 [DOI] [PubMed] [Google Scholar]
- 23.Phua J, Ngerng W, See K, et al. : Characteristics and outcomes of culture-negative versus culture-positive severe sepsis. Crit Care 2013; 17:R202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Vincent JL, Sakr Y, Sprung CL, et al. ; Sepsis Occurrence in Acutely Ill Patients Investigators: Sepsis in European intensive care units: Results of the SOAP study. Crit Care Med 2006; 34:344–353 [DOI] [PubMed] [Google Scholar]
- 25.Nannan Panday RS, Lammers EMJ, Alam N, et al. : An overview of positive cultures and clinical outcomes in septic patients: A sub-analysis of the prehospital antibiotics against sepsis (PHANTASi) trial. Crit Care 2019; 23:182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mellhammar L, Kahn F, Whitlow C, et al. : Bacteremic sepsis leads to higher mortality when adjusting for confounders with propensity score matching. Sci Rep 2021; 11:6972. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Klein Klouwenberg PM, Ong DS, Bos LD, et al. : Interobserver agreement of centers for disease control and prevention criteria for classifying infections in critically ill patients. Crit Care Med 2013; 41:2373–2378 [DOI] [PubMed] [Google Scholar]
- 28.Allami MK, Jamil W, Fourie B, et al. : Superficial incisional infection in arthroplasty of the lower limb. Interobserver reliability of the current diagnostic criteria. J Bone Joint Surg Br 2005; 87:1267–1271 [DOI] [PubMed] [Google Scholar]
- 29.Klompas M: Interobserver variability in ventilator-associated pneumonia surveillance. Am J Infect Control 2010; 38:237–239 [DOI] [PubMed] [Google Scholar]
- 30.Ter Avest E, van Munster BC, van Wijk RJ, et al. : Cohort profile of acutelines: A large data/biobank of acute and emergency medicine. BMJ Open 2021; 11:e047349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.ter Avest E, ter Maaten J, van Munster B, et al. : Acutelines: A Large Data-/Biobank of Acute and Emergency Medicine (Acutelines). 2020. ClinicalTrialsgov. Available at: https://clinicaltrials.gov/ct2/show/NCT04615065 Accessed November 4, 2020 [Google Scholar]
- 32.Li JS, Sexton DJ, Mick N, et al. : Proposed modifications to the Duke criteria for the diagnosis of infective endocarditis. Clin Infect Dis 2000; 30:633–638 [DOI] [PubMed] [Google Scholar]
- 33.Wilson J, Elgohari S, Livermore DM, et al. : Trends among pathogens reported as causing bacteraemia in England, 2004-2008. Clin Microbiol Infect 2011; 17:451–458 [DOI] [PubMed] [Google Scholar]
- 34.Vincent JL, Sakr Y, Singer M, et al. ; EPIC III Investigators: Prevalence and outcomes of infection among patients in intensive care units in 2017. JAMA 2020; 323:1478–1487 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Cai T, Mazzoli S, Mondaini N, et al. : The role of asymptomatic bacteriuria in young women with recurrent urinary tract infections: To treat or not to treat? Clin Infect Dis 2012; 55:771–777 [DOI] [PubMed] [Google Scholar]
- 36.Angus DC, Seymour CW, Coopersmith CM, et al. : A framework for the development and interpretation of different sepsis definitions and clinical criteria. Crit Care Med 2016; 44:e113–e121 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.