Pediatric Sepsis Definition—A Systematic Review Protocol by the Pediatric Sepsis Definition Taskforce

Supplemental Digital Content is available in the text.

Objectives:

Sepsis is responsible for a substantial proportion of global childhood morbidity and mortality. However, evidence demonstrates major inaccuracies in the use of the term “sepsis” in clinical practice, coding, and research. Current and previous definitions of sepsis have been developed using expert consensus but the specific criteria used to identify children with sepsis have not been rigorously evaluated. Therefore, as part of the Society of Critical Care Medicine’s Pediatric Sepsis Definition Taskforce, we will conduct a systematic review to synthesize evidence on individual factors, clinical criteria, or illness severity scores that may be used to identify children with infection who have or are at high risk of developing sepsis-associated organ dysfunction and separately those factors, criteria, and scores that may be used to identify children with sepsis who are at high risk of progressing to multiple organ dysfunction or death.

Data Sources:

We will identify eligible studies by searching the following databases: MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials.

Study Selection:

We will include all randomized trials and cohort studies published between January 1, 2004, and March 16, 2020.

Data Extraction:

Data extraction will include information related to study characteristics, population characteristics, clinical criteria, and outcomes.

Data Synthesis:

We will calculate sensitivity and specificity of each criterion for predicting sepsis and conduct a meta-analysis if the data allow. We will also provide pooled estimates of overall hospital mortality.

Conclusions:

The potential risk factors, clinical criteria, and illness severity scores from this review which identify patients with infection who are at high risk of developing sepsis-associated organ dysfunction and/or progressing to multiple organ dysfunction or death will be used to inform the next steps of the Pediatric Sepsis Definition Taskforce.

Infection accounts for almost one third of emergency department visits (1) and 40% of hospitalizations (2) and 25% of deaths in children globally (3). Although the majority of children have mild disease and recover quickly, almost 5 million children worldwide progress to life-threatening organ dysfunction and even death (4). The concept of sepsis has been traditionally used to characterize children with infection that manifest higher severity of disease with signs indicating presence of a systemic response to infection (5). The World Health Organization resolution on sepsis (6) stipulates quality improvement and public health interventions at local, national, and international level to reduce the burden of sepsis. Although recent meta-analyses indicate that sepsis is responsible for a substantial proportion of global childhood deaths from infection (7), evidence demonstrates substantial variability in the definition of sepsis and lack of precision when applying the term in clinical practice, coding, and research (8) which may lead to underestimation of the burden of sepsis.

The 1991 consensus conference on sepsis (5), and subsequent adult and pediatric sepsis definitions (9) have operationalized the disease definition using systemic inflammatory response syndrome (SIRS) (5) in the presence of presumed or proven infection as a requirement to meet criteria for sepsis, severe sepsis, or septic shock (9–11). SIRS was conceptualized as the body’s systemic response to an insult (e.g., infection) and included age-specific abnormalities in two or more of the following variables: 1) body temperature; 2) heart rate; 3) respiratory rate; or 4) WBC count. However, these criteria were nonspecific and do not reliably discriminate children with infections on a more severe trajectory or a higher risk of mortality (12). Ideally, sepsis criteria should meet both the need for sensitive early recognition as well as for specific separation of severe from mild infections. In the absence of a gold standard test to diagnose sepsis, the 2005 International Pediatric Sepsis Consensus Conference used expert opinion to provide age-specific cutoffs for SIRS criteria and operationalized definitions of organ failures (9). Since then, numerous studies have provided insights into the epidemiology, recognition, and treatment of sepsis in children (13–15). However, the diagnostic value of specific criteria to identify children with sepsis has never been rigorously evaluated.

In the 2016 update to the sepsis definition in adult patients (Sepsis-3), the Society of Critical Care Medicine (SCCM) and European Society of Intensive Care Medicine Sepsis Definitions Task Force conducted a systematic review of reported criteria used to identify adults with septic shock (11). This review only assessed hemodynamic criteria (hypotension, vasopressors, and hypoperfusion) and excluded pediatric studies, randomized controlled trials, and studies of specific pathogens and populations, thereby limiting its applicability to pediatrics. Furthermore, results of adult trials cannot be automatically generalized to the pediatric population because of significant differences in epidemiology (16), mortality rates (17), underlying diseases (18), disease-specific outcomes (13, 19), and differing results of septic shock trials in adults and children (20, 21).

Therefore, SCCM convened the Pediatric Sepsis Definition Taskforce in June 2019 to assess, develop, and validate criteria for the identification of sepsis in children. The Taskforce recognizes the need to assess both criteria for recognition of children with possible sepsis and for identification of sepsis leading to poor outcomes such as multiple organ failure or death. As part of this process, the Taskforce will conduct a systematic review, a survey, a Delphi process and a data-driven derivation and validation exercise. The goals of this systematic review are to synthesize evidence on individual factors, clinical criteria, or illness severity scores that may be used to identify children with infection who have or are at high risk of developing sepsis-associated organ dysfunction and separately to synthesize evidence on those factors, criteria, and scores that may be used to identify children with sepsis who are at high risk of progressing to multiple organ dysfunction or death.

OBJECTIVES

To determine whether 1) in children greater than or equal to 37 weeks postgestational age to less than 18 years with suspected or confirmed infection are individual factors, physiologic variables, laboratory tests, and illness severity scores associated with the development of sepsis, severe sepsis, or septic shock and 2) in children greater than or equal to 37 weeks postgestational age to less than 18 years with sepsis, severe sepsis, or septic shock are individual factors, physiologic variables, laboratory tests, and illness severity scores associated with development of new or progressive multiple organ dysfunction (NPMODS) and/or mortality.

MATERIALS AND METHODS

This protocol follows the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines (PROSPERO), is registered in the international prospective register of systematic reviews (November 12, 2019; PROSPERO—42019147932) and is funded, organized, and reviewed by SCCM. The multinational and multidisciplinary scientific panel responsible for development of the systematic review protocol includes subject matter experts from across the globe that were selected for the Pediatric Sepsis Definition Taskforce.

Eligibility Criteria

All cohort studies (prospective or retrospective), randomized and quasi-randomized trials published between January 1, 2004, and March 16, 2020, will be included. The search will be updated as needed such that the final submitted article will contain all articles published up to the previous 6 months. The start date of January 2004 was chosen to allow identification of studies published following the operational definition from the Pediatric Consensus Conference Sepsis Definition which was initiated in 2004 and published in 2005 (9). Literature before 2004 often lacked reference to specific sepsis criteria and as such, we anticipated it would be hard to reliably extract associations between sepsis criteria and outcomes prior to 2004. Studies with fewer than 50 children with sepsis, septicemia, severe sepsis, or septic shock will be excluded as they are unlikely to be adequately powered to describe criteria for recognition of possible sepsis or criteria for sepsis leading to multiple organ failure or death and are likely to have a high risk of bias due to their small sample size (22). Abstract only publications, case studies, narrative reviews, surveys, and study protocols will also be excluded.

Population.

The study population is defined as children greater than or equal to 37 weeks postgestational age to less than 18 years with suspected or confirmed infection presenting to any healthcare facility. Areas within the hospital where these patients are identified could include PICU/ICU/neonatal ICU, high dependency units, emergency departments, pediatric wards (general or specialized), or outpatient settings (e.g., oncology clinics). We will not include studies primarily focusing on adult patients with co-recruitment of adolescents or studies of premature infants. In addition, we will not include studies of full-term newborns that have never left hospital as the current criteria for organ dysfunction have not been validated in neonates during the first days of life (23). Specifically, Pao₂, creatinine, and bilirubin thresholds used in organ dysfunction scores do not apply in the first days of life due to physiologic adaptation; in addition, serum lactate levels following delivery may reflect placental circulation rather than ongoing problems in the newborn (24). All study populations (cardiac surgery, oncology, etc.) and pathogens (viruses, fungi, bacteria, parasites, etc.) will be included.

Exposure.

The exposures we will consider in this systematic review will be 1) suspected or confirmed infection with any pathogen (in children who have or are at high risk of developing sepsis-associated organ dysfunction) and 2) sepsis, septicemia, severe sepsis, or septic shock (in children at high risk of progressing to multiple organ dysfunction or death). Suspected or confirmed infections require either a diagnostic code of infection or the combination of a healthcare provider decision to treat with any length of antimicrobials and an attempt to obtain samples of body fluid cultures (e.g., blood, urine, cerebrospinal fluid, or other cultures). The article may define sepsis, severe sepsis, septicemia, or septic shock by any criteria provided the specific criteria used are explicitly stated.

Comparison.

The comparators will be patient criteria (individual factors, signs and symptoms, physiologic variables, laboratory tests) or illness severity scores (such as Pediatric Index of Mortality [25], Pediatric Logistic Organ Dysfunction [26], Pediatric Risk of Mortality [27], SIRS [9], pediatric Sequential [Sepsis-related] Organ Failure Assessment [28], and quick Sequential [Sepsis-related] Organ Failure Assessment [29]) assessed from 24 hours before diagnosis of sepsis/septicemia/septic shock until discharge from an acute-care setting. All versions of the listed illness severity scores will be included. Potential criteria to be included in our systematic review are detailed in Table 1; additional criteria will be added as dictated by the literature search. Studies focusing primarily on criteria that are only available on a research basis (e.g., gene-expression data) will not be eligible.

TABLE 1.

Potential Criteria to Identify in Eligible Studies

Outcomes.

The primary outcomes for 1) articles describing criteria to identify children with infection who have or are at high risk of developing sepsis-associated organ dysfunction are the diagnostic categories of sepsis, septicemia, severe sepsis, or sepsis shock. The primary outcomes for 2) articles describing criteria to identify children with sepsis who are at high risk of progressing to multiple organ dysfunction or death is development of NPMODS or healthcare facility mortality. Secondary outcomes for both types of articles include PICU admission, PICU mortality, 28-day mortality, PICU and hospital length of stay, and development of NPMODS (30). Although we acknowledge the importance of longer-term and quality of life-related outcomes, we will not be considering them for this review. We will assess the association of individual criteria and scores listed in Table 1 with each of the described outcomes in children with suspected or confirmed infection or sepsis, septicemia, severe sepsis, or septic shock. Studies must report on at least one of the primary or secondary outcomes in order to be included.

Data Sources

We will identify eligible studies by searching the following databases: MEDLINE (including Epub Ahead of Print), Embase, and the Cochrane Central Register of Controlled Trials. We will hand search references of the primary studies and systematic reviews for relevant studies. Data from abstracts or conference proceedings will not be searched.

Search Strategy

We will develop a search strategy with the help of a health information specialist (Appendix 1, Supplemental Digital Content 1, http://links.lww.com/CCX/A179). In addition to age, Medline MeSH search terms will include “sepsis,” “septicemia,” and “septic shock.” We will consider adding the following limiting variables “epidemiology,” “criteria,” “characteristics,” “variables,” “diagnosis,” “definition,” “decision rule,” “predictors,” and “recognition” if the original search yields greater than 10,000 articles. We will also use sentinel articles identified by members of the Systematic Review Working Group to validate the inclusiveness of the search strategy. There will be no language restrictions for the search.

Study Selection and Screening Process

The titles from the combined search will be downloaded and screened using a previously validated crowdsourcing platform Insight Scope (31). Citations will be screened for eligibility based on title and abstract, and full text using Insight Scope (www.insightScope.ca). Insight Scope is a web-based systematic review platform designed to facilitate citation screening, full-text upload, and conflict resolution. At each screening level, citations will be screened by two independent reviewers, and conflicts will be resolved by a member of the Systematic Review Working Group using the conflict resolution function. Insight Scope members who are interested in contributing to citation screening for this review will complete a test set of 100 citations and must achieve a sensitivity of 80% before they are given access to the project. At least 5–10 true positives will be purposely included in the test set. The test set will be created by the principal investigator (K.M.) and verified by one other Systematic Review Working Group member (L.S.) before the project is posted to recruit reviewers. Each title will be screened by two reviewers using the predetermined eligibility criteria. Titles eliminated by both reviewers will be rejected and the remaining titles moved to full-text screening. Each remaining full-text article will be screened by two members of the Systematic Review Working Group for inclusion in the final set of articles for data extraction. Conflicts will be resolved by discussion or a third member of the Systematic Review Working Group if consensus is not reached during the discussion. Kappa scores on the level of agreement reached between reviewers at each stage of screening will be reported.

Data Extraction and Management

Data from all included full-text articles will be extracted by two Systematic Review Working Group reviewers per citation using a Research Electronic Data Platform (REDCap) platform (32) hosted at the Children’s Hospital of Eastern Ontario Clinical Research Unit. Conflicts will be resolved by discussion or by a member of the Systematic Review Working Group if consensus cannot be reached. Data extraction will include information related to study characteristics (i.e., title, authors, year of publication, country, language, journal, study design, sample size, and inclusion/exclusion criteria); population characteristics (age, sex, admission diagnosis, level of care available, location of the admission within the hospital/clinic, disease classification of population studied, and patient comorbidities); exposure (pathogen, source, community-acquired vs hospital-acquired infection, or sepsis as per Centers for Disease Control and Prevention criteria [33], specific definition of sepsis used); clinical criteria (Table 1); and outcomes (PICU admission, 28-d mortality, PICU mortality, PICU and/or hospital length of stay, and NPMODS). Studies may report varying thresholds of “normal” for laboratory data such as serum glucose and bicarbonate; these thresholds will be documented in our data abstraction form. The authors will be contacted to obtain missing data on study methods or patient level data needed for computation of sensitivity and specificity of dichotomous outcomes. We will not contact authors for patient level data for calculation illness severity scores if these were not reported in the original article.

Risk of Bias in Individual Studies

We will assess the quality of selected articles using the Quality in Prognosis Studies tool for assessment of risk of bias in observational studies (34). We will perform a subgroup analysis to determine if there is a credible subgroup effect in which case, we would only include the low risk of bias studies in the final meta-analysis.

Data Synthesis and Analysis

We will summarize data on study characteristics, outcomes, and other related variables, criteria/scores reported as well as quality assessment using tables, graphs, and narrative summaries. Continuous outcomes will be summarized using mean and sds or medians with interquartile ranges as appropriate. Binary outcomes will be summarized using frequencies and percentages. We will construct a two-by-two table for each criterion identified in the included studies and will add 0.5 to all cells if any of the cells are zero. For each criterion, we will calculate its sensitivity and specificity for predicting sepsis, multiple organ failure, or death and report the corresponding 95% CIs. We will also calculate the likelihood ratios for the presence (positive likelihood ratio) or absence (negative likelihood ratio) of each criteria/score and pretest and posttest probabilities of the outcome (sepsis, multiple organ failure, or death). For interventional studies, only the outcomes in the usual care arm of the study will be included in these estimates.

When a criterion is assessed in four or more studies, we will perform a meta-analysis of diagnostic accuracy studies to calculate pooled estimates of sensitivity and specificity as well as provide the pooled area under the receiver operating characteristic curve. We will use a hierarchical random-effects bivariate logistic regression model, which accounts for heterogeneity as well as the correlation between sensitivity and specificity. We will examine sources of heterogeneity, including differences in methodology, setting, patient populations as well as differences in clinical presentations (e.g., different diagnosis or disease prognosis). Statistical heterogeneity will be assessed and reported using a chi-square test and the I² statistic. It is likely that there will be significant heterogeneity between studies and we will therefore pool results when studies are considered comparable (I² value < 40%) using a random-effects meta-analysis. We will perform subgroup analyses with respect to age, diagnostic category (e.g., oncology, severe acute malnutrition), community- versus hospital-acquired infections, healthcare setting (low vs high resource), and other relevant factors with the aim of providing more precise pooled estimates. We will present the results using sensitivity-specificity coupled forest plots as well as pooled receiver operating characteristic curves.

We will provide pooled estimates of overall hospital mortality and multiple system organ failure using a random-effects meta-analysis. Relative risk will be used as an effect measure in the meta-analysis. If data are available, we will also conduct meta-analysis to provide pooled estimates on sepsis-related mortality, PICU admission, PICU mortality, 28-day mortality, and PICU and hospital length of stay. We will report 95% CIs and provide graphical visualization of the results with forest plots. We will also consider subgroup analyses if we have adequate numbers of studies and/or patients within the included studies. All analyses will be performed the R statistical software (R Foundation for Statistical Computing, Vienna, Austria).

It is possible that different thresholds might have been used in calculating sensitivity and specificity, leading to high level of heterogeneity across the studies. As such, we will categorize studies with respect to similarity in thresholds used and only pool the estimates from similar studies. We will choose a threshold used in the majority of the studies and contact authors to receive individual-level data for the rest of the studies to calculate sensitivity and specificity. For studies with missing thresholds, we will impute missing data based on information provided in similar studies. Similarly, some studies report summary measures of the continuous outcome (e.g., mean and sd of lactate levels) rather than provide sensitivity and specificity by dichotomizing the outcomes. For such studies, we will request patient level data, and if provided, will calculate sensitivity and specificity based on a threshold used in the majority of the studies.

DISCUSSION

The goals of this systematic review are to synthesize evidence on individual factors, clinical criteria, and/or illness severity scores for the recognition of possible sepsis, sepsis, septicemia, and septic shock in children with suspected or documented infections and to synthesize evidence on factors, criteria, and scores that may be used to identify children with sepsis who are at high risk of progressing to multiple organ dysfunction or death. As this is a broad topic, we anticipate that there will be a large number of eligible citations. However, we will minimize the risk of missing key citations by hand searching references of primary studies and systematic reviews for relevant studies as well as soliciting sentinel articles from the Systematic Review Working Group.

It is important to note that the objective of this systematic review is to determine the evidence behind possible criteria previously used in the literature to identify patients with sepsis, septicemia, and septic shock and not to validate a new definition of sepsis. The determination of which variables have been correlated with the clinically important outcomes outlined in this systematic review is an important step toward the development and validation of new criteria for pediatric sepsis by the Pediatric Sepsis Definition Taskforce of SCCM.

ACKNOWLEDGMENTS

We would like to thank Kathy Vermoch and Lori Harmon from the Society of Critical Care Medicine and Katie O'Hearn at the Children's Hospital of Eastern Ontario for all their help with this project and article. We would also like to thank Margaret Sampson, librarian at the Children’s Hospital of Eastern Ontario for her invaluable help in creating and executing the search for this systematic review. We would also like to thank Ryan Sondarage and Supun Kotteduwa Jayawarden for their remarkable efforts in title/abstract and full-text screening. On behalf of the Pediatric Sepsis Definition Taskforce: Luregn J. Schlapbach (Co-Chair), Paediatric Critical Care Research Group, Child Health Research Centre, The University of Queensland, and Paediatric ICU, Queensland Children’s Hospital, Brisbane, QLD, Australia; R. Scott Watson (Co-Chair), Center for Child Health, Behavior and Development, Seattle Children’s Research Institute, Seattle Children’s Hospital, Seattle, WA; Andrew Argent (Vice-Chair), Department of Paediatrics and Child Adolescent Health, Red Cross War Memorial Children’s Hospital and University of Cape Town, Cape Town, South Africa; Lauren R. Sorce (Vice-Chair), Ann & Robert H. Lurie Children’s Hospital, and Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL; Elizabeth R. Alpern, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL; Fran Balamuth, Children’s Hospital of Philadelphia, Philadelphia, PA; Tellen D. Bennett, University of Colorado, Denver, CO; Paolo Biban, Major City Hospital, Verona, Italy; Joe Carcillo, Department of Critial Care Medicine, UPMC Children’s Hospital, Pittsburgh, PA; Enitan Carrol, University of Liverpool, Liverpool, United Kingdom; Kathleen Chiotos, Children’s Hospital of Philadelphia, Philadelphia, PA; Mohammod Jobayer Chisti, International Centre for Diarrhoeal Disease Research, Dhaka, Bangladesh; Idris Evans, Children’s Hospital of Pittsburgh of UPMC, Pittsburgh, PA; Lu Guoping, Children’s Hospital of Fudan University, Shanghai, China; Mark W. Hall, Nationwide Children’s Hospital, Columbus, OH; David Inwald, Addenbrooke’s Hospital, Cambridge University Hospital NHS Trust, Cambridge, United Kingdom; Paul Ishimine, University of California San Diego, San Diego, CA; Michael Levin, Imperial College London, London, United Kingdom; Niranjan Tex Kissoon, British Columbia Women and Children’s Hospital, Vancouver, BC, Canada; Rakesh Lodha, All India Institute of Medical Sciences, New Delhi, India; Kathryn Maitland, Imperial College, London, United Kingdom; Simon Nadel, St. Mary’s Hospital, London, United Kingdom; Satoshi Nakagawa, National Center for Child Health & Development, Tokyo, Japan; Claudio Flauzino Oliveira, Associação de Medicina Intensiva Brasileira, São Paulo, Brazil; Mark Peters, University College London Great Ormond Street Institute of Child Health, London, United Kingdom; Adrienne G. Randolph, Boston Children’s Hospital, Boston, MA; Suchitra Ranjit, Apollo Hospitals, Chennai, India; L. Nelson Sanchez-Pinto, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL; Halden F. Scott, Children’s Hospital of Colorado, Denver, CO; Daniela Carla Souza, University Hospital of The University of São Paulo, Sao Paulo, Brazil; Paul Tissieres, Hospital de Bicetre, Paris, France; Juliane Bubeck Wardenburg, Department of Pediatrics, Washington School of Medicine, St. Louis, MN; Scott L. Weiss, Children’s Hospital of Philadelphia, Philadelphia, PA; Wilson Milton Were, Department of Maternal, Newborn, Child and Adolescent Health, World Health Organization, Geneva, Switzerland; Matt Wiens, University of British Columbia, Vancouver, BC, Canada; James L. Wynn, University of Florida, Gainesville, FL; and Jerry J. Zimmerman, Seattle Children’s Hospital, Seattle, WA.