Key Points
Question
Is there an association between management guided by the neonatal early-onset sepsis calculator and reduction in empirical antibiotic therapy for newborns with suspected early-onset sepsis?
Findings
This systematic review and meta-analysis found that management guided by an early-onset sepsis calculator was associated with a significant reduction in empirical antibiotic therapy compared with conventional management, with a relative risk of 56% in before-after implementation studies. Safety data were limited, but no evidence was found of inferiority compared with conventional management strategies.
Meaning
Management guided by the neonatal early-onset sepsis calculator is associated with a substantial reduction in empirical antibiotic therapy, but more studies are needed to inform on safety.
Abstract
Importance
The neonatal early-onset sepsis (EOS) calculator is a clinical risk stratification tool increasingly used to guide the use of empirical antibiotics for newborns. Evidence on the effectiveness and safety of the EOS calculator is essential to inform clinicians considering implementation.
Objective
To assess the association between management of neonatal EOS guided by the neonatal EOS calculator (compared with conventional management strategies) and reduction in antibiotic therapy for newborns.
Data Sources
Electronic searches in MEDLINE, Embase, Web of Science, and Google Scholar were conducted from 2011 (introduction of the EOS calculator model) through January 31, 2019.
Study Selection
All studies with original data that compared management guided by the EOS calculator with conventional management strategies for allocating antibiotic therapy to newborns suspected to have EOS were included.
Data Extraction and Synthesis
Following PRISMA-P guidelines, relevant data were extracted from full-text articles and supplements. CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and GRADE (Grades of Recommendation, Assessment, Development and Evaluation) tools were used to assess the risk of bias and quality of evidence. Meta-analysis using a random-effects model was conducted for studies with separate cohorts for EOS calculator and conventional management strategies.
Main Outcomes and Measures
The difference in percentage of newborns treated with empirical antibiotics for suspected or proven EOS between management guided by the EOS calculator and conventional management strategies. Safety-related outcomes involved missed cases of EOS, readmissions, treatment delay, morbidity, and mortality.
Results
Thirteen relevant studies analyzing a total of 175 752 newborns were included. All studies found a substantially lower relative risk (range, 3%-60%) for empirical antibiotic therapy, favoring the EOS calculator. Meta-analysis revealed a relative risk of antibiotic use of 56% (95% CI, 53%-59%) in before-after studies including newborns regardless of exposure to chorioamnionitis. Evidence on safety was limited, but proportions of missed cases of EOS were comparable between management guided by the EOS calculator (5 of 18 [28%]) and conventional management strategies (8 of 28 [29%]) (pooled odds ratio, 0.96; 95% CI, 0.26-3.52; P = .95).
Conclusions and Relevance
Use of the neonatal EOS calculator is associated with a substantial reduction in the use of empirical antibiotics for suspected EOS. Available evidence regarding safety of the use of the EOS calculator is limited, but shows no indication of inferiority compared with conventional management strategies.
This systematic review and meta-analysis assesses the association between management of early-onset sepsis guided by the neonatal early-onset sepsis calculator (compared with conventional management strategies) and reduction in antibiotic therapy for newborns.
Introduction
Empirical therapy of newborns at risk for or with suspected early-onset sepsis (EOS) represents the main contributor to the use of antibiotics in early life.1 The reported number of newborns receiving antibiotic therapy for 1 episode of culture-proven EOS ranges from 18 to 118 in high-risk infants, and up to 1400 in well-appearing newborns born to mothers with chorioamnionitis.2,3,4 Thus, for each case of culture-proven EOS many newborns are exposed to potential harms related to empirical antibiotic therapy. Use of antibiotics in newborns is associated with early adverse consequences such as increased risk of necrotizing enterocolitis, fungal infections, and death in preterm infants.5,6 Moreover, antibiotics increase antibiotic resistance, mother-child separation, and health care costs.7,8 Antibiotic-induced microbiome alterations early in life, with downstream effects on the developing immune system,9,10 are also associated with increased risks of allergic diseases, obesity, and autoimmune diseases later in life.6,11,12
The neonatal EOS calculator is designed to improve the accuracy of empirical antibiotic administration in newborns with suspected EOS. It is based on a predictive risk model developed using a nested case-control design in a cohort of 608 014 newborns 34 weeks’ gestational age or older born at 14 hospitals in the United States, and further advanced using logistic regression and recursive partitioning.13,14 The EOS calculator (http://kp.org/eoscalc) estimates the EOS risk based on 5 objective maternal risk factors and 4 clinical neonatal risk factors. It stratifies newborns into 3 levels of risk with a corresponding recommendation for management, including to start or withhold empirical antibiotic therapy. Implementation of the EOS calculator at Kaiser Permanente Northern California hospitals almost halved the rates of antibiotic administration (from 5.0% to 2.6%) among term and late preterm infants in the first 24 hours after birth.15
The EOS calculator prediction model is based on a selected US population, and differences between health care settings may impede generalizability. For example, EOS incidence rates, maternal group B Streptococcus (GBS) screening policy, intrapartum antibiotic administration, and/or observation time in the hospital may differ between the United States and other countries. In view of the need to reduce unnecessary antibiotic use early in life, and the increasing use of the EOS calculator in many settings,3 there is urgency to summarize the best available evidence on the EOS calculator to guide policy making and further research.16,17,18
The purpose of the current systematic review and meta-analysis was to identify, critically appraise, and synthesize evidence from studies comparing management guided by the EOS calculator with conventional management strategies, and report the rates of empirical antibiotic therapy for suspected EOS. The second objective was to summarize the available safety data regarding the use of the EOS calculator.
Methods
We used a PRISMA-P (Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocols) review protocol for data collection, analysis, and reporting (the eAppendix in the Supplement contains full methodological details). We registered the review in advance (CRD42018116188, PROSPERO database).19,20
Study Eligibility Criteria
We prespecified eligibility criteria as follows: any study design with original data, comparing management guided by the EOS calculator with conventional management strategies, and reporting the rates of empirical antibiotic therapy for suspected EOS as an outcome. No eligibility criteria regarding safety data were set, and all eligible studies were screened for all safety outcomes. To ensure independence of outcome estimates, we excluded data sets that were used to develop the EOS calculator.
Information Sources and Search Strategy
We performed a systematic search of all available literature describing the EOS calculator in the Cochrane, Embase, and PubMed/MEDLINE databases, from 2011 (introduction of the EOS calculator model) through January 31, 2019. We searched in all search fields for eos calculator, eos risk calculator, sepsis calculator, or sepsis risk calculator. In the title and abstract fields, we searched for predictive, risk, quantitative, or stratification, combined with model or algorithm, and early onset sepsis, early onset neonatal sepsis, or EOS. Exact search engine strings are detailed in the review protocol (eAppendix in the Supplement). We limited our search results to peer-reviewed articles published in 2011 or later, because the multivariate model forming the basis of the EOS calculator was published in 2011.13 No other limits were applied. We examined reference lists of included studies and relevant reviews to identify additional eligible studies. We also reviewed all titles and abstracts of all articles citing original EOS calculator publications identified through Google Scholar and/or Scopus/Web of Science search engines. All citations were combined and duplicates were manually excluded.
Study Selection and Data Extraction
Search results were independently screened by 2 of us (N.B.A. and R.B.), who assessed each potentially eligible full-text article according to predetermined inclusion and exclusion criteria. In case of disagreement, a third researcher (F.B.P.) had the decisive vote. One of us (N.B.A.) extracted relevant data from articles as well as any available supplements. Two of us (R.B. and W.E.B.) verified data extraction for completeness and accuracy. The following general data were extracted: author, year, country, study design, populations, and inclusion criteria. We extracted data on the rates of newborns treated with empirical antibiotics for suspected or proven EOS within 72 hours or less after birth, both for management based on the EOS calculator and conventional management strategies. For these scenarios, we calculated the absolute and relative differences with 95% CIs. We extracted data on the following safety outcomes: missed cases of EOS (defined as newborns with culture-proven EOS not allocated antibiotic therapy within 24 hours after birth), changes in incidence of EOS, EOS morbidity and mortality, readmissions for neonatal sepsis, and timing of antibiotics, after EOS calculator implementation. We also noted any adverse events specifically reported by the authors. If multiple articles reported data from the same source study, results were combined to avoid overlap among results. For studies eligible for meta-analysis, we retrieved supplementary data from original authors if exact data on antibiotic use within 72 hours after birth were not present in the original publication. In addition, we surveyed original authors for updates on their data, and retrieved these when available.
Assessment of Methodological Quality
We assessed the risk of bias of individual studies using 8 applicable items of a dedicated checklist for assessment of studies evaluating prediction models (CHARMS [Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies]).21 Risk of bias for each item, including an overall risk-of-bias score, was classified as high, low, or unclear; disagreements were resolved via consulting with one of us (F.B.P.).
We used the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) tool to estimate the quality of evidence, from very low to high.22,23 This estimation was performed separately for the use of empirical antibiotics for EOS and for safety of use of the EOS calculator.
Synthesis of Results and Analysis
We classified studies according to their study design; studies evaluating cohorts before and after actual implementation of the EOS calculator, and studies performing hypothetical analysis of newborn databases. We pooled data from actual implementation studies with comparable homogeneous data before and after implementation, and calculated combined effect estimates. Subgroup analysis was performed for studies including newborns regardless of exposure to chorioamnionitis and for studies restricted to newborns exposed to chorioamnionitis. We quantified inconsistencies between the results of the studies by using the I2 test. Results were interpreted as representing either absence of heterogeneity (I2 < 25%), low heterogeneity (I2 = 25%-50%), moderate heterogeneity (I2 = 50%-75%), or high heterogeneity (I2 = ≥ 75%).24 Data entry and meta-analysis were performed using RevMan, version 5.3 (The Nordic Cochrane Centre). We calculated relative risk (RR) with 95% CIs. We present the effect estimates by using the random-effects model owing to assumption of clinical and methodological diversity among the studies, subsequently often leading to statistical heterogeneity. To compare proportions of missed cases of EOS, we used the Cochran-Mantel-Haenszel method to test for significance (α level P < .05), performed using R, version 3.5.0 (R Foundation for Statistical Computing).25
Results
Characteristics and Participants of Included Studies
After reviewing 354 identified publications for study eligibility, we selected and evaluated 56 full-text articles (Figure 1). Thirteen studies were included (Table 1).15,26,27,28,29,30,31,32,33,34,35,36,37 For 1 study,38 we used recently added data obtained through surveying authors for updated data.36 No randomized clinical studies were found. Six studies evaluated implementation of the EOS calculator in clinical practice using before-after analysis and were therefore eligible for meta-analysis.15,26,27,28,29,30 Seven studies estimated outcomes of using the EOS calculator by hypothetical analysis of newborn databases.31,32,33,34,35,36,37 Seven studies used a retrospective approach,30,31,32,33,34,35,36 3 used a prospective approach,15,28,37 and 3 studies used a combined approach.26,27,29 Ten of 13 studies were performed in the United States.15,27,29,30,31,33,34,35,36,37
Table 1. Characteristics and Use of Empirical Antibiotic Therapy in Included Studiesa.
Source | Location | Setting | Design | Births, No. | Included | EOS Calculator | Conventional Strategy | Reduction in Empirical AB | ||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
No. | Empirical AB, No. (%) | Strategy | No. | Empirical AB, No. (%) | Absolute % | Relative Risk, % (95% CI) | ||||||
Before-After Analysis | ||||||||||||
Kuzniewicz et al,15 2017 | United States | Mixed | Prospective | 204 485 | GA ≥35 wk | 56 261 | 1698 (3.0) | CDC informed | 95 543 | 5226 (5.5) | 2.5 | 55.2 (52-58) |
Achten et al,26 2018 | The Netherlands | Regional | Retrospective and prospective | 3953 | GA ≥35 wk | 1877 | 51 (2.7) | National guideline informed | 2076 | 100 (4.8) | 2.1 | 56.4 (40-79) |
Dhudasia et al,27 2018 | United States | Tertiary | Retrospective and prospective | 11 782 | GA ≥36 wk | 6090 | 222 (3.6) | CDC and AAP informed | 5692 | 356 (6.3) | 2.6 | 58.3 (49-69) |
Strunk et al,28 2018 | Australia | Tertiary | Prospective | 4233 | GA ≥35 wk | 2502 | 206 (8.2) | Adaptation AAP guideline | 1732 | 237 (13.7) | 5.5 | 60.2 (50-72) |
Hypothetical Database Analysis | ||||||||||||
Gievers et al,29 2018 | United States | Tertiary | Retrospective and prospective | 9039 | Chorioamnionitis, GA ≥35 wk | 143 | 13 (9.1) | CDC informed | 213 | 203 (95.3) | 86.2 | 9.5 (6-16) |
Beavers et al,30 2018 | United States | Tertiary | Retrospective | NR | Chorioamnionitis, GA ≥35 wk | 76 | 28 (36.8) | Preimplementation | 180 | 168 (93.3) | 57.0 | 39.3 (29-53) |
Shakib et al,31 2015 | United States | Tertiary | Retrospective | 20 262 | Chorioamnionitis, well-appearing, GA ≥34 wk | 698 | 39-86 (5.6-12.3)b | Actual practice (CDC and CFN informed) | NA | 430 (61.6) | 49.3-56.0b | 9.1-20.0b |
Kerste et al,32 2016 | The Netherlands | Regional | Retrospective | 2094 | AB for suspected EOS, GA ≥34 wk | 108 | 51 (47.2) | Actual practice (national guideline informed) | NA | 108 (100) | 52.8c | 47.2 (39-58)c |
Warren et al,33 2017 | United States | Tertiary | Retrospective | NR | AB for suspected EOS, GA ≥34 wk | 202 | 47 (23.3) | CDC guideline | NA | 188 (93.1) | 69.8c | 25.0 (19-32)c |
Money et al,34 2017 | United States | Tertiary | Retrospective | 19 525 | Chorioamnionitis, well-appearing for 24 h, GA ≥35 wkc | 362 | 9 (2.5) | Current protocol (CDC and AAP informed) | NA | 361 (99.7)d | 97.2d | 2.5 (1-5)d |
Carola et al,35 2018 | United States | Tertiary | Retrospective | 17 908 | Chorioamnionitis, GA ≥35 wk | 896 | 209 (23.3) | Actual practice (AB if chorioamnionitis) | NA | 896 (100) | 76.7 | 23.3 (21-27) |
Joshi et al,36 2019 | United States | Tertiary | Retrospective | 10 002 | Chorioamnionitis, well-appearing at birth, GA ≥34 wk | 596 | 53 (8.9) | Institutional practice (AB if chorioamnionitis) | NA | 596 (100) | 91.1 | 8.9 (3-11) |
Klingaman et al,37 2018 | United States | Tertiary | Prospective | 505 | GA ≥35 wk | 505 | 2 (0.4) | CDC informed | NA | 9 (17.8) | 1.4 | 22.2 (5-102) |
Abbreviations: AAP, American Academy of Pediatrics; AB, antibiotics; CDC, Centers for Disease Control and Prevention; CFN, Committee on the Fetus and Newborn; EOS, early-onset sepsis; GA, gestational age; NA, not applicable; NR, not reported.
Births = number of births in total study period in the eligible GA range; included = inclusion criteria used to select study population; chorioamnionitis = newborns with a mother who received a diagnosis of chorioamnionitis; reduction in AB = (hypothetical) reduction in empirical AB for EOS achieved by using the EOS calculator.
Reduction range reported (precluding calculation of meaningful CI), as depending on outcome of newborns in observe-and-evaluate category.
Studies limited to AB-treated infants; reported results represent estimations of maximum potential reduction of empirical AB by EOS calculator use.
Sampling of study excluded 41 infants who were symptomatic at birth and 38 infants developing symptoms after initial examination, resulting in an estimated reduction that does not reflect a potential implementation scenario.
The 13 included studies involved a total of 175 752 newborns. Of these, 172 385 were included in studies comparing cohorts before (66 949) and after (105 436) EOS calculator implementation, and 3367 were included in studies performing hypothetical database analysis. Inclusion criteria differed among studies. The minimal gestational age ranged from 34 to 36 weeks. Three studies were confined to well-appearing newborns, while the other 10 studies also included symptomatic newborns. Inclusion was limited to newborns with a diagnosis of maternal chorioamnionitis in 6 studies, and limited to newborns treated with antibiotics in 2 studies.
Risk of Bias and Quality of Evidence
The overall risk of bias was judged as high for 9 studies, low for 2 studies, and unclear for 2 studies (eTable 1 in the Supplement). We graded the overall quality of evidence for the primary outcome of reduction in use of empirical antibiotics as moderate, owing to the inclusion of very large observational studies that had large effect sizes and the consistency of results. We graded the quality of evidence regarding safety of use of the EOS calculator as very low, mainly owing to the small number of events across all studies.
Reduction in Use of Empirical Antibiotics When Using the EOS Calculator
All 13 included studies compared management guided by the EOS calculator with conventional management strategies and used the rate of empirical antibiotics prescribed for suspected EOS as a main outcome. All studies found a lower RR for antibiotic therapy, favoring use of the EOS calculator (range, 3%-60%) (Table 1). Studies evaluating the EOS calculator in newborns born to mothers with the risk factor of chorioamnionitis reported stronger reductions (RR, 3%-39%) compared with studies not limited to chorioamnionitis (RR, 25%-60%).
Meta-analysis results of data from before and after implementation of the EOS calculator favored use of the EOS calculator, with an overall RR of antibiotic use of 45% (95% CI, 35%-57%) among all 6 studies (Figure 2). We found an RR in antibiotic use of 56% (95% CI, 53%-59%) in the 4 studies including all newborns regardless of exposure to chorioamnionitis. We found no heterogeneity among results of these studies, of which 2 were from the United States,15,27 1 from Australia,28 and 1 from the Netherlands.26 For the 2 studies restricted to newborns exposed to chorioamnionitis,29,30 the RR in antibiotic use was lower (20%), but with a large 95% CI (4%-91%) and high heterogeneity (I2 = 96%) owing to large differences between the effect estimates.
Safety When Using the EOS Calculator
Three studies were specifically designed to evaluate the safety of the EOS calculator as a study objective or by calculating model performance, using before-after analysis.15,27,28 One or more safety outcomes were discussed in 12 of 13 included studies (eTable 2 in the Supplement). Across all studies, we found no indication of an increase in the incidence of EOS, readmissions, antibiotic use between 24 and 72 hours after birth, proportion of newborns requiring intensive care, or mortality associated with the use of the EOS calculator.
We reviewed all cases of EOS reported in the 13 included studies. Among before-after implementation studies, we found 5 of 18 missed cases of EOS (28%) in cohorts with EOS calculator–based management, compared with 8 of 28 missed cases of EOS (29%) in cohorts with conventional management strategies (pooled odds ratio, 0.96; 95% CI, 0.26-3.52; P = .95) (Table 2). Newborns with missed EOS were eventually started on antibiotics in all cases. Among studies performing only database analysis, we found 5 of 12 missed cases of EOS (42%) by hypothetical application of the EOS calculator (Table 3). Among all studies, almost half of newborns with missed EOS remained asymptomatic, regardless of management strategy (eTable 3 in the Supplement).
Table 2. Management of EOS Cases Using the EOS Calculator and Conventional Management Strategies in Before-After Studies.
Source | Management Guided by EOS Calculator | Conventional Management Strategy | P Value | ||||||
---|---|---|---|---|---|---|---|---|---|
Births | EOS Cases | AB <24 h | AB >24 h (Missed) | Births | EOS Cases | AB <24 h | AB >24 h (Missed) | ||
Kuzniewicz et al,15 2017 | 56 261 | 12 | 8 | 4 | 95 543 | 24 | 18 | 6 | NA |
Achten et al,26 2018 | 1877 | 2 | 2 | 0 | 2076 | 2 | 0 | 2 | NA |
Dhudasia et al,27 2018 | 6090 | 3 | 2 | 1 | 5692 | 1 | 1 | 0 | NA |
Strunk et al,28 2018 | 2502 | 1 | 1 | 0 | 1731 | 1 | 1 | 0 | NA |
Total, No. (%) | 67 019 | 18 | 13 (72) | 5 (28) | 105 365 | 28 | 20 (71) | 8 (29) | .95 |
Abbreviations: AB, antibiotics; EOS, early-onset sepsis; NA, not applicable.
Table 3. Cases of EOS in Database Studies and Hypothetical Management Using the EOS Calculator.
Sourcea | Included Population | EOS Cases, No. | AB <24 h | AB >24 h (Missed) |
---|---|---|---|---|
Shakib et al,31 2015 | GA ≥34 wk, chorioamnionitis | 1 | 1 | 0 |
Money et al,34 2017 | GA ≥37 wk, chorioamnionitis | 1 | 0 | 1 |
Carola et al,35 2018 | GA ≥35 wk, chorioamnionitis | 5 | 3 | 2 |
Joshi et al,36 2019 | GA ≥34 wk | 5 | 3 | 2 |
Total, No. (%) | NA | 12 | 7 (58) | 5 (42) |
Discussion
Reduction of antibiotic overtreatment in neonates is of paramount importance to avoid early and late adverse effects. In this systematic review and meta-analysis of all studies reporting the results of actual or hypothetical implementation of the EOS calculator including more than 175 000 newborns, we found that use of the EOS calculator is associated with a marked reduction in empirical antibiotic therapy compared with conventional management strategies. Studies restricted to newborns exposed to chorioamnionitis indicate an even larger potential for reduction in antibiotic use in such newborns. Data on safety were limited due to rarity of safety outcomes. However, when scrutinizing available data, we found no indications that use of the EOS calculator leads to an increase in missed cases of EOS, overall EOS incidence, readmissions, delay in antibiotic therapy, or EOS-related morbidity or mortality.
Safety is of critical importance and risk of missing cases of EOS is a major concern in the evaluation of management strategies for newborns at risk for or with suspected EOS. Risk management strategies for EOS need to balance the risk of a missed case of EOS against the harm of unnecessary antibiotics on a population level.5,15 Even well-appearing newborns with no risk factors can develop EOS. Thus, not every case of EOS is predictable, and clinical judgment and monitoring continue to be an essential part in early diagnosis.39 This is reflected in the observation period included in management guided by the EOS calculator, as well as in promising alternatives such as serial physical examinations after birth.38,39,40,41 For many EOS risk management strategies, the risk of missing a case of EOS is largely unknown. In contrast, the EOS calculator provides an individual EOS risk estimate for each newborn, and our review summarizes the current real-world evidence on this outcome in clinical practice. Depending on setting and strategies used, the EOS calculator can also serve as a safety net by flagging at-risk newborns overseen by conventional management strategies, which are more categorical in their recommendation for treatment.42,43 Altogether, although evidence of the safety of management guided by the EOS calculator is limited, it shows no indication of inferiority compared with conventional management strategies thus far.
Strengths and Limitations
Strengths of our systematic review include an exhaustive search strategy, systematic data extraction and analysis following an a priori specified and registered protocol, and surveying of authors of included studies to ensure completeness of data. It provides a synthesis of a novel tool in area of great current clinical interest and concern.
Our review also has some limitations. Meta-analysis was restricted to before-after implementation studies, but included many newborns. The use of a 24-hour post partum as the cutoff to designate a missed case of EOS is arbitrary, but it reflects a common timeframe for monitoring of at-risk newborns.3,15,38,44 Finally, owing to a limited scope, this review did not investigate potential secondary benefits of the EOS calculator, such as reductions in laboratory investigations, neonatal ward admissions, or related health care costs.15,28,29,45
Careful interpretation of the results from this systematic review and, in particular, consideration to local circumstances is warranted. Included studies were unrandomized, inducing a high risk of bias and limiting the quality of the evidence.46 Studies were conducted during a time span in which adjustments to the EOS calculator were made, which may skew results from contemporary effects of the EOS calculator.3 Furthermore, studies were performed predominantly with newborns born at 35 weeks’ gestation or later, in tertiary settings, and conducted within the United States. Because other settings and populations can carry differences that can possibly be associated with the performance of the model, this can limit the generalizability of findings in several ways.
First, the EOS calculator was derived from and validated within the setting of a US health care system, with an EOS incidence rate of 0.6 per 1000 live births, while EOS incidence rates vary across the world and setting.47,48 In this review, we observed similar effects of management by the EOS calculator in studies outside the United States.26,28 Furthermore, baseline EOS incidence rates reported in included studies varied between 0.2 and 1.0 per 1000 live births, and selecting at-risk populations resulted in significantly higher a priori risk of EOS.35 To accommodate for different incidence rates, the EOS calculator allows for a wide range in a priori risk of EOS (up to 4 cases per 1000 live births) to be used, since 2018.49 This range allows for customization of this aspect according to setting and populations, although this feature is controversial and has thus far not been validated.49,50
Second, profound differences are seen in current strategies of empirical antibiotic therapy for suspected EOS. Marked differences exist among guidelines as well as between practices under the similar guidelines.1,51,52 On average, approximately 5% of term newborns in the United States are treated with empirical antibiotics,53 while percentages vary between 2.3% and 7.9% across Europe.54,55 In settings with a high ratio of treated infants to confirmed cases of EOS, the opportunity for a reduction using the EOS calculator is likely larger than in settings where use of antibiotics is already limited. Our finding of relatively large reductions associated with management guided by the EOS calculator in newborns exposed to chorioamnionitis illustrates this scenario. Although use of the EOS calculator in these populations is controversial,35,49,50 epidemiologic data support the safety of limited use of empirical antibiotics.54,56 One study included in this review reported an RR of 22.2%, even though use of antibiotics without the EOS calculator would have been relatively low, at 1.8%.37
Finally, significant variation is seen among strategies for testing maternal GBS status. In the United States, routine GBS screening during pregnancy was implemented in 2002,43 whereas some other countries use strategies based on risk factors.57 However, the derivation cohort included a significant proportion of newborns born before implementation of routine maternal GBS screening.13 As such, the EOS calculator allows for “unknown” as a valid value for the GBS variable of the prediction model, allowing for a calculated EOS risk estimate even when GBS status is unavailable. In addition, the relative contribution of GBS as a predictive factor in the EOS calculator is only 2.3%, and therefore, changes in setting related to GBS status will by definition have a limited association with the model.13 Thus, differences in maternal GBS testing strategies are unlikely to impede implementation of use of the EOS calculator.
The EOS calculator was developed and validated using EOS defined as a positive (uncontaminated) blood culture within the first 72 hours of life.13 However, EOS can occur even when physicians are unable to isolate a pathogen, and antenatal antibiotics may decrease the likelihood of successful pathogen isolation at birth. Critically, a consensus definition of EOS is also lacking. Physicians label a case as presumed, suspected, or culture-negative EOS up to 16 times more often than EOS is confirmed by a positive blood culture, often resulting in treatment with 5 or 7 days of intravenous antibiotics.58,59 Concerns regarding such cases and the EOS calculator include the theory that antenatal antibiotics may interfere with blood culture results, creating false-negative blood cultures, and that reducing empirical antibiotics may allow for more cases of EOS to develop into severe disease.15,31 However, as we found no indications of increased incidence or severity of EOS after reduction of empirical antibiotic use in EOS calculator implementation studies, our findings correspond with the observation that concerns for false-negative blood cultures are based largely on fallacies.58,60
Our review shows that the results of the EOS calculator are promising and underscores the worldwide interest in its applicability in clinical practice. However, use of a predictive model as an algorithm to allocate treatment strategies to newborns represents a large deviation from conventional protocols, and implementation efforts report hesitation and concerns among current practitioners.29,35,61 Ideally, implementation of a prediction model in a different setting is preceded by validation in that setting.62 For the EOS calculator, this is impractical owing to the large number of newborns needed to validate for rare outcomes such as proven EOS. However, well-designed prospective studies can be used to overcome research gaps and ensure careful implementation of the EOS calculator. Before-after studies such as that by Kuzniewicz et al15 carry an inherent risk of historical bias. A multinational cluster randomized trial comparing conventional practices and/or guidelines with the EOS calculator, however, possibly using a stepped-wedge design, would represent the ideal design to investigate the question.14,15,63,64 This design would allow for randomization and comparison of results among institutions and countries, while preventing contamination of EOS calculator experience within institutions. The results of such a study can also provide feedback usable for setting-specific adjustments for the use of the EOS calculator, such as a priori risk of EOS. This is likely to further improve EOS calculator use and related outcomes. Finally, future research should best evaluate the EOS calculator not in isolation, but combined with methods such as serial physical examinations36,40 and laboratory marker candidates.59,65
Conclusions
Our systematic review and meta-analysis demonstrate that the use of the EOS calculator is associated with a substantial reduction in the use of empirical antibiotics for suspected EOS. Evidence regarding safety of use of the EOS calculator is limited, but we found no indication of inferiority compared with conventional management strategies. A risk of missing cases of EOS or delaying antibiotics exists, but should be weighed against the relatively large reductions in unnecessary use of empirical antibiotics. Large prospective intervention studies outside of the United States, preferably cluster randomized, will be paramount in comparing the EOS calculator with current and alternative strategies, and in implementing the EOS calculator as a tool to safely reduce unnecessary antibiotics in newborns.
References
- 1.Schulman J, Dimand RJ, Lee HC, Duenas GV, Bennett MV, Gould JB. Neonatal intensive care unit antibiotic use. Pediatrics. 2015;135(5):826-833. doi: 10.1542/peds.2014-3409 [DOI] [PubMed] [Google Scholar]
- 2.Wortham JM, Hansen NI, Schrag SJ, et al. ; Eunice Kennedy Shriver NICHD Neonatal Research Network . Chorioamnionitis and culture-confirmed, early-onset neonatal infections. Pediatrics. 2016;137(1). doi: 10.1542/peds.2015-2323 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kuzniewicz MW, Walsh EM, Li S, Fischer A, Escobar GJ. Development and implementation of an early-onset sepsis calculator to guide antibiotic management in late preterm and term neonates. Jt Comm J Qual Patient Saf. 2016;42(5):232-239. doi: 10.1016/S1553-7250(16)42030-1 [DOI] [PubMed] [Google Scholar]
- 4.Benitz WE, Wynn JL, Polin RA. Reappraisal of guidelines for management of neonates with suspected early-onset sepsis. J Pediatr. 2015;166(4):1070-1074. doi: 10.1016/j.jpeds.2014.12.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Esaiassen E, Fjalstad JW, Juvet LK, van den Anker JN, Klingenberg C. Antibiotic exposure in neonates and early adverse outcomes: a systematic review and meta-analysis. J Antimicrob Chemother. 2017;72(7):1858-1870. doi: 10.1093/jac/dkx088 [DOI] [PubMed] [Google Scholar]
- 6.Cotten CM. Adverse consequences of neonatal antibiotic exposure. Curr Opin Pediatr. 2016;28(2):141-149. doi: 10.1097/MOP.0000000000000338 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Fjalstad JW, Esaiassen E, Juvet LK, van den Anker JN, Klingenberg C. Antibiotic therapy in neonates and impact on gut microbiota and antibiotic resistance development: a systematic review. J Antimicrob Chemother. 2018;73(3):569-580. doi: 10.1093/jac/dkx426 [DOI] [PubMed] [Google Scholar]
- 8.Mukhopadhyay S, Lieberman ES, Puopolo KM, Riley LE, Johnson LC. Effect of early-onset sepsis evaluations on in-hospital breastfeeding practices among asymptomatic term neonates. Hosp Pediatr. 2015;5(4):203-210. doi: 10.1542/hpeds.2014-0126 [DOI] [PubMed] [Google Scholar]
- 9.Gensollen T, Iyer SS, Kasper DL, Blumberg RS. How colonization by microbiota in early life shapes the immune system. Science. 2016;352(6285):539-544. doi: 10.1126/science.aad9378 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Olin A, Henckel E, Chen Y, et al. . Stereotypic immune system development in newborn children. Cell. 2018;174(5):1277-1292. doi: 10.1016/j.cell.2018.06.045 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mitre E, Susi A, Kropp LE, Schwartz DJ, Gorman GH, Nylund CM. Association between use of acid-suppressive medications and antibiotics during infancy and allergic diseases in early childhood. JAMA Pediatr. 2018;172(6):e180315. doi: 10.1001/jamapediatrics.2018.0315 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rasmussen SH, Shrestha S, Bjerregaard LG, et al. . Antibiotic exposure in early life and childhood overweight and obesity: a systematic review and meta-analysis. Diabetes Obes Metab. 2018;20(6):1508-1514. doi: 10.1111/dom.13230 [DOI] [PubMed] [Google Scholar]
- 13.Puopolo KM, Draper D, Wi S, et al. . Estimating the probability of neonatal early-onset infection on the basis of maternal risk factors. Pediatrics. 2011;128(5):e1155-e1163. doi: 10.1542/peds.2010-3464 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Escobar GJ, Puopolo KM, Wi S, et al. . Stratification of risk of early-onset sepsis in newborns ≥ 34 weeks’ gestation. Pediatrics. 2014;133(1):30-36. doi: 10.1542/peds.2013-1689 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kuzniewicz MW, Puopolo KM, Fischer A, et al. . A quantitative, risk-based approach to the management of neonatal early-onset sepsis. JAMA Pediatr. 2017;171(4):365-371. doi: 10.1001/jamapediatrics.2016.4678 [DOI] [PubMed] [Google Scholar]
- 16.Ayrapetyan M, Carola D, Lakshminrusimha S, Bhandari V, Aghai ZH. Infants born to mothers with clinical chorioamnionitis: a cross-sectional survey on the use of early-onset sepsis risk calculator and prolonged use of antibiotics. Am J Perinatol. 2019;36(4):428-433. doi: 10.1055/s-0038-1668548 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Sniderman AD, D’Agostino RB Sr, Pencina MJ. The role of physicians in the era of predictive analytics. JAMA. 2015;314(1):25-26. doi: 10.1001/jama.2015.6177 [DOI] [PubMed] [Google Scholar]
- 18.Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood). 2014;33(7):1148-1154. doi: 10.1377/hlthaff.2014.0352 [DOI] [PubMed] [Google Scholar]
- 19.Shamseer L, Moher D, Clarke M, et al. ; PRISMA-P Group . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. doi: 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]
- 20.Moher D, Shamseer L, Clarke M, et al. ; PRISMA-P Group . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. doi: 10.1186/2046-4053-4-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Moons KG, de Groot JA, Bouwmeester W, et al. . Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014;11(10):e1001744. doi: 10.1371/journal.pmed.1001744 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Atkins D, Best D, Briss PA, et al. ; GRADE Working Group . Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490-1494. doi: 10.1136/bmj.328.7454.1490 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Ryan R, Hill S How to GRADE the quality of the evidence. https://colorectal.cochrane.org/sites/colorectal.cochrane.org/files/public/uploads/how_to_grade.pdf. Published December 1, 2016. Accessed January 16, 2019.
- 24.The Cochrane Collaboration In: Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. https://training.cochrane.org/handbook. Accessed November 9, 2018.
- 25.Armitage P, Berry G, Matthews JNS. Statistical Methods in Medical Research Third ed. London, UK: Blackwell Science; 1994. [Google Scholar]
- 26.Achten NB, Dorigo-Zetsma JW, van der Linden PD, van Brakel M, Plötz FB. Sepsis calculator implementation reduces empiric antibiotics for suspected early-onset sepsis. Eur J Pediatr. 2018;177(5):741-746. doi: 10.1007/s00431-018-3113-2 [DOI] [PubMed] [Google Scholar]
- 27.Dhudasia MB, Mukhopadhyay S, Puopolo KM. Implementation of the sepsis risk calculator at an academic birth hospital. Hosp Pediatr. 2018;8(5):243-250. doi: 10.1542/hpeds.2017-0180 [DOI] [PubMed] [Google Scholar]
- 28.Strunk T, Buchiboyina A, Sharp M, Nathan E, Doherty D, Patole S. Implementation of the neonatal sepsis calculator in an Australian tertiary perinatal centre. Neonatology. 2018;113(4):379-382. doi: 10.1159/000487298 [DOI] [PubMed] [Google Scholar]
- 29.Gievers LL, Sedler J, Phillipi CA, et al. . Implementation of the sepsis risk score for chorioamnionitis-exposed newborns. J Perinatol. 2018;38(11):1581-1587. doi: 10.1038/s41372-018-0207-7 [DOI] [PubMed] [Google Scholar]
- 30.Beavers JB, Bai S, Perry J, Simpson J, Peeples S. Implementation and evaluation of the early-onset sepsis risk calculator in a high-risk university nursery. Clin Pediatr (Phila). 2018;57(9):1080-1085. doi: 10.1177/0009922817751337 [DOI] [PubMed] [Google Scholar]
- 31.Shakib J, Buchi K, Smith E, Young PC. Management of newborns born to mothers with chorioamnionitis: is it time for a kinder, gentler approach? Acad Pediatr. 2015;15(3):340-344. doi: 10.1016/j.acap.2014.11.007 [DOI] [PubMed] [Google Scholar]
- 32.Kerste M, Corver J, Sonnevelt MC, et al. . Application of sepsis calculator in newborns with suspected infection. J Matern Fetal Neonatal Med. 2016;29(23):3860-3865. doi: 10.3109/14767058.2016.1149563 [DOI] [PubMed] [Google Scholar]
- 33.Warren S, Garcia M, Hankins C. Impact of neonatal early-onset sepsis calculator on antibiotic use within two tertiary healthcare centers. J Perinatol. 2017;37(4):394-397. doi: 10.1038/jp.2016.236 [DOI] [PubMed] [Google Scholar]
- 34.Money N, Newman J, Demissie S, Roth P, Blau J. Anti-microbial stewardship: antibiotic use in well-appearing term neonates born to mothers with chorioamnionitis. J Perinatol. 2017;37(12):1304-1309. doi: 10.1038/jp.2017.137 [DOI] [PubMed] [Google Scholar]
- 35.Carola D, Vasconcellos M, Sloane A, et al. . Utility of early-onset sepsis risk calculator for neonates born to mothers with chorioamnionitis. J Pediatr. 2018;195:48-52. doi: 10.1016/j.jpeds.2017.11.045 [DOI] [PubMed] [Google Scholar]
- 36.Joshi NS, Gupta A, Allan JM, et al. . Management of chorioamnionitis-exposed infants in the newborn nursery using a clinical examination-based approach. Hosp Pediatr. 2019;9(4):227-233. doi: 10.1542/hpeds.2018-0201 [DOI] [PubMed] [Google Scholar]
- 37.Klingaman C, King L, Neff-Bulger M. Improved newborn care: evidence-based protocol for the evaluation and management of early-onset sepsis. Am J Med Qual. 2018;33(1):106. doi: 10.1177/1062860617741437 [DOI] [PubMed] [Google Scholar]
- 38.Joshi NS, Gupta A, Allan JM, et al. . Clinical monitoring of well-appearing infants born to mothers with chorioamnionitis. Pediatrics. 2018;141(4):e20172056. doi: 10.1542/peds.2017-2056 [DOI] [PubMed] [Google Scholar]
- 39.Puopolo KM, Benitz WE, Zaoutis TE; Committee on Fetus and Newborn; Committee on Infectious Diseases . Management of neonates born at ≥35 0/7 weeks’ gestation with suspected or proven early-onset bacterial sepsis. Pediatrics. 2018;142(6):e20182894. doi: 10.1542/peds.2018-2894 [DOI] [PubMed] [Google Scholar]
- 40.Berardi A, Buffagni AM, Rossi C, et al. . Serial physical examinations, a simple and reliable tool for managing neonates at risk for early-onset sepsis. World J Clin Pediatr. 2016;5(4):358-364. doi: 10.5409/wjcp.v5.i4.358 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Good PI, Hooven TA. Evaluating newborns at risk for early-onset sepsis. Pediatr Clin North Am. 2019;66(2):321-331. doi: 10.1016/j.pcl.2018.12.003 [DOI] [PubMed] [Google Scholar]
- 42.National Institute for Health and Clinical Excellence Neonatal infection (early onset): antibiotics for prevention and treatment. https://www.nice.org.uk/guidance/cg149/resources/neonatal-infection-early-onset-antibiotics-for-prevention-and-treatment-35109579233221. Published August 22, 2012. Accessed June 19, 2018.
- 43.Verani JR, McGee L, Schrag SJ; Division of Bacterial Diseases, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention (CDC) . Prevention of perinatal group B streptococcal disease–revised guidelines from CDC, 2010. MMWR Recomm Rep. 2010;59(RR-10):1-36. [PubMed] [Google Scholar]
- 44.Jefferies AL. Management of term infants at increased risk for early-onset bacterial sepsis. Paediatr Child Health. 2017;22(4):223-228. doi: 10.1093/pch/pxx023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Gong CL, Dasgupta-Tsinikas S, Zangwill KM, Bolaris M, Hay JW. Early onset sepsis calculator-based management of newborns exposed to maternal intrapartum fever: a cost benefit analysis. J Perinatol. 2019;39(4):571-580. doi: 10.1038/s41372-019-0316-y [DOI] [PubMed] [Google Scholar]
- 46.Loke YK, Price D, Herxheimer A; Cochrane Adverse Effects Methods Group . Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol. 2007;7:32. doi: 10.1186/1471-2288-7-32 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Mukhopadhyay S, Puopolo KM. Neonatal early-onset sepsis: epidemiology and risk assessment. Neoreviews. 2015;16(4):e221-e230. doi: 10.1542/neo.16-4-e221 [DOI] [Google Scholar]
- 48.Shane AL, Sánchez PJ, Stoll BJ. Neonatal sepsis. Lancet. 2017;390(10104):1770-1780. doi: 10.1016/S0140-6736(17)31002-4 [DOI] [PubMed] [Google Scholar]
- 49.Degraeuwe P. Applying the neonatal early-onset sepsis calculator in cases of clinical chorioamnionitis at or after 34 weeks of gestation. J Pediatr. 2018;203:463-464. doi: 10.1016/j.jpeds.2018.07.077 [DOI] [PubMed] [Google Scholar]
- 50.Carola D, Greenspan J, Aghai ZH. Reply. J Pediatr. 2018;203:464-465. doi: 10.1016/j.jpeds.2018.07.084 [DOI] [PubMed] [Google Scholar]
- 51.van Herk W, el Helou S, Janota J, et al. . Variation in current management of term and late-preterm neonates at risk for early-onset sepsis: an international survey and review of guidelines. Pediatr Infect Dis J. 2016;35(5):494-500. doi: 10.1097/INF.0000000000001063 [DOI] [PubMed] [Google Scholar]
- 52.Mukhopadhyay S, Taylor JA, Von Kohorn I, et al. . Variation in sepsis evaluation across a national network of nurseries. Pediatrics. 2017;139(3):e20162845. doi: 10.1542/peds.2016-2845 [DOI] [PubMed] [Google Scholar]
- 53.Flannery DD, Puopolo KM. Neonatal antibiotic use: what are we doing and where shall we go? Neoreviews. 2018;19(9):e516-e525. doi: 10.1542/neo.19-9-e516 [DOI] [Google Scholar]
- 54.Fjalstad JW, Stensvold HJ, Bergseng H, et al. . Early-onset sepsis and antibiotic exposure in term infants: a nationwide population-based study in Norway. Pediatr Infect Dis J. 2016;35(1):1-6. doi: 10.1097/INF.0000000000000906 [DOI] [PubMed] [Google Scholar]
- 55.van Herk W, Stocker M, van Rossum AM. Recognising early onset neonatal sepsis: an essential step in appropriate antimicrobial use. J Infect. 2016;72(suppl):S77-S82. doi: 10.1016/j.jinf.2016.04.026 [DOI] [PubMed] [Google Scholar]
- 56.Duvoisin G, Fischer C, Maucort-Boulch D, Giannoni E. Reduction in the use of diagnostic tests in infants with risk factors for early-onset neonatal sepsis does not delay antibiotic treatment. Swiss Med Wkly. 2014;144:w13981. doi: 10.4414/smw.2014.13981 [DOI] [PubMed] [Google Scholar]
- 57.Homer CSE, Scarf V, Catling C, Davis D. Culture-based versus risk-based screening for the prevention of group B streptococcal disease in newborns: a review of national guidelines. Women Birth. 2014;27(1):46-51. doi: 10.1016/j.wombi.2013.09.006 [DOI] [PubMed] [Google Scholar]
- 58.Klingenberg C, Kornelisse RFRF, Buonocore G, Maier RF, Stocker M. Culture-negative early-onset neonatal sepsis—at the crossroad between efficient sepsis care and antimicrobial stewardship. Front Pediatr. 2018;6:285. doi: 10.3389/fped.2018.00285 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Stocker M, van Herk W, El Helou S, et al. ; NeoPInS Study Group . Procalcitonin-guided decision making for duration of antibiotic therapy in neonates with suspected early-onset sepsis: a multicentre, randomised controlled trial (NeoPIns). Lancet. 2017;390(10097):871-881. doi: 10.1016/S0140-6736(17)31444-7 [DOI] [PubMed] [Google Scholar]
- 60.Cantey JB, Baird SD. Ending the culture of culture-negative sepsis in the neonatal ICU. Pediatrics. 2017;140(4):e20170044. doi: 10.1542/peds.2017-0044 [DOI] [PubMed] [Google Scholar]
- 61.Rajbhandari S, La Gamma EF. Early-onset sepsis calculator—risk of delaying treatment. JAMA Pediatr. 2017;171(10):1015. doi: 10.1001/jamapediatrics.2017.2476 [DOI] [PubMed] [Google Scholar]
- 62.Moons KGM, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338(June):b606. doi: 10.1136/bmj.b606 [DOI] [PubMed] [Google Scholar]
- 63.Hendriksen JMT, Geersing GJ, Moons KGM, de Groot JA. Diagnostic and prognostic prediction models. J Thromb Haemost. 2013;11(suppl 1):129-141. doi: 10.1111/jth.12262 [DOI] [PubMed] [Google Scholar]
- 64.Moons KGM, Kengne AP, Grobbee DE, et al. . Risk prediction models, II: external validation, model updating, and impact assessment. Heart. 2012;98(9):691-698. doi: 10.1136/heartjnl-2011-301247 [DOI] [PubMed] [Google Scholar]
- 65.Newman TB, Draper D, Puopolo KM, Wi S, Escobar GJ. Combining immature and total neutrophil counts to predict early onset sepsis in term and late preterm newborns: use of the I/T2. Pediatr Infect Dis J. 2014;33(8):798-802. doi: 10.1097/INF.0000000000000297 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.