Early-Onset Sepsis Risk Calculator Integration Into an Electronic Health Record in the Nursery

Abstract

BACKGROUND AND OBJECTIVES:

An early-onset sepsis (EOS) risk calculator tool to guide evaluation and treatment of infants at risk for sepsis has reduced antibiotic use without increased adverse outcomes. We performed an electronic health record (EHR)–driven quality improvement intervention to increase calculator use for infants admitted to a newborn nursery and reduce antibiotic treatment of infants at low risk for sepsis.

METHODS:

This 2-phase intervention included programming (1) an EHR form containing calculator fields that were external to the infant’s admission note, with nonautomatic access to the calculator, education for end-users, and reviewing risk scores in structured bedside rounds and (2) discrete data entry elements into the EHR admission form with a hyperlink to the calculator Web site. We used statistical process control to assess weekly entry of risk scores and antibiotic orders and interrupted time series to assess trend of antibiotic orders.

RESULTS:

During phase 1 (duration, 14 months), a mean 59% of infants had EOS calculator scores entered. There was wide variability around the mean, with frequent crossing of weekly means beyond the 3σ control lines, indicating special-cause variation. During phase 2 (duration, 2 years), mean frequency of EOS calculator use increased to 85% of infants, and variability around the mean was within the 3σ control lines. The frequency of antibiotic orders decreased from preintervention (7%) to the final 6 months of phase 2 (1%, P < .001).

CONCLUSIONS:

An EHR-driven quality improvement intervention increased EOS calculator use and reduced antibiotic orders, with no increase in adverse events.

Antibiotic overuse may cause morbidity, mortality, and cost, including adverse drug events and antimicrobial resistance.^1–6 In newborns treated in a newborn nursery, antibiotics usually are discontinued when blood culture results remain negative and the infant has no signs or symptoms suggesting infection. However, safety concerns about even brief courses of antibiotics include potential effects on the microbiome and the development of allergic and inflammatory bowel diseases.^7,8

Until recently, antibiotic use in newborns was guided by Centers for Disease Control and Prevention (CDC) (2010) and American Academy of Pediatrics Committee on the Fetus and Newborn (2012) recommendations^9,10 (updated after the current study was performed^11,12). The recommendations include a low threshold for obtaining laboratory studies and initiating intravenous antimicrobial therapy in newborns who have risk factors including maternal group B Streptococcus (GBS) colonization, maternal intrauterine infection (chorioamnionitis), and signs or symptoms suggesting sepsis.

Antibiotic use is variable because the decision to treat depends on many factors.¹³ Maternal intrauterine infection typically is diagnosed during labor, but 26% of obstetricians make the diagnosis from maternal fever alone, leading to overdiagnosis because incidental fever occurs during labor in 7% of women and after perinatal epidural anesthesia in 15% of women.^14,15 The incidence of neonatal early-onset sepsis (EOS) decreased after publication of the 1996 CDC guidelines that recommended maternal intrapartum antibiotics for GBS colonization and other risk factors (range per 1000 live births: before publication, 2.0–3.7 cases; after publication, 0.5–1.59 cases).^16–23 However, the frequency of testing for EOS (15%–20%) and use of empirical antibiotics to prevent EOS (5%–8%) remain high.^24–27 Therefore, there is controversy about the 2010 guidelines, and following these recommendations may cause harm because of unnecessary interventions to healthy, low-risk infants.^13,28,29

Clinical decision support integrated into the electronic health record (EHR) aids clinicians to recognize sepsis and implement evidence-based treatment in adults,³⁰ but EHR integration of clinical decision support tools for neonatal sepsis has not been reported previously. In 2014, an open-source neonatal EOS risk calculator tool was described that begins with the previous probability of EOS and links 2 prediction models on the basis of (1) objective data known at delivery (gestational age, highest maternal antepartum temperature, GBS status, duration of rupture of membranes, and intrapartum antibiotic type and timing) and (2) infant clinical presentation to provide a final EOS risk estimate or posterior probability of infection.^20,31 Using this calculator to guide evaluation and treatment of infants who are at risk for developing sepsis reduces the proportion of infants receiving antibiotics by 50% without increased adverse outcomes.²⁵

The aims of this project were to integrate the neonatal EOS risk calculator into an EHR and evaluate the effect of EHR integration on calculator use and antibiotic treatment of infants admitted to a newborn nursery. We hypothesized that EHR integration of the calculator would decrease antibiotic order frequency by 50% within 1 year.

METHODS

Context

At University of Utah Hospital, 4000 infants are born annually, including 3500 infants at gestational age ≥34 weeks who transition to their mother’s room (couplet care) or are treated in an intermediate nursery and 500 infants who are transferred to the NICU for prematurity and neonatal comorbidities. Clinically stable infants ≥35 weeks’ gestation remain in couplet care. Infants 34 to 35 weeks’ gestation and infants who require intermediate care (oxygen via nasal cannula, continuous glucose infusion, close monitoring, or isolette for temperature instability) and who do not need NICU transfer receive intermediate nursery care. Infants not transferred to the NICU receive care by academic general pediatricians (75% of infants) or family medicine providers (25%). Antibiotic treatment alone is not an indication for NICU transfer. An EHR (Epic, Epic Systems Corp, Verona, WI) was implemented in the nursery 6 months before this study.

The pediatric nursery service includes 16 general pediatric attending physicians: 1 medical director, 7 additional core physicians (weekday service and overnight calls), and 8 other physicians (weekend coverage). The medical director and core physicians meet monthly to discuss protocols and resident education. Four resident physicians (pediatric and family medicine) and 2 to 4 medical students are supervised by pediatric attending physicians and cover the pediatric nursery service 24 hours per day. Residents examine patients, write medical record notes, and present all couplet care and intermediate nursery patients to the pediatric attending physician on morning rounds. Pediatric service resident and attending physicians attend to infants on the family medicine service when consultation is requested.

Family medicine nursery service providers are from 2 health care organizations: University of Utah and a Federally Qualified Health Center system. Physicians from the latter provide obstetric and newborn care at 2 hospitals. When family medicine service providers are outside the hospital, they observe patients through communication with nurses.

Planning the Intervention

Initial planning was on the basis of chart review of 698 infants born at gestational age ≥34 weeks to mothers who had chorioamnionitis. Calculator scores were applied retrospectively to assess the value and safety of calculator use.¹³ A multidisciplinary team was assembled, including medical and nursing leaders, nursery core attending physicians, and EHR analysts. Key drivers for calculator use were identified in group discussion, including physician awareness of the calculator, physician acceptance of calculator scores as an evidence-based alternative to CDC guidelines, resident understanding and use of calculator scores and treatment recommendations, and EHR design to facilitate calculator use. The study was reviewed and approved by the University of Utah Institutional Review Board.

Intervention

Preintervention data were collected about antibiotic orders for 6 months (defined as baseline; June 2, 2014 through November 17, 2014). Intervention phase 1 (November 18, 2014 through January 13, 2016) began after programming an EHR form (flow sheet) that contained the calculator fields and was external to the nursery admission note (Fig 1). Access to calculator fields was not automatic but required the user to click on a sidebar option (within the EHR but outside the encounter note) to display a form named Risk Assessments that contained fields to document calculator scores. This form was accessible through admission, rounding, and discharge EHR workflow activities and contained the calculator Internet address (https://neonatalsepsiscalculator.kaiserpermanente.org.) that was entered manually into a browser.³² During the intervention, our institution did not have the capability to autopopulate objective data elements into the flow sheet; manual entry was required. Calculator use was defined as provider entry of calculator scores into the EHR calculator fields.

FIGURE 1.

EHR form for phase 1 that was external to the nursery admission note and accessed from the sidebar (Risk Assessments). URL to the EOS risk calculator was entered manually into an Internet browser. Providers entered calculator scores manually into the 3 empty fields (rectangles).

At the beginning of phase 1, the EHR flow sheet was released. General pediatric physicians and maternal-newborn care nursing leaders attended an educational presentation about the calculator and supporting evidence. An e-mail was sent to all nursery providers that included calculator literature and the recommendation to use the tool. Provider questions about calculator use were addressed in person, on rounds, and by e-mail. In addition to vital sign checks every 30 minutes during transition to extrauterine life until 2 sets of normal vital signs were obtained, subsequent vital sign checks were increased from every 8 hours to every 4 hours for the nursery stay duration.³³ Nurses were educated about the increase in vital sign frequency by e-mail and at a staff meeting.

Infants of mothers who were diagnosed with intrauterine infection were transferred to the transition nursery for close nursing observation for the first hour after birth. All other infants without comorbidities stayed with their mothers during fetal-to-neonatal transition. The calculator scores and treatment recommendations were added to the pediatric resident sign-out sheets, and pediatric and family medicine residents on the pediatric service were required to present the scores and treatment recommendations during structured bedside rounds with the pediatric attending physician. Residents were expected to input calculator scores into the EHR at infant admission. Admission calculator scores automatically populated the resident sign-out sheets, ensuring that scores were available for patient care discussions and rounds.

After the first year of phase 1, it was shown in periodic data review that calculator fields were not used universally. We planned for phase 2 of the Plan-Do-Study-Act cycle³⁴ by identifying barriers to calculator use with methods described previously,³⁵ including the need to click outside the encounter note to access calculator fields, manual entry of the Internet address for calculator access, lack of automatic population of notes with calculator recommendations, and lack of reminders to access the calculator. The EHR analysts programmed discrete data entry elements into a structured documentation template (known in this EHR as a Smart Form) for the admission encounter, with a hyperlink to the calculator Internet page (Fig 2). Links were added to populate the admission, progress, and discharge notes with calculator recommendations. The major difference in phase 2 programming besides the hyperlink was the integration of calculator fields into the encounter note, avoiding the need for the provider to remember to open the Risk Assessments sidebar option.

FIGURE 2.

EHR Smart Form for phase 2, with discrete entry elements programmed into the admission encounter and URL hyperlink to the calculator Web site.

Phase 2 began January 14, 2016, when the Smart Form was released and flow sheet was discontinued. Balancing measures were infant positive culture results and NICU transfers before and after intervention.

Measures

Our primary outcome measures were provider documentation of calculator scores and infant antibiotic orders by pediatric or family medicine providers. Balancing measures included NICU transfers and positive blood culture results during the birth admission or readmission.

Analysis

Statistical process control (SPC) and interrupted time series were used to monitor the effect of interventions over time.³⁶ We used p-charts with the control set at 3σ to track the proportion of infants born weekly for whom the calculator was used to inform their care.³⁶ Another SPC technique, the cumulative summation (CUSUM) chart, was used to follow the effect of calculator use on the proportion of infants born weekly who had antibiotics ordered.³⁷ We used χ² test to compare calculator use and antibiotics ordered between phases and by provider type and NICU transfers before versus after starting the interventions. During the final 6 months of phase 2, antibiotic orders were analyzed separately because of observed increased accumulation of deviation below the goal on the CUSUM chart. Interrupted time series was used to evaluate trend in each phase, and confidence intervals were calculated.³⁸ Positive blood culture results were identified by querying laboratory results in the university data warehouse and billing data with sepsis diagnoses from readmissions to the local pediatric hospital. Statistical software was used (version 2017, QIMacros, KnowWare International, Denver, CO; Stata IC 15.1, StataCorp, College Station, TX), using Montgomery stability rules for SPC.³⁹

RESULTS

During the study period (June 2, 2014 through December 31, 2017), 12 111 newborns were admitted to the nursery. We excluded 187 infants (1.5%) who were transferred to the NICU before age 6 hours, primarily for continuous positive airway pressure per institutional policy, because treatment decisions in these infants were made by NICU clinicians who did not receive the intervention. We included the other 11 924 infants (98.5%) who remained in the nursery for at least 6 hours. During phase 1 (duration, 14 months), a mean of 59% of all infants had calculator scores entered into the EHR (Fig 3). There was wide variability around the mean with frequent crossing of the weekly means beyond the 3σ control lines (special-cause variation), indicating that the process was out of control and needed improvement. During phase 2 (duration, 2 years), the mean frequency of calculator use increased to 85% of all infants, and variability around the mean remained within the 3σ control lines.

FIGURE 3.

Calculator use and antibiotics ordered. A, SPC p-chart: proportion of nursery infants each week who had calculator scores entered into the EHR fields by the provider. The center line in each study phase was the calculated mean of the weekly proportion of infants with calculator used during the phase. The UCL and LCL were 3σ above and below the center line. The breaks in the center lines were determined by the process changes. B, SPC CUSUM chart: proportion of nursery infants each week who had antibiotics ordered within 24 hours after birth.

At baseline (preintervention), 7% of infants had antibiotics ordered, and steady accumulation of deviation above the ≤3% goal was shown in the CUSUM chart, which we set on the basis of previous studies (Fig 3).^25,40 During phase 1, the slope of the CUSUM line flattened (first half of phase 1) and declined steadily (second half of phase 1), and the CUSUM goal of ≤3% of infants receiving antibiotics was reached by the end of phase 1. During phase 2, the proportion of infants who received antibiotics was maintained at the 3% goal for 6 months and decreased to <3% during the final 18 months. During the final 6 months of the study, 1% of all infants had antibiotics ordered.

The interrupted time series of the proportion of infants born per week who received antibiotics showed a baseline increase of 0.16% per week (CI: 0.002% to 0.32%; P = .05) (Fig 4). The immediate treatment effect on the regression line (interruption of the line) at the start of phase 1 showed an average decrease in orders (−3.3%; CI: −1.7%; to −5.1%; P < .001). The trend of average antibiotic orders (slope) decreased in phase 1 (−0.08% per week; CI: −0.1% to 0.06%; P < .001). There was an immediate effect increase of 1.5% average antibiotic orders at the start of phase 2 (CI: 0.7% to 2.3%; P < .001), and the trend of average antibiotic orders in phase 2 decreased weekly (−0.01% per week; CI: −0.02% to 20.01%; P < .001).

FIGURE 4.

Interrupted time series of the proportion of infants receiving antibiotics each week.

The calculator was used more frequently for pediatrics than family medicine service infants in both phase 1 and 2, and the calculator was used during phase 2 almost universally by the pediatric service but for less than half of family medicine infants (Table 1). The frequency of antibiotics administered was similar between the pediatrics and family medicine services throughout the study (Table 1).

Table 1.

EOS Risk Calculator Use and Antibiotics Ordered in the Newborn Nursery by Family Medicine and Pediatric Service Providers

Phase	Calculator Used			Antibiotics Ordereda
	Family Medicine, n (%)	Pediatrics, n (%)	P<	Family Medicine, n (%)	Pediatrics, n (%)	P =
Preintervention	—	—	—	30 out of 406 (7)	82 out of 1213 (7)	.65
1	51 out of 962 (5)	2141 out of 2732 (78)	.001	43 out of 962 (4)	79 out of 2732 (3)	.02
2	685 out of 1602 (43)	4912 out of 5009 (98)	.001	20 out of 1602 (1)	93 out of 5009 (2)	.12

Data reported as number of infants who had the EOS risk calculator used or who received antibiotics per total number of infants (%). —, not applicable.

^aFor family medicine and pediatrics combined, infants with antibiotic orders were as follows: preintervention, 112 out of 1619 (7%); phase 1, 122 out of 3694 (3%); phase 2, 113 out of 6611 (2%). For the final 6 mo of phase 2, 18 out of 1741 (1%) infants had antibiotics ordered. Comparison of preintervention versus final 6 mo of phase 2, P < .001.

There were 3 infants who had positive blood culture results during the birth admission and were treated with antibiotics and discharged in good condition (Table 2). In addition, there was 1 infant who was discharged from the nursery and seen for poor feeding at 4 and 5 days after birth; this infant became febrile at 6 days after birth and was admitted to the pediatric hospital. Cerebrospinal fluid culture and imaging confirmed GBS meningitis with cerebral abscess. Because the mother was negative for GBS on prenatal testing, and both mother and infant were asymptomatic throughout the birth admission, this infant would not have received antibiotics using our preintervention protocol (Table 2).

TABLE 2.

Positive Blood Culture Results That Prompted Treatment in 4 Infants During the Study

Admission Type	Pathogen	EOS Risk per 1000 at Birth	EOS Risk per 1000 After Clinical Examination (Well Appearing, Equivocal, Clinical Illness)a	Phase	Reason for Treatment	Duration of Antibiotics
Birth admission	GBS	0.09	0.04, 0.43, 1.83	Preintervention	Clinically ill at 6 h	7d
Birth admission	Enterococcus faecalis	0.25	0.10, 1.27, 5.37	1	Maternal chorioamnionitis and prolonged rupture of membranes	7 d
Birth admission	Escherichia coli	0.07	0.03, 0.34, 1.43	1	Infant fever (38.3°C) at 45 min postpartum	14 d
Readmission	GBS	0.21	0.09, 1.07, 4.52	2	Fever at 6 d after birth; GBS meningitis with cerebral abscess	4 wk

Excluding 22 contaminant cultures that were not treated with antibiotics.

^aAdjusted for clinical condition using an EOS incidence of 0.3 per 1000 live births.

The frequency of NICU transfers was similar between the preintervention (no. of NICU transfers per no. of infants admitted to nursery, 14 out of 1605 [0.9%]) and intervention periods (79 out of 10 226 [0.8%]; P = .68).

DISCUSSION

In this academic couplet care and intermediate level nursery, EHR integration increased calculator use and reduced antibiotic orders without increased adverse events. Although reductions in antibiotic use and testing with calculator use were shown in previous studies,^25,40,41 we extended previous findings with the current study by showing that EHR integration of the calculator in a setting with diverse providers can be achieved effectively with minimal EHR design support.

Calculator use increased substantially in phase 2 by 44%, when the fields were integrated into the infant admission encounter with a hyperlink to the calculator Web site, and calculator recommendations were linked to populate the admission, progress, and discharge notes. Clinical practice is improved by automatic provision of decision support and recommendations as part of clinician workflow, whereas support that requires active initiation by the clinician is less likely to be adopted.^42,43 Although calculator scores in phase 2 were entered manually, increased score charting and reduced variability around the mean were observed after integration of the score fields into the encounter note, rather than requiring the user to click in the sidebar, and inclusion of a hyperlink instead of manual entry of the Web site address.

Periodic clinician feedback regarding use of decision support is associated with improved compliance with protocols.⁴² The requirement that pediatric service residents obtain calculator scores for the sign-out sheets used for morning rounds with attending physicians who evaluated their performance may explain why calculator use was greater for the pediatric than family medicine service. Attending physician feedback and evaluation of the residents may have outweighed the inconvenience of the nonautomatic decision support, whereas family medicine providers had no residents. Despite the difference in frequency of filled calculator fields between the 2 services, the decrease in antibiotic orders was similar (Table 1). As calculator use increased, antibiotic orders decreased. Family medicine providers may have used the calculator for clinical decision support without filling in the fields. In addition, presentation of the calculator as an alternative to CDC guidelines may have introduced a shift toward clinical observation instead of automatic antibiotic use, without close adherence to calculator recommendations.

Limited information is available about the effect of clinical decision support on clinical and economic outcomes.⁴⁴ We observed no adverse events associated with calculator-driven decisions, and the frequency of NICU transfers remained stable. In 2011, average hospital cost for newborn care in the United States was $3200 per infant.⁴⁵ In our study, 6% fewer infants received antibiotics, and direct hospital costs were 2.5-fold higher for infants receiving antibiotics. For 3500 nursery admissions per year at our institution, the potential annual savings may exceed $1 million. Scaled nationwide for 3 800 000 births annually,⁴⁵ potential cost savings may exceed $1 billion.

The 187 infants (1.5%) transferred to the NICU before age 6 hours, and excluded from the study, may have received antibiotics if care had been guided by the calculator definition of clinical illness. Addition of these patients to the 1% of infants treated empirically (final 6 months of study) may have resulted in antibiotic treatment in 2.5% of all infants, similar to the proportion treated in the calculator validation study.²⁵

Limitations of this study include limited generalizability because the study was performed at a single center that encourages quality improvement initiatives; a multicenter trial may be conducted to evaluate variation between different centers and populations. Some infants (1%) were lost to follow-up, and some positive blood culture results may have been missed because only billing data were available from the children’s hospital, without laboratory results. The frequency of calculator use without EHR documentation is unknown.

CONCLUSIONS

EHR integration of the EOS risk calculator reduced antibiotic orders from 7% to 1% of infants in the newborn nursery without increased adverse events. Further study may include comparing calculator scores with treatment decisions, and qualitative studies may be conducted to clarify how the calculator is used. Additional EHR automation and integration of calculator scores and recommendations may increase adoption. Furthermore, simplifying the provider documentation of discrete reasons for treatment decisions and deviation from protocols would be valuable measures to inform future educational and workflow interventions.

ABBREVIATIONS

CDC

Centers for Disease Control and Prevention

CUSUM

cumulative summation

EHR

electronic health record

EOS

early-onset sepsis

GBS

group B Streptococcus

SPC

statistical process control