Improving Recognition of Pediatric Severe Sepsis in the Emergency Department: Contributions of a Vital Sign Based Electronic Alert and Bedside Clinician Identification

Abstract

Objective

Recognition of pediatric sepsis is a key clinical challenge. We evaluated the performance of a sepsis recognition process including an electronic sepsis alert (ESA) and bedside assessment in a pediatric emergency department (ED).

Methods

Cohort study with quality improvement intervention in a pediatric ED. Exposure was a positive ESA, defined as 1) elevated heart rate or hypotension, 2) concern for infection, and 3) at least one: abnormal capillary refill, abnormal mental status, or high-risk condition. Positive ESA prompted team assessment/huddle to determine need for sepsis protocol. Clinicians could initiate team assessment/huddle based on clinical concern without positive ESA. Severe sepsis outcome defined as: 1) activation of the sepsis protocol in the ED or 2) development of severe sepsis requiring intensive care unit admission within 24 hours.

Results

There were 182,509 ED visits during the study period, with 86,037 pre-ESA implementation and 96,472 post-implementation, and 1112 (1.2%) positive ESAs. Overall, 326 patients (0.3%) were treated for severe sepsis within 24 hours. Test characteristics of the ESA alone to detect severe sepsis were sensitivity 86.2% (95%CI 82.0, 89.5), specificity 99.1% (95% CI 99.0, 99.2), positive predictive value 25.4% (95% CI 22.8, 28.0), and negative predictive value 100% (95% CI 99.9, 100). Inclusion of the clinician screen identified 43 additional ESA negative children with severe sepsis sensitivity 99.4% (97.8, 99.8%); specificity 99.1% (95% CI 99.0, 99.2). ESA implementation increased ED sepsis detection from 83% to 96%.

Conclusions

ESA for severe sepsis demonstrated good sensitivity and high specificity. Addition of clinician identification of ESA negative patients further improved sensitivity. Implementation of the ESA was associated with improved recognition of severe sepsis.

Introduction

Severe sepsis is a complex clinical syndrome resulting from the systemic inflammatory response (SIRS) to infection. Each year over 75,000 children are treated for severe sepsis in the United States resulting in substantial morbidity, up to 20% mortality, and over $4.8 billion in US health care expenditures.^1–3 Early and accurate recognition of pediatric severe sepsis is challenging since many children present initially with compensated shock without apparent hypotension.^4–6 Consequently, identifying the rare child with severe sepsis/septic shock from amongst many non-septic patients with fever and tachycardia presenting to a pediatric ED is truly akin to finding the proverbial “needle in a haystack.”

While recent reports demonstrate improved timeliness of severe sepsis therapy,^8,9 decreased sepsis related organ dysfunction, decreased hospital and intensive care unit (ICU) length of stay, and decreased mortality with protocol-guided care for pediatric severe sepsis/septic shock, ^10–12 determining which patients may most benefit remains problematic. Alerts based on clinical physiologic data embedded in an electronic health record (EHR) system have been studied as potential methods to facilitate sepsis recognition in adults.^13–15 Several investigators have evaluated candidate alerts based on SIRS criteria and signs of shock (hypotension, elevated lactate) implemented in adult clinical settings including the ED, ICU, and general inpatient ward with varying results on processes of care.^16.19 A recent inpatient pediatric study utilized vital sign based screening, but not an EHR based alert, for identification of sepsis, and demonstrated increased screening adherence and protocol utilization.²⁰ Each of these alerts rely on vital sign and/or lab abnormalities, without the addition of physical exam elements and clinician judgment about perfusion adequacy, which are crucial to identify the pediatric patient with severe sepsis amongst the many with SIRS who may rapidly improve with conservative therapy such as antipyretics and oral hydration alone. However, clinical judgment alone seems insufficient: a prior pediatric ED based study indicates that physician judgment of sepsis, without an electronic health alert, identified only 72% of children presenting with severe sepsis.²¹

We have previously studied the potential for EHR-based alerts to improve recognition of severe sepsis in children. We retrospectively applied an electronic alert based on criteria developed by the American Academy of Pediatrics (AAP) Septic Shock Collaborative²² and compared this to a prospectively applied physician identification screen for severe sepsis in children in the ED. The retrospectively applied alert increased sensitivity while reducing specificity compared to the physician identification screen alone.²¹ However, there has not been prospective application and study of a pediatric ED based electronic alert. In this study we prospectively implemented an EHR sepsis alert (ESA) in a pediatric ED and assessed both the test characteristics of the alert and the impact of alert implementation on severe sepsis recognition. We hypothesized that implementation of the ESA would improve recognition of pediatric sepsis in the ED.

Methods

Setting/Participants

The study was conducted in a free-standing academic children’s hospital ED with over 90,000 annual visits. The study period was June 1, 2013–May 31, 2015 with June 1, 2013 – May 31, 2014 providing pre-implementation data and June 1, 2014 – May 31, 2015 providing post implementation data. All patients presenting to the ED during the study period were included. Patients transferred from another institution to our ED were included if they received the initial dose of intravenous antibiotics at our institution.

Ethical Issues

This study was approved by the Institutional Review Board of the Children’s Hospital of Philadelphia, under a waiver of informed consent as a review of existing medical records. The ESA intervention was conducted as a quality improvement initiative conducted by our multi-disciplinary sepsis committee.

Planning the Intervention

The ED sepsis quality improvement (QI) team includes a pediatric emergency medicine attending physician and a pediatric emergency care nurse as co-chairs, a QI advisor, a QI data analyst, pediatric emergency medicine attending and fellow physicians, nurse practitioners, and bedside nurses.

Our center has taken a step-wise approach in attempting to improve recognition of pediatric patients with severe sepsis/septic shock in the ED. We implemented a sepsis protocol and order set in January 2012. At that time, our initial implementation used only bedside physician judgment to determine who required treatment with the sepsis protocol due to the concern that a vital sign only electronic alert would result in a high number of false positives, excessively disrupting workflow and potentially leading to overtreatment. We tracked patient outcomes including organ dysfunction, ICU and hospital length of stay during initial implementation and showed improved outcomes in patients treated with the sepsis pathway/order set in the ED compared to those who were not.¹¹ We developed a candidate electronic alert based on one proposed by the American Academy of Pediatrics (AAP) pediatric septic shock collaborative²² We next determined if the potential impact of this electronic alert in identifying patients that were missed using our initial recognition strategy of bedside clinical judgment, we retrospectively applied and evaluated the electronic alert and found that it had improved sensitivity compared to physician judgment without any electronic screening (92% vs. 72%), although specificity was lower. ²¹ Highest sensitivity was observed when patients were identified using either a positive electronic algorithmic alert OR identified by physician judgment. ²¹ Based on this data, we prospectively implemented an ESA including a physician judgment component as described here.

For this project, a two stage alert was built and implemented within the EHR (Epic Systems^™, Verona, Wisconsin) (Figure 1). The first stage alert is automatically deployed when an age-based elevated heart rate or hypotensive blood pressure is documented in the EHR at any time during the ED visit. Age-based heart rate and blood pressure cutoffs were selected to coincide with our institution’s modified pediatric early warning score (MPEWS).²³ If tachycardia/bradycardia and/or hypotension are identified, an automated alert immediately asks the provider (typically a triage or bedside nurse at our institution): “Is there fever, hypothermia, or concern for infection with this patient?” Nurses determine concern for infection in patients without fever based on clinical judgment. Examples would include: respiratory distress in patients at risk for aspiration pneumonia or increased seizure frequency in patients with an underlying seizure disorder. With an affirmative response, a subsequent screen prompts for additional assessment of peripheral capillary refill time and existing high-risk conditions. High risk conditions (asplenia, bone marrow or solid organ transplant, indwelling central venous catheter, malignancy, significant underlying central nervous system abnormality or technology dependence) were determined a priori and derived from the AAP sepsis alert²² with local modifications to include infants <56 days of age. This data combined with a mental status assessment, which is completed as part of standard triage assessment and automatically incorporated into the logic of the alert, is used to determine if the second stage fires. The mental status assessment is a dropdown menu completed by the triage nurse that is dichotomized by our algorithm into normal or abnormal and included in the ESA algorithm.

Figure 1.

Screen shots of the electronic sepsis alert (ESA) built into electronic health record. Panel A (1^st Stage Alert) automatically deploys in the electronic health record if the patient has tachycardia or hypotension. If assessment of yes to fever, hypothermia, or concern for infection (in Panel A), the screening questions in Panel B (2^nd Stage Alert) automatically deploy. If abnormal finding in capillary refill, high risk condition, or altered mental status (incorporated automatically from earlier in the triage process – not visible here), this is defined as ESA+ and Panel C appears prompting a sepsis huddle. This requires attending or fellow physician bedside evaluation to determine if the sepsis pathway is indicated (documented in Panel D). Panel E is a table describing these stages verbally.

The second stage fires if the patient has hypotension and/or tachycardia, risk of infection or fever, PLUS either a high risk condition OR altered mental status OR altered perfusion as measured by increased peripheral capillary refill time. Patients with positive first and second stage alerts were considered ESA positive. A positive ESA prompted a team assessment/sepsis huddle in which a pediatric emergency medicine attending and/or fellow evaluates the patient at the bedside along with the bedside nurse and determines if the sepsis protocol should be activated. A sepsis huddle was meant to be a brief, focused patient evaluation and discussion that could be completed in less than five minutes. If the sepsis protocol was activated, a templated computerized provider order entry ED sepsis order set was utilized. If the sepsis protocol was determined to not be needed (a “negative huddle”), nurses documented the assessment and the care proceeded as guided by the clinical presentation at hand. Clinicians could also call for a team assessment/sepsis huddle based on clinical identification at any time for any patient, even if the ESA was negative. Patients identified in this way are referred to as clinician-identified sepsis cases.

The alert is programmed to trigger at most once per visit for tachycardia. If the patient has normal vital signs at triage but develops tachycardia later in the visit, the alert would fire at the time of tachycardia, and the above process could occur at any time during the ED visit. Documentation of hypotension would trigger the alert at any time during the ED stay and could alert more than once. At our institution, triage can occur either in a triage booth or in a patient room, depending on room availability. For some critically ill patients, the sepsis protocol was initiated by the treatment team prior to the completion of triage. In these cases, if the ESA fired for those patients (even if it was after sepsis protocol activation), they were counted as ESA+. Details of the current sepsis protocol can be found at: http://www.chop.edu/clinical-pathway/sepsis-emergent-care-clinical-pathway. This protocol is updated annually and thus may differ slightly from the protocol that was in place at the time of this study, although the sepsis screening process is identical.

During the initial implementation of the intervention, the leaders of the ED Sepsis QI team met weekly for 3 months to review cases and identify issues and concerns. After this initial period, meetings continued monthly.

Methods of evaluation

This was a prospective cohort quality improvement study that evaluated both test characteristics of our ESA process, as well as improvements in the proportion of patients treated appropriately with the ED sepsis protocol over time. The primary outcome was treatment for severe sepsis, defined as either treatment on the ED sepsis protocol or the development of severe sepsis or septic shock requiring pediatric intensive care (PICU) care within 24 hours of ED presentation as defined by international consensus guidelines,⁵ and determined by daily screen of all PICU patients as part of routine clinical care and confirmed by medical record review. In addition, subjects with severe sepsis or septic shock who died prior to PICU screen and were not treated on the ED sepsis protocol were identified in monthly ED/PICU sepsis team meetings and by medical record review. We tracked our sepsis recognition process by monitoring the proportion of severe sepsis patients each month who were missed in the ED. A missed case was defined as any patient with severe sepsis who was NOT treated using ED sepsis clinical protocol and order set.

Analysis

We used standard descriptive statistics to describe the study population, using percentages for dichotomous variables and mean/median as appropriate for continuous variables. Sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios were calculated with 95% confidence intervals.

Controlling for confounding medical interventions (CMI)

We conducted additional analyses as described by Paxton et al. ²⁴ to account for potential overtreatment. This attempts to distinguish patients that were treated for severe sepsis but did not develop severe sepsis due to appropriate treatment and recovered (i.e. treatment prevented progression to severe sepsis in a patient that would have otherwise progressed to severe sepsis without treatment) versus over-treatment of patients that would have never progressed to severe sepsis even without treatment. To accomplish this, we modified the definition of severe sepsis to exclude all of the patients treated with the sepsis pathway who did not require PICU care within 24 hours of ED stay. We did sensitivity analysis and determined test characteristics as above using this definition. We also performed a second sensitivity analysis that included only patients requiring vasoactive agents as severe sepsis patients.

We utilized chi square testing and statistical process control charts (p charts) to measure the proportion of missed patients over time. For this measurement, we included outcomes for a 12-month period prior to alert implementation for comparison. Special cause variation was defined as any of the following: one or more points outside control limits, a run of 7 or more consecutive points on one side of the center line, or a trend of 7 consecutive points entirely increasing or decreasing. Special cause variation is a method of identifying a statistically significant change in a process following quality improvement intervention.^25,26 Limits were recalculated when improvements to the process resulted in special cause variation, once the new process has 7 or more data points. Median ED length of stay (LOS) for all ED patients was included as a balancing measure. In addition, we performed aggregate pre/post analysis using chi square testing.

Results

Nature of Improvement Intervention

This study occurred in the context of an existing sepsis QI program in our ED. The team planned the intervention based on prior data suggesting gaps in appropriate sepsis recognition. Prior to ESA implementation, the team presented data to the division about these gaps in care and associated patient outcomes.¹¹ There were ongoing educational updates to the ED division throughout the implementation period regarding successes and failures of ESA implementation, and feedback was sought from ED care providers throughout the implementation process through electronic mail, ED site visits from QI team members, and electronic surveys. A timeline of our ED sepsis quality improvement process, which includes the study period is shown in Supplemental Figure 1.

Determination of electronic alert performance

There were 182,509 ED visits during the study period, with 86,037 pre-ESA-implementation and 96,472 post-implementation. Characteristics of the ED population during the pre and post study period are presented in Table 1. A flow diagram of alert performance is shown in Figure 2. One thousand one hundred twelve patients (1.2%) had positive ESA. Of these, 265 (23.8%) had positive huddles and were placed on the sepsis protocol. Of total severe sepsis patients, there were 16 ESA positive patients who had negative huddles yet went on to have severe sepsis (4.9%). There were 43 (13.1%) cases of clinician-identified sepsis in patients who were ESA negative. There were two patients (0.6%) who were ESA negative, not clinician identified, and went on to have severe sepsis. This yields a total of 326 patients identified with severe sepsis during the study period. Clinical details of missed patients and clinician identified patients are provided in Supplemental Table 1.

Table 1.

Description of the emergency department (ED) population pre and post ESA implementation. Abbreviations: OR=operating room, Obs=observation unit, ICU= intensive care unit, LWBS= left without being seen, AMA=against medical advice

Patient Characteristic	Pre-Alert n(%)	Post-Alert n(%)
Age
<57 days	2,839 (3.3)	3,184 (3.3)
57 days– <1 year	10,669 (12.4)	11,866 (12.3)
1 year– < 4 years	25,036 (29.1)	28,170 (29.2)
4 years– <13 years	31,748 (36.9)	35,694 (37.0)
>= 13 years	15,745 (18.3)	17,558 (18.2)
Gender
Female	40,523 (47.1)	45,631 (47.3)
Disposition from ED
Admitted to floor/OR/Obs	13,851 (16.1)	14,856 (15.4)
Admitted to ICU	1,291 (1.5)	1,351 (1.4)
Died	13 (0.02)	17 (0.02)
Other (LWBS, AMA, transfer)	708 (0.8)	703 (0.7)
Total	86,037	96,472

Figure 2. STARD diagram for the performance of the ESA diagnostic test.

Flow diagram illustrating alert performance in the ED population during the study period. ESA+ indicates patients that had positive both first and second stage alerts which resulted in a sepsis huddle. Of the ED census, there were 15,292 patients with a positive first stage alert, and of these 1112 had positive answers to one of the screening questions (high risk condition, abnormal capillary refill, or abnormal mental status) which resulted in a positive ESA and huddle. Of these, 265 were treated on the sepsis protocol. There were 43 additional patients with clinician identified sepsis (CIS) who were treated with the sepsis protocol despite a negative ESA.

The test characteristics of the ESA are sensitivity 86.2% (95%CI 82.0, 89.5), specificity 99.1% (95% CI 99.0, 99.2), positive predictive value 25.4% (95% CI 22.8, 28.0), and negative predictive value 100% (95% CI 99.9, 100). Sensitivity was augmented to 99.4 (95% CI 97.8, 99.8) when we included clinician identified sepsis patients in the identification strategy. Detailed test characteristics for the performance of both the ESA alone and the ESA in combination with clinician identified sepsis are shown in Table 2a, with numbers of patients in each cell in Table 2b.

Table 2a.

Test characteristics of the electronic sepsis alert (ESA) and ESA with clinician-identified cases. Severe sepsis was defined either as development of severe sepsis defined by consensus definitions requiring PICU care within 24 hours of ED stay or utilization of the ED sepsis pathway. Confounding by medical treatment analysis assumes that all patients treated on the sepsis pathway but not requiring PICU care were false positives (ie were unnecessarily treated). Additional sensitivity analysis included only those patients requiring vasoactive mediations as true positives.

	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value	Likelihood Ratio Positive	Likelihood Ratio Negative
	Test Characteristic (95%CI)
ESA +	86.2 (82.0, 89.5)	99.1 (99.0, 99.2)	25.4 (22.8, 28.0)	100.0 (99.9, 100.0)	100.2 (92.59, 108.6)	0.14 (0.11, 0.18)
ESA + and Clinician Identified Sepsis	99.4 (97.8, 99.8)	99.1 (99.1, 99.2)	28.5 (25.9, 31.2)	100.0 (99.9, 100.0)	115.6 (107.9, 103.7)	0.01 (0.00, 0.02)
Confounding by Medical Intervention ESA + only	77.0 (70.5, 82.7)	99.0 (98.9, 99.1)	13.6 (11.7, 15.8)	100.0 (99.9, 100.0)	77.5(70.2, 85.6)	0.23 (0.2, 0.3)
Confounding by Medical Intervention ESA + and Clinician Identified Sepsis	99.0 (96.4, 99.9)	99.0 (98.9, 99.1)	16.9 (16.0, 17.8)	100.0 (99.9, 100.0)	99.6 (93.3, 106.2)	0.01 (0.00, 0.04)
True Positives Require Vasoactives: ESA+ only	83.9(66.3, 94.6)	98.9 (98.8, 99.0)	2.4(2.0, 2.8)	99.9(99.9, 100.0)	75.1(63.7, 88.6)	0.16 (0.07, 0.36)
True Positives Require Vasoactives: ESA+ and Clinician Identified Sepsis	95.1(83.5, 99.4)	98.9(98.8, 99.0)	3.5(3.2, 3.8)	100.0 (99.0, 100.0)	85.5 (78.0, 93.6)	0.05 (0.01, 0.19)

Table 2b.

A and B assume that all treated are disease +. C and D assume that all treated who did not require PICU care are disease negative.

A
	Severe sepsis +	Severe sepsis −
ESA+	281	831
ESA−	45	95315

B
	Severe sepsis +	Severe sepsis −
ESA+MD+	324	831
ESA+MD−	2	95315

C
	Severe sepsis +	Severe sepsis −
ESA+	151	961
ESA−	45	95315

D
	Severe sepsis +	Severe sepsis −
ESA+MD +	194	961
ESA+MD−	2	95315

To address confounding by medical interventions (the notion that it is difficult to determine if patients treated for severe sepsis truly had severe sepsis upon presentation and improved due to treatment or if did not truly have severe sepsis, improved due to natural course of illness and were treated unnecessarily), we performed additional analyses where we assumed that all patients who were treated on the sepsis protocol but who did not require ICU care to be false positives (130/326 patients met this criteria). This gave us an estimate of the lower bound of specificity in the instance that these patients were treated unnecessarily of 99.0 (95% CI 98.9, 99.1). In addition, we conducted a sensitivity analysis for patients requiring vasoactive agents. Detailed test characteristics for these analyses and patient counts are shown in Table 2 and supplemental Table 2.

Patient Outcomes Over Time: Determination of proportion of missed patients

We observed a decrease in the proportion of patients with severe sepsis who were not treated using ED sepsis clinical protocol and order set during the study period (missed patients) compared to a one year period prior to ESA implementation; from a mean of 17% pre to 4% post implementation. This difference met criteria for special cause variation (Figure 3). Patients were more likely to be correctly identified and treated for severe sepsis post ESA implementation (RR 2.4, (95% CI 1.4, 4.2). As a balancing measure we evaluated overall ED length of stay in all patients and observed no increase in median in the pre vs. post period (203 vs 196 minutes).

Figure 3.

Statistical process control chart demonstrating proportion of missed sepsis cases during the study. Black line is the proportion of missed cases during the study period. The total number of patients each month is indicated below the name of the month in parentheses. A missed case is a patient with severe sepsis in the PICU within 24 hours of their ED visit who was not treated with the ED sepsis protocol. The implementation of the ESA is marked with an arrow. Dashed lines are the upper and lower confidence limits defined as 2 standard deviations above and below the mean. Upper and lower control limits were recalculated each month, and a new mean was calculated when criteria were met for special cause variation. Aggregate proportions of missed cases for pre and post implementation period is indicated in the text box.

Limitations

There are several limitations to this study: First, it included a single pediatric academic center, and we do not know how it will perform in other settings, thus limiting generalizability. Larger scale efforts, including several ongoing multi center quality improvement collaboratives, will help to address this concern in the coming years. Second, the vital sign cut-offs utilized in this study were not empirically derived, and were chosen based on existing institutional practice. It is possible that we would have seen different results with a different set of vital sign cut points. Evaluating risk factors and a predictive model for identification of patients at risk for sepsis is an important future direction. Third, the main analysis defined severe sepsis as either sepsis protocol activation OR admission to the PICU for severe sepsis within 24 hours of ED arrival. We chose this definition because we could not determine which patients treated for suspected sepsis improved as direct results of appropriate treatment, or if they would have improved even without treatment. We conducted two sensitivity analyses to address this point: the first with only patients requiring PICU admission considered as true positives, and the second with only patients requiring vasopressors. Fourth, vital sign elements are completed in full and are “forced” in triage. We cannot rule out the possibility of some missing data in the second part of the alert: i.e. that the alert was not completely filled out by the team. In these cases, patients would be counted as ESA negative (first stage positive, second stage negative). We were able to identify instances where the alert was not properly filled out and the patient had severe sepsis through our process to identify missed patients (These patients are detailed in supplemental Table 1). However, there could have been instances where the alert was incompletely filled out and the patient did not have sepsis. These cases were not specifically reviewed in detail for this manuscript, but will be part of our ongoing quality improvement efforts in the future. Finally, our existing institutional screening process allows us to capture potentially missed patients only in the PICU. Thus, it is possible that patients requiring the neonatal or cardiac ICU would not have been identified as missed patients in this study.

Discussion

We have implemented a process that includes an ESA and bedside clinician assessment to facilitate identification of pediatric patients who developed severe sepsis/septic shock requiring PICU care within 24 hours of their ED stay. The ESA required bedside evaluation of just 1% of the ED census, and had 85% sensitivity and 99% specificity. Sensitivity was optimized to 99% when clinician identification was used to augment sepsis recognition of ESA negative patients. Following implementation of the ESA, we decreased the proportion of missed ED cases from 17% pre intervention to 4% post intervention.

Although the PPV in the study may appear low at 25–28%, this is almost an order of magnitude higher than the 2.5–4% PPV in other published pediatric sepsis screening studies. ^9,21 4 Further, given the cost (both financial and personal) of one missed case of sepsis, a 1:4 “hit rate” may be reasonable to trigger a rapid clinician evaluation, as in our electronic ESA. We agree, however, that this PPV may be insufficient to automatically trigger therapies, such as antibiotics and fluids. Of note, the PPV does decrease in both of our sensitivity analyses, suggesting a lower identification rate when only the sickest patients are considered true positives. As we continue to collect data on alert performance, we hope to refine the alert such that the PPV will improve in future iterations. In addition, future research efforts may help to identify sub populations at low risk of disease progression where initial treatment can be less aggressive

We had three years of sepsis quality improvement interventions in place prior to instituting the electronic alert and yet did not see improvements in sepsis recognition based on these prior interventions. In addition, we did see improvements in sepsis patient identification when applying a candidate electronic alert retrospectively.²¹ We therefore think that it is most likely the alert that led to improved performance. However, we cannot exclude that other actions by the sepsis QI team or other interventions during alert implementation also affected our results.

We performed detailed medical record review of the missed patients to identify themes that may improve future recognition efforts. The main theme that we identified was underlying patient complexity. We have since performed educational interventions to underscore the difficulty of sepsis recognition in this population, particularly in patients with severe developmental delay in whom changes in mental status can be difficult to ascertain. We have encouraged erring on the side of caution in these patients, and have encouraged treating for sepsis in these cases. In addition, we noted that 2 patients did not have their blood pressure checked prior to leaving the ED, and have instituted a rule that requires a full set of vital signs within 30 minutes of ED departure, and clearance by the attending physician before physically leaving the ED.

Interestingly, although our overall sepsis recognition improved following alert implementation, clinician identification remains an important modality for recognition of severe sepsis patients: identifying 43 patients that were ESA negative. This underscores the point that a vital sign based sepsis screen is not sufficient to fully capture all patients with severe sepsis, and that currently, a component of clinician identification remains critically important. There are clear needs for broad based educational initiatives to improve bedside sepsis recognition across any site that cares for children, as well as for bedside tools that can help to standardize the decision process in the huddle. Such a process would also allow us to identify barriers to sepsis recognition in instances where the protocol was inappropriately declined. In addition, it highlights the importance of developing additional novel diagnostic testing in the future to better detect this potentially deadly disease.

In conclusion, we tested an electronic sepsis alert that uses a combination of vital signs, risk factors, and clinician judgment to identify children with severe sepsis in a large academic ED that manages over 90,000 visits per year. This ESA improved recognition of severe sepsis, with a greater proportion of patients with sepsis being treated on the sepsis protocol. Sensitivity analyses for confounding by medical interventions do not suggest that the ESA resulted in overtreatment. Future efforts will focus on evaluating the ability to decrease unnecessary alerts while continuing to improve the sensitivity of the current system.

Abbreviations

AAP

American Academy of Pediatrics

CI

Confidence Interval

ED

Emergency Department

EHR

Electronic Health Record

ESA

Electronic Sepsis Alert

ICU

Intensive Care Unit

LOS

Length of Stay

MPEWS

Modified Pediatric Early Warning Score

PICU

Pediatric Intensive Care Unit

RR

relative risk

SIRS

Systemic Inflammatory Response Syndrome