Identification of Pediatric Sepsis for Epidemiologic Surveillance Using Electronic Clinical Data

Abstract

Objective

A method to identify pediatric sepsis episodes that is not affected by changing diagnosis and claims-based coding practices does not exist. We derived and validated a surveillance algorithm to identify pediatric sepsis using routine clinical data and applied the algorithm to study longitudinal trends in sepsis epidemiology.

Design

Retrospective observational study.

Setting

Single academic children’s hospital.

Patients

All emergency and hospital encounters from January 2011 to January 2019, excluding neonatal intensive care unit and cardiac center.

Exposure

Sepsis episodes identified by a surveillance algorithm using clinical data to identify infection and concurrent organ dysfunction.

Measurements and Main Results

A surveillance algorithm was derived and validated in separate cohorts with suspected sepsis after clinician-adjudication of final sepsis diagnosis. We then applied the surveillance algorithm to determine longitudinal trends in incidence and mortality of pediatric sepsis over 8 years. Among 93,987 hospital encounters and 1065 episodes of suspected sepsis in the derivation period, the surveillance algorithm yielded sensitivity 78% (95% CI 72, 84%), specificity 76% (95% CI 74, 79%), positive predictive value (PPV) 41% (95% CI 36, 46%), and negative predictive value (NPV) 94% (95% CI 92, 96%). In the validation period, the surveillance algorithm yielded sensitivity 84% (95% CI 77, 92%), specificity of 65% (95% CI 59, 70%), PPV 43% (95% CI 35, 50%), and NPV 93% (95% CI 90, 97%). Notably, most “false-positives” were deemed clinically-relevant sepsis cases after manual review. The hospital-wide incidence of sepsis was 0.69% (95% CI 0.67, 0.71%), and the inpatient incidence was 2.8% (95% CI 2.7, 2.9%). Risk-adjusted sepsis incidence, without bias from changing diagnosis or coding practices, increased over time (adjusted incidence rate ratio per year 1.07, 95% CI 1.06, 1.08; p<0.001). Mortality was 6.7% and did not change over time (adjusted odds ratio per year 0.98, 95% CI 0.93, 1.03; p=0.38).

Conclusions

An algorithm using routine clinical data provided an objective, efficient, and reliable method for pediatric sepsis surveillance. An increased sepsis incidence and stable mortality, free from influence of changes in diagnosis or billing practices, were evident.

INTRODUCTION

Sepsis, a life-threatening organ dysfunction caused by a systemic infection (1), is a leading cause of mortality and high health care costs (2, 3). Until recently, sepsis was defined as a systemic inflammatory response syndrome to infection, with severe sepsis indicating organ dysfunction using arbitrary thresholds and septic shock indicating cardiovascular dysfunction (4). In 2016, Sepsis-3 reframed adult sepsis as an infection causing an acute increase of ≥2 points on the Sequential Organ Failure Assessment (SOFA) organ dysfunction score and septic shock as sepsis requiring vasopressor therapy and hyperlactatemia (1). A similar framework for pediatric sepsis has been proposed, but not yet established (5–7).

The practical application of sepsis consensus criteria across clinical, research, quality improvement, and epidemiologic efforts is challenging. For example, because the 2005 International Pediatric Sepsis Consensus Conference (IPSCC) focused on criteria to aid in standardizing observational studies and clinical trials (4), existing consensus criteria fail to include many children diagnosed and treated for sepsis, particularly those with early or mild organ dysfunction (8, 9). In addition, tracking cases of sepsis within and across hospitals using consensus criteria requires laborious chart reviews, with variable interrater reliability (9, 10). Other data sources, particularly large administrative datasets commonly leveraged for research or epidemiologic surveillance, do not contain sufficient clinical data to apply consensus criteria. As a result, reliance on claims data has become a popular substitute. However, numerous studies have shown that claims data are neither reliable nor accurate in identifying episodes of sepsis (9, 11).

The wide availability of electronically recorded clinical data provides an opportunity to apply a surveillance algorithm to identify episodes of sepsis within and across health care systems. Using electronic health record (EHR) data from 409 hospitals, Rhee et al adapted clinical criteria from Sepsis-3 and the SOFA score to estimate the incidence of adult sepsis in the United States (12). Their surveillance approach yielded 69.7% sensitivity and 98.1% specificity and was overall more accurate than claims data. A similar algorithm has not been developed for pediatric sepsis. Therefore, we sought to derive and validate a surveillance algorithm to identify episodes of pediatric sepsis using routine clinical data available within the EHR and then apply the algorithm to study longitudinal trends in sepsis epidemiology.

METHODS

Study Design and Population

We performed a retrospective study using routine clinical data recorded in the EHR at a single academic children’s hospital. Patients treated in the emergency department or admitted to an inpatient ward at the Children’s Hospital of Philadelphia (CHOP) between January 1, 2011 and January 31, 2019 were eligible. To ensure an algorithm inclusive of the entire age spectrum cared for within the scope of pediatrics, we did not apply an upper age limit. We excluded patients in the neonatal intensive care unit and cardiac center (including cardiology ward and cardiac intensive care unit) because these populations were anticipated to have unique physiology that may require a more tailored surveillance approach. However, patients with cardiac conditions treated in other hospital locations were included. This study was approved by the CHOP Institutional Review Board (IRB) with a waiver of informed consent and assent (CHOP IRB #18–015197).

Derivation of the Pediatric Sepsis Surveillance Algorithm

A team of medical informatics experts in the Department of Biomedical and Health Informatics Arcus program at CHOP worked with clinical sepsis experts to develop a surveillance algorithm. Our goal was to include criteria indicative of a sepsis computable phenotype (13). The Arcus program is a strategic initiative of the CHOP Research Institute created to link clinical and research data and offers secure data archives, patient privacy protection, and access to clinical data on >2 million patients. While the IPSCC defined pediatric severe sepsis as 1) ≥2 systemic inflammatory response syndrome (SIRS) criteria, 2) suspected or confirmed invasive infection, and 3) cardiovascular dysfunction, acute respiratory distress syndrome, or ≥2 other organ dysfunctions (4), we focused on the Sepsis-3 framework of infection with acute organ dysfunction without the qualifying term “severe sepsis” or the need to include SIRS criteria. This approach is consistent with calls for a revised definition of pediatric sepsis to align with Sepsis-3 (14).

We derived our surveillance algorithm in a cohort of patients identified as suspected sepsis by our sepsis quality improvement (QI) program via electronic flags for antibiotics, blood culture, and/or fever (eTable 1). As part of this QI work, starting September 1, 2017, all patients flagged for suspected sepsis were also manually adjudicated through chart review by trained clinicians to confirm or reject the presence of severe sepsis or septic shock based on IPSCC criteria (eTable 2). Patients with non-bacterial causes of infection-mediated organ dysfunction (i.e., viral and fungal sepsis) were included as having severe sepsis or septic shock, as were patients in whom infection was suspected and treated by the clinical team in the absence of a definitive source of infection. Patients adjudicated as having severe sepsis or septic shock based on the 2005 IPSCC criteria between September 1, 2017 and June 30, 2018 served as the reference-standard for the derivation cohort.

For our surveillance algorithm, we a priori defined infection as a blood culture (ordered or collected) and sustained administration of new antibiotics, similar to Rhee et al (12). In order to capture patients for whom a blood culture may have been obtained during initial resuscitation at a referring facility, we allowed for transfer to CHOP from a referring facility to substitute for a blood culture. Patients were required to receive at least four consecutive days of antibiotics (though not necessarily the same antibiotics on all days), starting within ±2 calendar days of the blood culture or transfer and at least one dose of a parental antibiotic. The first antibiotic was required to be “new”, defined as that antibiotic not having been administered within the prior seven days. Four days of antibiotics were required in order to exclude patients for whom bacterial infection was suspected but not confirmed. Fewer than four days of antibiotics were allowed for patients who died or were discharged to hospice before four days.

We defined organ dysfunction based on criteria from the pediatric SOFA (pSOFA) score reported by Matics et al (5), with modifications derived from the adult sepsis surveillance criteria (12), another well-validated pediatric organ dysfunction score (Pediatric Logistic Organ Dysfunction Score [PELOD]-2) (15, 16), and three iterative revisions among our local sepsis experts. Sepsis-3 requires an increase in SOFA of ≥2 points, which equates to moderate dysfunction in a single organ system or mild dysfunction in two or more organ systems (1). Similar to Rhee et al (12), we required at least one moderate organ dysfunction within ±2 calendar days of blood culture or transfer, and selected criteria readily available within the EHR that corresponded to pSOFA ≥2 points in each organ system (except renal, see Table 1). Although not in SOFA or pSOFA, we included hyperlactatemia because it is a common manifestation of cardiovascular dysfunction in septic shock, associated with mortality, and demonstrated utility in the adult sepsis surveillance definition (12, 17, 18). We selectively included clinical variables likely to be available in the EHR for patients across a variety of hospitals. When data were not available in the EHR for a criterion, it was assumed to be normal.

Table 1:

Bioinformatics Criteria for Pediatric Sepsis

Sepsis Component	Surveillance Definition
1. Infection	Blood culture (ordered or collected) OR transfer from external health care facility, AND Antibiotics ≥4 days within ±2 days of blood culture/transfer day OR death/hospice before day 4
2. Acute organ dysfunction (any one criteria present within ±2 calendar days of blood culture or transfer):
Cardiovascular	>60 mL/kg isotonic fluid boluses within seven hoursa, OR New, additional, or increased dose of vasoactive medication, OR Blood lactate ≥2.0 mmol/L
Respiratory	New invasive/non-invasive mechanical ventilationb, OR Increase in non-invasive ventilation to >20 hours/dayb,c
Hematologic	Platelet <100,000 cells/μL and ≥50% decline from baseline (baseline required to be ≥100,000 cells/μL)d,e
Kidney	Serum creatinine ≥2x baseline and exceeding threshold for aged,f

^aOnly 0.9% saline or lactated Ringer’s fluid ordered as a bolus were included.

^bMechanical ventilation provided through an existing tracheostomy was excluded from this criteria.

^cNon-invasive ventilation included continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP). Patients using CPAP or BiPAP for ≥20 hours/day prior to blood culture or transfer were excluded from this criteria.

^dBaseline was defined as the average of the three lowest values over the preceding six months. If baseline was not available, then baseline was presumed “normal” and the requirement for “≥50% decline from baseline” was not required.

^eOncology patients, identified based on admission to the hospital’s oncology service or ICD-9/10 oncology codes within the preceding one year, were excluded from the platelet criteria due to a high rate of false-positives outweighing the unlikely scenario of truly having sepsis based on thrombocytopenia alone.

^fSerum creatinine thresholds for age were determined based on upper limit of normal values, at least renal pSOFA ≥1 score, and at least renal PELOD-2 score of 2: <1 month, 1.0 mg/dL; 1 to <12 months, 0.5 mg/dL; 12 to <24 months, 0.5 mg/dL; 2 to <5 years, 0.6 mg/dL; 5 to <12 years, 0.7 mg/dL; 12 to <18 years, 1.0 mg/dL; ≥18 years, 1.2 mg/dL

Each iterative review was focused on optimizing both sensitivity and specificity of our surveillance criteria relative to the IPSCC reference-standard with changes to organ dysfunction criteria based on manual review of the reasons for false-negatives and false-positives. For example, after the second iterative review, we excluded bilirubin because isolated hyperbilirubinemia as the defining organ dysfunction is rare in pediatric sepsis (19, 20), and this criterion led to substantial false-positives from non-septic hepatic dysfunction. After the third iterative review, we added fluid resuscitation >60 mL/kg to capture children with fluid-responsive shock who were otherwise relegated as “false-negatives” (21). We ceased with iterative changes to the surveillance algorithm once it was deemed unlikely that we could further reduce the number of false-positives or false-negatives without including criteria targeted to single patients.

Validation of the Pediatric Sepsis Surveillance Algorithm

Once derived, the surveillance algorithm was tested in a separate validation cohort that included patients adjudicated as having severe sepsis or septic shock based on the 2005 IPSCC criteria between July 1, 2018 and January 31, 2019. We did not a priori specify targets for the performance of the surveillance algorithm in the validation cohort.

Data Collection

Data were collected about demographics, chronic comorbid conditions (CCC) (22), admission to the pediatric intensive care unit (PICU), and mortality. Onset of sepsis was identified as the earliest of either the blood culture order or collection time, time of hospital arrival (for transfers), or first new qualifying antibiotic order. Sepsis was deemed community-acquired if onset occurred within two days of initial hospitalization or hospital-acquired if onset occurred on or after day three. A patient could account for >1 sepsis episode, either within the same or across different hospital encounters. However, to differentiate clinical decompensation from a new episode of sepsis, we required that at least 14 days to elapse from sepsis onset before classifying a new sepsis episode. Adjudicated cases confirmed to have sepsis that were not identified by the surveillance algorithm (false-negatives) and a random subset of the false-positives in the derivation cohort were manually reviewed by two authors to determine the cause of misclassification, with a third-party adjudication for discrepancies.

Statistical Analyses

Analyses were performed using R Version 3.3.3 (R Foundation) and STATA 12.1 (College Station, TX). Descriptive data are summarized as medians (interquartile range, IQR) or percentages. Test characteristics of the surveillance algorithm were calculated for derivation and validation cohorts with 95% confidence intervals (CI) using Wald estimates. We then determined the incidence and mortality of pediatric sepsis episodes across the eight-year study period using the surveillance algorithm. We used multivariable Poisson and logistic regression models to assess for longitudinal changes in incidence and mortality over time, respectively, adjusting for available patient-level characteristics likely to influence changes in sepsis epidemiology over time. Statistical significance was defined as a P <0.05.

RESULTS

Across the entire eight-year study period, derivation period, and validation period, there were 832,550, 93,897, and 64,388 eligible hospital encounters, respectively. The characteristics for the entire study population and derivation and validation periods were similar (eTable 3). The derivation and validation cohorts were derived from episodes flagged as suspected sepsis using the quality improvement electronic criteria within mutually exclusive time periods (eFigure 1).

Within the derivation period, there were 1065 suspected sepsis episodes flagged by the quality improvement electronic criteria, of which 187 were confirmed sepsis episodes after manual adjudication. Using these flagged/adjudicated episodes as the reference-standard, the surveillance algorithm yielded a sensitivity of 78% (95% CI 72, 84%), specificity of 76% (95% CI 74, 79%), positive predictive value (PPV) of 41% (95% CI 36, 46%), and negative predictive value (NPV) of 94% (95% CI 92, 96%) (eFigure 2a).

Within the validation period, there were 361 suspected sepsis episodes flagged by the quality improvement electronic criteria, of which 85 were confirmed sepsis episodes after manual adjudication. Using these flagged/adjudicated episodes as the reference-standard, the surveillance algorithm was validated to have a sensitivity of 84% (95% CI 77, 92%), specificity of 65% (95% CI 59, 70%), PPV of 43% (95% CI 35, 50%), and NPV of 93% (95% CI 90, 97%) (eFigure 2b).

All 42 episodes adjudicated as sepsis in the derivation cohort that were missed by the surveillance algorithm (“false-negatives”) and a random selection of 50 (24%) of the 207 episodes adjudicated as not sepsis that were identified as having sepsis by the surveillance algorithm (“false-positives”) were manually reviewed (Table 2). The most common reasons for surveillance algorithm “false-negatives” were <4 days of antibiotics (usually due to viral etiology), slow fluid resuscitation over >7 hours, and incorrect adjudication (“misclassification error”). The reasons for “false-positives” were largely due to single-organ dysfunction that met surveillance criteria, which were more inclusive than the IPSCC criteria. That some patients were identified with infection and at least one acute organ dysfunction but not adjudicated as having severe sepsis or septic shock by the 2005 IPSCC criteria indicates that the surveillance algorithm did not completely align with the IPSCC definition sepsis.

Table 2:

Reasons for Surveillance Algorithm “False-Negative” and “False-Positive” Determination of Sepsis

Reason	Number (%)
“False-Negatives”	42
<4 days of qualifying antibiotic days	21 (50)
60 mL/kg bolus fluid in >7 hours	5 (12)
Misclassification error (incorrectly adjudicated as sepsis)	4 (10)
Repeat episode within 14 days of prior sepsis episode	3 (7)
Organ dysfunction >2 days after blood culture	3 (7)
60 mL/kg bolus fluid prior to arrival	2 (5)
No blood culture or transfer	1 (2)
Neurologic dysfunction only	1 (2)
New mechanical ventilation in patient with tracheostomy	1 (2)
Thrombocytopenia as single organ dysfunction in oncology patient	1 (2)
“False-Positives”	50a
Single organ dysfunction – lactate ≥2 mmol/L	24 (48)
Single organ dysfunction – respiratory	18 (26)
Single organ dysfunction – ≥60 mL/kg bolus fluid in ≤7 hours	3 (6)
Single organ dysfunction – kidney	2 (4)
Misclassification error (incorrectly adjudicated as not sepsis)	2 (4)
Organ dysfunction unrelated to infection	1 (2)

^aOf the 207 episodes that were adjudicated as not severe sepsis/septic shock, we manually reviewed a random selection of 50 (24%) episodes that were identified as sepsis by the surveillance algorithm.

The hospital-wide incidence of pediatric sepsis episodes, including all 832,550 ED and inpatient visits, across the entire eight-year study period using the final surveillance algorithm was 6.9 episodes per 1000 hospital encounters (0.69%, 95% CI 0.67, 0.71%). Among 207,368 hospital admissions (excluding ED visits that did not result in admission), the incidence of sepsis was 27.8 episodes per 1000 hospital admissions (2.8%, 95% CI 2.7, 2.9%). The incidence of sepsis among all hospital encounters increased over time after controlling for age, sex, race, and number of CCC categories (adjusted incidence rate ratio per year 1.07, 95% CI 1.06, 1.08; P<0.001, Figure 1 and eTable 4). Although lactate measurement was increasingly emphasized during the study period, there was no change in proportion of sepsis episodes identified by the hyperlactatemia criterion over time (eFigure 3). Patient characteristics for all sepsis episodes are shown in Table 3.

Figure 1: Sepsis Incidence and Mortality, 2011–2018.

The incidence of sepsis among all hospital encounters (including all ED visits and hospital admissions, blue line), incidence of sepsis among hospital admissions (orange line), and the in-hospital mortality among sepsis episodes (green line) from 2011 through 2018. Sepsis episodes were identified using the surveillance algorithm. Data are presented as number of episodes per 1000 emergency/hospital encounters for incidence and percentage of episodes for mortality. Error bars indicate 95% confidence intervals.

Table 3:

Patient Characteristics for Sepsis Episodes Identified by Surveillance Algorithm

Variable	Total Study Period
Number of sepsis episodesa	5764
Age, years	6 (1–13)
Age, categories
<1 month	46 (1)
1 to <12 months	756 (13)
12 to <24 months	679 (12)
2 to <5 years	974 (17)
5 to <12 years	1479 (26)
12 to <18 years	1300 (23)
≥18 years	530 (9)
Sex, male	3066 (53)
Race
White	2,546 (44)
Black	1,533 (26)
Asian	274 (5)
Multiple races	150 (3)
Other	793 (14)
Number of CCC present
None	1077 (19)
One	973 (17)
Two	772 (13)
Three or more	2942 (51)
Type of CCC
Cardiovascular	1,371 (24)
Respiratory	1,271 (22)
Neurologic/Neuromuscular	2,029 (35)
Hematologic/Immunodeficiency	1,017 (18)
Oncologic	821 (14)
Renal/Urologic	641 (11)
Gastrointestinal	2,320 (40)
Metabolic	1,255 (22)
Other congenital/Genetic	1,196 (21)
Neonatal	391 (7)
Technology dependent	2,616 (45)
Solid organ transplantation	357 (6)
Admitted to hospital	5754 (>99)
Required PICU admissionb	3876 (67)
Location at sepsis onset
Community-acquired sepsis	4584 (80)
Hospital-acquired sepsis	1180 (20)
PICUc	693 (59)
Oncologyc	123 (10)
General pediatric or subspecialty wardc	364 (31)
Discharge disposition
Remained hospitalizedd	131 (2)
Routine discharge	4780 (83)
Inpatient rehabilitation facility	277 (5)
Transfer to short-term/acute-care facility	191 (3)
Deceased	385 (7)

CCC, chronic comorbid conditions; PICU, pediatric intensive care unit

Data are presented as median (interquartile range) or n (%)

^aPatients could have more than one sepsis episode per hospital encounter

^bDefined as admission to PICU within 72 hours of sepsis onset time

^cHospital location at time of onset of hospital-acquired sepsis

^dPatient did not leave the hospital prior to another sepsis episode during the same encounter

Overall mortality was 6.7% (95% CI 6.1, 7.3%). Mortality did not change over time after accounting for age, sex, race, community- versus hospital-acquired sepsis onset, presence of CCC, or number of organ dysfunctions (aOR per year 0.98, 95% CI 0.93, 1.03; P=0.38, Figure 1 and eTable 5). The distribution and number of organ dysfunctions, with associated mortality, among sepsis episodes identified using the surveillance algorithm are shown in Figure 2.

Figure 2: Organ Dysfunction and Associated Mortality in Sepsis Episodes.

A, Percentage of sepsis episodes identified by the surveillance algorithm with each organ dysfunction and associated in-hospital mortality for episodes that included each organ dysfunction. For example, cardiovascular dysfunction was present in 59% of sepsis episodes, and of those with cardiovascular dysfunction, 10% died prior to hospital discharge. B, Percentage of sepsis episodes identified by the surveillance algorithm by number of organ dysfunctions and associated in-hospital mortality for episodes that included each number of organ dysfunctions. For example, one organ dysfunction was present in 54% of sepsis episodes, and of those with one organ dysfunction, 2% died prior to hospital discharge. Organ dysfunctions included in the number count were >60 mL/kg isotonic fluid boluses within seven hours, new/increased vasoactive infusion, blood lactate ≥2.0 mmol/L, new invasive/non-invasive mechanical ventilation, increase in non-invasive ventilation to >20 hours/day, platelet <100,00 cells/μL with ≥50% decline from baseline, and serum creatinine ≥2x baseline and exceeding threshold for age.

DISCUSSION

We developed a surveillance algorithm to identify episodes of pediatric sepsis using routine clinical data available within the EHR. In both derivation and validation cohorts, the surveillance algorithm achieved comparable sensitivity (pediatric derivation 78%, validation 84% versus adult 69.7%) to a recent electronic surveillance definition for adult sepsis (12). Although our specificity was lower than that reported for adults (pediatric derivation 76%, validation 65% versus adult 98.1%), our reference-standard focused on suspected cases of sepsis rather than randomly selected hospitalizations with a much lower likelihood of including “sepsis-like” cases. Our surveillance algorithm was able to be applied retrospectively to generate eight years of epidemiological data about pediatric sepsis episodes using consistent clinical criteria without having to rely on laborious and expensive manual chart review or claims data that suffer from variability across providers and time.

A reliable method to identify episodes of pediatric sepsis has been elusive. We, and others, have relied on a combination of clinician self-report, medical record review, “home-grown” electronic flags, and billing codes to identify children with sepsis (3, 23–25). None of these methods provides an objective, efficient, and reliable approach, making epidemiologic comparisons across time and location difficult. For example, a review of 94 pediatric sepsis studies found substantial variability in patient characteristics between studies (26). The extent to which this variability reflected differences in local case-mix versus heterogeneity in the criteria used to identify sepsis was unclear. Moreover, there is a known disconnect between clinician diagnosis, fulfillment of consensus criteria, and assignment of billing codes for pediatric sepsis such that none of these methods provides a clear standard (8, 9, 11).

We chose an approach that parallels the Sepsis-3 framework rather than the 2005 IPSCC to ensure relevance to forthcoming pediatric sepsis updates (14) and to be consistent with the adult sepsis surveillance algorithm (12). Nonetheless, the best reference-standard for pediatric sepsis is currently the 2005 IPSCC criteria (4). Key differences are that the IPSCC criteria include a) clinician suspicion of infection irrespective of antibiotics), b) ≥2 SIRS criteria, and c) a complex set of criteria that can fulfill dysfunction among six organ systems. However, the 2005 IPSCC criteria were intended to identify a subset of children with sepsis at high-risk of mortality for clinical trials, while our surveillance algorithm was designed for retrospective surveillance of pediatric sepsis episodes with high reliability and efficiency (27). Because we compared a novel surveillance approach based on Sepsis-3 to an IPSCC-based reference-standard with a different framework, we expected differences in the patients identified with sepsis. Moreover, we applied an assumption that clinically-relevant sepsis can occur in children without meeting the 2005 IPSCC consensus definitions. For example, we deemed it acceptable that children treated with ≥4 days of antibiotics with isolated hyperlactatemia or respiratory dysfunction requiring invasive or non-invasive mechanical ventilation be included as having sepsis for the purposes of epidemiologic surveillance. This approach is consistent with clinical guidelines outlining less stringent criteria to clinically diagnose pediatric sepsis than IPSCC criteria, as well prior research showing that clinicians commonly diagnose and treat sepsis that has not met IPSCC criteria (8, 9, 21).

Ours is not the first attempt at a medical informatics-based approach to identify pediatric sepsis episodes. Matics et al also applied the Sepsis-3 framework to 8711 PICU encounters from a single institution, substituting age-based values into a pSOFA score (5). Notably, their pSOFA score was not developed or validated in children prior to utilization; rather, it simply adopted the same criteria used in adults. However, some features of SOFA may not be useful in children. For example, Glasgow coma score is not reliably scored or consistently recorded in pediatric hospitalizations (28), and hepatic dysfunction is rarely the only organ dysfunction in pediatric sepsis (19, 20). Moreover, Matics et al required hyperlactatemia to define septic shock—similar to Sepsis-3—but lactate is not always measured. Finally, it is not clear how accurately pSOFA would identify sepsis outside of the PICU, such as those with fluid-responsive septic shock. Our efforts expanded work by Matics et al to derive and validate an algorithm applicable to children in emergency, inpatient, and intensive care settings.

Applying our surveillance algorithm to eight years of EHR data revealed a rising incidence of sepsis that is less sensitive to changes in diagnosis and could not be attributable to variation in billing practices. This distinction is critical because, to this point, it has not been possible to disentangle the extent to which the reported rise in pediatric sepsis reflects intensified sepsis recognition, changes to coding practices, or a true increase in disease (3, 23, 24). Notably, the 2.8% incidence of sepsis among admissions was between prior estimates that used different claims-based strategies to identify sepsis (23, 29). Because strategies combining billing codes for infection and organ dysfunction have demonstrated trouble with specificity while sepsis-specific codes lack sensitivity (11, 30, 31), our surveillance approach seeks to achieve overall better accuracy than claims data. Rhee et al demonstrated that a similar EHR-based surveillance approach more accurately identified adult sepsis episodes than claims data (12); a similar comparison is warranted for our pediatric sepsis surveillance algorithm. Finally, the stable mortality we observed over time also contrasts with claims-based data that mortality has trended lower over time (3, 23, 24). Rather, our data indicate that the temporal rise in sepsis incidence translates into a higher absolute number of children dying, a finding with key public health implications.

There are several limitations. First, the surveillance algorithm excluded children not treated with four days of inpatient antibiotics, such as viral sepsis. Given that children with viral sepsis without bacterial co-infection have a low risk of death (32), we felt this omission was acceptable. Second, the applicability of this algorithm to the neonatal intensive care unit and cardiac center requires further study. Third, we did not include hepatic or neurologic dysfunction. Hepatic dysfunction was excluded because a) this criterion identified many children with primary liver disease treated for mild infections and b) most children with sepsis-associated hepatic dysfunction had other qualifying organ dysfunctions. Neurologic dysfunction was excluded because a reliable indicator was not ascertainable from the EHR. Fourth, since our surveillance algorithm was based on Sepsis-3 but compared 2005 IPSCC criteria and there was variability in the number of suspected sepsis episodes flagged by the quality improvement electronic criteria between the derivation and validation cohorts, interpretation of the test characteristics should be done with caution. Thus, an independent validation of our algorithm is needed at other institutions and in comparison to other criteria, such as claims data. Fifth, although the incidence was measured independently from clinician diagnosis and claims data, we cannot exclude the possibility that clinicians utilized organ dysfunction-defining therapies (e.g., vasoactives, mechanical ventilation) more aggressively over time. Six, this algorithm is limited to retrospective data only, and thus is not intended to assist in real-time clinical recognition of children with sepsis. Last, the epidemiologic estimates reflect the experience at a single institution in one geographic and healthcare setting. However, broader application of this algorithm, if validated in a multicenter study, could provide an objective, efficient, and reliable method for epidemiologic surveillance of pediatric sepsis.

CONCLUSION

An algorithm that uses routine clinical data available within the EHR can provide an objective, efficient, and reliable method for pediatric sepsis surveillance across emergency and inpatient hospital settings. Applying this algorithm to eight years of data from a single-center demonstrated an increase in sepsis incidence and stable mortality free from influence of changes in diagnosis or billing practices.

Supplementary Material

Supplemental Data File (.doc, .tif, pdf, etc.)_1

eFigure 1: Patient Flow Diagram

Total number of hospital episodes in the study period from January 1, 2011 through January 31, 2019, including number of hospital episodes included the in derivation and validation cohorts, number of suspected sepsis episodes flagged by the institutional quality improvement (QI) criteria, and the number of suspected sepsis episodes confirmed with or without sepsis after manual adjudication. All 832,550 hospital episodes were used for the epidemiology analyses while a subset of 93,987 hospital episodes were used for the derivation cohort and a separate subset of 64,388 hospital episodes were used for the validation cohort.

Supplemental Data File (.doc, .tif, pdf, etc.)_2

eFigure 2: Test Characteristics using Derivation and Validation Cohorts

The 2×2 tables comparing adjudication of suspected sepsis episodes with categorization of sepsis by the surveillance algorithm in the derivation (A) and validation (B) cohorts. Calculation of the test characteristics for the surveillance algorithm relative to the adjudicated reference-standard is shown.

Supplemental Data File (.doc, .tif, pdf, etc.)_3

eFigure 3: Proportion of Sepsis Episodes that Included Hyperlactatemia Over Time

The proportion of sepsis episodes identified by the surveillance algorithm in which criteria for organ dysfunction included hyperlactatemia (defined as blood lactate ≥2.0 mmol/L) with or without other organ dysfunctions (blue line) or included hyperlactatemia as the only organ dysfunction (orange line). There was no association between calendar year and the proportion of sepsis episodes that included hyperlactatemia as organ dysfunction (odds ratio 1.01, 95% confidence interval 0.99, 1.04; p=0.27). Error bars indicate 95% confidence intervals.