Surveillance Strategies for Tracking Sepsis Incidence and Outcomes

Abstract

Sepsis is a leading cause of death and the target of intense efforts to improve recognition, management and outcomes. Accurate sepsis surveillance is essential to properly interpret the impact of quality improvement initiatives, make meaningful comparisons across hospitals and geographic regions, and guide future research and resource investments. However, reliably tracking sepsis incidence and outcomes is challenging because it is a heterogeneous clinical syndrome without a pathologic gold standard, allowing for subjectivity and broad discretion in assigning diagnoses. Most epidemiologic studies of sepsis to date have used hospital discharge codes and have suggested dramatic increases in sepsis incidence and decreases in mortality over time. However, diagnosis and coding practices vary widely between hospitals and are changing over time, complicating the interpretation of absolute rates and trends. Other surveillance approaches include death records, prospective clinical registries, retrospective medical record reviews, and analyses of the usual care arms of randomized controlled trials. Each of these strategies, however, has substantial limitations. Recently, the U.S. Centers for Disease Control and Prevention released the “Adult Sepsis Event” definition that uses objective clinical indicators of infection and organ dysfunction to detect sepsis using routine clinical data from hospitals’ electronic health record systems. Emerging data suggest that electronic health record-based clinical surveillance such as Adult Sepsis Events is accurate, can be applied uniformly across diverse hospitals, and generates more credible estimates of sepsis trends than administrative data. In this review, we discuss the advantages and limitations of different sepsis surveillance strategies and consider future directions.

Sepsis is a leading cause of mortality and morbidity worldwide [1]. In the U.S. alone, sepsis afflicts an estimated 1.7 million adults each year and may contribute to up to 270,000 deaths [2]. Over the past two decades, sepsis has been the target of intense efforts to increase awareness and improve the timeliness and quality of care [3–5]. Accurately tracking trends in sepsis incidence and outcomes is essential in order to properly interpret the impact of quality improvement initiatives and guide future research and resource investments. A consistent surveillance method is also necessary to make meaningful comparisons across hospitals and geographic regions. This is particularly important today in light of the rise of state and national sepsis quality measures and a parallel desire to benchmark hospitals on their quality of care [6–8].

Reliably tracking sepsis incidence and outcomes, however, is difficult [9, 10]. The core challenge is that sepsis, currently defined as a dysregulated host response to infection leading to organ dysfunction [11], is a heterogeneous clinical syndrome without a pathologic gold standard [12]. Determining whether infection is present or whether organ dysfunction is due to infection versus other conditions (such as dehydration, volume overload, or medications) is often subjective [13]. Subjectivity complicates the interpretation of sepsis trends because it is unclear whether increases are due to a true rise in sepsis cases or to more patients being diagnosed, labeled, and coded for sepsis. This is a particular concern given ongoing vigorous efforts to promote sepsis awareness, early recognition, protocolized care, and optimal billing.

In this review, we summarize the most common approaches to sepsis surveillance, discuss the strengths and limitations of each strategy, and outline potential future directions.

Administrative Data

“Administrative data” refers to data generated in the process of patient care or hospital billing, such as hospital discharge diagnosis codes, physician claims, or hospital insurance claims. Hospital discharge diagnosis codes are commonly used for large-scale epidemiologic studies of sepsis, as well as internal tracking of sepsis outcomes within hospitals [14–18]. Sepsis diagnosis codes are also used for quality analyses and reporting, including screening cases to review for adherence to the Centers for Medicare and Medicaid Services’ Severe Sepsis/Septic Shock Early Management Bundle (“SEP-1”). Studies utilizing administrative data have consistently found that the incidence of sepsis has been sharply increasing over the past 25 years at a rate of 8 to 13% per year, and that in-hospital mortality rates are decreasing by as much as 2% per year [14–17, 19, 20].

There are clear advantages to administrative data (Table 1). The data are generated for every hospitalization as part of routine billing operations and therefore do not require dedicating additional resources specifically for sepsis monitoring. They are easy and inexpensive to obtain, relatively straightforward to analyze, and include useful corollary information such as comorbid conditions that can be used for risk-adjustment or other descriptive analyses. Furthermore, publicly available datasets such as the National Inpatient Sample allow estimation of nationwide rates that cannot be easily determined using other data sources.

Table 1.

Summary of Surveillance Strategies for Sepsis

	Advantages	Disadvantages	Reported Epidemiologic Trends
Administrative Claims Data	Convenient, easy to use, inexpensive Large, national data sets available High positive predictive value relative to medical record reviews	Susceptible to ascertainment bias from decreasing thresholds to diagnose and code for sepsis over time Limited ability to compare across institutions and regions due to variability in diagnosis and coding practices Limited ability to distinguish between community- and hospital-onset sepsis	Increasing sepsis incidence (8–13% per year in US) [14, 15, 19] Decreasing in-hospital mortality (1.4–2% per year) [14, 17]S
Death Records	Convenient, easy to use, inexpensive Captures deaths occurring outside of the hospital Gives insight into causal role of sepsis in mortality	Cannot be used to study incidence or other outcomes (unless paired with other data sources) Coding often inaccurate and variable Susceptible to ascertainment bias	Variable reports of trends in sepsis deaths (31% increase in US from 1999–2014, vs 30% global decrease from 1990–2017); number of deaths lower versus hospital discharge codes [1, 38]
Prospective Clinical Registries	More consistent inclusion criteria Allows identification of time zero and processes of care	Resource-intensive Susceptible to ascertainment bias due to decreasing threshold to diagnosis sepsis and organ dysfunction Limited ability to compare across institutions due to differing sepsis screening criteria	Decreasing in-hospital mortality (e.g., 35.0% to 18.4% from 2000–2012 in Australian/New Zealand ICUs) [44]
Retrospective Case Reviews	More accurate and rigorous than administrative data alone Allows identification of time zero and processes of care	Resource-intensive, infeasible to apply to large populations Potentially subjective
Sepsis Randomized Controlled Trials	More consistent inclusion criteria Other detailed clinical data often available to adjust for severity of illness No additional cost beyond expense of RCT	Relatively small numbers of patients Can only be used to study sepsis outcomes, not incidence Patients may not be reflective of general sepsis population Still susceptible to ascertainment bias as increased sepsis awareness leads to enrollment of less severely ill patients	Decreasing 28-day mortality, (e.g. 0.42% annual decrease from 2002–2016, but trend non-significant when controlled for severity of illness) [51]
Objective/Clinical criteria from EHR CDC “Adult Sepsis Event”	Objective and less susceptible to ascertainment bias (does not depend on diagnosis and coding practices) May be automated and applied to entire hospitals’ populations Distinguishes between community- and hospital-onset sepsis More objective hospital comparisons	Dependent on availability of electronic health records Requires information technology expertise and resources; only feasible in high-income countries High thresholds for organ dysfunction criteria may miss patients with lower severity of illness Susceptible to bias due to changes in practice patterns (e.g., lactate testing)	Sepsis incidence unchanged from 2009–2014 (without lactate criterion) or mild increase (with lactate criterion) [2] Mortality unchanged (without lactate criterion) or mild decrease (with lactate criterion) [2]

However, administrative data also have significant limitations. First, there is no standardized set of codes to identify sepsis. Administrative definitions are categorized as “explicit” when they rely on sepsis-specific ICD-9-CM or ICD-10-CM codes, such as sepsis, severe sepsis, septic shock, or septicemia. In contrast, “implicit” sepsis definitions, such as the Angus method, detect sepsis by seeking patients with diagnosis codes for both infection and organ dysfunction [18]. Some definitions, such as the Dombrovskiy method, require both explicit sepsis codes and organ dysfunction codes [14]. The precise set of codes used has a significant impact on observed sepsis incidence and outcomes. In 2013, Gaieski et al compared national estimates from four administrative definitions of sepsis and found that the annual incidence varied as much as 3.5-fold while mortality rates varied two-fold [19]. All administrative definitions, however, suggested parallel increases in incidence and decreases in mortality over time.

The accuracy of administrative definitions relative to medical record reviews also varies substantially [21]. Most studies indicate that explicit definitions have high specificity but low sensitivity, and tend to be preferentially applied to sicker patients, particularly critically ill patients with bacteremia, shock, and lactic acidosis [22–24]. Implicit definitions have better sensitivity but lower positive predictive value, likely because the post-hoc assignment of codes for infection and organ dysfunction does not imply a causal or even temporal relationship between the corresponding infection and organ dysfunction diagnoses [2, 23, 24]. As such, sepsis cohorts defined by explicit versus implicit definitions generally include fewer patients and have higher mortality rates.

Second, the use of administrative data for tracking sepsis trends is highly susceptible to bias from changing diagnosis and coding practices over time [8]. Many high-profile initiatives are increasing sepsis awareness amongst physicians, administrators, and consumers including the Surviving Sepsis Campaign, the CMS SEP-1 measure, ongoing educational initiatives, and the introduction of sepsis screening and management protocols in most hospitals. In addition, hospitals are encouraging providers to document sepsis more diligently since it allows hospitals to use more remunerative diagnosis-related groupings [25–27].

Clinicians’ increasing predilection to recognize, diagnose, and code for sepsis complicates the interpretation of studies based on administrative data that suggest that sepsis incidence is rising and case-fatality rates are falling [28]. While some of this may reflect true changes in sepsis epidemiology – more cases because the population is aging and becoming more medically complex, and fewer deaths due to earlier sepsis recognition and better management – several lines of evidence suggest ascertainment bias is also contributing. Jafarzadeh et al, for example, found that patients with similar characteristics and risk factors had a higher probability of being labeled with sepsis with each successive year from 2008–2012 [29]. Moreover, several studies using electronic health record data from single healthcare systems as well as nationally representative cohorts have demonstrated that trends in sepsis incidence and outcomes derived from clinical data are much more stable compared to administrative data [2, 30–32]. There is also evidence that thresholds to diagnose and code for organ dysfunctions such as acute kidney injury or acute respiratory failure are decreasing, complicating trends derived using implicit sepsis administrative definitions [33].

Lastly, variability in diagnosing and coding for sepsis may confound attempts to compare sepsis incidence and outcomes across hospitals and geographic regions, even when using data from the same time period. Rhee et al, for example, found that the sensitivity of sepsis codes relative to consistent clinical criteria drawn from electronic health records varied widely between institutions [34]. Hospital mortality rankings also changed substantially depending upon whether an administrative or clinical definition of sepsis was used, a finding also demonstrated by Walkey et al in a separate cohort of 308 hospitals [35]. While administrative data have been used to highlight institutional and geographic differences in sepsis mortality [36, 37], these potential sources of variability suggest great caution is warranted when trying to interpret these comparisons.

Death Records

Death records have also been used to track the incidence and characteristics of sepsis deaths over time, as well as to make inferences about geographical variation. Some studies have required explicit documentation of sepsis as a cause of death [38] while others have defined sepsis-associated deaths by documentation of any infection on the death certificate [39, 40]. Most analyses using death records have shown that annual counts of sepsis-related deaths are increasing over time. For example, one study reported that US sepsis-related deaths increased by 31% from 1999 to 2014 [38]. However, a recent study linking death records to hospitalization administrative data and vital statistics from 195 locations around the world estimated that global age-standardized sepsis incidence decreased by 37% from 1990 to 2017 while mortality decreased by 53% [1].

The advantages of this approach mirror those of other administrative data sources—the data are publicly available in large databases and easy to analyze. Data are easily grouped by geographic location, include demographic information, and may be combined with US Census Bureau and county-level community characteristic data [39] to facilitate assessments of geographic or social trends. They also capture deaths outside the hospital, which allows for a more comprehensive view of sepsis mortality. This is important since U.S. national data indicate that only 37% of all deaths (from any cause) occur in the hospital [41]. Capturing post-discharge mortality is particularly important in low-income countries where much of the burden of sepsis accrues outside the hospital; even in the US, an increasing number of patients with sepsis receive end-of-life care in hospice [2, 30]. In addition, death certificates give insight into the causal role of sepsis in a patient’s death, whereas administrative data and basic registries only indicate the presence of sepsis (without indicating if it directly contributed to death).

However, there are many limitations to the use of death records for sepsis surveillance. They shed light only on sepsis-related mortality rather than incidence or case-fatality rates, unless linked with hospital administrative data. Furthermore, to our knowledge, the accuracy of death certificates for identifying sepsis-related deaths has not been studied; however, accuracy of death certificates in general populations and for other common conditions has been shown to be poor [42, 43]. In particular, physicians differ (and often err) in their designations of the “immediate” and “underlying cause” leading to death. Furthermore, requiring explicit sepsis codes is likely to have low sensitivity, while using any infection code assumes that sepsis is present in all patients who die with infection, an assumption that may or may not be true. Using death records to identify sepsis-associated deaths is therefore susceptible to both false negatives and false positives. In general, death certificates appear to underestimate sepsis mortality. Epstein et al, for example, found that U.S. national estimates of sepsis mortality from hospital discharge codes were 15% to 140% higher than from death certificates [38]. Variability in local practices for completing death certificates may also confound comparisons across different regions. Finally, tracking sepsis mortality trends using death records is susceptible to the same ascertainment bias as hospital discharge diagnosis codes, namely that increasing sepsis awareness and decreasing thresholds to label infectious or non-specific syndromes as sepsis may be leading to more deaths being attributed to sepsis.

Prospective Clinical Registries

Tracking sepsis incidence and outcomes can also be accomplished using prospective clinical registries [44, 45]. This may take place at individual hospitals, or in large, multicenter registries such as the Surviving Sepsis Campaign (SSC) database, the Australian and New Zealand Intensive Care Society (ANZICS) Adult Patient Database, or the New York State Department of Health (NYSDOH) database of mandatorily reported sepsis cases. Cases are often identified via institutional screening protocols for sepsis. They may also be included via admission to certain hospital units, such as in the ANZICS database which is limited to ICU patients. Studies using data from prospective clinical registries have shown increasing sepsis incidence and decreasing sepsis mortality over time similar in magnitude to those seen with administrative data [44, 45].

At first glance, the use of registry data for sepsis surveillance seems advantageous compared to administrative data, as registry inclusion is done prospectively using consistent screening strategies and inclusion criteria. In addition, changes in billing strategies and discharge coding practices are less likely to affect registry data. However, heterogeneity and subjectivity in inclusion criteria results in significant risk of bias in registry data as well. Some registries allow for retrospective inclusion of additional cases via use of administrative codes or removal of cases based on review [6, 45]. Registries limited to ICU patients may also be biased by changing thresholds to admit patients to ICUs over time [46, 47]. This is a function of both changes in the availability of ICU beds and changing perceptions of what kinds of conditions merit ICU care; this may be particularly pertinent to sepsis since some hospitals’ protocols encourage ICU admission for all patients with sepsis. In addition, the SSC, NYSDOH, and other quality improvement registries are tied directly to aggressive screening protocols which are designed to increase sepsis recognition, increasing case counts and the likelihood of ascertainment bias. Different case definitions used in various registries may also preclude attempts to compare sepsis incidence and outcomes. Furthermore, the creation and maintenance of registries is expensive and time consuming, and data may be difficult for outside researchers to access.

Retrospective Case Reviews

Retrospective case reviews of medical records, usually performed by physicians, quality officers, or other trained clinicians, are generally the gold standard for sepsis identification. This method involves the careful review of all available clinical, laboratory, radiological, microbiological, and pathological information to apply consensus definitions of sepsis. Retrospective reviews are also performed as part of the SEP-1 measure, which requires hospitals to perform reviews of a sample of sepsis-coded hospitalizations monthly, confirm that CMS severe sepsis criteria are met, and abstract adherence rates to sepsis bundles. As above, clinical registries such as the NYSDOH registry may include cases gathered through retrospective review as well.

The primary advantage of chart review is that it involves more rigorous application of sepsis criteria, particularly compared to administrative definitions. It also enables determination of “time zero”, the time at which sepsis criteria were met and thus the starting point for measuring adherence with recommended processes of care.

The major limitation, however, is that chart reviews are highly resource intensive and often subjective [48] and therefore not a practical method for routine, ongoing sepsis surveillance across large numbers of institutions. Studies using case reviews as gold standards for sepsis diagnosis typically identify only a small random subset of cases for audit. In other uses, such as for SEP-1 or the NYSDOH registry, the cases are screened for review via administrative billing codes or prospective clinical screening. Therefore, studies of sepsis outcomes derived from retrospective review should be evaluated with attention to the method of case selection used, as they may still be susceptible to ascertainment bias. In addition, retrospective case reviews can be subjective and variable, even across highly trained clinicians. Several research studies, for example, have demonstrated acceptable yet imperfect levels of agreement between expert physicians applying consensus sepsis definitions [23, 30, 31]. Abstracting sepsis time zero for quality measures like SEP-1 may be even more variable given the complexity of the criteria [48].

Sepsis Randomized Controlled Trials

Several studies have analyzed the usual care arms of randomized controlled trials (RCTs) for sepsis to provide insight regarding overall mortality [49] and trends over time [50, 51]. Two large meta-analyses of RCT data both found declining sepsis mortality rates over a combined period of 1993–2016 [50, 51].

The primary benefits of using RCT data are that sepsis cases are more rigorously selected than in administrative cohorts, and should be free from confounding related to changing coding and billing practices. In addition, published data are readily available at no additional cost beyond the large expense of conducting the RCTs.

However, this approach is limited by relatively small numbers and can only be used to examine sepsis outcomes (generally mortality) rather than incidence. In addition, while case selection may be rigorous, study inclusion criteria vary widely between RCTs, resulting in heterogeneous populations that may or may not be reflective of the overall population of sepsis patients. Mortality rates may also depend upon institutional and geographic factors from where trials were conducted. Furthermore, the use of RCT data is also subject to the bias of increasing sepsis awareness and lower thresholds to diagnose sepsis, leading to the inclusion of patients with lower severity of illness over time. This point is supported by the meta-analysis of 44 sepsis RCTs by Luhr et al, which found decreasing crude 28-day mortality rates in the usual care arms between 2002–2016, but noted that the change was no longer significant after controlling for severity of illness [51].

Objective Clinical Criteria from Electronic Health Records

The widespread adoption of electronic health records (EHR) now allows for the possibility of population-level sepsis surveillance using objective clinical data rather than administrative data or registries. EHR-based sepsis definitions generally identify patients with clinical evidence of suspected infection (e.g., antibiotic administrations and concurrent sampling of clinical cultures) and concurrent organ dysfunction (e.g., vasopressors, respiratory support, or abnormal laboratory values) using information routinely stored in electronic records as structured data. Several early variations of EHR-based criteria have been published, which informed the development of the Adult Sepsis Event definition discussed in detail below [30, 31]. The Sepsis-3 task force used a similar approach in testing various clinical criteria for sepsis using electronic health record data, including the Sequential Organ Failure Assessment (SOFA) Score [52, 53]. However, their analyses were focused on estimating the prognostic validity of sepsis criteria rather than objectively measuring sepsis incidence and outcomes.

CDC “Adult Sepsis Event”

Recognizing the need of hospitals and policy-makers for a more objective measure to track sepsis incidence and outcomes, in 2018 the U.S. Centers for Disease Control and Prevention (CDC) created the “Adult Sepsis Event” (ASE) toolkit, which includes a validated definition that can be implemented using EHR data and supporting materials to help hospitals to create and maintain a robust, EHR-based sepsis surveillance operation [54]. The ASE definition was based on the Sepsis-3 framework of suspected infection with concurrent organ dysfunction, but optimized for simplicity and reproducibility across institutions. An Adult Sepsis Event is defined as 1) presumed serious infection, signified by drawing blood cultures and at least 4 consecutive days of antibiotics (or up until 1 day prior to death, discharge to hospice, transfer to another acute care hospital, or transition to comfort measures) starting within two calendar days of when blood cultures were drawn, plus 2) evidence of concurrent organ dysfunction, signified by any of six binary indicators of cardiovascular, pulmonary, renal, hepatic, coagulation, or perfusion dysfunction (Table 2). These organ dysfunction criteria parallel the organ systems included in the SOFA score, but are simplified in order to avoid potential variability and error that might arise from requiring calculations of the full SOFA score. In particular, neurologic dysfunction was excluded due to concerns that Glasgow Coma Scale scores are subjective and inconsistently measured across units and institutions [55, 56]. The ASE also eliminated other SOFA components that may be inconsistently measured, documented, and stored in EHRs, such as maximum daily vasopressor doses, urine output, blood gas results (which are often difficult to distinguish arterial versus venous), and fraction of inspired oxygen.

Table 2.

Centers for Disease Control and Prevention Adult Sepsis Event Criteria

Component	Detailed Criteria
Presumed Serious Infection	Blood culture obtained (regardless of result), and ≥4 “Qualifying Antibiotic Days” (QAD)a – starting within +/−2 days of blood culture day
Acute Organ Dysfunction (any one of the following within +/−2 days of blood culture day)	Vasopressor initiationb (norepinephrine, dopamine, epinephrine, phenylephrine, or vasopressin) Mechanical ventilation initiationb Doubling in serum creatinine or decrease by ≥50% of estimated glomerular filtration rate relative to baselinec (excluding patients with ICD-9 code for end-stage kidney disease, 585.6) Total bilirubin ≥ 2.0 mg/dL and doubling from baselinec Platelet count <100 cells/μL and ≥ 50% decline from baselinec (baseline must be ≥100 cells/μL) Serum lactate ≥ 2.0 mmol/Ld

Adapted from the U.S. Centers for Disease Control and Prevention Hospital Toolkit for Adult Sepsis Surveillance (www.cdc.gov/sepsis).

Sepsis = presumed serious infection + ≥1 acute organ dysfunction

^aQADs start with the first “new” antibiotic (not given in the prior 2 calendar days) within the +/−2 day period surrounding the day of the blood culture draw. Subsequent QADs can be different antibiotics as long as the first dose of each is “new.” Days between administration of the same antibiotic count as QADs as long as the gap is not >1 day. At least one of the first 4 QADs must include an intravenous antibiotic. If death or discharge to another acute care hospital or hospice occurs prior to 4 days, QADs are required each day until ≤ 1 day prior to death or discharge.

^bVasopressors and mechanical ventilation are considered to be “initiated” during the +/−2 day period surrounding the day of the blood culture draw if there were no vasopressors or mechanical ventilation administered on the prior calendar day.

^cFor presumed infection present-on-admission (blood culture day or first QAD occurring on hospital day 1 or 2), baseline lab values are defined as the best values during hospitalization. For hospital-onset infection (blood culture day and first QAD occurring on hospital day ≥ 3), baseline lab values are defined as the best values during the +/−2 day period surrounding the day of the blood culture draw.

^dSerum lactate captures additional cases of sepsis but may create ascertainment bias if lactate testing rates are increasing over time.

The ASE definition was developed by a CDC Prevention Epicenters-funded consortium that estimated the U.S. national burden of sepsis by applying these criteria to electronic health record data from a large and diverse set of hospitals. The definition was validated using over 500 medical record reviews and found to have superior sensitivity (70%) and similar positive predictive value (70%) compared to explicit sepsis diagnosis codes for identifying cases meeting Sepsis-3 criteria [2]. The surveillance definition also had comparable sensitivity and superior positive predictive value compared to implicit sepsis codes. Notably, the positive predictive value of ASE was higher (88%) when sepsis was defined as clinically suspected infection (rather than infection confirmed retrospectively by reviewers) with organ dysfunction. The investigators also examined trends between 2009 and 2014 and found no significant change in the incidence of sepsis nor the combined outcome of death or discharge to hospice when the lactate criterion was omitted (an a priori decision taken to account for to the known rise in lactate testing rates over time), in contrast to trends derived from administrative data from the same hospitals. When including lactate, sepsis incidence increased and death or discharge to hospice decreased, though both more modestly than with administrative data. These findings support the notion that surveillance via EHR-based techniques is less susceptible to confounding by clinical and policy trends in sepsis.

In addition to more credible assessment of sepsis trends, using Adult Sepsis Events offers several potential benefits for stakeholders. First, it is easier to apply EHR-based surveillance across hospitals’ entire populations compared to registries or retrospective case reviews. Indeed, if adopted widely across the U.S. healthcare system, as CDC recommends, it could enable accurate counting in near real-time of national sepsis incidence and outcomes. Second, ASE provides insight regarding the day of sepsis onset and allows easy distinction between community- and hospital-acquired sepsis compared to hospital discharge codes, which are limited to present-on-admission flags that may have variable accuracy [57, 58]. Third, widespread adoption of EHRs and simplification of organ dysfunction criteria to conform with data widely available in all EHRs make this approach promising for more meaningful comparisons of incidence and mortality rates between hospitals. Notably, differences in practice patterns such as blood culture and lactate orders may introduce confounding to inter-hospital comparisons using the ASE, and this potential application therefore requires further study. However, the increasing utilization of bundled care for sepsis is likely to reduce practice variability across hospitals. In addition, comparisons of hospitals’ sepsis outcomes will require rigorous risk adjustment to accommodate differences in patients’ demographics, comorbidities, and severity of illness.

Limitations of EHR-based clinical surveillance include the upfront costs of setting up data extraction routines, mapping data elements, and applying and validating algorithms, all of which require information technology expertise and resources. Once implemented, however, the marginal cost of ongoing surveillance should be low. In addition, this surveillance method can clearly only be applied in high-income countries; low and middle-income countries, where sepsis incidence and mortality are considerably higher, must currently still rely on administrative data and death records [1].

Furthermore, EHR-based surveillance is not completely free from bias because clinical practice patterns are changing over time. Clinicians may be becoming more prone to order blood cultures, check lactates, start antibiotics, and initiate vasopressors in response to sepsis education and quality improvement initiatives. Including lactate levels may be particularly prone to ascertainment bias because rates of lactate orders are increasing dramatically over time [59]. The ASE’s requirement of four or more qualifying antibiotic days is meant to increase specificity and help mitigate ascertainment bias related to the decreasing threshold to recognize and treat for sepsis by excluding those who received only short courses of antibiotics. However, some patients may be continued on empiric antibiotics for greater than four days even in the absence of true serious infection. An additional limitation is that ASE’s simplified organ dysfunction criteria may miss patients who meet Sepsis-3 criteria, such as those who have hypoxemia that does not require mechanical ventilation, or those with hypotension that does not require vasopressors. Patients meeting Sepsis-3 criteria that are missed by ASE, however, tend to have less severe disease and lower mortality rates compared to included patients [60]. Finally, like many of the methods discussed, EHR-based surveillance must be done retrospectively, as EHR-based definitions typically include requirements for a minimum number of antibiotic days which are only apparent in retrospect. EHR-based definitions that only require a single day of antibiotics may be more amenable to more real-time surveillance but this likely comes at some cost in specificity.

Future Directions of EHR-based Clinical Surveillance

EHR-based surveillance holds great promise as a method for comparing hospital performance in sepsis care and outcomes. However, these comparisons still need to be rigorously risk adjusted. Preliminary studies using the ASE definition suggest that EHR-based clinical risk-adjustment models accurately predict mortality and outperform models based on administrative data alone [61]. However, further work is needed in this area, in particular to assess the potential impact of different risk-adjustment methods on comparisons of hospitals’ sepsis mortality rankings.

In addition, the ASE was designed for and validated in the adult population, but pediatric sepsis is also a major public health concern with similar need for accurate tracking of incidence and mortality. The CDC convened a separate pediatric working group in 2018 which proposed a “Pediatric Sepsis Event” (PSE) case definition that parallels the ASE by incorporating blood culture and antimicrobial orders to signify confirmed or suspected infection and modified pediatric SOFA criteria for organ dysfunction [62]. Working group members noted the need for further refinement and validation of this definition with several planned sensitivity analyses. Eventually, a large, multicenter, national study is needed to estimate the burden of pediatric sepsis in the US.

While purposefully designed for simplicity and applicability across diverse EHRs, the ASE definition may also be useful as a foundation upon which additional functionality could be built. For example, inclusion of additional factors such as comorbid conditions may decrease misclassification of sepsis-associated organ dysfunction that is due to other factors such as chronic medical conditions, and therefore improve specificity [63]. Furthermore, it may be possible to automate identification of “time zero”, or the time from which process measures such as first antibiotics are measured, by utilizing first hypotension or elevated lactate within the ASE window period. This could significantly lessen the burden of SEP-1 case reviews by shifting away from the time-consuming and subjective work of manual abstraction.

Finally, some hospitals may elect to define suspected infection more broadly than ASE and identify organ dysfunction using the SOFA score rather than ASE’s criteria in order to more directly match Sepsis-3 criteria. Recently, Valik et al published the first validation of an EHR-based sepsis screening algorithm using Sepsis-3 criteria in a Swedish academic medical center, and found excellent sensitivity and specificity when compared to trained clinician medical record reviews[64]. However, the greater complexity of the SOFA score may introduce room for variability in electronic implementation that might limit inter-facility comparisons. Further research is needed to understand the generalizability of this approach and its applicability across diverse hospitals.

Summary and Conclusions

Most epidemiologic studies of sepsis have relied on administrative claims data, but this method is limited by ongoing changes in how clinicians screen, test, diagnose, and bill for sepsis. Death records, prospective registries, and analyses of sepsis randomized trials have also been used, but these remain subject to ascertainment bias and are limited by heterogeneous inclusion criteria. Retrospective medical record reviews by trained physicians are rigorous but are infeasible for population-level surveillance. Sepsis surveillance utilizing clinical data from electronic health records, such as CDC’s Adult Sepsis Event definition, is objective, applicable across large populations, and may offer improved clinical test characteristics compared to administrative data. While further work is needed to validate and refine this approach across diverse settings, electronic health record-based clinical surveillance is a promising new tool that may provide more reliable information on sepsis incidence and outcomes and thereby help drive further innovations and improvements in sepsis prevention, detection, and management.