Using objective clinical data to track progress on preventing and treating sepsis: CDC’s new ‘Adult Sepsis Event’ surveillance strategy

Sepsis is a leading cause of death and suffering, afflicting 1.7 million adults annually in the USA and contributing to over 250 000 deaths.¹ The high burden of sepsis has catalysed numerous performance improvement and policy initiatives, including mandatory sepsis protocols in a growing number of US states, the Centers for Medicare and Medicaid Services’ (CMS) ‘SEP-1’ measure, and WHO’s resolution declaring sepsis a global health priority.² Hospitals around the world are dedicating considerable resources to improving sepsis recognition and compliance with treatment bundles.

However, accurately measuring the impact of sepsis quality improvement efforts is challenging. The core problem is that diagnosing sepsis involves considerable subjectivity.³ Sepsis is a heterogeneous syndrome without a pathological gold standard. It is defined as infection leading to organ dysfunction,⁴ but it is often unclear whether a patient is infected and whether organ dysfunction is due to infection or other factors such as dehydration, medications, cancer, or inflammatory diseases.

The challenge of sepsis measurement is compounded by the rapidly changing clinical and regulatory milieu. Clinicians are being encouraged to screen for sepsis more aggressively and both clinicians and administrators are being encouraged to code for sepsis and organ dysfunction more diligently to maximise reimbursement. The net effect is that many patients that previously were never labelled with sepsis are now being counted.^3,5–8 These additional cases tend to have milder disease and lower mortality rates. If a hospital uses administrative codes to track sepsis, there is a high probability they will see higher sepsis case counts and lower sepsis mortality rates that are due at least in part to more ascertainment. ^{9, 10}

Some hospitals prospectively track cases that trigger sepsis screens or perform retrospective audits of hospitalisations flagged by administrative data.^11,12 New York state’s ‘Rory’s Regulations’ allow hospitals to use either method, while the CMS SEP-1 process measure requires retrospective case abstractions. These methods are more rigorous than using diagnosis codes alone but are still susceptible to suggesting misleading rises in sepsis rates and declines in mortality because of increasing sepsis awareness and recognition. Ascertainment bias is particularly problematic when all patients with suspected infection are counted, not just those in whom infection is confirmed or even probable. As an example, one healthcare system reported a reduction in sepsis mortality by more than 50% after implementing a quality improvement initiative, but tellingly twice as many sepsis cases were included in their case counts ¹³

These challenges in surveillance are a major impediment for hospital sepsis quality improvement initiatives. The uncertainty in knowing whether decreases in sepsis mortality are due to improvements in care versus simply expanding the pool of patients being classified as septic limits the ability of hospitals to know if their sepsis initiatives are bearing fruit. This in turn limits hospitals’ capacity to make informed decisions about how to optimise their sepsis programme.

Hospitals and policy-makers need a more objective measure to track progress and inform further improvements. Recognising this, the U.S. Centers for Disease Control and Prevention (CDC) created an ‘Adult Sepsis Event’ toolkit to help hospitals measure sepsis rates and outcomes using standardised clinical criteria that can be automated using routine electronic health record (EHR) data.¹⁴ This new tool offers hospitals a more reliable means to track sepsis since the criteria are clear and reproducible, have concrete rules for associating organ dysfunction with infection and focus on patients who receive sustained courses of antibiotics rather than simply suspected to have infection. Automated Adult Sepsis Event surveillance is more objective than tracking diagnosis codes and sepsis screens, easier to maintain than registries or retrospective audits, and able to track all hospitalised patients with sepsis rather than the small sample required by CMS.

The Adult Sepsis Event definition was initially developed as part of a multicentre study conducted to estimate the US national burden of sepsis using EHR data.¹ In accordance with the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3),⁴ the Adult Sepsis Event definition identifies patients with presumed serious infection and concurrent organ dysfunction. Presumed serious infection is defined as a blood culture order and administration of at least 4 days of antibiotics (fewer if death or discharge occurs before 4 days). Acute organ dysfunction is defined as the initiation of vasopressors or mechanical ventilation, elevated lactate, or changes in creatinine, total bilirubin or platelets relative to clearly delineated baseline values. A comparison of Adult Sepsis Event, Sepsis-3 and CMS SEP-1 criteria is shown in table 1, while a comparison with other sepsis surveillance methods is shown in table 2.

Table 1.

Adult Sepsis Event criteria and comparison with Sepsis-3 and SEP-1 criteria

	Sepsis-3 (EHR version)*	CMS ‘SEP-1’ Severe Sepsis	CDC Adult Sepsis Event
Infection	1. Body fluid culture obtained†, AND 2. Administration of ≥2 doses of antibiotics If a culture was obtained first, an antibiotic must be given within 72 hours. If antibiotic administration occurred first, a culture must be obtained within 24 hours	Documentation of suspected or confirmed infection ≥2 SIRS criteria (within 6 hours): temperature >38.3°C or <36.0°C; heart rate >90 beats/minute; respirations >20/minute; white blood cell count >12 or <4×109 cells/L or >10% bands	Blood culture obtained, AND New antibiotic starting within ±2 days of blood culture day, followed by a total of ≥4 consecutive antibiotic days (or until <=1 day prior to death, discharge to hospice or acute care hospital, or transition to comfort care)‡
Organ dysfunction	Increase in SOFA score by ≥2 points from up to 48 hours before to up to 24 hours after onset of infection (body fluid culture or antibiotic administration, whichever occurred first):	One of more of the following, within 6 hours of documentation of infection and SIRS (excluding organ dysfunction documented as chronic):	One or more of the following criteria within ±2 calendar days of blood culture day:
Cardiovascular	1. Mean arterial pressure <70 mm Hg 2. DA ≥5 μg/kg/min or any dobutamine 3. DA >5 or EPI ≤0.1 or NE ≤0.1 μg/kg/min 4. DA > 15 or EPI >0.1 or NE >0.1 μg/kg/min	Systolic blood pressure <90 mm Hg (or decrease by >40 mm Hg) or mean arterial blood pressure <65 mm Hg	Vasopressor initiation (NE, EPI, DA, phenylephrine or vasopressin)§
Pulmonary	1. PaO2/FiO2, 300–399 2. PaO2/FiO2 200–299 3. PaO2/FiO2 100–199 and ventilated 4. PaO2/FiO2 <100 and ventilated	Respiratory distress reguiring initiation of mechanical ventilation or non-invasive positive pressure ventilation	Mechanical ventilation initiation (>1 calendar day reguired between ventilation episodes)
Renal	1. Creatinine 1.2–1.9 mg/dL 2. Creatinine 2.0–3.4 mg/dL 3. Creatinine 3.5–4.9 mg/dL or <500 cc urine/day 4. Creatinine >5.0 mg/dL or <200 cc urine/day	Creatinine >2.0 mg/dL or urine output <0.5 mL/kg/hour for 2 hours	↑2× Creatinine or ↓≥50% of eGFR relative to baseline¶ (excluding patients with end-stage renal disease)
Hepatic	1. Bilirubin 1.2–1.9 mg/dL 2. Bilirubin 2.0–5.9 mg/dL 3. Bilirubin 6.0–11.9 mg/dL 4. Bilirubin >12.0 mg/dL	Bilirubin >2 mg/dL	Bilirubin ≥2.0 mg/dL and ↑2× from baseline¶
Coagulation	1. Platelets 100–149×109 cells/L 2. Platelets 50—99× 109 cells/L 3. Platelets 20–49×109 cells/L 4. Platelets <20×109 cells/L	Platelet count <100×109 cells/L, or INR >1.5 or aPTT >60 s	Platelet count <100×109 cells/L and ↓≥50% decline from baseline¶ (baseline must be ≥100)
Neurological (or perfusion	1. Glasgow Coma Scale score 13–14 2. Glasgow Coma Scale score 10–12 3. Glasgow Coma Scale score 6–9 4. Glasgow Coma Scale score <6	Lactate >2.0 mmol/L	Lactate ≥2.0 mmol/L**

^*Sepsis-3 EHR criteria are as defined by Seymour et al in the Sepsis-3 clinical criteria derivation analyses.¹⁵

^†Body fluid culture sites include abdomen, bronchoalveolar lavage, blood, bone, cerebrospinal fluid, catheters/devices, pleural space, skin/soft tissue, stool and urinary tract. Culture types include bacterial, fungal, viral, parasitic and Clostridium difficile toxin testing.

^‡A new antibiotic refers to one not given in the prior two calendar days. At least one antibiotic given within ±2 days of the blood culture day must be parenteral.

^§To count as a new vasopressor, that specific vasopressor cannot have been administered in the prior calendar day. Vasopressors given by bolus injection or in the operating room (or other procedural areas where anaesthesia is administered) are excluded.

^¶For presumed infection present-on-admission (blood culture day or first antibiotic occurring on hospital day 1 or 2), baseline laboratory values are defined as the best values during hospitalisation. For hospital-onset infection (blood culture day and first antibiotic occurring on hospital day ≥3), baseline laboratory values are defined as the best values during the ±2-day period surrounding the day of the blood culture draw.

^**Serum lactate captures more patients with sepsis but carries risk of ascertainment bias if rates of lactate testing in a hospital are increasing over time. CDC, Centers for Disease Control and Prevention; CMS, Centers for Medicare and Medicaid Services; DA, dopamine; EHR, electronic health record; EPI, epinephrine; FiO₂, fraction of inspired oxygen; INR, international normalised ratio; NE, norepinephrine; PaO₂ arterial partial pressure of oxygen; SIRS, systemic inflammatory response syndrome; SOFA, Seguential Organ Failure Assessment; aPTT, activated partial thromboplastin time; eGFR, estimated glomerular filtration rate.

Table 2.

Comparison of Adult Sepsis Events with other sepsis surveillance methods

Method	Advantages	Disadvantages
Administrative claims data	Convenient and easy to use Cases explicitly coded for sepsis usually have high positive predictive value relative to medical record reviews	Susceptible to ascertainment bias from decreasing thresholds to diagnose and code for sepsis over time Explicit sepsis codes have low sensitivity relative to medical record reviews Limited ability to distinguish hospital onset vs community-onset disease Limited comparability between hospitals due to variability in diagnosis and coding practices
Retrospective audits Example: CMS SEP-1 measure	More accurate and rigorous than using administrative data alone Allows for precise identification of time zero and processes of care relative to time zero	Resource-intensive; generally only applied to a fraction of a hospital’s sepsis population Still susceptible to ascertainment bias, particularly if cases are selected for audit based on administrative data Variability among abstractors and hospitals for determining presence of sepsis and time zero may limit interpretation of trends in a hospital and comparisons with other hospitals
Prospective registries Example: New York hospitals tracking patients triggering sepsis screens/protocols	More accurate and rigorous than using administrative data alone Allows for precise identification of time zero and processes of care relative to time zero Can capture majority of a hospital’s sepsis population if hospitals have structured electronic health record–based sepsis screens and protocols	Resource-intensive Still susceptible to ascertainment bias due to decreasing threshold to diagnose sepsis and organ dysfunction as well as more sensitive sepsis screens and protocols Sepsis screens or protocols can miss true sepsis cases and flag some patients who do not actually have infection or sepsis Limited comparability between hospitals due to differing sepsis screening criteria
Adult Sepsis Event surveillance using electronic health record data	More objective than other methods due to standardised criteria for presumed infection and organ dysfunction Focus on patients treated for infection with sustained courses of antibiotics (≥4 antibiotic days) mitigates ascertainment bias from increasingly aggressive sepsis screening over time capturing more patients with ‘suspected infection’ Can be automated and applied to an entire hospital's population Can identify day of sepsis onset May allow for more comparability across hospitals due to standardisation	Reguires electronic health record data with microbiology data, chemistry laboratories and medications to apply Implementation reguires information technology expertise and resources Misclassification possible (but higher sensitivity and comparable positive predictive value compared with administrative data) Still dependent on clinician interventions (blood cultures, antibiotics, vasopressors, mechanical ventilation, laboratory tests)

In contrast to Sepsis-3 clinical criteria,¹⁵ the Adult Sepsis Event definition was designed for retrospective surveillance rather than early detection and real-time clinical decision-making. The definition requires ≥4 days of antibiotics to minimise false positives from patients who get just a few days of empiric antibiotics that are stopped once infection is no longer suspected. In addition, it simplifies and adapts the Sequential Organ Failure Assessment (SOFA) score by eliminating components that are inconsistently measured, documented and stored in EHRs, such as mental status, vasopressor doses, urine output, blood gas results (which are often unreliably labelled as arterial vs venous) and FiO₂, at the time blood gases were drawn. This makes it more objective and suitable for widespread automated use among hospitals with different EHR systems.

Medical record reviews suggest that Adult Sepsis Events has superior sensitivity and similar positive predictive value compared with sepsis diagnosis codes for identifying patients meeting Sepsis-3 criteria.¹ It also generates more credible estimates of sepsis trends since the sensitivity of EHR-based clinical criteria is more stable over time than diagnosis codes.¹⁰ Applying this definition to EHR data from over 400 US hospitals showed that sepsis incidence and short-term mortality rates were stable from 2009 to 2014, in contrast to administrative codes from the same hospitals which gave the impression of steep increases in incidence and decreases in mortality.¹

We believe that implementing Adult Sepsis Event surveillance can yield important benefits for hospitals, clinicians and patients. First, it will enable hospitals to better measure the impact of their sepsis prevention and treatment initiatives since it identifies patients in a consistent manner independent of diagnoses assigned by physicians, hospital coders or quality auditors. This will provide more confidence to hospitals that mortality improvements are due to better care rather than better detection and labelling of less ill cases. Conversely, outcomes that remain static can help identify the need for further improvements in processes and care.

Second, Adult Sepsis Events can provide hospitals with a more complete picture of sepsis incidence, trends and outcomes than the CMS ‘SEP-1’ measure since the latter is a process measure and only includes a maximum of 20 cases per month. Applying Adult Sepsis Events electronically allows hospitals to track all sepsis cases. In addition, it sidesteps some of the variability inherent in SEP-1’s requirement to abstract time zero.¹⁶ This is a time-consuming and subjective task that requires reviewers to pore over multiple notes, vital signs, laboratory data, medications and nursing flowsheets to find the first documentation of suspected infection and judge whether organ dysfunction is present, new and related to infection.¹⁰ Adult Sepsis Events avoids these challenges by focusing instead on the day of sepsis onset as marked by concurrent blood culture draws, antibiotic starts and organ dysfunction on the same or adjacent calendar days.

Third, Adult Sepsis Events give more insight into when sepsis occurred during hospitalisation compared with discharge diagnosis codes, which are limited in their granularity to present-on-admission flags. Hospitals can use this to facilitate retrospective audits of processes of care. It can also give hospitals a better window into community-onset versus hospital-onset sepsis since present-on-admission codes are often inaccurate and variably applied by hospitals.¹⁷ Monitoring community-onset sepsis incidence rates could inform pre-hospital or ambulatory prevention initiatives to identify and treat infections before they progress to full-blown sepsis. Tracking hospital-onset sepsis incidence may help inform hospitals’ efforts to prevent nosocomial infections.

Fourth, objective sepsis surveillance is a first step towards more meaningful comparisons between hospitals. Currently, it is difficult to discern whether observed variations in sepsis outcomes reflect differences in case mix, quality of care, or diagnosis and coding practices. In New York state, where sepsis criteria and case-finding methods are not standardised, early analyses have demonstrated considerable variability in the percentage of hospitals’ sepsis cases found in administrative data that are reported to the state.¹⁸ Hospital networks may find Adult Sepsis Event surveillance helpful for internal quality improvement efforts by identifying hospitals with unexpectedly low or high sepsis mortality in order to elucidate the organisational or care factors that separate those hospitals. Credible comparisons will also require rigorous risk adjustment to account for differences in patient populations. This is not currently included in the Adult Sepsis Events toolkit, but CDC has flagged this as a priority for future development. Risk adjustment for Adult Sepsis Events could use methodology similar to New York state’s model¹⁹ but focus on data routinely found in EHRs.

Fifth, tracking Adult Sepsis Events provides another avenue to monitor and improve antibiotic use given that it relies on ≥4 antibiotic days to identify presumed infection. Audits could assess the choice, timeliness and duration of antibiotics. Integrating antimicrobial stewardship efforts with hospitals’ sepsis campaigns may help balance the need for early and aggressive antibiotics to begin treatment with aggressive de-escalation once a pathogen is identified or infection no longer seems likely.

To be sure, the Adult Sepsis Event definition has important limitations. Like all sepsis surveillance methods, the Adult Sepsis Event definition incorporates clinical judgements such as the decision to draw a blood culture, prescribe antibiotics or start vasopressors. It thus remains susceptible to variation and changes in practice patterns between clinicians and over time. It may also flag some patients who were not truly infected (despite receiving ≥4 days of antibiotics) or who had organ dysfunction unrelated to infection. This is also true of all other sepsis surveillance methods, but Adult Sepsis Events at least associate presumed infection with organ dysfunction in a consistent and reproducible manner. Validations of the accuracy of Adult Sepsis Events have thus far have been encouraging,¹ but further work is needed to assure consistency and generalisability across diverse settings. Adult Sepsis Events may miss some patients who meet Sepsis-3 criteria since the organ dysfunction criteria for the two definitions are similar but not identical. Patients missed by Adult Sepsis Events, however, tend to have milder cases of organ dysfunction that trigger SOFA organ dysfunction criteria but not Adult Sepsis Event criteria, such as hypoxaemia without need for mechanical ventilation or hypotension without need for vasopressors.¹ The Adult Sepsis Event also risks sowing confusion by adding another definition to a crowded field. However, we believe different sepsis definitions are appropriate for different purposes, particularly real-time guidance for clinical care (eg, Sepsis-3) versus retrospective surveillance for rigorous case-counting and outcome monitoring.²⁰ Additional research is needed to better understand the overlap and differences between Adult Sepsis Events and other sepsis criteria, and to adapt the definition for use in paediatric populations.

Lastly, from an operational standpoint, automating Adult Sepsis Event surveillance requires informatics expertise, resources and a reasonably sophisticated EHR. However, a common data specification and analytic code is available from CDC,^1,14 and the effort to maintain surveillance should be low after the initial programming and validation stage. If electronic implementation cannot be done, hospitals could alternatively choose to manually abstract Adult Sepsis Event cases. This would be time-consuming but more feasible if performed on a random sample of patients, and might still be simpler and more objective than applying SEP-1 or Sepsis-3/SOFA criteria.

In conclusion, the Adult Sepsis Event definition is an important step towards making sepsis surveillance more objective and providing more reliable information on sepsis incidence and outcomes to clinicians, quality officers, policy-makers and public health officials. This in turn carries the promise of helping drive further innovations and improvements in the prevention, detection and management of this devastating illness.