Improving Sepsis Outcomes in the Era of Pay-for-Performance and Electronic Quality Measures: A Joint IDSA/ACEP/PIDS/SHEA/SHM/SIDP Position Paper

Abstract

The Centers for Medicare & Medicaid Services (CMS) introduced the Severe Sepsis/Septic Shock Management Bundle (SEP-1) as a pay-for-reporting measure in 2015 and is now planning to make it a pay-for-performance measure by incorporating it into the Hospital Value-Based Purchasing Program. This joint IDSA/ACEP/PIDS/SHEA/SHM/SIPD position paper highlights concerns with this change. Multiple studies indicate that SEP-1 implementation was associated with increased broad-spectrum antibiotic use, lactate measurements, and aggressive fluid resuscitation for patients with suspected sepsis but not with decreased mortality rates. Increased focus on SEP-1 risks further diverting attention and resources from more effective measures and comprehensive sepsis care. We recommend retiring SEP-1 rather than using it in a payment model and shifting instead to new sepsis metrics that focus on patient outcomes. CMS is developing a community-onset sepsis 30-day mortality electronic clinical quality measure (eCQM) that is an important step in this direction. The eCQM preliminarily identifies sepsis using systemic inflammatory response syndrome (SIRS) criteria, antibiotic administrations or diagnosis codes for infection or sepsis, and clinical indicators of acute organ dysfunction. We support the eCQM but recommend removing SIRS criteria and diagnosis codes to streamline implementation, decrease variability between hospitals, maintain vigilance for patients with sepsis but without SIRS, and avoid promoting antibiotic use in uninfected patients with SIRS. We further advocate for CMS to harmonize the eCQM with the Centers for Disease Control and Prevention’s (CDC) Adult Sepsis Event surveillance metric to promote unity in federal measures, decrease reporting burden for hospitals, and facilitate shared prevention initiatives. These steps will result in a more robust measure that will encourage hospitals to pay more attention to the full breadth of sepsis care, stimulate new innovations in diagnosis and treatment, and ultimately bring us closer to our shared goal of improving outcomes for patients.

This multidisciplinary position paper describes why the Centers for Medicare & Medicaid Services (CMS) should retire the Severe Sepsis/Septic Shock Management Bundle (SEP-1) measure rather than make it pay-for-performance and provides recommendations to improve CMS’s planned electronic sepsis mortality measure.

Sepsis is a major public health problem. More than 1.7 million adults receive hospital care for sepsis in the United States each year, with over 250 000 deaths and $40 billion in Medicare expenditures [1, 2]. The burden of sepsis has appropriately spurred clinicians, hospitals, policy makers, and patient advocates to focus on improving sepsis care and outcomes.

The Centers for Medicare & Medicaid Services (CMS) Severe Sepsis/Septic Shock Management Bundle (SEP-1) is the most prominent national effort to improve sepsis care [3]. SEP-1 was implemented in 2015 as a pay-for-reporting measure (Box 1). Bundle compliance is “all-or-nothing,” and hospital SEP-1 compliance rates are publicly available. CMS is now proposing to make SEP-1 a pay-for-performance measure by incorporating it into the Hospital Value-Based Purchasing Program beginning in fiscal year 2026, raising the stakes associated with compliance [4]. Concomitantly, CMS is developing an electronic clinical quality measure (eCQM) to benchmark hospitals’ risk-adjusted sepsis mortality rates. Draft specifications for the Community-Onset Sepsis 30-day Mortality eCQM were released in June 2022 [5].

Box 1. The CMS Severe Sepsis/Septic Shock Management Bundle (SEP-1).

Severe Sepsis Bundle:

• 1. Measure lactate level within 3 h

• 2. Blood cultures (prior to antibiotics) within 3 h

• 3. Broad spectrum antibiotics within 3 h

• 4. Remeasure lactate if initial lactate elevated (>2.0 mmol/L) within 6 h

Septic Shock Bundle:

• 5. 30 cc/kg crystalloid bolus (normal saline or lactated ringers) within 3 h of hypotension, initial lactate ≥4.0 mmol/L, or clinician documentation of septic shock

• 6. Vasopressors to target mean arterial pressure ≥65 mmHg within 6 h if there is persistent hypotension after ≥30 cc/kg crystalloid bolus

• 7. Document repeat volume status and tissue perfusion assessment within 6 h:

  • Repeat focused exam: vital signs, cardiopulmonary, capillary refill, pulse and skin findings, OR

  • 2 of the following: Measure central venous pressure, central venous oxygen saturation, bedside cardiovascular ultrasound, or passive leg raise or fluid challenge

The SEP-1 measure is “all-or-nothing”: failure in any 1 bundle component means overall failure; no partial credit is given. Some bundle elements can be excluded if appropriate contraindications are explicitly documented in the medical record, eg, administering <30 cc/kg of crystalloid fluids due to concern for congestive heart failure and fluid overload.

SEP-1 brought welcome attention to sepsis. Nonetheless, there is considerable controversy regarding the strength of evidence supporting its bundle elements, whether bundle compliance improves outcomes, and whether there are unintended consequences that offset its potential benefits [6–11]. In 2020, a consortium of professional societies led by the Infectious Diseases Society of America (IDSA) published a position paper outlining concerns with SEP-1 and recommending several revisions, the most important of which was removing severe sepsis from the measure and focusing solely on septic shock [6]. In part 1 of the current position paper, these societies provide an updated analysis and now recommend retiring SEP-1 based on recent studies that document its real-world impact in adults, and the risk of exacerbating unintended consequences by shifting to pay-for-performance. In part 2, we outline our support for CMS's plan to adopt an electronic outcomes-based sepsis measure while offering suggestions to improve its reliability, efficiency, and credibility.

METHODS

This position paper was created by members of a task force initially assembled by IDSA in 2018 and expanded in 2020 to include representation from ACEP, PIDS, SHEA, SHM, and SIDP. The group developed public comments in response to the SEP-1 re-endorsement by the National Quality Forum in 2021, CMS's announcement in 2022 of their plan to transition SEP-1 to pay-for-performance, and CMS's release of the draft specifications of eCQM in 2022. The group aggregated, updated, and refined all public comments and added additional insights to create this document. The position paper was then shared with society boards in March 2023 for endorsement.

PART 1: REASONS TO RETIRE SEP-1 RATHER THAN MAKE IT A PAY-FOR-PERFORMANCE MEASURE

Real-world Evidence Indicates That SEP-1 Has Not Improved Patient Outcomes

Several time-series analyses using detailed clinical data from hundreds of hospitals elucidate the real-world impact of SEP-1 on patient outcomes (Table 1) [12–15]. Rhee et al analyzed 117 150 patients admitted to 114 academic and community hospitals with suspected sepsis between 2013 and 2017 and found that SEP-1 implementation in October 2015 was associated with an immediate increase in lactate testing but no improvement in the combined outcome of hospital death or discharge to hospice [12]. These findings persisted in several sensitivity analyses including one limited to patients with suspected septic shock. Barbash et al evaluated 54 225 patients with suspected sepsis admitted via emergency departments to 11 hospitals affiliated with the University of Pittsburgh Medical Center between 2013 and 2017 and found that SEP-1 was associated with a 50% increase in lactate measurements and a 30% increase in 30 cc/kg intravenous fluid infusions within 3 hours but no change in hospital mortality or discharge to home [13]. Anderson et al analyzed all adults with or without sepsis (n = 701 055) admitted to 26 hospitals between 2014 and 2016 and found that all-cause mortality per 1000 patients decreased by 39% during the study period [14]. However, mortality rates decreased by 5% each month during the year prior to SEP-1 implementation and then declined 2% each month during the year following SEP-1 implementation, suggesting SEP-1 implementation was associated with a blunting of a pre-existing decreasing mortality trend. Furthermore, in a subgroup analysis of patients with suspected sepsis, there was no change in mortality associated with SEP-1 implementation; rather, there were increases in 30-day readmissions, infection relapses, and acute kidney injury.

Table 1.

Major Multicenter Time Series Analyses Assessing the Impact of SEP-1

Study	Setting and Study Period	Study Population	Major Findings
Rhee et al [12]	114 hospitals within the Cerner HealthFacts dataset, October 2013–December 2017	117 510 adults admitted with suspected sepsis, defined as (1) blood culture drawn, (2) ≥2 systemic inflammatory response syndrome criteria, and (3) acute organ dysfunction within 24 h of hospital arrival	Immediate increase in lactate measurements within 24 h after SEP-1 implementation (55.1% to 76.7%, OR 1.34, 95% CI 1.04–1.74) Increases in empiric anti-MRSA antibiotics during study period (19.8% in Q4-2013 to 26.3% Q4-2017) as well as anti-Pseudomonal beta-lactam antibiotics (27.7% to 40.5%); trends occurred independent of SEP-1 implementation No change in short-term mortality rates (death or discharge to hospice): 20.4% vs 20.3% in post vs pre SEP-1 period
Barbash et al [13]	11 hospitals in the University of Pittsburgh Medical Center Health System, January 2013–December 2017	54 225 adults admitted from the ED with suspected sepsis, defined as (1) suspected infection (collection of a blood, urine, respiratory, or other body fluid culture) and (2) organ dysfunction (≥2 SOFA score points) within 6 h of ED arrival	50% increase in lactate measurements within 3 h after SEP-1 implementation (70.2% observed rate vs 46.5% expected rate based on pre-SEP-1 trends) 10% increase in broad-spectrum antibiotic use (49.8% observed vs 45.1% expected) 30% increase in 30 cc/kg intravenous fluid boluses (13.2% observed vs 9.9% expected) No change in trends for in-hospital mortality (absolute change: 0.1% [95% CI −.9%–1.1%]) or discharge to home
Anderson et al [14]	26 hospitals in 7 states, October 2014–October 2016	701 055 adults admitted for ≥24 h (with or without infection/sepsis); subgroup analysis among 31 013 patients with suspected sepsis, defined as ≥1 blood culture collected and subsequent receipt of broad-spectrum antibiotics for ≥48–72 h (conducted in 10 hospitals reporting microbiology data)	10% increase in antibiotic utilization in post-SEP-1 vs pre-SEP-1 periods (605 d of therapy/1000 patient days vs 546) and 24.4% increase in mean monthly days of therapy per 1000 patient-days over the study period 5% monthly decline in all-cause mortality during SEP-1 preparation period, followed by 19% increase during transition to SEP-1 implementation and 2% monthly decline during SEP-1 implementation (change in monthly mortality slope risk ratio between SEP-1 preparation vs implementation periods: 1.04, 95% CI 1.01–1.07) Among patients with suspected sepsis: no change in mortality rates but increase in 30-d readmissions, infection relapses within 30 d, and acute kidney injury
Pakyz et al [15]	111 hospitals participating in Vizient, October 2014–June 2017	7.3 million hospitalized adults; subgroup analysis among 293 665 patients with sepsis discharge diagnosis codes	Immediate 2.3% increase in broad-spectrum antibiotic use (P = .038) after SEP-1 implementation and 0.4% monthly increase in trend (P = .027) amongst all hospitalized patients Significant level increase in use of all antibiotic categories following SEP-1 implementation for patients with sepsis

Abbreviations: CI, confidence interval; ED, emergency department; MRSA, methicillin-resistant Staphylococcus aureus; OR, odds ratio; SEP-1, Severe Sepsis/Septic Shock Management Bundle; SOFA, Sequential Organ Failure Assessment.

Importantly, these 3 studies used slightly different definitions for suspected sepsis (detailed in Table 1), but all used objective clinical criteria (rather than diagnosis codes, which tend to be applied variably and only in patients ultimately confirmed to have sepsis) and all had similar findings. Convergent findings in different populations and data sets using a range of definitions support the conclusion that SEP-1 has not reduced sepsis mortality.

The SEP-1 Requirement to Administer Antibiotics to All Patients With Possible Sepsis Within 3 Hours Has Encouraged Unnecessary Antibiotic Use

The SEP-1 requirement to give antibiotics within 3 hours of sepsis onset pressures clinicians to act very quickly in all settings in which sepsis may be present, regardless of illness severity, and even when considerable uncertainty about the presence of sepsis exists. The signs and symptoms of sepsis are neither sensitive nor specific. Many common non-infectious conditions can mimic the clinical presentation of sepsis (eg, cancer, heart failure, arrythmias, adverse drug effects, toxidromes, drug withdrawal, thromboembolic disease, endocrine emergencies). Approximately one third of patients treated with antibacterial agents for possible sepsis are later found to have viral infections or non-infectious conditions [16, 17]. It is difficult to reliably differentiate between these conditions within the 3-hours permitted by SEP-1 before broad-spectrum antibiotics have to be given. This allows for the possibility that the pressure created by SEP-1 has increased premature and unnecessary antibiotic prescribing.

The time-series analyses assessing the impact of SEP-1 implementation described in the previous section also provide data on its effect on antibiotic prescribing patterns (Table 1). In the study by Rhee et al, empiric anti-methicillin-resistant Staphylococcus aureus (MRSA) antibiotic use for patients with suspected sepsis increased by 25% between 2013 and 2017, whereas anti-Pseudomonal beta-lactam use increased by 45% [12]. This trend occurred independent of SEP-1 implementation, yet the magnitude of increase in broad-spectrum antibiotic use during this relatively short time period (starting shortly before the preliminary adoption of SEP-1 by CMS in early 2014) is highly concerning. In the study by Barbash et al, SEP-1 implementation was associated with a 10% increase in broad-spectrum antibiotic administration within 3 hours among patients with suspected sepsis relative to expected trends [13]. In the study by Anderson et al, there was a 24.5% increase in antibiotic use amongst all hospitalized patients between October 2014 and October 2016, including increases in anti-MRSA and anti-Pseudomonal antibiotics [14]. A separate analysis by Pakyz et al of 111 hospitals also found that SEP-1 roll-out was associated with a 2.3% immediate increase in antibiotics targeting multi-drug-resistant organisms among all hospitalized patients followed by additional 0.4% increases per month thereafter; they also observed a significant increase in the use of all antibiotic categories at the time of SEP-1 implementation amongst patients with sepsis diagnosis codes [15]. Thus, there are considerable data suggesting that SEP-1 has accelerated the use of broad-spectrum antibiotics. Some hospitals have decreased time-to-antibiotics without unduly increasing unnecessary treatments [18], but this occurred independent of SEP-1 and appears to be the exception rather than the rule given studies from other hospital groups showing increases in antibiotic utilization. Finally, although SEP-1 does not target children specifically, its impact on processes of care in hospitals caring for both adults and children may contribute to antibiotic overuse in pediatric patients.

The Surviving Sepsis Campaign (SSC) guidelines advise clinicians to tailor the urgency and breadth of antibiotics to their certainty of infection and patients’ severity of illness (particularly the presence or absence of shock), in contrast to SEP-1's blanket 3-hour time-to-antibiotic goal for all patients with suspected sepsis [19, 20]. The SSC guidance notes that the urgency of antibiotics varies by severity of illness: short delays are associated with higher mortality rates in patients with septic shock but not in patients without shock [21–23]. The SSC's recommendation to administer antibiotics within 3 hours for possible but unconfirmed sepsis (vs 1 hour for possible septic shock) may still be overly aggressive given that several well-conducted studies show no difference in outcomes associated with intervals until antibiotics of 6 hours or longer for patients without shock [23, 24]. However, we believe the framework of allowing clinicians seeing a patient with possible but unconfirmed sepsis without shock the time and freedom to gather additional data to confirm or refute infection (including laboratory tests, imaging, and observing response to non-infectious treatments) before initiating empiric antibiotics is a step in the right direction for all patients, including those ultimately diagnosed with sepsis and those with sepsis-mimicking conditions.

Retrospective Analyses That Report SEP-1 Compliance Is Associated With Lower Mortality Rates Are Highly Confounded

The primary study cited as evidence that SEP-1 lowers mortality is a retrospective comparison of outcomes for 122 870 Medicare patients who received SEP-1 compliant care matched to 122 870 patients who received non-compliant care between October 2015 and March 2017 conducted by Townsend et al [25]. This study reported that bundle compliance was associated with lower 30-day mortality (22% vs 27%) and median hospital length of stay (5 vs 6 days). This study has been used to assert that even if SEP-1 implementation has not yet clearly lowered sepsis mortality rates, doubling down on efforts to increase bundle compliance (ie, through pay-for-performance) will do so.

This study is unreliable, however, because patients who receive bundle-compliant care tend to be different compared to patients who receive non-compliant care. For example, patients with sepsis without shock have a much lower risk of death compared to patients with septic shock but are also more likely to receive bundle-compliant care because fewer steps are required to pass the measure for patients without shock [26, 27]. This key baseline difference between patients who received SEP-1 compliant versus non-compliant care was evident in the study by Townsend et al [25]. Despite using propensity score matching to improve covariate balance between groups, those who received non-compliant care were much more likely to have septic shock (25.0% vs 15.1%), including persistent hypotension (6.8% vs 3.8%) or lactate levels ≥4.0 mmol/L (17.3% vs 9.4%) (as reported in e-Table 10 of the Townsend paper [25]). This was true in the primary standard-matched analysis and in a secondary analysis that used more stringent matching criteria (septic shock: 19.3% vs 15.7%, persistent hypotension 5.8% vs 4.2%, lactate ≥4.0 mmol/L 12.5% vs 10.8%, e-Table 13) [25]. In a subgroup analysis restricted to patients with septic shock, a more apples-to-apples comparison, mortality rates for patients who received care that was compliant versus non-compliant with the SEP-1 6-hour bundle were similar (38.0% vs 35.3%, P = .326 [Table 3 of the Townsend paper [25]]).

Additionally, younger and healthier patients tend to have clearer clinical presentations of sepsis (eg, fever, chills, rigors, productive cough), which ease diagnosis and management; conversely, older and more complicated patients (with greater baseline risk of death) often present with more ambiguous syndromes that lead to delays in sepsis care and may have comorbidities that make clinicians more cautious about administering large volumes of fluids [26, 27]. Importantly, patients with ambiguous presentations are at substantially higher risk of mortality even after accounting for age, comorbidities, illness severity, and time-to-antibiotics [26, 27]. These important nuances are not captured in the data abstracted for SEP-1 and therefore were not included in the analysis used by Townsend et al [25]. Another important confounder is the timing of sepsis onset: patients who develop sepsis while hospitalized are less likely to receive bundle-compliant care but are generally more severely ill than patients with community-onset sepsis and have at least 2-fold higher mortality rates that clearly are not attributable to bundle compliance rates alone [27–29]. This too was not included in the analysis by Townsend et al. Tellingly, studies that have used more comprehensive data for risk adjustment, including presenting symptoms, detailed comorbidities, and community- versus hospital-onset sepsis have found no association between SEP-1 compliance and mortality [27, 30].

Finally, the study by Townsend et al only focused on Medicare beneficiaries discharged with sepsis diagnosis codes who met the specific SEP-1 time zero criteria. In practice, clinicians often do not know in real-time whether a patient has sepsis but nonetheless may feel compelled to treat for the possibility. Clinicians also frequently treat patients who fall outside Medicare eligibility features but suffer from sepsis or a mimic; these patient were also not included in in the analysis. As such, this study fails to consider the impact of the SEP-1 bundles on many patients, especially those ultimately diagnosed with non-infectious conditions. For this reason, the best insight into the real impact of SEP-1 comes from the time-series analyses described earlier that analyzed the real-world impact of SEP-1 implementation in complete populations with suspected sepsis, including those ultimately found to have something other than sepsis, using objective clinical criteria and thus minimizing ascertainment bias. These studies found no effect on mortality rates.

Some studies assessing mandated bundles outside of SEP-1 (eg, the New York State bundle) have reported that sepsis mortality rates declined following implementation [31–33]. One potential explanation is that the New York State regulations were more effective than SEP-1 because they combined structure (ie, developing and submitting sepsis screening and treatment protocols), process (publicly reporting 3- and 6-hour bundle data), and outcomes (publicly reporting risk-adjusted mortality). However, the true impact of the New York State regulations is difficult to assess because sepsis bundle roll-outs were accompanied by efforts to increase sepsis recognition. This typically leads to an ascertainment bias as clinicians diagnose more patients with sepsis over time, including patients with milder syndromes and lower mortality rates, which in turn can give a misleading impression that bundles are lowering sepsis mortality rates [34].

There Are No High-Quality Data Demonstrating That the 30 cc/kg Crystalloid Fluid Bolus Threshold or Repeat Lactate Measurements Reduce Sepsis Mortality

Both the intravenous fluid bolus and repeat lactate requirements are common causes of SEP-1 compliance failures yet are supported by minimal data [11]. Two large observational studies, including approximately 50 000 patients with sepsis in New York State and 6000 patients in California, found no association between compliance with the fluid resuscitation bundle component and mortality [28, 35]. These results align with a multicenter randomized trial comparing liberal versus restrictive fluids for patients with septic shock that showed no difference in outcomes, further underscoring the lack of data to support a 1-size fits-all approach to fluid management [36]. A randomized trial also calls into question the value of serial lactate measures to guide fluid resuscitation for patients with septic shock: mortality rates were similar or lower among patients randomized to fluid resuscitation guided by physical exam (capillary refill time) versus serial lactate measurements [37]. Not surprisingly, the SSC Guidelines designated both these processes as “weak recommendations with low quality of evidence” [19]. Hospitals should not be denied payment and physicians deemed noncompliant for failing to follow non-evidence-based practices.

Focus on SEP-1 Diverts Attention and Resources From More Effective Measures and Comprehensive Sepsis Care

SEP-1 has had the unintended consequence of focusing hospitals’ and providers' attention on bundle compliance and documentation to the exclusion of other aspects of comprehensive sepsis care. In many hospitals, considerable time is spent discussing ways to improve documentation (ie, for repeat volume status and perfusion exams or potential exclusions for the 30 cc/kg fluid bolus) for the sole purpose of improving SEP-1 compliance scores rather than identifying and implementing changes in care that are more likely to improve patient outcomes.

We agree that initial care matters, but improving sepsis outcomes necessitates close attention to the full spectrum of patient care in addition to the first few hours of resuscitation, particularly as patients with sepsis are often hospitalized for long periods and are at high risk for complications of hospital care. Other opportunities to improve care include speeding identification of caustive pathogens and antibiotic susceptibilities, implementing processes to facilitate timely source control, optimizing antimicrobial dosing and administration regimens, encouraging timely antimicrobial de-escalation, minimizing sedation and delirium, using lung protective ventilation, preventing hospital-acquired infections, preventing pressure injuries, and improving rehabilitation programs [38].

The CMS proposal to shift SEP-1 from pay-for-reporting to pay-for-performance is a step backward in that healthcare systems will feel compelled to invest even more resources into the same limited set of processes that do not clearly improve outcomes. We believe that hospitals, clinicians, and patients will be best served by retiring SEP-1 and shifting to a measure focused on patient outcomes. This will encourage hospitals to pay more attention to the full breadth of sepsis care and stimulate further innovations in diagnosis and treatment. Hospitals could still choose to emphasize early resuscitation bundles based on internal assessments of gaps in care but they should not be forced to do so.

PART 2: RECOMMENDATIONS TO IMPROVE THE eCQM SEPSIS MORTALITY MEASURE

We support CMS's plan to implement a risk-adjusted sepsis outcome measure. Although there are multiple patient-centered sepsis outcomes that could be candidates, we believe that a focus on mortality is the right place to start. We also applaud CMS's plan to make the measure fully electronic, as this will improve efficiency, scalability, and objectivity compared to the current manual SEP-1 abstraction process which is highly resource-intensive and often variably applied [39–41].

The draft specification for the eCQM sepsis mortality measure identifies sepsis using three criteria (Table 2): (1) systemic inflammatory response syndrome (SIRS) criteria, defined using vital signs and white blood cell counts, (2) suspected infection, defined as antibiotic administrations or the use of present-on-admission (POA) ICD-10 codes for sepsis or infection, and (3) acute organ dysfunction, defined using vital signs, administered medications, use of respiratory support, and laboratory tests. We recommend the following modifications to the eCQM strategy for identifying sepsis to improve its credibility, efficiency, and reliability while diminishing the risk of unintended consequences.

Table 2.

Comparison of Definitions: CMS Community-Onset Sepsis 30-day Mortality Electronic Clinical Quality Measure (Draft Specifications) and CDC Adult Sepsis Event

CMS eCQM Community-Onset Sepsis	CDC Adult Sepsis Event
Systemic Inflammatory Response Syndrome (SIRS) criteria (≥2 of the following criteria within 6 h of presentation): Temperature >38 or <36°C Heart rate >90 beats/min Respiratory rate >20 breaths/min White blood cell count >12 000 or <4000 cells/mm3	Not Used
Suspected infection (any one of the following criteria): ICD-10 diagnosis code for infection (of suspected bacterial origin) POA, or ICD-10 diagnosis for sepsis POA, or Administration of antibiotics within 30 h of presentation and continuation for ≥3 d or until discharge	Presumed Serious Infection: Blood culture obtained (regardless of result), and Administration of antibiotics (including at least one parenteral antibiotic) within ±2 d of blood culture day and continued for ≥4 d or until ≤1 d prior to death, discharge to another acute care hospital or hospice, or transition to comfort measures
Organ dysfunction (≥1 of the following criteria within 6 h of presentation, in the absence of an alternative explanation): Administration of vasopressors MAP <65 mmHg or systolic blood pressure <90 mmHg Initiation of mechanical ventilation Creatinine >2.0 mg/dL (and at least 0.5 greater than, or 2×, baseline value) Total bilirubin >2.0 mg/dL Platelet count <100 000/mm3 INR >1.5 or aPTT >60 s	Organ dysfunction (≥1 of the following criteria within ±2 d of blood culture day): Initiation of vasopressors Initiation of mechanical ventilation Doubling in serum creatinine or decrease by ≥50% of estimated glomerular filtration rate relative to baseline (excluding patients with ICD-10 codes for end-stage kidney disease) Total bilirubin ≥2.0 mg/dL and doubling from baseline Platelet count <100 000/mm3 and ≥50% decline from baseline (baseline must be ≥100 000/mm3) Serum lactate ≥2.0 mmol/L

Abbreviations: aPTT, Activated Partial Thromboplastin Clotting Time; CDC, Centers for Disease Control and Prevention; CMS, Centers for Medicare & Medicaid Services; eCQM, electronic clinical quality measure; INR, international normalized ratio; MAP, mean arterial pressure; POA, present-on-admission.

Remove SIRS Criteria From the eCQM

SIRS criteria are common and nonspecific. They are present in up to 50% of hospitalized patients at some point during their stay, most of whom do not have sepsis [42]. Another study found that 18% of ED patients met SIRS criteria, but only 26% of that group had an acute infection [43]. SIRS criteria are also insensitive; one in eight critically ill patients with sepsis do not meet SIRS criteria [44]. Limiting the eCQM to patients with SIRS criteria therefore risks both over-detection and under-detection of sepsis.

Anchoring the eCQM to SIRS also risks promoting overreliance on SIRS as a screening tool. Using an insensitive and nonspecific trigger cannot drive improvements in care. Indeed, the evidence suggests SIRS-based alerts in the ED increase antibiotic use and Clostridioides difficile infections but do not improve mortality [45, 46]. SIRS-based prompts for sepsis recognition in the intensive care unit (ICU) or inpatient setting have also not improved patient outcomes in randomized trials [47–49]. These limitations of SIRS led to their exclusion from current international consensus criteria for sepsis (Sepsis-3) [50].

Including SIRS criteria also increases the eCQM's complexity and risks undermining comparability between hospitals. SIRS elements are prone to transient perturbations (heart rate, respiratory rate) that are variably recorded in the EHR or recorded in different ways in the EHR (eg, separate fields for temperature by axilla, mouth, rectum, bladder, etc.). This will likely lead to differences in the ways hospitals extract and curate SIRS criteria, introducing unnecessary additional variability between hospitals. Eliminating SIRS from the eCQM will simplify implementation, align CMS conceptually with Sepsis-3 criteria, decrease the risk of encouraging unnecessary antibiotics for patients with SIRS who are not infected, and prevent under-recognition of patients with sepsis but without SIRS.

ICD-10 Codes Should Not Be Used to Identify Patients With Infection

CMS proposed using antibiotic administrations or ICD-10 codes to identify patients with possible infection. Diagnosis codes will not increase sensitivity above antibiotic administrations since almost all meaningful bacterial infections are treated with antibiotics. A large medical record review-based study found that infection codes had a sensitivity of only 77% (95% confidence interval [CI] 75%–79%) for identifying infected patients [51]. Others report sensitivities below 50% for sepsis-specific codes [52].

Including diagnosis codes also risks introducing variability due to differences in code use amongst clinicians and between hospitals [53–55]. This is partly due to variability in the diagnosis of sepsis and partly due to differences in coding practices. One study asked intensivists to review 5 case vignettes describing patients with possible infection and organ dysfunction: 17% of respondents classified 1 case as sepsis, 28% deemed 2 of the 5 cases as sepsis, 33% classified 3 cases as sepsis, 19% flagged 4 cases as sepsis, and 3% thought all 5 patients had sepsis (kappa 0.29) [53]. Another study found that the median sensitivity of sepsis codes for clinical sepsis was 30% overall across 193 hospitals but ranged from 5% to 54% between hospitals [55]. Both diagnosis and coding practices for sepsis are changing over time and susceptible to both internal initiatives, such as quality improvement and sepsis awareness campaigns, and external pressures, such as changes in payment policies [1, 34, 56–59]. Lastly, present-on-admission codes are often inaccurate and variably applied across hospitals, especially when there are financial implications [60, 61].

The eCQM Should Be Harmonized With CDC's Electronic Surveillance Metric to Develop a Shared Federal Sepsis Measure

CDC invested considerable resources into developing and validating the Adult Sepsis Event (ASE) definition, an electronic surveillance metric modeled on the Sepsis-3 framework of infection with concurrent organ dysfunction but optimized for simplicity and reproducibility across institutions [62]. ASE defines suspected infection as a blood culture order and at least 4 days of new antibiotics (fewer if death or discharge occurs before 4 days). ASE defines organ dysfunction as initiation of vasopressors or mechanical ventilation, presence of an elevated blood lactate, or new changes in creatinine, total bilirubin, or platelet count. These organ dysfunction thresholds parallel the Sequential Organ Failure Assessment Score but eschew components that are inconsistently measured, documented, and stored in EHRs such as mental status, vasopressor doses, urine output, blood gas results, and fraction of inspired oxygen at the time of blood gas measurement. ASE does not include SIRS or diagnosis codes (see Table 2 for comparison of ASE vs draft eCQM criteria).

The ASE was developed to overcome the limitations of administrative data for sepsis surveillance and has been applied to hundreds of hospitals with diverse EHRs to estimate sepsis burden and characteristics [1, 63–68]. Studies show that ASE is more sensitive than sepsis diagnosis codes, has similarly high specificity, and is more reliable for assessing trends in sepsis incidence and mortality [1, 69]. ASE also can distinguish hospital-onset versus present-on-admission sepsis [29, 70], is strongly associated with poor outcomes [63, 71], and performs similarly in US and non-US hospitals [1, 63]. These key strengths of ASE make it well suited to serve as the basis for a national sepsis outcome measure in addition to an epidemiologic tool.

Despite its strengths, ASE can be updated and improved. The ASE infection criteria misses patients in whom blood cultures are not drawn and the elevated lactate criterion may distort temporal trends in hospitals that are checking lactates on more patients over time [51, 72]. Hypotension that does not require vasopressors and non-invasive respiratory support short of invasive mechanical ventilation were not included in ASE's organ dysfunction thresholds, in part because these data elements were not routinely available in many EHRs when ASE was first developed. With the current widespread adoption of Fast Health Interoperable Resources, including these important parameters is now feasible [73].

We encourage CMS and CDC to continue to collaborate on developing a single, harmonized measure based on the insights of both of their sepsis metric development teams. Harmonizing sepsis criteria across federal agencies will promote unity, increase credibility and efficiency, and facilitate shared prevention initiatives.

CONCLUSION

CMS has brought welcome attention to sepsis but SEP-1 itself has not catalyzed better clinical outcomes. We suggest retiring SEP-1 rather than using it in a payment model and support shifting to CMS's planned eCQM sepsis mortality measure. We further advocate removing SIRS criteria and diagnosis codes for infection from the eCQM and harmonizing it with CDC's ASE definition. These steps will result in a more robust measure that all stakeholders can embrace and bring us closer to our shared goal of improving outcomes for all patients.

Notes

Financial support. None.