Evidence Underpinning the Centers for Medicare & Medicaid Services’ Severe Sepsis and Septic Shock Management Bundle (SEP-1)

Abstract

Background:

The Severe Sepsis and Septic Shock Early Management Bundle (SEP-1), the sepsis performance measure introduced in 2015 by the Centers for Medicare & Medicaid Services (CMS), requires the reporting of up to 5 hemodynamic interventions, as many as 141 tasks, and 3 hours to document for a single patient.

Purpose:

To evaluate whether moderate- or high-level evidence shows that use of the 2015 SEP-1 or its hemodynamic interventions improves survival in adults with sepsis.

Data Sources:

PubMed, Embase, Scopus, Web of Science, and ClinicalTrials.gov from inception to 28 November 2017 with no language restrictions.

Study Selection:

Randomized and observational studies of death among adults with sepsis who received versus those who did not receive either the entire SEP-1 bundle or 1 or more SEP-1 hemodynamic interventions, including serial lactate measurements; a fluid infusion of 30 mL/kg of body weight; and assessment of volume status and tissue perfusion with a focused examination, bedside cardiovascular ultrasonography, or fluid responsiveness testing.

Data Extraction:

Two investigators independently extracted study data and assessed each study’s risk of bias; 4 authors rated level of evidence by consensus using CMS criteria published in 2013. High- or moderate-level evidence required studies to have no confounders and low risk of bias.

Data Synthesis:

Of 56 563 references, 20 studies (18 reports) met inclusion criteria. One single-center observational study reported lower in-hospital mortality after implementation of the SEP-1 bundle. Sixteen studies (2 randomized and 14 observational) reported increased survival with serial lactate measurements or 30-mL/kg fluid infusions. None of the 17 studies were free of confounders or at low risk of bias. In 3 randomized trials, fluid responsiveness testing did not alter survival.

Limitation:

Few trials, poor-quality and confounded studies, and no studies (with survival outcomes) of the focused examination or bedside cardiovascular ultrasonography. Use of the 2015 version of SEP-1 and 2013 version of CMS evidence criteria, both of which were updated in 2017.

Conclusion:

No high- or moderate-level evidence shows that SEP-1 or its hemodynamic interventions improve survival in adults with sepsis.

The Centers for Medicare & Medicaid Services (CMS) oversees Medicare and Medicaid, the 2 largest providers of health insurance in the United States. In 2015, CMS instituted the Severe Sepsis and Septic Shock Early Management Bundle (SEP-1) performance measure and began to monitor reporting of the measure by hospitals (1). In turn, the Joint Commission may eventually use SEP-1 completion as an element in hospital accreditation (2). To fulfill SEP-1’s 2015 through 2017 requirements for 1 patient, clinicians must do up to 7 interventions (Table 1) (3, 4); documentation for a single patient requires up to 141 tasks and takes as long as 3 hours. As a result, SEP-1 is one of CMS’s most complex performance measures (5–7). In time, CMS reimbursement may come to incentivize hospitals to complete all of the interventions. Thus, providers may be encouraged to fully adopt this protocol or jeopardize incentivized reimbursement (8). Because each of the interventions may become a compulsory part of U.S. health care practice, we believe they should be scientifically proved to improve meaningful outcomes.

Table 1.

2015 through 2017 CMS SEP-1 Performance Measure^*

Type of Sepsis	Within 3 Hours of Presentation	Within 6 Hours of Presentation
Severe	Measure initial lactate levels Administer broad-spectrum or other antibiotics Draw blood cultures before administering antibiotics	Repeat lactate level measurement only if initial lactate level is elevated
Septic shock	Resuscitate with crystalloid fluids, 30 mL/kg	If hypotension persists after fluid administration: Vasopressors If hypotension persists after fluid administration or initial lactate level is ≥4 mmol/L, perform a volume status and tissue perfusion assessment consisting of either of the following:
		A focused examination, including all of the following:  Vital signs
		Cardiopulmonary examination  Capillary refill evaluation  Peripheral pulse evaluation  Skin examination
		Any 2 of the following:
		Central venous pressure measurement  Central venous oxygen measurement  Bedside cardiovascular ultrasonography  Passive leg raise or fluid challenge

CMS = Centers for Medicare & Medicaid Services; SEP-1 = Severe Sepsis and Septic Shock Early Management Bundle.

^*To fulfill SEP-1’s requirements, clinicians must do up to 7 interventions, including 5 (in boldface) directed at hemodynamic support. This performance measure requires documentation of up to 141 tasks and takes up to 3 hours to complete per patient (3, 5–7). These tasks and the 20 flow diagrams that need to be followed are detailed over 45 pages in the SEP-1 specifications manuals (3, 4).

The Centers for Medicare & Medicaid Services uses published criteria to grade the evidence supporting its performance measures (See Supplement Table 1 for 2013 criteria, available at Annals.org) (9). To be considered high- or moderate-level evidence, studies must be free of confounders and have low risk of bias. The CMS specifications manuals describing SEP-1 for 2015 to 2017 for providers indicate that the measure (Table 1) is based on credible scientific evidence and that its interventions are “directly related to . . . reductions in hospital mortality, length of stay, and costs of care” (3, 4). The requirement in SEP-1 for early antibiotic administration fits these criteria because moderate-level scientific evidence shows that it improves survival in septic patients (10–13); however, the SEP-1 hemodynamic interventions do not. Three recent multicenter randomized controlled trials showed that resuscitation protocols based on central venous pressure (CVP) and oxygen saturation (ScvO₂) measurements increased costs and did not benefit septic patients (14–17). Editorials by sepsis experts note that the serial lactate measurements and fluid infusions of 30 mL/kg of body weight mandated by SEP-1 may not benefit septic patients and may sometimes be harmful (11, 18, 19). Finally, recent sepsis guidelines do not support or even mention several components of SEP-1’s volume status and tissue perfusion assessment (13). These observations raise concerns that scientific evidence does not support SEP-1’s hemodynamic interventions and that the basis for their inclusion should be independently evaluated.

To address these concerns, we searched for scientific evidence supporting the required hemodynamic interventions. We first communicated with CMS in November 2016 and were advised that the 62 references cited in the SEP-1 version 5.2 specifications manual (3) provided this evidence. On review, only 40 were original studies addressing sepsis, and none showed that any required hemodynamic intervention improved survival or other clinically meaningful outcomes (20). Thus, we did a systematic review evaluating whether high- or moderate-level evidence according to the 2013 CMS criteria shows that use of SEP-1 or its hemodynamic interventions improves survival in adults with sepsis. We analyzed version 5.2 of SEP-1. This version 5.2 was in effect to 2017 and had similar components to version 5.0a (3, 4).

METHODS

This systematic review, prepared according to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement, was registered in the PROSPERO database (CRD42016052716) on 2 December 2016.

Data Sources and Searches

We searched PubMed, Embase, Scopus, Web of Science, and ClinicalTrials.gov (Supplement, available at Annals.org) without language restrictions from each database’s inception to 28 November 2017. We also scanned the reference lists of analyzed studies and references cited in reviews and guidelines identified in the searches.

Study Selection

We included randomized trials and observational studies of adults (aged ≥16 years) with sepsis, severe sepsis, or septic shock that compared mortality rates in patients receiving versus not receiving 1 or more of the following SEP-1 hemodynamic interventions: serial lactate measurements, a 30-mL/kg fluid infusion, or assessment of volume status and tissue perfusion. For volume status and tissue perfusion assessments, we considered bedside cardiovascular ultrasonography; a passive leg raise or fluid challenge; and a clinician’s focused examination of all of the following: vital signs, cardiopulmonary function, capillary refill, peripheral pulse, and skin. Interventions could have been tested individually or as part of a treatment bundle. We excluded studies of CVP and ScvO₂ measurements because these have been proved to increase costs and not benefit septic patients (14–17). Only studies with recognized definitions of sepsis were included. Studies comparing 2 selected interventions (for example, resuscitation based on serial lactate vs. ScvO₂ measurements) without a usual care control were excluded. Included observational studies were nonrandomized and had a before–after or concurrent control design. In the before–after design, a control group of patients before the introduction of a particular intervention was compared with a treated group after the intervention. In the concurrent control design, nonrandomized patients who received a particular intervention were compared with concurrent patients who did not receive it.

Two authors (D.J.P. and P.Q.E.) reviewed searches using a 2-step process of title and abstract screening followed by full-text review of selected articles. Author consensus resolved any uncertainty about study inclusion.

Data Extraction and Quality Assessment

The same 2 authors extracted data using a standardized tool (Supplement), and 3 authors (D.J.P., J.S., and P.Q.E.) checked the extraction for accuracy. Extracted data included interval from admission to intervention, proportion of patients receiving the intervention, measured level (if the intervention was a measurement) and whether it correlated with subsequent treatment, amount administered (if the intervention was a treatment), and bundle composition and administration (if the intervention was part of a bundle). Three-hour or 6-hour bundles were defined on the basis of whether interventions required completion within a 3- or 6-hour period.

For each study, 2 investigators (D.J.P. and P.Q.E.) independently assessed potential confounders and risk of bias. Disagreements were resolved by consensus. First, we examined whether antibiotic administration was documented and similar (appropriateness and timing) in intervention and control groups. Second, we assessed the administration of adjunctive aids designed to improve participant management and outcomes. If adjunctive aids were used, we classified them as educational aids to improve provider recognition and care of septic patients (such as conferences, lectures, or posters) or prioritized care aids that directly affected management of septic patients (such as priority bed allocation, sepsis pagers or another alert system, a clinical sepsis team, expedited consultations or triage system for septic patients, or sepsis screening checklists). In before–after studies, we judged that patients in the after group would have been exposed to adjunctive aids initiated along with study interventions, whereas patients in the before group would not. For concurrent control studies, we judged that patients in groups compliant with all study interventions would have been exposed to adjunctive aids. Whether control patients not receiving the intended interventions were exposed was unknown. Absent or incomplete data about use of an intervention limited interpretation of any reported differences in survival between control and intervention patients. In studies investigating a measurement to guide therapy (such as serial lactate), we assessed whether lactate levels were reported and, if they were, whether they were associated with subsequent changes in fluid and vasopressor treatment. In studies investigating a treatment for hemodynamic support (such as a 30-mL/kg fluid infusion), we assessed whether the amount of fluid was recorded and whether it differed between intervention and control groups.

We assessed risk of bias for randomized trials and observational studies with the Cochrane Risk of Bias Tool and the Newcastle-Ottawa Scale, respectively (21, 22). Two investigators (D.J.P. and P.Q.E.) assessed this risk independently and settled disagreements by consensus. All components of either tool had to be graded as low risk of bias for a study to be rated low risk overall. For observational studies, comparability bias was based on whether severity of illness and presence of comorbid conditions were recorded and similar at baseline. The primary outcome was mortality, assessed as the relative risk (or odds ratio) of death and considered in the following hierarchy: 90-day, 60-day, 30-day, 28-day, hospital, or intensive care unit. We determined outcome bias on the basis of whether or not the study assessed mortality blindly or from record linkage and whether it reported 28-day or longer mortality and adequacy of follow-up.

Data Synthesis and Analysis

The level of evidence (high, moderate, low, or insufficient) for each intervention, and for SEP-1 overall, was based on 2013 CMS criteria (Supplement Table 1) and determined by author consensus (D.J.P., J.S., C.N., and P.Q.E.). We considered the effect estimates of individual studies to be confounded if 1 or more of the confounders described previously were present. If a study had high or unknown risk of bias, it was considered not to be at low risk of bias. For binary outcomes (mortality and appropriate antibiotics), odds ratios and their 95% CIs were calculated and plotted. For time to antibiotic administration, reported medians and interquartile ranges were converted to mean differences and SEs using Wan and colleagues’ method (23).

Role of the Funding Source

Intramural funding from the National Institutes of Health supported this work. The National Institutes of Health had no role in the design of the study or the collection, analysis, or interpretation of the data. The National Institutes of Health Clinical Center approved submission of the finished manuscript.

RESULTS

The literature search identified 56 563 references (Appendix Figure, available at Annals.org). After screening, we reviewed 527 full-text articles, including 4 translated from Chinese and 1 from Spanish. Of these, 18 reports describing 20 studies compared the effect on survival of 1 or more of the following nonantibiotic hemodynamic SEP-1 interventions in septic patients versus a control group: serial lactate measurements alone (5 articles), a 30-mL/kg fluid infusion alone (5 articles), serial lactate measurements and a 30-mL/kg fluid infusion (4 articles), a component of the volume status and tissue perfusion assessment (3 articles), and SEP-1 in its entirety (1 article) (24– 41). One article tested 30-mL/kg fluid infusions in 3 separate and distinct cohorts (31). These cohorts represent 3 individual studies and are distinguished here on the basis of their final year of enrollment. Three articles assessed fluid responsiveness—2 with a passive leg raise and 1 with a sequential fluid challenge (38– 40). We found no sepsis study that examined the effects on survival of the focused examination alone (that is, the combined vital sign, cardiopulmonary, skin, capillary refill, and peripheral pulse evaluations) or bedside cardiovascular ultrasonography alone versus a control group. Supplement Tables 2 to 9 (available at Annals.org) provide data on these 20 analyzed studies.

Assessment of Serial Lactate Measurements, a 30-mL/kg Infusion, or Both

Sixteen studies investigated serial lactate measurements, a 30-mL/kg fluid infusion, or both (Table 2). Of these, 2 were randomized controlled trials (n = 117 control participants and 118 intervention patients) and 14 were observational studies, including 8 concurrent control studies (n = 12 019 control participants and 4358 intervention patients) and 6 before–after studies (n = 6515 control participants and 7546 postintervention patients) (Table 2). One of the randomized trials was a subgroup of patients with sepsis (n = 135) from a larger trial investigating serial lactate measurements in critically ill patients with sepsis or nonsepsis presentations (24).

Table 2.

Study Design, Survival, and Antibiotic Administration

Study, Year (Reference)	Design	Patients, n				Odds Ratio of Survival (95% CI)
		Control		Intervention
		Survivors	Total	Survivors	Total
Studies with serial lactate measurements
Jansen et al, 2010 (24)	RCT	37	67	48	68	1.95 (0.96 to 3.96)
Xiaochun et al, 2015 (25)	RCT	22	50	30	50	1.91 (0.86 to 4.23)
Nguyen et al, 2007 (26)	OS-CCC	153	253	61	77	2.49 (1.36 to 4.57)
Dettmer et al, 2015 (27)	OS-CCC	67	111	101	132	2.14 (1.23 to 3.72)
McColl et al, 2017 (28)	OS-BACC	116	167	153	185	2.10 (1.27 to 3.48)
Studies with 30-mL/kg fluid infusion
Larosa et al, 2012 (29)	OS-CCC	17	24	31	34	4.25 (0.97 to 18.6)
Hayden et al, 2016 (30)	OS-BACC	90	108	110	130	1.10 (0.55 to 2.20)
Leisman et al, 2017 (31)
2012†	OS-CCC	3556	4769	826	1050	1.26 (1.07 to 1.48)
2014†	OS-CCC	787	958	640	739	1.40 (1.07 to 1.84)
2015†	OS-CCC	4046	5124	1732	2115	1.20 (1.06 to 1.37)
Ferreras Amez et al, 2017 (32)	OS-BACC	155	222	178	222	1.75 (1.13 to 2.71)
Teles et al, 2017 (33)	OS-CCC	25	46	90	121	2.44 (1.20 to 4.96)
Studies with serial lactate measurements and 30-mL/kg fluid infusion
Liu et al, 2016 (34)	OS-BACC	5104	5942	5723	6544	1.14 (1.03 to 1.27)
Rhodes et al, 2015 (35)	OS-CCC	473	734	65	90	1.43 (0.88 to 2.33)
Siontis et al, 2015 (36)	OS-BACC	38	51	36	41	2.46 (0.80 to 7.61)
Grek et al, 2017 (37)	OS-BACC	17	25	384	424	4.52 (1.83 to 11.1)
Studies assessing fluid responsiveness
Hou et al, 2016 (38)	RCT	31	32	32	32	3.10 (0.12 to 78.9)
Kuan et al, 2016 (39)	RCT	55	61	53	61	0.72 (0.23 to 2.22)
Cronhjort et al, 2017 (40)	RCT	16	18	14	16	0.88 (0.11 to 7.05)
Studies assessing SEP-1
Ramsdell et al, 2017 (41)	OS-BACC	35	48	94	110	2.18 (0.95 to 5.00)

		Antibiotic Administration
Patients With Appropriate Antibiotics, %		Time to Antibiotics, min
Control	Intervention	Control	Intervention
NR	NR	NR	NR
NR	NR	NR	NR
87*	100	NR	NR
NR	NR	NR	NR
NR	NR	101*	71
24*	91	NR	NR
NR	NR	Mean: 139 (SD, 74)*	Mean: 81 (SD, 39)
68‡	100	Median: 87 (IQR, −10 to 184)‡	Median: 38 (IQR, −19 to 95)
75‡	100	Median: 66 (IQR, 20 to 172)‡	Median: 32 (IQR, 1 to 73)
59‡	100	Median: 85 (IQR, 20 to 208)‡	Median: 29 (IQR, −4 to 66)
NR	NR	Median: 112 (IQR, 55 to 169)	Median: 89 (IQR, 41 to 166)
NR	NR	NR	NR
95	96	Mean: 2.4 (SD, 1.5)*§	Mean: 2.3 (SD, 1.5)§
NR	NR	NR	NR║
NR	NR	NR	NR║
60‡	100	NR	NR
NR	NR	NR	NR
84	90	Median: 61 (IQR, 33 to 93)	Median: 57 (IQR, 41 to 109)
NR	NR	NR	NR
81	90	NR	NR

IQR = interquartile range; NR = not reported; OS-BACC = before–after cohort control observational study; OS-CCC = concurrent cohort control observational study; RCT = randomized controlled trial; SEP-1 = Severe Sepsis and Septic Shock Early Management Bundle.

^*P < 0.050.

^†Leisman and colleagues (31) analyzed 3 cohorts of patients representing 3 individual studies. This year indicates the final year of the cohort’s enrollment.

^‡P < 0.001.

^§Values are presented in hours.

^║This study investigated a bundle in which early antibiotic administration was encouraged.

All 16 studies reported that the interventions or bundle of treatments improved survival (10 reached statistical significance). In all 10 studies that provided data, antibiotics were administered more quickly for intervention patients (reported as statistically significant for 9 studies) (Table 2 and Figure 1). McColl and colleagues (28) provided only a point estimate with no 95% CIs. In 3 studies investigating a bundle, early antibiotic administration was directed at the intervention group only. Eleven of the observational studies included educational or priority care aids (Table 3 and Supplement Table 7). Six of these were before–after studies in which the adjunctive aids were systematically provided to intervention and not control patients. In the 5 concurrent control studies, whether patients who did not receive SEP-1 interventions were exposed to adjunctive aids was unclear. None of the 9 studies examining serial lactate measurements provided the second or subsequent lactate measurements or data examining whether changes in lactate levels over time were associated with meaningful changes in treatment (for example, additional fluid administration for an increasing lactate level). Of 11 studies investigating a 30-mL/kg fluid infusion, 6 reported that a greater proportion of patients in the intervention than control group received the prescribed volume of fluid. Only 1 of the 11 studies reported the actual total or weight-based volume of fluid patients received. The mean weight-based volume administered to patients in this study was less than 30 mL/kg, with wide variability in both the intervention and control groups (26.3 mL/kg [SD, 17.8] and 24.6 mL/kg [SD, 18], respectively). None of the 16 studies had low risk of bias (Table 3 and Supplement Table 8).

Figure 1.

Summary of the proportion of patients receiving early antibiotics or the time to antibiotic administration among serial lactate and 30-mL/kg fluid infusion sepsis studies.

Ten of the 16 studies examining serial lactate measurements, a 30-mL/kg fluid infusion, or both provided data showing that a greater proportion of intervention than control patients received appropriate early antibiotics (6 studies) or that the time to antibiotic administration was shorter in intervention than control patients (7 studies).

* Antibiotic data were not reported.

† Only a point estimate with no 95% CIs for time to antibiotics was provided.

‡ Leisman and colleagues (31) analyzed 3 cohorts of patients representing 3 individual studies. This year indicates the final year of the cohort’s enrollment.

§ Intervention groups were treated with early antibiotics, but no data were provided.

Table 3.

Summary of Adjunctive Aids Used in Studies, Measurement and Administration of Targeted Interventions, and Risk-of-Bias Assessment

Study, Year (Reference)	Adjunctive Aids Used				Interventions
	Educational*		Priority Caret †		Serial Lactate Level Studies
	Control	Intervention	Control	Intervention	Second Lactate Level Reported		Relationship Between Changes in Serial Lactate Levels and Treatment Reported
					Control	Intervention
Studies with serial lactate measurements
Jansen et al, 2010 (24)	NR	NR	NR	NR	NA	No	No
Xiaochunetal, 2015(25)	NR	NR	NR	NR	NA	No‡	No
Nguyen et al, 2007 (26)	U	Yes	U	Yes	No	No§	No
Dettmer et al, 2015 (27)	NR	NR	NR	NR	No	No	No
McColl etal, 2017 (28)	No	Yes	NR	NR	No	No	No
Studies with 30-mL/kg fluid infusion
Larosa et al, 2012 (29)	NR	NR	U	Yes	NA	NA	NA
Hayden et al, 2016 (30)	NR	NR	No	Yes	NA	NA	NA
Leisman et al, 2017 (31)
2012║	NR	NR	U	Yes	NA	NA	NA
2014║	NR	NR	U	Yes	NA	NA	NA
2015║	NR	NR	U	Yes	NA	NA	NA
Ferreras Amez et al, 2017 (32)	No	Yes	No	Yes	NA	NA	NA
Teles etal, 2017 (33)	NR	NR	NR	NR	NA	NA	NA
Studies with serial lactate measurements and 30-mL/kg fluid infusion
Liu etal, 2016(34)	No	Yes	NR	NR	No	No	No
Rhodes et al, 2015 (35)	NR	NR	NR	NR	No	No	No
Siontis et al, 2015 (36)	No	Yes	No	Yes	No	No	No
Greketal, 2017 (37)	No	Yes	NR	NR	No	No	No

Interventions				Study Elements at Risk of Bias, n
30-mL/kg Fluid Infusion Studies				Total Possible	Scored as High or Unknown Risk
Patients Reported to Receive 30-mL/kg Fluid Infusion, %		Mean Weight-Based Fluid Volume Actually Administered (SD), mL/kg
Control	Intervention	Control	Intervention
NA	NA	NA	NA	6	4
NA	NA	NA	NA	6	5
NA	NA	NA	NA	8	5
NA	NA	NA	NA	8	4
NA	NA	NA	NA	8	2
50	71	NR	NR	8	3
NR	NR	NR	NR	8	4
36	100	NR	NR	8	2
36	100	NR	NR	8	2
31	100	NR	NR	8	2
26	57	NR	NR	8	3
NR	NR	NR	NR	8	3
NR	NR	24.6 (18)	26.3 (18)	8	3
NR	NR	NR	NR	8	6
NR	NR	NR	NR	8	6
33	81	NR	NR	8	5

NA = not applicable; NR = not reported; U = unknown.

^*Included the use of conferences, lectures, or posters to improve provider recognition and management of septic patients.

^†Included the use of sepsis alert systems, sepsis clinical teams, expedited sepsis consults, sepsis triage systems, or sepsis screening checklists to improve patient care.

^‡Lactate clearance (≥10% or lactate levels ≤2.0 mmol/L) was reported in 82% of intervention patients, but serial lactate levels and the effect on therapy were not provided.

^§Lactate clearance was reported but not defined in 79% of intervention and 39% of control patients, but serial lactate levels and the effect on therapy were not provided.

^║Leisman and colleagues (31) analyzed 3 cohorts of patients representing 3 individual studies. This year indicates the final year of 1 cohort’s enrollment.

Assessment of Fluid Responsiveness Testing

Three randomized controlled trials (n = 111 control participants and 109 treated patients) investigated whether testing fluid responsiveness during resuscitation in septic patients affects survival (Table 2). Two used a protocol incorporating passive leg raises, and 1 used sequential fluid challenges (Supplement Tables 4 and 5). Two studies assessed fluid responsiveness with noninvasive cardiac output monitors and 1 with a pulse index contour continuous cardiac output monitor. Control groups in all studies were usual care; no study reported that the interventions increased survival (Table 2). In 1 study providing data, use of appropriate antibiotics did not differ between groups (Table 2). Because survival did not differ in these studies, we did not further analyze potential confounders.

Assessment of SEP-1 in Its Entirety

One single-site study involving 48 patients treated before and 110 treated after implementation of the SEP-1 bundle reported higher in-hospital mortality before implementation (27% vs. 15%; P = 0.051) but stated that this difference did not reach the investigators’ targeted level of significance (P < 0.05) (40). The calculated odds ratio of survival with this bundle was 2.18 (95% CI, 0.95 to 5.00) (Table 2). Baseline organ dysfunction was worse in the control group (hypotension, P < 0.01; increased creatinine concentration, P = 0.01). Antibiotics were administered more quickly in intervention than control patients (not statistically significant) (Table 2), but time to antibiotic administration was not reported (Supplement Table 9). Educational and priority care aids (Supplement Table 7) were used in the intervention group but not the control group. The investigators did not report the actual volume of administered fluids; the change in therapy due to serial lactate measurements; or the proportion of patients in the intervention group receiving the focused examination, bedside ultrasonography, or fluid responsiveness assessments (Supplement Table 9). This study did not have low risk of bias (Supplement Table 8).

Level-of-Evidence Assessments

All 16 studies examining serial lactate measurements, a 30-mL/kg fluid infusion, or both reported improved survival in the intervention group. None provided high- or moderate-level evidence free of confounders with low risk of bias to support inclusion of these interventions in SEP-1 (Figure 2). Moreover, the 16 studies met 2013 CMS criteria for low-level evidence (Table 4). The 3 randomized trials assessing fluid responsiveness testing provided low- and moderate-level evidence showing no survival benefit (Table 4). No identified study examined the individual survival benefit of the focused examination or bedside cardiovascular ultrasonography (that is, insufficient evidence). One retrospective observational study with statistically significant baseline imbalances favoring the SEP-1 group reported a borderline survival benefit with the performance measure (low-level evidence) (Table 4).

Figure 2.

Summary of survival, confounders, and risk of bias among serial lactate and 30-mL/kg fluid infusion sepsis studies.

Although the 16 studies testing serial lactate measurements, a 30-mL/kg fluid infusion, or both reported increased survival (odds ratio of survival and 95% CI) in intervention compared with control patients, they all included confounders related to earlier antibiotic administration or adjunctive aid use in intervention patients or had incomplete intervention data. All 16 studies were at high or unknown risk of bias. A dash indicates that the study did not investigate the intervention and therefore data could not be reported. The × indicates that the study reported the actual volume of fluid administered.

* Leisman and colleagues (31) analyzed 3 cohorts of patients representing 3 individual studies. This year indicates the final year of the cohort’s enrollment.

Table 4.

Summary of the Overall Level of Evidence Provided by RCTs and Observational Studies to Support the Individual Hemodynamic Interventions Included in SEP-1 and SEP-1 Overall Based on CMS Criteria^*

Variable	Tested Individual Hemodynamic Interventions						Tested SEP-1 as Mandated by CMS
	Serial Lactate Measurements	30-mL/kg Fluid Infusion	Focused Examination†	Bedside Cardiac Ultrasonography	Fluid Responsiveness Testing
					With Passive Leg Raise	With Fluid Challenges
RCTs	2	None	None	None	2	1	None
Observational studies	7‡	11§	None	None	None	None	1
Level of evidence║	Low	Low	Insufficient	Insufficient	Moderate showing no benefit	Low showing no benefit	Low

CMS = Centers for Medicare & Medicaid Services; RCT = randomized controlled trial; SEP-1 = Severe Sepsis and Septic Shock Early Management Bundle.

^*Hemodynamic interventions included in SEP-1 indicate those not including central venous pressure or oxygen saturation measures.

^†Requires that all of the following be completed: vital signs, cardiopulmonary and skin examination, and capillary refill and peripheral pulse evaluation.

^‡Of these, 3 investigated serial lactate measurements only, whereas 4 investigated both serial lactate measurements and 30-mL/kg fluid infusion.

^§Of these, 7 investigated 30-mL/kg fluid infusion only, whereas 4 investigated both serial lactate measurements and 30-mL/kg fluid infusion.

^║Using CMS’s own categorization of evidence based on the number and quality of RCTs and observational studies (9).

DISCUSSION

Using CMS’s grading criteria from 2013, we found only low-level evidence supporting a survival benefit with serial lactate measurements or a 30-mL/kg fluid infusion. Low- or moderate-level evidence suggests that fluid responsiveness assessment does not improve sepsis survival. No studies examined the survival benefit of the focused examination or bedside cardiovascular ultrasonography. One study with low-level evidence reported a decrease in hospital mortality with SEP-1 in its entirety that bordered on statistical significance.

The absence of high- or moderate-level evidence to support or refute the hemodynamic interventions included in SEP-1 is probably due to the substantial resources required to do rigorous trials that provide such evidence. It took more than 5 years to plan and conduct each of the 3 multicenter randomized controlled trials (ProCESS [Protocolized Care for Early Septic Shock] [14], ARISE [Australasian Resuscitation in Sepsis Evaluation] [15], and ProMISe [Protocolised Management in Sepsis] [16]) that showed that CVP- and ScvO₂-directed hemodynamic support did not improve sepsis outcomes. These trials required many patients and substantial governmental buy-in and financial support.

Consistent with our findings, the 2016 Surviving Sepsis Campaign guidelines graded serial lactate measurements to guide resuscitation and administration of a 30-mL/kg fluid infusion as low-quality evidence. For assessment of fluid status, the guidelines graded a passive leg raise or fluid challenge compared with CVP and ScvO₂ measurements as low-quality evidence (13). The Surviving Sepsis Campaign guidelines do not include the focused examination. Cardiac ultrasonography is described as a “best practice statement,” which is equivalent to being “ungraded” in prior guidelines.

Not only does no high- or moderate-level evidence according to the 2013 CMS criteria support the benefit of SEP-1’s hemodynamic interventions, neither the specifications manuals describing SEP-1 from 2015 through 2017 nor this systematic review contain evidence that their use is actually safe. Relying on lactate measurements without accounting for a patient’s changing volume status in relation to cardiac, pulmonary, and renal function could lead to over- or underresuscitation. The safety of requiring that all septic patients regardless of comorbid conditions receive the same 30-mL/kg fluid infusion early in sepsis is unknown compared with titrated care. Although the fluid volumes administered in the ProCESS, ARISE, and ProMISe trials averaged close to 30 mL/kg (28 mL/kg and 34.7 mL/kg and a total of 1790 mL, respectively), they varied greatly among patients (SD, 21 mL/kg; SD, 20.1 mL/kg; and interquartile range, 1000 to 2500 mL, respectively). These data show that physicians titrate fluid volumes administered to septic patients. The safety of adjusting care on the basis of the focused examination or bedside cardiovascular ultrasonography is unknown, and SEP-1 does not direct actions based on the results of the focused examination. This absence of direction is appropriate because most of the examination’s components (that is, cardiopulmonary, peripheral pulse, skin, and capillary refill evaluations) provide only subjective data and have unproven benefit for guiding therapy. Completing the examination included in the 2015 Through 2017 SEP-1 reporting will consume valuable time and attention of health care providers and could be counterproductive. The 3 studies using passive leg raises or serial fluid challenges were small and did not rule out unsafe effects. The 1 study examining SEP-1 in its entirety provided no data about potential adverse effects associated with the administered hemodynamic interventions.

The United Kingdom’s National Institute for Health and Care Excellence released guidelines in 2016 for early sepsis management consistent with available literature (42). The U.K. guidelines do not stipulate the need for repeated lactate measurements and recommend smaller and more cautious volumes for sequential fluid infusions (two 500-mL infusions). They do not require that clinicians use specified interventions to guide hemodynamic support in persistently unstable patients. Instead, the clinician is allowed to determine which hemodynamic interventions to use in this complicated component of sepsis care that presently lacks a proven scientific approach.

Our literature searches identified 79 systematic reviews, guidelines, and review articles that investigated or described hemodynamic interventions for sepsis. None of these dealt with the bundle of hemodynamic interventions that SEP-1 addressed from 2015 through 2017 that we analyzed here. We identified 9 randomized trials registered in ClinicalTrials.gov that examined SEP-1 components (Supplement Table 10, available at Annals.org). Four of these trials are included in our systematic review (24, 38– 40); the other 5 are ongoing and will be completed within 5 years. Mortality is a primary end point in only 2 of these trials. Both will compare a fixed 30-mL/kg fluid infusion strategy versus a titrated fluid infusion strategy. One will rely on ultrasound guidance (NCT03020407) and the other on clinician judgment (NCT03214913) to titrate fluids.

Several studies report that sepsis bundles encourage early identification of sepsis and rapid administration of appropriate antibiotics and improve outcomes. The survival benefits of using the hemodynamic interventions remain unknown. In an analysis of 1 randomized trial and 8 observational studies of sepsis bundle use, we found that improved survival was associated with more rapid antibiotic use but not with elements of hemodynamic support (43). A more recent analysis of 31 observational studies and 6 randomized trials concluded that more rapid antibiotic administration, but not components of hemodynamic support, was likely associated with reported improvements in survival (12). An analysis of a 3-hour sepsis bundle used in more than 40 000 patients found improved survival with reduced time to antibiotic administration but not with time to fluid infusion, or volume of fluid (11). A recent survey of SEP-1 use in 2015 showed widespread compliance with the measure’s antibiotic requirement but wide variation in use of serial lactate measurements and 30-mL/kg fluid infusions (44). Simplified protocols that promote rapid identification, close monitoring, and administration of proven interventions to septic patients warrant investigation.

Our systematic review is limited by the number and quality of published studies available for analysis. We found no published randomized trials in patients with sepsis that investigated the survival benefits of SEP-1, a fixed 30-mL/kg fluid infusion strategy, bedside cardiovascular ultrasonography, or focused examination versus usual care. No studies adjusted their survival estimates for both antibiotic administration and use of adjunctive aids, and all had high or unknown risk of bias. We evaluated a version of SEP-1 published in effect from in 2015 through 2017; recent versions of SEP-1 recent versions of SEP-1 in effect in 2018 have been simplified but still include serial lactate measurements and a 30-mL/kg fluid infusion as well as an undefined volume status and tissue perfusion assessment and note similar references as prior versions (4). We used the 2013 CMS framework for evaluating evidence. A more recent CMS framework for evaluating evidence related to a composite measure, adopted in 2017, was not available when we began this project.

Our findings do not suggest that the hemodynamic interventions in SEP-1 are contraindicated. Rather, no credible evidence supports their beneficial effect on survival or mandated use for all septic patients. In individual patients, some providers may believe that these interventions help guide care when combined with their own clinical experience and judgment. Also, some patients with severe sepsis and without signs of fluid overload may benefit from resuscitation with an early fluid infusion of 30 mL/kg, or even larger volumes.

After contacting CMS, reviewing the 2015 through 2017 CMS specifications manuals for SEP-1, and doing a thorough review of the available scientific literature, we found no high- or moderate-level evidence free of confounding or bias to support the survival benefit of the SEP-1 hemodynamic interventions. Similar to other clinicians (5–7), we raise concerns that any future basing of hospital accreditation and reimbursement on use of SEP-1 interventions for all septic patients will transform unproven practices into universal care and potentially harm patients in whom they are not indicated. Moreover, requiring the reporting of these interventions as a CMS performance measure will consume personnel and financial resources that might be better directed to other, more effective therapies known to be beneficial and safe in most septic patients.

Appendix

Appendix Figure.

Evidence search and selection.

SEP-1 = Severe Sepsis and Septic Shock Early Management Bundle.

* 1 of these investigated 3 individual cohorts of patients, and each cohort is treated as a separate study in this analysis. Twenty total studies were analyzed.