Effectiveness and Safety of an Emergency Department Code Sepsis Protocol: A Pragmatic Clinical Trial

Abstract

Rationale

Sepsis care delivery—including the initiation of prompt, appropriate antimicrobials—remains suboptimal.

Objectives

This study was conducted to determine direct and off-target effects of emergency department (ED) sepsis care reorganization.

Methods

This pragmatic pilot trial enrolled adult patients who presented from November 2019 to February 2021 to an ED in Utah before and after implementation of a multimodal, team-based “Code Sepsis” protocol. Patients who presented to two other EDs where usual care was continued served as contemporaneous control subjects. The primary outcome was door-to-antimicrobial time among patients meeting Sepsis-3 criteria before ED departure. Secondary and safety outcomes included all-cause 30-day mortality, antimicrobial utilization and overtreatment, and antimicrobial-associated adverse events. Multivariable regression analyses used difference-in-differences methods to account for trends in outcomes unrelated to the studied intervention.

Results

Code Sepsis protocol activation (N = 307) exhibited 8.5% sensitivity and 66% positive predictive value for patients meeting sepsis criteria before ED departure. Among 10,151 patients who met sepsis criteria during the study, adjusted difference-in-differences analysis demonstrated a 13-minute (95% confidence interval = 7–19) decrease in door-to-antimicrobial time associated with Code Sepsis implementation (P < 0.001). Mortality and clinical safety outcomes were unchanged, but Code Sepsis implementation was associated with increased false-positive presumptive infection diagnoses among patients who met sepsis criteria in the ED and increased antimicrobial utilization.

Conclusions

Implementation of a team-based protocol for rapid sepsis evaluation and treatment during the coronavirus disease (COVID-19) pandemic’s first year was associated with decreased ED door-to-antimicrobial time but also increased antimicrobial utilization. Measurement of both patient-centered and off-target effects of sepsis care improvement interventions is essential to comprehensive assessment of their value.

Clinical trial registered with www.clinicaltrials.gov (NCT 04148989).

Prompt initiation of appropriate therapy—particularly antimicrobials—is fundamental to optimal sepsis outcomes (1–7). Although the adoption of care guidelines and sepsis bundles has helped (6, 8–12), many patients with sepsis still do not receive evidence-based care. Evaluation and treatment of patients with potential sepsis is a complex, multistep, and multidisciplinary process. Variation in sepsis care delivery across hospitals and physicians (13, 14) and the influence of resource availability (15–20) and nonpatient contextual cues (21, 22) indicate that the organization of sepsis care is a critical determinant of timely antimicrobial administration. Sepsis is a syndrome rather than a single disease, with a nonspecific presentation that challenges clinicians’ ability to quickly and accurately diagnose patients with sepsis and raises concerns that efforts to accelerate sepsis treatment may drive indiscriminate broad-spectrum antimicrobial use and increase associated adverse events for both patients with sepsis and bystanders (23–26).

Building on the success of dedicated response teams for stroke and other time-dependent medical emergencies (27–30), we hypothesized that reorganizing emergency department (ED) sepsis care around multidisciplinary “Code Sepsis” teams activated early in patients’ ED course would reduce door-to-antimicrobial times by mobilizing personnel, systematizing illness assessment of severity and infection probability, facilitating completion of diagnostic testing, and reducing gaps between treatment decision and antimicrobial infusion. We conducted a pragmatic pilot trial (31) to evaluate the effectiveness and unintended consequences of such a Code Sepsis program.

Preliminary results of this study were presented at the American Thoracic Society International Conference (San Diego, CA) in May 2024.

Methods

Study Design and Setting

This pragmatic pilot trial tested the effects of implementing modified ED sepsis care processes at an urban tertiary care/level I trauma hospital in Utah. Two regional referral/level II trauma hospitals belonging to the same healthcare system continued usual care throughout the study and served as contemporaneous controls. Study hospitals participated in ongoing health system–level sepsis quality improvement efforts and adhered to a shared protocol for sepsis care emphasizing guideline-recommended 3- and 6-hour care bundles (1, 32, 33). Three study periods were identified: a 12-month preintervention analysis period (November 13, 2018, to November 12, 2019), a 3-month wash-in period (November 13, 2019, to February 12, 2020), and a 12-month postintervention analysis period (February 13, 2020, to February 12, 2021; see Figure E1 in the data supplement); the postintervention analysis period substantially overlapped the coronavirus disease (COVID-19) pandemic’s first year. The Intermountain Health Institutional Review Board approved this study with a waiver of informed consent (IRB #1051053). The study was registered on www.clinicaltrials.gov (NCT 04148989) before the initiation of the intervention on November 13, 2019.

Subjects

Adult patients (ages ≥18 yr) who presented to a study ED during the pre- or postintervention analysis period were eligible for inclusion if the study hospital’s trauma team was not activated. The primary analysis (ED sepsis) cohort included the patients’ first eligible ED visit during which the patient met international sepsis-3 consensus criteria before ED departure based on the combination of acute organ failure (Sequential Organ Failure Assessment [SOFA] score ≥2 points above baseline) and both the collection of a body fluid culture and the intravenous administration of an antimicrobial (see Appendix E1) or oral administration of vancomycin, fidaxomicin, oseltamivir, or baloxavir for suspected or diagnosed infection. Encounters during which ED clinician documentation indicated that antimicrobials were given for reasons other than suspected infection (e.g., prophylaxis in cirrhosis-associated gastrointestinal bleeding) were excluded from the sepsis analysis cohort, as were encounters during the study wash-in period and subsequent eligible encounters for previously included patients. Safety outcome analyses were conducted in the ED sepsis (primary analysis) cohort and, to identify spillover effects (26), three additional cohorts:

• 1.All ED patients: all patients presenting to ED, regardless of sepsis status.
• 2.“High sepsis risk at triage”: all patients presenting to ED with indications of possible sepsis at ED triage, defined by a temperature of 38°C or higher and at least two of three criteria for abnormal vital signs (systolic blood pressure [SBP] <90 mm Hg; respiratory rate ≥22 or oxygen saturation ≤85%; and a Glasgow Coma Scale score <15).
• 3.“Sepsis mimics”: all patients presenting to ED who had a primary International Classification of Disease, 10th revision, Clinical Modification (ICD-10-CM) (34) discharge diagnosis code for heart failure or venous thromboembolism and no diagnosis codes for infection (see Appendix E2 for eligible ICD-10-CM codes).

Each of these three cohorts was restricted to an individual patient’s first eligible encounter for that cohort outside the wash-in period and, to avoid outcome misattribution, excluded patients who left the ED without treatment or who received antimicrobials in the 6 hours preceding ED arrival.

Intervention

The trial intervention (“Code Sepsis” protocol) was a restructuring of ED evaluation and care processes for patients with possible sepsis implemented at the intervention hospital on November 13, 2019. The Code Sepsis protocol incorporated six phases:

• 1.Screening: Patients with potential sepsis were identified by ED nurses and clinicians during triage using staged assessment criteria incorporating alternative diagnoses, infection suspicion, evidence of new organ failure, and clinician judgment (see Figures E2 and E3).
• 2.Code Sepsis activation: The Code Sepsis team (physician, one to two nurses, phlebotomist, patient care technician, pharmacist [if available], radiology technician) was called to the patient’s room within 5 minutes.
• 3.Patient evaluation: Team members performed patient evaluation tasks (see Figure E4) aided by a purpose-built patient evaluation order set, including bedside assessment, placement of intravenous line, point-of-care testing, laboratory and microbiological sample collection, and X-ray imaging (see Figure E5).
• 4.Decision huddle: The physician, nurse, and pharmacist met within 25 minutes to decide whether sepsis was present or absent (see Figure E6). If the diagnosis remained indeterminate, the team reconvened within 30 minutes to finalize their determination.
• 5.Treatment initiation: If sepsis was diagnosed, within 10 minutes, the physician and pharmacist selected and ordered antimicrobials, and the pharmacist and nurse coordinated administration.
• 6.Antimicrobial stewardship: An antimicrobial stewardship pharmacist reassessed antimicrobial-treated Code Sepsis patients on subsequent days and communicated recommendations to the treating team.

(For details of Code Sepsis protocol development, design, testing, education, and implementation, see the Supplemental Methods and Table E1).

Data Collection

Eligible patients were identified, and demographic and clinical data collected through the study health system’s electronic data warehouse (35). Postdischarge mortality data were obtained from the Social Security Death Index and Utah State death records through preexisting linkages. Values for the von Walraven weighted Elixhauser comorbidity index, SOFA score, and acute physiology score (APS) were calculated as previously described (17, 36–39). Potential antimicrobial-associated adverse events were identified using discharge diagnosis codes (see Appendix E3) adapted from Hohl and colleagues (40). Research coordinators used structured methods to identify the ED-diagnosed infection source; adjudicate the final presence and source of infection (Supplemental Methods); and verify, correct, and complete electronically captured data (4, 17, 41).

Exposures and Outcomes

The study period during which patients presented to the ED was the primary exposure. The primary outcome was door-to-antimicrobial initiation time among patients who met sepsis criteria in the ED, measured from ED arrival. The key clinical outcome was 30-day mortality among patients with sepsis. Other secondary outcomes measured in the ED sepsis cohort included a binary indicator for antimicrobial initiation within 3 hours of ED arrival (2–4), hospital and 1-year mortality, hospital charges, and hospital length of stay. Safety outcomes included 24-hour antimicrobial utilization within the “high sepsis risk at triage” and all ED patient cohorts; the summed spectrum score (42) for all antibiotics administered within 24 hours of ED arrival in the ED-sepsis and all–ED patient cohorts; false-positive presumptive infection diagnosis among patients with sepsis (41); ED antimicrobial treatment among sepsis-mimic patients; and new-onset Clostridioides difficile colitis occurring between 72 hours to 90 days after ED arrival, anaphylaxis within 72 hours of ED arrival, and antimicrobial-associated adverse events identified from the index hospitalization’s ICD-10-CM discharge diagnosis codes (40) among both the ED-sepsis and all–ED patient cohorts. Detailed definitions of secondary and safety outcomes are provided in the online Supplemental Methods and Appendix E3.

Missing Data

Study exposure and primary and secondary outcome covariates were nonmissing. Data were nonmissing for all outcomes other than hospital charges; patients who were missing data for hospital charges were excluded from analyses of this outcome. Because covariate data were complete for more than 95% of patients for all safety analyses, multivariable safety outcome analyses used complete case analysis. Sensitivity analyses using multiply imputed data were conducted for analyses where any eligible subjects had incomplete data (Supplemental Methods).

Statistical Analysis

Unadjusted between-groups comparisons used chi-square tests or Wilcoxon-Mann-Whitney tests as appropriate. The primary analysis used a multivariable γ regression to compare door-to-antimicrobial time before versus after Code Sepsis implementation at the intervention hospital, incorporating patient-level covariates and contemporaneous control data within a difference-in-differences framework to evaluate the effect of the intervention on the primary outcome while accounting for patient-level confounders and hospital-level trends in the outcome over time unrelated to the intervention. We tested the parallel trends assumption—the assumption that, in the absence of the intervention, the trends in the outcome would have been the same at both the intervention and control hospitals—using the method of Callaway and Sant’Anna (43). Adjustment variables for the primary and secondary analyses (age; sex; presentation from long-term care facility; comorbidity score; ED arrival by ambulance; preferred language; first-available temperature [<36°C, 36–38°C, or >38°C] and SBP; APS; ED SOFA score, and the ED-diagnosed source of infection [pulmonary, urinary tract, intrabdominal, skin/soft tissue, or other/multiple/unknown]) and the safety analyses (age; sex; comorbidity score; insurance type [private/commercial; Medicaid; Medicare; worker’s compensation; or uninsured/self-pay]; first available temperature and SBP; and APS score) were selected a priori using directed acyclic graphs (see Figures E7 and E8) that were created on the basis of literature review and expert input (44–46).

Prespecified exploratory analyses were used to examine the heterogeneity of treatment effect for the primary outcome by age (<65 vs. ≥65 yr), sex, presence of hypotension (SBP <90 mm Hg or mean arterial pressure <65 mm Hg) on ED arrival, ED arrival by ambulance, source of infection, and APS quartile. In the key sensitivity analysis, we used covariate-adjusted interrupted time series analysis (47–49) to investigate the intervention’s association with the primary outcome without reliance on the parallel trends assumption (Supplemental Methods). Additional prespecified sensitivity analyses targeted safety analyses that were affected by missing data as described earlier and repeated the primary analysis among 1) primary analysis cohort patients who also met criteria for having indications of possible sepsis at ED triage and 2) all ED encounters that met sepsis criteria while in the ED, using generalized linear models with a random effect for patient to account for nonindependence between multiple encounters for a single patient. A post hoc sensitivity analysis repeated the evaluation of antimicrobial-associated adverse events, using an expanded list of qualifying ICD-10-CM diagnosis codes (Appendix E3).

A two-sided P value ≤0.05 was considered statistically significant. Assuming that 1,650 intervention ED patients and 2,200 control ED patients would meet sepsis criteria annually, we estimated that we would have 90% power to detect a difference-in-differences change of ≥16 minutes in door-to-antimicrobial time on Code Sepsis implementation, assuming a baseline door-to-antimicrobial time of 170 ± 75 minutes and that 5–10% of patients would receive antimicrobials within 1 hour of ED arrival. Secondary and safety analyses were not adjusted for multiple comparisons and should be considered hypothesis generating. Analyses were performed in R Version 4.0.3 (R Foundation for Statistical Computing) and Stata Version 16.1 (StataCorp).

Results

Of 313,758 adult patient encounters at study EDs during the preintervention or postintervention analysis periods, 13,040 encounters (3.7%) met sepsis criteria while in the ED (see Table E2), and 10,151 unique patients were included in the primary analysis cohort (Figure 1). Intervention ED patients with sepsis were younger, were more often female, were more ethnically diverse, and had a higher proportion of nonpulmonary infections diagnosed in the ED and higher illness severity than patients with sepsis enrolled in the control EDs (Table 1). In a comparison of the pre- and postintervention sepsis populations, postintervention patients were less often female and had more pulmonary infections diagnosed. Control ED patients with sepsis had fewer comorbidities during the postintervention period, whereas intervention ED patients with sepsis had higher illness severity in the postintervention period. COVID-19 prevalence among patients with sepsis was similar between the intervention (14.4%) and control (17%) EDs. Both the intervention and control EDs treated 16% of their patients who had COVID-19 with antimicrobials. (For the characteristics of patients included in safety and sensitivity analyses, see Tables E2–E5).

Figure 1.

Consolidated Standards of Reporting Trials (CONSORT)-style patient inclusion and exclusion diagram. ED = emergency department.

Table 1.

Demographic and clinical characteristics of patients with sepsis included in the primary analysis cohort

Characteristic	Intervention ED				Control EDs				P Value for Intervention vs. Control ED
	Preintervention (n = 2,384)	Postintervention (n = 2,075)	P Value	Overall (n = 4,459)	Preintervention (n = 2,976)	Postintervention (n = 2,716)	P Value	Overall (n = 5,692)
Female sex, n (%)	1,276 (53.5)	1,037 (50.0)	0.02	2,313 (51.9)	1,440 (48.4)	1,229 (45.3)	0.02	2,669 (46.9)	<0.001
Age, yr, median (IQR)	64 (48–76)	65 (49–75)	0.44	64 (49–76)	69 (55–79)	69 (54–78)	0.31	69 (55–79)	<0.001
Race/ethnicity, n (%)			0.16				0.009		<0.001
Hispanic/Latino	331 (13.9)	294 (14.2)	—	625 (14.0)	207 (7.0)	249 (9.2)	—	456 (8.0)	—
Non-Hispanic/Latino White	1,851 (77.6)	1,572 (75.8)	—	3,423 (76.8)	2,638 (88.6)	2,352 (86.6)	—	4,990 (87.7)	—
Non-Hispanic/Latino other race	202 (8.5)	209 (10.1)	—	411 (9.2)	131 (4.4)	115 (4.2)	—	246 (4.3)	—
English as preferred language, n (%)	2,204 (92.4)	1,866 (89.9)	0.003	4,070 (91.3)	2,854 (95.9)	2,567 (94.5)	0.02	5,421 (95.2)	<0.001
Presented from long-term care facility, n (%)	144 (6.0)	179 (8.6)	0.001	323 (7.2)	195 (6.6)	147 (5.4)	0.08	342 (6.0)	0.01
Arrival to ED by ambulance, n (%)	803 (33.7)	800 (38.6)	0.001	1,603 (35.9)	1,011 (34.0)	982 (36.2)	0.09	1,993 (35.0)	0.34
Weighted Elixhauser comorbidity score, median (IQR)	11 (0–24)	11 (0–23)	0.65	11 (0–23)	12 (2–23)	10 (0–22)	0.002	11 (1–22)	0.92
First available clinical parameters
Systolic blood pressure, mm Hg, median (IQR)	127 (109–145)	126 (109–145)	0.94	126 (109–145)	128 (110–146)	128 (110–145)	0.83	128 (110–146)	0.008
Respiratory rate, breaths/min, median (IQR)	20 (18–22)	20 (18–24)	<0.001	20 (18–24)	18 (16–22)	20 (18–23)	<0.001	19 (17–22)	<0.001
Temperature, °C, median (IQR)	37.6 (36.7–38.5)	37.2 (36.3–38.3)	<0.001	37.5 (36.5–38.4)	37.2 (36.6–38.2)	37.1 (36.5–38.1)	0.002	37.1 (36.5–38.2)	<0.001
WBC count, 1,000/μl, median (IQR)*	11.6 (8.1–16.0)	11.7 (7.8–16.3)	0.82	11.6 (8.0–16.1)	11.6 (8.2–16.0)	11.4 (7.7–16.1)	0.09	11.5 (8.0–16.0)	0.79
Lactate checked & ≥2 mmol/dl, n (%)	894 (37.5)	897 (43.2)	<0.001	1,791 (40.2)	1,105 (37.1)	1,012 (37.3)	0.94	2,117 (37.2)	0.002
ED-diagnosed source of infection, n (%)			0.003				<0.001		<0.001
Pulmonary	653 (27.4)	641 (30.9)	—	1,294 (29.0)	964 (32.4)	1,005 (37.0)	—	1,969 (34.6)	—
Urinary	745 (31.2)	578 (27.9)	—	1,323 (29.7)	990 (33.3)	728 (26.8)	—	1,718 (30.2)	—
GI/abdominal	217 (9.1)	233 (11.2)	—	450 (10.1)	226 (7.6)	245 (9.0)	—	471 (8.3)	—
Skin/soft tissue	244 (10.2)	191 (9.2)	—	435 (9.8)	175 (5.9)	162 (6.0)	—	337 (5.9)	—
Other/multiple/unknown	525 (22.0)	432 (20.8)	—	957 (21.5)	621 (20.9)	576 (21.2)	—	1,197 (21.0)	—
Positive SARS-CoV-2 assay, n (%)†	N/A	299 (14.4)	N/A	299 (6.7)	N/A	461 (17.0)	N/A	461 (8.1)	0.008
Acute physiology score, median (IQR)	8 (5–11)	9 (6–12)	<0.001	8 (6–12)	7 (4–10)	7 (5–10)	0.37	7 (5–10)	<0.001
ED SOFA score, median (IQR)	3 (2–5)	4 (3–5)	<0.001	4 (2–5)	3 (2–5)	3 (2–5)	0.25	3 (2–5)	<0.001
ED length of stay, min, median (IQR)	270 (217–340)	279 (225–350)	<0.001	275 (220–346)	229 (185–280)	233 (189–288)	0.009	231 (187–284)	<0.001

Definition of abbreviations: ED = emergency department; GI = gastrointestinal; IQR = interquartile range; N/A = not applicable; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2; SOFA = Sequential Organ Failure Assessment; WBC = white blood cell.

^*Data missingness: WBC count, n = 114.

^† Includes one or more positive SARS-CoV-2 polymerase chain reaction or antigen assay occurring during the index encounter or within 14 days preceding ED arrival.

During the postintervention period, the Code Sepsis protocol was activated for 307 of 64,170 (0.5%) adult ED encounters in the intervention ED, including 203 (8.5%) of 2,396 encounters that met sepsis criteria before ED departure. Of 307 Code Sepsis protocol activation encounters, 203 (66.1%) met sepsis criteria before ED departure, and 265 (86.3%) had infection present on final adjudication (Table 2).

Table 2.

Code sepsis protocol process parameters in the intervention ED

Process Metric	Result
Activation rate among all ED encounters, n (%)	307/64,477 (0.5)
Activation rate among ED sepsis encounters (sensitivity), n (%)	203/2,396 (8.5)
Positive predictive value for sepsis in the ED, n (%)	203/307 (66.1)
Positive predictive value for infection presence on final adjudication, n (%)	265/307 (86.3)
Time (minutes) from ED arrival to Code Sepsis (n = 307), median (IQR)	34 (19–69)

Definition of abbreviations: ED = emergency department; IQR = interquartile range.

Among patients who met sepsis criteria after presenting to the intervention ED, median door-to-antimicrobial time was 160 minutes (IQR = 114–219) preintervention versus 146 minutes (IQR = 102–205) postintervention, with 60% receiving antimicrobials within 3 hours of ED arrival preintervention and 68% achieving this target postintervention. The primary adjusted difference-in-differences analysis demonstrated a decrease in average door-to-antimicrobial time for patients with sepsis of 12.8 minutes (95% CI = 7.0–18.6; P < 0.001) associated with Code Sepsis implementation and a corresponding 37% (95% CI = 14–64%; P = 0.001) increase in the adjusted odds of antimicrobial administration within 3 hours of ED arrival. Other secondary outcomes, including 30-day mortality (difference-in-differences adjusted OR [aOR] = 0.90; 95% CI = 0.68–1.19), did not change after Code Sepsis implementation (Table 3).

Table 3.

Primary and secondary outcomes for ED patients with sepsis

Outcome	Intervention ED		Control EDs		Unadjusted Difference-in- Differences Analysis		Adjusted Difference-in- Differences Analysis
	Preintervention (n = 2,384)	Postintervention (n = 2,075)	Preintervention (n = 2,976)	Postintervention (n = 2,716)	Change or Odds Ratio (95% CI)	P Value	Adjusted change or Odds Ratio (95% CI)	P Value
Door-to-antimicrobial time, min, median (IQR)	160 (114–219)	146 (102–205)	139 (102–176)	140 (105–177)	−12.5 (−18.7, −6.4)	<0.001	−12.8 (−18.6, −7.0)	<0.001
Door-to-antimicrobial time ≤3 h, n (%)	1,438 (60.3)	1,410 (68.0)	2,306 (77.5)	2,125 (78.2)	OR = 1.34 (1.12, 1.59)	0.001	aOR = 1.37 (1.14, 1.64)	0.001
In-hospital mortality, n (%)	113 (4.7)	148 (7.1)	149 (5.0)	190 (7.0)	OR = 1.08 (0.77, 1.51)	0.65	aOR = 0.76 (0.53, 1.10)	0.14
Mortality, 30 d, n (%)	201 (8.4)	262 (12.6)	280 (9.4)	332 (12.2)	OR = 1.17 (0.91, 1.51)	0.23	aOR = 0.90 (0.68, 1.19)	0.45
Mortality, 1 yr, n (%)	467 (19.6)	483 (23.3)	603 (20.3)	628 (23.1)	OR = 1.05 (0.87, 1.27)	0.60	aOR = 0.85 (0.68, 1.05)	0.13
Hospital charges (US$1000s), median (IQR)*	7.3 (3.6–14.3)	15.0 (7.0–32.7)	7.6 (3.8–14.7)	15.2 (7.6–30.1)	+3.9 (+1.3, +6.6)	0.003	+0.3 (−1.4, +2.1)	0.71
Hospital length of stay, d, median (IQR)	2.7 (1.0–5.0)	3.0 (1.1–6.1)	2.7 (1.1–4.8)	2.9 (1.2–5.6)	+0.4 (−0.01, +0.8)	0.097	−0.1 (−0.4, +0.1)	0.19

Definition of abbreviations: aOR = adjusted odds ratio; CI = confidence interval; ED = emergency department; IQR = interquartile range; OR = odds ratio.

^*Excludes 80 patients with missing data for hospital charges.

The sensitivity analysis that used a covariate-adjusted interrupted time series incorporating contemporaneous control data demonstrated a 19.3-minute (95% CI = 5.0–33.6) decrease in door-to-antimicrobial time (P = 0.007) associated with Code Sepsis implementation (Figure 2). Other sensitivity analyses including all ED sepsis encounters and patients with high sepsis risk on the basis of data from ED triage also yielded results similar to those of the primary analysis (see Table E6). There was no statistical heterogeneity for Code Sepsis implementation’s association with door-to-antimicrobial time on the basis of the presence of hypotension, infection source, illness severity, and other tested factors (Figure 3).

Figure 2.

Adjusted segmented regression analysis for door-to-antimicrobial time. Thin lines represent unadjusted 2-week average of door-to-antimicrobial times for control and intervention emergency departments (EDs). Values are anchored to the midpoint of the 2-week period that they represent. Thick lines represent estimated adjusted door-to-antimicrobial time (with shading indicating the 95% confidence interval) for a typical patient, defined by the median or most common categorical value for model covariates. Study Week 0 is the week during which the Code Sepsis protocol was implemented at the intervention hospital. The postintervention wash-in period is indicated in gray.

Figure 3.

Heterogeneity of treatment effect for Code Sepsis intervention on door-to-antimicrobial time. Adj. = adjusted; APS = acute physiology score; CI = confidence interval; ED = emergency department; GI = gastrointestinal; mult = multiple; unk = unknown.

Code Sepsis implementation was associated with increased utilization of antimicrobials within 24 hours of ED arrival among all ED patients (aOR = 1.08; 95% CI = 1.02–1.14; P = 0.01) and among the subset of patients in the “high sepsis risk at triage” cohort (aOR = 1.48; 95% CI = 1.10–1.98; P = 0.009). The adjusted odds of a false-positive presumptive diagnosis of infection (infection absent on final adjudication) among patients who met sepsis criteria in the ED were 38% higher (95% CI = 2–87; P = 0.04) in the intervention ED after Code Sepsis implementation, but the proportion of sepsis-mimic patients receiving antimicrobials did not change (Table 4; see Table E7). There was no significant adjusted difference-in-differences association between Code Sepsis implementation and the incidence of possible antimicrobial-related adverse events or new-onset C. difficile infection (Tables 4 and E7). Sensitivity analyses that used an expanded definition for antimicrobial-related adverse events (see Table E8) or multiple imputation for missing covariate data (see Table E9) yielded similar results. Among ED patients with sepsis at the intervention hospital, new-onset anaphylaxis occurred within 72 hours of ED arrival in 2/2,384 (0.08%) patients preintervention and 3/2,075 (0.14%) patients postintervention, compared with 1/2,976 (0.03%) and 2/2,716 (0.07%) patients, respectively, at the control hospitals.

Table 4.

Results of adjusted difference-in-differences analysis for safety outcomes

Outcome	Eligible Subjects		Adjusted Change or Odds Ratio (95% CI)	P Value
	Included in Analysis	Missing Covariate Data
Antimicrobial ≤24 h from ED arrival
All ED patients	173,879	6,523 (3.6%)	aOR = 1.08 (1.02–1.14)	0.01
“High sepsis risk at triage” patients	4,936	0	aOR = 1.48 (1.10–1.98)	0.009
Unnecessary antimicrobials in ED
Patients who met sepsis criteria in the ED with infection absent on final adjudication	10,151	0	aOR = 1.38 (1.02–1.87)	0.04
Receipt of antimicrobial in ED among “sepsis-mimic” cohort patients	3,850	122 (3.1%)	aOR = 0.77 (0.42–1.41)	0.39
Totaled spectrum scores for unique antibiotics received within 24 h of ED arrival*
ED patients with sepsis	10,027	0	+0.32 (0.06–0.58)	0.02
All ED patients	24,026	485 (2.0%)	+0.16 (0.02–0.29)	0.02
“High sepsis risk at triage” cohort patients	3,380	0	+0.02 (−0.43–0.48)	0.93
Possible antimicrobial-related AE
ED patients with sepsis	10,151	0	aOR = 1.09 (0.68–1.75)	0.73
All ED patients	173,879	6,523 (3.6%)	aOR = 1.01 (0.77–1.32)	0.97
New-onset Clostridioides difficile infection
ED patients with sepsis	10,038	0	aOR = 1.03 (0.49–2.17)	0.94
All ED patients	173,593	6,522 (3.6%)	aOR = 1.37 (0.90–2.08)	0.14

Definition of abbreviations: AE = adverse effect; aOR = adjusted odds ratio; CI = confidence interval; ED = emergency department.

^*Patients who did not receive an antimicrobial agent within 24 hours of ED arrival or who received only antiviral and/or antifungal agents during this window were excluded from analyses of antibiotic spectrum.

Discussion

In this pragmatic pilot trial, the adoption of a Code Sepsis protocol for ED sepsis care during the COVID-19 pandemic was associated with a small but significant decrease in door-to-antimicrobial time for the population of patients who met sepsis criteria in the ED. However, Code Sepsis implementation was also associated with increased overdiagnosis of infection among patients in this group and with more ED-wide antimicrobial utilization, especially among patients who were likely to be viewed as high risk for sepsis at ED triage. Clinical outcomes, including patient-centered safety outcomes, were unchanged after Code Sepsis implementation. Despite analyses incorporating contemporaneous usual care controls, we cannot exclude the possibility that the COVID-19 pandemic influenced trial findings.

Although numerous studies have reported on team-based sepsis care reorganization (50–54), rigorous measurement of the impact of systematic reorganizations of healthcare delivery has been lacking. Cluster-randomized trials of either parallel or stepped wedge design—generally the gold standard for such evaluations—are costly and complex to conduct. For this pragmatic pilot trial involving the reorganization of ED sepsis care at a single referral center, we used an array of methods to maximize the strength of causal inference. Most important, the deployment of difference-in-differences methods, formal validation of the parallel trend assumption, and sensitivity analysis with interrupted time series analyses also integrating contemporaneous control data reduce, but do not eliminate, the risk that we are misattributing to Code Sepsis implementation changes in study outcomes that actually resulted from other factors affecting sepsis care delivery, such as the COVID-19 pandemic. Other key features of our design and analysis that support causal inference include careful adjustment for prespecified potential confounders, the inclusion of all patients exposed to each care arm (avoiding potential selection bias), and careful prespecification of outcomes defined using objective data (minimizing risk of ascertainment bias in an unblinded study).

A common criticism of efforts to accelerate antibiotic initiation for suspected sepsis is the risk of inadvertently increasing unnecessary or unnecessarily broad-spectrum antibiotics and thereby causing patient- and population-level antibiotic adverse effects, including C. difficile infection, allergic reactions and other antibiotic treatment complications, and antibiotic resistance (24, 25, 55, 56). To date, however, these concerns have had little empirical support, with recent studies suggesting that neither adoption of sepsis protocol mandates (57) nor improvements in antimicrobial timing (58) were associated with increased antimicrobial utilization. Our findings, by contrast, do suggest that Code Sepsis implementation increased antimicrobial utilization and infection overdiagnosis. Given relatively low protocol utilization, nonintended spillover effects on patients who were not targeted by the intervention (26) were likely substantially due to implementation and education impacts on care systems (e.g., ED care team communication) and clinician practice patterns (e.g., antibiotic initiation thresholds) rather than direct treatment by means of the Code Sepsis protocol. It is important to note that we observed no evidence that measured patient-centered safety outcomes were affected by Code Sepsis implementation. However, we did not assess the development of new antimicrobial resistance or adverse events that were inaccessible through discharge diagnosis codes, and our analysis may be underpowered to detect small increases in antibiotic-associated adverse events. Given studies suggesting that as many as 20% of hospitalized patients receiving unnecessary antibiotics suffer an antibiotic-associated adverse drug event (59), additional research is needed to further investigate the off-target effects of sepsis care improvement interventions and to quantify the benefit of earlier antibiotic administration relative to the consequences of increased antimicrobial use.

The Code Sepsis activation rate was low, in part because of the COVID-19 pandemic’s interference with implementation efforts, potentially limiting the impact of our care redesign intervention. We did not capture factors that influenced Code Sepsis protocol utilization. Increased utilization of the Code Sepsis protocol might have augmented the implementation’s apparent effect on our antimicrobial delivery process outcome and thereby increased the likelihood of a measurable impact on sepsis mortality. However, it is also possible that doing so could have led to more dramatic increases in antimicrobial utilization, overtreatment, and adverse effects on downstream patient-centered outcomes. It is interesting that Code Sepsis activation exhibited a high positive predictive value for confirmed infection. Although this finding is reassuring in terms of resource utilization and the intervention’s direct impacts on safety outcomes, it further emphasizes the importance of remaining vigilant for unintended and bystander effects during implementation of change for systems of care.

Our study has several potential limitations. Despite the strategies discussed earlier, we cannot exclude the possibility that the observed findings result from changes in practice unrelated to the study intervention or that our effect estimates are subject to unmeasured confounding. In particular, the first two waves of the COVID-19 pandemic coincided with our postimplementation analysis period. The possibility that study findings result from differential pandemic impacts on study sites rather than the studied intervention remains an important consideration. Sites’ unified pandemic responses within an integrated health system alongside objectively similar severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) prevalence and treatment patterns reduce, but do not eliminate, the risk that unmeasured differences between control and intervention hospitals prevented difference-in-differences analyses from controlling for pandemic-related secular trends. The effects of Code Sepsis implementation may also have differed in an environment that experienced less disruption and resource strain.

While our Code Sepsis protocol is conceptually generalizable, we tested its implementation in a single ED and were unable to investigate the relative impact of either individual protocol components or education associated with protocol implementation. Differences in available resources or practice context or alternative activation mechanisms, team structures, or intervention protocols may have yielded different results. Although study hospitals shared sepsis protocols and quality improvement strategies throughout the study, control hospitals’ better baseline door-to-antimicrobial time potentially reduced motivation for practice improvement. Finally, we designed the Code Sepsis protocol and its evaluation to focus on improving antimicrobial initiation and did not target or measure impacts on fluid resuscitation and other components of sepsis care bundles.

Conclusion

Despite low utilization rates, implementation of a team-based Code Sepsis process early in the COVID-19 pandemic was associated with shorter door-to-antimicrobial times across all patients who met sepsis criteria in the ED. Antibiotic utilization also increased, but there was no change in measured patient-centered safety outcomes. Our findings demonstrate the importance of prospectively measuring patient-centered outcomes and off-target effects alongside process outcomes when evaluating sepsis care improvement interventions.