Risk of Misleading Conclusions in Observational Studies of Time-to-Antibiotics and Mortality in Suspected Sepsis

Abstract

Background

Influential studies conclude that each hour until antibiotics increases mortality in sepsis. However, these analyses often (1) adjusted for limited covariates, (2) included patients with long delays until antibiotics, (3) combined sepsis and septic shock, and (4) used linear models presuming each hour delay has equal impact. We evaluated the effect of these analytic choices on associations between time-to-antibiotics and mortality.

Methods

We retrospectively identified 104 248 adults admitted to 5 hospitals from 2015–2022 with suspected infection (blood culture collection and intravenous antibiotics ≤24 h of arrival), including 25 990 with suspected septic shock and 23 619 with sepsis without shock. We used multivariable regression to calculate associations between time-to-antibiotics and in-hospital mortality under successively broader confounding-adjustment, shorter maximum time-to-antibiotic intervals, stratification by illness severity, and removing assumptions of linear hourly associations.

Results

Changing covariates, maximum time-to-antibiotics, and severity stratification altered the magnitude, direction, and significance of observed associations between time-to-antibiotics and mortality. In a fully adjusted model of patients treated ≤6 hours, each hour was associated with higher mortality for septic shock (adjusted odds ratio [aOR]: 1.07; 95% CI: 1.04–1.11) but not sepsis without shock (aOR: 1.03; .98–1.09) or suspected infection alone (aOR: .99; .94–1.05). Modeling each hour separately confirmed that every hour of delay was associated with increased mortality for septic shock, but only delays >6 hours were associated with higher mortality for sepsis without shock.

Conclusions

Associations between time-to-antibiotics and mortality in sepsis are highly sensitive to analytic choices. Failure to adequately address these issues can generate misleading conclusions.

We document how common analytic choices to characterize associations between time-to-antibiotics and mortality for suspected sepsis can generate misleading conclusions. In optimized models, hourly delays in antibiotics were associated with higher mortality for septic shock but not sepsis without shock.

Graphical Abstract

Graphical Abstract.

This graphical abstract is also available at Tidbit: https://tidbitapp.io/tidbits/risk-of-misleading-conclusions-in-observational-studies-of-time-to-antibiotics-and-mortality-5f9a2c20-dfed-42e6-8bef-6ddd4b8c2d5d

Sepsis is a leading cause of death and disability worldwide [1, 2]. Quality-improvement initiatives, best-practice guidelines, and quality metrics emphasize the necessity of treating patients with possible sepsis with broad-spectrum antibiotics as quickly as possible, ideally within 1 hour of recognition, in order to reduce mortality [3–5]. These recommendations are based on observational studies suggesting that each additional hour until receipt of antibiotics is associated with increased mortality in patients with sepsis [6–11].

Recommendations to immediately treat all patients with sepsis with broad-spectrum antibiotics are controversial [12–20], however, because one-third or more of patients treated for possible sepsis turn out to have noninfectious conditions or viral infections [21–23]. These patients risk the potential adverse effects of antibacterial agents without their potential benefits [24–28].

Critical appraisals of observational studies on the association between time-to-antibiotics and mortality have identified several concerns, including the following: (1) limited adjustment for potential confounders, (2) inclusion of outlier patients with very long delays until antibiotics, (3) failure to differentiate between sepsis with and without shock, and (4) use of linear models that imply that each additional hour until antibiotics has an equal effect (Table 1) [12, 15, 37–39]. Failure to adequately address these issues may lead to misleading conclusions about the association between time-to-antibiotics and mortality.

Table 1.

Four Analytic Concerns With Observational Studies of the Association Between Mortality and Time-to-Antibiotics in Sepsis

No.	Concern	How This May Create Bias	Analytic Decisions and Relevant Data in Prior Studies
1	Insufficient confounding adjustment	In practice, time-to-antibiotics is not random. Patients with higher perceived mortality risk typically receive treatment earlier. Patient who receive antibiotics later may have different baseline characteristics and comorbidity profiles than those who receive early antibiotics. Inadequately adjusting for confounders may bias the inferred association with mortality.	Most prior studies have adjusted for few covariates, even though many factors influence both the decision to give antibiotics and mortality risk. Some influential studies did not include age [6–8], sex [6–8, 10], or race [6–9, 29, 30]. Others did not include comorbid diseases [6, 8] or only included an aggregate comorbidity score rather than modeling distinct individual comorbidities [9–11, 31].
2	Inclusion of patients with very long intervals until antibiotics in the analytic cohort	Patients treated >6 h after ED arrival are unusual. Current practice is to administer antibiotics early for suspected sepsis, so the best marginal evidence is likely from patients treated close to the recommended time frames (equipoise). Ordinary least-squares regression is highly sensitive to outlier data (leverage).	Most studies report that 75–80% of their patients received antibiotics before 6 h, but include patients receiving antibiotics well beyond 6 h in regression models that presuppose a uniform hourly odds ratio of mortality [10, 11, 31, 32]. Contrarily, in studies that create models with a separate effect for each hourly interval, there is often no significant increase in mortality for patients without shock until intervals beyond 3–5 h [11, 29, 31, 33].
3	Failure to differentiate between sepsis with shock and sepsis without shock	Sepsis encompasses a wide spectrum of disease severity. The presence of shock on arrival is known to affect both time-to-antibiotics and mortality. Combining sepsis and septic shock may inappropriately extrapolate the importance of early antibiotics from patients with imminently life-threatening illness to patients with less severe illness.	Prior studies used cohorts with very different proportions of patients in shock, ranging from 0% [30] to 100% [6, 7, 34]. Many studies created their hourly associations for “sepsis” using an unweighted, mixed cohort [8–11]. Examining subgroup analyses for the patients with and without shock, when reported, reveals dramatically different associations [9, 10, 35].
4	Using models that assume each hour until antibiotics has a single, uniform effect on mortality	Presenting linearized estimates for potentially nonlinear relationships may create the misleading impression that every 1-h delay until antibiotic treatment correlates with a constant increase in log odds of mortality, no matter which hour is examined; eg, 3–4 h from ED arrival has the same change in log odds of mortality as 21–22 h. Including patients with very long antibiotic delays necessitates an even stronger assumption.	Prior studies that examined the association between mortality and each hour until antibiotics separately (without assuming a constant change per hour) have often found nonlinearity in the trend [8, 9, 11, 31]. Other studies have described a J-shaped curve [36].

Abbreviation: ED, emergency department.

We therefore undertook a systematic evaluation of the impact of each of these analytic decisions on the estimated association between time-to-antibiotics and mortality using detailed clinical data from a large multihospital cohort.

METHODS

Study Design, Population, and Data Source

We conducted a retrospective cohort study using electronic health record data for all adults (≥18 y) admitted via the emergency departments (EDs) of 5 hospitals within the Mass General Brigham (MGB) system between June 2015 and August 2022, including 2 academic centers (Massachusetts General Hospital and Brigham and Women's Hospital) and 3 community hospitals (Brigham and Women's Faulkner Hospital, Newton-Wellesley Hospital, and Salem/North Shore Medical Center). The study was approved by the MGB institutional review board with a waiver of informed consent.

Definitions of Suspected Infection, Sepsis, and Septic Shock

We defined “suspected infection” as the collection of 1 or more blood cultures (regardless of result) and the administration of intravenous (IV) antibiotics within 24 hours from ED arrival [11, 32, 36]. We defined “suspected sepsis” as suspected infection plus organ dysfunction within 12 hours of ED arrival, defined as 1 or more of the following: lactate >2.0 mmol/L, initiation of noninvasive or invasive mechanical ventilation, creatinine >2.0 mg/dL and an increase of ≥50% from baseline, total bilirubin >2.0 mg/dL and an increase of ≥50% from baseline, or platelets <100 000/µL and a decrease of ≥50% from baseline. We defined “suspected septic shock” as suspected infection plus either hypotension (systolic blood pressure [SBP] < 90 mmHg) or lactate ≥4.0 mmol/L within 12 hours of ED arrival. Patients were only assigned to the highest severity cohort (cohorts were mutually exclusive).

Exclusion criteria included comfort measures or expiration within 6 hours of ED arrival; transfer from outside hospitals; admission to psychiatry or obstetric services; missing complete vital signs or missing creatinine, platelet count, or white blood count (WBC) result (based on laboratory collection time) within 12 hours of ED arrival; or receipt of oral or IV antibiotics prior to ED arrival (Supplementary Figure 1).

Exposure and Outcome

The exposure of interest was time to IV antibiotic administration from ED arrival (which was “time zero”). The primary outcome was in-hospital mortality.

Assessment of the Sensitivity of Results to 4 Common Analytic Decisions

We assessed the impact of (1) breadth of confounding adjustment, (2) maximum permitted interval until antibiotics, (3) suspected infection versus sepsis versus septic shock, and (4) assuming a single linear effect for each hour until antibiotics versus modeling each hour separately on the estimated association between time-to-antibiotics and mortality. For numbers 1, 2, and 3, we fit multivariable logistic regression models estimating the odds ratio (OR) of in-hospital mortality for hourly increases in time-to-antibiotics, similar to prior studies [6, 9–11].

For number 1 (breadth of confounding adjustment), we fit 5 models adjusted with progressively more covariates mirroring and exceeding those used in prior studies: (a) an unadjusted model; (b) adding demographics and basic encounter information including age, sex, race/ethnicity, year (of ED arrival), hospital type, arrival from a healthcare facility, arrival by ambulance, insurance type, and hospital discharge within the preceding 90 days; (c) adding comorbidities (chronic lung disease, diabetes, heart failure, liver disease, renal disease, leukemia, lymphoma, and solid tumor with and without metastases) and the Elixhauser comorbidity index [40]; (d) adding presenting laboratory data (platelets, hematocrit, WBC, lactate, total bilirubin, aspartate aminotransferase [AST], albumin, sodium, glucose, creatinine, and anion gap), and (e) a maximal model adding pre-arrival intubation and body mass index, first recorded temperature, heart rate, respiratory rate, SBP, highest respiratory support, and vasopressors. All covariates were determined a priori. Additional details are shown in the Supplementary Methods.

For number 2 (maximum interval until antibiotics included in the analysis), we calculated the fraction of each population treated within 6 hours, 12 hours, and 24 hours and then successively decreased the maximum time-to-antibiotics (24 h, 12 h, 6 h) permitted for cohort inclusion.

For number 3 (sepsis vs septic shock), we calculated hourly mortality ORs separately for “suspected infection,” “sepsis without shock,” and “septic shock.”

We applied the sequential confounding-adjustment models described above, including the maximal model, to each of these analyses. As a sensitivity analysis for confirmed infections, we repeated the above steps restricted to patients with positive blood cultures drawn within 24 hours of arrival, excluding common skin contaminants as defined by the National Healthcare Safety Network [41]. We also conducted sensitivity analyses (1) excluding all patients testing positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) within 2 days of ED arrival (given that many patients with coronavirus disease 2019 [COVID-19] pneumonia received antibiotics but did not have bacterial coinfections) and (2) including suspected sources of infection derived from previously described mappings of “present on admission” discharge International Classification of Diseases, Tenth Revision–Clinical Modification (ICD-10-CM) codes [25] added to the maximal confounding adjustment model as Boolean variables.

Finally, for number 4 (assuming each hour until antibiotics has a uniform effect), we fit a multivariable logistic regression model with separate indicators for each hour interval in order to generate log ORs of mortality for each separate hourly window compared to 0–1 hour, again stratified by sepsis severity. We used both an unadjusted model and a maximally adjusted model to assess the impact of confounding adjustment. As an additional sensitivity analysis, we used larger time windows until antibiotics (6–9 h, 9–12 h, 12–18 h, and 18–24 h) comparing these windows against the 0–6-hour interval, as well as comparing 3–6 hours versus 0–3 hours.

Wald 95% confidence intervals (CIs) were calculated for each OR using standard errors. We used 2-sided P values less than .05 to reject the null hypothesis that the OR equals 1. Statistical analyses were performed using R (version 4.0.2; R Foundation for Statistical Computing).

RESULTS

Patient Characteristics

Among 538 786 adults admitted from the ED, we identified 54 639 with suspected infection (no sepsis), 23 619 with suspected sepsis (without shock), and 25 990 with suspected septic shock (Supplementary Figure 1). Crude in-hospital mortality rates were highest for suspected septic shock (3484/25 990; 13%), followed by suspected sepsis (1423/23 619; 6.0%) and suspected infection (1049/54 639; 1.9%). Patients with suspected septic shock had the highest comorbidity burden and the most abnormal laboratory results; they were also most likely to arrive by ambulance, present to an academic hospital, and arrive from a healthcare facility (Table 2).

Table 2.

Characteristics of the Study Cohorts

	Cohort
	Suspected Infection (N = 54 639)	Suspected Sepsis (N = 23 619)	Suspected Septic Shock (N = 25 990)
Deaths, N (%)	1049 (1.9%)	1423 (6.0%)	3484 (13%)
Age (y), median (IQR)	64 (49, 77)	68 (56, 79)	67 (55, 78)
Sex, N (%)
Female	27 508 (50%)	10 275 (44%)	12 323 (47%)
Male	27 131 (50%)	13 344 (56%)	13 667 (53%)
Race/ethnicity, N (%)
Asian	1710 (3.1%)	860 (3.6%)	976 (3.8%)
Black	4128 (7.6%)	2272 (9.6%)	2233 (8.6%)
Hispanic	3282 (6.0%)	1477 (6.3%)	1633 (6.3%)
Other	770 (1.4%)	359 (1.5%)	401 (1.5%)
White	41 628 (76%)	17 373 (74%)	19 360 (74%)
Two or more categories	2453 (4.5%)	909 (3.8%)	915 (3.5%)
Missing	668 (1.2%)	369 (1.6%)	472 (1.8%)
Year of ED arrival, N (%)
2015	1155 (2.1%)	491 (2.1%)	583 (2.2%)
2016	4688 (8.6%)	1819 (7.7%)	2160 (8.3%)
2017	7181 (13%)	3135 (13%)	3338 (13%)
2018	7826 (14%)	3460 (15%)	3914 (15%)
2019	8216 (15%)	3797 (16%)	4218 (16%)
2020	8962 (16%)	4225 (18%)	4590 (18%)
2021	10 825 (20%)	4415 (19%)	4694 (18%)
2022	5786 (11%)	2277 (9.6%)	2493 (9.6%)
Arrival via EMS, N (%)	20 898 (38%)	12 569 (53%)	16 660 (64%)
Type of hospital, N (%)
Academic	32 058 (59%)	15 070 (64%)	17 238 (66%)
Community	22 581 (41%)	8549 (36%)	8752 (34%)
Insurance type, N (%)
Medicaid	6118 (11%)	2391 (10%)	2985 (11%)
Medicare	23 459 (43%)	11 221 (48%)	12 394 (48%)
Other	496 (0.9%)	159 (0.7%)	224 (0.9%)
Private	23 918 (44%)	9566 (41%)	10 050 (39%)
Missing	648 (1.2%)	282 (1.2%)	337 (1.3%)
Intubated pre-hospital arrival, N (%)	0 (0%)	7 (<0.1%)	89 (0.3%)
Admission from facility, N (%)	2565 (4.7%)	1609 (6.8%)	2282 (8.8%)
Hospitalization in past 90 d, N (%)	19 651 (36%)	9213 (39%)	10 426 (40%)
Any vasopressorsa within 12h of ED arrival, N (%)	31 (<0.1%)	217 (0.9%)	7436 (29%)
BMI, median (IQR)	26 (23, 31)	27 (23, 32)	26 (22, 30)
Missing	1278 (2.4%)	287 (1.2%)	359 (1.4%)
Elixhauser comorbidity score, median (IQR)	2 (−3, 17)	11 (0, 25)	14 (0, 28)
Selected Elixhauser comorbidities, N (%)
Cancer—Leukemia	1347 (2.5%)	707 (3.0%)	609 (2.3%)
Cancer—Lymphoma	1521 (2.8%)	840 (3.6%)	964 (3.7%)
Cancer—Metastatic	4255 (7.8%)	2297 (9.7%)	2814 (11%)
Cancer—Solid tumor without metastasis	6544 (12%)	3559 (15%)	3962 (15%)
Chronic pulmonary  isease	14 175 (26%)	6427 (27%)	7223 (28%)
Diabetes with chronic complications	9978 (18%)	6393 (27%)	5666 (22%)
Diabetes without chronic complications	4988 (9.1%)	2980 (13%)	3218 (12%)
Heart failure	10 166 (19%)	6528 (28%)	7999 (31%)
Liver disease, severe	571 (1.0%)	1289 (5.5%)	1448 (5.6%)
Renal failure, mild	8055 (15%)	5174 (22%)	4869 (19%)
Renal failure, severe	3150 (5.8%)	2327 (9.9%)	1988 (7.6%)
Vitalsb, median (IQR)
HR (beats/min)	94 (80, 108)	98 (82, 113)	100 (82, 117)
RR (breaths/min)	18 (18, 20)	20 (18, 22)	20 (18, 22)
SpO2 (%)	97 (95, 98)	97 (95, 98)	97 (94, 98)
SBP (mmHg)	133 (118, 150)	132 (116, 150)	109 (90, 132)
Temperature (°F)	98.4 (97.7, 99.4)	98.3 (97.6, 99.4)	98.1 (97.3, 99.2)
Highest O2 devicec, N (%)
None	39 704 (73%)	13 628 (58%)	10 873 (42%)
Nasal cannula	12 297 (23%)	5570 (24%)	7244 (28%)
High flow nasal cannula	331 (0.6%)	242 (1.0%)	337 (1.3%)
Oxygen conserving device	53 (<0.1%)	29 (0.1%)	46 (0.2%)
Simple mask	1031 (1.9%)	437 (1.9%)	557 (2.1%)
Advanced mask	1223 (2.2%)	731 (3.1%)	1283 (4.9%)
BiPAP	0 (0%)	1416 (6.0%)	813 (3.1%)
Ventilator	0 (0%)	1566 (6.6%)	4837 (19%)
ECMO	0 (0%)	0 (0%)	0 (0%)
Labsb, median (IQR)
Albumin (g/dL)	3.8 (3.4, 4.1)	3.7 (3.2, 4.1)	3.5 (3.0, 4.0)
Missing, N (%)	14 430 (26.4%)	2957 (12.5%)	2113 (8.1%)
Anion gap (mEq/L)	13.0 (11.0, 15.0)	15.0 (13.0, 17.0)	16.0 (13.0, 19.0)
Missing, N (%)	26 (<0.1%)	17 (<0.1%)	16 (<0.1%)
AST (U/L)	23 (17, 36)	30 (20, 56)	32 (20, 62)
Missing, N (%)	15 353 (28.1%)	3382 (14.3%)	2615 (10.1%)
Creatinine (mg/dL)	0.93 (0.74, 1.22)	1.14 (0.83, 1.95)	1.23 (0.87, 1.91)
Glucose (mg/dL)	116 (101, 141)	135 (110, 180)	131 (106, 181)
Missing, N (%)	1 (<0.1%)	0 (0%)	0 (0%)
Hematocrit (%)	36 (32, 40)	36 (31, 41)	36 (30, 41)
Missing, N (%)	4 (<0.1%)	6 (<0.1%)	7 (<0.1%)
Lactate (mEq/L)	1.2 (1.0, 1.5)	2.3 (1.6, 2.8)	2.6 (1.5, 4.5)
Missing, N (%)	17 827 (32.6%)	2201 (9.3%)	988 (3.8%)
Platelets (1000/µL)	235 (178, 309)	216 (147, 294)	219 (154, 301)
Sodium (mEq/L)	137 (134, 139)	137 (134, 140)	137 (133, 140)
Missing, N (%)	1 (<0.1%)	0 (0%)	0 (0%)
Total bilirubin (mg/dL)	0.50 (0.30, 0.80)	0.60 (0.40, 1.2)	0.60 (0.40, 1.0)
Missing, N (%)	14 472 (26.5%)	2978 (12.6%)	2140 (8.2%)
WBC (1000/µL)	10.4 (7.3, 14.2)	11.3 (7.6, 15.9)	11.5 (7.6, 16.5)

Abbreviations: AST, aspartate aminotransferase; BiPAP, bilevel positive airway pressure; BMI, body mass index; ECMO, extracorporeal membrane oxygenation; ED, emergency department; EMS, emergency medical services; HR, heart rate; IQR, interquartile range; RR, respiratory rate; SBP, systolic blood pressure; SpO₂, oxygen saturation; WBC, white blood cell count.

^aVasopressors included any intravenous administration of epinephrine, norepinephrine, phenylephrine, vasopressin, or dopamine.

^bFirst value measured within 12 hours of ED arrival.

^cHighest oxygen device within 12 hours of ED arrival.

Antibiotic Timing

The median time from ED arrival to antibiotic administration across all cohorts was 3.4 hours (interquartile range [IQR]: 2.0–5.7 h). Median times decreased with increasing severity of presentation: suspected infection, 3.8 hours (IQR: 2.4–6.2 h); suspected sepsis, 3.3 hours (2.0–5.6 h); suspected septic shock, 2.5 hours (1.4–4.6 h). Most patients were treated within 6 hours: 40 335 of 54 639 (74%) for suspected infection, 18 303 of 23 619 (77%) for suspected sepsis, and 21 586 of 25 990 (83%) for suspected septic shock (Figure 1). Crude mortality rates for patients receiving IV antibiotics after 6 hours were higher than those for patients receiving antibiotics before 6 hours: 2.1% versus 1.9% for suspected infection, 6.9% versus 5.8% for suspected sepsis, and 14% versus 13% for suspected septic shock. Treatment of suspected shock more than 6 hours after ED arrival was associated with later onset of hypotension, hyperlactatemia, and other signs of organ dysfunction (Supplementary Figure 2). Counts and additional characteristics of patients receiving antibiotics before versus after 6 hours are shown in Supplementary Table 1.

Figure 1.

Histogram of the distribution of time to first IV antibiotics (larger gray bars), measured from ED arrival, in 3 mutually exclusive cohorts of patients with increasing severity of presentation: (A) suspected infection, defined by blood culture drawn and IV antibiotics administered within 24 hours; (B) suspected sepsis, which also requires lab evidence of organ dysfunction or mechanical ventilation within 12 hours; and (C) suspected septic shock, which also requires evidence of hypoperfusion, specifically SBP <90 mmHg or lactate >4.0 mmol/L, within 12 hours. Each bar represents an hourly interval starting from 0. The count of patients in each interval experiencing in-hospital mortality or discharge to hospice is overlaid as a smaller red bar. The proportion of patients receiving antibiotics before 6 hours is indicated next to the vertical lines. Abbreviations: ED, emergency department; hrs, hours; IV, intravenous; SBP, systolic blood pressure.

Impact of Covariate Adjustment, Maximum Time-to-Antibiotics, and Sepsis Severity

Sequentially broader confounding adjustment led to changes in the direction, strength, and significance of effect estimates (Figure 2 and Supplementary Table 2). In unadjusted models including patients treated up to 24 hours after ED arrival, longer intervals until antibiotics were associated with decreased mortality for patients with septic shock. With maximal confounding adjustment, however, each additional hour until antibiotics was associated with increased mortality in all 3 cohorts (suspected infection, suspected sepsis, and septic shock). Even so, when limiting the maximally adjusted model to patients started on antibiotics within 6 hours, we only observed a statistically significant increase in mortality per hour until antibiotics in patients with suspected septic shock (adjusted OR [aOR]: 1.07 per hour; 95% CI: 1.04–1.11; P < .01). The association was not statistically significant for suspected sepsis (aOR: 1.03; 95% CI: .98–1.09; P = .23) or suspected infection (aOR: .99; 95% CI: .94–1.05; P = .75).

Figure 2.

Odds ratios of in-hospital mortality per hour delay in antibiotic administration under the assumption of a linear relationship with log odds, varying the covariates used for confounding adjustment (y axis), maximum time-to-antibiotics (A, B, and C), and stratification of illness severity (symbol and color). 95% Confidence intervals are depicted by horizontal lines, and the x axis is log scaled. Odds ratios greater than 1 indicate increasing mortality associated with later antibiotics. Note that the 3 populations are nonoverlapping. The top row shows the unadjusted analysis, with subsequent rows adding progressively more detailed sets of covariates. For details on confounding adjustment, see Methods.

Assuming a Linear Relationship Between Log Odds of Mortality and Time-to-Antibiotics

All of the preceding results were based on linear models that presumed each hour until antibiotics has a uniform impact on mortality. To assess the validity of this assumption, we calculated separate mortality ORs for each hourly interval until antibiotics relative to 0–1 hour. Figure 3 presents these results using both an unadjusted model (no covariates) and a fully adjusted model for each severity cohort. For patients with suspected septic shock, there was a significant increase in the aOR of mortality for intervals after more than 1 hour until antibiotics (eg, aOR: 1.27 for 1–2 h vs 0–1 h; 95% CI: 1.11–1.44; P < .01) (Figure 3E and 3F), whereas for patients with suspected sepsis without shock, a significant increase was not observed until intervals of 9 hours or more (aOR: 1.77 for 9–10 h vs 0–1 h; 95% CI: 1.16–2.70; P = .01) (Figure 3D).

Figure 3.

Odds ratios of in-hospital mortality when comparing each hourly interval of time-to-antibiotics against the 0–1-h interval without the assumption of a linear relationship between time-to-antibiotics and log odds, using a fully adjusted model (filled circles) or an unadjusted model (open circles), contrasting patients with suspected infection (A and B), suspected sepsis (C and D), and suspected septic shock (E and F). Point sizes are scaled to the number of patients in each hourly interval (scale N in legend). 95% CIs for each odds ratio are depicted by vertical lines; note that some data extend beyond the limits of the y axis, which is log scaled. The odds ratios of in-hospital mortality per hour delay under the assumption of a linear relationship (see Figure 2 and Supplementary Table 2) are drawn here as diagonal lines, contrasting the results when including patients with a maximum time interval until antibiotics of 6 hours (panels A, C, and E; solid lines with gray area depicting 95% CI) versus 24 hours (panels B, D, and F; dashed lines with dotted lines depicting 95% CI). Abbreviations: aOR, adjusted odds ratio; CI, confidence interval; IV, intravenous.

To further elucidate the impact of assuming linearity, we plotted the ORs for each separate hourly interval alongside trendlines assuming a uniform linear increase for every hour on the same figure (Figure 3). When limiting to patients receiving antibiotics before 6 hours, distinct hourly intervals until antibiotics were associated with significantly higher mortality for patients with suspected septic shock but not for sepsis without shock or for those with suspected infection alone (Figure 3A, 3C, and 3E).

After approximately 6 hours, there was an upward inflection in the separate hourly ORs for patients with suspected infection and suspected sepsis without shock (Figure 3B and 3D). However, fitting linear associations across the entire 24-hour study period would obfuscate this nonlinearity and produce similar significantly positive hourly associations for every cohort (dashed lines; Figure 3B, 3D, and 3F). In a sensitivity analysis using longer intervals, administering antibiotics 3–6 hours after ED arrival versus 0–3 hours was not significant for sepsis without shock (aOR: 1.03; 95% CI: .89–1.20; P = .66) or suspected infection without sepsis (aOR: .96; 95% CI: .82–1.13; P = .62) (Supplementary Table 3). There was, however, a significant association between intervals of 9–12 hours versus 0–6 hours for suspected sepsis without shock (aOR: 1.62; 95% CI: 1.26–2.08; P < .01) and suspected infection alone (aOR: 1.43; 95% CI: 1.10–1.86; P = .01) (Supplementary Table 3).

Blood cultures were positive in 9.6% of patients (10 038 of 104 248); 80% of these patients (8076 of 10 038) were treated within 6 hours (Supplementary Table 4). In a sensitivity analysis limited to this bacteremic population, ORs for mortality per hour until antibiotics were significant only in patients with suspected septic shock, and not the cohorts without shock, when including patients treated up to 6 hours or 12 hours after ED arrival (Supplementary Table 5). In sensitivity analyses that excluded patients with community-onset COVID-19 or added infection-source categories to the confounding adjustment model, ORs for mortality per hour until antibiotics were minimally changed, with no differences in statistical significance (Supplementary Figure 3).

DISCUSSION

Sepsis guidelines, quality metrics, and quality-improvement initiatives recommend immediate broad-spectrum antibiotics for all patients with possible sepsis based on observational studies suggesting that each additional hour until antibiotics is associated with increased mortality. Concerns have been expressed about 4 common analytic decisions in these studies: (1) insufficient confounding adjustment, (2) including nonrepresentative patients with very long delays until antibiotics, (3) failure to differentiate between sepsis with and without shock, and (4) assuming that each hour until antibiotics has an equal effect on mortality (Table 1). Our study examined these analytical concerns in a cohort of more than 100 000 patients with suspected infection, sepsis, or septic shock and found that each has the potential to substantially alter a study's conclusions.

We demonstrate that successively more detailed confounding adjustment has a very large effect on the estimated association between time-to-antibiotics and mortality, eventually shifting an association between longer times to antibiotics and decreasing mortality to an association with increasing mortality.

Likewise, including patients with a very long time-to-antibiotics (>6 h) in the analytic cohort can also fundamentally alter conclusions. When we included patients treated more than 6 hours from arrival, each additional hour until antibiotics was associated with increased mortality for both sepsis and septic shock, whereas restricting the analytic cohort just to patients treated within 6 hours of ED arrival generated a significant estimated hourly effect only for patients with septic shock. Patients treated more than 6 hours after arrival accounted for less than 20% of patients with sepsis and septic shock in our cohort and likely differ in important ways from patients receiving early antibiotics. Extrapolating the effect of delays of fewer than 6 hours from ED arrival using this small group of outlier patients treated after more than 6 hours is questionable. Recommendations to administer antibiotics within 1 hour or 3 hours of presentation (per current sepsis guidelines and quality metrics) should be informed by patients treated at intervals proximate to these time frames, not the minority of patients treated well beyond them. When these outliers were excluded, hourly estimates for death were only significant for suspected septic shock. This supports the latest Surviving Sepsis Campaign and professional society recommendations that highlight the greatest urgency for immediate antibiotics in patients with suspected septic shock [12, 19, 29].

Our finding that outlier patients can disproportionately affect results mirrors a recent analysis of 4792 patients treated for sepsis in 40 German hospitals [42]. Regression analyses including patients receiving antibiotics as late as 48 hours, despite 71% receiving treatment before 6 hours, generated a statistically significant increase in mortality ORs per hour until antibiotics of 1.019 (95% CI: 1.01–1.028). When the investigators directly compared patients treated after 1–3 hours versus 0–1 hour or 3–6 hours versus 0–1 hour, however, the results were not significant. Only when comparing patients treated after more than 6 hours versus 0–1 hour was there a significant association with mortality (aOR: 1.36; 95% CI: 1.12–1.63). An hourly OR limited to patients treated within 6 hours was not reported; our study demonstrates the importance of this sensitivity analysis, which substantially affected our results.

Strengths of our study include systematic, quantitative analyses of common analytic decisions that could impact the estimated association between time-to-antibiotics and mortality. We used a very large cohort of patients and highly detailed clinical data for confounding adjustment, including 40 different demographic, laboratory, and physiologic covariates. We purposefully selected the most common statistical approach used in prior large studies (multivariable logistic regression with time-to-antibiotics as the exposure variable) to facilitate direct comparison with prior studies [6–11]. We also generated fully adjusted estimates for each distinct hourly interval until antibiotics to characterize the appropriateness of models that assume each hourly interval has an equal impact.

Our study also has important limitations. Our results may be biased by residual confounding despite the number and detail of covariates we included in our models. We were limited to structured covariates extracted from electronic medical records. It is possible that qualitative information in clinical notes could further reduce confounding, such as vague versus explicit presenting symptoms [43, 44]. We did not evaluate whether patients had confirmed infections in retrospect or assess the appropriateness of ordered antibiotics [21, 22]. However, we conducted a sensitivity analysis of patients with positive blood cultures and found similar associations, acknowledging limited power in this smaller cohort (Supplementary Tables 4 and 5). Our exclusion criteria may have created cohorts with different disease severity than prior studies, and hospitals in this study were from 1 metropolitan region with relatively high proportions of White and non-Medicaid insurance; therefore, our cohort may not be generalizable to different settings [45]. We were not able to adjust for concomitant sepsis treatments, such as source control or fluid resuscitation. We used ED arrival time as time zero for antibiotic timing rather than trying to define time zero on physiologic grounds; some patients may have had sepsis for prolonged times prior to ED arrival [37]. Finally, the regression models we used do not accommodate time-varying confounders that are affected by past treatment; for example, clinicians may have modified their decision on whether and when to give antibiotics based on patients’ evolving clinical trajectories including both measured (eg, successive vital signs and test results) and unmeasured (eg, delirium, patient appearance) factors [46]. Emerging causal inference methods may better accommodate these more complex data interactions and are a promising route for future observational studies of time-varying treatment strategies [30].

In conclusion, we found that 4 analytic decisions in the existing time-to-antibiotics literature have a strong impact on the magnitude, direction, and significance of the perceived relationship between time-to-antibiotics and mortality and could substantially alter studies’ conclusions. Our findings point to the importance of critically evaluating time-to-antibiotics studies for their breadth of covariates, the maximum permitted interval until antibiotics, handling of sepsis with versus without shock, and whether models assume a single uniform effect for each hour until antibiotics. We found that, in maximally adjusted nonlinear models, each hour until antibiotics from 1–6 hours was associated with significantly higher mortality in patients with suspected septic shock but not in patients with suspected sepsis or infection alone. These findings have important implications for sepsis treatment guidelines, quality metrics, and quality-improvement initiatives.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Author Contributions. T. R. P. had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the National Institute of Allergy and Infectious Diseases, the Centers for Disease Control and Prevention, or the Agency for Healthcare Research and Quality. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Financial support. This work was supported by the National Institute of Allergy and Infectious Diseases (grant number T32AI007061 to T. R. P.), the Centers for Disease Control and Prevention (to C. R. and M. K.), and the Agency for Healthcare Research and Quality (grant number R01HS027170 to C. R., M. K., and M. R. F.).