Systematic Review and Meta-Analysis of the Efficacy of Appropriate Empiric Antibiotic Therapy for Sepsis

Abstract

Quantifying the benefit of early antibiotic treatment is crucial for decision making and can be assessed only in observational studies. We performed a systematic review of prospective studies reporting the effect of appropriate empirical antibiotic treatment on all-cause mortality among adult inpatients with sepsis. Two reviewers independently extracted data. Risk of bias was assessed using the Newcastle-Ottawa score. We calculated unadjusted odds ratios (ORs) with 95% confidence intervals for each study and extracted adjusted ORs, with variance, methods, and covariates being used for adjustment. ORs were pooled using random-effects meta-analysis. We examined the effects of methodological and clinical confounders on results through subgroup analysis or mixed-effect meta-regression. Seventy studies were included, of which 48 provided an adjusted OR for inappropriate empirical antibiotic treatment. Inappropriate empirical antibiotic treatment was associated with significantly higher mortality in the unadjusted and adjusted comparisons, with considerable heterogeneity occurring in both analyses (I² > 70%). Study design, time of mortality assessment, the reporting methods of the multivariable models, and the covariates used for adjustment were significantly associated with effect size. Septic shock was the only clinical variable significantly affecting results (it was associated with higher ORs). Studies adjusting for background conditions and sepsis severity reported a pooled adjusted OR of 1.60 (95% confidence interval = 1.37 to 1.86; 26 studies; number needed to treat to prevent one fatal outcome, 10 patients [95% confidence interval = 8 to 15]; I² = 46.3%) given 34% mortality with inappropriate empirical treatment. Appropriate empirical antibiotic treatment is associated with a significant reduction in all-cause mortality. However, the methods used in the observational studies significantly affect the effect size reported. Methods of observational studies assessing the effects of antibiotic treatment should be improved and standardized.

Sepsis affects 1.1 to 2.4 per 1,000 people per year and 20 to 42% of these patients die in hospital, with these rates probably underestimating the contribution of hospital-acquired infections (3, 16, 61). Septicemia and pneumonia combined are the sixth most common causes of death in the United States (36). Antibiotic treatment for the first 24 to 48 h is largely empirical (i.e., provided without evidence on the causative pathogen or its susceptibilities), and it is common wisdom that appropriate empirical antibiotic treatment (i.e., matching the in vitro susceptibilities of the isolated pathogens) reduces mortality. Physicians thus strive to achieve appropriate empirical antibiotic treatment for inpatients with suspected infections, and many times this is at the cost of administering superfluous and unnecessary antibiotics. Such treatment is associated with resistance development (83, 97) and side effects with no benefit.

Estimates of the potential benefit of appropriate empirical antibiotic treatment vary widely in the literature between no effect (21, 22, 48, 70, 84, 88) and adjusted odds ratios (ORs) above 6 (39). The effects might be truly variable and dependent on infection severity, the patient's immune status, and the type of bacteria. Alternatively, heterogeneity might stem from methodological factors in observational studies, since assessment of the effects of early treatment relies by necessity on nonrandomized studies (34). These may include the covariates collected and used for adjustment of the effect of antibiotic treatment on mortality and the methods used for adjustment.

We conducted a systematic review with meta-analysis of studies assessing the effects of appropriate empirical antibiotic treatment on mortality. We aimed to investigate the reasons for heterogeneity in the magnitude of this effect and to obtain a better estimate of the true effect in general or specific clinical scenarios. Such an estimate is crucial to the decision making regarding antibiotic treatment.

(Preliminary results have been presented at the European Congress of Clinical Microbiology and Infectious Diseases, oral presentation, 17 May 2009, Helsinki, Finland.)

MATERIALS AND METHODS

Study selection. (i) Study design.

We included prospective cohort studies, defined as those where cases were identified prospectively and data collection was started with identification. We judged that prospective data collection would result in the better and uniform availability of confounders for the adjusted analysis of mortality. We excluded studies published before 1975, using an arbitrary time point to denote an era in critical illness management that may be less relevant to current practice. We excluded studies that recruited less than 50 patients, assuming that with an average mortality of about 10%, an analysis including less than 5 outcomes has no power. We excluded studies assessing specifically meningitis and endocarditis, where treatment effects are expected to largely deviate from any common effect.

(ii) Patients.

The patients included were adults (age, >18 years) with sepsis and microbiologically documented infections.

(iii) Intervention.

The intervention was appropriate (versus inappropriate) empirical antibiotic treatment. “Empirical” treatment was defined as that administered prior to microbiological documentation of infection. “Appropriate” treatment had to be treatment matching the in vitro susceptibility of the pathogen. We permitted the inclusion of studies where up to 10% of pathogens were not tested in vitro (e.g., Mycoplasma pneumoniae); in these cases, the study definitions for appropriateness were accepted. We did not try to include antibiotic dosing, intrinsic antibiotic activity (e.g., vancomycin for methicillin-sensitive Staphylococcus aureus and aminoglycosides alone for Pseudomonas aeruginosa), or combination therapy in the definition of appropriateness, due to poor reporting of these definitions and the lack of evidence of their impact on mortality (73, 75), but we documented the definition and assessed its effect on outcomes.

(iv) Outcome.

The outcome assessed was all-cause 30-day mortality. If 30-day data were not available, we used mortality at another fixed point in time or in-hospital mortality and documented the outcome assessed in the study.

Data sources and searches.

We searched PubMed (January 1975 to November 2008) and references of all identified studies, using the following search strategy: ((antibiot* OR antimicrob* OR anti-bacter* OR antibacter*) AND (approp* OR inapprop* OR adequate OR inadequate) AND (mort* OR fatal* OR death OR dead OR alive OR survi*)). We did not include unpublished studies, since we needed a complete description of the study methods and analysis to investigate the reasons for heterogeneity. No language restrictions were applied.

Data extraction and quality assessment.

Two reviewers independently inspected each reference identified by the search and applied inclusion criteria. In cases where the same population studied was analyzed in more then one publication, the study's results were accounted for only once. Trials fulfilling the review inclusion criteria were assessed for risk of bias by two reviewers, independently, using the Newcastle-Ottawa score (NOS) (96), adapted for our review (see the information on adapted NOS in the supplemental material). The score assigns a study a maximum of 8 points, with higher scores indicating a lower risk of bias. In addition, we documented the definitions of “appropriate” and “empirical,” the timing of mortality assessment, and the prospective components of the study (planning, patient detection, and data collection).

Two reviewers independently extracted the data. In case of disagreement between the two reviewers, a third reviewer extracted the data. Trial authors were contacted for clarification and to complete missing data. We collected the raw, unadjusted number of deaths among patients given appropriate versus inappropriate empirical antibiotic treatment. We extracted the adjusted effect estimate of appropriate empirical treatment for mortality with its variance and documented the method used for adjustment, the covariates assessed, and terms for inclusion in multivariable analyses, which were the variables finally included in the analysis and their significance. We collected descriptive data on setting, study years, follow-up duration, patient characteristics, types of pathogens, sources of infection, and presence of bacteremia.

Data synthesis and analysis. (i) Unadjusted (univariate) analysis.

We computed odds ratios with 95% confidence intervals (CIs) for individual studies and pooled these in the meta-analysis. Null values precluding calculation of ORs were replaced by 0.5. We investigated heterogeneity through subgroup analyses and meta-regression on the basis of the study years; the prevalence of bacteremia, neutropenia, and pneumonia among the studied patients; the patients’ ages; the percentages of patients with septic shock and in an intensive care unit (ICU); the mean APACHE score; the prevalence of Pseudomonas aeruginosa, Staphylococcus aureus, and methicillin-resistant S. aureus (MRSA) infections; the study's adapted NOS score; and the other methodological variables assessed.

(ii) Adjusted analysis.

Out of all 70 studies included, 22 did not report an adjusted analysis: in 13 the univariate results for appropriate empirical treatment were nonsignificant, and in 9 no adjusted analysis was conducted, despite the significance observed on univariate analysis, usually due to a small sample size. In the primary analysis, these 22 studies were excluded, since we could not impute adjusted ORs. All 48 studies reporting an adjusted effect of appropriate empirical treatment used multivariable regression analysis. Most studies provided the numerical results of appropriate empirical treatment in the final model, whether it was significantly associated with mortality or not. Six studies reported qualitatively that appropriate empirical treatment was not significantly associated with mortality, with no numerical values being given. In the main analysis we imputed an OR of 1 for these studies and used the standard error (SE) of the univariate analysis as the dispersion measure. Thus, the main adjusted analysis includes all studies that assessed the adjusted effect of appropriate empirical treatment on mortality, using either reported numerical results from a multivariable analysis (42 studies) or an OR equal to 1 when appropriate empirical treatment did not remain significant on multivariable analysis (6 studies). We conducted a sensitivity analysis, where studies that did not perform a multivariable analysis because the univariate appropriate empirical treatment results were nonsignificant (13 studies) were included in the analysis, with OR equal to 1 with the univariate analysis results’ SEs. Heterogeneity was investigated as for the univariate analysis, with an added assessment of the types of covariates being included in the multivariable analysis (e.g., disease severity and background conditions). Odds ratios were pooled with 95% confidence intervals or standard errors calculated from reported P values.

Statistical methods.

All meta-analyses were conducted and reported using a random-effects model, assuming a priori significant heterogeneity resulting from diverse study populations and different models for adjusted analyses. Heterogeneity was assessed using a chi-square test of heterogeneity and the I² measure of inconsistency. Subgroup analyses were performed using a mixed-effects analysis, where a random-effects model is used to combine studies within each subgroup and the study-to-study variance is computed within each subgroup. Mixed-effect univariate meta-regression was conducted using the unrestricted maximum-likelihood method to assess individual variables. The proportion of between-study variance explained by the covariates (R²) was assessed using random-effect multivariable meta-regression (35). A funnel plot of standard errors against log(ORs) was constructed for the univariate analysis that included all studies, to assess for the effect of small studies; significance (2-tailed) of the Begg and Mazumdar rank correlation test is reported. Analyses were performed using the Comprehensive Meta-Analysis (version 2.2) and Stata (version 10.1) programs.

RESULTS

Seventy individual trials (2, 5-9, 11-15, 17, 19, 20, 23-33, 37-40, 42-47, 50-60, 62, 64-67, 69, 71, 76-82, 85-87, 90-93, 95, 98, 99), out of 2,800 identified references, fulfilled the inclusion criteria (Fig. 1). Overall, 46.5% of patients were given inappropriate empirical antibiotic treatment, and the mortality among them was 35%. Study characteristics are shown in Table 1. Twenty-six studies were conducted in an ICU. Fifteen assessed one specific pathogen, while others assessed all bacteria. Forty-two studies addressed only bacteremic patients, and the rate of bacteremia in the other studies ranged from 0 to 70%. The mean adapted NOS score was 6.7 (standard deviation, 1.0).

FIG. 1.

Study flow. References to excluded studies are available from the authors upon request.

TABLE 1.

Characteristics of included studies^a

Author(s), yr (reference)	Study yr(s)	Location	Setting	Main type of infection	Spectrum of bacteria assessed	No. of patients	% mortality (n/Nb)	Time of mortality assessment	% inappropriate empirical antibiotics (n/N)	Appropriate definition beyond in vitro coveragec	Adjusted analysis performed
Alvarez-Lerma, 1996 (2)	1988-1989	Spain	ICU	Pneumonia	All	565	33 (186/565)	72 h after discharge	34 (146/430)	No	No
Behrendt et al., 1999 (5)	1989-1993	Germany	All	Septicemia	All	983	18 (177/983)	28 days in hospital	30.3 (297/981)	Dose and route	No
Bodi et al., 2005 (6)	2000-2002	Spain	ICU	Community-acquired pneumonia	All	529	28 (48/529)	In ICU	14.8 (41/276)	No	Yes
Boots et al., 2005 (7)	1999-2000	Australia and New Zealand	ICU	Pneumonia	All	476	31 (148/476)	In ICU	12.6 (60/476)	No	Yes
Bouza et al., 2004 (8)	2000-2000	Spain	All	Bacteremia	All	297	23.5 (70/297)	In hospital	41.3 (120/290)	No	Yes
Byl et al., 1999 (11)	1994-1994	Belgium	NS	Bacteremia	All	428	20 (85/428)	NS	38 (159/417)	Dose and route	Yes
Bryan et al., 1983 (9)	1977-1981	USA	NS	Bacteremia	Enterobacteriaceae and Pseudomonas spp.	1186	36.7 (434/1,186)	In hospital	38.9 (461/1,186)	Dose and route	No
Candel et al., 2005 (12)	1991-2000	Spain	NS	Bacteremia	All	66	54 (33/66)	No definition	21 (14/66)	No	No
Cisneros et al., 1996 (13)	1993-1994	Spain	All	Bacteremia	Acinetobacter baumannii	79	51.9 (41/79)	In hospital	26.5 (21/79)	No	Yes
Clec'h et al., 2004 (14)	1997-2000	France	ICU	VAP	All	142	50 (71/142)	In hospital	55.6 (79/142)	In cases of P. aeruginosa, combination of 2 effective drugs	Yes
Depuydt et al., 2006 (15)	1992-2001	Belgium	ICU	Bacteremia associated with nosocomial pneumonia	All	110	50 (56/110)	In hospital	38 (42/110)	No	Yes
Dupont et al., 2003 (17)	1997-1998	France	All	Postoperative pneumonia	All	556	22.6 (126/556)	30 days in hospital	28.5 (92/322)	For nonfermenting Gram-negative bacilli, aminoglycoside alone considered inappropriate	Yes
El-Solh et al., 2001 (19)	1996-1999	USA	ICU	Pneumonia	All	104	54.8 (57/104)	In hospital	23.6 (13/55)	No	Yes
Fraser et al., 2006 (23)	2002-2004	Israel, Germany, Italy	All	All	All	895	14.7 (132/895)	30 days	35.6 (319/895)	No	Yes
Falguera et al., 2009 (20)	1995-2005	Spain	Non-ICU	CAP	Gram-negative bacteria	61	36 (22/61)	30 days	47.5 (29/61)	No	No
Garnacho-Montero et al., 2003 (25)	1997-2000	Spain	ICU	Sepsis	All	406	48 (196/406)	In hospital	17 (46/270)	Dose and route and 2 active antimicrobials were required when P. aeruginosa was isolated	Yes
Garnacho-Montero et al., 2005 (26)	1998-2003	Spain	ICU	VAP	All	81	64 (52/81)	In hospital	40.7 (33/81)	No	Yes
Garnacho-Montero et al., 2006 (24)	2002-2004	Spain	All	All	All	224	23 (52/224)	In hospital	10 (16/158)	Dose and route	Yes
Garrouste-Orgeas et al., 2000 (27)	1995-1996	France	All	Bacteremia	All	109	37.6 (41/109)	In hospital	24.8 (27/109)	Dose and route and duration	Yes
Gatell et al., 1988 (28)	1983-1986	Spain	All	Bacteremia	All	543	18 (98/543)	NS	39 (201/517)	Dose and route and duration	Yes
Gómez et al., 1999 (31)	1992-1996	Spain	All	Sepsis	Acinetobacter baumannii	58	32.7 (19/58)	1 mo after discharge	15.5 (9/58)	No	No
Gomez et al., 1993 (30)	1988-1992	Spain	All	Bacteremia	Anaerobic bacteria	61	37.7 (23/61)	1 wk after discharge	19.7 (12/61)	Duration	No
Gómez et al., 1995 (29)	1989-1993	Spain	All	Bacteremia	Streptococcus pneumoniae	71	19.7 (14/71)	1 mo after discharge	12.7 (9/71)	Dose	No
Gómez Gómez 2004 (32)	1992-1998	Spain	All	Bacteremia	P. aeruginosa	211	28 (59/211)	NS	10.9 (23/211)	No	Yes
Harbarth et al., 2003 (33)	NS	USA, Canada, Europe	All	Severe sepsis	All	904	27.6 (250/904)	28 days	23.3 (211/904)	Aminoglycoside monotherapy considered inappropriate for nonfermenting Gram-negative bacilli	Yes
Heyland et al., 1999 (37)	1992-1996	Canada	ICU	VAP	All	173	23.7 (41/173)	In hospital	21.8 (31/142)	Two antibiotics with activity required for Pseudomonas spp.	No
Hung 2005 (38)	2001-2002	Taiwan	Non-ICU	Bacteremia	Anaerobic	52	25 (13/52)	30 days in hospital	25.7 (9/35)	No	No
Ibrahim et al., 2000 (39)	1997-1999	USA	ICU	Bacteremia	All	492	38.4 (189/492)	In hospital	29.9 (147/492)	No	Yes
Iregui et al., 2002 (40)	2000-2001	USA	ICU	VAP	All	107	41 (44/107)	In hospital	30.8 (33/107)	No	Yes
Ispahani et al., 1987 (42)	1980-1983	UK	All	Bacteremia and candidemia	All	875	2.99 (252/875)	3 mo in hospital	45.8 (401/875)	No	No
Jamulitrat et al., 1994 (43)	1990-1991	Thailand	Non-ICU	Bacteremia	All	277	53.4 (148/277)	7 days from infection	29 (76/263)	No	Yes
Jang et al., 1999 (44)	1996-1997	Taiwan	ICU	Bacteremia	Gram-negative bacteria	147	36 (53/147)	30 days	41.5 (61/147)	Dose, route, and duration	Yes
Javaloyas et al., 2002 (45)	1989-1998	Spain	All	Bacteremia	All	773	14.3 (111/773)	In hospital	13.7 (106/772)	Dose, route, and duration	Yes
Jones and Lowes, 1996 (46)	1993, 1994	UK	Non-ICU	Bacteremia	All	63	38 (24/63)	28 days	42 (27/64)	No	No
Khatib et al., 2006 (47)	2002-2003	USA	NS	Bacteremia	S. aureus	174	28.7 (50/174)	In hospital	34.5 (60/174)	No	Yes
Leibovici et al., 1998 (50)	1988-1994	Israel	All	Bacteremia and candidemia	All	3413	25.4 (867/3,413)	In hospital	36.7 (1,255/3,413)	Aminoglycoside alone considered inappropriate for Pseudomonas spp.	Yes
Leone et al., 2003 (51)	1997-2000	France	ICU	All	All	107	58.8 (63/107)	30 days	11.5 (9/78)	No	No
Leone et al., 2007 (52)	2001-2004	France	ICU	VAP	All	115	23.5 (27/115)	In ICU	13 (15/115)	No	No
Leroy et al., 2003 (53)	1994-2001	France	ICU	VAP	All	132	43.9 (58/132)	In ICU	19.6 (26/132)	No	Yes
Lin et al., 2008 (54)	2001-2006	USA	All	Bacteremia	All	1523	8.5 (129/1,523)	30 days in hospital	35.5 (540/1,523)	Route and antibiotic matching the recommendations of the Sanford Guide to Antimicrobial Therapy	Yes
Lisboa et al., 2008 (55)	NS	Brazil and Spain	ICU	VAP	All	68	23.5 (16/68)	28 days	32.3 (22/68)	No	No
Luna et al., 2006 (56)	1999-2003	Argentina	ICU	VAP	All	76	52 (40/76)	28 days in hospital	68 (52/76)	No	No
Macfarlane et al., 1985 (57)	1982-1983	Jamaica	All	Bacteremia	All	222	27.5 (61/222)	NS	25.7 (57/222)	No	No
McDonald et al., 2005 (62)	2000-2001	USA	NS	Bacteremia	All	466	21.5 (100/466)	In hospital	22.7 (106/466)	Dose and route	No
Mallolas et al., 1991 (58)	1983-1989	Spain	All	Bacteremia	P. aeruginosa	274	42.7 (117/274)	NS	37.6 (103/274)	Dose, route, and duration	Yes
Marcos et al., 2008 (59)	1991-2006	Spain	All	Bacteremia	Enterobacter spp.	377	12.7 (48/377)	30 days	26 (82/314)	Dose and route	Yes
Marscall et al., 2008 (60)	2006-2007	USA	Non-ICU	Bacteremia	Gram-negative bacteria	250	14 (35/250)	In hospital	31.6 (79/250)	No	Yes
Metan et al., 2005 (64)	2003-2005	Turkey	All	Bacteremia	E. coli	53	26.4 (14/53)	30 days	77.3 (41/53)	Dose and route	No
Micek et al., 2005 (65)	2002-2004	USA	ICU	Severe sepsis	All	102	42 (43/102)	In hospital	25.5 (23/90)	No	Yes
Montravers et al., 1996 (66)	1987-1992	France	Surgical	Secondary peritonitis	All	100	39 (39/100)	In hospital	54 (54/100)	No	Yes
Nseir et al., 2006 (67)	1996-2001	France	ICU	COPD exacerbation requiring mechanical ventilation with positive tracheal aspirate	All	260	34.2 (89/260)	In ICU	27.3 (71/260)	No	Yes
Ortega et al., 2007 (69)	2003-2006	Spain	Non-ICU	Community-acquired bacteremia of unknown origin	All	200	13 (26/200)	30 days in hospital	19 (38/200)	Dose and route	Yes
Osmon et al., 2004 (71)	2001-2002	USA	All	Bacteremia	S. aureus and P. aeruginosa	314	17 (54/314)	In hospital	4 (13/314)	No	Yes
Pedersen et al., 1997 (76)	1992-1994	Denmark	All	Bacteremia	Gram-negative bacteria	815	24.4 (199/815)	30 days	25.8 (198/768)	No	Yes
Petrick et al., 2007 (77)	2005-2005	Malaysia	All	Bacteremia	All	191	27.2 (52/191)	In hospital	22 (42/191)	No	Yes
Pittet et al., 1996 (78)	1984-1989	Switzerland	ICU	Bacteremia	All	173	43.3 (75/173)	In hospital	9.2 (16/173)	Dose and route	Yes
Raineri et al., 2008 (79)	2001-2004	Italy	ICU	All	All	349	33.2 (116/349)	In hospital	27.8 (97/349)	Dose and duration	No
Rayner and Willcox, 1988 (80)	1986-1897	South Africa	All	Community-acquired bacteremia	All	239	29 (70/239)	In hospital	14.2 (34/239)	Dose, route, and duration	No
Rello et al., 1994 (81)	1988-1990	Spain	ICU	Bacteremia	All	111	31.5 (35/111)	NS	30.6 (34/111)	Dose, route, and duration	Yes
Rodriguez-Bano et al., 2003 (82)	1995-1989	Spain	All	Bacteremia	Acinetobacter baumannii	133	53.3 (71/133)	In hospital	57 (76/133)	Dose, route, and duration	Yes
Seidenfeld et al., 1986 (85)	1978-1982	USA	ICU	All	All	129	71.3 (92/129)	In hospital	15.8 (13/82)	Dose	No
Seligman et al., 2006 (86)	2003-2005	Brazil	ICU	VAP	All	75	38.6 (29/75)	28 days in ICU	26.6 (20/75)	Adequate when cultures were negative	Yes
Soriano et al., 2008 (87)	1991-1995	Spain	NS	Bacteremia	MRSA	414	28 (116/414)	30 days in hospital	59.4 (246/414)	No	Yes
Valles et al., 2003 (90)	1993, 1998	Spain	ICU	Bacteremia	All	339	41.5 (141/339)	In ICU	14.4 (49/339)	No	Yes
Vergis et al., 2001 (91)	1995-1997	USA	NS	Bacteremia	Enterococcus spp.	398	19.3 (77/398)	14 days in hospital	50.9 (106/208)	Duration	Yes
Vidal et al., 1996 (92)	1991-1994	Spain	All	Bacteremia	Pseudomonas aeruginosa	189	18 (34/189)	In hospital	33.3 (63/189)	Dose and route	Yes
Vidal et al., 2003 (93)	1991-2000	Spain	NS	Bacteremia	Glucose-nonfermenting Gram-negative bacteria other than P. aeruginosa	296	14.5 (43/296)	In hospital	22 (65/296)	Dose and route	Yes
Weinstein et al., 1997 (95)	1992-1993	USA	All	Bacteremia	All	843	22.5 (190/843)	In hospital	11 (87/791)	No	Yes
Zavascki et al., 2006 (99)	2004-2005	Brazil	NS	All	Pseudomonas aeruginosa	298	37.6 (112/298)	In hospital	73.15 (218/298)	No	Yes
Zaragoza et al., 2003 (98)	1995-1999	Spain	ICU	Bacteremia	All	166	51.8 (86/166)	In hospital	23.4 (39/166)	No	Yes

^aCAP, community-acquired pneumonia; VAP, ventilator-associated pneumonia; COPD, chronic obstructive pulmonary disease; NS, not stated.

^bn/N, number of patients with outcome/total number of patients.

^cIn addition to the requisition of in vitro coverage.

Unadjusted (univariate) analysis for mortality.

All studies but one (76) reported unadjusted results for the effect of inappropriate empirical antibiotic treatment on all-cause mortality. The pooled OR was 2.11 (95% CI, 1.82 to 2.44, 69 studies, 21,338 patients; see the figure in the supplemental material). Considerable heterogeneity was observed between studies (P < 0.001, I² = 72%). Three small studies (<70 patients each) were extreme outliers, with two reporting ORs of >70 (29, 31) and one reporting an OR of 0.046 (64). Excluding these, the OR in 66 studies was 2.10 (95% CI, 1.83 to 2.41), with heterogeneity being similar to that in all studies (P < 0.001, I² = 69%). Sensitivity analyses were conducted on these 66 studies. Exclusion of the largest study (50) in the meta-analysis (OR = 2.07) did not alter the results or heterogeneity (OR = 2.11; 95% CI = 1.83 to 2.45, 65 studies, 17,742 patients, I² = 69%).

Mortality was significantly higher with inappropriate empirical treatment in nearly all subgroups (Table 2 ). However, significant heterogeneity persisted in most subgroups, and none of the factors analyzed, except mortality time definition, yielded significantly different results between subgroups. Mortality defined at 28 to 30 days or some other fixed point of time was associated with lower ORs than in-hospital mortality or other time definitions, but the pooled ORs were statistically significant with all definitions. ORs were similar in studies conducted in or outside an ICU and with or without bacteremia. The OR was higher in studies assessing only P. aeruginosa infections and lower in studies assessing only MRSA infections compared to the OR in studies that assessed all bacteria; but only a few studies assessed individual pathogens, and the differences were not statistically significant.

TABLE 2.

Subgroup analysis to assess the effect of confounders on the association between appropriate empirical antibiotic treatment and all-cause mortality^a

Variable	Unadjusted			Adjusted
	OR (95% CI)	No. of studies	P value	OR (95% CI)	No. of studies	P value
Clinical
Setting
ICU	2.18 (1.0-2.79)	26		2.40 (1.51-3.81)	18
Non-ICU	2.06 (1.74-2.43)	40		1.78 (1.52-2.09)	30
Presence of bacteremia
All patients in the study	2.05 (1.70-2.47)	38		1.89 (1.49-2.41)	31
Some/none of the patients	2.16 (1.76-2.65)	28		2.41 (1.72-3.38)	17
Pathogen
MRSA	1.57 (0.95-2.61)	2		1.72 (0.50-5.99)	2
P. aeruginosa	3.25 (1.71-6.17)	4		2.03 (1.15-3.59)	4
Acinetobacter spp.b	7.37 (1.70-31.99)	3		7.59 (2.51-22.91)	2
Any infection assessed	2.00 (1.73-2.31)	54		2.02 (1.63-2.51)	38
Source of infection
Pneumonia only	2.10 (1.50-2.95)	17		2.17 (1.34-3.54)	10
Other/mixed	2.11 (1.81-2.46)	49		2.03 (1.64-2.51)	38
Methodological			0.026			0.004
Timing and location for mortality assessment
Fixed, 28-30 daysb	1.68 (1.32-2.14)	10		1.34 (1.08-1.68)	7
Fixed, other time point	1.59 (1.19-2.12)	9		1.74 (1.23-2.47)	6
In hospital or undefined	2.33 (1.96-2.77)	47		2.36 (1.84-3.02)	35
Appropriate empirical treatment assessment prospectively planned						0.007
Yesb	2.25 (1.92-2.63)	10		2.23 (1.78-2.79)	41
No	1.40 (0.92-2.15)	59		1.48 (1.22-1.80)	7
Appropriate empirical treatment definition						0.095
Only in vitro matching	2.13 (1.78-2.54)	34		2.30 (1.68-3.15)	24
Dose, route, and duration considerations	2.11 (1.58-2.83)	22		1.74 (1.31-2.30)	15
Single aminoglycosidesb,c	1.96 (1.69-2.65)	6		1.56 (1.33-1.82)	5
Other considerationsc	3.97 (1.10-14.36)	4		4.41 (1.00-19.45)	4
Total Newcastle-Ottawa score						0.003
<6b	1.40 (0.94-2.10)	3		1.09 (0.74-1.62)	2
6-8	2.15 (1.87-2.48)	63		2.12 (1.74-2.58)	46
No. of covariates included in multivariable analysis/no. of deaths (ratio) ≥ 10d			Not relevant
Yes				2.19 (1.55-3.08)	17
No				1.98 (1.57-2.51)	31
Reporting of terms of inclusion in multivariable modele			Not relevant			<0.001
Yes				2.55 (1.99-3.28)	28
Nonspecifically				1.70 (0.88-3.27)	8
Nob				1.37 (1.16-1.63)	12
Reporting of no. of patients included in multivariable analysis			Not relevant			0.003
Yes				2.67 (1.92-3.71)	26
Nob				1.53 (1.32-1.78)	22
Adjustment for sepsis severityf			Not relevant			0.070
Yes				2.16 (1.75-2.66)	43
No				1.46 (1.01-2.11)	5
Adjustment for background conditionsg			Not relevant			0.002
Yesb				1.57 (1.37-1.81)	32
No				3.26 (2.11-5.04)	16
Adjustment for neutropenia			Not relevant			0.013
Yes				1.55 (1.26-1.91)	19
No				2.41 (1.83-3.18)	29

^aORs of individual subgroups are shown with 95% confidence intervals and number of studies in each subgroup. Significant differences between subgroups are denoted by a P value.

^bNo significant heterogeneity in the subgroup (I² < 50%).

^cSingle aminoglycosides considered inappropriate for P. aeruginosa or non-fermentative Gram-negative bacteria or double coverage mandated for these bacteria. Other considerations included compliance with guidelines, MIC considerations, etc.

^dStudies that did not report on the type or number of variables included in the multivariable model were considered in the “No” category.

^eReporting of inclusion terms in multivariable model: terms clearly reported (e.g., P < 0.1 in univariate analysis), nonspecific reporting (e.g., all clinically significant variables), or no reporting.

^fDefined as the assessment of a severity score (such as the APACHE score) or septic shock for the adjusted analysis.

^gDefined as the assessment of a comorbidity score (such as the Charlson score) in the adjusted analysis or at least 6 variables out of the variables diabetes, malignancy, renal failure, neutropenia, heart disease, chronic lung disease, liver disease, and functional capacity.

Similarly, there was no association between the mean APACHE score, age, study year, or percentage of patients with septic shock or neutropenia in the meta-regression (Table 3). There was no significant association between risk ratios for mortality and the mortality rate in individual studies (ORs were not used for this analysis due to the inherent correlation between ORs and outcome rates). The funnel plot including all 69 studies was asymmetrical (P = 0.034), with small studies showing no benefit for appropriate empirical treatment possibly missing from the analysis (Fig. 2).

TABLE 3.

Meta-regression analysis to assess the effect of confounders on the association between appropriate empirical antibiotic treatment and all-cause mortality^a

Variable	Unadjusted			Adjusted
	ROR (95% CI)	No. of studies	P value	ROR (95% CI)	No. of studies	P value
Univariate analysis
Septic shock (% of patients)	0.98 (0.35-2.73)	44				0.033
				3.60 (1.11-11.65)	29
Neutropenia (% of patients)	0.49 (0.02-10.07)	16		0.20 (0.01-0.31)	15
Study year (1-yr increment)						0.092
	1.01 (0.99-1.04)	62		1.03 (0.99-1.07)	41
Age (yr [mean for study])	1.02 (0.99-1.05)	53		1.00 (0.96-1.03)	35
Multivariable analysis
Joint test, with septic shockb	Not relevant				34	0.047
Joint test, without septic shockb	Not relevant				48	0.015

^aRatio of ORs (ROR) are shown with 95% confidence intervals and number of studies available for analysis. RORs of >1 denote an increase in ORs positively associated with the confounder assessed and are provided for a 1% prevalence (septic shock, neutropenia) or a 1-year (study year, mean patient age) increment of the confounder assessed. Significant associations are denoted by a P value.

^bJoint test for significant covariates based on random-effects multivariable meta-regression. The P value is for the significance of the joint test on the basis of Knapp-Hartung modification; tau² estimates the between-study variance, and the tau² values were 0.124 and 0.233 for the unadjusted and adjusted analyses, respectively; I²_rest is the percentage of residual variation that is attributable to between-study heterogeneity, and the I²_rest values were 55.48% and 66.83% for the unadjusted and adjusted analyses, respectively; and R²_adj is the proportion of between-study variance explained by the covariates, and the R²_adj values were 52.48% and 36.02% for the unadjusted and adjusted analyses, respectively. The variables included were timing of mortality assessment, prospective plan to assess appropriate empirical treatment, adjustment for background conditions, and reporting of the terms of inclusion and number of patients included in the multivariable analysis. The prevalence of septic shock was reported in only 34 studies and was included in the top model.

FIG. 2.

Funnel plot, unadjusted analysis. Included studies (open circles) are asymmetrically distributed around the pooled odds ratio (vertical line). A more symmetric funnel can be obtain by imputing values for missing studies (black circles), and it is apparent that the missing studies are small studies with ORs of <1, i.e., favoring inappropriate empirical antibiotic treatment.

Adjusted (multivariable) analysis for mortality.

All studies reporting adjusted risk factors for all-cause mortality performed multivariable analysis. Two studies included a propensity score for appropriate empirical treatment in the multivariable analysis (33, 54), and one study performed a propensity-matched analysis (23, 74). Propensity-adjusted effects were slightly smaller than those obtained by multivariable analysis, but only two studies permitted this comparison (54, 74).

The studies collected and assessed various risk factors for mortality for potential inclusion in the multivariable analysis (see the table in the supplemental material). Nearly all studies assessed age, place of acquisition, and source of infection. Formal scores for sepsis severity (e.g., the APACHE score) and underlying conditions (e.g., the Charlson score) were each used in only about 50% of the studies. The median ratio between the number of covariates included in the multivariable model and the number of deaths in the cohort was 8.1 (range, 2 to 51.1). Nine studies did not provide information on the number or type of covariates included.

The pooled adjusted OR of the main analysis was 2.05 (95% CI, 1.69 to 2.49; 48 studies; Fig. 3). Considerable heterogeneity also remained in the multivariable analysis (P < 0.001, I² = 79.7%). In the sensitivity analysis, including “no-benefit” univariate studies, the OR was 1.79 (95% CI = 1.51 to 2.12, 61 studies, I² = 78.9%). In 41 studies reporting both unadjusted and adjusted numerical results, the ORs were 2.35 (95% CI, 1.99 to 2.78) on univariate analysis and 2.32 (95% CI, 1.88 to 2.87) on multivariable analysis.

FIG. 3.

Adjusted analysis of the effect of appropriate empirical treatment on mortality, subgrouped by adjustment to sepsis severity and background conditions (0, no adjustment; 1, covariates representing sepsis severity and background conditions included in adjusted analysis).

As for the unadjusted analysis, a significant advantage to appropriate empirical treatment was maintained in most subgroups assessed. Significant differences between subgroups were observed for several variables, including the time point for mortality assessment, as above, where the advantage was smallest (though still significant) when 28- to 30-day mortality was assessed (Table 2). Studies specifically designed to assess the effects of appropriate empirical treatment were associated with higher ORs than other studies. When the study definition of appropriate empirical treatment included dosing, route, or duration considerations or when single-aminoglycoside therapy was considered inappropriate for Pseudomonas aeruginosa, ORs were lower than those for studies that defined appropriate empirical treatment only by in vitro matching. A high adapted NOS score (lower risk of bias) was associated with larger ORs, but there was little variability in the total score. Similarly, reporting and methods of the multivariable model were associated with the effects reported.

Twenty-eight and 26 studies reported on terms for inclusion of variables and the number of patients included in the model, respectively. Reporting was associated with significantly higher ORs. Only five studies reported on the methods of handling missing values for the variables included. This and the ratio between the number of covariates and the number of deaths were not significantly associated with ORs. Adjustment for background conditions in general and neutropenia in particular were significantly associated with lower ORs, while adjustment for sepsis severity was associated with nonsignificantly higher ORs. The setting (ICU versus non-ICU), assessment of bacteremic patients, pneumonia, or specific pathogens did not significantly affect ORs.

In meta-regression (Table 3), only septic shock was positively associated with ORs, with the ratio of ORs being 3.60 for every 1% increase in the prevalence of septic shock in the study population (95% CI, 1.11 to 11.65). There was a trend for ORs to increase with the study year, but this did not reach statistical significance. All variables significantly associated with ORs explained only a small proportion of between-study variance, where R² was equal to 36.02% and rose to 52.5% in the set of studies that reported on the rate of septic shock at onset (Table 3, multivariable analysis). Only adjustment for background conditions was significantly associated with ORs in the multivariable meta-regression (coefficient, −0.53; standard error, 0.22).

Restricting the analysis to those trials that adjusted for background conditions (including neutropenia) and sepsis severity resulted in a pooled adjusted OR of 1.60 (95% CI, 1.37 to 1.86; 26 studies; Fig. 3), with moderate heterogeneity (46.3%).

DISCUSSION

Decision making regarding antibiotic treatment is unique. On one hand, no treatment equals the efficacy of antibiotics. To place the effect in context of other well-established interventions, the practice of administering aspirin in acute myocardial infarction is based on an OR of 1.30 (95% CI, 1.41 to 1.18) for 7 to 30 days of treatment (number of patients needed to treat [NNT] to prevent one fatal outcome, 41; 95% CI, 30 to 66 patients) (4, 41). The practice of administering low-molecular-weight heparin was estimated on the basis of an OR of 1.16 (95% CI, 1.05 to 1.28), and the NNT is 63 patients (95% CI, 37 to 193) (18). Most interventions in medicine are not based on improved crude survival (e.g., beta-blockers during acute myocardial infarction [1]). In comparison, the pooled odds ratio of appropriate antibiotic treatment during the first 48 h for all-cause mortality in our review was 1.60 (95% CI, 1.37 to 1.86), corresponding to an NNT of 10 (95% CI, 8 to 15), in the set of studies adjusting for background conditions and sepsis severity. Thus, the drive for prescription of antibiotics to patients with suspected infection is clear. On the other hand, there is no other instance in medicine where treatment given to the individual patient affects other patients and the society at large. Present prescription of an antibiotic or a policy to use an antibiotic might mean the loss of availability of this antibiotic and similar antibiotics for future patients (10). In an era of increasing antibiotic resistance, prescription of an antibiotic to one patient might mean no available treatment for future patients (83). The bulk of antibiotic consumption is empirical (72). The balance between preventing deaths from infections and using antibiotics judiciously to prevent resistance development is largely determined by our belief in the benefit of appropriate empirical antibiotic treatment and the magnitude of the benefit.

Estimation of this effect relies on observational studies, since a randomized trial would be unethical. It is difficult to predict the direction of bias caused by the nonrandom allocation of patients to appropriate versus inappropriate empirical treatment. Patients given appropriate empirical treatment might have been more critically ill and thus prescribed broader-spectrum treatment. Conversely, they might have been carriers of more susceptible bacteria and thus healthier (68). Patients with guarded short-term prognoses because of severe underlying conditions might be given inappropriate treatment because antibiotics (or broad-spectrum antibiotics) might be considered futile.

We observed considerable heterogeneity between the studies, with adjusted effects ranging between no effect and ORs above 15. We expected heterogeneity to stem from clinical variables related to patient and infection characteristics. However, only a few clinical variables could be shown to affect results. The percentage of patients with septic shock at onset of infection and adjustment for septic shock were associated with higher ORs, pointing at a larger benefit of appropriate empirical antibiotic treatment among patients with septic shock at infection onset. None of the other clinical variables affected the results, including the study year and setting, the patient's age, presence of bacteremia, source of infection, presence of neutropenia, and causative bacteria, although analysis of the last two variables was based on few studies.

Many methodological variables significantly affected the ORs. Prospective planning, intervention definitions, and follow-up duration impacted OR estimates. Less than half of the studies provided a clear description of the terms for inclusion of variables in the multivariable analysis and the number of patients included in the analysis, and nearly none described the methods used to deal with missing data. Adequate reporting was associated with higher ORs. The number of covariates was frequently high in relation to the number of outcomes in the cohort, and significance or the performance of the model was rarely presented (data not shown). The studies used different risk factors in the multivariable models. Adjustment for background conditions was the most significant variable affecting ORs, where adjustment was associated with smaller effects. It has previously been shown that adjustment for disease severity measures before infection onset (at admission and 24 h before infection onset) is associated with smaller effect estimates for the association between appropriate empirical antibiotic treatment and mortality (89). We could not assess the effects of disease severity measures before infection onset on the results because these were not reported (63), but our findings regarding background conditions probably reflect the same trend. The NOS, whose use is recommended for risk of bias assessment in cohort studies, was not very informative because of the small variability between the studies.

Several limitations of our analysis should be noted. We needed to use assumptions to be able to conduct the meta-analysis, such as the imputation of an OR of 1 for studies reporting qualitatively that appropriate empirical treatment was not significantly associated with mortality on multivariable analysis. For the main analysis, our assumptions were chosen to obtain a conservative effect estimate (it is likely that in these studies the OR was higher than 1 and statistically nonsignificant). Sensitivity analyses showed that results were robust with different assumptions. Publication bias was suggested in our analysis and is partially due to the fact that studies that did not find a significant effect of appropriate empirical treatment on mortality reported results qualitatively and could not be included because no numerical data were reported (22). Infections that are not typically documented microbiologically, mainly community-acquired pneumonia, are ill represented in our analysis. Finally, despite detailed analysis of clinical and methodological variables, we could not fully explain the observed heterogeneity between the studies.

In summary, we showed that, overall, inappropriate empirical antibiotic treatment is significantly associated with all-cause mortality in prospective studies. However, the estimated effect of appropriate empirical antibiotic treatment on mortality reported in observational studies is highly variable. The main determinants of the magnitude of the effect are methodological and relate to study design, outcome definitions, availability of risk factors for adjusted analysis, and the methods used in the multivariable analysis.

Future cohort studies should adhere to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for reporting of observational studies (94) and to existing guidance on reporting of multivariable logistic regression. Specifically, on the basis of our and previous analyses (63, 89), studies should assess 30-day mortality rather than in-hospital or other unfixed follow-up and adjust the effect of appropriate antibiotic treatment for underlying disorders, disease severity before infection onset, and sepsis severity at onset of infection. The same applies for randomized controlled trials of antibiotic or nonantibiotic treatments for sepsis. Future studies should attempt to quantify the negative ecological impact of unnecessary and superfluous antibiotic treatment using the same outcome measures by which appropriate empirical treatment is measured, loss-of-life years. The loss to both the individual treated and society should be accounted for (49).