Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jun 1.
Published in final edited form as: Acad Emerg Med. 2015 May 20;22(6):730–740. doi: 10.1111/acem.12672

Comparison of Clinical Prediction Models for Resistant Bacteria in Community-onset Pneumonia

Wesley H Self 1, Richard G Wunderink 1, Derek J Williams 1, Tyler W Barrett 1, Adrienne H Baughman 1, Carlos G Grijalva 1
PMCID: PMC4464985  NIHMSID: NIHMS677656  PMID: 25996620

Abstract

Objectives

Six recently published algorithms classify pneumonia patients presenting from the community into high- and low-risk groups for resistant bacteria. Our objective was to compare performance of these algorithms for identifying patients infected with bacteria resistant to traditional community-acquired pneumonia antibiotics.

Methods

This was a retrospective study of consecutive adult patients diagnosed with pneumonia in an emergency department and subsequently hospitalized. Each patient was classified as high- or low-risk for resistant bacteria according to the following algorithms: original health care-associated pneumonia (HCAP) criteria, Summit criteria, Brito and Niederman strategy, Shorr model, Aliberti model, and Shindo model. The reference for comparison was detection of resistant bacteria, defined as methicillin-resistant Staphylococcus aureus or gram-negative bacteria resistant to ceftriaxone or levofloxacin.

Results

Six hundred fourteen patients were studied, including 36 (5.9%) with resistant bacteria. The HCAP criteria classified 304 (49.5%) patients as high-risk, with an area under the receiver operating characteristic curve (AUC) of 0.63 (95% CI = 0.54 to 0.72), sensitivity of 0.69 (95% CI = 0.52 to 0.83), and specificity of 0.52 (95% CI = 0.48 to 0.56). None of the other algorithms improved both sensitivity and specificity, or significantly improved the AUC. Compared to the HCAP criteria, the Shorr and Aliberti models classified more patients as high-risk, resulting in higher sensitivity and lower specificity. The Shindo model classified fewer patients as high-risk, with lower sensitivity and higher specificity.

Conclusions

All algorithms for identification of resistant bacteria included in this study had suboptimal performance to guide antibiotic selection. New strategies for selecting empirical antibiotics for community-onset pneumonia are necessary.

INTRODUCTION

In the United States, pneumonia is the leading infectious cause of death, and one of the most common reasons for emergency department (ED) visits and hospital admissions.1–3 The etiology of pneumonia is usually unknown when antibiotics are initiated in the ED.4 Therefore, clinical practice typically involves empirical antibiotic selection targeting the most likely pathogens based on epidemiologic patterns.4,5 Historically, two categories of pneumonia were recognized: community-acquired pneumonia (CAP) and hospital-acquired pneumonia (HAP).6,7 In this paradigm, patients who developed pneumonia outside the hospital were treated with antibiotics targeting common bacteria circulating in the community and susceptible to multiple antibiotic classes, such as Streptococcus pneumoniae and Mycoplasma pneumoniae.6 Meanwhile, patients who contracted pneumonia while hospitalized were treated with broad-spectrum, multi-drug regimens to cover drug-resistant bacteria frequently circulating within hospitals, such as methicillin-resistant Staphylococcus aureus (MRSA) and Pseudomonas aeruginosa.7

During the past 20 years, pneumonia acquired outside the hospital but resistant to traditional CAP antibiotics has become an increasing concern.8–12 To assist with identification of community-onset pneumonia patients with increased risk for infection due to resistant pathogens, the 2005 pneumonia management guidelines from the Infectious Diseases Society of America (IDSA) and American Thoracic Society (ATS) adopted the concept of health care-associated pneumonia (HCAP).5 HCAP was defined as development of pneumonia outside the hospital but in patients who have at least one of six HCAP criteria, indicating recent exposure to health care facilities, or close contact to people carrying resistant pathogens.5 Recommendations for empirical antibiotic selection for HCAP mirrored those for HAP. Because the IDSA/ATS guidelines are the most prominent recommendations for pneumonia antibiotic selection in the United States, and the basis of antibiotic usage in many EDs, adoption of the HCAP criteria resulted in a major shift in antibiotic recommendations, with more patients receiving broad-spectrum antibiotics active against multi-drug resistant pathogens.4,5

At the time of HCAP criteria adoption, the need for further refinement to improve the accuracy of this paradigm was recognized.13–16 In particular, the HCAP criteria have been criticized for low specificity and positive predictive value for resistant bacteria, leading many patients to receive unnecessarily broad antibiotics.15–21 Five alternative algorithms have subsequently been developed with the goal of improving upon the HCAP algorithm for identifying patients with community-onset pneumonia infected with bacteria resistant to traditional CAP antibiotics: 1) Summit criteria,15 2) Brito and Niederman strategy,16,17 3) Shorr model,18,19 4) Aliberti model,20 and 5) Shindo model.21 However, the accuracies of these algorithms for identifying patients with resistant bacteria have not been compared in an independent population. Therefore, the goal of this study was to externally evaluate and compare the accuracy of six published algorithms for identifying patients infected with resistant bacteria in an independent cohort of adults with community-onset pneumonia.

METHODS

Study Design

This was an observational, retrospective study. The local institutional review board approved the study with waiver of informed consent.

Study Setting and Population

Consecutive adult patients (18 years and older) diagnosed with pneumonia in the ED and hospitalized at a single institution between January 1, 2010 and December 31, 2010 were considered. The study hospital is a tertiary-care, academic medical center in the United States with approximately 70,000 annual adult ED patient visits.

The study population included patients who presented to the ED, were diagnosed with pneumonia and hospitalized by the treating physicians, and were confirmed to have clinical and radiographic findings compatible with pneumonia on medical records review. Initially, we generated a list of all adult ED visits during the study period with an ED diagnosis of pneumonia by International Classification of Diseases 9th Revision, Clinical Modification (ICD-9-CM) codes 480.xx-486.xx, 487.0. A trained research coordinator reviewed the medical record corresponding to each of these ED visits. Patients were included in the study population if they met each of the following four criteria at the time of ED evaluation: 1) new radiographic evidence of pneumonia, defined as a chest x-ray or computed tomography (CT) scan interpreted by an attending radiologist as showing pulmonary consolidation, opacity, or infiltrate; 2) one or more of the following respiratory signs or symptoms: new cough, increased sputum production, dyspnea, pleuritic chest pain, respiratory rate > 25 breaths/minute, or respiratory failure requiring mechanical ventilation; 3) one or more of the following infectious signs or symptoms: temperature ≥ 100.4º F or ≤ 95.9º F, white blood cell count > 15,000 cells/mm3 or < 5,000 cells/mm3, chills, or altered mental status compared to baseline; and 4) hospitalized at the conclusion of the ED visit.

Study Protocol

Chart Reviews

For each patient who met inclusion criteria, a research coordinator performed a detailed medical record review of the ED visit and subsequent hospitalization using a standardized data abstraction instrument and review procedure.22 Data to calculate scores for each of the algorithms were collected, as well as demographic, laboratory, and pharmacy data. Two trained and experienced research coordinators performed these reviews; one coordinator completed a review for each patient. The coordinators were blinded to the purpose of the study. All variables were defined prior to the chart review portion of the study. Both coordinators had more than three years of experience conducting clinical pneumonia research, and met with the lead investigator on a biweekly basis throughout the study to resolve data collection questions. During chart reviews, absence of information for a particular variable was assumed to indicate an event did not occur. For example, the variable antibiotics in the prior 90 days was coded as “no” if there were no mention of antibiotics in the medical record, no prescription for an antibiotic, and no order for an antibiotic in the prior 90 days. To assess the quality of the data collection process, the lead investigator reviewed a 10% random sample of records and agreement between the investigator and coordinator was calculated for key variables. This random sample was selected using a random number generator function in Stata/IC 12.1.

Algorithms to Identify Patients with Resistant Bacteria

We evaluated six algorithms designed to identify patients with community-onset pneumonia who should be treated with broad-spectrum antibiotics targeting resistant bacteria. For each algorithm, patients were classified into two categories based on recommendations for antibiotic selection from the original description of each algorithm: traditional CAP antibiotics (algorithm indicated a low risk for resistant bacteria) or broad-spectrum antibiotics targeting resistant bacteria (algorithm indicated a high risk for resistant bacteria) (Table 1). Scoring criteria, definitions, and additional details for each algorithm are available in Data Supplement 1.

Table 1.

Description of the six algorithms designed to identify patients with community-onset pneumonia at risk for resistant bacteria.

Algorithm Method of Development Criteria / Prediction Variables Scoring Patients Classified as High-Risk
HCAP criteria5 Expert opinion

  1. Hospitalized in prior 90 days

  2. Nursing home resident

  3. Home infusion therapy

  4. Chronic dialysis

  5. Home wound care

  6. Family member with resistant bacteria

 Count number of criteria present ≥ 1 criterion
Summit criteria15 Expert opinion

  1. Severe pneumonia

  2. Prior antibiotics

  3. Poor functional status

 Count number of criteria present ≥ 2 criteria
Brito and Niederman strategy16,17 Expert opinion

  1. ICU admission

  2. Immunosuppression

  3. Hospitalized in prior 90 days

  4. Antibiotics in prior 6 months

  5. Poor functional status

 Count number of criteria present ≥ 2 criteria plus fulfillment of HCAP definition
Shorr model18,19 Clinical prediction model

  1. Hospitalized in prior 90 days

  2. Nursing home resident

  3. Chronic dialysis

  4. ICU admission

 Calculate score with differential weighting for variables Score ≥ 1
Aliberti model20 Clinical prediction model

  1. Comorbid conditions (cerebrovascular disease, diabetes, COPD)

  2. Antibiotics in prior 90 days

  3. Immunosuppression

  4. Home wound care

  5. Home infusion therapy

  6. Nursing home resident

  7. Hospitalized in prior 90 days

  8. Chronic renal failure

 Calculate score with differential weighting for variables Score ≥ 3
Shindo model21 Clinical prediction model

  1. Hospitalized in prior 90 days

  2. Immunosuppression

  3. Antibiotics in prior 90 days

  4. Gastric acid suppressive medication

  5. Tube feeding

  6. Non-ambulatory status

 Count number of criteria present ≥ 3 criteria
Open in a new tab

COPD = chronic obstructive pulmonary disease; HCAP = health care-associated pneumonia

In addition to a simple dichotomous algorithm (low-risk vs. high-risk for resistant bacteria), Shindo et al. also described a two-step algorithm that considered the risk for resistant gram negative bacteria and MRSA separately.21 In this two-step Shindo model, patients with three or more Shindo criteria were classified as high-risk for all resistant bacteria (MRSA and resistant gram negative bacteria); additionally, patients with two criteria were classified as high-risk for MRSA if they met one or more of the following MRSA-specific risk factors: prior MRSA infection, chronic hemodialysis, or history of congestive heart failure (Data Supplement 1).21,23

Pathogen Detection

All laboratory testing completed for clinical care within 72 hours of ED presentation was reviewed for each patient. Pathogen detection was limited to tests completed during the first 72 hours after ED presentation, to minimize the risk of identifying organisms not present at the time of initial presentation but acquired in the hospital after admission. Pneumonia etiology was assigned based on the following: blood cultures; high quality sputum cultures, defined as a Bartlett Q score ≥ 2+;24 bronchoalveolar lavage cultures with moderate (3+) or heavy (4+) growth of bacteria considered positive; tracheal aspirate cultures with moderate (3+) or heavy (4+) growth of bacteria considered positive; pleural fluid cultures; Legionella and pneumococcal urinary antigen tests; and polymerase chain reaction (PCR) of nasopharyngeal swabs for atypical bacteria and viruses.

Detection of resistant bacteria was used as the reference for comparison of the algorithms. Patients were classified as infected with resistant bacteria if any of the following were detected: MRSA, Pseudomonas aeruginosa, Acinetobacter baumannii, or other gram negative bacteria resistant to ceftriaxone or levofloxacin.5,19,20 Antibiotic susceptibility testing was performed with the Becton Dickenson (Franklin Lakes, NJ) Phoenix Automatic Microbiology System.25 Patients with other pathogens and those with no pathogens detected were classified as having non-resistant pneumonia.

Antibiotics

Antibiotics initiated in the ED were classified into regimens directed at resistant bacteria and those directed at non-resistant bacteria. An antibiotic regimen was considered to be directed at resistant bacteria if it included any of the following: anti-pseudomonal β-lactam, carbapenem, aztreonam, vancomycin, linezolid, or an aminoglycoside.4,5 Antibiotic regimens classified as directed against non-resistant bacteria included: macrolides, respiratory fluoroquinolones, doxycycline, and β-lactams without pseudomonal coverage.4,5

Data Analysis

For description of the study population, patients were classified either as fulfilling no HCAP criteria, or fulfilling one or more HCAP criteria. Descriptive statistics were used to report patient characteristics, pathogens detected, and antibiotics prescribed in the ED. The Wilcoxon rank sum test and chi-square test were used to evaluate for statistical differences in continuous and categorical data, respectively. Severity of illness was measured with the Pneumonia Severity Index (PSI),26 and modified ATS criteria for severe community acquired pneumonia.4

Inter-rater agreement between study coordinators and the lead investigator was calculated with Cohen’s kappa,27 and included the following variables: patient hospitalized for ≥ 48 hours in the prior 90 days, antibiotics within the prior 90 days, nursing home residence, and blood culture results.

Detection of resistant bacteria was used as the primary outcome to compare algorithms. Each algorithm was used to dichotomize patients into low- and high-risk categories for resistant bacteria based on the ordinal algorithm descriptions. Traditional 2x2 contingency tables were used to calculate sensitivity, specificity, positive and negative likelihood ratios (LR), and accuracy for each algorithm. Consistent with previous descriptions,28,29 a positive LR > 10 and/or negative LR < 0.1 were considered indicators of high potential for clinical utility. Accuracy was calculated with the following equation: ([number of true positives] + [number of true negatives]) / (all patients). Ninety-five percent confidence intervals (CI) were calculated with the exact method for a binomial proportion.

Each algorithm was directly compared to the HCAP criteria, the recommended strategy for antibiotic selection in ATS/ISDA guidelines. For these comparisons, we assumed patients classified as high-risk for resistant bacteria by an algorithm would be treated with broad-spectrum antibiotic regimens, such as piperacillin-tazobactam plus vancomycin plus azithromycin; and patients classified as low-risk for resistant bacteria would be treated with narrower-spectrum antibiotics, such as levofloxacin. For the algorithms that classified more patients as high-risk than the HCAP criteria, we calculated the number of additional patients that would be treated with broad-spectrum antibiotics needed to capture one additional patient with resistant bacteria appropriately treated (broad-spectrum antibiotics: number needed to treat, NNT).28 For example, if an algorithm resulted in ten additional patients receiving piperacillin-tazobactam plus vancomycin plus azithromycin compared to the HCAP criteria, and two of those patients were infected with resistant bacteria, the broad-spectrum antibiotics NNT would be five. The broad-spectrum antibiotics NNT for a particular algorithm represented the number of additional patients treated with piperacillin-tazobactam plus vancomycin plus azithromycin for each additional patient infected with resistant bacteria appropriately with this broad-spectrum regimen, and who would have been undertreated with levofloxacin if the HCAP criteria had been used to select antibiotics instead of this algorithm.

For algorithms that classified fewer patients as high-risk than the HCAP criteria, we calculated the number of additional patients treated with narrow-spectrum antibiotics per patient infected with resistant bacteria inappropriately treated with these narrow-spectrum antibiotics that did not cover the infecting pathogen (narrow-spectrum antibiotics: number needed to undertreat, NNU). For example, if, compared to the HCAP criteria, an algorithm resulted in 50 additional patients receiving levofloxacin instead of piperacillin-tazobactam plus vancomycin plus azithromycin, and five of those patients had resistant bacteria, the narrow-spectrum antibiotics NNU would be ten. In other words, for every ten additional patients treated with levofloxacin successfully, one additional patient would likely have a treatment failure due to bacteria resistant to levofloxacin.

In a separate analysis, scores for each algorithm were used as continuous variables, and receiver-operating characteristic (ROC) curves were developed for the identification of patients with resistant bacteria. Area under the curve (AUC) with 95% CIs was calculated for each curve and compared across algorithms.

Two-sided p values < 0.05 were considered statistically significant. Data were collected with the Research Electronic Data Capture (REDCap).30 Analyses were conducted with Stata/IC 12.1.

Sample Size Calculations

Sample size calculations were based on sensitivity of the Aliberti model using a cutpoint of three or more points to define the high-risk category for resistant bacteria. A priori assumptions for sample size calculations included: sensitivity of the Aliberti model of 0.75, 75% of ED visits with an ICD-9 code for pneumonia meeting our final study eligibility criteria, and 7.5% prevalence of resistant bacteria in the final study cohort. Our aim was to detect a 0.75 sensitivity for the Aliberti model with a 0.95 probability for the lower bound of the 95% CI being greater than 0.50. Using methods described by Flahault et al.,31 we calculated we would need to start the chart review component of the study with a minimum of 747 ED visits for pneumonia. Due to the seasonality of pneumonia, we also determined a priori to include at least one full calendar year of ED visits for pneumonia.

RESULTS

An ICD-9-CM code for pneumonia was given to 1,118 ED visits in 2010 at the study hospital. After detailed medical records review, 614 (54.9%) of these patients were confirmed to meet all eligibility criteria and included in the study population (Figure 1). This included 310 (50.5%) patients without any HCAP criteria, and 304 (49.5%) patients who fulfilled at least one HCAP criterion at the time of ED evaluation (Table 2). Pneumonia severity index scores were higher in patients with ≥ 1 HCAP criterion than in those without any HCAP criteria (median score 97 vs. 80, p < 0.01). A higher percentage of patients with ≥ 1 HCAP criterion also met the modified ATS criteria for severe CAP (20.4% vs. 12.9%, p = 0.01) and were admitted to an intensive care unit (32.2% vs. 21.9%, p < 0.01).

Figure 1.

Figure 1

Open in a new tab

Flow diagram of patient participation.

Table 2.

Patient characteristics.

Characteristic No HCAP Criteria (n = 310) ≥ 1 HCAP Criterion (n = 304) All Patients (N = 614)
Age in years 58 (45–70) 62 (48–75) 60 (46–72)
Female 138 (44.5) 156 (51.3) 294 (47.9)
Race
 White 209 (67.4) 243 (79.9) 452 (73.6)
 Black 92 (29.7) 55 (18.1) 147 (23.9)
 Other 9 (2.9) 6 (2.0) 15 (2.4)
Comorbidities
 Chronic lung disease 94 (30.3) 88 (29.0) 182 (29.6)
 Chronic heart failure 46 (14.8) 47 (15.5) 93 (15.2)
 Chronic renal disease 37 (11.9) 55 (18.1) 92 (15.0)
 Chronic liver disease 17 (5.5) 37 (12.2) 54 (8.8)
 Diabetes mellitus 71 (22.9) 95 (31.3) 166 (27.0)
 Cancer 68 (21.9) 97 (31.9) 165 (26.9)
 Cerebrovascular disease 26 (8.4) 46 (15.1) 72 (11.7)
Radiographic findings
 Multilobar pneumonia 95 (30.7) 84 (27.6) 179 (29.2)
 Pleural effusion 53 (17.1) 104 (34.2) 157 (25.6)
Pneumonia severity
 PSI score 80 (51–107) 97 (75–123) 90 (62–114)
 PSI risk class IV – V 130 (41.9) 175 (57.6) 305 (49.7)
 Altered mental status 44 (14.2) 77 (25.3) 121 (19.7)
 Severe CAP by ATS criteria 40 (12.9) 62 (20.4) 102 (16.6)
Short-term outcomes
 Hospital LOS in hours 74 (47–133) 108 (65–186) 89 (51–153)
 ICU admission 68 (21.9) 98 (32.2) 166 (27.0)
 Mechanical ventilation 24 (7.7) 23 (7.6) 47 (7.7)
 Vasopressor support 6 (1.9) 13 (4.3) 19 (3.1)
 In-hospital death 15 (4.8) 19 (6.3) 34 (5.5)
Open in a new tab

Data are reported as n (%) or median (IQR)

ATS = American Thoracic Society; CAP = community-acquired pneumonia; HCAP = health care-associated pneumonia; ICU = intensive care unit; IQR = interquartile range; LOS = length of stay; PSI = pneumonia severity index

Reviewer Inter-rater Agreement

Inter-rater agreement was calculated between the study coordinators and the investigator who performed a blinded over-read of 61 (10%) charts. Kappas with standard error (SE) between the coordinator review and investigator review for hospitalization in the prior 90 days, antibiotics in the prior 90 days, nursing home residence, and blood culture result were 0.90 (SE ±0.13), 0.90 (SE ±0.13), 1.0 (SE ±0.13), and 1.0 (SE ±0.10), respectively.

Pathogen Detection

At least one bacterial culture was completed in 552 (89.9%) study patients. Only three patients treated with antibiotics active against resistant bacteria did not have at least one bacterial culture completed. Ninety eight (16.0%) patients had pathogens detected, including 74 (12.1%) patients with bacterial pathogens (Table 3). Most of the detected bacteria were identified by blood culture (n = 37), or high-quality sputum culture (n = 30). Staphylococcus aureus, Streptococcus pneumoniae, and Pseudomonas aeruginosa were the most common bacteria identified. Resistant bacteria were detected in 36 (5.9%) patients, including 25 (8.2%) patients with ≥ 1 HCAP criterion, and 11 (3.5%) patients without any HCAP criteria (p < 0.01) (Table 3).

Table 3.

Pathogens detected.

Pathogens No HCAP criteria (n = 310) ≥ 1 HCAP criteria (n = 304) All patients (N = 614)
Any pathogen detected 50 (16.1) 48 (15.8) 98 (16.0)
Any bacteria 35 (11.3) 39 (12.8) 74 (12.1)
Resistant bacteria 11 (3.5) 25 (8.2) 36 (5.9)
 Methicillin-resistant S. aureus (MRSA) 3 (1.0) 12 (4.0) 15 (2.4)
 Pseudomonas aeruginosa 3 (1.0) 6 (2.0) 9 (1.5)
 Acinetobacter bumannii 0 1 (0.3) 1 (0.2)
 Serratia marcescens (resistant) 0 1 (0.3) 1 (0.2)
 Escherichia coli (resistant) 1 (.03) 0 1 (0.2)
 P. aeruginosa & MRSA 3 (1.0) 2 (0.7) 5 (0.8)
 P. aeruginosa & Stenotrophomonas maltphilia 0 2 (0.7) 2 (0.3)
 Stenotrophomonas maltphilia (resistant) 0 1 (0.3) 1 (0.2)
 E. coli (resistant) & Streptococcus pneumoniae 1 (0.3) 0 1 (0.2)
Non-resistant bacteria 24 (7.7) 14 (4.6) 38 (6.2)
 Streptococcus pneumonia 12 (3.9) 5 (1.6) 17 (2.8)
 Haemophilus influenza 2 (0.7) 1 (0.3) 3 (0.5)
 Methicillin-susceptible S. aureus (MSSA) 3 (1.0) 4 (1.3) 7 (1.1)
 Group A Streptococcus 1 (0.3) 0 1 (0.2)
 Group B Streptococcus 1 (0.3) 0 1 (0.2)
 Group C Streptococcus 2 (0.7) 0 2 (0.3)
 Streptococcus intermedius 1 (90.3) 0 1 (0.16)
 α-hemolytic Streptococcus 1 (0.3) 0 1 (0.16)
 Legionella sp. 0 1 (0.3) 1 (0.2)
 E. coli (sensitive) 1 (0.3) 1 (0.3) 2 (0.3)
 Klebsiella pneumoniae (sensitive) 0 1 (0.3) 1 (0.2)
 H. influenzae & α-hemolytic Streptococcus 0 1 (0.3) 1 (0.2)
Viruses without bacteria 12 (3.9) 8 (2.6) 20 (3.3)
Aspergillus fumigatus 1 (0.3) 1 (0.3) 2 (0.3)
Mycobacterium Avium Complex 2 (0.7) 0 2 (0.33)
Open in a new tab

Data reported as n (%)

Antibiotics

A majority of patients (62.4%) were treated in the ED with broad-spectrum antibiotics directed at resistant bacteria, with a higher percentage of patients fulfilling ≥ 1 HCAP criterion treated with antibiotics directed at resistant bacteria than patients without any HCAP criteria (83.9% vs 41.3%, p < 0.01). Specific antibiotic regimens are shown in Table 4.

Table 4.

Empirical antibiotics administered in the emergency department.

Antibiotic Regimen, n (%) No HCAP criteria (n = 310) ≥ 1 HCAP criteria (n = 304) All patients (N = 614)
Antibiotics directed at resistant bacteria 128 (41.3) 255 (83.9) 383 (62.4)
 Regimen A:

  1. antipseudomonal β-lactam, carbapenem, or aztreonam

  2. vancomycin or linezolid

  3. +/− fluoroquinolone, azithromycin, or doxycycline

 57 (18.4) 146 (48.0) 203 (33.1)
 Regimen B:

  1. antipseudomonal β-lactam, carbapenem, or aztreonam

  2. fluoroquinolone, azithromycin, or doxycycline

 21 (6.8) 28 (9.2) 49 (8.0)
 Regimen C:

  1. fluoroquinolone, azithromycin, or doxycycline

  2. vancomycin or linezolid

 13 (4.2) 31 (10.2) 44 (7.2)
 Regimen D:

  • Other regimen directed at resistant bacteria

 37 (11.9) 50 (16.4) 87 (14.2)
Antibiotics directed at non-resistant bacteria 180 (58.1) 48 (15.8) 228 (37.1)
 Regimen E:

  • Fluoroquinolone, azithromycin, or doxycycline monotherapy

 111 (35.8) 33 (10.9) 144 (23.5)
 Regimen F:

  1. β-lactam without pseudomonal coverage

  2. fluoroquinolone, azithromycin, or doxycycline

 68 (21.9) 15 (4.9) 83 (13.5)
 Regimen G:

  • β-lactam without pseudomonal coverage monotherapy

 1 (0.3) 0 (0) 1 (0.2)
No antibiotics in the ED 2 (0.6) 1 (0.3) 3 (0.5)
Open in a new tab

HCAP = health care-associated pneumonia

Algorithm Comparisons

Accuracy for Detection of Resistant Bacteria

Results of dichotomizing algorithm scores into high-risk and low-risk categories are summarized in Table 5. The HCAP criteria had a sensitivity and specificity of 0.69 (95% CI = 0.52 to 0.83), and 0.52 (95% CI = 0.48 to 0.56), respectively. Compared to the HCAP criteria, the Summit criteria, Brito and Niederman strategy, and Shindo model classified fewer patients as high-risk, leading to lower sensitivity and higher specificity. The Shorr and Aliberti models classified more patients as high-risk than the HCAP criteria, resulting in higher sensitivity but lower specificity. None of the new algorithms improved both sensitivity and specificity compared to the original HCAP criteria. Furthermore, none of the algorithms had a positive LR > 10 or a negative LR < 0.1.

Table 5.

Test characteristics of six algorithms for identification of patients with community-onset pneumonia who have resistant bacteria.

Algorithm (cutpoint) n (%) Classified as Low Risk n (%) Classified as High Risk Sensitivity (95% CI) Specificity (95% CI) Positive LR (95% CI) Negative LR (95% CI) Accuracy (95% CI)
HCAP criteria (cutpoint: ≥ 1 criterion) 310 (50.4) 304 (49.5) 0.69 (0.52–0.83) 0.52 (0.48–0.56) 1.44 (1.14–1.82) 0.59 (0.36–0.97) 0.53 (0.49–0.57)
Summit criteria (cutpoint: ≥2 criteria) 525 (85.5) 89 (14.5) 0.31 (0.16–0.47) 0.87 (0.83–0.89) 2.26 (1.33–3.86) 0.80 (0.65–1.00) 0.83 (0.80–0.86)
Brito and Niederman strategy (cutpoint: ≥ 2 criteria plus HCAP definition fulfilled) 347 (56.5) 267 (43.5) 0.67 (0.49–0.80) 0.58 (0.54–0.62) 1.59 (1.23–2.04) 0.58 (0.36–0.92) 0.58 (0.54–0.62)
Shorr model (cutpoint: score ≥ 1) 255 (41.5) 359 (58.4) 0.75 (0.57–0.87) 0.43 (0.39–0.47) 1.31 (1.07–1.60) 0.59 (0.33–1.04) 0.44 (0.40–0.48)
Aliberti model (cutpoint: score ≥ 3) 283 (46.1) 331 (53.9) 0.72 (0.55–0.85) 0.47 (0.43–0.51) 1.37 (1.10–1.70) 0.59 (0.35–1.00) 0.49 (0.45–0.53)
Shindo model (cutpoint: ≥ 3 criteria) 417 (67.9) 197 (32.1) 0.50 (0.33–0.67) 0.69 (0.65–0.73) 1.61 (1.14–2.29) 0.72 (0.52–1.00) 0.68 (0.64–0.72)
Open in a new tab

HCAP = health care-associated pneumonia;

Numbers Need to Treat or Undertreat

Compared to the HCAP criteria, the Summit criteria, Brito and Niederman strategy, and Shindo model resulted in fewer patients classified as high-risk overall, and fewer patients without resistant bacteria classified as high-risk (improved specificity). Compared to the HCAP criteria, using the Summit criteria resulted in 215 fewer patients classified as high-risk at the expense of 14 additional patients with resistant bacteria classified as low risk (narrow-spectrum antibiotics NNU = 15.4). The Brito and Niederman strategy resulted in 37 fewer patients classified as high-risk, at the expense of one additional patient with resistant bacteria classified as low-risk (narrow-spectrum antibiotics NNU = 37.0). The Shindo model classified 107 fewer patients as high-risk compared to the HCAP criteria, with seven additional patients with resistant bacteria incorrectly classified as low risk (narrow-spectrum antibiotics NNU = 15.3).

Compared to the HCAP criteria, the Shorr and Aliberti models classified more patients as high-risk overall, and correctly classified more patients with resistant bacteria as high-risk (improved sensitivity). Compared to the HCAP criteria, the Shorr model resulted in 55 additional patients classified as high-risk, with the benefit of correctly identifying two additional patients with resistant bacteria (broad-spectrum antibiotics NNT = 27.5). Similarly, the Aliberti model classified 27 additional patients as high-risk, with one additional patient correctly identified (broad-spectrum antibiotics NNT = 27.0).

ROC Curves

When scores for each of the algorithms were used in a continuous fashion, increasing score for each algorithm positively correlated with increasing prevalence of resistant bacteria (Figure 2). ROC AUC was similar for each of the six algorithms, ranging from 0.63 to 0.67 (p = 0.62) (Data Supplement 2).

Figure 2.

Figure 2

Open in a new tab

Prevalence of resistant bacteria in 614 patients with community-onset pneumonia stratified by the six algorithms designed to identify patients with resistant bacteria: (A) HCAP Criteria, (B) Summit Criteria, (C) Brito and Niederman Strategy (D) Shorr Model, (E) Aliberti Model, (F) Shindo Model.

Two-step Shindo Algorithm with MRSA-specific Risk Factors

Of the 157 patients with two Shindo criteria, 11 (7.0%) had resistant bacteria detected, including five with MRSA and six with resistant gram negative bacteria; 52 (33.1%) of these 157 patients had at least one MRSA-specific risk factor. Using the two-step Shindo algorithm, 21 (58.3%) of the 36 patients with resistant bacteria overall would have been correctly identified as high-risk (Data Supplement 3). Compared to the simple Shindo model (≥ 3 Shindo criteria signifying high risk for all resistant bacteria), using the two-step Shindo algorithm would have resulted in 52 additional patients receiving antibiotic therapy targeting MRSA; three of these 52 patients had MRSA infections.

DISCUSSION

Identifying which pneumonia patients presenting from the community have high enough risk for resistant bacteria to empirically initiate broad-spectrum antibiotics typically reserved for nosocomial infections is an important challenge. The prevalence of resistant bacteria among patients with community-onset pneumonia appears to range between approximately 3% and 30%, depending on geographic region and characteristics of the population.20,21,28,32 When resistant bacteria are responsible for infection, failure to initiate antibiotics that adequately treat the causative pathogen is a major risk factor for poor clinical outcomes.33–36 Therefore, rapid treatment with broad-spectrum antibiotics is essential for the management of pneumonia patients with a high risk of resistant bacteria.5 However, overuse of broad-spectrum antibiotics in patients without resistant bacteria may have significant negative consequences, including drug toxicity, antibiotic-associated infections, promoting antibiotic resistance, increased health care costs, and possibly increased mortality.37–43

In this observational study of 614 patients with community-onset pneumonia, 36 (5.9%) were found to have bacteria resistant to traditional CAP antibiotics. Despite this relatively low prevalence, clinicians frequently used empirical broad-spectrum antibiotics, with 62.4% of patients receiving antibiotics targeting resistant bacteria. This highlights a significant opportunity to reduce unnecessary antibiotic use by developing tools clinicians trust for identifying patients with resistant bacteria.

Current ATS/IDSA pneumonia guidelines recommended the HCAP criteria as a tool to assist clinicians with antibiotic selection.5 Application of the HCAP criteria in our study resulted in 25 of the 36 patients with resistant bacteria correctly identified as high-risk (sensitivity = 0.69). However, only 25 of the 304 patients classified as high-risk by HCAP criteria had resistant bacteria detected, resulting in a low positive likelihood ratio (1.44) and positive predictive value (0.082). The AUC for the HCAP criteria was low at 0.63, which is similar to the findings of Chalmers et al.28 in a recent meta-analysis of the HCAP literature.

Recently, the five alternative algorithms evaluated in this study were developed with the goal of improving accuracy beyond that of the original HCAP Criteria.15–21 In our study cohort, none of these newer algorithms significantly increased the AUC or improved both sensitivity and specificity compared to the HCAP criteria. Positive and negative likelihood ratios for each of the algorithms fell well short of the thresholds of 10 and 0.1, respectively, commonly used to identify potentially useful clinical tests.28,29 The Summit criteria15 and Brito and Niederman strategy,16 both based on expert opinion and designed to improve specificity above the HCAP criteria, did indeed classify fewer patients as high-risk for resistant bacteria, with improved specificity. However, these strategies were associated with declines in sensitivity. Among the three algorithms developed as clinical prediction models, the Shorr model18,19 and Aliberti model20 would have led to more patients receiving broad-spectrum antibiotics, while the Shindo model21 would have resulted in fewer patients with broad-spectrum antibiotics. The performance of each of these models was weaker in this external cohort compared with the study populations used to derive them.18,20,21

Our results highlight that all of the available algorithms, which rely on similar epidemiological risk factors, have suboptimal accuracy for guiding appropriate antibiotic selection. A new approach is needed. As technology for infectious disease diagnostics continues to mature, a more promising approach in the future may be rapid diagnostic testing for specific pathogens, with a shift away from epidemiological risk factors and toward direct pathogen testing to guide antibiotic selection.44,45

Our study had both strengths and limitations. Strengths include systematic and standardized data collection methods using a priori definitions for all variables. Additionally, a single study population was used to compare several clinical prediction models that were derived based on different patient populations. For example, the Shorr model18,19 was derived from a population composed of pneumonia patients who had identified bacterial pathogens, while the Aliberti20 and Shindo21 models were derived from more general populations of pneumonia patients that included patients without identified bacterial etiologies. Furthermore, the Shorr model was derived in the United States, while the Aliberti model was developed in Italy, and the Shindo model was developed in Japan. The current study evaluated these models in the same U.S. population of hospitalized pneumonia patients, facilitating direct comparison of the models in a population relevant for clinicians practicing in U.S. EDs.

LIMITATIONS

Limitations of our study include its retrospective design and reliance on standard diagnostic testing to identify pathogens. The initial step of generating our study population involved identifying ED visits with ICD-9-CM codes for pneumonia; a patient with pneumonia but who did not receive an ICD-9-CM code for pneumonia due to a coding error or coding of an alternative diagnosis, such as sepsis or respiratory failure, would not have been included. Some patients infected with resistant bacteria may have been misclassified due to the insensitivity of culturing for pneumonia pathogens and a full set of cultures not being performed on all patients. However, only three (0.5%) patients were treated with antibiotics active against resistant bacteria without bacterial cultures complete. This helps lessen concerns that substantially more resistant bacteria would have been identified if prospective, systematic bacterial culturing were performed.

Studies that rely on retrospective chart review for data collection are limited by the information available in the medical record. In this study, we extensively examined the medical record for data to calculate scores for each of the algorithms evaluated. We believe these data reflect the information clinicians had available at the time of patient management in the ED, and thus, estimate how each algorithm would have been employed in the clinical setting; however, these chart review data are subject to potential misclassification. Additionally, this was a single-center study at a tertiary care hospital, which may have a higher prevalence of patients with complex comorbidities and pneumonia caused by resistant bacteria than a typical community hospital. Further study with larger sample sizes and in diverse settings is indicated.

CONCLUSIONS

In this external evaluation of six algorithms developed to predict resistant bacterial infection among patients with community-onset pneumonia, accuracy was suboptimal for all the algorithms, and alternative approaches for selecting antibiotics are needed.

Supplementary Material

Supp FigureS1-S3

Acknowledgments

Funding: No industry funding was provided for this work. During the study period, WHS was supported by the National Center for Advancing Translational Sciences at the National Institutes for Health [KL2 TR000446]. DJW is support by a grant from the National Institute of Allergy and Infectious Diseases at the National Institutes of Health [K23AI104779]. TWB is supported by a grant from the National Heart, Lung and Blood Institute at the National Institutes of Health [K23HL102069]. Support for the Research Electronic Data Capture (REDCap) program was provided by National Center for Advancing Translational Sciences at the National Institutes of Health [UL1 TR000445].

Footnotes

Reprints: Not available from the authors.

Meetings: This study has not been previously presented.

Disclosures: WHS is a Scientific Advisory Board Member for BioFire Diagnostics. The authors have no other potential conflicts of interest or financial relationships to disclose.

References

  • 1.Kung HC, Hoyert DL, Xu J, Murphy SL. Deaths: final data for 2005. Natl Vital Stat Rep. 2008;56:1–20. [PubMed] [Google Scholar]
  • 2.Self WH, Grijalva CG, Zhu Y, et al. Rates of emergency department visits due to pneumonia in the United States, July 2006-June 2009. Acad Emerg Med. 2013;20:957–60. doi: 10.1111/acem.12203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Agency for Healthcare Research and Quality. [Accessed Mar 10, 2015];Pneumonia most common reason for hospitalization. Available at: http://archive.ahrq.gov/news/nn/nn070208.htm.
  • 4.Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44(Suppl 2):S27–72. doi: 10.1086/511159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.American Thoracic Society, Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med. 2005;171:388–416. doi: 10.1164/rccm.200405-644ST. [DOI] [PubMed] [Google Scholar]
  • 6.Niederman MS, Bass JB, Jr, Campbell GD, et al. Guidelines for the initial management of adults with community-acquired pneumonia: diagnosis, assessment of severity, and initial antimicrobial therapy. American Thoracic Society. Medical Section of the American Lung Association. Am Rev Respir Dis. 1993;148:1418–26. doi: 10.1164/ajrccm/148.5.1418. [DOI] [PubMed] [Google Scholar]
  • 7.American Thoracic Society. Hospital-acquired pneumonia in adults: diagnosis, assessment of severity, initial antimicrobial therapy, and preventive strategies. A consensus statement, American Thoracic Society, November 1995. Am J Respir Crit Care Med. 1996;153:1711–25. doi: 10.1164/ajrccm.153.5.8630626. [DOI] [PubMed] [Google Scholar]
  • 8.Friedman ND, Kaye KS, Stout JE, et al. Health care-associated bloodstream infection in adults: a reason to change the accepted definition of community-acquired infections. Ann Intern Med. 2002;137:791–7. doi: 10.7326/0003-4819-137-10-200211190-00007. [DOI] [PubMed] [Google Scholar]
  • 9.Kollef MH, Shorr A, Tabak YP, Gupta V, Liu LZ, Johannes RS. Epidemiology and outcomes of health-care associated pneumonia: results from a large US database of culture-positive pneumonia. Chest. 2005;128:3854–62. doi: 10.1378/chest.128.6.3854. [DOI] [PubMed] [Google Scholar]
  • 10.Muder RR, Aghababian RV, Loeb MB, Solot JA, Higbee M. Nursing home-acquired pneumonia: an emergency department algorithm. Curr Med Res Opin. 2004;20:1309–20. doi: 10.1185/030079904125004376. [DOI] [PubMed] [Google Scholar]
  • 11.Lobo LJ, Reed KD, Wunderink RG. Expanded clinical presentation of community-acquired methicillin-resistant Staphylococcus aureus pneumonia. Chest. 2010;138:130–6. doi: 10.1378/chest.09-1562. [DOI] [PubMed] [Google Scholar]
  • 12.Moran GL, Krishnadasan A, Gorwitz RJ, et al. Prevalence of methicillin-resistant Staphylococcus aureus as an etiology of community-acquired pneumonia. Clin Infect Dis. 2012;54:1126–33. doi: 10.1093/cid/cis022. [DOI] [PubMed] [Google Scholar]
  • 13.Fujitani S, Yu VL. A new category—healthcare-associated pneumonia: a good idea, but problems with its execution. Eur J Clin Microbiol Infect Dis. 2006;25:627–31. doi: 10.1007/s10096-006-0197-9. [DOI] [PubMed] [Google Scholar]
  • 14.Wunderink RG. Healthcare-associated bacteremia: stirring the mud. Crit Care Med. 2006;34:2685–6. doi: 10.1097/01.CCM.0000240789.93764.0F. [DOI] [PubMed] [Google Scholar]
  • 15.Kollef MH, Morrow LE, Baughman RP, et al. Health-care associated pneumonia (HCAP): a critical appraisal to improve identification, management, and outcomes—Proceedings of the HCAP summit. Clin Infect Dis. 2008;46(suppl 4):S296–334. doi: 10.1086/526355. [DOI] [PubMed] [Google Scholar]
  • 16.Brito V, Niederman MS. Healthcare-associated pneumonia is a heterogeneous disease, and all patients do not need the same broad-spectrum antibiotic therapy as complex nosocomial pneumonia. Curr Opin Infect Dis. 2009;22:316–25. doi: 10.1097/QCO.0b013e328329fa4e. [DOI] [PubMed] [Google Scholar]
  • 17.Maruyama T, Fujisawa T, Okuno M, et al. A new strategy for healthcare-associated pneumonia: a 2-year prospective multicenter cohort study using risk factors for multidrug resistant pathogens to select initial empiric therapy. Clin Infect Dis. 2013;57:1373–83. doi: 10.1093/cid/cit571. [DOI] [PubMed] [Google Scholar]
  • 18.Shorr AF, Zilberberg MD, Micek ST, Kollef MH. Prediction of infection due to antibiotic-resistant bacteria by select risk factors for health care-associated pneumonia. Arch Intern Med. 2008;168:2205–10. doi: 10.1001/archinte.168.20.2205. [DOI] [PubMed] [Google Scholar]
  • 19.Shorr AF, Zilberberg MD, Reichley R, et al. Validation of a clinical score for assessing the risk of resistant pathogens in patients with pneumonia presenting to the emergency department. Clin Infect Dis. 2012;54:193–8. doi: 10.1093/cid/cir813. [DOI] [PubMed] [Google Scholar]
  • 20.Aliberti S, Di Pasquale M, Zanaboni AM, et al. Stratifying risk factors for multidrug-resistant pathogens in hospitalized patients coming from the community with pneumonia. Clin Infect Dis. 2012;54:470–8. doi: 10.1093/cid/cir840. [DOI] [PubMed] [Google Scholar]
  • 21.Shindo Y, Ryota I, Kobayashi D, et al. Risk factors for drug-resistant pathogens in community-acquired and healthcare-associated pneumonia. Am J Respir Crit Care Med. 2013;188:985–95. doi: 10.1164/rccm.201301-0079OC. [DOI] [PubMed] [Google Scholar]
  • 22.Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews in emergency medicine research: where are the methods. Ann Emerg Med. 1996;27:305–8. doi: 10.1016/s0196-0644(96)70264-0. [DOI] [PubMed] [Google Scholar]
  • 23.Wunderink RG. Community-acquired pneumonia versus healthcare-associated pneumonia: the returning pendulum. Am J Respir Crit Care Med. 2013;188:896–8. doi: 10.1164/rccm.201308-1536ED. [DOI] [PubMed] [Google Scholar]
  • 24.Bartlett JG, Ryan KJ, Smith TF, Wilson WR. Laboratory Diagnosis of Lower Respiratory Tract Infections. In: Washington JA, editor. Cumitech 7A. Washington DC: American Society for Microbiology; 1978. [Google Scholar]
  • 25. [Accessed Mar 13, 2015];BD Phoenix Laboratory Procedure. Available at: https://www.bd.com/ds/technicalCenter/clsi/clsi-Phoenix_GramNegative_V5.15_V4.31.pdf.
  • 26.Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336:243–50. doi: 10.1056/NEJM199701233360402. [DOI] [PubMed] [Google Scholar]
  • 27.Landis JR, Koch CG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. [PubMed] [Google Scholar]
  • 28.Chalmers JD, Rother C, Salih W, Ewig S. Healthcare-associated pneumonia does not accurately identify potentially resistant pathogens: a systematic review and meta-analysis. Clin Infect Dis. 2014;58:330–9. doi: 10.1093/cid/cit734. [DOI] [PubMed] [Google Scholar]
  • 29.Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004;329:168–9. doi: 10.1136/bmj.329.7458.168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Flahault A, Cadilhac M, Thomas G. Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol. 2005;58:859–62. doi: 10.1016/j.jclinepi.2004.12.009. [DOI] [PubMed] [Google Scholar]
  • 32.Schreiber MP, Chan CM, Shorr AF. Resistant pathogens in nonnosocomial pneumonia and respiratory failure. Chest. 2010;137:1283–8. doi: 10.1378/chest.09-2434. [DOI] [PubMed] [Google Scholar]
  • 33.Kollef MH, Sherman G, Ward S, Fraser VJ. Inadequate antimicrobial treatment of infections: a major factor for hospital mortality among critically ill patients. Chest. 1999;115:462–74. doi: 10.1378/chest.115.2.462. [DOI] [PubMed] [Google Scholar]
  • 34.Moehring RW, Sloane R, Chen LF, et al. Delays in appropriate antibiotic therapy for gram-negative bloodstream infections: a multicenter, community hospital study. PLoS One. 2013;8:e76225. doi: 10.1371/journal.pone.0076225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Chen HC, Lin WL, Lin CC, et al. Outcome of inadequate empirical antibiotic therapy in emergency department patients with community-onset bloodstream infections. J Antimicrob Chemother. 2013;68:947–53. doi: 10.1093/jac/dks475. [DOI] [PubMed] [Google Scholar]
  • 36.Harbarth S, Garbino J, Pugin J, Romand JA, Lew D, Pittet D. Inappropriate initial antibiotic therapy and its effect on survival in a clinical trial of immunomodulating therapy for severe sepsis. Am J Med. 2003;155:529–35. doi: 10.1016/j.amjmed.2003.07.005. [DOI] [PubMed] [Google Scholar]
  • 37.Kett DH, Cano E, Quartin AA, et al. Implementation of guidelines for management of possible multidrug resistant pneumonia in intensive care: an observational, multicenter cohort study. Lancet Infect Dis. 2011;11:181–9. doi: 10.1016/S1473-3099(10)70314-5. [DOI] [PubMed] [Google Scholar]
  • 38.Paul M, Benuri-Silbiger I, Soares-Weiser K, Leibovici L. Beta lactam monotherapy versus beta lactam-aminoglycoside combination therapy for sepsis in immunocompetent patients: a systematic review and meta-analysis of randomized trials. BMJ. 2004;328:668. doi: 10.1136/bmj.38028.520995.63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Kelly CP, Pothoulakis C, LaMont JT. Clostridium difficile colitis. N Engl J Med. 1994;330:257–62. doi: 10.1056/NEJM199401273300406. [DOI] [PubMed] [Google Scholar]
  • 40.Arias CA, Murray BE. Antibiotic-resistant bugs in the 21st Century—A clinical super-challenge. N Engl J Med. 2009;360:439–43. doi: 10.1056/NEJMp0804651. [DOI] [PubMed] [Google Scholar]
  • 41.Graffunder EM, Venezia RA. Risk factors associated with nosocomial methicillin-resistant Staphylococcus aureus (MRSA) infection including previous use of antimicrobials. J Antimicrob Chemother. 2002;49:999–1005. doi: 10.1093/jac/dkf009. [DOI] [PubMed] [Google Scholar]
  • 42.Stevenson KB, Balada-Llasat JM, Bauer K, et al. The economics of antimicrobial stewardship: the current state of the art and applying the business case model. Infect Cont Hosp Epidemiol. 2012;33:389–97. doi: 10.1086/664910. [DOI] [PubMed] [Google Scholar]
  • 43.Attridge RT, Frei CR, Restrepo MI, et al. Guideline-concordant therapy and outcomes in healthcare-associated pneumonia. Eur Respir J. 2011;38:878–87. doi: 10.1183/09031936.00141110. [DOI] [PubMed] [Google Scholar]
  • 44.Caliendo AM, Gilbert DN, Cinocchio CC, et al. Better tests, better care: improved diagnostics for infectious diseases. Clin Infect Dis. 2013;57(Suppl 3):S139–70. doi: 10.1093/cid/cit578. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Restrepo MI, Aliberti S. Healthcare-associated pneumonia: where do we go next? Clin Infect Dis. 2014;58:340–1. doi: 10.1093/cid/cit738. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supp FigureS1-S3
NIHMS677656-supplement-Supp_FigureS1-S3.pdf (708.8KB, pdf)

RESOURCES