Abstract
OBJECTIVE
Development of a numerical score to measure the microbial spectrum of antibiotic regimens (spectrum score) and method to identify antibiotic de-escalation events based on application of the score.
DESIGN
Web-based modified Delphi method.
PARTICIPANTS
Physician and pharmacist antimicrobial stewards practicing in the United States recruited through infectious diseases–focused listservs.
METHODS
Three Delphi rounds investigated: organisms and antibiotics to include in the spectrum score, operationalization of rules for the score, and de-escalation measurement. A 4-point ordinal scale was used to score antibiotic susceptibility for organism-antibiotic domain pairs. Antibiotic regimen scores, which represented combined activity of antibiotics in a regimen across all organism domains, were used to compare antibiotic spectrum administered early (day 2) and later (day 4) in therapy. Changes in spectrum score were calculated and compared with Delphi participants’ judgments on de-escalation with 20 antibiotic regimen vignettes and with non-Delphi steward judgments on de-escalation of 300 pneumonia regimen vignettes. Method sensitivity and specificity to predict expert de-escalation status were calculated.
RESULTS
Twenty-four participants completed all Delphi rounds. Expert support for concepts utilized in metric development was identified. For vignettes presented in the Delphi, the sign of change in score correctly classified de-escalation in all vignettes except those involving substitution of oral antibiotics. The sensitivity and specificity of the method to identify de-escalation events as judged by non-Delphi stewards were 86.3% and 96.0%, respectively.
CONCLUSIONS
Identification of de-escalation events based on an algorithm that measures microbial spectrum of antibiotic regimens generally agreed with steward judgments of de-escalation status.
Antibiotic de-escalation has been proposed as a key component of antibiotic stewardship.1 De-escalation generally refers to a reduction in the spectrum of administered antibiotics through the discontinuation of antibiotics providing activity against nonpathogenic organisms, discontinuation of antibiotics with similar activity, or switching to more targeted therapy once a patient is clinically stable.1–3 De-escalation may also include stopping antibiotics altogether, on the basis of clinical criteria and negative culture results, or switching antibiotics from intravenous to oral routes.4,5 De-escalation has been defined and measured subjectively on the basis of individual opinions of what constitutes de-escalation or on the basis of objective but incomplete measures (eg, a reduction in the number of antibiotics administered).2,6–11 A fundamental problem is that conceptually, antibiotic spectrum remains poorly defined. Further, the ability to compare rates of antibiotic de-escalation between facilities is limited by a lack of standard objective measurement criteria.
We speculated that a numerical score based on an antibiotic regimen’s degree of microbial activity might be useful. Further, if antibiotic scores are calculated for each antibiotic administered during each day of therapy, a daily antibiotic regimen spectrum score could be calculated. Finally, we hypothesized that de-escalation could be measured by comparing spectrum scores early and later in treatment. (Figure 1).
FIGURE 1.
Concept of spectrum score for purpose of measuring antibiotic de-escalation. A, Hypothetical metric for calculating a spectrum score for an individual antibiotic. Clinically relevant species may refer to an organism such as Escherichia coli, whereas an organism group might refer to Enterobacteriaceae. B, Hypothetical application of a spectrum score to measure antibiotic de-escalation based on antibiotic regimen spectrum at different time points during antibiotic treatment. Patients frequently receive multiple antibiotics during a single hospital admission. Theoretically, a reduction in the daily summary spectrum score on hospital day 3 or 4 after diagnostic tests and clinical response are assessed might be used to signify de-escalation. Az, azithromycin; Ctx, ceftriaxone; Imp, imipenem; Vm, vancomycin.
Study aims include (1) developing a numerical spectrum score to compare the spectrum of antibiotic activity between treatment regimens and (2) defining de-escalation according to criteria that utilize the scoring metric. A long-term research aim is to construct an algorithm that can be applied to electronic medical records to measure facility-level de-escalation rates in patients with healthcare-associated pneumonia (HCAP).
METHODS
Between May 2012 and February 2013, experts in the field of antibiotic stewardship practicing in the United States participated in 3 rounds of a web-based modified Delphi method to aid in development of an antibiotic spectrum scoring metric and rules for application to identify de-escalation events. In the absence of evidence-based criteria to accomplish study aims, Delphi methods were used to guide decision making in development and application of the metric. This research complies with all federal guidelines and Veterans Affairs (VA) policies relative to human subjects and research.
Initially, we reviewed literature and established likely concepts of interest for exploration in the modified Delphi. Content domains were identified from which Delphi questions were created: (1) organisms and antibiotics to include in the spectrum score, (2) operationalization of rules for the score (ie, metric scoring and management of duplicate coverage in combination therapy), and (3) application then subsequent assessment of a prototype spectrum score method. Questions were tested by 2 nonparticipant, nonresearch infectious diseases clinicians prior to dissemination to the Delphi panel for intent and clarity.
Delphi panelists were recruited through a call for participants to members of listservs: Society for Healthcare Epidemiology of America Research Network, Society of Infectious Diseases Pharmacists, VA Society of Practitioners of Infectious Disease, and Northwestern Antimicrobial Stewardship Physicians’ Network. Interested respondents completed a screening survey regarding length of practice experience, postgraduate training, board certification, practice setting, and antimicrobial stewardship program participation. Respondents were invited to participate if they met the minimum prespecified criteria (postgraduate training or board certification, evidence of antimicrobial stewardship program participation, more than 2 years’ experience) and blocking requirements; the latter was used to ensure diversity across occupation (PharmD or MD), geographic region, and VA affiliation.
Each Delphi round was delivered through online survey software (Survey Monkey). Periodic e-mail reminders were sent to maximize the response rate. Results were aggregated and analyzed after each round. Prior to each new round, data summary reports were provided to participants that described their individual and aggregated group responses on survey items. Qualitative assessment of text-based responses for common themes was conducted independently by 2 investigators (K.M.-K., N.H.) between rounds and then compared and discussed. Common findings were evaluated and incorporated into questions in subsequent rounds to add clarity or to pursue an unexpected line of reasoning.
An analysis of survey responses was performed between rounds and upon completion of the Delphi. After each round, percentages of agreement, measures of central tendency, and dispersion were computed for all items. If 65% or greater of participants selected 6 or 7 on the 7-point Likert scale, then the item was deemed to have attained consensus. Likewise, if 65% or greater of participants scored an item as 1 or 2, the item was deemed to have attained negative consensus. Questions exhibiting minimal movement in measures of central tendency after 2 rounds were removed from the final round to allow for collection of opinion on additional content.12 Similarly, most items attaining stable consensus were also removed. Questions presented in all rounds were evaluated to determine whether wording changes were similar enough to compare across rounds; then, weighted analyses of Cohen’s κ were conducted to assess reliability between rounds as indicated. 13
Spectrum Score Development
Based on input from the first 2 Delphi rounds, a prototype procedure was developed to calculate spectrum scores (Table 1). To ensure that sufficient numbers of organisms were included in the spectrum score that would allow for differentiation between narrow and broad antimicrobial coverage of regimens, an organism was selected for inclusion if the majority of participants favored its use in the score (greater than 50% participant agreement, with Likert ratings of 5–7). A similar approach was taken with assessment for inclusion of antibiotics. The final spectrum score included 14 organism domains (19 species) and 10 antibiotic domains (27 antibiotics)
TABLE 1.
Summary of Prototype Spectrum Score Method Determinations
| Element | Approach | Comments |
|---|---|---|
| Selection of organisms and antibiotics for spectrum score inclusion | Criteria for inclusion of organisms and antibiotics in the spectrum score included those where a majority of participants indicated a positive response for inclusion in the score (more than 50% participant agreement, with Likert ratings of 5–7). Bacteroides spp., aminopenicillins, and first-generation cephalosporins were also included at the investigators discretion. | Final spectrum score included 14 organism domains (19 species) and 10 antibiotic domains (27 antibiotics). Organisms (% agreement for inclusion): Staphylococcus aureus (100), Pseudomonas aeruginosa (100), Klebsiella spp. (100). Escherichia coli (100), Enterobacter spp. (100), Streptococcus pneumonia (92), Serratia spp. (88), Proteus spp. (80), Acinetobacter spp. (80), Haemophilus influenzae (68), Enterococcus faecalis (68), Citrobacter spp. (68), Enterococcus faecium (68), Stenotrophomonas maltophilia (60), Legionella spp. (56), Providencia spp. (52), Morganella spp. (52), gram-negative anaerobes (Bactreroides spp.; 48). Antibiotics (% agreement for inclusion): vancomycin (100), piperacillin/tazobactam (100), cefepime/ceftazidime (100), levofloxacin (100), moxifloxacin (96), ceftriaxone/cefotaxime (100), ciprofloxacin (96), imipinem/meropenem (96), linezolid (96), ertapenem (96), aztreonam (92%), colsitin/polymyxin B (92), amikacin (84), tigecycline (84), ampicillin/sulbactam and amoxicillin/clavulanate (80), daptomycin (76), clindamycin (68), cefpodoxime/cefdinir (68), azithromycin/clarithromycin (60), metronidazole (60), trimethoprim/sulfamethoxazole (56), tetracyclines (56), ticarcillin/clavulanate (56), cefuroxime (52), nafcillin/oxacillin (52), ampicillin/amoxicillin (48), cefazolin/cephalexin (48). |
| Population of spectrum score with antibiogram data | To assign microbial spectrum to antibiotics, National VA susceptibility data (2008–2012) for organisms and antibiotics included in the spectrum score were retrieved from the VA Corporate Data Warehouse. Percent susceptibility was calculated for individual antibiotic-organism pairs for each cell in the spectrum score. | Data were available in qualitative values (sensitive/intermediate/resistant) and included antibiotics with testing performed but suppressed because of tiered reporting criteria. For antibiotic domains with more than 1 antibiotic (ie, ceftriaxone, cefotaxime), susceptibility results were reviewed individually and then combined if similar results were observed (± 10% agreement). Otherwise, individual antibiotic results were reported. A minimum of 50 results per antibiotic-organism pair was desired to attain less than 10% precision for percent susceptibility. Percent susceptibility was scored by quintile on a 4-point ordinal scale (0, lowest susceptibility; 4, highest susceptibility). |
| Assignment of spectrum score values without sufficient susceptibility data | Susceptibility data were reviewed in the context of CLSI recommendations and modified accordingly. Classification of NIA and assignment of values to organism-antibiotic pairs needing further confirmation of susceptibility results involved a combination of methods. | For example, the susceptibility of S. aureus to oxacillin was used to populate the susceptibility values for all β-lactams with staphylococcal activity, irrespective of reported results for this antibiotic. Organism-antibiotic pairs without susceptibility methods but with suppressed susceptibility results were handled on a case-by-case basis. CLSI documents provided estimates of microbial activity or NIA in some cases. Product labeling was used if it indicated that the antibiotic possessed NIA or, in some cases, if in vitro activity against more than 90% of isolates for a species was reported. Assignment of values for organism-antibiotic pairs for cases without sufficient VA susceptibility data was performed independently by 2 investigators (K.M.-K., B.H.), with adjudication of discrepancies by a third investigator (M.J.). |
NOTE. CLSI, Clinical Laboratory Standards Institute; NIA, no intrinsic activity; VA, Veterans Affairs.
National VA susceptibility data (2008–2012) for organisms and antibiotics included in the spectrum score were used to estimate microbial spectrum wherever possible. Percent susceptibility was calculated for individual antibiotic-organism pairs for each organism domain utilizing 1 isolate per patient per year. References used to assign spectrum score values in the absence of satisfactory VA susceptibility data included Clinical Laboratory Standards Institute testing and reporting rules (M100-S22; M076-A9), current antibiotic product labeling, primary literature, and (in the absence of suitable references; ~3% of organism-antibiotic pairs) investigator opinion.
Spectrum score values were converted into quintiles ranging from 0 points for susceptibilities less than 20% to 4 points for susceptibility of 80% or greater. To adjust for antibiotic coverage against intrinsically resistant organisms, a weight of 1.25 was applied to spectrum score values for Staphylococcus aureus, Escherichia coli, Klebsiella spp., Acinetobacter spp., and Enterococcus faecium, and a weight of 1.75 was applied to spectrum score values for Pseudomonas aeruginosa. The domain for each organism’s weighted score was then summed to create a spectrum score for each antibiotic.
To penalize duplicative coverage in combination therapy, we assumed that the probability of being susceptible to 1 antibiotic was independent of every other. Therefore, the probability of a species being susceptible to 1 or more antibiotics in the regimen equaled 1 minus the probability of being resistant to all antibiotics in the regimen. The unconditional proportion susceptible for each antibiotic in the regimen was calculated, and the product of these proportions was subtracted from 1 to obtain the probability that an organism was susceptible to the regimen. Spectrum scores for organism-antibiotic regimen pairs were then computed in an identical manner as individual antibiotics described above.
The prototype method was compared with Delphi participants’ judgments on de-escalation with a series of 20 antibiotic regimen vignettes created for the final Delphi round. Each vignette described daily antibiotic regimens both early and later during treatment. Antibiotic regimens with implicitly similar spectra were created. Participants were asked to rank each case on a 7-point Likert scale: de-escalation (greater than 4), no meaningful change in therapy (4), or escalation (less than 4). Spectrum scores were calculated for early and late therapy regimens, and the late therapy score was subtracted from the early therapy score, resulting in calculation of a change in spectrum score between regimens. Correlation between change in spectrum score and mean Likert scores was estimated with Pearson’s correlation coefficient. To summarize test characteristics, the sensitivity and specificity of the sign of change in spectrum score to predict expert de-escalation status (ie, gold standard) were calculated.
To further evaluate reliability of the spectrum score method to measure de-escalation, 300 vignettes were created on the basis of antibiotic regimen data obtained for hospitalization days 2 and 4 from a random sample of 14,000 veterans meeting criteria for HCAP, a disease for which both broad-spectrum empirical antibiotic therapy and de-escalation are recommended.14 Three antimicrobial stewards who were not Delphi participants and who were unfamiliar with the spectrum score ranked vignettes using the same 7-point Likert scale described above. Spectrum scores were determined for vignette antibiotic regimens, and the method was compared with expert opinion, as previously described.
RESULTS
One hundred and twenty-three individuals completed the screening survey, and 55 of those were sent invitations to participate in the Delphi process. Forty-one invited participants completed the initial Delphi round, 33 continued to complete round 2, and 24 participants completed all 3 rounds of the study (Table 2)
TABLE 2.
Characteristics of Modified Delphi Method Participants
| Participant description | Screening (n = 123) | Round 1 (n = 41) | Round 2 (n = 33) | Round 3 (n = 24) |
|---|---|---|---|---|
| Type of professional degree | ||||
| Physician | 40.7 | 48.8 | 48.5 | 40.0 |
| Pharmacist | 59.3 | 51.2 | 51.5 | 60.0 |
| Additional graduate degree | 13.8 | 17.1 | 18.0 | 24.0 |
| Current ABM board certifications (MD) | ||||
| ID | 98.0 | 100.0 | 100.0 | 100.0 |
| Training in ID (PharmD) | ||||
| BCPS-AQID | 20.3 | 14.3 | 17.6 | 13.3 |
| PGY2 | 42.0 | 57.1 | 58.9 | 60.0 |
| Accredited fellowship | 7.2 | 9.5 | 0.0 | 0.0 |
| PGY1 | 72.5 | 66.7 | 70.6 | 53.3 |
| Other postgraduate ID traininga | 33.3 | 19.0 | 17.6 | 13.3 |
| Hospital practice setting | ||||
| Community | 32.5 | 17.1 | 24.2 | 24.0 |
| Public or government | 7.3 | 4.9 | 3.0 | 0.0 |
| University | 40.7 | 39.0 | 39.4 | 52.0 |
| Veterans Affairs | 17.9 | 29.3 | 30.3 | 32.0 |
| Rehab or chronic care | 3.3 | 2.4 | 3.0 | 0.0 |
| Teaching | 42.3 | 41.0 | 48.5 | 44.0 |
| Bed size of practice setting | ||||
| <100 | 3.3 | 7.3 | 9.1 | 12.0 |
| >100–250 | 17.9 | 19.0 | 18.2 | 12.0 |
| >250–500 | 32.5 | 17.1 | 15.2 | 16.0 |
| >500–1,000 | 40.7 | 48.8 | 51.5 | 52.0 |
| >1,000 | 5.7 | 4.9 | 6.1 | 8.0 |
| Years in practice | ||||
| <2 | 4.9 | 0.0 | 0.0 | 0.0 |
| 2–5 | 26.0 | 26.8 | 27.3 | 28.0 |
| 5–10 | 17.9 | 17.1 | 18.2 | 24.0 |
| 10–15 | 16.3 | 14.6 | 15.2 | 12.0 |
| 15–20 | 8.1 | 9.5 | 9.1 | 8.0 |
| >20 | 26.8 | 29.0 | 30.3 | 28.0 |
| US region | ||||
| Midwest | 27.8 | 26.8 | 23.5 | 28.0 |
| Northeast | 38.3 | 37.5 | 36.4 | 24.0 |
| Southeast | 19.1 | 14.6 | 18.2 | 24.0 |
| West | 14.0 | 19.5 | 21.2 | 24.0 |
| Sex | ||||
| Female | 48.0 | 41.5 | 32.3 | 12.0 |
| Male | 52.0 | 58.5 | 66.7 | 88.0 |
| Antibiotic stewardship and de-escalation | ||||
| De-escalation decision maker when initiating antibiotic therapy | 55.3 | 61.0 | 60.6 | 48.0 |
| Consultation, mentoring, job responsibility related to others performing de-escalation | 91.9 | 90.0 | 97.0 | 96.0 |
| Member of antibiotic stewardship team | 88.6 | 95.0 | 100 | 96.0 |
| Developing policies related to antibiotic stewardship | 90.2 | 92.7 | 97.0 | 96.0 |
| Antibiotic stewardship research | 65.0 | 82.9 | 90.9 | 88.0 |
NOTE. Data are % of participants. BCPS-AQID, board-certified pharmacotherapy specialist with added qualifications in infectious diseases; ID, infectious diseases; MD, medical doctor; PGY1, completed general residency; PGY2, completed specialty residency in infectious diseases or related discipline.
Completed Society of Infectious Diseases Pharmacists or Making a Difference in Infectious Diseases Pharmacotherapy or other ID certificate programs.
Expert support for concepts utilized in development of the spectrum score method was identified (Table 3). In domain 1, consensus was established for inclusion of 6 organisms and 14 antibiotics in the spectrum score. Consensus also indicated agreement for assigning higher weight to coverage of organisms that were intrinsically multidrug resistant, particularly P. aeruginosa. In domain 2, consensus was identified for use of semiquantitative scores to calculate microbial activity over an all-or-nothing determination of susceptibility. Participants also indicated that accounting for duplicate coverage of organisms was important when calculating an antibiotic regimen’s spectrum score and that this should be informed by antibiogram data. Domain 3 questions sought to determine which clinical criteria were important when evaluating whether de-escalation is appropriate for consideration in the context of future model development for adjustment of de-escalation rates across facilities. Experts identified microbiology and susceptibility results, diagnostic certainty, infectious diagnosis, initial antibiotic selection, and severity of illness as important elements. The remaining questions sought to determine the optimal time to measure spectrum scores during hospitalization to calculate de-escalation rates. Participants indicated that the optimal time to measure baseline therapy was 24 hours after antibiotic initiation. Sixty-three percent of participants indicated that the optimal time after initial antibiotic administration to de-escalate therapy was 72 hours. A rephrased question from the perspective of assessing facility-level de-escalation rates asked for the optimal time to measure de-escalation rates in their facility. Fifty-four percent and 42% of participants selected 96 and 72 hours, respectively.
TABLE 3.
Spectrum Score Concepts of Interest with Consensus Support
| Questiona | Delphi round | Likert score, %
|
Median | Mean | SD | CV | Consensus | |
|---|---|---|---|---|---|---|---|---|
| 1–2 | 6–7 | |||||||
| Domain 1: organisms and antibiotic components of the spectrum score | ||||||||
| Species-level measurement (eg, Escherichia coli) as opposed to broader classifications of bacteria (eg, oxidase-positive gram-negative bacilli) are important to measure an antibiotic’s spectrum of activity. | 1 | 2 | 80 | 6 | 6 | 1.1 | 0.2 | |
| Pathogens with high potential for the development of resistance to many antibiotics (eg, Pseudomonas aeruginosa) should be given higher weight when measuring an antibiotic’s spectrum of activity. | 1, 2 | 0 | 88 | 6 | 6.3 | 0.9 | 0.1 | |
| Please consider the degree of weight that should be given to microorganisms with high potential for the development of resistance relative to all other organism domains in the spectrum score. Marking 1.00 indicates that the standard minimum weight should be applied, whereas larger values indicate a higher weight in calculating the spectrum score. | 3 | Consensus weight: 1.25–1.50 for Staphylococcus aureus (80%), Acinetobacter spp. (80%), E. coli (76%), Enterococcus faecium (72%), and Klebsiella spp. (68%); 1.75–2.0 for P. aeruginosa (80%) | ||||||
| Please rate the following microorganisms in terms of importance for inclusion in a spectrum score to measure the spectrum antibacterial activity. | 2, 3 | Consensus for inclusion: P. aeruginosa (100%), S. aureus (100%), Klebsiella spp. (96%), E. coli (96%), Streptococcus pneumoniae (79%), Enterobacter spp. (71%) | ||||||
| Please rate the following antibiotics in terms of importance for inclusion in a spectrum score to measure the spectrum antibacterial activity. | 2, 3 | Consensus for inclusion: piperacillin/tazobactam (100%), cefepime/ceftazidime (100%), imipenem/meropenem/doripenem (100%), vancomycin (92%), levofloxacin (88%), ciprofloxacin (83%), linezolid (83%), tigecycline (79%), moxifloxacin (75%), ertapenem (75%), aztreonam (71%), daptomycin (71%), ceftriaxone/cefotaxime (67%), amikacin (67%) | ||||||
| Domain 2: operational aspects of the spectrum score | ||||||||
| A nominal measure that dichotomizes based on a threshold of whether a bacterial species is usually sensitive or not sensitive to an antibiotic is better than an ordinal measure that weights the spectrum metric on the degree of sensitivity. | 1, 2 | 28 | 28 | 3 | 3.8 | 1.9 | 0.5 | |
| Accounting for duplicate coverage of organisms within a spectrum domain will be important when calculating a patient’s daily composite spectrum score. | 1, 2 | 3 | 66 | 6 | 5.7 | 1.3 | 0.2 | |
| Composite antibiogram data should be used to define antibiotic spectrum of activity. | 1, 2 | 3 | 72 | 6 | 5.9 | 1.1 | 0.2 | |
| A combination of approaches including use of available antibiogram data, creation of expert rules based on published literature, and consensus of opinion should be used to develop the spectrum score. | 2 | 0 | 100 | 7 | 6.7 | 0.5 | 0.1 | |
| Domain 3: application of the spectrum score | ||||||||
| Please score the following items in terms of clinical criteria needed to decide when antibiotic de-escalation is appropriate. | 1, 2, 3 | Consensus for microbiological results (94%),b susceptibility results (97%),b diagnostic certainty (84%),b infectious diagnosis (88%),b initial choice of antibiotics (92%),c and initial severity of illness (75%)c | ||||||
| Please select the optimal time after initial antibiotic administration that captures a patient’s initial empirical therapy but precludes the time when antibiotic de-escalation decisions are made.d | 1, 2, 3 | 24 hours (67%), 48 hours (17%), 72 hours (8%), and less than 24 hours (8%) | ||||||
NOTE. CV, coefficient of variation; SD, standard deviation.
In cases where the phrasing of a question had small changes across rounds, the final phrasing was tabled.
There was participant consensus on this item in round 2, and the item was not included in round 3.
Cohen’s weighted κ for comparing rounds 2 and 3 was larger than κ for comparing rounds 1 and 2. For initial choice of antibiotics, κ increased from 0.31 (P = .13) to 0.37 (P = .03). For initial severity of illness, κ increased from 0.27 (P = .18) to 0.63 (P < .001).
Cohen’s weighted κ was not significant for comparing rounds 1 and 2 (κ = 0.35; P = .07) or for comparing rounds 2 and 3 (κ = 0.21; P = .28), and this was consistent with the shift from no consensus in round 1 to near consensus in round 2 to consensus in round 3.
An example of spectrum score calculations for a common antibiotic regimen is provided (Figure 2). In this example, vancomycin has a spectrum score of 13.0, imipenem has a score of 41.5, and the antibiotic combination has a score of 45.25. Individual antibiotic spectrum score values ranged from 4.0 for metronidazole (indicating narrow-spectrum coverage) to 49.75 for tigecycline (indicating broad-spectrum antimicrobial coverage) on a possible 60-point scale (Table 4).
FIGURE 2.
Example of spectrum score calculation for individual and combination antibiotic regimens. A, Values populated with national Veterans Affairs susceptibility data wherever possible (50 isolates or more) supplemented with Clinical Laboratory Standards Institute testing and reporting criteria, current product labeling, and primary literature. B, Susceptibility estimates for combination obtained by calculating the unconditional proportion susceptible for each antibiotic in the regimen, and the product of these proportions was subtracted from 1 to obtain the probability that an organism was susceptible to the regimen. C, Other Enterobacteriaceae included Citrobacter spp., Enterobacter spp., Morganella spp., Proteus spp., Providencia spp., and Serratia spp. D, Ordinal values: 0, no intrinsic bacterial activity or susceptibility 20% or less; 1, greater than 20% but less than 40%; 2, 40% or greater but less than 60%; 3, 60% or greater but less than 80%; 4, 80% or greater. E, A weight of 1.25 was applied to ordinal domain values for Staphylococcus aureus, Escherichia coli, Klebsiella spp., Acinetobacter spp., and Enterococcus faecium, and a weight of 1.75 was applied to spectrum score values for Pseudomonas aeruginosa.
TABLE 4.
Prototype Spectrum Score Values for Individual Antibiotic Regimens
| Antibiotic group | Spectrum score |
|---|---|
| Aminoglycosides | |
| Amikacin | 35.50 |
| Gentamicin, tobramycin | 35.50 |
| β-lactamase inhibitors | |
| Ampicillin/sulbactam, amoxicillin/clavulanate | 29.50 |
| Piperacillin/tazobactam | 42.25 |
| Ticarcillin/clavulanate | 40.50 |
| Carbapenems | |
| Ertapenem | 30.25 |
| Imipenem, meropenem | 41.50 |
| Cephalosporins | |
| Cefazolin, cephalexin | 19.25 |
| Cefuroxime | 23.50 |
| Ceftriaxone, cefotaxime | 25.25 |
| Ceftazidime/cefepime | 33.25 |
| Ceftaroline | 26.00 |
| Fluoroquinolones | |
| Ciprofloxacin, levofloxacin | 39.75 |
| Moxifloxacin | 36.25 |
| Glycopeptides/lipopeptides | |
| Vancomycin | 13.00 |
| Daptomycin | 14.25 |
| Macrolides/lincosamides | |
| Azithromycin, clarithromycin | 12.25 |
| Clindamycin | 10.75 |
| Penicillins | |
| Ampicillin, amoxicillin | 13.50 |
| Nafcillin, oxacillin | 4.25 |
| Tetracyclines | |
| Tetracycline, doxycycline | 38.75 |
| Tigecycline | 49.75 |
| Miscellaneous | |
| Aztreonam | 21.50 |
| Colistin, polymyxin B | 34.00 |
| Linezolid | 18.00 |
| Metronidazole | 4.00 |
| Trimethoprim/sulfamethoxazole | 33.50 |
NOTE. Maximal theoretical score for any antibiotic regimen in 60.
The relationship between Delphi participant judgments and spectrum score in a set of 20 antibiotic regimen de-escalation scenarios is summarized in Table 5. Consensus of expert opinion regarding whether a specific regimen indicated de-escalation was identified in only 2 regimens; however, mean Likert scores favored de-escalation in 13 and escalation in 7 vignettes, respectively. The sign of change in spectrum score correctly classified mean Likert scores, indicating de-escalation in 9 of 13 vignettes. Three discordant vignettes classified by participants as de-escalation (vignettes 9, 16, 18) were classified as escalation by spectrum score. All discordant vignettes involved regimens where oral antibiotics could be substituted for intravenous formulations later in therapy. In this sample of 20 vignettes, change in spectrum score was not significantly correlated with mean participant Likert score (−0.34; P = .15).
TABLE 5.
Assessment of Antibiotic De-Escalation in 20 Antibiotic Regimen Vignettes: Comparison of Delphi Panelists and Prototype Spectrum Score Method
| Vignette ID | Antibiotic regimen
|
Likert score, %
|
Median | Mean | SD | CV | Spectrum score
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Initial | Subsequent | 1–2 | 6–7 | Initial | Subsequent | Δ | |||||
| 1 | Vancomycin and piperacillin/tazobactam | Ertapenem | 4 | 76 | 6.0 | 6.0 | 1.2 | 0.2 | 44.50 | 30.25 | −14.25 |
| 2 | Vancomycin and piperacillin/tazobactam and levofloxacin | Vancomycin and imipenem | 40 | 8 | 4.0 | 3.3 | 1.5 | 0.5 | 55.25 | 45.25 | −10.00 |
| 3 | Moxifloxacin | Ceftriaxone | 0 | 24 | 5.0 | 4.8 | 0.9 | 0.2 | 36.25 | 25.50 | −10.75 |
| 4 | Ceftriaxone and azithromycin | Levofloxacin | 0 | 0 | 4.0 | 3.9 | 0.6 | 0.2 | 30.75 | 39.75 | 9.00 |
| 5 | Cefepime and linezolid | Ceftaroline | 4 | 28 | 5.0 | 5.1 | 1.0 | 0.2 | 44.75 | 26.00 | −18.75 |
| 6 | Vancomycin and piperacillin/tazobactam | Vancomycin and piperacillin/tazobactam and levofloxacin | 56 | 4 | 2.0 | 2.3 | 1.1 | 0.5 | 44.50 | 48.50 | 4.00 |
| 7 | Ciprofloxacin and ampicillin/sulbactam | Ciprofloxacin and amoxicillin/clavulanate | 0 | 4 | 4.0 | 4.3 | 0.6 | 0.1 | 48.50 | 48.50 | 0.00 |
| 8 | Piperacillin/tazobactam | Ampicillin/sulbactam | 0 | 80 | 6.0 | 6.1 | 0.7 | 0.1 | 42.25 | 33.50 | −8.75 |
| 9 | Vancomycin | Trimethoprim/sulfamethoxazole | 4 | 56 | 6.0 | 5.5 | 1.3 | 0.2 | 13.00 | 40.75 | 29.75 |
| 10 | Vancomycin and piperacillin/tazobactam | Moxifloxacin and clindamycin | 0 | 36 | 5.0 | 5.3 | 0.9 | 0.2 | 44.50 | 40.75 | −3.75 |
| 11 | Ceftazidime and gentamicin | Gentamicin and imipenem | 28 | 0 | 3.0 | 3.0 | 1.0 | 0.3 | 41.75 | 50.00 | 8.25 |
| 12 | Imipenem | Moxifloxacin | 0 | 64 | 6.0 | 5.8 | 0.7 | 0.1 | 41.50 | 36.25 | −5.25 |
| 13 | Ceftriaxone | Piperacillin/tazobactam | 80 | 0 | 2.0 | 1.9 | 0.7 | 0.4 | 25.50 | 42.25 | 16.75 |
| 14 | Tigecycline | Ertapenem | 4 | 8 | 4.0 | 4.4 | 1.0 | 0.2 | 49.75 | 30.25 | −19.50 |
| 15 | Clindamycin | Vancomycin | 24 | 4 | 3.0 | 3.1 | 1.3 | 0.4 | 10.75 | 13.00 | 2.25 |
| 16 | Vancomycin and piperacillin/tazobactam | Levofloxacin and piperacillin/tazobactam | 8 | 4 | 5.0 | 4.4 | 1.1 | 0.3 | 44.50 | 48.50 | 4.00 |
| 17 | Levofloxacin | Moxifloxacin | 0 | 0 | 4.0 | 4.1 | 0.7 | 0.2 | 39.75 | 36.25 | −3.50 |
| 18 | Ceftriaxone and azithromycin | Cefpodoxime and doxycyline | 0 | 12 | 4.0 | 4.4 | 0.8 | 0.2 | 30.75 | 43.25 | 12.50 |
| 19 | Vancomycin and piperacillin/tazobactam | Piperacillin/tazobactam and metronidazole | 0 | 20 | 5.0 | 5.0 | 0.7 | 0.1 | 44.50 | 42.25 | −2.25 |
| 20 | Ciprofloxacin | Levofloxacin | 12 | 0 | 4.0 | 3.5 | 0.8 | 0.2 | 39.75 | 39.75 | 0.00 |
NOTE. A negative change in spectrum score implies de-escalation. CV, coefficient of variation; SD, standard deviation.
Non-Delphi participant antimicrobial stewards identified de-escalation and no meaningful change in therapy or escalation in 24.3%, 63.3%, and 12.3% of the 300 HCAP-based vignettes reviewed, respectively (average intraclass correlation coefficient, 0.929 [0.870–0.964]), whereas the spectrum score method identified de-escalation and no meaningful change in therapy or escalation in 24.0%, 62.0%, and 14.3% of these cases by day 4 of therapy, respectively. The sensitivity and specificity of the spectrum score method to identify de-escalation events as judged by antimicrobial stewards was 86.3% and 96.0%, respectively. Likert scores suggested that for some vignettes, experts made inferences regarding switches from intravenous antibiotic use on day 2 to oral antibiotic use on day 4. Further analysis indicated that mean Likert scores were 0.5 points higher for vignettes that could have included at least 1 antibiotic administered orally on day 4 but not on day 2 of therapy (P = .003), suggesting that experts viewed oral therapy favorably in classifying de-escalation decisions. Change in spectrum score was correlated with mean Likert score (0.66; P < .001).
DISCUSSION
The modified Delphi method provided critical insight into antibiotic stewards’ perceptions on antibiotic spectrum and de-escalation. The prototype spectrum score method that was developed reflects input from the Delphi participants. Important consensus items identified included the following: antibiotics used to treat intrinsically resistant organisms, particularly P. aeruginosa, should be weighted more heavily than other antibiotics; an ordinal scale of antibiotic susceptibility was preferable to categorically assigning susceptibility to an antibiotic; and accounting for duplicate coverage of organisms was important when measuring de-escalation. Regarding the optimal time to measure antibiotic de-escalation, 24 hours after initiation was considered the most appropriate time to measure baseline therapy, and while consensus was not achieved regarding the optimal time to measure antibiotic de-escalation, 96% of the participants indicated that day 3 or 4 was the optimal time to measure de-escalation rates in their facility. The spectrum score method that was developed and implemented generally agreed with panelist interpretations of de-escalation as well as antimicrobial stewards unfamiliar with the spectrum score.
Study strengths include the development of a novel approach to measure de-escalation which is based in part on opinions of antimicrobial stewards in formulation of the method. The construct behind de-escalation is that less selective pressure on non-disease-causing bacteria through the use of targeted narrow spectrum antibiotics is important. The spectrum score method, which uses susceptibility data in score formulation, may help facilitate objective measurement of antibiotic spectrum in de-escalation considerations. The method may also help to explore the association between antibiotic de-escalation and patient outcomes or antimicrobial resistance. Use of an algorithm to calculate spectrum scores is a new approach that takes advantage of the growing availability of electronic antibiotic use data. Current measurement of antibiotic de-escalation practice requires labor-intensive manual chart review, which is impractical for facility-level measurement or comparison.
Study limitations include the lack of consensus for select content areas, reliance on VA susceptibility and antibiotic use data, and the web-based modified Delphi design. Agreement between the algorithm and expert-recognized de-escalation was imperfect but on-par or superior to the level of agreement measured between experts. Panelists reached consensus for only a limited number of intrinsically resistant organisms and predominantly broad-spectrum antibiotics for inclusion in the score, and we may have introduced bias by informing panelists of our long-term goal to measure facility-level de-escalation rates in patients with HCAP. In the absence of consensus, we primarily included organisms and antibiotics that received majority affirmative rankings. The spectrum score was developed foremost from VA data, and the method was designed to measure de-escalation in VA electronic medical records, which may limit generalizability to community or university hospitals. However, the large nationwide sampling of recent data is also a potential strength. The rapid escalation in use of electronic medical records technology may allow adaptation for use in other healthcare systems in the future. A general limitation of the spectrum score approach is that in vivo susceptibility of many organisms exposed to antibiotics in the treatment of human infection as well as their clinical significance relative to antibiotic de-escalation are unknown. A final limitation involved the use of the web-based modified Delphi process. We observed a decrease in survey response rates over rounds, which is an inherent difficulty with written Delphi surveys; also, unlike a traditional Delphi method, where consensus may be sought through face-to-face deliberations, this study was unable to fully explore concepts of interest that arose in the process, such as the importance of intravenous to oral conversion.15
A variety of definitions of broad and narrow spectrum have been used in studies that measure de-escalation, yet objective characterization of de-escalation in terms of antimicrobial spectrum is limited.2–11 To date, 1 study has attempted to utilize a measure of antibiotic de-escalation on the basis of the intrinsic microbiological activity of antibiotics.6 A prospective observational cohort study of ventilator-associated pneumonia employed a scoring system that ranked antibiotic regimens on a scale of 1–5 on the basis of the intrinsic gram-negative activity of antipseudomonal β-lactam and fluoroquinolone antibiotics. Combination therapy regimens were scored on the basis of the most potent antimicrobial, and lesser antibiotics were ignored. De-escalation was classified as the switch to or addition of antibiotics with lower scores. The authors noted that de-escalation was uncommonly performed.
Future work will include further calibration of the spectrum score method to account for the importance of oral therapy in the assessment of antibiotic de-escalation and a full-scale VA facility-level assessment of de-escalation rates in HCAP. Upon completion, export and construct validation of the spectrum score method to measure de-escalation rates in other patient populations and healthcare settings may be warranted. Additional work should study the clinical significance of de-escalation on antimicrobial resistance and clinical outcomes as well as explanations for why de-escalation rates are low.
In summary, the modified Delphi method provided critical insight into antibiotic stewards’ perceptions on components of spectrum score development and operational aspects for applying the score to measure antibiotic de-escalation. While a clear consensus for all items was not identified, it is important to recognize that limited published data exist in the area of de-escalation, which is 1 of the main reasons this study used a Delphi process. On the basis of the Delphi results, we developed a method for measuring de-escalation in electronic medical data, which is based on the spectrum of microbial activity for antibiotic regimens that generally agrees with expert opinions of antibiotic de-escalation events.
Acknowledgments
We thank the following organizations for dissemination of the call for participants through their listservs: Society for Healthcare Epidemiology of America Research Network, Society of Infectious Diseases Pharmacists, Veterans Affairs Society of Practitioners of Infectious Disease, and the Northwestern Antimicrobial Stewardship Physicians’ Network.
We are also grateful for the time commitment and dedicated effort of Delphi participants. Initially, participant responses were tracked between rounds by their e-mail addresses, and we did not ask participants for their full contact information or if they wished to be identified as a participant until after completion of the Delphi procedure. The following individuals completed all Delphi rounds, responded to our exit survey request, and indicated their preference to be acknowledged as a participant: Deverick J. Anderson, MD, MPH, Duke University Medical Center, Durham, NC; Russell J. Benefield, PharmD, BCPS Clinical Pharmacist, Infectious Diseases, University of Utah Health Sciences Center, University of Utah College of Pharmacy, Salt Lake City, UT; Steven C. Ebert, PharmD, Clinical Professor of Pharmacy, University of Wisconsin–Madison, and Clinical Manager, Department of Pharmacy, Meriter Hospital, Madison, WI; Barry C. Fox, MD, Clinical Professor of Medicine, University of Wisconsin, and Director of Antimicrobial Stewardship Program, University of Wisconsin Hospitals, Madison, WI; Alan E. Gross, Clinical Assistant Professor, College of Pharmacy, University of Nebraska, and Pharmacist Coordinator, Antimicrobial Stewardship, Nebraska Medical Center, Omaha, NE; Timothy Gauthier, PharmD, BCPS (AQ ID), Assistant Professor of Pharmacy Practice, Nova Southeastern University College of Pharmacy, Jackson Memorial Hospital, Miami, FL; Christopher J. Graber, MD, MPH, Assistant Clinical Professor of Medicine, David Geffen School of Medicine at UCLA, VA Greater Los Angeles Healthcare System, Los Angeles, CA; Maximillian Jahng, Infectious Diseases Clinical Pharmacy Specialist, Antimicrobial Stewardship Program Manager, New Mexico VA Health Care System, Albuquerque, NM; Ken Klinker, PharmD, Clinical Specialist, Infectious Diseases, University of Florida Healthcare and Shands Hospital, Gainesville, FL; Darren R. Linkin, MD, MSCE, Assistant Professor of Medicine, Division of Infectious Diseases, Department of Medicine, and Associate Scholar, Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA; Conan MacDougall, PharmD, MAS, BCPS, Associate Professor of Clinical Pharmacy, University of California San Francisco School of Pharmacy, San Francisco, CA; Craig Martin, Director of Professional Practice Development, University of Kentucky College of Pharmacy, Lexington, KY; Stephanie Nagy-Agren, MD, Chief, Infectious Diseases, Salem VAMC, Associate Professor of Internal Medicine, Virginia Tech Carillion School of Medicine, Salem, VA; Michael Postelnick, RPh, BCPS (AQ ID), Senior Infectious Diseases Pharmacist, Northwestern Memorial Hospital, Chicago, IL; Patricia Saunders-Hao, PharmD, BCPS, Clinical Pharmacist, Infectious Disease, Mount Sinai Medical Center, New York, NY; Ed Septimus, MD, Professor Internal Medicine, Texas A&M Health Science Center, Houston, TX; Dora E. Wiskirchen, PharmD, BCPS, Assistant Professor of Pharmacy Practice, University of Saint Joseph School of Pharmacy, Clinical Infectious Diseases Pharmacist, Saint Francis Hospital and Medical Center, Hartford, CT
We would also like to thank all other Delphi participants, including 7 additional antimicrobial stewards (2 physicians and 5 pharmacists), who completed all Delphi Rounds but did not respond to our exit survey or who requested to remain anonymous on the exit survey.
We are also grateful to the non-Delphi participant antimicrobial stewards who reviewed healthcare-associated pneumonia vignettes: Allison Kelly, MD, National Infectious Diseases Service, VA Central Office, Cincinnati, OH, and Assistant Professor of Medicine, Department of Internal Medicine, Division of Infectious Diseases, University of Cincinnati, College of Medicine, and Cincinnati VA Medical Center, Cincinnati, OH; Kelly Echevarria, PharmD, Clinical Pharmacy Specialist–Infectious Diseases, South Texas Veterans Health Care System, San Antonio, TX; Birgir Johannsson, MD, Division of Infectious Diseases, University of Iowa Hospitals and Clinics, Iowa City, IA.
Financial support. K.M.-K. and N.H. were supported by the National Institute of Allergy and Infectious Diseases (grant 1 R15 AI098049-01) provided through Idaho State University. R.E.R. received payment for statistical services from the National Institute of Allergy and Infectious Diseases subcontracted through Idaho State University (grant 1 R15 AI098049-01). M.J. and M.S. are employed full-time by the Department of Veterans Affairs, and the National Institute of Allergy and Infectious Diseases provided payment for data management and analysis on this study to them, subcontracted through Idaho State University (grant 1 R15 AI098049-01). At the time of this study, B.H. was supported by a fellowship grant from Geneva University Hospitals. This work was also supported in part by resources and use of the Boise and George E. Wahlen Veterans Affairs Medical Centers. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.
Footnotes
Potential conflicts of interest. All authors report no conflicts of interest relevant to this article. All authors submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest, and the conflicts that the editors consider relevant to this article are disclosed here.
References
- 1.Dellit TH, Owens RC, McGowan JE, Jr, et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis. 2007;44(2):159–177. doi: 10.1086/510393. [DOI] [PubMed] [Google Scholar]
- 2.Masterton RG. Antibiotic de-escalation. Crit Care Clin. 2011;27(1):149–162. doi: 10.1016/j.ccc.2010.09.009. [DOI] [PubMed] [Google Scholar]
- 3.Silva BN, Andriolo RB, Atallah AN, Salomão R. De-escalation of antimicrobial treatment for adults with sepsis, severe sepsis or septic shock. Cochrane Database Syst Rev. 2013;3:CD007934. doi: 10.1002/14651858.CD007934.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Singh N, Rogers P, Atwood CW, Wagener MM, Yu VL. Short-course empiric antibiotic therapy for patients with pulmonary infiltrates in the intensive care unit: a proposed solution for indiscriminate antibiotic prescription. Am J Respir Crit Care Med. 2000;162(2 pt 1):505–511. doi: 10.1164/ajrccm.162.2.9909095. [DOI] [PubMed] [Google Scholar]
- 5.Schlueter M, James C, Dominguez A, Tsu L, Seymann G. Practice patterns for antibiotic de-escalation in culture-negative healthcare-associated pneumonia. Infection. 2010;38(5):357–362. doi: 10.1007/s15010-010-0042-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kollef MH, Morrow LE, Niederman MS, et al. Clinical characteristics and treatment patterns among patients with ventilator-associated pneumonia. Chest. 2006;129(5):1210–1218. doi: 10.1378/chest.129.5.1210. [DOI] [PubMed] [Google Scholar]
- 7.Alvarez-Lerma F, Alvarez B, Luque P, et al. Empiric broad-spectrum antibiotic therapy of nosocomial pneumonia in the intensive care unit: a prospective observational study. Crit Care. 2006;10(3):R78. doi: 10.1186/cc4919. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Labelle AJ, Arnold H, Reichley RM, Micek ST, Kollef MH. A comparison of culture-positive and culture-negative health-care-associated pneumonia. Chest. 2010;137(5):1130–1137. doi: 10.1378/chest.09-1652. [DOI] [PubMed] [Google Scholar]
- 9.Morel J, Casoetto J, Jospé R, et al. De-escalation as part of a global strategy of empiric antibiotherapy management: a retrospective study in a medico-surgical intensive care unit. Crit Care. 2010;14(6):R225. doi: 10.1186/cc9373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gonzalez L, Cravoisy A, Barraud D, Conrad M, Nace L, Lemarié J, Bollaert PE, Gibot S. Factors influencing the implementation of antibiotic de-escalation and impact of this strategy in critically ill patients. Crit Care. 2013;17(4):R140. doi: 10.1186/cc12819. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Schulz L, Osterby K, Fox B. The use of best practice alerts with the development of an antimicrobial stewardship navigator to promote antibiotic de-escalation in the electronic medical record. Infect Control Hosp Epidemiol. 2013;34(12):1259–1265. doi: 10.1086/673977. [DOI] [PubMed] [Google Scholar]
- 12.Holey EA, Feeley JL, Dixon J, Whittaker VJ. An exploration of the use of simple statistics to measure consensus and stability in Delphi studies. BMC Med Res Methodol. 2007;7:52. doi: 10.1186/1471-2288-7-52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70:213–220. doi: 10.1037/h0026256. [DOI] [PubMed] [Google Scholar]
- 14.American Thoracic Society; Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med. 2005;171(4):388–416. doi: 10.1164/rccm.200405-644ST. [DOI] [PubMed] [Google Scholar]
- 15.Hsu C-C, Sandford BA. The Delphi technique: making sense of consensus. [Accessed January 8, 2014];Practical Assess Res Eval. 2007 12(10) http://pareonline.net/pdf/v12n10.pdf. [Google Scholar]