Clinical Diagnoses and Antimicrobials Predictive of Pediatric Antimicrobial Stewardship Recommendations: A Program Evaluation

Abstract

BACKGROUND

The number of pediatric antimicrobial stewardship programs (ASPs) is increasing and program evaluation is a key component to improve efficiency and enhance stewardship strategies.

OBJECTIVE

To determine the antimicrobials and diagnoses most strongly associated with a recommendation provided by a well-established pediatric ASP.

DESIGN AND SETTING

Retrospective cohort study from March 3, 2008, to March 2, 2013, of all ASP reviews performed at a free-standing pediatric hospital.

METHODS

ASP recommendations were classified as follows: stop therapy, modify therapy, optimize therapy, or consult infectious diseases. A multinomial distribution model to determine the probability of each ASP recommendation category was performed on the basis of the specific antimicrobial agent or disease category. A logistic model was used to determine the odds of recommendation disagreement by the prescribing clinician.

RESULTS

The ASP made 2,317 recommendations: stop therapy (45%), modify therapy (26%), optimize therapy (19%), or consult infectious diseases (10%). Third-generation cephalosporins (0.20) were the antimicrobials with the highest predictive probability of an ASP recommendation whereas linezolid (0.05) had the lowest probability. Community-acquired pneumonia (0.26) was the diagnosis with the highest predictive probability of an ASP recommendation whereas fever/neutropenia (0.04) had the lowest probability. Disagreement with ASP recommendations by the prescribing clinician occurred 22% of the time, most commonly involving community-acquired pneumonia and ear/nose/throat infections.

CONCLUSIONS

Evaluation of our pediatric ASP identified specific clinical diagnoses and antimicrobials associated with an increased likelihood of an ASP recommendation. Focused interventions targeting these high-yield areas may result in increased program efficiency and efficacy.

Hospital-based antimicrobial stewardship programs (ASPs) have proven effective in improving the judicious use of antimicrobial therapy.^1–3 Stewardship strategies have demonstrated the ability to decrease inappropriate antibiotic use, enhance targeted therapy, reduce the duration of antibiotic therapy, and enhance patient safety, all resulting in cost savings.^2,4–9 Multiple professional organizations recommend ASPs in order to promote appropriate antimicrobial use and the implementation of stewardship programs in pediatric settings is growing.^10–12 However, few data exist on the evaluation of stewardship program resource utilization and the extent to which a pediatric ASP provides actual recommendations to prescribing clinicians. A better understanding of these factors could improve the efficiency and resource utilization of ASPs.

Previously, we described the impact of our prospective audit with recommendation and feedback (PAF) ASP. A decrease in hospital-wide antimicrobial utilization occurred following implementation of the program.⁵ However, the frequency of ASP reviews and associated recommendations by our program have not fully been evaluated.

In order to better understand the factors that are either barriers or facilitators to effective stewardship implementation, we sought to conduct an evaluation that identified the predictors of an ASP recommendation. The primary objective of this study was to determine the antimicrobials and diagnoses most strongly associated with a PAF recommendation. The secondary objective was to determine the likelihood of recommendation agreement by the prescribing clinician.

MATERIALS AND METHODS

Study Design and Setting

This was a retrospective cohort study of all hospitalized patients reviewed by the ASP at Children's Mercy Hospital from the start of the program, March 3, 2008, to March 2, 2013. Children's Mercy Hospital is a 354-bed, tertiary care, free-standing children's hospital in Kansas City, Missouri, serving a 5-state, 100-county region with approximately 15,000 admissions annually. The ASP has been previously described.^5,13 In brief, a patient review occurs when a monitored antimicrobial (Table 1) has been prescribed for 2 consecutive calendar days, triggering an ASP review. An ASP pharmacist and/or infectious diseases (ID) physician performs the medical chart review to determine the appropriateness of the prescribed antimicrobial(s) in regards to the indication, dose, and duration. Only a limited number of antimicrobials trigger a review (Table 1); however, all actively prescribed antimicrobials are assessed during the review process. On the basis of the review, the ASP may choose to provide a recommendation concerning antimicrobial prescribing to the primary medical team caring for the patient.

TABLE 1.

Antimicrobials Monitored by Our Antimicrobial Stewardship Program That Trigger a Review

Amikacina	Imipenem/cilastatinb
Amoxicillin/clavulanate	Levofloxacina
Ampicillin/sulbactam	Linezolida
Aztreonam	Meropenem
Cefepime	Moxifloxacinb
Ceftazidime	Piperacillin/tazobactam
Cefotaxime	Ticarcillin/clavulanateb
Ceftriaxone	Tigecyclineb
Ciprofloxacin	Tobramycin
Colistina	Vancomycin
Daptomycinb

^aPrior approval recommended.

^bNot on formulary.

A record for each reviewed antimicrobial was entered into an electronic database with the associated indication for the antimicrobial, planned duration of therapy, type of ASP recommendation if one occurred, verbal agreement or disagreement with the recommendation by the primary team, and adherence to the recommendation among those who agreed. The ASP pharmacist reviewed each medical chart following a recommendation to assess adherence.

Study Outcomes

The primary outcome was the probability and type of ASP recommendation for each antimicrobial and diagnosis. Antimicrobials used for prophylaxis (eg, trimethoprim-sulfamethoxazole for Pneumocystis jirovecii prevention) were excluded. ASP recommendations were categorized on the basis of recommendation status. If no recommendation was made, the category was “no recommendation.” Recommendations were further classified as (1) stop therapy, (2) modify therapy (ie, change antimicrobial), (3) optimize therapy (ie, alter dosing or route of administration), and (4) consult ID. If ID was already consulted at the time of an ASP review, this was categorized as “no recommendation.” In a secondary analysis, we determined the probability of disagreement with ASP recommendations on the basis of the same factors as for the primary analysis.

Indications for antimicrobial prescribing were determined by the ASP upon the review and categorized into diagnostic classes such as bloodstream infections, fever and neutropenia, and suspected sepsis. The diagnosis was assigned in the database by the ASP pharmacist. Respiratory infections and community-acquired pneumonia (CAP) were grouped as 2 mutually exclusive diagnosis classes. Respiratory infections included cystic fibrosis exacerbation, hospital-acquired pneumonia, and aspiration pneumonia (Table 2). ASP reviews with a documented antimicrobial for more than 1 diagnosis class, introduced complexity when examining the relationship between clinical presentation and ASP involvement. Consequently, reviews with 2 or more diagnosis classes were modeled as a separate group of 2 or more indications. If the indication was missing, the entire review was not included in the analysis.

TABLE 2.

Diagnostic Categories for Commonly Reviewed Antimicrobial Indications

ENT (%)	Respiratory (%)	Surgery(%)	Genitourinary (%)
Tracheitis (40)	CF exacerbation (62)	Intra-abdominal process (72)	UTI (94)
Otitis media (19)	Aspiration pneumonia (17)	Appendectomy (9)	Perirectal abscess (2)
Pharyngitis (12)	HAP/VAP (14)	Postoperative management (9)	Fistula (1)
Orbital cellulitis (9)	Sinusitis (5)	VPSI (5)	Renal abscess (1)
Adenitis (9)		Surgical site infection (2)
Mastoiditis (6)		Hardware infection (2)
Tonsillitis (3)

NOTE. CF, cystic fibrosis; ENT, ear, nose, and throat; HAP/VAP, hospital-acquired pneumonia/ ventilator-associated pneumonia; UTI, urinary tract infection; VPSI, ventriculoperitoneal shunt infection.

Data Analysis

The frequency and types of recommendations made by the ASP during the first 5 years following ASP implementation were calculated for each antimicrobial regimen and diagnosis class. A multinomial distribution model was used to determine the probability of each ASP recommendation group (mentioned previously), based on the specific treatment and diagnosis. Medical service, year of ASP program, presence of a chronic complex medical condition code,¹⁴ patient sex, and patient age were also included in the final model. A logistic model was used to determine the odds of disagreement with ASP recommendations, based on antimicrobial regimen, diagnosis class, and medical service. Both models accounted for the clustering effect of some patients having more than 1 ASP review. All analyses were completed using the statistical software of Stata, version 13 (StataCorp). The Children's Mercy Hospital Institutional Review Board approved this study.

RESULTS

ASP Reviews

A total of 15,016 ASP patient reviews were performed during the study period. We excluded 406 reviews (2.7%) with recommendation groups that were not concordant (eg, “stop” for antibiotic A, and “optimize” for antibiotic B) and 208 records (1.4%) with missing data on key explanatory factors. The remaining 14,402 reviews were included in our analyses. A relatively small percent (7%) of ASP patient reviews documented antimicrobial therapy for more than 1 diagnosis and therefore were classified into the group of 2 or more indications. The most common antimicrobial regimens reviewed by our ASP included ceftriaxone/cefotaxime, the combination of an ASP-monitored drug and penicillin, and the combination of 2 ASP-monitored drugs (Figure 1a). Some broad-spectrum agents such as ceftazidime, linezolid, fluoroquinolones, carbapenems, and vancomycin had low overall numbers of patient reviews.

FIGURE 1.

Number of total reviews performed by antimicrobial stewardship program (ASP) and the probability of an ASP recommendation based on antimicrobial (A) and diagnosis (B). Fluoro = fluoroquinolones; Clinda = clindamcyin; Metro = metronidazole; CNS = central nervous system; SSTI = skin and soft-tissue infections; ENT = ear, nose, and throat; CAP = community-acquired pneumonia; BSI = bloodstream infection.

Probability of an ASP recommendation

A total of 2,317 reviews (16%) resulted in an ASP recommendation (year 1, 20%; years 2–4, 15%; year 5, 14%). The antimicrobials with the highest probability of an ASP recommendation were ceftazidime (0.28), and the combination of an ASP drug with either gentamicin (0.25) or clindamycin (0.22). Linezolid (0.05), an ASP drug in combination with a penicillin (0.07), and cefepime (0.08) were the 3 drug classes with the lowest prevalence of a recommendation (Figure 1a). The percent of recommendations for antimicrobials requiring prior approval accounted for only 0.7% of all recommendations.

Several diagnoses were frequently identified during ASP reviews. Suspected sepsis, surgical conditions, and fever in the setting of neutropenia were the predominant diagnoses encountered (Figure 1b). However, these common diagnoses had a low probability of actually leading to an ASP recommendation compared with diagnoses such as ear/nose/throat (ENT) infections (0.29), gastrointestinal disorders (0.26), or CAP (0.27), all of which were associated with a high probability of recommendations. For example, the diagnosis of fever in the setting of neutropenia had approximately 1 ASP recommendation per 31 individual reviews compared with ENT infections, gastrointestinal disorders, or CAP, with approximately 1 ASP recommendation per 4 or fewer reviews.

Adjusted Predictive Probability of ASP Recommendation Category

After adjusting for medical service, review year, and patient-level clinical and demographic characteristics, the antimicrobial regimens with the highest predicted probability of a recommendation were ceftazidime (0.20), combination therapy of an ASP drug with clindamycin (0.20), and ceftriaxone/cefotaxime (0.21) (Figure 2a). Linezolid (0.06), an ASP drug plus metronidazole (0.09), cefepime (0.12), and fluoroquinolones (0.12) were the antimicrobials associated with the lowest adjusted predicted probability of an ASP recommendation. The diagnostic classes with the highest predicted probability of a recommendation were CAP (0.26), ENT (0.26), genitourinary (0.22), and respiratory (0.21) infections (Figure 2b). Tracheitis accounted for 40% of ENT diagnoses and cystic fibrosis exacerbations accounted for 62% of respiratory infections (Table 2). Diagnostic classes with the lowest predictive probabilities of an ASP recommendation included fever with neutropenia (0.04), central nervous system infections (0.09), and suspected sepsis (0.10). ENT infections had the highest probability of a recommendation to stop therapy (0.14 [95% CI, 0.12–0.17]; P < .001), followed by CAP (0.10 [0.08–0.12]; P < .001). ID consultation was most likely to be recommended with CAP (0.04 [95% CI, 0.03–0.05]; P < .001).

FIGURE 2.

Adjusted predictive probability of an antimicrobial stewardship program (ASP) recommendation by antimicrobial (A) and diagnosis (B). Fluoro = fluoroquinolones; Clinda = clindamcyin; Metro = metronidazole; CNS = central nervous system; SSTI = skin and soft-tissue infections; ENT = ear, nose, and throat; CAP = community-acquired pneumonia; BSI = bloodstream infection.

The frequency of recommendations was as follows: stop (45%), modify (26%), optimize (19%), and consult ID (10%). The combination of an ASP drug with clindamycin had the highest probability of a stop therapy recommendation (0.11 [95% CI, 0.09–0.13]; P < .001), followed by ASP drug with gentamicin (0.10 [0.07–0.12]; P < .001).

Disagreement With an ASP Recommendation

Of all the ASP recommendations, the medical team disagreed with 22% (519/2,317). Examination of the likelihood of disagreement by the primary medical team with the ASP revealed that recommendations focused on use of broad-spectrum antimicrobials including the carbapenems (odds ratio, 2.8) and linezolid (7.2) were the antimicrobials associated with the highest likelihood of disagreement (Table 3). Both CAP (odds ratio, 4.08) and ENT (4.19) were diagnoses also associated with a higher likelihood of disagreement. Suspected sepsis evaluation (odds ratio, 4.17) and patients in the neonatal intensive care unit clinical service unit (2.02) also had a high likelihood of disagreement with ASP recommendations.

TABLE 3.

Logistic Model Examining the Odds of Disagreement With ASP Recommendations

			Confidence limits
	Odds ratio	P value	Lower	Upper
Antibiotic class
ASP drug + clindamycin	1 [Reference]
Ceftazidime	0.777	.624	0.284	2.125
Cefepime	1.334	.384	0.696	2.556
Vancomycin	1.466	.281	0.731	2.941
Carbapenem	2.759	.002	1.434	5.309
Beta-lactam/inhibitor	2.026	.005	1.233	3.327
Fluoroquinolones	1.432	.399	0.620	3.307
Linezolid	7.231	.002	2.121	24.643
ASP drug + metronidazole	1.173	.666	0.567	2.423
ASP drug + penicillins	0.859	.638	0.457	1.614
ASP drug + gentamicin	1.737	.077	0.941	3.204
2 + ASP drugs	1.316	.263	0.813	2.128
Other	1.737	.063	0.969	3.113
Indication for treatment
Genito/urinary	1 [Reference]
CAP	4.084	<.001	2.393	6.970
ENT	4.192	<.001	2.411	7.291
Fever/neutropenia	2.791	.025	1.136	6.861
Gastrointestinal	2.414	.015	1.190	4.896
CNS	0.912	.886	0.259	3.202
Prophylaxis	2.535	.163	0.686	9.355
Respiratory (non-CAP)	4.243	<.001	2.376	7.578
Suspected sepsis	4.170	<.001	2.416	7.197
Skin/soft tissue	2.213	.073	0.929	5.267
Surgical	3.389	<.001	1.718	6.684
≥2 Indications	2.282	.007	1.249	4.167
Other indication	1.953	.095	0.889	4.290
Clinical service line
Hospitalists	1 [Reference]
PICU	1.432	.06	0.985	2.081
NICU	2.020	.001	1.317	3.098
Hematology/oncology	1.972	.002	1.288	3.021
Surgery	1.171	.457	0.772	1.776
Specialists	1.487	.053	0.994	2.224

NOTE. ASP, antimicrobial stewardship program; CAP, community-acquired pneumonia; CNS, central nervous system; ENT, ear, nose, and throat; NICU, neonatal intensive care unit; PICU, pediatric intensive care unit.

DISCUSSION

In this study, we evaluated the clinical factors associated with ASP recommendations and identified important factors associated with ASP recommendation disagreement. Our PAF ASP performs approximately 8 reviews daily. Sixteen percent of those reviews resulted in a recommendation regarding antimicrobial use prescribed in our pediatric hospital. Frequently reviewed antimicrobials were associated with a high probability of a recommendation (eg, ASP drug plus clindamycin, third-generation cephalosporins) and common diagnoses such as CAP and ENT infections had the highest predictive probability of an ASP recommendation. The types of recommendations (eg, stop therapy, consult ID) also varied by specific antimicrobials and/or diagnosis class. Finally we found that there was disagreement with a minority of our ASP recommendations. Disagreement by the primary medical service was more likely to occur when addressing broader-spectrum antimicrobial agents (eg, linezolid, meropenem) or certain commonly encountered diagnoses (eg, sepsis evaluation, CAP).

When assessing the probability of an ASP recommendation based on drug or diagnosis, several findings were striking. Overall, broader-spectrum antimicrobials had a relatively low probability of an ASP recommendation. This finding likely partially stems from having a prior authorization process for certain agents including linezolid and levofloxacin. Thus, prior authorization can minimize PAF ASP workload, supporting the value of combined PAF and prior authorization strategies as an effective approach for stewardship.^2,10 There is no prior approval required for cefepime, but it also had a low likelihood of triggering an ASP recommendation. Cefepime is the drug of choice for empirical therapy for fever in the setting of neutropenia hospital-wide per our local guideline. Successful implementation of hospital guidelines to standardize care in specific clinical settings can likely further decrease the frequency of ASP recommendations.

On the basis of national surveys of pediatric ASP programs, third-generation cephalosporins, aminoglycosides, ampicillin-sulbactam, and clindamycin are less likely to be monitored compared with the extended-spectrum antimicrobials such as vancomycin, linezolid, and carbapenems^11,12; however, these commonly prescribed antimicrobials were associated with frequent ASP recommendations at our hospital with 1 recommendation occurring per 5 reviews for ceftriaxone/cefotaxime. Although reviewing these commonly prescribed antimicrobials requires more stewardship resources, these reviews represented a high proportion of necessary recommendations. Interestingly, clindamycin and gentamicin, antimicrobials that are not actively monitored by our stewardship program, were both strongly associated with a recommendation to either stop therapy or obtain ID consultation, when used in combination with an ASP-monitored drug. This may reflect that combination therapy is indicated in a limited number of clinical situations.

CAP was the diagnosis with the highest probability of a stewardship recommendation, despite existence of both national and local hospital clinical practice guidelines.^15,16 Determining illness severity and differentiating viral from bacterial pneumonia remains challenging for the clinician. Useful biomarkers predicting CAP severity are lacking and even among pediatric ID specialists, variability in treatment is common.^15,17,18 Not only were ASP recommendations frequent with a CAP diagnosis, but also recommendations to consult ID were more likely to occur with CAP compared with other diagnoses, suggesting that the ASP potentially identified the clinical cases that were more severe or atypical and helped facilitate ID involvement in those circumstances. Fortunately, a clinical practice guideline for the treatment of CAP was implemented 4 months following ASP initiation at our hospital. Combined, these approaches resulted in a significant decrease in the use of ceftriaxone with a concomitant increase in ampicillin for the empirical treatment of CAP, confirming that a combined approach of stewardship activities can be very effective in changing clinical practice.¹⁶

Stop therapy accounted for the highest proportion of overall recommendations, raising the question about whether prior authorization or PAF might be more effective for certain antimicrobials. There are advantages to both prior authorization and the PAF approaches. Prior authorization offers an opportunity to prevent the initiation of unnecessary antimicrobials, determine the indication and planned duration of therapy, or recommend an ID consultation at the point of antimicrobial initiation.^6,19–21 Many of the diagnoses associated with a recommendation in our evaluation were those where diagnostic or therapeutic uncertainty exists and therefore allowing 48 hours to gather clinical information before review may be beneficial. However, because some broad-spectrum antimicrobials such as carbapenems (which were associated with a high likelihood of recommendation to stop therapy) should be used only in select clinical circumstances, our findings suggest that expanding use of prior authorization should be considered. Also, the majority of combination antibiotic therapy was frequently associated with the recommendation to stop, suggesting that combination therapy in general is a high-yield area for stewardship interventions.

Evaluation of both process and outcome measures is a critical part of stewardship.¹⁰ Our approach to evaluating the predictive probability of a review as well as the likelihood of agreement with the recommendation by the prescribing clinician is generalizable to other programs. Evaluation can provide key information on the distribution and utilization of ASP resources, as well as on potential areas for additional stewardship strategies. Our analysis revealed that review of commonly prescribed antimicrobials is critical in our program because these reviews result in a high number of recommendations. Further collaborations with physicians and allocation of pediatric stewardship resources that will address the most highly predictive antimicrobials and diagnoses identified in this study will likely result in a further reduction of inappropriate antimicrobial use and better use of available resources. Currently, we monitor only antibacterials. To expand the program to other anti-infectives such as antifungals and antivirals, we could consider eliminating reviews that infrequently result in an ASP recommendation.

This study has limitations. We evaluated a single institution's PAF ASP. Therefore the findings may not be generalizable to other institutions where prescribing practices differ. Specific ASP recommendations were excluded for 2.7% of the original sample, although this is unlikely to have influenced our findings. We did not look at the clinical outcomes associated with these recommendations. Stewardship has been shown to improve clinical outcomes.^{2,4–9,22,23} It is possible that reviews that had a low probability of a recommendation may have had a higher likelihood of clinical impact. Further evaluation of the clinical impact of stewardship recommendations is warranted. Classification of diagnoses was performed by the ASP reviewer and therefore misclassification could occur. Diagnoses are extracted from the medical chart and clarified by the medical team when unclear, making misclassification less likely. Finally, because the program does not currently review all prescribed antimicrobials, it is possible that opportunities for important stewardship recommendations may go unrecognized.

In summary, evaluation of both the types and distribution of ASP recommendations provides essential feedback to better understand our stewardship program. Continued evaluation of an ASP is critical so that strategies can be tailored to enhance stewardship efforts.²⁴ Although each hospital is unique in terms of clinical practice, our results provide an understanding of potentially high-yield targets for ASP recommendations. This approach can be adopted by other institutions that are initiating and continually evaluating their respective programs.