Skip to main content
PMC home page
The Canadian Journal of Hospital Pharmacy logoLink to The Canadian Journal of Hospital Pharmacy
. 2023 Jan 9;76(1):40–47. doi: 10.4212/cjhp.3258

Development of Quality Indicators to Evaluate the Appropriateness of Empiric Antimicrobial Use in Pediatric Patients

Holly MacKinnon 1, Kathryn Slayter 2, Jeannette L Comeau 3, Kathryn Timberlake 4, Michelle Science 5, Emily K Black 6,✉
PMCID: PMC9817230  PMID: 36683664

Abstract

Background

Use of quality indicators is one strategy recommended to assess antimicrobial prescribing for pediatric inpatients.

Objective

To achieve consensus from infectious diseases clinicians on quality indicators that characterize appropriate empiric antimicrobial use for the management of infectious syndromes in pediatric inpatients.

Methods

This study was completed using the Delphi technique. The research team developed an initial list of quality indicators, informed by a literature search. A multidisciplinary group of health care providers with expertise in infectious diseases was invited to participate. The list was disseminated to this panel of experts using Opinio survey software. The experts were asked to rate the indicators on a 9-point Likert scale in relation to the following criterion: “The importance of each item in determining appropriateness considering benefit or harm at the individual or population level”. Consensus was defined as at least 75% agreement and a median score of 7 or higher.

Results

Twelve of 31 invited experts completed at least 1 round of the survey, and 10 completed all rounds. Consensus was achieved on 28 of 31 proposed indicators after 3 rounds. Indicators with consensus were categorized under “empiric choice” (n = 12 indicators), “dose” (n = 5), “duration” (n = 2), “administration” (n = 4), “diagnosis” (n = 2), and “documentation” (n = 3). Six of the indicators for which consensus was achieved were rephrased by the experts.

Conclusions

Consensus was achieved on quality indicators to assess the appropriateness of empiric antimicrobial use in pediatric patients. Clinicians and researchers can use these consensus-based indicators to assess adherence to best practice.

Keywords: antimicrobial use, pediatrics, quality indicators

INTRODUCTION

Antimicrobial resistance is an increasing threat to human health worldwide. In Canada, more than a quarter of infections are currently resistant to the antimicrobial agents typically used to treat them, and this proportion is expected to rise to 40% by 2050.1 Antimicrobial resistance is also a growing concern in the United States. More than 2.8 million antimicrobial-resistant infections and 35 000 related deaths occur each year in the United States alone.2 Without action, many life-saving medical advances will no longer be available.1,2 Antimicrobial resistance also has significant socioeconomic implications. A decline in gross domestic product is projected to result from decreased labour productivity. Broader societal concerns have also been suggested, including a decrease in quality of life, social trust, and equality.1

Inappropriate antimicrobial use contributes to development of antimicrobial resistance and negative health outcomes, including increased risk of death.3,4 Antimicrobial stewardship, defined as “coordinated interventions designed to improve and measure the appropriate use of antibiotic agents”, is an important strategy to combat these negative outcomes.5 Data on the appropriateness of antimicrobial use, in addition to standard surveillance of antimicrobial utilization, is needed to inform stewardship efforts6,7; however, definitions of “appropriate use” or “appropriateness” are inconsistent.4

The proportion of antimicrobial use considered appropriate varies according to the investigators’ definition. A recent systematic review reported a large range of inappropriate antimicrobial use, from as low as 14.1% to as high as 78.9%, in hospitalized patients with severe infection.4 Data from a point prevalence survey of children’s hospitals in the United States showed that a quarter of pediatric patients were receiving suboptimal antibiotic therapy.8 Studies have often used a qualitative assessment of appropriateness based on clinician judgment; however, this approach may lead to differences in opinion because of the subjective nature of the assessment.9

The use of quality indicators is one strategy to objectively evaluate the appropriateness of antimicrobial use. A standardized list of indicators provides consistency in factors that should be considered when evaluating appropriateness. According to the Agency for Healthcare Research and Quality (US), quality indicators are “standardized, evidence-based measures of health care quality”.10 Quality indicators to evaluate the appropriateness of antibiotic use in a variety of settings have been published; however, these indicators were not specifically developed for use in the pediatric population after admission to hospital.11–15 When evaluating antimicrobial use in the pediatric population, unique considerations related to patient assessment (e.g., guidelines and recommendations for screening and diagnosis of infectious diseases, etiology of disease) and choice of antimicrobial agent (considering age, weight, and route of administration) should be considered, given the known differences between pediatric and adult populations. Monitoring of the appropriateness of antimicrobial use in the pediatric population, using standardized process measures, to assess the impact of stewardship efforts and to identify areas for quality improvement is therefore needed.

The objective of this study was to achieve consensus within a group of pediatric specialists on a list of quality indicators to evaluate the appropriateness of empiric antimicrobial use for pediatric patients admitted to hospital.

METHODS

Study Design

This study was completed using the Delphi technique, a method that uses a series of questionnaires to achieve consensus of opinion among individuals within an area of expertise. This technique allows participants to adjust their opinion after considering group feedback in successive rounds and also allows individuals to provide their opinions anonymously and without the influence of dominant individuals.16,17 This method was employed in our study as we aimed to obtain consensus virtually on a list of quality indicators of appropriate empiric antimicrobial use from experts throughout North America.

This study was approved by the IWK Health Research Ethics Board.

Questionnaire Development

A questionnaire was developed on the basis of published literature and the expertise of our research team. This team was composed of researchers and pediatric infectious diseases physicians and pharmacists (who are the authors of this article). A literature search was completed in the PubMed database to identify studies that described quality indicators suitable for evaluating the appropriateness of empiric antimicrobial prescriptions. The search strategy was designed by our team, which included clinicians and researchers with experience in completing systematic reviews. The search terms included combinations of antibiotic, antibacterial agent, or bacterial infection combined with terms for quality indicators. This search was consistent with the approach used by Kallen and Prins15 in completing a systematic review of quality indicators for determining the appropriateness of antibiotic use in adult inpatients. The reference lists of retrieved full-text articles were also searched by hand to identify relevant publications. Studies that reported on antimicrobial quality indicators or prescribing survey tools and checklists were retained for use in developing the questionnaire.

Based on the results of the literature search12,18–22 and the expertise of the research team, a questionnaire consisting of 25 indicators for evaluating the appropriateness of antimicrobial use in pediatric patients admitted to hospital was developed. The questionnaire was piloted by 5 pharmacists who had experience in infectious diseases or survey design and were not participating in the Delphi panel and was then adapted according to their feedback. Changes based on piloting of the survey included incorporation of formatting suggestions and rewording of some of the quality indicators for clarity and consistency. The pharmacists who peer reviewed the questionnaire also suggested reducing the basis for rating quality indicators from 2 criteria to a single criterion. This suggestion was accepted, and the criterion for rating indicators is described below, under “Data Collection”.

Participants

Potential participants were invited to complete the questionnaire on the basis of their expertise in providing care to pediatric patients with infectious syndromes. Experts were purposively sampled to obtain diverse representation from a multidisciplinary group of health care providers. Experts invited to participate included infectious diseases physicians, pharmacists, microbiologists, and administrators representing antimicrobial stewardship programs at various stages of implementation. Experts in Canada and the United States were considered for inclusion, to ensure broad geographic representation throughout North America. Experts were identified through the team’s professional networks and were invited to participate through email communication by a member of the research team. The initial list of experts was identified from previous work by members of our team (K.T., M.S.), who used the Delphi technique to develop quality indicators for evaluating antimicrobial stewardship programs.23 All participants provided consent through the online consent statement at the beginning of the questionnaire.

Data Collection

Experts who agreed to participate were asked to complete consecutive rounds of the Delphi process to establish consensus on the indicators. In each round, the experts were asked to review and rate each indicator listed in the questionnaire, which was disseminated through the survey tool Opinio, housed by Dalhousie University. The following criterion was used to rate the indicators: “The importance of each item in determining appropriateness considering benefit or harm at the individual or population level”.

During the first round, the experts were asked to rate the indicators on a scale from 1 to 9, where 1 = very unimportant and 9 = very important. The experts were also given the opportunity to comment on the indicators, suggest changes to wording, or add new indicators. In the second round, each expert received an individualized questionnaire that included, for each indicator, their previous rating, the aggregate mean rating (with standard deviation), the aggregate median rating (with interquartile range), the mode, and anonymous comments from the other experts. Newly suggested indicators were also added to the second-round questionnaire. The experts were asked to again rate the indicators on a scale from 1 to 9. In addition, the experts were provided with wording changes suggested by participants in round 1 and asked to indicate if they agreed or disagreed with the proposed changes (yes/no). An additional third version of the questionnaire was circulated for experts to rate the indicators with remaining disagreement after round 2.

Data Analysis

The results were summarized using descriptive statistics. Indicators with a median score of 7 or higher with no disagreement after 2 rounds of rating were retained and included in the final list of indicators for assessing appropriateness. Based on previously published related Delphi studies23–26 and published guidelines,27 disagreement was defined as less than 75% of panelists assigning a score of 7 or higher.

RESULTS

The 3 versions of the questionnaire were distributed to the expert panel in 3 rounds of the Delphi process between July and December 2018. Thirty-one experts were invited to participate in an attempt to recruit 15–20 participants, and 17 experts agreed to participate. Of those who agreed to participate, 12 completed at least 1 round of the questionnaire, with 10 of the 12 experts completing all rounds. Panelists who agreed to participate were infectious diseases physicians (n = 5) and infectious diseases pharmacists (n = 7). Most of the experts (n = 8) were practising in Canada.

A total of 25 indicators included in the initial questionnaire and 6 indicators suggested by the expert panelists were assessed during the 3 rounds (Table 1). After the initial round, consensus was achieved for 23 of the initial 25 indicators, and 6 new indicators were suggested. The 2 indicators with disagreement in round 1 were included in subsequent rounds. Six of the indicators with agreement also had suggested changes to wording; these indicators were rephrased, incorporated into the second round, and accepted by the experts (Table 2). After completion of 3 rounds, consensus was reached for 24 of the 25 quality indicators originally proposed and 4 of the 6 indicators suggested by expert panelists in the first round, and these 28 indicators were retained. The indicators for which consensus was reached were grouped under the categories of “empiric choice” (n = 12), “dose” (n = 5), “duration” (n = 2), “administration” (n = 4), “diagnosis” (n = 2), and “documentation” (n = 3).

TABLE 1.

Rating and Assessment of Quality Indicators by Experts

Indicatora Round 1 (n = 12) Round 2 (n = 10) Round 3 (n = 10) Conclusion
No. (%) Strong Agreement Median Score No. (%) Strong Agreement Median Score No. (%) Strong Agreement Median Score
Empiric choice
Empiric choice of antimicrobial agents for pediatric patients should be active against most likely causative pathogens. 12 (100) 9 – – – – Retain
Empiric choice of systemic antimicrobial therapy in pediatric patients should consider local susceptibilities (local antibiogram). – – 10 (100) 8.5 – – Retain
Pediatric patients with a history of anaphylaxis after penicillin therapy should be prescribed an alternative drug class. 10 (83) 8.5 – – – – Retain
Empiric choice of systemic antimicrobial therapy for pediatric patients should be prescribed according to local guidelines. If no local guidelines exist, choice of therapy should be prescribed according to national or international guidelines (where available). 12 (100) 8 – – – – Retain
Previous microbiology results should be considered in empiric choice of systemic antimicrobial therapy in pediatric patients. 11 (92) 8 – – – – Retain, rephrased
Contraindications (including medical conditions and medication use) should be taken into account when antimicrobials are prescribed to pediatric patients. 10 (83) 8 – – – – Retain, rephrased
Allergy status and history of adverse drug reactions should be taken into consideration when selecting empiric antimicrobial agents for pediatric patients. 9 (75) 8 – – – – Retain
Previous history of infection should be considered in empiric choice of systemic antimicrobial therapy in pediatric patients.b 8 (67) 8 – – 8 (80) 8 Retain, rephrased
Empiric choice of systemic antimicrobial therapy in pediatric patients should consider individual travel history. – – 10 (100) 7.5 – – Retain
Empiric choice of systemic antimicrobial therapy in pediatric patients should include data on local public health outbreaks. – – 8 (80) 7 – – Retain
Empiric choice of systemic antimicrobial therapy in pediatric patients does not include unnecessary duplication of therapy. – – 6 (60) 7.5 7 (78)
(n = 9)		 7 Retain
Empiric choice of systemic antimicrobial therapy should consider previous antimicrobial use in pediatric patients. 9 (75) 7 – – – – Retain, rephrased
Empiric choice of systemic antimicrobial therapy in pediatric patients should consider vaccination status. – – 5 (50) 7 4 (40) 6 Exclude
Empiric choice of systemic antimicrobial therapy in pediatric patients should consider previous environment exposures. – – 5 (50) 6.5 4 (40) 6 Exclude
Dose
Dose and dosing interval of systemic empiric antimicrobials should be adapted to pediatric patient renal function. 11 (92) 8 – – – – Retain
Antimicrobial agents that require therapeutic drug monitoring (such as vancomycin and gentamicin) should be managed according to guidelines. 10 (91)
(n = 11)		 8 – – – – Retain
Dose and dosing interval of systemic empiric antimicrobials should be adapted to the pediatric patient’s age. 10 (83) 8.5 – – – – Retain
Dose and dosing interval of systemic empiric antimicrobials should be prescribed according to guidelines. 10 (83) 8 – – – – Retain
Dose and dosing interval of systemic antimicrobials should be adapted to the pediatric patient’s weight. 10 (83) 8 – – – – Retain
Duration
Duration of surgical prophylaxis for pediatric patients should not exceed 24 hours. 11 (92) 9 – – – – Retain
Intended duration of systemic empiric antimicrobial therapy for pediatric patients should be compliant with guidelines. 10 (91)
(n = 11)		 8 – – – – Retain
Administration
Antimicrobial agents for pediatric patients with sepsis should be started intravenously. 12 (100) 9 – – – – Retain
Broad spectrum intravenous empiric antimicrobials should be administered to pediatric patients with severe sepsis and septic shock within 1 hour of identification. 12 (100) 9 – – – – Retain
Timelines of administration of antimicrobial therapy and prophylaxis for pediatric patients should be compliant with guidelines. 11 (92) 8.5 – – – – Retain
Empiric antimicrobial agents for pediatric patients should be administered via the appropriate route as recommended by guidelines. 11 (92) 8 – – – – Retain
Diagnosis
When starting systemic antimicrobial therapy for pediatric patients, specimens for culture from suspected sites of infection should be taken as soon as possible, preferably before antimicrobial agents are started (if applicable). 11 (92) 9 – – – – Retain
Microbiological investigations should be performed according to guidelines. 10 (83) 8 – – – – Retain
Two sets of blood cultures should be taken before antimicrobial administration when bacteremia is suspected in pediatric patients. 7 (58) 7 3 (30) 6 – – Exclude
Documentation
Allergy status (including nature and severity) should be documented in the medical records when antimicrobials are prescribed for pediatric patients. 12 (100) 8.5 – – – – Retain
Antimicrobial therapy for pediatric patients that deviate[s] from guidelines should be justified. 10 (83) 8.5 – – – – Retain, rephrased
An antimicrobial plan should be documented for pediatric patients in the medical record at the start of systemic antimicrobial treatment. (Antimicrobial plan in indication, name, dose, route, and interval of administration.) 9 (75) 8 – – – – Retain, rephrased
Open in a new tab

Note: Dashes are used for indicators not included in a particular round of the Delphi process.

a

For the 6 indicators with rephrasing (as noted in col. 3), the entry shown here incorporates the revised wording. See Table 2 for original wording.

b

Indicator omitted in error during round 2, but consensus was achieved in round 3.

TABLE 2.

Accepted Changes in Wording of Original Quality Indicators

Original Wording Suggested Wording Change No. (%) in Agreement
(n = 10)
Empiric choice of systemic antimicrobial therapy is appropriate for pediatric patients considering previous history of infection. Previous history of infection should be considered in empiric choice of systemic antimicrobial therapy in pediatric patients. 10 (100)
Empiric choice of systemic antimicrobial therapy is appropriate for pediatric patients considering previous antimicrobial use. Empiric choice of systemic antimicrobial therapy should consider previous antimicrobial use in pediatric patients. 10 (100)
Empiric choice of systemic antimicrobial therapy is appropriate for pediatric patients considering previous microbiology results. Previous microbiology results should be considered in empiric choice of systemic antimicrobial therapy in pediatric patients. 10 (100)
Contraindications (including concomitant medical conditions and medication use) should be taken into account when antibiotics are prescribed to pediatric patients. Contraindications (including medical conditions and medication use) should be taken into account when antimicrobials are prescribed to pediatric patients. 9 (90)
An antimicrobial plan should be documented for pediatric patients in the case notes at the start of systemic antimicrobial treatment. (Antibiotic plan is indication, name, dose, route, and interval of administration.) An antimicrobial plan should be documented for pediatric patients in the medical record at the start of systemic antimicrobial treatment. (Antimicrobial plan in indication, name, dose, route, and interval of administration.) 10 (100)
Antibiotic therapy for pediatric patients that deviate[s] from guidelines should be justified. Antimicrobial therapy for pediatric patients that deviate[s] from guidelines should be justified. 10 (100)
Open in a new tab

The highest-ranking indicators, which had 100% agreement by the experts and a median score of 9, were the following:

  • “Empiric choice of antimicrobial agents for pediatric patients should be active against most likely causative pathogens.”

  • “Antimicrobial agents for pediatric patients with sepsis should be started intravenously.”

  • “Broad spectrum intravenous empiric antimicrobials should be administered to pediatric patients with severe sepsis and septic shock within 1 hour of identification.”

The only indicator originally proposed to the experts that was ultimately rejected was “Two sets of blood cultures should be taken before antimicrobial administration when bacteremia is suspected in pediatric patients”.

Ratings for each indicator during the 3 rounds of the Delphi process are presented in Table 1.

DISCUSSION

To the authors’ knowledge, this study is the first to seek and achieve consensus on quality indicators to characterize the appropriateness of empiric antimicrobial use for the management of infectious syndromes in hospitalized pediatric patients. Panel representation included experts throughout North America with experience providing direct care to pediatric patients with infectious syndromes. All indicators included in this study were process-related measures that aimed to assess the quality of antimicrobial use. Most approved indicators were categorized as relating to “empiric choice”. High agreement after the first round by experts who worked as clinicians suggests that the indicators initially proposed are clinically important and relevant to improving the quality of patient care.

Indicators for determining appropriateness of use of antimicrobial agents in hospitalized patients have been developed by others. Monnier and others11 published 51 generic quality indicators for responsible antibiotic use in the inpatient setting. A broad range of stakeholders were included in that study; however, few participants were from North America (n = 5/25).11 In another study, a European panel of experts developed quality indicators for evaluating the appropriateness of antimicrobial use in hospitalized adults.12 Neither of these studies specifically focused on management of infectious diseases or antimicrobial use in pediatric patients.11,12 Considerations when determining appropriateness of antibiotic use in the management of infectious diseases in this patient population were therefore needed. Pediatric patients are not small adults: they exhibit differences in the spectra of infections that they may acquire, and their presentations differ from those of adults. Children, especially neonates, require special consideration when determining choice of antimicrobial therapy, including unique precautions and contraindications, as well as differences in dosing and formulation.

Many generic quality indicators related to empiric antimicrobial use that were published by Monnier and others11 and van den Bosch and others12 were used in development of our survey, with tailoring for the pediatric population. Retained quality indicators in our study overlap with previously published indicators; however, our expert panel also suggested additional indicators that focus on specific considerations in choosing the most appropriate empiric antimicrobial agent. These indicators are more tailored and may prompt further consideration of appropriateness at an individual patient level.

Our expert panel rejected the indicator “Two sets of blood cultures should be taken before antimicrobial administration when bacteremia is suspected in pediatric patients,” although this indicator was included and retained by previously published studies.11,12 Determining the rationale for indicator ranking was not within the scope of our study; however, it is postulated that respondents may have rejected the indicator given difficulty with venous access, especially in neonates. The need for adequate sample volume is the most important consideration for detection of bacteria when performing blood cultures. In children, the volume should be determined on the basis of the patient’s age and weight. In pediatrics especially, there must be a balance between volume of blood collected and the patient’s clinical condition.28,29

Our study had several strengths. The expert consensus panel comprised infectious diseases specialists and pharmacists with experience caring for pediatric patients with infectious syndromes within a North American context. The indicators presented in the first round were designed as process measures for clinicians and researchers to evaluate antimicrobial use and assess the impact of antimicrobial stewardship interventions. Furthermore, the indicators are detailed and provide opportunities to clearly identify areas for improvement in the processes of prescribing and administering antimicrobial agents.

Despite these strengths, a number of limitations should be considered. The expert panel included only pharmacists and physicians, as we were unable to recruit any microbiologists. Thus, our study yielded the perspectives of only pharmacists and physicians, although we recognize that other health care providers have valuable expertise to contribute to assessing appropriateness of antimicrobial use. Furthermore, the indicators were developed on the basis of evidence and guidelines current at the time. Since dissemination of our survey, pediatric guidelines in the Surviving Sepsis Campaign have been published, which recommend starting antimicrobials as soon as possible and within 3 hours for children with sepsis-associated organ dysfunction and no signs of shock.30 The Surviving Sepsis Campaign recommendation is reported to have a very low quality of evidence30; however, use of our indicators should be combined with consideration of the most recent evidence when evaluating appropriateness of antibiotic use. Finally, our study included only experts from North America and, as a result, the findings may not be generalizable to other geographic regions. Given overlap of retained indicators from the current study with those from European and other international Delphi studies, however, we expect that our findings may be relevant to other regions of the world.

CONCLUSION

To our knowledge, this is the first study to report on process-related quality indicators for assessing the appropriateness of empiric antimicrobial use in pediatric patients admitted to hospital on which consensus was achieved by an expert panel. Our findings may provide a standardized list of measures that infectious diseases clinicians, antimicrobial stewardship teams, institutions, and researchers can use when evaluating the effect of various interventions on the quality of antimicrobial use.

Footnotes

Competing interests: For activities unrelated to the study reported here, Emily Black has received research grants from the Drug Evaluation Alliance of Nova Scotia and Research Nova Scotia; consulting fees from Pear Health Solutions and the Ontario Pharmacists Association; presentation honoraria from the Continuing Pharmacy Education Division of Dalhousie University; and travel funding from the Canadian Pharmacists Association related to the Knowledge to Practice Advisory Circle. No other competing interests were declared.

Funding: This study was supported by the Dalhousie University Pharmacy Endowment Fund. Holly MacKinnon received salary support from the Dalhousie University Pharmacy Endowment Fund for work on this study.

References

  • 1.Expert Panel on the Potential Socio-Economic Impacts of Antimicrobial Resistance in Canada. When antibiotics fail. Council of Canadian Academies; 2019. [cited 2020 Apr 11]. Available from: https://cca-reports.ca/reports/the-potential-socio-economic-impacts-of-antimicrobial-resistance-in-canada/ [Google Scholar]
  • 2.Antibiotic resistance threats in the United States, 2019. US Department of Health and Human Services, Centers for Disease Control and Prevention; 2019. [cited 2020 Apr 11]. Available from: https://www.cdc.gov/drugresistance/pdf/threats-report/2019-ar-threats-report-508.pdf. [Google Scholar]
  • 3. Davey PG, Marwick C. Appropriate vs. inappropriate antimicrobial therapy. Clin Microbiol Infect. 2008;14(Suppl 3):15–21. doi: 10.1111/j.1469-0691.2008.01959.x. [DOI] [PubMed] [Google Scholar]
  • 4. Marquet K, Liesenborgs A, Bergs J, Vleugels A, Claes N. Incidence and outcome of inappropriate in-hospital empiric antibiotics for severe infection: a systematic review and meta-analysis. Crit Care. 2015;19(1):63. doi: 10.1186/s13054-015-0795-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Fishman N. Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS) Infect Control Hosp Epidemiol. 2012;33(4):322–7. doi: 10.1086/665010. [DOI] [PubMed] [Google Scholar]
  • 6. Barlam TF, Cosgrove SE, Abbo LM, MacDougall C, Schuetz AN, Septimus EJ, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. 2016;62(10):e51–e77. doi: 10.1093/cid/ciw118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Canadian Roundtable on Antimicrobial Stewardship. NCCID Project 279. HealthCareCAN and National Collaborating Centre for Infectious Diseases; 2016. Putting the pieces together: a national action plan on antimicrobial stewardship. [cited 2020 Apr 11]. Available from: http://www.healthcarecan.ca/wp-content/themes/camyno/assets/document/Reports/2016/HCC/EN/Putting%20the%20Pieces%20Together%20-%20Final%20Version.pdf. [Google Scholar]
  • 8. Tribble AC, Lee BR, Flett KB, Handy LK, Gerber JS, Hersh AL, et al. Appropriateness of antibiotic prescribing in United States children’s hospitals: a national point prevalence survey. Clin Infect Dis. 2020;71(8):e226–e234. doi: 10.1093/cid/ciaa036. [DOI] [PubMed] [Google Scholar]
  • 9. DePestel DD, Eiland EH, 3rd, Lusardi K, Destache CJ, Mercier RC, McDaneld PM, et al. Assessing appropriateness of antimicrobial therapy: in the eye of the interpreter. Clin Infect Dis. 2014;59(Suppl 3):S154–S161. doi: 10.1093/cid/ciu548. [DOI] [PubMed] [Google Scholar]
  • 10.Quality improvement and monitoring at your fingertips [website] Agency for Healthcare Research and Quality; [cited 2020 Apr 11]. Available from: https://www.qualityindicators.ahrq.gov/ [Google Scholar]
  • 11. Monnier AA, Schouten J, Le Maréchal M, Tebano G, Pulcini C, Stanic Benic M, et al. Quality indicators for responsible antibiotic use in the inpatient setting: a systematic review followed by an international multidisciplinary consensus procedure. J Antimicrob Chemother. 2018;73(Suppl 6):vi30–vi39. doi: 10.1093/jac/dky116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. van den Bosch CMA, Geerlings SE, Natsch S, Prins JM, Hulscher MEJL. Quality indicators to measure appropriate antibiotic use in hospitalized adults. Clin Infect Dis. 2015;60(2):281–91. doi: 10.1093/cid/ciu747. [DOI] [PubMed] [Google Scholar]
  • 13. Kallen MC, Roos-Blom MJ, Dongelmans DA, Schouten JA, Gude WT, de Jonge E, et al. Development of actionable quality indicators and an action implementation toolbox for appropriate antibiotic use at intensive care units: a modified-RAND Delphi study. PLoS One. 2018;13(11):e0207991. doi: 10.1371/journal.pone.0207991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Saust LT, Bjerrum L, Arpi M, Hansen MP. Quality indicators for the diagnosis and antibiotic treatment of acute respiratory tract infections in general practice: a RAND appropriateness method. Scand J Prim Health Care. 2017;35(2):192–200. doi: 10.1080/02813432.2017.1333305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Kallen MC, Prins JM. A systematic review of quality indicators for appropriate antibiotic use in hospitalized adult patients. Infect Dis Rep. 2017;9(1):6821. doi: 10.4081/idr.2017.6821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12:8. Article 10. [Google Scholar]
  • 17. Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41(4):376–82. doi: 10.1046/j.1365-2648.2003.02537.x. [DOI] [PubMed] [Google Scholar]
  • 18. Spivak ES, Cosgrove SE, Srinivasan A. Measuring appropriate antimicrobial use: attempts at opening the black box. Clin Infect Dis. 2016;63(12):1639–44. doi: 10.1093/cid/ciw658. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. James R, Upjohn L, Cotta M, Luu S, Marshall C, Buising K, et al. Measuring antimicrobial prescribing quality in Australian hospitals: development and evaluation of a national antimicrobial prescribing survey tool. J Antimicrob Chemother. 2015;70(6):1912–8. doi: 10.1093/jac/dkv047. [DOI] [PubMed] [Google Scholar]
  • 20. van den Bosch CMA, Hulscher MEJL, Natsch S, Wille J, Prins JM, Geerlings SE. Applicability of generic quality indicators for appropriate antibiotic use in daily hospital practice: a cross-sectional point-prevalence multicenter study. Clin Microbiol Infect. 2016;22(10):888.e1–888.e9. doi: 10.1016/j.cmi.2016.07.011. [DOI] [PubMed] [Google Scholar]
  • 21.Monnier A, Hulscher M, Schouten J, Kullberg BJ, Gyssens I, Stanic M, et al. DRIVE-AB WP1A quality indicators and quantity metrics of antibiotic use. DRIVE-AB; 2016. [cited 2020 Apr 11]. Available from: http://driveab.eu/wp-content/uploads/2014/09/WP1A_Final-QMs-QIs_final.pdf. [Google Scholar]
  • 22. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al. Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 2017;43(3):304–77. doi: 10.1007/s00134-017-4683-6. [DOI] [PubMed] [Google Scholar]
  • 23. Science M, Timberlake K, Morris A, Read S, Le Saux N. Groupe Antibiothérapie en Pédiatrie Canada Alliance for Stewardship of Antimicrobials in Pediatrics (GAP Can ASAP) Quality metrics for antimicrobial stewardship programs. Pediatrics. 2019;143(4):e20182372. doi: 10.1542/peds.2018-2372. [DOI] [PubMed] [Google Scholar]
  • 24. Morris AM, Brener S, Dresser L, Daneman N, Dellit TH, Avdic E, et al. Use of a structured panel process to define quality metrics for antimicrobial stewardship programs. Infect Control Hosp Epidemiol. 2012;33(5):500–6. doi: 10.1086/665324. [DOI] [PubMed] [Google Scholar]
  • 25. Bell CM, Brener SS, Comrie R, Anderson GM, Bronskill SE. Quality measures for medication continuity in long-term care facilities, using a structured panel process. Drugs Aging. 2012;29(4):319–27. doi: 10.2165/11599150-000000000-00000. [DOI] [PubMed] [Google Scholar]
  • 26. Fernandes O, Gorman SK, Slavik RS, Semchuk WS, Shalansky S, Bussières JF, et al. Development of clinical pharmacy key performance indicators for hospital pharmacists using a modified Delphi approach. Ann Pharmacother. 2015;49(6):656–69. doi: 10.1177/1060028015577445. [DOI] [PubMed] [Google Scholar]
  • 27. Loblaw DA, Prestrud AA, Somerfield MR, Oliver TK, Brouwers MC, Nam RK, et al. American Society of Clinical Oncology clinical practice guidelines: formal systematic review-based consensus methodology. J Clin Oncol. 2012;30(25):3136–40. doi: 10.1200/JCO.2012.42.0489. [DOI] [PubMed] [Google Scholar]
  • 28. Miller JM, Binnicker MJ, Campbell S, Carroll KC, Chapin KC, Gilligan PH, et al. A guide to utilization of the microbiology laboratory for diagnosis of infectious diseases: 2018 update by the Infectious Diseases Society of America and the American Society for Microbiology. Clin Infect Dis. 2018;67(6):e1–e94. doi: 10.1093/cid/ciy381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Huber S, Hetzer B, Crazzolara R, Orth-Höller D. The correct blood volume for paediatric blood cultures: a conundrum? Clin Microbiol Infect. 2020;26(2):168–73. doi: 10.1016/j.cmi.2019.10.006. [DOI] [PubMed] [Google Scholar]
  • 30. Weiss SL, Peters MJ, Alhazzani W, Agus MSD, Flori HR, Inwald DP, et al. Surviving Sepsis Campaign international guidelines for the management of septic shock and sepsis-associated organ dysfunction in children. Pediatr Crit Care Med. 2020;21(2):e52–e106. doi: 10.1097/PCC.0000000000002198. [DOI] [PubMed] [Google Scholar]

Articles from The Canadian Journal of Hospital Pharmacy are provided here courtesy of Canadian Society of Healthcare-Systems Pharmacy

RESOURCES