Quality indicators for responsible antibiotic use in the inpatient setting: a systematic review followed by an international multidisciplinary consensus procedure

Abstract

Background

This study was conducted as part of the Driving Reinvestment in Research and Development and Responsible Antibiotic Use (DRIVE-AB) project and aimed to develop generic quality indicators (QIs) for responsible antibiotic use in the inpatient setting.

Methods

A RAND-modified Delphi method was applied. First, QIs were identified by a systematic review. A complementary search was performed on web sites of relevant organizations. Duplicates were removed and disease and patient-specific QIs were combined into generic indicators. The relevance of these QIs was appraised by a multidisciplinary international stakeholder panel through two questionnaires and an in-between consensus meeting.

Results

The systematic review retrieved 70 potential generic QIs. The QIs were appraised by 25 international stakeholders with diverse backgrounds (medical community, public health, patients, antibiotic research and development, regulators, governments). Ultimately, 51 QIs were selected in consensus. QIs with the highest relevance score included: (i) an antibiotic plan should be documented in the medical record at the start of the antibiotic treatment; (ii) the results of bacteriological susceptibility testing should be documented in the medical record; (iii) the local guidelines should correspond to the national guidelines but should be adapted based on local resistance patterns; (iv) an antibiotic stewardship programme should be in place at the healthcare facility; and (v) allergy status should be taken into account when antibiotics are prescribed.

Conclusions

This systematic and stepwise method combining evidence from literature and stakeholder opinion led to multidisciplinary international consensus on generic inpatient QIs that can be used globally to assess the quality of antibiotic use.

Introduction

The loss of effectiveness of many antibiotics as a consequence of the emergence of antibiotic resistance has evolved to become a major threat to global public health. Unfortunately, this phenomenon coincides with a dry pipeline in antibiotic research and development (R&D).¹ The repercussions for patient care include increased mortality and limited effective therapy options for MDR and hospital-acquired infections.² In addition, treatment failure caused by antibiotic resistance has considerable financial consequences through, for example, prolonged hospital stay or more expensive antibiotic therapy.²

While all antibiotic use drives the emergence and dissemination of resistance to some degree, a major aggravating force is the inappropriate use of antibiotics.³ Therefore, reducing both overall antibiotic consumption and inappropriate use is a strategy to slow the pace of the emergence of resistant bacteria.^4–6 This strategy, commonly referred to as antibiotic stewardship, aims to measure and to improve antibiotic use.⁷ In order to be successful, antibiotic stewardship programmes (ASPs) should comprise distinct tools for measuring both the quantity and quality of antibiotic use.

Measuring (in)appropriateness of healthcare is typically done using quality indicators (QIs), defined as ‘measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality, and hence change the quality of care provided’.⁸ QIs of appropriate antibiotic use are valuable tools for ASPs as they guide the selection of improvement targets as well as help to establish the effectiveness of improvement interventions.⁹

The Driving Reinvestment in Research and Development and Responsible Antibiotic Use (DRIVE-AB) research consortium proposes the following definitions to distinguish between the assessment of quality and quantity of antibiotic use. A QI reflects the degree to which antibiotic use is correct or appropriate while, in contrast, a quantity metric reflects the volume or the costs of antibiotic use.¹⁰ Therefore, the QI has a value on its own while the quantity metric only gains value when comparisons are made between, for example, wards, hospitals or countries.

The DRIVE-AB project aims at reaching consensus on a standard for ‘responsible antibiotic use’, including QIs, between a large variety of stakeholders involved in antibiotic use: from prescribers through producers and regulators to patients. Indeed, the large scale of societal implications of the loss of antibiotic effectiveness as a result of resistance development requires combined cross-disciplinary efforts throughout the whole of society.

The aim of this study was to develop generic QIs for the inpatient setting taking into account different perspectives, including the medical community, public health, patients, developers and producers (antibiotic R&D), regulators and governments. Altogether, the QIs should serve as guiding principles for antibiotic use in the inpatient setting across diverse socioeconomic settings.

Materials and methods

A four-step RAND-modified Delphi method,^11–13 an iterative process of rating and soliciting expert input with multiple opportunities for feedback, was applied to develop generic QIs for antibiotic use in the inpatient setting. The consensus procedure combined the individual opinions of four groups of stakeholders. All the stakeholders consented to participate in the study and were aware that their answers would be used for research. The consensus procedure took place from mid-August 2015 till the end of January 2016.

Step 1 – Literature and web site search

A systematic review was performed in the MEDLINE database (since 1966) to identify papers describing inpatient QIs for antibiotic use. An inpatient was defined as a patient who is admitted to a hospital or healthcare facility for treatment. A healthcare facility was defined as any location where healthcare is provided, ranging from small clinics to hospitals. The search was performed on 5 February 2015. The search strategy is detailed in Figure S1 (available as Supplementary data at JAC Online). The aim was to identify QIs for antibiotic use that were either evidence-based (literature review, evidence-based guidelines) or consensus-based (formal and validated consensus, such as Delphi). Papers were included when written in English, on the use of systemically administered antibiotics drugs and when describing QIs for antibiotic use in the inpatient setting. Papers on the use of antiviral, antifungal, antiparasitic or antituberculosis drugs were excluded. Papers were also excluded when describing QIs for rare or orphan diseases, as reported by Orphanet.¹⁴ Finally, papers of which the full-text version was not accessible from Google Scholar® or one of the following libraries were also excluded: French National Institute of Health and Medical Research (INSERM), Radboud University Medical Center, University of Rijeka, University of Antwerp, University of Geneva, University of Leuven and University of Lorraine.

Two researchers (A. A. M. and I. C. G.) independently examined all titles and abstracts to select papers describing QIs for the inpatient setting using the Distiller^® software (Evidence Partners, Ottawa, Canada). Any disagreement on inclusion or exclusion of studies was resolved through discussion with a third author (M. E. H.). If no abstract was available or information was lacking for eligibility assessment, papers were selected for full-text screening. The exclusion of papers based on full-text screening was performed by one author (A. A. M.) and validated by a senior researcher (I. C. G.).

A complementary search was performed on English web sites of relevant (inter)national organizations and institutions active in the field of antibiotic stewardship, quality improvement and/or public health. Relevant web sites were selected in consensus by the authors, all working in the field of infectious diseases and/or antibiotic stewardship. Relevant sections of the web sites were searched by one reviewer (A. A. M.) using the search terms ‘indicator’ and/or ‘antibiotic/antimicrobial’.

The data extraction of QIs of antibiotic use was performed by one researcher (A. A. M.) using a standardized form. For the papers identified by the systematic review, the extraction process was repeated by the same researcher a second time for 10% of the references. Similarly, the data extraction from web sites was performed by the same researcher twice. The extracted QIs were then clustered into different non-overlapping logical themes based on the elements of the definition of responsible use.¹⁵ When a QI could be allocated to more than one theme, the predominant theme was chosen in consensus between two authors (A. A. M. and I. C. G.). Duplicates were removed. Overlapping QIs were then aggregated and, when applicable, disease- and patient-specific QIs were made more generic. Generic was defined as general and applicable to a large group of patients, not specific to any particular disease, country region or inpatient care setting.¹⁶ When needed, QIs were rephrased as a recommendation. The clustering, aggregation and rephrasing steps were undertaken in consensus between four authors (A. A. M., I. C. G., J. S. and M. E. H.).

Step 2 – First questionnaire

International stakeholders were invited by e-mail to participate. Stakeholders were invited based either on demonstrated experience and expertise in the topic of antibiotic use and/or stewardship, or in different perspectives of antibiotic use. Invited stakeholders originated from various countries across all continents. Stakeholders among the extended international network of academic and European Federation of Pharmaceutical Industries and Associations (EFPIA) partners of the DRIVE-AB project were solicited. Fifty-two stakeholders amongst four different groups, aiming at representing all parties involved in antibiotic use, were invited: medical community (n = 15); public health and patients (n = 12); antibiotic R&D (n = 14); and payers, policy makers, governments and regulators (n = 11).

A digital web-based questionnaire, with the potential inpatient QIs identified by the literature and web site searches, was designed using SurveyMonkey^® (Palo Alto, CA, USA). Together with the invitation e-mail, stakeholders received a document providing the scientific references for each of the identified QIs. The stakeholders were asked to appraise the relevance of each indicator for assessing the quality of antibiotic use. The 70 potential QIs were categorized and presented to the stakeholders. The assessment of relevance was done using a nine-point Likert scale (1 = clearly not relevant, 9 = clearly relevant). Stakeholders could also select the ‘cannot assess’ answer. Median scores were analysed across the four stakeholder groups. Relevance scores were interpreted as described elsewhere.^17,18 If the QI had a median ≥8 and if there was agreement between the stakeholders, the QI was selected. If the QI had a median ≥8 in combination with stakeholder disagreement, the QI was labelled for discussion. If the QI had a median <8, the QI was rejected. Agreement and disagreement were defined as ≥70% and <70% of the scores being in the upper tertile (score 7–9), respectively.^17,18 Stakeholders could comment on each indicator as well as propose new QIs. Newly proposed QIs that did not present any overlap with other QIs were selected for discussion in the consensus meeting.

Step 3 – Consensus meeting

Stakeholders that participated in the first questionnaire were asked to take part in a face-to-face consensus meeting on 29 September 2015. In addition, stakeholders from outside Europe could participate through a web-conferencing interface. The aim was to reach a balanced number of stakeholders across the four groups. Before the meeting, participants received a personal feedback report with the results of the first questionnaire including their own relevance scores together with the group scores. For the QIs labelled for discussion, the comments made by the stakeholders in the first online questionnaire were shown to expedite the discussion. During the meeting, the stakeholders discussed QIs labelled for discussion and newly proposed QIs. Discussed QIs could be selected, rejected or rephrased. For the rephrased QIs, modifications would typically be made to the new wording proposal until agreement was reached between all participating stakeholders. An audio recording of the meeting was made and used to make sure no relevant suggestions or comments were missed during the meeting.

Step 4 – Second questionnaire

A second web-based questionnaire including all selected and rephrased QIs was sent to all participating stakeholders together with a personal feedback report (providing the results of the previous two steps of the consensus procedure). Stakeholders were asked to appraise the QIs by answering the question ‘Do you agree with this indicator?’ with ‘yes’ or ‘no’. Stakeholders could select the ‘cannot assess’ answer and provide comments. Indicators were selected if >70% of the stakeholders agreed.¹⁸

The QIs were finally categorized as (i) structure indicators reflecting organizational aspects of healthcare, (ii) process indicators describing the care delivered to patients and (iii) outcome indicators specifying the effects of the care given to patients according to the Donabedian model.¹⁹

Results

Figure 1 shows the results of the selection process of the inpatient QIs (IQIs) after the different steps of the RAND-modified Delphi method.

Figure 1.

The results after each step of the RAND-modified Delphi method.

Step 1 – Literature and web site search

The systematic literature search identified 620 articles, of which 272 (44%) were considered eligible for full-text screening. After exclusion and inclusion criteria were applied, 139 articles (22%) were included and data extraction was performed. The flowchart of the systematic review is shown in Figure S2. The web sites of 26 institutions or organizations were searched for QIs. Eight web sites (31%) were ultimately included for data extraction (Table S1). The systematic review and the complementary web site search led to the identification of 518 QIs of antibiotic use in the inpatient setting. The web site search identified 62 QIs. Systematic aggregation and rephrasing into ‘generic’ indicators resulted in a set of 70 QIs for appraisal by the multidisciplinary stakeholder panel (Figure 1). To illustrate the aggregation and rephrasing process, an example is shown. The identified QIs ‘Blood cultures performed in the emergency department prior to initial antibiotic received in hospital’, ‘Proportion of community acquired pneumonia (CAP) patients who have blood cultures drawn and proportion whose initial blood cultures are performed prior to the administration or the first hospital dose of antibiotics’, ‘Blood sample taken before start of antibiotics’ and ‘Before starting systemic antibiotic therapy, at least 2 sets of blood cultures should be taken’ led to the generic IQI ‘IQI-31: Two sets of blood cultures should be taken before antibiotic administration when bacteraemia is suspected’. The 70 potential QIs were categorized into 20 themes: Access-Availability; Antibacterial Activity; Antibacterial Spectrum; Documentation; Dosing, PK/PD, Interval; Duration; Education; Expertise and Resources; Evidence-Based Guidelines; Indication; Interactions; Microbiological Diagnostics; Patient Outcome; Prescribing; Resistance Surveillance; Route; Surgical Prophylaxis; Therapeutic Drug Monitoring; and Timing and Toxicity.

Step 2 – First online questionnaire

In the online questionnaire, a multidisciplinary panel of 25 stakeholders (response rate 48%) from 15 countries across four continents (n = 15 from Europe, n = 5 from North America, n = 4 from Asia and n = 1 from Australia) appraised the relevance of the potential QIs for assessing the quality of antibiotic use. The online questionnaire is shown in Figure S3.

The 25 stakeholders were distributed as follows: n = 10 (response rate 67%) belonged to the medical community group, including board members of national and European professional societies, hospital pharmacists and infectious disease physicians; n = 3 (response rate 25%) to the public health and patients group, including the WHO, the Chennai Declaration Group and the Swedish public health institute; n = 7 (response rate 50%) to the antibiotic R&D group, including small and medium enterprises, large pharmaceutical companies and (health) economists; and n = 5 (response rate 46%) to the payers, policy makers, governments and regulators group, including the EMA, the US CDC, governments, a health technology assessment institute and a national health insurance advisor. A detailed list of all the stakeholders and their affiliations is shown in Table S2.

Based on relevance scores, 48 QIs were selected, 12 QIs were rejected and 10 QIs were labelled for discussion (Figure 1). Remarkably, the 12 rejected QIs included the two QIs relating to the theme Indication. Two new potential QIs were suggested by the stakeholders. IQIs with the highest relevance (median score of 9 and agreement score ≥96%) score included: IQI-8 ‘An antibiotic plan should be documented in the medical record at the start of the antibiotic treatment’; IQI-10 ‘The results of bacteriological susceptibilities should be documented in the medical record’; IQI-23 ‘The local guidelines should correspond to the national guidelines but should be adapted based on local resistance patterns’; IQI-26 ‘An antibiotic stewardship programme should be in place at the healthcare facility’; and IQI-47 ‘Allergy status should be taken into account when antibiotics are prescribed’. An overview of the results of the first online questionnaire is shown in Table 1.

Table 1.

Results of the consensus procedure on inpatient QIs for antibiotic use

QIs by theme	Type	First questionnaire			Consensus Meeting	Second questionnaire				Final selection
		median	fraction in higher tertile (%)	conclusion		agreement score (%)	conclusion
Access-Availability
1. Antibiotics from the antibiotic formulary should not be out of stock at the healthcare facility.	structure	9	84	selected	ND	86	selected			IQI-1
2. Prescribed antibiotics should actually be administered to the patients.	process	9	84	selected	ND	100	selected			IQI-2
Antibacterial Activity
3. The prescribed antibiotic should be active against all the likely causative pathogens.	process	9	80	selected	ND	80	selected			IQI-3
4. Antibiotic empirical therapy should be considered appropriate if the bacteria identified are susceptible to at least one of the antibiotics administered.	process	7.5	52	rejected
Antibacterial Spectrum
5. The microbiological laboratory should report individual selective susceptibility (or antibiogramsa) adapted to local guidelines.	structure	9	68	discussion	rephrased	95	selected			IQI-4
6. Broad-spectrum empirical antibiotic therapy should be changed to pathogen-directed therapy as soon as culture results become available.	process	9	84	selected	ND	96	selected			IQI-5
7. The choice of antibiotic treatment should be reviewed and modified based on clinical response.	process	9	88	selected	ND	96	selected			IQI-6
8. Antibiotics should be continued in the ICU until assessed within 48 h (before considering de-escalation).	process	7	44	rejected
9. Antibiotics for empirical therapy should be reviewed after the third day of treatment or when microbiological results become available.	process	newly suggested QI			rephrased	100	selected			IQI-7
Documentation
10. An antibiotic plan should be documented in the medical record at the start of the antibiotic treatment. (Antibiotic plan includes: indication, name, doses, duration, route, and interval of administration.)	process	9	96	selected	ND	100	selected			IQI-8
11. Clinical and laboratory sepsis parameters should be documented in the medical records when prescribing antibiotics.	process	9	80	selected	ND	86	selected			IQI-9
12. The results of bacteriological susceptibilities should be documented in the medical records.	process	9	96	selected	ND	95	selected			IQI-10
Dosing, PK/PD, Interval
13. Dosing and dosing interval of antibiotics should be prescribed according to guidelines.	process	8	80	selected	ND	96	selected			IQI-11
14. Dosing and dosing interval of renally eliminated antibiotics should be adapted to the patient’s renal function.	process	9	92	selected	ND	100	selected			IQI-12
15. Dosing of antibiotics should be adapted to the patient’s BMI.	process	8	64	discussion	rejected
16. Dosing of antibiotics should be adapted to the patient’s age.	process	7	52	rejected
17. The dosage regimen of antibiotics with an increased risk of toxicity (such as vancomycin or gentamicin) should be managed according to guidelines.	process	9	92	selected	ND	86	selected			IQI-13
Duration
18. Duration of antibiotic therapy should be compliant with guidelines.	process	8	80	selected	ND	86	selected			IQI-14
19. Antibiotic therapy should be discontinued based on the lack of clinical evidence of infection.	process	9	92	selected	ND	100	selected			IQI-15
20. Antibiotic therapy should be discontinued based on the lack of microbiological evidence of infection.	process	5	28	rejected
21. Antibiotic therapy should be discontinued on completion of the documented antibiotic course.	process	8	72	selected	ND	95	selected			IQI-16
22. Stopping antibiotic therapy should always be considered after three consecutive days of defervescence.	process	5	28	rejected
Education
23. Educational sessions about local practice guidelines should be organized for antibiotic stewardship teams and medical staff and should have a predetermined attendance target.	structure	8	68	discussion	rephrased	90	selected			IQI-17
Evidence-based Guidelines
24. Antibiotics should be prescribed according to local practice guidelines.	process	8	76	selected	ND	100	selected			IQI-18
25. Antibiotics should be prescribed according to national practice guidelines.	process	8	80	selected	ND	68	rejected
26. Antibiotics should be prescribed according to national guidelines when no local guidelines are available.	process	8	84	selected	ND	100	selected			IQI-19
27. Antibiotic prescriptions that deviate from guidelines should be justified.	process	9	76	selected	ND	96	selected			IQI-20
28. A local antibiotic guideline should be present at the healthcare facility.	structure	9	92	selected	ND	96	selected			IQI-21
29. An evaluation of whether an update should be considered for the local antibiotic guideline should be done every 3 yearsonce a year.	structure	8	68	discussion	rephrased	96	selected			IQI-22
30. The local guidelines should correspond to the national guideline but should be adapted based on local resistance patterns.	structure	9	100	selected	ND	96	selected			IQI-23
Expertise and Resources
31. An antibiotic formulary should be available and updated continuously at the healthcare facility.	structure	9	76	selected	ND	96	selected			IQI-24
32. An approval system should be in place for prescriptions of restricted antibiotics at the healthcare facility.	structure	9	84	selected	ND	96	selected			IQI-25
33. A computerized decision support system based on local guidelines should be available at the healthcare facility.	structure	7	60	rejected
34. An antibiotic stewardship programme (antibiotic prescribing control programme and/or antibiotic prescribing policy) should be in place at the healthcare facility.	structure	9	100	selected	ND	100	selected			IQI-26
35. Antibiotic prescribing should be compliant with recommendations from infectious disease and/or microbiology specialist(s).	process	8	80	selected	ND	76	selected			IQI-27
36. Audits of antibiotic use by the antibiotic stewardship team should be performed regularly at the healthcare facility.	structure	9	92	selected	ND	96	selected			IQI-28
37. A multidisciplinary antibiotic stewardship team appointed by the healthcare facility management should have meetings at least twice a year and make a report with objectives and selected performance indicators.	structure	8	80	selected	ND	91	selected			IQI-29
38. Patients with S. aureus bacteraemia should be seen by an infection disease specialista physician trained in infectious diseases.	process	newly suggested QI			rephrased	68	rejected
Indication
39. Antibiotics should be used only for strict indications.	process	7.5	64	rejected
40. A clinical scoring system should be used to determine if there is an indication for antibiotic use.	process	5	28	rejected
Interactions
41. Identified interactions between antibiotic regimen and concurrent medications should be documented in the medical record with a recommended management plan to deal with the interaction.	process	9	88	selected	ND	91	selected			IQI-30
Microbiological Diagnostics
42. Two sets of blood cultures should be taken before antibiotic administration when bacteraemia is suspected.	process	9	76	selected	ND	95	selected			IQI-31
43. Specimens for culture from suspected sites of infection should be collected before antibiotic administration.	process	9	88	selected	ND	95	selected			IQI-32
44. Microbiological investigations should be performed according to guidelines.	process	9	84	selected	ND	96	selected			IQI-33
Patient Outcome
45. Clinical outcomes of patients receiving antibiotics should be monitored at the healthcare facility.	outcome	8	80	selected	ND	86	selected			IQI-34
46. Bacterial outcomes of patients receiving antibiotics should be monitored at the healthcare facility.	outcome	7.5	64	rejected
47. Resistance outcomes of patients receiving antibiotics should be monitored at the healthcare facility.	outcome	8	64	discussion	rejected
48. Rates of nosocomial Clostridium difficile should be monitored at the healthcare facility.	outcome	9	80	selected	ND	100	selected			IQI-35
Prescribing
49. Antibiotics should be prescribed by generic name.	process	7	52	rejected
Route
50. The route of administration of antibiotics should be compliant with guidelines.	process	9	84	selected	ND	96	selected			IQI-36
51. Antibiotic therapy in adult patients with sepsis should be started intravenously.	process	9	76	selected	ND	84	selected			IQI-37
52. Switching from intravenous to oral antibiotic(s) should be performed according to guidelines.	process	9	80	selected	ND	91	selected			IQI-38
53. Switching from intravenous to oral antibiotic(s) should be done within 48–72 h based on the clinical condition and when oral treatment is adequate.	process	8	72	selected	ND	76	selected			IQI-39
Resistance Surveillance
54. Surveillance of antibiotic use and resistance should be performed at least once per year at the healthcare facility.	structure	9	92	selected	ND	100	selected			IQI-40
Surgical Prophylaxis
55. Prophylactic antibiotics should be available in the operating room and pre-operative admission units.	structure	9	68	discussion	rejected
56. Postoperative prophylactic antibiotics should be discontinued within 24 h after wound closure.	process	8	56	discussion	rejected
57. Prophylactic antibiotics should be added to a preoperative checklist.	structure	9	76	selected	ND	100			selected	IQI-41
58. A preoperative pause (time-out) should be implemented before administering antibiotic prophylaxis.	process	7	32	rejected
59. Prophylactic antibiotics should be redosed intra-operatively for surgeries longer than 3–4 h or significant blood loss (≥1500 mL).	process	8.5	60	discussion	rejected
Therapeutic Drug Monitoring (TDM)
60. TDM should be performed for antibiotics with a narrow therapeutic spectrum and an increased risk of toxicity (such as gentamicin and vancomycin) according to guidelines.	process	9	88	selected	ND	86		selected		IQI-42
61. At least 75% of TDM levels of antibiotics should be within the desired reference range.	process	8	44	discussion	rejected
62. If antibiotic TDM levels are not in the reference range, doses should be adjusted appropriately after the results become available.	process	9	76	selected	ND	91		selected		IQI-43
63. TDM levels of antibiotics should be documented in the medical records.	process	9	88	selected	ND	100		selected		IQI-44
Timing
64. Timeliness of administration of antibiotic therapy and prophylaxis should be compliant with guidelines.	process	9	92	selected	ND	100		selected		IQI-45
65. Antibiotic therapy should be started as soon as possible upon admission to the healthcare facility.	process	5.5	32	rejected
Toxicity
66. Duration of administration of intravenous antibiotics should be compliant with guidelines.	process	8	76	selected	ND	86		selected		IQI-46
67. Allergy status should be taken into account when antibiotics are prescribed.	process	9	96	selected	ND	100		selected		IQI-47
68. Allergy status (including nature and severity) of the patient should be documented in the medical records when antibiotics are prescribed.	process	9	92	selected	ND	100		selected		IQI-48
69. Patients with a history of anaphylaxis after penicillin therapy should be prescribed an alternative drug class.	process	9	88	selected	ND	100		selected		IQI-49
70. Medical staff should be educated regarding cross-allergy with cephalosporins in patients with penicillin allergy.	process	9	92	selected	ND	95		selected		IQI-50
71. Antibiotics should be changed in case of adverse reaction.	process	8.5	68	discussion	rejected
72. Contraindications should be taken into account when prescribing antibiotics.	process	9	88	selected	ND	96		selected		IQI-51

ND, not discussed; IQI, inpatient quality indicator; PK/PD, pharmacokinetic/pharmacodynamic.

The references are shown in Table S3. The strikethrough and underlined text show the rephrasing process of the quality indicators.

^aA selective susceptibility report (or antibiogram) is a report of a selection of antibiotic susceptibilities, based on bacteriological activity, broadness of spectrum or toxicity.

Step 3 – Consensus meeting

Fourteen stakeholders discussed the QIs for which there was disagreement as well as the newly suggested QIs. The stakeholders represented all groups: medical community (n = 4); public health and patients (n = 1); antibiotic R&D (n = 6); and payers, policy makers, governments and regulators (n = 3). Of the 10 QIs labelled for discussion, 3 were rephrased and 7 were rejected. The two newly suggested indicators were rephrased. The details of the consensus procedure including the selection and rejection as well as the rephrasing of the QIs are shown in Figure 1 and Table 1.

Step 4 – Second questionnaire

Twenty-two stakeholders answered the second questionnaire (response rate 88%). Based on agreement scores, out of the 53 potential QIs, 51 were selected and 2 were rejected (Figure 1). The final selected 51 IQIs included 36 (71%) process, 13 (25%) structure and 2 (4%) outcome indicators and have received a final numbering in Table 1. The 51 IQIs were classified into 19 themes of responsible antibiotic use (Table 1).

Discussion

This international and multidisciplinary consensus procedure led to the development of 51 generic QIs for antibiotic use in the inpatient setting. These QIs are intended to be universally applicable, regardless of infectious disease type, geographical or socioeconomic setting. Moreover, the broad background range of the stakeholders that selected them is expected to lead to widespread support for the QIs. Most of the IQIs were classified as process, about one-third as structure and only two as outcome indicators according to the Donabedian model. Altogether, the QIs covered a wide range of 19 different themes of responsible inpatient antibiotic use, of which the majority overlap with the elements of the definition of responsible use.¹⁵ During the consensus procedure, the two QIs relating to the theme ‘Indication’ were rejected by the stakeholder panel. This is surprising as ‘Using antibiotics for the correct indication’ is one of the main recommendations for prudent use by ECDC.²⁰ This also contrasts with the selection of the element ‘Indication’, phrased as ‘Using antibiotics only to prevent or cure infections for which antibiotic treatment provides a proven benefit’ for the global definition of responsible use by a similar international multidisciplinary panel.¹⁵ However, the stakeholders of the present study might have considered the requirement of a correct clinical indication for the use of antibiotics being covered by six QIs in the theme ‘Evidence-based guidelines’ (IQI-18 to IQI-22), e.g. IQI-18 ‘Antibiotics should be prescribed according to local practice guidelines’. In addition, several other QIs relate to the use of guidelines (e.g. IQI-11, IQI-14, IQI-33, IQI-36). Indeed, compliance with local hospital guidelines is a universal measure of healthcare quality.

Other researchers have developed generic (i.e. non-disease-specific) inpatient process QIs for antibiotic use. The QIs developed in this study overlap with all 11 QIs identified by van den Bosch et al.²¹ using a similar methodology with a European expert panel in which all the main medical specialties involved in antibiotic treatment were represented. However, the panel did not include (inter)national professional clinical societies or stakeholders from outside the medical community. In another initiative, the Transatlantic Taskforce on Antimicrobial Resistance (TATFAR), experts from the EU and the USA identified 17 core indicators and 16 optional indicators for inpatient antibiotic use addressing the organization of ASPs.²² While the van den Bosch study aimed at developing a concise set of non-disease-specific QIs, the list of TATFAR indicators resulted from comparisons between antibiotic stewardship programmes in EU and US hospitals. All these QIs should be seen as complementary output of international and cross-disciplinary efforts to improve antibiotic use.

A strength of this work is the use of a RAND-modified Delphi method combining both concepts from the international literature and international stakeholder opinions. This standardized method has been used previously for the identification of QIs of antibiotic use in the inpatient setting.^18,23,24 To our knowledge this is the first time that the perspectives of such a broad range of stakeholders involved with antibiotics were accounted for in the development of the QIs. The four stakeholder groups represented all parties involved in antibiotics from molecule to prescribed drug. The diversity in background and geography (15 countries across four continents) of the stakeholders emphasizes the potential for global acceptance of our consensus. The use of a broad definition for the inpatient setting should ensure that QIs are relevant for a large range of healthcare facilities, including e.g. acute care hospitals and long-term care facilities.

Methodological limitations of this study include the use of a single literature database (MEDLINE) for the systematic review. Both the complementary web site search and the opportunity given to the stakeholders to propose new QIs should have ensured that no relevant QI was missed. Another limitation is the lack of grading of the evidence for the QIs by the authors. Instead, stakeholders were provided with the original references for each of the QIs, offering the opportunity to assess the scientific evidence by themselves. Also, there was low participation of stakeholders of the public health and patients group compared with the three other groups. Explanations could include the perceived lack of knowledge for assessing QIs for the inpatient setting as public health typically focuses on community care. The relevance scores for the QIs were not analysed for each individual stakeholder group separately as the aim of the study was to identify QIs taking into account different perspectives. Therefore, the data analysis was performed across the four stakeholder groups. Finally, even though the scope of the study was global, an English language restriction was applied to the literature and site searches. We may have missed important papers in other languages. However, English is nowadays considered a major global vehicle in the scientific literature that is directed to a global audience.

Aspects relating to the applicability of the QIs in clinical practice were not assessed in the consensus procedure. However, this may rather have contributed to the simplicity of the QIs, required for a global scope, including coverage of low-income settings. Certainly, before these indicators can be used in daily medical practice, their applicability should be tested in the prevailing healthcare setting, including clinimetric properties relating to feasibility, validity and reliability.^23–25

How can the generic IQIs reported in this study be used? So far, the QIs have contributed to the ‘Proposals for EU guidelines for the Prudent Use of Antimicrobials in Human Medicine’ written by the ECDC.²⁰ Furthermore, educational materials derived from this study, e.g. a podcast of a train-the-trainer event organized in collaboration with the BSAC with delegates from 19 EU countries, are made available for use in ASPs and healthcare curricula.²⁶ In the future, the QIs, being one of the components of the DRIVE-AB standard of responsible antibiotic use, are expected to further guide (inter)national health policies.

In conclusion, the 51 generic QIs cover a broad scale of themes for responsible inpatient antibiotic use. They can be used globally to guide the use of both current and newly developed antibiotics in the future.