Antimicrobial stewardship using electronic prescribing systems in hospital settings: a scoping review of interventions and outcome measures

Abstract

Objectives

To identify interventions implemented in hospital electronic prescribing systems and the outcome measures used to monitor their impact.

Methods

We systematically searched CINAHL, EMBASE, Google Scholar and Medline using keywords in three strands: (i) population: hospital inpatient or emergency department; (ii) intervention: electronic prescribing functionality; and (iii) outcome: antimicrobial stewardship. The interventions were grouped into six themes: alerts, order sets, restriction of access, mandated documentation, embedded guidelines and automatic prescription stop. The outcome measures were organized into those that measure the quality or quantity of prescribing or clinical decision support (CDS) activity. The impact of each intervention reported was grouped into a positive, negative or no change.

Results

A total of 28 studies were eligible for inclusion. There were 28 different interventions grouped into the six themes. Alerts visible to the practitioner in the electronic health record (EHR) were most frequently implemented (n = 11/28). Twenty different outcome measures were identified, divided into quality (n = 13/20) and quantity outcomes (n = 4/20) and CDS activity (n = 3/20). One-third of outcomes reported across the 28 studies showed positive change (34.4%, n = 42/122) and 61.4% (n = 75/122) showed no change.

Conclusions

The most frequently implemented interventions were alerts, the majority of which were to influence behaviour or decision-making of the practitioner within the EHR. Quality outcomes were most frequently selected by researchers. The review supports previous research that larger well-designed randomized studies are needed to investigate the impact of interventions on AMS and outcome measures to be standardized.

Introduction

Antimicrobial resistance (AMR) is the innate, developed or acquired ability of a microorganism to survive in the presence of an antimicrobial agent. The excessive or inappropriate use of antimicrobials is widely recognized as a significant contributing factor to the development of AMR.¹ Antimicrobial stewardship (AMS) is an organizational approach to encourage optimal and evidence-based use of antimicrobials, and to monitor the outcomes of this on the development of resistance and patient outcomes.² AMS interventions are frequently multifaceted and encompass all elements that optimize the use of antimicrobials, including education, diagnostics, guidelines and prescribing interventions that are implemented during a patient’s contact with healthcare services. Surveillance of outcome measures is essential to the process, to monitor for positive or negative change, and to inform continuous improvement.

As hospitals move increasingly towards electronic health records (EHRs), many AMS interventions can be woven into the electronic processes. A specific area of interest is the rules that guide clinical processes (clinical decision support, henceforth CDS) within electronic prescribing (eP) systems that help practitioners to follow certain pathways when prescribing and monitoring antimicrobials during a patient’s inpatient journey. In 2017, Cresswell et al.³ conducted a review to identify approaches to AMS interventions in hospitals that use eP. The authors found variability between studies in the methodologies adopted and outcomes reported, and that the findings were not easily generalizable or transferable to other hospitals or settings. The authors concluded that future studies needed to be more rigorously developed, implemented, assessed and reported. A year later, DRIVE-AB—a multifaceted EU-wide initiative—published a systematic literature review to develop and propose a set of outcome measures as global standards for AMS interventions.⁴ The team identified 63 validated measures relevant to the inpatient setting: 51 quality indicators designed to measure the extent to which antibiotics are prescribed correctly or appropriately for a patient and indication; and 12 quantity metrics designed to measure the number or cost of antibiotics used.⁵

This scoping review has been conducted to update the research undertaken by Cresswell et al. (2016),³ searching the literature from August 2015–August 2020, starting when the previous published scoping review search ended. The aim was to identify AMS interventions in the eP setting, and to expand on this by identifying the outcome measures, the risk of bias of the studies and whether the studies reported a positive, negative or no change.

Materials and methods

A search of the literature was conducted from August 2015–August 2020, starting when the previous published review research ended the search, against defined inclusion and exclusion criteria (Table 1).

Table 1.

Inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria
Studies were included if they were conducted in the hospital setting, including the emergency departments treating adults or children	Studies were excluded if interventions focused solely in outpatients, dental or community settings, or they were systematic reviews
The following study designs were eligible for inclusion: Experimental studies (e.g. randomized controlled trials, controlled clinical trials, controlled before-and-after studies, interrupted time series) Epidemiological studies (e.g. descriptive and analytical studies) Qualitative studies (e.g. ethnographic, interviews, focus groups)	Studies were excluded if they fell outside the scope of interest, for example if they: Did not relate to both eP functionality and AMS Did not evaluate the effectiveness of an AMS intervention Evaluated technology that is not commonly associated with core electronic prescribing functionality
	Due to different contextual circumstances, the following studies were excluded: Studies that focused on the implementation of technology in community settings Studies that focused on the implementation of eHealth in developing countries

Systematic searches were conducted in CINAHL, EMBASE, Google Scholar, Medline and the Cochrane Central Register of Controlled Trials. The reference lists of included studies were hand searched for additional papers that had not been identified by electronic methods. The search strategy was comprised of keywords in three strands (Table S1, available as Supplementary data at JAC-AMR Online): (i) population: hospital inpatient or emergency department; (ii) intervention: eP functionality; and (iii) outcome: antimicrobial stewardship. Terms within the strands were combined using the Boolean term ‘OR’ after which the strands were combined using the Boolean term ‘AND’. Titles and abstracts of search outputs were independently reviewed for inclusion by two of the three authors (J.A.J., K.C. and S.K.P.) and discrepancies were resolved by arbitration to obtain a list for full text consideration. Two of the three authors reviewed each of the full-text citations with brief descriptions of reasons for exclusion documented. All disagreements for inclusion were discussed by the three authors until agreement was reached. Studies were restricted to those in the English language. An extraction tool was developed to facilitate the collection of data from each of the included studies, including the study design, setting, intervention, outcome measures, how outcome measures were calculated, the impact of the intervention (positive, negative or no change) and conclusions.

Following data extraction, the interventions implemented were grouped into themes. The outcome measures of each study were reviewed and thematically organized into those that measured: (i) the quality of prescribing covering the extent to which antibiotics are prescribed correctly or appropriately for a patient and indication; (ii) the quantity of prescribing, designed to measure the number or cost of antibiotics used; or (iii) CDS activity. The DRIVE-AB quality outcomes and quantity metrics were used to inform the analysis.^4,5

The studies were assessed for risk of bias by three independent researchers (J.A.J., S.K.P. and K.C.). The ROBINS-I tool was used to assess non-randomized intervention studies⁶ and the RoB2 tool for randomized crossover trials.⁷ Each score was checked by a second researcher to ensure consistency in the application of the tool and agreement in the assessment of quality determined.

Results

Our scoping review of published literature yielded 28 included papers of which 27 were observational studies and 1 a randomized controlled study (Figure 1, Table 2).⁸ The majority were conducted in the USA (n = 16/28), and then Australia (n = 4/28), Canada (n = 3/28) and one in each of Brazil, the Netherlands, New Zealand, South Korea and Spain. The 28 studies^9–36 reported combined data on more than 40 000 participants, including over 2000 children, and from study populations intentionally diverse for age, diagnosis and treatment. Most studies included a broad cross-section of hospital inpatients; however two studies were undertaken in the emergency department, four in paediatrics, one in maternity and one in an ICU.

Figure 1.

Literature search flow chart.⁸

Table 2.

Characteristics of included studies

Year, first author	Country of origin	Design	Intervention(s)	Quality outcome measure(s)	Quantity outcome measure(s)	CDS measure(s)
2019 Mostaghim et al.9	Australia	Retrospective clinical audit of medical records	Restriction of access Mandated documentation	Adherence to clinical guidelines Documentation	LOS
2019 Canovas-Segura et al.10	Spain	Retrospective notes review	Alert in eP system	Adherence to clinical guidelines		CDS alert frequency CDS alert override
2017 Bond et al.11	Australia	Interrupted time series study	Guidelines embedded into eP system	Mortality rate Resistance rates/acquisition of MDR organism Development of HCAI	LOS Antibiotic consumption
2018 Rosa et al.12	USA	Interrupted time series analysis	Pre-built order set	Mortality rate Adherence to clinical guidelines Clinical failure rate Expert referral rate	DOT Diagnostics
2017 Lambl et al.13	USA	Longitudinal outcome study over 3 years	Alert in eP system	Resistance rates/acquisition of MDR organism Development of HCAI Expert referral rate	DOT	Use of CDS order sets
2017 Metcalfe et al.14	Australia	Retrospective comparative study	Restriction of access	Adherence to clinical guidelines
2019 O’Sullivan et al.15	USA	Quasi-experimental interrupted time-series analysis	Alert in eP system		LOS DOT Antibiotic consumption
2019 Colletti et al.16	USA	Before-and-after study	Alert in eP system	Adherence to clinical guidelines		Use of CDS order set
2017 Hincker et al.17	USA	Historical cohort before-and-after study	Pre-built order set Alert in eP system	Adherence to clinical guidelines
2020 Bowers et al.18	New Zealand	Before-and-after study	Restriction of access Alert in eP system Mandated documentation	Adherence to guidelines Documentation
2020 Mouwen et al.19	The Netherlands	Historically controlled prospective intervention design	Restriction of access	Mortality rate Clinical failure rate Readmission rate
2016 Burgess et al.20	USA	Pre-intervention and post-intervention study	Pre-built order set Alert in eP system Guidelines embedded into eP system	Adherence to clinical guidelines Expert referral rate	Diagnostics
2016 Kandel et al.21	Canada	Prospective	Alert in eP system	Resistance rates/acquisition of MDR organism Development of HCAI	DOT	CDS alert frequency CDS alert override
2020 Ridgway et al.22	USA	Randomized controlled trial with crossover design	Alert in eP system	Mortality rate Readmission rate Resistance rates/acquisition of MDR organism Development of HCAI	LOS Antibiotic consumption
2020 Brotherton et al.23	USA	Retrospective quasi-experimental study	Pre-built order set Automatic prescription stop	Mortality rate Clinical failure rate Readmission rate Time to active therapy	LOS Diagnostics
2019 Wolfe et al.24	USA	Retrospective cohort study	Automatic prescription stop	Mortality rate Adherence to clinical guidelines Resistance rates/acquisition of MDR organism Change of route of administration	DOT Antibiotic consumption
2019 Downes et al.25	USA	Before-and-after study	Automatic prescription stop		DOT
2015 Kitchlu et al.26	Canada	Before-and-after study, chart review	Pre-built order set Guidelines embedded into eP system	Adherence to clinical guidelines Expert referral rate	Diagnostics
2018 Dassner et al.27	USA	Before-and-after retrospective, observational study	Restriction of access	Adherence to clinical guidelines Clinical failure rate Readmission rate Time to administration
2019 Johnk et al.28	USA	Retrospective comparative study	Restriction of access	Mortality rate Clinical failure rate Development of HCAI
2017 Baysari et al.29	Australia	A controlled before-and-after study and qualitative interviews	Pre-built order set Mandated documentation	Adherence to clinical guidelines Documentation
2016 Huh et al.30	Republic of Korea	Interrupted time series with segmented regression analysis	Pre-built order set	Mortality rate; resistance rates/acquisition of MDR organism LOS DOT Antibiotic consumption
2016 Bonella et al.31	Brazil	Before-and-after intervention design	Automatic prescription stop	Mortality rate Adherence to clinical guidelines Readmission rate Change of route of administration	LOS DOT
2018 Fargo et al.32	USA	Retrospective chart review	Pre-built order set	Clinical failure rate	LOS	Use of CDS order set
2015 Revolinski33	USA	Quasi-experimental study	Pre-built order set Alert in eP system	Adherence to clinical guidelines		Use of CDS order set
2019 Kan et al.34	Canada	Quasi-experimental study	Alert in eP system	Adherence to clinical guidelines Change of route of administration
2019 Nichols et al.35	USA	Retrospective cohort study	Pre-built order set	Adherence to clinical guidelines
2017 Sauberan et al.36	USA	Retrospective chart and EHR review	Pre-built order set	Clinical failure rate LOS DOT

Interventions

Six out of the 28 studies used CDS intervention tools developed by external or commercial agencies to optimize stewardship^9–14 whilst the remaining 22 used those developed in house. The interventions were grouped into six themes: alerts, order sets, restriction of access, mandated documentation, embedded guidelines and automatic prescription stop (Table 3). Eight studies reported outcomes following implementation of one intervention, 5 studies reported on outcomes of two interventions, 11 on three interventions and 4 studies reported the outcomes of four interventions (Table 2). Most interventions in the studies (n = 22/28) were developed by researchers in house and designed for a specific hospital within the functionality of the eP system.

Table 3.

Antimicrobial intervention themes

Intervention	No. of studies	No. outcomes reported	Positive change, n (%)	No change, n (%)	Negative change, n (%)
Pre-built order set
Simple	8	17	2 (12)	14 (82)	1 (6)
Complex	6	20	6 (30)	13 (65)	1 (5)
Alerts
Alert visualized in EHR (medical)	8	30	12 (40)	18 (60)	0
Alert visualized in EHR (surgical)	3	5	5 (100)	0	0
External notification	4	10	5 (50)	5 (50)	0
Others
Restriction of access	7	16	6 (38)	9 (56)	1 (6)
Mandated documentation	4	10	1 (10)	9 (90)	0
Guidelines/pathway in eP system	2	8	3 (38)	4 (50)	1 (12)
Automatic prescription stop	1	6	2 (33)	3 (50)	1 (17)

Alert interventions comprised automated messages to practitioners to provide information, warning or notification that action is required. Two types of alerts were identified in the studies: interruptive ‘pop-ups’ visualized in a patient’s EHR (n = 11) or an electronic notification generated in the background to inform a pharmacist or specialist team (n = 4). Eleven studies reported outcomes following introduction of EHR pop-ups, three in the peri-operative setting^14,16,17 and eight in the medical setting.^10,18–24 Peri-operative alerts focused on reminding theatre staff to administer prophylactic antimicrobials at the correct dose and time, whilst in the medical setting alerts were more varied, used to inform healthcare practitioners of the presence of a care bundle or guideline or the risks associated with a particular therapy, or to remind prescribers to complete all elements of the gold standard antimicrobial authorization to administer e.g. antimicrobial name, dose, frequency, route, indication and duration.

An order set, in its simplest form, includes all the essential elements for an antimicrobial prescription (or authorization) to administer therapy. These were implemented in 8 of the 28 studies. More complex order sets extending beyond the ‘authorization to administer’ were found to include a range of requests required for improved diagnosis, management or follow-up—for example, clinical severity scoring, dose adjustments for special populations, laboratory test requests and monitoring. These more complex order sets were adopted in five studies.^{12,13,20,23,25,26} Interventions to restrict access to ‘non-formulary’ or reserve line antimicrobials were reported in five studies.^{9,11,13,14,27} Methods to permit access were granted in one of two ways: electronically where the prescriber inputted details such as the indication or justification to gain approval,^9,11,13 or secondly by getting another healthcare professional to countersign the prescription.^14,27

The outcomes of mandated documentation of indication and/or duration were reported in three studies.^18,28,29 Whilst mandating documentation reduced the incidence of incomplete fields in the eP system, it did not increase the rate of accurate documentation. Two studies investigated the impact of embedding the infection management or antimicrobial prescribing guidelines into eP.^11,20 In both studies, guidelines were accessible to practitioners, but prescribers were not forced to adhere to them. Finally, one study reported the impact of introducing an automatic stop to antimicrobials if a specific criteria set was fulfilled.²²

Overall, a third of outcomes reported across the 28 studies showed positive change (34.4%, n = 42/122) and 61.4% (n = 75/122) showed no change (Figure 2).

Figure 2.

Impact of intervention on outcome measures.

Outcome measures

There was a total of 122 outcome measures reported across the 28 studies. These were divided into 19 themes: quality measures (n = 12/20), quantity measures (n = 4/20)⁴ and CDS-aligned measures (n = 3/20) (Table 3).

Quality outcomes

Out of the 12 quality indicators used across the studies, adherence to clinical guidelines was most frequently used (n = 18/28) (Table 3). Adherence assessed elements such as prescription of the recommended drug, dose, frequency, route, timing, monitoring and duration. Adherence to clinical guidelines was assessed by the research team and could only be determined if information in the patient record was accurate.

Nine out of the 28 studies used mortality as an outcome measure. Mortality was found to be measured from a fixed time period from the intervention (n = 3/9)—one at 14 days (n = 1/9)¹ and two at 30 days (n = 2/9),^21,22 whilst five studies adopted the more flexible ‘in-hospital mortality’ determined as patient death within that episode of inpatient care.^{11,12,24,28,30} One additional paper did not provide information regarding how mortality rate was determined.³ Readmission rates were used in seven studies. The length of time to readmission varied between studies, ranging from 14 to 90 days of the intervention. Six papers reported readmission rates due to all causes^{9,20,22,25,28,31} whilst one reported those only related directly to the infection.¹⁴

Five studies report the rate of healthcare acquired Clostridioides difficile infection (Table 3), however a different method was used in each case. Denominators were either per 100, 1000 or 10 000 occupied bed days or patient days. One study reported acquisition rates of MDR Gram-negative organisms within 180 days.²² Clinical failure rate was used in four studies, but different methods were used to qualify clinical failure, with two of the four studies describing the measure as ‘the need to resume intravenous therapy’ (having de-escalated to oral therapy). One paper described failure as the ‘requirement to broaden therapy from initial choice’²⁰ and one used the number of monthly episodes of clinical failure per 1000 at-risk patient days.¹² For measures of mortality, readmission rates, acquisition of another infection and clinical failure, there was considerable divergence in the methods used to calculate these, whilst for those reported as rates there is further variation in the denominator used in the included papers.

Appropriate and accurate documentation regarding an antimicrobial’s indication, duration and stop/review date was investigated in four studies (Table 3). Time to administration of antimicrobial was used as a measure in studies focusing on peri-operative prophylaxis and sepsis. Three studies using this measure reported time from initial prescription to administration of the first dose of antibiotic^16,32 whilst two reported the time of peri-operative administration of prophylactic antimicrobials.^15,16

Appropriateness for IV-to-oral switch (IVOS) was reported in three studies (n = 3/28) (Table 3), with consideration of IVOS in the de-escalation process of antimicrobials reported in all three. Two out of 28 studies investigated the impact of their intervention on resistance rates (Table 3), with one reporting ESBL and carbapenem-resistant Enterobacterales rates²⁸ and one monitoring the resistance rates³³ of ‘major pathogens’.

Two outcome measures were used in one study each: time to active therapy, described as time from index culture to therapy with at least one active antibiotic,²⁵ and disposition,³² the setting to which the patient is discharged following admission. Finally, frequency of referral for expert review was reported in three studies before and after the intervention (Table 3). In each of these, referral rates were reported with no further description to indicate if the intervention had an impact on the appropriateness of referrals.

Over the 28 studies, quality outcomes were assessed 51 times. A large proportion showed a positive change (n = 21/51, 41.2%), and just over half (n = 28/51, 54.0%) showed no change (Figure 3).

Figure 3.

Impact of intervention on quality measures.

Quantity measures

A total of 7/28 studies adopted at least two quantity outcome measures (Table 3). The most reported quantity measures used in the studies were length of stay (LOS) (n = 12/28), days of antimicrobial therapy (DOT) (n = 9/28) and antimicrobial consumption (n = 4/28) (Table 3). There was some consistency in the methodology used to measure LOS with 10/12 papers reporting total duration of inpatient hospital stay and the remaining 2 determining ‘infection-related length of stay’.^11,25 Divergence was observed in the reporting of DOT, with three studies reporting total DOT and seven reporting this outcome as a rate using one of four denominators: 100 patient days,²¹ 1000 patient days,^13,25,28,31 1000 at-risk patient days¹² and 1000 occupied bed days.¹¹ Three studies investigated the antibiotic consumption reported in all cases as antibiotic expenditure. Finally, three studies reported on the quantity of diagnostics utilized, but did not report the appropriateness of their use (Table 3). Diagnostics measured include laboratory assays, blood cultures and imaging.

Over the 28 studies, 25 quantity outcomes were assessed. The majority of these showed no change (n = 18/25, 72.0%), with five showing a positive change (n = 5/25, 20.0%) (Figure 4).

Figure 4.

Impact on quantity measures.

CDS outcomes

Outcomes relating to CDS were reported in four studies (Table 3). Two studies presented data on workarounds employed by prescribers whose access was limited. One study presented data on the use of an order set in the management of C. difficile infection.³³ In this study, uptake of use of the order set was poor with only 1.2% (n = 2) of prescriptions generated this way. The addition of an alert informing prescribers of the presence of the order set significantly increased uptake to 30.8% (n = 45). Finally, one study reported how to achieve optimal response to alerts with minimizing alarm fatigue, with alerts delayed for 48–72 h so errors or mistakes could be picked up by the clinical team thereby reducing the overall number of alerts.¹⁰

Twelve studies reported the impact of alerts on adherence to clinical guidelines (Table 3). All studies implementing external notification,^12,13,34 or EHR visualized peri-operative alerts^15–17 demonstrated a significant improvement (Figure 2). Six studies introduced HER-visualized alerts in the medical setting with three showing a statistically significant improvement.^20,21,24 The three studies that showed no change to guideline adherence all required the prescriber to review and/or modify the antimicrobial therapy^10,18,23 in response to the alert, whilst the three studies that showed positive change to guideline adherence were interventions that required no change to the therapy prescribed.^20,21,24

Of the four studies introducing simple order set interventions, only one¹⁷ demonstrated a clinical benefit to adherence to clinical guidelines following implementation. Four of the five studies demonstrated improved adherence using a complex order set intervention.^12,13,20,26 Five studies that implemented a system to ‘restrict access to certain antimicrobials’ determined adherence to clinical guidelines as an outcome measure, of which four showed a significant improvement.^11,13,14,27 Each of these four studies required prior approval, either electronically or by a second professional before treatment could continue. One study reported a non-significant improvement but was the only study that permitted up to 24 h use of restricted antibiotics whilst approval was sought.⁹

None of the studies that implemented mandated documentation demonstrated an improved adherence to guidelines.^28,29,32 Following intervention implementation, improved rates of documentation were observed in two studies, but neither demonstrated an increase in the appropriateness of therapy.^18,29 An increase in the use of workarounds following implementation of mandated documentation was also observed in one study.²⁹

Assessment of risk of bias

Risk of bias was assessed in the 27 non-randomized studies using ROBINS-I.⁶ Risk of bias was judged as critical in 14 cases, serious in 6, moderate in 4 and low in 3 papers (Table S2). This outcome was largely influenced by the number of studies assessed with a critical or serious potential for bias due to confounding factors (n = 17/27) and moderate bias in the selection of reported results (n = 17/27). Furthermore, the bias in measurement of outcomes was reported as moderate or serious in 20/27 studies. This was largely a result of the subjective assessment of quality outcome measures, such as with case note review to determine adherence to guidelines The randomized study assessed using RoB2⁷ was judged to have a high risk of bias (Table S3) due to the unblinded nature of the study, the randomization and measurement of the outcome.

Discussion

The aim of this review was to identify AMS interventions implemented in hospitals with eP systems, the outcome measures aligned to monitor the impact of these and the reported outcomes. Six intervention themes were found to be used across 28 studies, and most CDS tools had been developed in house. Consistent with findings from Cresswell et al. (2016),³ the most frequently implemented intervention was alerts in the eP system, the majority of which were to influence behaviour or decision-making of the practitioner as part of their workflow. Alert notifications were either visualized in the patients EHR directly to the practitioner, or external notifications with interventions designed to prompt review by the specialist team followed by individual interaction with the prescriber. Alert interventions were found to have the highest rate of reported positive outcome, with a positive change reported for alerts visualized in the EHR in the surgical setting. Surgical alerts (n = 3), implemented to remind practitioners to administer prescribed antimicrobials within an optimal window for peri-operative prophylaxis, were the only interventions where modification of prescribing behaviour was not the intention. Second and third highest rates for outcome measure positivity were external notifications and pop-up alerts visualized in the EHR in the medical setting. External notification interventions, which triage the patient for specialist review, had a higher percentage of positive outcome measure than EHR visualized (medical) interventions (50% versus 40%) even though both were used in comparable settings and triggered by similar inputs. Differences in outcome positivity may suggest that clinicians are more likely to change prescribing behaviour when they perceive that a named specialist is offering patient-specific advice rather than that offered by CDS, which may be perceived as generic. The involvement of the multidisciplinary team, as well as monitoring of electronic prescriptions to prevent or detect errors or suboptimal treatment, are behaviour change techniques that have been adopted in successful intervention studies relating to prescribing.³⁷ Additionally, the interaction with a specialist can provide education and an opportunity to respond to prescriber concerns and to share the responsibility of the decision-making, which have also been found to influence behaviour.³⁷ A higher rate of positive outcome measures was reported in studies where changes to prescribing or decision-making behaviour were not required (5/5 versus 39/124). This may highlight the lack of published research focusing on integrating behaviour change techniques alongside other interventions when implementing health technologies.³⁷

Despite being widely recommended as a core principle of AMS, appropriate and accurate documentation regarding an antimicrobial indication, duration and stop or review date was only investigated in a few studies. This may suggest systems do not have the functionality for practitioners to add such information, or systems prevent such in-house configuration. This may change as technical design standards are put in place for systems, which prioritize patient safety.³⁸

Most studies reported outcomes from interventions designed to improve antimicrobial prescribing, which was consistent with findings from Cresswell et al (2016).³ To change prescribing behaviour, an intervention is usually required at the point of decision-making. Prescribers working in emergency or admission units will frequently be reviewing and commencing therapy for patients, therefore intervening at the point of prescribing will be effective. Following admission, treatment review is frequently undertaken during ward rounds or team meetings in the presence of multidisciplinary colleagues and senior medical staff. Decisions and outputs from discussions may be delegated to more junior medical team members to implement and, if appropriate, to prescribe.³⁹ For the inpatient setting, interventions activated at the point of prescribing may be too late since junior staff may feel uncertain about deviating from prescribing decisions already made by the senior team.⁴⁰ It is not surprising therefore that many of the included studies report no significant change to practice following implementation of these interventions. This is exemplified by the interventions of mandated documentation and ‘simple order sets’, which are tools used to ensure the appropriate prescription of antimicrobial dose, duration and indication are documented but are focused on the point of prescribing long after the decision-making has happened. Findings from these studies show not only little significant change in outcome measures, but poor uptake of the implemented tools and a high rate of workarounds. The evidence suggests that to make significant positive change to AMS outcomes, interventions may need to be earlier in the decision-making pathway and focused on the senior decision-makers in clinical teams.⁴⁰

Successfully implemented AMS interventions not only support the stewardship agenda but facilitate optimal antimicrobial prescribing, making the process quicker, easier or simpler.⁴¹ Order sets are built to ensure all the requirements of an antimicrobial prescription are present including indication, dose, route and duration. To implement order sets, numerous sets are required for each indication, for example IV therapy, oral therapy using tablet/capsules and liquid formulations, alternative for allergies, modifications for renal or hepatic impairment and adjustments for children. Due to the potential volume of sets in the prescribing catalogue of the eP system, an effective search function is required to promptly find the desired order set by drug name or indication. Difficult to locate sets will affect usability and be seen as a barrier to prescribing, with a consequence of poor uptake of the intervention.⁴² Positive outcomes for simple order set interventions were lower than for complex order sets (12% and 30%, respectively). The rationale for better outcomes may be that complex order sets have ‘extras’ such as appropriately timed blood tests, monitoring and other ancillary medicines, all of which can be perceived as labour saving and therefore preferred by prescriber. In one study, the use of the order set was only found to increase when an alert was added to the system to remind prescribers of its existence.³³ System alerts that prompt the prescriber are useful behaviour change techniques,³⁷ but their optimal impact is also highly dependent on factors such as user-design and engagement.

The appropriate selection of outcome measures is essential for driving the availability of reliable data from which time periods can be compared, and impact measured. The DRIVE-AB review highlighted the disparities that exist between published studies in identifying impact and thus recommended that at least two quantity factors should be reported by researchers to determine change.⁴ In this review, fewer than half of studies (n = 7/28) used two quantity outcome measures. The goal of AMS interventions is to minimize the development and spread of AMR in microorganisms, but only 2 of the 28 studies reported rates of resistance as an outcome measure and neither demonstrated significant change during the study follow-up.^28,30 Resistance has been shown to be rapid to acquire, but slow to lose.⁴³ As a result, surrogate outcome measures are frequently observed to be adopted to demonstrate change within a defined study duration.

Half of the outcome measures demonstrated a significant positive change following implementation of the interventions. However, in some cases, no change may be the desired outcome to demonstrate no harm or no unintended consequences. For example, none of the interventions showed a change in mortality, which is the desired outcome as it demonstrates that patients were caused no harm following implementation of the intervention. Since lack of positivity should not always be considered adversely, caution should be exercised when comparing numbers or percentages of outcome measures that showed positive change since the relevance will be dependent on the outcome measures selected by the studies.

The assessment of quality outcomes can be subjective and relies on the knowledge and skills of the researcher, absence of bias and accessibility and availability of data required to make a judgement. Additionally, these outcomes rely on organizational policies and procedures, for example, up-to-date evidence-based antimicrobial guidelines.⁴⁴ Subjective outcome measures identified in this review include adherence to clinical guidelines, appropriate documentation, clinical failure and IVOS. The methods used to calculate outcome measures across the studies were also found to be inconsistent, particularly for mortality rate, clinical failure rate and days of therapy. This means that such outcomes are not comparable between studies and are only useful when applied locally and the same measure is adopted each time. It is worth noting though that a larger proportion of studies in this review used mortality as an outcome compared with the Cresswell et al. (2016)³ review (32% versus 0.5%). This shows positive steps in the inclusion of quantity outcome measures and assessment of unintended consequences of interventions.

It is important to note that the studies included in this review did not monitor the impact of interventions over a prolonged period, and so it is not clear whether outcome measures were sustained or changed over time. Staff may become reliant on systems to inform their decisions or may become fatigued to the presence of these—both of which can have an impact on practitioner decision-making and long-term behaviour change.^37,45

Quality of the evidence

The quality of the evidence was assessed to be low across the studies, reducing the certainty that the interventions had a positive impact on quality and quantity outcome measures. High risk of bias was largely a result of the potential for confounding factors and the reporting of results. Some confounding factors are to be expected in non-randomized trials conducted in the hospital setting, and therefore more randomized controlled trials are needed to generate higher quality evidence to investigate the impact of AMS interventions. As many quality outcomes studies required subjective researcher review, there was some bias in the measurement of outcomes. This reinforces the need for quantity outcomes to be measured and reported alongside quality outcomes, to reduce the risk of bias in determining the impact of interventions.

Conclusions

Most interventions implemented to improve AMS were configured in house, with the most frequent type being alerts in the eP system. This is consistent with previous findings and may suggest that organizations select interventions based on the technology available to them, what can be feasibly and acceptably implemented with local resource and data that can be used to benchmark against local or national targets. Most studies used more than one outcome measure, with quality outcomes making up the majority of these. Despite recommendations for AMS studies, few adopted two quantity outcome measures to determine change. The methods used to calculate some measures were also found to be inconsistent across studies, meaning that reliable and useful comparisons cannot be made between settings to assess the impact of interventions on AMS. This review supports previous research that larger well-designed randomized studies are needed to investigate the impact of interventions on AMS. It also highlights the need for outcome measures to be standardized, and for more quantity measures to be included by researchers to measure the number or cost of antibiotics being used.

Funding

This study is funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (grant reference number RP-PG-0617-20009).

Transparency declarations

None to declare.

Disclaimer

The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any author accepted manuscript version arising from this submission.

Supplementary data

Tables S1 to S3 are available as Supplementary data at JAC-AMR Online.