Abstract
Background
Computerised decision-support systems (CDSSs) for antibiotic stewardship could help to assist physicians in the appropriate prescribing of antibiotics. However, high-quality evidence for their effect on the quantity and quality of antibiotic use remains scarce. The aim of our study was to assess whether a computerised decision support for antimicrobial stewardship combined with feedback on prescribing indicators can reduce antimicrobial prescriptions for adults admitted to hospital.
Methods
The Computerised Antibiotic Stewardship Study (COMPASS) was a multicentre, cluster-randomised, parallel-group, open-label superiority trial that aimed to assess whether a multimodal computerised antibiotic-stewardship intervention is effective in reducing antibiotic use for adults admitted to hospital. After pairwise matching, 24 wards in three Swiss tertiary-care and secondary-care hospitals were randomised (1:1) to the CDSS intervention or to standard antibiotic stewardship measures using an online random sequence generator. The multimodal intervention consisted of a CDSS providing support for choice, duration, and re-evaluation of antimicrobial therapy, and feedback on antimicrobial prescribing quality. The primary outcome was overall systemic antibiotic use measured in days of therapy per admission, using adjusted-hurdle negative-binomial mixed-effects models. The analysis was done by intention to treat and per protocol. The study was registered with ClinicalTrials.gov (identifier NCT03120975).
Findings
24 clusters (16 at Geneva University Hospitals and eight at Ticino Regional Hospitals) were eligible and randomly assigned to control or intervention between Oct 1, 2018, and Dec 31, 2019. Overall, 4578 (40·2%) of 11 384 admissions received antibiotic therapy in the intervention group and 4142 (42·8%) of 9673 in the control group. The unadjusted overall mean days of therapy per admission was slightly lower in the intervention group than in the control group (3·2 days of therapy per admission, SD 6·2, vs 3·5 days of therapy per admission, SD 6·8; p<0·0001), and was similar among patients receiving antibiotics (7·9 days of therapy per admission, SD 7·6, vs 8·1 days of therapy per admission, SD 8·4; p=0·50). After adjusting for confounders, there was no statistically significant difference between groups for the odds of an admission receiving antibiotics (odds ratio [OR] for intervention vs control 1·12, 95% CI 0·94–1·33). For admissions with antibiotic exposure, days of therapy per admission were also similar (incidence rate ratio 0·98, 95% CI 0·90–1·07). Overall, the CDSS was used at least once in 3466 (75·7%) of 4578 admissions with any antibiotic prescription, but from the first day of antibiotic treatment for only 1602 (58·9%) of 2721 admissions in Geneva. For those for whom the CDSS was not used from the first day, mean time to use of CDSS was 8·9 days. Based on the manual review of 1195 randomly selected charts, transition from intravenous to oral therapy was significantly more frequent in the intervention group after adjusting for confounders (154 [76·6%] of 201 vs 187 [87%] of 215, +10·4%; OR 1·9, 95% CI 1·1–3·3). Consultations by infectious disease specialists were less frequent in the intervention group (388 [13·4%] of 2889) versus the control group (405 [16·9%] of 2390; OR 0·84, 95% CI 0·59–1·25).
Interpretation
An integrated multimodal computerised antibiotic stewardship intervention did not significantly reduce overall antibiotic use, the primary outcome of the study. Contributing factors were probably insufficient uptake, a setting with relatively low antibiotic use at baseline, and delays between ward admission and first CDSS use.
Funding
Swiss National Science Foundation.
Translations
For the French and Italian translations of the abstract see Supplementary Materials section.
Introduction
Avoiding emergence and dissemination of multidrug-resistant pathogens remains a global priority.1, 2, 3 Efforts to promote appropriate use of antimicrobials in general and antibiotics in particular through antibiotic stewardship programmes have been advocated by WHO, professional societies, and governments.4 Although there is increasing evidence that antibiotic stewardship interventions can generally reduce antimicrobial use, costs, Clostridioides difficile infection, and ultimately antimicrobial resistance in the hospital setting,5, 6 we still have insufficient evidence on which particular antibiotic stewardship interventions provide the largest, most sustainable, and most cost-effective improvement in antibiotic prescribing.
Research in context.
Evidence before this study
Antimicrobial resistance is a substantial threat to human wellbeing and the recently published GRAM study estimated that about 1·3 million deaths per year in 2019 were attributable to bacterial antimicrobial resistance. Assuring that antibiotics are used appropriately is key for preventing further emergence and spread of antimicrobial resistance and a crucial determinant of the health of the patients of today and tomorrow. Antibiotics are among the most frequently prescribed medicines to patients and among the medicines most often used incorrectly. Changing the behaviour of prescribers towards a more appropriate use of antibiotics is a challenge and there is no clear consensus on how best to achieve this goal. There is therefore an urgent need to develop more effective tools to better support physicians in the complex task of choosing the most appropriate antibiotic treatment, including when not to prescribe antibiotics. Computerised decision-support systems (CDSSs) hold great promise to address the issue of improving antimicrobial prescriptions. We searched MEDLINE for articles published from inception until Jan 1, 2022, using the terms “antimicrobial” or “antibiotic” combined with “randomised trial” or “randomized trial” and “computerized decision support” or “computerised decision support”. We also analysed studies identified in recent systematic reviews on CDSSs for antimicrobial prescribing. Most studies were before-and-after studies without a control group, and only a few cluster randomised trials were identified, mostly targeting the primary care setting or specific syndromes. Systematic reviews consistently concluded that there is the need for higher-quality studies to answer the question of effectiveness of CDSSs for antimicrobial stewardship.
Added value of this study
Our study is one of the few multicentre cluster randomised trials assessing CDSS for antibiotic prescribing in inpatients. In this cluster randomised trial of more than 20 000 patients admitted to 24 units in three Swiss hospitals, the intervention was not associated with a significant decrease in antibiotic use measured in days of therapy per admission in the 12 intervention units compared with the 12 control units that were exposed to standard-of-care antibiotic stewardship. Switch from intravenous to oral therapy was more frequent in the intervention group. Insufficient uptake of the tool related to important aspects of its design could have contributed to this negative result. Nevertheless, computerised decision-support tools will probably gain increasing importance in the future and this study offers some insights in how best to design and implement them.
Implications of all the available evidence
The fact that we were not able to demonstrate an effect on the primary outcome of antibiotic use shows that there is still a lot to learn on how to best leverage the potential of computers in health care. Furthermore, there is no way other than high-quality randomised trials to facilitate this learning journey and to avoid overly optimistic conclusions about the impact of these systems. CDSSs need to be user friendly, and our study provides clues for the improvement of the design of multimodal antibiotic-stewardship interventions and to maximise uptake of computerised systems by prescribers.
New technologies are becoming an integral part of modern medicine and computerised tools essential for the safe and effective delivery of health care. Computerised physician-order entry (CPOE) has become a standard in many settings, offering the possibility to electronically guide prescribers at the time of prescription, independent of direct human intervention. Computerised decision-support systems (CDSSs) integrated into CPOE have thus gained increasing interest as tools to improve antibiotic prescribing, but high-quality evidence for the impact of these digitalised antibiotic-stewardship interventions is still scarce.7, 8, 9 The vast majority of studies in this area are before-and-after studies without a control group, which have a higher risk of bias and lower internal validity than randomised controlled trials.10 Multinational consensus groups stressed the necessity to optimise research in the field of antibiotic stewardship, including using more appropriate study designs and judicious selection of outcomes.8, 11, 12
The Computerised Antibiotic Stewardship Study (COMPASS) trial aimed to address this evidence gap through a randomised multicenter trial assessing whether a CDSS can reduce antimicrobial use measured in days of therapy per admission in patients who are admitted to hospital.
Methods
The study protocol has been published previously.13 Deviations from the published protocol are outlined in this Article (appendix p 1). We followed the CONSORT extension for cluster randomised trials for the reporting of this study14 (appendix 3 p 2).
Study setting and population
This study was done in 24 acute-care wards in three Swiss hospitals (table 1 ; appendix 3 p 5). Geneva University Hospitals is a primary tertiary-care centre in the French-speaking part of Switzerland. The 16 participating wards in Geneva (eight in internal medicine and eight in geriatrics) were included from the internal medicine department and geriatric department. Ente Ospedaliero Cantonale is a regional hospital network in the Canton of Ticino, the Italian-speaking part of Switzerland, of which two hospitals, Bellinzona and Lugano regional hospitals, participated in the study. The eight participating wards in Ente Ospedaliero Cantonale (two surgery wards and two internal medicine wards per site) were recruited from the internal medicine and surgical departments. Antibiotic stewardship programmes have been implemented previously in the three participating hospitals (table 1), which have a similar electronic health record and electronic prescribing system with CPOE. Apart from the introduction of the CDSS, there were no changes in the antibiotic stewardship programmes over the course of the study.
Table 1.
Characteristics of the participating hospitals
| Geneva University Hospitals |
Ente Ospedaliero Cantonale |
||
|---|---|---|---|
| Lugano | Bellinzona | ||
| Type of hospital | University tertiary-care hospital | Regional hospital | |
| Number of acute-care beds in 2019 | 1100 | 306 | 229 |
| Approximate overall admissions to acute-care medicine or surgery wards in 2019 | 26 000 | 8000 | 6000 |
| Acute care defined daily doses per 100 patient days, 2017 | 48 | 50 | 42 |
| Electronic health record | In-house development of EHRs and first elements of electronic health records in place since the 1970s, current clinical part of the EHR implemented since 2000 | Based on the in-house system from Geneva University Hospitals | |
| Computerised Physician Order Entry | Since 2006 | Since 2016 | |
| Antibiotic-stewardship activities | Antibiotic-stewardship programme since 2007: local guidelines updated every 2 years; infectious disease consultations on demand; review of positive blood cultures; dedicated rounds in some divisions and real-time review of antibiotic prescriptions (ICU, HSCT, and SOT units); internal and external benchmarking of antibiotic usage and resistance; regular teaching sessions for physicians; advice on therapeutic drug monitoring on demand; no dedicated rounds in geriatric and internal medicine departments; and no real-time review of antibiotic prescriptions in geriatric and internal medicine departments | Local guidelines updated every 2 years; review of every positive blood culture; regular teaching sessions for physicians; real-time review of antibiotic prescriptions during infectious disease specialists rounds, once per week in Lugano and in selected wards in Bellinzona; and advice on therapeutic drug monitoring on demand in Lugano | |
EHR=electronic health record. ICU=intensive care units. HSCT=haematopoietic stem-cell transplantation. SOT=solid-organ transplantation.
Participants
Eligibility criteria for the wards were at least 150 admissions per year and having CPOE implemented. Exclusion criteria for the wards were outpatient clinics, overflow wards, absence of a matchable ward regarding specialty and baseline antibiotic use, intensive care units, and emergency rooms.
Any physician in charge of prescribing antimicrobials in CPOE (usually physicians in training) was directly exposed to the CDSS, whereas other physicians in the ward who were not actively prescribing were indirectly exposed to the CDSS (for example through discussion with physicians in training). All physicians in the participating wards received quarterly feedback reports by e-mail aggregated at the ward level. Inclusion and exclusion criteria for the participating wards are listed in the published protocol.13
Trial design and randomisation
The COMPASS trial was an open-label, cluster-randomised trial. Before randomisation, wards were paired according to location (Geneva or Ticino), specialty (medicine or visceral surgery or orthopaedic surgery in Ticino and medicine or geriatrics in Geneva), and in Geneva only (since all wards were already matched in Ticino with the first criteria), baseline antibiotic use in days of therapy per admission. Within each of the 12 pairs, wards were then randomised (1:1) to the intervention or control group using an online random-sequence generator. Allocation of the intervention was not concealed to physicians or to patients. The study intervention period was 12 months. The intervention was implemented on Sept 7, 2018, in the four intervention wards at Ente Ospedaliero Cantonale (Ticino) and on Dec 17, 2018, in the eight intervention wards in Geneva University Hospitals (Geneva). A run-in period of 2 weeks to 3 weeks was considered and the study period for the primary analysis in Ticino was Oct 1, 2018, to Sept 30, 2019, and in Geneva was Jan 1, 2019, to Dec 31, 2019. The trial was approved by the responsible institutional review boards (Commission Cantonale d'Éthique de la Rercherche de Genève, approval number 2017-00454 and Comitato Etico Cantonale, Ticino).
Intervention
The intervention was implemented at the ward level. The overall framework for the COMPASS intervention was identical in all study sites (figure 1 ; appendix 3 p 28), but the informatics development was done in parallel at Geneva University Hospitals and Ente Ospedaliero Cantonale.
Figure 1.
Framework of the multimodal computerized intervention
The computerised decision-support system is embedded into the electronic-prescribing system and triggered by the prescription of an antimicrobial in the computerised physician-order entry. The intervention contains four components: decision support for antimicrobial treatment and request for an accountable justification in case of deviation from the recommended duration; alert for self-guided re-evaluation of the prescription on calendar days 3–5; decision support for the duration and request for an accountable justification in case of deviation from the recommended duration; and feedback of quality indicators of antimicrobial prescriptions delivered at the ward level. CAP=community-acquired pneumonia. IV=intraveinous. PO=per os.
The framework of the computerised intervention consisted of four components (described in detail in the appendix 3 p 8). First, decision support for the choice of antimicrobial treatment on the basis of indication entry (from a list with the possibility to enter free text) and the corresponding treatment recommendation (if available for the chosen indication) in the local guidelines. An accountable justification was requested in case of guideline deviation (from a list with the possibility to enter free text; appendix 3 p 24). Second, alert for self-guided re-evaluation of antimicrobial therapy on calendar day 2–4 of therapy (the alert was only triggered by the delay from the time of the initial prescription and was only visible to physicians, and was scheduled by default at 3 days from the initial prescription but could be anticipated or postponed by 1 day at the time of the initial prescription to give the prescriber the possibility to avoid re-evaluation alerts on weekends). To reassess the treatment, the prescriber had three choices, stop, revalidate, or modify, represented by three specific buttons. By clicking on stop, the treatment would be stopped. By clicking on revalidate, the treatment would be confirmed and continue unchanged until the end dates set initially or until a new order was made. By clicking on modify, the prescriber would be directed to the support system and would have the possibility to select a new indication, a new antimicrobial, a new route of administration, or several of these options. Third, decision support for the duration of antimicrobial treatment based on guideline-recommended duration for the selected indication. Fourth, quarterly automated-feedback reports of unit-wide antimicrobial prescribing indicators sent by email to all physicians working in participating wards.
The computerised intervention was integrated into the electronic prescribing system and was only visible for physicians. The prescriber was directed into the decision-support tool for any new antimicrobial (on the basis of prespecified Anatomical Therapeutic Chemical codes) that was ordered. The prescriber could also enter the system by clicking on a specific icon termed COMPASS on the electronic prescribing homescreen.
The control wards received standard-of-care antimicrobial stewardship (table 1). Screenshots of the two CDSS (Ticino and Geneva), the underlying algorithms, and examples of feedback reports are provided in the appendix 3 (pp 9–17, 28).
A new feature of the CDSS was developed and implemented in September, 2019, in Geneva (after 9 months) to block a revalidation of the ongoing prescriptions and force the prescriber to prescribe antimicrobials through the CDSS from the day the patient was admitted to the ward if they were already receiving antimicrobials at the time of admission (appendix 3 p 5).
Outcomes
The primary outcome was measured by the difference in overall systemic antibiotic use measured in days of therapy of systemic antibiotic use per admission based on electronically recorded antimicrobial administration data. Secondary outcomes are presented in the appendix 3 (p 1), and deviations from the study protocol are highlighted. Unplanned 30-day readmission was replaced by unplanned 18-day readmission, only in-hospital 30-day mortality is provided instead of 30-day mortality (data outside the hospital were not available), hospital length of stay (LOS) has been replaced by ward length of stay, because a patient can be admitted to wards belonging to the two groups over their hospital stay. Clinical outcomes are included to demonstrate the safety of the intervention, the improvement of quality of care and the absence of unintended consequences.
Statistical analysis
Outcome variables were summarised across intervention and control groups using descriptive statistics. The unit of analysis was an admission to a study ward. If a patient was admitted several times in the same or in a different ward, the admissions were considered as independent observations. The analysis populations were the intention-to-treat (ITT) population and the per-protocol population. The ITT set included all admissions. The per-protocol population was a subset of the ITT population, which included admissions that received the intervention of the group they were randomly assigned to.
Analyses for the primary outcome were done for the ITT and the per-protocol populations on the basis of the randomisation of the ward in which the admission occurred. In addition, subgroup analyses were done for the ITT and the per-protocol populations for the primary outcome by region (Geneva and Ticino) and by medical specialty (surgical, medical, and geriatric). Furthermore, sensitivity analyses were done assessing only the 6 months with the new CDSS feature, which forced use of the CDSS by the physician from the day of patient admission in intervention wards in Geneva (Sept 1, 2019, to Feb 29, 2020).
Negative-binomial mixed-effects hurdle models were applied for the primary analyses to account for the excess of zeroes. These models account for the clustering effect.
The models adjusted for the following fixed effects on the cluster and patient level: intervention group; study site; age (<65 years vs ≥65 years); antibiotic used on the first day of admission (as a surrogate for community-acquired infections); type of ward; and comorbidities (defined by ICD-10 codes). A ward-specific random effect was added to account for clustering. Ward length of stay was not included, neither as an offset nor as a fixed effect, because it might be affected by the intervention (shorter antibiotic treatment could result in shorter length of stay.15 For qualitative outcomes (appropriateness) and clinical and microbiological outcomes, we used mixed-effects logistic regressions with logit links and the same structure as for the primary outcome, and a ward-specific random effect also added to account for clustering. For process outcomes (use of the system), restricted mean survival times were computed until the maximal time observed for cases in which CDSS was not used from the first day the patient received an antibiotic. The amount of missing data was negligible (less than 5%) and thus complete case analysis was done. Analyses were done using R, version 4.0.2. This trial is registered with clinicaltrials.gov (identifier NCT03120975).
Role of the funding source
The funder of the study had no role in study design, data collection, data analysis, data interpretation, writing of the report, or the decision to submit the paper for publication.
Results
During the 12-month study period, 24 clusters (16 at Geneva University Hospitals and eight at Ticino Regional Hospitals) were eligible and randomly assigned to control or intervention. There were 11 384 admissions representing 8717 unique patients in the intervention group and 9673 admissions representing 7450 unique patients in the control group (figure 2 ). In the intervention group, 4578 (40·2%) of 11 384 admissions received at least one dose of a systemic antibiotic and 4142 (42·8%) of 9673 received at least one dose in the control group. Among patients who received antibiotics, the proportion of patients who received them from the first day in the ward was 2551 (55·7%) of 4578 in the intervention group and 2251 (54·3%) of 4142 in the control group.
Figure 2.
Flow-chart of the study participants, according to study arm and cluster
An admission was defined as any admission to a ward. If a patient was admitted several times in the same or in a different ward, the admissions were considered as independent observations. The populations defined are the ITT population and the per-protocol population. ITT=intention to treat.
Baseline characteristics of study participants were well balanced between study groups (table 2 ). Characteristics of participants including only patients who received antimicrobials during their stay in the participating wards are presented in the appendix 3 (ITT and per-protocol populations; appendix 3 p 7).
Table 2.
Baseline characteristics of the study participants
| Control (N=9673 in 12 clusters) | Computerised decision-support system (N=11 384 in 12 clusters) | Total (N=21 057 in 24 clusters) | ||
|---|---|---|---|---|
| Age, years | 76 (63–85) | 75 (61–84) | 76 (62–85) | |
| Gender | ||||
| Female | 4811 (49·7%) | 5438 (47·8%) | 10 249 (48·7%) | |
| Male | 4862 (50·3%) | 5946 (52·2%) | 10 808 (51·3%) | |
| Comorbidities | ||||
| Chronic cardiac disease | 2774 (28·7%) | 3641 (32·0%) | 6415 (30·5%) | |
| Chronic lung disease | 2069 (21·4%) | 2232 (19·6%) | 4301 (20·4%) | |
| Diabetes | 1952 (20·2%) | 2132 (18·7%) | 4084 (19·4%) | |
| Chronic kidney disease | 1865 (19·3%) | 1979 (17·4%) | 3844 (18·3%) | |
| Neoplasia | 390 (4·0%) | 542 (4·8%) | 932 (4·4%) | |
| Chronic liver disease | 272 (2·8%) | 292 (2·6%) | 564 (2·7%) | |
| Immunosuppression | 146 (1·5%) | 161 (1·4%) | 307 (1·5%) | |
| HIV/AIDS | 2 (<0·1%) | 20 (0·2%) | 22 (0·1%) | |
Data are n (%) or median (IQR). All admissions were to a participating ward, regardless of whether they received antibiotics during their stay in the ward.
The unadjusted overall mean days of therapy per admission was slightly lower in the intervention group than in the control group (p<0·0001; table 3 ). The unadjusted overall mean antimicrobial day per admission was also lower in the intervention group than in the control group (2·5 antimicrobial days per admission, SD 4·5, vs 2·8 antimicrobial days per admission, SD 4·9; p<0·0001). When only considering patients receiving antibiotics on admission, the unadjusted overall mean days of therapy per admission was 7·9 (SD 7·6) in the intervention group and 8·1 (SD 8·4; p=0·50) in the control group (6·3 antimicrobial days per admission, SD 5·1, vs 6·6 antimicrobial days per admission, SD 5·6; p=0·081). Whenever the patients received antibiotics during their stay in the ward, the number of days of therapy received was similar between the control and intervention groups (table 3; appendix p 20). The per-protocol analysis showed similar results (appendix 3 p 20). Use of the CDSS at least once during ward stay occurred in 3466 (75·7%) of 4578 patients receiving antimicrobials overall, 1317 (80·0%) of 1689 in Ticino, 2149 (74·4%) of 2889 in Geneva, 800 (77·8%) of 1028 in geriatrics wards, 1930 (75·7%) of 2551 in medical wards, and 674 (67·5%) of 999 in surgical wards. In Geneva, the CDSS was used from the first day of antibiotic treatment for 2696 (58·9%) of 4578 admissions overall, 1431 (56·1%) of 2551 admissions in medical wards, and 657 (64·0%) of 1028 admissions in geriatric wards (data not available for Ticino). A substantial proportion of patients switched groups (1775 [12·3%] of 14 392) over the study period (considering only unique patients independent of the number of admissions). When only considering the same hospital stay (eg, a transfer from a ward in the intervention group to a ward in the control group, or vice versa, during the same hospital stay) the proportion of patients who switched groups during the same hospital admission was 513 (2·7%) of 19 199 (in these cases the denominator was admissions and not unique patients).
Table 3.
Summary statistics for the primary outcome for the ITT population and effect of the intervention
| Control | Computerised decision-support system | |
|---|---|---|
| DOT by admission for the entire population | ||
| Number of observations | 9673 | 11 384 |
| Mean (SD) | 3·5 (6·8) | 3·2 (6·2) |
| Median (IQR) | 0 (0–5·0) | 0 (0–5·0) |
| DOT by admission only for patients who received antibiotics | ||
| Number of observations, n (%) | 4142 (42·8%) | 4578 (40·2%) |
| Mean (SD) | 8·1 (8·4) | 7·9 (7·6) |
| Median (IQR) | 6·0 (4·0–10·0) | 6·0 (3·0–10·0) |
| Geometric mean (SD) | 5·8 (2·3) | 5·6 (2·3) |
| Effect of the intervention | ||
| Any antibiotic | 1·12* | 0·94–1·33† |
| DOT for those who received antibiotics | 0·98‡ | 0·90–1·07† |
Calculation based on non-missing values. The DOT presented was based on strictly positive values. DOT=days of therapy. ITT=intention to treat.
Odds ratio.
95% CI.
Incidence rate ratio.
Analyses done by study region, by type of wards, and, for Geneva only, for the period during which additional features were implemented in the CDSS, showed no statistically or clinically significant difference for the primary outcome (appendix 3 pp 20–21). Sensitivity analyses considering rates and adjustment for unit LOS (ITT and per-protocol populations) showed similar results (appendix 3 p 21). Analysis considering only the first admission of a patient in a participating ward showed similar results (IRR 1·02, 95% CI 0·94–1·10; appendix 3 pp 21–22). There was no apparent trend over time for decreasing days of therapy during the intervention period in comparison to the year before the implementation of the CDSS in Geneva (appendix 3 p 23).
Overall, 1195 charts of patients who received at least one antibiotic dose during their stay in a participating unit (948 [79·3%] in Geneva and 247 [20·7%] in Ticino) were manually reviewed for quality indicators of antibiotic prescribing (defined in appendix 3 pp 1, 18). Of these, 548 (45·9%) admissions concerned general medical wards, 520 (43·5%) geriatric wards, and 127 (10·6%) surgical wards. The most frequent indication reported for antibiotic prescription based on the chart review were pneumonia (462 [38·7%]), followed by lower-urinary-tract infection (127 [10·6%]), intra-abdominal infection (100 [8·4%]) and upper-urinary-tract infection (98 [8·2%]). These indications were representative of the most frequent indications entered into the Geneva CDSS (which were pneumonia with 1529 [36·5%] of 4184, followed by lower-urinary-tract infection with 258 [6·2%] and upper-urinary-tract infection with 247 [5·9%]). Switch to oral therapy by day 7 was more frequently done in the intervention group than in the control group (table 4 ). By contrast, there was no statistically significant difference between the intervention and control groups regarding other appropriateness criteria (table 4). Among the 133 prescriptions with deviation from guidelines for which a justification was provided, the most frequent reasons were recent pretreatment with antimicrobials (32 [24·1%]), recommendations by an infectious-disease specialist (31 [23·3%]), and allergy or other contraindication (26 [19·5%]; appendix 3 p 24). In both groups, the most frequently prescribed antimicrobials (in days of therapy) belonged to the Watch category (53·6%), followed by the Access (44·8%) and then the Reserve (1·5%) categories (appendix 3 p 24).
Table 4.
Effect of intervention on qualitative antimicrobial outcomes, clinical outcomes, and microbiological outcomes
| Control* | Computerised decision-support system* | Total* | OR† | 95% CI | |
|---|---|---|---|---|---|
| Qualitative antimicrobial outcomes | |||||
| Appropriate choice of the molecule‡ | 337/455 (74·1%) | 370/503 (73·6%) | 707 (73·8%) | 1·03 | 0·71–1·49 |
| Appropriate duration | 356/430 (82·8%) | 389/460 (84·6%) | 745 (83·7%) | 1·12 | 0·78–1·60 |
| De-escalation done whenever possible | 90/115 (78·3%) | 98/121 (81·0%) | 188 (79·7%) | 1·05 | 0·53–2·05 |
| Oral switch by day 7 | 154/201 (76·6%) | 187/215 (87·0%) | 341 (82·0%) | 1·91 | 1·12–3·26 |
| Treatment adapted to microbiological results | 203/228 (89·0%) | 228/245 (93·1%) | 431 (91·1%) | 1·60 | 0·83–3·07 |
| Clinical outcomes | |||||
| 30-day in-hospital mortality | 368/6142 (6·0%) | 444/7808 (5·7%) | 812 (5·9%) | 1·02 | 0·86–1·21 |
| Readmission within 18 days | 413/7276 (5·7%) | 448/8680 (5·2%) | 861/15 956 (5·4%) | 0·90 | 0·74–1·09 |
| Transfer to ICU or to IMC | 284/9619 (3·0%) | 370/11 269 (3·3%) | 654 (2·7%) | 1·20 | 0·80–1·79 |
| Infectious disease consultation§ | 405/2390 (16·9%) | 388/2889 (13·4%) | 793 (15·0%) | 0·86 | 0·59–1·25 |
| Length of stay in the ward, median¶ | 7 | 6 | 6 | 0·95 | 0·84–1·08 |
| Microbiological outcomes | |||||
| Facility onset of Clostridioides difficile infection per 1000 admissions | 2 | 2·8 | 2·2 | 1·17 | 0·81–1·68 |
Length of stay shows the results of all available data. ICU=intensive care unit. IMC=intermediate care unit.
Denominators vary by outcomes.
Adjusted.
Assessed only indications for which local guidelines are available.
Geneva only, the denominator is admissions receiving antimicrobials.
For the analysis, 0·5 days was added to length of stay and then log transformed. A linear mixed-effect model was used. Endpoint was log (length of stay plus 0·5). Estimate was then a ratio of geometric means.
In the multivariate model for secondary outcomes, there was no difference between the two groups in clinical and microbiological outcomes (table 4). Hospital length of stay and the number of multidrug-resistant organism bloodstream infections were also similar in the two groups (appendix 3 p 25).
The number of ID consultations in Geneva was lower (OR 0·86, 95% CI 0·59–1·25) in the intervention group (388 [5·7%] of all 6769 admissions and 388 [13·4%] of 2889 admissions receiving antimicrobials) compared to the control group (405 [6·9%] of all 5842 admissions and 405 [16·9%] of 2390 admissions receiving antimicrobials; table 4). 90 physicians participated in the online satisfaction survey. The median rating of the system on a 5-points Likert scale was 3·0 in Geneva and 3·4 in Ticino (appendix 3 p 26).
Discussion
In this large multicentre cluster randomised trial involving more than 20 000 patients from three hospitals in two Swiss-language regions, the implementation of a multimodal computerised intervention for improving antimicrobial prescriptions did not lead to a statistically significant decrease of antibiotic use. However, switch from intravenous to oral therapy occurred more frequently in the CDSS group.
Interpretation of this finding merits several considerations. First, insufficient adherence to the multicomponent computerised intervention was a major issue. Low uptake of CDSS has been reported previously,16 along with frequent overriding of the system.17 In our case, the CDSS was never used during an antimicrobial course in about one of four admissions receiving antimicrobials. Furthermore, when it was used in about 40% of the admissions, the CDSS was not used from the start of the antibiotic course but with a delay of several days. This can partially be attributed to the trial design. A large proportion of antimicrobial prescriptions for patients who were hospitalised are initiated in emergency rooms, which were excluded from the study because of contamination issues (patients on antibiotics started in the emergency room before being transferred to control wards) and because of the absence of a matchable ward for pairing. Therefore, for any patient transferred from the emergency room to a participating ward, instead of a simple validation of the prescriptions through the electronic prescribing system, prescribers in the intervention ward had to stop and represcribe antimicrobials through the CDSS. This was perceived as additional workload and therefore frequently not performed, unless the prescriber intended to change the initial empiric treatment. To address this issue, an additional feature was developed during the course of the study in Geneva to enforce the use of the system from the day of the patient admission in the ward if they were already receiving antibiotics at admission (the automatic revalidation of already prescribed antimicrobials was blocked). Nevertheless, the per-protocol analysis, taking into account only ward admissions for which the CDSS was used at least once and a sensitivity analysis taking into account only the period starting from the new feature implementation in Geneva did not show different results.
Another factor potentially jeopardising the results was that the re-evaluation process was left optional for the prescriber. Mainly for safety concerns, we designed the re-evaluation process to not trigger automatic stop orders if the scheduled delay for re-evaluation was exceeded. We were not able to measure precisely the delay between the first appearance of the re-evaluation alert and the action by the prescriber (if ever done) but overall re-evaluation was underperformed, which highlights the difficulty to properly design automatic alerts.18
The potential impact of the tool might have also been limited by its focus on empiric antimicrobial therapy. Most of the local guidelines concern empirical treatments that are frequently started in the emergency room. Therefore, the CDSS was not primarily designed to provide support based on microbiological results available, and this point could have also limited end-user satisfaction and adoption. Additionally, turnover of junior physicians between units belonging to different study groups could have introduced some cross-over effects and improved prescriptions in the control group. Finally, by international comparison, there was relatively low antibiotic use in the participating hospitals, which could have limited the margin for improvement and contributed to an insufficiently powered trial.
Positive findings in the context of computerised-antibiotic-stewardship studies are mostly coming from studies with weak, non-randomised study designs and should be interpreted cautiously. A high risk of bias is associated with greater intervention effect in antibiotic-stewardship studies19 and the quality of the reporting of interventions in published antibiotic-stewardship studies is generally poor,19 making a real assessment of the impact of CDSS alone difficult. Over the course of our study, we intentionally respected the automated aspect of the intervention and did not add manual or human interventions to reinforce the uptake or modify behaviours. We felt that this would have made results more difficult to apply to real-life settings, given that such extra interventions are difficult to sustain outside a study setting. Although the best way to maximise the impact of CDS systems remains unclear, our experience suggests that some kind of human intervention in addition to optimal design of the tools might be necessary to achieve this goal.20
The objective of antimicrobial stewardship is not only to avoid unnecessary antimicrobial exposure, but also to improve the quality of antimicrobial prescribing by assuring that the correct antibiotics are used at the correct dose, route, and duration.21 In that context, an interesting positive finding of our study is the increased transition from intravenous to oral antimicrobials in the intervention group. Several studies demonstrated that increased use of oral antibiotics is associated with reduced drug costs and length of hospital stay without compromising efficacy or safety.4, 15, 22 The median ward length of stay was also reduced (albeit not significantly) in the intervention group, and de-escalation, adaptation to microbiological results, and right choice of molecules were all better performed in the intervention group than the control group. The consistent effects in favour of the intervention, even not statistically significant, are encouraging. Again, the underuse of the tool might explain, at least partially, these inconclusive results. The high adherence to guidelines (around 75%), above the adherence reported by other studies19, 23 underscores already good antimicrobial prescribing practices, even if adaptation to microbiological results and de-escalation were suboptimal in both groups, providing targets for antimicrobial stewardship interventions.21, 24 It should also be noted that optimal indicators and target values for these aspects of antimicrobial stewardship have yet to be defined.21
Our study has several important strengths. We used a robust study design with cluster randomisation and inclusion of several centres. The CDSSs were developed in two study sites with different electronic-health records and languages. Although the CDSSs were based on the same algorithms,25 they were developed independently, which improves external validity and suggests feasibility of implementation of the algorithms elsewhere. Collection of clinical, microbiological, and process outcomes was done rigorously, using predefined and validated criteria. These important metrics are frequently absent in clinical trials investigating antibiotic-stewardship intervention in acute-care settings.11, 17, 26, 27 In addition, the CDSSs were built considering findings from qualitative studies and systematic reviews,18 such as involvement of interprofessional teams during the development and implementation stages,28 integration into the workflow, and limitation of data entry.18
Our study brings key elements to develop and implement CDSS for antibiotic-stewardship programmes. We recommend the following: spending sufficient time and money on the design of the user interface and integration into the workflow, including extensive usability testing to assure widespread adoption by the end user; the limitation of possibilities to circumvent the system and making certain processes mandatory, such as the re-evaluation of the prescription; designing the tool to provide support not only for basic situations, but also to provide individualised suggestions for more complex situations, ideally by integrating patient-level microbiological and other laboratory data; and carefully planning the collection and retrievability of process data to correctly assess the effect of each component and adjust the system accordingly.
This study had also some limitations. First, even though the CDSS was developed in parallel in two different study sites, replication of the computerised algorithms in other settings would require important resources. Secondly, process measures for some components of the intervention, such as the re-evaluation process, were not appropriately collected precluding a detailed analysis of the uptake and the effects of each component.29 Finally, we showed in a previously published qualitative study that efficiency and ease of use are two key facilitators for the adoption of CDSS.8 Even though the tools were developed iteratively with a testing process involving clinicians, we probably should have invested more time to involve end-users through an extensive user-experience process to optimise these aspects and improve user acceptance and adoption.30
The effectiveness of the COMPASS intervention to reduce overall antimicrobial use remains inconclusive. However, some positive signals regarding quality of antibiotic us are encouraging to further explore the CDS system for antibiotic stewardship. Improvements in the design of the user interface, a mandatory re-evaluation process, and integration of microbiology data seem to be key elements to maximise end-user adoption of such a system. The continued expansion of CDSS and AI into health care seems unavoidable. This study illustrates that learning curve might be steeper than expected.
Data sharing
The study protocol and statistical analysis plan will be available immediately following publication with no end date for anyone who wishes to access these documents for any purpose. Patient-level data collected for the study will not be made available to others.
Declaration of interests
We declare no competing interests.
Acknowledgments
Acknowledgments
We would like to thank all members of the COMPASS study team for the valuable contribution in setting up the study. We would also like to thank all the physicians in the participating wards and the head of participating divisions and departments. We would like to thank Carolina Frankhauser for providing data on Clostridoides difficile. We would like to thank Constance Robert-de-Saint-Vincent for her work with data entry and file revisions. This work was supported by the Swiss National Science Foundation (grant number 407240_167079) in the context of the Swiss National Research Programme 72 Antimicrobial Resistance.
COMPASS study group members
Carlo Balmelli, Emmanuel Durand, Olivia Fahrni, Damien Grauser, Marie-Françoise Piuz, Pietro Benedetto Faré, and Mickaël Tognon.
Contributors
BDH conceived study and obtained funding, and wrote the first draft of the protocol. SH, LK, EB, RM, and GC provided input on the protocol. BDH, GC, JP, NV, SDS, BWS, NSC, RV, VC, and FP developed CDSS. GC, RV, CRSV, OF, and MT collected data on quality of antibiotic use. GC, BDH, and JS wrote the statistical analysis plan. GC prepared the data for analysis. GC and JS analysed the data, and directly accessed and verified the underlying data reported in the manuscript. GC and BDH wrote the first draft of the manuscript. All authors critically reviewed the manuscript and provided input, and all authors confirm that they had full access to all the data in the study and accept responsibility to submit for publication.
Supplementary Materials
References
- 1.Dyar OJ, Huttner B, Schouten J, Pulcini C, ESCMID Study Group for Antimicrobial stewardshiP What is antimicrobial stewardship? Clin Microbiol Infect. 2017;23:793–798. doi: 10.1016/j.cmi.2017.08.026. [DOI] [PubMed] [Google Scholar]
- 2.Nathwani D, Varghese D, Stephens J, Ansari W, Martin S, Charbonneau C. Value of hospital antimicrobial stewardship programs [ASPs]: a systematic review. Antimicrob Resist Infect Control. 2019;8:35. doi: 10.1186/s13756-019-0471-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Murray CJ, Ikuta KS, Sharara F, et al. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet. 2022;399:629–655. doi: 10.1016/S0140-6736(21)02724-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. 2016;62:e51–e77. doi: 10.1093/cid/ciw118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Baur D, Gladstone BP, Burkert F, et al. Effect of antibiotic stewardship on the incidence of infection and colonisation with antibiotic-resistant bacteria and Clostridium difficile infection: a systematic review and meta-analysis. Lancet Infect Dis. 2017;17:990–1001. doi: 10.1016/S1473-3099(17)30325-0. [DOI] [PubMed] [Google Scholar]
- 6.Hulscher MEJL, Prins JM. Antibiotic stewardship: does it work in hospital practice? A review of the evidence base. Clin Microbiol Infect. 2017;23:799–805. doi: 10.1016/j.cmi.2017.07.017. [DOI] [PubMed] [Google Scholar]
- 7.Baysari MT, Lehnbom EC, Li L, Hargreaves A, Day RO, Westbrook JI. The effectiveness of information technology to improve antimicrobial prescribing in hospitals: a systematic review and meta-analysis. Int J Med Inform. 2016;92:15–34. doi: 10.1016/j.ijmedinf.2016.04.008. [DOI] [PubMed] [Google Scholar]
- 8.Rzewuska M, Charani E, Clarkson JE, et al. Prioritizing research areas for antibiotic stewardship programmes in hospitals: a behavioural perspective consensus paper. Clin Microbiol Infect. 2019;25:163–168. doi: 10.1016/j.cmi.2018.08.020. [DOI] [PubMed] [Google Scholar]
- 9.Charani E, McKee M, Ahmad R, et al. Optimising antimicrobial use in humans: review of current evidence and an interdisciplinary consensus on key priorities for research. Lancet Reg Health Eur. 2021;7 doi: 10.1016/j.lanepe.2021.100161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.de Kraker MEA, Abbas M, Huttner B, Harbarth S. Good epidemiological practice: a narrative review of appropriate scientific methods to evaluate the impact of antimicrobial stewardship interventions. Clin Microbiol Infect. 2017;23:819–825. doi: 10.1016/j.cmi.2017.05.019. [DOI] [PubMed] [Google Scholar]
- 11.Schweitzer VA, van Werkhoven CH, Rodríguez Baño J, et al. Optimising design of research to evaluate antibiotic stewardship interventions; consensus recommendations of a multinational working group. Clin Microbiol Infect. 2020;26:41–50. doi: 10.1016/j.cmi.2019.08.017. [DOI] [PubMed] [Google Scholar]
- 12.Schweitzer VA, van Heijl I, van Werkhoven CH, et al. The quality of studies evaluating antimicrobial stewardship interventions: a systematic review. Clin Microbiol Infect. 2019;25:555–561. doi: 10.1016/j.cmi.2018.11.002. [DOI] [PubMed] [Google Scholar]
- 13.Catho G, De Kraker M, Waldispühl Suter B, et al. Study protocol for a multicentre, cluster randomised, superiority trial evaluating the impact of computerised decision support, audit and feedback on antibiotic use: the COMPuterized Antibiotic Stewardship Study (COMPASS) BMJ Open. 2018;8 doi: 10.1136/bmjopen-2018-022666. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.CONSORT Cluster trials. http://www.consort-statement.org/extensions?ContentWidgetId=554
- 15.Mouwen AMA, Dijkstra JA, Jong E, Buijtels PCaM, Pasker-de Jong PCM, Nagtegaal JE. Early switching of antibiotic therapy from intravenous to oral using a combination of education, pocket-sized cards and switch advice: a practical intervention resulting in reduced length of hospital stay. Int J Antimicrob Agents. 2020;55 doi: 10.1016/j.ijantimicag.2019.07.020. [DOI] [PubMed] [Google Scholar]
- 16.Mitchell SL, D'Agata EMC, Hanson LC, et al. The Trial to Reduce Antimicrobial Use in Nursing Home Residents With Alzheimer Disease and Other Dementias (TRAIN-AD): a cluster randomized clinical trial. JAMA Intern Med. 2021;181:1174–1182. doi: 10.1001/jamainternmed.2021.3098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Nageeti T, Al-Harbi R, Al-Malki K, et al. Improving antimicrobial prescriptions with computerized decision support systems: where are we? Stud Health Technol Inform. 2019;262:138–141. doi: 10.3233/SHTI190036. [DOI] [PubMed] [Google Scholar]
- 18.Olakotan OO, Mohd Yusof M. The appropriateness of clinical decision support systems alerts in supporting clinical workflows: a systematic review. Health Informatics J. 2021;27 doi: 10.1177/14604582211007536. 14604582211007536. [DOI] [PubMed] [Google Scholar]
- 19.Davey P, Marwick CA, Scott CL, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev. 2017;2 doi: 10.1002/14651858.CD003543.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wathne JS, Kleppe LKS, Harthug S, et al. The effect of antibiotic stewardship interventions with stakeholder involvement in hospital settings: a multicentre, cluster randomized controlled intervention study. Antimicrob Resist Infect Control. 2018;7:109. doi: 10.1186/s13756-018-0400-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Wathne JS, Skodvin B, Charani E, et al. Identifying targets for antibiotic stewardship interventions through analysis of the antibiotic prescribing process in hospitals - a multicentre observational cohort study. Antimicrob Resist Infect Control. 2020;9:114. doi: 10.1186/s13756-020-00749-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Omidvari K, de Boisblanc BP, Karam G, Nelson S, Haponik E, Summer W. Early transition to oral antibiotic therapy for community-acquired pneumonia: duration of therapy, clinical outcomes, and cost analysis. Respir Med. 1998;92:1032–1039. doi: 10.1016/s0954-6111(98)90351-1. [DOI] [PubMed] [Google Scholar]
- 23.Devchand M, Stewardson AJ, Urbancic KF, et al. Outcomes of an electronic medical record (EMR)-driven intensive care unit (ICU)-antimicrobial stewardship (AMS) ward round: assessing the ‘five moments of antimicrobial prescribing’. Infect Control Hosp Epidemiol. 2019;40:1170–1175. doi: 10.1017/ice.2019.218. [DOI] [PubMed] [Google Scholar]
- 24.Howard P, Huttner B, Beovic B, et al. ESGAP inventory of target indicators assessing antibiotic prescriptions: a cross-sectional survey. J Antimicrob Chemother. 2017;72:2910–2914. doi: 10.1093/jac/dkx243. [DOI] [PubMed] [Google Scholar]
- 25.Catho G, Centemero NS, Waldispühl Suter B, et al. How to develop and implement a computerized decision support system integrated for antimicrobial stewardship? Experiences from two Swiss hospital systems. Front Digit Health. 2021 doi: 10.3389/fdgth.2020.583390. published online Feb 16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Davey P, Peden C, Charani E, Marwick C, Michie S. Time for action-Improving the design and reporting of behaviour change interventions for antimicrobial stewardship in hospitals: early findings from a systematic review. Int J Antimicrob Agents. 2015;45:203–212. doi: 10.1016/j.ijantimicag.2014.11.014. [DOI] [PubMed] [Google Scholar]
- 27.Moehring RW, Anderson DJ, Cochran RL, Hicks LA, Srinivasan A, Dodds Ashley ES. Expert consensus on metrics to assess the impact of patient-level antimicrobial stewardship interventions in acute-care settings. Clin Infect Dis. 2017;64:377–383. doi: 10.1093/cid/ciw787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Bezemer T, de Groot MC, Blasse E, et al. A human(e) factor in clinical decision support systems. J Med Internet Res. 2019;21 doi: 10.2196/11732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40. doi: 10.1186/1748-5908-2-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Richardson S, Mishuris R, O'Connell A, et al. ‘Think aloud’ and ‘near live’ usability testing of two complex clinical decision support tools. Int J Med Inform. 2017;106:1–8. doi: 10.1016/j.ijmedinf.2017.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The study protocol and statistical analysis plan will be available immediately following publication with no end date for anyone who wishes to access these documents for any purpose. Patient-level data collected for the study will not be made available to others.