Time to evaluate decision support systems for antimicrobial prescribing outside the hospital

In this issue of The Lancet Infectious Diseases, Gaud Catho and colleagues¹ reported a cluster-randomised trial in 24 wards of three Swiss hospitals to measure the effect of a computerised decision-support system (CDSS) integrated with a computerised physician-order entry (CPOE) system on improving prescribing and reducing hospital antibiotic use before the COVID-19 pandemic.

This trial provides an in-depth examination of the feasibility and generalisability of such a programme in a real-world setting. The in-house CDSS was carried out with attractive dynamic indicators. The randomisation procedure was adequate, and stratified by unit type. There is a long history of using electronic tools in Geneva alongside antimicrobial stewardship activities. Indeed, the hospital had been implementing electronic health records since the 1970s, with the current version established nearly 20 years before the CDSS intervention was deployed. At the start of the study, the CPOE and the antimicrobial stewardship programme had been in place for 13 years. This might have diminished the effect of CDSS as an additional layer to an existing, comprehensive, and well implemented system. Thus, the data from the Lugano and Bellinzona hospitals, which were naive sites, are interesting because they provide external validation to the trial. The confidence in data is also high because the authors did a random selection and qualitative review of about 10% of the medical records of the two groups of patients who received at least one dose of antibiotic. There was also a low level of intercluster contamination (about 10%). The uptake of intervention was moderate, with nearly a quarter of patients who received antimicrobial therapy did not receive the intervention of CDSS when they were eligible.

The trial shows that in a setting with extensive experience of electronic tools and well established antimicrobial stewardship programme, the addition of a CDSS for (empirical) antibiotic prescribing does not reduce overall antibiotic use, perhaps because of poor adherence to the features of the support system, its ergonomics, or its recommendations; although, based on chart review, more patients in units that received the intervention were switched to oral therapy.

This is an important and contrasting finding, as CDSS have been used successfully to increase adherence to guidelines in hospital cohorts evaluating their effect in treating urinary-tract infections and in patients who are critically ill.2, 3 The CDSS also improved appropriateness of empirical antibiotic treatment in a multicountry cluster-randomised trial in patients with suspected bacterial infections.⁴ This trial⁴ was done in 2004, before the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America recommended in 2016 that CDSSs be integrated at the point of care,⁵ and before the massive deployment of antimicrobial stewardship policies. The trial⁴ is also the only randomised trial to have been included in the systematic review with meta-analysis done in 2017 by Davey and colleagues,⁶ which showed that antimicrobial stewardship interventions consistently improved compliance with recommendations and reduced the duration of antibiotic treatment. It is therefore important to have a recent randomised trial to assess the value of this technology, almost 20 years later, because it is unlikely to overstate the benefit of the CDSS in an obviously different context.

Despite integration into the workflow through CPOE, the success of CDSS in this trial might have been limited by a design that was not sufficiently human centred to fully understand and incorporate end-user needs, and how end users interacted with CDSS before its actual implementation.⁷

There is no ideal framework for evaluating CDSS or antimicrobial stewardship programmes. Cluster randomisation, including stepped-wedge designs, is favoured because it allows causal inference. However, these designs are logistically and analytically complex, costly, and potentially subject to numerous biases.⁸ As cluster size increases, these trials have diminishing returns in power and precision, and to increase trial efficiency, the number and size of the cluster should be determined simultaneously, not independently. Because of the absence of allocation concealment, contamination can occur, and the effect of the intervention is likely to be influenced by the amount of adoption of the intervention at the cluster level. In addition, CDSS are, in essence, multimodal interventions, and the process of selecting the parameters that should be measured and retrievable from the system for each component of the intervention is essential to ensure a fine-grained evaluation of the trial results. In the near future, it might be worthwhile to use alternative causal-inference methods to evaluate interventions that have already diffused, to avoid the use of interrupted time-series designs, and to overcome the classic pitfalls of the cluster randomisation framework. To that end, real-world data at the individual level could be used to emulate target trials to produce real-world evidence.⁹

Future improvements of CDSSs in the hospital setting will come from integrating individual level laboratory and microbiological data, and possibly personal medical history through electronic health records, to personalise CDSS-derived recommendations in complex cases of infection. However, two challenges remain for integrating and evaluating CDSS to improve antimicrobial prescribing. First, paediatrics, and in particular neonatal sepsis, for which extrapolation of results from studies done in adults is not possible, and for which many broad-spectrum antibiotics are used off label.¹⁰ Second, primary care, including nursing homes, which accounts for the bulk of antimicrobial prescribing, and for which even a small effect could prevent a substantial number of antimicrobial exposures, but for which implementation of the interventions is complex.7, 11

I declare no competing interests.