Abstract
Background
Prospective review and feedback (PRF) of antibiotic prescriptions and compulsory computerized decision support system (CDSS) are 2 strategies of antimicrobial stewardship. There are limited studies investigating their combined effects. We hypothesized that the use of on-demand (voluntary) CDSS would achieve similar patient outcomes compared with automatically triggered (compulsory) CDSS whenever broad-spectrum antibiotics are ordered.
Methods
A parallel-group, 1:1 block cluster randomized crossover study was conducted in 32 medical and surgical wards from March to August 2017. CDSS use for piperacillin-tazobactam or carbapenem in the intervention clusters was at the demand of the doctor, while in the control clusters CDSS use was compulsory. PRF was continued for both arms. The primary outcome was 30-day mortality.
Results
Six hundred forty-one and 616 patients were randomized to voluntary and compulsory CDSS, respectively. There were no differences in 30-day mortality (hazard ratio [HR], 0.87; 95% CI, 0.67–1.12), re-infection and re-admission rates, antibiotic duration, length of stay, or hospitalization cost. The proportion of patients receiving PRF recommendations was not significantly lower in the voluntary CDSS arm (62 [10%] vs 81 [13%]; P = .05). Appropriate indication of antibiotics was high in both arms (351/448 [78%] vs 330/433 [74%]; P = .18). However, in geriatric medicine patients where antibiotic appropriateness was <50%, prescription via compulsory CDSS resulted in a shorter length of stay and lower hospitalization cost.
Conclusions
Voluntary broad-spectrum antibiotics with PRF via CDSS did not result in differing clinical outcomes, antibiotic duration, or length of stay. However, in the setting of low antibiotic appropriateness, compulsory CDSS may be beneficial.
Keywords: antimicrobial stewardship, appropriate antibiotics, cluster randomized controlled trials, computerized decision support, prospective review and feedback
Increasing antimicrobial resistance due to inappropriate antimicrobial use is a global concern, and antimicrobial stewardship teams have become an integral part of the response to this issue [1, 2]. Through prospective review of antibiotic prescriptions and feedback (PRF) to doctors, patients have improved clinical response, reduced adverse effects, and reduced mortality [3–5]. However, this strategy is labor-intensive, and skilled health care workers are expensive and scarce resources [6, 7].
Antibiotic computerized decision support systems (CDSS) have been used to facilitate these processes to circumvent the lack of manpower. In observational studies, implementation of a CDSS has been correlated with an overall reduction in broad-spectrum antibiotic use and increased susceptibility of Pseudomonas aeruginosa to imipenem and Enterobacteriaceae to gentamicin and ciprofloxacin [8, 9]. CDSS also improved clinical outcomes in a randomized controlled trial [10]. While PRF and CDSS guidance are designed to be in accordance with institutional guidelines, there are differences in acceptance of the recommendations between the 2 systems [3, 10–13]. In previous studies, PRF recommendations had an acceptance rate of 60%–70%, while CDSS acceptance was only 40% [3, 4, 13, 14]. Currently, there are limited studies comparing the combined effects of these 2 strategies [2].
At Tan Tock Seng Hospital (TTSH) in Singapore, antimicrobial stewardship has focused on PRF by a team of infectious disease doctors and pharmacists. Since 2009, this team has reviewed piperacillin-tazobactam and carbapenem orders according to hospital antibiotic guidelines from day 2 of antibiotic prescription during office hours. In 2011, we implemented a CDSS, which was triggered around the clock and at the point of antibiotic ordering of piperacillin-tazobactam and carbapenem in the electronic medical record. The compulsory CDSS provides guidance on antibiotic use and infection management based on hospital guidelines. Theoretically, compulsory CDSS may improve the timeliness of appropriate antibiotic and clinical outcomes such as mortality, but doctors may find it cumbersome and intrusive, preferring on-demand CDSS use [10]. We hypothesized that together with prospective review and feedback, voluntary, on-demand use of CDSS in ordering antibiotics would achieve similar patient outcomes compared with compulsory, automatically triggered CDSS use at the point of antibiotic ordering.
We aimed to investigate in a real-world cluster randomized controlled trial if, compared with a compulsory CDSS, voluntary use of the CDSS when piperacillin-tazobactam and carbapenem were prescribed would achieve similar clinical outcomes, antibiotic prescribing, and requirement for subsequent PRF in the individual patient.
METHODS
Study Design and Patients
ARIES is a parallel-group, 1:1 block, real-world open labeled cluster randomized crossover study conducted at TTSH, a 1700-bed teaching hospital. Waiver for informed consent was approved by the local institutional review board (DSRB/F/2015/00671). The study was conducted in 32 medical and surgical wards over a 24-week period from March 2017 to August 2017. Intensive care unit (ICU), high-dependency, and step-down care wards were excluded, as we had a specialized infectious disease team for ICU and high-dependency patients. Piperacillin-tazobactam and carbapenems are rarely used in our step-down care wards. Patients in the clusters were enrolled at first prescription of piperacillin-tazobactam or a carbapenem. Each participant was only included once.
Randomization
As modifications to the CDSS could only be done at the ward level, we clustered the patients by wards. This would also reduce contamination of intervention effects within wards, which would be present in individual patient randomization. Twenty-five wards (range) had 30 (30–41) or more patients, while 7 wards had fewer than 30 (12–29) patients. Clusters were stratified into 2 blocks based on size and allocated to an intervention or control group using a random number generator. Crossover of the study arms occurred at week 12 without a washout period.
Procedures
To promote understanding and acceptance of CDSS guidance, an educational campaign was conducted. This began 24 weeks before initiation of the study and continued until completion (October 2016 to August 2017). The campaign was comprised of a monthly package of 3-minute videos with an accompanying short quiz. This was disseminated to all doctors via email, hospital intranet, and Facebook, with weekly reminders and complimentary coffee cards to 2 doctors each month as an incentive for participation. The educational materials were developed following focus groups conducted in previous studies and a 1-day prospective evaluation of compulsory CDSS use in 81 patients [12, 14, 16]. The intervention group was comprised of 15 clusters where piperacillin-tazobactam and carbapenem could be ordered by voluntary use of the CDSS (doctors were allowed to order without using the CDSS too), while the control group had 17 clusters with compulsory use of the CDSS when ordering both types of antibiotics. In both arms, PRF occurred on day 2 of prescription and was available only during office hours. Patients’ electronic medical records at our hospital were reviewed prospectively for 6 months from the first prescription based on the periods specified in the primary and secondary outcomes. The CDSS provided antibiotic recommendations adjusted for renal function, and drug allergies were accounted for by available clinical laboratory data or manually entered data such as the type of infection, severe penicillin allergy, and dialysis status. It also provided alerts and clues to help clinicians decide on diagnosis and management. Compulsory CDSS was triggered at the point of antibiotic ordering, while voluntary CDSS was used on demand at the point of antibiotic ordering. Review of CDSS guidance is necessary once activated for the antibiotic order to be completed, but doctors are free to accept or reject its recommendations. Differences in terms of clinical workflows between CDSS and PRF are described in Supplementary Table 5.
Outcomes
To demonstrate the impact of education on acceptance of CDSS guidance, we monitored acceptance rate of CDSS recommendations for the first 12 weeks of the baseline period (April to October 2016, 24 weeks) and compared it with that of the first 12 weeks of the educational campaign (October 2016 to March 2017, 24 weeks). Data were collected for the first 1280 patients during each period.
The primary outcome was 30-day mortality from the date of the first piperacillin-tazobactam or carbapenem prescription. Secondary outcomes included number and types of recommendations from PRF, clinical response at day 7, 30-day re-infection rate, 30-day re-admission rate, length of stay, diarrhea during hospitalization, 6-month incidence of multidrug-resistant organisms, duration of index piperacillin-tazobactam or carbapenem use (days of therapy), overall hospitalization cost, and appropriateness of antibiotic use according to institutional guidelines. Trained pharmacists assigned appropriateness independent of whether CDSS were used, and they were not privy to the randomization process. Clinical response was defined as resolution of systemic inflammatory response syndrome [17]. Recommendations from the CDSS and PRF were classified into de-escalation (switch to a narrower-spectrum antibiotic), dose optimization, antibiotic spectrum optimization (increase in the spectrum of antibiotic therapy), infectious disease referral, additional investigation, and setting antibiotic duration. Multidrug-resistant organisms were defined as methicillin-resistant Staphylococcus aureus, vancomycin-resistant enterococci, third-generation cephalosporin or carbapenem-resistant Enterobacterales, and multidrug-resistant Acinetobacter baumannii or Pseudomonas aeruginosa [18] and Clostridiodes difficile. Overall, hospitalization cost was determined from the final hospital bill size in Singapore dollars (1 Singapore dollar = 1.3822 US dollars on August 8, 2019).
Statistical Analysis
Sample size calculation accounted for intention-to-treat analysis and the following assumptions: 30% of patients in control arm may be transferred out to the intervention arm; recommendations were accepted 50% of the time; the mortality rate of the intervention group was 15% [4]. Simulation using 32 clusters indicated that a total of 1280 patients (16 clusters and 640 patients in each study arm) would have a power of 80% to detect a 5% difference in mortality rates [19]. Uncertainty was set at a 5% level of significance. Equal cluster sizes were assumed. As the primary outcome was binary, intracluster correlation was not needed to simulate power. Analysis for primary and secondary outcomes was carried out by intention-to-treat and per-protocol analyses. Univariate analyses using the chi-square test for categorical variables and Mann-Whitney test for continuous variables were performed. To report time to death, re-admission, and re-infection, Kaplan-Meier estimators were calculated and plotted across time strata. Log-rank tests were performed to test for equality across interventions. Cox proportional hazards models were used to calculate the risk of various outcomes. Subgroup analysis of common departments and sources of infections was performed for 30-day mortality, re-admission, re-infection, and length of stay to identify possible confounders. All tests were done at a 5% significance level. Sample size calculation was performed in R using the clusterPower package. All other analyses were performed using STATA 13. A data safety monitoring board was convened to review the interim results of the study before crossover at week 12. The study was registered at ClinicalTrials.gov (NCT04011657).
Role of the Funding Source
The study was funded by the National Medical Research Council, Ministry of Health, Singapore (CNIG14MAY005). The funder of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and final responsibility for the decision to submit for publication.
RESULTS
During the educational campaign, the number of patients who had at least 1 CDSS recommendation accepted was not significantly different during the baseline and educational phases (746/1213 [62%] vs 796/1240 [64%]; P = .11). There were more recommendations accepted during the educational phase (1300/3611 [36%] vs 1571/3640 [43%]; P < .01). Acceptance of recommendations was significantly improved for antibiotic spectrum optimization (343/894 [38%] vs 423/857 [49%]; P < .01), dose optimization (361/430 [84%] vs 442/482 [92%]; P < .01), and setting antibiotic duration (410/728 [56%] vs 509/725 [70%]; P < .01). Acceptance of recommendations was not significantly different after the campaign for de-escalation (19/347 [5%] vs 27/362 [7%]), infectious disease referral (14/228 [6%] vs 15/222 [7%]), and additional investigations (153/984 [16%] vs 155/992 [16%]).
Intraclass correlation was low and insignificant, and the percentage of total variance accounted for by the wards for 30-day mortality was 0.7% (95% CI, 0.01%–23%), for 30-day re-admission it was 2% (95% CI, 0.2%–16%), and for 30-day re-infection it was 1% (95% CI, 0.2%–12%) (Supplementary Figure 1). Therefore, we proceeded to analyze our data using the survival approach.
During the cluster randomized study from March 28, 2017, to August 28, 2017, a total of 4060 patients were prescribed piperacillin-tazobactam or carbapenems over the course of the study and screened for eligibility. One thousand two hundred fifty-seven patients from 32 clusters were randomized to voluntary (n = 641) and compulsory CDSS (n = 616) (Figure 1). Recruitment was stopped after 24 weeks. The baseline characteristics of the patients in both study arms were similar (Table 1). Most patients (92% [1161/1257]) received antibiotics for empirical therapy, with piperacillin-tazobactam accounting for 86% (1076/1257). Most patients were in the departments of General Medicine (29% [365/1257]), Geriatric Medicine (18% [223/1257]), Respiratory Medicine (9% [118/1257]), and General Surgery (7% [84/1257]). Respiratory, urinary tract, intra-abdominal, and skin and soft tissue infections were common.
Figure 1.
Trial profile. a5 antibiotics not served, 4 excluded due to inclusion for another trial, 1 excluded due to age <21 years. Abbreviations: CDSS, computerized decision support system; ICU/HD, intensive care unit/high-dependency; IQR, interquartile range; PRF, prospective review and feedback; TTSH, Tan Tock Seng Hospital.
Table 1.
Baseline Characteristics
| Characteristics | Voluntary CDSS (n = 641) | Compulsory CDSS (n = 616) |
|---|---|---|
| Demographics | ||
| Age, median (IQR), y | 74 (45–93) | 76 (48–93) |
| Male | 376 (59) | 333 (54) |
| Surgical discipline | 103 (16) | 101 (16) |
| Charlson’s score, median (IQR) | 7 (2–13) | 7 (2–13) |
| APACHE II, median (IQR) | 15 (6–28) | 16 (6–29) |
| Transferred to ICU | 38 (6) | 47 (8) |
| Transferred to step down care | 102 (16) | 104 (17) |
| Transferred out of randomized study arm | 59 (9) | 61 (10) |
| Index antibiotic | ||
| Piperacillin-tazobactam | 557 (87) | 519 (84) |
| Carbapenem | 84 (13) | 97 (16) |
| Empiric therapy | 600 (94) | 561 (91) |
| Targeted therapy | 41 (6) | 55 (9) |
| Positive microbiology | 138 (22) | 129 (21) |
| Active empiric therapy | 122/138 (88) | 107/129 (83) |
| Source of infection | ||
| Respiratory | 415 (65) | 420 (68) |
| Urinary | 109 (17) | 121 (20) |
| Intra-abdominal | 38 (6) | 34 (6) |
| Hepatobiliary | 29 (5) | 23 (4) |
| Bone and joint | 19 (3) | 8 (1) |
| Skin and soft tissue | 41 (6) | 44 (7) |
| Vascular catheter | 7 (1) | 6 (1) |
| Neutropenic sepsis | 16 (3) | 11 (2) |
| Unknown source | 66 (10) | 55 (9) |
| Othersa | 14 (2) | 5 (1) |
Data are No. (%), unless otherwise indicated.
Abbreviations: CDSS, computerized decision support system; ICU, intensive care unit; IQR, interquartile range.
aIncluding neurological source, ear, nose, and throat, infective endocarditis, eye, paraspinal abscess.
In the voluntary CDSS arm, 132 (21%) patients had their first course of piperacillin-tazobactam or carbapenems ordered using CDSS, compared with 612 (99%) patients in the compulsory CDSS arm. A similar proportion of patients in both study arms were reviewed subsequently by the PRF team: voluntary CDSS (488 [70%]) vs compulsory CDSS (443 [72%]). In the voluntary CDSS arm, 154 (24%) patients had broad-spectrum antibiotics started without CDSS or PRF review. The number and types of CDSS and PRF recommendations are summarized in Table 2. Overall, there were fewer CDSS recommendations in the voluntary arm compared with the compulsory arm (425 vs 1733), and about half of these were accepted (49% vs 46%). Antibiotic spectrum optimization, additional investigations, and setting antibiotic duration were the most common recommendations. De-escalation was more often recommended in the compulsory arm but had lower acceptance than in the voluntary arm. There were fewer PRF recommendations provided in the voluntary arm compared with the compulsory arm (74 vs 99), and >75% of these were accepted. The most common recommendations were de-escalation, additional investigations, and setting antibiotic duration. Among those with subsequent positive microbiology, patients with active empiric therapy were similar: voluntary CDSS (122/138 [88%]) vs compulsory CDSS (107/129 [83%]; P = .20).
Table 2.
Type of Recommendations and Their Acceptance (in Percentages) Provided for the Use of Broad-Spectrum Antibiotics Guided by Voluntary or Compulsory CDSS and PRF
| Characteristics | Voluntary CDSS | Compulsory CDSS |
|---|---|---|
| Total CDSS recommendations | 425 (49) | 1733 (46) |
| De-escalation | 15 (47) | 174 (9) |
| Dose optimization | 63 (98) | 199 (95) |
| Antibiotic spectrum optimization | 111 (51) | 403 (48) |
| Infectious disease consult referral | 24 (0) | 89 (5) |
| Additional investigations | 117 (11) | 495 (17) |
| Setting antibiotic duration | 95 (74) | 373 (83) |
| Total PRF recommendations | 74 (76) | 99 (80) |
| De-escalation | 37 (76) | 42 (91) |
| Dose optimization | 1 (0) | 2 (50) |
| Antibiotic spectrum optimization | 3 (100) | 5 (60) |
| Infectious disease consult referral | 3 (33) | 3 (33) |
| Additional investigations | 14 (64) | 18 (67) |
| Setting antibiotic duration | 16 (94) | 29 (83) |
Data are for the intention-to-treat population and are presented as No. (%).
Abbreviations: CDSS, computerized decision support system; PRF, prospective review and feedback.
Appropriateness of antibiotic use was similar between study arms in terms of indication, dose, and duration (Table 3). There were significantly fewer patients who received CDSS recommendations in the voluntary CDSS arm compared with the compulsory CDSS arm (132 [21%] vs 612 [99%]; P < .01). There were fewer patients who received PRF recommendations in the voluntary CDSS arm compared with the compulsory arm, but it was not statistically significant (62 [10%] vs 81 [13%]; P = .05). When either CDSS or PRF recommendations were provided, most patients (>90%) had at least 1 recommendation accepted. Per-protocol analysis showed similar trends (Table 4).
Table 3.
Appropriateness of Antibiotic Use, Acceptance of Recommendations, and Outcomes of Patients who Received Broad-Spectrum Antibiotics Guided by Voluntary or Compulsory CDSS and PRF Recommendations
| Characteristics | Voluntary CDSS (n = 641) | Compulsory CDSS (n = 616) | P |
|---|---|---|---|
| Reviewed by PRF | 448 (70) | 443 (72) | .43 |
| Appropriate indication under PRF reviews | 351/448 (78) | 330/443 (75) | .18 |
| Appropriate dose | 625 (98) | 599 (97) | .77 |
| Appropriate duration | 587 (92) | 548 (89) | .12 |
| Recommendations | |||
| Received CDSS recommendations | 132 (21) | 612 (99) | <.01 |
| Accepted CDSS recommendationsa | 130 (20) | 556 (90) | <.01 |
| Received PRF recommendations | 62 (10) | 81 (13) | .05 |
| Accepted PRF recommendationsa | 51 (8) | 71 (12) | .03 |
| Outcomes | |||
| 30-d mortality | 123 (19) | 102 (16) | .22 |
| 30-d re-infection rate | 132 (21) | 142 (23) | .29 |
| 30-d re-admission rate | 92 (14) | 87 (14) | .91 |
| Clinical response at day 7 | 535 (83) | 517 (84) | .82 |
| Length of stay, median (IQR), d | 15 (5–64) | 15 (4–70) | .92 |
| 6-mo multidrug-resistant organismsb | 152 (24) | 171 (27) | .10 |
| Diarrhea this admission | 89 (14) | 86 (14) | .96 |
| Index antibiotic days of therapy, median (IQR) | 4 (3–5) | 4 (3–5) | .47 |
| Index antibiotic days of therapy ≤3 | 295 (46) | 297 (48) | .45 |
| Gross hospitalization costs, median (IQR), S$ | 13 301 (7184–24 079) | 13 308 (6743–24 904) | .96 |
Data are presented as No. (%), unless otherwise indicated.
Abbreviations: CDSS, computerized decision support system; IQR, interquartile range; PRF, prospective review and feedback.
aPatients were considered to have recommendations by CDSS or PRF accepted if at least 1 of the recommendations provided by the respective service was accepted.
bMultidrug-resistant organisms were defined as methicillin-resistant S. aureus, vancomycin-resistant enterococci, third-generation cephalosporin or carbapenem-resistant Enterobacterales, and multidrug-resistant A. baumanii or P. aeruginosa and Clostridiodes difficile diarrhea. Data are for the intention-to-treat population.
Table 4.
Appropriateness of Antibiotic Use, Acceptance of Recommendations, and Outcomes of Patients who Received Broad-Spectrum Antibiotics Guided by Voluntary or Compulsory CDSS and PRF Recommendations
| Characteristics | Voluntary CDSS (n = 455) | Compulsory CDSS (n = 416) | P |
|---|---|---|---|
| Reviewed by PRF | 324 (71) | 293 (70) | .80 |
| Appropriate indication under PRF reviews | 259/324 (80) | 215/293 (73) | .05 |
| Appropriate dose | 443 (97) | 402 (97) | .52 |
| Appropriate duration | 424 (93) | 373 (90) | .06 |
| Recommendations | |||
| Received CDSS recommendations | 91 (20) | 412 (99) | <.01 |
| Received PRF recommendations | 41 (9) | 50 (12) | .15 |
| Outcomes | |||
| 30-d mortality | 85 (19) | 85 (20) | .52 |
| 30-d re-infection rate | 113 (25) | 106 (26) | .83 |
| 30-d re-admission rate | 85 (19) | 79 (19) | .91 |
| Clinical response at day 7 | 383 (84) | 344 (83) | .56 |
| Length of stay, median (IQR), d | 12 (4–41) | 12 (4–35) | .26 |
| 6-mo multidrug-resistant organisms | 91 (20) | 94 (23) | .35 |
| Diarrhea this admission | 59 (13) | 61 (15) | .47 |
| Index antibiotic days of therapy, median (IQR) | 4 (3–5) | 3 (3–5) | .23 |
| Index antibiotic days of therapy ≤3 | 213 (47) | 209 (50) | .31 |
| Gross hospitalization costs, median (IQR), S$ | 10 520 (5826–18 430) | 9671 (5734–17 576) | .43 |
Data are for the per-protocol population and are presented as No. (%), unless otherwise indicated.
Abbreviations: CDSS, computerized decision support system; IQR, interquartile range; PRF, prospective review and feedback.
There were similar 30-day mortality (hazard ratio [HR], 0.87; 95% CI, 0.67–1.12), 30-day re-infection (HR, 1.15; 95% CI, 0.91–1.46), and 30-day re-admission rates (HR, 0.99; 95% CI, 0.74–1.33) between voluntary and compulsory CDSS (Figure 2, Table 3). There was no difference in clinical response at day 7 between the voluntary and compulsory arms (106 [17%] vs 99 [16%]; P = .22). The median length of hospital stay (interquartile range [IQR]) was similar (15 [5–64] days vs 15 [4–70] days; P = .92). Incidence of diarrhea during admission and 6-month acquisition of multidrug-resistant organisms were not significantly different. The median days of therapy of index antibiotic use (IQR) were similar (4 [3–5] days vs 4 [3–5] days; P = .47). The overall median hospitalization cost (IQR) was not significantly different between the voluntary and compulsory CDSS arms in intention-to-treat (SG$13 302 [S$3221–S$67 110] vs SG$13 307 [S$3064–S$64 666; P = .91) (Table 3).
Figure 2.
Survival analysis of 30-day mortality in patients who received broad-spectrum antibiotics guided by a voluntary or compulsory computer decision support system and prospective review and feedback recommendations. Data are for the intention-to-treat population. Abbreviations: CDSS, computerized decision support system; HR, hazard ratio.
Subgroup analysis of the top 4 common departments, namely General Medicine, Geriatric Medicine, Respiratory Medicine, and General Surgery, was performed. There was no difference in clinical outcomes, length of stay, hospitalization cost, or duration of index antibiotics between the study arms in patients of General Medicine and Respiratory Medicine (Supplementary Table 1). Subgroup analysis of common infections, namely respiratory, urinary tract, intra-abdominal, and skin and soft tissue infections did not identify any differences between the study arms (Supplementary Table 2). Among Geriatric Medicine patients, the median length of stay (IQR) was significantly higher (19 [5–83] days vs 14 [4–43] days; P = .03) in the voluntary CDSS arm, corresponding to a significantly higher median overall hospitalization cost (S$13 945 [S$3706–S$57 133] vs S$10 444 [S$3099–S$31 276]; P = .02). There was no difference in patient characteristics between both arms, except fewer patients in the voluntary CDSS arm received (29 [27%] vs 114 [100%]; P < .01) and accepted (24 [22.0%] vs 84 [74%]; P < .01) CDSS recommendations (Supplementary Table 3). Among General Surgery patients, the median length of stay (IQR) was not significantly higher in the voluntary CDSS arm (20 [7–74] vs 16 [6–40]; P = .075); however, the median overall hospitalization cost (IQR) was significantly higher in the voluntary CDSS arm compared with the compulsory CDSS arm (S$35 303 [S$5249–S$82 634] vs S$20 994 [S$4333–S$61 243]; P < .01). Notably, appropriate indications of antibiotics were lower in both departments between voluntary and compulsory CDSS compared with the overall study (351 [78%] vs 330 [75%]; Geriatric Medicine: 52 [48%] vs 57 [50%]; P = .50; General Surgery: 26 [68%] vs 33 [72%]; P = .34). There were no significant differences in patient characteristics in the study arms, other than more CDSS recommendations and acceptance in the compulsory CDSS arm (Supplementary Tables 3 and 4).
DISCUSSION
Piperacillin-tazobactam and carbapenems prescribed in the setting of voluntary CDSS use had similar clinical outcomes and appropriateness of use when compared with compulsory CDSS. It did not increase the need for PRF recommendations by the AMS team. However, compulsory CDSS was associated with significant reduction in hospital length of stay and hospitalization costs for patients when antibiotic appropriateness was low.
In the setting of high appropriate antibiotic use and PRF, it is likely that compulsory CDSS may not have clinical benefits and may inconvenience doctors by causing delay, distraction, or irritation. A more sophisticated CDSS that uses artificial intelligence or machine learning to diagnose an infection rather than relying on doctors to enter their clinical diagnosis may be better. However, benefits may be magnified in settings with lower appropriateness of antibiotic use and when CDSS is implemented as a new system [9]. Notably, voluntary CDSS with PRF did not lead to differences in appropriateness of antibiotic use, duration of index antibiotic use, or clinical outcomes. A recent meta-analysis concluded that CDSS improved the adequacy of antibiotic coverage (measured as compliance with guidelines) and marginally lowered mortality [21]. In addition, a separate report from our group conducted just after CDSS implementation between 2011 and 2012 reported mortality benefit with receipt of CDSS antibiotic recommendations [11]. Our study was conducted several years after implementation of hospital antibiotic guidelines and PRF in 2009 and compulsory CDSS in March 2010. Doctors in the hospital had substantial experience with these interventions before the start of our study in March 2017. The high coverage of PRF may have addressed any possible difference between voluntary and compulsory CDSS use too. Further studies are needed to study the impact of mixed strategies of antibiotic stewardship in hospitalized patients [20].
Our study provided novel insights on the concurrent use of 2 common antimicrobial stewardship strategies of CDSS and PRF deployed in different ways. Piperacillin-tazobactam and carbapenems were mainly used for empiric therapy, and most CDSS recommendations to optimize the antibiotic spectrum suggest additional investigations and setting antibiotic duration. CDSS rarely provided de-escalation recommendations compared with PRF. PRF occurred on day 2 and subsequent days until the antibiotic was stopped. Additional clinical information or changes in patient condition could have driven these differences. Although <50% of CDSS recommendations were accepted in both the compulsory and voluntary CDSS arms, it was interesting to note that >90% of patients in both arms had at least 1 recommendation accepted. A separate cohort study on our CDSS for piperacillin-tazobactam and carbapenem prescriptions found that almost 50% were ordered after office hours [16]. Dose and antibiotic spectrum optimization were the most frequently accepted CDSS recommendations; it is important that these factors be correct early in the treatment of infection.
There are limited randomized studies on CDSS in AMS with mortality as a primary outcome, and these have focused on other surrogate outcome measures and have not reported on the nonexpert end-user workflow [20–22]. We studied mortality as the primary outcome, and our CDSS is integrated with clinical workflow as it is made available at the time of antibiotic prescription [14]. It provides recommendations for investigations and referrals in addition to antibiotics. We studied the implementation hurdles of our CDSS, patients’ and physicians’ predictors, and the psychosocial determinations of physicians’ acceptance of CDSS recommendations previously [11, 12, 14]. We then designed an educational campaign aimed to optimize the nonexpert end-user usage of our CDSS.
Our study did not evaluate the effect of CDSS on other antibiotics such as fluoroquinolones and third-generation cephalosporins. Between clusters, there may be contamination, possible Hawthorne effect, and bias between the study arms due to doctors’ rotations, doctors managing patients in both study arms concurrently on different wards, and patient transfers. We adopted a crossover design to adjust for these effects [15], as CDSS and PRF were considered standard of care at our hospital and were recommended by AMS guidelines [1]. We were unable to introduce a washout period before crossover or have a standalone CDSS or PRF study arm. Hence, we were not able to fully address effects of CDSS on mortality because of the concurrent use of PRF. COMPASS, a cluster randomized controlled trial focusing only on CDSS use, is ongoing. However, the primary outcome is overall antibiotic use [23]. Further studies are needed to evaluate the effects of CDSS on AMS and mortality.
CONCLUSIONS
Voluntary CDSS for piperacillin-tazobactam and carbapenem prescriptions with PRF did not result in differing clinical outcomes, antibiotic duration or length of stay, or PRF recommendations compared with compulsory CDSS and PRF. However, in geriatric medicine patients, where appropriateness of antibiotics was lower, compulsory CDSS with PRF resulted in lower length of stay and overall hospitalization cost.
Supplementary Data
Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Acknowledgments
We thank members of the data safety monitoring board, Tan Tock Seng Hospital antimicrobial stewardship team, and our information technology partner, Integrated Health Information Systems.
Financial support. This work was supported by the National Medical Research Council, Ministry of Health, Singapore, Singapore (CNIG14MAY005, CS-IRG NIG).
Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
Author contributions. T.M.N., D.C.L., B.Y., J.W., H.L.T., and T.W.L. contributed to the study design. H.L.T., S.H.T., M.Y.Y., C.B.T., B.A., D.C.L., and T.H.L. were part of the prospective review and feedback team. S.T.H. and J.W. did the statistical analysis. S.T.H. collected all the data. T.M.N. provided the first draft of the manuscript, and all authors contributed to the final manuscript.
References
- 1. Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis 2016; 62:e51–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Davey P, Marwick CA, Scott CL, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev 2017; 2:CD003543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Lew KY, Ng TM, Tan M, et al. Safety and clinical outcomes of carbapenem de-escalation as part of an antimicrobial stewardship programme in an ESBL-endemic setting. J Antimicrob Chemother 2015; 70:1219–25. [DOI] [PubMed] [Google Scholar]
- 4. Teng CB, Ng TM, Tan MW, et al. Safety and effectiveness of improving carbapenem use via prospective review and feedback in a multidisciplinary antimicrobial stewardship programme. Ann Acad Med Singapore 2015; 44:19–25. [PubMed] [Google Scholar]
- 5. Schuts EC, Hulscher MEJL, Mouton JW, et al. Current evidence on hospital antimicrobial stewardship objectives: a systematic review and meta-analysis. Lancet Infect Dis 2016; 16:847–56. [DOI] [PubMed] [Google Scholar]
- 6. Howard P, Pulcini C, Levy Hara G, et al. ; ESCMID Study Group for Antimicrobial Policies (ESGAP); ISC Group on Antimicrobial Stewardship An international cross-sectional survey of antimicrobial stewardship programmes in hospitals. J Antimicrob Chemother 2015; 70:1245–55. [DOI] [PubMed] [Google Scholar]
- 7. Dickstein Y, Nir-Paz R, Pulcini C, et al. Staffing for infectious diseases, clinical microbiology and infection control in hospitals in 2015: results of an ESCMID member survey. Clin Microbiol Infect 2016; 22:812.e9–17. [DOI] [PubMed] [Google Scholar]
- 8. Yong MK, Buising KL, Cheng AC, Thursky KA. Improved susceptibility of gram-negative bacteria in an intensive care unit following implementation of a computerized antibiotic decision support system. J Antimicrob Chemother 2010; 65:1062–9. [DOI] [PubMed] [Google Scholar]
- 9. Thursky K. Use of computerized decision support systems to improve antibiotic prescribing. Expert Rev Anti Infect Ther 2006; 4:491–507. [DOI] [PubMed] [Google Scholar]
- 10. Leibovici L, Kariv G, Paul M. Long-term survival in patients included in a randomized controlled trial of TREAT, a decision support system for antibiotic treatment. J Antimicrob Chemother 2013; 68:2664–6. [DOI] [PubMed] [Google Scholar]
- 11. Chow AL, Lye DC, Arah OA. Mortality benefits of antibiotic computerised decision support system: modifying effects of age. Sci Rep 2015; 5:17346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Chow A, Lye DC, Arah OA. Psychosocial determinants of physicians’ acceptance of recommendations by antibiotic computerised decision support systems: a mixed methods study. Int J Antimicrob Agents 2015; 45:295–304. [DOI] [PubMed] [Google Scholar]
- 13. Ng TM, Phang VY, Young B, et al. Clinical impact of non-antibiotic recommendations by a multi-disciplinary antimicrobial stewardship team. Int J Antimicrob Agents 2017; 50:166–70. [DOI] [PubMed] [Google Scholar]
- 14. Chow AL, Ang A, Chow CZ, et al. Implementation hurdles of an interactive, integrated, point-of-care computerised decision support system for hospital antibiotic prescription. Int J Antimicrob Agents 2016; 47:132–9. [DOI] [PubMed] [Google Scholar]
- 15. van Duijn PJ, Bonten MJ. Antibiotic rotation strategies to reduce antimicrobial resistance in gram-negative bacteria in European intensive care units: study protocol for a cluster-randomized crossover controlled trial. Trials 2014; 15:277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Chow AL, Lye DC, Arah OA. Patient and physician predictors of patient receipt of therapies recommended by a computerized decision support system when initially prescribed broad-spectrum antibiotics: a cohort study. J Am Med Inform Assoc 2016; 23:e58–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Balk RA. Systemic inflammatory response syndrome (SIRS): where did it come from and is it still relevant today? Virulence 2014; 5:20–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Magiorakos AP, Srinivasan A, Carey RB, et al. Multidrug-resistant, extensively drug-resistant and pandrug-resistant bacteria: an international expert proposal for interim standard definitions for acquired resistance. Clin Microbiol Infect 2012; 18:268–81. [DOI] [PubMed] [Google Scholar]
- 19. Reich NG, Myers JA, Obeng D, et al. Empirical power and sample size calculations for cluster-randomized and cluster-randomized crossover studies. PLoS One 2012; 7:e35564. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Rawson TM, Moore LSP, Hernandez B, et al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately? Clin Microbiol Infect 2017; 23:524–32. [DOI] [PubMed] [Google Scholar]
- 21. Carracedo-Martinez E, Gonzalez-Gonzalez C, Teixeira-Rodrigues A, et al. ; Galician Pharmacoepidemiology Research Group Computerized clinical decision support systems and antibiotic prescribing: a systematic review and meta-analysis. Clin Ther 2019; 41:552–81. [DOI] [PubMed] [Google Scholar]
- 22. Curtis CE, Al Bahar F, Marriott JF. The effectiveness of computerised decision support on antibiotic use in hospitals: a systematic review. PLoS One 2017; 12:e0183062. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Catho G, De Kraker M, Waldispühl Suter B, et al. Study protocol for a multicentre, cluster randomised, superiority trial evaluating the impact of computerised decision support, audit and feedback on antibiotic use: the COMPuterized Antibiotic Stewardship Study (COMPASS). BMJ Open 2018; 8:e022666. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.