Integrated antibiotic clinical decision support system (CDSS) for appropriate choice and dosage: an analysis of retrospective data

Abstract

Introduction

Decision-making for inpatient antibiotic prescribing is complex due to many considerations to be taken. So far, clinical decision support systems (CDSS) have been rarely used in antibiotic stewardship (ABS) and even less integrated in computerized physician order entry systems (CPOE).

Methods

We developed a guideline-based, CPOE-integrated CDSS (ID ANTIBIOTICS) to support antibiotic selection and dosing. We compared routine antibiotic inpatient prescribing data with CDSS-generated recommendations in the initial antibiotic selection, the duration of therapies, and costs. Finally, we assessed possible benefits of the CDSS by its performance in German ABS-guideline quality indicators (ABS-QIs).

Results

The requirements of several ABS-QIs can be supported with ID ANTIBIOTICS: electronic local guidelines, electronic decision-support, renal dosage adjustments, local guideline-based initial selection (all not quantified), and therapy durations for the treatment of pneumonia (significantly) without increasing costs. Performance in ABS-QIs for extensive therapies for community-acquired pneumonia could be improved with the CDSS by 20.2% (OR 0.134; 95% CI: 0.101-0.178); for hospital-acquired pneumonia by 3.7% (OR 0.742; 95% CI: 0.629-0.877). There was no difference in median daily drug costs between real-world prescriptions and CDSS recommendations (both: € 4.78, p=0.081).

Conclusions

In retrospective analyses, antibiotic CDSS can show possible performance in antibiotic stewardship through quality indicators (ABS-QIs). Further research and pilot testing of the software are needed to provide more insights into ABS-QI evaluation, user acceptance, and real-world effectiveness. Deep integration of antibiotic CDSS into existing medication processes without using multiple systems could contribute to the necessary acceptance of clinical practitioners.

Introduction

An estimated 50% of antibiotics are prescribed inappropriately.¹ Multiple, complex parameters contribute to the decision on prescribing the most suitable antibiotic drug.^2,3 Furthermore, patient-individual medication management is essential to minimize contraindications, side effects, and interactions to ensure appropriate antibiotic therapy, including dosing in renal impairment.^4,5 Almost a quarter of adverse drug reactions are antibiotic side effects.² Besides, non-adherence to antibiotic guidelines is associated with higher mortality,⁶ a longer length of stay,⁷ higher costs,⁸ and a higher likelihood of inappropriate therapy and broad-spectrum antibiotics.⁹ In turn, unnecessary untargeted treatment is associated with enhancing resistances and results in fewer available reserve antibiotics. Hence, antibiotic stewardship (ABS) programs are increasingly used to sustainably optimize and ensure rational antibiotic prescribing in hospitals.¹⁰ For this purpose, the German ABS-guideline describes quality indicators (ABS-QIs), which differentiate between structural indicators and process indicators.¹¹

Structural indicators include "a multidisciplinary team of infectiologists, pharmacists, microbiologists, ABS experts," or the "electronic availability of in-house guidelines" or a Clinical Decision Support System (CDSS) for antibiotic prescribing. Process factors include the "Requirement of a blood culture at the start of a calculated therapy" or specific maximum durations of therapy (DOT) for pneumonia. A maximum of seven days for community-acquired pneumonia (CAP) and ten days for hospital-acquired pneumonia (HAP) is indicated. Our focus was put on the indicators for "initial therapies (substances, dosage) according to local/national guidelines".¹¹

Enormous efforts can arise from implementing these ABS-QIs and measuring their performances.¹² Appropriate software support on pathogen statistics and Computerized Physician Order Entry (CPOE) are essential for implementing ABS measures and assessing ABS-QIs.¹³ Due to the shortage of professional ABS staff, easy-to-implement measures are needed, especially for hospitals with smaller budgets. Therefore, preference should be given to widely useable support methods like CDSSs, which are playing an emerging role in ABS.^1,14

Antibiotic CDSSs showed reductions in antibiotic consumption overall and for certain antibiotics, like carbapenems. Likewise, they reduced mortality, length of stay, and costs, albeit inconsistently. However, the initial antibiotic therapy was more likely to be appropriate and guideline-adherent for most studies.^1,14

Nevertheless, many guideline-based CDSSs seem to be stand-alone applications and not integrated directly into the electronic medication prescribing process or CPOEs, nor using patient data from hospital information systems (HIS). Consequently, these are fundamental reasons why prescribing workflows on normal wards do hardly involve antibiotic CDSSs so far. Point-of-care electronic documentation with HIS and CPOE represents the workflow of normal wards nowadays in many places. However, workflow integration with point-of-care availability would contribute to higher acceptance and higher quality of antibiotic CDSSs.

Generally, specific factors with mutual influences should be considered when introducing new software into healthcare, as described by Yusof, Kuljis¹⁵ in the HOT-fit framework.¹⁵ Technology aspects, including system quality, information quality, and service quality, must fit into the human and organizational environment to convert the technology's advantages into a net benefit. However, the net benefit also influences the other dimensions, e.g., the system usage and user satisfaction as human factors.¹⁵ With similar considerations, new ABS measures and ABS-QIs require initial pilot testing before implementation.¹²

We intended to develop an entirely CPOE-integrated CDSS for normal wards focusing on initial antibiotic choice for prescription, including recommendations for dosage, duration of therapy, and an integrated drug safety check. We aimed to assess possible benefits of the CDSS on antibiotic stewardship through its performance in quality indicators (ABS-QIs).

Methods

Development and implementation of the CDSS

ID ANTIBIOTICS was designed as a rule-based CDSS with a web-based user interface integrated into the existing CPOE ID MEDICS^® (Figure 1). We extracted recommendation information from the local antiinfective guideline of University Medical Center Hamburg-Eppendorf (UKE),¹⁶ and created a standardized knowledge base. Therefore, we mapped the used terms for diagnoses, severities, and risk factors onto the Wingert nomenclature, an ontology contained in ID MACS^®, which allows an interpretation of such terms and the processing within our rules engine. Furthermore, routine data from the patient health record is being used to apply the knowledge from the CDSS to the specific patient. The ontology contains not only diagnosis taxonomies but also resistance patterns extracted from the drug's Summary of Product Characteristics.

Figure 1. ID ANTIBIOTICS: user entry, data source, program features, and program output.

CPG – clinical practice guideline; CPOE – computerized physician order entry; HIS – hospital information system; HL7 – health level seven; SPC – summary of product characteristics; XML – extensible markup language.

• –
  User entry: suspected infection, severity, and risk factors are necessary user entries, besides the selection of an antibiotic drug and confirmation of dosage and therapy length.
• –
  Data sources are available patient-specific data, e.g., diagnoses and constant data in ID ANTIBIOTICS like guideline information stored in the knowledge base.
• –
  Program features show the CDSS-internal processes and the connections to other systems like the CPOE: ID MEDICS^®
• –
  The resulting program output is an antibiotic therapy suggestion, including dosage, therapy length, and costs.

A specific combination of diagnoses, severities, and risk factors are prerequisites for recommendations from the knowledge base, leading to recommended substances for prescription, including first-/second-/third-line priority, duration of therapy, and individualized dosage. Dosage information was integrated into a separate dosage database named ID TheraOpt^®.

Microbiological findings, e.g., antibiograms, communicated from the HIS or the Laboratory Information System (LIS) via HL7 messages (Health Level Seven Standard), are continuously saved in XML-based data containers (Extensible Markup Language) that already comprise all patient information used in ID MEDICS^®.

CDSS functionality

ID ANTIBIOTICS combines the patient's microbiological findings with recommendations generated by the knowledge base offering a ranking of the antibiotic suggestions. Antibiotic substances showing resistance or intermediate resistance are sorted out, while substances with susceptibility are ranked higher. Without resistance testing, standard resistance information of the detected pathogen is used.

Equally susceptible substances are ranked according to guideline priority (first-line vs. second-line therapy), reserve antibiotic status, daily drug costs, indication check, and standard resistances (Figure 1). Patient-specific alerts of the integrated drug therapy safety CDSS ID PHARMA CHECK^® for contraindications, allergies, and interactions enhance the recommendations.

We developed and evaluated ID ANTIBIOTICS’ general design and modeling involving clinical physicians and pharmacists. The generated recommendations are by design guideline-based. Therefore, they reflect the manually specified, human-made, ABS-curated in-house guideline. Thus, the evaluation to date focused on the retrospective analysis presented in this publication.

Analysis using retrospective data

The analysis was performed as a retrospective proof-of-concept study (POC) for the CDSS using existing electronic prescribing data from UKE's CPOE ID MEDICS^®. The primary outcomes were the performances in ABS-QIs.¹¹ Secondary outcomes involved whether the CDSS can prevent unnecessary use of antibiotics by shortening the duration of therapy (DOT) and whether costs can be saved.

Anonymized data of initially 738,467 patient cases from July 2015 to February 2017 (585 days) was used (Figure 2). We included cases with an antibiotic prescription (ATC J01*) and a guideline-listed infection: urinary tract infections (UTI), community-acquired pneumonia (CAP), hospital-acquired pneumonia (HAP), ear, nose, and throat infections (ENT), or acute exacerbations of COPD. We excluded cases with antibiotics that were unavailable before the guideline's publication and cases assignable to multiple recommendations. A total of 3,279 cases with 3,664 infections and 6,651 antibiotic prescriptions resulted (Figure 2). The integrated UKE antiinfective guideline was not updated or changed during the study period. Every change or update of guidelines would have led to a necessary knowledge base update.

Figure 2. Flowchart of patient cases.

Case selection for retrospective analysis, also serving as control cases.

*Cases with non-guideline diagnoses, duplicated diagnoses in documentation, and cases with more than one recommendation were excluded.

After patient selection, a retrospective application of ID ANTIBIOTICS was performed to classify the cases into clinical situations based on documented infection diagnoses, severities, and risk factors. Antibiotic recommendations with DOTs for these situations were generated and compared to real-world antibiotic prescriptions of the identical cases, serving as the study's controls. As these controls were the identical cases as for the retrospective intervention, the two groups had identical demographic characteristics: the median age was 67 years (IQR 51-76 years), 56.4% were female, and the median length of stay was 9.00 days (IQR 4.32-18.09). Table 1 shows all suggested substances and their guideline-based classification to first-, second-, and third-line and substances recommended for a possible sequence therapy. The difference between discontinuation time and the prescription start was used as DOT of real-world prescriptions; CDSS DOTs were obtained directly from recommendations. We compared ABS-QI performances¹¹ for not exceeding maximum therapy durations for HAP and CAP between real-world prescriptions and CDSS recommendations. In addition, we calculated actual costs using minimum and maximum prices, compared them with calculated costs of CDSS suggestions, and compared median drug daily costs DOT-independently. Finally, we summarized whether ID ANTIBIOTICS supports the performance in selected ABS-QIs applicable for software and the functions of ID ANTIBIOTICS (10 out of 46 of the German ABS-guideline) to show possible benefits.¹¹

Table 1.

Antibiotic prescriptions generated by ID ANTIBIOTICS in total and per active ingredient and combination therapy, including classification to first/second/third line, as well as possible sequence therapy

ATC	Substance	1st-line [n]	1st-line [%]	2nd-line [n]	2nd-line [%]	3rd-line [n]	Sequence therapy [n]
J01CA04	Amoxicillin	0	0.0%	0	0.0%	0	0
J01CA12	Piperacillin	0	0.0%	23	0.4%	0	0
J01CE02	Phenoxymethylpenicillin	4	0.1%	0	0.0%	0	0
J01CR04	Sultamicillin	210	3.5%	7	0.1%	0	696
J01CR21	Ampicillin and sulbactam	1415	23.5%	773	12.8%	0	0
J01CR25	Piperacillin and tazobactam	645	10.7%	132	2.2%	0	0
J01DC02	Cefuroxime	782	13.0%	245	4.0%	0	696
J01DD04	Ceftriaxone	2170	36.1%	1	0.0%	0	0
J01DD13	Cefpodoxime	0	0.0%	121	2.0%	0	0
J01DH02	Meropenem	0	0.0%	645	10.7%	0	0
J01EE01	Sulfamethoxazole and trimethoprim	0	0.0%	121	2.0%	0	0
J01FA09	Clarithromycin	0	0.0%	3	0.1%	0	0
J01MA02	Ciprofloxacin	139	2.3%	137	2.3%	0	0
J01MA14	Moxifloxacin	16	0.3%	3085	50.9%	0	0
J01XE01	Nitrofurantoin	121	2.0%	0	0.0%	0	0
J01XX01	Fosfomycin	121	2.0%	0	0.0%	0	0
J01DD02 + J01MA02	Ceftazidime + ciprofloxacin	129	2.1%	0	0.0%	0	0
J01CR25 + J01MA02	Piperacillin/tazobactam + ciprofloxacin	3	0.1%	129	2.1%	0	0
J01DD02 + J01GB03	Ceftazidime + gentamicin	0	0.0%	129	2.1%	0	0
J01DH02 + J01MA02	Meropenem + ciprofloxacin	0	0.0%	132	2.2%	0	0
J01DH02 + J01XX01	Meropenem + fosfomycin	0	0.0%	0	0.0%	129	0
J01CR25 + J01XX01	Piperacillin/tazobactam + fosfomycin	0	0.0%	0	0.0%	129	0
J01DD04 + J01FA09	Ceftriaxone + clarithromycin	34	0.6%	0	0.0%	0	0
J01CR25 + J01FA09	Piperacillin/tazobactam + clarithromycin	30	0.5%	0	0.0%	0	0
J01MA02 + J01CR04	Ciprofloxacin + sultamicillin	0	0.0%	329	5.4%	0	0
J01MA02 + J01GB03	Ciprofloxacin + gentamicin	1	0.0%	42	0.7%	0	0
J01DC02 + J01FA09	Cefuroxime + clarithromycin	4	0.1%	0	0.0%	0	0
J01CR21 + J01FA09	Ampicillin/sulbactam + clarithromycin	0	0.0%	4	0.1%	0	0
	No antibiotic therapy	191	3.2%	0	0.0%	0	0
	Total	6015	100%	6058	100%	258	1392

We performed SQL queries for patient selection with Microsoft SQL Management Studio 2014 / MS SQL Server 2014. ID ANTIBIOTICS' services were used for generating therapy recommendations in the same SQL database. We used IBM SPSS Statistics 25 to analyze the results, including statistical tests and boxplot graphs. All other steps were performed using Microsoft Excel 2016.

The Ethics Committee of the Medical Association in Hamburg confirmed our submission on 28.06.2017, saying that the conduct of the study was justifiable (processing no.: WF-037/17).

Results

Quality indicators (ABS-QIs)

ID ANTIBIOTICS with the integrated guideline¹⁶ meets the following ABS-QIs¹¹ (Table 2):

The structural indicators "local guidelines and ABS documents electronically available," "electronically available decision support for the use of antiinfectives according to locally consented guidelines," and "designation of standard versus special/reserve drugs in the house list" can be fulfilled with ID ANTIBIOTICS. The CDSS's structural design and ranked recommendations can mitigate the ABS-QI "use of selective antibiograms" if non-selective antibiograms are used.

For CAP and HAP, "initial therapy (substances, dosage) according to local/national guideline" could be achieved, and extensive therapies (HAP > ten days, CAP > seven days) could be reduced. With the CDSS, 87.5% ABS-QI compliance would be possible for HAP and 95.9% for CAP. The process indicator "dose adjustment in patients with impaired renal function within two days" is supported with the initial dosage suggestion, including adjustment for renal impairment. Furthermore, the integrated drug safety check shows alerts to prevent "[..] simultaneous administration of oral fluoroquinolones with multivalent cations" (among others) and therefore supports achieving this ABS-QI.

Table 2.

Performance in quality indicators for antibiotic stewardship (ABS-QIs) with the clinical decision support system (CDSS) ID ANTIBIOTICS

Quality indicator for antibiotic stewardship (ABS-QI)11	Structural or process indicator	Guideline authors11 ranking	Performance with ID ANTIBIOTICS and the integrated guidelines16
Local guidelines and ABS documents electronically available	Structural indicator	1	Supported
Electronically available decision support for the use of antiinfectives according to locally consented guidelines	Structural indicator	3	Supported
Designation of standard versus special/reserve drugs in the house list	Structural indicator	2	Supported*
Use of selective antibiograms (reduced reporting of findings adapted according to local guidelines)	Structural indicator	1	Partially supported**
Community-acquired pneumonia (CAP)
Initial therapy (substances, dosage) according to local/national guideline	Process indicator	1	Supported
Duration of therapies not longer than seven days (normal ward patients)	Process indicator	1	Possible degree of performance: 95.9%
Hospital-acquired pneumonia (HAP)
Initial therapy (substances, dosage) according to local/national guideline	Process indicator	1	Supported
Duration of therapies not longer than ten days (normal ward patients)	Process indicator	1	Possible degree of performance: 87.5%
Antibiotic dosage and application
Dose adjustment in patients with impaired renal function within two days	Process indicator	2	Supported
No simultaneous administration of oral fluoroquinolones with multivalent cations	Process indicator	1	Supported+

ABS – antibiotic stewardship; ABS-QI – antibiotic stewardship quality indicators.

We used 10 ABS-QIs that were applicable for software and the functions of ID ANTIBIOTICS from a total of 46 ABS-QIs of the German ABS guideline.¹¹ Performance in ABS-QIs is only stated as “supported,” “partially supported,” or with a “possible degree of performance,” as the real-world performance of ID ANTIBIOTICS is still to be determined.

Definitions:

• -
  supported: this ABS-QI’s degree of performance was not quantified, but ID ANTIBIOTICS supports herewith
• -
  partially supported: this ABS-QI’s degree of performance was not quantified, but ID ANTIBIOTICS supports partly herewith
• -
  possible degree of performance: this ABS-QI’s degree of performance was quantified for ID ANTIBIOTICS

^*Supported with adaptation/configuration to individual hospital conditions.

^**Supported only partially because the use of selective antibiograms is not achieved, but the risk is mitigated with ontology integrated resistance patterns.

⁺Supported with the integrated drug safety check.

Duration of therapy (DOT)

ID ANTIBIOTICS generated 3,664 recommendations, resulting in 6,015 first-line, 6,058 second-line, 258 third-line antibiotics, and 1,392 sequence therapy substance recommendations (Table 1).

Real-world prescription's DOT was lower than suggestions of ID ANTIBIOTICS (median 3.11 days, IQR 1.03-6.50 vs. median 7.00 days, IQR 6-9). Although recommended therapies were longer, the CDSS could reduce extensive treatments for CAP by 20.2% and HAP by 3.7%. CDSS recommendations were significantly less likely to exceed seven days for CAP therapy and ten days for HAP than real-world prescriptions (CAP: OR 0.134, 95% CI: 0.101-0.178; HAP: OR 0.742, 95% CI: 0.629-0.877).

Costs

Results were inconsistent for summed up antibiotic drug costs and showed reductions of -184,377.32 € (-79.4%) or increases of +16,097.31 € (+13.7%) per year depending on scenarios and methods used. This extensive range and consistently longer DOTs for CDSS suggestions led us to DOT-independent calculations.

There was no difference in median daily antibiotic costs (both medians 4.78 €, Mann-Whitney, p=0.081) between real-world prescriptions (IQR 0.86-6.57 €) and CDSS suggestions (IQR 0.83-6.43 €) (Figure 3). In conclusion, median drug daily costs for antibiotics would not rise using ID ANTIBIOTICS, while a reduction possibility is given.

Figure 3. Median daily therapy costs for antibiotics in € between real-world prescriptions (median 4.78 €, IQR 0.86-6.57 €, n=6,651) and CDSS suggestions generated by ID ANTIBIOTICS (median 4.78 €, IQR 0.83-6.43 €, n=12,073, only 1st/2nd-line recommendations were used).

Comparison with Mann-Whitney showed no differences between the two groups (p=0.081).

• –
  Suggestions not to perform antibiotic therapy (0 €) were omitted.
• –
  For both groups, all minimum and maximum prices were used.
• –
  No analysis on cost-effectiveness has been performed on either group.

Discussion

Quality indicators (ABS-QIs)

ID ANTIBIOTICS meets important structural ABS-QIs for rational antibiotic prescribing¹¹ by design. In 69% (49-82%) of German hospital patients with impaired renal function, the process ABS-QI "dose adjustment […] within two days" was achieved.¹¹ ID ANTIBIOTICS meets this ABS-QI already during initial prescribing if laboratory values or renal status are available. With antibiotics being the most common group requiring dose adjustment,¹⁷ we see more potential to support this effectively with CDSS usage. However, the present retrospective study did not use dosage suggestions because a structured evaluation of ID MEDICS^® dosing regimens would have required a very high manual effort. Therefore, our focus was on initial antibiotic selection and prescription recommendation. Nevertheless, we consider this essential for ID ANTIBIOTICS' benefit assessment and encourage further research.

Various other process ABS-QIs are met with ID ANTIBIOTICS:

Oral fluoroquinolones should not be administered with multivalent cations to avoid decreased efficacy (process ABS-QI).¹¹ The integrated drug safety check generates alerts concerning interactions, contraindications, allergies, and other drug-related problems, helping users avoid these issues upon prescribing. The present analysis did not determine how often users would ignore these alerts in reality and prescribe interacting drugs simultaneously. Nevertheless, it can be assumed that CDSSs with explicit alerts could increase ABS-QI compliance, which has an actual fulfillment rate of 68% (41-80%) in German hospitals.¹¹ Other drug-related issues – not indicated as ABS-QIs – could presumably be prevented, too.

HAP's "duration of therapy should not exceed ten days" (process ABS-QI).¹¹ The UKE guideline specified for HAP "up to 3-5 days after improvement of clinical symptoms, no longer than 8-10 days" and, for more severe cases, "no longer than 8-15 days".¹⁶ Using the CDSS, HAP prescriptions over ten days reduced by 3.7% (OR 0.742; 95% CI: 0.629-0.877). Moreover, this 87.5% ABS-QI compliance would considerably improve Germany's current 64% (40-75%).¹¹

CAP's "duration of therapy should not exceed seven days (patients on normal wards)" (process ABS-QI).¹¹ The UKE guideline recommended "5-7 days" and up to "8-15 days".¹⁶ We found a reduction of 20.2% using CDSS-suggestions (OR 0.134; 95% CI: 0.101-0.178). Germany's current 40% (29-50%) ABS-QI-compliance¹¹ could be increased up to 95.9% with ID ANTIBIOTICS (status quo in UKE real-world prescriptions was 75.7%). However, the prerequisites for this – as for HAP – would be that users do not increase durations beyond the recommended maximum when initially prescribing and do not extend therapy in the course of treatment. An advantage of ID ANTIBIOTICS' direct prescribing into the CPOE is that limited durations can be specified from the outset. Consequently, the high likelihood of extensive antibiotic therapies persisting without continuing indication is reduced. For instance, Vaughn, Flanders¹⁸ showed that two-thirds of antibiotic therapies for pneumonia are administered extensively. These extensive therapies did not show worse outcomes (death, re-hospitalization, a.o.) but were 5% more likely to show antibiotic-associated adverse events.¹⁸

The result on the ABS-QI "initial therapy (substances, dosage) according to local/national guideline" for pneumonia (HAP / CAP) cannot be considered as valuable as expected because the local UKE guideline was the sole basis for the knowledge base. Furthermore, it was unclear whether the diagnosis was documented immediately for many cases. In future evaluations, it will be essential to examine whether the detection of clinical situations by the CDSS meets the requirements of the guideline.

Duration of therapy (DOT)

Although long therapies for pneumonia were less likely in our data, medians of DOTs were significantly higher in CDSS-suggestions. We assume that reasons could include: 1) continuing outpatient therapies after discharge (24.3% of inpatient antibiotic prescriptions lasted at least until discharge); 2) switching to alternative substances due to low efficacy of initial antibiotics (50.6% of cases received more than one, 26.0% more than two, 11.9% more than three antibiotics); 3) de-escalating of efficacious initial antibiotics to more targeted antibiotics with narrower spectrum.

In addition, retrospective investigations with CPOE data based only on the duration of therapy have the following weaknesses: they do not sufficiently take into account loading doses, additional single doses, or doses at daily intervals (e.g., every two days), the last two of which tend not to occur with antibiotics. Likewise, paused therapy regimens would be entirely detected if no actual discontinuation of the prescription was documented in the CPOE. Furthermore, we did not investigate if different infection severities affect DOTs, e.g., for CAP, as we focused on the ABS-QI for extensive therapies.

Although shorter antibiotic therapy durations generally are favored to reduce selection pressure on resistance development in bacteria, searching for clinical evidence can be challenging.¹⁹ For us, likewise, it was impossible to investigate the effects on antibiotic resistance or the quality of care. Furthermore, it was impossible to foresee actual therapy durations using the CDSS in a prospective setting.

Altogether, the documentation quality of the routine data was a limiting factor, and an evaluation in a realistic setting with up-to-date guidelines is necessary. As mentioned above, the HOT-fit framework¹⁵ supports this thesis: the present retrospective study performed testing only in the dimensions of system quality, information quality, and service quality, i.e., exclusively within the "technology" domain. However, system use and user satisfaction ("human" domain), and structure and environment within the "organization" are essential for successful implementation and net benefit evaluation. Usability tests can help to evaluate system use and user satisfaction initially.²⁰ However, the system's actual acceptance and its real use only get visible with all domains' interaction.²¹ Essential for a CDSS to fulfill ABS-QIs is high user acceptance and wide-ranging use.²² Therefore, benefit-risk evaluations should conclude every CDSS development with software piloting on a test ward.²⁰

Implementing ABS programs involves considerable costs and time for hospitals.²³ Although amortization and measure benefits usually appear quickly,²³ hospitals prioritize "low-hanging fruit" measures achieving higher benefits with less effort like prescription restrictions of specific antibiotics.²⁴ We consider this an important reason for ID ANTIBIOTICS still missing pilot testing.

Costs

Our analysis showed that ID ANTIBIOTICS' suggestions would not increase median daily antibiotic drug costs (both medians: € 4.78 per day, no significant differences, p=0.081). Although not significant, the summed up costs of all antibiotic prescriptions showed a slightly decreasing tendency. Guideline-adherent antibiotic treatment⁸ and antibiotic CDSSs with good user acceptance^1,14 are associated with lower costs. Evans, Pestotnik²⁵ showed increasing antibiotic costs and length of stay when CDSS recommendations were ignored while following the recommendations resulted in a decrease.²⁵

Our approach can be considered an approximation for cost change with guideline-based, CDSS-supported antibiotic therapy. However, this does not allow a statement on actual costs or the quality of care since we could not perform any cost-effectiveness analysis due to the retrospective study design. Nevertheless, we showed that using ID ANTIBIOTICS would not be more expensive than prescribing without clinical decision support. That should encourage further investigations on total costs, as costs arising from inappropriate treatment often exceed direct drug costs.

Relevance and further limitations

Finally, many positive outcomes of antibiotic CDSSs can be found in the literature: Neugebauer, Ebert²⁶ showed that adequate UTI therapy through using a CDSS is possible. The CDSS was superior to conventional information sources and showed significant improvements in diagnosis and therapy recommendations without increasing prescribing time; such investigations are still to be done for ID ANTIBIOTICS.²⁶

Curtis, Al Bahar¹⁴ and Rittmann and Stevens¹ showed reductions in mortality, prescription amount, resistance, and increases in guideline adherence and adequate antibiotic selection related to the detected pathogen. Positive effects on costs – as shown retrospectively with ID ANTIBIOTICS – were often present, though not consistently demonstrable.¹⁴ The levels of integration into existing workflows and decision support vary widely among different CDSSs: 42.2% of CDSSs provided specific recommendations; only 6.7% allowed direct CPOE prescriptions.¹ Nevertheless, there is a generally positive benefit assessment on antibiotic CDSS. However, as the benefit of a CDSS depends on the supported process steps, its specific integration, and many other factors, we suggest that every individual system needs to show a real-world benefit on its own.

Due to the retrospective design, our analysis cannot disclose many essential aspects for evaluating a knowledge-based CDSS. Implementing a CDSS in a clinical environment and investigating impacts on clinical outcomes and the quality of patient care are among them. Furthermore, our selection of ABS-QIs was adapted explicitly to ID ANTIBIOTICS. Because of the retrospective data, it was impossible to evaluate ABS-QI performance in a real setting, i.e., how often the system affects the physicians' choice of antibiotic, dosage, duration of therapy, or resulting costs.

Nevertheless, we see a significant advantage of ID ANTIBIOTICS in its direct CPOE integration into ID MEDICS^®. The physician user can stay in his familiar environment for prescribing. The generated antibiotic suggestions are directly saved as prescriptions in the CPOE, so there is no need for user re-entry. Hence, ID ANTIBIOTICS accomplishes the requirement for integration in existing workflows, which is considered essential for hospital CDSS implementations.²¹

Conclusions

A knowledge-based decision support system - especially for antibiotic stewardship – is valuable if backed by helpful information and is practicable in use. It should recognize clinical situations specifically and sensitively and be easily integrated into the intended workflow. Ultimately, decision-maker's and user's acceptance based on demonstrable routine utility is essential.

Clinical decision support in antibiotic stewardship is still far from widespread use due to insufficient digitization and dissemination of CPOEs in hospitals and a lack of willingness to use CDSSs in practice. Further research on performing retrospective analyses with routine prescription data is needed to optimize the CDSS evaluation on a broader database.

This retrospective analysis does not replace actual patients' care evaluations not performed to date with ID ANTIBIOTICS. However, it was possible to show prospects and possible performances in quality indicators for antibiotic stewardship (ABS-QIs), e.g., less extended therapies for community-acquired and hospital-acquired pneumonia, with median daily antibiotic costs not increasing. Therefore, we encourage pilot testing to prove net benefit and usability in practice. This study's quality indicators and other parameters like adequate dosing and optimized drug therapy safety could serve as target variables for future investigations. Further research is required to generate evidence that antibiotic CDSS and comparable IT solutions can support clinical staff in their decision to improve the quality of antibiotic prescribing.