Skip to main content
Journal of the American Medical Informatics Association: JAMIA logoLink to Journal of the American Medical Informatics Association: JAMIA
. 2019 May 11;26(10):1010–1019. doi: 10.1093/jamia/ocz057

Helping GPs to extrapolate guideline recommendations to patients for whom there are no explicit recommendations, through the visualization of drug properties. The example of AntibioHelp® in bacterial diseases

Rosy Tsopra 1,2,, Karima Sedki 3, Mélanie Courtine 3, Hector Falcoff 4, Antoine De Beco 4, Ronni Madar 1, Frédéric Mechaï 6,7, Jean-Baptiste Lamy 3
PMCID: PMC7647204  PMID: 31077275

Abstract

Introduction

Clinical decision support systems (CDSS) implementing clinical practice guidelines (CPGs) have 2 main limitations: they target only patients for whom CPGs provide explicit recommendations, and their rationale may be difficult to understand. These 2 limitations result in poor CDSS adoption. We designed AntibioHelp® as a CDSS for antibiotic treatment. It displays the recommended and nonrecommended antibiotics, together with their properties, weighted by degree of importance as outlined in the CPGs. The aim of this study was to determine whether AntibioHelp® could increase the confidence of general practitioners (GPs) in CPG recommendations and help them to extrapolate guidelines to patients for whom CPGs provide no explicit recommendations.

Materials and Methods

We carried out a 2-stage crossover study in which GPs responded to clinical cases using CPG recommendations either alone or with explanations displayed through AntibioHelp®. We compared error rates, confidence levels, and response times.

Results

We included 64 GPs. When no explicit recommendation existed for a particular situation, AntibioHelp® significantly decreased the error rate (−41%, P value = 6x10−13), and significantly increased GP confidence (+8%, P value = .02). This CDSS was considered to be usable by GPs (SUS score = 64), despite a longer interaction time (+9–22 seconds). By contrast, AntibioHelp® had no significant effect if there was an explicit recommendation.

Discussion/Conclusion

The visualization of weighted antibiotic properties helps GPs to extrapolate recommendations to patients for whom CPGs provide no explicit recommendations. It also increases GP confidence in their prescriptions for these patients. Further evaluations are required to determine the impact of AntibioHelp® on antibiotic prescriptions in real clinical practice.

Keywords: clinical decision support system, clinical practice guidelines, antibiotics, infectious diseases, primary care, visualization

BACKGROUND AND SIGNIFICANCE

Inappropriate antibiotic prescriptions may have harmful consequences, such as increasing antimicrobial resistance,1 or causing patient complications—or even death.2 It also has an impact on health care system costs.2

The prescription of antibiotics is particularly complex in primary care because general practitioners (GPs) usually have to make decisions empirically, that is, without knowing the identity of the causal agent.3,4 They also have to consider other factors such as bacterial resistance rates5 and patient preferences.6

National health authorities provide narrative clinical practice guidelines (CPGs) to help GPs to prescribe the most appropriate antibiotic.7 CPGs include evidence-based recommendations in the form of “clinical situation/antibiotic” associations, which are then implemented in clinical decision support systems (CDSS) in the form of “If/then” rules8,9 (eg, “If acute uncomplicated cystitis, then prescribe fosfomycin trometamol”).3 However, despite the existence of CDSS, 30% of antibiotic prescriptions for outpatients remain inappropriate.10–12 This may be due to a lack of GP confidence in the recommendations13 and the difficulties GPs face when trying to extrapolate the recommendations to real patients.14,15 Indeed, CPGs provide no explicit recommendations for many clinical situations,16 and GPs must then combine knowledge from multiple sources and their own experience to guide their decisions.

We designed a CDSS, AntibioHelp®, to overcome these shortcomings. AntibioHelp® displays both the recommended and nonrecommended antibiotics, together with their properties, weighted by degree of importance. We hypothesized that the visualization of these weighted antibiotic properties would help GPs to have greater confidence in the recommendations due to a better understanding of the underlying rationale and extrapolate the recommendations to patients for whom CPGs provide no explicit recommendations but only partial guidance. For example, CPG experts recommend fosfomycin trometamol in uncomplicated cystitis in women because it has the following properties: 1) it is not contraindicated, 2) it is microbiologically active against Escherichia coli, and 3) it causes few adverse effects. If fosfomycin cannot be prescribed (eg, due to contraindications), the GP could consider another antibiotic with similar properties, such as pivmecillinam. Furthermore, it has been shown in other studies that visualization is a potentially valuable approach to decision support.17

The aim of this study was to evaluate whether an interface displaying the antibiotics and their weighted properties could increase the confidence of GPs in CPG recommendations and help GPs to extrapolate recommendations to patients for whom the CPGs provide no explicit recommendations.

Previous work - design of AntibioHelp®

We previously identified the properties used by CPG experts to decide which antibiotics to recommend18,19 and the degree of importance (or weight) of each property.20 This information is not explicit in CPGs. We first had to analyze the pieces of justifications occasionally given by CPG experts to explain their recommendations (eg, “fosfomycin should be preferred because it causes few side effects”). This analysis showed that experts establish their recommendations on the basis of 11 antibiotic properties (eg, “few side effects”).18,19 Some of these properties are necessary, and only antibiotics that satisfy these properties can be prescribed (eg, absence of contraindication). Others are preference properties, and antibiotics that satisfy these properties are preferred over those that do not (eg, antibiotics with few side effects are preferred over those with more side effects).

We built a knowledge base describing these 11 properties for 50 antibiotics, 11 infectious diseases, and 21 patient profiles.20 Each property is Boolean (true or false) and its value can also be unknown. The “true” value always corresponds to an advantage, that is, an argument in favor of the prescription of the antibiotic. The values depend on the antibiotic but also the causal bacteria, patient profile, and the infectious disease involved. The knowledge base was structured as an OWL 2.0 ontology, including 144 038 resource description framework triples describing 5696 classes, 19 properties, and 34 483 axioms. It belongs to the ALC(D) family of description logics.

For a given clinical situation (ie, a given infectious disease in a patient with a given profile), the antibiotics and their properties can be modeled as sets of elements: the antibiotics are the elements, and we can define sets for each value of each property (eg, the set of antibiotics that are contraindicated, the set of antibiotics with few adverse effects, and the set of antibiotics with many adverse effects). For each necessary property prop, we consider a single set, SetNecess, prop, including all antibiotics that do not satisfy this property (ie, the value is false or unknown). For each preference property prop, we consider 2 sets: SetPrefAdv, prop, including all antibiotics for which the property is true, and SetPrefDis, prop, including all antibiotics for which the property is false.

For the visualization of these sets, we used a recently developed technique for set visualization, called rainbow boxes.21–23 In the resulting interface (Figure 1), AntibioHelp®, all the antibiotics are displayed in columns, and the sets describing the properties are displayed in colored rectangular boxes. Each box covers the columns of the antibiotics belonging to a given set (ie, sharing a given value for a given property). When the columns covered by a box are not contiguous, holes are present in the box: the box is split into several parts connected by a thin horizontal line (eg, in Figure 1, the box labeled “Moderate efficacy” has 1 hole of size 2 corresponding to the columns “ceftriaxone” and “amoxicillin clavulanate”). Antibiotics are separated into 2 groups:

Figure 1.

Figure 1.

Visualization of the weighted drug properties with rainbow boxes. Example of otitis in a child over the age of 6 years, without allergy at the top and with allergy to beta-lactam and sulfonamide at the bottom.

  • The antibiotics on the right, with a dark gray column header, are those that should not be prescribed because they do not have the necessary properties. The sets SetNecess, prop, describing the necessary properties not satisfied, are displayed in light red boxes.

  • The antibiotics on the left, with a light gray column header, are those that could be prescribed because they have all the necessary properties. The sets SetPrefAdv, prop and SetPrefDis, prop describing the preference properties are displayed in a light green box if the property is advantageous (ie, true), or a light orange box if the property is disadvantageous (ie, false). No box is displayed if the value of the property is unknown. Box height is proportional to the weight attributed by the experts in the CPGs. These weights were learnt in a preference model,20 using a metaheuristic algorithm.24 For each preference property prop, 2 weights, WeightPrefAdv, prop and WeightPrefDis, prop, are considered for SetPrefAdv, prop and SetPrefDis, prop, respectively. The preference model ranks the antibiotics that have the necessary property. The model computes a score (the utility function u), corresponding to a linear combination of the property values for antibiotic a:
    u(a)=prop{WeightPrefAdv,propWeightPrefDis,prop0ifaSetPrefAdv,propifaSetPrefDis,propotherwise

Antibiotics with a higher score are ranked as better than those with a lower score. The metaheuristic algorithm finds the optimal values for WeightPrefAdv, prop and WeightPrefDis, prop, that is, those ranking the antibiotics in the same order as in the CPGs. Properties with higher weights are considered to be more important than those with lower weights. The preference model is generic and covers the 11 infectious diseases considered.

Table 1 lists the 11 properties, their type (necessary or preference) and their weights (for preference properties).

Table 1.

Properties used by CPG experts to determine which antibiotics to recommend. Preference properties have weights derived from a preference model learnt from CPGs with a metaheuristic algorithm (20). For each preference property prop, 2 weights, WeightPrefAdv, prop and WeightPrefDis, prop, were learnt for SetPrefAdv, prop (ie, the set including all antibiotics for which the property is true), and SetPrefDis, prop (ie, the set including all antibiotics for which the property is false), respectively

Properties Type WeightPrefAdv, prop WeightPrefDis, prop
Naturally active against the causal bacterium Necessary
Probably active against the causal bacterium Necessary
Clinical efficacy proven in the disease Necessary
Absence of contraindication for the patient Necessary
Convenient protocol Preference 4 7
Nonprecious class Preference 2 2
Absence of serious and frequent side effects Preference 2 5
High level of efficacy Preference 2 2
Narrow antibacterial spectrum Preference 2 2
Low level of ecological adverse effects Preference 2 3
Acceptable taste Preference 2 3

Abbreviation: CPG, clinical practice guideline.

In addition, AntibioHelp® displays:

  • The rank of recommendation of the antibiotic in the CPGs, indicated as a number displayed below the name of the antibiotic.

  • Detailed information about antibiotics and their properties is provided in an interactive information bubble on demand, when the cursor is moved over the name of the antibiotic (mouseover). Icons are also associated with each antibiotic property to facilitate the reading of the interface.

  • Brand names or generic names of antibiotics, which can be requested by ticking a checkbox at the bottom left of the screen.

  • The pharmacological class of each antibiotic, indicated by a brightly colored bar above the antibiotic name. At the bottom right of the screen, 6 checkboxes can be used to hide the antibiotics belonging to a given pharmacological class (eg, if the patient is allergic to the antibiotics of the class concerned).

The interface is generated automatically for all clinical situations modeled in the ontology. For example, Figure 1 displays the antibiotics and their properties for otitis in children over the age of 6 years. In this case, 6 antibiotics have all the necessary properties and could therefore be prescribed. At a glance, we can see that 1 of these antibiotics, amoxicillin, has the greatest total height of green boxes and the smallest total height of orange boxes. It is, therefore, the most appropriate antibiotic in this clinical situation. Furthermore, in the column header, we can see that this is the top-ranking antibiotic recommended in the CPGs.

This interface can also be helpful for extrapolating recommendations to patients for whom CPG recommendations do not apply due to contraindications or allergy (eg, for otitis in a child over the age of 6 years with allergies to beta-lactams and sulfonamide). In such cases, the GP can tick the checkboxes at the bottom of the screen to gray out the antibiotics belonging to the classes to which the patient is allergic (the beta-lactam and sulfonamide classes in this case). The GP can then see, at a glance, which antibiotics have the necessary properties and their preference properties and can, therefore, decide which antibiotic is the most appropriate. Here, only 1 antibiotic, pristinamycin, has all the necessary properties (Figure 1). This antibiotic is, therefore, the most appropriate.

MATERIALS AND METHODS

We investigated whether the visualization of weighted antibiotic properties via AntibioHelp® could increase GP confidence in CPG recommendations and help GPs to extrapolate the recommendations to patients for whom CPGs provided no explicit recommendations.

To achieve this goal, we asked GPs to respond to clinical cases on the basis of CPG recommendations either alone or with the assistance of AntibioHelp®. We tested both 1) clinical cases for which CPGs provided explicit recommendations, to determine whether the interface increased GP confidence; and 2) clinical cases for which CPGs provided no explicit recommendations to determine whether GPs extrapolated recommendations better with the interface. GPs were reminded of the CPG recommendations relating to the disease of the clinical case.

Evaluation design

We carried out a 2-stage crossover study in which the participating GPs were asked to respond to clinical cases presented in a random order with and without access to AntibioHelp® (Figure 2). The evaluation was performed online with Firefox or Chrome.

Figure 2.

Figure 2.

Diagram of the crossover study (SUS: System Usability Scale).

GP recruitment

Participants were contacted by e-mail via medical associations. For involvement in the study, the GPs had to be in training or practicing in primary care and they had to have an internet connection. The evaluation was free and anonymous.

Constitution of clinical cases

We first extracted 57 clinical situations from 2 CPGs for respiratory infections. We then randomly chose 2 clinical situations for set A, and 2 for set B. Randomization was stratified by type of disease (eg, otitis) and patient profile. For each clinical situation, we built 2 clinical cases: a simple clinical case for which a recommendation was provided in the CPGs and a complex clinical case for which there was no explicit recommendation. For example, for the clinical situation “otitis in a child over the age of 6 years,” we built 2 clinical cases: 1) “otitis in an 8-year-old child without allergy” for which a recommendation exists and 2) “otitis in an 8-year-old child with beta-lactam and sulfonamide allergy” for which no explicit recommendation was provided in the CPGs.

This process yielded 2 sets of clinical cases: A and B. Set A contained 2 simple and 2 complex cases of otitis and pneumonia. Set B contained 2 simple and 2 complex cases of sinusitis and pharyngitis.

For training, we also built 2 simple and 2 complex clinical cases of cystitis and pyelonephritis. The results for these cases are not analyzed here.

Crossover design

GPs were randomly assigned to 2 groups:

  • In group 1, GPs first considered 2 of the training cases and set A without AntibioHelp®, and they then considered the other 2 training cases and set B with AntibioHelp®.

  • In group 2, GPs first considered 2 of the training cases and set A with AntibioHelp®, and they then considered the other 2 training cases and set B without AntibioHelp®.

In both groups, when GPs had to answer without AntibioHelp®, they were shown a recap of the CPG recommendations relating to the disease concerned in the form of a simple table displaying the recommended antibiotics and their rank in the CPG recommendations (eg, uncomplicated acute cystitis → fosfomycin trometamol in rank 1).

After using AntibioHelp®, GPs completed the System Usability Scale (SUS). SUS is a 10-item scale for evaluating usability.25,26 For each item, the degree of agreement is assessed with a 5-point Likert scale. A total SUS score between 0 and 100 was then calculated.

At the end of the evaluation, GPs were asked whether they preferred to visualize the weighted drug properties, and they were invited to write optional comments in free text.

Measurement criteria

We measured 5 criteria:

  • Error rate was measured by determining the number of inaccurate responses for each clinical case. A gold standard was derived by a medical doctor (RT) and a specialist in antibiotics (FM). For simple clinical cases, the prescription of any antibiotic recommended in rank 1 in the CPG was considered correct. For complex clinical cases, the prescription of any antibiotic that could safely be used to treat the patient was considered correct.

  • Confidence level was measured on a 7-point scale27: totally unsure (2% certainty), unsure (10% certainty), doubtful (25% certainty), more or less sure (50% certainty), sure enough (75% certainty), sure (90% certainty), very sure (98% certainty).

  • Response time was measured as the time taken to give a response for the clinical case.

  • Perceived usability was measured by calculating the SUS Score.

  • Preference was measured by determining the proportion of GPs preferring to have access to AntibioHelp®.

Statistical analysis

The error rates were compared in Fisher’s tests. The impact of GP characteristics (such as age, sex, time in practice, last medical training in antibiotics, and frequency of external resource use), and their interaction with access to the interface (with/without AntibioHelp®) were assessed with a generalized linear model. Confidence levels and response times were compared in student’s t tests. R V.3.5.1 statistical software was used.

RESULTS

We included 64 GPs in the study: 59% were women, and almost half were under the age of 40 years and had been in practice for less than 20 years. More than half of the GPs declared using an external resource for antibiotic prescription more than once per week (Table 2).

Table 2.

Sociodemographic characteristics of the GPs

Group 1 (Percentage % (number)) Group 2 (Percentage % (number)) Total (Percentage % (number))
Sex
Male 13% (8/64) 28% (18/64) 41% (26/64)
Female 34% (22/64) 25% (16/64) 59% (38/64)
Age
20-29 years old 2% (1/64) 6% (4/64) 8% (5/64)
30-39 years old 23% (15/64) 14% (9/64) 38% (24/64)
40-49 years old 5% (3/64) 6% (4/64) 11% (7/64)
50-59 years old 8% (5/64) 9% (6/64) 17% (11/64)
Over 60 years old 9% (6/64) 17% (11/64) 27% (17/64)
Time in practice
In training 3% (2/64) 3% (2/64) 6% (4/64)
0-4 years 6% (4/64) 8% (5/64) 14% (9/64)
5-9 years 11% (7/64) 9% (6/64) 20% (13/64)
10-19 years 11% (7/64) 5% (3/64) 16% (10/64)
20-29 years 5% (3/64) 9% (6/64) 14% (9/64)
Over 30 years 11% (7/64) 19% (12/64) 30% (19/64)
Last medical training in antibiotics
Less than 1 year ago 9% (6/64) 6% (4/64) 16% (10/64)
Between 2 and 5 years ago 23% (15/64) 25% (16/64) 48% (31/64)
Between 6 and 10 years ago 8% (5/64) 14% (9/64) 22% (14/64)
Over 11 years ago 6% (4/64) 8% (5/64) 14% (9/64)
Frequency of external resource use for antibiotic prescriptions
Never 0% (0/64) 2% (1/64) 2% (1/64)
Rarely (<once/month) 2% (1/64) 3% (2/64) 5% (3/64)
Sometimes (1 to 3 times/month) 13% (8/64) 14% (9/64) 27% (17/64)
Frequently (>once/week) 19% (12/64) 20% (13/64) 39% (25/64)
Almost all the time 14% (9/64) 14% (9/64) 28% (18/64)

Abbreviation: GP, general practitioner.

Error rate

For simple clinical cases, the error rate was 19.5% without AntibioHelp® vs 20.3% with AntibioHelp®. This difference was not significant (P value = 1, Fisher’s test).

For complex clinical cases, the error rate was 91% without AntibioHelp® vs 50% with AntibioHelp®. This difference was significant (P value = 6x10−13, Fisher’s test).

The generalized linear model showed that 3 factors had a significant impact on error rate: 1) access to the interface (P = 6x10−6), 2) the complexity of the clinical case (P < 3x10−16), and 3) the interaction of these 2 factors (P = 3x10−7). These results are consistent with the results of Fisher’s exact tests. Other factors (age, sex, time in practice, last medical training in antibiotics, and frequency of external resource use) had no significant impact on error rate.

Thus, when no explicit recommendations were available, the visualization of the weighted antibiotic properties significantly improved antibiotic prescription by GPs (Figure 3).

Figure 3.

Figure 3.

Error rate for simple and complex clinical cases by type of disease. The error rate was 24%–38% higher for clinical cases relating to pneumonia and pharyngitis than for other diseases. The missing bar for otitis and sinusitis corresponds to an error rate of 0%.

GP confidence

For simple clinical cases, GP confidence level was 86% without AntibioHelp® vs 87% with AntibioHelp®. This difference was not significant (P value = .5, Student’s t test).

For complex clinical cases, GP confidence level was 63% without AntibioHelp® vs 71% with AntibioHelp®. This difference was significant (P value = .02, Student’s t test).

Thus, when no explicit recommendations were available, the visualization of the weighted antibiotic properties significantly improved GP confidence level (Figure 4).

Figure 4.

Figure 4.

Confidence level for simple and complex clinical cases by type of disease.

Response time

For simple clinical cases, the response time was 37 seconds without AntibioHelp®versus 46 seconds with AntibioHelp®. This difference was significant (P value = .0007, Student’s t test).

For complex clinical cases, the response time was 61 seconds without AntibioHelp®versus 83 seconds with AntibioHelp®. This difference was significant (P value = .001, Student’s t test).

Thus, the visualization of weighted antibiotic properties significantly increased the response time (Figure 5).

Figure 5.

Figure 5.

Response time for simple and complex clinical cases by type of disease.

Usability

The mean SUS score for AntibioHelp® was 64. The perceived usability of the interface was therefore “good,” according to the scale proposed by Bangor et al.28,29 The interface seemed to be easy to use, and no technical support was required for 77% of GPs (Figure 6).

Figure 6.

Figure 6.

Distribution of answers to each question of the SUS (System Usability Scale).

GP preferences and free comments

Overall, 58% of GPs reported preferring having access to the weighted antibiotic properties, vs 42% who preferred not to have such access.

Some GPs wrote free comments highlighting the usefulness of AntibioHelp® for the management of complex patients. It was reported by 1 GP that the interface could be used to study antibiotic properties. Another 2 explained that AntibioHelp® makes it easier to understand antibiotic choice and to take into account the clinical reasoning behind guidelines. Conversely, 1 GP said that in clinical practice, he/she only needed to see which antibiotic is recommended—not its properties. A few of the GPs said that they appreciated the content displayed (eg, bacterial susceptibility), whereas others found the tool too complex. Several GPs suggested that AntibioHelp® could be improved by adding antibiotic doses, the names of the causal bacteria, filters for pregnancy and renal failure, and by removing some antibiotics that were displayed but not used in primary care.

DISCUSSION

Our results show that the visualization of weighted antibiotic properties helped GPs to extrapolate recommendations to patients for whom CPGs provide no explicit recommendations. It also increased the confidence of GPs in their prescriptions for these patients. Conversely, the visualization of antibiotic properties had no significant effect when explicit recommendations were available. The presentation of weighted antibiotic properties in the form of rainbow boxes was considered usable by GPs despite the longer interaction time required (an additional 9–22 seconds).

Methodology used for evaluation

We conducted a rigorous study by randomly assigning GPs and clinical cases and conducting a 2-period crossover study. We also used 2 different sets of clinical cases to prevent memory effects (GPs beginning the second part of the study might recall the responses they gave during the first part of the study). Our study design was also subject to several limitations: 1) the GPs were quite young, probably because of the mode of evaluation (we carried out an online evaluation which may have selected for the youngest GPs who were the most comfortable with computers); 2) training in the use of AntibioHelp® was limited to an online tutorial, with no possibility of asking questions; 3) the study was carried out online, so the conditions of the evaluation were not, therefore, controlled. Nevertheless, we deliberately chose to perform a study of this type because it made it possible to include more GPs (GPs are more likely to participate in studies if they can perform the evaluation anytime and anywhere), and it was suitable for evaluation of prototypes under development.

Error rate

No significant difference in error rate was observed for simple clinical cases. This result was expected because CPGs provide clear recommendations for such cases, and the GPs were reminded of these recommendations even when they had no access to AntibioHelp®. By contrast, AntibioHelp® significantly decreased the error rate by 41% for complex clinical cases. For these cases, CPGs provided no explicit recommendations, and the visualization of weighted drug properties helped GPs to extrapolate the recommendations. Other studies on CDSS for antibiotic treatment have also reported significant improvements in antibiotic prescription with the use of a CDSS.30 However, most studies also considered the decision as to whether or not to prescribe antibiotics, whereas we focused exclusively on which antibiotic to prescribe. Furthermore, even with the visualization of weighted antibiotic properties, the error rate remained high for complex cases (50%). This high error rate may be due to 1) the complexity of the clinical cases for which the CPGs provided only partial guidance and/or 2) the complexity of the proposed visual interface. However, the results of the SUS test are not consistent with the second of these explanations: only 13% of GPs reported requiring technical assistance to use the interface, and only 31% found the interface unnecessarily complex.

GP confidence

We expected Antibiohelp® to increase GP confidence, but this effect was observed only for complex cases (+8%). For simple cases, GPs already had high levels of confidence in the recommendations of CPGs (86% without AntibioHelp®). This is quite surprising, because most studies have reported a lack of GP confidence in CPGs.13 However, for complex cases in which CPGs provided no explicit recommendation and only partial guidance, the visualization of weighted drug properties helped GPs to feel even more confident in their choice. Confidence in the CDSS is an important factor for increasing the chances of its adoption by GPs in clinical practice.31

Response time

Response times were longer when GPs had access to the interface: +9 seconds for simple cases and +22 seconds for complex cases. Given the large amount of information displayed on the interface, this time increase remains moderate. Furthermore, GPs used the interface for the first time during this study. Their response times would be expected to be shorter once they had become familiar with the interface. Short interaction times are an essential feature for ensuring the adoption of CDSS by GPs.32

Comparison with other CDSS

Most CDSS that are designed to improve antibiotic prescription in primary care33,34 display the list of therapeutic options recommended by CPGs for a given patient according to the scientific evidence. These CDSSs have several limitations: 1) updates are slow, as they depend on updates of the CPGs; 2) they provide no guidance for some patients; and 3) they do not provide the rationale underlying the recommendations displayed, which may hinder adoption of the CDSS. Here, we propose a new approach involving the design of a CDSS displaying both the list of therapeutic options recommended by CPGs and those not recommended. For each option, the interface also displays the antibiotic properties weighted by degree of importance as learnt from the CPGs. This approach has several advantages: 1) GPs can use the properties to extrapolate recommendations for patients for whom there are no explicit recommendations within CPGs, 2) GPs can understand the rationale underlying the recommendations displayed, and 3) GPs do not have to wait for CPGs to be updated to be informed of advances in medical knowledge. The visualization of weighted antibiotic properties provides instantaneous information, which could be updated automatically through external resources, such as microbiological observatories. For example, if the rate of E coli resistance to fosfomycin trometamol increases considerably, GPs could be informed by the visualization of a red box for the property “probably active.” This would make it easy for GPs to understand that this antibiotic should not be prescribed anymore, without having to wait a few years for the publication of a new CPG for GPs to be informed of this change. Furthermore, the presentation of weighted properties should encourage GPs to be involved in the decision process rather than to be passive and just accept the decision suggested by the CDSS. The preservation of GP autonomy is an important factor to be taken into account when designing CDSS.35

Perspectives

In the future, we aim to increase the usability of the “rainbow box” interface, by making use of existing guidelines in heuristics.36 We also aim to integrate this interface into the AntibioHelp® CDSS, which is designed to guide antibiotic prescription in primary care.37 Ideally, the CDSS should be connected to the patient’s electronic health record38,39 and would be activated every time the GP diagnoses an infectious disease. From the diagnosis and patient conditions encoded with medical terminologies (eg, ICD 10, SNOMED CT) in the electronic health record, the CDSS would be able to deduce the corresponding clinical situation and display the appropriate “rainbow box” interface if antibiotic prescription is recommended. However, because the coding of diagnosis is often delayed and not available at the time of prescription, we also plan to integrate AntibioHelp® into the electronic prescribing workflow (GPs would have to document diagnosis and some patient conditions at the time of prescription). We also aim to connect AntibioHelp® to microbiological observatories and drug databases to facilitate the automatic updating of weighted drug properties. This would provide local teams with the opportunity to manage the adaptation and updating of the CDSS themselves, according to local conditions. For example, we plan to adjust the properties relating to drug microbiological activity according to local bacterial susceptibilities.39,40 Likewise, we plan to extend the ontology with new antibiotic properties considered relevant by local experts (eg drug cost, drug reimbursement). The local adaptation of the CDSS may reinforce GP confidence in the system, and thereby increase its chances of adoption by GPs. This might also lead to a better adoption/respect by GPs of the recommendations provided by the CDSS.

Potential impacts

AntibioHelp® is a promising solution for improving antibiotic prescription in primary care, particularly for patients for whom there are no explicit recommendations within CPGs.30,41 It could have a positive impact on 1) antimicrobial resistance,42 2) patient safety,43 3) compliance (AntibioHelp® could be used in a shared decision-making process to explain the antibiotic choice to patients), and 4) health care system costs (fewer prescriptions of expensive broad-spectrum antibiotics44 and shorter hospital stays43). Further evaluations will be conducted in real clinical practice conditions to determine the impact of AntibioHelp® on these 4 aspects.45,46 However, AntibioHelp® would have a greater impact if disseminated as part of multi-faceted interventions (eg, education, training) at different levels (eg, health care professionals, the population) rather than on its own.

FUNDING

Funding was obtained from the Agence Nationale de Sécurité du Médicament et des Produits de santé (ANSM). APP 2016—RaMiPA Project (Raisonner pour Mieux Prescrire les Antibiotiques [Reasoning for a better antibiotic prescription]). Project leader: Dr Rosy Tsopra.

AUTHOR CONTRIBUTIONS

Design of the study protocol: RT, JBL

Building of the preference model: RT, JBL, KS

Building of the clinical cases and gold standard: RT, FM

Setting up of the study: RT, JBL, ADB, HF, RM

Statistics and data analysis: RT, JBL, MC

Writing of the first full version of the manuscript: RT

Agreement with all aspects of the work and approval of the final version for publication: RT, KS, MC, HF, ADB, RM, FM, JBL

ACKNOWLEDGMENTS

We thank all the GPs for taking the time to participate despite their high workload.

We thank SFTG (Société de Formation Thérapeutique du Généraliste) for supporting the study.

We thank Manon Cabal for designing the icons of AntibioHelp®.

We thank the anonymous reviewers for their valuable advice, which enabled us to improve the manuscript.

CONFLICT OF INTEREST STATEMENT

None declared.

REFERENCES

  • 1. Ventola CL. The antibiotic resistance crisis. P T 2015; 404: 277–83. [PMC free article] [PubMed] [Google Scholar]
  • 2. Davey PG, Marwick C.. Appropriate vs inappropriate antimicrobial therapy. Clin Microbiol Infect 2008; 14 (Suppl 3): 15–21. [DOI] [PubMed] [Google Scholar]
  • 3.Gupta K, Hooton TM, Naber KG, et al. International clinical practice guidelines for the treatment of acute uncomplicated cystitis and pyelonephritis in women: A 2010 update by the Infectious Diseases Society of America and the European Society for Microbiology and Infectious Diseases. Clin Infect Dis 2011;52:e103–120. doi:10.1093/cid/ciq257. [DOI] [PubMed]
  • 4.Woodhead M, Blasi F, Ewig S, et al. Guidelines for the management of adult lower respiratory tract infections–full version. Clin Microbiol Infect 2011;17 Suppl 6:E1–59. doi:10.1111/j.1469-0691.2011.03672.x. [DOI] [PMC free article] [PubMed]
  • 5. Pulcini C, Williams F, Molinari N, et al. Junior doctors’ knowledge and perceptions of antibiotic resistance and prescribing: a survey in France and Scotland. Clin Microbiol Infect 2011; 171: 80–7. [DOI] [PubMed] [Google Scholar]
  • 6. Wood F, Simpson S, Butler CC.. Socially responsible antibiotic choices in primary care: a qualitative study of GPs’ decisions to prescribe broad-spectrum and fluroquinolone antibiotics. Fam Pract 2007; 245: 427–34. [DOI] [PubMed] [Google Scholar]
  • 7. Woolf SH, Grol R, Hutchinson A, et al. Potential benefits, limitations, and harms of clinical guidelines. BMJ 1999; 3187182: 527–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Rawson TM, Moore LSP, Hernandez B, et al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately? Clin Microbiol Infect 2017; 238: 524–32. [DOI] [PubMed] [Google Scholar]
  • 9. Wanger P, Martin L.. Algorithms for optimizing drug therapy. BMC Med Inform Decis Mak 2004; 4: 10.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of Inappropriate Antibiotic Prescriptions Among US Ambulatory Care Visits, 2010-2011. JAMA 2016; 31517: 1864–73. [DOI] [PubMed] [Google Scholar]
  • 11. Linder JA, Stafford RS.. Antibiotic treatment of adults with sore throat by community primary care physicians: a national survey, 1989–1999. JAMA 2001; 28610: 1181–6. [DOI] [PubMed] [Google Scholar]
  • 12. Steinman MA, Gonzales R, Linder JA, et al. Changing use of antibiotics in community-based outpatient practice, 1991–1999. Ann Intern Med 2003; 1387: 525–33. [DOI] [PubMed] [Google Scholar]
  • 13. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 28215: 1458–65. [DOI] [PubMed] [Google Scholar]
  • 14. Farquhar CM, Kofa EW, Slutsky JR.. Clinicians’ attitudes to clinical practice guidelines: a systematic review. Med J Aust 2002; 1779: 502–6. [DOI] [PubMed] [Google Scholar]
  • 15. Carlsen B, Glenton C, Pope C.. Thou shalt versus thou shalt not: a meta-synthesis of GPs’ attitudes to clinical practice guidelines. Br J Gen Pract 2007; 57545: 971–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Bouaud J, Seroussi B, Falcoff H, et al. Consequences of the verification of completeness in clinical practice guideline modeling: a theoretical and empirical study with hypertension. AMIA Annu Symp Proc 2009; 2009: 60–4. [PMC free article] [PubMed] [Google Scholar]
  • 17. Greenes RA, Bates DW, Kawamoto K, et al. Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134–43. [DOI] [PubMed] [Google Scholar]
  • 18. Tsopra R, Venot A, Duclos C.. Towards evidence-based CDSSs implementing the medical reasoning contained in CPGs: application to antibiotic prescription. Stud Health Technol Inform 2014; 205: 13–7. [PubMed] [Google Scholar]
  • 19. Tsopra R, Venot A, Duclos C.. An algorithm using twelve properties of antibiotics to find the recommended antibiotics, as in CPGs. AMIA Annu Symp Proc 2014; 2014: 1115–24. [PMC free article] [PubMed] [Google Scholar]
  • 20. Tsopra R, Lamy J-B, Sedki K.. Using preference learning for detecting inconsistencies in clinical practice guidelines: methods and application to antibiotherapy. Artif Intell Med 2018; 89: 24–33. [DOI] [PubMed] [Google Scholar]
  • 21. Lamy JB, Berthelot H, Favre M. Rainbow boxes: a technique for visualizing overlapping sets and an application to the comparison of drugs properties. In: Proceedings of the International Conference Information Visuzalization 2016; July 2016; Lisbonne, Portugal.
  • 22. Lamy J-B, Berthelot H, Capron C, et al. Rainbow boxes: a new technique for overlapping set visualization and two applications in the biomedical domain. J Vis Lang Comput 2017; 43: 71–82. [Google Scholar]
  • 23. Tsopra R, Kinouani S, Venot A, et al. Design of a visual interface for comparing antibiotics using rainbow boxes. Stud Health Technol Inform 2017; 235: 529–33. [PubMed] [Google Scholar]
  • 24. Lamy J-B. Artificial feeding birds (AFB): a new metaheuristic inspired by the behavior of pigeons In: Advances in Nature-Inspired Computing and Applications. Springer International; 2019: 43–60. [Google Scholar]
  • 25. Brooke J. SUS-A quick and dirty usability scale. Usabil Eval Ind 1996; 189: 194. [Google Scholar]
  • 26. Brooke J. SUS: a retrospective. J Usabil Stud 2013; 8: 29–40. [Google Scholar]
  • 27. Bruttomesso D, Gagnayre R, Leclercq D, et al. The use of degrees of certainty to evaluate knowledge. Patient Educ Couns 2003; 511: 29–37. [DOI] [PubMed] [Google Scholar]
  • 28. Bangor A. An empirical evaluation of the system usability scale. Int J Hum-Comput Interact 246: 574–94. [Google Scholar]
  • 29. Bangor A, Kortum P, Miller J.. Determining what individual SUS scores mean: adding an adjective rating scale. J Usabil Stud 2009; 4: 114–23. [Google Scholar]
  • 30. Holstiege J, Mathes T, Pieper D.. Effects of computer-aided clinical decision support systems in improving antibiotic prescribing by primary care providers: a systematic review. J Am Med Inform Assoc 2015; 221: 236–42. doi: 10.1136/amiajnl-2014-002886 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Goddard BL. Termination of a contract to implement an enterprise electronic medical record system. J Am Med Inform Assoc 2000; 76: 564–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Moxey A, Robertson J, Newby D, et al. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc 2010; 171: 25–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Linder JA, Rose AF, Palchuk MB, et al. Decision support for acute problems: the role of the standardized patient in usability testing. Journal of Biomedical Informatics 2006; 396: 648–55. [DOI] [PubMed] [Google Scholar]
  • 34. Litvin CB, Ornstein SM, Wessell AM, et al. Adoption of a clinical decision support system to promote judicious use of antibiotics for acute respiratory infections in primary care. Int J Med Inform 2012; 818: 521–6. [DOI] [PubMed] [Google Scholar]
  • 35. Esmaeilzadeh P, Sambasivan M, Kumar N, et al. Adoption of clinical decision support systems in a developing country: antecedents and outcomes of physician’s threat to perceived professional autonomy. Int J Med Inform 2015; 848: 548–60. [DOI] [PubMed] [Google Scholar]
  • 36. Johnson CM, Johnson T, Zhang J.. Increasing productivity and reducing errors through usability analysis: a case study and recommendations. Proc AMIA Symp 2000: 394–8. [PMC free article] [PubMed] [Google Scholar]
  • 37. Tsopra R, Jais J-P, Venot A, et al. Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics. J Am Med Inform Assoc 2014: 21 (e1): e107–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Zielstorff RD. Online practice guidelines. J Am Med Inform Assoc 1998; 53: 227–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis 2016; 6210: e51–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Thursky KA, Mahemoff M.. User-centered design techniques for a computerised antibiotic decision support system in an intensive care unit. Int J Med Inform 2007; 7610: 760–8. [DOI] [PubMed] [Google Scholar]
  • 41. Shebl NA, Franklin BD, Barber N.. Clinical decision support systems and antibiotic use. Pharm World Sci 2007; 294: 342–9. [DOI] [PubMed] [Google Scholar]
  • 42. Yong MK, Buising KL, Cheng AC, et al. Improved susceptibility of Gram-negative bacteria in an intensive care unit following implementation of a computerized antibiotic decision support system. J Antimicrob Chemother 2010; 655: 1062–9. [DOI] [PubMed] [Google Scholar]
  • 43. Curtis CE, Al Bahar F, Marriott JF.. The effectiveness of computerised decision support on antibiotic use in hospitals: a systematic review. PLoS One 2017; 12: e0183062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Thursky KA, Buising KL, Bak N, et al. Reduction of broad-spectrum antibiotic use with computerized decision support in an intensive care unit. Int J Qual Health Care 2006; 183: 224–31. [DOI] [PubMed] [Google Scholar]
  • 45. Yen P-Y, Bakken S.. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 193: 413–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Mainous AG, Lambourne CA, Nietert PJ.. Impact of a clinical decision support system on antibiotic prescribing for acute respiratory infections in primary care: quasi-experimental trial. J Am Med Inform Assoc 2013; 202: 317–24. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES