Lower alert rates by clustering of related drug interaction alerts

Abstract

Objective: We aimed to investigate to what extent clustering of related drug interaction alerts (drug-drug and drug-disease interaction alerts) would decrease the alert rate in clinical decision support systems (CDSSs).

Methods: We conducted a retrospective analysis of drug interaction alerts generated by CDSSs in community pharmacies. Frequently generated combinations of alerts were analyzed for associations in a 5% random data sample (dataset 1). Alert combinations with similar management recommendations were defined as clusters. The alert rate was assessed by simulating a CDSS generating 1 alert per cluster per patient instead of separate alerts. The simulation was performed in dataset 1 and replicated in another 5% data sample (dataset 2).

Results: Data were extracted from the CDSSs of 123 community pharmacies. Dataset 1 consisted of 841 572 dispensed prescriptions and 298 261 drug interaction alerts. Dataset 2 was comparable. Twenty-two frequently occurring alert combinations were identified. Analysis of these associated alert combinations for similar management recommendations resulted in 3 clusters (related to renal function, electrolytes, diabetes, and cardiovascular diseases). Using the clusters in alert generation reduced the alert rate within these clusters by 53–70%. The overall number of drug interaction alerts was reduced by 11% in dataset 1 and by 12% in dataset 2. This corresponds to a decrease of 21 alerts per pharmacy per day.

Discussion and conclusion: Using clusters of drug interaction alerts with similar management recommendations in CDSSs can substantially decrease the overall alert rate. Further research is needed to establish the applicability of this concept in daily practice.

BACKGROUND AND SIGNIFICANCE

Clinical decision support systems (CDSSs) are a useful tool in detecting potential drug therapy–related problems.^1–3 However, in daily practice, physicians and pharmacists override up to 95% of the alerts.^4–7 When only a minority of alerts lead to action, this can lead to “alert fatigue,” with the risk of missing important alerts.

Several strategies to reduce alert fatigue have been investigated, both in simulations and in daily practice.^7–20 These strategies involve changes in the usability design of CDSSs (eg, changing the presentation of alerts from interruptive to noninterruptive),^17–22 reassessment of the clinical relevance of alerts in order to turn off irrelevant alerts,^8–11 and incorporation of more clinical characteristics in the algorithms generating the alerts (eg, lab measurements, duration of use, prophylactic medication in use).^7,12–16 In general, the majority of these strategies were targeted at increasing the specificity of individual alerts, often in small subsets of alerts. These investigations led to varying results, but override rates tended to stay considerably high.²³ Therefore, there is a need for exploration of additional strategies.

In community pharmacy, the majority of drug-drug interaction alerts and drug-disease interaction alerts (hereafter referred to as drug interaction alerts) are generated for a minority of patients, namely patients with multimorbidity and polypharmacy.²⁴ When many alerts are generated for the same patient at the same moment, overlooking one of them is conceivable. Moreover, a few therapeutic drug groups are responsible for the majority of the alerts.²⁴ Besides, it is known that a small number of drug-drug interactions account for the majority of the generated drug-drug interaction alerts.^5,25,26 Based on these findings, it is likely that the majority of drug interaction alerts concern a limited subset of potential problems. This can lead to the generation of several alerts for the same patient at the same moment related to the same risk (eg, several drug-drug and drug-disease interactions pointing out the risk of increased potassium levels). One integrated alert could potentially replace individual alerts, which would reduce the alert rate without changing the content of the recommendations presented to health care providers.

OBJECTIVE

In this study, we aim to investigate whether there are associations between the drug-drug interaction alerts and drug-disease interaction alerts that occur for a patient, and to what extent clustering of related drug interaction alerts that are concurrently generated would decrease the alert rate in clinical decision support systems.

METHODS

Setting

Dutch community pharmacies use CDSSs from a limited number of software suppliers. The pharmacy information system Pharmacom® (by Total Specific Solutions (TSS) PharmaPartners®) is used by approximately 55% of Dutch community pharmacies. The electronic patient record in the system includes data on dispensed medications and coded chronic diseases. Prescriptions can be sent electronically from physician to pharmacy (for most general practitioners), or printed prescriptions can be used (for most other prescribers). Clinical decision support with drug therapy alerts, including drug-drug interactions and drug-disease interactions, is an integral part of the pharmacy information system. Alerts are generated during the processing of the prescription in the community pharmacy, before dispensing takes place. Identical drug interaction alerts are generated for first-time and repeat prescriptions. Every alert is displayed in a separate popup window with specific management recommendations. Background information on the alert and management recommendations is available on a website and in a reference book.²⁷

Pharmacists regularly contact prescribers about the management of drug interactions. The CDSS of Dutch general practitioners is similar to the pharmacists’ CDSS, and shared electronic patient records are quite common. For other prescribers, the availability of information is mostly different.

Data collection

Invitations to participate in this study were mailed to 250 randomly chosen pharmacies from the 1080 community pharmacies using the Pharmacom system (with a reminder after 3 weeks). Pharmacists had to authorize TSS PharmaPartners to extract anonymous patient data. For participating pharmacies, the following patient data were extracted over the period August 2012 to July 2014: basic patient characteristics (including coded chronic diseases), dispensed medications (including dispensing date, dose, dosing regimen), and all generated drug therapy alerts.

Data analysis

The data were analyzed using Microsoft Access and SPSS (version 20.0; SPSS Inc., Chicago, IL, USA). Two random samples of 5% of patients per pharmacy to whom at least 1 drug was dispensed in the period August 2013 to July 2014 were selected (dataset 1 and dataset 2). Dataset 1 was used to identify clusters of drug interaction alerts and to simulate generation of drug interaction alerts based on these clusters. The simulation of clustered generation of drug interaction alerts was replicated in dataset 2 to test for consistency of the results.

Cluster identification

Clusters were identified in 5 steps (Figure 1):

1. The most frequently generated drug interaction alerts were selected. A cutoff of 1% of the total number of generated drug interaction alerts was used.

2. For the alerts selected in step 1, the most frequently generated combinations of alerts within a pharmacy visit were determined (ie, combinations of alerts generated for prescriptions dispensed to the same patient on the same day in the same pharmacy). From this step on, first-time prescriptions were excluded from the analysis. The management recommendation texts for drug interactions differentiate between first-time prescriptions and repeat prescriptions. For example, when adding an angiotensin-converting enzyme (ACE) inhibitor to a therapy with diuretics, the recommendation is to start with half the normal starting dose for 3 days, and to take the first dose just before bedtime. But when both drugs are used chronically, the recommendation is to monitor renal function and potassium levels. Because of the very specific management recommendations on therapy start, the potential for clustering of alerts on first-time prescriptions is limited.

3. All combinations of alerts on repeat prescriptions with an average frequency of at least once per day per pharmacy were selected.

4. Overrepresentation of the combinations of 2 alerts relative to the frequency of the individual alerts was determined, analogous to the reporting odds ratios used in the analysis of spontaneous report databases.²⁸ Alert combinations with an odds ratio significantly above 1 were considered associated alert combinations (P < .05 was considered statistically significant).

5. Steps 2 and 3 were repeated to identify whether the selected combinations of 2 alerts were associated with a third alert, etc.

6. For the associated alert combinations from step 3, the management recommendations were assessed for similarities. Combinations where all included alerts had similar management recommendations in case of chronic use were selected. For example, the drug-drug interaction renin-angiotensin system (RAS) inhibitors–potassium-sparing agents/potassium and the drug-disease interaction renal impairment–diuretics are accompanied by a similar recommendation: monitor renal function and potassium levels.

7. All drug interaction alert combinations with similar management recommendations were combined into clusters. For example, when there were 3 combinations of drug interactions, all relating to potassium monitoring, all drug interaction alerts belonging to these combinations were aggregated into 1 cluster.

Figure 1.

Steps of cluster identification

Simulation of clustered alert generation

Alert generation based on the identified clusters was simulated in both datasets 1 and 2. The frequency of the occurrence of alerts within a cluster was assessed by determining the frequency of alerts in any of the potential underlying alert combinations (eg, in a cluster of 4 alerts, there are 6 possible combinations of 2 alerts, 4 combinations of 3 alerts, and 1 combination of 4 alerts; all these combinations were part of the cluster). As in the identification of the clusters, first-time prescriptions were disregarded.

The number of simulated clustered alerts was subtracted from the originally generated number of alerts to calculate the potential reduction of alerts.

For example, in our original database there was a visit with 3 alerts: (1) RAS inhibitors–diuretics, (2) renal impairment–diuretics, and (3) renal impairment–ACE inhibitors. All alerts were part of the same cluster. In the simulation (the clustered alert generation), we counted 1 alert for this situation, compared to 3 alerts in the original situation.

Ethics and confidentiality

As this was a retrospective database analysis, the study was exempt from ethical review. To protect the privacy of patients and pharmacists, only anonymous data were extracted from the CDSS, to prevent the identification of individual patients or pharmacies.

RESULTS

Of the 250 invited pharmacies, 123 (49%) agreed to participate in the study. The 2 random samples of 5% included data on over 80 000 patients who received over 1.6 million prescriptions (Figure 2). The prescriptions generated nearly 0.6 million drug interaction alerts, corresponding to an average of 185 drug interaction alerts per pharmacy per day. Eight percent of these alerts were generated on first-time prescriptions. The top 10 dispensed drugs in our sample included the same drugs as the nationwide top 10 dispensed drugs in community pharmacies.²⁹

Figure 2.

Dataset characteristics

Identified clusters

Dataset 1 contained 298 261 generated drug interaction alerts. Twenty-eight drug interactions had a frequency over 1% of all generated drug interaction alerts. This selection consisted of 16 drug-drug interactions and 12 drug-disease interactions (Table 1). These 28 drug interactions accounted for 59.5% of all generated drug interaction alerts.

Table 1.

Drug interaction alerts with a frequency >1% of all drug interaction alerts (n = 298 261)^a

Drug interaction	Type	Percentage of drug interaction alerts (%)
RAS inhibitors–diuretics	Drug-drug	7.6
Antidiabetics–beta-blocking agents	Drug-drug	4.6
Diabetes–ACE inhibitors	Drug-disease	4.5
Obstructive pulmonary disease–beta- blocking agents	Drug-disease	4.2
Renal impairment–diuretics	Drug-disease	3.9
NSAIDs–RAS inhibitors	Drug-drug	2.3
RAS inhibitors–potassium-sparing agents/potassium	Drug-drug	2.3
Bisphosphonates–polyvalent cations	Drug-drug	2.2
Vitamin K antagonists–(es)omeprazole	Drug-drug	2.1
Renal impairment–ACE inhibitors	Drug-disease	2.1
Heart failure–beta-blocking agents	Drug-disease	2.0
Beta-blocking agents–NSAIDs	Drug-drug	1.9
Thyroid drugs–polyvalent cations	Drug-drug	1.7
Salicylates (antithrombotic)–SRIs	Drug-drug	1.6
Renal impairment–antidiabetics	Drug-disease	1.5
Diabetes–thyroid drugs	Drug-disease	1.5
Vitamin K antagonists–metformin	Drug-drug	1.4
Digoxin–diuretics	Drug-drug	1.4
Salicylates (antithrombotic)–NSAIDs	Drug-drug	1.3
SRIs–diuretics	Drug-drug	1.3
Diabetes–SRIs	Drug-disease	1.2
P2Y12 inhibitors–salicylates (antithrombotic)	Drug-drug	1.2
Corticosteroids–salicylates (antithrombotic)	Drug-drug	1.1
Renal impairment–minerals	Drug-disease	1.1
Diabetes–antipsychotics	Drug-disease	1.1
Gout–diuretics	Drug-disease	1.1
Ulcus pepticum–antithrombotic agents	Drug-disease	1.0
NSAIDs–SRIs	Drug-drug	1.0

ACE = angiotensin-converting enzyme; NSAID = nonsteroidal anti-inflammatory drug; RAS = renin-angiotensin system; SRI = serotonin reuptake inhibitor; ^aBased on dataset 1

Twenty-two combinations of alerts (20 combinations of 2 alerts and 2 combinations of 3 alerts) occurred at least once per pharmacy per day. All combinations were significantly associated: the combination of alerts was overrepresented relative to the frequency of the individual alerts (Appendix 1). Fourteen of these associated combinations had similar management recommendations (Appendix 1). Based on these management recommendations, 3 clusters of alerts were identified: monitoring of potassium levels and renal function, monitoring of potassium levels and renal function plus monitoring of diabetes, and monitoring of blood pressure and/or heart failure (Table 2).

Table 2.

Comparison of number of alerts for clustered generation of drug interaction alerts and the original CDSS^a

Cluster	Management recommendation (for repeat prescriptions)	Alerts in cluster		Number of alerts		Change in alert rate (%)
				Original CDSS	Clustered alert generation
Cluster A	Monitor renal function + potassium	RAS inhibitors–diuretics RAS inhibitors–potassium(-sparing diuretics) Renal impairment–diuretics Renal impairment–ACE-inhibitors	Dataset 1	13 694	4920	−64
			Dataset 2	14 014	5151	−63
Cluster B	Monitor renal function +  potassium + diabetes	RAS inhibitors–diuretics Renal impairment–diuretics Renal impairment–ACE inhibitors Renal impairment–antidiabetics Diabetes–ACE inhibitors Antidiabetics–beta-blocking agents	Dataset 1	30 999	9369	−70
			Dataset 2	31 918	9484	−70
Cluster C	Monitor blood pressure /heart failureb	NSAIDs–RAS inhibitors Beta-blocking agents–NSAIDs	Dataset 1	4077	1866	−54
			Dataset 2	5226	2463	−53

ACE = angiotensin-converting enzyme; CDSS = clinical decision support system; NSAID = nonsteroidal anti-inflammatory drug; RAS = renin-angiotensin system; ^aFirst-time prescriptions excluded; ^bDepending on the indication of the RAS inhibitor and the beta-blocking agent

Alert rates

The alert simulation using clusters resulted in reductions of 64%, 70%, and 54%, respectively, for the alerts included in the 3 clusters compared to the original situation in dataset 1 (Table 2). Replication in dataset 2 showed comparable reductions.

Clustered alert generation for these 3 clusters reduced the overall number of drug interaction alerts from 298 261 to 265 464 (−11%) in dataset 1 and from 292 812 to 258 752 in dataset 2 (−12%). This corresponds to a decrease of 21 drug interaction alerts per pharmacy per day.

DISCUSSION

In this investigation, we identified 3 frequently occurring clusters of drug interaction alerts, consisting of 8 drug interaction alerts. The use of these clusters in a CDSS simulation led to a decrease of over 50% for the alerts in the clusters. The overall drug interaction alert rate decreased by more than 10%, corresponding to a decrease of 21 alerts per pharmacy per day. The clusters were related to renal function, electrolytes, diabetes, and cardiovascular diseases, common chronic conditions with extensive chronic drug use.

Several strategies to reduce alert fatigue have been investigated, most of them targeted at increased alert specificity.^7–16 The results of these strategies are diverse, ranging from limited success to decreases in the alert rate ranging from 50–90% within specific subsets of alerts.^7,12,15,16 All of those previous attempts to reduce the alert burden were different from our approach, as we focused on an overview of all alerts with the same management recommendations within 1 patient. In contrast to other strategies, our method does not focus on individual alerts or subsets, and therefore can be extended to all alerts without adaption or reassesment of individual alerts.

Our study shows that the concept of clustering of alerts has potential. Even when using only 3 clusters together consisting of only 8 alerts, a reduction of more than 10% of the total drug interaction alert rate was realized. We see potential for extension of this strategy in 4 directions. First, more drug-drug interactions and drug-disease interactions could be incorporated in these 3 clusters. For example, many other less frequently generated alerts advise to monitor renal function and potassium levels.

Second, new clusters can be created for other management recommendations; for example, clusters on monitoring sodium levels or monitoring the international normalized ratio.

Third, other alert types could also be included in the alert clustering. Lab-drug alerts, age-drug alerts, duplicate medication alerts, and dosing alerts are all eligible candidates for clustered alerting.

Fourth, for alerts with similar management recommendations for first-time and repeat prescriptions, first-time prescriptions could be included in the clusters. For the purpose of consistency, this would be useful. The effect on alert rate would be limited, because alerts on first-time prescriptions concerned only 8% of the alerts.

More generally, this study suggests 2 new areas for CDSS improvement.

The first one concerns a further extension of the concept of clustering. In our investigation, we started with selection of frequently generated drug interaction alerts (with one or more management recommendations each) and looked for the potential of combining the alerts of several drug interactions. For the health care provider, the recommendation is more important than the interaction itself. Therefore, we propose a reverse approach, realizing recommendation-based alerting: for every recommendation, rather than for every drug interaction, an alert is generated. This alert can be based on 1 or more drug interactions. This approach supports a patient-centered way to do clinical risk management instead of the current drug interaction–centered way. The need for an integrated approach is underlined by the fact that comorbidity clusters can be defined, and that multimorbidity reduces the applicability of monodisciplinary guidelines.^30–33

The second area for improvement relates to the moment of alerting. In our study, it was noticed that alert timing was a recurring aspect in the assessment of similarity of management recommendations. For example, the management recommendations of the clusters we identified all advised (half-)yearly reassessment or monitoring. There is no need to generate these alerts for every repeat prescription, unless there is a change in the patient’s health condition. In addition, some drug interactions are mainly relevant at the start of the therapy. Examples in our clusters are the drug-drug interaction between RAS inhibitors and diuretics and the drug-disease interaction between ACE inhibitors and diabetes (recommending intensified blood glucose monitoring at the start).²⁷ Generation of these alerts could be limited to first-time prescriptions. Therefore, the combination of specific alert timing and alert clustering could further reduce the alert rate.

This study simulated a clustered generation of drug interaction alerts and demonstrated the potential of clustered alert generation in one CDSS. The study has several limitations. In our investigation we focused on all generated alerts, without assessing their clinical relevance. Obviously, it is also important to increase the specificity of alerts to reduce the number of irrelevant alerts. Therefore, alert clustering is a complementary strategy.

A second limitation is that our study is based on data from only one CDSS. In other settings other CDSSs will be used, leading to differences in the specific alerts that will be generated. However, a comparable pattern of overlapping management recommendations can be expected. We therefore believe our strategy can be applied to most CDSSs.

Lastly, the decrease in alert rate by clustered alert generation in clinical practice can be different from our simulation. We have not investigated how a clustered alert should be displayed or how health care providers would perceive clustered alerts. Therefore, the applicability of this concept still has to be proven. In future investigations, the design and management of alerts are of major importance. It has been shown that the design of alerts affects their efficiency and results.^21,22 A clustered alert is an alert with one recommendation, shown in one window, but based on more than one drug interaction. All drugs, diseases, and other risk factors related to the concerning recommendation should be concisely shown to enable the health care professional to make a proper judgment. Because of the combination of information in one window, paying special attention to the alert design is advised in order to prevent data overload. Moreover, the actual management of clustered alerts should be investigated. With fewer alerts, there is less risk of confusion about several comparable alerts and overseeing one of them. However, there is a clear need to manage the single new alert correctly, and to judge its relevance properly. The effect of clustered alert generation on interventions by health care professionals must be established to rule out any unexpected results and to optimize the concept before implementing it in daily practice.

CONCLUSION

The use of 3 clusters of drug interactions with similar management recommendations in alert generation decreased the alert rate of the alerts in clusters by more than 50%. The overall alert rate for drug interactions was reduced by more than 10% (corresponding to a decrease of 21 drug interaction alerts per pharmacy per day). Extension of drug alert clustering can potentially further reduce the alert rate.

CONTRIBUTORS

All authors contributed to the study design, the study protocol, and the manuscript. M.H. and H.S. performed the data analysis. All authors approved the final manuscript.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector.

COMPETING INTERESTS

The authors have no conflicts of interest that are directly relevant to the content of this study.

SUPPLEMENTARY MATERIAL

Supplementary material is available online at http://jamia.oxfordjournals.org/.