Contextualized Drug–Drug Interaction Management Improves Clinical Utility Compared With Basic Drug–Drug Interaction Management in Hospitalized Patients

Abstract

Drug–drug interactions (DDIs) frequently trigger adverse drug events or reduced efficacy. Most DDI alerts, however, are overridden because of irrelevance for the specific patient. Basic DDI clinical decision support (CDS) systems offer limited possibilities for decreasing the number of irrelevant DDI alerts without missing relevant ones. Computerized decision tree rules were designed to context‐dependently suppress irrelevant DDI alerts. A crossover study was performed to compare the clinical utility of contextualized and basic DDI management in hospitalized patients. First, a basic DDI‐CDS system was used in clinical practice while contextualized DDI alerts were collected in the background. Next, this process was reversed. All medication orders (MOs) from hospitalized patients with at least one DDI alert were included. The following outcome measures were used to assess clinical utility: positive predictive value (PPV), negative predictive value (NPV), number of pharmacy interventions (PIs)/1,000 MOs, and the median time spent on DDI management/1,000 MOs. During the basic DDI management phase 1,919 MOs/day were included, triggering 220 DDI alerts/1,000 MOs; showing 57 basic DDI alerts/1,000 MOs to pharmacy staff; PPV was 2.8% with 1.6 PIs/1,000 MOs costing 37.2 minutes/1,000 MOs. No DDIs were missed by the contextualized CDS system (NPV 100%). During the contextualized DDI management phase 1,853 MOs/day were included, triggering 244 basic DDI alerts/1,000 MOs, showing 9.6 contextualized DDIs/1,000 MOs to pharmacy staff; PPV was 41.4% (P < 0.01), with 4.0 PIs/1,000 MOs (P < 0.01) and 13.7 minutes/1,000 MOs. The clinical utility of contextualized DDI management exceeds that of basic DDI management.

Study Highlights.

• WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

☑ Basic drug–drug interaction (DDI) alerts suffer from high override rates due to their sheer number. Despite many attempts to reduce the number of DDI alerts; basic clinical decision support (CDS) systems have shown to have limited possibilities for decreasing the number of irrelevant DDI alerts without also reducing the number of relevant alerts.

• WHAT QUESTION DID THIS STUDY ADDRESS?

☑ Is it possible to improve clinical utility, i.a., to reduce the number of irrelevant DDI alerts using contextualization without reducing the number of relevant alerts.

• WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?

☑ This study shows that contextualized DDI management compared with basic DDI management in a clinical pharmacy setting can considerably decrease the number of irrelevant DDI alerts and thereby increase the time available to interpret relevant DDI alerts, thus leading to more relevant interventions without missing relevant DDI alerts.

• HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?

☑ Introduction of contextualized CDS‐DDI management can drastically improve clinical utility of DDI management for hospitalized patients.

Drug–drug interactions (DDIs) frequently occur in hospitalized patients: 65–90% of these patients are exposed to one or more potential DDIs. ¹ , ² , ³ Mismanagement of DDIs can lead to adverse drug events or reduce the efficacy of drugs involved. ⁴ Healthcare providers cannot be expected to memorize the thousands of known DDIs and the management thereof. ⁵ , ⁶ , ⁷ Clinical decision support (CDS) systems have been added to computerized physician order entry (CPOE) and pharmacy information systems to assist healthcare professionals in alerting and managing the risks of potentially harmful medication combinations. Such DDI‐CDS systems, also called basic medication‐related CDS systems, ⁸ trigger alerts for the pair‐wise combination of the drugs involved. In practice, however, such DDI alerts are frequently overridden as most alerts are considered to be irrelevant for specific patients. ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³

Too many irrelevant alerts can lead to alert fatigue, ⁹ , ¹³ , ¹⁴ , ¹⁵ described as a “mental state being the result of too many irrelevant alerts consuming time and mental energy, which can cause important alerts to be ignored along with clinically unimportant ones.” ¹⁶ Basic DDI‐CDS systems have limited options for suppressing irrelevant DDI alerts, other than turning DDI alerts off for specific drug combinations. This approach has limited opportunities for improving specificity without compromising sensitivity. ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ Recent studies concluded that CDS systems should have greater flexibility to customize DDI alerting especially by adding contextual modulation, also called specificity modulation. ²⁵ , ²⁶ , ²⁷ The term contextual modulation comes from neurobiology, being the change in the neurons’ responsiveness to a stimulus caused by context. ²⁸ In the setting of medication‐related CDS alerting, contextual modulation changes whether or not a triggered alert is displayed and how, based on context.

Contextualized CDS systems, also known as advanced CDS systems, offer more possibilities for suppression of irrelevant alerts since these systems incorporate context. The most important forms of context are workflow context and clinical context, which, by using information available in the electronic health records (EHRs), can prioritize alerts and/or suppress them. ²⁹ , ³⁰ , ³¹ , ³² Improving the specificity of DDI alerts has been studied using different types of clinical contexts: admission wards or treating medical specialties ³³ , ³⁴ (e.g., not showing QT prolongation DDI alerts for intensive care patients); patient parameters such as age, blood pressure and/or laboratory results ¹² , ³³ , ³⁵ , ³⁶ , ³⁷ (e.g., not showing DDI alerts for potassium + potassium‐sparing diuretics in patients with hypokalemia); only showing alerts above a specific dose (e.g., fluconazole >100 mg + immunosuppressant) ³⁸ ; and coadministered drugs. ³⁸ , ³⁹ CDS systems using the workflow context can be programmed to better fit workflow and usability, ³² , ³³ , ⁴⁰ , ⁴¹ improve DDI alert triggering, ⁴² and stop or reduce the repetition of already presented alerts. ³³ , ⁴³ Overall, these studies showed reductions of 50–92% of the DDI alert burden. ³³ , ³⁵

Most studies have focused on reducing the number of irrelevant DDI alerts in either using clinical or workflow contexts. ²² , ²³ , ³⁹ Only two studies investigate the effects of combining these two. ³³ , ³⁵ Moreover, only two studies have explored the impact of such optimizations on the sensitivity or negative predictive value (NPV). ¹⁷ , ¹⁸ Therefore, a set of computerized decision tree rules (algorithms) was designed to combine clinical and workflow contexts to suppress irrelevant DDI alerts. These rules were programmed on top of a regular DDI knowledge base. The contextualized DDI‐CDS management system utilizes different patient parameters, laboratory values, drug doses, and previous evaluations to contextualize and assess triggered DDI alerts. This study aimed to compare the clinical utility of contextualized DDI management with that of basic DDI management in hospitalized patients.

METHODS

Study design

A single‐center prospective crossover study was performed to compare basic DDI‐CDS management with contextualized DDI‐CDS management in a clinical pharmacy setting. The first phase was the basic DDI phase, the second phase being the contextualized DDI phase. The basic DDI phase included basic DDI alerts used in clinical practice, while contextualized DDI alerts were gathered in the background, referred to as background data collection. In the contextualized DDI phase, this process was reversed. The basic DDI management phase lasted from August 10, 2020, to September 13, 2020, (35 days), followed directly by the contextualized DDI management phase from the of September 14, 2020, to November 2, 2020, (50 days); this last phase was longer to perform an adequate number of time measurements of the pharmacy interventions. Prescribers were not informed about the change in the clinical pharmacy DDI management process. The study was declared not subject to Research Involving Human Subjects Act (non‐WMO) by the ethics committee of the Catharina Hospital.

Study setting

The study was performed in the clinical pharmacy at the Catharina Hospital Eindhoven, a 700‐bed teaching hospital in the Netherlands. The hospital used HiX (version 6.1 HF105 and HF108, Chipsoft BV, Amsterdam, The Netherlands) as its EHR system. All relevant medical data are ordered and stored in this system, including CPOE for inpatient and outpatient settings. Smart CPOE ordering, which uses predefined orders and order sets, prevents the occurrence of predominantly time‐dependent DDIs. The integrated CDS system offered by the CPOE was used as the basic DDI‐CDS system. The basic DDI‐CDS system had one type of alert‐suppression possibility: turning specific DDI alerts off. Before the start of the study, a set of DDI alerts was already suppressed, having been considered irrelevant by a team of prescribers and hospital pharmacists, included in Table  S1 .

Gaston Pharma (version 2.8.2.100, Gaston Medical, Eindhoven, The Netherlands) was used to develop and generate the contextualized DDI alerts. Before the start of the study, as a technical validation, two weeks of DDI alerts were matched, assuring that without contextualization both systems triggered the same alerts. This was followed by three months of technical validation of the developed meta‐rules. Gaston Pharma has been used to provide medication‐related CDS in addition to G‐standard knowledge base in the Catharina Hospital content since 2006. However, it has not been previously used for DDI management. Costs and return on investment analysis of this system have been published previously. ³³ Previously developed clinical rules partially overlapped basic DDI alerts, included as italic type in Table  S1 . The contextualized CDS system can suppress DDI alerts based on the clinical context, including laboratory values, medication order (MO) details, outpatient drug–drug combinations, and patient demographics. The CDS system could also suppress alerts based on previous evaluations of the DDI, cluster alerts based on advice, medication type, or other characteristics.

Hospital pharmacy staff consisted of 12 (senior) hospital pharmacists, 4 hospital pharmacists in training, and around 40 pharmacy technicians. Daily pharmaceutical services, including DDI management, were performed by four 0.3 full‐term equivalent hospital pharmacists and 12 full‐term equivalent pharmacy technicians on a ward basis. All of the pharmacy staff were already trained to use the contextualized CDS system before the start of the study.

Study inclusion

The study included all MOs of patients hospitalized with at least one DDI alert triggered by the basic DDI‐CDS system during basic as well as contextualized DDI management phase. An MO was defined as a new prescription or any change to an existing prescription (i.e., a dose adjustment).

Basic drug–drug interaction management process

Table  1 (left‐hand side), shows the key system and process details for the basic DDI management process. Figure  S1A–C gives an overview of DDI alert presentation in the basic DDI‐CDS system. If the MO for one or both interacting drugs was changed, the DDI alert was shown for each MO during evaluation. Therefore, in most cases, the same DDI alert was shown twice.

Table 1.

Similarities and differences between DDI‐CDS management process and system for both phases

Basic DDI management phase	Contextualized DDI management phase
Same for both phases
Basic DDI‐CDS system in use for physicians
G‐standard knowledge basea
Pharmacy technicians adjusted administration times of drugs without consulting physician or hospital pharmacist
Different for both phases for clinical pharmacy staff
Basic DDI‐CDS system integrated into EHR system	Contextualized DDI‐CDS system on top of EHR system
“Real‐time” DDI alert generation	“Batch‐wise” DDI alert generation
DDI alerts were shown to prescriber, pharmacy technician, and hospital pharmacist in that order	DDI alerts were shown only to pharmacy technician or only to hospital pharmacist depending on applicability
Communication possible between physician and clinical pharmacy staff in the DDI alert note	No communication possible between physician and clinical pharmacy staff
Workflow context suppression
DDI alerts were shown paired to MOsb	DDI alerts were shown independent of MOsc
With each type of MO change (prescriber, administration time; route of administration, etc.) the DDI alert was shown	DDI alerts were shown once to pharmacy staff and thereafter only if daily dose of one of the interactions drugs was changedd
MOs of actionable absorption time‐dependent DDI alerts were changed by pharmacy technician; changes checked by second and thereafter by hospital pharmacist	MOs of actionable absorption time‐dependent DDI alerts were changed by pharmacy technician and then checked by contextualized DDI‐CDS system
Clinical context suppression e
DDI alerts could be turned off	DDI alerts could be suppressed depending on specific clinical context (Table  2 )
DDI alerts were shown if one of the interacting drugs was already stopped in last 24 hours	DDI alerts were not structurally shown when one of the interacting drugs was already stopped; only if applicablef
Clinical rules alerts overlapping part of DDIs were partly showng	Clinical rules alerts overlapping part of DDIs were not showng

CDS, Clinical decision support; DDI, drug–drug interaction; EHR, electronic health record; MOs, medication orders.

^a G‐standard knowledge base, which, among others, includes DDI assessments used nationally in the Netherlands. ⁴⁴

^b Evaluation of medication alerts, including DDI alerts, was shown paired to an MO (e.g., DDI alert is shown when evaluating metoprolol MO and DDI alert is shown when evaluating paroxetine MO).

^c The contextualized system showed DDI alerts not paired to MOs (e.g., DDI alert for metoprolol + paroxetine was shown once) as general suppression was done based on the Anatomical Therapeutic Chemical (ATC) codes of drugs appearing in multiple MOs and DDI alerts were only shown “again” if daily dose of one of the interactions drugs was changed.

^d DDI alerts only reappeared if medication and daily dose were changed, changing from “if necessary use” to regular use or from “one‐time use” to regular use were defined as dose changes. Multiple MOs for the same drug were always clustered to show one alert; e.g., haloperidol 10 mg + 1 mg + amiodarone 200 mg clustered to show a single alert for haloperidol with amiodarone.

^e Contextualization included suppressing DDI alert not applicable within the given context and also adding additional information form the electronic health record (EHR) to manage the DDI alert if shown. All general as well as specific clinical contextualization is shown in Tables  2 and S1 .

^f DDIs found to be clinically relevant when one of the drugs was stopped (perpetrator) included cytochrome P450 (CYP) inhibitors with long half‐life: hydroxychloroquine (100 days), chloroquine (28 days), fluoxetine (15 days), and amiodarone (150 days) and CYP inducers (28 days): rifampicin, primidone, phenytoin, phenobarbital, carbamazepine, efavirenz, hypericum, and ritonavir.

^g Full list of clinical rules used and the overlap with DDI alerts is included at the bottom of Table  S1 in italic type.

The basic DDI‐CDS system offered the option to add text to the alert using the alert comment box. Alerts, including comments, were shown to the prescriber, pharmacy technician, and hospital pharmacist, in that order. The alerts themselves were the same for each healthcare professional; text was differentiated. The alert text presented to the prescriber and hospital pharmacist were those included in the G‐standard knowledge base. Pharmacy technicians were presented a locally written alert text. The local alert text contained instructions regarding which specific parameters should be checked in the EHR and added to the alert comment box. If instructed in the local text, the pharmacy technician could forward the DDI alert to a hospital pharmacist. Based on all gathered information, the hospital pharmacist could then decide whether to contact the prescriber.

Drug–drug interactions affecting drug absorption are called absorption time‐dependent DDI alerts (e.g., oral ciprofloxacin binding calcium). If administration times needed to be changed to prevent this type of DDI, a pharmacy technician would change administration times without contacting the prescriber. A different pharmacy technician and thereafter the hospital pharmacist evaluated if administration times were changed correctly. The process of DDI evaluation was continuous between 8:00 a.m. and 5:30 p.m.; however, MOs triggering DDI alerts prescribed after 2:00 p.m. were evaluated by clinical pharmacy staff the next day.

Contextualized drug–drug interaction management process

Table  1 (right‐hand side), shows the key system and process details for the contextualized DDI management process. In contrast to the basic DDI process, the contextualized DDI alerts were triggered in batches at 6:00 a.m., 9:00 a.m., 11:00 a.m., and 3:00 p.m. and evaluated between 8:00 a.m. and 5:30 p.m. For the prescribers the MO process was not changed during the study, i.e., basic DDI alerts were also directly presented to the prescriber. DDI alerts triggered using the contextualized DDI‐CDS system were evaluated by the hospital pharmacist. Preliminary alert evaluation by a pharmacy technician was omitted and mostly replaced by the contextualized DDI‐CDS system. Table  S1 provides a complete list of CDS evaluations replacing pharmacy technician evaluations. Absorption time‐dependent DDI alerts were shown only to the pharmacy technician; the correctness of the adjustments made by the pharmacy technician was checked by the contextualized DDI‐CDS system in the first following batch.

Contextualized DDI specifics: alert contextualization and suppression

Basic as well as contextualized DDI‐CDS systems used the Dutch G‐standard knowledge base content, which includes DDI assessments used nationally in the Netherlands. ⁴⁴ During the study, the knowledge base was updated monthly, following a regular schedule. As of November 2020, G‐standard knowledge base contained 1,058 DDIs, incorporating a total of 32,676 drug pairs. A drug pair was defined as Anatomical Therapeutic Chemical (ATC) code A + ATC B.

In addition to basic suppression (turning a specific alert off), the contextualized DDI‐CDS system combined two overarching types of alert contextualization and alert suppression. The first type being suppression based on the workflow context, the second being suppression based on the clinical context. Workflow suppression included (i) suppression based on previous DDI alert evaluation (i.e., DDI alerts only reappeared if medication and daily dose were changed, changing from “if necessary use” to regular use or from “one‐time use” to regular use were defined as dose changes), (ii) suppression based on previously unevaluated DDI alerts, and (iii) suppression based on the ATC codes of drugs appearing in multiple MOs (e.g., haloperidol 10 mg + 1 mg + amiodarone 200 mg clustered to show a single alert for haloperidol with amiodarone).

General clinical context suppression, performed on all DDI alerts, included suppressing DDI alerts where one of the drugs was already stopped at the time of showing the DDI alert. DDI alerts including stopped cytochrome P450 (CYP) inhibitors with long half‐life and CYP inducers, however, were shown; a full list of included inhibitors and inducers is included at the bottom of Table  2 .

Table 2.

The types of clinical context suppression, including examples, applied in this study

Clinical context suppression based on modulator: (number of drug pair combinationsa to which the modulator was applied)b	Explanation (examples)
Prescription of interaction drug pair (3,591)
Dose (152)	Suppression when DDI was not applicable for a specific dose, e.g., simvastatin ≤40 mg combined with ticagrelor
Route of administration (94)	Suppression when DDI was not applicable for a combination of drugs using different routes of administration, e.g., midazolam nasal spray combined with verapamil
Chronology and lag time between administrations (1,318)	Suppression when drug administration times were sufficiently spaced to prevent absorption DDI, e.g., ciprofloxacin at 8:00a.m. and 8:00 p.m. combined with calcium at 3:00 p.m.
No alternative available (275)	Suppression when no therapeutic alternative was available in the setting, e.g., labetalol intravenous combined with insulin intravenous
Course of therapy (1,752)
“Only” if necessary use (127)	Suppression when drug use was used once or only used if necessary, e.g., haloperidol if necessary <5 mg combined with amiodarone
Short duration of administration (45)	Suppression when drug combination was present only for a short duration, including once, e.g., verapamil 2.5 mg once during percutaneous coronary intervention combined with digoxin
Drug–drug combination existing prior to admission (1,580)	Suppression when DDI existed prior to admission, e.g., metoprolol combined with paroxetine
Drug pair combination stopped c , d  (all) (General rule)	Suppression when one of the drugs was already stopped, c , d e.g., starting mirabegron while metoprolol was previously stopped
Comedication (4,033)
Pharmacodynamic counter‐DDI (828)c	Suppression when DDI increased risk is mitigated by comedication, c e.g., naproxen and dexamethasone when pantoprazole was coadministered
Pharmacodynamic risk modifiers e (e.g., DDIs only relevant in case multiple drugs involved) (3,205)	Suppression when only two of the three drugs increasing the risk of clinically significant DDI were present, e e.g., perindopril combined with furosemide with ibuprofen already in use
Patient characteristics (1,663)
Patients age (835)	Suppression when DDI was only applicable to a certain age category, e.g., ceftriaxone intravenously administered combined with calcium‐containing intravenous fluid in patients >1 month old
Comorbidity (828)	Suppression when DDI was only applicable combined with comorbidities, e.g., naproxen + dexamethasone in 30‐year‐old patient with previous gastric ulcer
Dynamic patient information (2,311)
Lab resultse (1,833)	Suppression when laboratory value monitoring was ordered or result was knowne, e.g., hydrochlorothiazide combined with citalopram when sodium was ordered
Vital signs (478)
Actual measurements (9)	Suppression when vital signs where above or below certain values, e.g., metoprolol + fluoxetine and heart rate >60 beats per minute
Routine monitoring f (469)	Suppression when routine monitoring was performed, e.g., alpha‐blocker combined with beta‐blockers on all medical wards
Deemed not clinically relevant in all cases (2)
Clopidogrel + (es)omeprazole	Suppression of a specific DDI rated not clinically relevant

Contextual modulators which were used for suppression as described by Seidling et al. 2014 were applied. ²⁷ Grouping modulators are shown in gray rows. (Sub)modulators, presented in italic type in the left‐hand column, were added to an existing modulator or modulator group. Several contextual DDI alerts showed different content or advice dependent on contextual modulators. The number of contextual modulators includes modulators used in the clinical rules in use previous to the study.

ATC, Anatomical Therapeutic Chemical; DDI, drug–drug interaction.

^a Definition of a single drug was done based on ATC code. A drug pair was therefore defined as ATC A + ATC B; e.g., N06AB03 (fluoxetine) + C07AB02 (metoprolol).

^b Multiple contextual modulators could be applied to a single drug pair (e.g., N06AB03 (fluoxetine) + C07AB02 (metoprolol), e.g., drug combination used prior to hospitalization and routine monitoring (continuous cardiac monitoring if admitted to the intensive care unit)).

^c A general rule applied to all drug pairs was used to suppress all DDI alerts where one of the drugs from the drug pair was already stopped, excluding pharmacokinetic interactions^d relevant after stopping (e.g., amiodarone stopped one day before starting digoxin).

^d Drugs (perpetrators) included as relevant after stopping included cytochrome P450 (CYP) inhibitors with long half‐life: hydroxychloroquine (100 days), chloroquine (28 days), fluoxetine (15 days), and amiodarone (150 days) and CYP inducers (28 days): rifampicin, primidone, phenytoin, phenobarbital, carbamazepine, efavirenz, hypericum, and ritonavir.

^e Previous to the current study, clinical rules were already in use monitoring ordering of timely therapeutic drug monitoring when applicable, drug‐induced electrolyte disorders or electrolyte disorders without proper drug management, international normalized ratio (INR) monitoring and use of gastric protection dependent on multiple risk factors including patients’ age, pharmacodynamic DDIs, and monitoring including pharmacodynamic counter‐DDI. Table  S1 (bottom) includes a full list of drug pairs included, including contextual modulators used.

^f Routine monitoring modulator included advice to monitor heart rate and blood pressure on regular wards and DDI alerts advising electrocardiography on wards that performed continuous cardiac monitoring. These wards included the intensive care unit, cardiac medium care unit and cardiac lounge.

Table  2 shows the contextual modulators used for suppression based on clinical context. In total, 15 different types of contextual modulators were used for specific suppression of DDI alerts, grouped into 5 major contextual modulators. ²⁷ Specific clinical context suppression spanned out over 93 out of the total 1,058 DDIs, including a total of 8,739 drug pairs using 11,627 modulators. This included basic DDIs previously overlapping clinical rules which could be turned off for pharmacy staff in its entirety. Table  S1 provides details on all meta‐rules applied.

Contextualized DDI alerts presented additional contextual information to pharmacy staff. The information provided was based on recommendations by Payne et al. ⁴⁰ Figure  S1 panel D gives an example of a contextualized DDI alert.

Clinical utility

Four outcome measures were used to determine clinical utility for both CDS management processes, namely:

• 1Positive predictive value (PPV) of the DDI alerts shown

An alert was considered to be a true positive (TP) if found to be clinically relevant. Clinical relevance in the context of this study was defined as an alert intervened upon by a pharmacy professional. DDI alerts not considered clinically relevant were counted as false positives (FPs). PPV was calculated as TPs/(TPs + FPs). PPV was calculated for the basic and contextualized DDI phases.

• 2NPV of the DDI alerts shown

Pharmacy interventions not shown in the background data collection were considered false negatives (FNs). The remaining DDI alerts not shown in the background data were considered true negatives (TNs). Hence, NPV was calculated as TNs/(FNs + TNs). The NPV was calculated only for the basic phase.

• 3Number of pharmacy interventions (PIs)/1,000 MOs

Pharmacy interventions (PIs)/1,000 MOs was calculated using the total number of PIs in a phase divided by the total number of included MOs for that phase times a 1,000. Pharmacy interventions were included if registered as such in the EHR or the CDS system.

• 4Time spent on DDI management/1,000 MOs

The time measurements needed for all separate steps of the DDI management process in the basic as well as contextualized DDI phase was measured for MOs with DDI alert using a stopwatch. MOs with DDI alerts were measured. If multiple DDI alerts appeared on one MO, the total time spent evaluating the MO was measured, and the number of DDI alerts was noted and total time spent on the MO was divided by the number of DDI alerts noted.

Total time spent on DDI management in basic and contextualized DDI phase was the addition of the median time of each step of the DDI management process multiplied by the frequency of occurrence. This was done separately for absorption time‐dependent DDIs and remainder of the DDIs; irrelevant and relevant DDI management evaluation were also measured.

Data analysis

Statistical tests were performed using SPSS statistics for Windows (version 27.0.0; IBM Corp, Armonk, NY). A Mantel‐Haenszel test was performed to test for difference in frequency in the number of DDI alerts/1,000 MOs triggered and DDI alerts/1,000 MOs shown to pharmacy st. ⁴⁵ An additional Bonferroni correction was used to test for the difference in DDI alerts shown for the different medical specialties. A general estimation of equations was used to test the difference in PPV between both methods. ⁴⁶ A two‐proportion Z‐test was used to test the difference in number of pharmacy interventions (PIs)/1,000 MOs. ⁴⁷ A P value of <0.05 was considered statistically significant. No statistical comparison was done to compare time spent on DDI management per 1,000 MOs.

RESULTS

Medication order inclusion and DDI alert characteristics

The basic DDI phase (35 days) included 67,188 MOs with 14,787 triggered basic DDI alerts belonging to 1,528 patients, i.e., a mean of respectively 1,920 MOs/day and 423 triggered basic DDI alerts/day. The contextualized DDI phase (50 days) included 92,659 MOs triggering 22,626 basic DDI alerts (in the background) belonging to 2,077 patients, i.e., a mean of respectively 1,853 MOs/day triggering 453 basic DDI alerts/day. Surgery patients accounted for most of the triggered DDI alerts, 4,764 (32%) and 6,647 (29%) of the basic and contextualized DDI phases, respectively. Table  S2 shows the top 30 DDI alert characteristics of both phases. Most triggered DDI alerts were of drugs increasing the risk of gastric ulcers (~ 30%), DDIs involving CYP/P‐glycoprotein/uridine diphosphate glucuronosyltransferase followed (~ 18%), and time‐dependent absorption DDIs (~ 15%). There was no substantial difference in the number of triggered basic DDI alerts/1,000 MOs: 220 and 244 for the basic and contextualized DDI phase, respectively. There were no statistical differences in respect to patients’ age, gender, treating specialty, and number of drugs used at hospitalization between both phases.

Clinical utility

Comparing basic DDI management process with contextualized DDI management in the basic DDI phase

The left‐hand side of Table  3 presents the results of the basic DDI phase for both DDI‐CDS systems, including the differences. During this phase, 3,835 DDI alerts were shown to pharmacy staff using the basic DDI‐CDS system, leading to 107 interventions, resulting in an overall PPV of 2.8%. Using the background data, the contextualized DDI‐CDS system would have shown 498 DDI alerts, resulting in a reduction in displayed DDI alerts of 88.1%, compared with the basic DDI‐CDS system, which would have resulted in a PPV of 23.5% (107/456). Seven of the 107 relevant alerts were not triggered by the contextualized DDI‐CDS system because they had been resolved before starting the batch run. Assuming these seven alerts would have been triggered without previous intervention, no relevant alerts were likely to have been missed in the contextualized DDI‐CDS system, resulting in a 100% NPV.

Table 3.

A comparison of basic DDI‐CDS alerts and contextualized DDI‐CDS alerts for both DDI phases

	Basic DDI phase (35 days)				Contextualized DDI phase (50 days)
	Basic DDI‐CDS system	Contextualized DDI‐CDS system	Difference between systems		Basic DDI‐CDS system	Contextualized DDI‐CDS system	Difference between systems		Difference between phases in clinical practice
	Clinical practice	Background data collection	Change	P value	Background data collection	Clinical practice	Change	P value	Change	P value
Absolute results
MOs	67,188		n/a	n/a	92,659		n/a	n/a	n/a	n/a
Basic DDI alerts triggered	14,787		n/a	n/a	22,626		n/a	n/a	n/a	n/a
DDI alerts shown	3,835a	456	n/a	n/a	5,824 a , b	902	n/a	n/a	n/a	n/a
Pharmacy interventions	107	107 c	n/a	n/a	363 b	373	n/a	n/a	n/a	n/a
Normalized results
Basic DDI alerts triggered/1,000 MOs	220.1		n/a	n/a	244.2			n/a	+10.9%	n/a
DDI alerts shown/1,000 MOs	57.1	6.8	−88.1%	< 0.01	62.9	9.6	−84.1%	< 0.01	−83.1%	< 0.01
PPV	2.8%	23.5%	+20.7%	< 0.01	6.3%	41.4%	+35.1%	< 0.01	+38.6%	< 0.01
NPV	n/a	100%	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
PIs/1,000 MOs	1.6	n/a	n/a	n/a	n/a	4.0	n/a	n/a	+250%	< 0.01
Median time spent on DDI management/ 1,000 MOs	(37.2 minutes) 2233.3 seconds				(13.7 minutes) 821.9 seconds				−63.2%	n/a

In both phases, DDI alerts from the basic DDI‐CDS system and the contextualized DDI‐CDS system are shown. DDI alerts shown in clinical practice are presented in black font; those from the background data collection are shown in gray italic type. The outcome measures used to assess clinical utility are presented in bold font.

DDI, drug–drug interaction; CDS, clinical decision support; MOs, medication orders; n/a, not applicable; NPV, negative predictive value; PPV, positive predictive value.

^a Only a portion of the basic DDI alerts triggered were shown to prescriber as well as pharmacy staff: 14,787 DDI alerts were triggered in the basic DDI phase and 22,626 alerts were triggered in the contextualized DDI phase. Most DDI alerts were not shown based on national assessment of clinical relevance. ⁴⁴

^b Ten DDI alerts were suppressed using basic suppression and were therefore unavailable for intervention in the basic DDI‐CDS system (triple whammy [renin–angiotensin–aldosterone system + diuretic + nonsteroidal anti‐inflammatory drug] n = 9, beta‐blocker + antidiabetic n = 1).

^c Of the 107 DDI alerts with intervention in the basic DDI phase, 7 would have been revealed by the contextualized DDI‐CDS system; however, DDIs were resolved before starting the contextualized DDI‐CDS system batch run.

Comparing the contextualized DDI‐CDS system with the basic DDI‐CDS system in the contextualized DDI phase

The right‐hand side of Table  3 presents the results of the contextualized DDI phase for both DDI‐CDS systems. During this phase, 902 DDI alerts were shown to pharmacy staff using the contextualized DDI‐CDS system, leading to 373 interventions, resulting in an overall PPV of 41.4%. The basic DDI‐CDS system would have shown 5,824 DDI alerts, of which 363 could have led to an intervention, as 10 alerts were suppressed using basic suppression. Background data collection PPV was 6.3%. The difference in PPVs between both systems was 38.6%, P < 0.01.

Comparing clinical practice between phases

A significantly higher PPV of 41.4% was achieved during the contextualized DDI phase, compared with 2.8% during the basic DDI phase, P < 0.01. The same was true for the number of PIs/1,000 MOs: 24.6 during the contextualized DDI phase compared with 9.9 in the basic DDI phase, P < 0.01. The number of PIs was higher for all types of DDIs. Total time spent on DDI management/1,000 MOs was reduced from 37.2 minutes in the basic DDI management phase to 13.7 minutes in the contextualized DDI management phase. For pharmacy technicians, the median time spent performing DDI management/1,000 MOs was reduced from 23.6 minutes in the basic DDI phase to 9.0 minutes in the contextualized DDI phase. For hospital pharmacists this was 13.7 minutes in the basic DDI phase and 4.7 minutes in the contextualized DDI phase. Table  S3 shows the median time spent on DDI management in both phases, based on actual practice and stratified according to type of healthcare professional.

Contribution of different suppression techniques during the contextualized phase

During the contextualized DDI phase, workflow suppression made the most significant contribution to DDI alert suppression (38.9%, n = 42,766), the highest contribution (98.4%) of the previously evaluated DDI alerts. During the same phase, 10,686 unique DDI alerts were suppressed using clinical context suppression. Ignoring workflow suppression, pharmacy technician context suppression would have suppressed 67.9% (n = 36,402) of all triggered DDI alerts by the contextualized DDI‐CDS system. The most significant contributions to suppressed alerts were (i) no alternative available in a hospital setting (30%), (ii) pharmacodynamic risk modification; three‐way DDIs involving only two drugs (24.6%), and (iii) lag time between administrations in absorption time‐dependent DDIs (18.0%).

DISCUSSION

Principal findings

This study demonstrated that contextualized DDI management has greater clinical utility than basic DDI management regarding hospitalized patients in a clinical pharmacy setting. Clinical utility improved in all the outcome measures; PPV was 35.3% higher in the contextualized DDI management process, and NPV was 100% for the contextualized DDI‐CDS system. Furthermore, the number of PIs increased from 1.6/1,000 MOs with basic DDI management to 4.0/1,000 MOs with contextualized DDI management, suggesting a high degree of alert fatigue with basic CDS‐DDI management.

Comparison of this research with other studies

Several previous studies have shown that adding clinical and/or workflow context can significantly reduce the DDI alert burden. Helmons et al. and Daniels et al. reduced the number of alerts by ~ 50%. ¹² , ³³ Improvement in PPV was also demonstrated, i.a., by Eppenga et al., increasing from 9.9% to 14.8% after introducing a contextualized DDI‐CDS. ³² , ³⁵ While all previous studies find improvements in reduction of DDI alerts, the effect size in reduction of DDI alerts (93%) and achieved PPV (41.4%) achieved in this study have not been demonstrated before. Moreover, this study shows that this PPV improvement can be achieved without sacrificing NPV (100%). No previous studies were found that show that decreasing the number of DDI alerts shown to pharmacy staff actually increases the number of pharmacy interventions. Moreover, this was done concurrently with a considerable time reduction to pharmacy staff (37.2 to 13.7 minutes/1,000 MOs).

Study limitations

An obvious limitation of the study was the research design being nonrandomized and open‐label to pharmacy staff and only performed in a clinical pharmacy setting. A crossover design can be sensitive to seasonal and healthcare professional influences. However, in this case the crossover design provided data for both periods using both systems, making these possible influences insightful. The rate of DDIs triggered/1,000 MOs as well as the number of displayed DDIs/1,000 MOs was different between the two phases. However, the rate was higher in the contextualized DDI phase, making the measured difference in clinical utility an underestimation rather than an overestimation. Furthermore, the process change was studied in one hospital and used one EHR. Nonetheless, adopting a similar approach and using the same contextualized DDI‐CDS system, researchers investigating different EHRs have obtained similar results. ³³ The study could also be subject to researcher bias as three of the authors were also hospital pharmacists (in training) performing DDI management in both phases. Nevertheless, these were only three of the 16 members of the hospital pharmacy staff and none had any conflicts of interest.

A technical limitation was that the contextualized DDI process triggered DDI alerts in batches, which could, in theory, leave patients vulnerable to DDIs as pharmacy interventions are not timely performed. However, compared with basic DDI management practice, hospital pharmacists received DDI alerts sooner, as there was no delay in pharmacy technician evaluation. To enhance clinical utility for prescribers, it is necessary to perform this contextualized DDI alerting in real time. Fortunately, this is currently possible and is already being used in several other hospitals across the Netherlands. Another limitation of the study was that there was no expert review of DDI alerts to assess NPV. In clinical practice, however, staff and time constraints inhibit the expert review of each DDI alert. Using PIs in clinical practice resulted in an NPV of 100% under the contextualized DDI management process.

Considerations for current practice and future improvements

This study shows that significant decrease in DDI burden can been achieved by using simple contextual modulators. Based on this study additional more sophisticated contextual modulators don’t seem to be a top priority for a general hospital setting. Analysis of the residual FPs also showed that many FPs could be traced back to workflow‐related technical issues, most of which were resolved by implementing an auto‐refresh function and waiting for the CDS to display the alert after all meta‐rules had been executed. Gaston Pharma has been shown to be easily combined with different knowledge bases and providing contextualized CDS linked to several different CPOEs and/or EHRs.

The greatest benefit as it comes to time savings in the current study was removing the pharmacy technician from the primary DDI alert evaluation on nonabsorption time‐dependent DDIs and replacing them by the contextualized CDS system and the other way around for absorption time‐dependent DDIs.

Basic pair‐wise DDI‐CDS systems in hospital practice are common in Western countries. ²¹ , ³⁰ , ⁴⁸ In the Netherlands, CPOE including basic pair‐wise DDI‐CDS is mandatory in all medical settings as of January 1, 2014. No references have been found using a contextualized CDS‐DDI system in clinical practice on a larger scale. Since October 2020, the Netherlands has, however, moved its DDI knowledge base from a pair‐wise combination model to a decision tree model, comparable to the models used in this study, thus enabling all healthcare providers to benefit from contextualized DDI management in a clinical context. However, it is important to consider that the greatest percentage of suppression during this study was achieved by workflow context suppression. Therefore, consideration should also be given to if and how to deploy and implement workflow contextualization improvements.

CONCLUSION

Contextualized DDI management compared with basic DDI management in a clinical pharmacy setting can considerably decrease the number of irrelevant DDI alerts and thereby increase the time available to interpret relevant DDI alerts, leading to more relevant interventions without missing relevant DDI alerts.

FUNDING

No funding was received for this work.

CONFLICT OF INTEREST

The authors declared no competing interests for this work.

AUTHOR CONTRIBUTIONS

A.T.M.W. wrote the manuscript. A.T.M.W., M.K., T.C.G.E., R.J.E.G., and E.K. designed the research. A.T.M.W., T.M., and B.W.M.v.d.B. performed the research. A.T.M.W. and W.N.C. analyzed the data. A.T.M.W., T.M., and W.N.C. contributed to new analytical tools.

ETHICS APPROVAL

Declared not subject to Research Involving Human Subjects Act (non‐WMO).

Supporting information

Supinfo S1

DATA AVAILABILITY STATEMENT

Freely available after contact with first author.