Practitioners’ Views on Computerized Drug–Drug Interaction Alerts in the VA System

Abstract

Objectives

To assess Veterans Affairs (VA) prescribers’ and pharmacists’ opinions about computer-generated drug–drug interaction (DDI) alerts and obtain suggestions for improving DDI alerts.

Design

A mail survey of 725 prescribers and 142 pharmacists from seven VA medical centers across the United States.

Measurements

A questionnaire asked respondents about their sources of drug and DDI information, satisfaction with the combined inpatient and outpatient computerized prescriber order entry (CPOE) system, attitude toward DDI alerts, and suggestions for improving DDI alerts.

Results

The overall response rate was 40% (prescribers: 36%; pharmacists: 59%). Both prescribers and pharmacists indicated that the CPOE system had a neutral to positive impact on their jobs. DDI alerts were not viewed as a waste of time and the majority (61%) of prescribers felt that DDI alerts had increased their potential to prescribe safely. However, only 30% of prescribers felt DDI alerts provided them with what they needed most of the time. Both prescribers and pharmacists agreed that DDI alerts should be accompanied by management alternatives (73% and 82%, respectively) and more detailed information (65% and 89%, respectively). When asked about suggestions for improving DDI alerts, prescribers most preferred including management options whereas pharmacists most preferred making it more difficult to override lethal interactions. Prescribers and pharmacists reported primarily relying on electronic references for general drug information (62% and 55%, respectively) and DDI information (51% and 79%, respectively).

Conclusion

Respondents reported neutral to positive views regarding the effect of CPOE on their jobs. Their opinions suggest DDI alerts are useful but still require additional work to increase their clinical utility.

Introduction

Adverse drug events (ADEs) are a significant cause of mortality, hospitalization, and emergency department visits. ^1–4 One important contributing factor to ADEs is drug–drug interactions (DDIs). In one study conducted at two tertiary care hospitals, DDIs accounted for 3% of ADEs observed in adult patients. ⁵ The presence of specific DDIs has been associated with a 20-fold increase in the risk of hospitalization among elderly individuals. ⁶ The overall prevalence of potentially serious DDI combinations in adult outpatients has been estimated at less than 1%. ⁷ However, for specific DDIs, the prevalence may be much higher. For example, the warfarin—NSAID interaction has been reported to be as high as 24% of patients receiving warfarin. ⁸

Numerous studies have demonstrated that the use of computerized physician order entry (CPOE) can be an efficient means for decreasing omission errors, ⁹ transcription errors, ¹⁰ serious medication errors, ¹¹ and injury from adverse drug events. ¹² The use of CPOE can also substantially reduce medication errors when clinical decision support features such as drug-disease contraindications and DDI alerts are incorporated into the system. ^13–17 However, implementation of CPOE can have drawbacks as well. Ash and colleagues provided many examples of how implementing patient care information systems, which included CPOE, could foster rather than reduce errors. ¹⁸ In addition, further negative emotions such as shame, guilt, anger, and annoyance, can arise in clinicians as a result of various CPOE features and implementation strategies. ¹⁹

Users of CPOE systems have reported both positive and negative views of these systems. A survey of health providers at two military health care facilities found that CPOE was generally well liked, with an overall satisfaction of 3.8 on a five-point scale where “5” represented the highest satisfaction level. ²⁰ Another survey conducted by Lee et al. at Brigham and Women’s Hospital in Boston found that respondents had reported a reasonable level of overall satisfaction with their CPOE system (mean = 5.1 on a seven-point scale). ²¹ A prospective study conducted by Rind et al. indicated that 44% of physicians considered computerized reminders helpful while 28% felt the reminders were annoying. ²² In an evaluation of the impact of an inpatient CPOE system on patient care, most nurses had more positive views than physicians. ²³ More recently, Murff and Kannry assessed physician satisfaction with two CPOE systems and reported that respondents were more satisfied with the Department of Veterans Affairs (VA) computerized patient record system (CPRS) (mean = 7.2 on a nine-point scale) than with a commercially available product (mean = 3.7). ²⁴ Several challenges in the introduction of an electronic medical record (EMR) system were identified by Scott et al. ²⁵ Many EMR users felt excluded from the selection process for the EMR program, leading to doubts and resistance to use the program. There were also concerns about reduced clinician productivity.

Although CPOE has been evaluated in numerous studies, relatively few published studies have been conducted to specifically examine computerized DDI alerts. ^26–28 Glassman et al. surveyed VA clinicians and found 55% of respondents believed that drug interaction alerts improved their ability to prescribe safely; whereas only 9% disagreed. ²⁷ Nevertheless, 55% of clinicians perceived that poor signal-to-noise ratio moderately or greatly limited use of the alerts. In a survey of general practitioners in the UK, 90.4% of respondents agreed that drug interaction alerts were a useful tool in prescribing but 73.5% agreed that the alerts were sometimes not applicable or relevant to the patient. ²⁸

With the implementation of CPRS throughout the VA health care system in the late 1990s, prescribers now enter prescription orders electronically for review and verification by a pharmacist before dispensing. As a part of the order entry system when two products are prescribed that may interact, the prescriber is alerted to the potential problem. The Department of Veterans Affairs National Drug File Support Group is responsible for the drug interaction package. This working group is responsible for identifying and maintaining the clinical decision rules to trigger DDI alerts. The VA classifies interactions into two groups: “significant” or “critical.” Combinations are considered candidates for DDI alerting if the interaction is pharmacokinetic in nature, such as alterations in absorption, plasma protein binding, enzyme induction or inhibition, or interference with renal excretion. An example of a significant interaction is coprescribing of ciprofloxacin and phenytoin. An example of a critical interaction is the combination of fluvoxamine and phenelzine. All critical alerts require a reason to override the alert and allow the order to be placed. A common complaint among VA practitioners is that many DDI alerts are erroneous because the alerts are based on VA drug classes, not necessarily specific drug products. For example, ophthalmic erythromycin is not viewed differently from oral erythromycin in terms of the potential for a drug interaction. Therefore, prescribers may get multiple nuisance DDI alerts on the same order which can be very aggravating to busy clinicians.

DDI alerts are presented first when an order dialog is accepted and again when the order is actually signed. It is at signature that a reason for overriding a critical DDI would be required, but there is no way to enter a reason for overriding a less than critical DDI. In addition to the alert being provided to the prescriber, the pharmacist will also be subsequently alerted during the verification process if the prescriber decides to continue with the prescription order despite the DDI alert. Unlike many settings where CPOE has just been recently introduced, VA health practitioners have had sufficient experience with CPOE to identify general likes and dislikes. The purpose of this study was to assess VA prescribers’ and pharmacists’ adaptation to the CPRS and their views on a series of statements about computer-generated inpatient and outpatient DDI alerts. To make suggestions for improving the alerts, we also assessed and compared prescribers’ and pharmacists’ preferences of possible changes to DDI alerts.

Methods

A postal survey was used to obtain data from outpatient pharmacists and prescribers within the Veterans Affairs Medical Centers (VAMCs) across the United States. A convenience sample was identified by inviting VAMCs with over 250,000 ambulatory patient visits per year to participate based on previous participation in research projects and/or expressed interest in DDI research. An effort was also made to recruit VAMCs from different geographic areas. At the time of this study, approximately 80 VAMCs had 250,000 or more visits annually. Ten VAMCs were asked to participate in the study but only seven sites (San Francisco VAMC, San Francisco, CA; Southern Arizona VA Healthcare System, Tucson, AZ; Carl T. Hayden VAMC, Phoenix, AZ; Boston VA Healthcare System, Boston, MA; Iowa City VAMC, Iowa City, IA; Ann Arbor VAMC, Ann Arbor, MI; VA Puget Sound Healthcare System, Seattle, WA) contributed data. Within each VAMC, a random sample of 100 to 125 prescribers and 20 to 22 pharmacists was selected to receive a survey. Eligible participants in this survey were identified by a site-specific principal investigator at each local VAMC. Prescribers had to have prescriptive authority within the VAMC (e.g., physician, nurse practitioner, physician assistant) and have an active outpatient practice (e.g., ambulatory care clinic, specialty clinic) at the VAMC. Residents in training were also included as long as they met the previous inclusion criteria. Prescribers who practiced primarily in procedure-driven fields (e.g., surgery) were excluded from this study. Pharmacists had to be employed at the VAMC and have pharmacist privileges in the outpatient or ambulatory care setting. Per diem pharmacists were not eligible to participate.

A separate survey instrument for prescribers and pharmacists was developed for this study by the research team. The survey instruments contained original questions developed specific to the objective of this study and questions from previously published research on DDIs and health information systems. ^27,29–32 Draft versions of the prescriber and pharmacist survey instruments were developed by the research team and presented to staff from the Southern Arizona VA Healthcare System in Tucson, Arizona. Feedback was received for both instruments and incorporated in the final versions of the instruments. The prescriber and pharmacist surveys used the same format and had many overlapping questions although unique questions were included to delineate the prescribers’ and pharmacists’ roles in CPOE. The survey was a one-page, double-sided form. The prescriber survey instrument contained 33 items covering demographics, work characteristics, adaptation to the computer system, sources of drug information, DDI alert burden, DDI alert content, outcomes of DDI alerts, and opinions concerning modification of DDI alerts. The pharmacist survey contained 39 items covering the same areas with the exception of outcomes of DDI alerts and the addition of questions regarding their clinical interventions with prescribers regarding DDI alerts and their proficiency managing DDIs.

Distribution of the surveys followed a modified Total Design Method approach. ³³ Pharmacist surveys were distributed once through VA interoffice mail with the exception of one site (San Francisco) where a second distribution of the surveys occurred due to a low response rate following the initial distribution. Prescriber surveys were also distributed though VA interoffice mail. Approximately eight weeks following the first distribution, a second survey was distributed to prescribers. For both pharmacists and prescribers, the survey was accompanied by a cover letter explaining the purpose of the survey and containing the essential elements of informed consent, and a postage-paid return envelope. Surveys were anonymous and no unique participant identifiers were used. However, a unique code identifying each specific VAMC was included on the surveys. There were no specific incentives provided to participants.

This study was approved by the University of Arizona Human Subjects Protection Committee and by the individual institutional review boards and Research and Development Committees at each participating VAMC. Each VAMC identified a site principal investigator who served as the lead person for the study at the site and was the liaison with the University of Arizona.

Frequency distributions and means were used to describe categorical and continuous variables, respectively. The two-sample t-tests and Mann-Whitney U tests were used to compare prescribers’ and pharmacists’ ratings of the impact of the CPRS on their jobs. For the items regarding possible changes to DDI alerts, a weighted preference score (WPS) was calculated for each statement choice by summing up the preference points. The preference point was calculated by multiplying the percentage of respondents who gave that rank by the corresponding weight of the rank, where a rank of “1” was given a weight of three points, “2” was given two points, and “3” given one point. A higher WPS score suggested a greater preference for the statement choice. An alpha level of 0.05 was used to determine statistical significance. All analyses were performed using SPSS 13.0.

Results

Summary Statistics

The demographic and practice characteristics of the prescribers and pharmacists who responded to the questionnaire are summarized in ▶. ▶ Of the 725 prescribers sampled, 258 returned the questionnaire, giving a response rate of 36%. Of the 258 respondents, 54% were female, 71% worked full time, and 60% reported internal medicine or primary care as their primary area of practice (▶). On average, the respondents were 46.6 ± 10.1 years of age, had practiced as a licensed prescriber for 14.8 ± 10.6 years, and wrote 97.7 ± 155.5 prescriptions in VA outpatient clinics per week (▶). Approximately 97% of the respondents considered their proficiency in using a computer to be at least at the “Average” level (▶).

Table 1.

Table 1. Characteristics of Prescribers

Characteristic	No. (%) ∗ (n = 258)
Gender
Male	117 (45)
Female	139 (54)
Full time/Part time
Full time	183 (71)
Part time	70 (27)
Primary area of practice
Cardiology	16 (6)
Emergency	6 (2)
Endocrinology	10 (4)
Geriatrics	5 (2)
Internal medicine	80 (31)
Primary care	75 (29)
Psychiatry/Mental health	20 (8)
Rheumatology	6 (2)
Other	32 (12)
Self-rated computer proficiency
Novice	1 (<1)
Below average	4 (2)
Average	64 (25)
Good	150 (58)
Expert	36 (14)
Characteristic	mean (SD)
Age	46.6 (10.1)
Years of practice	14.8 (10.6)
Number of half-days/week spent in VA outpatient clinics	5.3 (4.2)
Number of Rx written/week	97.7 (155.5)

^∗ Percentages may not total 100 because of rounding and missing data.

SD = standard deviation.

Table 2.

Table 2. Characteristics of Pharmacists

Characteristic	No. (%) (n = 84)
Gender ∗
Male	36 (43)
Female	47 (56)
Full time/Part time ∗
Full time	73 (87)
Part time	9 (11)
Education/Training†
BS pharmacy	35 (42)
Pharmacy residency	31 (37)
BCPS board certified	6 (7)
PharmD	56 (67)
Specialty residency	8 (10)
Post-graduate degree	6 (7)
Responsibility area in the VA pharmacy†
Outpatient pharmacy	50 (60)
Ambulatory care clinic	39 (46)
Inpatient pharmacy	10 (12)
Specialty clinic	15 (18)
Self-rated computer proficiency
Below average	2 (2)
Average	19 (23)
Good	56 (67)
Expert	7 (8)
Characteristic	mean (SD)
Age	40.4 (10.1)
Years of practice	14.9 (10.2)
Number of years working in the VA health care system	9.5 (7.9)
Number of CPOE medication orders verified/day	119.9 (122.2)

^∗ Percentages may not total 100 because of rounding and missing data.

^† Percentages may be over 100 because of multiple selections.

SD = standard deviation; CPOE = computerized physician order entry.

Of the 142 pharmacists sampled, 84 (59%) returned the questionnaire. Of the 84 respondents, 56% were female, 87% worked full time, 60% reported outpatient pharmacy as their primary area of responsibility, and 67% had a Doctor of Pharmacy degree (▶). On average, the respondents were 40.4 ± 10.1 years of age, had practiced as a licensed pharmacist for 14.9 ± 10.2 years, and verified 119.9 ± 122.2 CPOE medication orders per day (▶). Similar to prescribers, almost all (98%) of the respondents considered their computer proficiency to be average or better (▶).

Adaptation to the CPRS

Respondents were asked to rate the effects of the CPRS on their jobs on one of three five-point scales depending on the question, with ratings ranging from “1” (more difficult, less interesting, or more stressful) to “5” (less difficult, more interesting, or less stressful). In general, both prescribers and pharmacists indicated that the CPRS had a positive impact on their jobs. In particular, most respondents agreed that the system had made their jobs less difficult. Pharmacists revealed statistically more favorable attitudes toward CPRS than prescribers (p < 0.05) (▶), but the difference was relatively small, 0.3 to 0.6 on a one to five rating scale. Prescribers almost never found it necessary to bypass CPRS and use “the old way of doing things” (i.e., handwritten prescription orders) and, on average, had problems with the system only some of the time (▶).

Table 3.

Table 3. Adaptation to the VHA Computerized Patient Record System (CPRS)

	Prescribers Mean ∗ (SD)	Pharmacists Mean ∗ (SD)	Mean Difference
More difficult/Less difficult	3.8 (1.1)	4.1 (0.9)	−0.3†
Less interesting/More interesting	3.6 (1.1)	4.1 (0.8)	−0.6†
More stressful/Less stressful	3.4 (1.1)	3.7 (0.9)	−0.3

^∗ Responses from a five-point scale, with ratings ranging from “1” (more difficult, less interesting, or more stressful) to “5” (less difficult, more interesting, or less stressful).

^† Mann-Whitney U test indicated statistical significance (p < 0.05).

SD = standard deviation.

Table 4.

Table 4. Prescribers’ Views on Statements about Computerized Drug–Drug Interaction Alerts

View	Mean ∗ (SD)	% Often‡
I am satisfied with the accuracy of the DDI alerting system.	3.1 (1.3)	44
How frequently do you have problems with the computerized patient record system (CPRS)?	1.8 (0.8)	3
How frequently do you feel like hitting the computer terminal?	1.7 (0.9)	4
DDI alerts provide me with information that I already know.	3.4 (1.1)	52
The DDI system provides alerts that seem to be just about exactly what I need.	2.7 (1.2)	29
Of all the medication orders you enter during an average day, how often do you receive alerts about potential DDIs?	2.5 (1.0)	16
DDI alerts change my initial prescribing decisions.	1.9 (1.0)	8
How frequently do you find it necessary to bypass the CPRS and use the old way of doing things?	1.3 (0.7)	2
	mean† (SD)	% agree§
DDI alerts should be accompanied by management alternatives.	3.8 (1.0)	73
DDI alerts should be accompanied by more detailed information about the interaction.	3.7 (1.1)	65
DDI alerts should only appear once during the order entry process.	3.6 (1.1)	59
DDI alerts are presented in a useful format.	3.2 (1.1)	46
I feel confident in the computer’s ability to provide me with meaningful DDI alerts.	3.2 (1.1)	43
A DDI alert should not be generated for individual patients who have already had an alert overridden.	3.1 (1.2)	39
It should be more difficult for prescribers to override alerts for potentially lethal interactions.	3.0 (1.2)	36
Prescribers should have the ability to tailor which DDIs generate alerts when they are entering orders.	2.9 (1.1)	26
Prescribers should not be required to enter a reason for overriding a DDI alert.	2.6 (1.2)	24
DDIs considered only significant (vs. critical) should not generate an alert.	2.6 (0.9)	17
DDI alerts, such as those in CPRS, have increased my potential for prescribing medications safely.	3.6 (1.1)	61
The DDI alert system is good at alerting me to clinically important interactions.	3.5 (1.0)	61
DDI alerts are essentially meaningless, a waste of time.	2.2 (1.1)	13

^∗ 1 = Almost never; 2 = Some of the time; 3 = About half of the time; 4 = Most of the time; 5 = Almost always.

^† 1 = Strongly disagree; 2 = Disagree; 3 = Neither disagree nor agree; 4 = Agree; 5 = Strongly agree.

^‡ Includes those who responded most of the time and almost always.

^§ Includes those who responded agree or strongly agree.

SD = standard deviation.

Prescribers’ and Pharmacists’ Views about Computerized DDI Alerts

▶ presents prescribers’ mean rating on each of a series of statements about computerized DDI alerts. Prescribers agreed that DDI alerts should only appear once during the order entry process, and should be accompanied by management alternatives and more detailed information about the interaction. Prescribers also agreed that DDI alerts had increased their potential for prescribing medications safely and that the system was good at alerting them to clinically important interactions. In addition, about half of the time prescribers were satisfied with the accuracy of the alerting system. However, prescribers found that DDI alerts often provided them with information that they already knew.

▶ presents pharmacists’ mean ratings on the statements about computerized DDI alerts. Congruent with prescribers’ views, pharmacists agreed that DDI alerts should be accompanied by management alternatives and more detailed information. They also agreed that it should be more difficult for prescribers to override alerts for potentially lethal interactions. Most pharmacists felt confident in their ability to speak to prescribers about DDIs as well as their ability to determine clinically meaningful DDI alerts. In addition, pharmacists agreed that both prescribers and pharmacists should be required to enter a reason for overriding DDI alerts. The results also indicated that most of the time pharmacists would contact the prescriber about a potentially lethal DDI even if it had been overridden by the prescriber. In addition, pharmacists felt comfortable contacting prescribers about CPOE medication orders that involved an overridden DDI alert. Finally, two-thirds found DDI alerts most of the time or almost always a useful tool in verifying the appropriateness of CPOE medication orders.

Table 5.

Table 5. Pharmacists’ Views on Statement about Computerized Drug–Drug Interaction Alerts

View	Mean ∗ (SD)	% Often‡
I am satisfied with the accuracy of the DDI alerting system.	3.3 (0.9)	45
How frequently do you have problems with the computerized patient record system (CPRS)?	2.0 (0.7)	4
How frequently do you feel like hitting the computer terminal?	1.6 (0.8)	2
When presented with a potentially lethal DDI, I contact the prescriber even though it has been overridden.	4.2 (1.1)	79
I feel comfortable contacting prescribers about CPOE medication orders that involve an overridden DDI alert.	4.1 (0.9)	82
I find DDI alerts a useful tool in verifying the appropriateness of CPOE medication orders.	3.7 (0.9)	67
	mean† (SD)	% agree§
DDI alerts should be accompanied by management alternatives.	4.0 (0.9)	82
DDI alerts should be accompanied by more detailed information about the interaction.	4.4 (0.9)	89
DDI alerts should only appear once during the order entry process.	3.1 (1.1)	44
DDI alerts are presented in a useful format.	3.1 (1.0)	40
I feel confident in the computer’s ability to provide me with meaningful DDI alerts.	3.2 (0.9)	42
A DDI alert should not be generated for individual patients who have already had an alert overridden.	2.7 (1.0)	24
It should be more difficult for prescribers to override alerts for potentially lethal interactions.	4.2 (1.0)	85
Prescribers should have the ability to tailor which DDIs generate alerts when they are entering orders.	2.3 (1.0)	12
Prescribers should not be required to enter a reason for overriding a DDI alert.	1.6 (0.8)	2
DDIs considered only significant (vs. critical) should not generate an alert.	2.5 (0.9)	18
I have confidence in my ability to speak to prescribers about DDIs when they are identified.	4.1 (0.8)	89
I feel confident in my ability to determine which DDI alerts are clinically meaningful.	3.9 (0.8)	79
The large volume of DDI alerts makes it difficult to differentiate clinically important from unimportant interactions.	3.4 (1.0)	48
Clinically important DDI alerts are easily differentiated from other warning messages and drug utilization review (DUR) alerts.	3.0 (0.9)	35
The level of attention that I give a DDI alert depends on the individual prescriber of the CPOE medication order.	2.6 (1.0)	18
The level of attention that I give a DDI alert depends on the type of practitioner (e.g., MD, nurse practitioner, physician assistant).	2.5 (1.0)	18
Prescribers are generally not receptive when I contact them about DDIs.	2.2 (0.8)	7
Pharmacists should not be required to enter a reason for overriding critical DDI alerts.	2.0 (0.9)	6
Computer generated DDI alerts are essentially meaningless, a waste of time.	2.0 (0.8)	2

^∗ 1 = Almost never; 2 = Some of the time; 3 = About half of the time; 4 = Most of the time; 5 = Almost always.

^† 1 = Strongly disagree; 2 = Disagree; 3 = Neither disagree nor agree; 4 = Agree; 5 = Strongly agree.

^‡ Includes those who responded most of the time and almost always.

^§ Includes those who responded agree or strongly agree.

SD = standard deviation.

Possible Changes to DDI Alerts

Respondents were presented with seven potential changes to DDI alerts and were asked to select the three most favorable ones (▶) and rank them from “1” to “3,” with “1” being the most favorable. The results indicated that prescribers most preferred having management options for DDIs; whereas, pharmacists most preferred making it more difficult to override lethal interactions. Other preferable changes included showing DDI alerts one time per patient and customization of DDI alerts. Based on the weighted preference score, the top three favorable changes were the same among prescriber and pharmacist groups but the ranks differ. The least interest was expressed for elimination of all DDI alerts.

Table 6.

Table 6. Respondents’ Preference of Possible Changes to DDI Alerts

	Prescribers WPS ∗	Pharmacists WPS ∗
Eliminate all DDI alerts	0.05	0.05
Eliminate only significant (vs. critical) DDI alerts	0.35	0.36
Eliminate the requirement to provide an override reason	0.49	0.10
Make it more difficult to override lethal interactions	0.95	2.20
Show DDI alerts one time per patient	1.24	1.04
Allow customization of DDI alerts for medications I commonly use	0.84	0.63
Provide management options for DDIs	1.53	1.96

^∗ WPS = weighted preference score, which is the sum of preference points, where a 1% of rank of “1” was given three points, “2” was given two points, and “3” given one point.

Use of Drug Information Sources

▶ presents the frequency with which the respondents reported their most often used sources of general drug and DDI information. Both prescribers and pharmacists relied on electronic references more frequently than any other information source. The next most frequently used source for prescribers was pharmacists; whereas, pharmacists reported printed references as their second most frequently used drug information source. Similar patterns of frequency were observed for general drug information and DDI information. In addition, the results indicated that prescribers relied more heavily on pharmacists to obtain DDI information relative to general drug information. In contrast, few pharmacists consulted a non-pharmacist clinician for DDI information and they were more likely to seek such information from electronic references than for general drug information.

Table 7.

Table 7. Sources of General Drug Information and DDI Information Used by Respondents

	Prescribers		Pharmacists
	General Drug Information	DDI Information	General Drug Information	DDI Information
A non-pharmacist clinician	17%	2%	32%	2%
Pharmacist	30%	51%	25%	23%
Printed reference (e.g., AHFS® †, Facts & Comparisons®)	24%	9%	38%	24%
Electronic reference (e.g., MicroMedex®, Internet, PDA, UpToDate®)	62%	51%	54%	79%

^† American Hospital Formulary Service.

* Column percentages may be over 100 because of multiple selections.

Discussion

This study assessed prescribers’ and pharmacists’ perceptions about CPRS and computer-generated DDI alerts within VA medical centers. In general, perceptions of prescribers and pharmacists were consistent across several categories. Although prescribers and pharmacists self-reported comparable computer proficiency, pharmacists rated the impact of the CPRS on their jobs more positively. The mean differences in ratings were statistically significant; however, the differences did not reach a one-point difference on the rating scale. Also, the clinical importance of the differences in ratings is unknown. Few prescribers (3%) and pharmacists (4%) reported frequently having problems with CPRS. As such, we may conclude that prescribers and pharmacists in outpatient settings at the seven VAMCs were generally well adapted to using CPRS, which at the time of the survey (2004–2005), CPRS had been implemented for several years within the VA health care system. The finding is consistent with previous studies, which demonstrated clinicians’ satisfaction with the implementation of the CPOE system. ^20,21,24 Despite the positive feedback from health providers, the challenges of implementation of CPOE remain. The determinants of successful implementation and acceptance of CPOE in health care facilities have been well studied. ^34–38 In order to exploit the advantages of applying this new technology to health care, lessons should be learned from previous implementations.

The results of this study also suggest that, in general, DDI alerts are viewed favorably by prescribers and pharmacists in the outpatient setting of the VA health care system. Respondents disagreed that DDI alerts were a waste of time. When asked to rank potential changes to the DDI alerts, eliminating them completely ranked the lowest. The VA health care system uses two levels to rate the severity of DDIs: significant and critical. Of the two, a critical severity rating is considered to be of greater clinical importance. When asked to consider the elimination of only significant DDI alerts, and not critical DDI alerts, the majority of respondents also did not favor this option. Given the volume of potential DDI alerts that could be generated, decreasing the burden of alerts would seem like an option that many would favor. However, only 16% of prescribers reported that they were alerted often about DDIs suggesting that they may not feel overwhelmed by them. In fact, 61% of prescribers responded that DDI alerts had increased their potential to prescribe safely and over two-thirds of pharmacists thought that DDI alerts were often useful when verifying prescription orders. Previous research of attitudes towards computerized DDI alerts among community pharmacy managers found almost identical findings with respect to whether DDI alerts were a waste of time. ²⁹ Previous research at a large VA system in Southern California also found that the majority (55%) of prescribers reported that DDI alerts had increased their potential to prescribe safely. ²⁷ Our study extends these previous findings to prescribers and pharmacists across a wider population within the VA.

While it is clear that respondents are positive towards DDI alerts, their responses on other items made it clear that there is a disconnect between what prescribers and pharmacists would like and what they are being provided with the current DDI alerting system. Only 30% of prescribers indicated that DDI alerts provided them with exactly what they needed the majority of the time. Items asking about the format of DDI alerts and the accuracy of DDI alerts found that the majority of respondents was dissatisfied with these dimensions of the alerts. Previous research has found similar results. ³⁹ It also should be noted that DDI alerts are presented to end-users as part of a series of other alerts related to the prescription order entry process (e.g., drug allergy, therapeutic duplication, dose alerts), which can affect how the alerts are perceived and recognized by prescribers and pharmacists. In practice, the majority of these alerts is overridden by prescribers and pharmacists and does not affect the final prescribing decision. ^40–43 There is concern that the low signal-to-noise ratio of alerting systems can desensitize users to all alerts, which may result in important alerts being ignored or missed. ⁴⁴ An additional concern is that the amount of time devoted to unnecessary alerts is a lost opportunity to focus on more important patient care issues. Responses from pharmacists in this study were mixed (i.e., disagree, neutral, agree) on whether DDI alerts were easily differentiated from other types of prescription order entry alerts or warning messages. A potential solution to improving the signal-to-noise ratio for DDI alerts might be to evaluate the usefulness of the concurrent alerts that are provided and to eliminate those that do not contribute to improving the prescribing process. In any case, it is clear that additional research and efforts need to be devoted to improving the signal-to-noise ratio of DDI alerting systems.

Several suggestions that were posed to respondents produced consistent responses from both pharmacists and prescribers and are worth noting. First, providing management alternatives along with DDI alerts was rated most favorably by a majority of respondents. Lack of information about DDI alerts has been previously reported as limiting the usefulness of alerts in the VA system. ²⁷ It could be an opportunity to improve medication prescribing by building up an alert system that educates prescribers about important DDIs and on preferred management strategies in real time when the issue is most relevant to the user. While this type of system would be ideal, it should also be recognized that creating such a system will require substantial effort to develop and maintain. For example, within the VA system, there are over 1,800 significant and critical DDIs that are alerted in the CPRS. DDI references contain monographs for these interactions that could be used to provide background on each interaction, but management alternatives are typically not provided. Developing specific management alternatives for each DDI, while taking into account formulary and other site-specific issues, would be a formidable endeavor. However, it could have a significant effect on improving the utility of DDI alerts and decreasing potential adverse drug events from DDIs. The majority of prescribers also favored having DDI alerts shown only once during the order entry process, as opposed to the current approach where DDI alerts can be displayed multiple times during the order entry process for the same DDI. In contrast, less than half of pharmacists favored having DDI alerts shown only once during the order entry process. Eliminating a DDI alert for a patient after it has been overridden only once per patient was not favored by prescribers and even less favored by pharmacists. Currently, the VA system requires a prescriber and pharmacist to enter a reason when overriding a critical DDI alert. Surprisingly, the majority of respondents did not favor removing this requirement even though anecdotal reports from prescribers and pharmacists have described it as a nuisance for users. Adding the ability for prescribers to customize alerts for commonly used medications was not favored by a majority of prescribers. Other research has found that 31% of prescribers have reported the inability to tailor alerts as a limitation, ²⁷ which is similar to the 26% of prescribers that favored this suggestion in the present study. This study did not attempt to further evaluate the respondents’ reasoning behind their responses. However, it appears that there is reluctance among prescribers and pharmacists to remove or make edits to the DDI alerting system that would disable a potential alert. Further research evaluating reasons for these responses could be useful in developing modifications to DDI alerts.

There are several limitations to this study that warrant cautious interpretation of the study results. The selection of VAMCs was non-random and the results may not be generalizable to all VAMCs. However, we did attempt to select VAMCs from various geographic regions. The overall response rate was low and may limit the representativeness of the respondents. As we had limited information about non-respondents, it was impossible to assess how representative the respondents were, particularly for prescribers. Also, it is possible that those who were most or least satisfied with the DDI alerting system were more likely to return the questionnaire.

Conclusion

This study provides a better understanding of health practitioners’ attitudes toward CPOE in general and DDI alerts. Prescribers and pharmacists generally do not consider DDI alerts a waste of time and find them useful. However, improvements in the DDI alerting such as providing more detailed information, streamlining how DDI alerts are presented, and improving the signal-to-noise ratio of these systems warrant further research for successful implementation.