Physicians’ response to computerised alerts for psychotropic drugs in older persons: a multilevel analysis of the associated alert, patient and physician characteristics

Abstract

Objective

Computerised drug alerts are expected to reduce patients’ risk of adverse drug events. However, physicians over-ride most drug alerts, because they believe that the benefit exceeds the risk. The purpose of this study was to determine the drug alert, patient and physician characteristics associated with the: (1) occurrence of psychotropic drug alerts for elderly patients and the (2) response to these alerts by their primary care physicians.

Setting

Primary care, Quebec, Canada.

Design

Prospective cohort study.

Participants

Sixty-one physicians using an electronic prescribing and drug alert decision-support system in their practice, and 3413 elderly patients using psychotropic drugs.

Primary and secondary measures

Psychotropic drug class, alert severity, patient risk for fall injuries and physician experience, practice volume and computer use were evaluated in relationship to the likelihood of having: (1) a psychotropic drug alert, (2) the prescription revised in response to an alert. Cluster-adjusted alternating logistic regression was used to assess multilevel predictors of alert occurrence and response.

Results

In total 13 080 psychotropic drug alerts were generated in 8931 visits. Alerts were more likely to be generated for male patients at higher risk of fall-related injury and for physicians who established the highest alert threshold. In 9.9% of alerts seen, the prescription was revised. The highest revision rate was for antipsychotic alerts (22.6%). Physicians were more likely to revise prescriptions for severe alerts (OR 2.03; 95%CI 1.39 to 2.98), if patients had cognitive impairment (OR 1.95; 95%CI 1.13 to 3.36), and if they made more visits to their physician (OR 1.05 per 5 visits; 95%CI 1 to 1.09).

Conclusions

Physicians view and respond to a small proportion of alerts, mainly for higher-risk patients. To reduce the risk of psychotropic drug-related fall injuries, a new generation of evidence-based drug alerts should be developed.

Article summary.

Article focus

• Computerised decision support is expected to reduce prescribing errors that may increase the risk of adverse events but most alerts are ignored.

• We assessed whether psychotropic drug alerts were more likely to be generated for higher-risk patients, and for physicians in relationship to their demographic and practice characteristics. We also assessed whether these characteristics were associated with the likelihood that a prescription would be revised in relationship to an alert.

Key messages

• Commercial drug decision-support systems generate many alerts for psychotropic drugs, 97% of which are of moderate or serious severity (not severe), and the majority of these alerts will be ignored by primary care physicians.

• Patients at greater risk of fall-related injuries are more likely to have psychotropic drug alerts, as are male physicians, and physicians who screen out the majority of alerts.

• Physicians are more likely to change a prescription in response to an alert if the patient is a frequent visitor at the clinic and if they have cognitive impairment.

Strengths and limitations of this study

• This is the first study to use empirically generated audit trails within a computerised medical record to simultaneously evaluate the drug, patient and physician characteristics that are associated with the likelihood of having a psychotropic drug alert generated by a commercial drug alert system, as well as the physicians’ response to an alert, in a large cohort of primary care physicians and their elderly patients.

• We were unable to assess all possible alert-related predictors of physician response (eg, type of alert) as the majority of physicians elected to view only the severe alerts.

Introduction

Psychotropic medication use is associated with an increased risk of adverse drug events such as fall-related injuries in older persons.^1–4 Fall-related injuries among older adults are a significant source of morbidity, mortality, and healthcare costs. In the USA, about a third of adults 65 years and older experience a fall annually, resulting in US$19 billions of healthcare service use.^{5 6} Half of the adverse drug events associated with fall-related injuries in older persons are considered preventable and a majority of errors that contribute to preventable adverse drug events occur at the time of prescribing.⁷

Improving the management of psychotropic medications has been identified as a potentially cost-effective means by which to reduce fall-related injuries in older persons.^{7 8} Computerised prescribing systems integrated with clinical decision support systems could serve as a means to more effectively prescribe psychotropic medications because these systems have been found to prevent medication errors by providing physicians with alerts at the point of prescribing.^9–11 However, evidence is mounting that physicians override the majority of computerised drug alerts they receive from such systems—reported at upwards of 90% in some studies⁴—which may limit the utility of such systems in improving medication management of psychotropic medications in older adults.

Physicians’ high rates of over-riding computerised drug alerts may be associated with ‘alert fatigue,’ which has been theorised to occur when a high volume of drug alerts of limited clinical relevance are generated.^12–14 Interventions have been developed to address ‘alert fatigue’ by modifying how alerts are presented and viewed, including reducing the volume of clinically irrelevant alerts through an expert panel review¹⁵ or by providing alerts on-demand to physicians.¹⁶ However, these interventions have met with varying levels of success. Furthermore, evaluations of physicians’ response to alerts suggest that a complex set of factors may shape physicians’ usage of these systems including features of the computerised drug alert system and alerts^{15 17 18} the complexity of patients’ health needs¹⁹ and physician characteristics and practice environment.¹⁹

Identifying the multilevel determinants of response to alerts may help refine the content and delivery of alerts in a manner that is better attuned to physicians’ practice and patients’ needs.

We conducted a study to identify factors associated with physicians’ response to alerts for older adults on psychotropic medications. We also sought to characterise the key steps leading to a physician's response to alerts, including factors associated with the generation and viewing of the alert, as these steps may be linked to whether physicians subsequently respond to an alert.

Methods

Study design and population

We conducted a prospective cohort study with 61 primary care physicians, and all of their patients aged 65 years of age or older who were prescribed psychotropic drugs (n=3413). All computerised drug alerts generated for these patients between 1 January and 31 December 2007 that involved their psychotropic medications were retrieved. This study was approved by the McGill University Institutional Review Board.

Electronic prescribing system

All physicians used a bilingual (English/French) electronic health record and drug management system (MOXXI) that is, integrated with the provincial health insurance agency (Régie de l'assurance maladie du Québec, RAMQ). The RAMQ provides daily downloads of all new medical services and prescriptions for all consenting patients in the practice from its population-based health insurance databases, enabling new health problems, procedures, emergency department visits, hospitalisations and medications to be automatically updated in the patient's health record. The MOXXI system includes a health problem list, an electronic prescriber, information on all medications dispensed including the drug, strength, quantity, duration and prescribing physician, and automated alerts for potential prescribing problems.²⁰

The automated prescribing alerts and drug knowledge database, maintained by Vigilance Santé (Montreal, Canada), generates alerts using information from the patient's current medication list and new electronic prescriptions for all major types of prescribing problems, including excess dose, drug–allergy, drug–drug, drug–disease, drug–age contraindications and therapeutic duplication (www.vigilance.ca). The drug knowledge base classifies all types of alerts by the level of clinical importance on a three-level severity scale, with level 1 alerts defined as ‘severe alerts—avoid completely’, level 2 alerts defined as ‘serious alerts, avoid if possible’ and level 3 alerts as ‘moderate alerts, use with caution’. Alerts are presented in an interruptive manner, forcing physicians to either change the prescription to remove the alert, or enter a reason for ignoring the alert before being able to print the prescription (see the online appendix for screenshots). Physicians may view a detailed description of the interactions or contraindications identified in the alert by clicking in a detail panel. In addition, physicians have the option to suppress and not view serious and/or moderate alerts (eg, level 2 and/or 3) as well as individual drug alerts that he or she disagrees with at the time the alert is generated. In addition, all alerts, current and previously suppressed, can be viewed by the physician in a non-interruptive manner by accessing the alert history section of the application.

Data sources

Audit trails in the MOXXI database were used to measure physicians’ usage of the system, including date and time of use, screens viewed and functions used, prescriptions written, alerts generated, alerts seen by the physician (physician alert level preference) and the physician's response to alerts he or she viewed. Physicians’ characteristics and e-prescribing experience was assessed at the completion of training using questionnaire and standardised e-prescribing skill assessment.¹¹

Patient and practice characteristics were assessed using data from Quebec's universal public health insurance program (RAMQ). For each consenting patient and physician, data were available from the beneficiary, prescription and medical services claims databases to measure beneficiary age, sex and neighbourhood measures of socio-economic status as well as all medical visits, hospitalisations, prescriptions and health problems and injuries recorded in the year prior to enrolment and throughout the study period.²¹ The accuracy of these data have been validated in prior studies.^22–28

Outcomes

Alert generated

Using data from the MOXXI database, we determined, at each patient visit, whether one or more psychotropic drug alerts was generated. An alert could be generated on the basis of existing active medication (the patient's profile of dispensed medication) and/or a newly prescribed psychotropic medication. To evaluate patient and physician characteristics associated with the generation of alerts, each visit was classified as a visit with or without a psychotropic drug alert.

Alert medication revised

For each psychoactive drug alert that was generated at a visit, we first assessed whether the physician saw the alert as a function of the physician's ‘alert level setting’. Among alerts that were seen, we assessed whether the physician made an effort to modify, stop or change the dose of one or more of the drugs implicated in the alert, or alternately over-rode the alert in which case they were required to record the reason for doing so.

Multilevel predictors

Physician-level predictors

Physician demographic characteristics examined included gender and practice experience, defined as the number of years since graduation from medical school. Both were measured at enrolment as these characteristics are associated with prescribing behaviour and may be predictive of alert generation and response. Practice characteristics such as practice volume and the proportion of elderly in the practice are also associated with prescribing behaviour.²⁹ Daily practice volume was calculated based on medical service billing claims made by each physician during 2006, calculated by counting the number of visits the physician billed for that year and dividing by the number of workdays with one or more billings. The percentage of patients 65 years and older was measured as the number of patients 65 years or older that the physician billed a service for in 2006, divided by the total number of patients the physician billed a service for in 2006. Experience and skills related to electronic prescribing was also measured as a factor in alert response as it may influence the speed and confidence in modifying prescriptions in response to an alert. The electronic prescription rate was measured by calculating the number of electronic prescriptions (e-RXs) per 100 visits, and the electronic prescription speed was defined as the time (in minutes) to write three standardised prescriptions electronically. Lastly, the physician-selected alert-level setting was measured as a possible determinant of alert response. Physicians could choose to view severe alerts only (level 1 alerts), view severe and serious alerts (levels 1 and 2 alerts), or view all alerts (levels 1, 2 and 3 alerts) as ‘interruptive alerts’ during the prescribing process.

Patient-level predictors

Five patient attributes were measured as factors that may influence the likelihood of alerts as well as physician response to alerts. Patient age was measured as of 1 January 2007, and patient gender was identified. In addition, patient risk factors for fall-related injuries were measured from ICD9 diagnostic and procedure codes in their medical service claims from the years 2005 and 2006, using a previously validated set of codes for each risk factor.^{21 24 30 31} Risk factors measured included the presence of a prior fall-related injury, gait and balance problems, lower-extremity weakness, and cognitive impairment. We also assessed each patient's medication complexity by counting the number of active medications (prescribed or dispensed, including psychotropic medications) at each visit. Lastly, we measured visit frequency as a reflection of the opportunity to modify treatment, defined as the number of ambulatory care visits made in the year prior to the visit date.

Alert-level predictors

The attributes of each alert was characterised by type of alert, severity and therapeutic class. Type of alert was classified into one of nine mutually exclusive categories: drug–disease contraindication, cumulative side effects, drug–age contraindication, therapy duplication, drug interaction, excess dose, renal dose adjustment, drug or potential cross allergy and other. Alert severity was classified as moderate, serious or severe using the classification established by the vendor (www.vigilance.ca). Therapeutic class of drug(s) causing the alert included antidepressants, antipsychotics, anticonvulsants, benzodiazepines, opioids and other psychoactive classes.

Analysis

To estimate the relationship between the alert, patient and physician predictors and (1) alert generation and (2) response to alerts we used alternating logistic regression. Alternating logistic regression models provide a method of deriving unbiased estimates of association as it permits non-independence of observations with more than one level of clustering to be managed within the context of a dichotomous outcome. For the outcome, alert generated (yes/no), the unit of analysis was visit and the outcome was modelled with two levels of clustering: visits within patients, and patients within physician. For the outcome, alert revised (yes/no), the unit of analysis was alert and the outcome was modelled with two levels of clustering; alerts within patient and patients within physician. For the generation of alerts outcome, sample size was sufficient to include all variables in the multivariate model using alternating logistic regression. However, for the revision of alerts outcome the sample size of alerts that were actually seen by physicians was not sufficient to include all variables in the multivariate model and have it converge. Therefore, only variables found to have a bivariate association with a level of statistical significance of p=0.20 or less were included in the final model predicting revision of alerts.

Results

Overall, 44% of physicians were female. On average, physicians had 23.4 years of practice experience, 21% (SD 11.7%) of patients in the physicians’ practice population were 65 years of age and older, 21 patients were seen per day and 53.7 electronic prescriptions were written per 100 visits (table 1). Physicians varied in computer-based prescribing proficiency with a range in the time required to write three standardised prescriptions of 2.8–10.4 min. Most physicians (82%) set the drug alert preference to view only severe alerts. Eligible patients had a mean age of 75.5 years, 9.4 active drugs and 12.2 visits. In total 67.5% were female, and up to 24.1% had at least one risk factor for fall-related injuries, the most common being gait and balance problems (table 1).

Table 1.

Physician and patient characteristics

Physicians (n=61)
Demographic characteristics	N	%
Female	27	44
	Mean	SD
Practice experience (years)	23.4	7.1
Practice characteristics
Daily practice volume	21.2	7.4
Percent of patients 65 years and older	21	11.7
Experience and skills related to electronic prescribing
Electronic prescription rate (e-RXs per 100 visits)	53.7	45.7
Electronic prescription speed (minutes per 3 e-RXs) (Range)	5.5 (2.8–10.4)	1.7
Alert level setting	N	%
View severe alerts only	50	82
View serious and severe alerts only	9	14.8
View all alerts (moderate, serious and severe)	2	3.2
Patients (n=3413)	Mean	SD
Age	75.5	6.6
	N	%
Female	2303	67.5
Risk factors for fall-related injuries (in prior 2 years)
Prior fall-related injury	147	4.3
Gait and balance problems	822	24.1
Lower-extremity weakness	272	8
Cognitive impairment	288	8.4
Visits (n=8931)	Mean	SD
Number of ambulatory care visits (in year prior to visit)	12.2	9.1
Number of active medications (at visit)	9.4	4.2

In 2007, 13 080 psychotropic drug alerts were generated in the 8931 visits made by study patients; an average of 1.5 psychotropic drug alerts per visit. The majority of alerts were drug–disease contraindications (42.9%) followed by cumulative side effects (23.8%; table 2). Most alerts were of serious (52%) or moderate (45.1%) clinical importance. Antidepressants (45%) and benzodiazepines (30.8%) were most common drugs implicated. Only a minority of the alerts generated (6.2%) were seen because the majority of physicians had set their preferences to see only severe alerts. As a result, alert types most likely to be seen were for excess dose (69.8%), drug or potential cross allergies (29%) and drug–age contraindications (18.3%). Among those that were seen, 9.9% were revised, with the highest revision rate being for excess dose (14.8%), renal dose adjustment (16.7%), severe alerts (16.2%) and for antipsychotics (22.6%).

Table 2.

Characteristics of alerts by stage in alert generation, viewing and revision process

	Alerts generated (n=13080 in 8931 visits)		Alerts seen* (n=807)		Alerts revised (n=80)
	n	%	n	Percentage seen (of alerts generated)	n	Percentage revised (of alerts seen)
Type of alert
Drug–disease contraindication	5606	42.9	144	2.6	16	11.1
Cumulative side effects	3109	23.8	112	3.6	8	7.1
Drug–age contraindication	1600	12.2	293	18.3	30	10.2
Therapy duplication	1345	10.3	102	7.6	8	7.8
Drug interaction	1087	8.3	43	4	1	2.3
Excess dose	126	1	88	69.8	13	14.8
Renal dose adjustment	122	0.9	12	9.8	2	16.7
Drug or potential cross allergy	31	0.2	9	29	1	11.1
Other	54	0.4	4	7.4	1	25
Severity of alert
Severe	387	3	315	81.4	51	16.2
Serious	6797	52	462	6.8	29	8.8
Moderate	5896	45.1	30	0.5	0	0
Therapeutic class of drug(s) causing alert†
Antidepressants	5886	45	239	4.1	9	3.8
Antipsychotics	1531	11.7	62	4	14	22.6
Anticonvulsants	990	7.6	42	4.2	4	9.5
Benzodiazepines	4033	30.8	250	6.2	33	13.2
Opioids	1781	13.6	56	3.1	5	8.9
Other psychoactive classes	2725	21	272	10	24	8.8

*All alerts generated by a patient's profile of active or newly prescribed medication are available for a physician to view, and those shown in an ‘interruptive’ manner are a function of the alert setting preferences that is, set by the physician in accordance with severity (mild, moderate and severe).

†The sum of the alerts by drug class may exceed the total as an alert may be associated with more than one drug class.

Factors associated with alert occurrence

Alerts were more likely to be generated at a visit for patients who had a prior fall-related injury (OR 1.44; 95% CI 1.03 to 2.01; table 3). Female patients after adjusting for greater visit frequency were less likely than males to have a psychotropic drug alert generated during the visit (OR 0.85; 95% CI 0.75 to 0.98). Female physicians (OR 0.61; 95% CI 0.42 to 0.91), and physicians who saw more patients per day (OR per 5 patient increase: 0.89, 95% CI 0.79 to 0.99) were less likely to have psychotropic drug alerts, whereas physicians who set their alert preferences to view only the severe alerts were 99% (95% CI 1.04 to 3.81) more likely to have a psychotropic drug alert generated for their patients.

Table 3.

Patient and physician level predictors of alert generation

Number of visits=8931	Descriptive statistics				Multivariate logistic regression analysis*
	No alert (n=3863)		Alert (n=5068)		OR	95% CI	p Value
Patient
Demographic characteristics	n	%	n	%
Female	2730	70.7	3547	70	0.85	0.75 to 0.98	0.02
Male	1133	29.3	1521	30	Reference	Reference	Reference
	Mean	SD	Mean	SD
Age (per year)	75.7	6.5	75.9	6.6	0.99	0.98 to 1	0.17
Risk factors for fall-related injuries	n	%	n	%
Fall-related injury	302	7.8	580	11.4	1.44	1.03 to 2.01	0.04
Gait and balance problems	938	24.3	1447	28.6	1.15	0.99 to 1.33	0.06
Lower-extremity weakness	317	8.2	435	8.6	0.95	0.75 to 1.21	0.68
Cognitive impairment	193	5	208	4.1	0.91	0.64 to 1.29	0.59
	Mean	SD	Mean	SD
Number of ambulatory care visits (year prior to visit) (OR per 5 visit increase)	11.2	7.7	12.9	10	1.08	1.05 to 1.12	<0.01
Number of active medications (at visit)	8.3	3.8	10.3	4.4	1.12	1.10 to 1.14	<0.01
Physician
Demographic characteristics	n	%	n	%
Female	1411	36.5	1462	28.8	0.61	0.42 to 0.91	0.01
Male	2452	63.5	3606	71.2	Reference	Reference	Reference
	Mean	SD	Mean	SD
Practice experience (OR per 5 year increase)	26.2	5.4	26.5	5.7	0.89	0.76 to 1.03	0.12
Practice characteristics
Daily practice volume (OR per 5 patient increase)	23.1	6.7	21.7	7	0.89	0.79 to 0.99	0.03
Percent of patients 65 years and older (OR per 10% increase)	25.7	10.8	25.5	11.4	1	0.89 to 1.11	0.96
Experience and skills related to electronic prescribing
Electronic prescription speed (minutes per 3 e-RXs) (per minute increase)	5.3	1.6	5.6	1.6	1.01	0.93 to 1.10	0.78
Electronic prescription rate (e-RXs per 100 visits) (OR per 10 e-RX increase)	25.7	13	27.6	12.3	1.06	0.96 to 1.17	0.28
Alert level setting
View severe alerts only	3143	41	4527	59	1.99	1.04 to 3.81	0.04
View serious and severe alerts only, or view all alerts	885	44.9	1085	55.1	Reference	Reference	Reference

*The dataset comprised 8931 visits clustered within 3413 patients clustered within 61 physicians. Alternating logistic regression was used to account for clustering (visits within patients and patients within providers).

Factors associated with alert response

As only 80 alerts were revised, we were unable to include all potential predictors of alert response in a multivariate alternating logistic regression model. Therefore, we included only predictors with a p value ≤0.20 in the bivariate association (table 4). Severe alerts were twice as likely to be revised when compared to serious or moderate alerts (OR 2.03; 95% CI 1.39 to 2.98). Alerts for renal dose adjustment were the most likely type of alert to be revised (OR 1.64; 95% CI 1.09 to 2.46), with a significantly higher likelihood of revision than drug–age contraindications (reference group) and drug interactions (OR 0.49; 95% CI 0.22 to 1.08). Drug alerts for patients with cognitive impairment were 95% more likely to be revised than those with no cognitive problems (OR 1.95; 95% CI 1.13 to 3.36), and the likelihood that an alert would be revised increased for patients who made more visits to their physician (OR 1.05 per 5 visit increase; 95% CI 1.004 to 1.09). Physician sex, practice experience, practice characteristics, computer skill and alert level setting were not associated with the likelihood of responding to an alert.

Table 4.

Factors associated with revising a prescription in response to an alert

Alerts seen=807	Descriptive statistics				Regression analysis*
	Over-ride		Revise		Bivariate			Multivariate
	(n=727)		(n=80)		OR	95% CI	p Value	OR	95% CI	p Value
Alert characteristics
Severity of alert	n	%	n	%
Most severe	264	36	51	64	2.21	1.48 to 3.29	<0.01	2.03	1.39 to 2.98	<0.01
Moderate or less severe	463	64	29	36	Reference	Reference	Reference	Reference	Reference	Reference
Type of alert
Drug–age contraindication	263	36	30	38	Ref	Ref	Ref	Ref	Ref	Ref
Cumulative side effects	104	14	8	10	0.97	0.73 to 1.30	0.86	1.06	0.70 to 1.60	0.79
Drug–disease contraindication	128	18	16	20	0.94	0.64 to 1.38	0.75	0.74	0.44 to 1.23	0.24
Therapy duplication	94	13	8	10	0.97	0.57 to 1.67	0.92	1.14	0.47 to 2.79	0.77
Drug interaction	42	6	1	1	0.67	0.49 to 0.93	0.02	0.49	0.22 to 1.08	0.08
Excess dose	75	10	13	16	0.98	0.50 to 1.92	0.95	0.67	0.34 to 1.34	0.26
Renal dose adjustment	10	1	2	3	1.55	1.07 to 2.25	0.02	1.64	1.09 to 2.46	0.02
Drug or potential cross allergy	8	1	1	1	0.93	0.49 to 1.74	0.81	0.66	0.37 to 1.19	0.17
Other	3	1	1	1	1.42	0.67 to 3.02	0.36	1.41	0.80 to 2.49	0.24
Patient characteristics
Female	475	65	50	63	1.02	0.70 to 1.47	.94	–	–	–
Male	252	35	30	38	Ref	Ref	Ref
	Mean	SD	Mean	SD
Age (years)	74.5	6.2	74.5	6.5	1.02	0.99 to 1.04	0.25	–	–	–
Risk factors for fall-related injuries	n	%	n	%
Fall-related Injury	32	4.4	5	6.3	0.89	0.35 to 2.26	0.80	–	–	–
Gait and balance problems	212	29.2	29	36.3	1.18	0.78 to 1.78	0.44	–	–	–
Lower extremity weakness	61	8.4	7	8.8	1.31	0.68 to 2.51	0.42	–	–	–
Cognitive impairment	103	14.2	12	15	1.67	1.03 to 2.70	0.04	1.95	1.13 to 3.36	0.02
Number of ambulatory visits (year prior to visit) (OR per 5 visit increase)	16.2	23.2	18.6	34.1	1.02	0.99 to 1.06	0.19	1.05	1.004 to 1.09	0.03
Number of active medications (at visit)	10.8	4.9	10.9	4.8	1.01	0.97 to 1.06	0.52
Physician characteristics
Demographic characteristics:	n	%	n	%
Female	119	16	25	31	1.63	0.73 to 3.65	0.23	–	–	–
Male	608	84	55	69	Reference	Reference	Reference
	Mean	SD	Mean	SD
Practice experience (OR per 5 year increase)	24.2	5.4	24.7	6.7	0.96	0.60 to 1.55	0.88	–	–	–
Practice Characteristics
Daily practice volume (OR per 5 patient increase)	20.4	6.1	19.3	6.8	088	0.65 to 1.02	0.38	–	–	–
Percentage of patients 65 years and older (OR per 10% increase)	19.5	9.1	19	8.3	0.83	0.44 to 1.56	0.56	–	–	–
Experience and skills related to electronic prescribing
Electronic prescription speed (minutes per 3 e-RXs) (per minute increase)	5.5	1.6	5.8	1.8	1.18	0.90 to 1.55	0.23	–	–	–
Electronic prescription rate (e-RXs per 100 visits) (per 10 e-RX increase)	31.4	12.4	28.8	15.9	0.84	0.65 to 1.10	0.20	0.86	0.66 to 1.13	0.27
Alert level setting	n	%	n	%
View severe alerts only	323	44.4	48	60	1.80	0.91 to 3.57	0.09	1.11	0.54 to 2.28	0.78
View serious and severe alerts only, or view all alerts	404	55.6	32	40	Reference	Reference	Reference	Reference	Reference	Reference

*The above dataset contained 807 alerts clustered within 303 patients, who were clustered within 47 physicians. For patients clustered within physicians, minimum to maximum cluster size was 1 to 141, respectively. Alternating logistic regression was used to account for two levels of clustering (alerts within patients; patients within physicians).

Discussion

This is the first study to employ empirically generated audit trails with a computerised medical record to investigate the alert, patient and physician characteristics associated with both the generation of psychotropic alerts from a commercially available drug alert system and physician response in primary care. We found that psychotropic drug alerts were common, with 1.5 alerts occurring per visit, that the majority (97%) were moderate or serious alerts (not severe alerts) and that physicians saw a minority (6.1%) of these alerts because they (82%) set their threshold for viewing alerts to see only the severe alerts. Patients at higher risk of fall-related injuries were more likely to have alerts for their psychotropic drugs, and were also more likely to have medications revised in response to an alert. Alerts were more likely to be generated for patients’ psychotropic medications if their physician was male and had established an alert threshold to view only the severe alerts. Additionally, the severity and type of the alert were associated with a physician's response to an alert.

Our findings are coherent with prior research,^{12–17 19 32–34} as well as a recent Delphi assessment of important attributes of drug alert systems to improve clinical utility.³⁵ Alerts are considered clinically useful when graded by severity, when patient risk is considered, and when the probability of an adverse event is significant. Physicians in this study responded in a similar manner; preferentially revising severe alerts and in patients who were at higher risk of an adverse medication-related event. Both Isaac and Grizzle noted a similar pattern.^{17 34} Physicians were more likely to respond to the most severe alerts, and for patients who were being newly exposed to a drug or drug combination, where the risk would be higher than those who had already demonstrated that they could tolerate the medication. These findings as a group suggest that alerts will be most useful if graded by clinical importance, and if patient-specific estimates of risk are calculated and presented, so that only the most clinically relevant alerts could be presented.

However, one of the greatest barriers to developing more clinically useful alert systems is the paucity of empirical evidence of the risk associated with the vast majority of alerts. Most current alert systems, both commercial and home-grown, base the grading of alerts as well as the probability of harm on lowest level of evidence—expert opinion,^36–38 although there are notable recent attempts³⁹ to grade alerts by the strength of the evidence. To make substantial gains in improving drug safety, and reducing the risk of adverse drug events with computerised decision support, real-world evidence needs to be generated on the risks associated with drug alerts, and this evidence needs to be incorporated into a new generation of alert systems. For example, with the increasing use of electronic medical records, an opportunity exists to use clinical and administrative data to monitor adverse drug events and model the actual risks of adverse events associated with prescribed medications. Moreover, priority should be placed on drugs and drug combinations that have the highest population attributable risk, which is a function of the prevalence of use, magnitude of the risk and severity of adverse effects. This would likely include the most prevalent classes of medication—cardiovascular drugs, antibiotics, psychotropic medications, anti-inflammatories and analgesics, as well as those with established high risks such as anticoagulants.

This is the first study to document that physicians who are most likely to generate alerts are also more likely to establish the highest threshold for viewing alerts. This finding, while new, fits with prior observations on the high frequency of overriding computerised prescribing alerts by physicians.^{12 15 16 19} Alert fatigue created by too many drug alerts is a common complaint,^{17 34} and methods of over-riding, or suppressing alerts are highly valued, particularly when alerts are presented in an interruptive manner. However, if physicians with a high volume of drug alerts are more likely to have a higher rate of adverse drug events in their patient population, then facilitating alert over-rides defeats the purpose of implementing computerised decision support in the first place. It has been assumed that reducing the sheer number of alerts to the most serious problems will solve the problem of alert fatigue and alert over-rides. But this assumption may not be correct. A growing body of literature in educational psychology suggests that physicians who have the highest prevalence of quality and safety problems in practice may also be most likely to over-rate the quality of their performance.⁴⁰ If these factors influence alert response, then the physicians who are most likely to generate severe drug alerts may be least likely to respond to them. This possibility needs to be assessed in future research, as more comprehensive feedback, such as comparative benchmarking for quality of drug management as well as patients outcomes, may need to be included with drug alerts to alter physician's self-assessment of performance so that safety and quality problems will be addressed.⁴¹

Our results need to be considered in light of several limitations. First, the number of alerts that were actually seen by physicians and revised was small, and we had insufficient power to identify factors that may have increased the likelihood of responding to an alert by 50% or less. Our sample is limited to primary care physicians and psychotropic drug alerts. The generalisability of these findings to alerts related to other classes of drugs, to other alerting systems, and to other types of physicians or health professionals is not known and would be an important avenue for future research. Lastly, we studied physicians in one jurisdiction, and were unable to assess contextual factors that may modify physician response to alerts such as jurisdictional differences in malpractice litigation risk, funding for the quality of performance rather than for services alone, or the existence of support systems for quality improvement.

In conclusion, to reduce the risk of psychotropic drug-related fall injuries, a new generation of drug alerts should be developed that incorporates empirical estimates of the risk of injury related to psychotropic drugs, as well as patient characteristics. Eventually, these empirical risk estimates could be calculated using clinical and administrative data by systems capable of modelling such data in real time, providing earlier warnings about new drugs associated with increased risks of fall injuries. In addition, the need to incorporate more comprehensive feedback on the quality and outcomes of drug prescribing for physicians that generate a high proportion of alerts should be evaluated in subsequent research.