The national cost of adverse drug events resulting from inappropriate medication-related alert overrides in the United States

Abstract

Objective

To estimate the national cost of ADEs resulting from inappropriate medication-related alert overrides in the U.S. inpatient setting.

Materials and Methods

We used three different regression models (Basic, Model 1, Model 2) with model inputs taken from the medical literature. A random sample of 40 990 adult inpatients at the Brigham and Women’s Hospital (BWH) in Boston with a total of 1 639 294 medication orders was taken. We extrapolated BWH medication orders using 2014 National Inpatient Sample (NIS) data.

Results

Using three regression models, we estimated that 29.7 million adult inpatient discharges in 2014 resulted in between 1.02 billion and 1.07 billion medication orders, which in turn generated between 75.1 million and 78.8 million medication alerts, respectively. Taking the basic model (78.8 million), we estimated that 5.5 million medication-related alerts might have been inappropriately overridden, resulting in approximately 196 600 ADEs nationally. This was projected to cost between $871 million and $1.8 billion for treating preventable ADEs. We also estimated that clinicians and pharmacists would have jointly spent 175 000 hours responding to 78.8 million alerts with an opportunity cost of $16.9 million.

Discussion and Conclusion

These data suggest that further optimization of hospitals computerized provider order entry systems and their associated clinical decision support is needed and would result in substantial savings. We have erred on the side of caution in developing this range, taking two conservative cost estimates for a preventable ADE that did not include malpractice or litigation costs, or costs of injuries to patients.

BACKGROUND AND SIGNIFICANCE

Adverse drug events (ADEs) result in more than 770 000 injuries or deaths in U.S. hospitals each year.^1–3 These specific injuries result from medical interventions related to a drug. In addition to the human impact, which is great, ADEs that occur during hospitalisation are also costly for hospitals to treat, with expenses estimated to be between $1.56 and $5.6 billion annually.^4–6 The individual cost of a significant or life-threatening ADE has been estimated to range between $2852 and $8116 in community hospitals.⁷ These costs are likely to be conservative estimates, as they did not include malpractice or litigation costs, or the costs of injuries to patients.

Many of these ADEs are preventable, and the number could be reduced if hospitals made changes to their systems for drug ordering and administration.⁴ Computerized provider order entry (CPOE) with clinical decision support (CDS) has the potential to improve the drug ordering process by guiding healthcare providers with their prescribing. These systems can alert providers to potential hazardous drug-drug interactions (DDIs),^8,9 or prompt them to adjust the drug dose based on a patient’s renal function.^10,11 Computerized systems can also detect ADEs early, even if they are not preventable, so that appropriate action can be taken to lessen the severity of the event.

However, the success of these computerized systems depends on how well they have been designed and used. Too many alerts may lead to “alert fatigue” for the user, which can result in providers overlooking even important medication-related alerts.¹² Much of our recent work has focused on evaluating the appropriateness of providers’ alert overrides;^8,9,13,14 few data are available to quantify the additional resource utilization associated with inappropriately overriding these alerts. In this study, we estimated the national cost of preventable ADEs resulting from inappropriate alert overrides in the U.S. inpatient setting.

MATERIALS AND METHODS

We used the following approach to estimate the national cost of preventable ADEs:

• Step 1: Estimate the number of national inpatient medication orders. This involved drawing a random sample of patient data from the Brigham and Women’s Hospital (BWH), Boston, Massachusetts, U.S., and the national inpatient sample (NIS).

• Step 2: Calculate the number of inappropriate alert overrides and avoidable ADEs. This involved drawing on estimates from the published medical literature.^4,14,15

• Step 3: Estimate the national cost of ADEs resulting from inappropriate medication-related alert overrides. Two values were taken from the medical literature.^5,7

We next describe each of these steps in turn, providing further details about how our estimates were calculated.

Step 1: Estimate the number of national inpatient medication orders

We estimated the number of national inpatient medication orders by multiplying the average number of orders per inpatient in our BWH sample by the number of discharges from the NIS in 2014. The NIS is part of the Healthcare Cost and Utilization Project, sponsored by the Agency for Healthcare Research and Quality (AHRQ), and is the largest all-payer inpatient care database in the U.S. It contains clinical and resource use information on all individuals covered by Medicare, Medicaid, private insurance, or uninsured.

We recognized that patient characteristics may have differed between both samples and time periods, and wanted to ensure consistency. We obtained a sample of BWH inpatient data from July 1, 2015, to June 30, 2016, and used this single year of data to estimate the relationship between patient characteristics (age, gender, length of stay [LOS], and acuity) and BWH inpatient medication orders. We used Medicare Severity-Diagnosis Related Group (MS-DRGs) and All Patient Refined DRGs (APR-DRGs) weights in the regression as a proxy for patient acuity. APR-DRGs can be thought of as a refinement of MS-DRGs (finer gradations of conditions by disease and severity) to include all payers and all patients. We obtained data on age, gender, and LOS from the NIS “Inpatient Core File” in our analysis, applying weights as necessary. We obtained MS-DRGs from the “Inpatient Core File” and APR-DRGs from the “Disease Severity Measures File” from the NIS. DRG weights were also updated annually, with the elimination of old DRGs and the creation of new DRGs in NIS data (as old treatments are discontinued, and new treatments emerge), and we accounted for this in our economic analysis.

We used the entire BWH sample to develop the regression model and then extrapolated the results to the most recent year of NIS data available (2014). As the patient characteristics of our BWH sample differed from the national population (eg, gender, age, LOS, and DRG weight, with further details presented in our results section below), we used three different regression models (Basic, Model 1, Model 2) to extrapolate BWH mediation orders to NIS data. For the basic model, we first ran a regression on the BWH sample to estimate the importance of each of these factors (the coefficients for gender, age, LOS, and DRG weight) on the number of medication orders the patient had, and then secondly ran the NIS data through the regression equation (ie., each observation with the patient’s age, gender, LOS, and DRG weight) to predict the number of medication orders we would expect for each patient after adjusting for these sample characteristics. However, regressions such as this with limited data are not an exact science, and the interaction between the four variables and the number of medication orders might not be purely linear, so we also produced variations of the Basic model to see how that affected predicted medication orders, eg, Model 1, we included an interaction between age and gender; Model 2, we included an interaction between age and gender, and we also included age squared. The results presented below are similar for all three.

Supplementary Appendix 1, which is available on request, contains details on how we cleaned, merged, and adjusted BWH and NIS data. For the primary results, we have chosen to present our estimates using the Basic model and MS DRG extrapolation.

Step 2: Calculate the number of inappropriate alert overrides and avoidable ADEs

We drew on the published medical literature to estimate the number of drug-allergy, drug-drug, duplicate drug, and renal-dosing alerts triggered, and inappropriately overridden.^14,15 A drug allergy interaction alert was displayed if the patient had a definite (the drug being ordered was an exact match to the allergen), probable (the drug being ordered was in the same allergen group), or possible reaction (cross-sensitivity was considered likely), as well as the specific type of reaction that the patient may be at risk of developing, eg, hives. A drug-drug interaction alert was generated if an undesirable interaction was likely to occur between two drugs and cause serious injury to the patient. A duplicate drug alert was displayed if there were two or more orders for the same medication or for medications in the same therapeutic class. The renal-based medication substitution alerts were based on a calculation of the patient’s creatinine clearance using weight, height, age, sex, and most recent creatinine level, and suggested substituting a particular medication. All alerts were interruptive and required the user to give a reason if he or she decided to override the alert. We also calculated the number of avoidable ADEs resulting from inappropriate overrides in U.S. hospitals nationally.⁴

Step 3: Estimate the national cost of ADEs resulting from inappropriate medication-related alert overrides

We used two estimates for the cost of a preventable ADE in the inpatient setting from the published medical literature. The first value was taken from a study conducted in two tertiary care centers by Bates et al⁵, which found an average avoidable ADE cost of $4685 in 1993. The second estimate was taken from a study conducted in six community hospitals by Hug et al⁷, which found that a preventable ADE cost $3511 in 2005/2006. We inflated both reported costs per ADE to 2016 dollars ($8968 and $4430, respectively) using the Producer Price Index for primary services at general medical and surgical hospitals.¹⁶

Estimating the time and cost of responding to CDS alerts

We also estimated the time that a clinician and pharmacist can spend viewing and responding to each medication-related alert (approximately 4 seconds), and the implicit hourly wage ($90 and $58, respectively) to determine the opportunity cost of that time.^16,17 We also calculated a loaded hourly rate ($117 and $76, respectively) to include cover for additional charges (eg, insurance, sick time, vacation time), and payroll taxes.

RESULTS

Step 1: Estimate the number of national inpatient medication orders

The full BWH inpatient sample contained 40 990 patients. Using three regression models to extrapolate BWH medication orders (1 639 294) to NIS data, we estimated that 29.7 million adult inpatient discharges in 2014 resulted in 1.02 billion to 1.07 billion medication orders (see Table 1). These extrapolations also showed how there was only 4.25% difference between assigning APR-DRGs compared to MS-DRGs to all patients. Within each extrapolation, there was less than a 0.5% difference between the smallest (eg, Model 2_Column 1) and the largest estimate (eg, Basic model_Column 1).

Table 1.

Number of estimated national inpatient medication orders in 2014 by DRG type

Regression Model Specification	2014 NIS Data Using MS-DRG Weights		2014 NIS Data Using APR-DRG Weights
	Total (billions)	Per patienta	Total (billions)	Per patienta
Basic model	1.073	36.1	1.028	34.6
Model 1	1.070	36.0	1.024	34.4
Model 2	1.069	35.9	1.023	34.4

^acalculated from BWH sample.

Comparison of BWH data sample and NIS

We compared the characteristics of the BWH inpatient sample to the NIS sample (Table 2). Compared to the NIS sample, the BWH inpatient sample was more female (61% versus 59%), younger (average age 55.2 versus 57.3), more severely ill (as measured by a longer hospital LOS; 5.2 days versus 4.7 days), and with a higher average MS-DRG weight (2.21 versus 1.49) and higher average APR-DRG weight (1.54 versus 1.14).

Table 2.

A comparison of BWH and NIS patient samples

Variable	BWH Sample (1st July 15–30th Jun 16)						2014 NIS [a]
	Full Sample [a]		Sample Patients with MS-DRGsa		Sample Patients with APR-DRGsa
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Female (%)	61	49	52	50	68	47	59	49
Age	55.16	18.90	68.51	15.01	44.75	14.58	57.26	20.48
LOSb	5.22	6.74	5.87	6.69	4.72	6.74	4.74	6.26
MS-DRG weightb,c	1.84	2.18	2.21	2.19			1.49	1.38
APR-DRG weightb,c					1.54	2.12	1.14	1.32
Med. orders per patient	39.99	44.69	44.93	46.12	36.12	43.16	NA	NA
Total number of patients in each sampleb	40 990d		17 954		22 878		5 949 232e

^aExcluding patients under the age of 18.

^bPlease refer to Supplementary Appendix 1 for details on how the data were cleaned and adjusted.

^cFor full BWH inpatient sample, DRG weights are a weighted average of patients with MS DRGs and patients with APR DRGs; for NIS sample, DRG weights are the average if all patients are assigned MS DRGs or all patients are assigned APR DRGs, respectively.

^dThe full BWH inpatient sample included about 158 BWH patients (0.004%) for whom TRIS-DRGs were reported; thus, the numbers of MS DRG patients and APR DRG patients do not sum to the total sample size.

^e5.9 million observations in the NIS represented 29.7 million hospital patient discharges.

We found MS-DRGs and APR-DRGs were reported for about 43% and 56% of the BWH inpatient sample, respectively. We compared the characteristics of the full BWH inpatient sample to the subset of the BWH sample that had MS-DRGs only. The two BWH samples (full sample and subsample) of patients were similar to NIS patients in terms of age and gender, but were both more severely ill than the national average measured by LOS and DRG weight. As the BWH is a large, urban, teaching, tertiary care hospital, it was reasonable to expect that the full BWH inpatient sample would be more severely ill than the national average.

The full BWH inpatient sample also included 158 (0.004%) patients for whom TRIS-DRGs were reported. TRIS-DRGs are for TriCare patients, which is the healthcare provider for members of the U.S. military, and their DRG weights were found to be very similar (but not identical) to MS-DRGs.

Step 2: Calculate the number of inappropriate alert overrides and avoidable ADEs

Starting with the projected number of inpatient medication orders (1.02 billion to 1.07 billion) estimated in the previous step, we multiplied this by the percent (7.3) of medication orders that had triggered alerts in the BWH sample. We concentrated on only four main types of medication-related alerts: drug-allergy, drug-drug, duplicate drug, and renal-dosing, using rates from the medical literature.^14,15 This resulted in an estimated 75.1 million to 78.8 million medication alerts in U.S. hospitals in 2014, depending on which model and set of DRG weights were used. Taking the Basic model with MS-DRG weights (78.8 million), we then used this to estimate the total number of alert overrides (57.6 million). Of these, we estimated that 5.5 million might have been inappropriately overridden, resulting in approximately 196 600 ADEs (see Table 3). Using all three regression models and both types of DRG extrapolations, the projected national number of ADEs ranged from 196 600 (Basic model and MS-DRG weights) to 187 400 (Model 2 and APR-DRG weights).

Table 3.

Estimated number of national alerts and overrides by alert type, appropriate and inappropriate overrides, and ADEs using Basic model and MS-DRG weights for NIS

Alert Type	No. of Alerts (millions)	No. of Overrides (millions) (%)a	Appropriate Overrides (millions) (%)b	Inappropriate Overrides (millions) (%)c	Inappropriate Overrides Resulting in ADEs (thousands) (%)d
Drug allergy	48.23	39.51 (81)	38.12 (96.5)	1.38 (3.5)	13.8 (1.0)
Drug–drug interaction	13.77	9.39 (68.2)	5.82 (62.0)	3.57 (38.0)	107.0 (3.0)
Duplicate drug	16.15	8.38 (51.9)	8.21 (98.0)	0.17 (2.0)	1.7 (1.0)
Renal-dosinge	0.61	0.39 (62.4)	0.008 (2.2)	0.37 (97.8)	74.0 (20.0)
Total	78.75	57.65 (73.3)	52.16 (90.5)	5.49 (9.5)	196.6 (3.6)

^aThe number of alert overrides (column 3) was calculated by multiplying the number of alerts, eg, 48.23 for drug allergy (column 2) by the percentage of alerts overridden, eg, 81% for drug allergy.

^bThe number of appropriate overrides (column 4) was calculated by multiplying the number of overrides, eg, 39.51 million for drug allergy (column 3) by the percentage of appropriate overrides, eg, 96.5% for drug allergy.

^cThe number of inappropriate overrides for drug allergy (column 5) was calculated by multiplying the number of overrides, eg, 39.51 million for drug allergy (column 3) by the percent of overrides deemed inappropriate, eg, 3.5% for drug allergy (column 5) taken from Slight et al.¹⁴ Percentages for the other alert types were taken from Nanji et al.¹⁵

^dSince 1.0% of inappropriate drug allergy overrides resulted in ADEs, we ended up with 13.8 thousand ADEs.

^eTo calculate the % of renal-dosing alert overrides, we drew on the results of Chertow et al’s study where clinicians overrode 9012 (62.4%) of the 14 440 alerts generated. If the clinician’s final order fell within the CDS parameters, we deemed this as “appropriate.”

Step 3: Estimate the national cost of ADEs resulting from inappropriate medication-related alert overrides

We multiplied the estimated 196 600 ADEs expected to occur in U.S. hospitals because of inappropriate alert overrides by the estimated cost per preventable ADE based on Bates et al.⁵ This was projected to result in costs of $1762.6 million for treating preventable ADEs. Using the estimated cost of a preventable ADE published by Hug et al, this was projected to result in costs of $871 million for treatment.

Estimating the time and cost of responding to CDS alerts

The 78.8 million alerts were projected to cost 87 500 hours of clinician time and 87 500 hours of pharmacist time over the course of a year, with an opportunity cost of $10.2 million and $6.6 million, respectively. The avoided costs associated with the averted ADEs exceeded the opportunity cost of clinicians’ and pharmacists’ time responding to the alerts by more than two orders of magnitude using the Bates et al’s⁵ cost per ADE, and by more than 1.5 orders of magnitude using the Hug et al’s⁷ cost per ADE. Thus, the intervention would “break even,” even if clinicians and pharmacists each took almost seven minutes per alert using the Bates cost per ADE, and almost seven minutes per alert using the Hug cost per ADE.

For the basic regression model (see Table 4), we estimated that if clinicians and pharmacists each spent an average of 4 seconds to respond to each alert and the average implicit wage for clinicians and pharmacists is $117 and $76 per hour, respectively, then clinicians and pharmacists would have jointly spent 175 000 hours (87 500 hours each) responding to the 78.8 million alerts (MS-DRG extrapolation) compared to 167 600 hours (83 800 hours each) responding to the 75.5 million alerts (APR-DRG extrapolation). This would result in an opportunity cost of $16.9 million (MS-DRG extrapolation) compared to $16.2 million (APR-DRG extrapolation). Using our cost estimates of an ADE and extrapolating using MS-DRG weights, 5.5 million inappropriate alert overrides resulted in 196 600 ADEs costing the healthcare system $1760 million based on Bates et al, and $871 million based on Hug et al. Extrapolating using APR-DRG weights, 5.26 million inappropriate alert overrides resulted in 188 300 ADEs, costing the healthcare system $1690 million based on Bates et al, and $834 million based on the Hug et al study. (see Table 5).

Table 4.

Estimated time responding to alerts and total cost of ADEs resulting from inappropriate alert overrides

Alert Type	Estimated Time Responding to Alerts by Alert Type, Basic Model, and MS-DRG Weights for NIS		Total Cost of ADEs by Alert Type, Basic Model, and MS DRG Weights for 2014 NIS (millions of 2016 $$)
	Hours responding to alerts (thousands)	Opportunity cost of responding to alerts (millions 2016 $$)	Annual cost of ADEs, Bates, et al5(millions)	Annual cost of ADEs, Hug, et al7(millions)
Patient allergy	107.2	$10.3	$124.0	$61.3
Drug–drug interaction	30.6	$2.9	$959.6	$474.0
Duplicate drug	35.9	$3.5	$15.0	$7.4
Renal-dosing	1.3	$0.13	$664.0	$328.0
Total	175.0	$16.9	$1762.6	$871

Table 5.

The estimated cost of ADEs and time responding to alerts

Estimated time responding to alerts by model specification and DRG type
Regression model specification	NIS MS-DRG Weights		NIS APR-DRG Weights
	Hours Responding to Alerts (thousands)	Opportunity Cost Responding to Alerts (millions 2016 $$)	Hours Responding to Alerts (thousands)	Opportunity Cost Responding to Alerts (millions 2016 $$)
Basic model	175.0	$16.9	167.7	$16.2
Model 1	174.4	$16.8	167.0	$16.1
Model 2	174.3	$16.8	166.9	$16.1
Estimated Cost of ADEs by Model and DRG Type (millions of 2016 $$)
Regression model specification	NIS MS-DRG Weights		NIS APR-DRG Weights
	Cost of ADEs		Cost of ADEs
	Bates, et al	Hug, et al	Bates, et al		Hug, et al
Basic model	$1762.6	$871	$1688.8		$834.3
Model 1	$1756.9	$868	$1682.4		$831.1
Model 2	$1755.1	$867	$1680.8		$830.3

DISCUSSION

We estimated that 29.7 million adult inpatient discharges in 2014 resulted in approximately a billion medication orders, which in turn generated about 75 million medication alerts in 2014. We then estimated that approximately 5.5 million medication-related alerts were inappropriately overridden, resulting in approximately 196 600 ADEs nationally, costing between $871 million and $1763 million. These data suggest that further optimization of hospitals’ CPOE systems and their associated CDS is needed and would result in substantial savings.

This model-based economic analysis has focused primarily on setting out a range of plausible estimates for the total cost of ADEs resulting from inappropriate medication-related alert overrides in the U.S. inpatient setting. We have erred on the side of caution in developing this range, taking into account two cost estimates for a preventable ADE from the published medical literature. One cost estimate was taken from a study conducted at the same large tertiary care hospital from which we drew our two inpatient samples (full sample and subsample).⁵ However, this BWH cost estimate ($4685) could be considered an overestimation, as our data showed how this hospital treated patients who were more severely ill compared to the national average. That said, our second cost estimate ($3511) was taken from a study conducted in six community hospitals by Hug et al⁷, where the costs of a preventable ADE appeared slightly lower. These costs are also likely to be conservative estimates, as they did not include malpractice or litigation costs, or the costs of injuries to patients. As the majority of healthcare expenses (diagnoses and intervention) are usually incurred early in a patient’s hospital stay, Taheri et al¹⁸ estimated only the incremental cost of the final hospital day for a patient. This value can reasonably be assumed to be an underestimate of the true costs in most cases, as ADEs are likely to result in an increased LOS of between 2.2 days and 3.1 days.^5,7

Our economic analysis focused only on the consequences for hospitals of providers inappropriately overriding medication-related alerts. We considered the more common types of medication-related alerts likely to have been implemented in hospitals in this analysis (drug-allergy, drug-drug, duplicate drug, renal suggestion). One question that emerges is where should hospitals focus their efforts to help reduce these preventable costs in the future? We know from the published literature that the rate of inappropriate alert overrides varies substantially by alert type.⁸ Drug-drug interaction alerts in particular were found to be often inappropriately overridden by healthcare providers. A key issue here is that too many drug-drug warnings are overridden because providers are bombarded with these alerts, and a key approach will be to display only the most important warnings. The most common override reason cited for drug-drug interactions was “will monitor as recommended.”^7,8 Drug monitoring is another task that is often not carried out as intended,⁹ with some providers possibly less clear about what specific parameters should be monitored or indeed when such monitoring should be performed. Hospital CPOE systems with CDS could be further optimized to address this by including more specific information in the content of their alerts or facilitating the ordering of a particular drug level at the same time these alerts are presented. Although it had some of the highest values, adjustment of drug dosing according to level of renal function and age is also not yet done in most institutions,¹⁰ which represents another major missed opportunity. We acknowledge that our results could be interpreted in many different ways, eg, institutions could interpret the low opportunity cost of the alert to the high cost of avoiding an ADE and be encouraged to add more medication-related alerts. However, our analysis was conducted at one point in time and does not observe the rates of change over time. If the numbers of medication-related alerts were to increase, alert fatigue among system users may also increase. We anticipate that this “diminishing returns” to additional alerts could actually increase the cost in terms of frequency of ADEs per order, and so we would advise caution in interpreting our results in this way.

Limitations

The evidence base from which we obtained our model inputs has limitations. First, our two cost estimates were obtained from studies conducted in the state of Massachusetts, and it is possible that these costs may not be generalizable to other states due to differences in patient populations. Second, our analysis is also dependent upon the quality and accuracy of the hospitals’ cost accounting systems from which we obtained our cost estimates for a preventable ADE. Third, as mentioned above, we obtained data on age, gender, and LOS from the NIS “Inpatient Core File” in our analysis, applying weights as necessary. We noted that a very small amount of patient observations (n = 3507) out of a total of 5 952 739 patient observations was missing at least one of these variables, and so the number of medication orders was predicted for 5 949 232 patient observations representing 29.7 million hospital discharges. Fourth, we considered the more common types of medication-related alerts in this analysis (drug-allergy, drug-drug, duplicate drug, renal suggestion), but acknowledge that there are other types of medication-related alerts, such as age-based and formulary alerts, which have not been included. Furthermore, our inappropriate alert override rates were obtained from studies conducted at a large, academic healthcare center.

CONCLUSION

The cost of ADEs resulting from inappropriate medication-related alert overrides is substantial. We estimated a projected total cost of $871 million for U.S. hospitals to treat these preventable ADEs; this cost did not include the costs of injuries to patients or malpractice costs, which can be considerable from a societal perspective. There is scope to further optimize hospitals’ CPOE systems with CDS to improve patient safety; additional resource invested in this area is warranted.

FUNDING

This study was supported by the Agency for Healthcare Research and Quality (AHRQ), grant number U19HS021094.

CONTRIBUTORS

DWB conceived this study and secured the funding for this work. SPS and CF conducted the data collection, analysis, and interpretation. DWB also contributed to the analysis and interpretation of data. SPS led the writing of this manuscript with all coauthors commenting on drafts of the paper. All authors gave their approval for the final version to be published. DWB and SPS are guarantors.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Conflict of interest statement. None declared.