Recognition and Knowledge of Medications with Black Box Warnings Among Pediatricians and Emergency Physicians

Abstract

“Boxed warnings” (BW), sometimes referred to as “black box warnings,” are the most serious level of warning provided by the Food and Drug Administration (FDA). We aimed to assess physician awareness and knowledge of BW, and to gain a better understanding of where physicians obtain information about serious adverse drug reactions for commonly prescribed medications. A cross-sectional survey instrument was administered to emergency medicine (EM) and pediatrician (Peds) attending and resident physicians. The main outcome measures were physician performance in identifying medications with and without black box warnings and the content of the warnings. The survey response rate was 81/198 (41 %). Respondents correctly identified medications with BW only 36.3 % of the time, but were able to correctly identify medications without such warnings 83.8 % of the time. Attending physicians were better able to identify medications with or without BW when compared with residents (p < 0.05). Among residents, there was a statistically significant increase in the ability to identify medications with or without BW with increasing year of training (p < 0.01). Correct identification of the content of BW was low in both groups (13.3 %). Only 19/50 (37 %) EM physicians and 16/31 (52 %) Peds reported that they consider BW when prescribing medications. 23/81 (29 %) respondents indicated that they did not stay current or had no method of staying current with black box information. EM and Peds attending and resident physicians at a single institution had limited ability to identify medications containing BW or the content of such warnings. A significant number reported that they did not stay current or had no consistent method for staying current with BW.

Electronic supplementary material

The online version of this article (doi:10.1007/s13181-015-0519-3) contains supplementary material, which is available to authorized users.

Introduction

First instituted in 1979, “Boxed warnings” (BW), sometimes referred to as “black box warnings,” are the most serious level of warning provided by the Food and Drug Administration (FDA). The FDA typically reserves such warnings for medications in which a severe adverse drug reaction may lead to death or serious injury. They are aimed at the prescribing health care practitioner and are prominently displayed in the packaging insert surrounded by a black box. The contents of the warnings typically highlight vulnerable patient populations, drug-drug interactions and dosing risks, or information on early detection of reversible conditions [1]. The approval of a new drug by the FDA may be contingent upon the placement of a BW. Alternatively, the warning may be added in the post-marketing interval due to the identification of a serious adverse effect through post-marketing surveillance.

An increasing number of medications have been labeled with BW. In a review of new drug approvals and existing medications from 1996 to 2012, the number of BW increased by twofold after 2004 [2]. As of June 2011, there were more than 400 medications with BW [3]. With the increasing number of BW, it remains to be seen how health care providers will remain up to date, and whether the volume of such warnings will dilute its intended purpose.

Awareness of and “adherence” to BW may be poor. Several studies have examined BW “adherence” in the inpatient and outpatient settings. Kloet et al reported 34 non-adherences out of 293 medications (11.6 %) with BW prescribed to a large group of inpatients. The authors identified two adverse drug reactions, both related to drug-drug interactions [3]. Adherence in the outpatient setting may be better. In a study by Lasser et al., there were 2354 instances of BW “violations” in 33,778 outpatients (7 %) prescribed medications with BW. [4]. A study by Moeller et al. assessed the awareness of BW in pharmacy students. The results indicated that the awareness of BW was proportional to the education level of the student but the knowledge level was inconsistent across many drug classes [5].

The purpose of this study was to assess physician awareness and knowledge of BW, and to gain a better understanding of where physicians obtain information about serious adverse drug reactions for medications they commonly prescribe. To our knowledge, this is the first published survey attempting to assess this information among physicians. We hypothesized that awareness and knowledge of such warnings would be low and that physicians would use diverse resources.

Methods

This study was approved by the University of California San Francisco (UCSF) institutional review board. A cross-sectional survey instrument was administered to attending and resident physicians in their first, second, third, and fourth years of training, within the Department of Emergency Medicine and Pediatrics at UCSF. The survey instrument was aimed to assess physician awareness of medications possessing a BW and familiarity with the warning content for 15 medications (5 with and 10 without BW).

The survey instrument was developed with input from the medical toxicology faculty at the California Poison Control System–SF Division and clinical pharmacists at UCSF.

The medications were selected from a list of the most commonly prescribed medications within our institution and representing a broad range of medication classes (Table 1). We confirmed the presence of a BW and the content of the warning for each of the medications listed by directly reviewing the FDA-approved packaging insert. We collected data on the respondents’ level (attending or resident) and year of training. Respondents were asked to provide the definition of a BW (free-text response). They were then asked to correctly identify those medications possessing a BW (yes or no response), as well as to correctly identify the content of the warning (free-text response). Respondents were also asked in general terms to report how they stayed current with BW information, and whether they routinely considered BW in their decision to prescribe a particular medication (free-text response, eTable 1 for the survey).

Table 1.

Survey medications with boxed warnings and their content

Medication	Black box warning
Ciprofloxacin	Increased risk of tendinitis and tendon rupture. Avoid in patients with known history of myasthenia gravis.
Midazolam (IV)	Respiratory depression and respiratory arrest, especially when used for sedation in noncritical care settings.
Naproxen	Increased risk of serious cardiovascular thrombotic events, myocardial infarction, and stroke. Increased risk of serious gastrointestinal adverse events.
Haloperidol	Increased mortality in elderly patients with dementia-related psychosis.
Metformin	Lactic acidosis is a rare, but serious, complication.

Medications without boxed warnings included in the survey: ondansetron, aspirin, acetaminophen (PO), albuterol, prednisone, glyburide, vancomycin, hydrochlorothiazide, atropine, and acylovir

Reference: http://dailymed.nlm.nih.gov/dailymed/ for FDA package insert details

The survey was administered from October to November of 2014. It was distributed anonymously by e-mail, and included instructions to complete the survey independently and without the use of any reference materials. Free-text responses for BW content were reviewed by two of the authors (CS and RL), each of whom independently graded the responses. A free-text response was considered correct if it matched any part of the BW abbreviated definition listed in Table 1. The authors then met to review the results and any areas of discrepancy were adjudicated and a consensus was reached. The authors disagreed on only 2 out of 55 free-text responses, and were able to reach a consensus on the two disputed cases with further discussion.

Descriptive statistics were performed, including percentage of correct responses for each individual drug, the mean percent correct for presence/absence of a BW for all 15 drugs, and the mean percent correct for the warning content cited for each of the 5 drugs carrying warnings. Scores were recorded, and means and standard deviations were calculated. Comparisons between attending and resident scores were tested by t test. Comparisons between level of training were performed using ANOVA and simple linear regression. Data were analyzed using Stata 13.0.

Results

The survey was distributed to a total of 198 physicians, and there were 81 completed responses (survey response rate of 41 %). Of the respondents, 28 % were attending physicians, 1 % fellows, and 71 % residents. Emergency physicians and pediatricians made up 62 % and 36 % of respondents, respectively. Ninety percent of respondents were able to correctly describe the definition of a BW.

The ability of respondents to correctly identify medications with or without a BW is summarized in Tables 2 and 3. Respondents correctly identified only 36.3 % of medications with BW. At best, 65 % of respondents correctly identified haloperidol as having a BW, compared with only 12 % for midazolam. Respondents were better able to identify medications that did not have a BW (83.8 % accuracy). With the exception of ondansetron, medications without a BW were identified correctly with greater than 75 % accuracy. Interestingly, over 50 % of respondents incorrectly identified ondansetron as having a BW.

Table 2.

Correct identification of medications with black box warning

	Medication (% of respondents correct)					Average total score
	Ciprofloxacin	Haloperidol	Metformin	Naproxen	Midazolam
Specialty (no.)
Emergency medicine (50)	40.0	82.0	50.0	20.0	10.0	40.4
Pediatrics (31)	22.6	38.7	38.7	32.2	16.1	29.7
Level of training
PGY1 (16)	25.0	56.3	43.8	25.0	12.5	32.5
PGY2 (20)	35.0	60.0	55.0	30.0	10.0	38.0
PGY3 (16)	12.5	50.0	43.8	25.0	12.5	28.8
PGY4 (5)	40.0	80.0	60.0	0.0	20.0	40.0
Attending/fellow (24)	50.0	83.3	37.5	25.0	12.5	41.7
Total (81)	33.3	65.4	45.7	24.7	12.3	36.3

PGY postgraduate year

Table 3.

Correct identifications of medications without black box warning

	Medication (% of respondents correct)										Average total score
	APAP	Acyl	Albut	ASA	Atrop	Glyb	HCTZ	Pred	Vanco	Ondan
Specialty (no.)
Emergency medicine (50)	80.0	92.0	94.0	78.0	88.0	80.0	100.0	92.0	90.0	40.0	83.4
Pediatrics (31)	83.9	90.3	100.0	72.4	83.9	71.0	96.8	93.5	87.1	64.5	84.5
Level of training (no.)
PGY1 (16)	75.0	87.5	100.0	75.0	75.0	68.8	100.0	81.3	75.0	68.8	80.6
PGY2 (20)	75.0	90.0	90.0	75.0	75.0	65.0	95.0	90.0	85.0	45.0	78.5
PGY3 (16)	87.5	93.8	100.0	75.0	87.5	81.3	100.0	93.8	87.5	37.5	74.5
PGY4 (5)	80.0	80.0	80.0	60.0	100.0	100.0	100.0	100.0	100.0	60.0	76.0
Attending/fellow (24)	87.5	95.8	100.0	83.3	100.0	83.3	100.0	100.0	100.0	45.8	89.6
Total (81)	81.5	91.4	96.3	76.5	86.4	76.5	98.8	92.6	88.9	49.4	83.8

APAP acetaminophen, Acyl acyclovir, Albut albuterol, ASA aspirin, Atrop atropine Glyb glyburide, HCTZ hydrochlorothiazide, Pred prednisone, Vanco vancomycin, Ondan ondansetron

Table 4 summarizes the composite score for all respondents in identifying medications with and without BW. There was a statistically significant difference in the ability of attendings and fellows to identify medications with and without BW when compared to residents (p < 0.05). Among the residents, there was a statistically significant increase in the ability to identify medications with and without BW with increasing year of training. With each year of training, scores went up by 0.36, 95 % CI 0.15, 0.57 (p < 0.05).

Table 4.

Physician ability to identify the presence or absence of BW (maximum score=15)

	Mean (max = 15) (95 % CI)	Std. dev.
Level of training (no.)
PGY1 (16)	9.69	1.25
PGY2 (20)	9.75	1.92
PGY3 (16)	9.88	1.45
PGY4 (5)	10.6	0.89
All residents (57)	9.84 (9.44, 10.25)	1.53
Attending/fellow (24)	11.04 (10.55,11.53)	1.16
Total	10.2	1.53

p < 0.05 for attending/fellow vs. all residents, p < 0.05 simple regression with increasing level of training

The ability to correctly identify the content of the BW is illustrated in Table 5. Respondents correctly identified the content of the BW with an accuracy of only 13.3 %. Twenty-eight percent of respondents identified the correct content of the BW for metformin, while only 2.5 % of respondents correctly identified haloperidol’s BW concerning risks in elderly patient with dementia-related psychosis. There was no statistically significant difference in the ability of attendings compared with residents to correctly identify the content of a BW. There was also no statistically significant difference in the ability of residents to identify the content of such warnings by year of training.

Table 5.

Ability to correctly identify the content of black box warnings

	Medication (% of respondents correct)					Average total score
	Ciprofloxacin	Haloperidol	Metformin	Naproxen	Midazolam
Specialty (no.)
Emergency medicine (n = 50)	22.0	4.0	38.0	12.0	6.0	16.4
Pediatrics (n = 31)	10.3	0.0	13.0	10.3	13.0	9.3
Level of training
PGY1 (n = 16)	12.5	0.0	25.0	6.3	0.0	8.8
PGY2 (n = 20)	15.0	0.0	30.0	10.0	5.0	12.0
PGY3 (n = 16)	6.3	0.0	31.3	12.5	12.5	12.5
PGY4 (n = 5)	0.0	0.0	40.0	20.0	0.0	12.0
Attending/fellow (n = 24)	33.3	8.3	25.0	12.5	12.5	18.3
Total (81)	17.2	2.5	28.4	11.1	7.4	13.3

Only 37 % of emergency physicians and 52 % of pediatricians reported that they consider BW when prescribing medications. However, of the group of attending physician completing the survey, 74 % indicated that they do consider such warnings. Furthermore, 29 % of all respondents reported that they did not stay current or had no method of staying current with BW information. Methods for staying current on BW information were diverse. Thirty-one percent of respondents used web sites or mobile applications such as Lexicomp or Epocrates, 22 % used clinical pharmacists, and 9 % used word-of-mouth. Other resources identified included the FDA website, newsletters, CME courses, podcasts, journals, and e-mail notifications.

Discussion

The results of this pilot survey establish that attendings and residents from a single academic institution had limited knowledge of medications containing BW or the content of such warnings. With higher levels of training, providers were better able to identify which drugs contained a BW, but were no better at determining the content of such warnings. It is interesting to note that many free-text answers scored as incorrect did mention important drug safety concerns. For example, 63 % of respondents cited QT interval prolongation as the boxed warning for haloperidol. While, the risk of QT prolongation has been emphasized by the FDA and is included as an important potential cardiovascular adverse effect, the boxed warning content is actually related to increased mortality in elderly patients with dementia-related psychosis [6]. It is possible that more experienced physicians attain greater awareness of medications with BW over time, and are able to “brand” them as such. However, they may also become aware of other important significant adverse drug effects for the same medications, and do not distinguish them from those that resulted in the BW.

The majority of respondents reported that they had no consistent method of staying current with BW, and reported a diverse range of resources that they draw upon to obtain such information. Through the DailyMed website, the National Library of Medicine is the official provider of FDA label information [6]. The website provides comprehensive and up-to-date information compiled from drug labeling information submitted to the FDA. The information is provided free of charge as a public service, and the website does not accept advertisements. There were no respondents who indicated that they used this website as a resource when prescribing medications. However, independent databases such as Epocrates, Lexicomp, and Micromedex were cited as resources used by some of the respondents. There is some data indicating that independent databases may contain inconsistent information. Cheng et al found that of 416 marketed prescription drugs with a BW, only 135 (32 %) were covered in all 8 different BW databases evaluated in the study. The sensitivity for identifying a BW drug ranged widely (42–98 %) for each individual database [7].

The results of our study call into question the effectiveness of the BW, and suggest that improvements can be made in the way the FDA communicates serious adverse drug reactions to prescribing physicians. While BW appear prominently in the drug insert, physicians are unlikely to read them. The lack of a well recognized centralized database among our survey respondents, and the increasing numbers of medications with identified serious drug reactions may make it difficult for physicians to stay abreast of these warnings. While the FDA website includes a free e-mail alert service that allows subscribers to receive direct communications from the FDA regarding drug safety, none of our survey respondents indicated that they used this service. All of these factors may contribute to an inadequate consideration of serious adverse drug reactions by prescribing physicians.

This study has several important limitations. While we aimed to maximize survey participation through multiple reminders, the survey’s response rate was only 41 %. For convenience, the study was performed at a single academic institution and only among physicians in the department of emergency medicine and pediatrics. The results could differ among a sampling of community-based physicians or those within different specialties. Additional studies should be performed at other institutions (both academic and community practice) and with a broader range of physicians of various specialties in order to determine whether these results are generalizable.

The survey instrument itself could also introduce bias in a variety of ways. For example, physicians may feel reluctant to answer questions honestly if their responses will reflect poorly on the quality of care they are providing to their patients. The choice of medications included in the survey could be less familiar to emergency medicine (EM) or Peds physicians resulting in a decreased ability to identify BW than if other more familiar medications had been included. We aimed to mitigate such bias by administering the survey in an anonymous fashion, and by choosing medications more commonly prescribed among the respondents. Finally, we also requested that respondents complete the survey without the use of smart phones, or other resources because we wanted to understand physician’s inherent knowledge of BW. In reality, the modern physician actively uses these adjunct resources in real time when prescribing medications to patients. Future studies could consider repeating this survey with and without the use of ancillary resources.

Conclusions

A single cross-sectional pilot survey of emergency physicians and pediatricians at an academic institution showed limited ability to identify medications with BW. With higher levels of training, providers were better able to identify which drugs contained a BW, but were no better at determining the content of the warning. Resident physicians less frequently reported that they considered BW when prescribing medications compared with attendings. Both resident and attending physicians had no consistent resources for obtaining information about BW.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with ethical standards

Conflict of interest

Authors Craig Smollin, Jonathan Fu and Ross Levin declare that they have no conflicts of interest.