Abstract
Objective
To develop a practical approach for implementing clinical decision support (CDS) for medication black box warnings (BBWs) into health information systems (HIS).
Methods
We reviewed all existing medication BBWs and organized them into a taxonomy that identifies opportunities and challenges for implementing CDS for BBWs into HIS.
Results
Of the over 400 BBWs that currently exist, they can be organized into 4 categories with 9 sub-categories based on the types of information contained in the BBWs, who should be notified, and potential actions to that could be taken by the person receiving the BBW. Informatics oriented categories and sub-categories of BBWs include – interactions (13%) (drug-drug (4%) and drug-diagnosis (9%)), testing (21%) (baseline (9%) and on-going (12%)), notifications (29%) (drug prescribers (7%), drug dispensers (2%), drug administrators (9%), patients (10%), and third parties (1%)), and non-actionable (37%). This categorization helps identify BBWs for which CDS can be easily implemented into HIS today (such as drug-drug interaction BBWs), those that cannot be easily implemented into HIS today (such as non-actionable BBWs), and those where advanced and/or integrated HIS need to be in place to implement CDS for BBWs (such a drug dispensers BBWs).
Conclusions
HIS have the potential to improve patient safety by implementing CDS for BBWs. A key to building CDS for BBWs into HIS is developing a taxonomy to serve as an organizing roadmap for implementation. The informatics oriented BBWs taxonomy presented here identified types of BBWs in which CDS can be implemented easily into HIS currently (a minority of the BBWs) and those types of BBWs where CDS cannot be easily implemented today (a majority of BBWs).
Keywords: Black box warning, BBW, clinical decision support, CDS, health information systems, HIS, taxonomy, patient safety
1. Background
Black box warnings (BBWs) are the Food and Drug Administration’s (FDA’s) strongest warning for medicines that carry risk of special problems, especially death or serious injury [1]. Currently, BBWs exist for more than 400 prescription medications [2] (►Table 1). Medications with BBWs are regularly prescribed, but BBW recommendations are not routinely followed. Of the 40% of patients who receive BBW medications, up to 40–50% do not get the recommended laboratory testing recommended in the BBW [3–4]. BBWs for possible medication interactions were not followed 36% of the time [5]. Estimates of the number of patients with a contraindicated diagnosis who received drugs despite a BBW range from 1–25% [5–6]. Studies of risk communication for cisapride showed that labeling changes, including BBWs, failed to change prescribing behavior [7–8].
Table 1.
Black box warning (BBW) drugs/class (Source: Formulary Productions, Accessed May 2011)
| BBWs Drug class | # of drugs |
|---|---|
| Analgesics | 41 |
| Anticonvulsants | 7 |
| Antidiabetics | 4 |
| Anti-Infective (Aminoglycosides) | 5 |
| Anti-Infective (Antifungals) | 4 |
| Anti-Infective (Antituberculins) | 2 |
| Anti-Infective (Antivirals) | 24 |
| Anti-Infective (Fluoroquinolones) | 6 |
| Anti-Infective (Miscellaneous) | 18 |
| Antineoplastics | 67 |
| Cardiovascular Agents | 51 |
| CNS (Stimulants) | 7 |
| CNS (Depressants) | 1 |
| NS Drugs (Anesthesia) | 7 |
| CNS Drugs (Skeletal Muscle. Relaxants) | 3 |
| CNS Drugs (Antiparkinson) | 3 |
| Contrast Agents | 10 |
| Dermatologic Agents | 9 |
| Gastrointestinal Agents | 8 |
| Hematologic Agents | 29 |
| Hormones (Sex Hormones) | 44 |
| Immunologic Agents and Biologics | 21 |
| Psychiatric Agents | 51 |
| Renal Agents | 11 |
| Respiratory Agents | 5 |
| Vaccines | 1 |
| Vitamins/Iron | 5 |
| Miscellaneous | 32 |
| Total | 476 |
Informatics clinical decision support (CDS) tools can be deployed in health information systems (HIS) to improve compliance with other types of best practice recommendations, when implemented thoughtfully [9–11]. To date, no systematic approach for effective CDS for all types of BBWs within HIS exists. As HIS become more prolific, advanced, and integrated, the ability to incorporate informatics tools to assist with BBW compliance will become increasingly important. Furthermore, federal and state governments are focusing more on ensuring that healthcare systems have mechanisms in place to minimize adverse drug events (ADEs) from medications, including BBWs related ADEs [12].
Here we present a taxonomy for conceptualizing BBWs from an informatics perspective. BBWs are very heterogeneous and different BBWs apply to medications used in different setting. Also, BBWs are directed at different types of professionals and even sometimes patients in the medication chain from prescribing a medication to taking the medication. Our goal was to create an all-encompassing BBW taxonomy. We developed this taxonomy to formulate our overall approach to trying to implement CDS tools for all BBWs within our healthcare system. To our knowledge this is the first informatics oriented BBW taxonomy ever developed.
2. Objectives
To develop a practical approach for implementing clinical decision support (CDS) for medication black box warnings (BBWs) into health information systems (HIS).
3. Methods
Beginning in the first half of 2010, we examined the more than 400 prescription medications with BBWs [2] (►Table 1). First, all BBWs were reviewed and summarized by the one of the physician authors and the two pharmacist authors. Secondly, the research team identified two general BBW themes – what action the BBW was recommending and to whom the BBW was aimed. After further review of all BBWs, four major categories and ten sub-categories were identified. Finally, each BBW was categorized into one of the ten sub-categories of the taxonomy. For the final categorization, each BBW was reviewed independently by one of the physician and one of the pharmacist authors. Any discrepancies in these two independent reviews were discussed among all four authors and a consensus was reached. All BBWs were able to be categorized into one of the four major categories and one of the ten major sub-categories. ►Table 2 shows the final informatics-oriented BBW taxonomy.
Table 2.
Informatics oriented black box warning (BBW) taxonomy.
| Category | Description | % of BBW |
|---|---|---|
| Interactions | Black Box Warnings recommending interaction checking | 13% |
| Drug-Drug | Avoid or carefully monitor in patients taking certain other drugs | 4% |
| Drug-Diagnosis | Avoid or carefully monitor in patients with certain diagnoses | 9% |
| Testing | Black Box Warnings recommending testing | 21% |
| Baseline | Tests that need to be conducted before the drug can be initiated or restarted | 9% |
| Ongoing | Tests that need to be conducted on a on-going basis while the patient is taking the drug | 12% |
| Notifications | Black Box Warnings recommending notification | 29% |
| Drug Prescriber | Information/instructions needed for the drug prescriber | 7% |
| Drug Dispenser | Information/instructions needed for the drug dispenser | 2% |
| Drug Administrator | Information/instructions needed for the drug administrator | 9% |
| Patient | Information/instructions needed for the patient (including informed consent) | 10% |
| Third Party | Information/instructions needed for a third party | 1% |
| Non-Actionable | Black Box Warnings that do not appear to be actionable within health information systems | 37% |
4. Results
The informatics oriented BBW taxonomy includes four broad BBW categories – interaction, testing, notification, and non-actionable BBWs. Each primary category is described below, including specific examples.
4.1 Details of BBW Taxonomy
4.1.1 Interaction BBWs
Interaction BBWs (13% of BBWs) are interactions between a drug with a BBW and another drug (drug-drug), or between a drug with a BBW and diagnosis (drug-diagnosis), at the time of initiation and throughout the course of taking the BBW drug. BBWs in this categories should typically be shown to the drug prescriber at the time of prescribing and/or renewing/reordering and/or when other medications or diagnoses or tests change and possibly, as a back-up, to others in the medication delivery chain.
4.1.1.1 Drug-Drug Interaction BBWs
Drug-drug interaction BBWs (5% of BBWs), are the most common type of BBW in HIS today. A robust system for checking drug-drug BBWs should not only identify the potential problem at the time of ordering, but also provide alternatives such as medication changes and/or increased monitoring recommendations. Examples of drug-drug interaction BBWs include ritonavir, a CYP450 inhibitor, increasing concentrations of other drugs such as amiodarone, flecainide, and quinidine, as well as drugs that may interact with warfarin.
4.1.1.2 Drug-Diagnosis Interaction BBWs
Drug-diagnosis BBWs (9% of BBWs) in HIS exist, although are less common then drug-drug interaction BBW checking systems. Drug-diagnosis BBWs in HIS would cross-check each drug for any contraindicated diagnoses, such as pregnancy. Diagnosis information could either be based on ICD-9 codes in the HIS (for example in problems list, diagnosis list, or past medical history) or objective data, such as a positive pregnancy test within the last 9 months for the diagnosis of pregnancy or abnormal hemoglobin A1C for the diagnosis of diabetes or cardiac echocardiogram for the diagnosis of congestive heart failure. Example of drug-diagnosis BBWs include angiotensin converting enzymes inhibitors (ACE inhibitors), angiotensin receptor blockers (ARBs), retinoids, and statins which are all contraindicated in pregnancy, as well as pioglitazone, a thiazolidinedione used to treat diabetes, which is contraindicated in New York Heart Association (NYHA) class III and IV heart failure.
4.1.2 Testing BBWs
Testing BBWs (21% of BBWs) has two sub-categories – baseline (prior to initiation or re-initiation) and ongoing. As the names imply, baseline tests should occur before a BBW medication is begun and/or restarted and ongoing indicates tests while the patient is taking the medication. As with interaction BBWs, BBWs in this categories should typically be shown to the drug prescriber at the time of prescribing and/or renewing/reordering and possibly, as a back-up, to others in the medication delivery chain.
4.1.2.1 Baseline Testing BBWs
Baseline testing BBWs (12% of BBWs) recommend baseline testing when a drug is started. A computerized physician order entry (CPOE) system could automatically detect if baseline laboratory testing existed in assessable HISs. CPOE systems could help providers decide if previous tests indicate the medication should or should not be given or if dose adjustments need to be made, for instance in someone with renal insufficiency. If appropriate baseline testing did not exist, then the appropriate orders could be generated. One challenge that exits with current BBW recommendations is that many lack the specifics for a baseline testing timeframe. For example, does the baseline test need to be conducted on the day the medication is started, within 1 week before or after initiation, 1 month before or after initiation, etc. To be fully actionable within HISs, the details of timeframes need to be specified in the BBW. Many medications have BBWs that fall into this category including cholesterol lowering statin medications which should have liver function testing before initiation and carbamazepine which requires baseline monitoring of the complete blood count to look for immunosupression.
4.1.2.2 Ongoing Testing BBWs
Ongoing testing BBWs (10% of BBWs) refer to drugs that carry a risk of organ system toxicity or require dosage adjustment in the presence of organ dysfunction. A CPOE system, could check to see if recommended ongoing testing had occurred, if any changes in the medication should be made based on the ongoing test results, and generate the appropriate order and/or future orders and/or standing protocol orders to facilitate ongoing testing. Alerts could appear with the appropriate information and/or order when refilling or renewing patient medications or anytime an ongoing lab tests are due. Even if the patient is not present, tools within HIS could alert drug prescribers, drug dispensers, and drug administrators of the need for ongoing testing or that ongoing test results are missing. Information and reminders about ongoing testing could also be presented within personal health records, automated emails and/or telephone/text messages.
4.1.3 Notification BBWs
Notification BBWs (29% of BBWs), are BBWs where additional information needs to be provided to someone involved with the medication. This additional information would not include information regarding interactions or testing, which are covered in the previous to categories. Depending on the BBW, notifications could be directed towards one of five parties: drug prescribers, drug dispensers, drug administrators (generally nurses in the inpatient setting), patients (drug administrators in the outpatient setting), and/or third parties. Informatics tools can be utilized at the time of drug prescribing, drug dispensing, or drug administration.
4.1.3.1 Drug Prescriber Notification BBWs
In this category of BBWs (7% of BBWs), the prescriber would be notified of a BBW regarding specific information or instructions needed in conjunction with a drug. One example is doxorubicin, an antineoplastic agent, where there is a maximum dose range BBW. Prescribers would be notified when cumulative doses are greater than 300–500 mg/m2 due to the risk for cardiac impairment2. Docetaxel, an antineoplastic drug, is another example with a BBW to the prescriber to premedicate the patient with dexamethasone three days prior to starting therapy.
4.1.3.2 Drug Dispenser Notification BBWs
This category of BBWs (2% of BBWs) includes warnings to drug dispensers. The pharmacy information system could automatically print out labels with special notification that the medication is “not for injection” and “for oral use only”, based on the route of administration ordered during CPOE. An example of a drug with a BBW in this category is acetylcysteine for inhalation. This drug comes in a vial that resembles a vial of solution for injection, but it is not suitable for injection. The BBW states that the drug is not for injection.
4.1.3.3 Drug Administrator Notification BBWs
The drug administrator notification BBWs (9% of BBWs) includes warnings to personnel administering the medication, typically a nurse in the inpatient setting. Many BBWs included in this category involve drugs that may have infusion-related reactions or drugs that require certain equipment or a particular setting during administration. To be effective, these notifications need to be seen just before or in conjunction with medication administration, ideally in a closed loop medication administration with bar code medication administration. Alemtuzumab, an antineoplastic agent, can cause acute reactions while the drug is being infused, which generally happen during the time the patient is receiving the medication. Other BBWs drugs in this category include antineoplastics such as carboplatin and cisplatin.
4.1.3.4 Patient Notification BBWs
Patient notification BBWs (10% of BBWs) indicate BBW information or actions involving the patient. In this category, informatics tools would provide the patient with information or automatically print necessary forms, such as informed consent forms, that a patient may need to complete to satisfy the requirements of the BBWs. One example of when this notification would appear is with the use of irinotecan, an antineoplastic agent, which causes severe diarrhea. Patients could be presented with specific anticipatory guidance regarding signs and symptoms to look for and actions to take to minimize these effects or address them if they occur. A loperamide prescription could also automatically be generated for the patient when irinotecan is used. Another example of a medication that is used more commonly in this category is warfarin. Point of care information reinforcing patient counseling to minimize the risk of bleeding, reporting any signs/symptoms of bleeding to their physician, and awareness of precautions/risk factors such as drug-drug interactions. The warfarin FDA Patient Medication Guide could be automatically printed for a patient and/or incorporated into the patient’s personal health record.
4.1.3.5 Third Party Notification BBWs
Third party notification BBWs (1% of BBWs) occur if a medication needs information sent to a third party, such as a prescriber and/or patient to be enrolled in a national registry. These national registries are established to closely monitor specific medications with heightened safety concerns. Advanced HISs could potentially automatically enroll that patient in the appropriate registry at the time of medication ordering. An example of this is clozapine, an antipsychotic drug, which has a significant risk for agranulocytosis. Due to the severity of this risk, the National Clozapine Registry was established to ensure regular monitoring of white blood cell count and absolute neutrophil count prior to the delivery of the next medication supply.
4.1.4 Non-actionable BBWs
Finally, a significant percentage (over 1/3–37%) of all BBWs are not written in a way that can be actionable within HISs and are designated non-actionable. These BBWs, if presented to the appropriate person, do not provide clear guidance on what the person receiving the BBW should do. Although these BBWs could be built into HIS from a technical perspective, the messaging related to these BBWs, based on the information provided in the BBW itself, would not be able to provide the person receiving the message clear guidance on what to do and so would not be effective messages and would contribute to alert fatigue. The prototypical example of this category of BBW is selective serotonin re-uptake inhibitor (SSRI) anti-depressant medications. These medications are almost always given to treat depression, but carry a black box warning that they may increase suicide risk. Although it would be easy for a HIS to determine when a new SSRI was being initiated, it is very unclear what the prescriber receiving this alert should do. The BBW would be actionable if the instructions were clear. For example, before prescribing an SSRI, have the patient sign a suicide contract and schedule weekly follow-up with the patient. Another example is fluoroquinolone antibiotic agents. The BBW for these agents states that these agents may increase the risk of tendon rupture and tendonitis. The BBW would be actionable if clear instructions such as possibly giving the patient tendon stretching exercise instructions or clear instruction for the patient to call immediately with any joint or muscle pain. These BBWs do not provide clear guidance on what how this information should be used when prescribing or considering prescribing these medications.
5. Discussion
Implementing BBWs in HIS presents an opportunity for informatics to improve patient safety and thereby improve health care quality and decrease costs. The informatics tools to capitalize on this opportunity, primarily CDS tools and other alerting tools, which can be role based, already exist in HIS. However, keys to maximizing BBW implementation and effectiveness revolve around how the information of BBWs is represented, organized, and managed, as well as the existence of fully interoperable HIS throughout the medication delivery system, from medication prescribing to medication administration. To our knowledge, this is the first informatics oriented BBW taxonomy.
This taxonomy highlights some of the opportunities and challenges for implementing BBWs through HIS. Almost 2/3 of BBWs could be implemented within HIS because the BBWs can be implemented using tools already existing with HIS, providing clear guidance to the person receiving the BBW on what to do. Approximately 1/3 of non-actionable BBW cannot be implemented easily in HIS today because best practice CDS guidelines state that effective CDS provides clear action that the person receiving the information should follow [13-15]. ►Table 3 depicts the relative ease with which different categories of BBWs can be implemented and the issues involved in their implementation today.
Table 3.
Clinical decision support (CDS) for black box warnings (BBWs).
| Category | Implementation | Issues |
|---|---|---|
| Interactions | Easy | |
| Drug-Drug | Easy |
|
| Drug-Diagnosis | Medium |
|
| Testing | Medium | |
| Baseline | Medium |
|
| Ongoing | Medium |
|
| Notifications | Medium | |
| Drug Prescriber | Medium |
|
| Drug Dispenser | Medium |
|
| Drug Administrator | Medium |
|
| Patient | Medium |
|
| Third Party | Hard |
|
| Non-Actionable | Hard |
|
Currently, BBW implementation has generally only involved BBWs for drug-drug interactions because commercial databases (such as Medi-Span™ and First Data Bank™) exist with this information. One of the primary reasons why actionable BBWs have not been implemented in other areas is lack of structured, maintained, trusted, and generally available databases with drug-diagnoses, drug-testing, or notification information. Some of these databases could be developed because standard terminologies exist not only for drugs, but also diagnoses (International Classification of Disease (ICD) – 9) and tests (Logical Observation Identifiers Names and Codes (LOINC)). Standard terminology for notification information does not exist. Lack of standardized data and terminology systems has been identified as a barrier to CDS in CPOE systems [16].
Approaches should be taken that do not require individual instances of HIS to develop and maintain their own knowledge repository for BBWs. Rather, HIS vendors and/or third parties should develop and integrate standard mechanisms for BBW knowledge representation, organization, and management, as had been done with drug-drug interaction knowledge. Finally, approximately 1/3 of BBWs are currently non-actionable within HIS. FDA standards for BBW development [17] should incorporate best practice CDS guidelines [15, 16]. Even some BBWs that appear actionable, such as baseline testing, need additional specificity to be fully and easily actionable within HIS. The FDA should review current BBWs and develop BBW standards that enable and facilitate their implementation within HIS. Additionally, health information technology standards and certification bodies should help catalyze full implementation of CDS for BBWs in HIS, which this taxonomy should facilitate.
These efforts at BBW data representation, organization, and standardization should occur with overall efforts to integrate HIS generally and CDS specifically into the overall continuum of patient care. In order to be successful, these efforts need to address technical (terminology, data structure, etc.) and non-technical issues (content, regulatory policies, etc.). For example, ideally BBW HIS CDS modules and terminology would have the ability to integrate with the FDA’s Adverse Events Reporting System (AERS), however, to do this would require the use of common standardized terminologies such as MedDRA [17, 18].
Once BBW representation, organization, and standardization issues are addressed and all BBWs can be supported with HIS, their successful implementation will only occur if full data exchange exists between all HIS that have information about a patient and all parts of medication delivery system use HIS. Complete medication, diagnosis, and testing information across time and all points of care will be critical if BBW implementation through HIS is to reach its full potential. CPOE and physician documentations systems need to freely exchange information with pharmacy information systems and laboratory information systems and ideally with personal health records; and in the inpatient setting, with closed loop bar code administration systems. Given that only about 1.5% of hospital systems have complete, fully deployed HIS [19] and 4% of ambulatory physicians have extensive fully functioning HIS [20] significant work needs to be done in HIS implementation and interoperability to gain the full benefit of CDS for BBWs in HIS. However, given the current low compliance with BBWs and the effective of well-designed CDS in other areas of clinical areas [21–22], CDS for BBW in HIS has the potential to significantly improve BBW compliance and therefore patient safety.
6. Conclusion
BBWs present an informatics opportunity to improve patient safety. To maximize this opportunity, BBWs should be organized and categorized in a systematic way such as presented here to facilitate implementation of informatics CDS tools within HIS. Additionally, all BBWs need to be constructed in a way to facilitate CDS implementation for them in HIS.
Clinical Relevance Statement
The study provides a framework through which institutions can determine where clinical decision support (CDS) for black box warnings (BBWs) can be implemented within their health information systems (HIS) currently. It also identifies areas where work needs to be done to make CDS for all BBWs possible.
Conflicts of Interest
The authors declare that they have no conflicts of interest in this research.
Protection of Human and Animal Subjects
Human and/or animal subjects were not included in this project.
Acknowledgements
This study did not involve any external funding.
References
- 1.21 code of Federal Regulations paragraph 201.57(e) (2000). [Google Scholar]
- 2.Generali J.Black Box Warning. www.formularyproductions.com/blackbox Accessed May 2011.
- 3.Wagner AK, Chan KA, Dashevsky I., MA FDA drug prescribing warnings: Is the black box half empty or half full? Pharmacoepidemiol Drug Saf 2006; 15 (6): 369–386 [DOI] [PubMed] [Google Scholar]
- 4.Raeble MA, Lyons EE, Andreade SE, Chan KA, Chester EA, Davis RL, et al. Laboratory monitoring of drugs at initiation of therapy in ambulatory care. J Gen Intern Med 2005; 20: 1120–1126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Lasser KE, Seger DL, Yu DT, Karson AS, Fiskio JM, Seger AC, et al. Adherence to black box warnings for prescription medications in outpatients. Arch Intern Med 2006: 166(3): 338–344 [DOI] [PubMed] [Google Scholar]
- 6.Horlen C, Malone R, Bryant B, Dennis B, Carey T, Pignone M, et al. Frequency of inappropriate metformin prescriptions. JAMA 2002; 287: 2504–2505 [DOI] [PubMed] [Google Scholar]
- 7.Smalley W, Shatin D, Wysowski DK, Gurwitz J, Andreade SE, Goodman M, et al. Contraindicated use of cisapride: impact of FDA Regulatory Action. JAMA 2000; 284: 3036–3039 [DOI] [PubMed] [Google Scholar]
- 8.Guo JJ, Curkendall S, Jones JK, Fife D, Goehring E, She D.Impact of cisapride label changes on codispensing of contraindicated medications. Pharmacoepidemiol Drug Safe 2003; 12: 295–301 [DOI] [PubMed] [Google Scholar]
- 9.Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14(1): 29–40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kuperman GJ, Teich JM, Gandhi TK, Bates DW. Patient safety and computerized medication ordering at Brigham and Women’s Hospital. Jt Comm J Qual Improv 2001; 27(10): 509–521 [DOI] [PubMed] [Google Scholar]
- 11.Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10(6): 523–530 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Thompson CA.Hospital inspectors eye black-box warnings. AM J Health Syst Pharm 2008; 65: 890–894 [DOI] [PubMed] [Google Scholar]
- 13.Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14(1): 29–40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wright A, Phansalkar S, Bloomrosen M, Jenders RA, Bobb AM, Halamka JD, et al. Best practices in clinical decision support: the case of preventive care reminders. Appl Clin Inform 2010; 1(3): 331–345 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Osheroff JA, Teich JM, Levick D, Saldana L, Velasco FT, Sittig DF, et al. Improving outcomes with clinical decision support: An implementer’s guide. Second edition. HIMSS. 2012. [Google Scholar]
- 16.Ahmadian L, van Engen-Verheul M, Bakhshi-Raiez F, Peek N, Cornet R, de Keizer NF. The role of standardized data and terminology systems in the computerized clinical decision support systems: literature review and survey. Int J Med Inform 2011; 80(2): 81–93 [DOI] [PubMed] [Google Scholar]
- 17.Food and Drug Administration Guidance for industry – warnings and precautions, contraindications, and boxed warning sections of labeling for human prescription drug and biological products – content and format. Food and Drug Administration. October2011. [Google Scholar]
- 18.MedDRA Maintenance and Support Services Organization Introductory guide to MedDRA Version 14.0. Chantilly, Virginia: March, 2011. [Google Scholar]
- 19.Harpaz R, Chase HS, Friedman C.Mining multi-item drug adverse effect associations in spontaneous reporting systems. BMC Bioinformatics 2010; 11 (Suppl 9): S7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jha AK, DesRoches CM, Campbell EG, Donelan K, Rao SR, Ferris TG, et al. Use of electronic health records in U.S. hospitals. N Engl J Med 2009; 360(16): 1628–1638 [DOI] [PubMed] [Google Scholar]
- 21.Jha AK, Bates DW, Jenter C, Orav EJ, Zheng J, Cleary P, et al. Electronic health records: use, barriers and satisfaction among physicians who care for black and Hispanic patients. J Eval Clin Pract 2009; 15(1): 158–163 [DOI] [PubMed] [Google Scholar]
- 22.Kawamoto K, Houklihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330(7494): 765 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Damiana G, Pinnarelli L, Colosimo SC, Almiento R, Sicuro L, Galasso R, et al. The effectiveness of computerized clinical guidelines in the process of care: a systematic review. BMC Health Serv Res 2010; 10 :2 [DOI] [PMC free article] [PubMed] [Google Scholar]