Abstract
Objectives: This study compared the potential for personal digital assistant (PDA)–based drug information sources to minimize potential medication errors dependent on accurate and complete drug information at the point of care.
Methods: A quality and safety framework for drug information resources was developed to evaluate 11 PDA-based drug information sources. Three drug information sources met the criteria of the framework: Eprocrates Rx Pro, Lexi-Drugs, and mobileMICROMEDEX. Medication error types related to drug information at the point of care were then determined. Forty-seven questions were developed to test the potential of the sources to prevent these error types. Pharmacists and physician experts from Creighton University created these questions based on the most common types of questions asked by primary care providers. Three physicians evaluated the drug information sources, rating the source for each question: 1 = no information available, 2 = some information available, or 3 = adequate amount of information available.
Results: The mean ratings for the drug information sources were: 2.0 (Eprocrates Rx Pro), 2.5 (Lexi-Drugs), and 2.03 (mobileMICROMEDEX). Lexi-Drugs was significantly better (mobileMICROMEDEX t test; P = 0.05; Eprocrates Rx Pro t test; P = 0.01).
Conclusion: Lexi-Drugs was found to be the most specific and complete PDA resource available to optimize medication safety by reducing potential errors associated with drug information. No resource was sufficient to address the patient safety information needs for all cases.
INTRODUCTION
Drug information sources readily available at the point of care is one practice-improvement intervention that may reduce medication errors [1]. Access to drug information sources in health systems and hospitals is considered a minimum standard. Typically, these resources are found in a version in print and for main frame or Internet access throughout patient care areas and professional areas such as the pharmacy and patient care units. Network access to drug information sources is also common in chain community pharmacies. Physician clinic practices have been less rapid to incorporate computerized drug information source access, however, and generally maintain a local centralized print library for drug information in the clinic itself. Despite the seemingly close proximity of many drug information sources, these sources are often not accessible under the efficiency constraints and clinical pressures most clinicians face. The process of readily accessing drug information in daily clinical practice needs systematic improvement [2]. The use of personal digital assistants (PDAs) has emerged as a technology that promises to improve the process of accessing drug information at the point of care for clinicians. Drug information resources have rapidly become available for use on the PDA.
Access to drug information at the point of care via PDA may have great potential to reduce medication errors associated with prescribing. Prescribing is the step in the medication-use process associated with the greatest proportion of documented errors. Further, it has been pointed out that the prescriber is the first individual in the medication-use process who can take steps to prevent error [3]. Most preventable adverse drug events occur in the prescribing stage of the medication-use process and have been attributed to inappropriate prescribing decisions and inappropriate monitoring [4]. What is the potential for PDA-based drug information sources to improve medication safety? The potential depends on the sufficiency of information available in software for these small, handheld computers.
This study compared the potential for PDA-based drug information sources to assure medication safety associated with specific, accurate, and complete drug information at the point of care. The objective of this study was to determine the optimal drug information resource available on a PDA to use in daily clinical practice to meet this purpose.
METHODS
The project was conducted during the fall of 2002. A framework to describe the safety of drug information resources was developed and included traditional standards for evaluating the quality of drug information sources. The framework was used to evaluate eleven PDA-based drug information sources available at the time (Appendix A). The first step was to determine the indicators of a quality drug information source. Three of the eleven resources met all of the quality indicators and were selected for further study.
Next, safety indicators of a drug information source were determined. These two sets of indicators were used to develop a comprehensive set of questions to test the capacity of the drug information sources to answer drug information questions involving medication safety. Three general practice physicians agreed to serve as raters of the drug information sources. They were provided the questions and PDAs with the three drug information sources. The most sufficient drug information source to answer the safety-based questions was determined from these ratings. The specifics of this process are described here.
Quality and drug information sources
Quality indicator criteria were established based on the concept that the drug information resource must be sufficient to meet the drug information need when a question is posed. For a resource to be sufficient, it must be accurate, be complete, and meet the breadth, depth, and scope of information needed to answer a question. A comprehensive listing of general quality indicators of a drug information source was compiled by the primary investigator based on prior literature and evaluation of established published source books with general drug information [5–9]. A specific listing of the types of drug information that would be required to meet breadth, depth, and scope of a comprehensive drug information source was also compiled using the same technique.
These two listings were reviewed using a delphi group technique composed of three general practice physicians and three pharmacists. The final listing of quality indicators was transformed into a self-administered written survey to confirm those quality indicators that physicians believed were important in drug information sources, with the types of information specifically identified. Physicians were chosen to evaluate this standard, because physicians generate the vast majority of prescriptions. A convenience sample of ten physicians in general primary care medical practice (five family practice, five internal medicine) completed the survey. The physicians in the convenience sample did not participate in the delphi group.
Safety and drug information sources
Drug information resource indicators for medication safety were determined. The classification system for medication errors from the National Coordinating Council Medical Error Reporting Program (NCCMERP) was assessed to determine the most common types of medication error likely to occur because of the lack of sufficient drug information at the point of care [10]. Safety indicators were derived based on this review. Error types were matched against the quality criteria to determine if each criterion predominantly related to a medication safety concern. In addition, each of the quality indicators was analyzed to determine their value to safety. Upon completion of this review, it was found that several indicators of quality represented the resources' likelihood of improving patient safety when prescribing took place (Appendix A).
Evaluation of personal digital assistant (PDA) drug information sources for quality and safety
Forty-seven drug information questions were developed to test the ability of the sources to meet the quality and safety criteria. The questions were based on three criteria: (1) information from the literature on the most common types of questions asked by primary care providers related to drug therapy, (2) content of questions representing commonly encountered clinical situations, and (3) representativeness of the questions of quality and safety indicators [5, 11–13]. Pharmacists who have extensive practice experience answering questions posed by physicians initially developed the questions using these three criteria. A delphi group of general practice physicians and pharmacists reviewed and finalized the questions (Appendix B).
Analysis
The survey responses from the ten physicians to evaluate the quality indicators (Table 1) and the types of content that drug information sources should contain (Table 2) were summed and placed in rank order of agreement. The rank order of agreement was determined to assess what physicians thought were the most important areas of quality and content for a drug information source. This order was likely to represent the perceived frequency with which physicians believed they answered certain types of questions. All indicators were kept for this evaluation, because all indicators were necessary to assure that patient safety was optimized with the information source. Each of the eleven resources was evaluated using the quality indicators. Three resources—Eprocrates Rx Pro, Lexi-Drug, and mobileMICROMEDEX—met all of the identified quality indicators and were selected for comparative evaluation.
Table 1 Physician-determined quality indicators for drug information sources
Table 2 Physician-determined type of information needed in a drug information source
The questions were then used to test each of the three drug information sources. Three general practice physicians served as raters. Each was instructed to independently answer all forty-seven questions using each reference. The physician raters recorded answers to the questions and were then asked to assign a rating to each answer based on the extent to which the resource met the drug information need. The physicians were also asked to document discrepancies between the resources that might challenge the accuracy of the information. The ratings used were: 1 = no information available in the resource relevant to the information need; 2 = some information available, inadequate to meet the information need; or 3 = right amount of information available to meet the information need. The ratings were totaled, and an average score was calculated per physician rater for the resources. If a reference received an average score of 3.0, the reference would provide the right amount of information available for all of the questions used to test it. A perfect score of 3.0 would be the optimal reference.
Intra- and inter-rater reliability
An intra-rater comparison of the three references was performed and the significance set at P = 0.05 analysis of variance (ANOVA). The purpose of this comparison was to determine if each rater determined a difference between the three drug information sources. An inter-rater comparison for each reference was also performed, and the significance set at P = 0.05 (ANOVA). This comparison was performed to determine if the raters differed on a particular drug information source. The non-parametric Kruskal-Wallis test was also conducted to compare to the ANOVA because of the small sample size.
RESULTS
The mean ratings for sufficiency to meet the information need related to patient safety were: 2.00 (Eprocrates Rx Pro), 2.50 (Lexi-Drugs), and 2.03 (mobileMICROMEDEX), respectively; with Lexi-Drugs significantly better when compared to mobileMICROMEDEX (t test; P = 0.05) and Eprocrates Rx Pro (t test; P = 0.01). No resource was sufficient for all 47 questions. The number of episodes with no or insufficient information were 19 (Eprocrates Rx Pro), 18 (mobileMICROMEDEX), and 5 (Lexi-Drugs), respectively. None of the references were found to be inaccurate.
The intra-rater comparison revealed that Lexi-Drugs was rated significantly higher by two of the three raters when compared among the three references (ANOVA P < 0.05; Kruskal-Wallis P = 0.063). The third rater rated Lexi-Drugs higher, with the difference detected at P = 0.07. There was no inter-rater difference in the ratings of the three physicians when each drug information reference was compared (Eprocrates Rx Pro P = 0.99, Lexi-Drugs P = 0.32, mobileMICROMEDEX P = 0.49; Kruskal-Wallis P = 0.49) (Tables 3 and 4).
Table 3 Intra-rater comparison of drug information sources
Table 4 Inter-rater comparison of drug information sources
The physicians differed in their anecdotal impressions about the use of the information sources. Two of the three indicated they liked the completeness of the Lexi-Drugs resource; one stated, “it complements my knowledgebase,” and another indicated it was so full of information that another resource would be sought because of a desire to have something quick. All commented that they found Lexi-Drugs' side scroll bar difficult to use. The evaluation of the three resources indicated that Lexi-Drugs was the most specific and complete drug information resource available via PDA to optimize medication safety in patient care at the time of the evaluation.
The small sample size is a consideration in this study. To be a comprehensive evaluation, a much larger sample of questions should be addressed. However, the goal was to make a sound, pragmatic recommendation to practice-based clinicians about the optimal PDA-based drug information service for use at the point of care when prescribing takes place. The evaluation achieves this goal.
DISCUSSION
Traditionally, drug information sources have been viewed from the context of optimizing drug therapy use. Enhanced access to drug information via the PDA has the potential to further this objective. However, optimizing drug use is not the same as promoting medication safety. Public awareness of the safety of medication use and the high priority society places on this subject suggest information sources should also be viewed as a form of intervention to improve patient safety.
The goal of safety is to eliminate risk, thus, eliminating harm. Toward this end, the Institute of Medicine called for a reduction in medical errors in the United States by 50% over a 5-year period [14]. This call to action is a direct result of a substantial body of literature about medical errors in the hospital setting, with the majority of this knowledge related to medication errors. What is an error? A reasonable definition has been offered that states, “An error is any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional or patient” [14]. Conceptually, error should be considered as an event that occurred that should not have occurred. If the event led to inappropriate medication use, it is an error. This event may or may not have resulted in patient harm.
Examination of the concept of medication error from this point of view makes clear that drug information sources must have specific information that is sufficient to meet the basic information needs of each clinical decision made in the process of medication use. “Sufficient” means that the information needs can only be met if the information is accurate, specific, and complete enough to meet clinical practice needs. Therefore, listing dosage forms for an oral antibiotic, such as ampicillin, as “capsule, suspension, injection” would not be considered sufficient. Although accurate, the description is not specific or complete enough for a clinician to actually prescribe or recommend the regimen using the suspension product. What is the concentration of ampicillin in the suspension? How does one accurately recommend a dose, based on milligrams of ampicillin and volume to administer per dose? In this example, the drug information reference is insufficient to minimize dosing errors. The information should include strength and/or concentration to be sufficient.
A single drug information product cannot stand alone to fulfill all critical patient care information needs. The need for a complete library that is readily accessible to clinicians continues. The comprehensiveness of Lexi-Drugs was found to be a trade off for quickness of information retrieval. However, all physician raters commented that they found the resource easy enough to use in exchange for the depth of information. This comment is important in the context of medication safety at the point of prescribing, when the information must adequately provide the basic detail necessary to correctly write a complete prescription. It is noteworthy that another independent study found Lexi-Drugs an excellent source for assessing drug interactions compared to other PDA software applications [15].
One valuable safety addition to PDA drug information sources would be product identification, such as information about capsule or tablet color, markings, and sizes. This information was not available through any of the three resources evaluated but was commented on by the physician raters. Pictures of each dosage form would be ideal but not practical because of the large amount of memory that the typical PDA would require. Another suggestion was to provide a bibliography to support clinical information. Having access to the literature citations would strengthen the drug information conclusion.
Adverse drug events that are simply listed in the drug information source do not provide enough information to improve decisions related to a medication's safety. They should be listed by severity and/or frequency. Listing adverse events by system would enable the user to quickly find important information, and information about the likelihood of adverse events was commented on as useful. Lexi-Drugs lists adverse drug events by frequency.
The cost of drugs is not related to medication safety directly but is related to medication adherence. Lexi-Drugs and Eprocrates Rx Pro both list the average wholesale price (AWP) of each drug. The AWP is not the price charged to patients, distributors, or other intermediaries and does not consider the patient's prescription drug benefit or actual cost of the medication. However, practitioners find this feature useful when prescribing medication.
Computerized practitioner order entry, integrated electronic medical records, and clinical decision support systems must be designed with patient safety as a core framework. The implications of this work are applicable to these developing systems. Patient safety improvements that depend on information systems will only be as effective as the potential they are designed to meet.
CONCLUSION
Lexi-Drugs was the most specific and complete resource available via PDA to improve medication safety by reducing potential errors associated with insufficient or incomplete drug information. However, no resource sufficiently addressed the patient safety information needs for all cases. As with print libraries, utilizing more than one resource is often necessary to provide the most accurate and complete information. Specific suggestions for improving any drug information source to improve medication safety are to include information on the stability of drug products and descriptions of the drug dosage form for identifying the product. Clinicians should evaluate future drug information sources by their ability to meet medication safety needs.
APPENDIX A
APPENDIX B
Drug information questions used to compare drug information sources
What is the recommended dosage regimen for use of reserpine in treatment of hypertension in an adult male who is fifty-eight years old?
What is the initial recommended starting dose for tolbutamide in an adult who has adult-onset diabetes mellitus?
What are the active ingredients in Cosopt?
Is it possible that diltiazem causes gingival hyperplasia?
What are the most common side effects of isosorbide dinitrate?
What are the top drug interactions I should worry about for a man who is on Aldomet?
Are there any cautions or concerns about the use of isoniazid in a woman who is thirty-eight years old?
What should be done about a patient's warfarin therapy; the patient is scheduled to have a wisdom tooth extraction two weeks from now?
Is there a commercially available garlic concentrate capsule? I have a patient who has heard of this and wants to start taking one.
Is there a drug interaction between warfarin and garlic?
My patient wants to take vitamin E concentrate. Is there a commercially available product?
My patient is on warfarin. Is there a drug interaction between warfarin and vitamin E?
I would like to restart Prozac and delivered a baby two weeks ago. Will this drug go into my breast milk?
I have been on Dilantin and just learned that I am pregnant. Will I be able to continue taking Dilantin throughout the entire pregnancy? Is there any evidence about teratogenicity?
Should I adjust the dose of this patient who is on Dilantin? Based upon her serum creatinine, I estimate her creatinine clearance to be about twenty milliliters per minute (mls/min).
Is hydrochlorothiazide a good choice in the management of edema in my congestive heart failure patient? She has a creatinine clearance estimated at twenty to twenty-five mls/min (she is eighty-three years old).
My patients renal function is deteriorating, and I have her on tobramycin. Are there any concerns about adverse effects or precautions?
My patient's liver function tests are rising quickly; I have her on itraconazole. Are there any concerns about adverse effects or precautions?
My patient is taking warfarin 7.5 milligrams per day orally and has come to clinic with an international normalized ratio (INR) of six. What should I do?
A little boy who is eight years old just ate a bottle of FeSo4 tablets his mother had in the kitchen. She thinks he ate forty or fifty of them. What should I do?
This adult will not swallow capsules, and it is essential that he be treated with a macrolide antibiotic. Is there an injectable, long acting macrolide that can be given in the office?
What is the cost of Ditropan XL?
What are the key counseling points that are important about the use of topical nitroglycerin paste?
What is the mechanism of action of Paxil?
What is the stability of ampicillin oral suspension once it is reconstituted?
A patient just asked me if I could compound a ketorolac (or EMLA or Brompton's cocktail) topical cream. Is there a formulation for this?
My patient is taking OxyContin thirty milligrams by mouth (per os) every (mg po q) twelve hours for severe pain. How should I prescribe it?
What is the dosage of Augmentin suspension in a four-year-old female patient (fifty pounds) with acute otitis media?
What is the dose of Tranxene for an eighty-five-year-old woman with anxiety?
What is the maximum daily dose of Darvocet N 100?
How do you prescribe Percocet for the pain associated with an acute ankle injury (twisted ankle) from playing tennis? The ankle was twisted yesterday and is now swollen and tender. The patient is forty-five years old.
What is the equivalent dose of Lipitor to Zocor?
How long should I wait after a myocardial infarction to start warfarin therapy in a fifty-five-year-old man?
Does Zithromax work for community acquired pneumonia?
How effective is Zoloft in the management of premenstrual dysphoric disorder?
How long does the sexual dysfunction (decreased libido) associated with Zoloft last after stopping it?
Above what dose of potassium chloride as a daily oral supplement does hyperkalemia develop?
Is Verapamil contraindicated in pregnancy?
Are there any interactions between amoxicillin and food?
What is the brand name for sertraline?
How much hydrochlorothiazide is in Dyazide?
What is the active ingredient in Lanoxicaps?
What are the available dosage forms and strengths of digoxin?
How do the absorption and duration characteristics of Calan and Calan SR compare?
What is the mechanism of action of Zithromax?
What is the cost of Humulin N?
How often should a phenytoin serum level be drawn in a patient with grand mal seizures who is stable and well controlled on phenytoin?
Footnotes
* Support for this project was provided in part by the Agency for Healthcare Research and Quality, grant number 1-R18HS11808-01.
Contributor Information
Kimberly A. Galt, Email: kgalt@creighton.edu.
Ann M. Rule, Email: arule@creighton.edu.
Bruce Houghton, Email: Houghton@creighton.edu.
Daniel O. Young, Email: dyoung@creighton.edu.
Gina Remington, Email: gremingt@creighton.edu.
REFERENCES
- Institute of Medicine. Crossing the quality chasm. Washington, DC: Committee on Quality of Healthcare in America, Institute of Medicine, National Academies Press, 2000. [Google Scholar]
- Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Vinson DC, Stevermer JJ, and Pifer EA. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. BMJ. 2002 Mar 23; 324(7339):710. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen MR. Medication errors. Washington, DC: American Pharmaceutical Association, 1999. [Google Scholar]
- Kanjanarat P, Winterstein AG, Johns TE, Hatton RC, Gonzalez-Rothi R, and Segal R. Nature of preventable adverse drug events in hospitals: a literature review. Am J Health Syst Pharm. 2003 Sep 1; 60(17):1750–9. [DOI] [PubMed] [Google Scholar]
- Enders SJ, Enders JM, and Holstad SG. Drug information software for Palm operating system personal digital assistants: breadth, clinical dependability, ease of use. Pharmacotherapy. 2002 Aug; 22(8):1036–40. [DOI] [PubMed] [Google Scholar]
- Sackett DL, Straus SE, Richardson WS, Rosenberg W, and Haynes RB. Evidence-based medicine: how to practice and teach EBM. 2nd ed. London, UK: Churchill Livingstone, 2001. [Google Scholar]
- Watanabe AS, Conner CS. Principles of drug information services: a syllabus of systematic concepts. Hamilton, IL: Drug Intelligence Publications, 1978. [Google Scholar]
- Slaughter RL, Edwards DJ. Evaluating drug literature: a statistical approach. New York, NY: McGraw-Hill Medical Publishing Division, 2001. [Google Scholar]
- Galt KA. Analyzing and recording a drug information request. Bethesda, MD: American Society of Health-System Pharmacists, 1994. [Google Scholar]
- National Coordinating Council For Medication Error Reporting and Prevention. Taxonomy of medication errors. [Web document]. Rockville, MD: Secretariat, US Pharmacopoeia, 1998. [cited 9 Dec 2004]. <http://www.nccmerp.org/pdf/taxo2001–07–31.pdf>. [Google Scholar]
- Gaither CA, Bagozzi RP, Ascione FJ, and Kirking DM. The determinants of physician attitudes and subjective norms toward drug information sources: modification and test of the theory of reasoned action. Pharm Res. 1997 Oct; 14(10):1298–308. [DOI] [PubMed] [Google Scholar]
- Ely JW, Osheroff , Ebell MH, Bergus GR, Levy BT, Chambliss ML, and Evans ER. Analysis of questions asked by family doctors regarding patient care. BMJ. 1999 Aug 7; 319(7206):358–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Pifer EA, and Stavri PZ. A taxonomy of generic clinical questions: classification study. BMJ. 2000 Aug 12; 321(7258):429–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kohn LT, Corrigan JM, and Donaldson MS. To err is human: building a safer health system. Washington, DC: Committee on Quality of Healthcare in America, Institute of Medicine, National Academies Press, 2000. [PubMed] [Google Scholar]
- Barrons R. Evaluation of personal digital assistant software for drug interactions. Am J Health Syst Pharm. 2004 Feb 15; 61(4):380–5. [DOI] [PubMed] [Google Scholar]