A Conceptual Framework for Evaluating Outpatient Electronic Prescribing Systems Based on Their Functional Capabilities

Abstract

Objective: Electronic prescribing (e-prescribing) may substantially improve health care quality and efficiency, but the available systems are complex and their heterogeneity makes comparing and evaluating them a challenge. The authors aimed to develop a conceptual framework for anticipating the effects of alternative designs for outpatient e-prescribing systems.

Design: Based on a literature review and on telephone interviews with e-prescribing vendors, the authors identified distinct e-prescribing functional capabilities and developed a conceptual framework for evaluating e-prescribing systems' potential effects based on their capabilities. Analyses of two commercial e-prescribing systems are presented as examples of applying the conceptual framework.

Measurements: Major e-prescribing functional capabilities identified and the availability of evidence to support their specific effects.

Results: The proposed framework for evaluating e-prescribing systems is organized using a process model of medication management. Fourteen e-prescribing functional capabilities are identified within the model. Evidence is identified to support eight specific effects for six of the functional capabilities. The evidence also shows that a functional capability with generally positive effects can be implemented in a way that creates unintended hazards. Applying the framework involves identifying an e-prescribing system's functional capabilities within the process model and then assessing the effects that could be expected from each capability in the proposed clinical environment.

Conclusion: The proposed conceptual framework supports the integration of available evidence in considering the full range of effects from e-prescribing design alternatives. More research is needed into the effects of specific e-prescribing functional alternatives. Until more is known, e-prescribing initiatives should include provisions to monitor for unintended hazards.

Prescription medications are central to health care, accounting for 13% of health care expenditures and being used by 65% of the U.S. public in a given year.¹ Errors in the use of prescription medications are common, and they often result in patient injuries.^2,3,4 Many of these injuries would be preventable with better information management.⁵ Electronic prescribing (e-prescribing), which we define as clinicians' computerized ordering of specific medication regimens for individual patients, offers the potential to substantially reduce medication errors and also to improve health care efficiency. However, some e-prescribing efforts have met unexpected challenges,^6,7 and faced with the uncertainties, few provider organizations have adopted e-prescribing.^8,9 Nonetheless, inpatient e-prescribing is now being encouraged by a coalition of large employers,¹⁰ and provisions for outpatient e-prescribing are likely to be part of a new Medicare prescription drug benefit.^11,12 Provider organizations that choose to adopt e-prescribing can select from a wide variety of commercial e-prescribing options, but they have no coherent framework to guide their choices.

A few studies have found benefits from e-prescribing, but so far these evaluations have involved only “home-grown” systems implemented by “pioneers” at academic medical centers.¹³ New studies will likely produce additional evidence for effects of commercial systems, but even large investments in controlled trials would be unlikely to produce direct evidence for every type of prescribing system in every type of health care organization.

We sought to develop a conceptual framework for comparing the potential benefits and risks of e-prescribing systems based on their component functional capabilities. This effort is intended as a step toward a design theory¹⁴ for e-prescribing that would consist of empirically grounded principles for predicting the effects of design alternatives within specific clinical environments. This approach is consistent with guidance on the evaluation of complex health interventions, which calls for using a conceptual framework to model how the components of an intervention contribute to its overall effects.^15,16 Though the primary focus of our conceptual framework is on evaluating outpatient e-prescribing systems, many of the same principles should also apply to inpatient systems. In identifying specific functional capabilities, we focused in particular on features that would influence the prescribing step of medication management.¹⁷

Background

Failures of Current Medication Management Processes

Recognizing that medications can cause harm as well as benefits, federal law requires many medications to be dispensed only with a prescription from a licensed practitioner.^18,19 However, even with this basic safety mechanism in place, patients remain at significant risk of injury from the erroneous use of medications. Injuries from medications in general are termed “adverse drug events” (ADEs), and the subset that are due to errors are termed “preventable ADEs.” The health benefits of an e-prescribing system would be a function of its ability to reduce preventable ADEs and increase the appropriate use of medications in comparison with the existing medication management processes that the system would replace.

Net Risk of Preventable ADEs

▶ shows preventable ADE rates found in studies of traditional inpatient and outpatient settings. The table also highlights methodological differences that might contribute to the variation observed among these estimates. An early study, which followed both inpatient and outpatient general internal medicine patients, found that 4.3% of patients experienced ADEs, 83% of which resulted from outpatient prescriptions. The majority of ADEs were preventable and at least serious in severity (▶, first row).²⁰ Other studies, which have focused on inpatients or outpatients only, have found preventable ADE risks ranging from 0.4% of hospital admissions²¹ to 3.0% of outpatients within three months of receiving a prescription.²² As shown in the table, studies that found higher ADE rates also tended to include less-serious events. Other studies that reviewed emergency department charts found potential drug interactions or allergies in 4% to 21% of visits.^23,24,25 These studies are not included in ▶ because they did not assess how often errors led to ADEs, but they suggest that the preventable ADE risk is also high in emergency departments.

Table 1.

Net Risks of Preventable Adverse Drug Events (ADEs) among Studies Having a Defined Base Population

Setting	Study	Detection Method	Study Population	Total ADE Rate	Preventable ADE Rate	Proportion Serious, Life-threatening, or Fatal
Inpatient + outpatient	Burnum, 197620	Physician self-report	1,000 inpatients and outpatients*	4.3 per 100 patients	2.3 per 100 patients	83% of preventable ADEs
Inpatient	Leape, 199121	Chart review	30,195 admissions at 51 hospitals	0.7 per 100 admissions	0.4 per 100 admissions†	100% of total ADEs
	Bates, 19954	Chart review plus nurse and pharmacist reporting	4,031 adult hospital admissions	6.5 per 100 admissions	1.8 per 100 admissions	43% of total ADEs
	Kaushal, 200176		1,120 pediatric hospital admissions	2.3 per 100 admissions	0.5 per 100 admissions	80% of preventable ADEs
Outpatient	Honigman, 200177	Automated EMR screening	15,665 patients	5.5 per 100 patients in one year	2.0 per 100 patients in one year	23% of total ADEs
	Gurwitz, 200363	Clinician reporting, automated & manual record screening	27,617 Medicare + Choice enrollees	5.0 per 100 person-years	1.4 per 100 person-years	58% of preventable ADEs
	Gandhi, 200322	Patient interview	661 patients who received a prescription	27 per 100 patients	3.0 per 100 patients	10% of preventable ADEs

NOTE. This table includes all studies identified in the literature review (described in Model Formulation Process, in text) that systematically identified preventable ADEs among a defined base population.

^*Consecutive inpatients and outpatients in a three-physician community general internal medicine practice. Of the ADEs, 83% arose from outpatient prescriptions.

^†The study did not report the specific proportion of ADEs that were preventable. Rather, the estimate shown is based on assuming that the preventability of ADEs was the same as that of adverse events overall.

An additional group of studies has used emergency department visits or hospital admissions as sentinel events to identify possible outpatient ADEs. A study of 62,216 emergency department visits found that 1.7% resulted from outpatient ADEs.²⁶ A meta-analysis of 36 studies concluded that 5% of hospital admissions resulted from outpatient ADEs, with 23% of these due to patient errors.²⁷ Because about 7% of the U.S. population is hospitalized annually,²⁸ this study suggests that the population risk of being hospitalized due to an outpatient ADE is on the order of 0.35%. Since only 70% of the population receives outpatient care annually,²⁸ an outpatient's risk of an ADE requiring hospitalization would be about 0.5%, an estimate consistent with the serious ADE rates found in other studies.

In addition to their health impacts, ADEs increase work and resource consumption. In a study of inpatient ADEs, patients who suffered preventable ADEs had 4.6 additional hospital days and $5,857 in additional costs, compared with matched control patients on the same unit.²⁹ The study's authors projected that for a 700-bed teaching hospital, preventable ADEs led to additional direct costs of $2.8 million per year, exclusive of any malpractice losses or additional costs to patients. Another analysis found that few patient-level risk factors for ADEs could be identified, indicating that improved medication management is needed for all patients rather than select subgroups.³⁰

Nature and Causes of Preventable ADEs

For e-prescribing systems to optimally reduce ADEs, their design elements should be matched with the types of errors made when manual prescribing processes are used.¹⁴ A few studies have provided data on the nature and causes of preventable ADEs, though at varying levels of detail. ▶ shows how errors that contributed to ADEs were distributed across stages of the medication management process, among studies that provided these data. Overall, prescribing, administration, and monitoring appear to present much larger risks than dispensing. An additional study has reported that 45% of preventable ADEs were related to errors in medication choice, with another 20% being due to errors in dose or frequency.²²

Table 2.

Distribution of Errors Associated with Preventable Adverse Drug Events (ADEs) across Stages of Medication Management

	Burnum, 197620	Leape, 199578	Gurwitz, 200363
Setting	Inpatient and outpatient*	Inpatient	Outpatient
Total preventable ADEs (n)	23	70	421
Stage	Errors Associated with Preventable ADEs, n (%)†
Prescribing	10 (43)	41 (59)	246 (58)
Transcribing	Not included	2 (3)	Not included
Dispensing	1 (4)	4 (6)	<8 (<2%)§
Administration‡	12 (52)	40 (57)	89 (21)
Monitoring	6 (26)	Not included	256 (61)

^*An inpatient and outpatient general internal medicine practice—see footnote for ▶.

^†Percentages within columns sum to more than 100% because some preventable ADEs were attributed to errors at more than one stage.

^‡In the inpatient setting, administration errors are generally nursing errors; in the outpatient setting, they are generally patient adherence errors.

^§Number reported only as “less than 2%” in the published report.

The root causes of medication errors have been cataloged by two studies, both set in the inpatient environment. Though one study included only pharmacist-intercepted medication errors³¹ and the other used multiple methods to identify both intercepted and non-intercepted errors,⁵ the most common root causes were similar, as shown in ▶. In addition to the causes identified in the studies, illegible handwriting is also often cited anecdotally as a cause of medication errors. Though neither study found handwriting to be a frequent root cause, a 1994 report by the American Medical Association found that misinterpreted prescriptions were the second most prevalent and expensive malpractice claim category.³² Two hospital-based studies found legibility problems with 20% to 33% of orders.^33,34 Pharmacists usually avert harm from illegible prescriptions by calling the prescriber, but when illegible prescriptions are simply misread, the wrong medication is dispensed without a callback. In one fatal case, a prescription for Isordil (a long-acting nitrate for angina) was misread and dispensed as Plendil (an antihypertensive).³⁵ Drug names that look or sound like those of other commonly used drugs are the most susceptible to misinterpretation.³⁶

Table 3.

Root Causes of Medication Errors and the Resulting Error Types in Two Inpatient Studies

	Leape, 199578	Lesar, 199731
Error detection method	Chart review plus nurse & pharmacist reporting	Pharmacist interception
Root cause
Lack of knowledge about the drug	22%	33%
Lack of knowledge about the patient	14	29
Calculation, decimal point, units errors		18
Nomenclature errors (slips in drug name, dosage form, or abbreviation)		13
Faulty administrative processes		3
Rule violations	10
Slips and memory lapses	9
Transcription errors	9
Other*	34
Unclassified		3
Total	100%	100%
Error type
Wrong dosage	44%	58%
Wrong drug choice, other than allergy	25	22
Known allergy	12	13
Wrong route of administration	2	3
Other†	17	4
Total	100%	100%

^*Includes faulty drug identity checking, faulty dose checking, faulty interaction with other services, infusion pump and parenteral delivery problems, inadequate monitoring, drug stocking and delivery problems, preparation errors, and lack of standardization.

^†Includes orders written for or administered to the wrong patient, errors in timing of administration, failures to monitor drug therapy or to act on monitoring results, and “miscellaneous” errors.

Effects of Electronic Prescribing Systems

E-prescribing is one form of computer-based physician order entry (CPOE), which also includes computer-based orders for tests and other medical interventions. CPOE has been available in some hospital systems since the 1970s,³⁷ but it is seldom used by physicians.⁸ Evidence for the effects of CPOE is limited,¹³ but the available studies suggest that CPOE can improve patient safety and reduce hospital costs. The Regenstrief Medical Record System, at Wishard Memorial Hospital in Indianapolis, includes inpatient CPOE with drug-interaction and allergy warnings.³⁸ In a randomized controlled trial, inpatient medical teams assigned to CPOE generated 12.7% lower charges and a 0.9-day shorter length of stay than teams using handwritten orders.³⁹ The HELP system,⁴⁰ at LDS Hospital in Salt Lake City, has been associated with reductions in allergy, dosing, and antibiotic selection errors,⁴¹ and with reduced errors of omission for preoperative antibiotics.⁴² An inpatient CPOE system at Brigham and Women's Hospital in Boston included alerts for drug interactions, allergies, redundant medications, and relevant laboratory results. Following its implementation, the rates of serious medication errors (preventable ADEs + potential ADEs) fell by 55%.⁴³ After subsequent improvements, the rate of non-intercepted serious medication errors fell further, to a rate that was 86% below the baseline.⁴⁴ In contrast, a few distinctly negative experiences with CPOE have also been described. The University of Virginia Medical Center in Charlottesville experienced a houseofficer work-action in response to a new CPOE system, resulting in the system's being taken offline until user demands were addressed.⁶ Recently, a CPOE system at Cedars-Sinai Medical Center in Los Angeles was taken offline due to physician complaints.⁷

Evidence for the effects of outpatient e-prescribing is much more limited. PRODIGY, an outpatient system developed in Britain, involves prescribing from a list of medication/dose combinations recommended for the diagnoses entered on an encounter note. Preliminary studies have shown modest improvements in adherence to guidelines.⁴⁵ In a small study of outpatient ADEs, use of an elementary e-prescribing system, which did not include standardized medication or dose selection, was not associated with differences in preventable ADE rates.⁴⁶

Model Formulation Process

Literature Review

To systematically identify published literature related to the potential effects of e-prescribing, we searched the Medline database, including articles from 1966 through 2001, using combinations of the following terms: Medication errors; Drug therapy, computer assisted; Drug interactions; (Prescriptions, drug OR pharmaceutical preparations); (Reminder systems OR point of care systems); (Computer^* OR Internet OR software OR online systems); (Clinical pharmacy information systems OR medication systems). Unique citations were combined into a single data set (n = 2,591). The title and journal name of each citation in the data set were manually reviewed, and 1,609 citations were classified as “not relevant” to understanding the effects of electronic prescribing, leaving 982 potentially relevant articles. We reviewed abstracts from this set and retrieved full-text articles that might contribute to hypotheses about the effects of e-prescribing systems.

The literature review identified two models that have been used for analyzing failures of medication management processes. A model used by the Joint Commission for Accreditation of Healthcare Organizations considered five steps in medication management: prescribing, dispensing, administering, monitoring, and systems control.¹⁷ A model used by the Adverse Drug Event Prevention Study (ADEPS) considered four steps in medication management: ordering, transcription, dispensing, and administration.⁵ We formed a process flow model that integrated the steps of these two models, recasting the “transcription” step from the ADEPS model as the “transmission” step in our model in order to encompass either handwritten or electronic prescribing.

Review of Existing Systems' Functional Capabilities

After viewing live demonstrations of three commercial e-prescribing systems, we designed a structured telephone interview to elicit from vendor representatives the important functional capabilities offered in their e-prescribing product. A list of 127 companies offering electronic prescribing was abstracted from a published resource guide.⁴⁷ Based on an examination of each company's Website by at least two reviewers, we determined that 57 of these companies were resellers or otherwise not in the business of producing an original e-prescribing product. We telephoned the remaining companies and conducted a structured interview with a company representative who was qualified to answer technical questions about their e-prescribing products. The interview script (Appendix, available as an online data supplement at www.jamia.org) focused primarily on eliciting features of the prescribing step. Vendors that were difficult to reach by telephone were also contacted using a structured e-mail request for information. The project was approved by institutional review boards at RAND and UCLA.

The vendor interviews identified 56 distinct e-prescribing products from 49 companies. Results of the interviews were compiled into a list of major functional capabilities, which was then organized using our process model of medication management. Evidence for the effects of these functional capabilities was sought in the literature review results. Additional hypotheses were generated for possible effects of functional capabilities.

Model Description

Process Model of Medication Management

▶ shows the process model of medication management that we propose for organizing prescribing system evaluations. The model is expressed at a level intended to make it applicable for both handwritten and electronic prescribing, across patient settings. Central to the model are the five major activities involved in medication management: prescribe, transmit, dispense, administer, and monitor. For each activity, the diagram shows inputs, outputs, resources that may be occupied (which are drivers of the activity's direct costs), and information that may regulate the activity (which largely influences its quality).

Figure 1.

A function model of medication management. The major activities involved in medication management are shown as boxes. For each box, arrows on the left show the activity's inputs, those on the right show its outputs, those above show information that may influence the activity's performance, and those below show resources that the activity may occupy. This notation is based informally on the Integrated Definition for Functional Modeling (IDEF0).⁷⁹ In addition, the dashed lines indicate the potential unifying effects of system integration, making the same patient data available across activities. Black shading or bold lettering indicates an element that is mandatory for the particular activity. Gray shading indicates an element that is usually involved in the activity but is not mandatory. “e-Rx” is an abbreviation for electronic prescribing.

The “prescribe” activity is defined, in part, by the involvement of a prescribing clinician. The other major resources that may be used include paper records and e-prescribing systems. The activity's mandatory inputs are a clinician's assessments about the need for prescription medications. The patient's preferences regarding medications are also a desirable input. The “prescribe” activity may be informed by drug information, patient data (such as known allergies), and drug formulary restrictions, which may be available from print or electronic resources. The activity's mandatory output is a completed prescription. The prescribing step may also output information that helps the patient adhere to the prescribed regimen.

The “transmit” activity delivers a completed prescription for fulfillment. In traditional outpatient environments, patients would often complete this step without assistance, but clinicians or office staff may also participate, for example by telephoning prescriptions. The “dispense” activity involves a pharmacist except when medications are dispensed in the clinician's office. Pharmacists may also use information systems, and they may access the same types of information used in the prescribing step. Problematic prescriptions may require a call to the clinician; as a result, prescriptions may be changed or cancelled rather than being dispensed. The “administer” activity always involves the patient, at least as recipient, and it could involve a range of health care providers in some settings. “Monitoring” as we define it always involves the patient, at least as the subject of observation, and the clinician, whose assessments feed back into prescription changes.

E-prescribing Functional Capabilities

An e-prescribing system would influence the quality and efficiency of prescribing through functional capabilities intended to alter aspects of each medication management activity. ▶ summarizes potential effects from 14 individual e-prescribing functional capabilities. We identified seven of these in existing outpatient e-prescribing systems; seven others were identified only in the literature review. This section reviews the potential effects of each functional capability.

Table 4.

Potential Effects of Individual Electronic Prescribing Functional Capabilities

Step	Functional Capability	Potential Effects of Capability*
Prescribe	1. Patient selection or identification†	Wrong-patient errors (+ or −), clinician labor (+ or −)
	2. Diagnosis selection and diagnosis-based reminders†	Omission and drug-choice errors (likely −),45,49 accuracy of diagnosis data (+ or −), clinician labor (likely + or neutral)45
	3. Medication selection menus†	Wrong-dose and wrong-drug errors (+ or −),44,51 clinician labor (+ or −)54,55,56
	4. Safety alerts, based on:†	Drug-choice errors, including allergies (+ or −),41,44,58 clinician labor (+ or −)
	• allergies
	• drug–drug interactions
	• drug–disease interactions
	• drug–lab (renal, hepatic function)
	• body size, age (child, elder, …)
	5. Formulary alerts†	Formulary adherence (likely +),51 clinician labor (likely −), health care costs (+ or −)
	6. Dosage calculation	Dosage errors (likely −),60,61 clinician labor (+ or −)
Transmit	7. Data transmission to inpatient, retail, and/or mail-order pharmacy†	Transcribing errors (likely −), clinician labor (+ or −)
Dispense	8. Physician in-office dispensing†	Drug-choice errors (+ or −), clinician labor (+ or −)
Administer	9. Patient education materials, coordination of education activities	Outpatient adherence (+ or −), pharmacist, clinician, & staff labor (+ or −)
	10. Medication administration aids	Outpatient adherence (likely +)
	11. Refill and renewal reminders	Outpatient adherence (likely +)
Monitor	12. Corollary orders (e.g., for monitoring tests)	Monitoring errors (likely −)64
	13. Automated patient questionnaires to detect adverse effects; other structured follow-up communication	Patient failures to report significant adverse effects (likely −), patient adherence (likely +), clinician & staff labor (+ or −)
	14. Alerts for patients' failure to refill	Patient adherence (likely +), clinician & staff labor (+ or −)

^*Explanations of potential effects appear in the article text. Citation numbers indicate specific supporting evidence.

^†Functional capability identified in an existing commercial e-prescribing product.

Patient selection, often from a menu of patients in the clinician's practice, is usually the first e-prescribing step. The literature review did not identify specific evidence for the effects of this feature. However, since users often slip⁴⁸ when selecting from menus, the wrong patient may be inadvertently selected. Unless the system makes these slips easy to detect and correct, patient selection menus could increase the likelihood of wrong-patient errors. In the outpatient environment, wrong-patient prescriptions transmitted to a mail-order pharmacy could cause injury for patients who aren't vigilant. Patient selection could also increase or decrease clinicians' labor, depending on its workflow integration.

Diagnosis selection, when it is used to initiate prescribing, enables reminders that can increase adherence to practice guidelines, as demonstrated by the PRODIGY system⁴⁵ and others.⁴⁹ However, if clinicians are forced to enter a diagnosis for each prescription, they may enter incorrect or embellished diagnoses to access the medications they want to prescribe. This inaccurate information could then interfere with the system's subsequent performance. Furthermore, the diagnostic codes used for billing often fail to express clinically important distinctions⁵⁰; thus, relying on the same codes to guide billing and prescribing may compound inaccuracies. Requiring diagnosis selection would also add a step to the prescribing process, possibly increasing the clinician's labor unless this step also contributes to documentation and billing requirements. Users of the PRODIGY system, which combines diagnosis documentation with prescribing, perceived that the system made office visits longer, but the measured duration of visits was no different.⁴⁵

Selecting medication regimens using menus decreases wrong-dose errors by disallowing invalid combinations. The structured prescription data obtained also enable other safety measures such as alerts. With the institution of menu-based CPOE at Brigham and Women's Hospital, the proportion of orders with doses above the recommended maximum fell from 2.1% to 0.6%.⁵¹ However, errors in intravenous potassium orders markedly increased, returning to pre-CPOE levels only after potassium-ordering screens were modified.⁴⁴ This finding demonstrates that flawed prescribing menus can introduce new errors. Look-alike medication errors^52,53 might potentially increase if clinicians select an adjacent name on an alphabetized list and complete the prescription without recognizing the error. The resulting well-formed prescriptions would be very difficult for pharmacists to recognize as an error and intercept. Finally, menu-based prescription writing may increase physician labor. At Brigham and Women's, CPOE took substantially longer than handwriting, and the extra time was offset only modestly by time saved in looking for charts and responding to pharmacy calls.⁵⁴ CPOE with the Regenstrief System initially took twice as long as handwritten ordering,⁵⁵ but ordering speed improved substantially with experience.⁵⁶

Safety alerts may reduce the risks of preventable ADEs considerably. Following the institution of CPOE at Brigham and Women's Hospital, 50 to 80 orders per day were changed because of allergy alerts.⁴⁴ Alerts from the HELP system also dramatically reduced allergy and drug selection errors.⁴¹ However, drug interaction programs vary in their sensitivity and specificity,⁵⁷ and the thresholds set for triggering alerts could significantly influence their safety impact and costs. Highly sensitive systems that generate many “false alarms” could lead users to disregard future alerts; signs of this extinguishing effect have been observed.⁵⁸ Overly specific alerting could also degrade performance if prescribers expect their errors to be detected and review their prescriptions less carefully. Incomplete data on the patient's current regimen would also lead to alerting failures, even by systems set for high sensitivity. Thus, the adequacy of alerting could be particularly dependent on the prescribing system's integration with electronic medical records, not only within the prescribing clinician's practice but also with records from other clinicians who care for the patient.

Formulary alerts can also influence prescribing significantly. At Brigham and Women's, adherence to the formulary for intravenous H2 blockers increased from 14% to 88% with the implementation of a formulary alert.⁵¹ In most outpatient settings, formulary alerts would primarily reduce the work that results when payment is rejected at the pharmacy. To the extent that patients pay out of pocket rather than waiting for a prescriber to respond to a formulary rejection, formulary alerts could save money and reduce hassle for patients. Finally, formulary alerts may or may not reduce overall health care costs, since restrictive formularies can result in expenditures' being shifted from drugs to other health care services.⁵⁹

Computer-assisted dose calculations, which may rely on patients' body size, age, renal function, and other metabolic parameters, have increased prescribing accuracy in several studies.^49,60 The addition of automated renal function monitoring to the HELP system was associated with reductions in excessive antibiotic doses and in antibiotic-related ADEs.⁶¹ However, appropriate calculations may require e-prescribing systems to have access to medical record data such as age, body weight, and laboratory test results that reflect renal and hepatic function. If these data are only intermittently available, the resulting intermittent failure of dosing calculations could potentially increase dosing errors.

Transmitting data electronically from prescribing systems to pharmacies should eliminate human transcription errors, generally improving safety and efficiency. However, errors or clinician work might increase if transmissions are unreliable or if prescribing data must be re-entered manually at the pharmacy because of incompatible systems. Though this step is the least frequent source of preventable ADEs (▶), developing the required linkages with pharmacy systems may facilitate systems integration, resulting in more complete patient data being available to drive alerts and reminders.

Dispensing from physicians' offices is possible with some e-prescribing systems, potentially eliminating community pharmacists from the outpatient medication management process. If the prescribing system is less capable than pharmacists are at intercepting medication errors or educating patients, or if physicians are induced to prescribe from an in-office inventory, then drug-choice and patient-administration errors might increase.

The administration step is a frequent source of errors and might also be improved by e-prescribing functions. Well-designed educational materials should reduce outpatients' self-administration errors. Systems that facilitate physician, nurse, and pharmacist collaboration in patient education could further augment adherence.⁶² However, poorly designed patient educational materials may result in inconsistent instructions or misunderstandings that actually increase errors. Systems that help schedule and track drug administration should also reduce administration errors. Currently, inpatient systems usually feature drug administration tracking, but outpatient systems could also foster tracking, for example, by providing patients with printed daily administration schedules.

Features that enhance monitoring may be particularly important for outpatient prescribing, given the frequency of errors found at this step.⁶³ Automatic orders for laboratory monitoring tests more than doubled physicians' adherence to recommended testing.⁶⁴ Prescribing systems with access to pharmacy data could inform the clinician when patients fail to fill prescriptions on time, allowing clinicians to address patient nonadherence, a task that physicians currently perform poorly.⁶⁵ Prescribing systems might also improve monitoring by administering patient questionnaires that detect important adverse reactions and trigger more timely appropriate action.⁶² Although clinicians might need to devote additional time to monitoring tasks, their time might be saved overall if this information led to better patient communication. However, in most clinical settings, each of these monitoring capabilities would require extensive integration with external information systems, thus making them possibly less achievable in the near term.

Applying the Conceptual Framework

To apply the proposed conceptual framework in evaluating specific e-prescribing systems, an evaluator would first use the process model (▶) to assess prescribing in the intended clinical environment at baseline, before the implementation of any new systems. Then, for each activity in the process model, the evaluator would list the functional capabilities of the proposed e-prescribing system that might influence the activity's outputs. These capabilities might include those shown in ▶ as well as other features that may be novel among e-prescribing systems. Finally, each functional capability's potential effects would be hypothesized, using evidence from the literature where it is available.

Validation by Example

As an example of applying this conceptual framework, we assess the potential effects of two specific, unnamed e-prescribing systems in a community general internal medicine office. ▶ summarizes the comparison of each system with the baseline features of medication management in the practice before e-prescribing.

Table 5.

Summary of Results from Applying the Conceptual Framework to Evaluate Two Alternative Outpatient Electronic Prescribing (E-Rx) Systems

	System A (Handheld)		System B (EMR Module)
Baseline Capabilities	E-Rx Capabilities	Potential Effects	E-Rx Capabilities	Potential Effects
Prescribe
Stickers used to identify patient on Rx	Select patient from schedule; patient identity not displayed in later steps	Clinician labor (minimal change)	Selection from list of all patients; patient identity displayed throughout	Clinician labor (possible slight +)
		Wrong-patient errors (possible unintended +)		Wrong-patient errors (likely −)
Paper chart sometimes missing	(No diagnosis entry or reminders)		Paper chart is replaced; diagnosis-based reminders	Omission errors (likely −)
				Clinician labor (possible +)
Meds, doses handwritten	Medication menus present valid doses	Wrong-dose errors (likely −)55,56	Medication menus present valid doses	Wrong-dose errors (likely −)55,56
Drug reference in exam room	(No safety alerts or dosage calculations)		Safety alerts (allergies & drug interactions)	Drug-choice errors & allergies (likely −)
				Clinician labor (likely +)
No formulary information	Formulary status icons on medication menu	Clinician labor (likely−from preventing callbacks)	(No formulary information)
Transmit
Patient's responsibility	Electronic fax to inpatient, retail, and/or mail-order pharmacy	Transcribing errors (likely − or no change)	Electronic fax to inpatient, retail, and/or mail-order pharmacy	Transcribing errors (likely − or no change)
Refills by telephone		Clinician & staff labor (slight + from selecting pharmacy, possible − telephone time)		Clinician & staff labor (slight + from selecting pharmacy, possible − telephone time)
Dispense, Administer, Monitor
Limited interaction checking at pharmacy	(No support)		(No support)
Verbal patient instructions

In the baseline environment of this office, a practice management system produces stickers that physicians use to identify the patient on paper prescriptions. Patient history information (e.g., allergies) may be available directly from the patient or in handwritten medical records, though the medical record is sometimes not available due to filing problems. Medications and dosages are then handwritten, based on drug information in the clinician's memory and in reference books available in the examination room. Drug formulary information is not readily accessible. Explicit drug safety checking is performed occasionally, using the available reference information. Clinicians' labor is the primary resource used. Transmitting prescriptions to the pharmacy relies largely on patients' labor, but prescription refills are commonly handled by telephone. Errors are rarely introduced at this step. Dispensing takes place at a variety of community pharmacies, where third-party formulary adherence is enforced. Formulary mismatches are common for new prescriptions, resulting in additional labor for pharmacists and physicians. Pharmacy computer systems also perform checks for drug interactions, but interactions are rarely discovered. Patients self-administer medications based largely on verbal instructions from the pharmacist and the physician. Monitoring relies on physicians' or patients' initiative. Overall, medication management in this environment likely leads to preventable ADEs at rates similar to those summarized in ▶ for outpatient prescribing.

One e-prescribing alternative is a standalone system that uses a handheld computing platform. Prescribing begins with clinicians' selection of a patient from their schedule, which can be loaded daily into the e-prescribing system from a practice management system. The patient selection process is quick, but the selected patient's name is not displayed through subsequent prescribing screens. Thus, the system misses an opportunity to help clinicians intercept wrong-patient errors. The system does not support diagnosis selection or diagnosis-based suggestions, removing the potential positive and negative effects of that feature from consideration. The system does support medication selection menus that enforce appropriate doses, thus likely decreasing wrong-dose errors. The system also indicates the formulary status of medications within the menus, likely reducing the clinician's time spent on pharmacy callbacks. Safety alerts and dosage calculations are not supported, thus omitting a large category of potential safety benefits. The system can transmit prescriptions to local or mail-order pharmacies via electronic fax, possibly saving clinicians and staff time on the telephone, at the cost of extra labor in choosing a pharmacy and dealing with failed transmissions. At the pharmacy, faxed prescriptions would still need to be manually entered into an information system; thus the rate of errors at this step may be unchanged. The system has none of the capabilities shown in ▶ that could support the dispensing, administration, or monitoring steps. In summary, this system offers some potential benefits in decreasing wrong-dose errors and decreasing time spent on formulary-related telephone communications, but it might increase the risk of wrong-patient errors and it omits many possibly beneficial features, thereby missing many potential benefits as well as some risks.

The other e-prescribing alternative is an integrated module of an electronic medical record system. The system is intended for use during the clinical encounter, replacing the paper medical record. The patient's identity is selected from a list of patients in the practice, not from the clinician's schedule, since the system does not interface with the office's practice management system. Thus, patient selection might be slightly less efficient for the prescriber; however, the patient's identity is displayed in the interface throughout, likely decreasing wrong-patient errors. Diagnosis selection is performed as part of documenting the current encounter, and the system provides diagnosis-based reminders, likely improving guideline adherence. However, clinicians may prescribe without a diagnosis, and they are not limited to the medications recommended for the diagnosis. Medication selection is generally quick, but clinician labor may be slightly increased by the extra step involved in linking documentation with prescribing. Medications and doses are selected from menus presenting only valid combinations, likely decreasing wrong-dose errors. The system provides safety alerts based on allergies and drug–drug interactions, but not body size, age, drug–disease, or drug–laboratory interactions. Current internist users estimate that about 10% of prescriptions generate an alert and that 10% of those are useful, suggesting that the specificity of alerts may be relatively low. Nonetheless, these alerts would likely prevent some proportion of drug-choice errors, at a cost of increased clinician labor. The system provides no formulary information, thus missing an opportunity to prevent pharmacy calls. For the remaining steps of medication management, the system is no different from the first system reviewed—completed prescriptions can be transmitted to pharmacies by electronic fax, and the system has no capabilities to support dispensing, administration, or monitoring. In summary, this system's integration with an electronic medical record enables features that would likely reduce drug-choice errors and omission errors, compared with the current environment or with the first e-prescribing system reviewed. However, this system may have fewer benefits for office efficiency, given the apparent low specificity of alerts and the lack of formulary information that could prevent calls from pharmacies.

Discussion

Health care organizations have compelling reasons for interest in outpatient e-prescribing. The evidence summarized in ▶ and ▶ indicates that a primary care provider with 3,000 patients could expect 45–90 preventable ADEs per year among his or her patients, with more than half of these resulting from a prescribing error. In academic hospitals, CPOE with e-prescribing has been associated with dramatic improvements in medication errors and in guideline adherence. However, the generalizability of these findings to commercial systems and community outpatient settings remains uncertain.¹³ Moreover, the structural and cultural variance among health care organizations combined with the variance in functional capabilities among e-prescribing systems will likely make any evidence about overall e-prescribing effects difficult to generalize.

This paper describes a conceptual framework for evaluating e-prescribing that accounts for variation among clinical settings and among e-prescribing systems. Using our approach, evaluators first construct a version of the medication management process model (▶) that is localized for the intended patient care setting. Next, they identify the e-prescribing functional capabilities that might influence each activity in the process (▶). Finally, they consider the effects from each functional capability, integrating specific evidence where it is available. Since e-prescribing systems can introduce errors that did not exist with handwritten prescribing,⁴⁴ the framework specifically guides users to consider both the positive and the negative effects of each system feature.

Our framework organizes potential e-prescribing features using a process model of the activities in medication management. We believe that this organization of the framework provides a comprehensive view of a system's potential effects within its environment, using a sequence of activities that are familiar to most clinicians and administrators. Other investigators have used a similar process model to simulate the results that could be expected from different degrees of ADE reduction within each medication management activity.^66,67 Our approach differs in that it opens the “black box” of each activity to consider how system design features would alter the activity.

One limitation of the proposed conceptual framework is the relative paucity of evidence available to support judgements about the effects from different e-prescribing design features. Although we found some evidence to support 8 specific effects for six of the 14 functional capabilities we catalogued (five effects related to safety, one to formulary adherence, and two to clinician labor), all of this evidence needs to be considered preliminary given the limited number of studies. Overall, the evidence for functional capabilities is not yet sufficient to support quantitative modeling of effects. Thus, the statements resulting from an application of our framework to individual vendor systems need to be considered hypotheses—of varying strength—rather than conclusions.

Users of our model should also be aware that the 14 functional capabilities we identified do not represent an exhaustive list. In identifying functional capabilities, we focused primarily on the “prescribing” step of medication management. Capabilities that we have not explicitly considered, especially involving later steps of the process model, may also be important in determining the effects of e-prescribing systems. However, the proposed evaluation framework has the virtue of being able to accommodate additional functional capabilities that we have not considered.

Though the current evidence is incomplete, it nonetheless supports a set of e-prescribing design priorities. Because errors in medication selection are most common, functional capabilities such as medication selection menus and safety alerts may be the most important for preventing ADEs. Because the administration and monitoring steps are also common sources of outpatient preventable ADEs, adequate support of outpatient education and follow-up monitoring may also reduce health risks significantly. Because the transmission and dispensing steps accounted for fewer than 5% of preventable ADEs, electronic transmission of prescriptions may have a lesser impact on patient safety than it does on process efficiency. When evidence is incomplete, a Delphi panel process can help to codify expert opinion about best practices.⁶⁸ We are currently conducting an expert panel process to develop quality standards for e-prescribing.

Another limitation in the evidence base is the lack of information about factors that can lead to unintended hazards. The experience with potassium ordering screens at Brigham and Women's Hospital⁴⁴ demonstrated that occult hazards can exist in e-prescribing systems, but the specific factors that caused this hazard were not identified. Analysis of the Therac-25 radiation therapy accidents found that rare, fatal overdoses arose from interactions between software and hardware flaws coupled with frequent, cryptic error messages to which users became insensitive.⁶⁹ Some unintended hazards may be prevented by better software engineering and evaluation,⁷⁰ but identifying rare hazards may still depend on user vigilance.⁶⁹ Future work integrating the functional capability approach with cognitive models of human errors^71,72 may prove valuable for predicting user interface hazards. Until then, our framework may still help evaluators to systematically consider possible hazards by applying basic principles of user interface design⁷³ when examining each system capability. More importantly, implementations of e-prescribing should be accompanied by specific efforts to monitor for and respond to unexpected quality and efficiency problems.

Clearly, more research is needed in e-prescribing effects. Given the high costs of controlled trials, some experts have argued that less-rigorous evidence must be accepted for safety-related information technology (IT).⁷⁴ However, others have countered that the harms potentially arising from IT interventions justify significant investments in evaluation before the nation commits to the high costs of implementation.⁷⁵ We posit that experimentation might focus most productively on alternative functional capabilities within different clinical environments rather than on e-prescribing vs. traditional prescribing. As more complete evidence is accumulated, our framework may serve as a starting point for quantitative models. Future work might also expand the framework to deal with other aspects of CPOE, such as computerized test ordering and results retrieval.

Finally, little information is available about the costs of creating and maintaining e-prescribing systems. These costs would likely be greatest for institutions starting with less-developed informatics infrastructures. Whether e-prescribing systems will be cost-saving or whether their potential health benefits will require additional net investments in health care remains unknown. Future studies of e-prescribing systems will be most useful if they can capture implementation costs.

Conclusions

We propose a conceptual framework for evaluation of e-prescribing that facilitates the consideration of systems' functional capabilities in projecting their expected effects. The framework uses a process model of medication management to comprehensively identify the relevant functional capabilities. The 14 functional capabilities we have identified through our review of the literature and of existing systems can serve as an initial list of features to look for. However, rather than stopping with a simple checklist, the proposed framework guides the integration of existing evidence in judging the potential effects of each feature within a specific health care environment.

This proposed methodology of focusing assessments on functional capabilities may have utility for assessing a broader range of information systems. Future research should focus on collecting evidence for the effects of functional capabilities and also on the factors that may lead to unintended hazards from e-prescribing. The collection of evidence at the level of functional capabilities may provide a way to generalize results from one system evaluation to another.