The Development and Evaluation of an Integrated Electronic Prescribing and Drug Management System for Primary Care

Abstract

Objective: To develop and evaluate the acceptability and use of an integrated electronic prescribing and drug management system (MOXXI) for primary care physicians.

Design: A 20-month follow-up study of MOXXI (Medical Office of the XXIst Century) implementation in 28 primary care physicians and 13,515 consenting patients.

Measurement: MOXXI was developed to enhance patient safety by integrating patient demographics, retrieving active drugs from pharmacy systems, generating an automated problem list, and providing electronic prescription, stop order, automated prescribing problem alerts, and compliance monitoring functions. Evaluation of technical performance, acceptability, and use was conducted using audit trails, questionnaires, standardized tasks, and information from comprehensive health insurance databases.

Results: Perceived improvements in continuity of care and professional autonomy were associated with physicians' expected use of MOXXI. Physician speed in using MOXXI improved substantially in the first three months; however, only the represcribing function was faster using MOXXI than by handwritten prescription. Physicians wrote electronic prescriptions in 36.9 per 100 visits and reviewed the patient's drug profile in 12.6 per 100 visits. Physicians rated printed prescriptions, the current drug list, and the represcribing function as the most beneficial aspects of the system. Physicians were more likely to use the drug profile for patients who used more medication, made more emergency department visits, had more prescribing physicians, and lower continuity of care.

Conclusion: Primary care physicians believed an integrated electronic prescribing and drug management system would improve continuity of care, and they were more likely to use the system for patients with more complex, fragmented care.

Background

Prescription Drug Use and Drug-related Illness

Drug-related illness accounts for 5% to 23% of hospital admissions^1,2,3,4 and is now claimed to be the sixth leading cause of mortality.⁵ Hospital-based studies of adverse events systematically identify errors in prescribing and drug management among the leading causes of preventable injury or death.^{6,7,8,9,10,11,12,13,14} Preventable adverse drug events in ambulatory practice, where the vast majority of prescriptions are generated, are estimated to occur in 2% to 3% of patients treated per year, of which 58% are related to prescribing errors.^15,16,17,18 A variety of potential causes of ambulatory prescribing problems have been identified including: a rapid increase in the number of drugs, contraindications, interactions, and side effects^19,20,21; multiple prescribing physicians and dispensing pharmacies^22,23; and transcription errors.²⁴ Indeed, there is sufficient concern over mistakes made in transcribing the written prescription that the U.S. Medicare Prescription Drug and Modernization Act of 2003 requires the nationwide implementation of an electronic prescription drug program by January 1, 2006.²⁵

Improving Drug Safety by Computerized Prescribing and Drug Management Systems

It is widely accepted that electronic prescribing and integrated drug information systems can reduce avoidable errors in prescribing and dispensing.^{16,19,20,26,27,28,29,30,31,32,33,34,35} The specific functionalities needed to improve the safety and quality of drug management have been defined^16,33 based on an analysis of the causes of preventable adverse drug events. These include the (1) integration and display of patient demographic information from office management systems, (2) retrieval and display of all currently active drugs from community pharmacy systems, (3) automated alerts for relevant prescribing problems (therapeutic duplication, excess dose, dose-adjustment for weight and renal impairment, drug–disease, drug–drug, drug–age, and drug–allergy contraindications) prioritized by importance, (4) integration of electronic prescriptions into pharmacy software to avoid transcription errors, (5) transmission of orders to discontinue medication to dispensing pharmacies, and (6) monitoring of patient adherence and treatment outcomes.

Yet, for the vast majority of ambulatory-care physicians, available prescribing systems do not provide these critical functions. Most systems offer stand-alone prescribing capability or integration with patient demographics, drug interaction information, and the capacity to fax prescriptions to a pharmacy selected by the physician.^35,36 Drug histories are not integrated from dispensing pharmacies, disease and allergy information must be manually entered and even then may not be saved or integrated into decision-support systems.³⁶ Faxed prescriptions are not legally accepted in many jurisdictions because the source of the prescription cannot be adequately authenticated and selection of the dispensing pharmacy by the prescribing physician compromises the patient's right to choose the dispensing pharmacy.³⁷

Integrating these required components into an electronic prescribing system requires a secure information technology (IT) infrastructure. Physicians can then obtain information on dispensed prescriptions from community-based pharmacists, transmit prescriptions electronically, and manage refill requests efficiently.³⁵ However, unlike the hospital environment where a single pharmacy is involved in dispensing prescriptions for hospital patients, the ambulatory setting is more complex as up to 40% of patients may obtain their prescriptions from more than one retail pharmacy and 60% to 80% obtain prescriptions from different physicians.²³ This vital IT infrastructure has not been established,³⁵ in part because of a lack of common governance structure in ambulatory care, inadequate funding, and the absence of legislative or regulatory requirements. The lack of integration of electronic prescribing systems is considered to be the single most important barrier to physician adoption.^35,38 If progress is to be made in improving patient safety, we need to investigate whether the provision of an IT infrastructure for electronic prescribing in ambulatory care results in the expected increase in physician use of these systems, particularly for high-risk patients.

We took advantage of a unique situation in a population-based health insurance system in Quebec, Canada, to develop and evaluate the acceptability and use of an electronic prescribing and integrated drug management system among community-based primary care physicians. Specifically, we assessed system performance, speed of skill acquisition with training, pre- and postimplementation perceptions of benefit, the use rate of different components of the system, and the use in higher risk populations.

Methods

Context

The Canadian province of Quebec provides a universal health insurance program that covers the costs of essential medical care for 8.5 million provincial residents. The Quebec health insurance agency (RAMQ) is responsible for beneficiary enrollment and reimbursement of all physicians and pharmacies. The RAMQ maintains a database of all Quebec beneficiaries (name, age, sex, residence), all medical services received by beneficiaries (date, diagnosis, type and location of service and provider), and medications received (medication, date dispensed, prescribing physician and dispensing pharmacist, quantity and duration of prescription) with data validated in prior research.³⁹ Public drug insurance is provided to approximately 50% of Quebec residents (the elderly, welfare recipients, and persons not insured through their employer). The remainder must, by legislation, be covered by employer-sponsored private drug insurance. All private insurers need to provide the same formulary for insured drugs as the public insurance program.

Development of an Electronic Prescribing and Integrated Drug Management System: Overview

Most ambulatory physicians have minimal resources to manage local computer systems.²⁷ Therefore, we developed MOXXI (Medical Office of the XXIst Century) as a portable electronic prescribing and integrated drug management system that allowed physicians to write and transmit prescriptions from any location using a personal digital assistant (PDA), retrieve the drug history, recent emergency department visits, and hospitalizations and have a patient's drug, disease, and allergy profile reviewed for potential prescribing problems through wireless synchronization with a central server that retrieved medical and pharmaceutical data through interfaces with pharmacy and provincial insurance systems. The MOXXI IT infrastructure is based on multitiered client server architecture (see Web-based technical appendix). In addition to providing contacts, calendar, Internet access, and PDA versions of drug monographs, six key functionalities were developed to enhance safety and physician adoption.

Functionalities to Enhance Safety and Physician Adoption

Leveraging Existing Electronic Health Data to Facilitate Physician Adoption

Preloading and integration of patient demographic information: Once physicians were recruited into the study, demographic data including the provincial unique person identifier (NAM) for all patients they saw and billed for in the past year were retrieved from the RAMQ and automatically populated into their information tables. Patients seen for the first time would need their information to be manually entered. This process minimized data entry requirements for physicians and improved the accuracy of patient information.

The automated health problem list was developed to facilitate the detection of drug–disease therapy problems and avoid burdening physicians with data entry. Information on potential problems was extracted from all ICD-9 diagnostic codes recorded by any physician on medical services billings for a given patient. Based on prior research, these codes have high specificity but lower sensitivity.⁴⁰ To improve sensitivity, a single indication drug–disease table was created and the list of medications each patient was dispensed in the past year was analyzed and used to convert all single-indication drugs into the related disease (e.g., insulin = diabetes, type 1 or 2). Potential problems (those records generated through external sources) are presented to the physician for validation. In addition, when physicians prescribe through the electronic prescription (e-Rx) they are required to record a therapeutic indication. All therapeutic indications selected for prescribed therapy are added to the problem list. Physicians can flag problems that are no longer active or manually add problems by menu selection.

Allergy information cannot be retrieved from historical files. To facilitate entry of data, physicians can add allergies in the health problem entry screen by selecting the relevant drug to which the person is allergic. All drugs with related chemical properties are identified by the drug knowledge database and included in the surveillance for potential drug allergy problems.

Electronic Prescription (e-Rx) Capabilities

Electronic prescriptions are generated by the user finding and selecting a drug name (by entering the first three letters of either the generic or brand name), and then from a list of standardized directives (▶A). These directives populate the prescription builder fields (drug name, dose, route, frequency, instructions for administration, duration of treatment, repeats) with values that can be subsequently modified. Quantity to be dispensed is calculated automatically. Physicians can also select a drug order set from a personalized list of previously prescribed medication. A novel feature of the MOXXI e-Rx prescribing function is the requirement for physicians to record at least one treatment indication from a list of all approved or documented indications for each drug prescribed. Capability of free-text entries for unlisted or off-label indications also exists. Pharmacists' knowledge of treatment indication was considered to be essential to improve the safety of dispensing and appropriateness of patient counseling. Renewals of previously prescribed medications are facilitated by providing a list of current medication that automatically generates new prescriptions to be sent, when selected.

Figure 1.

MOXXI System Functionality. A, Electronic prescription pad with mandatory documentation of therapeutic indication. B, Medication profile display of information on dispensed prescriptions retrieved through the integrated MOXXI system.

Capability of Electronic Prescription Retrieval into Pharmacy Management Systems

When the patient takes the printed copy of a MOXXI e-Rx prescription to a participating retail pharmacy the pharmacist can enter the unique prescription number contained in the printout and access the associated electronic prescription transaction. After verifying a match with the correct patient identification information, the electronic prescription transaction is checked out and locked as a “filled” prescription and is integrated into the pharmacy management software, thereby eliminating requirements for data entry.

Transmission of Orders to Discontinue Medication to Dispensing Pharmacies

Stop and change medication orders were added as a key safety feature. Physicians can select a currently dispensed medication and issue a stop or change (dose or drug) order, along with recording a reason for their decision using a drop down list selection. These orders are transmitted to the original dispensing pharmacy by electronic transmission along with all new and refilled medications, and are recorded on the printed prescription.

Medication Profile Display and Patient Medication Adherence and Outcome Monitoring Tool

A graphical representation of a patient's medication profile was developed to display a summary view of current drug therapy, as well as medical services used from all available data sources (▶B). Medications prescribed or dispensed in the past six months are shown with a color legend corresponding to (1) whether a medication that was prescribed has been dispensed, (2) medications prescribed by other physicians, (3) lapses in treatment as calculated by drug by day exposure, and (4) therapy overlap days. Drug cost information, ER visits and hospitalizations are also displayed. By clicking on any drug in the list, the physician can view details about the records of prescriptions dispensed, the mean prescription costs in the past 6 months, and a bar graph representation of calculated refill compliance in the past three months.^41,42 The profile view also highlights (by color-coded severity) those drugs that have been identified as potential prescribing problems. This graphical medication “radar screen” allows the physician to assimilate an enormous amount of information quickly and concisely to enable better decision making.

User-selectable Alerts Filtering for Potential Prescribing Problems

The drug knowledge base uses documented allergies, verified diseases, and all active prescribed and dispensed medications as input to assess potential prescribing problems related to: drug–disease, drug–age, and drug–allergy contraindications, drug interactions, duplicate therapy, cumulative toxicity (multiple drugs with the same side effect, e.g., sedation), and drug dose (too high, too low) for the specific patient selected. The drug knowledge base screening is triggered with each prescription generated and can also be run manually at any time. Alerts are displayed as pop-up messages and are categorized into three levels of severity: (1) absolutely contraindicated, (2) should be avoided if possible, and (3) use with caution. Physicians can selectively filter the severity of alerts to be displayed and can suppress any given alert for a particular patient or for all patients if they consider it to be clinically irrelevant. Alerts are displayed when opening a patient file, when viewing the drug profile, and when generating a prescription. When problems are identified as part of the automated surveillance, the physician must either revise the prescription or document from a drop-down menu the reason for ignoring an alert (e.g., benefit greater than risk). An audit trail is maintained to document the alert setting, alerts that are seen, the patient involved, and the physician response (revision or rationale for ignoring).

Evaluation of the Electronic Prescribing and Integrated Drug Management System

Design and Study Population

The performance, acceptability, and use of the MOXXI system were assessed in a longitudinal 20-month follow-up study (March 2003 to November 2004) of primary care physicians who were in full-time fee-for-service practice in a large metropolitan area, geographically proximal, community-based pharmacies, and consenting patients from the physicians' practice population. Primary care physicians and pharmacists were identified by professional association master lists and contacted by letter and telephone to determine their interest in participating. The list of potentially eligible patients comprised of those who visited in the past year was determined from the provincial RAMQ medical services claims, physician, and beneficiary files. Physicians consented patients using the electronic patient list provided by the RAMQ that was also used to prepopulate their practice population in the MOXXI system. The study was approved by the Faculty of Medicine Institutional Review Board, the Access to Information Commission, RAMQ legal counsel, the College of Physicians, and the Order of Pharmacists.

Physician Training

Physicians were trained in a three-hour small group session to use the MOXXI system. Presentation and demonstration were followed by supervised one-on-one support. A follow-up visit was made to the physician's practice setting within 72 hours of training to provide assistance in using the drug management system in daily practice. Follow-up visits were subsequently conducted every one to two months or more frequently as needed to address unique issues of individual physicians.

Technical Performance

Slow technical performance is an established barrier to physician use of computerized systems.⁴³ A standardized system response time assessment was developed to test the technical performance of the system in eight fundamental task areas: logging in, calling up a patient file, accessing the medication history, accessing the drug menu for a prescription, entering the drug directive for a prescription, entering an indication for a prescription, conducting an automated drug review for potential problems, and saving and printing a prescription. As performance may vary by such factors as the size of the physician's practice database, and reliability of telecommunications transmission, speed of performance on the eight tasks was assessed in each physician's practice by the field coordinator using the system log to record response time in seconds. The assessment was conducted for the Visual Basic and C++ versions of the MOXXI system.

Physician Skill Acquisition

To assess the speed with which physicians were able to conduct basic drug management tasks, we conducted a standardized assessment of their performance in writing new prescriptions, adding a problem or allergy to the problem list, and represcribing medication. Physicians were asked to complete the standardized performance task at the end of the training session, after the first two weeks of system implementation, and after three months. With each test administration, a research assistant observed and recorded the time it took for each physician to complete each component of the standardized task.

Acceptability of the System

Using an adaptation of the technology acceptance model questionnaire,^44,45 physicians rated the ease of use, intent to use, expected impact, and perceived value of the MOXXI system on a five-category Likert scale. Physicians completed the questionnaire at the end of training and eight months after implementation. In the follow-up questionnaire, physicians were also asked to rate the perceived value of 16 components of the MOXXI system.

Use of the MOXXI System

To assess use, we measured the proportion of times the electronic prescribing and drug profile components of the system were used in visits made by patients who had at least one dispensed prescription and visited each study physician during the 20-month implementation period. Use assessment was based on the number of visits as it provided a more precise and comprehensive representation of the potential number of opportunities the system could have been used by each physician for prescribing and drug profile retrieval than dispensed prescriptions.

Visit information was retrieved from RAMQ databases, where the medical service claim beneficiary, physician, and date of service data were used to create a visit count for each patient and physician, and the pharmacy claim beneficiary and date of service data were used to restrict the visit count to patients with active prescriptions. The numerator for each rate was the number of times the physician wrote a prescription or accessed the drug profile during the implementation period for all patients who made a visit with an active prescription. These data were retrieved from audit trails maintained by the MOXXI system that records the date, beneficiary, and component of the system accessed by each physician.

Assessment of Use in Relationship to Patient Characteristics

To determine whether physicians were more likely to use the electronic prescription or drug profile components of the system for higher risk patients, we measured the rate of use by the patient's number of active medications and by the extent of care fragmentation (number of prescribing physicians, number of emergency department visits, proportion of visits to the primary care study physician). The RAMQ medical services and prescription claims databases were used to calculate the number of medications and pattern of service use for each patient using methods developed previously.^{39,41,42,46,47,48} Analysis of this outcome was confined to individuals with RAMQ drug insurance to ensure completeness of drug history information.

Analysis

Descriptive statistics were used to summarize information on technical performance, physician skill acquisition, and acceptability ratings. Rates of use were graphically summarized bimonthly during the implementation period, overall, and by physician and patient characteristics. To test whether physicians were more likely to use the drug management system for patients with a greater number of medications and more fragmented care, we used a Poisson regression model within a generalized estimating equation (GEE) framework to estimate the association. Patient was the unit of analysis, physician was the clustering factor, and an exchangeable correlation structure was used to account for dependence among the residuals. A Poisson GEE regression model was also used to determine whether baseline assessment of the intention to use the MOXXI system was associated with overall use rates in the first 24 months, where physician was the unit of analysis, bimonthly use was the repeated measure for each physician, and an autoregressive order 1 correlation structure was used to account for dependence among residuals.

Results

The mean age of the 28 physicians (22% of 127 eligible physicians) who implemented the MOXXI system in their practice was 43.4 years; 46% were female. All but two had prior experience using a PDA or computer at the time of enrollment, although only 10.7% used a computer for clinical activities prior to the MOXXI implementation; the mean number of hours of weekly computer use was 7.1 hours. 13,515 (19.1%) of the physicians' 70,805 eligible practice patients consented. Physicians were more likely to obtain consent from patients who were older, visited more frequently, and were covered by the public drug insurance plan.⁴⁹ The mean age of participating patients was 55.0 years; 59.8% were female and 34.3% of their 704,034 prescriptions were written by other physicians. The 38 pharmacies (92.7% of eligible pharmacies) and 90 pharmacists who participated in the evaluation represented five pharmacy information software systems. All but one pharmacy information system provider was willing to establish an interface to retrieve information on dispensed prescriptions from the pharmacy systems. Although all pharmacists wanted to receive electronic prescriptions, only two pharmacy system providers were willing to establish the interface to retrieve electronic prescriptions and receive stop orders. Cost, priority, lack of standard requirements, absence of government mandate, and impact on business practices were all factors that delayed or prevented comprehensive collaboration from the retail pharmacy community. Incomplete pharmacy participation influenced the type of patients for whom the physicians sought consent.⁴⁹

Technical Performance

Between March 2003 and May 2004, physicians were issued the initial solution using the iPaq model 3870 (206 MHz, 64 MB RAM) running the Microsoft Visual-Basic version of the application. In July 2004, users were upgraded to a model 5550 iPaq (400 MHz, 256 MB RAM) and the much faster C++ version of the software. ▶ shows the significant performance enhancements evidenced by time required to retrieve the electronic medication history, enter a prescription, and receive an alert. Several factors contributed to this enhanced performance: a PDA with a faster CPU and more RAM, a more efficient PDA operating system, transition from an application that ran in an interpreted mode to one that was compiled, improved processing overhead through better Oracle Lite database drivers, and optimization of the wireless modem functioning with better bandwidth access.

Table 1.

Technical Performance of the MOXXI System

	Version 1 (28 Trials)	Version 2 (28 Trials)
Task	Mean in Seconds (SD)	Mean in Seconds (SD)
Logging in	50.69 (43.03)	49.53 (1.30)
Calling up a patient's file (tap name from list)	7.56 (3.81)	4.37 (2.86)
Accessing drug profile	23.13 (13.50)	1.44 (0.85)
Prescribing a drug
Calling up prescription pad	5.10 (6.11)	0.99 (0.20)
Search for a drug (e.g., ati)	2.33 (3.12)	1.26 (0.09)
Entering directives	3.86 (2.78)	5.80 (0.54)
Entering indication	0.58 (0.40)	0.29 (0.05)
Viewing a prescription alert	8.75 (4.18)	1.53 (0.63)
Calling up the prescription review system	16.69 (3.74)	9.67 (5.86)
Saving and printing a prescription	49.93 (145.83)	28.31 (2.02)

SD = standard deviation.

Physician Skill Acquisition

Physicians took the greatest length of time to complete basic prescribing functions using the computerized system immediately after training; two weeks after training, the time required to use the system was reduced by more than 50% (▶). By three months post-implementation, systematic but more modest reductions in performance time were achieved. In comparison to time taken to produce a handwritten prescription for the three new medications, the computerized prescribing system (Visual-Basic version) took more than twice the time, even at three months. However, renewal of medications was substantially faster using the computer-based prescribing system, even in the immediate posttraining assessment (▶).

Table 2.

Standardized Task Performance Speed of the 28 Primary Care Study Physicians on the Visual-Basic Version of the MOXXI System Immediately after Training and at Two Weeks and Three Months Post-implementation

	After Training	2 Weeks Post-implementation	3 Months Post-implementation
	Mean in Seconds (SD)	Mean in Seconds (SD)	Mean in Seconds (SD)
Task using the PDA
Search for a patient	34.1 (48.7)	12.5 (6.9)	11.3 (5.7)
Verify problem in problem list	24.8 (22.1)	12.7 (5.5)	14.4 (3.5)
Add allergy to problem list	50.5 (28.1)	37.5 (20.7)	30.3 (22.2)
Write a prescription for these three medications and send it electronically	281.4 (127.8)	176.8 (58.3)	140.5 (50.8)
Represcribe these eight medications and send electronically	66.3 (61.6)	70.8 (29.0)	51.6 (18.6)
Handwritten prescription comparison
Write a prescription for these three medications by hand	61.9 (15.2)
Represcribe these eight medications	109.2 (31.4)

SD = standard deviation.

Acceptability of the System-baseline Expectations

At the completion of training, 61.5% of physicians strongly agreed that they would use the MOXXI system for most of their patients (▶). Physicians rated the usefulness of the system higher than the perceived ease of use. Expected benefits of the system that achieved the highest mean ratings were improvements in communication, continuity, and quality of care. Physician perceptions that were most strongly correlated with the expectation that they would use the MOXXI system for most of their patients were confidence in using the system and the expectation that the system would improve professional autonomy, satisfaction, and continuity of care (▶).

Table 3.

Baseline of Computer Experience, Expected Use, and Benefit of the MOXXI System for Primary Care Study Physicians

Baseline Assessment Question	Mean Rating (SD) (1 = strongly disagree, 5 = strongly agree)	% Strongly Agree (No.)	Correlation with Expected Use (p-value)
Expected use
Expect to use with most patients	4.46 (0.81)	61.5% (16)	-
Expected benefits/attributes
Will be useful	4.42 (0.64)	50.0% (13)	0.63 (< 0.001)
Will be easy to use	3.96 (0.84)	24.0% (6)	0.42 (0.04)
Will improve professional satisfaction	4.57 (0.57)	61.5% (16)	0.51 (< 0.008)
Will improve professional autonomy	3.96 (0.87)	30.8% (8)	0.62 (< 0.001)
Will improve quality of care	4.20 (0.91)	48.0% (12)	0.32 (0.13)
Will improve communication	4.23 (0.76)	42.3% (11)	0.28 (0.17)
Will improve continuity of care	4.38 (0.69)	46.2% (12)	0.71 (< 0.001)
Will increase patient satisfaction	4.11 (0.82)	34.6% (9)	0.41 (0.04)
Will not increase depersonalization	4.23 (0.86)	46.2% (12)	0.57 (< 0.003)
Computer self-efficacy
Comfortable with computers	4.04 (0.82)	30.8% (8)	0.43 (0.03)
Confident I can use MOXXI	3.81 (0.94)	30.8% (8)	0.52 (< 0.007)
Computer experience	Mean (SD)	Range
Hours per week in using a computer	7.13 (9.14)	0–35	0.30 (0.14)

Results are based on responses received from 26 of the 28 physicians.

Spearman rank-order correlations were estimated between physician self-rating of expected use of the MOXXI system and self-ratings of expected benefit, computer experience, and computer self-efficacy.

SD = standard deviation.

Acceptability of the System Postimplementation Assessment

After using the system for eight months, physicians rated the printed prescription, the current medication list, drug profile, and represcribing function as the most useful aspects of the system (▶). Greater variation in perceived benefit between physicians existed for the drug problem surveillance and alert system (standard deviation of rated benefit >1.0). Drug monographs and drug cost information were considered to be the least beneficial aspects of the system.

Table 4.

Primary Care Study Physicians' Ratings of the Benefit of Components of the MOXXI System Eight Months after Implementation

	Mean (SD) (1 = strongly disagree, 5 = strongly agree)	% Agree or Strongly Agree (No.)
Printed prescription	4.54 (0.59)	95.8% (23)
Current prescription list	4.22 (0.67)	87.0% (20)
Represcribing function from current drug list	4.21 (0.98)	79.2% (19)
List of medication prescribed by me and dispensed	4.13 (0.74)	87.5% (21)
Electronic prescription pad	4.13 (0.74)	79.2% (19)
List of medication prescribed by others	4.04 (1.0)	79.2% (19)
Drug-allergy contraindication screening	3.75 (0.99)	75.0% (18)
Refill compliance calculator	3.50 (0.98)	54.2% (13)
Drug interaction screening	3.42 (1.18)	66.6% (16)
Problem/allergy list	3.33 (1.05)	58.3% (14)
Information on emergency department visits and hospitalizations	3.29 (0.69)	41.7% (10)
Drug-disease contraindication screening	3.26 (1.18)	52.2% (12)
Therapeutic duplication screening	3.17 (1.17)	50.0% (12)
Drug age contraindication screening	3.08 (1.28)	45.8% (11)
Drug monographs	3.04 (1.26)	43.4% (10)
Drug cost information	2.82 (0.85)	18.2% (4)

On the basis of a paired t-test, the mean difference in physicians' ratings in the follow-up questionnaire compared to the baseline questionnaire was significantly lower for intent to use (mean: −0.78, p = 0.006), usefulness (mean: −0.84, p = 0.001), made work easier (mean: −1.61, p < 0.0001), professional satisfaction (mean: −1.74, p < 0.0001), communication (mean: −1.53, p < 0.0001), and patient satisfaction (mean: −1.32, p = 0.0002), but there was no significant change in ratings for perceived benefits for quality of care (mean: −0.72, p = 0.06) and ease of use (mean: −0.61, p = 0.07).

SD = standard deviation.

Use of the MOXXI System

In the first 20 months, the mean biweekly use rate for the electronic prescription was 36.9/100 visits in comparison to 12.6/100 visits for the drug profile. In the initial month, the use rate of the drug profile surpassed the electronic prescribing rate, likely because it provided physicians with the first opportunity to view the complete drug, visit, and compliance profile for their patients (▶A). In the subsequent months, the use rate for the electronic prescribing system was systematically higher than the use rate of the drug profile, likely because patients who were visiting for the first time during the follow-up period declined from 95.3% in the first month to 5.8% in the last. Physicians varied in their use of the MOXXI system (mean: 32.9/100 visits (SD: 20.3); interquartile range: 14.0; 45.0). The physician's rating of the expected frequency of use of the system at baseline was significantly associated with subsequent use rate (▶B). Physicians who strongly agreed with the statement that they would use the MOXXI system for most of their patients had electronic prescribing rates that were twofold higher throughout the implementation period in comparison to physicians whose expectation of use was rated lower (relative rate: 2.26; 95% confidence interval: 1.43–3.46). Prior computer experience was also significantly associated with use (baseline weekly computer use less than four hours per week [mean, 30.1/100 visits] vs. four or more hours per week [mean, 49.1/100 visits] [relative rate: 1.46; 95% confidence interval: 1.07–2.11]). Physician sex, age, and ratings of expected benefits were not associated with use rates.

Figure 2.

Utilization of the MOXXI system in the first 20 months after training. A, Bi-monthly utilization rate of the electronic prescription system and the electronic medical history. B, Bi-monthly utilization rate of the electronic prescription system by expected intention to use for the majority of practice patients prior to implementation.

Use in Higher Risk Patient Populations

During the follow-up period, physicians were significantly more likely to use the drug profile for patients who had more complex drug therapy, lower continuity of care, more emergency department visits, and a greater number of prescribing physicians (▶). In contrast, patient characteristics were not significantly associated with the use of the electronic prescribing system.

Table 5.

Utilization of the MOXXI System by Complexity of Patient Drug Treatment and Fragmentation of Care

Patient Characteristics	No. (%)	Drug Profile Utilization Rate	e-Rx Utilization Rate
Complexity of drug treatment
1 drug/day	628 (15.0%)	5.9	27.0
2–4 drugs/day	1,757 (41.9%)	9.5	33.4
>4 drugs/day	1,811 (43.1%)	12.2	32.1
Relative rate per additional medication [95% CI] (p-value)		1.14 [1.06–1.21] (p = 0.001)	1.01 [0.99–1.10] (p = 0.54)
Fragmentation of care
No. of prescribing MDs
1	838 (20.0%)	10.7	37.3
2–3	1,007 (24.0%)	9.8	34.4
4–5	1,405 (33.5%)	10.2	30.9
>5	946 (22.5%)	11.4	28.2
Relative rate per additional physician [95% CI] (p-value)		1.09 [1.04–1.15] (p = 0.007)	0.99 [0.96–1.02] (p = 0.60)
No. of emergency department visits
0	3,366 (80.2%)	9.9	32.9
1	347 (8.3%)	11.8	29.3
2	245 (5.8%)	11.9	31.1
3–4	110 (2.6%)	13.2	29.3
>4	128 (3.1%)	13.1	26.8
Relative rate per additional emergency department visit [95% CI] (p-value)		1.04 [1.01–1.09] (p = 0.006)	1.0 [0.98–1.03] (p = 0.90)
Continuity of care
>68% visits to study MD	1,039 (24.8%)	8.9	32.6
51%–67% visits to study MD	1,056 (25.2%)	9.5	31.5
34%–50% visits to study MD	1,043 (24.8%)	9.9	30.8
≤33% visits to study MD	1,058 (25.2%)	15.0	34.0
Relative rate per 50% decrease in continuity [95% CI] (p-value)		1.48 [1.07–2.04] (p = 0.02)	1.0 [0.95–1.05] (p = 0.96)

Relative rates were estimated by Poisson regression models within a generalized estimating equation (GEE) framework. Models were estimated separately for each attribute of fragmented care and drug use, adjusted for age and gender in relation to drug profile use, and electronic prescribing (e-Rx) use. The mean age of patients was 68.2 (standard deviation 15.4), and 60.8% were female. For number of drugs used per day, adjusted estimates by age were not possible as age is collinear with drug use; therefore, estimates presented are adjusted for patient gender only.

The estimated working correlation for patients clustered within physicians was r = 0.26 for drug profile use and r = 0.59 for e-Rx use. GEE estimates of the corrected standard error could not be obtained for the model of the e-Rx and number of medications association because of a lack of convergence; thus, uncorrected standard errors and p-values are presented.

e-Rx = electronic prescribing; CI = confidence interval; MD = physician.

Discussion

This study showed that physicians were more likely to use an integrated electronic prescribing and drug management system,^16,33,38 compared to previously reported studies involving stand-alone systems.⁵⁰ Although physicians did not achieve use rates of 60% to 80% that would have been expected if they prescribed electronically in all patient visits, we showed that physicians selectively use such a system to access comprehensive drug histories for patients at greater risk of drug-related adverse events, including those using a greater number of medications, with multiple prescribing physicians, emergency department visits, and poorer continuity of care.^51,52 This may be because electronic prescribing was only faster than handwritten prescriptions for patients with multiple medications. However, greater use of this system for more vulnerable patients also suggests that the integration of drug profile information from disparate sources may increase use by improving safety, completeness, and efficiency in acquiring a complete drug history for more complex patients.³⁵ Broad-scale adoption of electronic prescribing by all physicians and for all patients will likely require additional value-added benefits that would compensate for a less efficient process than the handwritten prescriptions, faster, more adaptive technology, and possibly legislative requirements.

Considerable efforts are required to deploy an integrated system in community-based primary care practices. Overall, we estimate that approximately 2,427 person-days (PD) were required to negotiate (267 PD), develop (1,800 PD), and implement (360 PD) our prototype. We tried to accomplish this integration using two different approaches to health information exchange. The first method was to actively implement a pharmanet that retrieved information from individual community-based pharmacies and make it available to physicians through a centralized data repository. The advantage of this approach was that information on all dispensed prescriptions was captured for all patients regardless of drug insurance coverage. The main limitation was the significant cost required to establish an independent infrastructure to link each pharmacy in order to retrieve a complete medication list. As patients often used multiple pharmacies,²³ these requirements were critical to avoid incomplete information and were undermined by the lack of priority for this endeavor from pharmacy management. The second approach focused on extracting data from the existing prescription claims adjudication process. In our context, this was the provincial health insurance program, but in other settings, this would be the private insurance adjudicators. For a comparatively lower cost, the existing claims adjudication infrastructure could be used to provide rapid access to drug information for all patients insured by the respective carrier. When a large proportion of the population is covered by a relatively small number of carriers, this approach could be more readily adopted and implemented to provide the infrastructure for integrated drug systems than a pharmanet model. The primary limitation of existing claims adjudication systems is that they only capture drugs insured by the respective formularies of individual carriers and thus will always be, to differing extents, incomplete. However, these systems could be readily used to capture all drug information if mechanisms were instituted to permit patient authorization of complete capture of drug information.

Handheld computing was barely adequate for an application of this level of complexity. The hardware was a major constraint because of limited screen size, memory, and battery life. Even though the portability and intuitive interface were attractive to physicians, the slow performance weighed heavily against more frequent use. The future challenge is to find the right balance of adequate performance in a form that is portable and nonintrusive to physician-patient interactions. Ultra-portable laptops or tablet PCs may represent an ideal solution.

Our study is limited to physicians who are early adopters of new technology. The perceived benefits of improved continuity of care and professional autonomy that were associated with the use of the technology by this group may not be applicable to other physicians. Considerable variation in use rates existed between physicians that appeared to be related to physician perception of expected benefit and experience in computer use. However, small sample size limited the ability to assess physician-level determinants of use. Future research should assess physician characteristics that influence use, the generalizability of our findings to other settings, and the possible reduction in adverse drug events that may be achieved by using integrated drug information management systems for higher risk, complex patients. Future development should focus on the integration of community-based and in-hospital pharmacy information systems⁵³ and the incorporation of disease management decision support into prescribing systems.

MOXXI Technical Appendix 1

The MOXXI e-Rx application is a custom-built application initially developed using Microsoft Visual-Basic 6.0 then re-developed under Microsoft C++ 8.0 to enhance performance (▶). The prototype was designed to enhance the safety, efficiency, and effectiveness of drug management based on research and development conducted by the MOXXI research team over the past ten years. The development team included a physician who worked in the primary care setting and followed a rapid application development process. The e-Rx application resides on a handheld device used by the participating physicians to generate electronic prescriptions transactions.

The MOXXI e-Rx client was initially deployed on HP-Compaq iPaqs (model 3870) with an Intel StrongARM SA-1110 206 MHz CPU and 64 Mb RAM initially running the Microsoft Windows for the Pocket PC 2002 operating system. These devices were later upgraded to HP-Compaq iPaqs (model 5550) with an Intel XScale 400-MHz CPU and 256 MB RAM running Microsoft Windows for Pocket PC 2003. The handheld computers were equipped with Sierra Wireless Aircard Model 555 CDMA2000 network cards. Office printing of prescriptions was via Bluetooth communications to an HP model 1250 laser jet printer. Wireless Internet connectivity was via the CDMA2000 digital spread-spectrum service provided by Bell Mobility 1X. Security was established by: embedded Oracle 9i encryption at the database level, implementation of a user personal identification number (PIN), iPaq device ESN, Aircard MAC Address, and Bell virtual private network (VPN) services to the central MOXXI server farm. Each iPaq is authorized for use within the system to a single physician. Since wireless network connectivity could not be guaranteed, the iPaq maintained a local encrypted database (Oracle 9i lite) of that physician's patients so that transactions can be generated and queued until a connection to the central server is reestablished. The iPaq local database synchronizes with the server automatically whenever the PDA is turned on, and a warning is displayed to the physician in the event of loss of wireless network connectivity or failure of the database synchronization process.

The MOXXI e-Rx application server component was implemented on a SunFire 280R (2 × 900 MHz SPARC III CPUs, 6 GB RAM, dual 36-GB hard drives) server running Solaris 8. This server provides access to data from pharmacy systems, the RAMQ system, and the drug knowledge base server (see ▶), and routes data to the client application. Physicians are able to view information about dispensed prescriptions from participating pharmacies, the provincial RAMQ pharmacy databases, and disease and medical visit information derived from the RAMQ medical services claims database.

The database server (Oracle 9i) was implemented on a second SunFire 280R server with 2 GB RAM and a 240-GB RAID5 disk storage array. All MOXXI transactions are transmitted securely to this central database server. Transactions include new consenting patients, electronic prescriptions, prescription stop orders, requests for drug profile review, and health problem/allergy confirmations. This server is the central repository for data exchange for all data generated by the physicians, pharmacies, and RAMQ for participating patients. The database server communicates with the RAMQ provincial health administrative databases via HTTPS protocol and downloads information on dispensed prescriptions, emergency department visits, and hospitalizations for consenting patients. Transactions between the MOXXI e-Rx application and the retail pharmacies are also managed on this server. The participating retail pharmacies modified their pharmacy management software to upload electronic prescriptions transactions from the database server and to transmit information on dispensed prescriptions for consenting patients. VPN connections with each pharmacy through a 56-kbps dial-up modem or via a pharmacy chain central server were used for this secure data exchange. To comply with provincial legislation forbidding the direct transmission of a prescription to any particular retail pharmacy, electronic prescription transactions were deposited in the database server. Only when the patient presented the printed prescription bearing the unique MOXXI prescription number would the electronic prescription be able to be “pulled” from the database server, processed, and dispensed. Data exchange with pharmacies used the CPhA-3 (Canadian Pharmacy Association version 3) format already in operation for pharmacy billing and transaction adjudication. In keeping with provincial legislation, patients must provide a written consent for a physician to have access to their complete drug profile. To facilitate immediate access to complete drug information, an electronic notification of the presence of a written consent was considered legislatively acceptable because the physician was accepted as a cyber notary.

The Drug Knowledgebase was implemented on an IBM server (Xseries model 232, Intel Xeon 1.26-GHz CPU with 1 GB RAM, dual 36-GB hard drives) running a Microsoft Windows 2000 server. This server processes requests from the MOXXI application server. It analyzes the active dispensed medication set and any newly generated prescriptions and patient profiles and issues real-time alerts to the prescriber if a potential problem is identified. Data inputs include all electronic and dispensed prescriptions, allergies, diseases, and patient drug insurance coverage. The automated review produces output sent to the e-Rx handheld client as an alert containing information on the problem(s) identified, the specific drug(s)/diseases implicated, possible corrective measures, and severity classification. The Drug Knowledge Module is activated each time a prescription is sent or at the request of the physician.

Figure 3.