Higher accuracy of complex medication reconciliation through improved design of electronic tools

Abstract

Objective

Investigate the accuracy of 2 different medication reconciliation tools integrated into electronic health record systems (EHRs) using a cognitively demanding scenario and complex medication history.

Materials and Methods

Seventeen physicians reconciled medication lists for a polypharmacy patient using 2 EHRs in a simulation study. The lists contained 3 types of discrepancy and were transmitted between the systems via a Continuity of Care Document. Participants updated each EHR and their interactions were recorded and analyzed for the number and type of errors.

Results

Participants made 748 drug comparisons that resulted in 53 errors (93% accuracy): 12 using EHR2 (3% rate, 0–3 range) and 41 using EHR1 (11% rate, 0–9 range; P < .0001). Twelve clinicians made completely accurate reconciliations with EHR2 (71%) and 6 with EHR1 (35%). Most errors (28, 53%) occurred in medication entries containing discrepancies: 4 in EHR2 and 24 in EHR1 (P = .008). The order in which participants used the EHRs to complete the task did not affect the results.

Discussion

Significantly fewer errors were made with EHR2, which presented lists in a side-by-side view, automatically grouped medications by therapeutic class and more effectively identified duplicates. Participants favored this design and indicated that they routinely used several workarounds in EHR1.

Conclusion

Accurate assessment of the safety and effectiveness of electronic reconciliation tools requires rigorous testing and should prioritize complex rather than simpler tasks that are currently used for EHR certification and product demonstration. Higher accuracy of reconciliation is likely when tools are designed to better support cognitively demanding tasks.

INTRODUCTION

Research studies done over the last 2 decades provide strong evidence that medication injuries to patients in hospitals and in the community are common.¹ Many of these adverse drug events (ADEs) are directly related to inaccurate or missing medication histories that are often needed for clinicians to make informed treatment decisions.² An important strategy to reduce the incidence of ADEs is a regular review and reconciliation of all the medications a patient is taking to obtain the true current record. The process, however, could be a daunting task for polypharmacy patients with complex medication histories, as clinicians need to aggregate accurate information from multiple sources, including the patient or a caregiver, primary care and specialist physicians, outpatient medical records, records from remote systems, hospital discharge summaries and community pharmacies, and complete it in the limited time available during office visits.³ Patients treated with multiple medications also tend to be older,⁴ require more care, often have prescriptions from many providers⁵ and are at higher risk of ADEs.⁶ Acquiring and reconciling complex medication histories is a laborious process with many opportunities for error.⁷ Recognizing both the importance and difficulty of this process, the Joint Commission declared medication reconciliation a National Patient Safety Goal in 2004.⁸

Electronic medication reconciliation (eMedRec) tools are vital components of a technology-based strategy to prevent errors.⁹ Hospitals have led the effort to make reconciliation more accurate and less difficult at key care transition points such as admission and discharge. Keeping medication records up to date and error-free is just as important in ambulatory and long-term care.¹⁰ Primary and specialty care providers often need to review the list in their electronic health record (EHR) system to determine whether any medication the patient was taking prior to the visit has been discontinued or altered or any new medications have been prescribed. Many current eMedRec tools allow clinicians to compare the best available medication history with orders or prescriptions, identify discrepancies by displaying medication lists from EHRs and Continuity of Care Documents (CCDs), and support corrective actions such as changing, discontinuing, or adding medications to the permanent reconciled record.

A recent systematic review of the effect eMedRec has had on reducing discrepancies and associated ADEs at hospital transitions concluded that while discrepancies were significantly reduced, clear evidence of measurable safety gains is not yet apparent, although high-quality studies with rigorous design were lacking.¹¹ There are numerous reports that do show a net benefit of reconciliation but those were almost exclusively based on interventions that rely heavily on pharmacy staff resources.¹² Other studies, such as the Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS), showed that a new vendor EHR more than doubled the rate of medication discrepancies, an effect likely attributable to a combination of factors including design, local implementation and variations of use by clinicians.^13–15 Electronic reconciliation is a powerful tool requiring high-quality design and careful integration into clinical workflows in order to perform effectively and avoid unintended effects that may increase the risk of ADEs.^16,17

Academic, trade, and regulatory bodies recommend applying human factors approaches to health information technology (HIT) design as a pathway to safer and better-quality care.^18–21 The Office of the National Coordinator for Health Information Technology (ONC) requires all developers to follow a user-centered design (UCD) process²² to meet certification criteria for the safety-enhanced design component of stage 2 of the meaningful use EHR incentive program.²³ The National Institute of Standards and Technology formulated the UCD process as a collection of design methods that emphasize insight into cognitive tasks and decision-making, iterative refinement, and frequent testing of prototypes with clinicians starting at the earliest stages of development.^24,25

Despite these efforts, UCD criteria are not consistently met.^26,27 Many eMedRec tools have been found to lack recommended features and characteristics, such as highlighting medication discrepancies, allowing the possibility to organize, sort, and filter medications as needed or displaying lists side-by-side to minimize cognitive effort during visual comparison.²⁸ For example, clinicians limited to viewing one screen of relevant information at a time (eg, by having to scroll to see off-screen content) have to store in their working memory any information of interest displayed on screens no longer in view and recall it when needed.²⁹ This display fragmentation³⁰ reduces efficiency³¹ and increases cognitive load which could lead to more frequent medical errors.^32,33 There is scant data available to describe the current design practices of HIT vendors and the variability in the development process that may partially account for the poor usability of some EHR products.²⁷

The quality of eMedRec tools has recently been studied with cognitive design and usability methods.³⁴ Studies with standardized scenarios and simulations indicated that there were substantial differences in time and accuracy in reconciling medications.³⁵ For example, one study showed that a key design aspect of a tool that reduced discrepancies at admission by more than half was its ability to display preadmission medications on the left side of a split screen and allow clinicians to manipulate the list on the right to generate admission orders and review their decisions about potentially problematic additions or changes.³⁶

The objective of our study was to investigate the effect of different design concepts of a reconciliation module on human performance and error during a cognitively complex task. We asked clinicians to reconcile identical, standardized sets of medication lists with 2 different tools integrated in EHRs and analyzed the rate and character of observed errors. EHR1 was a recently implemented vendor system and EHR2 was developed internally.³⁷

The effects of interface design on human performance measured in error rates and time are often evaluated in crossover studies, where the same group of participants completes standardized clinical tasks with 2 or more tested systems.^38–41 This is the only study, to our knowledge, that applied these techniques to eMedRec to describe the process and the potential for error. Our findings may inform the design and testing of interventions and electronic tools that effectively alleviate the cognitive burden of clinicians during complex reconciliations and reduce the number of errors leading to adverse events.

MATERIALS AND METHODS

We simulated the reconciliation of a complex medication record in an outpatient setting in a standardized-task study with repeated measures. A list of medications for a hypothetical patient whose records were stored in 2 different ambulatory EHRs was transmitted between the systems via an electronic CCD. Participants were asked to reconcile the received CCD with the EHR record (Table 1). A test administrator in the role of patient answered questions about the active use of each medication or its dose when asked.

Table 1.

Medications entered in EHR records (*denotes discrepancies)

Medications recorded in EHR systems
Both EHRs	EHR1 Only	EHR2 Only
Abacavir (ziagen) 300 mg tablet	*Lisinopril 40 mg tablet	*Zestril (lisinopril) 20 mg tablet
by mouth two times a day	by mouth daily	by mouth daily
take with or without food
Adefovir (hepsera) 10 mg tablet	*Clonidine (catapres) 0.1 mg tablet	*Clonidine (catapres) 0.1 mg patch
by mouth every other day	by mouth two times a day	transdermal once a week
Klonopin (clonazepam) 0.5 mg tablet	*Crestor (rosuvastatin) 40 mg tablet	*Lipitor (atorvastatin) 40 mg tablet
by mouth two times a day	by mouth daily	by mouth daily
as needed for anxiety attacks
Lamivudine (epivir) 150 mg tablet	Aspirin 81 mg chewable tablet	Humulin (insulin 70/30 human) 70-30/mL vial
by mouth daily	by mouth daily	10 mL subcutaneously every morning before breakfast
Lasix (furosemide) 20 mg tablet	Ambien (zolpidem) 5 mg tablet	Amlodipine 10 mg tablet
by mouth every other day	by mouth nightly	by mouth daily
	as needed for sleep
MiraLax (polyethylene glycol) 17 g packet	Zoloft (sertraline) 100 mg tablet	Tramadol 50 mg tablet
by mouth daily	by mouth daily	by mouth two times a day
as needed for constipation		as needed for pain
Novolog (insulin aspart 70/30) 100 unit/ml injection vial
inject 4 Units under the skin three times a day before meals
Prilosec (omeprazole) 40 mg tablet
by mouth daily
Raltegravir (isentress) 400 mg tablet
by mouth two times a day
Zofran (ondansetron) 4 mg tablet
by mouth three times a day

There were 16 medications in each record: 10 were entered identically in both systems, including dose and frequency, and 6 were unique to each. Reconciling the EHR record required 22 comparisons: the 10 identical medications had to be verified and confirmed as taken by the patient; 6 medications in the EHR but not on the CCD list were either confirmed or discontinued, and 6 that were only on the CCD would either be added to the record or disregarded if not confirmed. Our intent was to simulate a realistic ambulatory scenario where there was a partial overlap of records and several discrepant entries. The reconciliation task and required actions are shown in Figure 1 for EHR1 and in Figure 2 for EHR2. Differences in typography (capitalization, bold type, multiple line entry) and in sig abbreviations are represented in the figures exactly as they appeared on the screen. Shaded parts of each figure contain medication entries that are identical in the EHR record and on the CCD list.

Figure 1.

Reconciliation of medication records with a CCD list for EHR1.

Figure 2.

Reconciliation of medication records with a CCD list for EHR2.

The study task was developed from an actual medication record of a patient with multiple morbidities treated with polypharmacy. We edited medication records in the tested systems to create a balanced task with an equal number of actions required to reconcile the record in each EHR: verify 16 medications, add 4, and discontinue 2.

The lists contained 3 discrepant medications (marked by asterisks in Table 1) requiring the following changes: (1) The dose of lisinopril was increased from 20 to 40 mg and the prescription changed from brand name (Zestril) to generic; (2) the route of clonidine was changed from an oral pill to a transdermal patch; (3) Lipitor was substituted for Crestor (same therapeutic class); and (4) tramadol was discontinued. Participants were not asked to make adjustments to therapy or to comment on its appropriateness. They could ask the “patient” which medications he was actively taking and to confirm the dose, route, and frequency.

Participants using EHR1 had to verify the 10 medications that were common to the EHR and the CCD, had to add amlodipine and Humulin, had to replace clonidine oral with clonidine patch and had to replace Crestor with Lipitor. In EHR2, participants had to verify the 10 medications in the EHR and the CCD, had to add Ambien, aspirin, and Zoloft, had to replace Zestril with lisinopril, and had to discontinue tramadol. In both cases, the reconciled record would have a total of 18 medications. Each participant used both systems to reconcile the scenario (within-subject, repeated measures design) in an alternating, counterbalanced order, and completed an interleaving task to minimize possible learning effect. The instructions to the participants were to reconcile the local medication record with a CCD received from the patient’s prior primary care provider.

The tested modules were specifically designed to support medication reconciliation by formatting and preparing CCD lists for visual comparison with the medication record. EHR1 was a vendor-developed system that was recently implemented to replace a legacy, institution-developed EHR2. The design of the modules was substantially different: in EHR1, the CCD list was placed above the EHR record in a single-column layout and sorted alphabetically (Figure 3); EHR2 (Figure 4) had both lists arranged side-by-side and matched by therapeutic class. Due to the high number of medications, the CCD was only partially displayed in EHR1 and the system record below was entirely off the screen, necessitating scrolling to compare individual entries. EHR2 had both lists directly visible for visual inspection (see Horsky and Ramelson⁴² for a detailed description of the design).

Figure 3.

Representation of screen layout for EHR1.

Figure 4.

Representation of screen layout for EHR2.

Twenty-two ambulatory care physicians and one physician assistant responded to e-mail solicitation and agreed to participate. Twelve worked with EHR1 for 6 months or longer and 10 between 2 and 6 months after completing standard training. The clinicians reconciled medications daily or at least several times per week in practice, and some had medical assistants to help. However, they had no previous training or experience using the EHR2 module as it was released one year prior to the institution’s transition to the vendor system, and, in that time, very few CCD documents were exchanged. They received a short (1–2 min) demonstration prior to the study. Participants were instructed to verbalize their thoughts during the reconciliation in a standard think-aloud protocol. Time to task completion was therefore not analyzed as an outcome measure due to pauses and interruptions for comments.^43,44 The screens were video- and audio-recorded with Morae 3.3 suite, and interactions of participants with the modules were later analyzed.⁴⁵

Analyses

We quantified the number and type of errors in the final reconciled EHR record and tested group means for statistical significance. The order of EHRs used for the task (EHR1 then EHR2, and vice versa) and observed strategy (pairwise or list-wise medication comparison) were also noted. Analyses were done to evaluate 3 effects: (1) the error rate for all medications (22 × number of participants × 2 EHRs) by McNemar’s test for proportions of related (within-subject) results and a dichotomous outcome (correct-incorrect decision); (2) the error rate per participant and task, calculated as a proportion of incorrect decisions out of 22, and the difference of means for EHR type compared using a paired t-test (within-subject effect); and (3) a mixed-model analysis of variance for repeated measures to assess the effect of 2 within-subject factors (EHR type, reconciliation strategy) and one between-subject factor (order of EHR in task completion). All analyses were done with SAS v. 9.4.⁴⁶

RESULTS

Of the 23 recruited clinicians, 17 completed the study task using both EHRs. Five used only one due to unplanned time and work constraints or because of technical difficulties (network connection problems). Their results were excluded from the analysis to maintain the within-subject design. There was no systematic cause of noncompletion and group analyses showed similar effect size and statistical significance with and without the excluded data. Eleven participants were female and 12 were primary care physicians. All clinicians practiced for 10 years or more.

Participants made a total of 748 drug comparisons (22 medications × 17 participants × 2 EHRs) that resulted in 53 errors (93% accuracy): 12 errors using EHR2 (3% rate, 0.7 average, 0–3 range) and 41 errors using EHR1 (11% rate, 2.4 average, 0–9 range). Results are shown in Table 2.

Table 2.

Number of reconciliation decision errors and mean error rate per clinician

EHR	Correct, N (%)	Error, N (%)	Error µ (StD)	Range	Odds Ratio (95% CI)	McNemar's M
EHR2	362 (97)	12 (3)	0.7 (1.2)	3 (0–3)
EHR1	333 (89)	41 (11)	2.4 (2.6)	9 (0–9)	3.7 (1.9–7.2)	P < .0001

Twelve clinicians made completely accurate reconciliations with EHR2 (71%) and 6 with EHR1 (35%). Four made multiple errors working with EHR2 (24%) and 11 working with EHR1 (65%). The number of errors per participant is plotted in Figure 5. Five participants made accurate reconciliations using both EHRs (29%). Seven, the largest proportion, made no errors using EHR2 and one or more errors using EHR1 (41%). The distribution is shown in Figure 6.

Figure 5.

Number of errors per participant.

Figure 6.

Participants with correct and erroneous reconciliations.

Table 3 lists all the reconciliation actions required for participants using each EHR and where the errors occurred. Most errors (28, 53% of total) were made when reconciling medications containing discrepancies: 4 occurred in EHR2 and 24 in EHR1, a significant difference (P = .008). Medications already entered in the EHR record had the smallest number of errors (5, 9% of total, all in EHR1), and those that were entered identically in EHR and CCD lists had 7 errors (13% of total): 1 in EHR2 and 6 in EHR1, a difference approaching significance (P = .0557). Participants erred about equally often with each system for medications listed only on the CCD documents (13 errors, 25% of total, 6 in EHR1 and 7 in EHR2). Analysis of reconciliation actions showed that errors of omission in adding a medication to the EHR were more frequent (23, 43%) than failing to discontinue one already in the record (14, 26%) or to verify one (16, 30%). Failure to discontinue or verify medications occurred mostly when participants used EHR1: 12 vs 2 in EHR2 for discontinue (significant difference; P = .0132) and 15 vs 1 in EHR2 for verification (significant difference; P = .022).

Table 3.

Reconciliation actions and errors by EHR used

EHR1			EHR2
Medications	Action	Errors	Medications	Action	Errors	Pr > Paired |t|
Discrepant
Clonidine (catapres) 0.1 mg tablet by mouth two times a day	D/C	6	Clonidine (catapres) 0.1 mg tablet by mouth two times a day	Verify	0
Clonidine (catapres) 0.1 mg patch transdermal once a week	Add	6	Clonidine (catapres) 0.1 mg patch transdermal once a week	Verify	0
Crestor (rosuvastatin) 40 mg tablet by mouth daily	D/C	6	Crestor (rosuvastatin) 40 mg tablet by mouth daily	Verify	0
Lipitor (atorvastatin) 40 mg tablet by mouth daily	Add	2	Lipitor (atorvastatin) 40 mg tablet by mouth daily	Verify	0
Zestril (lisinopril) 20 mg tablet by mouth daily	Verify	2	Zestril (lisinopril) 20 mg tablet by mouth daily	D/C	2
Lisinopril 40 mg tablet by mouth daily	Verify	2	Lisinopril 40 mg tablet by mouth daily	Add	2
Subtotal, N		24			4
Mean per clinician		1.41			0.24	P = .008
In EHR records
Aspirin 81 mg chewable tablet by mouth daily	Verify	3	Amlodipine 10 mg tablet by mouth daily	Verify	0
Ambien (zolpidem) 5 mg tablet by mouth nightly as needed for sleep	Verify	1	Humulin (insulin 70/30 human) 70-30/ml vial 10 ml subcutaneously every morning before breakfast	Verify	0
Zoloft (sertraline) 100 mg tablet by mouth daily	Verify	1	Tramadol 50 mg tablet by mouth two times a day as needed for pain	D/C	0
Subtotal, N		5			0
Mean per clinician		0.29			0	P = .136
On CCD documents
Amlodipine 10 mg tablet by mouth daily	Add	3	Aspirin 81 mg chewable tablet by mouth daily	Add	3
Humulin (insulin 70/30 human) 70-30/ml vial 10 mL subcutaneously every morning before breakfast	Add	3	Ambien (zolpidem) 5 mg tablet by mouth nightly as needed for sleep	Add	2
Tramadol 50 mg tablet by mouth two times a day as needed for pain	Verify	0	Zoloft (sertraline) 100 mg tablet by mouth daily	Add	2
Subtotal, N		6			7
Mean per clinician		0.35			0.41	P = 0.805
On both EHR and CCD lists
Abacavir (ziagen) 300 mg tablet by mouth two times a day take with or without food	Verify	0	Abacavir (ziagen) 300 mg tablet by mouth two times a day take with or without food	Verify	0
Adefovir (hepsera) 10 mg tablet by mouth every other day	Verify	0	Adefovir (hepsera) 10 mg tablet by mouth every other day	Verify	0
Klonopin (clonazepam) 0.5 mg tablet by mouth two times a day as needed for anxiety attacks	Verify	0	Klonopin (clonazepam) 0.5 mg tablet by mouth two times a day as needed for anxiety attacks	Verify	1
Lamivudine (epivir) 150 mg tablet by mouth daily	Verify	1	Lamivudine (epivir) 150 mg tablet by mouth daily	Verify	0
Lasix (furosemide) 20 mg tablet by mouth every other day	Verify	0	Lasix (furosemide) 20 mg tablet by mouth every other day	Verify	0
Miralax (polyethylene glycol) 17 g packet by mouth daily as needed for constipation	Verify	2	Miralax (polyethylene glycol) 17 g packet by mouth daily as needed for constipation	Verify	0
Novolog (insulin aspart 70/30) 100 unit/ml injection vial inject 4 Units under the skin three times a day before meals	Verify	2	Novolog (insulin aspart 70/30) 100 unit/ml injection vial inject 4 Units under the skin three times a day before meals	Verify	0
Prilosec (omeprazole) 40 mg tablet by mouth daily	Verify	1	Prilosec (omeprazole) 40 mg tablet by mouth daily	Verify	0
Raltegravir (isentress) 400 mg tablet by mouth two times a day	Verify	0	Raltegravir (isentress) 400 mg tablet by mouth two times a day	Verify	0
Zofran (ondansetron) 4 mg tablet by mouth three times a day	Verify	0	Zofran (ondansetron) 4 mg tablet by mouth three times a day	Verify	0
Subtotal, N		6			1
Mean per clinician		0.35			0.06	P = .056
Total		41			12
Mean per clinician		2.41			0.71	P = .008
Reconciliation action
Add to EHR	Add	14			9	P = .415
Discontinue in EHR	D/C	12			2	P = .013
Verify with patient	Verify	15			1	P = .022

Participants used 2 principal strategies to reconcile medications: a pairwise comparison by verifying one drug on the CCD list at a time with the EHR record (or in the opposite direction) and with the “patient,” making changes as needed, or a listwise comparison by verifying all medications on one list first and then repeating the process with the other list. Pairwise strategy was preferred to listwise by a wide margin on EHR2 (13–4) and to a lesser degree on EHR1 (10–7). We used a mixed-model approach to estimate the effects of 2 within-subject factors (EHR type, comparison strategy) and one between-subject factor (reconciling EHR2 or EHR1 first) in a repeated-measures analysis of variance. In this model, the EHR type had a significant effect on error rate (P = .0289), while the comparison strategy (pairwise vs listwise) had no effect (P = .7598). Task completion order (9 started with EHR1 and 8 with EHR2) also had no effect on the error rate (P = .8373).

DISCUSSION

We asked a group of highly skilled clinicians, most of whom have been caring for patients for a decade or more, to do a complex medication reconciliation with 2 different electronic tools specifically designed for that purpose. The reconciled records were significantly more accurate when clinicians used EHR2, even as they followed the same standardized scenario for both systems. Remarkably, we observed that they made 75% fewer errors (12 vs 41) on the institution-developed EHR2 with no prior training or experience with its function except for a 1–2 min demonstration.

This finding suggests that the better outcome was strongly facilitated by a design that was more effective in supporting a cognitively demanding task consisting of a detailed comparison of clinical information between 2 electronic sources that included interviewing a “patient” about current medication use. Improved human performance is often reported in studies where systems provide adequate cognitive support to clinicians. For example, a study that compared original and redesigned drug-drug interaction alerts showed that a new interface was more efficient in resolving alerts; participants made fewer errors and reported greater satisfaction with the CDS interventions.³⁹

The distribution of accurate results and errors within the group of participants was consistent with expected positive effects of intuitive design that leads to quick learning of tasks.⁴⁷ Twice as many clinicians produced completely accurate reconciliations using EHR2 (12, 70%) than EHR1 (6, 35%), suggesting that most participants had working proficiency after minimal training. Almost all errors in EHR2 (11 out of 12) were made by a minority of participants (5, 29%), but errors in EHR1 were more widely distributed: the majority of participants made errors (11, 65%), with 9 (53%) making between 2 and 5 errors (Figure 5). The effect of intuitive learning is also apparent in the observation that 7 participants (41%, the largest proportion in Figure 6) accurately reconciled using EHR2 but made errors using EHR1. In contrast, only one clinician made an error in EHR2, while being accurate in EHR1 (Figure 6).

The lists contained 3 kinds of discrepancy: the presence or absence of a medication, multiples in the same therapeutic class, and differences in medication details such as dose and route. We found that errors overall clustered within the 6 discrepant entries (28, 53%) rather than in the 10 identical ones, indicating that subtle differences were more difficult to notice and that the tool lacked sufficient cognitive support, especially in EHR1. The 6-fold difference in the number of errors in this subset (24 vs 4 medications, Table 3) suggests that the accuracy of reconciliation could be increased significantly by improving design. For example, when lisinopril 40 mg was added to EHR1, Zestril 20 mg had to be discontinued (and the reverse in EHR2), but clinicians failed to discontinue the inactive entry at a much lower rate in EHR2. One possible explanation is that in EHR1, the presence of an automatically identified duplicate (eg, Zestril 20) was indicated by placing a checkbox at the bottom of a densely populated dialog box. This was easily missed by participants. EHR2 employed a more effective intervention for the same purpose by triggering an alert within the tool workflow, reducing such errors to a minimum.

An error that was prevalent in both systems was the failure to add a medication from the CCD list to the EHR record (see Table 3). Errors of omission have been reported in other reconciliation studies as more frequent than other types of error, such as wrong dosage or route, although no single reason was clearly identified.^11,48,49 We can hypothesize that omissions observed in this study would be responsive to design changes but further studies would be needed to propose specific interventions.

Both systems represented medications with notable signature (sig) and modality variations between local (EHR) and CCD lists. For example, EHR2 (Figure 2) placed brand name first and generic name in parentheses in the record, eg, Lasix (furosemide), but in the reverse order on the CCD, eg, furosemide (Lasix), and used an abbreviation in the first case (PO QOD) and a verbal description in the latter (by mouth every other day). Both EHRs also used frequency abbreviations (QD, QOD) that are discouraged by the Joint Commission.⁵⁰ The effect of these differences on cognitive effort when comparing whether 2 entries are identical is unknown but practical guidelines for readability and comprehension of text on screen often recommend using similar or identical typography and terms to visually emphasize similarity and association.

The single-column layout of the tool in EHR1 required frequent scrolling and memory recall to compare medications. Most usability experts and industry guidelines for safe and efficient HIT design strongly discourage this approach in favor of memory recognition that can be more effectively used on information aggregated on one screen, a core principle of optimal cognitive support.^{22,33,47,51–54} The side-by-side (or split-screen) view of the lists in EHR2 was further enhanced by automatic grouping and matching of medications by therapeutic class, which effectively transformed a large, complex task into a series of smaller, less difficult ones. Although EHR1 also allowed grouping and sorting of medications, it could be applied only to the CCD list, not to the medication record, and therefore was less useful. Excessive scrolling impedes flexibility, efficiency, and the use of recognition and is frequently identified as a usability barrier.²⁹

Opinion of clinicians on design and function

The study format allowed participants to verbalize their thoughts and provide commentary during the task. We also asked them to share their concerns and opinions in a short debriefing and interview period after they completed working with each system.⁵⁵ Their comments were transcribed and organized according to recurrent themes, shown in Box 1. The list is not exhaustive and some quotes could be attributed to several participants but it reviews the most salient points that emerged from reviewing all verbalizations and narratives that we recorded.

Box 1.

Comments by participants

Regarding EHR1

Layout and visual characteristics

• Difficult and dense on the eyes.

• Visually it does not work for me at all, everything is spread out, I cannot compare drugs.

• It would be nice to have a marker where I was [reviewing an alphabetical medication list] in case the phone rings and I get interrupted or distracted.

Side-by-side vs one-column list arrangement

• It would be great to have both on one screen, side by side; I don’t like scrolling

• This is a nightmare; you can’t see both lists.

• The lists should be next to each other so I can see them both and compare, this going up and down, at this point I realize I have no time to do the examination.

• I don’t want to go back and forth, so I will just ask you what you are on and then we’ll go over the other list.

Two-stage process [workaround] and potential for error

• I just add them all and sort it out later in the EHR medication record.

• I would go to the medication record and simply add what the patient is telling me so I don't have to go through this hassle [reconciliation tool]. I hate going through the two lists, so I add everything and then go into the medication section and weed out the duplicates and add those that need to be added.

• Now I would go over the final list again with the patient, because there is so much room for error in this process, and change the dose and details in the medication record section.

• You have to go here over both lists individually, because in the local record there may be drugs that are not on the incoming list, like Ambien and aspirin here.

• So far I have spent 20 [minutes] of my 25-minute visit doing this.

Medical assistants and potential for error

• The entire CCD is merged in by a medical assistant, and then I go through it in the medication record, weed out duplicates, and confirm with the patient whether they are taking them or not.

• My medical assistant does the reconciliation, and then I have to review it because sometimes it is wrong.

• Medical assistant brings medications over [from the CCD to the EHR] and reviews them with the patient, but I will need to review them again; some people don't [clinicians].

• I don’t know how the medical assistants doing this [reconciliation] are expected to pick up on the subtle differences. I think the people doing med rec now are the least likely to do it accurately.

• Only physicians and pharmacists will know these are the same thing [Crestor and Lipitor, same-class statins].

• Sometimes the medical assistant goes too fast and the patient just says “yes, yes, yes” and kind of yeses them to death when I know we have made changes; or they don’t admit to the MA taking anxiety meds or Viagra and the like, and I have to go over it again.

• The medical assistants are often college kids and they are very smart and look up the drug names and spelling on their phones or google them. I don’t think there is a “sounds-like” lookup, and if you don’t know the name correctly you are out of luck.

• This is where mistakes are made [reconciliation tool]. All these meds coming over to [EHR1]; we have seen a bunch of mistakes. Oftentimes my nurse would go to refill it and I look at the old system and there’s a mistake. I have to go back to [EHR2] and say no, it’s supposed to be the long-acting med and so forth.

Regarding EHR2

Side-by-side vs one-column list arrangement

• This screen is much easier because it is side-by-side.

• I did not see the clonidine patch and pill difference on the other one [EHR1] but picked it up here.

• Significant improvement over the other one, and I feel the accuracy of this list is probably much better.

Ordering medications by therapeutic class

• Clonidine tablet in its own category?

Participants pointed out that eyestrain due to the dense presentation of information in EHR1 negatively affected their ability to quickly glean important differences in the lists, and that, somewhat ironically, relevant data were often too widely separated on the screen to compare effectively in context. They almost universally considered the single-column layout and scrolling a severe hindrance. Many also described how their own workflow would differ in practice, partially motivated as a workaround to avoid using the tool. It was usually a 2-stage process in which they imported the entire CCD list in the tool and then reconciled on the medication record screen. About a third of the clinicians had medical assistants do the first part of the process before seeing the patient but were also aware of the risk posed by the limits of their training. For example, studies showed that pharmacy technicians were significantly more accurate than nurses in reconciling medication histories in an emergency department⁵⁶ and that medical assistants with little pharmacology training often obtained medication histories in the outpatient setting.¹⁶ The clinicians would have to review the list again later with the patient, duplicating effort and extending completion time.

A support intervention in EHR1 provided visual cues of correspondence between medications on the lists. When the cursor hovered over an entry in the CCD, a green highlight would identify a matching EHR record. However, while brand and generic names were recognized as being the same drug, different doses were still marked as a “match.” Only a few participants knew about this decision aid but were unaware that doses were not checked and failed to update the record. The risk of error was compounded in this case by the inability to see both highlighted items at the same time, although on shorter lists they would be in view.

The results of this study have important implications for testing EHRs for patient safety considerations. The ONC Stage 2 test dataset for the medication reconciliation requirement of the Clinical Information Reconciliation measure contains 3 medications in each medication list.⁵⁷ This relatively simple scenario would not identify important safety considerations as demonstrated by our study. In order to properly assess potential error rates with medication reconciliation functionality in EHRs as applied to polypharmacy patients, complex medication scenarios must be utilized.

LIMITATIONS

We observed a group of physicians from one health care system complete a simulated task. This study format allowed us to compare human performance without potential environmental bias, but it also constrained the interpretation of findings that need to be contextualized to work conditions in ambulatory practice. For example, we could not observe cognitive errors that are known to arise as the result of interruptions, multitasking, and high volume of work and may therefore be undercounted. Other errors that we did describe could in fact be corrected or reduced by secondary reviews, variations in workflows and further workarounds that the participants did not do in the study. All participants provided primarily outpatient care, but some may have occasionally used an inpatient reconciliation tool embedded in EHR2 that shared several design features with the tested outpatient tool and therefore their familiarity with it may have been greater than what would be expected after a short demonstration at the beginning of the test. There were several automatic interventions built into each EHR designed to assist with medication comparison that we did not explicitly demonstrate to the participants. For example, the existence of a visual cue of correspondence in EHR1 (highlight) was known to some, although they may not have interpreted its meaning correctly. The effect of these interventions on the measured rate of accuracy is therefore uncertain.

CONCLUSION

Reconciling complex medication histories is a laborious and cognitively demanding process. A group of expert clinicians in this study, using 2 specialized EHR tools, reconciled the same patient record with very different results. The higher accuracy of reconciled records in EHR2 was likely attributable to its favorable design and usability characteristics and effective cognitive support. Opinions and comments of the participants strongly supported this finding.

The reconciliation was more complex than what is typically needed in practice. Depending on the specialty and demographic variations, however, many clinicians routinely encounter patients with polypharmacy and difficult reconciliations with inefficient tools are a persistent problem for them. Patients who take many medications would greatly benefit from improved reconciliation, which would potentially result in improved medication safety overall.

Accurate assessment of the safety and effectiveness of electronic reconciliation tools requires comprehensive and rigorous testing and should prioritize complex rather than simpler tasks that are currently used by the ONC for certification or by vendors in demonstrating their products.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector. It was supported internally by Partners HealthCare.

COMPETING INTERESTS

The authors have no competing interests to declare.

CONTRIBUTORSHIP

JH and HR contributed equally to the conception and design of the study, data acquisition, and analysis and interpretation, and to drafting the manuscript and revising it for important intellectual content. ED substantially contributed to the acquisition, analysis, and interpretation of data and to critical revision of the manuscript. All authors provided final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.