Medication Reconciliation: Work Domain Ontology, Prototype Development, and a Predictive Model

Abstract

Medication errors can result from administration inaccuracies at any point of care and are a major cause for concern. To develop a successful Medication Reconciliation (MR) tool, we believe it necessary to build a Work Domain Ontology (WDO) for the MR process. A WDO defines the explicit, abstract, implementation-independent description of the task by separating the task from work context, application technology, and cognitive architecture. We developed a prototype based upon the WDO and designed to adhere to standard principles of interface design. The prototype was compared to Legacy Health System’s and Pre-Admission Medication List Builder MR tools via a Keystroke-Level Model analysis for three MR tasks. The analysis found the prototype requires the fewest mental operations, completes tasks in the fewest steps, and completes tasks in the least amount of time. Accordingly, we believe that developing a MR tool, based upon the WDO and user interface guidelines, improves user efficiency and reduces cognitive load.

1. Introduction

Medication errors occur during transitions in care, such as when patients are admitted to, or discharged from, hospitals and from the use of multiple medications to treat one or more disorders^1,2,3,4,5. Medication errors can result from administration inaccuracies at any point of care and remain a major cause for concern; one study identified medication issues in 93.3% of patients⁴. Some medication errors can be prevented by carefully maintaining the list of currently prescribed medications^3,4. One study showed that utilizing a medication reconciliation process reduced medication errors by 80%⁶.

While there are a number of standalone and embedded Medication Reconciliation (MR) tools available, there are few studies demonstrating the success of these tools. Furthermore, despite the fact that MR has been identified as an important process for reducing medication errors, there is no consensus on how best to create a MR tool. We hypothesize that a systematic user interface design process, accomplished by first identifying the essential task requirements using a Work Domain Ontology (WDO) and then incorporating user-centered interface design principles, will dramatically improve the efficiency and quality of the MR process. A WDO defines the explicit, abstract, implementation-independent description of the task by separating the task from work context, application technology, and cognitive architecture. In this paper, we show how this process results in an MR tool that is more efficient than two existing tools.

1.1. What is Medication Reconciliation?

Although The Joint Commission (TJC) designated medication reconciliation a National Patient Safety Goal (NPSG) in 2009, there is no precise definition of medication reconciliation (MR) to guide the design and assessment of MR processes and tools. The Joint Commission defines MR as, “the process of comparing a patient’s medication orders to all of the medications that a patient has been taking⁵.” In contrast, The National Institute of Standards and Technology (NIST), as part of the use cases for Stage 1, Meaningful Use Criteria, defines MR as the process of electronically comparing two or more medication lists for a single patient⁷. There is also little consensus on the relationship between medication order entry, the identification of medication allergies and interactions, and medication reconciliation. As a result, both the input to the MR task and the output or goal of the task are vaguely defined, providing almost no objective criteria for improving MR. Despite this lack of guidance in the national standards, there is a growing consensus among clinicians that the reconciliation process should address therapeutic duplication, medication allergies and interactions, medication exclusion, dosage form, dosage regiment, differences in medication strength, and an up-to-date list of medications⁸.

1.2. Issues with Medication Reconciliation

One study of primary care practices found that 25% of patients suffered from an adverse drug event, 11% of which were preventable and 28% ameliorable. In many cases, adverse drug events are the result of errors that occur in the prescribing stage; one study found this to be true in 246 out of 421 cases⁸. Many of these errors occur due to the complexity of the MR process.

TJC has faced numerous challenges since it deemed MR a NPSG. Despite allocating funds, healthcare organizations have been unable to develop a tool that fully meets the needs of the MR process^9–11. The MR process has also proven to be resource intensive. Current systems have proven to be complex and to be implemented successfully, the current MR process demands trained personnel and may require a full-time, multi-disciplinary team^{7–9,12–15}.

The duration of the MR process has also proven itself to be unpredictable. Indeed, time taken to complete the MR process varies widely amongst healthcare professionals¹⁶. In particularly labor-intensive cases, the MR task has been shown to take up to a full hour¹⁷. We believe that creating a tool based upon a Work Domain Ontology, that defines the MR task in terms of its goals, operators, objects, relations among objects, and constraints, will provide better guidance for improved MR processes and tools.

1.3. Establishing the Work Domain Ontology for the Medication Reconciliation process

A WDO defines the basic structure of work that a system, in conjunction with human users, will perform¹⁸. A WDO defines the explicit, abstract, implementation-independent description of the task by separating the task from work context, environment, application technology, and the human cognitive architecture¹⁸. By separating a given task from artifacts used to perform the work, proper attention is placed on the necessary functionality of a task. In other words, the WDO separates the hard constraints and demands of the task from those imposed by the particular tools and workflow used to do the task. Accordingly, rather than focusing on technical details and innovative features, the WDO highlights the inherent nature of the work.

The primary step in creating a WDO is to establish the goals, objects, relations among objects, constraints, and operations of the task. Once the aforementioned have been defined, it is necessary to identify the design space in which the task will be performed. Next, the variety of possible constraints, and their respective relationships, amongst the objects must be determined. Finally, it is necessary to identify the possible implementation-independent operations for the previously defined relevant relationships.

Once defined, analysts use a WDO to determine how best to distribute tasks among humans and tools. This distribution defines the basic functionality of a tool and the required functional interactions between the tool and its human user(s). The detailed design of the user interface requires more traditional human factors user interface design guidelines.

1.4. Design Guidelines for an Electronic Medication Reconciliation Tool

Graphical user interface (GUI) guidelines have been developed to reduce problems and prevent negative consequences. For example, the following “Eight Golden Rules of Interface Design” have been shown to improve usability and ensure error reduction¹⁹:

1. Strive for consistency

2. Enable frequent users to use shortcuts

3. Offer informative feedback

4. Design dialog to yield closure

5. Offer simple error handling

6. Permit easy reversal of actions

7. Support internal locus of control

8. Reduce short-term memory load

These guidelines are used in both the development and evaluation stages of GUI design. By taking such design issues into consideration, one can reduce interface usability issues and decrease time spent in the development process. Such guidelines overcome challenges of designing systems that have an intuitive interface and incorporate essential principles of human-computer interaction²⁰.

1.5. Prototyping

Effective prototyping allows detrimental issues to be identified and addressed prior to product release. In short, a prototype is a draft version of an application that represents the range of the product. Prototypes can be high-fidelity, medium-fidelity, or low-fidelity, ranging from those that are fully functional to paper mock-ups²¹. Despite the lack of functionality associated with low-fidelity, paper mock-ups, such prototypes are often valuable in visualizing design solutions that lead to innovation and improvement. While some question the validity of utilizing low-fidelity prototypes to determine usability, Catani and Biers, in a landmark study, used low- and high-fidelity prototypes to perform library search tasks and found that there was no reliable difference in the usability issues discovered²⁰. In fact, there was a high commonality in the usability issues uncovered using the different prototypes. Further studies demonstrated no significant difference in the number of usability issues identified when analyzing low- and high-fidelity prototypes^22–24. We developed a low-fidelity MR prototype, based on the WDO and the GUI design guidelines.

2. Methods

2.1. Determining the Work Domain Ontology for the Medication Reconciliation Process

To develop the WDO for the MR process, we started with the NIST meaningful use case for MR⁷ and the literature review of MR discussed above. We defined the goal of the MR process to be the creation of a list of medications that the patient should be taking when presented with two different lists of medications for a patient: one older list and one newer. In addition, the final list should be therapeutically appropriate and free of undesired side-effects and medication interactions.

Consider a patient who visits his primary care physician (PCP) after discharge from the hospital. The patient presents his PCP with his post-discharge medication list that may or may not differ from the PCP’s list of current patient medications. Neither list may accurately reflect what the patient is currently taking. Furthermore, the PCP may acquire new information during the visit that would render one, or both, of the lists ineffectual for optimal patient treatment. Additional problems may also arise if the independent lists contain medications that cause drug-drug interactions, adverse drug reactions, or errors in dosage or frequency.

Regardless of the context of care, the objects in the MR process are two lists: one older and one newer. Each list consists of one, or more, medications and its associated name (brand or generic), dosage, frequency, route of administration, active ingredients, start date, and end date.

The basic operations of MR allow a medication to be continued, discontinued, or changed. We chose not to include adding a new medication, because this is best handled as part of physician order entry, which is a separate task that requires its own analysis and user interface. One may assume that the need to compare lists adds additional information operations, such as determining possible medication interactions. At the level of the WDO, however, such needs are not operations, but rather artifacts of the tools used to do the task and the distribution of effort between the tools and user(s). In the WDO, these considerations are specified as a combination of goals and relations among objects. The goals of the final list are that the prescribed medications are therapeutically appropriate and free of undesired side-effects and interactions. How the aforementioned goals are distributed between the tool and the user is part of the application design process. The relations among objects specify correspondences between medications on the two lists. We established that the following relationships could occur between medications on separate lists:

1. Strict Equivalence—The medications are identical in all respects.

2. Form Equivalence—The medications are identical in form; i.e. brand versus generic medications.

3. Functional Equivalence—The medications are different in form, but have equivalent functions.

4. Partial Equivalence—Medications may possess form or functional equivalence, but differ in dosage, frequency, or route. In compound medications, partial equivalence correspondences occur if an ingredient in one compound medication matches the ingredient in a separate medication.

5. No Equivalence—The medication is unique in both form and function.

Creating a thorough WDO for the MR task allowed us to determine both the necessary medication information and the possible inter-medication relationships that could occur.

2.2. Prototyping

Prior to full-scale implementation of an application, it is important to develop a prototype in order to gain insight about the design prior to committing the additional resources required for a final working application. Accordingly, for the primary stage of this study, a low-fidelity, electronic prototype was created to simulate an actual MR task. In order to maximize usability, the constructed prototypes were designed with “The Eight Golden Rules of Interface Design” in mind¹⁹.

In an effort to ensure consistency and reduce short-term memory load, we represent the correspondences found in the MR WDO using spatial proximity and a series of colors throughout all stages of the MR process. The main reconciliation interface, showing the spatial and color-coding of correspondences, is shown in Figure 2. Medications that are equivalent in some way are grouped vertically with faint horizontal separating lines, so that they appear as part of the same visual group. In addition, medications with strict equivalence are green; medications with form, functional, or partial equivalence are yellow; and medications with no equivalence are red. In a further effort to reduce short-term memory load, the relationship between medications and the origin of each medication is explicitly stated. Placing all medication information on a single screen prevents the need to commit medications, dates, and possible interactions to memory.

Figure 2.

Prototype of the initial reconciled record.

To support the internal locus of control, the interface allows direct interaction. Rather than having the user respond to a series of prompts, she can traverse through the MR process in any order. The user is provided with feedback after each decision. The color-coding, relation, and alerts column all dynamically adapt to the actions of the user. Accordingly, the user can view the progress of the MR task step-by-step on a single screen.

The MR task is designed in an effort to allow easy reversal of actions. Primarily, the interface allows the user to switch views without losing any data. For example, if the user is in the midst of the MR process and wants to view the two original lists, she may simply click on the “Original Lists” tab without losing any progress in the “Reconcile Lists” tab. Furthermore, the drag-and-drop interface allows the user to reverse her decision without having to traverse through past actions or start the MR task again.

In order to address the issues that arise in the MR process, a low-fidelity prototype was created using the principles of user interface design (Figures 1, 2 and 3). The prototype went through over 20 iterations in an attempt to bolster usability, harness efficiency, and reduce the probability of error.

Figure 1.

Two, original, unreconciled medication lists.

Figure 3.

Prototype of the final reconciled record.

2.3. GOMS/KLM Analysis

Evaluating the interface can be done in a number of ways after prototype development. One technique used in usability testing is GOMS (Goals, Operations, Methods, and Selection rules) analysis. The purpose of GOMS is to determine the functionality of the system and provide estimates of task performance time²⁵. Proposed by Card and Moran, the Keystroke-Level Model (KLM) is a simplified version of a GOMS analysis in which the overall execution time of a task is estimated by assigning a time to each operation. The KLM model has six operators: K for pressing a key or mouse button, P for pointing to a location on screen with the mouse, H for moving hands to home position on the keyboard or to the mouse, B for a button press, M for mentally preparing to perform an action, T(n) for typing a string of characters (n * K seconds)²⁶. While the exact time associated with each action in the KLM model varies, David Kieras at The University of Michigan has produced estimated execution times for each action: K is 0.20 seconds, P is 1.10 seconds, H is 0.40 seconds, B is 0.10 seconds, M is 1.20 seconds, and T(n) is 0.20 *n seconds²⁷.

Three tasks were analyzed via the KLM analysis: reconciling two identical medications, removing a medication from the record, and editing the dosage of a medication. We compared our prototype to two other systems that are currently in use and claim MR capabilities: Legacy Health System (LHS) with RxPad and the Pre-Admission Medication List (PAML) Builder. Formed in 1989, LHS is an Oregon-based corporation that offers an integrated network of healthcare services through the northwestern United States. According to LHS, RxPad is a, “self-learning module designed to instruct the pharmacist in the process of medication reconciliation and its implementation at Legacy Health System and to define the role of the pharmacist in this interdisciplinary process²⁸.” Information regarding LHS with RxPad can be found at http://www.legacyhealth.org. Similarly, spurred by the growing interest in MR, Partners Information Systems began developing the PAML Builder application in 2004 to assists clinicians with the process. PAML Builder was designed to build the PAML, store information collected through the PAML Builder, provide services to mediate access to the collected information, and support the analytical and supporting needs surrounding the MR process²⁹. Between late 2005 and early 2006, the PAML Builder tool underwent promising pilot studies at both Massachusetts General Hospital and Brigham and Women’s Hospital²⁹. Further information about PAML Builder can be found in EG Poon’s paper, “Design and implementation of an application and associated services to support interdisciplinary medication reconciliation efforts at an integrated healthcare delivery network,” and in the presentation, “It is Not Just a List,” located on the website for the American Society of Health-System Pharmacists (http://www.ashp.org/).

3. Results

3.1. KLM Analysis

Using available screen shots, training manuals, and walkthroughs for both the LHS and PAML Builder systems, a KLM analysis was conducted for three common MR tasks: reconciling two identical medications (Table 1), removing a medication from the record, and editing the dosage of a medication. For each of the three tasks examined, we recreated the necessary operations for each task and, thus, were able to determine the number of required actions, mental operations, and completion times. In each instance, the prototype was found to be the most efficient.

Table 1:

KLM analysis for reconciling two identical medications

Prototype
Description	Operation	Internal or External	Time (Sec)
Reach for mouse	H [mouse]	External	0.40
Locate Reconcile Lists button	M [locate]	Internal	1.20
Point to Reconcile Lists button	P [point]	External	1.10
Click on Reconcile Lists button	B [mouse]	External	0.10
			2.80
PAML Builder
Reach for mouse	H [mouse]	External	0.40
Locate medication	M [locate]	Internal	1.20
Point to medication	P [point]	External	1.10
Click on medication	B [mouse]	External	0.10
Locate Add button	M [locate]	Internal	1.20
Point to Add button	P [point]	External	1.10
Click on Add button	B [mouse]	External	0.10
			5.20
LHS with RxPad
Locate identical medications	M [locate]	Internal	1.20
Determine which duplicate medication to eliminate	M [determine]	Internal	1.20
Reach for mouse	H [mouse]	External	0.40
Point to medication to remove	P [point]	External	1.10
Click on D/C Button	B [mouse]	External	0.10
Determine reason for discontinuation (duplication)	M [determine]	Internal	1.20
Point mouse to menu	P [point]	External	1.10
Click on Discontinue Reason menu	B [mouse]	External	0.10
Select Reason	B [mouse]	External	0.10
Point to Discontinue Rx button	P [point]	External	1.10
Click on Discontinue Rx button	B [mouse]	External	0.10
			7.80

The number of actions, the number of required mental operations, and overall task completion time were lower for the prototype than either LHS RxPad or PAML (Table 2). Specifically, the prototype required an average of seven operations per task compared to the 12.6 and 9.3 operations required for the electronic MR tools of LHS RxPad and PAML Builder, respectively. Furthermore, the average task completion time for each tool was 5.46 seconds for the prototype, 9.5 seconds for LHS and RxPad, and 6.9 seconds for the PAML Builder. Additionally, the prototype demands equal or fewer mental operations from the user than required by either the LHS RxPad or the PAML Builder systems.

Table 2:

Comparative Results from the KLM analysis

	Prototype	LHS	PAML Builder
Average number of operations per task	7.0	12.6	9.3
Average completion time per task (seconds)	5.46	9.5	6.9
Average number of mental operations per task	7.0	12.0	8.0

3.2. Comparative Analysis

The prototype required fewer operations, fewer mental operations, and less time to complete a task. We believe there are two main reasons why neither the LHS or PAML Builder tools were able to achieve the efficiency of the prototype: the operations defined by the WDO are not clear in the two alternative interfaces and neither interface adheres to the aforementioned principles of GUI design.

One use of a WDO is to provide important requirements for guiding GUI design.³⁰ Research into the psychology of human problem-solving has shown that different implementations of a single problem can result in a drastically different efficiencies and cognitive loads. Since the WDO places functionality as the top priority in the design process, it lends itself to promote the creation of highly efficient and usable interfaces. When viewing both the LHS and PAML Builder MR tools, a number of issues arise due to a lack of consideration for both the objects and their respective constraints and relationships.

Consider the task of medication dosage modification. While both tools allow the user to modify the dosage of a medication, the user is given no information as to what, or why, she should modify a medication. Indeed, neither the LHS nor the PAML Builder tool provides visual indication of form equivalence between medications. Furthermore, in the LHS tool, the user must open a new screen to modify the medication. By guiding the user to an alternative screen, both the task completion time and cognitive load of the user is increased; the user must not only remember what other medications the patient is taking, but also the possible interactions. In PAML Builder, the user can modify a medication when adding it to the reconciled list, but is given no information to the possible constraints and relationships amongst the medications.

The WDO in conjunction with the GUI and cognitive engineering guidelines, promoted the creation of an efficient MR interface. The main reasons why the efficiency of both the LHS and PAML Builder tools was lower than the prototype is due to the prototype’s support for an internal locus of control, user feedback, consistency, and ability to reduce cognitive load. In the prototype, the entirety of the MR process is supported through direct interaction with three screens (Figures 1, 2, and 3). Throughout all stages of the reconciliation process, we depict the possible constraints and relationships through a standard set of colors that change when a medication is acted upon. The three main screens are dynamically updated throughout the MR process to provide the user with constant feedback. Accordingly, the user can view the progress of the MR task step-by-step on a single screen. In contrast, the LHS and PAML Builder tools require the user to traverse numerous screens, pop-ups, and prompts. Furthermore, the lack of color-coding and dynamically updated relationships fails to provide the user with feedback on her progress. This leads to an increase in the number of physical and mental operations required to do the analyzed tasks in LHS and PAML.

4. Discussion

We hypothesized that a systematic user interface design process, accomplished by first identifying the essential task requirements and then incorporating user interface design principles in the design process, would dramatically improve the efficiency of the MR process. First, the WDO for the MR task was developed and used as the framework for a prototype design. The prototype was developed according to the WDO and the user interface design guidelines with an overall goal to improve efficiency and reduce the user’s cognitive load. When the prototype was compared to the LHS and PAML electronic MR tools by means of a KLM analysis, we found that the prototype required the fewest number of mental operations, the fewest number of physical steps, and the least amount of time.

The strengths of this study revolve around the utilization of a WDO, in conjunction with design guidelines, to inform prototype creation. Creating a WDO ensures that the essential goals, operations, objects, relations and constraints of a health-related information task are supported by the resulting tool. However, this study has several limitations. Primarily, while the predictive nature of a KLM analysis gives us insight into the tool’s efficiency, we did not conduct an actual usability study. As a result, we are only able to predict task times if we assume that user actions are optimal. In order to get a greater sense of the prototype’s value, we need to construct a higher-fidelity prototype and put it through rigorous usability and learnability testing.

Other studies have created MR tools and tested them in a variety of healthcare settings. Unlike our prototype, however, these tools were either paper-based or combined a number of different interventions such as home-visits, surveillance, computer prescription order entry, and intervention programs^{11,13,30–32}. Other studies were focused on clinical improvement, rather than efficiency, medication adherence, or went beyond the scope of MR as defined by NIST.^14–15 Generally, the aforementioned studies have identified MR as necessary, but do not have a definitive solution to current issues. Unfortunately, the variety of solutions means that our outcomes cannot be easily compared to other MR tools. The strength of our study lies in the fact that we have developed a design process for creating a tool that matches the fundamental requirements of MR and then embeds these requirements in a user-centered tool.

While the prototype predicts improved user efficiency, considerable work remains. In addition to creating a fully functional prototype, it is imperative to conduct usability testing. Doing so will allow for both interface improvement and reduced likelihood of inefficiency and errors. While the MR tool predicts increased efficiency and reduced cognitive load, there are still unanswered questions regarding its effectiveness at reducing the number of medication related adverse events. Accordingly, future research will resolve around creating a high-fidelity prototype, integrating the tool into a working system, and performing a clinical study documenting the number of medication related adverse events that occur. The outcomes from the study will then be compared to the number of medication related adverse events occurring via traditional means of MR. This will allow us to understand if a tool created with a WDO and modeled on design guidelines not only improves efficiency and reduces cognitive load, but also decreases the number of medication related adverse events.

5. Conclusion

This study demonstrates the importance of using a WDO plus GUI design guidelines for designing health information technology that supports fundamental task requirements while simultaneously improving user efficiency.