We’re Lost, But We are Making Good Time: Navigating Complex Pathways in a Patient-Order Management Task

Abstract

Patient order management (POM) is a mission-critical task for perioperative workflow. Interface complexity within different EHR systems result in poor usability, increasing documentation burden. POM interfaces were compared across two systems prior to (Cerner SurgiNet) and subsequent to an EHR conversion (Epic). Here we employ a navigational complexity framework useful for examining differences in EHR interface systems. The methodological approach includes 1) expert-based methods—specifically, functional analysis, keystroke level model (KLM) and cognitive walkthrough, and 2) quantitative analysis of observed interactive user behaviors. We found differences in relation to navigational complexity with the SurgiNet interface displaying a higher number of unused POM functions, with 12 in total whereas Epic displayed 7 total functions. As reflected in all measures, Epic facilitated a more streamlined task-focused user experience. The approach enabled us to scrutinize the impact of different EHR interfaces on task performance and usability barriers subsequent to system implementation.

Introduction

A primary objective of human-computer interaction research in healthcare is to gain a clear understanding of how health information technology (IT) facilitates or hinders clinical workflow, specifically within EHR use (1). Complexity within an interface often results in poor usability of a system, generating barriers to efficient workflow. There is universal dissatisfaction with electronic health records (EHRs), specifically increasing safety challenges and burden of use (2). One significant recommendation of the AMIA 2020 Task Force was decreasing documentation burden (3), of which poor system usability can significantly contribute. This poor usability can create barriers to clinical workflow due to a lack of consistency across interface design, generating interaction complexities during data entry and navigation. Inefficient EHR system navigation can result from having patient information scattered across multiple screens, unwieldy interfaces, and data access and data entry processes that require a complex set of steps (4). By modeling the steps in clinical workflow and assessment of associated interfaces, areas with high cognitive load and usability barriers and challenges can be further explored. An assumption guiding much of the work in this area is that small interface changes can significantly reduce bottlenecks in workflow and improve task performance (5). Comparative analyses can yield valuable insights into the sorts of changes needed to streamline navigation and enhance user performance (6).

The objective of our research is to understand EHR-mediated workflows and how the different facets of interface elements impact task performance and cognition. Task performance based on interface design is assessed by applying the navigational complexity framework to the pre-post implementation of a new EHR while comparing used and unused interface elements. This paper presents a comparison of the different EHR interface design elements. It analyzes their frequency of use in clinical workflow and how these design elements influence task performance and efficiency.

Background

Healthcare institutions have invested significantly in the implementation of new EHRs. The expectations that EHRs will enhance productivity and streamline workflow have met with equivocal results. The persistent problem is that EHR interface designs are frequently unnecessarily intricate, compromising the user experience (8). There is a myriad of factors that contribute to the usability challenges experienced by clinicians. Some of them, for example, policies mandating increasing volumes of documentation, are beyond the scope of human-computer interaction research. However, there has been a growing body of research that has expanded the range of usability research and situated within the broader spectrum of EHR-mediated workflow (2,7,9). Recent efforts have been made to characterize, operationalize, and reduce navigational complexity (4,5). EHR-based navigation can be construed as an interaction with user interface presentation and controls that allow users to locate and access needed information(4). Navigation describes the path taken to complete a task, including the actions (e.g., mouse clicks) and the traversal through space (e.g., negotiating a sequence of screens) (5). In this paper, we employ a cognitive engineering (CE) framework for the development of principles, methods, and tools used to assess and guide the design of systems to support human performance (10). User behavior can be characterized as a combination of elementary cognitive, perceptual, and motor behavior(11). Certain task-system combinations may be more memory-intensive, or, require more in the way of perceptual and motor behavior (11). Problematic navigation creates significant cognitive overhead with efforts devoted to managing the interface and fewer resources available for completion of EHR-mediated clinical tasks.

The study reported in this paper builds on our prior research into task-based navigational complexity. Duncan and colleagues studied vital signs documentation, medication reconciliation, and medication administration record tasks in the EHR, characterizing how different charting interfaces mediate performances across clinical sites (5,12,13). The study documented how the configuration of interface elements created unnecessary complexities when interacting with the system, as reflected in time on task and interactive behavior complexity measures. Based on this work, Duncan et al. developed a navigational complexity theoretical and methodological framework for examining differences in EHR interface systems and their impact on task performance (5). The framework employs both expert review and user testing methods to explore differences in task-specific EHR interfaces. The approach also draws on Calvitti et al., which applied methods for capturing, analyzing, and visualizing EHR use and clinical workflow (7). We also incorporate techniques from the TURF EHR usability framework, which operationalize and explains usability differences employing a range of methods (8). Specifically, we draw on a functional analysis approach to categorize the relative instrumental value of interface elements (realized as functions) in completing a task (14).

Methods

The navigational complexity methodological framework is described in detail in Duncan et al. (12). In this paper, the mentioned framework is applied in the process of identifying the complexities in navigation variation across clinical sites and individuals. There are two main categories of analysis: expert-based methods (e.g., representational analysis) mainly involve the evaluation and judgment of trained analysts. User-based methods rely on empirical data derived typically from observational or experimental studies. In this case, the observations were in-situ as nurses performed tasks before surgery. The Registry of Operations and Tasks (ROOT) Project was launched to characterize current workflows focusing on EHR use in the surgical services department. One of the primary goals of the ROOT is to understand and interpret variation across different sites and systems.

Settings

Observations took place at four Mayo Clinic hospitals: Phoenix, AZ, Rochester, MN, Jacksonville, FL, and Eau Claire, WI. The analysis was performed on video capture of 18 different patient cases involving 11 nurses across all sites. At the Arizona and Florida campuses, the primary tool used for charting was Cerner SurgiNet. . A total of 14 hours of video recordings were captured across 11 different nurses over 10 days at the three different clinical sites, which are presented in this work The primary focus of data capture was on the preoperative (PreOp) nursing assessment performed by nursing staff in-situ on a variety of patients. Morae™ 3.3 video analytic software was used to capture workflow, where the software records the clinician’s on-screen activities.

Data Analysis

Video recordings of individual patient encounters were reviewed for integrity and noticeable gaps in time then were segmented into different tasks based on an established clinical workflow task list. Once segmented, the specific task of interest was isolated. The navigational complexity framework applied here includes both expert-based and user-based analysis (5) on the POM task. Expert-based methods included representational analysis of interfaces to evaluate the appropriateness of representations for performing a selected task (11) and process modeling to represent the ideal sequence of steps involved in task completion. User-based methods such as interactive behavior measures for each clinician were used to understand what actions users performed and to identify and explain variation across users, systems, and clinical sites. Functional analysis was performed to gain a thorough understanding of the functionalities that are required to meet specific work requirements (TURF). User interface elements were categorized as objects or operations and situated within the task completion where each function was utilized.

Results

We compared patient order management documentation across two different systems: Cerner (SurgiNet), which was the legacy system, and the newly implemented Epic system. Schematic representations of the interfaces used for patient order management in SurgiNet and Epic are presented in Figures 1 and 2. In the surgical setting, this is a particularly important task as no processes involving patient care can be completed unless an order is entered for that process. The method of managing patient orders includes activating and deactivating orders for execution, releasing orders from holding for various clinical tasks, and creating verbal orders for emerging tasks for specific patients that need to be completed. The results are organized as follows: schematic representations of the interfaces used in POM to provide a visualization of the interface elements, an excerpt from a cognitive walkthrough/KLM to show the goals, actions and cognitive processes of task completion, a workflow model aligning goals and subgoals with the used functions and interactive behavior measures showing the perceptual-motor effort required from users.

Figure 1:

Schematic representation of the POM interface presented in Cerner SurgiNet

Figure 2:

Schematic representation of the POM interface presented in Epic

Interface Schematic Representation Descriptions

Figures 1 and 2 present schematic representations of the individual interfaces used to complete the POM task. There is a general universal protocol for releasing and activating orders across all surgical settings. However, the process in which these steps are completed varies substantially across systems. In Cerner SurgiNet, there was a menu column that allowed for navigation between various sections of the EHR, and one of the available options was the “Orders-Charges” tab. Information was displayed in a list form with the ability to shift between various sections of orders through a navigation pane (see Section A, Figure 1). Active outstanding orders were displayed in bold. Although the steps involved in task completion are nearly identical, there are differences in the representations of orders, in particular, the headers used to categorize orders. Epic presents a significantly smaller number of available functions when accessing the POM interface. There is a menu column that allows for navigation between various sections of the EHR. One of the main options is the “Orders” button, as seen in Figure 2 Section A. Information was displayed in list format with the ability to sort orders into labeled sections such as “Signed and Held” (see Section B, Figure 1).

Functional Analysis of EHR Interface Elements

When comparing the different functions available within the two different systems, there are variations between the different modes of interactions as well as the available functions within the interface. Cerner SurgiNet has a more significant subset of functions available when navigating through the system to the POM interface that is not related to the task of focus. These can be seen in Figure 1, where a total of 29 functions are available in the left-hand panel, covering a range of tasks, including POM. The multitude of elements likely increases the visual search for the user when navigating to the needed function and adding more steps to complete a task. Epic has a much higher ratio of overall used functions relative to those visible, where a total of 10 functions are initially available to users to navigate to the POM interface, as seen in Figure 2 Section A. In both SurgiNet and Epic, these functions are static buttons. When navigating to the appropriate section of orders, there are numerous different functions within SurgiNet that users have access to, 16 in total. Although these are used for categorization and sorting, clinicians uniformly employ one function, as seen in Figure 1 Section B. All of these functions are either checkboxes or dropdown selections. There is a more significant visual search effort required for users to locate the correct function. In Epic, there are a total of 7 different selectable functions when navigating to the appropriate section of orders. The available functions provide a narrower subset of options, making task completion more streamlined, as seen in Figure 2 Section B. During the process of activating and releasing orders, SurgiNet offers several different available functions within the interface that are pertinent to the task but not utilized. In Epic, there is an overall smaller subset of functions available to users within the interface for activating orders. By restricting the displayed functions, there is a more streamlined workflow that makes the overall process simpler but at the expense of not having proximal access to other desired functions.

Cognitive Walkthrough

To efficiently interact with the functions on-screen to complete the task, clinicians need to situate the process as goals and subgoals with associated actions and cognitive processes. While there are physical actions (mouse clicks), these do not show the complexities involved with task completion. To understand these complexities and how functions influence the cognitive processes, a combination of a cognitive walkthrough and KLM was created to show the goal and actions required to complete the POM task, as seen in Tables 1 and 2. The purpose of this analysis was to use the cognitive walkthrough to show the milestones of task progress and then leverage the KLM to model the physical and mental actions taken by users to reach these milestones. Additionally, cognitive processes were also aligned with these actions that correspond with the functions used for task completion. Table 1 shows a portion of a primary goal, subgoals, actions, and cognitive processes associated with POM. Being that the methods are mostly identical for all sites, this table reflects the general procedure with slight variations existing for site-specific details. The primary goal of all systems was to navigate to the appropriate interface and activate applicable patient orders.

Table 1:

Combination of KLM and Cognitive Walkthrough for the Patient Order Management Task in Cerner SurgiNet

Goal 2: Activate Appropriate Order sets
Subgoal A: Locate Appropriate Section of Orders
Action: Search Header options for the Signed and Held” section
Cognitive Process	Description	Operation	Time (Sec)
Working Memory	Decide what order section is appropriate	M [Locate]	1.2
Visual Search Working Memory	Locate the Signed and Held” section	M [Locate]	1.2
Subgoal B: Select Signed and Held” section
Action: Click on Signed and Held” section
Cognitive Process	Description	Operation	Time (Sec)
Perceptual-Motor Visual Search	Point to Signed and Held” section	P [Point]	1.1
Perceptual-Motor	Click on Signed and Held” section	B [Mouse]	0.1
System Response: Show all Orders in section
Subgoal C: Determine Appropriate Orders for Patient
Action: Select appropriate order sets to release
Cognitive Process	Description	Operation	Time (Sec)
Visual Search Working Memory	Locate the order sets to release	M [Locate]	1.2
Perceptual-Motor Visual Search	Point to the checkbox of the order set to release	P [Point]	1.1
Perceptual-Motor	Click on the checkbox of the order set to release	B [Mouse]	0.1
Perceptual-Motor Visual Search	Point to the “Release” button	P [Point]	1.1
Perceptual-Motor	Click on the “Release” button	B [Mouse]	0.1
System Response: Ordered released to be executed
Subgoal D: Activate Appropriate Orders
Action: Select Orders to Activate
Cognitive Process	Description	Operation	Time (Sec)
Visual Search Working Memory	Locate the orders order section to activate	M [Locate]	1.2
Perceptual-Motor	Point to order section to activate	P [Point]	1.1
Visual Search
Perceptual-Motor	Click on order section to activate	B [Mouse]	0.1
Visual Search Working Memory	Locate “Activate” button	M [Locate]	1.2
Perceptual-Motor Visual Search	Point to “Activate” button	P [Point]	1.1
Perceptual-Motor	Click on “Activate” button	B [Mouse]	0.1
System Response: Activated list of orders for review

Table 2:

Combination of KLM and Cognitive Walkthrough for the Patient Order Management Task for Epic

Goal 2: Activate Appropriate Orders
Subgoal A: Locate Appropriate Section of Orders
Action: Search Header options for the “SURG General PreOp AZ” Section
Cognitive Process	Description	Operation	Time (Sec)
Working Memory	Decide what order section is appropriate	M [Locate]	1.2
Visual Search Working Memory	Locate the SURG General PreOp AZ section	M [Locate]	1.2
Subgoal B: Select “SURG General PreOp AZ” Section
Action: Click on “SURG General PreOp AZ” Header
Cognitive Process	Description	Operation	Time (Sec)
Perceptual-Motor Visual Search	Point to SURG General PreOp AZ section	P [Point]	1.1
Perceptual-Motor	Click on SURG General PreOp AZ section	B [Mouse]	0.1
System Response: Show all Orders in section
Subgoal C: Determine Appropriate Orders for Patient
Action: Search Order Set for Relevant Orders
Cognitive Process	Description	Operation	Time (Sec)
Working Memory	Decide what order appropriate to activate for patient	M [Locate]	1.2
Visual Search Working Memory	Locate the orders to activate	M [Locate]	1.2
Subgoal D: Activate Appropriate Orders
Action: Select Orders to Activate
Cognitive Process	Description	Operation	Time (Sec)
Visual Search	Locate the orders order section to activate	M [Locate]	1.2
Working Memory
Perceptual-Motor Visual Search	Point to order section to activate	P [Point]	1.1
Perceptual-Motor	Click on order section to activate	B [Mouse]	0.1
Visual Search Working Memory	Locate the “Activate” button	M [Locate]	1.2
Perceptual-Motor Visual Search	Point to “Activate” button	P [Point]	1.1
Perceptual-Motor	Click on “Activate” button	B [Mouse]	0.1

For SurgiNet, in goal 2, activating appropriate orders, there are a total of 5 different subgoals used as milestones. Subgoals A-D requires that the user perform one action while subgoal E requires two separate actions by the user to complete. Each of the steps associated with a subgoal involves a series of cognitive processes, however, there is no uniformity between these processes. Subgoal D and E are the most complex with 6 steps involved in each of the actions for both subgoals. Although Subgoal D requires one action, subgoal E required 2 separate actions. There was considerable visual search, dependence on working memory, and extra perceptual-motor activity associated with these steps. Once this was completed, the remaining steps were consistent across all sites where orders were identified for signature. For Epic, in goal 2, activating appropriate orders, there are a total of four subgoals used as milestones for task completion. Subgoal D is the most complex with 1 action, however, there are multiple steps involved in this action, 6 in total, and several cognitive processes. The most prominent are visual search and perceptual-motor activity.

Navigations Paths

Figure 3 shows a sunburst diagram of the different interface design functions hierarchy SurgiNet (blue) and Epic (green) require. Each section where the layout expands shows where the user would navigate to get to the POM interface. When interacting with the system, there are several interactive behaviors and cognitive processes involved in task completion. When completing the individual steps and procedures, there are different interface functions within the EHR that users interact with and navigate through to complete the task. Users go through a hierarchy of functions to get to their desired interface, with different systems having different navigational hierarchies. Bother Cerner SurgiNet and Epic have three levels of navigation to get to the desired interface. However, the SurgiNet system requires users to navigate through a higher number of unused functions, with SurgiNet displaying 12 different functions while Epic displays 7 functions overall. This shows that although the number of steps to navigate through the interface is similar, the cognitive effort to navigate through these interfaces is different.

Figure 3:

Sunburst diagram for Cerner SurgiNet (blue) and Epic (green)

Interactive Behavior

Tables 3 and 4 represent the different interactive behavior measures (mouse clicks, screen changes) that are required by a user when completing the POM task in the pre-implementation system across various clinical sites, all using the same system. These measures show average values across users. These measurements provide some insight into the effort required by users to complete a task as well as task performance to quantify better the burden of navigational complexity placed on clinicians. The functionalities and navigation were nearly identical, with only the presentation of data varying. KLM showed that Arizona required 20.1 seconds, Florida required 23.57, and Eau Claire required 15.61 seconds, respectively.

Table 3:

Patient Order Management summary of Interactive Behavior measures, the KLM predicted task duration and the actual task duration for Cerner SurgiNet

Location (Task completed/ All cases)	Mean (SD) of Time (sec)	Mean (SD) of Mouse Clicks	Mean (SD) of Screen Changes	KLM
Arizona (6/8)	198 (138)	61.4 (47.72)	18.6 (11.9)	20.4
Florida (4/7)	98 (59)	12.67 (13.67)	9.00 (8.49)	16.8
Eau Claire (3/3)	36 (29)	7.0 (5.1)	4.5 (3.0)	16.8

Table 4:

Patient Order Management summary of Interactive Behavior measures, the KLM predicted task duration and the actual task duration for Epic

Location (Task completed/ All cases)	Mean (SD) of Time (sec)	Mean (SD) of Mouse Clicks	Mean (SD) of Screen Changes	KLM
Arizona (6/8)	46.25 (49.72)	6.75 (3.63)	3.5 (1.65)	9.6
Florida (4/7)	35.8 (14.4)	9.33 (3.2)	5.3 (2.0)	9.6
Eau Claire (3/3)	28.02 (5.24)	6.2 (2.6)	2.2 (1.37)	9.6

In contrast, KLM ranged from 20.1 seconds for Arizona, 16.8 seconds for both Florida and Eau Claire. As mentioned above, Arizona required additional steps in the second action of subgoals E, yielding a higher KLM value than elsewhere. This resulted in a higher amount of mouse clicks, 61.4, while Florida and Eau Claire required 12.67 and 7.0 to complete POM. Arizona and Eau Claire observed versus KLM times mostly aligned, while there was a significant difference in Florida observed versus KLM times. In Florida, nearly one-fifth of the observed time was spent waiting for the system to load and populate the screen with patient information, accounting for the difference between the observed and KLM. Table 4 shows mean values for time, mouse clicks, screen changes, and the noted times for Epic in comparison.

Discussion

The study documented changes to EHR-mediated workflow post-conversion as reflected in measures of interactive behavior. EHR conversion to different systems is increasing, which has a discernible impact on workflow and mission-critical tasks such as POM. Currently, there are substantial challenges to comparing EHRs (2) and few frameworks to inform the usability comparison(14), resulting in a lack of information to guide new EHR selection decision-making (2). This paper aimed to contrast the EHR-mediated workflow and how to interface design, including the distribution of functional elements, can affect navigational complexity and task efficiency. The process of POM across two charting systems from Mayo Clinic hospitals were compared through the application of the navigational complexity framework. We observed differences in modes of interaction as mediated by differences in interface design.Epic utilized an overall more streamlined interface, consisting of a more focused and task-centered approach with fewer functional options available that were not directly associated with the POM task.

We observed a shorter mean duration, fewer mouse clicks, and screen changes per order during the POM task post-implementation. When compared to the pre-implementation system, ease of access to the POM interface and the modes of interactions were generally more straightforward and required fewer perceptual-motor actions. Also, the fewer screen changes lessened the burden on working memory for users, thus reducing the overall cognitive load. This was also evidenced in the functional analysis and the sunburst representation, indicating a higher ratio of task-centered functions. The cognitive walkthrough revealed that fewer subgoals were necessary to achieve the goals of activating order sets in Epic relative to SurgiNet. The KLM analyses also predicted a substantial difference of almost 15 seconds to navigate and enable a single order. The convergence of different data analyses incorporating expert and user data suggests that these differences may be robust, even given the small sample size.

Differences in navigation and interface design contribute to poor task efficiency. Providers often perceive EHRs as challenging to use, and usability analysts have cited issues with difficult-to-read interfaces, confusing displays, and iconography that lacks consistency and intuitive meaning(1). These usability issues can often lead to increased cognitive load. These challenges may surface when transitioning from one system or interface to another, causing disruptions to workflow and efficiency (15). By applying the navigational complexity framework, we can better understand how different EHR interfaces differentially mediate task performance and document changes after system implementation. This also allows for a deeper understanding of task complexities at a more granular level.

There are several limitations to this work. The study employed a small sample size, and many uncontrollable factors can have an impact on EHR-mediated workflow. However, by applying the navigational complexity framework leveraging both analytic methods and user analysis, we can validate and even anticipate some of the findings. EHR navigation for a given task is a relatively finite space, and there are a small number of routes to task completion. The modeled pathways represent the most commonly observed rather than a complete set of possible trajectories. Also, we cannot conclude that one EHR is superior to another for the task of patient order management. There may be advantages to have a more extensive array of functions available at a given time. However, we believe that a more streamlined approach reduces navigational complexity and can alleviate some of the documentation burden issues and enhanced user experience.

Conclusions

The use of a navigational complexity framework helped identify interface design differences and how they can contribute to cognitive load and documentation burden. Expert-based and user-based analyses were applied to the POM task to understand better usability barriers at a more granular level and their effect on task performance and efficiency. The analyses completed in this paper identified interface design elements that differentially mediate task performance. By establishing system comparison tools to identify potential usability barriers in a system, issues can be identified to enhance the user experience, leading to a higher quality of care in workflow while informing optimization efforts.