Skip to main content
AMIA Annual Symposium Proceedings logoLink to AMIA Annual Symposium Proceedings
. 2018 Apr 16;2017:790–799.

Using Process Mining Techniques to Study Workflows in a Pre-operative Setting

Adela Grando 1, Danielle Groat 1, Stephanie K Furniss 1,2, Joshua Nowak 1,3, Regina Gaines 1, David R Kaufman 1, Karl A Poterack 4, Tim Miksch 2, Richard A Helmers 5
PMCID: PMC5977611  PMID: 29854145

Abstract

Information technologies have transformed healthcare delivery and promise to improve efficiency and quality of care. However, in-depth analysis of EHR-mediated workflows is challenging. Our goal was to apply process mining, in combination with observational techniques, to understand EHR-based workflows.

We reviewed nearly 76,000 event logs from 15 providers and supporting staff, and 142 patients in a pre-operative setting and we inspected 3 weeks of interviews and video observations. We found that on average 44 minutes were spent per patient interacting with the EHR, 55% of the time of the patient visit was spent by personnel interacting with the EHR and for over 5% of the time personnel used or reviewed paper-based artifacts. We also discovered the handover-of-care network and compared frequency of interactions between personnel.

This study suggests that applying process mining in combination with observational techniques has vast potential for informing Mayo Clinic in the forthcoming EHR conversion.

Introduction

Current health information technologies (HIT), such as electronic healthcare records (EHRs), aim to improve healthcare delivery through increased access to patient information and better organization1. However, recent HIT implementation and use have yielded mixed results in different clinical settings. Problems with EHR integration with existing clinical work processes impact workflow2,3. This has resulted in extending the time required to complete tasks and has also resulted in the development of workarounds which can result in adverse events that compromise patient safety or quality of care delivered4. With these new problems arising, we need a meaningful way to record and analyze clinicians’ interactions with HIT. This need coincides with Mayo Clinic’s interest to collect and analyze current workflow on clinician’s interactions in order to facilitate a large-scale EHR conversion.

In advance of an upcoming enterprise wide conversion to a converged EHR, the Mayo Clinic launched the ROOT (Registry Of Operations and Tasks) project, which aims to understand EHR-mediated workflows and clinician-HIT interaction5. Surgical settings have been prioritized as the testbed for this project. Surgery is one of the most clinical and resource intensive activities in health care, constituted by multiple care transitions and a high incidence of adverse events6. A main goal of the ROOT project is to create a repository of current high-definition snapshots of clinical and informational workflows at Phoenix, AZ, Rochester, MN, Jacksonville, FL, and in the Mayo Clinic Healthcare System before the transition to a new EHR.

Process mining techniques describe a family of a-posteriori automatic analysis techniques that exploit event logs, which record information about the sequences (traces) of events (activities) executed using electronic information systems. While process mining has been extensively used in business management, medical process mining is a relatively newer research field7. In our prior research, which laid the methodological foundation for this work, we investigated variations in clinicians’ EHR workflow by integrating quantitative analysis of patterns of users’ EHR-interactions with in-depth qualitative analysis of user performance8,9. This paper summarizes preliminary outcomes from the ROOT project resulting from observations, interviews, video ethnography and process mining analysis at the pre-operative (PreOp) setting at Mayo Clinic Hospital, Phoenix, Arizona.

Objectives

The objective of this study was to employ an integrative methodological approach leveraging process mining, observations, interviews and video ethnography, to understand patient workflows, clinical workflows, and interactions of personnel with electronic systems at the PreOp setting of one of the Mayo Clinic facilities. Four questions were posed to guide this research: Can we validate observed handover-of-care based on PreOP personnel’s sequential interactions with patient cases?; How much time do personnel spend in electronic information systems?; How much time do personnel spend on documenting and/or reviewing data using paper based artifacts vs. electronic systems?; Is there a correlation between care paths followed and the time spent in the PreOp by different patient groups?.

Methods

We integrated outcomes of clinical observations, interviews or video ethnography with process mining techniques to increase our study sample and, in some cases, validate outcomes. The interviews, observations and videos guided the process mining and helped us to identify relevant questions to be answered by process mining techniques in order to better understand clinical and patient workflows. In turn, the process mining can be used to discern the distribution of patterns evident in our workflow.

For this study, two different teams collaborated to carry out different data gathering techniques and analysis.

Observations, Interviews and Video Ethnography

The first team conducted mixed-method data collection on-site over a 3-week period in 2016. Data collection methods included semi-structured interviews and multiple observational techniques to surface specifics about all roles and activities performed in PreOp. In this paper, we refer to the following set of methods collectively as “observational techniques”: shadowing, video ethnography using hand-held video camera and Morae™ video capture. Morae™ software is used for usability studies and it records user activity with no interruption to the user’s work. Using a webcam, it was also used to record audio of participants verbalizing their thoughts and video recording of the participant’s hands and desk space. Participants were asked to think-aloud as they performed HIT-based tasks to reveal activity, goals and cognitive processing, and allow for retrospective analysis.

The mixed-method approach was used by the team to identify, describe and validate the clinical workflow components and processes. The study presented here, draws on a subset of this analysis for an in-depth description of patient flow, personnel, and personnel’s coordination processes integral to efficient patient flow.

Process Mining Analysis

The second team was responsible for collecting and analyzing, using process mining techniques and tools, the event logs generated at the PreOp during two days that the first team was on-site. The team had access to nearly 76,000 event logs generated during the selected days. For those days, there was a daily average of 15 providers and supporting staff, and 71 patients.

The event logs mainly captured actions performed within the Cerner SurgiNet environment. SurgiNet is the EHR application used for tracking cases in the perioperative settings (i.e. pre-op, intra-op and post-op) and for most perioperative documentation. In addition, when applications outside the EHR system such as Synthesis, Smart Template Wizard, Schappt Book and Outlook email were launched, there were few if any event logs created.

Each row in the log file was considered to be an event. Each event in the log file included 16 attributes: 1)End Date: date of the log file event; 2)End Time: accurate to the second when the log file event was completed; 3)Node Name: networking information; 4)Timer Name: high-level description of the log file event; 5)Elapsed Time: amount of time in seconds for the system to execute the log file events; 6)Pass: unknown; 7)Exec Name: name of the execution software; 8)Process ID: unknown; 9)Thread ID: unknown; 10)Username: provider identifier; 11)Client PC: workstation identifier; 12)Client IP: networking identifier; 13)Fail Code: unknown; 14)Subtimer: additional background regarding the log file event; 15)Result Count: unknown; 16) Metadata 1, 2 and 3: may contain a patient identifier; the patient identifier is not related to the patient medical record number or any other identifier that could be used to re-identify the patient. We provide in Figure 1 an example of an event in a log file. From all the information contained in the log files the most relevant for process mining analysis was end date and time, elapsed time, timer name in conjunction with the subtimer, username (provider identifier) and the metadata (patient identifier).

Figure 1.

Figure 1.

All of the attributes provided in a Cerner event log are split into two rows. The columns of interest in the first row provide the ending timestamp of the event (End Date, End Time), duration of the event (Elapsed Time), name of software function that triggered the event (Timer Name) within an information system (Exec Name), and the identifier of the provider (User Name) using the software. The second row contain the identifier of the patient (Metadata 1) that is related to the event and additional information regarding the software’s function (Subtimer).

The Mayo Clinic also made available an event log dictionary that provided, for each potential event log that could be collected, a description of its interpretation. To better understand and associate users’ behavior and clinical workflow, the second team reviewed selected portions of the Morae™ video recordings and closely collaborated with the first team to generate the results reported here.

Morae™, Microsoft Excel, Java, and Disco™ were used for preprocessing and cleaning the EHR event logs. While the process mining analysis presented in this paper is based on two of the observed days, the full data set of Morae™ video recordings and researchers’ observations, were reviewed for log event analysis.

In Excel, event logs were sorted based on date and time. Files were converted from .csv to .xls format. The .xls files were then cleaning using a Java program. Once the files were cleaned, they were manually merged with Excel. Files were merged according to days so that multiple personnel were all in the same file. This organization allows for the process mining analysis to be done from either the personnel or patient perspective.

Once the .xls files were organized by days, they were imported into Disco™ and exported as XES ProM files. Disco™, a processing mining and automated process discovery software, was used for additional data cleaning and file conversion10. The resulting XES files were imported to ProM for analysis. ProM is an open-source, free software, considered the “de-facto” process mining software11. It contains hundreds of plug-ins (algorithms) for process mining analysis. Several ProM plug-ins were used to further refine the dataset for analysis and produce visualizations.

Results

First, we present outcomes from the application of observational techniques, in particular descriptions of PreOp personnel roles and responsibilities, patient-based workflows and personnel’s coordination process integral to patient flow in PreOp. Then, we build on those outcomes to apply process mining techniques to address three relevant questions posted by the observational team: Can we validate observed handover-of-care based on PreOP personnel’s sequential interactions with patient cases?; How much time do personnel spend in electronic information systems?; How much time do personnel spend on documenting and/or reviewing data using paper based artifacts vs. electronic systems?; Is there a correlation between care paths followed and the time spent in the PreOp by different patient groups?.

Personnel and their interactions, captured through observational techniques

PreOp relies on a number of personnel and roles to ensure safe patient care delivery and timely preparation for surgery. Roles specific to PreOp include the health unit coordinator (HUC), patient care associate (PCA), pre-operative nurse (RN), float nurse (Float RN), Desk nurse (Desk RN), and Team Lead (TeamLead RN).

HUCs are non-clinical staff who are stationed at the check-in desk in the Surgery Waiting Room. For patients arriving to the hospital on their day of surgery, the HUC station is their first stop. The HUC facilitates communication between patient’s family and clinicians during the patient’s perioperative care (i.e., pre-op, intra-op and post-op). For example, the HUC is responsible for notifying the Desk RN when the patient has arrived in the Surgery Waiting Room.

The PCA is the patient’s second point of contact in PreOp. PCAs guide the patient from the Surgery Waiting Room to the patient’s PreOp room, orient the patient to the room, and collect some of the patient’s vital signs measures (written on paper). For some patients, PCAs may help with surgery preparation activities. PCA’s do not use HIT in support of patient care.

Each day, RNs are assigned two PreOp Rooms and are responsible for ensuring the Nurse Assessment, preparatory care and associated documentation are complete for each patient assigned to those rooms. Activities include: validate the patient and surgical procedure, activate anesthesia, surgery and nursing orders, reconcile medications, administer medications, validate procedure and signature on the informed consent document, validate presence of a recent history and physical note is documented, ensure pre-operative testing is completed, coordinate outside resources, complete assessment questions with the patient (e.g., neurological assessment), collect patient contact information and post-surgery transportation plans and others. The RN’s last activity for each patient, is to complete a face-to-face verbal handoff to members of the patient’s surgical team, who then transport the patient to the operation room.

Each day, several nurses fill the role of Float RN. Unlike RNs, Float RNs are not assigned to rooms or patients. Instead, they provide assistance to the RN as needed. The assigned RN is responsible for the RN Assessment and required pre-operative documentation. Therefore, the Float RN will assist with non-documentation activities allowed by their training (e.g., place IV catheter and administer IV fluids).

The Desk RN and TeamLead RN roles are each filled daily by a nurse who is more experienced in the unit. The Desk RN manages patient flow through the PreOp unit and day-of staffing schedules. Responsibilities include, assign patients to PreOp Rooms with consideration to the RN’s workload and the patient’s scheduled surgery start time,identify patients who may not be prepared for the scheduled surgery start time and communicate with the surgical team (e.g., the patient arrives late to the unit), and coordinate patients and PreOp staff in response to emergent changes to the surgery schedule (e.g., contact patients to notify of change). Experience is important because the Desk RN needs to anticipate changes to the surgery schedule (e.g., based on a surgeon’s anticipated efficiency in prior procedures, the surgeon is likely to ask for a patient to be prepared of surgery earlier than scheduled) and resource constraints (availability of RNs and Float RNs). The TeamLead RN role is typically filled by one experienced RN. When this RN is not working, another experienced RN fills this role. Otherwise, the TeamLead RN’s daily responsibilities are similar to a Float RN where they assist PreOp RNs with safe and timely preparation of the patients for surgery.

Patient-based workflow, captured through observation techniques

Table 1 provides a schematic summary of observed patient-workflow in terms of patient’s physical location in PreOp setting, personnel involved in patient care, artifacts used (paper-based or electronic) and SurgiNet perioperative tracking status. The patient’s first interaction with the PreOp unit, is at the Surgery Waiting Room on the hospital’s second floor. Prior to this, upon arriving at the hospital, the patient checks in at the Admission Desk. In the Surgery Waiting Room, the patient checks in with the HUC at the check-in desk. Patients are asked to arrive two hours before their scheduled surgery. To inform other PreOp personnel that the patient has arrived, the HUC uses Perioperative Tracking in SurgiNet. Specifically, the HUC changes the patient’s tracking status to “Arrival Waiting Room”.

Table 1.

Observed correlation of patient’s physical location in PreOp setting, personnel involved in patient care, artifacts used (paper-based or electronic) and SurgiNet perioperative tracking status.

Patient’s Location Surgery Waiting Room PreOp Rm# Operating Room
Personnel/Role HUC Desk RN PCA Desk RN RN HUC OR Coordinating RN Desk RN
PreOp Location Surgery Waiting Room Desk RN work space Desk RN work space, Surgery Waiting Room, PreOp Rm Desk RN work space PreOp Rm# Surgery Waiting Room Hallway/ Desk RN workspace Desk RN workspace
Artifacts Used SurgiNet; Paper logbook Staff Scheduler; SurgiNet; Patient chart (w/ stickers, purple paper) Patient chart (w/ stickers, purple paper) SurgiNet; patient sticker Powerchart; SurgiNet; Patient chart (w/ purple paper) SurgiNet; Paper logbook Handover of Purple paper Purple paper, SurgiNet, Powerchart
SurgiNet tracking status “Arrival Waiting Room”     “In PreOp” “RN Assess Comp” “Family” Remove “Family”   Remove “In PreOp”
Other SurgiNet updates   “PreOp Rm#” “[RN name]”           Remove “PreOp Rm #”

The Desk RN monitors the Perioperative Tracking display, which most often remains displayed on one of the person’s desktop monitors. When the Desk RN sees the indication that the patient is in the Waiting Room, the Desk RN retrieves the patient’s prepared chart from a file rack behind him/her, which holds prepared paper charts (PC) for all surgical cases planned for the day. The Desk RN interacts with the Perioperative Tracking display at SurgiNet to select the room and to type the name of the RN assigned to that patient case. To make the room assignment, the Desk RN views PreOp RN staff and the availability of their assigned rooms. The room number is also written on the front of the patient’s chart and on a small purple paper attached to the chart. PreOp RN staffing and their assigned rooms are viewed and managed on a locally-developed screen view, Staff Scheduler, that is always displayed on the Desk RN’s second desktop monitor.

When a chart becomes available in the file rack, a PCA picks up the chart on her way to get the patient. The PCA meets the patient in the Surgical Waiting Room and guides the patient to the assigned PreOp room. On the way, they stop at a scale and measure to get the patient’s height and weight, which the PCA records on the purple paper. To signal to the Desk RN that the patient is on the way to a room, the PCA takes a patient identification sticker off the chart and leaves it with the Desk RN. This triggers the Desk RN to change the patient status in Perioperative Tracking display in SurgiNet from “Arrival Waiting Room” to “In PreOp”. The patient is placed in the PreOp room one or two hours before the surgery.

In the PreOp room, the PCA collects additional patient vital signs (e.g., blood pressure, temperature, etc.). To signal to the RN when the patient is in the room, the PCA moves the rolling table in the room to the hallway just outside the room, and places the patient chart on the table. Before entering the room, the RN will often get the patient’s paper chart from the table and access patient data in the PowerChart to assess pre-operative care needs and initiate processes, such as activating surgical, anesthesia and nursing orders. The duration, number and type of system interactions the RN performs to complete the Nurse Assessment, preparatory care and associated documentation for each case, are affected by a number of factors, such as the patient’s acuity, surgical service and surgical procedure.

When finished with the RN Assessment and pre-operative care and the patient is ready to go to the operating room, the RN changes the patient’s Perioperative Tracking status in SurgiNet to “RN Assess Comp”. If the patient would like to see family who are waiting in the Surgery Waiting Room, the RN would also add “Family” item to the list signaling that the patient’s family can be brought to the patient’s PreOp room. The HUCs monitor their Perioperative Tracking display for the brown “Family” status. When they see this, a HUC guides the family to the patient’s room. Upon returning to their desk, the HUC removes the “Family” status event from the Perioperative Tracking display, which returns the display back to the message “RN Assess Comp”.

After the RN completes a face-to-face verbal handoff with members of the patient’s surgical team, they transport the patient to the assigned operating room for surgery. On their way past the Desk RN, one of the surgical team members drops off the purple card with the Desk RN. This transfer of the purple card is used to signal to the Desk RN that the patient is leaving the PreOp unit. After several minutes have passed and the patient’s Perioperative Tracking status reads “In Surgery”, the Desk RN removes “In PreOp” from the patient’s tracking list. Of note, the purple card has the patient’s vital signs on it that were hand-written by the PCA. The Desk RN is responsible for entering these data into the patient’s chart in PowerChart.

Provider and patient-based workflows captured through process mining

Using a combination of process mining techniques, and outcomes from observational data collection we endeavored to answer four questions relevant for workflow analysis:

1. Can we validate observed handover-of-care based on PreOP personnel’s sequential interactions with patient cases?

We examined how an analysis of team interactions using the event logs compares to the above observational-based description of patient-based workflow. To achieve this, we created a sequential visualization and performed a social-network analysis of the event logs where team interactions are measured by personnel’s sequential access to a patient chart. For example, in the sequence of events associated with a patient chart, if there is an event associated with the HUC followed by an event associated with the Desk RN, then the sequential visualization would show a dot associated to a HUC preceding a dot associated to the Desk RN role, and the social-network analysis would identify a HUC-to-Desk RN interaction for this patient case.

We used the dotted chart PROM plug-in to automatically discover and visualize the sequence of personnel’s interactions for each patient case for Days 1 and 2. Dotted char for Day 2 is shown in Figure 2. Of note, each row is for a single patient and the dot colors are associated to the personnel’s role. The HUC and the Desk RN usually generate the first event log for each patient. These events are often followed by an RN. This is consistent with the observed workflow.

Figure 2:

Figure 2:

Dotted chart for Day 2. The x-axis is divided into hours of the day and each row represents a patient. The colored dots represent log file events from all the type of providers in PreOp. The colors are based on provider roles.

The handover-of-work social network PROM plug-in provides an alternative visualization of team interactions on Days 1 and 2, in which the role-to-role interactions with SurgiNet are visualized as a relative probability. The event logs were filtered to contain four events types that best represented user interactions with SurgiNet for patient documentation or information review activities. These four events were among the 15 most-frequently occurring events for both Days 1 and 2. For example, “Build SDL” event represents personnel clicking check boxes on the screen when completing documentation or reviewing information, and “Load Set Events” event represents personnel changing the PreOp tracking status of the patient. The other two events, “Display Plans” and “Start Chart Search” indicate that the provider/staff has accessed a patient’s care plan and chart, respectively.

In Figure 3, the social network of handover-of-work analysis shows that generally all the roles reciprocate handover-of-work to one another, except for the TeamLead RN who only interacts with the Desk RN. The most common hand-offs occurred from the HUC-to-Desk RN and then back and forth between the Desk RN and the RNs (i.e., Desk RN-to-RN and RN-to-Desk RN). The next most common interactions occurred from the Desk RN–to-HUC, back-and-forth between the HUC and the RNs (i.e., HUC-to-RN and RN-to-HUC). Also, consistent with observations there were fewer event logs generated by float RNs as they generally assist with non-documentation related care.

Figure 3:

Figure 3:

The social network ot nandover-ot-work snows tiiat generally all tiie roles reciprocate handover-of- work to one another except for the Team Lead RN only hands-off work to the Desk RN. The most common hand-offs occur from the HUC-to-Desk RN and then bi-directionally between the Desk RN and the RNs. Circle size correlates to the number of event logs associated with the role—a larger circle size indicates that role generated a larger number of event lows. Arrow thickness correlates to the number of interactions. A larger arrow indicates that interaction occurred more frequently when compared to other interactions.

2. How much time do personnel at PreOp spend using electronic information systems?

To answer this question, two things had to be quantified for each patient: a) how much time personnel spent in electronic information systems, and b) how much time personnel spent with a patient.

After manually reviewing event logs, we found that SurgiNet was almost exclusively the executed program recorded in the provided event logs. There were few events outside SurgiNet. For example, on Day 1, there were only 40 events of ~38,000, or 0.1%, not associated with SurgiNet. Two other programs identified, other than SurgiNet, were Smart Template Wizard and Schappt Book. We were not able to interpret these events, as they did not coincide with the Morae video capture. On the other hand, using Morae video recordings, we could associate an event from Day 2 labeled “web launch” with the launch of Synthesis. Synthesis is a home-grown system to facilitate the navigation of the EHR. It offers users a single, integrated view of information from many disparate sources. This log file event occurred an additional 11 times on Day 2 and only once on Day 1. These other “web launch” events from Days 1 and 2 did not coincide with Morae™ video recordings; therefore, we could not assess if this event name always referred to the user accessing the Synthesis system. Also, there were instances in the Morae™ video recordings where users launched other systems or tools, such as web browsers or email, and these events were not identifiable in the event logs. As a result, we concluded that event logs do not sufficiently identify when users access electronic systems outside SurgiNet; therefore, the event logs could not be analyzed to measure the amount of time personnel spent on other clinical information.

To quantify how much time personnel spent in electronic information systems, we assumed that all activity in the event logs was attributed to time spent using SurgiNet. We analyzed the amount of time the personnel spent in the system per patient by generating patient-centered workflows using ProM. The data from Days 1 and 2 was visualized from the patient perspective with the dotted chart PROM plug-in (Figure 4). From the 142 identified patients we selected a subset of 60 patients. For instance, we removed patients when we identified that personnel spent less than 15 minutes interacting with the information systems to review information of that patient. For the given example, the assumption was that the personnel were accessing patient charts for individuals not scheduled for the surgery that day. For each of the 60 patients we created the corresponding dotted chart and we discovered that the average amount of time a RN spent interacting the patient’s chart using SurgiNet was 44 minutes. On average the time the patient spent in the PreOp room, excluding time in the waiting room and waiting time for surgery, was 3 hours and 33 minutes and 27.2% of that time was spent by personnel accessing SurgiNet.

Figure 4:

Figure 4:

Dotted chart of Day 1. The x-axis is divided into hours of the day (4:00am to the end of the day is displayed) and each line represents one of the 77 patients. Each colored dot represents one of the 39,363 log file events from all the personnel and the colors are based on 274 unique even types. For instance, dark purple represents events where the doctor read clinical notes and light green corresponds to an ICU band load.

The time spent by personnel with a patient could be different from the time spent in electronic information systems. To quantify the time spent by personnel with a patient we excluded event logs generated by HUC, Desk RN and other personnel to eliminate variability in patient arrival times with respect to the PreOp schedule. The dataset was reduced to only include RNs and patients for whom we had Morae video recordings available, which was 6 patient cases. For that subset, total time in the system was found looking at the difference between the first event log from the RN and the last event log from the RN with respect to the patient of interest. This was assumed to closely represent the amount of time a patient spent in a PreOp room before being wheeled into surgery and did not include time spent in the waiting room. For the patient of interest, activity groups associated to the RN role were defined as two or more events that occurred within close temporal proximity to each other. Gaps of 5 minutes or more in the event logs indicated the beginning of a new activity group. The start time from the first event log in an activity group was subtracted from the end time of the last event log from the same activity group, to determine the total amount of time the RN spent reviewing/updating patient information in SurgiNet. The duration of each activity group was considered an approximate representation of the time the RN spent using SurgiNet, which was then compared to the total time in the system. We found that the average amount of time a patient spent in the PreOp Room, excluding time in the waiting room and waiting time for surgery, was 2 hours and 12 minutes. One hour and 12 minutes (55%) of that time was attributed to RNs using SurgiNet to record information relevant to the patient’s clinical case.

3. How much time do personnel at PreOp spend on documenting and/or reviewing data using paper based artifacts?

In Question 1, we used event logs to quantify the amount of time a patient is in PreOp and amount of time personnel use the computer-based artifact, SurgiNet. Morae™ video analysis was required to identify how much time personnel spent per patient reviewing and/or documenting with paper artifacts. The RN spent an average of 6 minutes and 40 seconds using paper-based artifacts per patient, which accounted for 5.2% of the average total time a patient is in PreOp. The remaining time (40%) was not spent using paper-based artifacts or interacting with SurgiNet.

4. Is there a correlation between care paths followed and the time spent in the PreOp by different patient groups?

In our case, patient groups could be defined in terms of several factors, such as the patient’s acuity, surgical service and surgical procedure. By filtering the event logs based on common PreOp patient tracking events described in Figure 1 (e.g., those associated with check-in and check-out events), it is possible to estimate how much time a patient spent in the PreOp unit. The amount of time each patient spent in PreOp was determined by first visualizing the filtered event logs (i.e., a sub-set that only contains the event type associated with tracker status update events for the patients with the dotted chart analysis), see Figure 4. The data was cleaned to only include personnel and patients for whom we had Morae video analysis available. Then the dotted chart plug-in was used to extract the time spent in PreOp in the same fashion that was done to find the amount of time personnel were in SurgiNet as in Figure 4. While we could estimate the care paths and time spent by patients at PreOp, correlating them with the patient’s clinical case requires a manual review of the patient’s chart, which is outside the scope of the methodological approaches chosen for the ROOT project. Therefore, while very relevant, this question remains unanswered.

Discussion

The application of process mining into healthcare is a relatively new, but growing area of research. Zheng et al investigated user’s interaction with an EHR by uncovering hidden navigational patterns in EHR event logs12. They identified repeated feature access in successive action sequence. Various patterns deviated from optimal clinical management pathways. Kannampallil et al used event logs to compare the information-seeking strategies of clinicians in critical care settings13. Specifically, they characterized how distributed information was searched, retrieved and used during clinical workflow. They found that there are costs (e.g. effort, time and cognitive load) associated with particular strategies. In our prior work, process mining was applied to characterize clinician’s patterns of information gathering8,9. Kaufman et al. proposed the novel use of process mining techniques in combination with qualitative methods to help discover, analyze and visualize records of human computer interactions to lead to improved EHR designs. Applying similar methods, Furniss et al. found that participants’ EHR-interactive behavior was associated with patient case complexity, preferred information sources, and EHR default settings.

A 2016 review by Rojas et al. found 74 papers associated with the use of process mining in healthcare domains14. In 2013 a review of the state of the art by Mans et al. indicated that only 40 publications referred to the use of process mining in real clinical contexts15. From those, only one publication focused on the use of process mining in the context of PreOp processses16. The review surfaced the five questions that are most frequently asked when process mining techniques are used in the healthcare context17: Q1) What are the most commonly followed paths and what exceptional paths are followed?; Q2) Where are the bottlenecks in the process?; Q3) Is there compliance with external and internal guidelines?; Q4) Are there any differences between care paths followed by different patient groups?; Q5) What are the roles and social relationships between medical staff?. Besides the questions addressed in the Result section, the ROOT project also attempted to answer frequently asked questions Q1-Q5. A central goal of our research was to apply a hybrid methodology for addressing similar questions. Below we explain in detail successes and challenges encountered trying to answer those questions. We also describe how questions that could not be fully addressed in this study can be investigated in future research.

Questions Q1-Q3: unfortunately, the event logs available in the ROOT project were created to quantify performance measurements. This constraint makes it difficult to accurately infer the clinical context of the related workflow activities. Therefore, applying process mining techniques to the available event logs did not lead to answering questions Q1-Q3.

Question Q4: the research teams wanted to know if there was a correlation between the care paths followed and the time spent in PreOp by different patient groups. Understanding patient’s clinical case would require having access to the patient’s chart. The ROOT project did not envision review of the patient’s medical records, and therefore we could not answer this question.

Question Q5: We successfully answered this question by validating results from observational techniques with the discovered handover-of-care (Figure 3). Consistent with observations, the social network shows that generally the roles reciprocate handover-of-work to one another except for the Team Lead RN who only hands-off work to the Desk RN. In addition, we compared the frequency of interactions between role types. The most common hand-offs occur from the HUC-to-Desk RN and then bi-directionally between the Desk RN and the RNs. This analysis could be replicated considering individual personnel or groups of personnel.

In the process of trying to answer frequently asked questions Q1-Q5 the ROOT project encountered some limitations:

  • Event logs were created for performance analysis and not for workflow analysis: we only had access to event logs being recorded for tracking the performance of electronic information systems (e.g., system response time). Example: time required to load a new screen, time required to register a click in a box in a screen, time required to close a screen, etc. The event logs that we had access to did not contain the level of granularity and event types needed to answer some relevant clinical workflow questions posted by the research team, or to answer the four questions chosen in the Objective section using only process mining techniques. It would have been desirable to have access to event logs related to personnel activities as described in the observed provider-based and patient-based workflows. Example: check-in, assessment, upload vital signs, etc. What we have learned will help to better identify the type of event logs that we wish to collect in the new EHR ecosystem to better understand and discover clinical workflows and to guide workflow improvements.

  • Lack of access to event logs from all electronic information systems used: We only had event logs from a single information system used in PreOp unit, the EHR-based SurgiNet system. While it was the primary system used for patient documentation and coordination activities in this setting, RNs referenced other information systems to complete key patient care activities. In addition, when applications outside the EHR system such as Synthesis, Outlook, Smart Template Wizard and Schappt Book were launched, there were few if any event logs created. As a result, we could only approximate the total time personnel spent with all HIT systems (Question 1). We are in the process of understanding the implications of these limitations.

  • Lack of access to patient’s medical records: because the ROOT project has been framed as a quality improvement research, we had no access to EHR data for the observed patients. Access to this information could have provided additional insights on the discovered patient-based workflows.

The challenges and successes encountered while answering the questions described in the Result section and questions Q1-Q5 helped the ROOT team to identify lessons learned:

  • Access to event log dictionaries was a very valuable initial resource to help interpreting logs

  • Review of observational data could be time consuming but essential to further interpret event logs and outcomes from process mining. The need to access to observational data is based on the limitation that event logs available for the ROOT project were created for performance analysis, instead of for workflow analysis.

  • Mine both patient-centered and provider-centered workflows provided complementary insights for the clinical workflow analysis.

  • Understand differences between the type of events captured through the enterprise, as oppose to the type of desirable events could guide future event log capturing.

Conclusions and Future Work

The Mayo Clinic is embarking on a large-scale effort to standardize workflows across its four geographical clinical networks, and thus move from multiple disparate EHR platforms to a single EHR system.

Here we reported preliminary results that demonstrate the value of applying process mining techniques, in combination with observational techniques, to better understand clinical and electronic workflows before they disappear due to the transition of Mayo Clinic to the new EHR. Most importantly, lessons learned on the limitations encountered during the process of mining existing log events to answer relevant workflow questions will guide future recommendations on type of information and levels of granularity required to be captured by log events to be generated by the new Mayo Clinic EHR.

We have conducted similar multi-method data collection observational techniques at four additional Mayo Clinic locations. In addition, we have completed and we are in the process of analyzing 35 interviews (21 of them involving leadership) across the enterprise. We plan to apply the methodologies described in this paper to answer similar questions, and to compare results between locations. Similarities and differences between identified workflows could inform the upcoming process of workflow standardization and future process improvements across the Mayo Clinic enterprise.

We suggest that determination of event types in system-generated event logs is made with consideration for clinical workflow analysis, such as for the goals of the ROOT project.

Acknowledgements

Mayo Clinic, for financial support through BMS0148.

References

  • 1.Blumenthal D. Stimulating the Adoption of Health Information Technology. N Engl J Med. 2009 Apr 9;360(15):1477–9. doi: 10.1056/NEJMp0901592. [DOI] [PubMed] [Google Scholar]
  • 2.Ash JS, Berg M, Coiera E. Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. J Am Med Inform Assoc JAMIA. 2004;11(2):104–12. doi: 10.1197/jamia.M1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors. JAMA. 2005 Mar 9;293(10):1197–203. doi: 10.1001/jama.293.10.1197. [DOI] [PubMed] [Google Scholar]
  • 4.Unertl KM, Weinger MB, Johnson KB, Lorenzi NM. Describing and Modeling Workflow and Information Flow in Chronic Disease Care. J Am Med Inform Assoc JAMIA. 2009;16(6):826–36. doi: 10.1197/jamia.M3000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Doebbling Brad, Burton M, Kaufman D, Poterack K, McCullough M, Grando A, et al. Integrated Workflow Capture in an EHR Conversion: Standardizing on Best Practice Methods. In:. AMIA Conference 2016; Chicago, IL. 2016. [Google Scholar]
  • 6.Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003 Feb 4;138(3):161–7. doi: 10.7326/0003-4819-138-3-200302040-00007. [DOI] [PubMed] [Google Scholar]
  • 7.Process Mining - Data Science in Action | Wil M.P. van der Aalst | Springer [Internet]. [cited 2017 Mar 2]. Available from: http://www.springer.com/us/book/9783662498507.
  • 8.Kaufman DR, Furniss SK, Grando MA, Larson DW, Burton MM. A Sequential Data Analysis Approach to Electronic Health Record Workflow. Stud Health Technol Inform. 2015;218:40608. [PubMed] [Google Scholar]
  • 9.Furniss S, Burton M, Grando A, Larson D, Kaufman D. Integrating Process Mining and Cognitive Analysis to Study EHR Workflow. In:. AMIA Conference 2016; Chicago, IL. 2016. [PMC free article] [PubMed] [Google Scholar]
  • 10.Fluxicon. Process Mining and Automated Process Discovery Software for Professionals - Fluxicon Disco. [Internet]. Disco. [cited 2017 Feb 14] Available from: https://fluxicon.com/disco/ [Google Scholar]
  • 11.start | ProM Tools [Internet]. [cited 2017 Feb 14]. Available from: http://www.promtools.org/doku.php. [Google Scholar]
  • 12.Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc JAMIA. 2009 Apr;16(2):228–37. doi: 10.1197/jamia.M2852. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kannampallil TG, Franklin A, Mishra R, Almoosa KF, Cohen T, Patel VL. Understanding the nature of information seeking behavior in critical care: implications for the design of health information technology. Artif Intell Med. 2013 Jan;57(1):21–9. doi: 10.1016/j.artmed.2012.10.002. [DOI] [PubMed] [Google Scholar]
  • 14.Rojas E, Munoz-Gama J, Sepulveda M, Capurro D. Process mining in healthcare: A literature review. J Biomed Inform. 2016 Jun;61:224–36. doi: 10.1016/j.jbi.2016.04.007. [DOI] [PubMed] [Google Scholar]
  • 15.Mans, Ronny RS, Aalst, van der WMP Wil, Vanwersch, RJB Rob. Process mining in healthcare: opportunities beyond the ordinary [Internet]. BPMcenter.org; 2013 [cited 2017 Mar 2]. (BPM report) Available from: http://repository.tue.nl/761533. [Google Scholar]
  • 16.Fei H, Meskens N. Discovering Patient Care Process Models from Event Logs. In:. 8th International Conference of Modeling and Simulation; Hammamet, Tunisia. 2010. [Google Scholar]
  • 17.Mans RS, Aalst WMP, der van, Vanwersch RJB, Moleman AJ. Process Mining in Healthcare: Data Challenges When Answering Frequently Posed Questions. In: In: Lenz R, Miksch S, Peleg M, Reichert M, Riaño D, ten Teije A, editors. Process Support and Knowledge Representation in Health Care. Berlin Heidelberg: Springer; 2013. (Lecture Notes in Computer Science) [Google Scholar]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association

RESOURCES