Multi-Method Study of Electronic Health Records Workflows

Abstract

EHRs transform work practices in ways that enhance or impede the quality of care. There is a need for in-depth analysis of EHR workflows, particularly in complex clinical environments. We investigated EHR-basedpre-operative workflows by combining findings from 18 interviews, 7 days of observations, and process mining of EHR interactions from 31 personnel caring for 375 patients at one tertiary referral center. We provided high-definition descriptions of workflows and personnel roles. One third (32.2%) of the time with each patient was spent interacting with the EHR and 4.2% using paper-based artifacts. We also mined personnel social networks validating observed personnel’s EHR-interactions. When comparing workflows between two similar pre-operative settings at different hospitals, we found significant differences in physical organization, patient workflow, roles, use of EHR, social networks and time efficiency. This study informs Mayo Clinic’s enterprise-wide conversion to a single EHR and will guide before and after workflow comparisons.

Introduction

The Mayo Clinic is undergoing an enterprise-wide conversion to a common electronic health record (EHR). In advance of that conversion, the Mayo Clinic launched the Registry Of Operations and Tasks (ROOT) project in the summer of 2016. ROOT involves an interdisciplinary team with expertise related to Mayo Clinic’s EHR system and clinical environments, clinical workflow analysis, human-computer interaction, and automated business workflow analysis. An aim of ROOT is to capture and document at a granular level EHR-mediated workflows and health information technology (HIT) interactions before the conversion to the new EHR. The methodological approach consists of applying a novel multi-method approach (1-5) that combines video ethnography, staff shadowing, semi-structured interviews (leadership, key clinician and clinical activity) and process mining of event logs. Each of the methods surfaces different dimensions of EHR-mediated workflow and enables us to triangulate.

A hospital’s perioperative workflow is highly complex, involving multiple interconnected sub-workflows that provide surgical care for inpatients and outpatients during pre-assessment, pre-operative (PreOp), intra-operative (IntraOp) and post-operative (PostOp) activities. (6) The perioperative workflow is tightly coupled to patient safety, patient quality of care, patient flow, and stakeholders’ satisfaction (i.e., patient, surgeon, nurse, perioperative staff, and hospital administration). Perioperative settings are complex multi-disciplinary environments; they require a high level of workflow efficiency and account for a substantial share of revenue and cost. Inefficiencies occur when perioperative workflows are fragmented, disjointed or ineffective, leading to delays, interruptions, lack of communication, staff member workarounds and medical error. (7) The perioperative environment is the source of a significant percentage of adverse events. (8)

The ROOT project aims to help improve health care delivery at the Mayo Clinic by providing data-driven insights into current perioperative workflows at Phoenix, Arizona (PHX), Jacksonville, Florida (JAX), Rochester, Minnesota, and the Mayo Clinic Healthcare System before current workflows are modified by the transition to the new EHR. Grando et al. studied the PreOp setting at PHX. (3) The research presented in this paper leverages on PHX PreOp study, by involving an expanded research team and methods to the analysis of PreOp workflows at a second site, JAX. The JAX facility was chosen due to its similarities with the PHX facility; both are destination medical centers of similar patient size, using the same EHR. Both PHX and JAX PreOp aim to provide quality care and have the patient ready for surgery at the appropriate time.

The objective of this research is to study JAX PreOp workflows and compare discovered JAX and PHX PreOp workflows in terms of: personnel roles and their interactions, EHR systems, patient EHR-mediated workflow, clinician/staff social network, and time spent by personnel interacting with the EHR and with paper-based artifacts. This is novel research, given the lack of studies on clinical workflow comparison and standardization. (9-12) It opens future interesting research directions, such as comparison of workflows before and after conversion to a new EHR.

Methods

Setting

The JAX hospital includes two surgical areas—1^st Floor Surgery and 4^th Floor Surgery. Each floor has its own PreOp, IntraOp, and PostOp areas. PreOp and PostOp care are provided by the Perianesthesia Unit staff. The Perianesthesia Unit is part of the larger Surgical Services Department. On average, the Perianesthesia Unit provides PreOp and PostOp care for 65 cases per day—approximately 20 in 1^st Floor Surgery and 45 in 4^th Floor Surgery. The 1^st Floor Surgery contains nine PreOp spaces, nine PostOp spaces, and six operating rooms (ORs). Surgical case types on the 1^st Floor include pain procedures, eye, breast cancer, the more ambulatory cases for orthopedics, general surgery, ear, nose and throat, colorectal, gynecology, and urology. Approximately 80% of these patients go home the same day of the surgery, and 20% are admitted to the hospital. Surgical case types in 4^th Floor Surgery are more acute cases for all surgical specialties.

Multi-method approach

Data collection and curation was based on the clinical workflow capture and analysis methodology developed previously (MB) and adopted as a standard approach in the Applied Clinical Informatics program at Mayo Clinic. (4,5) Our multi-method approach to study JAX PreOp and compare it with PHX PreOp consisted of integrating outcomes from:

Leadership, Key Clinician and Clinical Activity Interviews: Four team members (SF, SH, RS, JB) conducted semi-structured interviews of personnel in various perioperative roles at JAX. The leadership and key clinician interviews were developed to capture operational and information system use, challenges and opportunities for improvement. The clinical activity interviews were developed to capture the “day in the life” of the front-line personnel’s role to include information system use, goals and activities, and challenges.

The interview analysis utilized the software tool NVivo^TM to identify key steps that occurred in the PreOp process. The objective for this analysis was to code and identify tasks in the workflow, and the information tools that they used to help complete job tasks. Interviews were also coded with key structured elements such as the position of the health care personnel and the location within the hospital.

Observational Techniques: Three informatics analysts (SF, SH, RS) conducted mixed-method data collection at JAX perioperative setting over 7 days in April and May 2017. Observational techniques were resource intensive, including video ethnography, shadowing, artifact collection, and contextual inquiry. Shadowing was captured using handheld audio recorders and artifacts were captured using a digital camera. Video ethnography involved capturing HIT interactions with a handheld video camera and with the Morae^TM software. Morae^TM is a reference standard software for usability studies and it records user interactions with information systems without interrupting the user’s work. Via the use of a web cam, Morae^TM is also able to video record participants’ hands and desk space, and record audio of participants verbalizing their thoughts and conversations in the immediate vicinity. The video recording of the desk space enabled the capture of paper-based artifacts that were used during interactions with the EHR. Data collected from observational techniques were analyzed by a biomedical informatics graduate student (BD) and an expert in human-computer interaction (DK.)

Process Mining Analysis: Two computer scientists specialized in clinical workflow modeling (AM, AG) analyzed the event logs generated at the JAX PreOp during the observed days, using process mining methods and tools. Process mining allows for the analysis of workflows based on event logs. Event logs are automated recordings of system events maintained for reasons including system checks, quality and safety. During process mining, analysts apply specialized data-mining algorithms to event logs in order to identify trends and patterns.(13) Process mining aims to improve process efficiency and understanding of processes.

The team had access to the event logs generated at JAX PreOp during the observation dates. Each event log contained information on: a) date, time and duration of the event, b) software used, c) identifier of the person responsible for the event, d) identifier of the patient to whose medical record is been accessed. In addition, the Mayo Clinic provided an event log dictionary that contained human-readable interpretations or each potential event log that could be collected. At the beginning, the process mining team retrieved the event logs for which we have the corresponding Morae^TM video recordings, to acquire a better understanding of how actions in the EHR were captured by event logs and explained in the event log dictionary.

Morae^TM, Microsoft Excel, Java, and Disco^TM were used for preprocessing and cleaning the EHR event logs. In Excel, event logs were sorted based on date and time. For uniformity, the Excel files were then cleaned and organized using the same Java program that was written and used for Grando et al. study. (3) The Excel files were imported into Disco^TM, a process mining and automated process discovery software that allows additional data cleaning and file conversion.(14) The output of Disco^TM are XES files that can be imported to ProM for analysis. ProM is an open-source software, considered the “de-facto” process mining software.(15) ProM contains hundreds of plug-ins (algorithms) for process mining analysis. Several ProM plug-ins were used to further refine the dataset for analysis and produce visualizations. Because the event logs contained personnel and patient identifiers, we were able to use process mining tools to analyze clinician or staff-based workflow and discover patient-based workflow.

Results

We conducted 12 clinician and key personnel interviews, totaling 10 hours. In addition, we completed six leadership interviews during almost six hours. In PreOp, we collected nine Morae^TM video recordings of eight Registered Nurses (RN) using the EHR totaling six hours, one Morae^TM video recording of a PreOp Team Leader RN (Lead RN) at the PreOp Nurse Station for four and a half hours, and one Morae^TM video recording of a Clinical Assistant (CA) at the PreOp Check-in Desk for three and a half hours. We also obtained Morae^TM video recordings of seventeen clinicians at the IntraOp and PostOp settings, totaling 44 hours.

The team had access to 116,706 event logs generated during seven days in the PreOp setting. For those days, there were 31 clinicians and supporting staff, and 375 patients. The event logs mainly captured actions performed with the SurgiNet system (107,392 logs.) SurgiNet is an EHR application used for tracking cases in the perioperative settings and for most perioperative documentation. In addition, there were fewer event logs created by other EHR applications: PowerChart (7,021 logs), PmdbDocs (2,213 logs), SchApptBook (41 logs) and PM Office (39 logs.) PowerChart is the clinician’s desktop solution for viewing, ordering and documenting in the EHR for a patient. Clinicians use this application to view and update the patient medical record. SchApptBook is designed to manage all outpatient appointment attendance details. It also manages the bookings of follow-up clinic appointments. This application does not focus on clinical content. PM Office is designed to manage patient elective bookings, waitlists and other clerical non-ward related activities. This application does not focus on clinical content. PmdbDocs is used for registration purposes, and also by nurses to print out patients’ face sheets and allergy bands.

By combining outcomes from the interviews, observational techniques and process mining methods, we were able to answer the following research questions in the context of JAX PreOp:

1. Personnel roles and their interactions, captured through interviews and observational techniques

In JAX PreOp setting, surgical patients and their family members interact with personnel in five roles for timely preparation for surgery. These roles are volunteer (Volunteer), CA, patient care technician (PCT), RN and Lead RN.

Volunteers: Are stationed at the volunteer desk in the 4^th Floor Surgery family waiting area. Volunteers facilitate communication between the patient’s family and clinicians during the patient’s perioperative care. They are also a resource to patients’ family members for availability of hospital resources (e.g., cafeteria, reading the Surgical Patient Tracking Board visible in the family waiting area.)

CA: For patients arriving at the hospital on the day of surgery, the CA is the patient’s first point of contact at the surgery check-in desk for the given floor. Additional responsibilities include communicating with the Lead RN about patients as needed, making phone calls to absent patients when advised by the Lead RN, and printing out and putting together paper charts for next day surgical cases.

PCT: Five PCTs work each day. They may assist with PreOp or PostOp care. In the PreOp setting, they assist with bringing patients into the PreOp space (from the waiting room or in-house patients from the hospital floor), help RNs get the patient ready for surgery (get patient’s height, weight and temperature, connect patient to monitors, assist patient to bathroom and dress in gown, and surgery specific prepping), and secure patient’s belongings. “Prepping” is specific to the patient surgical case and is ordered by the surgeon. It typically includes clipping hair and scrubbing with antiseptic cloths.

Lead RN : There are five Lead RNs among Perianesthesia Department staff. Lead RNs are responsible for managing the staff and patient workflow. If necessary, the Lead RN can serve as an RN caring for a patient; however, they are encouraged to remain in the Lead RN role and manage the flow. To manage staff and patient flow in PreOp, the Lead RN interacts with SurgiNet Perioperative Tracking to change the patient status, enter assigned PreOp bay and RN name, initiate request for PCT to bring a patient into PreOp. This role also relies on several paper-based information tools to manage patient flow and staff workload.

RN: On most days, nine RNs are staffed across 1^st Floor Surgery (PreOp and PostOp), and 18 RNs across 4^th Floor Surgery (PreOp and PostOp). There are multiple shift start times, supplying RN coverage from 05:00 to 21:00. RNs are cross-trained for all PreOp and PostOp care a patient may need. They may monitor patient load in their unit by looking at the SurgiNet Perioperative Tracking display on their individual workstations and on the shared displays in the PreOp and PostOp units.

2. Patient-based EHR-mediated workflow, captured through interviews and observational techniques

Table 1 provides a schematic summary of patient workflows in terms of the patient’s physical location in the PreOp setting, personnel involved in patient care, artifacts used (paper-based or electronic) and SurgiNet perioperative tracking status. The SurgiNet Perioperative Tracking Board status is the primary reference used by clinical and administrative personnel, volunteers, and patients’ family members to monitor patients’ progress. To better link the information provided in Table 1 with Figure 1, patient-based workflow steps are listed at the bottom of the table.

Table 1:

Observed correlation of patient’s physical location in PreOp setting, personnel involved, artifact used (paper-based or electronic), SurgiNet tracking status and patient-based workflow steps.

Patient’s Location	Waiting Room		PreOp Rm			Operating Room
Personnel/Role	CA	Lead RN	PCT	Lead RN	RN	OR RN
PreOp Location	Waiting Room	PreOp Nurse Station	PreOp Nurse Station; Waiting Room; PreOp Rm	PreOp Nurse Station	PreOp Rm; Pyxis Station	PreOp Rm; Hallway; OR
Artifacts Used	SurgiNet Case Selection; SurgiNet Tracking Board; Powerchart; Partial paper chart (PreOp checklist, labels, etc)	SurgiNet Tracking Board; SurgiNet (Orders, MAR, Clinical Notes); Partial paper chart (labels, H&P); Post-it note	Post-it note	SurgiNet Tracking Board SurgiNet; PowerForms; Full paper chart (PreOp checklist, consents, H&P, labels, etc); Post-it note; Pyxis	Full paper chart (PreOp checklist, consents, H&P, labels, etc)
PeriOp Tracking status (SurgiNet)	“Arrival Waiting Room” (grey)	“Picking up from Waiting Rm” (teal)		“In Preop” (pink)	“Preop RN Assess Comp” (pink)	“In Surgery” (green)
Other SurgiNet updates		“PreOp Rm#” “[RN name]”
Patient-based workflow steps	(1)	(2)	(3), (4)		(5), (6), (7)	(9)

Figure 1:

Patient-based EHR-medicated PeriOp workflow

Figure 1 describes the patient-based workflow steps presented in Table 1. The flow chart, expressed as a UML activity diagram, describes the workflow of a patient from being checked into the hospital until being ready for surgery:

Arrive at Hospital: Patients may come to PreOp from the emergency department, from a hospital floor, or may enter the hospital on the day of surgery and check in directly at PreOp. This patient-based workflow describes the latter.

(1) Register patient: The patient reports to the check-in desk at the surgery waiting area on the 1^st or 4^thFloor. The CA retrieves the patient’s prepared chart from a file rack of charts which holds prepared partial paper charts for all surgical cases planned for the day. The CA separates the ID wristband from other papers in the chart and then compares the patient’s medical record number (MRN) and the financial number (FIN) printed on the wristband to those on the printed patient labels. The patient is asked to verify the patient’s information on the wristband. To check the patient into the system, the CA enters data into a practice management screen that is accessed through the SurgiNet Case Selection screen. The CA changes the patient’s status in SurgiNet Perioperative Tracking Board to “Arrival Waiting Area” (grey) to inform other PreOp personnel that the patient has arrived. The CA’s last steps are to open PowerChart to confirm that the patient is checked in, apply ID wristband to the patient’s wrist, give the patient and family the Patient Case Card with a de-identified tracking number that the family can use to reference the Tracking Board display in the waiting area and the Contact Card which they are instructed to fill out and provide to a volunteer.

(2) Assign PreOp Room and RN: The PreOp Lead RN, monitoring the Perioperative Tracking display from the desk at the PreOp nurse station inside the PreOp unit, sees the indication that the patient is in the waiting room. The display most often remains visible on the person’s desktop monitor. If there are specific anesthesia needs for the patient—; such as a regional block—that would affect resource assignment, this would be communicated to the Lead RN from anesthesia on the Tracking Board. When the Lead RN is ready to room the patient, the Lead RN retrieves a partial patient paper chart (i.e., patient stickers, history & physical printout (H&P)) and a post-it note with a patient label on it from a stack of similar partial patient charts prepared for all surgical cases that day, and writes the assigned PreOp bay number on the post-it note and on the sheet of labels. In the SurgiNet Tracking Board, the Lead RN changes the patient’s status to “Picking up from Waiting Rm”, selects the room and types the name of the assigned RN. Then the Lead RN opens the patient chart from the Perioperative Tracking Board display to initiate the surgeon’s orders (in Orders screen), print off the patient medication list (from medication administration record (MAR) screen), and, if an H&P was not already printed for the patient, to print the recent H&P (from Clinical Notes screen). During this process, the Lead RN notates patient name, room number and RN name on two paper documents, which are maintained throughout the day to manage patient flow, staff resources, and to maintain awareness of patients who are still to go through PreOp. The Lead RN signals to the PCT that a patient needs to be brought from the waiting room to their PreOp room by placing the post-it note with the patient label on the front of the PreOp nurse station desk. Lastly, the Lead RN takes the patient labels and H&P printout to the assigned PreOp RN.

(3) Bring patient to PreOp area: The PCT carries the post-it note with the patient label to the check-in desk to retrieve the partial patient chart, then to the waiting room to collect the patient. Before escorting the patient to the assigned PreOp Room, the PCT verifies that he/she has the correct patient by making sure that the patient name, date of birth, and ID number on the post-it note matches the patient’s wristband and what the patient states.

(4) Prepare patient: On the way to the assigned PreOp bay, the PCT records the patient’s height and weight on the post-it note. At the patient’s PreOp bay, the PCT records the patient’s temperature on the post-it note. The PCT gives the post-it note to the RN to document these vital signs. The PCT or RN then instructs the patient to change into the hospital gown and, if needed, use the antiseptic wipes. Once completed, the RN, occasionally aided by the PCT, connects the patient monitoring devices to gather additional information about the patient’s current state. The PCT often assists with additional surgical preparation tasks ordered by the surgeon, such as clipping hair.

(5) Verify patient, procedure and consent forms: The PreOp RN verifies the patient’s identity, procedure, surgical site identified, accuracy of the ID wristband and allergy status, and that the correct paper consent forms are complete. The RN verifies the information from the preoperative evaluation clinic H&P (including meds), and gathers additional information needed from the patient for chart review and assessments.

(6) Complete orders: The RN completes any outstanding PreOp orders listed in SurgiNet—such as initiate intravenous (IV) access, place patient stockings and compression cuffs, administer medications retrieved from the Pyxis machine, and deliver patient education.

(7) Complete charting and documentation: The RN enters data across multiple windows in SurgiNet and PowerForm to complete PreOp documentation to include chart review, skin assessments, IV access, patient education, and medication administration. Only the PreOp RN completes patient charting in the electronic systems. When the RN completes the PreOp assessment, the RN signals to others by finalizing the perioperative document in SurgiNet, which causes the patient’s status in the Tracking Board to change to “Preop RN Assess Complete”.

(8) Sign off: Before the patient can be transported to the OR, the Stop Sheet needs to be completed. The Stop Sheet is a paper checklist with tasks that have to be initiated by the RN, OR circulating RN (OR RN), anesthesiologist, and surgeon. These other individuals may stop by as they are able while the RN is completing the nursing tasks, or after. The care team sign off task can occur at any point after (3) and in unison with (4) - (7). However, (8) must occur before the patient can be transported to the OR for surgery (9).

(9) Transport to OR: The OR RN or the Certified Registered Nurse Anesthesiologist (CRNA) get the patient from the patient’s PreOp bay when the surgical team is ready to transport the patient to the OR. There is no face-to-face handoff between the PreOp RN and IntraOp personnel. The OR RN enters “time in” operating room in SurgiNet documentation, and this changes the patient’s status in SurgiNet Perioperative Tracking Board to “In Surgery” (green). Ready for surgery: the patient is in the OR and ready for the procedure.

3. Social network based on personnel’s interactions with patient cases, captured through observational techniques and process mining

In this section, we analyze team interactions with patient cases at JAX PreOp using the event logs in the context of the patient-based workflow sequence and locations described above. In particular, we employed process mining to discover patient-based workflows that include information on the personnel roles associated with each event.

Dotted chart analysis: was conducted using the dotted chart ProM plug-in. The dotted chart visualization in Figure 2 shows the sequence of personnel’s interactions for each patient case for Day 2. The x-axis represents time, while the y-axis represents the different patient cases. Each colored dot depicts an event in the log, while the color of the dot corresponds to the role of the person that created the event. Based on this chart, the majority of patient-based workflows start with an event generated by the CA, followed by a few events generated by the Lead RN, which are then followed by a larger number of sequential events generated by the RN. This is consistent with the observed workflow. Similar sequential interactions were identified based on the dotted charts for the other days.

Figure 2:

Dotted chart for Day 2. The x-axis is divided into hours of the day and each row represents a patient. Each colored dot corresponds to a log file event from personnel in PreOp. The colors are based on personnel roles.

The dotted chart in Figure 2 also conveys the duration of each patient-based workflow. Note that we have filtered out patient cases that had duration shorter than 15 minutes because PreOp workflow takes considerably longer, thus we assumed that, in these instances, personnel were accessing or reviewing information for patients not scheduled for surgery that day. The majority of patient cases depicted in Figure 2 include a large number of events that occurred across several hours. However, there are a few patient cases that include only a single or very few events, typically generated by the CA. These cases likely reflect the CA’s brief EHR interactions to print patient-specific documents when preparing paper charts for the next day’s surgical cases.

Social network analysis: was conducted using the social network ProM plug-in. This analysis provides an insight on the relative frequency of role-to-role interactions with a patient case, as illustrated in Figure 3. Circles in a social network correspond to roles, with circle size denoting the number of event logs associated with the role. Arrows correspond to directed interactions between two roles, and arrow thickness denotes the number of interactions. In Figure 3, the social network shows that generally all the roles reciprocate interactions to one another, except for the PCT who only interacts with the RN. The most common interactions occurred from the CA to the Lead RN and from the Lead RN to the RN, which is consistent with the observed behavior. The next most common interactions occurred from the RN to the Lead RN, from the Lead RN to the CA, as well as back-and-forth between the CA and the RN. In addition, consistent with observations, there were considerably fewer event logs generated by PCTs as they generally assist with non-documentation related care.

Figure 3:

Clinician and staff social network. Circles correspond to roles, and larger circles indicate that the role generated a large number of event logs. Arrows correspond to interactions, and thicker arrows indicate that an interaction occurred more frequent

4. Time spent by personnel interacting with the EHR, captured through process mining

To answer this question, we analyzed 116,706 event logs generated by electronic information systems used in JAX PreOp, namely Surginet, PowerChart, PmdbDocs, SchApptBook and PM Office. Similar to our previous study on PreOp workflow in PHX (3), we focused our analysis on personnel in the PreOp room (i.e., RNs) and we calculated two metrics: i) absolute time spent by each RN interacting with the systems for each patient and ii) proportional time spent by each RN interacting with the systems for each patient over the entire stay of that patient in the PreOp room. In order to calculate the first measure, we first discovered all patient-based workflows for each RN on each day. By visualizing the workflows with the use of dotted charts, it became clear that there were different activity groups in the workflows. In other words, there were several events with close time proximity, but at the same time there were also relatively large time gaps between these groups of events. We attributed gaps of 5 minutes or more to the RN not using the EHR (e.g., interacting with the patient) and, therefore, we considered gaps of this length to indicate the beginning of a new activity. For each patient-based workflow, we identified the different activity groups and we calculated their duration by subtracting the start time of the first event log in an activity group from the end time of the last event log from the same activity group. By adding the durations of all activity groups in each patient-based workflow, we were able to identify the total time spent by the RN using the EHR for that patient.

The dotted chart and the list of times for each patient workflow included 9 cases where the RN used the EHR for less than 1 minute over the course of the five days. This is an indication that these patient cases were for individuals not scheduled for surgery on any of the days of interest and, hence, these cases were removed. For the remaining patient based workflows, we found that the average amount of time an RN spent interacting with the patient’s chart using electronic systems was 25 minutes and 6 seconds.

Calculating the second value requires measuring the time that the patient stayed in the PreOp room. Since this information was not directly available from the event logs, we assumed that this can be approximated by the difference between the first and the last event log from the RN with respect to the patient of interest, which we will refer to as “overall time in the system” for the remainder of this paper. By using the first metric and the overall time of the RN in the system for each patient-based workflow, we were able to identify the proportional time spent by each PreOp personnel interacting with the EHR for each patient. We found that, on average, the amount of time a patient spent in the PreOp Room was 1 hour and 40 minutes, and 32.2% of that time was spent on average by the RN interacting with the patient’s chart using electronic systems.

5. Time spent by personnel documenting and/or reviewing data using paper-based artifacts vs. using EHR, captured through observational techniques

Morae^TM video analysis was utilized to identify how much time personnel spent per patient reviewing and/or documenting with paper artifacts. Over the seven video recordings, the RN spent an average of 1 minute and 42 seconds using paper-based artifacts per patient. The average time that the RNs were observed to interact with the EHR was 31 minutes and 56 seconds. The remaining time (10 minutes and 7 seconds) was not spent using paper-based artifacts or interacting with the EHR.

6. Comparisons between PreOp settings at Jacksonville and Phoenix

Similar location-based patient flow: When comparing the JAX PreOp workflow (Table 1) to the PHX PreOp workflow (3), we can see that the patient moves through similar locations in the same sequence—from the surgery waiting room to the PreOp room to the operating room.

Similar EHR applications and different paper artifacts: Both sites use SurgiNet Perioperative Tracking Board and other SurgiNet screens, and a paper patient chart to include patient labels and consent forms. However, the sites differ in which paper artifacts they use (e.g., PreOp checklist, post-it note, and purple paper.)

Different physical organization: In JAX, the Perianesthesia Department has integrated the staff of their two surgical areas, 1^st Floor Surgery and 4^th Floor Surgery. PHX has not integrated staffing of their two surgical areas. Therefore, our previous analysis at PHX was restricted to the PreOp area that handles most of the acute cases; did not include the PreOp area that handles most of the ambulatory cases.

Different use of PeriOp tracking system: Some of the SurgiNet Perioperative Tracking Board patient status labels used in the two sites are common (i.e., “Arrival Waiting Room”, “In Preop”, and “RN Assess Comp”), while others are different (i.e., “Family” is used in PHX but not in JAX, and “Picking up from Waiting Rm” is used in JAX but not in PHX.)

Different personnel roles: The same care is delivered at JAX and PHX PreOp sites, but by individuals of different roles. JAX PreOp has Volunteer, CA, PCT, RN, and Lead RN. PHX has health unit coordinator (HUC), patient care associate (PCA), pre-operative nurse (RN), float nurse (Float RN), Desk nurse (Desk RN), and Team Leader nurse (Lead RN.) The PCT role in JAX is very similar to the PCA role in PHX. The distribution of responsibilities differs for the remaining roles. For example, the Lead RN in JAX is more similar to the Desk RN role in PHX than to the Lead RN role in PHX. Furthermore, according to the JAX Perianesthesia Nurse Manager, the CA role is about 75% similar to a unit secretary or HUC and 25% similar to a PCT or PCA.

Different role-patient and role-HIT interactions: Differences in personnel roles lead to different patient and EHR interactions. For example, in PHX PreOp, the HUC, Desk RN, and RN interact with the SurgiNet Tracking Board, while in JAX the CA, Lead RN, and occasionally the RN interact with the SurgiNet Tracking Board. In PHX, the HUC and the Desk RN usually generate the first EHR interactions, which are often followed by interactions from the RN. In JAX, the CA usually generates the first EHR interactions, which are followed first by EHRs interactions from the Lead RN and later by interactions from the RN.

Different social network: The differences in personnel roles and workflow are seen in the social networks. In PHX, generally all the roles reciprocated interactions to one another, except for the Lead RN who only interacted with the Desk RN. At JAX, generally all the roles reciprocated interactions to one another, except for the PCT who only interacted with the RN. In PHX, the most common interaction occurred from the HUC-to-Desk RN and the back-and-forth between the Desk RN and the RN. The next most common interactions occurred from the Desk RN-to-HUC, back-and-forth between the HUC and the RN. In JAX, the most common interactions occurred from the CA-to-Lead RN and from the Lead RN-to-RN. The next most common interactions occurred from the RN-to-Lead RN-to- CA, as well as back-and-forth between the CA and the RN.

Different time spent by RNs interacting with the EHR: An RN in JAX PreOp typically had less documentation to complete compared to RN in PHX because approximately 80% of surgical patients in JAX are evaluated at the PreOp evaluation clinic prior to their day of surgery. This was reflected in our findings—the average amount of time an RN spends interacting with the patient’s chart using SurgiNet was larger in PHX compared to in JAX.

Different time spent by RNs using paper-based artifacts: in PHX, RNs spent an average of 6 minutes and 4 seconds using paper-based artifacts per patient, while in JAX they spent an average of 1 minute and 42 seconds.

Discussion

By combining outcomes from interviews and observations, we obtained high definition descriptions on JAX PreOp workflows, which were validated with larger data sets through the use of process mining. EHR workflow is a complex multi-dimensional phenomenon in that it involves inputs from several agents and is situated in complex dynamic environments. No single method can capture workflow in all its dimensions. Some of the authors have previously proposed an improved solution for workflow discovery and monitoring that consists of integrating quantitative with qualitative methodologies. (1-5) We applied this methodological approach to the study of JAX PreOp workflows.

Important contributions of this research include providing the Mayo Clinic data analysis to inform the EHR conversion process, and introducing and testing a well-grounded multi-method approach. The proposed method will be used to compare workflows before and after the new EHR implementation to understand its impact on health care delivery. This is important given that EHRs can support but also inhibit collaboration (16). For instance, the nodes of the discovered social network (Figure 3) provide quantifiable information on which PreOp roles interact the most with the EHR and how heavily they could be impacted by the conversion to the new EHR. In addition, the strength of the arcs on the social network provide data on the frequency of role-based EHR-mediated collaborations. We will apply the same methodological approach before and after the EHR conversion to provide valuable insights on how personnel communication and coordination are affected. In addition, the metrics that we computed on time spent by the patient in the PreOp setting, and time spent by personnel using paper-based artifacts and the EHR will help to quantify the impact that the new EHR will have on time-related effectiveness metrics and documentation. Also, the patient workflows discovered at JAX PreOp (Figure 1) and PHX PreOp (3) could be compared against the workflows resulting after the new EHR implementation to understand workflow convergence across sites after adoption of the same EHR. The high definition snapshots of EHR workflow can be used to chart course corrections, for example, if there are breakdowns in communication, bottlenecks in workflow or frequent use of costly workarounds.

In addition to studying JAX PreOp workflows, the outcomes from this study were used with the outcomes from Grando et al. research (3) to conduct preliminary comparisons between PHX and JAX PreOp settings. This is novel research, given that there is a lack of studies on comparison and standardization of clinical workflows. (9-12) While both facilities share the same EHR applications, we noticed major differences in physical organization of surgical areas and personnel roles. Differences in personnel roles led to differences in personnel interactions with the patient and the EHR, clinician/staff social network, and time spent using EHR and paper-based artifacts. Our findings reflect the potential challenges of integrating a single EHR into multiple PreOp settings with different clinical workflows. Though provisional, the outcomes of our study suggest that the conversion to the same EHR could benefit from the standardization of personnel roles, which in turn could affect clinical tasks, EHR-based workflows, personnel and patient interactions, paper-based artifacts, social network, time metrics and workflow effectiveness.

A limitation of the comparative study is that the discovered PHX workflows were based on data collected only from the PreOp area that handles most of acute cases, and did not include most of the ambulatory cases. In contrast, our analysis of JAX workflows included all the surgical cases. These differences in data capture could have impacted the comparative analysis. As future work, we would like to conduct observations at PHX focused on the PreOp settings that were not observed. We are in the process of analyzing interview data, to contrast objective workflow analysis outcomes with stakeholders’ perceptions to better inform comparative analysis.

Given that we discovered significant differences across two geographical locations within a single health system, more in-depth comparisons are planned in order to inform Mayo Clinic’s workflow standardization efforts required for the conversion to a new EHR. In addition, future work will attempt to map workflow components discovered at JAX and PHX PreOp sites, into a work domain ontology to specify workflow-based clinical applications. The use of the ontology could help to define semantics and semantic closeness of workflow components in terms of standard clinical terminologies, nomenclatures or ontologies. We also plan to use the ProM tool to map the discovered PreOp workflows into a mathematical formalism, called Petri Net, to attempt to obtain precise process comparisons. (17)

Conclusions

Our research team has applied the same multi-method workflow analysis approach to two similar PreOp settings at different Mayo Clinic geographical locations. The main contribution has been providing initial data on differences and similarities of PreOp workflows between two sites and to guide future research on workflow standardization.