Using a medical simulation center as an electronic health record usability laboratory

Abstract

Usability testing is increasingly being recognized as a way to increase the usability and safety of health information technology (HIT). Medical simulation centers can serve as testing environments for HIT usability studies. We integrated the quality assurance version of our emergency department (ED) electronic health record (EHR) into our medical simulation center and piloted a clinical care scenario in which emergency medicine resident physicians evaluated a simulated ED patient and documented electronically using the ED EHR. Meticulous planning and close collaboration with expert simulation staff was important for designing test scenarios, pilot testing, and running the sessions. Similarly, working with information systems teams was important for integration of the EHR. Electronic tools are needed to facilitate entry of fictitious clinical results while the simulation scenario is unfolding. EHRs can be successfully integrated into existing simulation centers, which may provide realistic environments for usability testing, training, and evaluation of human–computer interactions.

Introduction

Electronic health records (EHRs) are becoming increasingly common in hospitals and ambulatory practices.^{1 2} While most evaluation studies have demonstrated positive effects of health information technology (HIT),³ some studies have identified unintended consequences after the introduction of HIT (eg, increased time for antibiotic delivery after a hard stop during order entry, misreported data, and information loss).^4–7 Healthcare providers have also expressed concerns that HIT is not easy to use and increases their workload.^7–9 As a result, increased attention is being devoted to making HIT both safer and more usable.^{10 11}

Usability evaluations performed by vendors/developers and researchers before, during, or after implementation can enhance the safety, efficiency, and user satisfaction of EHRs.¹² To conduct HIT usability evaluations with users, researchers are increasingly using medical simulation to standardize clinical scenarios, simulate specific scenarios, and collect usability metrics, as these tasks are difficult if not impossible to perform in the clinical environment.^13–22 Several research groups have created HIT usability laboratories to perform usability evaluations^{12 23}; however, these dedicated facilities require large resource investments solely for HIT usability evaluation.

Over 130 US health care institutions have developed medical simulation centers to train and evaluate health professionals.^24–27 Many of these centers have made large investments in physical space and expert staff. Some medical device manufacturers already use simulation centers as part of their product development processes, testing human factors, device features, and interaction with the clinical environment.²⁸ Medical simulation centers can also be used as HIT usability laboratories, contributing to new product development, vendor content configuration, end user training, and evaluation.^{21 22}

In order to study HIT usability and safety in medical simulation centers, EHRs must be integrated into them. However, previous reports have only described the use of virtual EHRs that have been specifically designed for the simulation environment.^{29 30} As part of a project to understand how emergency physicians use electronic documentation, we describe our approach and experience setting up, configuring, and using an actual emergency department (ED) EHR in a medical simulation center.

Case report

Setting

We partnered closely with our institution's multidisciplinary medical simulation center which is comprised of a variety of high-fidelity simulation suites, including two patient examination rooms, an operating room, conference and debriefing rooms, and virtual reality and technical skills laboratories. The simulation center is staffed by both simulation specialists and experts in simulation education. It is regularly used to train medical students, residents, and other hospital staff on resuscitation, critical care, team training, invasive procedures, and core content in a safe environment that replicates the clinical setting. The center is located on the hospital campus and is easily accessible by hospital staff. Our local institutional review board exempted this project from review.

Planning

We assembled an interdisciplinary team with expertise in clinical informatics, medical simulation, medical education, emergency medicine, and usability testing. Working closely with these experts in medical education and simulation, we designed a clinical scenario representative of a typical ED visit; the scenario was intentionally straightforward as the goal was to study emergency physician use of electronic documentation, not to assess medical knowledge or technical skills. The clinical scenario developed was that of a 55-year-old man with a history of atrial fibrillation on warfarin who presented to the ED with a headache after being hit in the head by a softball (see online supplementary appendix A). To increase the fidelity, we incorporated an interruption by a nurse requesting the participant to review an electrocardiogram for another ED patient, representing parallel processing that is typical in the ED.

Four research staff helped run the simulation scenario. A physician-actor played the role of the patient using a standardized script (see online supplementary appendix A). One research analyst facilitated and moderated the session. A second research analyst took notes during the encounter and played the role of nurse interrupter. A simulation center technician assisted with session setup and audiovisual recording.

Physical space

We configured the physical space of the simulation center to represent a typical ED examination room and used portable room dividers to delineate space inside and outside the patient room. Study staff monitored the session from a control room, separated from the examination room by one-way glass (figure 1).

Figure 1.

Adjacent simulation center control room allows research analysts to observe clinical simulation and collect data with usability software.

Computers

Our clinicians often use workstations on wheels (WOWs) in the ED. We used a similar WOW running the same software—Microsoft Windows XP and Microsoft Internet Explorer 8 (Microsoft, Redmond, Washington, USA) for the web-based ED EHR (figure 2). At their discretion, participants could freely move the WOW into the examination room for documenting at the patient bedside or leave the WOW outside while they performed their history and physical examinations.

Figure 2.

Participant gathering patient history and entering electronic clinical documentation using a workstation on wheels during a simulated patient encounter.

Electronic health record

We integrated the custom-developed, web-based ED EHR used in one of our EDs into the simulation scenario, focusing on the patient tracking and electronic physician documentation features. We used the quality assurance (QA) environment of the ED EHR, a fully functional version of the production system but without real patient data, that is primarily used by the information systems (IS) team to test new releases. Given this existing EHR testing environment, minimal IS resources were required to configure the QA EHR in the simulation setting. While there were no direct IS costs to our project, we needed to coordinate with IS to ensure concurrent use of the QA environment would not interfere with development activities and that the functionality would be the same as the current production system. The IS team also granted temporary access to the QA environment to research analysts and simulation session participants. The research analysts then created fictitious test patients with chief complaint and triage data specific to the scenario (see online supplementary appendix A) in the ED EHR (figure 3A).

Figure 3.

(A) Participant reviews simulated patient's nursing triage note in the emergency department (ED) electronic health record (EHR) (captured using Morae Recorder). (B) Participant enters history of present illness in the ED EHR (captured using Morae Recorder).

Computerized provider order entry (CPOE), electronic result review, and clinical document review are also available in our ED EHR, but handled by separate enterprise applications. Making these applications available for the simulated patients required additional coordination with enterprise application teams. Since our primary objective was to understand how emergency physicians used electronic documentation, we did not invest the additional resources to enable CPOE and results review features for the simulation encounter. Participants were therefore instructed to verbalize all orders (medications, laboratory, and imaging), the standard practice during simulation training in our facility. Further, there was no method for simulation staff to enter laboratory or imaging results into the EHR during the session. Consistent with current simulation center practice, Microsoft PowerPoint slides with laboratory and imaging results were displayed on a liquid crystal display monitor in the simulation room for participants to review.

Participants

We recruited participants from our emergency medicine residency program, offering a US$50 incentive payment. Participation was voluntary and did not impact performance evaluations or employment. All participating resident physicians had substantial experience with the ED EHR (residents use this EHR in their clinical practice) as well as the simulation environment (residents participate in biweekly simulation training as part of their residency training).²⁵

Simulation scenario

Participants received a short, scripted orientation from research analysts explaining the logistics of the session and were then given 20 min to perform a complete ED visit, including history taking, physical examination, diagnostic testing, and disposition, as well as electronic documentation. All documentation was completed on the ED EHR system running on the WOW. Participants were permitted to proceed with patient evaluation as they deemed appropriate. The session was terminated when the participant completed clinical documentation or after 20 min, whichever endpoint was reached first.

Data collection

We collected multifaceted data on the participants’ use of the ED EHR during this session. The medical simulation environment enabled the research analysts to observe each simulation session from the control room, taking notes on the participant actions, workflow, and challenges using the ED EHR. We installed Morae Recorder (TechSmith, Okemos, Michigan, USA) onto the WOW to capture participant screen actions and interactive behavior in real time (figure 3B). Morae Observer was installed on a control room workstation, which allowed our trained research staff to monitor the participant's actions on the WOW and to record key interactions with the EHR interface.

Since simulation scenarios often need to be recorded for later debriefing, our medical simulation center is equipped with a state-of-the-art audiovisual system. We used existing wireless microphones and multiple cameras to record audio and video of the sessions for subsequent review.

Pilot testing

We performed a dry run with our project team and then pilot tested the entire simulated scenario, including the EHR and audiovisual equipment, with a physician participant. These pilot tests, well-established simulation best practices,^{31 32} identified issues with displaying laboratory and imaging results and WOW network connectivity that were resolved before additional physicians participated.

Results

Resident physicians successfully completed this simulation scenario and ED EHR electronic documentation. Each session generated quantitative and qualitative data on the participant's workflow and use of the ED EHR that will be analyzed to understand and improve ED EHR electronic documentation features.

We summarize the key tasks and resources needed to set up and use an EHR in a medical simulation center clinical scenario in table 1. We also held study team debriefings after every participant session, collecting feedback on operations and fidelity. We compiled cumulative lessons learned into a pre-simulation checklist (box 1) that was used before each simulation session to help ensure all systems were operational and previous missteps were not replicated.

Table 1.

Timeline with key resources for using an electronic health record in a medical simulation center clinical scenario

Project stage	Task	Key required resources* (people and equipment)
Planning	Formulate operational or research question
	Establish partnership with simulation center and IS teams	Simulation leadership, IS leadership
	Understand available simulation center resources (ie, facilities, technicians)	Simulation staff
	Understand available IS resources (ie, EHR test environments)	IS analysts
Simulation configuration	Design the clinical simulation scenario, including standardized test script and participant instructions	Clinical education experts, simulation staff
	Identify staff required to run the scenario, including actors, project staff, and simulation center technicians	Simulation staff
	Identify simulation center physical space and props needed for scenario	Simulation staff
	Plan for how tests will be ordered and results viewed, within the scenario	Simulation staff, medical education experts
	If audio/video recording is needed, consider using simulation center resources (if available)	Simulation staff
Information systems configuration	Plan for computer hardware in the simulation center (use existing or add new)	Simulation staff, computer workstation
	Identify and coordinate use of an EHR test environment	IS analysts, EHR test environment
	Obtain access to the EHR test environment for study staff and participants	IS analysts
	Configure EHR for scenario, if needed
	Consider installation of specialized software to capture all participant interactions with the EHR	Usability testing software
	Test configuration and operation of computers, EHR, and usability testing software in simulation center
Pilot testing	Dry run with project team	Simulation staff
	Pilot test	Simulation staff, patient actor, participant
Execute session	Recruit participants	Participants
	Consent participants
	Train participants on simulation setting and EHR, as needed
	Perform pre-simulation session checklist (box 1) to verify EHR, AV equipment, and scenario are ready	Simulation staff
	Execute scenario	Simulation staff, patient actor, participant

*Key resources also include one to two research analysts who will participate in all tasks.

AV, audiovisual; EHR, electronic health record; IS, information systems.

Box 1. Pre-simulation session checklist.

1. Prepare and test emergency department (ED) electronic health record (EHR) in simulation center

   1. Boot workstation on wheels

      1. Login
      2. Open ED EHR application (verifies successful network connection)
   2. Select and confirm patient in ED EHR tracking board to be used for scenario
2. Prepare and test audiovisuals

   1. Portable microphones for participant and interviewer
   2. Preset camera views/angles
   3. Test and queue PowerPoint presentation with laboratory and imaging results to project into simulation room after ordered by participant
3. Set up Morae Recording environment

   1. Install Morae Recorder module on participant workstations
   2. Install Morae Observer module on control room workstation
   3. Connect Morae Observer and Recorder via network
   4. Confirm connection and live recording
4. Prepare simulation

   1. Moulage standardized patient with hat and make-up to simulate head injury from softball game
   2. Ensure roles are known by standardized patient and nurse confederate
   3. Assemble necessary props (eg, stethoscope, otoscope, penlight, etc)
   4. Prepare paper electrocardiogram for interruption
   5. Position room dividers

Discussion

We successfully integrated an actual ED EHR into our medical simulation center, creating an EHR usability laboratory. Our methods and experiences serve as a model for others seeking to integrate real EHRs into their simulation centers and sessions. Meticulous planning, using existing facilities and infrastructure, and strong partnerships with clinical educators, medical simulation experts, and IS will facilitate integration of the EHR into the simulation setting. Once established, simulation centers with EHR capabilities have numerous applications, including workflow analysis, testing and evaluating clinical systems, and training clinical staff on IS.

Using our QA environment allowed us to rapidly integrate the EHR clinicians use in clinical practice into the medical simulation center. A similar approach can be taken by other simulation centers looking to add to their facilities’ EHR regardless of system type (home-grown, vendor, modular, or enterprise). IS routinely creates and maintains different versions or environments of their system such as development, testing/QA, training, and production. Working with the institution's IS team, simulation centers can create independent, dedicated simulation EHR environments or use an existing application environment, as we did.

Limitations

This simulation was not a formal evaluation or validation of the EHR in the simulation environment. In this case report, we did not analyze workflow or usability data on the participants’ use of the EHR during the simulated scenario. Participants had considerable experience with both the EHR and simulation environment. Alternative approaches, such as additional training time, may be needed if participants are unfamiliar with simulation or the system being tested. While we used a custom-developed ED EHR system, the general principles we described for integrating an EHR into a simulation scenario apply to any EHR system. Additional resources may also be required for more complex simulation scenarios and for institutions without comprehensive medical simulation centers and robust EHR testing environments. Finally, an important challenge is the inability of simulation scenario facilitators to electronically enter results in the simulated patients’ electronic records. Tools are needed to allow manual entry (or predefined automatic entry) of fictitious clinical results (such as laboratory and imaging results) to the patient record in real time while the simulation scenario is unfolding.

Conclusion

Medical simulation centers can be used as EHR usability laboratories. We described one approach to integrating clinical IS into a medical simulation center using the QA environment of our EHR and working closely with our IS and simulation center teams. With greater interest in EHR usability and simulation, challenges with real-time simulation results entry and review are likely to be overcome. With the integration of real EHR systems, medical simulation centers can become usability laboratories to facilitate analysis of clinician workflow, test and evaluate IS, and train clinical staff on IS.