Using an integrative mock-up simulation approach for evidence-based evaluation of operating room design prototypes

Abstract

This paper describes the process and tools developed as part of a multidisciplinary collaborative simulation-based approach for iterative design and evaluation of operating room (OR) prototypes. Full-scale physical mock-ups of healthcare spaces offer an opportunity to actively communicate with and to engage multidisciplinary stakeholders in the design process. While mock-ups are increasingly being used in healthcare facility design projects, they are rarely evaluated in a manner to support active user feedback and engagement. Researchers and architecture students worked closely with clinicians and architects to develop OR design prototypes and engaged clinical end-users in simulated scenarios. An evaluation toolkit was developed to compare design prototypes. The mock-up evaluation helped the team make key decisions about room size, location of OR table, intra-room zoning, and doors location. Structured simulation based mock-up evaluations conducted in the design process can help stakeholders visualize their future workspace and provide active feedback.

INTRODUCTION

Operating rooms (ORs) present high-risk to physicians and other clinicians. Thus, their design requires a deep understanding of the roles of clinical stakeholders, critical care processes, technology, and equipment utilized during care as well as the myriad technical aspects of the healthcare building process itself. A recently published literature review highlights a range of built environment problems in ORs including contaminated surfaces, inadequate workspaces, poor adjacencies, trip and bump hazards, poor furniture and equipment ergonomics, loud noises and uncomfortable work environments (Joseph, Bayramzadeh, Zamani, & Rostenberg, 2017). These environmental factors contribute to a range of adverse outcomes during surgery including injury to staff, flow and task disruptions during surgery and surgical site infections.

Many of these problems can be avoided or mitigated through a careful and shared understanding of the work systems that includes the organization, people in the space, required tasks, technology and equipment, and the built environment. The built environment is a critical component of the healthcare system and has the potential to impact patient and staff safety and quality of care. Decisions made during the healthcare facility design process have the potential to create latent conditions that may adversely impact work practices of clinicians and contribute to adverse outcomes (Joseph & Rashid, 2007; Reason, 2000). The 2014 Guidelines for Design and Construction of Hospital and Outpatient facilities recommends a safety risk assessment (SRA) as part of any healthcare project (Facility Guidelines Institute, 2014). The SRA is intended to be a multi-disciplinary process that engages team members (including architects, clinicians, patient advocates, and risk managers) in a discussion about key design decisions that may have an adverse impact on outcomes such as patient falls, healthcare acquired infections, and medical errors (Taylor, Joseph, Quan, & Nanda, 2014).

It is absolutely critical for healthcare facility design teams to proactively evaluate the impact of key design decisions before facilities are built to avoid unsafe and unhealthy spaces that are harmful for patients and staff (Reno et al., 2014). However, it is hard for clinicians to imagine these complex interactions in a future state through reviews of building plans and perspectives that are typically used to communicate proposed design to end-users. While there are many different ways of engaging teams during the design process, evaluations of physical mock-ups have been shown to be particularly effective in identifying and remediating potential safety concerns (Taylor et al., 2014), as they improve understanding and communication between healthcare providers and designers (Keys, Silverman, & Evans, 2016).

As such, mock-ups are increasingly being used in the healthcare facility design process to support and validate design decisions. Mock-ups are defined as full-scale models of a design that are used for teaching, demonstration, evaluation, or other purposes to enable testing of a design (HQCA, 2016). Simulation-based mock-up evaluation involves testing various aspects of a proposed design by simulating clinical scenarios or enactment of relevant clinical tasks (HQCA, 2016).

The Health Quality Council of Alberta (HQCA) has published a framework for conducting simulation based mock-up evaluations during the healthcare facility design process (HQCA, 2016). The HQCA framework puts forth a systematic approach for collecting and analyzing data from full scale- mock-ups using simulation. This framework includes recommendations on timing and planning for the mock up, construction of the mock-up, development of scenarios, running the evaluations as well as analyzing the evaluations to obtain evidence-based feedback for design. However, this framework does not include specific tools or methods for conducting the evaluation or for developing design recommendations to support an iterative design process.

The current study builds upon the HQCA framework to develop protocols and evaluation tools that were utilized while conducting simulation-based mock-up evaluations to test a range of design ideas. This work was conducted as part of a four-year multidisciplinary patient safety-learning lab focused on designing safer and more ergonomic ORs. The findings from this learning lab, including the prototype OR design, are informing the development of two new ambulatory surgical centers (pediatric and orthopedic) for the health system partner on this project. The purpose of this paper is to present the protocols and tools developed to evaluate prototype OR designs and to present the findings from this process.

METHODS AND PROCEDURES

To evaluate OR design prototypes, an evaluation toolkit was developed based on the mock-up evaluation framework from the Health Quality Council of Alberta (HQCA, 2016). The HQCA framework includes six guiding principles:

1. A simulation-based mock-up evaluation should be considered, and if applicable, planned, as part of the pre-design stage for inclusion in the design stage.

2. The mock-up evaluation should be thoroughly planned to maximize effectiveness.

3. Building of the mock-up should align with evaluation timing and objectives.

4. Roles and responsibilities for those involved in the evaluation should be clearly defined.

5. The simulation scenarios that are created and enacted should test the evaluation objectives.

6. Recommendations should be informed by evidence-based data from scenario enactments.

The HQCA framework was expanded and customized for the current project based on the specific objectives around OR design. The mock-up evaluation protocol was also designed to facilitate the maximum involvement and input from the surgical teams that might eventually use the spaces being designed. The key steps that were undertaken based on the HQCA framework include:

• Timing and planning

• Tool development

  • Overall goals

  • Master protocol

  • Note Taker’s template

  • Simulation director guide

• Constructing the physical mock-up and conducting the simulations

• Evaluation: Conducting the mock-up evaluation with clinicians

Timing and Planning

The research findings from the first year of the learning lab were used to create a knowledge base document to support a design project focused on OR design. This document includes a literature review on OR environments, OR workflows, and best practice case studies, to understand roles and workflows for all key surgical personnel. Second year graduate Architecture students were tasked with working with this multidisciplinary learning lab team to develop evidence-based design solutions for an innovative, safe, operationally efficient, and flexible OR that could fit any OR suite configuration. The design project began with a multidisciplinary design workshop, which included students, the multidisciplinary learning lab team, and national experts in architecture and healthcare. In this workshop a systematic brainstorming session about the OR environment was conducted to facilitate exploration of ideas through structured questions and answers on post-it notes. Architects and clinicians helped students think through the issues pertaining to the OR and how those issues may be addressed through potential solutions.

The team of researchers and educators determined the scope of the mock-up construction and evaluation as two phases (tape-on-the-floor and cardboard mock-up) as well as parallel iterative mock-up evaluation phases in a semester-long studio course. The studio project predetermined the timing of the mock-up in consideration with conceptual design, design development, and design strategies refinement phases.

Parallel to the studio assignment, the research team developed a systematic simulation-based mock-up evaluation protocol to support the iterative evaluation of the proposed designs in a timely manner to facilitate the next round of design strategies refinements.

Tool Development: Developing an evaluation protocol to obtain clinician input

Overall goals

The first and most critical step in developing the evaluation tool for the mock-up simulations was to establish key design guidelines based on the research objectives to guide the development and evaluation of design prototypes. Using the evidence base that was developed prior to the workshop, the input from the brainstorming session as well as additional literature reviews and case studies of surgery centers, the students developed a set of overarching objectives and evidence-based design guidelines to support the design process.. The design guidelines included five overarching goals (see Figure 1):

Figure 1.

Evidence-based design guidelines

• optimize the ability to change over time

• optimize sustainable strategies

• optimize clinical outcomes and health & safety

• optimize positive experience for all users

• optimize efficiency

Each of the 9 evidence-based design workshop are linked to one or more goals and address key aspects of OR design that impact workflow, disruptions, and other safety concerns, such as surface contamination (Figure 1). The design guidelines were stated such that they could be achieved through implementation of one or more design strategies.

The design strategies that were evaluated (evaluation objectives) had to align with the design phase and mock-up fidelity; therefore, some of them were repeated across three rounds of evaluation and some were phase-specific. Also, the evaluation tool evolved with each consecutive mock-up phase to better respond to the evaluation objectives and mock-up fidelity. Table 1 shows how the overall goals and evidence-based design guidelines that were developed, the rationale for each guideline, the design strategies that address the different guidelines, and the design strategies we tested in the mock-up process. Given the low level of mock-up fidelity, the evaluation only focused on the evidence-based design guidelines presented in table 1. A future high-fidelity mock-up will allow testing of additional design features and equipment.

Table 1.

Overall goals and evidence-based design guidelines

OVERALL GOALS	EVIDENCE-BASED DESIGN GUIDELINES	WHY IS IT IMPORTANT?	DESIGN STRATEGIES	DESIGN ELEMENTS EVALUATED
Optimize the ability to change over time
	Incorporate plug and play system	Increase adaptability 1	Flexible furnishings Prefabricated wall/ceiling systems and modules	CN workstation
	Employ a flexible room/suite chassis	Increase adaptability of OR and suite 1	Room zoning/planning	Parallel processing
Optimize sustainable strategies
	Incorporate plug and play system	Expand life cycle	Flexible furnishings Prefabricated wall/ceiling systems and modules	CN workstation Parallel processing
	Employ a flexible room/suite chassis	Expand life cycle of OR and suite Reduce waste in construction and renovation	Room zoning/planning	Parallel processing
Optimize clinical outcomes and health and safety
	Plan the OR to optimize movement and flow	Reduce door openings 2 Reduce flow disruptions and distraction 3 Reduce risk of infection 4	Establish clear functional room zoning Parallel processing Locate adequate storage in convenient locations within the OR	Doors Storage CN workstation Anesthesia workstation Surgery Sidedness Parallel processing
	Provide integrated digital information displays in multiple locations	Enhance safety 5 Enhance situational awareness 6	Built-in modular displays	Displays
	Incorporate plug and play system	Decrease tripping hazards 7 Enhance staff performance 8	Flexible furnishings Prefabricated wall/ceiling systems and modules	CN workstation Parallel processing
Optimize positive experience for all users
	Provide integrated digital information displays	Enhance teamwork and communication 9		Displays
	Plan the OR to optimize movement and flow	Decrease unnecessary movement 10	Establish clear functional room Zoning Parallel processing Locate adequate and storage in convenient locations within the OR	Doors Storage CN workstation Anesthesia workstation Surgery Sidedness Parallel processing
Optimize efficiency and effectiveness
	Plan the OR to optimize movement and flow	Reduce costs 11 Reduce travel time 12 Enhance throughput and turnaround 13	Establish clear functional room Zoning Parallel processing Locate adequate storage in convenient locations within the OR	Doors Storage CN workstation Anesthesia workstation Surgery Sidedness Parallel processing
	Provide integrated digital information displays in multiple locations	Enhance teamwork and communication 14	Built-in Modular displays	Displays

¹Allison et al., 2015; Kelts, McMains, Chen, & Weitzel, 2015

²Panahi, Stroh, Casper, Parvizi, & Austin, 2012

³Ahmad et al., 2016; Broom, Capek, Carachi, Akeroyd, & Hilditch, 2011; Elbardissi & Sundt, 2012

⁴Beldi, Bisch-Knaden, Banz, Muhlemann, & Candinas, 2009; Chauveaux, 2015

⁵Rostenberg & Barach, 2011

⁶Sanderson, Watson, & Russell, 2005

⁷Matern & Koneczy, 2007; Patkin, 2003

⁸Matern & Koneczy, 2007; Patkin, 2003; Watkins et al., 2011

⁹Moss & Xiao, 2004; Van Det et al., 2009

¹⁰Panahi et al., 2012; Patkin, 2003; Watkins et al., 2011

¹¹Marjamaa, Torkki, Hirvensalo, & Kirvela, 2009; Pelly, Zeallear, Reed, & Martin, 2013; Sandberg et al., 2005

¹²Panahi et al., 2012

¹³Pelly et al., 2013; Sandberg et al., 2005; Torkki, Marjamaa, Torkki, Kallio, & Kirvela, 2005

¹⁴Rostenberg & Barach, 2011

Master Protocol

A master document was developed to help formalize the execution of the simulation scenarios. The master document included: (1) pre-briefing information for simulation participants, evaluation objectives, (2) scenario tasks, (3) equipment and supplies list for each scenario, (4) design features to be evaluated, and (5) outline of structured evaluation questions. This document served as an overall reference document for the team throughout the mock-up evaluation process and formed the basis of two additional documents that were created to facilitate the mock-up evaluation – a simulation director’s worksheet and a note-takers’ worksheet.

To evaluate the design prototypes, the research team developed task-based scenarios, each of which aimed to evaluate multiple design features (evaluation objectives) by engaging the participants with the design features under evaluation. The scenarios were developed in collaboration with the research team’s clinical experts, and were focused on Orthopedic and Pediatric surgeries, since these were the service lines planned for the two new surgery centers. The scenarios were further vetted to only include those that represent the most common activities performed in an OR and those that are feasible to simulate given the low level of mock-up fidelity. Table 2 shows a sample section from the master document customized for evaluation of work zones using simulated scenarios in the intraoperative phase.

Table 2.

Scenario tasks and evaluation tool in intraoperative phase

INTRAOPERATIVE
Evaluation Objectives	Work Zones
Design Features	Work Zone Configuration
Equipment and supplies List	Patient gurney Instruments’ cart Instruments’ main table Instruments’ tray table (Mayo stand) Appropriate surgical table (with arm support, proper size for pediatric) 2 trash cans Kick bucket Anesthesia Machine with workstation Circulating nurse (CN) workstation (computer desk + chair) Chair/stool for surgeons (pediatric) 1 chair/stool for anesthesiology personnel
Scenario Tasks	Players: 1–2 anesthesiology personnel The anesthesiology personnel are monitoring patient The anesthesia personnel interact with the anesthesia storage and mayo stand as necessary in an intraoperative phase. The anesthesia personnel try sitting down when possible.	Player: 1 Scrub Nurse The scrub nurse is assisting the surgeon Pulls up and positions back table/s Complete/refine back table set-up et-up/connect equipment Anticipate and pass instruments, sponges, fluids, etc.	Player: 1 Circulating Nurse Constantly monitors for any breaches in sterile technique, positioning mishaps, etc. Dispense additional supplies/instruments to back table as needed (may need to leave the room for unanticipated supply item) Prepares and sends specimens	Players: 1 Surgeon The surgeon and residents are at the OR table performing surgery. The surgeon leaves the OR table and hands off the patient care to the resident. The surgeon goes to the surgeon’s workstation to enter information in a computer
Evaluation Questions	Is the location of the storage convenient when the anesthesia team member is trying to access storage at the same time as monitoring patient? Are there any obstructions in the way from the anesthesia chair to the storage?	Is there adequate space to position the instrument table relative to the surgeon so that scrub nurse can see the surgery and pass instruments as needed?	Are there any obstructions in the way to the door as the nurse heads out to gather supplies? Is there enough horizontal surface available to prepare specimens? Can the CN view the entire surgical field from where they have positioned their workstation?	Does the location and design of the surgical workstation support the needs of the surgeon?

Simulation Director’s Guide

The simulation director is key to the management of the mock-up evaluation as he/she manages the activities of the participants as they evaluate different options and keeps the evaluation on track. The simulation director’s guide provides an overview of the mock-up evaluation process including plans of all the different prototypes that were to be evaluated, organized in the order of evaluation. It includes a script for pre-briefing, a description of planned scenarios, and the design features under evaluation for each scenario.

Aligned with the evaluation objectives, the simulation director’s guide also includes a series of prompts for different team members for performing different tasks and questions for the participants to obtain their feedback. For example, one of the issues under consideration was the effectiveness of the layout for surgeries performed on either side of the patient. The simulation director prompted the surgical team members to set up the room as they would for surgeries on the right side and left side of the patient and to simulate the positions and movements during these different sided surgeries. The director then followed up the simulation with specific questions that probed adequacy of space for performing tasks, access to supplies and equipment, visibility to team members, and optimal location of monitors in these different surgical configurations (different prototypes). Once all the scenarios were completed, the simulation director facilitated a focus group discussion among the participants in order to obtain additional feedback regarding the different prototypes that were evaluated as well as specific design features included in the prototypes. The set of debriefing questions developed in response to specific evaluation objectives for the phase 3 mock-up evaluations are provided in table 3. Usually a small subset of these questions was used during the focus group discussion to prompt discussion on a range of different issues.

Table 3.

Example of structured debriefing questions from the Simulation Director’s Guide

DESIGN ELEMENT	EVALUATION QUESTIONS
Doors	Does the location of the doors facilitate an easy movement of patient in and out of the room? Does the location of the door accommodate bringing in the equipment and carts in and out of the room? Do you think the location of doors in this layout will facilitate easy movement to supplies? Does the number of doors allow for effective separation of the CN’s travel path from patient?
Scrub Sink Location	Does the location of the scrub sink work for different team members? What do you think about the length of the surgeon’s travel from the scrub sink to the patient bed after scrubbing? What do you think about the views into the OR from the scrub sink and door under different configurations?
Surgery Sidedness	(specify the side of the surgery in the notes) When the surgery located in the left/right/head/foot of the surgical table: Is there enough space for scrub nurse and instrument tables in different bed positions? Is there enough space for anesthesia team? Is access to supplies, workstation and patient convenient for anesthesia team? Does the CN have visibility to surgical field and team members? What do you think about the location and direction of the monitors for surgeon?
CN Workstation	Would the chargeable battery-powered mobile workstation work in an OR? Does this model work well for different types of surgeries?
Displays	If wall displays change over the course of the surgery, who would coordinate the displays? Where should the controls for the displays be located? What types of information would you like to see on a wall display? Where should the information be displayed? For different surgical locations and team positions, would the preferred location of displays need to change?
Storage (general and anesthesia)	Is the location of storage for anesthesia personnel easy to access by the anesthesia team? How much space is adequate for the anesthesia team to avoid interference with anesthesia storage? How often during a surgery, would you need to get those? Is there anyone else on the team who would need to get to the supplies located in this anesthesia cabinet? Is the location of storage near the CN convenient for everyone on the team? Are there any obstructions to using the CN storage? Where do you want the storage in the room? How much needs to be in the room?

Note Taker’s Template

Parallel to the simulation director’s guide, a note taker’s template was developed. This template includes the floor plans for each of the design prototypes being evaluated, an observation checklist, and the set of debriefing questions that the simulation director directed to the participants during the focus group discussion. The note-takers were trained to annotate the plans to indicate any adjustments made during the scenario enactments. The observation checklist includes the scenario tasks and a series of evaluation questions. For example, for the task where the stretcher with the patient is brought into the OR, an evaluation question asked, “Are there any bumps or challenges in maneuvering the stretcher?” The note taker was asked to check a box if the answer was yes and then add notes in the notes column. The debriefing questions are also included in the same order as the simulation director’s guide. Note takers used printed sheets to annotate the plans and also took notes directly on a laptop computer. Table 4 shows a sample of note taker template for the intraoperative phase.

Table 4.

Note taker’s template

INTRAOPERATIVE PHASE
Scenario Task	Evaluation question	check	Notes
SN pulls up and positions back table	Is there adequate space to position the instrument table relative to the surgeon so that scrub nurse can see the surgery and pass instruments as needed?
SN Connect equipment	Can the scrub nurse reach all necessary equipment that she needs to connect from her location?
CN monitors sterile field	Can the circulating nurse view the entire surgical field from where they have positioned their workstation?
	Does the zones’ configuration allow for safe and effective sharing between CN and surgeon’s zones?
CN heads out for additional supplies	Are there any obstructions in the way to the door as the nurse heads out to gather supplies?
CN retrieving items from storages in the OR	Where do you want the storage in the OR? How much needs to be in the room?
CN receives page/phone call	Is the location of the phone convenient?
CN prepares and send specimens?	Is there enough horizontal surface available to prepare specimens?
Anesthesia personnel get up to prepare supplies	Are there any obstructions in the way from the anesthesia chair to the storage?

Constructing the Physical Mock-Up and Conducting the Simulations

The constructed physical mock-up was evaluated in three phases. For the first mock-up, students developed four design options, two of which were selected for testing in the mock-up. The design options for this phase included two OR sizes as a result of different room proportions. Different proposed locations for OR doors that varied in size were considered as they might influence the flow of the equipment and personnel in and out of the OR. Doors with different widths were proposed to determine the adequate width for a door that allows seamless entry of equipment, patient gurney, and personnel.. Blue tape was used to mark the location of the walls and doors on the floor of the space. Some real OR equipment (e.g. OR table) and some placeholder equipment was brought into the space to allow users to understand space availability and circulation patterns.

For the second phase of the mock-up construction, four primary design options using the same footprint were selected for testing. Students pre-fabricated cardboard modules to be assembled on site. The ‘plug and play’ feature of the mock-up allowed for relocation of sections of the wall, scrub sink, storage, and doors to facilitate testing of multiple design options (see Figure 2).

Figure 2.

Plug and Play feature of the physical mock-up

The third mock-up phase evolved with more detailed design refinements to the OR design and physical construction. Additional elements tested involved the location of supplies, wall mounted displays, and a customized mobile circulating nurse (CN) workstation.

Two different surgical teams (pediatric and orthopedic), including anesthesia personnel, nurses, and a surgeon, were recruited for enactment. The teams and individual participants remained consistent across all evaluation phases.

One of the research team members, an anesthesiologist, served as the simulation director to guide enactment players and ensure that feedback was collected for each design option. A clinically trained research team member was selected to lead the simulations to ensure that the scenarios were clinically relevant and that the simulation director could effectively address any questions that arose during the evaluation. To minimize interruptions with the flow of the enactment, only the simulation director, players, and two note-takers remained inside the boundaries of the mock-up, with note-takers positioned in the corners of the mock-up space.

A 30-minute pre-briefing session was held before the simulation to introduce the roles, simulation goals, enactment process, and procedures for design evaluation. Participants were reminded to share their input during the discussions held following each simulation. A safety briefing informed participants about the mock-up structure stability and the designated emergency evacuation routes. After the pre-briefing, the simulation director instructed the participants to take their positions, as defined by the requirements of the scenario, and then announced the start of the enactment. On completion of the scenarios, the simulation director requested feedback from players based on structured debriefing questions and a dynamic discussion followed involving all stakeholders, including clinicians, architects, researchers, and architecture students involved in the design of the new ambulatory surgical center. Figure 3 illustrates how clinicians were engaged to test the design features through simulation.

Figure 3.

Testing of the design features through simulations

Application of simulation-based mock-up evaluation

To demonstrate the application of the simulation-based mock-up evaluation process, a session conducted during the third phase is described. The room size and proportions, configuration of zones, and OR table angle were developed based on findings from the first and second phase of simulations. The room is 22′ by 26′, with a net area of 572 square feet. The OR has a support room adjacent to one of the long walls of the OR whose purpose was to increase future flexibility by allowing for OR expansion in response to changes in surgery type, technology, or equipment demands. The OR has three main work zones including the circulation zone (C.Z.), scrub zone (S.Z.), and anesthesia zone (A.Z.). The A.Z. is always located at the head of the OR table, except for cases where the surgery occurs in the upper-neck region of the patient. The S.Z. is positioned on either side of the OR table, depending on the sidedness of the surgery, to provide direct support to the surgeon and surgical assistant. The C.Z. is preferably separate from the OR table and S.Z. The C.Z. can expand as the S.Z. alternates to either side of the OR table. The C.Z. is positioned far from the A.Z. and S.Z., yet the location still provides easy access to the S.Z. for needed support. One of the key factors that impacted the organization and placement of work zones was door location. The iterative mock-up simulations used in phase one and two of the evaluation process validated the overall configuration of the work zones.

In order to evaluate this design option a multidisciplinary group was engaged including two architects, two anesthesiologists, one nurse, eight graduate architecture students and two researchers. Total of 10 surgical personnel (five pediatric and five orthopedic) participated in mock-up evaluations as participants. The surgical personnel included surgeons, anesthesiologists, and nurses. The pediatric and orthopedic OR simulations were run separately. Since the simulations were only focused on specific activities, such as bringing in the patient gurney and positioning it next to the OR table (as opposed to complete surgery procedure), each session took no more than 20 minutes including the focus groups following the simulations. A total of six design variations were evaluated for each group. The clinicians enacted simulation scenarios that included tasks aligned with the evaluation objectives. The design features that were evaluated during the third design phase of the study were door location, OR table sidedness, scrub sink location, anesthesia storage location, general storage location, anesthesia workstation, CN workstation, and displays. During the simulation, note takers recorded responses to the evaluation questions based on their observation of the clinicians’ interaction with the OR elements, input from clinicians, and follow-up discussions. To allow for further examination of the simulations, the process was video-recorded and photographs were taken to capture the interaction of clinicians with the OR mock-up during scenario enactment and evaluation.

The detailed notes and plan annotations taken throughout the scenario enactment and debriefing session were transcribed and summarized immediately following the sessions conducted in each of the phases. This information was shared with all stakeholders including students, clinicians, and architects. The students participated as observers throughout the simulation and engaged in discussion with the clinicians during the debriefing session. Following the simulation based mock-up evaluation, the notes were reviewed internally by the student design team and changes made to the prototype design in response to the clinicians’ input.

RESULTS

The final version of the of the evaluation tool evolved into a comprehensive toolkit addressing all evaluation objectives. The design features that were continuously evaluated across all three phases included OR table placement and orientation, door location, storage location, and work zones. Design elements that were evaluated included, but were not limited to, built-in modular displays and OR table sidedness.

The iterative design process integrated with the repeated simulation based mock-up evaluation and involvement of the multidisciplinary teams resulted in a dialogue around key patient and staff safety issues that was invaluable in the evolution of the OR prototype design. The evaluation tool allowed for structured and focused recording of the input relevant to the evaluation objectives. Figure 4 shows a summary of the iterative design and mock-up evaluation process and results. Some of the key design decisions for a safer and more ergonomic OR that were made as a result of this process included:

Figure 4.

The iterative design and mock-up evaluation process

Door attributes

Door location was tested repeatedly in all three mock-up phases, as it played a key role in directing flow within the OR. The evaluation focused on how the location of doors affected the number of turns that the gurney would need to make until it was positioned next to the OR table as well as any potential bumps along the way. The surgical staff preferred the location of the door in a zone adjacent to the C.Z. and CN workstation zone to minimize the possibility of people entering the room into the sterile area around the OR table and the associated risk of contamination in those areas. During focus group discussions, the surgical staff also explained that based on their experiences, swing doors interfere with staff and equipment transitioning through the door. To reflect on this comment, the design evaluation focused on the use of sliding doors, which eliminate the required space needed for a swing door opening.

OR table placement and orientation

The position of the OR table influences the patient gurney’s ease of transfer into the room, equipment positioning, position of the anesthesia and scrub nurse’s areas, and surgery sidedness. Given that different surgery types may require varied locations of equipment and surgical personnel, the design prototype tested the OR table in an angled position which was eccentrically off-center in the OR (see figure 5). The angled position allowed the surgical field to flip for flexible support of the scrub nurse on either side of the table and also facilitated the placement of the equipment, such as the C-arm at either side of the table in case of an orthopedic surgery. Sidedness is interdependent with the door location, availability of outlets, and adequate space for additional equipment and bed rotation. The positioning of the OR table and the location of the door influences traffic flow that could interfere with the scrub nurse’s work area and threaten the sterility of the surgical field.

Figure 5.

Final design of the operating room based on the feedback gained through iterative mock-up evaluations.

Scrub sink location

During all three phases of the simulation, scrub nurses repeatedly emphasized the benefits of line of sight from the scrub sink area into the OR, specifically the area around the OR table. Visibility provided scrub nurses with a sense of control over events occurring in the OR and a means of ensuring that the sterile field was not violated during the pre-operative phase. Therefore, the scrub sink was strategically located outside of the OR’s long wall and across from the OR table to maximize visibility to the area surrounding the OR table. Also, the size of the scrub sink provided sufficient space for surgical masks and scrub brushes.

Anesthesia workstation

The anesthesia personnel expressed concerns about workspace that is rather limited due to the number of personnel and equipment housed in the area. Also, anesthesia personnel expressed the need for distance between the anesthesia area and doors to minimize interference with anesthesia-related tasks. Accordingly, the design prototypes placed the A.Z. in the corner of the OR, away from all the options for door locations. This was accomplished through the angled orientation and eccentric placement of the OR table. The A.Z. provided ample space, by eliminating the wasted space behind the surgical boom, to allow anesthesia personnel to work with less flow disruptions related to space constraints as well as to accommodate a storage cabinet for anesthesia supplies and an anesthesia cart and printer for specimen labeling. According to the simulation evaluation, the location and amount of space provided for anesthesia-related tasks worked well for the anesthesia personnel. Also, the door location away from the A.Z. seemed to prevent interruptions in this zone from other surgical personnel who walk in and out of the OR. During scenario enactments, the anesthesia personnel indicated the need for a wall-mounted glove dispenser and a hardwired phone in the OR, specifically for anesthesia personnel.

Anesthesia storage

Anesthesia personnel are the primary group using the anesthesia storage. Given that the anesthesia workstation has to accommodate a number of personnel, equipment, and storage, the workspace is often cluttered with overflow of items to be stored. The anesthesia personnel indicated that not only should there be adequate storage but also that the storage should be strategically located for easy and quick access. Often, the location of the storage in a constrained space leads to congestion when multiple people are working in this space and poses difficulties with opening the cabinet door. The evaluation of the design prototype showed that positioning the storage in the area in front of the monitors, rather than adjacent and behind them, provides easier access for all anesthesia personnel with minimal interference with the storage door. Also, the placement of the storage in front of the monitors allows easier access for the CNs in rare occasions that they need to access the storage (see figure 5 for the proposed storage location).

Circulating nurse workstation

The design prototype included a battery-powered mobile CN workstation. Since the station battery was chargeable, it eliminated a need to be connected to an electricity outlet via a cord. The designated parking spot for the mobile CN workstation in the prototype was in the corner of the OR; however, the mobility of the station allowed the CN to reposition the station as well as station monitor angle to maximize their view to the OR. The chargeable-battery feature of the workstation allowed the workstation to remain mobile within areas away from an electricity outlet. The flat surface of the station allowed for temporary storage of items.

After the simulation-based evaluation, CNs expressed value in having adequate visibility to the overall OR. However, CNs noted concerns with the lack of space for a printer and the potential clutter in the OR because of the mobility of the workstation.

General storage

General storage is often used by various OR personnel but primarily by the CN. General storage should be located in a way that prevents traffic congestion, obstruction, and interference with staff tasks. The mock-up evaluation helped identify the location for the general storage in the C.Z. This location helps keep the CN and other personnel away from the sterile S.Z. on either side of the OR table and anesthesia area. Given that the key role of the CN is to support the surgical team, an independent location for general storage seemed desirable. The simulation and mock-up evaluation confirmed this expectation.

Displays

The mock-up provided an innovative approach for the display of information in the OR by simulating continuous built-in modular displays. The goal was to evaluate the effectiveness and convenience of information presented to all OR personnel. One limitation of the prototype at the cardboard mock-up stage was that the overhead lights and booms were not in place. So potential sightline conflicts with overhead equipment could not be evaluated.

The surgical team members preferred lower displays to improve sightlines, as the height of the displays in the prototype design required excessive head and neck movement. The evaluation of the displays location showed that the continuous built in modular displays were not as effective for staff, because of overload of information. However, staff expressed the importance of placing the displays within their visual field where they can see the displays with minimal repositioning of the body (see figure 5 for staff preferred location for built-in modular displays). The mock-up evaluation process revealed that staff favored the presence and abundance of information that could be made available to them through displays. Information displays are known to improve work speed and communication in the OR (Webster & Cao, 2006); similarly, staff expressed that the display of different information such as the surgical checklist, X-rays, patient information, scheduling information, static information (weight, allergies, etc.), and time during different phases of the surgery could potentially improve their situational awareness of the OR and patient. Also, staff preferred having control over the displayed information to allow them to retrieve information they needed during different times of the surgery.

Parallel processing

During the phase 3 mock-up evaluations, debriefing focus groups with the clinicians led to a conversation about improving efficiency through parallel processing, which is simultaneous care for two patients including induction of one patient outside of the OR, as the second patient is under surgical care inside the OR (Marjamaa, Torkki, Hirvensalo, & Kirvelä, 2009). The pediatric surgical team suggested anesthesia induction outside the OR to make it more patient and parent friendly. The idea of induction room to make an efficient OR was inspired by a case study at Seattle Children’s Hospital that the students and research team had evaluated earlier in the project. Other studies have also shown that separate induction rooms helps to reduce surgeon time in the OR and increases the number of surgeries that can be performed in a day especially for short surgeries (Fong, Smith, & Langerman, 2016). The orthopedic teams, on the other hand, did not see a benefit of induction rooms given that their surgeries are typically very long. They recommended that the team consider including an instrument preparation room adjacent to the OR so that much of the preoperative instrument set-up could take place outside the OR. Some rudimentary simulations were conducted in the existing phase 3 mock-up to test these potential flows. As a result, the entry points into the OR were ultimately repositioned from the short wall to the long wall of the OR to accommodate multiple entry points for various configurations of induction room, instrument preparation room and direct entry into the OR from the circulation side of the OR.

The different concepts that emerged during the three phases of design and evaluation around parallel processing, door design and placement, OR monitor placement, storage and workstation design were integrated into a final design concept (see figure 5).

DISCUSSION

This study aimed to develop a toolkit to evaluate physical mock-ups of design prototypes using scenario-based simulations. Although mock-up evaluation of design prototypes is becoming a common practice in architectural projects related to healthcare (Traversari, Goedhart, & Schraagen, 2013; Peavey, Zoss, & Watkins, 2012), they lack rigor and are not systematic in their approach. Also, mock-up evaluations are often conducted later in the design process once many design elements are finalized. There is a huge opportunity to conduct simple mock-up evaluations early in the design process to obtain critical feedback from end-users. While evaluations of more refined design options can also be helpful, they are less effective in generating new ideas and designs. Given how costly mock-up construction is, it is important to effectively utilize physical mock-ups to engage end-users in evaluating design features at different points in the design process. The current study allowed for evaluation of ergonomics, which is defined as the adjustment of the environment to humans (Vink, Imada, Zink, 2008), within a simulated OR environment throughout the design process.

The mock-up evaluation toolkit built upon a previously developed framework for conducting scenario based evaluations (HQCA, 2016). The HQCA framework was effective as a starting point for guiding both the toolkit development as well as structuring the actual mock-up evaluation. Basing the evaluation criteria on evidence helped in developing evidence-based design prototypes through an iterative process. The evaluation toolkit included purposeful scenarios that targeted design features of interest. Parallel to structured scenarios, a set of evaluation criteria in form of questions were used to direct discussions for an effective evaluation. Scenarios were tailored to design features that were under evaluation as well as the end-users who would be interacting with those design features. Further, findings from each stage of the evaluation guided the refinement of design. Through an iterative process, design prototypes were refined and evaluated with an updated evaluation toolkit, which was focused to evaluate the newly refined design features.

While this toolkit and its components were developed specifically for the OR, this tool structure could easily be modified to support evaluations of other healthcare spaces, for example patient rooms. The structure of evaluation tool allows flexibility for evaluating different stages of design as well as types of spaces. When adapting the toolkit for other types of spaces, the overall goals may be modified based on the type of space and be reinforced by incorporating previous literature. Further, the evaluation questions must be formulated to target the effectiveness of the proposed design features to be tested.

The evaluation tool developed in this study systematized the simulation process and, parallel to that, the evaluation of mock-up design prototypes. The early phase of the mock-up evaluation showed that an open-ended approach to data collection provided additional advantages. The qualitative approach allowed for a holistic understanding of different perspectives on design implementation. Therefore, flexibility in data collection methods and strategies is key to capturing the needs of all stakeholders. Despite the fluidity of the discussions, the evaluation tool provided an efficient and effective way for systematically and comprehensively gathering data. The evaluation tool also guided note takers, which prevented excessive and irrelevant note taking. The evaluation tool was adapted and customized for each design phase to better address the overall goals of that specific iteration. The iterative process was valuable for building on lessons learned that informed appropriate decisions for each design simulation and evaluation phase.

The process and tools that were developed as part of this study helped in obtaining clinician feedback as well as allowed for objective assessment of the space to actively inform the design process within a fast-paced project timeline. The engagement of the stakeholders in the design process is invaluable as it allows them to not only participate in design decision-making but also to learn by exchange of information with the design team (Mackrill, J., Marshall, Payne, Dimitrokali, & Cain, 2017). The mock-up evaluation process enabled a deeply collaborative design process to emerge that allowed active engagement of students, architects, researchers, and clinicians.

The evolution of the evaluation tool over the three phases of design resulted in a solid structure for data collection and evaluation of mock-ups. The evaluation tool can be adapted for the evaluation of other OR mock-ups. If the overall goals are adapted based on literature, the toolkit can be also utilized for evidence-based evaluation of mock-ups of other types of settings.

Limitations

From a research perspective, the current findings on OR design were reasonable based on repeated evaluation of scenarios across phases; however, they may not be generalizable to other OR settings due to the limitation of the study sample. Clinicians working in different health systems might have different priorities and work practices leading to different solutions. While the design solutions may not be generalizable, the process and tools are relevant and transferable to other projects. The scope of evaluations was focused on the design of the OR itself and did not include the larger context of a hospital and its influences on the OR design. This limitation is simply due to the physical restrictions of developing a mock-up in the scope of a real hospital. Simulation-based mock-up evaluations are geared toward qualitative studies given the limited number of participants; however, future research should explore challenges and opportunities on expanding such evaluation to a scope appropriate for quantitative analysis.

While the evaluation helped the team in making decisions regarding the evaluation objectives (such as room size, door location, zoning) that were relevant in the early design phases, some decisions regarding finishes (type of flooring) or equipment selection (functionality of overhead boom) could not be addressed through the process presented in the paper due to the relatively low level of mock-up fidelity. These types of decision are typically made later in the design process when the design (and the associated mock-up) can be of higher fidelity and can be used to answer detailed design questions. A future high-fidelity OR mock-up being planned as part of this project will include all proposed equipment and finishes such overhead booms, flooring, recessed cabinets and sliding doors and will allow for higher fidelity surgical scenarios as well as evaluation of additional design features. The toolkit will likely be refined further to include additional methods of data collection such as video recording and objective performance metrics related to flow (bumps, trips). The current structure of the toolkit is flexible enough to include new evaluation objectives as well as additional methods of data collection.

CONCLUSION

The integrative simulation based mock-up evaluations served to better inform, educate, and engage all stakeholders, including architects, surgery teams, and patients, and develop a mutual and consensus-based approach to design decision-making. The integrative process allowed students to benefit from end users’ feedback in refinement of their design. The process also allowed for active and structured engagement of clinical end users. The evaluations helped to validate design features and their implications for patients and staff. The process and tools developed as part of this study were effective in supporting a structured dialogue between design and healthcare disciplines and can easily be adapted and applied to other healthcare facility design projects.

Highlights.

• Simulation-based mock-up evaluations can enhance communication between end-users of healthcare facilities and design teams

• The design of the operating room was developed through the highly collaborative process by a multidisciplinary team of architects, clinicians, and researchers.

• The mock-up evaluation toolkit can be adapted and used in similar design projects engage multidisciplinary stakeholders

• The toolkit can be used for effective evaluation of healthcare facility design prototypes.