Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Sep 2.
Published in final edited form as: Int J Med Inform. 2022 Mar 2;162:104727. doi: 10.1016/j.ijmedinf.2022.104727

Human-Centered Design of Team Health IT for Pediatric Trauma Care Transitions

Pascale Carayon 1,2, Bat-Zion Hose 1,2,3, Abigail Wooldridge 4, Thomas B Brazelton III 5,6, Shannon M Dean 7, Ben L Eithun 5, Michelle M Kelly 5,6, Jonathan E Kohler 8, Joshua Ross 5,6,9, Deborah A Rusy 5,10, Peter LT Hoonakker 2
PMCID: PMC9437147  NIHMSID: NIHMS1791482  PMID: 35305517

Abstract

Background:

As problems of acceptance, usability and workflow integration continue to emerge with health information technologies (IT), it is critical to incorporate human factors and ergonomics (HFE) methods and design principles. Human-centered design (HCD) provides an approach to integrate HFE and produce usable technologies. However, HCD has been rarely used for designing team health IT, even though team-based care is expanding.

Objective:

To describe the HCD process used to develop a usable team health IT (T3 or Teamwork Transition Technology) that provides cognitive support to pediatric trauma care teams during care transitions from the emergency department to the operating room and the pediatric intensive care unit.

Methods:

The HCD process included seven steps in three phases of analysis, design activities and feedback.

Results:

The HCD process involved multiple perspectives and clinical roles that were engaged in inter-related activities, leading to design requirements, i.e., goals for the technology, a set of 47 information elements, and a list of HFE design principles applied to T3. Results of the evaluation showed a high usability score for T3.

Conclusions:

HFE can be integrated in the HCD process through a range of methods and design principles. That design process can produce a usable technology that provides cognitive support to a large diverse team involved in pediatric trauma care transitions. Future research should continue to focus on HFE-based design of team health IT.

Keywords: human-centered design, human factors and ergonomics, team health IT, pediatric trauma, care transition

1. Introduction

Clinicians need health information technology (IT) that supports their activities. Unfortunately, often health IT is poorly designed, does not support a clinician’s workflow and increases their cognitive workload [1–3]. Incorporating human factors and ergonomics (HFE) methods and principles in health information technology (IT) design can improve the technologies by providing better cognitive support to clinicians. HFE methods can be used to identify information needs for decision making in clinical processes. HFE principles can help to better organize and present information to clinicians. However, we have limited knowledge about how to incorporate HFE methods and principles in health IT design [3, 4]. For example, research has described the HFE process to develop information requirements for a novel newborn resuscitation device [5], and to enhance the usability of clinical decision support technologies for colorectal cancer screening by primary care physicians [6], and in emergency departments (EDs) to support buprenorphine prescribing [7] and pulmonary embolism diagnosis [8, 9]. These rare studies focus on technologies used by one group of clinical users, such as physicians. As team-based care is increasingly deployed, we need to address health IT design to support teamwork.

We know little about how to design health IT to support the interconnected work of health care teams [3]. Using contextual design [10], Wu and colleagues [11] developed an integrated display to support intensive care unit (ICU) resuscitation teams. Schiro et al. [12] adapted the ISO (International Organization of Standardization) human-centered design (HCD) process for a pediatric ED patient prioritization tool. Using cognitive systems engineering methods, an interdisciplinary team developed an integrated display to support the work of ED physicians and nurses in tracking patient care and managing ED resources [13, 14]. These unique studies describe application of HFE to the design of health IT for relatively small teams comprised of physicians and nurses. Using cognitive systems engineering methods, Nemeth et al. [15] developed a cooperative communication tool to support individual and team work of staff (e.g. physician, nurse, respiratory therapist) in a burn ICU. Building on this research, we describe the HCD process for a technology aimed at supporting team cognition during care transitions of pediatric trauma patients. The team involved in these transitions is much larger than those described in previous studies with different roles from multiple disciplines and services; therefore, adding to the complexity of the design process.

Incorporating HFE methods and principles in the design of team health IT is particularly important for distributed care teams sharing information in high-stress environments. This is the case for the large pediatric trauma teams [16] who often experience challenges with information management [17]. Over 30 months, we implemented an HCD process to develop T3 or Teamwork Transition Technology. T3 aims to support team cognition during emergent transitions of pediatric trauma patients from the ED to the operating room (OR) and the pediatric intensive care unit (PICU). Our objective is to describe the HCD process used to develop a usable team health IT (T3 or Teamwork Transition Technology). This paper focuses on the overall HCD process, which represented a relatively long, complex process; some specific research activities of the HCD process are further described in separate papers [18–20].

2. Background: Human-Centered Design of T3

According to the ISO [21], “Human-centered design is an approach to interactive systems development that aims to make systems usable and useful by focusing on the users, their needs and requirements, and by applying human factors/ergonomics, and usability knowledge and techniques” (page vi). The ISO framework of human-centered design (HCD) activities includes interactive, iterative phases of (1) understanding and specifying the context of use, (2) specifying the user requirements, (3) producing design solutions, and (4) evaluating the design. We adapted the ISO framework to the design of T3. As reviewed above, studies of team health IT have used different HCD models and approaches, such as contextual design [11] and cognitive systems engineering [14, 15]. Similarly to Schiro et al. [12], we adapted the ISO framework as it allows flexibility regarding which methods to apply while providing an overall, iterative framework to design. The HCD process for T3 included seven steps in three phases of analysis, design activities and feedback, which produced design requirements for T3 and a list of HFE design principles (Figure 1).

Figure 1 –

Figure 1 –

Open in a new tab

Human-Centered Design Process for T3 (Teamwork Transition Technology)

Adapted from the ISO framework of human-centered design activities [21]

Note: SEIPS = Systems Engineering Initiative for Patient Safety, HFE = Human Factors and Ergonomics.

The HCD process for T3 included steps that are often and typically used, such as initial work system and process analysis (step 1) and feedback via usability evaluations (steps 4 and 7). Our design process also facilitated input from leaders, including operational, medical and informatics leaders who participated in the collaborative design (step 3) and nursing leaders (step 5). Initially our HCD process did not include Step 6 of patient safety analysis; this was added based on a suggestion from nursing leaders (Step 5). In Step 3, we chose a collaborative design approach, which, in contrast to user-centered design [22], directly involves users in design [23]. We created a design team that included clinicians who represented multiple disciplines and perspectives. Such an HCD process can be challenging when designing a technology for a team (e.g. T3 or Teamwork Transition Technology) because of the large number of user groups, variation in information needs and potentially divergent perspectives.

3. Methods

Table 1 provides an overview of the HCD steps and their main outcomes. HFE researchers organized the HCD process with the help of a design team that included seven clinicians who represented multiple perspectives involved in pediatric trauma care: emergency medicine, surgery, anesthesia, intensive care medicine, nursing & trauma management, hospitalist, and medical informatics.

Table 1 –

Detailed Description of the Human-Centered Design Process for T3 (Teamwork Transition Technology)

Steps Activities When and how long? Outcomes
ANALYSIS 1. SEIPS analysis of pediatric trauma care transitions [20] 17 interviews with 18 clinicians from the ED, OR and PICU who represented the following roles: attending, fellow, resident, nurse

  • Between May 2017 and November 2017

  • Average interview duration: 52 minutes (range: 29 minutes to 66 minutes); total of 14 hours and 44 minutes.

  • Process maps for ED-OR, OR-PICU and ED-PICU care transitions.

  • Multiple work system barriers and facilitators; in particular team cognition, which is the focus of design process.
2. Suggestions from clinicians for sociotechnical solutions Secondary data analysis performed by HFE researchers using interviews with 18 clinicians (see step 1) September 2017: results of analysis presented to design team in a 1-hour meeting. Main suggestions for sociotechnical system solutions to support team cognition in ED-OR transition:

  • document for ED nurse to bring to OR

  • identification of care team members

  • physical presence of all pediatric trauma care team members in the ED

  • auto-population of vitals and other patient-related data into EHR

  • timely and concise charting of care provided to child in the ED

  • recording of EMS report and/or story of injury in EHR.
DESIGN 3. Collaborative design sessions Four interdisciplinary co-design sessions with involvement of multiple disciplines: HFE, emergency medicine, surgery, anesthesia, intensive care medicine, nursing & trauma management, hospitalist, medical informatics. Survey of information elements.
Feedback survey after each design session.

  • Between January and October 2018

  • Total of 4 hours (range: 53 minutes to 64 minutes).

  • Focus on team health IT to support team cognition during ED-OR-PICU care transitions: goals, features, and information elements.

  • Survey data on priorities for information elements.

  • Iterative design of T3 mock-up on paper, and in PowerPoint, LucidChart and then Adobe InDesign.

  • Survey data on quality of design sessions.
FEEDBACK 4. Individual feedback from clinicians Feedback from 4 physicians from ED, OR (surgery and anesthesia) and PICU on initial T3 mock-up using interview and usability survey

  • Four meetings in July 2018

  • Average duration: 55 minutes (range 43 minutes to 66 minutes); total of 3 hours and 43 minutes.

  • Usability survey data on T3.

  • Qualitative information, including what to change on T3.
5. Input from nursing leadership Two meetings with nursing leadership; with participation of members of the design team (HFE, nursing & trauma management, medical informatics) Two 1-hour meetings in July 2018 and July 2019. 2018 meeting:

  • brought up issues of patient safety (which led to patient safety analysis; see step 6)

  • suggestions about weight (in kg) and blood products

2019 meeting:

  • suggestion for including surgical techs in the mock-up evaluation (see step 7)

  • discussion of implementation (e.g. cost for purchasing displays, physical location)
6. Patient safety analysis [18] PRA meeting with 8 clinicians from ED, OR and PICU who represented the following roles: attending, nurse 2-hour PRA meeting in December 2018.

  • Total of 9 vulnerabilities identified; top 3 vulnerabilities: reliability of information, unidentified patients appearing younger, visibility of protected health information.

  • Solutions: identifying source of data, last time updated, indicating whether data validated, high refresh rate; highlighting unknown or estimated; clear rule about access to T3.
7. Evaluation of T3 mock-up [19] Scenario-based evaluation of T3 with 36 clinicians from ED, OR and PICU using interview and usability survey

  • Between August 2019 and October 2019

  • Average duration: 25 minutes (range 10 to 52 minutes); total of 15 hours and 21 minutes.

  • Qualitative and quantitative data on usability of T3.

  • Ideas for improving design of T3.

  • Suggestions for implementation of T3: e.g. physical location, change management.
Open in a new tab

Note: ED = Emergency Department, OR = Operating Room, PICU = Pediatric Intensive Care Unit, HFE = Human Factors and Ergonomics, EMS = Emergency Medical Services, EHR = Electronic Health Record, PRA = Proactive Risk Analysis.

3.1. Step 1 – SEIPS Analysis of Pediatric Trauma Care Transitions

Using the SEIPS (Systems Engineering Initiative for Patient Safety) model [24, 25] and process modeling [26], we analyzed data on pediatric trauma care transitions between the ED, OR and PICU [20] and data from the pediatric trauma registry to characterize the complexity of these transitions [16]. The extensive analysis of work system barriers and facilitators is reported in a separate publication [20] and shows that the dimension of ‘team cognition’ was most frequently cited by the interviewees. The main decision of Step 1 was the focus on sociotechnical solutions to support team cognition of distributed pediatric trauma care teams who manage children during ED-OR-PICU care transitions.

3.2. Step 2 – Identification of Sociotechnical Solutions

Using interview data from Step 1, we identified sociotechnical solutions to support team cognition. Based on the SEIPS analysis, the list of sociotechnical solutions, and literature on integrated displays for health care teams [11, 27], the design team decided to develop a patient status board to support team cognition during ED-OR-PICU transitions.

3.3. Step 3 – Collaborative Design Sessions

The collaborative design process consisted of 4 design sessions with the 7 clinicians of the design team (Table 2). A core group of 4 HFE researchers prepared and facilitated the design sessions; 2 additional HFE students helped when necessary. When designing a team health IT, a challenge is to decide which information elements to display on the technology to provide cognitive support to the entire team. Therefore, we organized the design sessions to facilitate this decision-making, e.g. using a survey of the design team on information elements. The iterative collaborative design produced about 20 different versions of T3. At the end of each design session, we administered a survey to get just-in-time feedback from the design team about the quality of the design sessions.

Table 2 –

Step 3: Collaborative Design Sessions for Developing T3 (Teamwork Transition Technology)

What and when? Who? Activities
Design session 1 (January’2018)

  • 6 HFE researchers

  • 4 clinicians representing: emergency medicine, surgery, anesthesia, nursing & trauma management

  • Present results from SEIPS analysis of pediatric trauma care transitions (step 1)

  • Review research on team health IT design

  • Discuss structure of T3

  • Vote and discuss which information elements to include on T3
Design session 2 (April’2018)

  • 4 HFE researchers

  • 7 clinicians representing: emergency medicine, surgery, anesthesia, intensive care medicine, nursing & trauma management, hospitalist, medical informatics

  • Ask clinicians for ideas on timeline mock-up

  • Discuss ideas for timeline
Design session 3 (June’2018)

  • 4 HFE researchers

  • 6 clinicians representing: emergency medicine, surgery, anesthesia, intensive care medicine, nursing & trauma management, hospitalist

  • Review and discuss T3 mock-up

  • Discuss results of information elements survey
Design session 4 (October’2018)

  • 4 HFE researchers

  • 6 clinicians representing: emergency medicine, surgery, anesthesia, intensive care medicine, nursing & trauma management, hospitalist

  • Review and discuss T3 mock-up

  • Discuss feedback from Step 4 usability evaluation
Open in a new tab

Note: SEIPS = Systems Engineering Initiative for Patient Safety, HFE = Human Factors and Ergonomics.

3.4. Step 4 – Individual Feedback from Clinicians

During the collaborative design sessions (step 3), we produced multiple version of T3, including a mock-up in Adobe InDesign that was evaluated in Step 4. In collaboration with the clinicians on the design team, we developed a clinical scenario to show how T3 would evolve over the course of the scenario. The scenario was integrated in the collection of data on usability of T3 and qualitative comments on the cognitive support offered by T3 and its impact on patient safety. Four clinicians from the design team who represented medical perspectives from the ED, OR and PICU participated in this step. We used the SUS (System Usability Scale) [28] to gather quantitative usability data. The SUS is a reliable tool for measuring usability of products and services. It consists of 10 questions with a response scale from 1–5 that can be recoded into a usability scale from 0 (low usability) to 100 (high usability).

3.5. Step 5 – Input from Nursing Leadership

We gathered input on T3 from hospital nursing leadership in two meetings. The first meeting occurred between design sessions 3 and 4; nursing leaders reviewed a version of the T3 mock-up and provided feedback. They suggested a few design changes, including displaying weight in kilogram and providing additional information on blood products. We also discussed the impact of T3 on patient safety as nursing leaders expressed concern about the accuracy of the displayed information on T3. This led us to add a step in the HCD process and conduct a patient safety analysis (step 6). At the second meeting, nursing leaders reviewed the evaluation plan and suggested to add surgical technicians to the sample for Step 7.

3.6. Step 6 – Patient Safety Analysis

We conducted a proactive risk assessment (PRA) of T3, which is described in a separate publication [18]. We first identified vulnerabilities of T3 that could lead to patient safety problems. Then, through a voting process, we focused on three vulnerabilities with the highest patient safety consequences and discussed possible solutions. The main vulnerabilities were related to (1) reliability of information displayed on T3, (2) unidentified patients appearing younger, and (3) visibility of protected health information.

3.7. Step 7 – Evaluation of Mock-up of T3

We conducted a scenario-based evaluation of the T3 mock-up with 36 participants. Details on the methodology are reported in Hoonakker et al. [19]. We used the SUS [28] to evaluate T3 usability, which had a Cronbach alpha score of 0.90.

3.8. Overall HCD Process for T3

The entire HCD process was spread over 30 months (Figure 2). As described in Table 1, it included diverse inter-related activities in seven distinct steps. The first two steps of analysis produced data that were used in subsequent steps of design activities and feedback. Steps 3–7 helped to define the T3 design requirements and produce the final mock-up of T3.

Figure 2 –

Figure 2 –

Open in a new tab

Timeline of Human-Centered Design Process of T3 (Teamwork Transition Technology)

4. Results

4.1. Feedback on Design Sessions

The feedback survey after each of the design sessions showed that, on a scale from 1 (poor) to 5 (excellent), the four co-design sessions were rated between 4 and 5. On a scale from 1 (not useful) to 5 (very useful), the sessions were rated between 4 and 5. Qualitative comments highlighted the importance of “multidisciplinary input from several different services involved in trauma” and the consideration of different perspectives. Challenges included time constraints as meetings were scheduled for only 1 hour and not all clinicians could fully participate in all design sessions.

4.2. Design Goals for T3

An outcome of Steps 1 and 2 of the HCD process was the agreement on developing a technology for care team members involved in ED-OR-PICU transitions. The design team identified that the main goal of T3 was to support team cognition and shared mental models in these transitions for pediatric trauma patients [20]. During the collaborative design of Step 3, the design team iteratively developed the following specific design goals of T3:

  1. to provide timely, up-to-date summary of information about patient status to distributed care team members during patient transition from ED to OR and PICU.

  2. to support communication, coordination and anticipation between sending (ED) and receiving (OR, PICU) units, such as when the OR or PICU is ready to receive the patient, if the patient is still in the ED, and what medications need to be prepared.

  3. to help identify care team members involved in the ED to OR and PICU transitions.

4.3. Information Elements on T3

A challenge of the HCD process was deciding which information elements to include on T3, especially given the various users and roles with different information needs. Decisions about which information elements to include on T3 were made by the design team during the four collaborative design sessions (step 3). This was an iterative process with multiple activities, such as voting on information elements during the first design session and discussing results of the survey on information elements during the third design session (see Table 2). The final version of T3 (see Figure 3), which was evaluated in Step 7, includes 47 information elements in 10 categories (see Appendix for the complete list of information elements):

  1. patient information

  2. information about what happened prior to admission

  3. information on the patient family or caregiver

  4. time elapsed since the injury and in the ED

  5. current status of the patient

  6. information on the mannequin

  7. information on the timeline: e.g. start and stop times of CPR, vital signs

  8. members of the care team

  9. status of the transition to OR

  10. status of the transition to PICU.

Figure 3 –

Figure 3 –

Open in a new tab

Mock-up of T3 (Teamwork Transition Technology)

4.4. HFE Design Principles for T3

A key output of the HCD process was a list of HFE principles that were applied to T3. The HFE design principles emerged iteratively with input from clinicians involved in pediatric trauma care transitions (steps 1, 2, 4 and 6), collaborative design (step 3), and hospital leadership meetings (step 5). Because the goal of T3 was to support team cognition during care transitions, a major design principle was situation awareness. The structure of T3 was, therefore, organized around three panels of (1) what happened to the child and background information (left panel), (2) current status (middle panel), and (3) next steps of transition to the OR, PICU or floor (right panel), as well as a timeline in the bottom part. Because HFE researchers organized the HCD process, they used their expertise and knowledge of the literature to also provide input to the design process and the list of HFE design principles. For instance, we integrated several usability heuristics [29, 30] in the design of T3 such as consistency and minimization of workload. Safety is another important usability heuristic, which was also emphasized by nursing leaders (step 5) and included in the list of design principles. The design principles for T3 addressed all three domains of cognitive, physical and organizational HFE (see Table 4).

Table 4 –

Human Factors and Ergonomics (HFE) Design Principles for T3 (Teamwork Transition Technology)

HFE Design Principles Application in T3
Situation awareness

  • Information is presented to support three levels of situation awareness: perception, comprehension, projection; therefore, information on T3 in different panels (structure)

  • Time and time elapsed. T3 shows clock with current time, and two running clocks for time since injury and time in ED.

  • Refreshed automatically and continuously. T3 refreshes automatically information; e.g. when a parent/family/caregiver arrives and provides their phone number.
Physical and visual access

  • Available in multiple locations. T3 is available in multiple locations, e.g. trauma bay, OR, PICU. For instance, users in the OR can view T3 and know the patient’s status in order to prepare for surgery.

  • Mobile. T3 is available on a display that can be moved and follow the care team during patient transport.

  • Viewed on either a large screen or a small screen. A large screen could be available in a trauma bay. T3 could also be viewed on a computer display or laptop from any location.

  • Visible. T3 is in an optimal location in the trauma bay so that team members can view information easily and quickly.
Visual support
  • Information is displayed in a visual format whenever applicable and relevant:
    • Mannequin: T3 includes a visual representation of the patient’s front and back with relevant information (e.g. location of injuries, access points).
    • Timeline: T3 shows trends in vital signs and their relationships.
    • Traffic lights: T3 shows status of various activities needed for transition to OR using red/green traffic lights.
Information integration and interpretation

  • Integrated information. T3 displays integrated, synthesized information that is available in various elements of the EHR.

  • Duplicated information. A few information elements are duplicated to support interpretation; e.g. vital trends on timeline and current values of vital signs on right side of timeline.
Consistency

  • Consistent with other parts of the EHR. The banner in T3 is consistent with the banner in the EHR. Information on unidentified patient (e.g. “xxAruba, Unident13”) is consistent with convention used by hospital.
Minimization of workload

  • Auto-populated. T3 automatically pulls data from the EHR and devices.

  • Minimum set of information elements. T3 shows 47 information elements that are most useful and important for members of the pediatric trauma team. Only a small number of information elements are duplicated.
Safety

  • Ensuring accuracy and timeliness of data displayed on T 3

Open in a new tab

Note: ED = Emergency Department, OR = Operating Room, PICU = Pediatric Intensive Care Unit, EHR = Electronic Health Record.

4.5. Usability Evaluation

Usability of T3 was first evaluated in individual feedback sessions with 4 clinicians (step 4). The SUS scores varied from 60 to 98 with a mean of 85 and a standard deviation of 14. When asked to describe three things they liked about T3, clinicians mentioned that T3 displayed a lot of well-organized information that showed trends and provided timely updates on the patient status in a single display. Two of the four clinicians mentioned that some text boxes had a lot of information, e.g. detailed information on individual physicians. This feedback was used to revise the T3 mock-up.

Table 5 provides information on participants in the second usability evaluation conducted in Step 7. Most were positive about T3 (Figure 4). The overall mean SUS score for T3 was 74.4, which is above the cut-off score for “good” (71.4) but below excellent (85.5) [31].

Table 5 –

Step 7: Participants in the Usability Evaluation of T3 (Teamwork Transition Technology) (N=36)

User groups 3 people from each of the following 12 groups:

  1. ED attending physician

  2. ED resident

  3. ED nurse

  4. Anesthesiology attending

  5. Anesthesia resident

  6. Anesthetist

  7. OR nurse

  8. Surgery attending

  9. Surgery resident (trauma chief)

  10. Surgical technician

  11. PICU attending physician

  12. PICU nurse.
Gender 50%: women
Age 61%: younger than 40 years
Open in a new tab

Note: ED = Emergency Department, OR = Operating Room, PICU = Pediatric Intensive Care Unit.

Figure 4 –

Figure 4 –

Open in a new tab

Step 7: Usability of T3 (Teamwork Transition Technology)

Note: The last 4 items (below the line) are negatively worded.

5. Discussion

This study shows how an HCD process can produce a usable health IT (T3 or Teamwork Transition Technology) for multiple team members involved in pediatric trauma care transitions. We provided a detailed description of the HCD process, which is rarely done especially for the design of team health IT. The HCD process unfolded over 30 months with activities aimed at analyzing pediatric trauma care transitions, co-designing the technology with clinicians, and gathering and integrating feedback from multiple stakeholders. The seven steps of the HCD process led to the creation of a technology mock-up with high usability.

5.1. Participation of Multiple Perspectives via Multiple Methods

Designing team health IT can be particularly challenging because team members have different information needs. In this study, we paid particular attention to the systematic integration of multiple perspectives in the design process. This was accomplished with both direct and indirect user participation in all phases of the HCD process (i.e., analysis, design and feedback), using multiple methods.

User participation in an HCD process for technology design can take different forms, such as direct participation in decision-making for technology design and indirect participation via data collection [32]. While users of T3 were involved directly in the technology design process as seven clinicians participated in the design team and made decisions about the design of T3, others were involved indirectly in multiple stages of the HCD process. For instance, in Step 1, we interviewed 18 clinicians involved in the ED-OR-PICU pediatric trauma care transitions; interview data were used to define the goals of T3. Users were also involved in the patient safety analysis of Step 6 and the evaluation of the T3 mock-up in Step 7. Involving all groups of users along with other stakeholders (e.g. nursing leadership, trauma management) is critical for developing a usable technology [33], but can be challenging. For instance, in Step 3 of collaborative design, not all clinicians were able to participate in all design sessions; this is a major challenge for participatory methods in health care as clinicians are busy with clinical responsibilities. This calls for the development of efficient design methods [34]. Because not all groups of users were directly involved in the design of T3, it is possible that important input may have been missed. While we found a high usability score for T3 in Step 7, a few survey respondents provided negative feedback. For instance, 23% of survey respondents either disagree (6%) or neither agree or disagree (17%) with the statement on overall satisfaction with T3. In informal debriefings after the usability evaluation, some participants indicated that T3 may not be useful in the trauma bay, in particular if the trauma chief says things out loud and help everyone maintain a shared mental model. Other negative feedback related to the issue of information accuracy, which was also identified in the patient safety analysis of Step 6. These issues are important to address in the next phase of design and implementation of T3.

The participatory HCD process integrated multiple perspectives through multiple methods and mechanisms: interviews (steps 1 and 2), design sessions (step 3), individual feedback (steps 4, 5 and 7) and a structured PRA or proactive risk analysis (step 6). This variety of participatory approaches ensured that multiple perspectives provided input in all stages of the HCD process. In particular, the co-design sessions (step 3) allowed for collaboration among different clinical roles; therefore, facilitating the emergence and integration of multiple perspectives through discussion, confrontation and convergence [35].

5.2. Integrating HFE in Team Health IT Design

Our study showed how HFE can be integrated into the process of team health IT design, i.e., through the use of HFE methods and the application of HFE design principles. Our HCD process included the following HFE methods: SEIPS analysis (step 1), co-design sessions (step 3), feedback and usability evaluation of T3 mock-ups (steps 4 and 7), and proactive risk analysis (step 6). Experts recommend the use of multiple and mixed HFE methods, in particular in health IT design [36, 37]. Both quantitative and qualitative data are needed to develop a usable technology [38]. In our study, we collected both quantitative survey data (i.e. SUS or System Usability Scale [28]) and qualitative interview data in Steps 4 (individual feedback from 4 clinicians) and 7 (scenario-based evaluation of T3 mock-up with 36 clinicians).

The HFE discipline has developed a large compendium of “principles” [39], in particular for designing usable technologies. In the cognitive HFE domain, design principles include usability heuristics [29, 40]. In the physical HFE domain, design of technologies tackles visual access and physical dimensions. In the organizational HFE domain, technologies need to fit with organizational and team processes. In this study, we produced HFE design principles for T3 (see Table 4), which cover the domains of cognitive (i.e. situation awareness, visual support, information integration and interpretation, minimization of workload), physical (i.e. physical and visual access) and organizational (i.e. consistency and safety) HFE. These design principles emerged with input from clinicians and nursing leaders and integrated expertise of the HFE researchers. Because HFE researchers were actively involved in the HCD process, they provided an important perspective as well as knowledge of HFE design. The list of design principles in Table 4 may be applicable to other team health IT, e.g. integrated displays and status boards.

5.3. Adaptability of HCD Process

We adapted the HCD process from the ISO [21], which has similarities with the process Schiro et al. [12] applied to design a pediatric ED patient prioritization tool. We conducted a SEIPS analysis of care transitions, similar to the work system analysis of Schiro and colleagues. Another similarity was the evaluation stage; we conducted an evaluation of the T3 mock-up and Schiro et al. conducted user testing of their mock-up and prototype. Our HCD process was based on the co-design model [35], which included a series of four collaborative design sessions in which clinicians were directly involved in designing the team health IT. Schiro et al. presented their mock-up to a multidisciplinary focus group for feedback and design suggestions, which is more closely aligned with a user-centered design process and indirect participation [22].

Feedback in HCD is critical. Similar to Schiro et al. [12], our HCD process included multiple feedback loops, in particular between Steps 3 to 7 in the phases of design activities and feedback (Figure 1). For instance, 4 clinicians provided feedback on an early mock-up of T3 (step 4), which was discussed in a design session (step 3) and led to revisions to the design. Iterative feedback is important as many different opinions and perspectives need to be assessed and integrated during the design process.

The HCD process itself is adaptable and may change over time. We adapted the HCD process as we gathered input, in particular from nursing leadership (step 5). We conducted the proactive risk analysis (step 6) after nursing leaders expressed concern about the safety of T3, which could provide wrong or delayed information. Safety was one of the HFE design principles of T3 (see Table 4). This analysis will be particularly important in the implementation of T3: issues of where the data come from will need to be addressed. In a separate analysis, we found that about 90% of the information elements on T3 could be found in the EHR; the proactive risk analysis brought up the important issue of reliability of and timely access to information in the EHR.

5.4. Limitations

Our study provides detailed information on an HCD process; however, we did not compare our process and its outcome to a non-HCD process or to another form of HCD. Such comparative studies could be useful, but may be difficult to practically conduct. Our study adds to the literature on the benefits of HCD and integration of HFE methods and design principles. Finally, because we developed and evaluated a technology mock-up, we do not have information on the actual use of the technology. Evaluation after implementation is critical to fully assess the fit of the technology in the entire work system and workflow.

6. Conclusion

We showed how an HCD process integrated multiple HFE methods, applied design principles in the cognitive, physical and organizational HFE domains, and produced a usable team health IT. Integrating multiple perspectives in the design process is particularly important as T3, or Teamwork Transition Technology, aims to support the work of a large, distributed team involved in pediatric trauma care transitions. Future research should focus on the design of team health IT as patient care is increasingly performed by fluid teams that are often distributed spatially, organizationally and temporally.

Summary table.

What is already known:

  • Human-centered design (HCD) is a useful approach to develop usable health information technologies; but its application has focused on technologies used by individual users.

  • HFE methods and principles are increasingly used; but how they are incorporated in a human-centered design process for team health IT has been rarely addressed.

What this study added to our knowledge:

  • Human-centered design (HCD) of team health IT should rely on multiple HFE methods and apply various design principles in all three HFE domains of cognitive, physical and organizational HFE (e.g. situation awareness, physical and visual access, information integration).

  • Multiple perspectives can be integrated in the human-centered design (HCD) of team health IT with the use of various methods (e.g. interviews, collaborative design sessions, usability evaluation) and mechanisms (e.g. direct and indirect participation) at all stages of the design process.

Acknowledgments

We would like to thank all of the clinicians and health care leaders who participated in various steps and activities of the human-centered design process of T3.

Funding

Funding for this research was provided by the Agency for Healthcare Research and Quality (AHRQ) [Grant No. R01 HS023837]. The project described was supported by the Clinical and Translational Science Award (CTSA) program, through the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (NCATS), [Grant UL1TR002373]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies. We thank the study participants, as our research would not be possible without them.

Appendix –. Information Elements of T3

No. Information elements Description
Patient information
1 Patient name Patient’s name
2 Gender Patient’s gender
3 Age Patient’s age
4 Birthday Patient’s birthday
5 Weight Patient weight
6 Allergies Patient’s allergies to medication, e.g. penicillin
7 Pertinent medical history Patient’s chronic conditions; e.g. history of asthma and patient’s medication(s) taken at home
8 Home medications Patient’s home medications
9 Level Criteria patient met prior to arrival (i.e. level 1 or 2)
10 Current time Actual time
Prior to admission (PTA)
11 Pre-hospital report Includes radio report, interventions and EMS report
12 Time of injury Time the injury occurred
Patient family/caregiver
13 Power of Attorney (POA) presence Parent/guardian arrived to the hospital or not
14 Contact information Parent/guardian name and phone number to contact
Time elapsed
15 Time since injury Time since injury occurred
16 Time in ED Time since patient in ED
Current
17 Current injuries Patient’s traumatic injuries
18 Total in Patient’s total inputs, including crystalloid, packed red blood cells (PRBC), fresh frozen plasma (FFP) and platelets (PLT)
19 Total out Patient’s estimated blood loss
Mannequin
20 Location of injuries Location of patient’s traumatic injuries, including spinal precautions (i.e. c-collar and thoracic lumbar spine (T&L))
21 Lines, drains and airway (LDAs) Patient’s lines (including IV type and size), drains and airway
Timeline
22 ED arrival Time when the patient arrived to the ED
23 Massive transfusion protocol (MTP) Time when MTP
24 Drips Patient’s continuous infusions (e.g. paralytic, sedation, vasoactive infusions)
25 Last analgesic Last analgesic administered for pain management
26 CPR start Time since CPR started
27 CPR stop Time since a change in provider, stop to do pulse check, or administer medication
28 Total CPR time Sum of CPR time
29 Heart rate Patient’s heart rate trends
30 Oxygen saturation Patient’s oxygen saturation trend
31 Systolic, diastolic and mean arterial blood pressure Patient’s systolic, diastolic and mean arterial blood pressure trends
32 Temperature Patient’s temperature trend
Care team members
33 Trauma attending Trauma attending caring for patient
34 EM attending EM attending caring for patient
35 Anesthesia attending Anesthesia attending caring for patient
36 Trauma chief Trauma chief caring for patient
37 Primary ED nurse Primary ED nurse caring for patient
38 Neurosurgery consult Neurosurgery consult caring for patient
39 Orthopedic consult Orthopedic consults caring for patient
Transition to OR
40 Admit order placed Yes/no admit order placed
41 OR card dropped Yes/no OR card dropped
42 OR ready Yes/no OR ready
43 PICU notified Yes/no PICU notified
Transition to PICU/floor
44 Anticipated unit Location where the patient is anticipated to go after the OR
45 PICU attending PICU attending name
46 PICU fellow PICU fellow name
47 PICU care team leader PICU care team leader name
Open in a new tab

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Ratwani RM, Reider J, and Singh H, A decade of health information technology usability challenges and the path forward. JAMA, 2019. 321(8): p. 743–744. [DOI] [PubMed] [Google Scholar]
  • 2.Sittig DF, et al. , Current challenges in health information technology–related patient safety. Health Informatics Journal, 2018. 26(1): p. 181–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Carayon P and Hoonakker P, Human factors and usability for health information technology: Old and new challenges. Yearbook of Medical Informatics, 2019. 28(1): p. 71–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Turner P, Kushniruk A, and Nohr C, Are we there yet? Human factors knowledge and health information technology - the challenges of implementation and impact. Yearbook of Medical Informatics, 2017. 26(1): p. 84–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Pickup L, et al. , Development of a clinical interface for a novel newborn resuscitation device: Human factors approach to understanding cognitive user requirements. JMIR Human Factors, 2019. 6(2): p. e12055. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Militello LG, et al. , Designing colorectal cancer screening decision support: A cognitive engineering enterprise. Journal of Cognitive Engineering and Decision Making, 2016. 10(1): p. 74–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Ray JM, et al. , Computerized clinical decision support system for emergency department-initiated buprenorphine for opioid use sisorder: User-centered design. JMIR Human Factors, 2019. 6(1): p. e13121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Carayon P, et al. , Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study. BMJ Quality & Safety, 2020. 29: p. 329–340. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Babione JN, et al. , Human-centred design processes for clinical decision support: A pulmonary embolism case study. International Journal of Medical Informatics, 2020. 142. [DOI] [PubMed] [Google Scholar]
  • 10.Beyer H and Holtzblatt K, Contextual Design: Defining Customer-Centered Systems. 1997, San Francisco, CA: Morgan Kaufmann Publishers. [Google Scholar]
  • 11.Wu P, et al. , Supporting emergency medical care teams with an integrated status display providing real-time access to medical best practices, workflow tracking, and patient data. Journal of Medical Systems, 2017. 41(12): p. 186. [DOI] [PubMed] [Google Scholar]
  • 12.Schiro J, et al. , Applying a human-centered design to develop a patient prioritization tool for a pediatric emergency department: Detailed case study of first iterations. JMIR Human Factors, 2020. 7(3): p. e18427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Wang X, et al. , Design and evaluation of an integrated, patient-focused electronic health record display for emergency medicine. Applied Clinical Informatics, 2019. 10(04): p. 693–706. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Guarrera TK, et al. , Cognitive engineering design of an emergency department information system, in Cognitive Systems Engineering in Health Care, Bisantz AM, Burns CM, and Fairbanks RJ, Editors. 2014, CRC Press. p. 43–74. [Google Scholar]
  • 15.Nemeth C, et al. , Developing a cognitive and communications tool for Burn Intensive Care Unit clinicians. Military Medicine, 2016. 181(suppl_5): p. 205–213. [DOI] [PubMed] [Google Scholar]
  • 16.Wooldridge AR, et al. , Complexity of the pediatric trauma care process: Implications for multi-level awareness. Cognition, Technology & Work, 2019. 21: p. 397–416. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Hoonakker PLT, et al. , Information flow during pediatric trauma care transitions: Things falling through the cracks. Intern Emerg Med, 2019. 14(5): p. 797–805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Hoonakker PLT, et al. , Proactive risk assessment of team health IT for pediatric trauma care transitions (T3), in Health and Social Care Systems of the Future: Demographic Changes, Digital Age and Human Factors, Cotrim TP, et al. , Editors. 2019, Springer International Publishing: Cham. p. 213–221. [Google Scholar]
  • 19.Hoonakker PLT, et al. , Scenario-based evaluation of team health IT to support pediatric trauma care transitions. Applied Clinical Informatics, To be published. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Wooldridge AR, et al. , Work system barriers and facilitators in inpatient care transitions of pediatric trauma patients. Applied Ergonomics, 2020. 85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.International Organization for Standardization, ISO 9241–210 Ergonomics of Human-System interaction - Part 210: Human-Centred Design for Interactive Systems. 2010, ISO: Geneva, Switzerland. [Google Scholar]
  • 22.Norman DA and Draper SW, User-Centered System Design: New Perspectives on Human-Computer Interaction. 1986, Hillsdale, NJ: Lawrence Erlbaum Associates. [Google Scholar]
  • 23.Kushniruk A and Nohr C, Participatory design, user involvement and health IT evaluation, in Evidence-Based Health Informatics, Ammenwerth E and Rigby M, Editors. 2016, IOS Press: Amsterdam, The Netherlands. p. 139–151. [PubMed] [Google Scholar]
  • 24.Carayon P, et al. , Work system design for patient safety: The SEIPS model. Quality & Safety in Health Care, 2006. 15(Supplement I): p. i50–i58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Carayon P, et al. , Human factors systems approach to healthcare quality and patient safety. Applied Ergonomics, 2014. 45(1): p. 14–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Wooldridge AR, et al. , SEIPS-based process modeling in primary care. Applied Ergonomics, 2017. 60: p. 240–254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Parush A, et al. , Can teamwork and situational awareness (SA) in ED resuscitations be improved with a technological cognitive aid? Design and a pilot study of a team situation display. Journal of Biomedical Informatics, 2017. 76: p. 154–161. [DOI] [PubMed] [Google Scholar]
  • 28.Brooke J, SUS: A “quick and dirty” usability scale, in Usability Evaluation in Industry, Jordan PW, Werdmeester BA, and McClelland AL, Editors. 1996, Taylor & Francis: London. p. 189–194. [Google Scholar]
  • 29.Zhang J, et al. , Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics, 2003. 36: p. 23–30. [DOI] [PubMed] [Google Scholar]
  • 30.Molich R and Nielsen J, Improving a human-computer dialogue. Communications of the ACM, 1990. 33(3): p. 338–348. [Google Scholar]
  • 31.Bangor A, Kortum PT, and Miller JT, An empirical evaluation of the System Usability Scale. International Journal of Human–Computer Interaction, 2008. 24(6): p. 574–594. [Google Scholar]
  • 32.Haines H, et al. , Validating a framework for participatory ergonomics (the PEF). Ergonomics, 2002. 45(4): p. 309–327. [DOI] [PubMed] [Google Scholar]
  • 33.Kujala S, User involvement: A review of the benefits and challenges. Behaviour & Information Technology, 2003. 22(1): p. 1–16. [Google Scholar]
  • 34.Hundt AS, et al. , Conducting an efficient proactive risk assessment prior to CPOE implementation. International Journal of Medical Informatics, 2013. 82: p. 25–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Détienne F, Collaborative design: Managing task interdependencies and multiple perspectives. Interacting with Computers, 2006. 18(1): p. 1–20. [Google Scholar]
  • 36.Carayon P, et al. , A systematic review of mixed methods research on human factors and ergonomics in health care. Applied Ergonomics, 2015. 51: p. 291–321. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Valdez RS, McGuire KM, and Rivera AJ, Qualitative ergonomics/human factors research in health care: Current state and future directions. Applied Ergonomics, 2017. 62: p. 43–71. [DOI] [PubMed] [Google Scholar]
  • 38.Carayon P and Salwei ME, Moving toward a sociotechnical systems approach to continuous health information technology design: The path forward for improving electronic health record usability and reducing clinician burnout. Journal of the American Medical Informatics Association, 2021. 28(5): p. 1026–1028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Salvendy G, Handbook of Human Factors and Ergonomics. 2012, Hoboken, NJ: John Wiley & Sons. [Google Scholar]
  • 40.Scapin DL and Bastien JMC, Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behaviour & Information Technology, 1997. 16(4): p. 220–231. [Google Scholar]

RESOURCES