A systematic review of Human Factors and Ergonomics (HFE)-based healthcare system redesign for quality of care and patient safety

Abstract

Healthcare systems need to be redesigned to provide care that is safe, effective and efficient, and meets the multiple needs of patients. This systematic review examines how Human Factors and Ergonomics (HFE) is applied to redesign healthcare work systems and processes and improve quality and safety of care. We identified twelve projects representing 23 studies and addressing different physical, cognitive and organizational HFE issues in a variety of healthcare systems and care settings. Some evidence exists for the effectiveness of HFE-based healthcare system redesign in improving process and outcome measures of quality and safety of care. We assessed risk of bias in 16 studies reporting the impact of HFE-based healthcare system redesign and found varying quality across studies. Future research should further assess the impact of HFE on quality and safety of care, and clearly define the mechanisms by which HFE-based system redesign can improve quality and safety of care.

Practitioner Summary

Existing evidence shows that HFE-based healthcare system redesign has the potential to improve quality of care and patient safety. Healthcare organizations need to recognize the importance of HFE-based healthcare system redesign to quality of care and patient safety, and invest resources to integrate HFE in healthcare improvement activities.

1. Introduction

Since the US Institute of Medicine (IOM) released the report “To Err is Human: Building a Safer Health System” in 1999 (Kohn, Corrigan and Donaldson 1999), quality of care and patient safety have become major concerns in the U.S. and around the world. To provide care that is safe, effective and efficient, and meets the multiple needs of patients, healthcare systems are in great need of redesign (Institute of Medicine Committee on Quality of Health Care in America 2001). Considerable efforts and substantial resources have been invested in the past decade to prevent medical errors and improve patient safety (Bates et al. 1999; Landrigan et al. 2004; Pronovost et al. 2006); however, evidence on the effectiveness of these efforts is ambiguous and limited (Landrigan et al. 2010; Leape and Berwick 2005; Shekelle et al. 2011; Vincent et al. 2008). The 2005 report by the IOM and the National Academy of Engineering highlighted Human Factors and Ergonomics (HFE) as a key systems engineering approach to improve healthcare work systems and processes and, therefore, quality of care and patient safety (Reid et al. 2005). Evidence shows that lack of attention to HFE in the design and implementation of healthcare technologies, processes, workflows, jobs, teams and sociotechnical systems can result in poor quality of care and patient safety incidents, such as medication errors and adverse drug events (Institute of Medicine 2006; Leape et al. 1995), as well as undesirable employee and organizational outcomes, such as job dissatisfaction, burnout, injuries and turnover (Carayon et al. 2006).

Although HFE is now recognized as important to healthcare quality and patient safety (Carayon, Alyousef and Xie 2012; Gurses, Ozok and Pronovost 2012), we have limited empirical information on HFE applications in healthcare system redesign (Carayon, Xie and Kianfar 2014). We conducted a systematic review, including assessment of risk of bias, to examine how HFE has been applied to redesign healthcare work systems and processes. We also collected evidence on the effectiveness of HFE in improving quality of care and patient safety. After reviewing the results of the systematic review, we discuss gaps in current research on HFE in healthcare system redesign, and propose recommendations for future research.

2. HFE-based healthcare system redesign

2.1 Definition

We adapted the SEIPS (Systems Engineering Initiative for Patient Safety) model of work system and patient safety (Carayon et al. 2006; Carayon et al. 2014) to examine the application of HFE to healthcare system redesign. The SEIPS model is a systems engineering model anchored within HFE. As it integrates Donabedian’s structure-process-outcome model (Donabedian 1988) and the work system model developed by Carayon and Smith (Carayon 2009; Carayon and Smith 2000; Smith and Carayon 2000; Smith and Carayon-Sainfort 1989), the SEIPS model highlights how work system design (structure) is linked to patient safety (outcome) through care processes. As shown in Figure 1, the SEIPS model is used to describe an existing healthcare system as well as the redesigned healthcare system. The dashed lines represent four major phases of healthcare system redesign: analysis, design, implementation and evaluation (Meister and Enderwick 2001; Parker and Wall 1998; Wilson and Morrisroe 2005). For instance, the dashed lines from outcomes to process and from process to work system in the upper part of Figure 1 represent the phase of system analysis where information is collected to understand how negative outcomes can result from process deficiencies, which in turn are influenced by the work system.

Figure 1.

SEIPS Model of Healthcare System Redesign (adapted from Carayon et al. (2006)).

HFE applications in the four phases of system redesign for quality of care and patient safety can be categorized as: (1) use of HFE tools, (2) use of HFE knowledge, and (3) direct involvement of HFE professionals in healthcare organizations (Carayon 2010). Therefore, we defined HFE-based healthcare system redesign as the deployment of HFE tools, knowledge and professionals in the analysis, design, implementation and evaluation of healthcare work system changes to improve care processes and patient, employee and organizational outcomes. Table 1 provides examples of HFE-based healthcare system redesign.

Table 1.

Examples of HFE-based healthcare system redesign.

Phases of healthcare system redesign	Examples of HFE application
	Use of HFE tools	Use of HFE knowledge	Direct involvement of HFE professionals
Analysis	Cognitive task analysis to identify information requirements for using a patient-controlled analgesia (PCA) pump (Lin et al. 1998) Workflow analysis to identify deficiencies of radiotherapy treatment delivery process (Chan et al. 2010)	Learning from aviation HFE experience for healthcare physical environment redesign (de Korne et al. 2012) Education conference organized to teach HFE knowledge related to facility design and safety (Reiling et al. 2004)	Heuristic evaluation conducted by HFE experts to identify HFE issues of a clinical information system for anesthesiology (Beuscart-Zephir et al. 2005) Field observation conducted by HFE experts to understand tasks performed by radiation therapists (Chan et al. 2010)
Design	User testing of prototypes in the iterative design of computerized provider order entry (CPOE) (Beuscart-Zephir, Pelayo and Bernonville 2010) Proactive risk assessment to identify and address potential process failures (Reiling et al. 2004)	Development of checklist prototypes based on HFE design principles (e.g., format, length, graphic layout) (de Vries et al. 2009) Participatory design of an operating room scheduling system with main focus on usability and user acceptance (Hasvold and Scholl 2011)	Involvement of HFE experts in a Delphi process to identify HFE design specifications for an emergency department (ED) telemetry system (Kobayashi et al. 2013) Development of design principles and recommendations for an ED charting system with HFE experts (Kobayashi et al. 2011)
Implementation	Field observation, interview and focus group to identify barriers to user acceptance of a decision support system (Lesselroth et al. 2011) Proactive risk assessment to identify vulnerabilities associated with implementation of CPOE in intensive care units (ICUs) (Hundt et al. 2013)	System implementation informed by HFE implementation principles (e.g., participation) – a team of healthcare providers leading the implementation of a surgical checklist (de Vries, Prins, et al. 2010) Use of a sociotechnical framework (fit between individuals, task and technology framework) to guide implementation of a decision support system (Lesselroth et al. 2011)	Training of nurses on health and safety of patient handling provided by HFE experts (Smedley et al. 2003) Regular communication with HFE experts to solve problems encountered in implementation of patient lift and transfer equipment (Schoenfisch et al. 2011)
Evaluation	User testing to evaluate usability of a redesigned CPOE (Chan et al. 2011a) Survey to examine user experience of healthcare providers with a redesigned code cart medication drawer (Rousek and Hallbeck 2011)	Heuristic evaluation of smart infusion pump conducted by trained clinical and technical experts (Namshirin, Ibey and Lamsdale 2011) Field observation guided by cognitive engineering frameworks (e.g., expert decision-making, human-computer interaction, mutual awareness) (Patterson, Cook and Render 2002)	Walk-through assessments of the use of patient lift and transfer equipment by HFE experts (Schoenfisch et al. 2011) HFE experts identifying medication errors and associated work system factors resulting from implementation of CPOE in ICUs (Wetterneck et al. 2011)

2.2 Characteristics

HFE-based healthcare system redesign addresses a range of physical, cognitive and organizational HFE issues that can potentially affect quality of care and patient safety (Carayon 2006, 2007; Gurses, Ozok and Pronovost 2012). Examples of physical HFE issues include mismatches between task requirements and physical characteristics of healthcare professionals (e.g., nurses performing strenuous patient handling tasks), healthcare technologies with inappropriate physical dimensions (e.g., too small font size on computer screen), and physical layout and environment that do not support clinical tasks (e.g., central nursing station in a location that limits line of sight for patient monitoring). Examples of cognitive HFE issues include limited information for clinical decision-making (e.g., care transition with incomplete patient records), clinical tasks resulting in high cognitive workload (e.g., distraction of nurses during medication administration), and medical devices designed without considering cognitive abilities of healthcare professionals (e.g., drug library of smart infusion pump with multiple concentrations). Examples of organizational HFE issues include job stress and burnout of healthcare professionals (e.g., turnover and understaffing in nursing homes), ambiguous roles and responsibilities of healthcare professionals (e.g., role ambiguity in care coordination), and ineffective teamwork in health care (e.g., miscommunication between care team members).

HFE-based healthcare system redesign differs from general quality improvement interventions as it incorporates the three core characteristics of HFE: (1) use of a systems approach, (2) design-driven approach, and (3) focus on both system performance and well-being (Dul et al. 2012). First, HFE-based healthcare system redesign applies a systems approach, which highlights interactions among work system elements and levels, the dynamic impact of individual work system elements on the whole system, and links between work system, care processes and system outcomes (Carayon et al. 2006; Waterson 2009; Wilson 2000). According to the work system model (Carayon 2009; Carayon and Smith 2000; Smith and Carayon 2000; Smith and Carayon-Sainfort 1989), a healthcare system consists of multiple people (e.g., patients, physicians, nurses) performing different tasks (e.g., direct patient care, maintaining clinical documentation) with various tools and technologies (e.g., medical devices, electronic health record) in a physical environment (e.g., hospital unit, patient room) under certain organizational conditions (e.g., safety culture, work schedule, teamwork). These interrelated work system elements influence care processes, which further influence patient, employee and organizational outcomes (Carayon et al. 2006). The objective of HFE-based healthcare system redesign, therefore, is to achieve better patient, employee and organizational outcomes through the improvement of healthcare work systems and care processes.

Second, while HFE can contribute to all phases of healthcare system redesign, we propose that the core of HFE-based healthcare system redesign is to involve HFE in the phases of system design and implementation. HFE applications to system analysis and evaluation are important but have limited influence on actual redesign. HFE analysis focuses on the existing healthcare system and aims to understand how undesired system outcomes (e.g., harmful medication errors) are associated with process deficiencies (e.g., duplicate medication ordering) and HFE issues in the work system (e.g., poor design of CPOE interface, limited communication between healthcare providers) (see dashed line 1 in Figure 1). Performing an HFE-based system analysis, however, does not fully answer the question of how to redesign the work system to improve care processes and system outcomes. HFE evaluation, on the other hand, focuses on the redesigned healthcare system and examines whether HFE issues in the work system have been addressed and how an intervention influences care processes and system outcomes (see dashed line 4 in Figure 1). HFE-based system evaluation can inform future iterations of healthcare system redesign. To ensure that the redesign of a healthcare system is HFE-based, HFE needs to be considered at the minimum in the phases of system design and implementation (see dashed lines 2 and 3 in Figure 1). These phases of system design and implementation respectively define the content (what is the redesign) and process (how the redesign is implemented) of work system redesign (Carayon 2007).

Finally, HFE-based healthcare system redesign aims to improve both system performance and human well-being. The ultimate goal of this redesign is to improve quality of care and patient safety, which are components of system performance. The impact of HFE-based healthcare system redesign on quality of care and patient safety can be assessed with measures of care processes and patient outcomes (Donabedian 1988). Measures of care processes include, for example, efficiency, processing time, usability/acceptance of changes, compliance with clinical protocols, ordering and performance of tests, and screening for disease (Carayon et al. 2010). Measures of patient outcomes include, for example, mortality, complications, quality of life, medical errors and patient satisfaction. From an HFE viewpoint, in addition to quality of care and patient safety, healthcare system redesign should enhance well-being of healthcare professionals, such as job satisfaction and motivation. Therefore, quality improvement interventions that would increase the workload of already busy healthcare professionals and lead to burnout are not considered HFE-based healthcare system redesign.

Empirical studies of HFE-based healthcare system redesign are limited and scattered over a wide range of clinical topics. A systematic review, therefore, is necessary to collect existing evidence on HFE-based healthcare system redesign and assess its impact on quality of care and patient safety.

3. Methods

This systematic review follows the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (Liberati et al. 2009; Moher et al. 2009). Methods proposed by the Cochrane Effective Practice and Organization of Care Group (Higgins and Green 2011) and the Centre for Reviews and Dissemination (2009) were also adapted and applied for study search (e.g., design of search strategy, set of restrictions), study selection (e.g., process for selecting studies) and data extraction (e.g., development of data collection form).

3.1 Exclusion and inclusion criteria

The literature review was limited to peer-reviewed journal articles written in English. Studies were excluded if: (1) they were not related to healthcare system redesign (e.g., Casler and Cook 2003; Wong and Richardson 2010), (2) they described best practices or methods (e.g., Beuscart-Zephir et al. 2007; Henriksen, Joseph and Zayas-Caban 2009), (3) they only presented a research protocol (e.g., Hysong et al. 2009; Marshall et al. 2011), and (4) they reported on general quality improvement (e.g., Amoore and Ingram 2002; Mutter 2003).

Due to the complexity of HFE-based healthcare system redesign, multiple studies could be related to one project. For example, a study might involve a work system analysis and usability evaluation to develop an intervention, while other studies might implement the intervention and assess its impact on care processes and/or patient outcomes. Therefore, studies from the same project were grouped and then screened for inclusion. A project was included if it met all four inclusion criteria:

• The project applied HFE to the phases of system design or implementation;

• The project described the HFE tools, knowledge or professionals employed;

• The project described the intervention; and

• The project reported the impact of the intervention on care processes or patient outcomes.

3.2 Study search and selection

After discussion with the other researcher and upon agreement on inclusion criteria, one researcher conducted the literature search and selected studies (see Figure 2). The search was conducted in three databases from their inception to February 2013: PubMed (from 1950), PsycINFO (from 1872), and Web of Science (from 1965). The search combined terms in four areas: (1) HFE, (2) intervention, (3) healthcare work system elements and (4) care processes and outcomes (see Table 2). While terms from different areas were combined using the Boolean operator “AND”, terms within each area were combined using the Boolean operator “OR”. After removing duplicates, a total of 6863 studies were identified. The title and abstract of each study were screened. A total of 6380 studies were excluded based on the exclusion criteria: 6115 were not related to healthcare system redesign, 216 described best practices or methods, 8 only presented a research protocol, and 41 reported on general quality improvement. Full-text articles were retrieved for the remaining 483 studies. Their reference lists and the later studies citing them were manually searched. Thirty-eight additional studies were found. Studies related to the same project were then grouped for further screening. Among the 451 identified projects, 439 did not meet the inclusion criteria: 135 only reported the analysis phase, 176 only reported the evaluation phase, 37 did not describe the HFE tools, knowledge or professionals being employed, 6 did not describe the intervention, and 85 did not evaluate the impact of intervention on care processes or patient outcomes. We included a total of 12 projects or 23 studies in the review (see Table 3).

Figure 2.

Flow diagram of study search and selection.

Table 2.

Terms used for literature search.

Areas	Search terms
HFE	Human factors, ergonomic(s), macroergonomic(s), socio-technical, task analysis, work analysis, human engineering, cognitive engineering, engineering psychology, usability, usefulness, human-computer interaction, biomechanics
Intervention	Intervention(s/studies), change(s), program(s), recommendation(s), design, implementation
Healthcare work system elements	Work system(s), work situation(s), procedure(s), task(s), healthcare worker(s)/provider(s), health information technology (IT), medical device(s), environment
Care processes and outcomes	Process(es), performance, efficienc(y/ies), patient safety, error(s), adverse event(s), satisfaction, injur(y/ies), quality of care

Table 3.

Projects and studies on HFE-based healthcare system redesign included in the systematic review.

Projects	Studies	References
1	1.1	Chan et al. (2010)
	1.2	Chan et al. (2012)
2	2.1	Chan et al. (2011a)
	2.2	Chan et al. (2011b)
3	3.1	Christofidis et al. (2013)
	3.2	Preece et al. (2012)
	3.3	Preece et al. (2013)
4	4.1	de Korne et al. (2012)
5	5.1	de Vries et al. (2009)
	5.2	de Vries, Dijkstra, et al. (2010)
	5.3	de Vries, Prins, et al. (2010)
6	6.1	Eames et al. (2013)
	6.2	Hoffmann, Russell and McKenna (2004)
	6.3	Hoffmann and Worrall (2004)
	6.4	Hoffmann and McKenna (2006)
	6.5	Hoffmann et al. (2007)
7	7.1	Johnson, Johnson and Zhang (2005)
8	8.1	Kobayashi et al. (2011)
9	9.1	Kobayashi et al. (2013)
10	10.1	Lesselroth et al. (2011)
11	11.1	Lin et al. (1998)
	11.2	Lin, Vicente and Doyle (2001)
12	12.1	Rousek and Hallbeck (2011)

3.3 Assessment of risk of bias

All 12 projects evaluated the impact of HFE-based healthcare system redesign on care processes or patient outcomes; 16 of the 23 studies in the 12 projects actually reported data on care processes or patient outcomes. Two researchers assessed independently the risk of bias in the 16 studies using a specially-developed 30-question instrument (see Table 7 in section 4.4). The instrument was developed based on the checklist of Downs and Black (1998) for assessing quality of randomized and non-randomized healthcare intervention studies. It also included three questions from Tullar et al. (2010): (1) “Were concurrent comparison (control) group(s) used?” (2) “Was the calendar duration of the intervention documented?” and (3) “Was the participation rate reported for employees?” Whenever researchers disagree on the assessment of a study, they met and discussed their assessment until consensus was achieved.

Table 7.

Assessment of risk of bias.

Questions	Studies~
	1.1	2.1	3.1	3.2	4.1	5.2	5.3	6.1	6.5	7.1	8.1	9.1	10.1	11.1	11.2	12.1
0. Were concurrent comparison (control) group(s) used?	Y	Y	Y	Y	Y	N	Y	Y	Y	Y	N	N	N	Y	Y	Y
1. Is the hypothesis/aim/objective of the study clearly described?	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
2. Are the main outcomes to be measured clearly described in the Introduction or Methods section?	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
3. Are the characteristics of participants included in the study clearly described?	N	Y	Y	Y	N	Y	Y	Y	Y	N	N	N	N	N	N	Y
4. Are the interventions of interest clearly described?	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
4a. Was the calendar duration of the intervention documented?	N	Y	N	N	Y	Y	Y	Y	Y	N	Y	Y	Y	N	N	N
5. Are the distributions of principal confounders in each group of participants to be compared clearly described?	N	N	Y	N	N	N	Y	Y	Y	N	N	N	N	N	N	N
6. Are the main findings of the study clearly described?	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
7. Does the study provide estimates of the random variability in the data for the main outcomes?	Y	N	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	N	N	N	N
8. Have all important adverse events that may be a consequence of the intervention been reported?	Y	Y	Y	Y	N	N	N	N	N	N	N	N	N	N	N	N
9. Have the characteristics of participants lost to follow-up been described?	NA	NA	NA	NA	NA	NA	NA	Y	Y	NA	NA	NA	NA	NA	NA	NA
10. Have actual probability values been reported (e.g., 0.035 rather than <0.05) for the main outcomes except where the probability value is less than 0.001?	N	Y	Y	Y	Y	Y	Y	Y	Y	N	N	Y	Y	N	N	Y
11. Were the participants asked to participate in the study representative of the entire population from which they were recruited?	U	U	U	U	U	U	Y	U	U	U	U	U	U	N	U	U
12. Were those participants who were prepared to participate representative of the entire population from which they were recruited?	U	U	U	U	U	U	U	U	U	U	U	U	U	U	U	U
12a. Was the participation rate reported?	N	N	N	N	N	NA	NA	Y	Y	N	N	N	N	N	N	N
13. Were the staff, places, and facilities where the patients were treated, representative of the treatment the majority of patients receive?	Y	U	U	U	Y	Y	Y	Y	Y	U	Y	Y	Y	U	U	Y
14. Was an attempt made to blind participants to the intervention they have received?	Y	Y	Y	Y	U	NA	N	Y	Y	U	NA	NA	NA	U	U	U
15. Was an attempt made to blind those measuring the main outcomes of the intervention?	U	U	U	U	U	NA	U	Y	Y	N	NA	NA	NA	U	U	U
16. If any of the results of the study were based on “data dredging”, was this made clear?	NA	Y	Y	Y	Y	Y	Y	Y	Y	NA	Y	NA	NA	N	NA	NA
17. In trials and cohort studies, do the analyses adjust for different lengths of follow-up of patients, or in case-control studies, is the time period between the intervention and outcome the same for cases and controls?	NA	NA	NA	NA	N	NA	Y	Y	Y	NA	NA	NA	NA	NA	NA	NA
18. Were the statistical tests used to assess the main outcomes appropriate?	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	N	Y	Y	Y
19. Was compliance with the intervention/s reliable?	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
20. Were the main outcome measures used accurate (valid and reliable)?	U	U	U	U	U	Y	Y	Y	Y	U	U	U	Y	U	U	U
21. Were participants in different intervention groups (trials and cohort studies) or were the cases and controls (case-control studies) recruited from the same population?	NA	NA	NA	NA	Y	NA	Y	Y	Y	NA	NA	NA	NA	NA	NA	NA
22. Were participants in different intervention groups (trials and cohort studies) or were the cases and controls (case-control studies) recruited over the same period of time?	Y	Y	Y	Y	Y	NA	Y	Y	Y	Y	NA	NA	NA	Y	Y	Y
23. Were participants randomized to intervention groups?	N	Y	Y	Y	N	N	N	Y	Y	Y	N	Y	N	Y	Y	Y
24. Was the randomized intervention assignment concealed until recruitment was complete and irrevocable?	N	U	U	U	N	N	N	Y	Y	U	N	U	N	N	N	U
25. Was there adequate adjustment for confounding in the analyses from which the main findings were drawn?	N	N	Y	N	N	N	Y	Y	Y	N	N	N	N	N	N	N
26. Were losses of participants to follow-up taken into account?	NA	NA	NA	NA	NA	NA	NA	Y	Y	NA	NA	NA	NA	NA	NA	NA
27. Did the study have sufficient power to detect a clinically important effect where the probability value for a difference being due to chance is less than 5%?	U	U	U	U	U	U	U	Y	Y	U	U	U	U	U	U	U

Y=yes; N=no; U=unable to determine; NA=not applicable.

^~The studies’ numbers refer to the list in Table 3.

3.4 Data extraction

We developed a data collection form, which was pilot-tested on three projects and revised subsequently. One researcher used the data collection form to extract the following data from each project: (1) the setting and country of the project, (2) the healthcare system redesigned, (3) HFE issues (e.g., physical, cognitive, organizational) addressed by the redesign, (4) work system elements (e.g., people, tasks, tools and technologies, organization, environment) affected by the redesign, (5) phases of the redesign process and HFE tools, knowledge and professionals applied in each phase, and (6) impact of the intervention on care processes and patient outcomes, as well as other system outcomes (e.g., employee, organization). The other researcher reviewed the extracted data, and consensus was achieved between the two researchers. The complete data collection form is in Supplementary A.

4. Results¹

4.1 Healthcare system redesign focus

The twelve projects addressed various HFE issues in a wide range of healthcare systems, and developed different interventions to target single or multiple elements of the work system (see Table 4).

Table 4.

Healthcare system redesign focus

Projects~	Healthcare system redesign focus	HFE issues addressed*			Work system elements related to intervention#
		P	C	O	T	T/T	O	E
1	User interface of radiotherapy treatment delivery system		X			X
2	CPOE order set system		X			X
3	Paper medical observation chart used by hospital staff to document physiological observations over time for an individual patient	X	X			X
4	Operating room floor marking	X						X
5	Checklist for surgical pathway from admission to discharge		X	X		X
6	Booklet for providing information to stroke patients		X		X	X	X
7	Redesign of family history-tracking and pedigree drawing program for assessing genetic risk of hereditary cancer syndrome		X			X
8	Paper-based ED charting system	X	X			X
9	Interface of ED telemetry system	X	X	X		X	X	X
10	Computerized decision support system to improve adherence with deep venous thrombosis prophylaxis		X	X		X
11	Interface of a patient-controlled analgesia (PCA) pump		X			X
12	Hospital code cart medication drawer	X	X			X

^~The projects’ numbers refer to the list in Table 3.

^*HFE issues: P=physical; C=cognitive; O=organizational.

^#Work system elements: T=tasks; T/T=tools and technologies; O=organization; E=environment.

4.1.1 HFE issues addressed in healthcare system redesign

Four projects focused on the redesign of medical devices. Project 11 redesigned the interface of a PCA pump to address cognitive HFE issues, such as information and psychological requirements of programming the PCA pump. Project 3 considered both physical (e.g., size, orientation, color) and cognitive (e.g., display of vital signs, detection of abnormal observations) HFE issues in the redesign of a medical observation chart used by hospital staff to document patients’ physiological stability. Similarly, project 8 considered both physical (e.g., labeling, spacing, dimension) and cognitive (e.g., pattern recognition, perception from visual cue) HFE issues in the redesign of a paper-based ED charting system. Project 12 redesigned a hospital code cart medication drawer by addressing both physical (e.g., visibility, grouping, layout) and cognitive (e.g., standardization to reduce mental workload) HFE issues.

Five projects focused on the redesign of health IT, including treatment delivery system (project 1), CPOE (project 2), clinical information system (project 7), clinical telemetry system (project 9) and clinical decision support system (project 10). These five projects addressed cognitive HFE issues associated with health IT design, such as information integration and presentation, and interface usability. In addition, project 10 emphasized organizational HFE or macroergonomic issues (e.g., heterogeneous cohort of clinicians, discordance between clinical guideline and local practice) during the implementation of a clinical decision support system. Project 9 considered both physical (e.g., location of displays, muted alarm speakers, limited workspace) and organizational (e.g., process for system maintenance) HFE issues of an ED telemetry system.

Two projects focused on the redesign of care processes. Project 5 considered both cognitive (e.g., content and format of checklist) and organizational HFE issues (e.g., time constraints, logistics) in the redesign of a surgical pathway from admission to discharge. Project 6 examined cognitive HFE issues (e.g., information needs, reading ability) associated with providing information to stroke patients.

Finally, project 4 focused on physical HFE issues in the operating room, including positioning of surgical devices.

4.1.2 Work system interventions

The interventions implemented in ten of the twelve projects targeted single elements of the work system. Project 4 focused on the physical environment and implemented floor markings in operating rooms. The other nine projects focused on tools and technologies in various healthcare systems: four projects redesigned medical devices (projects 3, 8, 11 and 12), four projects redesigned health IT (projects 1, 2, 7 and 10), and one project redesigned the surgical pathway by implementing a checklist (project 5).

Interventions in two projects targeted multiple elements of the work system. To provide tailored information to stroke patients, project 6 implemented an education and support package, which consisted of (1) a computer system generating tailored written information booklet and (2) support by an occupational therapist with clinical experience in stroke rehabilitation; the occupational therapist reinforced some information pre-hospital discharge and contacted patients via phone post-hospital discharge. Project 9 implemented a multi-element intervention to improve the overall ED telemetry system: (1) installation of distributed telemetry large-screen displays, (2) repositioning of central telemetry displays and speakers, (3) adjustment of alarm volume for audible, less obtrusive notification, (4) adjustment of alarm parameters to reduce false alarms, (5) placement of touchpad input devices for intuitive interaction, (6) coordination of institutional infrastructure for routine maintenance, (7) announcement of study and intervention at ED personnel meetings, (8) group training and on-shift training of ED personnel, and (9) integration of the telemetry system into nurse charting workflow.

4.2 Process of healthcare system redesign

The twelve projects used HFE in different phases of the process of healthcare system redesign (see Table 5). While six projects described HFE in all four phases of the redesign process, i.e. analysis, design, implementation and evaluation (projects 4, 5, 6, 8, 9 and 10), six projects did not report information on the phase of system implementation (projects 1, 2, 3, 7, 11 and 12).

Table 5.

HFE applications in the process of healthcare system redesign.

Phases of redesign process	Objectives	HFE applications (tools, knowledge, professionals)	Projects~
Analysis	Assessment of current system Identification of deficiencies in current system Identification of system constraints and design requirements Learning from HFE literature and other systems or industries	Observation Interview Survey Workflow analysis Task analysis (hierarchical, cognitive) Layout analysis Heuristic evaluation User testing User needs analysis Review of HFE literature Learning from HFE experts	1, 4, 8, 9, 10, 11 6, 7, 11 7, 9, 1, 4, 10 2, 3, 7, 9, 10, 11 4 1, 3, 7, 11 12 6, 7, 9 3, 9 4, 9
Design	Development of design recommendations and solutions Assessment and refinement of prototype	HFE design principles Input from healthcare stakeholders and HFE experts Heuristic evaluation User testing Observation Focus group	1, 2, 3, 5, 6, 8, 11 1, 4, 6, 8, 9, 10, 12 2, 3, 7 6, 7, 8, 10, 12 5 1
Implementation	Implementation of intervention	HFE implementation principles: Communication and feedback Learning and training Continuous improvement Project management	5, 9, 10 9, 10 5, 6, 10 4, 5, 6, 8, 10
Evaluation	Assessment of impact of HFE-based healthcare system redesign	User testing Observation Interview Survey Review of clinical data	1, 2, 3, 11, 12 4 4 6, 7, 9, 12 5, 8, 9, 10

^~The projects’ numbers refer to the list in Table 3.

4.2.1 System analysis

Main activities of HFE-based system analysis included:

• Assessment of current system (studies 1.1, 2.1, 3.3, 4.1, 6.4, 7.1, 9.1 and 10.1),

• Identification of deficiencies in current system (studies 1.1, 1.2, 3.3, 5.1, 7.1, 8.1, 11.1 and 12.1),

• Identification of system constraints and design requirements (studies 2.1, 7.1, 9.1 and 10.1), and

• Learning from HFE literature and other systems or industries (studies 3.3, 4.1 and 9.1).

The most frequently used HFE data collection methods were observation, interview, survey and review of archival data. Data were collected to assess workplace layouts (study 4.1), user needs (studies 7.1 and 9.1) and clinical tasks and workflow (studies 1.1, 1.2, 2.1, 4.1 and 10.1). Specific HFE techniques applied in system analysis included hierarchical task analysis (studies 3.3 and 7.1), cognitive task analysis (study 10.1), workflow analysis (studies 1.1 and 1.2), heuristic usability evaluation (studies 3.3, 7.1 and 11.1) and user testing (studies 7.1 and 12.1). Studies also applied HFE knowledge (e.g., HFE principles for interface design) (studies 3.3 and 11.1) and involved HFE experts (e.g., informal discussion with HFE experts) (study 9.1) in system analysis.

4.2.2 System design

HFE-based system design emphasized the iterative process for creating and refining design recommendations and solutions. To address HFE problems identified in system analysis, researchers proposed design recommendations and solutions based on HFE design principles (e.g., principles for interface design) (studies 1.1, 2.1, 3.3, 5.1, 6.2, 8.1 and 11.1) and input from healthcare stakeholders and HFE experts (e.g., focus group, participatory design) (studies 1.1, 4.1, 6.2, 8.1, 9.1, 10.1 and 12.1). Recommendations were then implemented through the development of prototypes, which were further assessed (e.g., heuristic evaluation, user testing, focus group, observation) and refined until all HFE issues (e.g., usability) were addressed (studies 1.1, 2.1, 3.3, 5.1, 6.2, 7.1, 8.1, 10.1 and 12.1).

4.2.3 System implementation

Various principles for successful HFE-based system implementation have been proposed, such as participation, communication and feedback, learning and training, top management commitment, and project management (Carayon, Alyousef and Xie 2012; Karsh 2004; Smith and Carayon 1995). Communication with stakeholders about the redesign was reported in three projects (studies 5.1, 9.1 and 10.1). Different means of communication (e.g., presentations, information flyers) were used to share information with end users and decision makers to reduce uncertainty and answer questions about the implementation. Two projects described user training designed to promote transfer of knowledge and skills into work practice (studies 9.1 and 10.1). Continuous improvement was mentioned in three projects (studies 5.1, 6.2 and 10.1). Observations, interviews and focus groups were conducted to explore user experience with the redesigned system, identify barriers to system implementation, and develop solutions to address identified problems. In addition, five projects described project management during system implementation, including formation of an implementation team (study 5.3), identification of champions (study 10.1), pilot testing of redesigned system (studies 5.1 and 6.2), and structured implementation (e.g., phased implementation, parallel implementation) (studies 4.1, 5.1 and 8.1).

4.2.4 System evaluation

HFE-based system evaluation assessed the impact of system redesign on care processes, such as task performance, compliance with best practices and response time (studies 1.1, 1.2, 2.1, 3.1, 3.2, 4.1, 5.2, 7.1, 8.1, 9.1, 10.1, 11.2 and 12.1), as well as patient outcomes, e.g., complication, in-hospital mortality and diagnostic error (studies 3.1, 3.2, 5.3, 6.1, 6.5 and 10.1), and employee outcomes, e.g., required physical demands, back injuries, perceived exertion, user satisfaction and safety awareness (studies 1.1, 1.2, 4.1 and 7.1). Multiple HFE data collection methods were used in system evaluation, including observation, interview, survey, review of clinical data and user testing.

Six of the twelve projects included in this review conducted user testing to evaluate the usability of interventions (studies 1.1, 2.1, 3.1, 3.2, 7.1, 11.1, 11.2 and 12.1). These six projects applied a within-subject design; however, one project did not randomize or balance out the order of tasks performed by participants (study 1.1). The other six projects applied different types of study design to evaluate the impact of interventions (Grimshaw et al. 2000): studies 5.2 and 8.1 applied a before-and-after design with no control group; studies 4.1, 9.1 and 10.1 applied a time series design; study 5.3 applied a before-and-after design with a control group; and studies 6.1 and 6.5 applied a randomized control design.

4.3 Impact of HFE-based healthcare system redesign

The twelve projects provided some evidence on the effectiveness of HFE-based healthcare system redesign to improve care processes, patient outcomes and employee outcomes (see Table 6).

Table 6.

Impact of HFE-based healthcare system redesign.

Processes/Outcomes	Measures	Impact	Projects~
Care processes	• Task completion time	↓	1, 2, 3, 7, 11, 12
	• Error rate	↓	1, 3, 8, 9, 11
	• Compliance with recommended best practices	↑	4, 5, 10
	• Acceptance of system changes	↑	10
Patient outcomes	• Complication rate	↓	5
	• In-hospital mortality	↓	5
	• Self-efficacy of patients for accessing medical information	↑	6
	• Veterans Health Administration’s EPRP (External Peer Review Program) performance indicators	↑	10
Employee outcomes	• Mental workload	↓	11
	• Satisfaction	↑	1, 7
	• Safety awareness	↑	4

^~The projects’ numbers refer to the list in Table 3.

4.4 Assessment of risk of bias

Table 7 shows results of risk of bias assessment. Studies 6.1 and 6.5 addressed the largest number of quality criteria (27 of 30), while studies 10.1, 11.1 and 11.2 addressed the lowest number of quality criteria (9 of 30). The average number of quality criteria addressed was 14.

Ten quality criteria were used to assess the quality of reporting (questions 1 to 10). While all studies clearly described their research objectives, outcome measures, interventions and main findings (questions 1, 2, 4 and 6), about half of the studies provided limited data on characteristics of participants, distribution of principal confounders and adverse events associated with interventions (questions 3, 5 and 8). Regarding external validity, twelve studies did not report participation rate (question 12a), and only one study provided information on the representativeness of participants (questions 11 and 12). Internal validity was assessed in terms of bias and confounding. Main issues related to bias were: (1) unclear description of the process for blinding participants and researchers to the intervention² (questions 14 and 15), and (2) lack of information on the reliability and validity of outcome measures (question 20). The main issue related to confounding was inadequate adjustment of confounding in statistical analyses (question 25). Finally, most studies (14 out of 16) did not provide information about statistical power (question 27).

5. Discussion

We reviewed studies on HFE-based healthcare system redesign to assess the impact of HFE on quality of care and patient safety. We identified a total of twelve projects with 23 studies that showed the breadth of HFE applications to healthcare system redesign. We found evidence of the effectiveness of HFE in improving quality of care, such as reduced task completion time (Chan et al. 2010; Chan et al. 2011a; Christofidis et al. 2013; Johnson, Johnson and Zhang 2005; Lin, Vicente and Doyle 2001; Preece et al. 2012; Rousek and Hallbeck 2011), decreased error rate (Chan et al. 2010; Christofidis et al. 2013; Kobayashi et al. 2011; Kobayashi et al. 2013; Lin, Vicente and Doyle 2001; Preece et al. 2012), and improved compliance with best practices (de Korne et al. 2012; de Vries et al. 2009; Lesselroth et al. 2011). There was also evidence of HFE-based interventions’ impact on patient safety, such as decreased complication rate (de Vries, Prins, et al. 2010), decreased in-hospital mortality (de Vries, Prins, et al. 2010), and increased self-efficacy of patients (Eames et al. 2013; Hoffmann et al. 2007). These studies are important as they show the positive impact of HFE-based healthcare system redesign on healthcare quality.

Because work systems, care processes and patient outcomes are interrelated (see Figure 1), evidence on the effectiveness of HFE-based interventions should include data on changes in the work system (redesign or intervention), changes in the care process (impact of intervention on care processes) and changes in patient outcomes (impact of intervention on patient outcomes). The twelve projects included in this review described interventions; eleven of the twelve projects assessed the impact of intervention on care processes; but only three projects assessed the impact of intervention on patient outcomes.

Only a small number of studies collected data on the impact of HFE-based healthcare system redesign on patient outcomes; this highlights the challenge of linking work system to patient outcomes. To determine the causal relationship between healthcare system redesign and improvement in patient outcomes, future research needs to describe the pathways or mechanisms between the system redesign on the one hand and patient safety and other care quality outcomes on the other hand (Carayon, Alvarado and Hundt 2007). For example, Carayon and Gurses (2008) describe the mediating role of workload between the work system, on the one hand, and care processes and patient outcomes, on the other hand. Studies should use a combination of proximal and distal process and outcome measures to document the impact of interventions on care processes and possibly patient outcomes (Carayon et al. 2010; Holden et al. 2013). For example, de Vries and colleagues (de Vries, Dijkstra, et al. 2010; de Vries, Prins, et al. 2010) showed that a checklist designed and implemented based on HFE principles could improve compliance with recommended timing of antibiotic prophylaxis administration in a surgical pathway (process measure), and reduce complications and in-hospital mortality (outcome measures).

Besides quality of care and patient safety, HFE-based healthcare system redesign should also improve well-being of healthcare professionals and develop healthy work organizations (Carayon et al. 2006; Sainfort et al. 2001). Among the twelve projects reviewed, only three examined the impact of redesign on employee outcomes, such as reduced mental workload (Lin, Vicente and Doyle 2001), improved satisfaction (Chan et al. 2010; Johnson, Johnson and Zhang 2005), and enhanced safety awareness among healthcare professionals (de Korne et al. 2012). None of the twelve projects examined the impact of redesign on organizational outcomes. Future research needs to complement measures of quality of care and patient safety with systematic data on employee and organizational outcomes.

Most projects (9 out of 12) focused on physical and cognitive HFE issues of medical devices and health IT, and were limited to microergonomics (Hendrick and Kleiner 2001). These projects, although important, do not consider complex sociotechnical system characteristics that need to be addressed to improve overall system outcomes from the viewpoint of patients, employees and organizations (Carayon et al. 2006; Carayon et al. 2014; Hendrick 1980; Hendrick and Kleiner 2001). Future research on HFE-based healthcare system redesign needs to connect microergonomics with macroergonomics and to jointly consider physical, cognitive and organizational HFE issues (Carayon et al. 2013). An example of this macroergonomic approach is the study conducted by Kobayashi et al. (2013) who designed and implemented a multi-element intervention to address physical (e.g., location of displays, muted alarm speakers, limited workspace), cognitive (e.g., poor signal/noise ratio, low yield of system access) and organizational (e.g., widespread knowledge deficit of system presence, availability, features and operation) HFE issues of an ED telemetry system. This study shows the feasibility and value of a systematic approach to healthcare system redesign that integrates multiple system elements and levels (Karsh and Brown 2010; Karsh, Waterson and Holden 2014; Waterson 2009). Furthermore, the application of HFE should be broadened to other domains of patient safety (e.g., human error, performance of healthcare workers, system resilience) and the redesign of other elements of healthcare system (e.g., clinical tasks, workflow, physical environment).

We also need to understand factors that influence the adoption of HFE in healthcare system redesign, to develop and test HFE tools and methods for healthcare system redesign, and to conduct empirical studies illustrating how HFE tools and methods can be adapted to and adopted by healthcare organizations (Carayon 2010). For example, HFE applications to healthcare system redesign may involve collaboration between HFE professionals and healthcare stakeholders (Carayon and Xie 2011); therefore, a participatory ergonomics approach can actively involve end users (e.g., patients, healthcare providers) in the design and implementation of HFE solutions (Noro and Imada 1991; Wilson and Haines 1997). Existing applications of participatory ergonomics in healthcare, however, focus on individual tasks of specific jobs (Bohr, Evanoff and Wolf 1997; Fragala and Santamaria 1997; Udo et al. 2006). Research is needed to expand the application of participatory ergonomics to redesign other elements of healthcare work systems (e.g., workstation, care process) in different care settings (e.g., pediatrics).

In this review, we assessed risk of bias in the sixteen studies that evaluated the impact of HFE-based healthcare system redesign on care processes and patient outcomes. We found varying quality across the studies, but did not exclude studies based on the risk of bias assessment. In general, the quality of the studies is limited as the average number of criteria addressed by the studies was only 14 out of 30. The main quality issues include: (1) limited data on participation rate and characteristics of participants, which limits our ability to determine generalizability; (2) unclear description of process for blinding participants and researchers to the intervention; (3) inadequate information on reliability and validity of outcome measures; (4) lack of information on confounders and their adjustment; (5) no information on adverse events associated with interventions; and (6) failure to report statistical power. The sixteen studies applied various study designs, such as before-and-after design with or without control group, time series design, and randomized control design. Studies with a before-and-after design without a control group are open to a range of validity concerns (Robson 2011), such as possibility of historical trends affecting observed changes over time. Studies with a randomized control design present fewer validity concerns (Higgins and Green 2011), but may be more challenging to implement when evaluating HFE-based healthcare system redesign. Researchers interested in conducting an HFE-based intervention study can use the 30-question instrument that we developed for assessing the scientific quality of HFE-based intervention studies (see Table 7).

Because we limited our systematic review to peer-reviewed journal articles, we may not have included all HFE-based intervention studies. It is possible that a project on HFE-based healthcare system redesign is not reported in a single study, and that additional information is reported in publications such as project reports, working papers or conference presentations. A search of the grey literature, conference proceedings and dissertations may yield further evidence on the application of HFE to healthcare work system redesign. However, limiting the review to peer-reviewed published literature is typical in health services research systematic reviews as publication in a peer-reviewed journal is considered as an indicator of scientific quality. We limited the systematic review to studies written in English, with the potential to omit relevant international studies on HFE-based healthcare system redesign.

6. Conclusion

As indicated by Norris (2012), HFE should be widely applied and integrated into “the design, implementation and change management of sociotechnical systems in health care”. There is growing recognition of the importance of HFE-based healthcare system redesign to quality of care and patient safety among healthcare professionals, leaders and researchers (Carayon, Xie and Kianfar 2014; Gurses, Ozok and Pronovost 2012; Hignett et al. 2013; Leape and Berwick 2005; Reid et al. 2005). We reviewed HFE applications to the redesign of various healthcare systems and found some empirical evidence for the effectiveness of HFE-based healthcare system redesign, such as improving patient safety and reducing errors. Further research is needed to continue developing the empirical evidence on the impact of HFE in healthcare system redesign. This research should clearly explain the mechanisms or pathways between the redesign and expected outcomes. It also needs to address scientific criteria such as those used in our quality assessment (see Table 7), and use appropriate study designs and various approaches such as mixed methods research. Additional efforts are necessary to support HFE applications to healthcare system redesign and further disseminate HFE in health care (Carayon 2010).

Appendix A. Summary of Studies on HFE-Based Healthcare System Redesign.

Projects~	Setting	Healthcare system being redesigned	HFE issues*	Work system elements related to intervention	Phases of redesign process				Impact
					Analysis	Design	Implementation	Evaluation	Care process and patient outcome	Other outcomes
1	A 118-bed hospital with 150 radiation therapists and 16 radiation treatment machines (Canada)	User interface of radiotherapy treatment delivery system	C	Tools and technologies	-- Analysis of the radiotherapy treatment delivery process 30 hours of field observation of radiation therapists performing their regular tasks conducted by an HFE expert Workflow analysis to identify tasks regularly performed by radiation therapists and areas associated with high likelihood of incidents Heuristic evaluation by 1 radiation therapist and 2 HFE experts to identify HFE issues of the system and evaluate their severity	-- Redesign of the existing radiotherapy treatment delivery system Redesign of the system based on HFE design principles 2 focus groups with experienced radiation therapists to collect feedback on the redesigned system Refining the system based on results of focus groups		-- Comparison of the redesigned and the existing radiotherapy treatment delivery systems User testing with 16 radiation therapy students using four scenarios related to typical treatment delivery tasks	-- Impact of redesigned system on care processes Decreased error rates for overlooking an important note and changes in approval dates Reduced mean task completion time	-- Impact of redesigned system on employee outcomes Improved user satisfaction
2	A 1200-bed academic hospital (Canada)	CPOE order set system	C	Tools and technologies	-- Analysis of existing paper admission order sets to Identify tasks for completing an admission order set Identify necessary functionalities of CPOE order set system	-- Design and redesign of the CPOE order set system Configuration of the system based on the content of existing paper order sets Heuristic evaluation by 3 HFE experts and 1 physician to identify HFE issues of the system Iterative redesign of the system based on HFE design principles		-- Comparison of the existing paper order sets, the original CPOE order set system and the redesigned CPOE order set system User testing with 27 end-user representatives	-- Impact of redesigned CPOE order set system on care processes Reduced mean task completion time No need for assistance No significant differences in number of errors
3	A 929-bed quaternary and tertiary referral teaching hospital, a 600-bed tertiary referral teaching hospital, a 330-bed community hospital, and a 80-bed referral center (Australia)	Paper medical observation chart used by hospital staff to document physiological observations over time for an individual patient	P, C	Tools and technologies	-- Development of usability heuristics for observation charts Informal task analysis carried out by an HFE expert to examine how charts are filled out and used by a range of clinicians Review and adaptation of usability heuristics from the software and web design domains -- Analysis of existing observation charts Heuristic evaluation of 25 general observation charts that were in use by 5 evaluators to identify usability problems and develop design guidelines	-- Design of a new observation chart Creation of a new observation chart using the newly developed design rules Heuristic evaluation of the newly developed observation chart to confirm that usability problems had been addressed as much as possible		-- Comparison of the redesigned observation chart and 4 existing observation charts User testing with experienced and novice chart users to examine the impact of the observation charts on the ability of health professionals to recognize patient deterioration	-- Impact of redesigned observation chart on care processes Reduced response time for detecting abnormal observations Reduced error rate for detecting abnormal observations
4	A major referral center handling 140,000 outpatient visits and 14,000 surgical cases annually (Netherlands)	OR floor marking	P	Environment	-- Benchmarking with aviation Site visits of airport conducted by a multidisciplinary team to learn experience of runway operators regarding marking, position of materials, traffic flows, safety rules and regulations, and incident management Field observation of ORs by the multidisciplinary team to learn surgical workflow and workplace layout	-- Decision made by the multidisciplinary team to use floor marking to facilitate consistency in the correct positioning of surgical devices and to minimize infection risks Integration of field experience from ORs with knowledge learned from aviation	-- Implementation of floor marking in a step-by-step fashion Temporary floor marking implemented in two of four ORs for six months Temporary floor marking implemented in all ORs for six months Permanent floor marking implemented in all ORs	-- Evaluation of system redesign Observation to evaluate compliance with positioning of surgical devices within the clean airflow Interviews with ophthalmic surgeons, surgical and anesthesia nurses, and managers	-- Impact of floor marking on care processes Increased compliance with recommended positioning of surgical devices in the clean airflow	-- Impact of floor marking on employee outcomes Enhanced safety awareness among surgical staff
5	Design and pilot testing: a 1,002-bed academic hospital Implementation and evaluation: 11 academic hospitals (6 intervention hospitals and 5 control hospitals) (Netherlands)	Checklist for surgical pathway from admission to discharge	C, O	Tools and technologies	-- Analysis of the surgical pathway and related critical safety risks Literature review to describe surgical pathway into distinct stages and to identify critical safety risks in each of the stages	-- Development of a checklist prototype based on the results of analysis and HFE literature on checklist design-- Validation of the checklist First set of observations of 41 surgical procedures by an independent researcher to identify process deviations Creation of an observation form based on the first set of observation Second set of observation of 130 surgical procedures to evaluate the extent of agreement between process deviations occurring in reality and items on the checklist Revision of the checklist based on results of observation	-- Pilot testing of the checklist with 350 surgical procedures during a 5-month period Instructive presentations about the checklist given to all users Introduction of the checklist in daily practice Structured interviews with 21 surgeons and 17 anesthesiologists at the end of the pilot test to assess usability of the checklist and to identify barriers to using the checklist Revision of the checklist based on the results of interviews -- Implementation of the checklist Formation of an implementation team consisting of a surgeon, an anesthesiologist and a quality control officer in each participating hospital Implementation of the checklist in a step-by-step fashion in the surgical wards and operating rooms	-- Evaluation of the impact of checklist on patient outcomes using a controlled, multi-center, prospective design Review of data on complication rate and in-hospital mortality collected during a 3-month period both pre- and post-intervention Review of data on compliance with the checklist -- Retrospective analysis of the effect of the checklist on care process (timing of antibiotic prophylaxis administration) in a single hospital	-- Impact of checklist on care processes Better compliance with regard to the timing of antibiotic prophylaxis administration -- Impact of the checklist on patient outcomes Decreased complication rate Complication rate positively associated with rate of compliance Decreased in-hospital mortality
6	Acute stroke unit of two public, tertiary hospitals (Australia)	Booklet for providing information to stroke patients	C	Tools and technologies, tasks, organization	-- Analysis of current practice in providing written information Interviews with 57 stroke patients and 12 care providers to (1) assess the content and design characteristics of existing written materials, (2) compare the reading level of existing written materials with the reading ability of stroke patients, and (3) assess information needs of stroke patients	-- Identification of system design considerations Literature review to identify design principles for health education materials A multidisciplinary focus group to identify factors that influence design and adoption of the computer system -- Design of the graphical user interface and associated database using a HFE approach Focus groups to identify graphical user interface requirements Interface development to incorporate all system and human functions Usability evaluation using feedback from clinicians Final design of the computer system	-- Pilot test of the computer system with 8 stroke patients and their care providers Interview with pilot test participants to evaluate their satisfaction with the system and to identify opportunities for improvement	-- Evaluation of the impact of the computer system Randomized control trial to evaluate the effects of providing stroke patients with computer-generated tailored written information compared to generic written information Randomized control trial to evaluate the effects of an education package (including computer-generated tailored written information, verbal reinforcement of information pre-discharge, and telephone contact post-discharge) on the knowledge, health, psychosocial and satisfaction outcomes of stroke patients	-- Impact of providing computer-generated tailored written information on patient outcomes Higher self-efficacy for accessing stroke information More positive of being informed Higher satisfaction with information received
7	A 594-bed academic hospital (USA)	Redesign of family history-tracking and pedigree drawing program for assessing genetic risk of hereditary cancer syndrome	C	Tools and technologies	-- Analysis of the original application Open-ended interviews with end users to determine tasks Hierarchical task analysis conducted by reviewing interview data with end users through scenarios Heuristic evaluation and user testing to uncover usability problems and the discrepancies between the users’ and the designers’ conceptual model -- User analysis of the redesigned application Survey to determine needs of potential users Comparative analysis of three commercial products available on the market Functional analysis of the redesigned interface	-- Creation of prototypes Paper prototypes built based on the results of analysis -- Small-scale usability studies Iterative heuristic evaluation based on usability principles and guidelines User testing with think-aloud method to validate interface design decisions and test alternative interfaces -- Modification of prototypes based on results of usability studies		-- Comparison between new and old systems using a controlled experiment Objective measures and questionnaire to determine whether the redesigned application decreases error rate, increases productivity, and increases user satisfaction	-- Impact of redesigned system on care processes Reduced mean task completion time No usability problems encountered	-- Impact of redesigned system on employee outcomes Improved user satisfaction
8	An adult ED in a 719-bed regional referral center and level 1 trauma center (USA)	Paper-based ED charting system	P, C	Tools and technologies	-- Analysis of existing charting system Observation to determine incidence of chart binder placement into the wrong chart rack slot and frequency of inappropriate chart binder transposition into an adjacent clinical unit Identification of deficiencies of the existing charting system, including inconsistent and small font labeling, a single-color scheme, and absence of HFE cues	-- Redesign of charting system Use of HFE principles and input from clinical providers and HFE experts to (1) improve labeling of chart binder for rapid identification of assigned patient room and clinical area, (2) enable easy distinction of a chart binder from others in proximity, and (3) highlight the misplacement of charts through design features that relied on intuitive pattern recognition -- Iterative redesign of charting system with ad hoc discussions with clinician end users	-- Implementation of redesigned charting system System phased in 86 individual patient care spaces in the ED over a 1-month period	-- Evaluation of the impact of redesigned charting system Comparison of pre- and post-implementation data on incidence of chart binder placement into the wrong chart rack slot and frequency of inappropriate chart binder transposition into an adjacent clinical unit	-- Impact of redesigned charting system on care processes Reduced number of chart binder location problems
9	Two 16-bed adult ED units in a 719-bed academic regional referral hospital (USA)	Interface of ED telemetry system	P, C, O	Tools and technologies, environment, organization	-- Assessment of baseline ED telemetry system performance in detecting life-threatening cardiac arrhythmias using on-site simulation -- Development of HFE knowledge base to define pre-intervention system state (e.g., hardware, task, process, user, organizational and environmental factors and issues) and HFE objectives Literature review Hardware inventory Functions diagnostic Real-time clinical use observation End-user survey of needs analysis Informal discussions with small user groups and institutional experts	-- Identification and grouping of design specifications for ED telemetry system in HFE categories (physical, cognitive and organizational HFE) Modified Delphi process involving investigators and ED stakeholders Iterative discussions with institutional biomedical engineers, device manufacturer and end users to verify feasibility, processes and value of the design specifications -- Integration of design specifications into a multi-element intervention to improve (1) system accessibility, (2) system relevance with enhanced signal/noise ratio and system utility for real-world ED practice, and (3) organizational processes for system sustainment	-- Incrementally implementation of the intervention over a period of 17 months Announcement of study conduct and intervention at ED personnel meetings Study simulation sessions Group training and on-shift training of ED personnel	-- Interim and post-intervention assessment of ED telemetry system performance in detecting life-threatening cardiac arrhythmias using on-site simulation -- Review of live environment alarm log records -- Collection of unsolicited anecdotal provider reports of system utility -- Informal survey of end users to assess matching of post-intervention system functions with user needs and to collect suggestions and feedback for future improvements	-- Impact of redesigned ED telemetry system on care processes Increased number of cardiac arrhythmias (e.g., ventricular tachycardia and sinus bradycardia) detected
10	A 230-bed tertiary care academic hospital (USA)	Computerized decision support system to improve adherence with deep venous thrombosis (DVT) prophylaxis	C, O	Tools and technologies	-- Assembly of a knowledge base on DVT prophylaxis Literature review on published clinical guidelines Collection of protocols from academic affiliates Discussion with subject matter experts -- Analysis of DVT prophylaxis process Observation with think-aloud method to understand the cognitive tasks, ordering patterns, and workflow of users Description of clinical workflow and design requirements	-- Design of computerized decision support system Creation of mock-ups of computerized decision support system Usability testing of mock-ups with surgeons Development of specialty-specific order menus based on feedback from surgeons	-- First implementation cycle Release of the new order menus Presentations to residents, pharmacists and nurses to describe the purpose and design of the new menus Informational flyers posted in all surgical staff and resident working areas and break rooms -- Second implementation cycle Observation, semi-structured interviews, focus groups and chart reviews of the prophylaxis orders to understand sociotechnical barriers to user acceptance of the system Reconfiguration of the order menus Training during resident orientation Engagement of clinical champions to support in-service trainees	-- Evaluation of the use of new order menus and the impact of new order menus on compliance with prophylaxis recommendations -- Evaluation of potential relationship between use of new order menus and organizational performance	-- Impact of new order menus on care processes Low acceptance and use of the new order menus during the first implementation cycle Improved acceptance and use of the new order menus during the second implementation cycle Improved compliance with prophylaxis recommendations after reconfiguring the order menus -- Impact of new order menus on patient outcomes Improved medical center indicator performance after reconfiguring order menus
11	A 404-bed academic hospital (Canada)	Interface of a patient-controlled analgesia (PCA) pump	C	Tools and technologies	-- Cognitive task analysis to identify information requirements of programming PCA pump Field observation and interviews with experienced nurses to collect feedback on the current device Bench tests to identify characteristics of the device that make its operation prone to error Assessment of the current device based on HFE design principles	-- Redesign of PCA pump Development of a new prototype interface based on HFE principles and identified information requirements		-- Evaluation of redesigned interface Experiment with nurse students and experienced nurses to compare redesigned interface and current device	-- Impact of redesigned system on care processes Reduced programming time Fewer errors	-- Impact of redesigned system on employee outcomes Lower mental workload ratings
12	Not specified	Hospital code cart medication drawer	P, C	Tools and technologies	-- Analysis of current medication drawer User testing with nurses, pharmacists and nursing managers to identify usability problems	-- Redesign of medication drawer Development of several modified drawers based on comments and suggestions from user testing participants User testing of modified drawers until the final prototype drawer was created		-- Evaluation of redesigned medication drawer User testing to compare medication retrieval time and number of wasteful actions associated with current and redesigned drawers Questionnaire to examine user experience with redesigned medication drawer	-- Impact of redesigned medication drawer on care processes Lower medication retrieval time Fewer wasteful actions Improved perceptions of medication drawer visibility, usability and organization

^~The projects’ numbers refer to the list in Table 3.

^*HFE issues. P: physical; C: cognitive; O: organizationa