Human factors in healthcare IT: Management considerations and trends

Abstract

A range of human factors issues are recognized as critical to the success of projects involving Health Information Technology (HIT). Problems related to the usability of HIT have come to the fore, with continued reports of systems that are non-intuitive and difficult to use and that may even pose safety risks. In this article, we consider a number of approaches from usability engineering and human factors that can be applied to improve the chances of system success and adoption. A range of methods focused around human factors can be employed throughout the system development cycle of HIT. The purpose of this article is to discuss human factors approaches that can be used to improve the likelihood of successful system adoption and also provide input into the selection and procurement process of HIT. The article concludes with recommendations regarding how understanding of human factors can be integrated into healthcare organizational decision making.

Introduction

Human factors related to the use and usability of Health Information Technology (HIT) has emerged as an issue of critical importance in the adoption of technology in healthcare. Human factors as a field is focused on the study of the relationship amongst humans and the tools they use, including computer systems. ¹ This includes work towards better understanding, designing, and optimizing the interaction between health professionals and HIT. Human factors considerations include addressing issues at the cognitive, social, and organizational level. ² In this article, we discuss the critical need to more fully understand and consider human factors in designing, testing, implementing, procuring, and evaluating HIT. We also provide recommendations for healthcare leadership and management about how human factors considerations can be incorporated into critical organizational decision making around HIT.

Over the past several decades, there have been continued reports of problems with a range of HIT and health information systems. ³ Some HIT have been found to be unusable by end users, others have been reported as inadvertently affecting workflow in suboptimal ways, and it has been reported that some might even be considered safety hazards. ⁴ This is despite the already huge investment in HIT not only in Canada but also globally. Indeed, the usability of HIT has been the focus of considerable interest as it affects both the effectiveness and adoption of a wide range of HIT deployments and applications.

In this article, we will discuss the importance of considering human factors in health informatics. In particular, we will focus on issues related to usability and ultimate adoption of HIT by end users. The purpose is to summarize approaches that have emerged from human factors engineering and to present recommendations for increasing awareness of the importance of usability engineering at all organizational levels. We will argue for the need for increased consideration of human factors at multiple levels and times, including during the design, testing, optimization, implementation, procurement, and evaluation of HIT. We also touch on some critical issues related to education and regulatory issues as they pertain to human factors in health informatics and healthcare more generally.

In arriving at our conclusions, we conducted an overview of the literature applying the review methodology, as described by Grant and Booth’s typology of reviews ⁵ using articles from human factors engineering and health informatics for improving HIT usability. This review was conducted with a view to considering the literature in the context of potential application to management and organizational decision making for improving end user adoption of HIT. Bringing increased consideration of methods and results from human factors research to all levels of management and organizational decision making is an area that has remained to be more fully explored. In this article, we describe some novel ways knowledge translation can be achieved for improving end user satisfaction and adoption of HIT through application of human factors research and methods.

Human factors, human computer interaction, and usability of health information systems

Human Computer Interaction (HCI) is the subfield of the broader field of human factors that deals with understanding the interaction between humans and computers, as well as with cognitive and social phenomena around that interaction, which may take place in an organizational context. ² HCI has become a well-developed field of study, and methodologies from HCI have been incorporated in the design and implementation of countless information systems. As a subfield of HCI, usability engineering has focused on methods and approaches to making systems easier to use, more effective, and more enjoyable to use. Indeed, in recent years, User Experience (UX) has emerged as an important aspect of human interaction with information systems and technologies. ⁶

In healthcare, the areas of usability and UX have seen considerable interest in light of the reported difficulties that have been encountered in trying to implement and deploy a range of health information systems. ⁷ This has included systems such as the Electronic Health Record (EHR) and Clinical Decision Support Systems (CDSS), which have in many cases proven difficult to design in such a way that they are both effective and deemed to be usable by their end users. ⁷ To address this, formal definitions of usability have emerged that have allowed for guiding efforts at improving the situation. For example, according to the International Standards Organization (ISO), usability refers to “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ⁸ The specific dimensions of usability which can be broadly considered as a measure of “ease of use” of a system or interface are the following: (1) learnability––for example, how easy is it to learn how to use a HIT? (2) effectiveness––for example, does the HIT allow the user to carry out the tasks it is supposed to? (3) efficiency––for example, does the HIT support the task in an efficient way? (4) user satisfaction––for example, is the HIT satisfying to use, and (5) safety––for example, is the HIT safe to use?^5,7

A range of methods can be applied to assess the usability of HIT. The application of these approaches is critical both in the design of new systems as well as in the implementation of systems (including commercial vendor-based products). These include usability inspection methods which involve having one or more trained usability specialists inspecting or systematically stepping through a user interface using well-developed guidelines (heuristics) in order to identify and rectify usability problems. ⁹ The most well known of the inspection method is heuristic evaluation, pioneered by Jacob Nielsen, which has been applied in numerous studies of HIT. ¹⁰ This approach involves analyzing an interface in concordance with 10 specific heuristics that are associated with effective user interface design and can be effectively applied to identify usability problems with HIT. ⁷

In addition, the approach known as usability testing involves observing end users of systems as they interact with a technology to carry out representative tasks (e.g. observing and recording physicians interacting with EHRs to record patient information).^7,11 Researchers and human factors practitioners have shown usability testing to be an important tool in designing more usable and acceptable HIT. ¹¹ In addition, the approach can help to identify serious usability problems that may be associated with safety issues.^12-15 Feedback from such work can identify a wide range of specific usability problems (and a large variety of problems have now been reported in the literature), some of which could lead to user dissatisfaction with the system, while others could potentially lead to medical error and patient harm (see Table 1).

Table 1.

Examples of usability problems.

Usability problem	Example
Visibility problem	User (e.g. physician) is unable to see an alert presented by an electronic record user interface, as it is not prominently displayed 16
Understandability problem	User does not understand the onscreen instructions for patient treatment options 16
Unclear log on/off	User is not sure if they have logged off as the status of the system is not clearly indicated in the user interface 4
Documenting on the wrong patient record	User is unable to determine what patient they are documenting on due to multiple records open, and as a result inadvertently enters data into the wrong patient record 7
Navigational problem	User is unable to see how to navigate through a complex set of computer screens to get to a desired screen and section of the system 17
Screen-driven behaviour	The screen layout and organization of information on it leads health professional to miss key patient data (in following the order on the screen) 17
Consistency problem	Day, month, and year are entered (and displayed) in different formats (orders) in different parts of the system due to lack of consistency 7

Other methods include deployment of usability questionnaires to users of HIT, interviews with end users, focus groups as well as a wide range of ethnographic observational methods (see Table 2). In addition, clinical simulation methods extend usability testing by conducting usability evaluations under more realistic conditions often conducted “in-situ” (i.e. in real clinical, healthcare, and home settings where the HIT will ultimately be deployed). ¹⁶ In order to provide guidance in analyzing the resultant data from human factors and usability studies, a range of qualitative coding schemes have also been developed. ¹⁸ These schemes provide checklists and categories that can be used to help the analyst or researcher identify particular usability problems and issues, which can then be rectified through an iterative process of system optimization.

Table 2.

Human factors approaches for evaluating and analyzing HIT.

Method	Example references (using the approach)
Usability testing	7,16,17,19,20
Usability inspection (heuristic evaluation and cognitive walkthrough)	7,21
Clinical simulations	12-15
Focus groups	22,23
Interviews	22,23
Surveys/Questionnaires	24-26
Observation and shadowing	27,28

Application of usability engineering throughout the system development life cycle: When and where to apply the methods?

There is increasing recognition of the need to apply usability engineering methods and related human factors approaches and methods throughout the design, development and deployment of all HIT. The Systems Development Life Cycle (SDLC) is a well-known software engineering concept that refers to the stages that Information Technology (IT) goes through. ²⁹ Early stages include planning and analysis (including requirements gathering), followed by system design, implementation, and then deployment in healthcare settings. Along this entire life cycle, it has been demonstrated that the application of usability methods can lead to implementation of more usable and ultimately acceptable HIT (see Figure 1). This has included application of these methods to provide feedback to early design, prototypes, and mature systems. Also, once deployed, usability evaluation of implemented systems (including commercial, vendor-based systems) can provide specific feedback to system customization and improvement, as well as to training departments within healthcare organizations.

Figure 1.

Usability engineering and human factors methods across the phases of the systems development life cycle.

Based on the literature, the recommended approaches to testing HIT prior to widespread implementation and release are depicted in Figure 1. The figure shows a range of methods that can be applied, from early testing of HIT applying usability inspection and usability testing. Later in the development process, this can include usability testing of systems in limited deployments, which could involve usability evaluation of mock-ups, early designs, and prototypes. This can be followed by in-situ (in real settings) usability testing of more completed products under real or realistic simulation conditions (“near-live”) or actual limited real-life application.

In one of our studies, this involved conducting an initial usability test which involved observing and recording several physicians as they interacted with a new clinical guideline-based Decision Support System (DSS). ¹⁶ After analyzing the resultant digital recordings of user interactions with the system (involving eight representative providers who served as study participants), the user interface was modified accordingly to correct defaults and usability problems. A second phase of testing was then conducted whereby conditions of use were simulated involving observing use of the tool in interviewing a simulated patient. After modifying the system to take into account findings from the second phase (which focused on optimizing the triggering of the guidelines within the system), a final third phase of testing was undertaken, where recordings of a limited number of “live” patient interactions were analyzed. As a result of this work, which was conducted in the three phases, the resultant system achieved a high level of user adoption and satisfaction. ¹⁶

Application of human factors methods in HIT and system selection

An important application for applying human factors methods and approaches (and in particular usability engineering methods) is the area of HIT system selection and procurement. Healthcare organizations spend millions in selecting and procuring a wide range of HIT, ranging from EHRs to Decision Support Systems (DSSs) to bedside equipment and infusion pumps. To date, evidence-based approaches to making such selection decisions have been limited in scope, often resulting in selection of systems that do match the requirements, capabilities, or needs of end users in the buying healthcare organization. ³⁰ In our work, ³¹ we have detailed how we have applied usability engineering methods in the selection and procurement process to provide stronger evidence for fit between candidate systems and the buying organization (in particular, fit with the end users within that organization as well as organizational processes). To explore this, a continuum was described by the authors that reflects the degree of evidence for “fit” between different candidate systems (i.e. for possible selection) and the buying organization. ³¹ This continuum ranges from weak evidence (gathered using conventional selection processes) to stronger evidence that can include analysis of how users would interact with and react to different candidate systems by applying usability methods. It was argued that to ensure that a system that might be selected would be acceptable and adopted by end users (e.g. health professionals) in an organization that the candidate systems should undergo extensive human factors analyses (undertaken by the buying organization) as a required part of the selection process. Based on these recommendations, several major national EHR procurements in Denmark and Finland applied such an approach in selection of their respective regional EHRs. ³² This work highlighted the need to get a better understanding of potential human factors and usability issues with particular HIT products even before selecting them, and furthermore how using this information can inform the system selection process in healthcare. ³¹

Layers of usability testing to ensure system usability and safety

The application of human factors and usability engineering approaches has been shown to improve the process not only of designing and developing new HIT applications and systems but also for ensuring both the usability and safety of the implementation of commercial vendor-based systems. In a series of studies the authors have been involved in, human factors methods were applied in the refinement and customization of EHR implementations at several major hospital systems. ³³ This included conducting a sequence of usability tests, beginning with basic usability testing, followed by simulation studies and finally by “near-live” testing of new systems prior to widespread implementation and deployment. Each of these methods identified a range of customizations and optimizations needed in order to improve the ultimate usability and acceptance of the systems and HIT applications prior to widespread release within the hospital setting. ³⁴ In much of our work, the application of the human factors usability engineering methods resulted in improved adoption of the final system by end users (e.g. physicians, nurses, and patients). This work has led to international recommendations regarding the process of applying usability evaluation within system development as well as implementation life cycle of HIT products to ensure system usability and safety.^33,34

Management issues related to human factors and usability

There is a need to train and hire health informatics specialists with backgrounds in the areas related to human factors outlined above. These professionals could provide critical advice on issues related to human factors, including how to best deploy HIT to end users in organizations such as hospitals, clinics, and the home. They may also provide advice and support for decision making for selecting HIT in a way that takes into account user needs and capabilities with regard to HIT. In some organizations, such individuals have started to develop subgroups within organizational IT and quality assurance departments that are devoted to human factors and usability of HIT. ³⁵

A high priority should be given to ensuring usability and safety throughout the whole organization. This is evidenced by a number of reported cases where systems failed to be adopted due to lack of user input into design, testing, and deployment of HIT and this is now recognized as being a global phenomenon. Although national and international efforts have been initiated to promote certification and regulation of usability in response, there are a number of limitations of this work, which need to be further examined and considered by the management of healthcare organizations responsible for selecting, buying, customizing, and deploying HIT. ³⁶ There are currently a number of HIT certifications internationally designed to allow vendors to “certify” that their products (e.g. EHRs) have met certain standards regarding their development process. Management needs to be aware of the certification requirements, processes of commercial vendor-based products such as EHRs (as well as the limitations of these processes). Indeed, analysis of the certification process in the United States^37,38 has shown that a certified product may not necessarily be considered “usable” in local contexts (this is the case particularly for EHRs product developed outside of Canada, but purchased and used in a particular Canadian health authority). This is as the current certification processes are limited in that they typically do not require reaching a specific level of usability, but rather certify the process of system development (i.e. to certify that the process includes, for example, some limited amount of usability testing). Furthermore, centralized certification of a vendor product does not mean a complex system will be found to be learnable, effective, efficient, and safe for end users, when it is deployed within a local healthcare setting or context (particularly if the context and setting of where the system is deployed is different from the context and setting from where the system was designed). This argues for the need for not only centralized certification of system usability (e.g. by regional or national bodies) but also for local hands-on usability testing of systems within specific hospital or organizational settings to ensure usability. This in turn will require greater awareness at all management levels of critical issues related to usability, methods, and approaches to ensuring HIT usability and safety, as well as current trends and regulation around human factors and usability of systems and applications.

Education, training, and knowledge translation: Who should apply the methods?

There is a need for an understanding of not only the importance of human factors for the success of our HIT efforts but also a wider understanding of the methods and approaches that can be applied to lead to improved usability and ultimately improved HIT adoption. There are multiple levels of understanding, from the level of health informatics professionals (e.g. working in hospitals, health authorities, ministries of health, and commercial vendors and companies) to understanding required by management of health organizations. To address this current knowledge gap, key aspects of human factors work and training in the area of usability engineering methods have emerged. At the University of Victoria School of Health Information Science, for example, several courses on usability in healthcare have become a key part of both the undergraduate and graduate education programs in health informatics. Such courses provide hands-on examples of how human factors and usability of HIT can be considered, understood, and improved in a range of healthcare settings. Graduates have been exposed to human factors and usability engineering methods in their courses, and as result upon graduation, many have adopted the methods and applied them in their institutions and specific work places (e.g. hospitals, clinics, and ministries of health). Graduates of health informatics programs now hold key positions ranging from managers and directors to chief information officers in key healthcare organizations in Canada and internationally. The application of the knowledge gained around human factors has begun to be translated into day-to-day practices and decision making approaches around selection, design, and deployment of HIT. However, there is still the need for further training, education, and awareness of the importance of human factors throughout the healthcare industry. To address this, efforts at increasing exposure to human factors we have developed webinars, short courses, fellowship training, and micro-credentialing focused on human factors and usability engineering in healthcare. Through health informatics degree programs, short courses, and continuing education, these efforts have resulted in increased uptake of specific human factors approaches in healthcare and this trend is expected to continue.

Discussion

Addressing issues related to human factors in HIT is of critical importance to ensuring the effective deployment of HIT in healthcare organizations. Indeed, the success or failure of IT projects in healthcare is largely predicated on end user acceptability of the systems and technologies we introduce. This includes need for increased understanding of user needs, usability, the safety of HIT, and their ability to support human activities in healthcare. As such, human factors needs to receive considerably more attention. As noted, this is evidenced by the continued reports of unusable or unsafe HIT being deployed globally, despite the large amount of spending in this area. The field of human factors and the subfield of usability engineering offer a range of scientific methods and approaches that can be applied throughout the entire design, development, deployment, and implementation processes of HIT (a number of which have been described in this article). These methods have been shown to be cost-effective within the context of HIT and are not difficult to employ.^39-41 However, improved awareness and education about these methods and approaches will be critical. Along these lines, expansion of educational programs in health informatics and specific training in human factors aspects of the field will be a basis for needed knowledge translation in this area. Leaders of health organizations also need to understand human factors issues and potential problems in order to effectively lead IT-related projects. The expected benefits of HIT will only be reaped if end users of systems procured are found to be effective, efficient, usable, and safe by the end users of the systems we deploy. There are a number of key points in the life cycle of HIT where human factors considerations are critical. Although organizations may acquire systems that have been certified for usability, this will not be a guarantee that when implemented within the context of a particular health organization that it will be found to be usable and useful by end users. Greater emphasis will need to be placed on understanding the capabilities as well as the limitations of specific HIT in terms of fit between the system and the organization. Furthermore, education and training around best practices in selecting, customizing, and deploying HIT from a human factors perspective is necessary. This will help to ensure improved decision making around HIT along the entire system development life cycle, which will be required in order to ultimately achieve the full potential and benefits of technology in healthcare, to maximize the organizational benefits of technology.

Conclusion

In this article, it has been argued that increased consideration of human factors is needed at all levels in healthcare organizations, from the health informatics professionals procuring and implementing HIT projects all the way up to the Chief Information Officer and Chief Executive Officer. Decisions around selection, procurement, and implementation of HIT are critical and account for a large expenditure in healthcare organizations today. Such decisions affect the organization long after the original selection process is complete and are critical to efficient organizational functioning. To maximize the benefits of this investment, a number of recommendations have been made in this article, including applying human factors approaches to increasing the level of evidence for system-organization fit when selecting systems. In addition, the application of human factors methods for optimizing and iteratively improving the usability of HIT on an ongoing basis is needed in order to increase end user satisfaction with systems. This has been shown to be critical at all stages in the system development life cycle, and not only during development processes by vendors but also in the implementation of vendor-based products when HIT such as EHRs are implemented within healthcare organizations. It was further recommended that in order to achieve this improved level of understanding and use of evidence around human factors issues, further education and training on methods that have emerged from outside of healthcare (specifically from the disciplines of human factors and usability engineering) will need to become incorporated into HIT practices and healthcare decision making by management. Along these lines, further education and training about how methods and approaches described in this article can be applied will be needed.

ORCID iD

Andre W. Kushniruk https://orcid.org/0000-0002-2557-9288

Elizabeth M. Borycki https://orcid.org/0000-0003-0928-8867