Debunking Health IT Usability Myths

Abstract

Poor usability is a threat to patient safety and linked to productivity loss, workflow disruption, user frustration, sub-optimal product use and system de-installations. Although usability is receiving more attention nationally and internationally, myths about usability persist. This editorial debunks five common myths about usability (1) usability only concerns the look and feel of a product and is, therefore, only a minor concern, (2) usability is not measurable, (3) usability stifles innovation, (4) vendors are solely responsible for product usability, and (5) usability methods are not practical for use in healthcare.

Poor usability of health information technology (HIT) products can have profound, negative impacts. Usability has been linked to patient safety issues, productivity loss, workflow disruption, user frustration, sub-optimal product use and system de-installation [12, 14, 18, 49, 51]. For example, usability issues with computerized provider order entry (CPOE) facilitated 22 types of medication error risks [21]. Other authors provided single cases where CPOE caused a severe overdose of potassium chloride [14] and that hospital staff transplanted an infected kidney after not being able to detect over multiple occasions a donor’s positive Hepatitis C result buried in their electronic health record (EHR) [47].

EHR usability concerns have reached a tipping point nationally [33, 51] and internationally [26, 53] resulting in initial efforts to improve the user experience for HIT. In the US, the Office of the National Coordinator added new language in Meaningful Use Stage II requiring initial processes to address usability for EHR certification [5, 31]. Yet, worldwide health organizations, including vendor organizations, can be reluctant to adopt usability processes to improve the user experience [32] in part due to common misconceptions. Karsh and colleagues described HIT fallacies in 2010 [18], although only a few had usability implications. As user experience experts, we have encountered many usability myths including ones publically proclaimed by HIT leaders. We outline five of these common usability myths and dispel them.

Usability Myth #1: Usability only concerns the look and feel of a product

Myth

Usability deals primarily with the users’ satisfaction while interacting with a HIT product and is, therefore, a minor consideration.

Reality

Usability is distinctly more than look and feel or user satisfaction. Usability is defined as efficiency, effectiveness and satisfaction in accomplishing specific goals in a specific context [15]. User experience or UX is a newer, broader term dealing with all aspects of users’ interactions, based on understanding user needs, requirements, workflow and involves iterative design and testing during product development [19]. UX can mean either a quality indicator of a product or the defined processes to evaluate a product; usability has at least these components classically defined by Nielsen [36]:

• •Learnability – how easy the product is to learn for the tasks users require, especially during firsttime use
• •Memorability – how easy the product is to remember, especially for intermittent users or after a period of not using a product
• •Low usability error rate – how many errors are generated, their severity and the ease of recovery from errors
• •Efficiency – how quickly tasks can be completed
• •End-user satisfaction – how pleasant the product is to use, including visual appeal

While a frequent perception in healthcare is that user satisfaction is the sole indicator of product usability, the most critical component is its direct relationship to patient safety, the major element of effectiveness. Poor usability in highly computerized health settings led to patient safety issues such as negative workarounds [22, 42]; high rates of adverse drug events [34]; emergent process failures [55]; inability to detect important medications on an electronic medication administration record [11]; e-documents that do not support nursing handoffs, a task crucial to patient safety [50]; and well-publicized, unintended consequences of Health IT [1, 2].

These kinds of usability problems are often a result of incomplete requirements, unsupported workflow and poorly designed environment-to-software interactions. Thus, usability is foremost about how a product functions and how users interact with it. Satisfaction or visual appeal is important as it can impact performance, but attractive visuals will not improve performance if the design does not address the underlying needs of clinicians and tasks. Therefore, the primary component of usability in HIT is its effectiveness related to patient safety.

Usability Myth #2: Usability is not measurable

Myth

Usability is based upon subjective information such as perceptions. It is different for each person, is “in the eye of the beholder” and cannot be easily calculated.

Reality

Core usability issues are measured by assessing users’ performance, underlying tasks and workflow. Usability testing is conducted to decrease risk/errors, ensure task completeness and increase effectiveness. It is more than asking HIT users to examine a screen shot or “play” with an application to render an opinion. These informal activities may not include comprehensive user tasks, a cross-section of end-users or a range of interaction outcomes to measure accurately the usability of an application. The results can be incomplete, subjective and biased. Instead, user experience testing includes systematic evaluations that consider major stakeholders, representative tasks and performance outcomes [35, 37, 52].

Usability has been measured outside healthcare, notably in high risk displays used in nuclear power plants, military equipment, air traffic control, and airplane cockpits [6, 12, 17, 30, 38, 46]. Moreover, usability measures on medical devices have been required for over a decade [10]. Designers begin by understanding critical user tasks under realistic conditions and then measuring how well a product fulfills those tasks. Examples of objective usability measures include task time, errors, accuracy, and task completion [7, 8, 20, 24, 48, 54]. Other examples used widely outside health care include numbers of keystrokes and mouse clicks, eye-gazes, and mental workload [23, 28, 35, 39, 52]. Usability can be measured in descriptive, experimental and multi-factorial studies. The importance of measuring usability was made clear during national testimony by a leading usability expert at the University of Maryland College Park in the US, Professor Ben Shneiderman,

“Usability measurements are to user interface design what medical exams are to patient care.”

Like patient examinations in healthcare, measuring usability is fundamental to product design because it provides objective data and identifies major areas for errors and productivity loss. These measures are a diagnostic component for improving the “health” and functioning of a HIT product.

Usability Myth #3: Usability stifles innovation

Myth

Usability is a rigid process that results in identical and blandly undistinguishable products. It will eliminate vendors’ competitive edge and disallow innovation. Companies, such as Apple, do not place priority on usability. In fact, "If Apple had employed usability, the iPhoneTM would never have been developed," according to a national HIT leader at the 2011 Summer Institute of Nursing Informatics.

Reality

User-centered design drives innovation because it is a systematic method of understanding user needs and a process that leads to the discovery of true pain points and unmet needs. Usability investigation encourages a deep understanding of how users’ backgrounds, experience, goals, and environmental context affect their responses and perceptions. The result is a strong foundational platform for companies to apply creative problem-solving and innovative approaches.

Many companies such as Apple are mentioned as quintessential examples of innovation excellence but a common misperception suggests that usability is absent in these environments. Unlike the HIT leader's quote and misperception, Apple has a long history of recognizing the essential role of usability in product design, emphasizing human behaviors and users’ interactions with products. Apple published noted guides for developers on how to design usable software [27]. In any organization, all great design teams have a deep understanding about users’ cognitive processes, their interactions with current products, what is considered intuitive for users and why current products do not fulfill users’ needs. Design teams innovate based upon that basic understanding. Finally, extensive case studies and research point to many barriers in innovation with product usability cited as one of the remedies [43].

Usability Myth #4: Vendors are solely responsible for product usability.

Myth

Because vendors develop commercial software, health organizations can only have minimal impact on product usability. If organizations buy HIT certified software, usability is assured.

Reality

Usability is a responsibility across at least these entities: vendor health organizations, healthcare facilities, professional health IT organizations, and federal agencies. Obviously, vendor organizations shoulder considerable responsibility to create usable HIT products and incorporate known usability principles and processes. Vendors’ responsibilities are foremost for iterative HIT design improvements based upon the findings of usability testing. This alone could make an enormous difference in product design. Unfortunately, the uptake of usability processes has been uneven to minimal among EHR vendors [32]. However, blaming vendors as solely responsible for poor designs and placing the complete onus on them for improving the user experience is not optimal. These tactics make other health stakeholders seem powerless and disregard other potential solutions. Thus, a one-sided effort will not solve HIT user experience issues.

At the local, provider level, health facilities’ responsibility for usability is a continuous obligation rather than just a one-time event at the time of purchase. Health organizations make usability decisions starting at product purchase and extending into tailoring, implementation, and maintenance phases. For example, decisions about tailoring EHR products can have a direct impact on usability of electronic health records because health organizations can embrace or decide to ignore vendors’ best practices, potentially undermining the user experience [50]. User experience responsibilities extend across the systems life cycle for all organizations, including provider organizations. Health organizations need resources devoted to usability and onsite expertise to evaluate and to improve the user experience after purchase decisions are made for any HIT product, to incorporate best practices of usability on an on-going basis. Likewise, vendor organizations need to respond promptly to fix critical usability issues. Resources devoted to usability are essential to understand workflow integration (e.g., computing support for rounds), measure usability for products, and guide choices for devices (mobile devices or pumps) for all organizations. Purchasing a HIT-certified product by itself will not ensure usability because current requirements are not robust or do they incorporate local needs.

Systems integration is currently a provider health organization responsibility that affects the user experience. For example, lack of integration across legacy, outpatient and inpatient EHRs negatively affected providers’ effectiveness and efficiency [49]. Unfortunately, systems integration can be costly and time-consuming so organizations may not integrate systems due to lack of resources. This decision can negatively impact providers. Also, as noted by Karsh and colleagues [18] well-designed HIT products, such as EHRs, still require user training especially for first-time users. However, training should not be used to compensate for poor usability. Vendors and health organizations hold joint responsibility for user training and support tailored to unique requirements of individual users.

Professional organizations such as the Health Information Systems Society and Usability Professionals Association have roles to define processes and promote improved user experiences across products, For example, HIMSS has published freely available documents on EHR usability [12, 13]. Last, federal agencies have a role to promote improving the user experience. A beginning is the recent Meaningful Use Stage II certification criteria [5] and documents from the National Institute of Standards and Technology [31, 44, 45]. Yet, we argue that more robust requirements are needed for user experience performance testing, making test results transparent and for developers to employ user-centered design techniques.

Usability Myth #5: Usability methods are not practical for use in healthcare.

Myth

Usability is a process suitable for consumer products or aviation but not for the fast-paced clinical arena. Methods are too time-consuming, too expensive, expend too many human resources and will delay development and product releases.

Reality

Usability processes are upfront in a product lifecycle to reduce overall effort and promote safer, more effective product purchase and use [12]. These processes are built in early to reduce the need for rework; to reduce surprisingly unusable products and to result in products that better satisfy users’ needs, aid users’ work performance, and promote patient safety.

Usability processes are feasible and proven in health product development. Efficient, low cost methods are available for adoption by all organizations [11, 16, 24, 39, 41, 56]. Organization exemplars are also available. The Toronto University Health Network, the U.S. Veterans Administration and the Mayo Clinic built usability laboratories to evaluate healthcare technologies and determine their role in difficult interactions and adverse events [9, 12, 40]. These efforts resulted in improved product design/redesign, safer product purchases and evaluations of how technologies relate to patient care incidents [12, 40]. The expense of formal laboratories may not be as necessary today because many users’ interactions can be captured using laptops with cameras with installed keystroke software. Clinical simulation laboratories can also function as usability laboratories [3, 25], so any existing simulation laboratories can serve a dual purpose for conducting usability studies as well as traditional clinical simulations [4, 29].

For organizations interested in improving usability, a 5-phase model describes a usability maturity model and activities at each phase (► Table 1) [12]. Dimensions with each phase include a focus on users, management, processes and infrastructure, resources (e.g., human, budget) and education (e.g., about the rationale, concepts and methods of usability.). With organizational attention to usability, activities can shift from reactive to proactive, from complaints about product usability after deployment to preventing unsafe, naive product purchase or release. Organizational planning must take into account local priorities, level of risk, product redesign versus new development and concomitant timelines.

Table 1.

Five Phases of the Health Usability Maturity Model [12]

Phase	Title	Definition
1	Unrecognized	Lack of awareness of usability. No practices, policies or resources
2	Preliminary	Sporadic inclusion of usability. Very limited resources
3	Implemented	Recognized value of usability. Small team doing usability
4	Integrated	All benchmarks of usability implemented including a dedicated user experience team
5	Strategic	Business benefit well understood, usability mandated, budget and people part of each year’s budget, results used strategically throughout the organization

Improving the user experience is critically needed in healthcare because of its direct correlation with patient safety, clinician productivity and organizational efficiency. Misperceptions about usability are delaying progress. Incorporating user experience processes and principles into health organizations of all types and levels can improve clinician experiences with health IT products, and especially patient safety.

Clinical Relevance

Poor usability of EHRs is well documented; yet vendors and organizations can be reluctant to employ known user experience due to myths about usability. Debunking these top myths could result in improved uptake of user experience techniques and ultimately improved health IT products.

Conflicts of Interest

The authors have no known conflicts of interest.

Protection Of Human Subjects

No human subjects were used as part of this work.