Asking the Right Questions—Human Factors Considerations for Telemedicine Design

Abstract

Purpose of Review

The goal of this paper was to provide a narrative review of human factors considerations for telemedicine. It also sought to provide readers a foundation of human factors thinking and methods that could be employed within their own practice.

Recent Findings

There are only a handful of articles that discuss the importance of user-centered design and human factors principles in relation to telemedicine systems.

Summary

Most articles come to the conclusion that design flaws could have been avoided by involving stakeholders in the design and implementation of telemedicine. However, many of them lack the guidance for those who find themselves having to choose, implement, or use unwieldy systems. With this in mind, this paper provides a series of human factors principles, real-world questions, methods, and resources for those who may find themselves considering, implementing, sustaining, or using telemedicine in their own healthcare settings.

Introduction

Human factors (HF) is the practice of fitting the work system to the human, not the other way around. It focuses on designing various factors of the system according to how humans think and behave rather than forcing the human to adapt their behavior to the system as whole. As a professional discipline, there are three fundamental principles that help guide how HF is practiced. First, it emphasizes the human-centered aspects of the work environment. Second, as a scientific discipline, HF practitioners apply what we know of human behavior to the design of the work. Third, HF takes a systems approach and emphasizes how the human interacts with all of the other elements within a work system. For instance, as with so many other sciences, the formal origins of HF are often considered to appear during the World War II, when skilled pilots were making erroneous actions with the flight controls. The first HF practitioners carefully examined the context and overall system in which the errors occurred and made recommendations on how to improve the design of controls and displays to accommodate pilot performance. HF has continued to make meaningful contributions to aviation, as well as other domains like consumer technology, transportation, and nuclear power to name a few. Though HF continues to flourish across various professions, it is still in its relatively nascent stages as a formal profession within healthcare. Not until 2000, for instance, was To Err Is Human: Building a Safer Health System published by the National Academy of Sciences in which the authors devoted an entire chapter to advocating for systems thinking principles—similar to those applied by the WWII-era HF practitioners—needing to be applied in healthcare [1]. Today, telemedicine is one specific area of healthcare in need of receiving HF attention.

In short, telemedicine is healthcare from a distance. Telemedicine encompasses many modalities including, but not limited to, virtual visits with healthcare providers, remote patient monitoring, robotic surgery which allows a surgeon to operate from a different location, provider messaging within patient portals, and facilitated visits within a clinic setting conducted with specialists in another. Telemedicine is also increasingly becoming an essential aspect of medical practice. Factors like the recent advances in technology, the rise in Internet accessibility, and the recent boom in its application due to the COVID-19 pandemic [2] have propelled telemedicine into an easy and cost-effective solution for patients seeking treatment for the “most common, most irritating, and most inconvenient” ailments [3]. It is crucial, however, that providers and their institutional leadership consider the needs of their end users—both patients and providers—when implementing a telemedicine program rather than assuming that any service will suffice [4]. If history is any indication, blindly throwing technology at a problem does not guarantee a solution, and, more likely, it will lead to a variety of unintended issues. Take electronic medical records (EMRs) for instance. Though the Food and Drug Administration does provide some recommendations and guidance for the design of EMR interfaces, there does not yet exist any formal regulations or requirements mandating that EMRs be designed using formal human factors principles or methods [5]. This blind spot has encouraged rapid, unregulated, and incentivized development, leading many to voice their concerns over poor usability and its impact on patient safety [6].

The purpose of this paper is to provide healthcare administrators, directors, managers, and providers who are looking to incorporate telemedicine into their health system with pragmatic questions and problem-solving strategies towards implementing user-friendly telemedicine services and avoid imposing cumbersome and inefficient products on their users. These questions and recommendations are informed by a narrative review of the most recent and relevant literature regarding human factors and telemedicine programs. Adopting, integrating, and sustaining telemedicine programs is challenging, but our hope is that this paper will be useful to any institution utilizing telemedicine, whether they be in the early exploration phase or have existing legacy systems in need of updating.

Methods

When developing these questions, we reviewed the most recent literature on HF design considerations for telemedicine, specifically focusing our search on interface design. We conducted a PubMed search on articles published within the last 5 years (2016 through 2020) using the following key terms: human factors, ergonomics, cognitive engineering, human engineering, and telemedicine. This initial search resulted in 47 total publications. We then implemented inclusion criteria for only full-text articles written in English and those directly relevant to human factors issues associated with telemedicine. In total, 26 articles were used for the narrative review.

Results

The small number of articles that we ultimately used for this review demonstrated the current lack of empirical work within telemedicine and design. While some articles focused on specific patient populations using telemedicine, others focused on implementation in entire hospitals and others yet on the implementation in smaller settings such as clinics or ambulances. This information was used to build a framework for the telemedicine considerations we felt would be most beneficial to focus on for this review (Fig. 1). This framework coincides with the following questions that we suggest should be asked at each phase of the process.

Fig. 1.

An infographic on a telemedicine implementation timeline, key points, and a sample of appropriate methods

General Outline for Each Question Section

Each of the following questions is accompanied with a well-established and fundamental HF principle that provides rationale and motivation for the question. The body of each section discusses the importance of why the question should be explored, how information that may help answer the question can be gathered, and insightful publications collected from the narrative review. Additionally, a small sample of methods that may be used to help answer the question are also provided. At the conclusion of each section are additional resources which will highlight the most relevant articles from our review, as well as other helpful publicly available resources and examples of application. The expectation is not that you will become a human factors expert overnight but that you will have a foundation of understanding with a small toolkit to guide you.

Question #1: What should we know about our telemedicine users?

Key HF Principles: Know the needs, capabilities, and environment of your users

Methods: Focus groups, surveys, interviews, and personas

This first question is driven by the first principle of human factors: every element of the system should be user centered. Prior to a telemedicine program being implemented, there must be a firm understanding of user needs. This can be accomplished by conducting a needs assessment. Broadly speaking, a needs assessment describes any effort to identify and understand the characteristics of each user group (e.g., providers, patients). Though there are multiple means in conducting a needs assessment, all with varying levels of effort and amount of data collected, the essential component of any needs assessment involves talking with end users. It is imperative to focus on their existing skills and expectations for a telemedicine program rather than assuming they will seamlessly adapt to whatever system is implemented. For example, telemedicine in a children’s hospital must meet the needs of families (not just patients), while telemedicine targeted for older adults should consider the digital divide that may exist among this user population. Identifying these distinctions is critical, as a notable theme that emerged from our review is the lack of a one-size-fits-all solution. The current research on cancer patients, aging adults, and condition-specific needs (e.g., stroke patients) reveals the difficulties of designing telemedicine systems to meet the unique needs of such a diverse patient populations [7–11]. Within these articles, however, factors such as patient accessibility, system integration, and environmental constraints were identified as potential barriers or facilitators across all user groups. Systematic reviews and technology-focused research highlighted another important theme: the need for telemedicine companies to conduct their own needs assessment when considering telemedicine system design. Two review papers examining systems design flaws each concluded that frontline stakeholders should be involved in the design from the beginning; a third paper demonstrated how using user personas could also help designers identify and understand the end user perspective [8, 11, 12]. Now that we have discussed the importance of needs assessments, we will discuss what to consider when conducting one.

A needs assessment can be as simple as sending out a survey to existing patients asking about their perceptions of telemedicine, their concerns, and their likelihood of use. The existing literature covers several topics that should be addressed when developing your needs assessment for a telemedicine program. For instance, are there any language barriers among the end users? Are there any issues with technology literacy, both among providers and patient users? Are there limitations or concerns regarding connectivity or access? Though the answers to some of these questions may seem intuitive at first, it is important to keep in mind that the telemedicine systems should be designed for other people to use, not those who are developing and selling it. Among the most frequently observed barriers to telemedicine adoption, for instance, is a lack of technology literacy among the hospital staff [13]. As the needs assessment is underway, certain themes will emerge that may indicate barriers to a large portion of your user group (e.g., privacy or access concerns, language barriers). This is an indication that further digging may be required, in which case interviews or focus groups might be helpful.

Resources:

• Pelayo et al. (2019) provide an informative example of how a needs assessment was conducted using a variety of methods, including a preliminary questionnaire, semi-structured interviews, and focus groups [14].

• Aldossary et al. (2017) demonstrate a useful framework for telemedicine planning based on health needs assessment [15].

Question #2: What should we know about the telemedicine system?

Key HF Principle: Make choices based on data, not on sales pitches

Methods: Benchmark testing, user testing, heuristic analysis, Failure Modes and Effects Analysis (FMEA), and observations in other healthcare settings

Though there is a great deal of empirical HF research on what makes “good” design, unfortunately, we still encounter poorly designed technologies and materials on a daily basis. One of the challenges in incorporating HF principles during technology development and implementation is that HF is often considered to be a common sense practice by the general public. Experiences with user-friendly products typically go unnoticed by the end user, since they were likely able to accomplish their task without any setbacks. But rest assured, when HF is not present in the design, thus making a product user-unfriendly, it then becomes blatantly obvious. As the IBM Design Team put it, “Ease of use may be invisible, but its absence sure isn’t” [16].

Other complementary and more specific questions about the telemedicine system include the following: What usability considerations should we look for or consider testing ourselves? How well will this integrate with the EMR, if at all? How do we pick between vendors? Whether implementing telemedicine for the first time, considering upgrading a current telemedicine system, or switching to a new vendor, there are important HF challenges to consider. Recently, one primary care physician was tasked with implementing telehealth tools at more than two dozen primary care practices and pointed out that accessing telemedicine through a patient portal can be as cumbersome as filling out a credit report [17]. Though there are multiple aspects of technology that you may wish to consider, we will focus primarily on user interface design.

Though our narrative review included a couple of case studies identifying human factors considerations for telemedicine systems, such as the value of both initial education sessions [18] and real-time video conferencing [19], the overall amount of empirical research on the topic is scarce. Other literature reviews examining design and usability within telemedicine systems have reported a similar lack of HF research necessary for telemedicine system development [20]. In total, we reviewed two experimental studies of telemedicine systems; both examined design issues specific to older adults [10, 21]. Regardless of the user population, however, we summarized the relevant findings from these studies into the following key points that should be considered when evaluating telemedicine platforms:

• Avoid requiring users to create an account or download applications; these requirements create a barrier to system acceptance and oftentimes lead to user frustration.

• Invitation correspondence, whether via e-mail or text, should be written in concise, coherent, and non-technical language.

• The location and appearance of the button to initiate the telemedicine appointment should be prominent and obvious to help the user easily locate it.

• An electronic instruction manual should be made available.

• The size and contrast of interface typeface should be adjustable to accommodate for the visually impaired.

When evaluating telemedicine platforms, it is important that not all of the information should be prioritized equally. Though the Federal Trade Commission and Food and Drug Administration make efforts to regulate unproven claims of various health products, very little enforcement exists for consumer technologies. Technology companies are notorious for making extravagant (and often false) claims that their products are usable, flexible, and effective. When it comes to telemedicine platforms, vendors should be able to provide data that supports such claims. Specifically, they should demonstrate what type of testing was conducted to identify usability issues. Ideally, vendors should conduct rigorous testing with end users as participants who are completing representative tasks within the system. A less thorough, yet sufficient method for identifying usability issues would be to conduct a heuristic evaluation. Here, a usability expert would apply a well-established criteria for identifying basic usability issues (e.g., when using a video conference interface, users should be able to readily and correctly identify the microphone status as being muted or not) [22]. Regardless, vendors should be able to provide documentation of what methods were used to identify usability issues and how those issues were resolved. If they cannot, ask to bring it in house to test the product with representative users. Though quick, single-day testing may not allow you to identify all of the constraints and barriers; it will reveal any major issues that may have otherwise been overlooked. Vendors should also provide documentation of how HF was considered and implemented during design. Note that the vendors may call HF by another name, such as user experience, user-centered design, usability, or ergonomics.

Resources:

• www.usability.gov—A helpful website that provides a wide range of resources for both novices interested in learning more about usability in general and experts staying informed of the latest developments.

• https://www.nngroup.com/articles/ten-usability-heuristics—Developed by interface design and human factors expert Jakob Nielsen, this list of 10 usability heuristics for interface design is helpful rules-of-thumb for identifying major usability issues within an interface design, such as recommendations for the interface to help users identify and recover for errors. Though this list of heuristics is most effective when applied by a usability expert, they can serve as a helpful resource when examining a telemedicine user interface [23].

Question #3: How can we integrate what we know about our users and the technology when implementing telemedicine?

Key HF principles: Fit the tech to the person, not the person to the tech

Methods: Pilot testing, task analysis, and reporting mechanism

Other questions along these lines include the following: Which telemedicine technology is right for us? How should we introduce the telemedicine system to our users? What level of support should we give, such as training and education resources? Whether you are implementing for an entire hospital or focused specifically on your allergy clinic, start small [24]. The implementation of telemedicine will have downstream effects that cannot always be predicted. In a recent conversation with a provider whose clinic recently moved to telemedicine, he admitted that they were not prepared for all the scheduling conflicts that would occur once patients began signing up. Appointments made in the EMR did not automatically update the calendar within the telemedicine program, as it was a standalone software. This required the scheduler to create two calendar events for each visit in order to connect the provider to the virtual appointment. In addition, if patients alter their appointments via the patient portal, this change would not be reflected in the telemedicine appointment. This is just one example where the benefits of starting small, such as with a pilot group or through phased implementation, can help identify issues that could potentially turn into very large barriers to adoption.

Though training the users is also an important aspect of introducing a new system, it is important to keep in mind that training has limitations. Training is effective when introducing unfamiliar technologies, when developing and maintaining proficiency of sensorimotor skills that may degrade over time, or when needing to provide practice opportunities for infrequently used procedures. But training is not a viable intervention for addressing poor design, particularly for users who have already been trained on the equipment or are already familiar with the technology (e.g., video conferencing) [25].

Lastly, if you discovered irreconcilable barriers in your answers to questions 1 and 2, it is important to make those issues transparent to the users. If there are limits to the telemedicine access or delays for certain providers, for instance, telemedicine coordinators and managers should inform users of those issues, rather than relying on users to discover these issues themselves. Failure to be inclusive of the users by keeping them informed and providing them with a sense of control of their work environment, even during and after implementation, risks promoting user distrust of the system [26].

Resources:

• www.telehealthresourcecenter.org supported by the National Consortium of Telehealth Resource Centers is a helpful website that aggregates resources from across 12 regional and 2 national Telehealth Resource Centers (TRCs), including “assistance, education, and information to organizations and individuals who are actively providing or interested in providing health care at a distance” [27].

• JEWC van Gemert-Pijnen et al. (2011) conducted an excellent review of hands-on guidelines to improving telemedicine implementation, which they then used to develop a holistic framework and a systems approach to telemedicine implementation [28].

• Baker and Stanley (2018) provide an excellent review of the technical aspects of establishing a telemedicine program, including detailed discussions on specific types of telemedicine visits (e.g., comparing and contrasting consultative and facilitated visits) and staffing considerations [4].

Question #4: In the case of monitoring and sustaining existing or newly implemented telemedicine programs, what else should we consider?

Key HF principles: Understanding work-as-imagined often differs from work-as-done

Methods: Pre- and post-testing, contextual inquiry, and safety and hazard reporting

Other questions to consider regarding the monitoring and support of a telemedicine program include the following: What aspects of telemedicine should we be monitoring? What happens when things go wrong? How can we support staff and patients 6 to 12 months down the road? The heart of this question is appreciating that true understanding for how the telemedicine program is being used will not be observed until after it has been implemented. The driving principle behind this question began with the human factors perspective of cockpit design during WWII discussed at the outset of this paper—the realization that work as imagined largely relies on the assumption that work is conducted the same way it is described in protocols and policies. The reality is that work-as-done is very different; it is what people actually do in an environment that no policy can describe. Work-as-done occurs in environments with competing demands, varying resource availability, interruptions, etc. These all have an impact on how work is actively done.

One way to measure efficacy would be to conduct a pre- and post-assessment. If you collected user input prior to implementation, then you could conduct a post-assessment by collecting data complimentary to what was collected earlier. Ideally, before telemedicine is introduced, you can focus on the needs you were hoping to solve and the metrics you could use to measure their success. This gives you the ability to compare a wide range of data from the number of patients seen, number of no-shows or late arrivals, satisfaction, perceptions of access by patients and staff, and more. For example, in a post-assessment of pilot program for physical therapy telehealth, Lee and Billings (2016) reported the following HF issues that were encountered and addressed for successful sessions [18]:

• Education and training to the practitioners promoted program acceptance.

• Environmental distractions, such as background noise and lighting, needed to be controlled.

• Audio and video communication adjustments were necessary for users with sensory impairments; headphones were helpful for those with hearing difficulties, and larger computer monitors with improved lighting aided those with visual impairments.

While management of telemedicine programs is not addressed in this paper, it is important to support users beyond implementation. The actual context of use will reveal unforeseen challenges and constraints that influence how the product is being used. Questions HF experts often ask frontline workers are, “What makes your job difficult?” or “If you could change one thing about the process, what would it be?”. This approach is beneficial because it gives a voice to those at the point of care and it demonstrates that you are willing to listen. Promoting an inclusive and collaborative work environment helps build trust and facilitates effective communication among team members. Providers, nurses, schedulers, and staff will often find issues long before management knows of their existence. When elements of the system do not meet the needs of the user, the user gets creative and adapts accordingly to complete the task. This is how both paper and computer-based workarounds are born. Instead of focusing on the workaround—which is a symptom of a flaw in the system—focus instead on the element of the system that became inefficient and provoked the user to create a workaround. Similar to the methods in the needs assessment (question 1), it is useful to talk to people, collect their input by using surveys, and ask what challenges they may be facing now that part of their work experience has changed.

Resources:

• Taylor, Capling, and Portnoy (2018) provide an excellent overview of important considerations for maintaining ongoing support for a telemedicine program, including discussion about staffing, measuring patient satisfaction, observing telemedicine outcomes (both clinical and non-clinical), and promoting effective teamwork [29].

• www.safetydifferently.com is a human factors and safety blog providing a variety of resources and posts on how to approach safety, including more detailed descriptions on the differences between work-as-imagined and work-as-done [30].

Conclusion

The goal of this paper was to provide administrators, supervisor, providers, facilitators, and coordinators of telemedicine programs with important questions and recommended solutions for their respective services. Our initial review of the literature revealed a general lack of human factors considerations for the design and implementation of telemedicine. One systematic review article found only 15 articles reporting telemedicine system evaluation with user input, and of those 15, only four actually measured usability [8]. A conclusion within that review was that stakeholders, such as providers, patients, EMTs, nurses, and support staff, should be central to the design process.

Human factors was born of this very principle—that users must be the primary drivers for design, implementation, and sustainability. Unfortunately, HF principles are often overlooked when new technologies are being introduced for a variety of reasons. Not everyone has the opportunity to work alongside HF experts, and while these professionals are gaining traction with careers inside hospital systems, there still remains a lot of work to be done. It is with this in mind that though you may not have a human factors resource, you have some understanding of the principles, methods, and application.

As human factors professionals, we understand that too often, providers and patients are at the mercy of software that is not user centered or who struggle to meet the needs of patients without understanding why. It is our hope that the information and recommendations in this paper equip you with the resources to ask the right questions, dig deeper on the questions that go unanswered, and continuously look for ways to make healthcare access easier for everyone.

Compliance with Ethical Standards

Conflict of Interest

The authors declare no conflicts of interest relevant to this manuscript.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.