Table 2.

Overview of Usability Evaluation Methods

Usability Evaluation Method	Description	Strengths	Limitations
Eye-tracking[45]	Eye-tracking involves a device recording the motion of the eye as a participant views the eHealth intervention. The device traces the pupil movement within the eye and determines the direction and focus of the participant‟s gaze.	This method can provide information about the duration of the participant‟s focus on a spot, movement of the participant‟s focus, specific items on the screen that draw attention, and participant‟s navigation of the page. With this method, researchers can compare patterns of eye movements among participants.	This method cannot tell you the intent behind a participant‟s gaze. Eye-tracking cannot capture peripheral vision such that you cannot be certain that the participant did not see an item outside the immediate scope of vision. It can be expensive as software is required to track eye motion and it may require specialists to conduct eye-tracking sessions and interpret the results.
Questionnaires [46]	A questionnaire involves participants scoring items using a predetermined scale. There are numerous standardized and validated questionnaires to quantitatively evaluate usability of eHealth interventions.	Validated questionnaires can be quick and easy to administer to participants. It can differentiate between a usable vs. unusable systems.	Questionnaires cannot identify which components of the technology need to be fixed and the scoring systems for questionnaires can be complex.
Semi-structured Interviews [47, 48]	Semi-structured interviews involve a face-to-face interview focusing on usability-related topics such as functionality, navigability, and ease of use of the eHealth intervention.	These interviews are useful for gathering information regarding an individual‟s attitudes, beliefs, practices, and experiences. The confidential setting of an interview allows for the interviewer to ask sensitive questions about social or personal experiences.	The limitations of this approach are that it requires preparation time to design questions and probes and potential cost to train the interviewer.
Contextual Interview[45]	A contextual interview is a usability assessment that involves observation of end-users as they work with the eHealth program in their own environment.	This method allows researcher to identify issues that users are facing and learn more about the setting of the user environment; researchers are able to ascertain the speed of internet, the layout of the space, and what additional resources are needed for optimal use of the eHealth program.	Researcher needs to travel to the user environment to conduct the contextual interview. This method provides data based only on researcher observations.
Think-aloud [49, 15]	The think-aloud method is a usability assessment that gathers information on functionality, navigability, and ease of use while end-users interact with the eHealth intervention technology. End users and/or experts (as part of a heuristic evaluation) express their thoughts and questions in real time as they perform tasks, allowing the observers to see the cognitive processes associated with task completion. During the think-aloud, trained research staff usually uses software to record the end-user‟s responses including verbal comments and physical responses such as eye movements during this assessment.	This method provides insight into usability flaws by identifying actual problems encountered by end-users and the causes underlying the encountered problems.	An expensive evaluation method as recording software is involved. In addition, this method can only reveal usability problems of the specific tasks given to end-users.
Cognitive Walkthrough [50, 49]	A cognitive walkthrough determines whether the end-user‟s background knowledge and the technological cues embedded in the computer system interface are enough to facilitate successful completion of a task. This method evaluates of the learnability of the eHealth tool. The method involves a monitored task simulation with the end-user going through the sequence of actions necessary to complete a task.	A cognitive walkthrough focuses on cognitive actions such as identifying icons and behavioral or physical actions such as mouse clicks needed for task completion. This method can identify usability problems that can hinder completion of a task and especially useful when developing technology for populations with low literacy.	The limitations of this approach include time for intensive preparation needed to detail correct sequence of actions needed to complete a task. These written sequences may limit the usability evaluation of the whole eHealth system.
Heuristic Evaluation & Expert Review [49, 15]	A heuristic evaluation is a method involving a small group of experts to evaluate the user interface design against a list of usability principles. The heuristic principles include the following: 1) simple to use and natural dialogue, (2) speak the end-user‟s language, (3) minimize end-user‟s memory load, (4) be consistent, (5) provide feedback, (6) provide clearly marked exits, (7) provide shortcuts, (8) provide good error messages, (9) prevent errors, and (10) provide help and documentation. The purpose of the heuristic evaluation is to uncover usability problems of the eHealth technology by identifying unmet usability principles. The think-aloud method, scenarios, and cognitive walkthrough may be used with experts as part of the heuristic evaluation.	This method can provide quick feedback and experts can provide suggestions to correct any problems related to usability.	Multiple experts may be hard to recruit and potentially expensive. There may be a lack of consistency or overlap in detected usability problems between experts.
Focus Group Discussion [45]	Focus groups are a moderated discussion on a range of preset topics that comprise approximately 5 to 10 participants from the target population. Consideration should be taken to conduct a focus group among participants who are representative of the target population. When recruiting participants, it is important to consider specific traits of the target population such as age, experience, gender, education.	Focus groups are a way to assess attitudes, beliefs, knowledge, practices, desires and reactions to topics.	The content from the focus groups discussions cannot be verified. In addition, the setting of a group may influence the way that respondents speak about topics.
Scenario [45]	Scenarios are used to situate tasks in realistic settings for the use of an eHealth program. The scenarios contain key tasks to provide representative data on usability of the system. These tasks should encourage an action without providing instructions on how to use the interface.	This method shows how the end-user accomplishes a task and gives information on whether the interface facilitates completing the scenario.	Poorly written scenarios can lead to poor usability data. This method requires a trained facilitator and intensive preparation time to create the scenarios. Scenarios must be used in the context of the think-aloud method and heuristic evaluation to obtain feedback from both end-users and experts.