A Methodological Framework for Evaluating Mobile Health Devices

Abstract

Mobile devices such as PDAs are increasingly used as a part of healthcare interventions. Although they can provide powerful communication and self-management tools for patients with chronic illness, relatively little is known about their usability. A methodological framework for evaluating mobile health devices is proposed. It adapts the well-known methodologies of cognitive task analysis and usability testing. The framework is illustrated in the context of an analysis of an advanced function glucose meter. The analyses documented a range of problems which highlights the need to carefully scrutinize the usability of mobile health devices.

Introduction

In recent years, mobile devices such as PDAs, cell phones, and mobile computers have become vital resources for communication and personal information management [1]. These devices have increasingly permeated the health care arena enabling clinicians with rapid access to patient data as well as providing patients with powerful self-management tools. Although the enhanced functionality of modern glucose meters is clearly a promising development for the burgeoning diabetes population, relatively little is known about their usability. The small screen and complex system of hierarchical menus present challenges for users, especially older adults who constitute a sizable segment of this patient population. This paper presents a methodological framework for the usability analysis of mobile health devices with a particular focus on older adults.

Methodological Framework

We have developed a cognitive framework for characterizing and reducing barriers between seniors and productive use of health information technologies to manage their health [2]. The framework has three interdependent analytic foci: a) the system resources presented to the end user, b) the tasks supported by the system and c) the profile of the individual users in terms of skills and competencies. The current framework adapts our prior work to make it suitable for small mobile devices. This includes developing usability tools for capture of interaction on small devices and development of new interaction models that take into account the increased working memory demands of small devices. Because of the extensive use of navigation buttons in these devices, new data representations were also required. Figure 1 shows one example of a user interaction with an advanced glucose monitor. The task analysis represents the goals for each task, each step to be taken, the necessary knowledge, and the feedback presented to the user after an action has been completed.

Figure 1.

Novel graphical presentation of user action history. Time proceeds vertically from top to bottom. The icons on the horizontal axis are buttons on the glucose meter. Points on the graph indicate button presses. Optimal task completion requires only 5 steps, while user required over 15

Conclusions

New mobile devices offer a wide range of valuable resources to assist patients in managing their health. However given the complexity of navigating hierarchical menus, it is reasonable to hypothesize that such devices may not be suitable for many older adults. The evaluation framework was successfully applied to the analysis of an advanced glucose meter and was able to identify both usability problems as well as possible solutions.