Skip to main content
AMIA Annual Symposium Proceedings logoLink to AMIA Annual Symposium Proceedings
. 2011 Oct 22;2011:795–804.

Assessing the Usability of a Telemedicine-based Medication Delivery Unit for Older Adults through Inspection Methods

Frank M Ligons 1,2, Katrina M Romagnoli 1,2, Suzanne Browell 3, Harry S Hochheiser 1,2, Steven M Handler 1,2
PMCID: PMC3243181  PMID: 22195137

Abstract

Polypharmacy and medication non-adherence are common in older adults, potentially leading to medication-related problems and increased healthcare expenditures. Medication Delivery Units (MDUs) may improve adherence, but their interfaces may present usability challenges for older adults with age-related impairments. We used a combination of three inspection methods – heuristic evaluation, cognitive walkthrough, and simulated elderly interaction, to identify potential concerns with the usability of a commercially available telemedicine MDU. Each method revealed different problems, with repeated discoveries via different methods providing triangulated evidence. Despite the MDU’s general usability, issues of varying severity were discovered. Significant usability issues associated with physical interactions with the MDU included loading and unloading the medication blister packs, and opening the delivered medication prior to administration. Less severe issues centered on small text sizes and poor user feedback. Further usability testing, involving older adults with a variety of impairments, is needed to validate findings.

Introduction

Polypharmacy, or multiple prescription drug use, is common in older adults, with one recent study suggesting that 29% use at least 5 prescription medications concurrently.1 Although polypharmacy is sometimes necessary, the potential difficulty of managing these medications can lead to non-adherence.2 Estimates of medication non-adherence vary widely in the literature, ranging from 0% to 100%3, 4, depending on the characteristics of the patient and the underlying disease(s) being treated. However, one estimate of community-dwelling older adult medication non-adherence is approximately 50%.5

A recent review assessed the potential barriers to medication adherence in the elderly and found that several patient-specific factors, including disease-related knowledge, health literacy, and cognitive function, affected rates of non-adherence.6 Non-adherence is of particular importance in older adults, as it can lead to various medication-related problems such as medication errors, adverse drug events, adverse drug withdrawal events, and therapeutic failures.7 These medication-related problems contribute to increases in hospitalization, readmission rates, emergency department visits, and healthcare costs.810

A wide variety of behavioral and educational interventions aimed at improving medication adherence have been investigated, including: 1) simplifying medication regimens; 2) identifying medication side effects; 3) improving patient self-efficacy; and 4) providing alerts or reminders to take medication as prescribed. Medication Delivery Units (MDUs) can simplify the process of managing complicated medication regimens, providing reminders as to when and how a drug should be taken, and automating the medication delivery process. More advanced “integrated” MDUs have telecommunication capabilities, including additional health monitoring and management functionality such as medication adherence reporting and remote prescription management.11

To improve adherence, MDUs must be usable, particularly by older adults who may have cognitive, visual, and/or fine motor impairments, any of which might complicate the processes of interpreting and interacting with the MDU’s computer interface. Although we initially hypothesized that interactions with the MDU’s touchscreen interface would be the source of most potential difficulties, our task-oriented inspections would allow us to identify potential difficulties with other aspects of the system, including loading and unloading of the medications and removal of pills from the blister packs. The potential identification of these important, but not computer-related, usability concerns illustrates the contribution of these inspections.

There are unique methodological challenges in evaluating how older adults may use MDUs. Usability challenges that are easily met by technologically savvy users may pose serious difficulties for older users who are less accustomed to using computers and other types of technology. There are also specific best practices to address difficulties that older adults may have with communication, attention, speed of comprehension, location/environment and moderator selection.12 Finally, for the inexperienced user, unfamiliar technology of any kind may produce anxiety and reticence that may not be tied to the specific details of any given device.13

As an initial effort to assess the usability of a telemedicine MDU for older adults, three different evaluation methods were employed to identify problems that may impact their use of these devices. Usability was assessed using two well-established usability inspection methods: heuristic evaluation14 and cognitive walkthrough analysis.15 The third method involved completion of common tasks while wearing a kit that simulated visual and motor impairments frequently experienced by older adults. In addition to providing insight into the usability of the device, this approach provides a preliminary model for triangulated assessments of usability for older adult users.

Methods

The EMMA® (Electronic Medication Management Assistant, INRange Systems (http://www.inrangesystems.com)), is an integrated telemedicine MDU developed to address the need for home-based medication management.16 This MDU has an integrated touchscreen user interface and cellular wireless communication capabilities used for remote monitoring and management of medications. The EMMA® dispenses medications that come from the pharmacy in sheets of single-dose blister packs, which are loaded into the front of the machine. The machine is capable of storing a 30-day supply of a once-daily medication using up to 10 different blister packs (i.e., 300 pills). A second unit, without a touchscreen interface, can be daisy-chained to the primary machine, increasing the total capacity to 20 blister packs (i.e., 600 pills). Remote medication management tools can be used to adjust dosage and schedules as needed. At the times indicated by the medication prescription, medications are “dropped” (i.e., not dispensed) into an open front drawer and remain in a blister pack cell. The patient then uses a special tool, the Pill-Dini medicine extractor, provided to patients along with the EMMA® to remove the pill from the blister pack. PRN dosages can be delivered as well, if allowed by the prescription, and advance “drops” for vacations or time away from the MDU can be requested as needed. The machine does not deliver liquid or powder preparations, nor does it deliver medicines that are prefilled in syringes.

There are two categories of usability evaluations from which to choose: inspection methods and usability (or user) testing. We selected inspection methods because they can be carried out much less expensively than usability testing, while still capturing the majority of the most important issues that may affect use of this device.17 Three human computer interaction (HCI) inspection methods were applied to the EMMA® MDU: 1) heuristic evaluation; 2) cognitive walkthrough; and 3) simulated interaction with age-related visual and motor impairments. These three methods were used together to provide a more comprehensive picture than would have been derived from any single approach.

Heuristic Evaluation

Heuristic evaluation is an inexpensive usability inspection method that involves applying a set of validated principles for interface design to a system such as a MDU. Specifically, evaluators examine various aspects of an interface with respect to validated principles such as “speak the user’s language” and “minimize memory load,” identifying violations as potential sources of usability challenges.18 Heuristic evaluation has been shown to effectively find issues at a relatively low cost19, even with a few evaluators20 who are not trained in human-computer interaction or interface design.18

Accepted heuristics for these evaluations have been developed for a general user population. However, they do not account for usability challenges associated with cognitive, visual, and fine motor impairments that may lead to increased usability difficulties for older users.21 To address this shortcoming, our evaluation augmented a subset (8 of the 9) heuristics promoted by Nielsen22 with additional guidelines from the National Library of Medicine regarding website design for older adults.23 We purposely chose not to include Nielsen’s “provide shortcuts” heuristic. Our rationale for omitting this heuristic was that as the EMMA® MDU interface task pathways are very short, involving few opportunities for shortcuts, a category critiquing this aspect of the device was unnecessary. The full list of heuristics applied as part of this study is given in Table 1.

Table 1:

Heuristics used to assess the EMMA® Telemedicine Delivery Unit

Nielsen National Library of Medicine
Simple and natural dialogue* Only use sans-serif typeface, 12–14 point, medium or bold weight, double spaced, left justified
Speak the user’s language
Minimize the user’s memory load* Avoid blue, yellow and green in close proximity
Be consistent
Provide feedback Use light background with dark text or dark background with light text and avoid patterned backgrounds
Provide clearly marked exits
Use good error messages
Prevent errors
Provide shortcuts
*

Heuristics also found in the NLM list of heuristics;

Nielsen heuristic intentionally not studied.

We evaluated the MDU with respect to each of these heuristic guidelines. For each heuristic, the interface was thoroughly explored, looking for instances where that heuristic was violated. Evidence of violations was recorded as a potential usability problem. The authors who conducted the evaluations (FML, KMR, and SB) discussed findings cooperatively, merging redundant findings and generating consensus descriptions for them. Majority vote was used to determine the relevance and severity of each issue.

Cognitive Walkthrough

Cognitive walkthroughs are descendants of the traditional walkthroughs pioneered by the software engineering community.15, 24 This approach focuses on the cognitive processes involved in task completion, instead of the functionality of the system. Cognitive walkthrough was chosen as a well-established evaluation method that has been employed in a wide variety of studies and extended considerably for a plethora of contexts and needs.15, 25 Cognitive walkthroughs were deemed particularly appropriate for the EMMA®, because the universe of possible tasks is small, allowing tasks of all levels of importance and frequency of use to be evaluated.26

The cognitive walkthrough consisted of several stages. First, a user context was developed by the evaluator. Consideration was given to the specific targeted user group (i.e. older adults) and their potential challenges in using the system in question. In this study, the context used in the evaluation was that of older adults with limited experience and comfort using computers or similar technology. This user profile was then used as the background for scrutiny of a set of tasks and the actions needed to complete those tasks. These actions were evaluated with a series of questions aimed at developing compelling explanations as to why the user might (or might not) complete the actions (and therefore the tasks) successfully.24 If the questioning lead to the conclusion that a given action might not be completed correctly, that step in the process was noted as a potential usability problem.

A series of representative tasks were selected, with a focus on covering all likely interactions (Table 2). Step-by-step instructions for each walkthrough task were elicited from the instruction manual provided by the device manufacturer. The research team collectively performed these tasks supported by the EMMA® machine, step-by-step. Each step was discussed with respect to any impediments to performing the step including: 1) confusion around instructions; 2) cognitive or physical challenges; 3) the impact of wait times; and 4) the congruence between the expectations of the consequences of performing a step and the actual outcome. Once a task was complete, the research team came to a consensus on how to rank any issues discovered and how best to describe them.

Table 2:

Cognitive Walkthrough Tasks

1. Load a blister pack*
2. Unload a blister pack*
3. Deliver a scheduled medication
4. Deliver an as-needed (PRN) medication*
5. Schedule a vacation drop
6. Schedule an advance drop for scheduled medication
7. Schedule an advance drop for as-needed medication
8. Schedule a manual drop
9. View inventory
10. Get help
*

Tasks also performed using the elderly kit (see below).

Elderly Kit

Simulations of limitations faced by older users may provide insight into potential usability problems. This study used an “elderly kit,” developed by the University of Pittsburgh’s Institute on Aging (http://www.aging.pitt.edu), as a geriatric-sensitivity training aid. The kit contains components designed to simulate some of the most common medical conditions that affect older adults. The kit includes glasses that mimic common visual impairments including cataracts and macular degeneration, as well as rubber gloves to simulate a loss of fine motor sensitivity and control caused by conditions such as arthritis. The simulation kit was used to provide preliminary insights into usability problems that older adults with these particular common medical conditions might face using this MDU. Similar approaches have been used by several groups in an effort to make everyone from engineers to architects more sensitive to the realities of the impairments faced by their older customers27, 28, and to uncover difficulties visually impaired users may face during ambulatory navigation through a hospital campus.29

One member of the evaluation team (KMR) acted as the “impaired elderly user,” performing a subset of the tasks included from the cognitive walkthrough (Tasks 1, 2 and 4 from Table 2), along with two additional tasks: receiving a blister pack and extracting the pill from it. These five tasks were chosen to represent the range of interactions required of device users, in the hopes of identifying all of the difficulties that might be faced by older users with physical limitations. While the opening of the blister pack cell does not involve direct interaction with the EMMA® MDU, pill extraction was evaluated because visual or fine motor impairments may affect this essential task. The evaluator tested each of the three elements of the elderly kit (1 pair of gloves and 2 pairs of glasses) independently, performing each of the five tasks with each of the three elements. As with the other inspection methods, problems discovered during interactions were noted and a consensus set of usability difficulties identified.

Analysis

The evaluators categorized the usability issues discovered during each of the three evaluations as major, medium, or minor. They defined problems as major if they were likely to result in task failure, medium if they were likely to confuse the user, but unlikely to result in task failure, and minor if they were annoyances that would not interfere with task success.

Results

Examination of the usability issues discovered led to the identification of four categories of usability problems: 1) physical and perceptual problems with the unloading/loading of the medication blister packs and the small font size of the medication information imprinted on each blister cell; 2) organizational problems with task design and information layout; 3) feedback problems involving shortcomings in the communication of system state to the user; and 4) inconsistencies in design elements and terminology. Each of the usability issues identified in the inspections were assigned to one or more of these categories. A summary of these problems, along with their categories and severity, can be found in Table 3.

Table 3:

Usability Challenges Encountered using the EMMA®

Major Medium Minor
Physical/Perceptual Narrow medication loading slot (C, S)
Hard to read type (S)
Difficult to identify, retrieve, and open delivered medication (S)
Blister pack type is small (H, C, S)
Small font size of medication instructions (H, S)
Small screen and button size (H,S)
Difficult to select buttons (H,S)
Organization None Poorly organized menus (C) “View Inventory” is not available on the home screen (C)
“Drops” are poorly organized (H,C)
Feedback The help menu is cryptic (C) No feedback from buttons (H,C)
Not all processes have progress bars (H,C)
No feedback when a drop is initiated (C)
None
Consistency None Inconsistent use of progress bars (C) Commands are used inconsistently (H, C)
“Drop” menu poorly organized (H,C)

Note: Letters refer to the analysis method that identified a particular problem: H= Heuristic Evaluation; C= Cognitive Walkthrough; S= Simulation of Age-Related Impairments.

Heuristic Analysis

Violations of several heuristics were found, including those related to memory load, consistency, feedback, errors, and typeface, and are listed below.

Minimize memory load

We identified major issues with the help menu. As there is no “Help” button, but merely a button with a question mark, it is not clear how the help menu should be accessed. Once a user presses the “?” button they are taken to an intermediate screen where they are presented with several more potentially confusing buttons, including one labeled “English” and another labeled “Press.” A third button, labeled with a question mark, is the correct button to press for accessing the help system. This presents a memory load issue because users have to employ trial-and-error to figure out how to get to the help system.

The other major memory load comes from the “Drop” menu. The drop function is intended for delivering a set amount of medication based on the number of hours, or days, that a patient will not have access to the MDU (i.e., while away from home on vacation). Memory load becomes an issue because the user must remember why they are doing the drop, which dates the drop includes, and then keep up with the schedule of taking the medications while they are away from the EMMA®.

Be Consistent

Two minor usability issues were found. First, all “medication drops” are not on the same page, with “advance drops” being unexpectedly located on a second page, separated from the other drops (vacation drops and manual drops) on the first page of the menu, which has room for 2 more buttons. Additionally, some task pathways require the use of a “back” button to move to a prior screen, whereas others require use of a “cancel” button.

Provide Feedback

Several medium usability issues were discovered. There is no tactile, visual or auditory feedback indicating when a button is pressed or which button was pressed. This may result in users accidentally, or repeatedly, pressing buttons leading to an unintended result, such as ending up on the wrong screen. Lack of feedback is exacerbated by the long processing times required by some of the internal tasks (when the machine is communicating with the server, moving blister packs during loading and unloading, or during medication delivery) performed by the EMMA®. These long waits are further complicated when no information is provided about what precisely is happening or how long it will take to complete.

Prevent Errors

The analysis identified two medium issues with error prevention, including the potential for incorrect button selection and difficulty loading and unloading the blister packs into the MDU. The most important button, the “Touch Here” button to take a medication, is well designed. However, the remaining buttons are relatively small (∼4 cm), with a narrow (<1 cm) border separating them, making incorrect button selection a real possibility. The load and unload tasks are also susceptible to error. For example, it is unclear how to load the blister packs. Uncertainty about how to best align the directional arrows on the blister pack for insertion, and how far to insert the blister pack, may cause confusion and anxiety. Even when one is already aware of how to load and unload the blister packs, the precise requirements for three-dimensional positioning and force make it challenging even for users without motor impairments.

Typeface

The EMMA® uses sans-serif, bold, left-justified, double-spaced type for all on-screen commands and prompts. Blister packs for the device use sans-serif font in a variety of sizes, some of which is bolded. Most of the text is readable with two exceptions, which were rated as medium in severity: 1) the on-screen directions for taking medication and 2) the detailed drug information printed on the back of the blister packs indicating the name and dose of the medication. In both cases, the small font size makes it difficult to read, even for persons with uncompromised vision.

Cognitive Walkthrough

The cognitive walkthrough identified major usability issues with loading the blister packs of medication and accessing the help menu. In addition to the difficulties with the blister packs noted above, difficulties exist with the insertion of the blister pack into the loading slot. The slot that accepts the blister pack appears large, but it contains a second loading tray within that is difficult to see because it is black and narrow (less than 0.5 cm high). The blister pack must be precisely inserted into this inner loading tray to be accepted into the machine. Given the small size of the loading tray, and the fact that the blister pack is almost exactly as wide as the tray, there is little room for error in alignment. This is a major issue, as even unimpaired users may find this task difficult.

The second major issue - difficulty in accessing the help system – reiterates related findings from the heuristic analysis.

Problems considered medium in severity include difficulty seeing text, poorly organized information and tasks within the interface, lack of feedback, and inconsistency throughout the system. Text size varies throughout the interface with some text appearing large and easy to read (i.e. the “Touch Here” button which is used to deliver the medications) and other text appearing small and difficult to read even with perfect vision. The screen (7.5 inches diagonal) and the on-screen buttons are relatively small, particularly for those with imperfect vision. The blister packs are also difficult to read. Text elements describing the contents of each pack, including the medication name, dose, patient name, and the correct orientation of the pack for loading into the device, are all very small. The functionality behind the “Options” and “More” buttons is not clearly organized, with unclear labels and categorizations presenting the possibility of confusion in menu navigation.

Some tasks such as navigating to the drop menu, deciding amongst the various drops, and successfully configuring a drop, can be confusing. Feedback problems identified in the heuristic evaluation, including lack of response when buttons are pushed and lack of progress or status bars indicating the state of long-running processes, arose during the course of the cognitive walkthrough. A problem was identified with drops, as the interface does not clearly describe the differences between various types of drops (vacation, manual, and advance). Vacation drops are used when a user is going on vacation and needs to take his medication(s) along. A manual drop delivers an entire blister card of medication. Advance drops are utilized when the user is going away for a few hours and needs to take his medications with him. Although this information is in the user manual, this is not considered sufficient for avoiding potential problems. Drops also lack appropriate feedback when a drop is in progress. No feedback is provided about how the device is responding to the user’s request or how long it will take to complete.

A number of other minor issues were uncovered. First, pressing a specific on-screen button requires visual acuity as well as fine motor dexterity, as buttons are relatively small and grouped closely together. Second, the user cannot view the medication inventory from the main screen, which can make finding this function inconvenient. Lastly, as noted in the heuristic analysis, there is variance in the interface’s terminology. Both “enter” and “confirm” are used interchangeably as well as “back” and “cancel.”

Usability Tests with Age-Related Impairment Kit

The usability tests with the elderly kit revealed potential difficulties faced by older persons with visual or fine-motor impairments while using the EMMA®. Major problems identified include those seen in the other inspections, including difficulties with the size of the insertion slot for the blister pack and small type on the screen. The evaluators also found major difficulties with retrieving and opening the medication once it had been delivered. These problems were found using all three of the simulations enabled by the elderly kit. The cataract simulation glasses made reading text on the screen, and distinguishing the boundaries of buttons, very difficult. Loading and unloading the blister packs was particularly challenging. Retrieving the delivered medications, and opening them with the Pill-Dini™, requires good visual acuity to see the pill in the unit, and to correctly align the blister pack and cutting blade. It was also almost impossible to accomplish this with the cataract glasses. The macular degeneration glasses presented similar difficulties, but to a slightly lesser degree. The gloves impeded the handling of the delivered blister packs and added even more challenge to opening the delivered medications with the Pill-Dini™.

Problems of medium severity include the small font size on both the blister packs and the on-screen medication administration instructions. The cataract glasses made these elements impossible to read, with readability only slightly improved with the macular degeneration glasses. Being unable to read a medication’s name and dosage would usually be rated as a major problem. Still, this is ranked as a medium issue because each blister pack is scanned to ensure that it is the correct medication and strength when first loaded into the MDU. However, difficulty in reading the on-screen instructions for how to take the medication may lead to administration errors.

Discussion

The EMMA® telemedicine medication delivery unit (MDU) was evaluated using three different inspection methods to identify problems that may impact the ability of older adults to use this device. The three inspection techniques identified different, but complementary, usability concerns at a relatively low cost.

The results of our inspections highlight the importance of treating usability inspections as end-to-end evaluations, involving all aspects of the use of a system, as opposed to narrowly focusing on interactions with computer interfaces only. For example, the loading and unloading of the medication blister packs require interaction with a narrow slot in the device, and correct orientation of the packs, despite sparse orientation cues and the use of small type. Our findings illustrate the “major” severity of these potential usability issues and run counter to our initial assumption that interactions with the touchscreen interface would be the cause of the most significant difficulties.

Removal of the medication from the blister pack cells presents similar concerns. Using the provided Pill-Dini™ to remove pills from the blister pack cell requires good visual acuity and fine-motor control for placing the individual blister pack in the device, slicing through the blister pack, removing the pack from the device, and retrieving the pill from the sliced pack.

Further investigation, likely including usability tests with older users, will be needed to fully understand the impact of these potential usability concerns. As for difficulty with loading and unloading the EMMA®, because each of the EMMA® MDUs can be loaded with up to 300 pills (usually enough for a 30-day supply of 10 unique solid-form medications), as well as daisy chained with another of the EMMA® devices with the same capacity, this should be a relatively infrequent task, possibly handled by a caregiver. The removal of the pills from the blister pack may be the task most likely to prevent successful use of the MDU. As some State and Federal regulatory requirements prohibit the appealing alternative of delivering the pill without the blister pack, alternative mechanisms of extracting the pill might be helpful for some users, particularly those with poor motor control.

Small screen size, lack of touchscreen feedback, and sluggish software responsiveness also present concerns. For users with visual impairments, larger screens and selection targets are likely to improve performance. Feedback cues using haptic elements, which confer a vibration to the user upon approaching or touching a button, may prevent errors by making users more aware of what they are touching on-screen and when. Other audio or visual cues might also be useful. There were also noticeable time lags in the responsiveness of the interface after the user presses a button. The elderly kit provided additional perspective that would not have arisen from the heuristic evaluation or cognitive walkthrough. Even design guidelines specifically created for older adults23 do not include detailed advice for designs that might be effective for users with low vision, cataracts, macular degeneration or other visual impairments. Use of various, potentially redundant, feedback channels (i.e. auditory or haptic) may benefit these users.

Minor problems were identified with the gloves that simulate fine motor impairment. Difficulties with on-screen button selection suggest the possibility that users with impaired fine motor control may have difficulty using some aspects of the EMMA® effectively.

Limitations

The inspection techniques used in this study provide insights into the usability of the EMMA®, but these analyses are not definitive. Although heuristic evaluations and cognitive walkthroughs have proven useful in identifying usability problems, they are not necessarily comprehensive, sometimes capturing only a subset of the difficulties identified in usability tests with human participants.17, 30 Further, the heuristics chosen from the NLM guidance for web sites for older users23 have not been validated for use in heuristic evaluations.

This study has several additional limitations worth noting. First, each of the evaluators performed their inspections working together as a group. Second, each of the evaluators completed the same three evaluations in the same order. Both of these limitations could have lead to observations that were potentially biased, largely as a result of learning effects. For example, having all three evaluators present during the inspections may have sensitized evaluators to potential errors, and subsequent inspections may have been influenced by confirmation bias. The use of a simulated elderly kit has another potential limitation. Although the use of proxy users in informing design can be appropriate, particularly for users with impairments, testing with real users is preferred. The role that these proxy simulations play in the current study is directly analogous to that of usability inspections: as low-cost exploration of usability issues that may uncover major concerns without the overhead of full usability studies.31

Future Work and Implications

As usability inspections identify potential, not actual, usability problems, usability tests, possibly including think-aloud evaluations32, will be needed to identify those issues that are most troublesome for users. Given the proposed application for use with older patients, these studies should ideally involve this cohort with a variety of cognitive, visual and fine motor impairments. As a starting point for an effort aimed at exploring the utility of MDUs for improving medication adherence in older adults, these results have informed the design of study participant screening processes and exclusion criteria. Our data is also being used by INRange Systems, to design the next version of the EMMA® MDU, which will address many of the concerns identified in this study. Future studies assessing short and long-term medication adherence are needed, as well as the impact of the MDU on medication-related adverse events and healthcare utilization. Future studies should also determine which patients are most likely to benefit from a MDU based on their: 1) degree and type (i.e., intentional vs. non-intentional) of non-adherence; 2) clinical context (e.g., assisted living, home-based primary care, or during transition from hospital to home); and 3) ability to operate the device; predicted likelihood of successful use; and degree of cognitive impairment (i.e., mild cognitive impairment vs. dementia).

Conclusions

Three different human computer interaction inspection methods evaluated the EMMA® MDU, and they identified problems that may make it difficult for older adults to use the device. Triangulation unearthed repeated findings which further confirmed their validity. Despite the device’s general usability, issues of varying severity were discovered. Although our inspections suggest that individuals with advanced cataract disease may not be able to use the device successfully, we did not find sufficient evidence that users with other visual and fine motor impairments would not be able to use the device.

Acknowledgments

This project was supported, in part, by a National Institutes of Health (NIH) grant 5KL2RR024151 (NIH Roadmap Multidisciplinary Clinical Research Career Development Award Grant), an Agency for Healthcare Research Quality (AHRQ) R01HS018721, a National Institute on Aging K07AG033174, and a NLM Fellowship 5 T15 LM007059-24. The authors also wish to thank INRange Systems for providing the EMMA® Medication Delivery Unit used for the duration of this study. None of the authors have any financial relationship with INRange Systems.

References

  • 1.Qato DM, Alexander GC, Conti RM, Johnson M, Schumm P, Lindau ST. Use of prescription and over-the-counter medications and dietary supplements among older adults in the United States. JAMA - Journal of the American Medical Association. 2008;300(24):2867–2878. doi: 10.1001/jama.2008.892. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Rifkin DE, Laws MB, Rao M, Balakrishnan VS, Sarnak MJ, Wilson IB. Medication adherence behavior and priorities among older adults with CKD: A semistructured interview study. American Journal of Kidney Diseases. 2010;56(3):439–446. doi: 10.1053/j.ajkd.2010.04.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Haynes RB, Taylor DW, Sackett DL. Compliance in health care. Baltimore: 1979. [Google Scholar]
  • 4.DiMatteo MR. Variations in patients’ adherence to medical recommendations: A quantitative review of 50 years of research. Medical care. 2004;42(3):200–209. doi: 10.1097/01.mlr.0000114908.90348.f9. [DOI] [PubMed] [Google Scholar]
  • 5.Murray MD, Darnell J, Weinberger M, Martz BL. Factors contributing to medication noncompliance in elderly public housing tenants. Drug Intelligence and Clinical Pharmacy. 1986;20(2):146–152. doi: 10.1177/106002808602000210. [DOI] [PubMed] [Google Scholar]
  • 6.Gellad WF, Grenard JL, Marcum ZA. A systematic review of barriers to medication adherence in the elderly: Looking beyond cost and regimen complexity. American Journal Geriatric Pharmacotherapy. 2011;9(1):11–23. doi: 10.1016/j.amjopharm.2011.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Handler SM, Wright RM, Ruby CM, Hanlon JT. Epidemiology of medication-related adverse events in nursing homes. American Journal Geriatric Pharmacotherapy. 2006;4(3):264–272. doi: 10.1016/j.amjopharm.2006.09.011. [DOI] [PubMed] [Google Scholar]
  • 8.Colt HG, Shapiro AP. Drug-induced illness as a cause for admission to a community hospital. Journal of the American Geriatrics Society. 1989;37(4):323–326. doi: 10.1111/j.1532-5415.1989.tb05498.x. [DOI] [PubMed] [Google Scholar]
  • 9.Col N, Fanale JE, Kronholm P. The role of medication noncompliance and adverse drug reactions in hospitalizations of the elderly. Archives of Internal Medicine. 1990;150(4):841–845. [PubMed] [Google Scholar]
  • 10.Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients: Excess length of stay, extra costs, and attributable mortality. Journal of the American Medical Association. 1997;277(4):301–306. [PubMed] [Google Scholar]
  • 11.Center for Technology and Aging Technologies for Improving Post-Acute Care Transitions. http://www.techandaging.org/PACTdraftPositionPaper.pdf. Accessed 7/2/10.
  • 12.Barrett J, Kirk S. Running focus groups with elderly and disabled elderly participants. Applied Ergonomics. 2000;31(6):621–629. doi: 10.1016/s0003-6870(00)00031-4. [DOI] [PubMed] [Google Scholar]
  • 13.Goebel M. Ergonomic design of computerized devices for elderly persons - The challenge of matching antagonistic requirements. 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007; 2007. pp. 894–903. Vol 4554 LNCS; [Google Scholar]
  • 14.Nielsen J, Mack RL. Usability inspection methods. New York: Wiley; 1994. [Google Scholar]
  • 15.Lewis C, Polson P, Wharton C, Rieman J. Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. CHI Conference on Human Factors in Computing Systems; 1990. pp. 235–242. (Journal Article): [Google Scholar]
  • 16.Williams C, Honeywell M, Robertson N, Robinson T, Welch T. Novel Device for Medication Management. http://www.uspharmacist.com/content/t/technology/c/13778/. Accessed 7/28/10.
  • 17.Karat CM. Usability inspection methods. John Wiley & Sons, Inc; 1994. A comparison of user interface evaluation methods; pp. 203pp. 203–233. [Google Scholar]
  • 18.Nielsen J. Finding usability problems through heuristic evaluation. Paper presented at: ACM Conference on Human Factors in Computing Systems - CHI ’92; 1992; New York, NY, United States: 1992. [Google Scholar]
  • 19.Nielsen J. Cost-justifying usability. Orlando, FL, USA: Academic Press, Inc.; 1994. Guerrilla HCI: using discount usability engineering to penetrate the intimidation barrier; pp. 245pp. 245–272. [Google Scholar]
  • 20.Lazar J, Feng JH, Hochheiser H. Research methods in human-computer interaction. Chichester, West Sussex, UK: Wiley; 2010. [Google Scholar]
  • 21.Hawthorn D. Possible implications of aging for interface designers. Interacting with Computers. 2000;12(5):507–528. [Google Scholar]
  • 22.Nielsen J, Molich R. Heuristic evaluation of user interfaces. Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people; Seattle, Washington, United States: ACM; 1990. pp. 249–256. [Google Scholar]
  • 23.National Library of Medicine Making Your Website Senior Friendly. http://www.nlm.nih.gov/pubs/staffpubs/od/ocpl/agingchecklist.html. Accessed 7/21/10.
  • 24.Polson PG, Lewis C, Rieman J, Wharton C. Cognitive walkthroughs: a method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies. 1992;36(5):741–773. [Google Scholar]
  • 25.Mahatody T, Sagar M, Kolski C. State of the art on the cognitive walkthrough method, its variants and evolutions. International Journal of Human-Computer Interaction. 2010;26(8):741–785. [Google Scholar]
  • 26.Shneiderman B, Plaisant C. Designing the user interface : strategies for effective human-computer interaction. 5th. Boston: Addison-Wesley; 2010. [Google Scholar]
  • 27.Agnes Lab at MIT AGNES (Age Gain Now Empathy System) http://agelab.mit.edu/agnes-age-gain-now-empathy-system. Accessed 7/25/10.
  • 28.BBC Online News The Suit that Makes You Feel Old. http://news.bbc.co.uk/2/hi/health/3538220.stm. Accessed 7/10/10.
  • 29.Rousek JB, Koneczny S, Hallbeck MS. Simulating visual impairment to detect hospital wayfinding difficulties. Paper presented at: 53rd Human Factors and Ergonomics Society Annual Meeting 2009, HFES 2009; 2009. [Google Scholar]
  • 30.Desurvire HW. Usability inspection methods. John Wiley & Sons, Inc; 1994. Faster, cheaper!! Are usability inspection methods as effective as empirical testing? pp. 173pp. 173–202. [Google Scholar]
  • 31.Jeffries R, Miller JR, Wharton C, Uyeda K. User interface evaluation in the real world: a comparison of four techniques. Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology; 1991. [Google Scholar]
  • 32.Ericsson KA, Simon HA. Protocol analysis: verbal reports as data. Cambridge, Mass: MIT Press; 1984. [Google Scholar]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association

RESOURCES