Abstract
Purpose
To examine the relationship between cognitive status and the usability of an integrated medication delivery unit (MDU) in older adults who reside in an Assisted Living Facility (ALF).
Methods
Subjects were recruited from a single ALF in Pittsburgh, PA. Usability testing sessions required subjects to execute tasks essential to using EMMA® (Electronic Medication Management Assistant), a Class II Federal Drug Administration (FDA) approved integrated MDU. Video coding allowed for quantification of usability errors observed during the testing sessions. Each subject's cognitive status was assessed using the Mini Mental State Exam (MMSE®) with scores <24 indicating cognitive impairment. Functional status was assessed using the Lawton Instrumental Activities of Daily Living (IADL) questionnaire, and a global assessment of subjective usability was assessed by completing the System Usability Scale (SUS). Non-parametric statistics and correlation analysis were used to determine whether significant differences existed between cognitively impaired and non-impaired subjects.
Results
Nineteen subjects were recruited and completed the protocol. The subject pool was primarily white, female, 80+ and in possession of above average education. There was a significant relationship between MMSE® scores and the percentage of task success (z = −2.03, p = 0.04). Subjects with MMSE® scores of 24+ (no cognitive impairment) successfully completed an average of 69.0% of tasks vs. the 34.7% performance for those in the cognitively impaired group (<24). Six of the unimpaired group also succeeded at meeting the 85% (6 out of 7 correct) threshold, while none of the impaired group was able to. No subject with cognitive impairments (<24 MMSE®) completed more than 5/7 (71.4%) of their tasks. Two of the impaired subjects failed all of the tasks. Three of the MMSE®'s subsections (Date, Location and Spell ‘world’ backwards) were found to be significantly related (p < 0.05) to the percentage of task success. Tasks success rates were related with IADL scores (z = −3.826, p < 0.0001), and SUS scores (r = 0.467, p = 0.0429).
Conclusions
Medication delivery units like EMMA® have the potential to improve medication management by combining reminder systems with telemedical monitoring and control capabilities. However, subjects judged to be “cognitively impaired” (<24 MMSE®) scored a significantly smaller percentage of task success than the “unimpaired” (> = 24), suggesting that cognitive screening of patients prior to the use of EMMA® may be advisable. Further studies are needed to test the use of EMMA® amongst ALF residents without cognitive impairment to see if this technology can improve medication adherence.
Keywords: Medication adherence, Aged, Telemedicine, Medication systems/organization & administration, Usability
1. Introduction
The older adult population in the United States is large and growing, with approximately 20% of Americans projected to be 65 or older by 2030 [1]. Although older adults consume more medications than any other segment of the population [2,3], their ability to manage their medication regimen decreases with age [4–6]. Some older adults employ medication management strategies, such as the use of self-filled med-sets, easy-open vials, pill boxes, alarm clocks, reminder alarms, watches, vibrating/alarmed pill boxes, and automated pill dispensers [7–9]. These and other approaches to improve medication adherence have yielded mixed results [10–13].
A patient's ability to self-administer medication is the strongest predictor of whether he/she will be placed in an Assisted Living Facility (ALF), eclipsing other indicators such as age, cognitive status, ability to perform instrumental activities of daily living (IADL) [14], depression, and frequency of falling [15]. The vast majority (77%) of ALF residents in the U.S. receive some form of medication management assistance [16], which may include administration by nurses (i.e. RNs/LPNs) or medication technicians (depending on state regulations).
Integrated telemedical medication delivery units (MDUs) present a potential new approach to providing medication management services in ALFs. MDUs are devices that deliver medication to patients at specific times, and in specific doses, while being programmed, monitored, and controlled remotely [17]. In addition to providing reminders (to patients and/or providers) [10], MDUs also keep logs, record patient self-administration, and act as electronic medication administration records (eMARs) [18]. Remote access facilities provide clinicians and pharmacists with access to medication adherence logs, along with new capabilities, including remote dosage adjustment and the delivery of alerts to formal and informal care providers.
The potential benefits of this new technology will not be realized if the MDU cannot be effectively used by older users. Our earlier work [17] used combination of static techniques (heuristic evaluation and cognitive walkthrough) to assess the usability of a Class II Federal Drug Administration (FDA) approved integrated MDU called EMMA® (INRange Systems, http://www.inrange.com). Although several potential usability issues were identified, we did not find any barriers that would prevent successful task completion. A recent survey of 96 older users of the MD.2 medication delivery device found that substantial majorities of respondents reacted favorably to the device, with more than 80% indicating that they would like to use the device in the future [19]. However, we are not aware of any work that addresses specific issues that might impact the ability of older patients to use medication delivery devices.
To gain understanding of some of these factors, our study asked older users to use the EMMA® MDU to complete tasks necessary for medication management. The participation of potential users of EMMA allows for the identification of difficulties actually experienced by these subjects, which may be as distinct from those predicted by the usability evaluations that relied upon student evaluators. We also explored the hypothesis that subjects with impaired cognitive function may experience usability difficulties that may interfere with successful use of the MDU for medication management. To study this hypothesis, we used valid and reliable measures of cognitive and functional status, a generic usability questionnaire, and video analysis [20] of task performance to identify specific difficulties experienced by study participants.
2. Materials and methods
Following Institutional Review Board approval by the University of Pittsburgh, we recruited subjects from a single 100-bed independently-owned ALF in Western Pennsylvania. The majority (75/79) of residents at this facility receive medication management services.
2.1. Medication delivery unit
EMMA® delivers medications from single-dose blister cards, according to schedules programmed remotely by prescribing pharmacies. Blister cards containing 30 wells (approximately one month's supply of a single medication) are inserted via a loading tray in the front of the device, with up to 10 different blister cards available in a single machine, and additional medications available through a second daisy-chained device. User interaction with EMMA® is accomplished via a touch-screen interface. Medications are delivered into a drawer at the bottom of the device. Remote control functionality is available via a secure online web connection that communicates data from EMMA® through an embedded cellular modem or LAN connection.
When the programmed schedule indicates that the patient is due to take a scheduled medication, EMMA® will beep and display a flashing message on the touch screen indicating which medication(s) are available and should be taken. The user must press a button on the touch screen and confirm that they are ready to take the medication(s). The user must then retrieve the delivered blister card from the drawer and extract the medication. This can be accomplished by pushing the medication out through the foil, using scissors, or utilizing the Pill-DiniTM medication extractor – a small, handheld plastic device containing a razor blade that will cut the blister seals surrounding medications placed in the slot. As needed, “pro re nata” (PRN) doses are delivered by the prescription programmed into the device, and can be accessed via a series of menu options selected from the touch screen interface. Advance delivery of needed dosage is available for travel away from home and from the EMMA® interface.
2.2. Experimental protocol and survey
All data collection procedures were conducted by the first author (FML) at the Asbury Heights Villas ALF in Pittsburgh, PA. Following informed consent, each subject was asked 14 questions concerning their socio-demographic disposition (e.g., age, gender, education, etc.). Assessment of each subject's cognitive status was conducted using the Mini-Mental State Exam (MMSE®). Functional status was assessed via Lawton and Brody's Instrumental Activities of Daily Living (IADL) [14]. Subjective impressions of the usability of the EMMA® were assessed using the System Usability Scale (SUS) [21].
2.2.1. Mini-Mental State Exam (MMSE®)
The MMSE® is a brief, easily administered screen for cognitive impairment. We used previously [22] validated cut-points to classify MMSE® scores, with scores ≤23 considered indicative of some degree of cognitive impairment [22,23].
2.2.2. Instrument activities of daily living (IADL)
IADLs assess each respondent's functional status as related to their ability to perform certain daily activities required to function independently in a community setting, such as laundry, money management, and telephone use, without assistance [14]. This instrument results in a score (0–8), one point for each of the eight subsections of activities. Higher scores indicate more independence.
2.2.3. System Usability Scale (SUS)
After usability testing, we employed the SUS to collect data concerning each subject's opinions and impressions following the use of a technical device. This brief verbal interview provides insight into a users’ perceptions, attitudes, and feelings. The calculation of the SUS results in an overall score ranging from 0 to 100, with higher values indicating better perceived usability [21]. Descriptive statistics were used to summarize all scores.
2.2.4. Usability testing
We chose seven tasks necessary for successful use of the EMMA® for medication management, including Load, Unload, Deliver as-needed (PRN) medication, Remove Pill from Blister Pack, Manual drop, View inventory, and Get help (Table 1).
Table 1.
Usability tasks.
| Task Name | Description |
|---|---|
| Load | Load blister cards (new prescriptions or refills) |
| Unload | Remove empty blister cards |
| Deliver | Deliver medications |
| Remove pill | Extract medication from blister pack |
| Manual drop | Pre-deliver medications for use during travel away from EMMA® |
| View Inventory | Check availability of medications |
| Help | Get assistance with use of EMMA® |
Prior to administering the usability test to each subject, FML demonstrated how the required tasks are best accomplished on EMMA®. Subjects were instructed to watch this demonstration carefully in order to replicate the process on their own without any help.
Subjects were positioned in front of the EMMA® device, with the touchscreen within reach. A video camera was positioned behind the subjects and focused on the touchscreen, to capture both touchscreen actions and participant utterances.
Subjects were given a maximum of 5 min to successfully complete each task. During this time, subjects could repeatedly attempt to complete the task until successful, or it would count as a task failure. Tasks were counted as successes if the subject ultimately finished the task before the 5 min were up, even if there were errors along the way. Subjects also had the option to “pass” on a task, at any time, simply by telling the investigator that they were unable to continue.
2.3. Data analysis
2.3.1. Data coding
Video recordings were reviewed and coded for task completion success and for error conditions. Usability errors were defined as either difficulties or slips. Tasks without errors were coded as “No Difficulties” (ND) or “No Slips” (NS).
Difficulties were defined as incidences of medical conditions that interfered with the participant's ability to accomplish a task. Cognitive difficulties included apparent trouble with memory, attention, and problem solving. Difficulties were described as cognitive when subjects asked the investigator questions, made declarative statements indicating they did not know what to do next, or made incorrect declarative statements about their next action. Physical/Perceptual (P/P) difficulties included physical problems such as hand tremors or poor vision that might prevent successful task completion. Physical challenges were cited as difficulties when the difficulty prevented task completion in the time allowed. Verbalizations that indicated vision problems (i.e. “I can't see those buttons” or “My eyes aren't what they used to be.”) were counted as perceptual difficulties.
Slips are deviations from a task's optimal path – the shortest possible sequence of actions required to successfully complete the task. Some tasks, such as Manual Drop, required navigation through several pages of responses on EMMA®'s interface. Other tasks like “Get help” may be fulfilled by circuitous (not optimal) routes through the interface. There were three types of slips: Organizational slips involved the selection of the wrong button on the screen, thereby departing from the optimal path for that task. These mistakes were associated with either the design of the task pathway or the subject's understanding of the pathway. These errors may also have been related to poor presentation of key information (i.e. small font sizes). Feedback slips were recorded whenever EMMA® failed to provide useful feedback in response to a user action. Physical/Perceptual slips occurred when the participant knew what to do, but was hindered due to physical or perceptual limitations. Physical/perceptual slips differ from physical/perceptual difficulties in extent. If a physical/perceptual problem did not prevent the participant from eventually returning to the correct path, the problem was labeled a slip. Physical/perceptual problems that made task completion nearly or completely impossible were categorized as difficulties.
2.3.2. Video coding
A two-step process was used to code difficulties and slips for each task. In the first phase, individual difficulties and slips were identified. To avoid confounding effects associated with slips or difficulties that may have arisen in attempting to recover from prior problems, each slip/difficulty was associated with an extent in time. Specifically, after a difficulty or slip was initially recorded, no other difficulties/slips could be recorded for that task until the user (i) returned to the optimal path, (ii) returned to the home screen and restarted the task, or (iii) cited another difficulty or slip of the same type but with a different underlying cause. The result of this phase was a list of zero or more slips and difficulties for each task.
The second phase involved selecting a consensus slip and difficulty for each task. Specifically, each task was assigned slip and difficulty labels for the categories most frequently associated with that task. For example, if a subject's task scored three Physical/Perceptual difficulties and one cognitive difficulty, the Physical/Perceptual difficulty was the category chosen to represent the difficulties associated with the task. One investigator (FML) scored all 19 of the videos and a second author (KMR) independently scored 10 of the 19 videos.
2.4. Statistical analysis
The Mann–Whitney U test was used to investigate whether unimpaired subjects (MMSE® score of ≥24) had higher percentages of task success with EMMA® than those who were cognitively impaired (MMSE® score < 24) [22,23].
To determine whether any of the MMSE® subsections had statistically significant relationships with the percentage of task success, we calculated correlations between responses to MMSE® questions and task success rates as the independent variable and the percentage of task success as the dependent variable.
The Pearson correlation was used to determine if there was a statistically significant relationship between the SUS scores and the percentage of task success. The Wilcoxon Signed rank test was used to determine the relationship between the IADL scores and the percentage of task success [24].
To examine the impact of participant education, we calculated the Pearson correlation between education and MMSE® scores. We also used the Mann–Whitney U test to evaluate the relationship between education (stratified as “lower” for participants with no college education and “higher” for all others) and IADL, task success, and SUS scores.
To validate our scoring rubric we calculated Cohen's kappa agreement levels [25] between the two scorers.
Data analysis was carried out via SAS/STAT® software versions 9.2, 9.3 (SAS Institute, Inc., Cary, NC), and MS Excel 2007 (Microsoft, Inc., Redmond, WA). For all analyses, statistical significance was set at p < 0.05.
3. Results
3.1. Demographics
The mean age of the 19 subjects who volunteered to participate was 87.1 years of age, the majority were female (63.2%) and all but one had at least a high-school level of education and declared English as their primary language. Almost all subjects (94.7%) identified themselves as “White/Caucasian” (Table 2).
Table 2.
Participant demographics.
| Characteristics | n | % |
|---|---|---|
| Age groups | ||
| 60–79 | 1 | 5.3 |
| 80–89 | 12 | 63.2 |
| 90+ | 6 | 31.6 |
| Gender | ||
| Female | 12 | 63.2 |
| Education | ||
| Less than high school graduate | 1 | 5.3 |
| High school graduate/GED | 6 | 31.6 |
| Vocational training | 2 | 10.5 |
| Some college/associate's degree | 4 | 21.1 |
| Bachelor's | 4 | 21.1 |
| Master's or post-grad | 1 | 5.3 |
| Doctoral | 1 | 5.3 |
3.2. MMSE-related findings
Table 3 describes the population statistics stratified by the MMSE®. Fig. 1 shows the distribution of number of tasks completed successfully by each group of subjects. Subjects with MMSE® scores of 24+ (no cognitive impairment) completed 63% of the tasks, whereas subjects with cognitive impairment only completed 37% of the tasks. This difference was statistically significant (z = −2.03, p-value = 0.04).
Table 3.
MMSE® scores stratified by published cut points.
| MMSE® range | n | % of subjects | Mean | Range | SD |
|---|---|---|---|---|---|
| 0–23 | 7 | 36.8 | 18 | 13–23 | 3.3 |
| ≥24 | 12 | 63.2 | 27.5 | 24–30 | 2.2 |
Fig. 1.
Distributions of task success rates stratified by cognitive status.
As shown in Fig. 1, six individuals without cognitive impairment reached the 85% (6/7) task completion level. One impaired subject completed only one out of seven (1/7) tasks correctly. Two of the impaired subjects failed at all tasks (0/7).
Table 4 reports the correlation between items in the MMSE® and the percentage of task success.
Table 4.
Correlations between MMSE responses and task completion success rates.
| MMSE task | Correlation | p-value |
|---|---|---|
| Orientation–Date | 0.53 | 0.018* |
| Orientation–Location | 0.47 | 0.041* |
| Attention and Calculation – Spell ‘world’ backwards | 0.80 | <0.0001* |
| Recall – Repeat 3 objects | 0.40 | 0.089 |
| Language – Repeat short phrase | –0.12 | 0.63 |
| Language – Carry out a 3 step command | 0.11 | 0.67 |
| Language – Write a sentence | 0.40 | 0.087 |
| Executive Function/Motor Skills – Copy Design | 0.20 | 0.43 |
Statistically significant values are marked with asterisks.
MMSE® scores were not significantly correlated with education level (Pearson's r = 0.12, p = 0.54). There was also no relationship between task success rates and education (z = −1.17, p = 0.24).
3.3. Activities of daily living
Sixteen percent of subjects (3 subjects) had an IADL score of 4, whereas 36.8% (7 subjects) had an IADL score of 5. Six subjects scored 6, and the remaining 15.8% (3 subjects) scored 7. The relationship between a subject's IADL score and their percentage of task success is significant (z = −3.826, p < 0.0001).
There was no significant correlation between IADL and MMSE® scores (Mann–Whitney z = −0.47, p = 0.64).
3.4. SUS
Two participants (10.5%) had SUS scores ≤20, three subjects scored between 20 and 40 (15.8%), nine had a score between 40 and 60 (47.4%), two between 60 and 80 (10.5%), and the remaining three scored greater than 80 (15.8%). There was a significant correlation between SUS scores and task completion success rates (r = 0.467, p = 0.0429). SUS scores were not correlated with MMSE® scores (z = −0.029, p = 0.77).
Correlations between individual SUS questions and task success rates are given in Table 5.
Table 5.
Correlations between SUS questions and task success rates.
| Score | r-value | p-value |
|---|---|---|
| Would like to use system | –0.060 | 0.81 |
| Found system complex | –0.481 | 0.036* |
| Found system easy to use | –0.036 | 0.89 |
| Need tech support to use system | –0.60 | 0.0060* |
| Functions are well integrated | 0.15 | 0.547 |
| Too many inconsistencies | –0.56 | 0.012* |
| Many would learn quickly | –0.045 | 0.86 |
| System is cumbersome | –0.48 | 0.035* |
| Felt confident using system | 0.39 | 0.099 |
| Would need to learn a lot | –0.45 | 0.051 |
Statistically significant values are marked with asterisks.
3.5. Usability errors
Usability error percentages and task success rates, categorized by task and error type, are in Table 6. Each cell in the table displays both the count and frequency for the number of subjects that incurred each error.
Table 6.
Frequencies of usability errors and task success rates.
| Cognitive | Physical/perceptual | No Difficulties | Organizational | Feedback | Physical/perceptual | No Slips | Task successa | |
|---|---|---|---|---|---|---|---|---|
| Deliver | 14 (73.6%) | 0 (0%) | 5 (26.3%) | 1 (5.3%) | 2 (10.5%) | 0 (0%) | 16 (84.2%) | 5 (26.3%) |
| Help | 9 (47.4%) | 1 (5.3%) | 9 (47.4%) | 13 (68.4%) | 4 (21.1%) | 0 (0%) | 2 (10.5%) | 9 (47.4%) |
| Load | 13 (68.4%) | 3 (15.8%) | 3 (15.8%) | 3 (15.8%) | 2 (10.5%) | 12 (63.2%) | 2 (10.5%) | 3 (15.8%) |
| Manual drop | 16 (84.2%) | 1 (5.3%) | 2 (10.5%) | 9 (47.4%) | 4 (21.1%) | 0 (0%) | 6 (31.6%) | 2 (10.5%) |
| Remove pill | 7 (36.8%) | 1 (5.3%) | 11 (57.9%) | 0 (0%) | 0 (0%) | 10 (52.6%) | 9 (47.4%) | 11 (57.9%) |
| Unload | 10 (52.6%) | 1 (5.3%) | 8 (21.0%) | 6 (31.6%) | 3 (15.8%) | 1 (5.3%) | 9 (47.4%) | 8 (21.0%) |
| View inventory | 7 (36.8%) | 1 (5.3%) | 11 (57.9%) | 5 (42.1%) | 5 (47.4%) | 0 (0%) | 9 (47.4%) | 11 (57.9%) |
*The frequencies were calculated such that every task, subject, and error type was either present (1) or absent (0).
Task Success is the number of subjects that successfully completed each task. Percentage of task success among the 19 subjects is shown in parenthesis.
3.6. Inter-rater reliability (IRR) for video coding
Inter-rater reliability measures are given in Table 7.
Table 7.
Inter-rater reliability measures (Cohen's Kappa).
| Category | Type | Kappa |
|---|---|---|
| Difficulties | Cognitive | 0.50 |
| Physical/Perceptual | 0.51 | |
| No Difficulties | 0.60 | |
| Slips | Organizational | 0.81 |
| Feedback | 0.49 | |
| Physical/Perceptual | 0.80 | |
| No Slips | 0.65 |
4. Discussion
4.1. MMSE-related findings
We found a statistically significant relationship between MMSE® scores and the percentage of task success. Participants with lower scores on the MMSE had lower task success rates than participants with higher scores, suggesting that cognitive difficulties may play a role in influencing an individual's ability to successfully use the EMMA®. This is a specific instance of a more general observation that mental acuity and cognitive performance affect a patient's usability of an assistive device [26,27].
Three of the MMSE® subsections were found to have a statistically significant relationship with the percentage of task success: (1) Orientation–Date; (2) Orientation–Location; and (3) Spell ‘world’ backwards. For the first two, it is possible that non-cognitively impaired subjects took advantage of cues present in the environment, as the ability to covertly guide visual attention does not change with aging [28]. For the third, those who performed poorly when asked to spell ‘world’ backwards may have been hindered by deficits in attention and concentration [29]. In other words, top performers might have relied on top-down attentional control, which has been shown to be effectively used by older adults to compensate for changes in working memory [30]. The overall trend suggests that subjects better able to take advantage of attentional control, as opposed to those that may have sought to rely on visual working memory, performed better with EMMA®.
Although participants with higher MMSE® scores completed more tasks successfully, no subject was able to complete all tasks correctly. Particularly given the relatively simple nature of the most important tasks – pill delivery and extraction from the blister-pack – this result indicates the importance of combining simplified task and interface design with assessments aimed at understanding individual characteristics that might influence the likelihood of successful task completion [27,31].
There is a well-established relationship between amount of education and performance on the MMSE, with those with more education tending to have higher composite MMSE scores [22,32]. However, in our study we did not see an independent relationship between level of education and performance. This could a result of the small sample size and the non-normal distribution of education level (94.7% having at least a high school degree or equivalent). Further study will be necessary to fully understand the relationship between age, education, MMSE scores, and task performance with the EMMA®.
4.2. Tasks with the highest failure rates
A closer look at the two tasks that caused the most task failures gives insight into the range of issues that may underlie failures in all of the seven tasks. The tasks with the lowest success rates are “Manual Drop” (10.5% succeeded) and “Load blister card or new prescription pills” (15.8% succeeded). These tasks challenged the subjects in a variety of ways and were especially difficult for cognitively impaired subjects. For instance, the unfamiliar name of the “Manual Drop” task did not appear to stimulate recognition with any of the subjects. This unfamiliarity may be indicative of decreased cue availability, which has been shown to significantly impair performance in older adults [33]. Sometimes subjects would repeat the question as if to make sure they were on the right task, asking questions like “You want me to do a manual drop?” This question, combined with a confused tone, suggests difficulties in determining which actions to take next. The manual drop task also involved more steps than a scheduled medication, requiring more concentration, attention, and short-term memory.
Loading the blister cards into the EMMA® was another task in which subjects performed poorly. However, this result is not as concerning, as in a home application of the device trained staff would likely be dispatched every month to the patient's home to load the machine, thus avoiding any related difficul-ties for patients and families.
4.3. Subjective responses
The wide range of SUS scores and the weak correlation between SUS scores and task completion rates limit the conclusions that can be drawn regarding participants’ subjective responses to the EMMA®. The absence of any clear relationship between task completion success and subjective response may be an indication of initial enthusiasm that was not dampened by frustration in task completion, or, conversely, of skepticism un-swayed by successful task completion. Examination of individual questions revealed a potentially interesting pattern: questions that were associated with poor subjective responses (“found system complex”, “need tech support to use system”, too many inconsistencies”, and “system is cumbersome”) were significantly associated with task success. Further study, including observations and more detailed assessment of user responses, will be needed to better understand these effects.
4.4. Limitations
Our recruitment strategy was not randomized and may have been subject to bias. Participants were relatively homogeneous: 94.7% were White/Caucasian, and of above average educational and financial attainment, which is associated with higher MMSE® scores [22,32]. Study demographics may influence how MMSE® scores are interpreted and can even affect the meaning and placement of the cut points [32]. As these factors may limit the generalizability of our results, further study might be needed to investigate the efficacy of the EMMA® MDU for older adults with more varied ethnic, racial, and educational backgrounds.
Most categories yielded low to moderate kappa, which indicates some variation in coding choices between the two raters (FML and KMR). However, both organizational slips and P/P slips reported substantial agreement (kappa ≥ 0.80) [34].
5. Conclusion
MDUs like EMMA® represent a potential solution to improve medication management by combining reminder systems with telemedical monitoring and control capabilities. However, this technology presents significant usability challenges, particularly for the cognitively impaired. Subjects judged to be “cognitively impaired” (<24 MMSE®) scored a significantly smaller percentage of task success than the “unimpaired” (≥24). Cognitive screening should be considered prior to assigning EMMA® to a patient. This may prevent an assignment of EMMA® to a patient likely to fail using the device. Future studies are needed to test the use of EMMA® amongst ALF residents without cognitive impairment to see if this technology can improve medication adherence, and to address the other contextual challenges associated with the complexities of medication management for older adults [35].
Summary points.
Electronic medication delivery units (MDUs) may help older adults struggling with medication adherence, but barriers such as impaired cognitive function that might impede successful use.
Users with impaired cognitive function may have to have lower rates of successful task completion when using MDUs.
Cognitive abilities should be considered when exploring the use of these devices by older patients.
Acknowledgments
David A. Nace MD, MPH, CMD (Medical Director – Asbury Heights), Zachary A. Marcum, PharmD, MS, BCPS (Asst. Professor–Division of Geriatric Medicine, University of Pittsburgh), Janet Torregrosso LPN, PCHA (Personal Care Home Administrator – Asbury Villas), Linda Campbell ADPC (Activities Manager Coordinator – Asbury Villas), Ciayan Zhang MS and Zlatana Nenova BS (University of Pittsburgh PhD students).
This project was supported, in part, by a National Institutes of Health (NIH) grant 5KL2RR024151 (NIH Roadmap Multidisciplinary Clinical Research Career Development Award Grant), an Agency for Healthcare Research Quality (AHRQ) R01HS018721, a National Institute on Aging K07AG033174, and a NLM Fellowship 5 T15 LM007059-26. The authors also wish to thank INRange Systems for providing the EMMA® MDU used for the duration of this study, and the anonymous reviewers for their helpful comments. None of the authors have any financial relationship with INRange Systems.
Footnotes
Author's contributions
FML collected the data and contributed to data analysis. KMR assisted with video coding. CMT conducted regression analyses and oversaw other statistical analyses. SMH, HH, and CMT contributed to every phase except data collection. All authors had final manuscript review authority.
Conflicts of interest
There are no conflicts of interest to report. Study sponsors had no involvement with the study design, data collection/analysis, manuscript development, or the submission destination.
REFERENCES
- 1.Day JC. Population Projections of the United States by Age, Sex, Race, and Hispanic Origin: 1995–2050. Census USBot/U.S. Government Printing Office; Washington, DC: 1996. [Google Scholar]
- 2.Qato DM, Alexander GC, Conti RM, Johnson M, Schumm P, Lindau ST. Use of prescription and over-the-counter medications and dietary supplements among older adults in the United States. J. Am. Med. Assoc. 2008;300(24):2867–2878. doi: 10.1001/jama.2008.892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wilson IB, Schoen C, Neuman P, Strollo MK, Rogers WH, Chang H, et al. Physician–Patient communication about prescription medication nonadherence: a 50-state study of America's seniors. J. Gen. Intern. Med. 2007;22(1):6–12. doi: 10.1007/s11606-006-0093-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cline CMJ, Björck-Linné AK, Israelsson BYA, Willenheimer RB, Erhardt LR. Non-compliance and knowledge of prescribed medication in elderly patients with heart failure. Eur. J. Heart Fail. 1999;1(2):145–149. doi: 10.1016/s1388-9842(99)00014-8. [DOI] [PubMed] [Google Scholar]
- 5.Okuno J, Yanagi H, Tomura S, Oka M, Hara S, Hirano C, et al. Compliance and medication knowledge among elderly Japanese home-care recipients. Eur. J. Clin. Pharmacol. 1999;55(2):145–149. doi: 10.1007/s002280050609. [DOI] [PubMed] [Google Scholar]
- 6.Beers MH, Sliwkowski J, Brooks J. Compliance with medication orders among the elderly after hospital discharge. Hosp. Formul. 1992;27(7):720–724. [PubMed] [Google Scholar]
- 7.Center for Technology and Aging [1.07.14];Technologies for Improving Post-Acute Care Transitions. 2010 Available from: http://www.techandaging.org/PACTdraftPositionPaper.pdf.
- 8.Lakey SL, Gray SL, Borson S. Assessment of older adults’ knowledge of and preferences for medication management tools and support systems. Ann. Pharmacother. 2009;43(6):1011–1019. doi: 10.1345/aph.1L704. [DOI] [PubMed] [Google Scholar]
- 9.Lee ML, Dey AK. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; Vancouver, BC, Canada: 2011. Reflecting on pills and phone use: supporting awareness of functional abilities for older adults; pp. 2095–2104. [Google Scholar]
- 10.Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst. Rev. 2. 2008 doi: 10.1002/14651858.CD000011.pub3. [DOI] [PubMed] [Google Scholar]
- 11.Gellad WF, Grenard JL, Marcum ZA. A systematic review of barriers to medication adherence in the elderly: looking beyond cost and regimen complexity. Am. J. Geriatr. Pharmacother. 2011;9(1):11–23. doi: 10.1016/j.amjopharm.2011.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Marek KD, Stetzer F, Ryan PA, Bub LD, Adams SJ, Schlidt A, et al. Nurse care coordination and technology effects on health status of frail older adults via enhanced self-management of medication: randomized clinical trial to test efficacy. Nurs. Res. 2013;62(4):269–278. doi: 10.1097/NNR.0b013e318298aa55. http://dx.doi.org/10.1097/NNR.0b013e318298aa55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lee ML, Dey AK. Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM; Toronto, ON, Canada: 2014. Real-time feedback for improving medication taking; pp. 2259–2268. [Google Scholar]
- 14.Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179–186. [PubMed] [Google Scholar]
- 15.Lieto JM, Schmidt KS. Reduced ability to self-administer medication is associated with assisted living placement in a continuing care retirement community. J. Am. Med. Dir. Assoc. 2005;6(4):246–249. doi: 10.1016/j.jamda.2005.04.002. [DOI] [PubMed] [Google Scholar]
- 16.Hawes C, Phillips CD, Rose M. High Service or High Privacy Assisted Living Facilities, their Residents and Staff: Results from a national survey. Services DoHaH; Washington, DC: 1999–2003. [1.07.14]. A national study of assisted living for the frail elderly. http://aspe.hhs.gov/daltcp/reports/hshp.htm. [Google Scholar]
- 17.Ligons FM, Romagnoli KM, Browell S, Hochheiser HS, Handler SM. Assessing the usability of a telemedicine-based medication delivery unit for older adults through inspection methods. AMIA Annual Symposium proceedings/AMIA Symposium AMIA Symposium. 2011 [PMC free article] [PubMed] [Google Scholar]
- 18.Skibinski KA, White BA, Lin LIK, Dong Y, Wu W. Effects of technological interventions on the safety of a medication-use system. Am. J. Health Syst. Pharm. 2007;64(1):90–96. doi: 10.2146/ajhp060060. [DOI] [PubMed] [Google Scholar]
- 19.Reeder B, Demiris G, Marek KD. Older adults’ satisfaction with a medication dispensing device in home care. Inform. Health Soc. Care. 2013;38(3):211–222. doi: 10.3109/17538157.2012.741084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kushniruk AW, Patel VL, Cimino JJ. Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces. Proceedings: a conference of the American Medical Informatics Association/AMIA Annual Fall Symposium AMIA Fall Symposium. 1997:218–222. [PMC free article] [PubMed] [Google Scholar]
- 21.Brooke JSUS. A quick and dirty usability scale. Usability Eval. Ind. 1996;18:9–94. [Google Scholar]
- 22.Tombaugh TN, McIntyre NJ. The mini-mental state examination: a comprehensive review. J. Am. Geriatr. Soc. 1992;40(9):922–935. doi: 10.1111/j.1532-5415.1992.tb01992.x. [DOI] [PubMed] [Google Scholar]
- 23.Dick JPR, Guiloff RJ, Stewart A. Mini-mental state examination in neurological patients. J. Neurol. Neurosurg. Psychiatry. 1984;47(5):496–499. doi: 10.1136/jnnp.47.5.496. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Martin WE, Bridgmon KD. Quantitative and Statistical Research Methods: From Hypothesis to Results. Jossey-Bass; 2012. [Google Scholar]
- 25.Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam. Med. 2005;37(5):360–363. [PubMed] [Google Scholar]
- 26.Ziefle M, Bay S. How older adults meet complexity: aging effects on the usability of different mobile phones. Behav. Inform. Technol. 2005;24(5):375–389. [Google Scholar]
- 27.Kaufman DR, Patel VL, Hilliman C, Morin PC, Pevzner J, Weinstock RS, et al. Usability in the real world: assessing medical information technologies in patients’ homes. J. Biomed. Inform. 2003;36(1–2):45–60. doi: 10.1016/s1532-0464(03)00056-x. [DOI] [PubMed] [Google Scholar]
- 28.Greenwood PM, Parasuraman R, Haxby JV. Changes in visuospatial attention over the adult lifespan. Neuropsychologia. 1993;31(5):471–485. doi: 10.1016/0028-3932(93)90061-4. [DOI] [PubMed] [Google Scholar]
- 29.Ganguli M, Ratcliff G, Huff FJ, Belle S, Kancel MJ, Fischer L, et al. Serial sevens versus world backwards: a comparison of the two measures of attention from the MMSE. J. Geriatr. Psychiatry Neurol. 1990;3(4):203–207. doi: 10.1177/089198879000300405. [DOI] [PubMed] [Google Scholar]
- 30.McAvinue LP, Habekost T, Johnson KA, Kyllingsbaek S, Vangkilde S, Bundesen C, et al. Sustained attention, attentional selectivity, and attentional capacity across the lifespan. Atten. Percept. Psychophys. 2012;74(8):1570–1582. doi: 10.3758/s13414-012-0352-6. [DOI] [PubMed] [Google Scholar]
- 31.Shneiderman B, Hochheiser H. Universal usability as a stimulus to advanced interface design. Behav. Inform. Technol. 2001;20(5):367–376. [Google Scholar]
- 32.Murden RA, McRae TD, Kaner S, Bucknam ME. Mini-mental state exam scores vary with education in black and whites. J. Am. Geriatr. Soc. 1991;39(2):149–155. doi: 10.1111/j.1532-5415.1991.tb01617.x. [DOI] [PubMed] [Google Scholar]
- 33.Paleja M, Spaniol J. Spatial pattern completion deficits in older adults. Front. Aging Neurosci. 2013:2013. doi: 10.3389/fnagi.2013.00003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 35.Dalgaard LG, Grönvall E, Verdezoto N. Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering); Venice, Italy: 2013. Accounting for medication particularities: designing for everyday medication management; pp. 137–144. [Google Scholar]