Abstract
Background/objectives:
The ability to manage medications independently may be affected in older adults due to physical and cognitive limitations. Numerous electronic medication adherence products (eMAPs) are available to aid medication management. Unfortunately, there are no available guidelines to support clinicians in recommending eMAPs. The objective of this study was to create and validate a clinician tool to guide use of eMAPs.
Methods:
Pharmacists who previously tested the usability of the eMAPs participated in a focus group to provide feedback on 5 metrics of the clinician guide: unassisted task completion, efficiency, usability, workload and an overall eMAP score. Participants were asked semistructured questions on how they would use the tool to inform recommendations of medication aids to patients. The discussions were audio-recorded and transcribed verbatim and qualitatively analyzed. The clinician guide was modified to reflect feedback.
Results:
Five pharmacists (80% female, mean years of practice: 15.8) participated in the focus group. The clinician guide was modified by removing 2 metrics and adding an additional 8 metrics: maximum number of alarms, number of days the product can accommodate for based on a daily dosing regimen, price, monthly subscription, portability, locking feature, average time to set the device and number of steps required to set the device. The definition and calculation for unassisted task completion were modified. Additional instructions and specific patient case examples were also included in the final clinician guide.
Conclusion:
Since significant variability exists between eMAPs, it is imperative to have a tool for frontline clinicians to use when appropriately recommending the use of these products for medication management in older adults.
Introduction
For patients to take their medications, several steps have to be undertaken in a specific order. The patient has to identify the correct medication to take, open the appropriate container or packaging, know how many tablets or capsules to retrieve, be able to retrieve the correct number of pills and know the process by which to self-administer the medications. 1 This process, the ability to “self-administer a medication regimen as it has been prescribed,” is labeled medication management capacity and requires both physical and cognitive capabilities. 2
Knowledge into Practice.
The ability to appropriately manage medications is important for independent living among older adults.
Numerous electronic medication storage and dispensing devices are available for older adults to purchase, but there are no guides to assist in choosing a particular dispensing device.
The clinician guide for the use of electronic medication adherence products can enable clinicians to assist patients in choosing a product based on the needs of the patient and the features of the product.
MISE EN PRATIQUE DES CONNAISSANCES.
La capacité à gérer correctement les médicaments est importante pour le maintien d’une vie indépendante chez les personnes âgées.
De nombreux appareils électroniques de distribution et de stockage de médicaments sont offerts sur le marché pour les personnes âgées, mais il n’existe aucun guide pour les aider à choisir un appareil de distribution en particulier.
Le guide du clinicien sur l’utilisation des appareils électroniques d’observance thérapeutique peut permettre aux cliniciens d’aider les patients à choisir un appareil en fonction des besoins du patient et des caractéristiques du produit.
Older adults’ capacity to manage medications has been shown to be affected in several studies. In 1 cross-sectional survey of 317 patients aged 65 years and older, 28.4% experienced challenges with opening their medication packaging; of these older adults, the relative risk of facing 1 or more challenges with opening peel-off blisters was 3.7 times (95% confidence interval [CI], 2.5-5.5) and with push-through blisters was 1.9 times (95% CI, 1.2-2.8) that of opening a bottle. 3 Aside from the physical strength and dexterity required to open packaging, vision impairment can significantly increase the risk of exceeding the maximum dose, improper dosing and improper dose spacing, as reported by 1 study investigating the impact of visual acuity on the use of nonprescription single and multi-ingredient acetaminophen products. 4 Cognitive impairment has also been correlated with declining medication management capacity in several studies.2,5-7 In addition to these patient-related factors, medication-related factors such as formulation of the medication, polypharmacy and complexity of medication regimens also affect medication management capacity. 8
Older adults, the majority of whom continue to self-manage their medications,9,10 try to address these challenges with medication management by devising a multitude of strategies to improve medication-taking. Patients may use personal systems and routines to remember to take their medications. For example, they may store their medications in specific locations (e.g., the kitchen counter) or take medications at times correlated with certain activities (e.g., while watching television).11,12 Many older adults also use reminder systems such as calendars or pill boxes to organize their medications.10,13
A variety of medication organization systems are available on the market. A previous systematic search for electronic products promoted to improve medication-taking among seniors revealed 80 such products available for purchase by the Canadian senior. 14 These electronic medication adherence products (eMAPs) have a variety of features such as integrated alarms and cloud-based dispensers with real-time monitoring. However, an investigation into the usability of 22 of these products demonstrated significant variability in the usability of and workload required to use these products. 13 In addition, this study demonstrated that participants were only able to complete all the steps required to set up a product for use in 103 of 186 tests completed (55.3%; range, 0%-100%). 13
Visual examination of these products as well as the variability in usability, workload and the number of steps required to set up the products revealed several differentiating features. These included different number and size of compartments for medication storage and organization, as well as different number and type of alarms and cost. These differentiating features could be used to guide clinicians in assisting older adult patients and their caregivers in adopting the appropriate product to address medication-taking. Therefore, the objective of this study was to develop and validate a guide for clinicians to use in clinical practice when determining which electronic medication adherence product may be suitable for an older adult by comparing the different features of the 22 eMAPs tested.
Methods
Study design
This study was part of a larger mixed-methods research project testing the usability and workload of 22 eMAPs (Table 1). For the validation of the clinician guide, we used qualitative research methods using focus groups.
Table 1.
Electronic medication adherence products (eMAPs) tested
NA, not applicable; NR, not reported.
Ethical review
This study received approval from the University of Waterloo Office of Research Ethics. All participants were informed of the study and provided consent prior to enrolling.
Development of clinician guide
The initial version of the clinician guide was developed by examining the results from the larger study, which tested the usability and workload required to use eMAPs. Twenty-two products were tested by a sample of older adults (n = 23), caregivers (n = 5) and health care professionals (n = 11) for usability, workload, time taken to set up and use the product and problems encountered while using the product. Each participant was asked to set up and use 5 products. Participants were provided with manufacturer instructions and a series of tasks that researchers requested they complete to use the product. 13
We used the System Usability Scale (SUS)15,16 and the NASA–Task Load Index (NASA-TLX) questionnaire17,18 to determine and interpret usability and workload, respectively, and cognitive walkthrough 19 to examine the problems encountered and time taken for setting up and using the product. 13 We chose to report the following features in the clinician guide: mean unassisted task completion, mean efficiency, mean usability, mean workload and an overall eMAP score.
Unassisted task completion
Unassisted task completion was defined as the number of steps a participant completed without assistance from a research team member. 20 When assistance was required, it included providing hints, explaining a step or physically assisting the participant to set up the product. Unassisted task completion was calculated by the following formula: 20
2. Efficiency
Efficiency was defined as the time spent on task completion. 20 It is measured by dividing the proportion of steps completed successfully, whether assisted or unassisted, by each participant divided by the total time spent by the participant on all steps: 20
3. Usability
Usability was determined by the SUS score of each product. SUS consists of 10 statements that assess an individual’s level of agreement with concepts such as complexity of the system, technical ability to use the system, integration of the functions and learnability of the system, among others. SUS scores range from 0 to 100, where higher scores indicate higher usability. 15 Both SUS scores and a colour coding scale were used to display usability of each product in the clinician guide. 15 SUS scores of 70.01 to 100.00, defined as “acceptable usability,” were colour coded green; SUS scores of 50.01 to 70.00 were defined as “marginally acceptable” and displayed in yellow; and SUS scores of <50.00 were defined as “not acceptable” and represented in red.
4. Workload
Workload was determined by the NASA-TLX score, where scores range from 0 to 100, with higher scores indicating more work involved in using the product. Similar to SUS, both the scores and a colour coding scale were used to display NASA-TXL score ranges in the clinician guide. 18 The colour green indicated NASA-TLX scores that were >75th quartile, yellow indicated between 50th and 75th quartile and red indicated <50th quartile.
5. eMAP score
Based on these 4 metrics, we determined an overall eMAP score. Each metric for each eMAP was given a score between 1 and 3, where 3 indicated that the eMAP for the particular metric fell within the highest quartile, or was coloured green. A score of 2 indicated that the eMAP fell within the 50th to 75th quartile or was coloured yellow, and a score of 1 indicated that it fell below the 50th quartile or was coloured red. The 4 metrics were then added up for each eMAP and divided by 1.2 to obtain a total score out of 10. Overall scores for the eMAPs ranged from 3.33 to 10. A corresponding image was used to showcase eMAP scores (see Table 2).
Table 2.
Electronic medication adherence product grading scale
| Score range | Corresponding image |
|---|---|
| ≥8.5 |
|
| <8.5 and ≥7 |
|
| <7 and ≥5.5 |
|
| <5.5 and ≥4 |
|
| <4 |
|
Focus group sample
Once the initial version of the tool was finalized with the 4 metrics identified above for each of the 22 eMAPs tested, the 11 health care professionals who completed the usability and workload testing for eMAPs in the larger study were invited to participate in a focus group.
Participants were provided with an explanation for the development of the tool, the definitions and formulas used to calculate the different metrics and how the overall score was determined. Participants were asked semistructured and probing questions regarding the use of the different metrics in the clinician guide, whether the tool accurately represented their experiences with the testing of the tool, whether the tool could be useful in clinical practice and whether additional factors could be reported (Appendix 1, available online at www.cpjournal.ca).
Data analysis
The focus group session was audio-recorded using a Sony Digital Voice Recorder ICD-PX470. Three research team members (AM, CL and JI) took detailed notes during the focus group meeting. The audio recording was then transcribed verbatim by one researcher (JI) into a Microsoft Word document (Microsoft Office 365 ProPlus Version 1907), and notes from the focus group were used to supplement the recording. The transcript was then independently verified by another member (AM). The transcripts were thematically coded to identify themes and subthemes. Transcripts were coded independently by 2 researchers (AM and JI). Any disagreement was resolved by discussion. The results of this analysis were used to develop a final version of the tool.
Results
Of the initial 11 health care professionals who participated in the larger study, 5 pharmacists agreed to participate in this study. The focus group discussion consisted of 5 pharmacist participants and 6 research team members (Table 3).
Table 3.
Participant demographics
| Variable | Pharmacists (N = 5) |
|---|---|
| Gender (n, %) | |
| Male | 1 (20) |
| Female | 4 (80) |
| Years of practice | |
| Mean ± SD | 15.8 ± 12.7 |
| Mode | 15 |
| Median | 15 |
| Range | 5-37 |
| Older adults worked with/dispensed prescriptions for (n, %) | |
| <10 | 0 (0) |
| 10-20 | 1 (20) |
| 20-30 | 1 (20) |
| >30 | 3 (60) |
| Assist older adults with medication-taking | |
| Yes | 5 (100) |
| Medication aids recommended to patients (n, %) | |
| Yes | 5 (100) |
| Blister pack | 5 (100) |
| Pill box/dosette | 1 (20) |
Qualitative analysis
Thematic coding of the transcript generated 5 themes, including 1) interpretation, 2) appropriateness of the metrics, 3) independent scoring, 4) multiple product end-users and 5) further research (see Figure 1 for participant quotes associated with themes and subthemes).
Figure 1.
Participant quotes associated with themes and subthemes
Interpretation
Participants indicated difficulty with understanding the overall eMAP score, usability, workload, unassisted task completion and efficiency, as they were unfamiliar with these metrics.
They also indicated that users may require additional training to use the clinician guide and to revise the guide to allow those without any background knowledge to use it (Figure 1).
Appropriateness of the metrics
Several subthemes were identified within this theme. Participants discussed the appropriateness of the different metrics as factors by which to recommend products. For example, they felt comparing eMAPs based on the 5 metrics presented without a comparison of the number of compartments was not helpful in determining which product was appropriate for a patient with a complex medication regimen. Participants also indicated that all 4 metrics should be not be weighted equally for every patient and found interpretation of the weighted scoring problematic. Participants also discussed the appropriateness of units representing the different metrics, cutoff values and colour coding and recommended other factors such as cost and portability to be included in the clinician guide.
Independent scoring
Participants noted that as clinicians, they would like to determine which product is best for their patient based on the metrics, instead of an overall score (Figure 1).
Separation of metrics by product end-users
Participants also reported that the guide should be used to help clinicians and caregivers determine which product may be usable by the end-user filling the product with medication as well as the person retrieving the medication for administration (Figure 1). In the study, usability was determined for both filling and using the product as one metric, but participants in this study reported that 2 separate usability metrics—one for filling the adherence product and one for retrieving the medication for administration—should have been determined.
Further research
Finally, participants noted further research needs to be conducted to measure other aspects of using eMAPs that may guide selection of the product, such as product learnability.
General takeaways
Based on the discussions with the pharmacist participants and the qualitive analysis of the focus group, we determined participants wanted a tool that they, as clinicians, can use to aid them in independently determining which product is best for their patient, based on patient need and wants. Participants also wanted more information about the products, for example, whether the product is portable, the cost of the product, the number of compartments and number of days of supply each product can accommodate. Participants also wanted visual consistency in the formatting of the guide, for example, all values in percentages, all values colour coded and all values to the same number of decimal points. Participants did not appreciate the efficiency metric as well as the overall score, as these were not readily understood and could be easily misinterpreted.
Final clinician guide
The initial version of the clinician guide was modified with the feedback and suggestions received, and the final version is available as Figure 2. Of the original 5 metrics included in the initial version, the metrics of efficiency and the overall eMAP score were removed, and both SUS and NASA-TLX were modified by removing the scores. The colour coding used to represent degree of usability and workload was retained. Eight additional metrics were added: maximum number of alarms, number of days the product can accommodate for based on a daily dosing regimen, price, monthly subscription, portability, locking feature, average time and number of steps required to set the device. The metric unassisted task completion was recalculated to represent the proportion of individuals who were able to set, fill and remove medications from the device without any help from researchers. Additional instructions and patient case examples were included to guide the appropriate use of the tool (see Appendix 2, available online at www.cpjournal.ca).
Figure 2.
The final clinician guide
Discussion
Our clinician guide is the first tool of its kind. There are no other guides that enable clinicians to recommend or assist patients with purchasing the most appropriate electronic medication adherence aid in medication-taking. By using this guide, clinicians, including pharmacists, physicians, nurses and occupational therapists, can compare the number of days a product can accommodate based on the frequency of dosing, the number of alarms that can be set as reminders, the cost of the product, whether a subscription fee is required for real-time electronic monitoring, a device’s portability and if the device can be locked, as well as average usability and average workload required to use the product. These features can be used by clinicians to recommend products based on the needs of the patient and by gauging whether the usability and workload required will affect independent use.
The features chosen to be represented in the clinician guide were also determined to be of importance in a qualitative analysis of interviews conducted with participants who tested the usability of the products. 21 They indicated that the initial impression of simplicity or complexity in the setup affected the overall impression of the product. Participants found products that required repetitive reviews of instructions for setup unfavourable. Other product features that were discussed by participants were the availability of alarms, portability, storage capacity and affordability.
Many, if not all, community pharmacies offer the service of dispensing medications in blister packaging, also commonly known as compliance packaging. Blister packaging is useful for those patients on complex medication regimens, as the organization of the medications is completed by the pharmacy. This type of packaging addresses the errors a patient may make when filling their own multidose dispensing aid with medications from multiple prescription vials, each with its own set of instructions. 22 It may improve efficiency in taking medications. 22 Furthermore, it permits caregivers and health care providers to visually examine the packaging to determine whether doses are being taken on the days and times dispensed. However, older adults may not be able to peel the tabs or push medications through the blister packaging. 3 In addition, while blister packaging reduces cognitive workload related to organizing medications, it does not address forgetfulness. It may be useful for patients with declining executive functioning 23 but not declining memory. 24 Older adults living within a fixed income may not be able to afford this service in every province. 25 Patients who are unaware of the service or those who cannot afford the dispensing fees charged with each blister pack fill may opt to buy their own multidose dispensing aids, whether they have electronic features or not. Therefore, it is imperative that products be examined for appropriateness as they are not equally usable. Older adults and caregivers may struggle with setting up the product for use at home. 13
While our guide compares 22 of the products tested, it is not comprehensive. Numerous other products are available for purchase. 14 Many, if not all, of these products are marketed without an assessment of effectiveness in improving medication-taking or adherence. 26 Furthermore, features of the products vary widely and can affect the usability of the products. Patients with impaired dexterity, impaired cognition and vision problems may find them difficult to use. 24 Indeed, physical and cognitive capability was noted as crucial for the appropriate use of the products. 21 Many products required good vision, hearing and dexterity to press small buttons, flip switches, rotate devices and respond to alarms, among others. Similarly, technology literacy and learnability may also drive the appropriate use of the product.
Limitations
While our guide offers some features to consider when determining which product is most suitable, more research is required to establish the effectiveness of the products in improving and sustaining adherence to medications. Furthermore, the guide is limited in its applicability as we tested our products with older adults who did not report any physical or cognitive challenges. We also did not test learnability. These user features have to be considered when recommending medication adherence devices, and future studies should investigate the usability of these products in patient populations with physical and cognitive limitations.
Another limitation of this study was the small sample size of 5 pharmacists. While we recruited 11 health care professionals to participate in the larger study, only 5 pharmacists agreed to participate in the focus group. The participation of the 5 pharmacists increases the robustness of the guide compared to one where the guide was developed solely by a research team; however, a larger sample size and additional interprofessional participants would have increased the robustness even further and enabled other disciplines, such as occupational therapists, to use the tool.
Conclusion
Numerous electronic products are marketed to address medication-taking. However, there is significant variability in the features, cost, usability and workload among these products. This variability necessitates a comparison of features of the products so that an end-user, caregiver or clinician choose the right product for their use. Our Clinician Guide to Recommending Electronic Medication Adherence Products (eMAP Clinician Guide) provides a comparison of the features of 22 such products to guide clinicians in recommending the appropriate product based on the specific needs of the patient. ■
Acknowledgments
The authors thank the pharmacist participants who provided invaluable insight and advice in the development of this guide.
Footnotes
Author Contributions: Project initiation and supervision: Tejal Patel; Design and methodology: Tejal Patel; Data collection: Aidan McDougall, Jessica Ivo, Feng Chang, Jillian Bauer; Data analysis: Aidan McDougall, Jessica Ivo, Feng Chang, Jillian Bauer, Sarah Pritchard; Writing first draft: Tejal Patel; Review: all members.
Declaration of Conflicting Interests: The authors do not have any conflicts of interest to declare.
Funding: This research was funded by the Centre for Aging and Brain Health Innovation SPARK Grant. The sponsors had no role in study design, conduct, analysis/interpretation of the data or decision to publish and did not have access to the study data.
ORCID iD: Tejal Patel 
https://orcid.org/0000-0003-3002-8306
Contributor Information
Tejal Patel, University of Waterloo School of Pharmacy.
Jessica Ivo, University of Waterloo School of Pharmacy.
Aidan McDougall, Centre for Family Medicine Family Health Team, Kitchener.
Catherine Lee, Centre for Family Medicine Family Health Team, Kitchener.
Feng Chang, University of Waterloo School of Pharmacy; Schlegel–University of Waterloo Research Institute of Aging, Waterloo, Ontario.
Jillian Bauer, Centre for Family Medicine Family Health Team, Kitchener.
Sarah Pritchard, Centre for Family Medicine Family Health Team, Kitchener.
References
- 1. Notenboom K, Beers E, van Riet-Nales DA, et al. Practical problems with medication use that older people experience: a qualitative study. J Am Geriatr Soc 2014;62:2339-44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Maddigan SL, Farris KB, Keating N, Wiens CA, Johnson JA. Predictors of older adults’ capacity for medication management in a self-medication program. J Aging Health 2003;15:332-52. [DOI] [PubMed] [Google Scholar]
- 3. Philbert D, Notenboom K, Bouvy M, et al. Problems experienced by older people when opening medicine packaging. Int J Pharm Prac 2014;22:200-4. [DOI] [PubMed] [Google Scholar]
- 4. Mullen RJ, Curtis LM, O’Conor R, et al. Visual acuity, literacy and unintentional misuse of nonprescription medications. Am J Health-Syst Pharm 2018;75:e213-20. [DOI] [PubMed] [Google Scholar]
- 5. Sino CGM, Sietzema M, Egberts TCG, Schuurmans MJ. Medication management capacity in relation to cognition and self-management skills in older people on polypharmacy. J Nutr Health Aging 2014;18:44-9. [DOI] [PubMed] [Google Scholar]
- 6. Edelberg HK, Shallenberger E, Wei JY. Medication management capacity in highly functioning community-living older adults: detection of early deficits.J Amer Geriatr Soc 1999;47:592-6. [DOI] [PubMed] [Google Scholar]
- 7. Sumida CA, Vo TT, Van Etten EJ, Schmitter-Edgecombe M. Medication management performance and associated cognitive correlates in healthy older adults and older adults with MCI. Arch Clin Neuropsych 2019;34:290-300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Yap AF, Thirumoorthy T, Kwan YH. Medication adherence in the elderly. J Clin Gerontol Geriatr 2016;16:1093-101. [DOI] [PubMed] [Google Scholar]
- 9. Hutchinson LC, Jones SK, West DS, et al. Assessment of medication management by community-living elderly persons with two standardized assessment tools: a cross-sectional study. Am J Geriatr Pharmacother 2006;4:144-53. [DOI] [PubMed] [Google Scholar]
- 10. Raehl CL, Bond CA, Woods T, Patry RA, Sleeper RB. Individualized drug use assessment in the elderly. Pharmacotherapy 2002;22:1239-48. [DOI] [PubMed] [Google Scholar]
- 11. Palen L, Aaløkke S. Of pill boxes and piano benches: “home-made” methods for managing medications. In: Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. 2006:79-88. Available: https://dl.acm.org/doi/10.1145/1180875.1180888 [Google Scholar]
- 12. O’Quin KE, Semalulu T, Orom H. Elder and caregiver solutions to improve medication adherence. Health Educ Res 2015;30:323-35. [DOI] [PubMed] [Google Scholar]
- 13. Patel T, Ivo J, Faisal S, et al. A prospective study of usability and workload of electronic medication adherence products by older adults, caregivers and health care providers. J Med Intern Res 2020;22(6):e18073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Farooqi M, Carter C, Patel T. Electronic medication adherence technologies: classification to guide use in older adults. Can Pharm J (Ott) 2017;150:S26. [Google Scholar]
- 15. Brook J. SUS—a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability evaluation in industry. London (UK): CRC Press; 1996. [Google Scholar]
- 16. Bangor A, Staff T, Kortum P, Miller J, Staff T. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009;4(3):114-23. [Google Scholar]
- 17. Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical approach. Adv Psychol 1988;52:139-83. [Google Scholar]
- 18. Grier RA. How high is high? A meta-analysis of NASA-TLX global workload scores. Proc Hum Factors Ergon Soc 2015;59(1):1727-31. [Google Scholar]
- 19. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004;37(1):56-76. [DOI] [PubMed] [Google Scholar]
- 20. Albert W, Tullis T. Measuring the user experience: collecting, analyzing and presenting usability metrics. 2nd ed. Waltham (MA): Elsevier; 2013. [Google Scholar]
- 21. Faisal S, Ivo J, McDougall A, et al. Stakeholder feedback of electronic medication adherence products: qualitative analysis. J Med Intern Res 2020;22 (12):e18074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Lecouturier J, Cunningham B, Campbell D, et al. Medication compliance aids: a qualitative study of users’ views. Br J General Prac 2011;61:93-100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Etty-Leal MG. The role of dose administration aids in medication management for older people. J Pharm Prac Res 2017;47:241-7. [Google Scholar]
- 24. Elliot RA. Appropriate use of dose administration aids. Aust Prescriber 2014;37:46-50. [Google Scholar]
- 25. Parminder MS. Compliance packaging: be aware of the hidden cost. Can Med Assoc J 2019;191:E1313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Faisal S, Ivo J, Lee C, Carter C, Patel T. The usability, acceptability and functionality of smart oral multidose dispensing systems for medication adherence: a scoping review. J Pharm Pract. Epub 2020. Dec 17. [DOI] [PubMed] [Google Scholar]