Abstract
Context
Evaluation of usability and mobile health content is critical for ensuring effective implementation of technology utilizing interventions tailored to the needs of hospice care providers for people living with dementia in community-based settings.
Objectives
To evaluate the usability, content, and “readiness to launch” of the Aliviado mobile health app for interdisciplinary team members participating in the Hospice Advanced Dementia Symptom Management and Quality of Life.
Methods
Usability of the Aliviado app was assessed in 86 respondents with an adapted IBM Computer Usability Satisfaction Questionnaire following Hospice Advanced Dementia Symptom Management and Quality of Life training and implementation of the mobile app.
Results
More than half of users receiving training employed the mobile app in practice. Users reported use as: Daily-6.3%, Weekly-39.6%, monthly-54.2%. The highest measured attributes were usefulness, value, and effectiveness. Over 90% deemed the app “ready to launch” with no or minimal problems.
Conclusion
This study shows that a newly-developed mobile app is usable and can be successfully adopted for care of people living with dementia.
Keywords: telemedicine, mobile applications, hospice, dementia, usability testing
Introduction
Hospice providers deliver care to 1.55 million patients and family members each year of whom just less than half have dementia as a primary or secondary diagnosis.1 Mobile health apps for hospice interdisciplinary team (IDT) members are promising, supportive resources that may enhance delivery of hospice care. This is especially true for persons living with dementia (PLWD) and their caregivers as these individuals tend to have longer average lengths of stay, greater complexity, and require different assessments and treatments than many other serious illnesses due to the substantial cognitive impairment present. This type of care can thus be especially challenging.2 Inadequate hospice care for this population can lead to caregiver burden, unchecked dementia behaviors, and poor symptom control including agitation, pain management, and depression and anxiety3, which in turn can lead to greater risk for hospitalization and live discharge.4–6 The hospice IDT can work with families and PLWD to identify resources that address these behaviors. However, there is a lack of readily accessible, evidence-based information, assessment tools, and care plans that target end of life care for PLWD.7 Additionally, for hospice providers, there are structural challenges that impede dementia-focused hospice care. These include cumbersome and inflexible electronic health records (EHRs), difficulties documenting real-time care in the home and long-term care sites, and communication challenges among providers across different disciplines and organizations.8,9
The Aliviado Dementia Care Hospice-Edition program is a quality assurance performance improvement effort within a 25-site embedded pragmatic clinical trial for hospice IDT training (Appendix 1). We created a mobile health app (Figure 1) to support IDT hospice providers receiving Aliviado training and measured its usability among a convenience sample of users. As part of the trial, providers were trained to integrate evidence-based assessments and care plans into hospice delivery to PLWD and their caregivers with unmet needs and challenging behaviors.10,11 The Aliviado Dementia Care mobile health app supports care delivery through providing clinicians easy access at the patient’s residence to a toolbox of assessment instruments, care plans, and caregiver educational materials with the option to document these embedded tools within the EHR.
Figure 1. Aliviado Screen Shots.
Screen shots demonstrating features and functionality of the app. Features include search function, the ability to select from 4 different informational tabs featuring content and tools to support care (assessment, care plans, caregiving support, and news and blogs), and the functionality of being able to select the most useful features into a “favorites” section.
Usability of health technologies is a critical factor to evaluate implementation of apps targeting the care of older adults.12,13 While there are numerous individual instruments to measure usability, many are author-developed surveys for specific use cases.14 To address this challenge, we have adapted the widely used and validated IBM Computer Usability Satisfaction Questionnaire.15 The purpose of this study was to a) evaluate the usability of the Aliviado app and b) determine what components are needed to support care delivery using the Aliviado app.
Methods
Study Design.
This is a cross-sectional sub-study of home hospice IDT members participating in the Hospice Advanced Dementia Symptom Management and Quality of Life (HAS-QOL) Trial (NCT04175977).10,11 Ethical approval of the HAS-QOL trial has been approved by the Institution Review Board at the New York University Grossman School of Medicine.
Briefly, the HAS-QOL trial is an embedded pragmatic stepped-wedge clinical trial in 25 hospices that evaluates the effectiveness of Aliviado Dementia Care-Hospice Edition, a dementia-focused quality improvement program for hospice IDT members.10,16,17 This sub-study focuses on the first 5 hospice agencies enrolled and trained between January and March 2020. These hospices were located in California, Florida, Maryland, Nevada, and North Carolina and had an average daily census ranging from 60 to 2100. In addition to training, participants were given access to the Aliviado app. The app reinforces evidence-based dementia care training as well as evidence-based assessment instruments, symptom management algorithms, care plans, and caregiver education. The app had been previously conceptualized through co-design with pilot hospices who shared their needs for such an app and functionality they wished to see. The app was designed using Google’s Material Design System.18
In this sub-study we evaluated the usability of the Aliviado mobile health app through a Qualtrics survey between June and July 2020. Upon completion, respondents received $50. Inclusion criteria were English proficiency and completion of the Aliviado program discipline-specific training course for hospice. Participants were home health aides and skilled IDT members (nurses, social workers, chaplains, physicians, nurse practitioners, nursing assistants, occupational or physical therapists). Once participants demonstrated that they successfully downloaded the Aliviado app to a personal device they were permitted to make their own decision with respect to utilization of the mobile app during care of hospice patients. Weekly push notifications were sent to remind IDT members to complete the dementia care training and tips on using Aliviado tools. Within the training, IDT members were provided with instruction and were prompted to complete tasks when a PLWD was admitted to hospice and then monthly.
Aliviado mobile app.
The Aliviado app is available for iPhone and Android platforms (Figure 1). It was created for use in home health (additional existing trial NCT03255967)17 and hospice for home health aide and skilled IDT members. App features for home health aide users included informational resources (Aliviado Dementia Care training, caregiver education articles, Aliviado news/blog articles) which could be read in the app or emailed. App features for skilled IDT members also included evidence-based assessment instruments and Aliviado care plans. Findings from our pilot phase suggested that EHR standard resources were difficult to modify and use in the field. Thus having them accessible through a frequently updated mobile health app was preferrable.10 Additionally, given the pragmatic nature of the intervention, guidelines from the Aliviado program recommended usage of certain tools on a per-visit, monthly or as needed basis, though whether on paper, the EHR (if feasible) or within the app was purposely left flexible for the hospice.
Theoretical framework.
Evaluation of usability was grounded in the Morville User Experience honeycomb framework.19 The framework describes core components that determine if new technology will be effective and adopted into a workflow. According to this framework, interventions will most likely be adopted if they are useful, desirable, valuable, accessible, credible, findable and usable. We sought to apply these principles to develop an evaluation survey for the Aliviado app.
Usability Survey Development.
To assess app usability, a research survey workgroup convened, selected an established technology evaluation survey, the IBM Computer Usability Satisfaction Questionnaire and adapted it to assess perceived usability by hospice IDT members.15 While a variety of technology assessment instruments were considered, we selected this instrument for the following reasons: a) usability focus; b) strong psychometrics; and c) historical breadth of use in the extant literature potentially allowing for benchmarking of our application against similar interventions. The workgroup consisted of 10 research team members with knowledge of home health and hospice care, care for PLWD, and/or mobile health development. The group met weekly to design, test, and refine the questionnaire prior to distribution to the IDT participants. The process yielded a survey that evaluated the usability of the Aliviado app and identified what components needed refinement.
Usability Survey.
The survey consisted of 5 sections (Appendix 2):
Frequency of use
App usability
App content & interface
Overall readiness for use
Barriers to use – open response
Analysis
Quantitative.
For the purpose of analysis, respondents were sorted into three groups based on frequency of use of the app: Never, Training Only, and Users (daily, a few times a week, weekly, at least once a month). Differences between respondent characteristics were evaluated by chi-square for nominal variables and t-test for continuous variables. Group means and 95% confidence intervals were reported for usability scores. Responses were compared between Training Only and Users groups. Statistical differences between groups were evaluated with ordinal logistic regression.
Qualitative.
In order to identify reasons associated with non-use, open responses for Barriers to use were reviewed. A coding scheme was developed according to Braun & Clarke’s method for thematic content analysis.20 This method of thematic analysis involves 6 steps: familiarizing, coding, generating themes, reviewing themes, defining and naming themes and formalizing the themes in a written document. Accordingly, 2 PhD-trained nurse researchers with training in palliative care and qualitative methods individually read responses to the question: “What features were missing (i.e., features that you would like to see)?” These responses were individually coded for content, reviewed, and discussed by the two researchers who then inductively categorized themes and summarized them across cases to allow for in-depth descriptive analysis.
Results
Characteristics.
We contacted 671 IDT staff across five hospices that participated in the HAS-QOL trial of whom 553 had logged into the app at least once. Of the total 553 participants, 101 (18.3%) completed the usability survey. Group percentages of participants’ self-reported racial/ethnicity were White (74.0%), Black (18.8%), Hispanic (6.3%), and Pacific Islander (1.0%). Overall, 44.8% were nurses, 29.2% home health aides, 17.7% were social workers, 7.3% were chaplains, and 1.0 % were physicians, in line with overall hospice IDT populations. Participants reported 6.1, 8.4, and 16.8 years of practice in their hospice organization, hospice care, and profession, respectively. Participants reported their highest educational degree as: graduate (19.8%), bachelor’s (26.0%), associate’s (28.1%), and no degree (25.0%).
After excluding 15 respondents who self-reported never downloading or using the app, 86 were sorted into two groups, those that used the app for “Training only” (n=38) and those that used the app in practice–“USERS” (n=48). There were no significant differences in characteristics between “Training only” and “USER” groups. Those in the “Training only” group were exposed to the form and function of the app in simulated training yet never adopted use of the app in practice. Those in the USERS group reported the frequency of use as: Daily-6.3%, few times/week-10.4%, Weekly-29.2%, At least once a month-54.2%.
Usability survey results.
Table 1 shows respondent scores for Usability and Content & Interface sub-scores divided into 3 groups (all respondents, Users, and Training Only). Respondents report positive experiences for all categories of usability, content and interface. For the usability subscale, highest scoring items were usefulness, value, and effectiveness. For the content and interface subscale, highest scoring items were clarity, mistakes, and task effectiveness. Overall satisfaction was scored lowest on both subscales. There was a significant difference in overall satisfaction in usability between Likert group scores for Users (3.78, 3.54–4.03) vs. Training-only group (3.23, 2.93–3.53), p <0.05.
Table 1.
Usability Survey
| Overall (n=86) | Users (n=48) | Training Only (n=38) | |
|---|---|---|---|
| USABILITY | |||
| USEFULNESS | 4.29 (4.08 – 4.49) | 4.14 (3.91 – 4.37) | 4.47 (4.12 – 4.82) |
| VALUE | 4.28 (4.06 – 4.50) | 4.19 (3.91 – 4.46) | 4.39 (4.04 – 4.74) |
| EFFECTIVENESS | 4.28 (4.08 – 4.47) | 4.10 (3.85 – 4.35) | 4.50 (4.19 – 4.80) |
| EASE OF LEARNING | 4.23 (4.00 – 4.46) | 4.12 (3.83 – 4.41) | 4.36 (3.99 – 4.74) |
| ACCESSIBILITY | 4.21 (3.95 – 4.46) | 4.08 (3.76 – 4.40) | 4.36 (3.95 – 4.78) |
| SIMPLICITY | 4.20 (3.95 – 4.44) | 4.10 (3.80 – 4.41) | 4.31 (3.92 – 4.70) |
| PRODUCTIVITY | 4.17 (3.94 – 4.40) | 4.12 (3.83 – 4.41) | 4.23 (3.87 – 4.60) |
| EFFICIENCY | 4.16 (3.93 – 4.39) | 4.10 (3.83 – 4.37) | 4.23 (3.82 – 4.64) |
| COMFORT | 4.03 (3.78 – 4.28) | 3.87 (3.56 – 4.17) | 4.23 (3.82 – 4.64) |
| SPEED | 3.85 (3.58 – 4.12) | 3.76 (3.44 – 4.08) | 3.97 (3.51 – 4.43) |
| OVERALL SATISFACTION WITH EASE OF USABILITY | 3.54 (3.34 – 3.73) | 3.78 (3.54 – 4.03) | 3.23 (2.93 – 3.53)* |
| CONTENT & INTERFACE | |||
| CLARITY | 4.33 (4.15 – 4.51) | 4.30 (4.10 – 4.50) | 4.37 (4.04 – 4.69) |
| MISTAKES | 4.31 (4.09 – 4.52) | 4.29 (4.04 – 4.54) | 4.33 (3.96 – 4.70) |
| TASK EFFECTIVENESS | 4.29 (4.07 – 4.51) | 4.23 (3.99 – 4.47) | 4.37 (3.96 – 4.77) |
| INFORMATION ORGANIZATION | 4.26 (4.05 – 4.47) | 4.15 (3.89 – 4.41) | 4.41 (4.07 – 4.76) |
| FUNCTIONALITY | 4.26 (4.03 – 4.49) | 4.17 (3.87 – 4.47) | 4.38 (4.02 – 4.74) |
| EASE OF FINDING INFO | 4.24 (4.01 – 4.46) | 4.08 (3.79 – 4.37) | 4.44 (4.10 – 4.78) |
| LIKE USING | 4.22 (3.99 – 4.46) | 4.06 (3.74 – 4.38) | 4.44 (4.10 – 4.78) |
| EASE OF UNDERSTANDING | 4.22 (4.03 – 4.41) | 4.13 (3.92 – 4.33) | 4.34 (3.99 – 4.69) |
| INTERFACE | 4.20 (4.00 – 4.40) | 4.08 (3.81 – 4.35) | 4.36 (4.04 – 4.67) |
| OVERALL SATISFACTION WITH CONTENT & INTERFACE | 4.09 (3.86 – 4.33) | 3.89 (3.57 – 4.20) | 4.36 (4.01 – 4.70) |
Usability Scores - mean and 95% CI for “All” respondents, “Users”, and “Training-only” groups. Scores represent 5-point Likert scores measuring level of agreement to functional statements of user perceptions of app usability, content & interface. 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree.
p<0.05 by Ordinal logistic regression between user and trainer group.
Readiness to Launch and Missing Features.
Table 2 shows assessment of problems that may interfere with successful usability of the app. In response to the prompt, 90.6% of respondents perceived the app ready to launch with “minor” or “no problems.” No differences in usability were found between the five hospice sites, χ2 (12, N=85) =13.9, p=0.31). When asked what useful missing features would enhance usability, 23 respondents provided open-ended suggestions which were then analyzed according to Braun & Clarke’s method for thematic content analysis.20 The top themes with respondent examples were: Accessibility-the need for offline access, printable care plans and assessments that could be shared with family care partners; Assessment–easier access to more assessment tools that some thought should be “more thorough” and others “simpler,” and Additional Dementia resources–especially those directed at family and caregivers and for patients with severe dementia.
Table 2.
Overall Mobile App Readiness to Launch
| TYPE OF PROBLEMS | Overall n=85 (%) | User (n=48) (%) | Training Only (n=37) (%) |
|---|---|---|---|
| NO PROBLEMS | 31 (36.5%) | 15 (31.3%) | 16 (43.2%) |
| MINOR | 46 (54.1%) | 29 (60.4%) | 17 (45.9%) |
| MAJOR | 7 (8.2%) | 4 (8.3%) | 3 (8.1%) |
| CATASTROPHIC | 1 (1.2%) | 0 (0.0%) | 1 (2.7%) |
Usability Problem Evaluation – frequency count and column percentages of user responses to the prompt: “OVERALL USABILITY: The Aliviado App has: No Problems, Minor Problems, Major Problems, or Catastrophic Problems”
Discussion
Hospice providers of PLWD require accessible, evidence-based resources to guide care at the end of life. Findings from this study suggest that the newly developed Aliviado mobile app was well-received by members of hospice IDT and can be successfully incorporated into real-world hospice care clinical workflow to support evidence-based care. User testing demonstrated strengths in usability and content & interface. Furthermore, hospice providers suggested the inclusion of features to support further assessment, family resources, and care plans for severe dementia.
The Aliviado app is tailored to support hospice providers with evidence-based resources from a dementia symptom management training program.10,11 While other mobile apps address complementary areas in home- and community-based settings, most are not evidence-based.21 Likewise, even though clinician-targeted mobile apps have also been used in palliative care to assist with advanced care planning, provision of training materials, and display of clinical guidelines, they provide insufficient information to appraise their evidence base.22 Mobile apps can be successfully used to monitor symptoms, capture web-based patient-reported outcomes, and utilize artificial intelligence for patient data insights.23 Although mobile technology use has gone through expansive growth, providers prefer they complement palliative care efforts rather than replace person-to-person IDT hospice services.24 Our evaluation of the usability of this app is an effort to guide mobile health development in a manner that supports rather than replaces palliative care delivery.
Evidence-based content and usability evaluation are essential components of mobile health app development. Usability testing has rarely been reported in apps designed for geriatric or palliative care providers according to a systematic review,22 though several patient and caregiver facing apps have done so.25,26 Instruments to measure usability are numerous. A recent systematic review of usability questionnaires for mobile health app development found 15 unique questionnaires across 35 studies.14 The strength of this work is not only the delivery of evidence-based resources within a mobile health app but the simultaneous evaluation of usability. Collectively, this approach ensures the best evidence is accessible and usable in hard-to-reach environments where they are needed.
Not every user was drawn to use the Aliviado App on a regular basis. Although over half of the users reported using the app at least monthly, there is room to make this tool more accessible and helpful. As indicated by the majority of users stating there were no or minor problems, we believe that this innovative tool has the potential to currently be used in practice to support IDTs. Added to the non-technological resources currently used in hospice care, this innovation offers an additional tool to improve care.
The overall readiness for launch was rated high among all participants. Usability scores for those who were exposed to the app only in training sessions slightly differed from those who used it more frequently. Consequently, usability may serve as a surrogate indicator of those likely to implement the technology beyond a training session (overall satisfaction with ease of usability). In order to enhance functionality, we have adopted an agile process to update the app based on the feedback and requests (e.g., the ability to store the data within the app). Further evaluation is needed to assess its integration in multidisciplinary team approaches. Additionally, caregivers’ perspectives are an important component of care delivery. Future development of this application should include their perspective on receiving care from IDT members who use mobile applications.
There were limitations to this study. First, a limited portion of IDT members completed the survey. The low response rate was anticipated given the nature of the study (voluntary participation of those providing care in a pragmatic trial), care demands and survey timing (responses were solicited during the earlier phases of COVID-19 pandemic), and desirability of providing feedback for a developing technology. While all individuals were required to be trained on the app, they were not required to provide feedback. This may reflect self-selection for those more technology savvy or who found the app useful. Another limitation is that some disciplines were under-represented in our analysis of usability (i.e., physicians). Future app development will include targeted sampling of usability perspectives from these disciplines. Despite the limited survey response rate, we have integrated the feedback and seen significant uptake of the app. Further usability testing is warranted for future iterations of the app, especially with those that report challenges, minor, and major problems with use.
Second, while the participating hospices were diverse in profit status, size and region, they had all agreed to participate in a clinical trial, which may indicate lack of generalizability to hospice agencies as a whole, though most of these hospices had never participated in a trial before. Despite these limitations, these results provide strong baseline evidence for the usability of the app and its potential to serve as a reinforcement for implementation of Aliviado Dementia Care-Hospice Edition as part of the HAS-QOL Trial. We have since utilized these results to add additional features including in-app care plan and assessment score saving, biometric login (e.g. Face ID), mobile push nudges to increase adherence to program components, with future plans to include secure offline access and other features for the 20 additional hospices participating in HAS-QOL in 2021. As the content and features of this prototype undergo further development, future semiotic (meanings people attribute to symbols) and heuristic (expert evaluation and refinement) evaluation will enhance the user interface and usability of this tool for hospice care providers.27
Acknowledgements
This study was conducted with the support of the following funders:
National Palliative Care Research Center (NPCRC, Kornfeld Scholars Fellowship), National Institute of Aging (NIA, R61/R33AG061904, R01AG056610)
Appendix 1. Intervention Description
The Aliviado Dementia Care Hospice-Edition program (Aliviado) is an agency-wide, quality assurance and performance improvement program. The Aliviado program is adapted from an evidence-based, complex intervention program designed to improve dementia symptom management in home healthcare settings.
The goal of the Aliviado program is to help hospice agencies improve symptom management for people living with dementia (PLWD) and care satisfaction of the bereaved caregivers. The Aliviado program consists of the following components: 1. dementia care training; 2. Aliviado toolbox; 3. clinical workflow change; 4. Aliviado mobile application; and 5. mentorship.
Dementia care training
The Aliviado dementia care training consists of 6 real-time training modules for the Aliviado champions and 5 self-paced online training modules for the non-champions. The Aliviado champions refer to local hospice staff who serve as leaders and facilitators to implement the Aliviado program at their agency. The composition of the Aliviado champions include hospice administrators; team managers; bedside clinicians including nurses, social workers, and chaplains; and sometimes physicians, nurse practitioners, and home health aides. The non-champions refer to the rest of the hospice interdisciplinary team (IDT) members. As an agency-wide program, all hospice IDT members are enrolled in either the Aliviado champion training or the non-champion training as appropriate. To ensure sustained training effect, new hospice employees are enrolled in the Aliviado training program on a rolling basis throughout the study period after the initial implementation of the program.
The real-time champion training is led by the Aliviado implementation specialists uniformly trained by the Principal Investigator. Before the outbreak of COVID-19, the champion training was provided in person. Post-COVID, the champion training is provided live via Zoom. The self-paced non-champion training can be completed in the Aliviado learning center on the Aliviado.org website or in the Aliviado mobile application.
The champions, and non-champions other than providers and home health aides, receive training on the following topics: 1. differential diagnosis of dementia, delirium, and depression; 2. pain assessment and management in older adults living with dementia; 3. assessment of behavioral and psychological symptoms of dementia (BPSD); 4. pharmacological and non-pharmacological interventions for BPSD; and 5. effective communication with people living with dementia, caregivers, and health care providers. Demonstration of the Aliviado toolbox materials and functionalities of the Aliviado mobile application are interspersed throughout these training topics as relevant.
For the Aliviado champions, they receive additional training on how to drive quality assurance and performance improvement initiative in hospice settings, including the implementation of the plan-do-study-act cycle, a non-punitive culture, and audit and feedback. They also receive hands-on training on how to use the Aliviado toolbox materials and navigation of the Aliviado mobile application.
For providers (i.e., physicians, nurse practitioners, and physician’s assistants), they receive the non-champion training in the form of two 30-minute online modules on overview of the Aliviado program, BPSD and de-prescribing.
For home health aides, they receive non-champion training in the form of 17 brief online videos (5 to 10 minute per video), covering topics including dementia definition, subtypes, and staging; BPSD (e.g., anxiety, depression, sleep disturbances); communication with PLWD; personal care tips; non-pharmacological interventions; physical environment; and nutrition and hydration. The home health aide training videos are available in both English and Spanish, considering that some aides’ primary language is Spanish and may have lower English literacy.
Aliviado toolbox
The Aliviado toolbox provides hospice agencies and IDT members with:
Two dementia symptom management algorithms (i.e., BPSD and terminal delirium)
Eight standardized assessment tools (e.g., the Mini-Cog, the Neuropsychiatric Inventory Questionnaire, Pain Assessment in Advanced Dementia Scale, etc.)
Seven core Aliviado care plans for specific BPSD (e.g., agitation and aggression care plan, depression care plan, etc.) and two supplementary care plans on caregiver stress and acute delirium
Twenty caregiver education articles on BPSD management and other caregiving issues (e.g., end of life care, advance care planning, caregiver stress, etc.). The caregiver education articles are available in both English and Spanish.
The assessment tools are recommended to be administered at admission and then monthly. The participating hospices are encouraged to start with integration of two to three Aliviado assessments first and then gradually incorporate more materials. Whether or not a care plan should be administered depends on the assessment results.
Clinical workflow change
The Aliviado implementation specialists closely work with the local hospice administrators and champions to identify specific Aliviado toolbox materials to integrate into the local clinical workflow (e.g., embedding the Aliviado care plans into the hospice’s EHR or paper charting system). Because the Aliviado program is being tested in a pragmatic clinical trial, each participating hospice has the flexibility to select the Aliviado toolbox materials that are deemed most relevant to their local context to integrate into their clinical workflow.
Aliviado mobile application
The Aliviado mobile application provides hospice IDT members with the following core functionalities:
Create patient profiles
Complete, store, review, and email Aliviado assessments and care plans in the app
Read and email Aliviado caregiver education articles in the app (Language options: English and Spanish)
“Favorite” any Aliviado assessments, care plans, and caregiver education articles
Read Aliviado news and blogs
Review the Aliviado symptom management algorithms
Access the Aliviado Dementia Care Learning Center to complete the assigned online dementia care training
Weekly push notifications to remind IDT members to complete the dementia care training and tips on Aliviado tools (“Tool of the week”)
The skilled IDT members (nurses, social workers, chaplains, and providers) have full access to all functionalities. The home health aides have access to the Learning Center, news and blogs, the caregiver education articles (including the ability to read, email, change language preference, and “favorite” a caregiver education article). The recommended frequency of assessment administration (at admission and then monthly) remains the same for the mobile application version. Whether or not a care plan should be administered depends on the assessment results.
Mentorship
The implementation specialists meet with Aliviado champions one week post champion training and then monthly after for a full year to provide mentorship on implementation of the Aliviado program. Meanwhile, the champions are expected to provide mentorship to the non-champions at their agency, regarding how to implement the Aliviado program to fit their local context. Additional mentorship to the champions is available through the Aliviado Support Center. The champions can email the Support Center for both clinical and implementation related questions and their emails are forwarded to the appropriate study team member to answer. The Aliviado Support Center also provide technical support to all hospice staff using the Aliviado mobile application or receiving the Aliviado dementia care training.
Appendix 2.
Usability Survey
| Please rank your level of agreement to the following statements: | Strongly disagree | Disagree | Neither agree nor disagree | Agree | Strongly agree | |
| SATISFACTION: Overall, I am satisfied with how easy it is to use the Aliviado App. | 1 | 2 | 3 | 4 | 5 | |
| ACCESSIBILITY: The App is easy to access when I am in the field/at bedside. | 1 | 2 | 3 | 4 | 5 | |
| VALUE: The App is a valuable addition to your practice. | 1 | 2 | 3 | 4 | 5 | |
| USEFULNESS: This App helps me care for my patients. | 1 | 2 | 3 | 4 | 5 | |
| SIMPLICITY: The App is simple to use. | 1 | 2 | 3 | 4 | 5 | |
| EFFECTIVENESS: The App is effective in directing me to information I could use. | 1 | 2 | 3 | 4 | 5 | |
| SPEED: I could use the App quickly. | 1 | 2 | 3 | 4 | 5 | |
| EFFICIENCY: I could use the App efficiently. | 1 | 2 | 3 | 4 | 5 | |
| COMFORT: I feel comfortable using this App. | 1 | 2 | 3 | 4 | 5 | |
| EASE OF LEARNING: It is easy to leam how to use this App. | 1 | 2 | 3 | 4 | 5 | |
| PRODUCTIVITY: Using this App will enhance my productivity. | 1 | 2 | 3 | 4 | 5 | |
| OVERALL USABILITY ASSESSMENT | ||||||
| Please choose one | No Usability Problems | Minor Usability Problems | Major Usability Problems | Catastrophic Usability Problems | ||
| OVERALL USABILITY: The Aliviado App has: | 1 | 2 | 3 | 4 | ||
|
CONTENT and INTERFACE
Please rank your level of agreement to the following statements: |
Not Applicable | Strongly disagree | Disagree | Neither agree nor disagree | Agree | Strongly agree |
| ERROR MESSAGES: The App gives error messages that clearly tell me how to fix problems. | NA | 1 | 2 | 3 | 4 | 5 |
| MISTAKES: Whenever I make a mistake using the App, I recover easily and quickly. | NA | 1 | 2 | 3 | 4 | 5 |
| CLARITY: The information provided with this App is clear. | 1 | 2 | 3 | 4 | 5 | |
| EASE of FINDING INFO: It is easy to find the information I need. | 1 | 2 | 3 | 4 | 5 | |
| EASE of UNDERSTANDING: The information provided with the App is easy to understand. | 1 | 2 | 3 | 4 | 5 | |
| TASK EFFECTIVENESS: The information is effective in helping me comDlete mv work. | 1 | 2 | 3 | 4 | 5 | |
| INFORMATION ORGANIZARON: The organization of information on the system screens is clear. | 1 | 2 | 3 | 4 | 5 | |
| INTERFACE: The interface of the App is pleasant. | 1 | 2 | 3 | 4 | 5 | |
| LIKE USING: I like using the interface of the App. | 1 | 2 | 3 | 4 | 5 | |
| FUNCTIONALITY: This App has all the functions and capabilities I expect it to have. | 1 | 2 | 3 | 4 | 5 | |
| SATISFACTION: Overall, I am satisfied with this App. | 1 | 2 | 3 | 4 | 5 | |
| What features were the most useful? | ||||||
| What features were the least useful? | ||||||
| What features were missing? (i.e., features you would you like to see) | ||||||
Footnotes
There are no conflicts of interest to report.
*We certify that this work is novel clinical research
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Hospice N, Organization PC. NHPCO Facts and Figures: 2018 Edition. In: National Hospice and Palliative Care Organization; Alexandria, VA; 2019. [Google Scholar]
- 2.Fetherston AA, Rowley G, Allan CL. Challenges in end-of-life dementia care. Evidence-based mental health. 2018;21(3):107–111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Amjad H, Snyder SH, Wolff JL, Oh E, Samus QM. Before hospice: symptom burden, dementia, and social participation in the last year of life. Journal of palliative medicine. 2019;22(9):1106–1114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Luth EA, Russell DJ, Xu JC, et al. Survival in hospice patients with dementia: the effect of home hospice and nurse visits. J Am Geriatr Soc. 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hunt LJ, Harrison KL. Live discharge from hospice for people living with dementia isn’t “graduating”-It’s getting expelled. J Am Geriatr Soc. 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Luth EA, Russell DJ, Brody AA, et al. Race, Ethnicity, and Other Risks for Live Discharge Among Hospice Patients with Dementia. J Am Geriatr Soc. 2020;68(3):551–558. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lee RP, Bamford C, Poole M, McLellan E, Exley C, Robinson L. End of life care for people with dementia: The views of health professionals, social care service managers and frontline staff on key requirements for good practice. PLoS One. 2017;12(6):e0179355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Shalev A, Phongtankuel V, Lampa K, et al. Examining the role of primary care physicians and challenges faced when their patients transition to home hospice care. American Journal of Hospice and Palliative Medicine®. 2018;35(4):684–689. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Washington KT, Alaniz Staner LM, Collie BE, Craig KW, Demiris G, Oliver DP. Toward a Practice-Informed Agenda for Hospice Intervention Research: What Are Staff Members’ Biggest Challenges? American Journal of Hospice and Palliative Medicine®. 2020:1049909120948222. [DOI] [PubMed] [Google Scholar]
- 10.Lin S-Y, Schneider CE, Bristol AA, et al. Sequential Pilot Trials of Aliviado Dementia Care to Inform an Embedded Pragmatic Clinical Trial. The Gerontologist. 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Schneider CE, Bristol A, Ford A, et al. The Impact of Aliviado Dementia Care-Hospice Edition Training Program on Hospice Staff’s Dementia Symptom Knowledge. J Pain Symptom Manage. 2020;60(2):e7–e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Radhakrishnan K, Jacelon CS, Bigelow C, Roche JP, Marquard JL, Bowles KH. Association of comorbidities with home care service utilization of patients with heart failure while receiving telehealth. J Cardiovasc Nurs. 2013;28(3):216–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Phongtankuel V, Adelman RD, Reid M. Mobile health technology and home hospice care: promise and pitfalls. Progress in palliative care. 2018;26(3):137–141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sousa VE, Lopez KD. Towards usable e-health: A systematic review of usability questionnaires. Applied clinical informatics. 2017;8(2):470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lewis JR. Measuring perceived usability: The CSUQ, SUS, and UMUX. International Journal of Human–Computer Interaction. 2018;34(12):1148–1156. [Google Scholar]
- 16.Jones TM, Brody AA. Adaptation and Piloting for Hospice Social Workers of Aliviado Dementia Care, a Dementia Symptom Management Program. American Journal of Hospice and Palliative Medicine®. 2020:1049909120962459. [DOI] [PubMed] [Google Scholar]
- 17.Bristol AA, Convery KA, Sotelo V, et al. Protocol for an embedded pragmatic clinical trial to test the effectiveness of Aliviado Dementia Care in improving quality of life for persons living with dementia and their informal caregivers. Contemp Clin Trials. 2020;93:106005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Google. Design Guidance and Code. Google Design. https://material.io/. Published 2021. Accessed April 20, 2021, 2021. [Google Scholar]
- 19.Morville P, Sullenger P. Ambient findability: libraries, serials, and the internet of things. The serials librarian. 2010;58(1–4):33–38. [Google Scholar]
- 20.Terry G, Hayfield N, Clarke V, Braun V. Thematic analysis. The Sage handbook of qualitative research in psychology. 2017:17–37. [Google Scholar]
- 21.MacKenzie Greenle M, Morgan B, Sayani S, Meghani SH. Identifying mobile apps targeting palliative care patients and family members. Journal of Palliative Medicine. 2018;21(10):1380–1385. [DOI] [PubMed] [Google Scholar]
- 22.Meghani SH, MacKenzie MA, Morgan B, Kang Y, Wasim A, Sayani S. Clinician-Targeted Mobile Apps in Palliative Care: A Systematic Review. J Palliat Med. 2017;20(10):1139–1147. [DOI] [PubMed] [Google Scholar]
- 23.Calton BA, Rabow MW, Branagan L, et al. Top Ten Tips Palliative Care Clinicians Should Know About Telepalliative Care. J Palliat Med. 2019;22(8):981–985. [DOI] [PubMed] [Google Scholar]
- 24.Portz JD, Ford KL, Doyon K, et al. Using Grounded Theory to Inform the Human-Centered Design of Digital Health in Geriatric Palliative Care. J Pain Symptom Manage. 2020;60(6):1181–1192 e1181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kamal AH, Wolf S, Nicolla JM, et al. Usability of PCforMe in Patients With Advanced Cancer Referred to Outpatient Palliative Care: Results of a Randomized, Active-Controlled Pilot Trial. J Pain Symptom Manage. 2019;58(3):382–389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Hochstenbach LM, Zwakhalen SM, Courtens AM, van Kleef M, de Witte LP. Feasibility of a mobile and web-based intervention to support self-management in outpatients with cancer pain. Eur J Oncol Nurs. 2016;23:97–105. [DOI] [PubMed] [Google Scholar]
- 27.Islam MN, Karim MM, Inan TT, Islam A. Investigating usability of mobile health applications in Bangladesh. BMC Med Inform Decis Mak. 2020;20(1):19. [DOI] [PMC free article] [PubMed] [Google Scholar]