Piloting of the virtual telecare technology ‘Addison Care’ to promote self-management in persons with chronic diseases in a community setting: protocol for a mixed-methods user experience, user engagement and usability pilot study

Abstract

Introduction

Chronic diseases in older adults are one of the major epidemiological challenges of current times and leading cause of disability, poor quality of life, high healthcare costs and death. Self-management of chronic diseases is essential to improve health behaviours and health outcomes. Technology-assisted interventions have shown to improve self-management of chronic diseases. Virtual avatars can be a key factor for the acceptance of these technologies. Addison Care is a home-based telecare solution equipped with a virtual avatar named Addison, connecting older persons with their caregivers via an easy-to-use technology. A central advantage is that Addison Care provides access to self-management support for an up-to-now highly under-represented population—older persons with chronic disease(s), which enables them to profit from e-health in everyday life.

Methods and analysis

A pragmatic, non-randomised, one-arm pilot study applying an embedded mixed-methods approach will be conducted to examine user experience, usability and user engagement of the virtual avatar Addison. Participants will be at least 65 years and will be recruited between September 2022 and November 2022 from hospitals during the discharge process to home care. Standardised instruments, such as the User Experience Questionnaire, System Usability Scale, Instrumental Activities of Daily Living scale, Short-Form-8-Questionnaire, UCLA Loneliness Scale, Geriatric Depression Scale, Stendal Adherence with Medication Score and Self-Efficacy for Managing Chronic Diseases Scale, as well as survey-based assessments, semistructured interviews and think-aloud protocols, will be used. The study seeks to enrol 20 patients that meet the criteria.

Ethics and dissemination

The study protocol has been approved by the ethic committee of the German Society for Nursing Science (21-037). The results are intended to be published in peer-reviewed journals and disseminated through conference papers.

Trial registration number

DRKS00025992.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This pilot study provides an opportunity to explore the acceptability of and experiences with a potentially beneficial e-health technology in the under-represented population of chronically ill older persons in a telecare setting.

• The mixed-methods study design will provide a deep and broad insight on usability, user experience and user engagement of Addison Care as a German-speaking, culturally adapted virtual avatar.

• This investigation evaluates the efficacy of a sophisticated virtual avatar, Addison, in assisting with many crucial health management tasks—including medication management and health vitals monitoring.

• A focus on barriers to user-engagement for those who are technologically hesitant will provide rich information concerning how best to design virtual avatars and e-health technologies to match user needs and mental models.

• The primary limitation of this study is the relatively small sample size due to our selective inclusion criteria, which may diminish the ethnic and socioeconomic diversity of our sample.

Background

Societies across the globe are facing a significant shift in age demographics whereby older adults are becoming an increasingly larger group within their population. This phenomenon is one of the most salient economic, social, and medical issues of current times.¹ Ageing increases both the risk for most chronic diseases and for multimorbidity. Between 34% and 61% of older adults are multimorbid,² which can have consequences such as disability and functional decline, poor quality of life, social isolation, depression and high healthcare costs.^{3 4}

Patients themselves have an integral role in the management of their chronic disease.⁵ Factors that influence effective self-management of chronic disease include: experience, skill, motivation, culture, confidence, habits, physical and mental function, social support, and access to care.⁶

Self-management of chronic diseases is defined as the response to signs and symptoms when they occur, with the goal that patients play an active role in optimising health outcomes and minimising the impact of their conditions.⁶ Self-management support refers to patient, healthcare professional and healthcare system interventions aimed to improve self-management behaviours.⁷ Self-monitoring vitals⁸ and medication adherence have been recognised as two of the most essential self-management activities performed by patients to promote their health.⁹

Although interventions designed to promote self-management in chronic diseases have traditionally been offered in-person, delivering these interventions remotely using available technology (eg, mobile smart phones, internet, interactive voice response, telephone, virtual reality) has become more prevalent.¹⁰ These technology-assisted interventions have shown to improve self-management and health status.^{11 12}

Digital information technologies support people with care requirements to maintain their independence, improve quality of life, increase health literacy and aid caregivers in their duties.^{13 14} Telehealth is one of the fastest-growing sectors in healthcare. The term refers to a broad array of provider-to-patient communication and has been defined as using telecommunications, information technologies and devices to share information and to provide clinical, population health and administrative services at a distance.¹⁵ Remote patient monitoring (RPM) is a widely used telehealth intervention that can effectively support self-management in patients with chronic diseases.⁷

Remote patient monitoring

RPM is a promising solution for facilitating the patient–physician relationship while addressing the shortage of healthcare workers today. Studies concerning the efficacy of RPM has spanned the topics of postoperative rehospitalisation, chronic disease management, medication adherence and quality of life and has shown promising results.^16–20 However, RPM technology can only benefit patients who choose to actively interact with the devices. As compared with younger users, elderly users also face unique challenges that are a direct result of ageing—such as declines in dexterity, hearing and vision. As a result, researchers have identified that improving ease of navigation for task completion, ensuring appropriate size and colour of font, and properly configuring the size of the hardware itself are paramount in addressing technological hesitancy.²¹

Virtual avatars

Graphic user interfaces, which can improve the user experience and personalise the experience for the user through virtual avatars, have begun to be incorporated into RPM systems. Virtual avatars are an emerging feature in RPM that has shown propitious results in terms of user engagement, health education and self-care behaviour.²²

One important factor in the receptiveness of patients to virtual avatars is the avatar’s appearance. Bott et al²³ investigated the impact of a virtual pet avatar to deliver surveys to older clients. They found that those who interacted with the avatar experienced lower rates of delirium, fewer falls and decreased loneliness. However, research has generally shown that anthropomorphic characteristics are often preferable for virtual healthcare avatars²⁴—as well as similarities in appearance between the avatar and the user.²⁵ Previous literature has revealed that when designing virtual agents for older persons, key factors related to acceptance of technology include conversational latency, gamification and artificially intelligent lexicon.²⁶

User experience and technology acceptance among older persons

Understanding how older adults perceive technology and virtual avatars may lead to improvements in the accessibility, acceptability and adoption of virtual avatars among older persons with chronic diseases. This can be accomplished through user experience (UX) research, wherein the overall experience of the user is assessed through measures related to usability, user engagement, usefulness, function, credibility and satisfaction with the technology.²⁷ While behaviour, cognition and affect are important defining components of user engagement,²⁸ learnability, efficiency, memorability, few errors and satisfaction are defining components of usability.²⁹ UX is based on User-Centred Design, wherein the needs and characteristics of the end user become the focus of technology design and development, with the intention of higher acceptance and fewer user errors.³⁰

Theories that predict and explain health technology acceptance and use can help to tailor the technology to specific patient needs. One of the more recent models, the Unified Theory of Acceptance and Use of Technology (UTAUT),³¹ posits that a person’s intent to use (acceptance of technology) and usage behaviour (actual use) of a technology is predicated by the patient’s performance and effort expectancy of the technology. The UTAUT also suggests social influence and facilitating conditions as determinants of behavioural intention to use the technology.^{31 32} Most older persons are significantly less adept at technology use than the general population, with technology anxiety being a major influence on older users’ intent to use technologies.³³ However, older adults are interested in integrating new technologies into their healthcare.³⁴ Studies confirm the applicability of the UTAUT in the context of Telecare services among older persons.³⁵

Intervention: Addison Care tablet personal computer (PC)

The present research pilots an intervention provided by Addison Care,³⁶ which is an innovative home-bound connected virtual RPM platform for individuals living with chronic disease. A 3D-animated nurse named ‘Addison’ is the centre of interaction between the system and its users, personifying the telehealth experience for the user. The pilot study encompasses two health-related functions of Addison Care: ‘Addison’ supporting the user in self-monitoring relevant vitals (blood pressure, weight, pulse and oxygen saturation) as well as medication schedule adherence. This is achieved by offering reminder and monitoring functionalities (see figure 1).

Figure 1.

Addison Care functions in German version (reproduced with permission from https://electroniccaregiver.com). ACBM, Addison Care base monitor.

The Addison Care hardware consists of a tablet PC with a speaker, a microphone module and a touch screen (see figure 1). The tablet connects with Bluetooth vitals measuring devices and can be installed in a user’s home. Avatar technology combined with natural language understanding and automatic speech recognition provides users with effective natural interaction with the assisting technology.^{22 26} Subtitles, vital signs and medications are graphically illustrated on the Addison Care interface for clear communication between the virtual agent and the user.

The Addison Tablet PC is connected to a web-based dashboard that allows access to user data, including vitals measurements and medication reminders. For the pilot study, medication plans, reminder options and contact information are managed by members of the study team, who also act as a support team for the technical setup and in case of technical problems. The intervention in this study involves voice-driven audiocentred interaction between Addison and users in German, as well as the implementation of a German touch screen interface. Introduction of Addison Care to German users requires adaption of the original technology to ensure a good cultural fit. Adaptations were made to the surroundings of the avatar, as well as to Addison’s mannerisms. Additionally, changes were made to the system to ensure a good fit between system and real life in terms of interactive elements (from basics ensuring appropriate data and time formats to more complex elements like making sure the avatar interacts in a culturally appropriate manner with the user). Voice and touch interaction modes are currently adapted from English into German. All piloted features of Addison Care are shown in figure 1.

Objectives

While other studies have provided insight into the potential of digital health technology and virtual avatars, the vast majority have been tested within laboratory settings, where older adults were unable to interact with the technology in a natural environment. Additionally, the digital health systems and virtual avatars were not culturally adapted after development.

The study aims to explore the feasibility, acceptability, experience, engagement and usability of the culturally tailored health technology and the virtual avatar Addison for self-management for older patients with chronic diseases in their own home.

Methods and analysis

A pragmatic, non-randomised, one-arm pilot study applying an embedded mixed-methods approach will be conducted to examine the primary outcomes ‘user experience’, ‘usability’ and ‘user engagement’ of the virtual avatar Addison three times within the use span. ‘Embedded’ refers to the integration of qualitative methods into a quantitative methodology framework or vice versa, to provide enriched insights or understanding into the phenomena of interest.^{31 37} The study design is pluralistic, problem-centred, real-world applicable and focused on the consequences of actions, stemming from pragmatism as a research paradigm.³⁷ The present protocol followed the SPIRIT guidelines (see online supplemental file 1).³⁸ Data collection will take place between September 2022 and November 2022.

Recruitment criteria and process

Eligible patients will be identified by medical specialists in in German hospitals. The inclusion criteria are as follows:

• Planned patients transition from hospital to extramural care.

• Three to nine drugs (regular intake of drugs, no status of hypermedication).

• Sixty-five years or older with a chronic health condition.

• Ability to speak and understand German language.

The exclusion criteria are:

• Ten or more drugs per day.

• Younger than 65 years.

• Moderate to severe cognitive impairment or severe psychiatric disorders.

Provided that these criteria are met and general interest in using health technology is expressed, information about the pilot study and the intervention will be shared. If a patient declares the will to participate, a meeting with the support team will be arranged while the patient is still at the hospital. Potential participants will be informed of all aspects of the study through verbal instruction and written materials (figure 2, encounter 1). After written informed consent (see online supplemental file 2) is provided, living situation and sociodemographic data will be assessed by research assistants.

Figure 2.

Study flow, phenomenon of interest, instruments, data sets and settings.

Setting and sample size

Addison Care will be piloted in participants’ homes, located in a community setting, after their discharge from hospital for two consecutive weeks. In encounter 2 (see figure 2) within 1 day after the informed consent is provided, the support team will give first instructions on Addison Care while the participant is still hospitalised. First adjustments of reminder, medication plan and vital measurements will be provisioned for the use of Addison Tablet PC at home. This study seeks to enrol 20 patients. The sample size is an adequate number to evaluate study feasibility, test the study procedures and explore the user experience.^{39 40}

Patient and public involvement

In advance of the pilot study, older adults assisted in the development of the data collection materials and pretesting of Addison Care. However, patients and the public were not involved in the development of the research question, outcome measures and the design of the study.

Outcomes, instruments and variables

Building on the theoretical concepts of technology acceptance (UTAUT), we will assess user experience, usability, and user engagement (primary outcomes), as well as participant background information (eg, sociodemographic and care provision) and health status-associated phenomena (functional status, quality of life and well-being, loneliness, depression, medication adherence and self-management) using standardised, quantitative and semistandardised qualitative research instruments (see figure 2).

Standardised research instruments

User experience. The German version of the User Experience Questionnaire (UEQ)⁴¹ will be used to assess user experience. The UEQ consists of 26 items along 6 scales: attractiveness (6 items, Cronbach’s alpha α=0.89), perspicuity (4 items, α=0.82), efficiency (4 items, α=0.73), dependability (4 items, α=0.65), stimulation (4 items, α=0.76) and novelty (4 items, α=0.83).^{41 42} Each item represents a 7-point rating scale (−3 most negative rating, +3 most positive rating) of properties that the product under study may have. An average score is computed for each scale.

Usability. To assess the usability of Addison Care, the validated German version of the System Usability Scale (SUS) will be applied.⁴³ The SUS⁴⁴ consists of 10 items and is a standardised, generic instrument for assessing the usability of technical applications, mobile applications or devices. Internal consistency has been reported to range between α=0.70 and 0.95.⁴⁵ The SUS consists of 10 items, each with 5-point rating scales (1-strongly disagree to 5-strongly agree). The standardised scoring of the SUS results in a total score between 0 to 100 points using a given norm-based scoring algorithm.⁴⁵

User engagement. Automatic system and data logging information will be used to measure user engagement in terms of intensity and type of interactions between users and Addison Care. This non-participatory data collection, for example, documenting data using automatically protocolled technical variables without having asked questions or the presence of an observer, will provide essential information on the actual use, used functions, and user engagement with certain contents of the product of interest.^46–48

Functional status. The German translation⁴⁹ of the Instrumental Activities of Daily Living (iADL) scale⁵⁰ will be applied to assess patients' functional status in terms of activities of daily living. The iADL is a standardised instrument that measures functionality related to eight domains of daily living. It has reported reliability coefficients ranging from 0.85 to 0.91.⁵¹ Each domain is measured using either three or four ability levels with 0 or 1 point per domain, resulting in a summary score of 8 points at maximum. Due to a strong reference of some items to household aspects, gender-specific scores will be used, for example, 0 (low function, dependent) to 8 (high function, independent) for women and 0 to 5 for men, respectively.⁵¹

Quality of life. Health-related quality of life will be measured by the German version of the Short-Form-8-Questionnaire (SF-8).⁵² The SF-8 assesses the eight dimensions physical functioning, role physical (role limitations because of physical health), bodily pain, general health, vitality, social functioning, role emotional (role limitations because of emotional problems) and mental health, by one item each, and along two scales ‘physical component summary score’ and ‘mental component summary score’. The items comprise of five-point or six-point response scales that verbalise the extent to which each dimension is present. In addition to single-item analysis, the two summary scores will be measured using a given norm-based scoring method. Next to an adequate test-retest reliability,⁵² an overall internal consistency between α=0.86 and 0.92 have been reported.⁵³

Loneliness. To assess participants’ perception of social isolation and loneliness, the shortened, three-item German version^{54 55} of the UCLA (University of California, Los Angeles) Loneliness Scale will be applied. Each item exhibits a five-level response scale (very often, often, sometimes, rarely, never) and will be analysed item-by-item. Cronbach’s alpha for the three-item loneliness scale was 0.72.⁵⁴

Depression. The German translation⁵⁶ of the Geriatric Depression Scale (GDS) will be used to evaluate the presence of depression.^{57 58} The eight-item version will be applied to make the survey as time-efficient as possible.⁵⁹ Participants are asked about selected symptoms of depressive states over the past week using a dichotomous response format (no vs yes). The total sum score of the GDS-8 is 0–8 points. Internal consistency with α>0.80 has been shown.⁵⁹ A recommended cut-off score of GDS ≥3 indicating relevant indications of depression will be applied.

Medication adherence. Participants’ adherence to their medication regimen will be measured by the Stendal Adherence with Medication Score (SAMS).⁶⁰ SAMS consists of 18 items on a five-level response scale (0–4) assessing fully adherent to nonadherent medication behaviour per item.⁶¹ Responses are summarised into a cumulative point scale (0–72), which can be categorised as fully adherent (0), moderately adherent (1–10) and not adherent (>10). An overall internal consistency of α=0.83 has been reported.⁶¹

Self-management. To assess participant’s Self-Efficacy for Managing Chronic Diseases (SESG6), the German version of the 6-item scale will be used.⁶² The six items are rated with a 10-level Likert-type scale (1 ‘not at all confident’ to 10 ‘totally confident’). A mean score over at least four of the six items will be calculated, thus allowing a maximum of two missing item responses. SESG6 has been attested a high internal consistency measure of α=0.93.⁶²

Technology proficiency, readiness and expectations. A standardised face-to-face interview prior to the use of Addison Care (‘pre-use interview’) will be performed to collect information on participant technology proficiency and readiness (seven items) in terms of experience with and use of general information and communication technologies (three items) as well as expectations regarding the upcoming use of the Addison Care technology (six items). These closed-ended questions were derived from empirical and theoretical literature^{31 32 63} and further adapted by the research team.

Sociodemographic and care provision variables. Sociodemographic and care-relevant variables will be collected by means of a short, standardised 9-item questionnaire. Age of participants, gender, living situation, place of residence in terms of urbanisation, care provision by relatives, and care provision by ambulant/mobile care service will be assessed using closed-ended questions. Information on documented primary diagnoses and existing additional chronic diseases will be collected using open-ended questions and categorised applying the 11th revision of the International Classification of Diseases and Related Health Problems (ICD-11).⁶⁴

Semistandardised research instruments

First experiences and encountered technical obstacles. A qualitative, semistructured brief telephone interview (‘mid-use interview’) with users after 1 week of Addison interaction will be conducted. Information about users’ experiences to date, as well as previous effort and encountered challenges in using the Addison Care technology will be collected. The user reports are to be recorded in an open-ended documentation sheet.

User experience, fulfilled expectations, perceived enabling conditions for use and technology’s social influence, and health behaviour. A comprehensive qualitative, semistructured, face-to-face interview will explore participants’ perspectives with reference to the fulfilled expectations after the use of Addison Care (‘postuse interview’), perceived enabling conditions and social influence in the use of the technology, as well as the participant’s experiences and adaptions of health behaviour. The interview guide questions on user experience are based on the respective literature on UX research,⁶⁵ those on conditions and technology’s social influence along the main factors of the UTAUT model,^{31 32} and those on health behaviours were developed against the background of the Health Action Process Approach (HAPA).⁶⁶ The interview will be audiorecorded and transcribed. With reference to the embedded mixed-methods approach, the four most striking individual ratings of the previously collected standardised UEQ will be thematised and perceived changes in secondary outcomes (functional status, quality of life, loneliness, depression, medication adherence) will be assessed using open-ended questions. To address their perspectives on the use of Addison Care, an optional topical block of guided questions will be operationalised.

Task performance scenario and think-aloud protocol. Finally, to gain insight into user thoughts, decision-making processes and how they experience the Addison Care technology, a structured observation with an accompanying think-aloud protocol will be applied.⁶⁷ Participants will be asked to perform a set of specific tasks with Addison Care while verbally expressing their immediate thoughts, and explaining their reactions during system interaction. Task performance and participant comments will be documented using a structured observation sheet.

User safety and data management

During the 2-week study period, medical emergencies, acute deterioration in health or care needs, patients' feelings of insecurity, or hospital admissions will constitute reasons to end the participation early. Formal health services in the community setting will be informed about the use of Addison Care by their clients. Informal caregivers of the participants will be educated about Addison Care and are instructed to contact the support team in need of help (see figure 2).

Figure 2 provides detailed information on the different data retrieved during participants’ enrolment. Personal information of participants will be accessed by the support team only, who will monitor the dashboard and assist with any user problems. Dashboard access is granted by login data provided by Addison Care USA.

All data retrieved empirically (see figure 2) will be saved on study-specific computers during data collection and stored in password-protected folders on the support team storage after completed data collection. User engagement data will be stored on Addison Tablet PC for short periods of time being but regularly exported onto the server from the clinical dashboard and after the end of the pilot study transferred to study-specific computers. All personal data will be stored at a server in Berlin in Germany and encrypted. According to European Union General Data Protection Regulations, participants have the right to view all stored data or choose to delete their data at any given time as long as their data has not been anonymised by code yet.

Analysis

Various data will be organised and triangulated in data sets Quan 1–4 and Qual 1–3 (see figure 2) for analysis that fit the relevant phenomenon of interest. Final integration of overall results will take place on conclusion of the study³⁷ and will be summarised with a joint display by using a mixed-methods matrix.⁶⁸

Participants’ characteristics will be statistically described using information on sociodemographics, living and care provision, quality of life, health literacy, activities of daily living and medication adherence (Quan 1, figure 2).

A thematic content analysis of the qualitative data gained from interviews and observations in encounters 4 and 5 (see figure 2) will be performed, expanding the deductively developed code by inductive inputs.⁶⁹ Deductive codes prepared from theoretical preconsiderations will include the concepts of user experience as well as usability. Coding strategy will separate the two phenomena during the coding process. User experience results will be produced by triangulating the results of the UEQ(Quan 3) as well as code system elements gathered in qualitative data sets (Qual 1, 2, 3). These three data sets will provide usability results after interviews are transcribed and coded. The codes will then be merged with the SUS results (Quan 3) to get a clear picture of obstacles and acceptance. User Engagement data will track usage events like logins, reminders and overall Addison-user-interaction over the 2-week usage period—resulting in data set Quan 4 (see figure 2). To facilitate the subsequent main study, deductive codes for the area of a feasibility study are also included in the coding strategy.⁷⁰ All quantitative data will be analysed using common descriptive statistics.

Ethics and dissemination

Ethical considerations

This pilot study was approved by the ethics committee of the German Society for Nursing Science (21-037) to ensure that the research is done in accordance with the Declaration of Helsinki and in line with the current legislation authority (see online supplemental file 3). The pilot study is registered in the German Clinical Trials Register (ID: DRKS00025992).

Dissemination

The results are intended to be published in peer-reviewed journals and disseminated through conference papers.

Overview

This protocol presents research that assesses the feasibility, acceptability, experience, engagement and usability of Addison Care—a health technology and virtual avatar for older persons with chronic diseases in their own home.

For this purpose, we culturally adopted the Addison Care technology and its functions (tutorial, medication management, testing vital signs) to explore participants’ acceptance and experiences of the health technology and the virtual avatar.

For older adults with chronic diseases, the overarching goal of self-management is to enhance their quality of life and maintain independence, all while supporting formal and informal caregivers.

The goal of this pilot study is to further our understanding of the potential issues and challenges that will be used as the foundations for a larger randomised control study.

One of the strengths of this study is the use of the health technology for a longer period of time and with real patients in a natural setting. Another strength lies in the cultural adaption of the health technology and its integration in a telecare framework. The integrated voice and touch interaction with the avatar ‘Addison’ should also contribute to improve the human–computer interaction.

Limitations

Possible limitations of the pilot study are the lack of results on usability or acceptance of the US American version of Addison Care that we can refer to. Cultural adaption and translation into German therefore might not be the only reason for a suboptimal user experience. Interviews allow to gain insight into this issue. The effectiveness of the extensive data collection process has to be proven as well as the recruitment process. The highly selective sample of the pilot study will diminish ethnical or socioeconomic diversity which will be introduced thoroughly in the study following the pilot. Within the qualitative branch of the mixed-methods study we seek sufficient richness of data but do not expect to achieve a data saturation. The study’s time line may be influenced by COVID-19 pandemic recruitment-wise as well as by pandemic regulations in Germany which cannot be foreseen at the current situation. Because we do not have an influence on the stability of the Internet connection, this could be another source of uncertainty. Finally, it is not the aim of the pilot study to show effects on the health status of the users. But the multiple instruments for testing health status-associated phenomena should provide adequacy to show such effects in a subsequent main study.

Ethics statements

Patient consent for publication

Not applicable.