Voice-Activated Self-Monitoring Application (VoiS): User Acceptance and Satisfaction in the Field

Abstract

This paper illustrates the processes and results of user acceptance and satisfaction tests for the voice-activated self-monitoring (VoiS) application conducted in the field. VoiS was designed and developed for individuals with diabetes (DM) and hypertension (HTN) to support their routine and convenient self-management using a smart speaker platform. VoiS is also accessible on users’ mobile devices to visualize user-generated data. A total of nine adults with DM and HTN participated and were asked to use the VoiS system at home for a week to identify its acceptability in real-world conditions. Participants completed phone interviews to report operational errors and a structured survey to assess the acceptability of VoiS. The results showed that participants agreed or strongly agreed that VoiS was easy to use and useful, and they intended to continue using it. They were highly satisfied with VoiS. They also reported operational errors and challenges including device-related technical issues and the smart reminder feature of VoiS. Overall, participants perceived VoiS as an easy and useful tool for managing their conditions and felt motivated to monitor their biomarkers routinely.

I. Introduction

Chronic conditions such as diabetes (DM) and hypertension (HTN) are significant global health challenges. As of 2022, an estimated 830 million people worldwide live with DM, with fewer than half achieving effective medication-based control [1]. Similarly, 1.28 billion adults have HTN, yet 46% remain undiagnosed and only 21% have controlled blood pressure [2]. The coexistence of DM and HTN is common due to shared underlying mechanisms, affecting a substantial proportion of patients [3]. For instance, in the United States, 80% of individuals with DM also have HTN [4], increasing the risk of complications such as heart disease, stroke, retinopathy, and kidney damage [3].

Managing multiple chronic conditions (MCC) adds complexity, often resulting in reduced quality of life, increased emergency visits, hospitalizations, and functional decline [5, 6]. Self-management has emerged as a critical approach for improving the health outcomes of individuals with MCC [7], requiring patients to engage actively in daily tasks such as problem-solving, decision-making, and health monitoring [8]. Among these, self-monitoring is particularly essential as it enables patients to track their health status and make informed decisions [9].

Digital health technologies, including mHealth apps, wearable devices, and telehealth, are increasingly being adopted to support self-management efforts [10]. To address the needs of individuals with MCC, particularly those with DM and HTN, we developed the Voice-Activated Self-Monitoring Application (VoiS) [11]. Grounded in the individual and family self-management theory [9] and prior research on user needs and barriers [12, 13], VoiS is an innovative mHealth tool designed to support self-monitoring through voice interaction.

VoiS operates on a smart speaker platform (e.g., Amazon Alexa) to enhance user convenience through hands-free interaction and multi-platform accessibility. Its main features include: (1) self-monitoring and tracking of glucose, blood pressure (BP), and health behaviors, (2) clinical decision support, and (3) shared decision-making support. The self-monitoring function allows for voice-based data entry and smart reminders, while the clinical decision support component leverages an AI-embedded knowledge base for real-time biomarker evaluation. The shared decision-making function offers both data visualization through a responsive web app and voice-based data summaries for efficient communication. Detailed design and feature descriptions are available in our prior publication [11].

Given its novel design, usability testing is essential to assess user needs, preferences, and barriers, and to refine the tool for optimal experience [14, 15]. An initial mixed-method usability test with 14 patients in a laboratory setting evaluated specific features through task-based assessments and structured surveys. Most participants found the system useful and easy to use, offering valuable feedback for further development [16].

However, while laboratory tests offer controlled, distraction-free environments, they do not fully capture the complexities of real-world usage, where factors such as interruptions, multitasking, and network issues may impact performance [17]. Field-based usability testing is therefore critical to assess real-world usage and user acceptance more accurately [18]. This paper presents the methodology and findings of a field usability and acceptance study of VoiS, offering insights into its effectiveness and user experience in home settings.

II. Related Work

A. Usability Testing in the Field

Field testing is a common method for evaluating the usability and acceptance of mobile applications in real-world settings. Unlike laboratory studies, field tests capture authentic user behavior by allowing interaction with technology in daily environments. This approach accounts for variables such as interruptions, distractions, and network variability, which are difficult to replicate in controlled settings [18]. Additionally, field testing offers greater flexibility and comfort for users, often leading to more natural engagement and reliable feedback. The extended duration of such tests also enables the collection of longitudinal data, providing insights into sustained use and evolving user experiences.

B. Usability Testing for mHealth Applications in the Field

Field testing is increasingly used to assess mHealth applications, focusing on ease of use, functionality, and user satisfaction in real-life contexts. For example, a 6-week home-based study evaluated an app for medication self-management among adolescent transplant recipients, highlighting its role in supporting patient-caregiver communication [19]. This approach provided valuable insights into the app’s practical usability and its impact on patient adherence. Similarly, a 2-week field test, following an initial lab study, evaluated the feasibility and user acceptance of an mHealth app aimed at supporting alcohol self-management, highlighting the importance of context in user engagement and sustained usage [20]. A more extensive, one-year field test of a diabetes self-management app revealed that practical and social acceptability can sometimes compensate for minor usability deficiencies, underscoring the importance of real-world validation for long-term app success [21]. In another example, Wedel et al. conducted a 4-week field study to assess the ASTHMAXcel PRO app, designed for asthma education and patient symptom tracking. Their findings indicated that interactive, personalized interfaces and tailored educational content significantly enhanced user satisfaction and app adoption, demonstrating the critical role of personalization in mHealth usability [22]. These studies underscore the importance of field testing in capturing meaningful, user-centered insights that drive the success of mHealth interventions.

III. Method

A. Study Design

This study employed a mixed-method design, a common approach in field studies that integrates both qualitative and quantitative research methods. Mixed-method designs often include surveys and semi-structured interviews to capture both numerical data and in-depth user experiences. Our approach was modeled after the field test methodology of Kaikkonen [17] et al. and aligned with Level 4 of Henson’s evaluation framework for mHealth apps, which emphasizes usability, user acceptance, and accessibility [23]. The primary goal was to assess the usability and acceptability of the VoiS system in real-world home settings. This involved system installation, task execution, and structured feedback through interviews and surveys.

1). System Installation and Demonstration

On the scheduled installation day (Day 0), a pair of research assistants (RAs) visited each participant’s home to set up the VoiS system. Each participant received a standardized test packet, which included: an Alexa Echo device, a BP monitor, a glucometer, a box of lancets, a box of testing strips, and a paper-based instruction manual. The RAs installed the VoiS app on participants’ smartphones using test accounts, allowing them to track health data (e.g., BP and glucose levels) through both the app and a web-based dashboard. One participant opted to use only the web app without a smartphone, demonstrating the system’s flexibility.

The instruction manual included step-by-step guidance for device setup, Alexa and VoiS operation, personal identification number creation, and task execution. It also provided sample voice commands, such as “Record my blood pressure” or “Enter my BP,” to guide participants through daily health tracking. Additionally, the manual outlined the schedule for phone interviews and provided contact information for technical support.

After installation, the RAs conducted a hands-on demonstration, walking participants through each task and addressing any questions to ensure comfort with the system. Participants practiced using the system until they felt confident, reinforcing their understanding before the independent testing phase.

2). Home-Based Task Execution

Participants were asked to complete five core tasks during the home testing period: 1) BP monitoring: measure BP once daily; 2) Glucose monitoring: check glucose levels twice daily (fasting and post-meal); 3) Health behavior monitoring: log daily health behaviors; 4) Data review: use either the audio briefing feature or the web app to review logged data; and 5) Usage logging: document any system errors or usability issues.

To support task execution, participants set up personalized smart reminders on Day 0 with RA assistance. They were also instructed to record any technical issues or challenges in a structured usage log, which included fields for date, time, feature errors, potential causes, and user comments. This approach provided a comprehensive view of real-world usability and potential system improvements.

3). Phone Interviews and Surveys

Participants completed three phone interviews on Days 2, 4, and 7 to gather qualitative feedback on their experiences using VoiS at home. Interview questions covered topics such as system adoption, ease of use, environmental challenges, and overall satisfaction. This step aimed to capture real-time user perceptions and identify barriers to effective system use.

At the end of the test period, participants completed a survey designed using the second version of the Unified Theory of Acceptance and Use of Technology (UTAUT2) [24] and Lund’s USE questionnaire [25]. The survey included five key themes: ease of use, usefulness, social influence, use intention, and satisfaction. Each theme was measured on a 5-point Likert scale (strongly disagree to strongly agree), providing structured, quantitative insights into user acceptance and satisfaction. The survey and interview questions were aligned with the core features of the VoiS prototype [16], which include self-monitoring, clinical decision support, and shared decision-making, supported by sub-features such as voice interaction, smart reminders, evidence-based knowledge bases, data visualization, and audio briefings.

B. Sample and Setting

We employed purposive sampling to recruit 10 participants, consistent with standard usability testing practices recommending 5–10 users [17, 26]. Eligibility criteria included: (1) age between 45–80 years; (2) diagnosis of both type I or II diabetes and hypertension; (3) English proficiency; (4) passing score on the Six-Item Screener for cognitive impairment [27]; (5) no speech, hearing, or visual impairments; (6) ownership of an electronic device (smartphone, tablet, or computer); and (7) home Wi-Fi access.

Participants were recruited via flyers through two channels: previous usability study participants [16] and clinician referrals from primary care and internal medicine units at a Midwestern tertiary hospital. Initial screening assessed eligibility based on criteria 1–5, after which interested individuals were screened further using criteria 6 and 7. The study protocol was approved by the institutional review board of a Midwestern university (IRB#21.317).

C. Data Collection and Procedure

On Day 0, at least two research team members visited participants’ homes to connect their Alexa accounts, install the VoiS system, and provide usage instructions and demonstrations. Participants then signed informed consent forms and completed a baseline questionnaire.

Data collection occurred between April and May 2024 and included phone interviews on Days 2, 4, and 7, along with structured usage logs. The phone interviews were recorded and transcribed using Microsoft Teams. The usage log was designed with predefined categories for common errors and issues, along with space for open-ended notes. Participants completed the logs while using the system and returned them by mail. Fig. 1 outlines the full data collection process.

Fig 1.

The process of data collection for the acceptability test

D. Data Analysis

We conducted a thematic analysis of the interview transcripts and usage logs to identify patterns relevant to our research questions. This method is suited for uncovering themes without reliance on predefined categories or theoretical frameworks [28]. Descriptive statistics were calculated from survey responses using SPSS 28.0. Triangulating qualitative and quantitative data provided a comprehensive understanding of participant experiences.

IV. Results

A. Participant Characteristics

Nine individuals participated in the field test to assess user satisfaction and acceptance of VoiS. Five participants were recruited from a previous lab-based usability test, and five through physician referrals from clinical settings. Although ten participants were initially enrolled, one (P029) withdrew on Day 2 due to health complications; data of P029 were excluded. All remaining participants completed the study tasks and survey, except P026, who missed the Day 7 interview due to a clinic visit. Despite multiple follow-up attempts, no response was received, but data from P026—including demographics, usage logs, and interviews from Days 2 and 4—were included in the analysis.

1). Demographics and Technology Experience

Participants ranged in age from 41 to 79 years, with six over the age of 60. Most participants were female (66.7%) and White (66.7%), and more than two-thirds were retired. All had diagnoses of hypertension and either type 1 or type 2 diabetes, and over half also reported high cholesterol. Full demographic details are provided in Table 1. All participants owned and regularly used smartphones. Fewer than half (44.4%) had a smart speaker at home. Most (77.8%) had prior experience with voice assistants such as Siri, Alexa, or Google Assistant.

Table 1.

Participant Characteristics (N=9)

Variables	Category	Mean/n	Range/Frequency
Age (year)		61.22	41–79
Gender	Male	3	33.3%
	Female	6	66.7%
Race/Ethnicity	Black/African American	2	22.2%
	White	6	66.7%
	Hispanic	1	11.1%
Education	College (no degree)	5	55.6%
	Associate degree	2	22.2%
	Bachelor’s degree	2	22.2%
Health Conditions	Diabetes type 1	2	22.2%
	Diabetes type 2	7	77.8%
	High blood pressure	9	100.0%
	Obesity	4	44.4%
	Heart disease	3	33.3%
	High cholesterol	5	55.6%
Recruitment	VoiS usability lab test	5	55.6%
	Clinic referral	4	44.4%

B. Acceptability test at home

1). VoiS Usage Patterns

Participants used the VoiS system between one and three times daily, with an average frequency of twice per day over the one-week testing period. Most participants set up 2 to 4 smart reminders to support their monitoring routines. Over half scheduled the maximum of four reminders, for example, for BP at 7 a.m., fasting glucose at 8 a.m., post-meal glucose at 3 p.m., and health behavior tracking at 7 p.m.

2). Perceived Acceptability of VoiS

Eight participants completed the acceptability questionnaire during the final phone interview. As shown in Table 2, the majority found VoiS to be both easy to use (78%) and useful (89%). Most participants (89%) expressed high satisfaction and a strong intention to continue using the system. Most participants either agreed or remained neutral regarding social influence (i.e., whether others encouraged its use).

Table 2.

Acceptability of VoiS (N=8)

Category	# of Item	Median	Mean	Range 1–5	SD
Ease of Use	4	4.12	4.06	2.75 – 5.00	0.65
Perceived Usefulness	4	4.50	4.34	3.50 – 5.00	0.48
Social Influence	3	3.83	3.83	3.00 – 4.33	0.50
Use Intention	3	4.0	4.16	3.67 – 4.67	0.35
Satisfaction	6	4.50	4.25	3.50 – 4.83	0.50

When asked about specific features, participants identified BP and glucose monitoring, along with data representation, as the most important for managing their health. Smart reminders and data visualization were rated as the most helpful features. Interestingly, despite their usefulness, smart reminders were also cited as the most inconvenient component of the system.

3). System Errors in the Real Setting

Data from usage logs and phone interviews revealed three main categories of challenges encountered with VoiS in home settings: (1) device-related technical issues, (2) smart reminder inconsistencies, and (3) environmental factors.

1. Device-related technical issues

   1. Difficulty understanding lay language: Six participants reported that Alexa had trouble recognizing non-standard or lay expressions when initiating or completing tasks. Comments included: “The hardest part is to get Alexa to recognize my words (P007),” “Alexa did not understand when I said my BP (P021),” “Alexa stopped talking sometimes (P022),” and “I don’t know whether it understands me or not. She didn’t know what I was trying to say and gave me the right response (027).”
   2. Insufficient training and familiarization time: Five participants were new to smart speakers and needed additional time to become comfortable with Alexa. Difficulties were noted particularly on Day 0 and Day 2 interviews but were no longer reported by Days 4 and 7. Participants cited language barriers and lack of tech experience: “Language, because I am from another country, she doesn’t understand me (P021),” “We are not very tech-savvy. We don’t have an Alexa in our home. We haven’t really studied the whole app yet, we need time (P026),” “Maybe I was speaking too fast (P027),” and “The first day was a little cracky for me. It was the first time. After that, everything was fine (P028).”
   3. Challenges with device control: Some participants struggled with basic device controls, such as adjusting volume or restarting the system:, “Alexa did not work. So, turned off the machine then backed it (P027)” and “One thing I don’t like it is I can’t progress the volume, it’s extremely loud (P024).” As with the previous issue, these concerns were only reported early in the study period and appeared to resolve with continued use.

2. Smart reminder inconsistencies

   Issues related to VoiS’s smart reminders were mentioned across all interviews and logs. Six participants reported receiving repeated reminders even after submitting their data: “Because keep reminding me. And I said, I already did it (P021),” “But reminder at 1:30 pm then at 2 pm reminder played again after I did my readings (P022),” “When I record my data earlier, it still reminds me and says repeatedly (P028),” and “The reminder didn’t stop (P030).” These reminders stem from VoiS’s rule-based AI, which prompts users when scheduled data entries appear missing. The mismatch between actual usage and scheduled reminders likely contributed to this issue, especially since participants used VoiS about twice daily while setting 2–4 reminders.

3. Environmental factors

   Most participants reported no environmental interference. However, two specific issues were identified: i) connectivity problems: one participant experienced intermittent internet due to a modem failure; and ii) background noise: another participant noted that overlapping voices from a visiting grandchild interfered with Alexa’s voice recognition: “My grandson is in the room. Maybe his voice plus my voice is a disturbing (P027).”

C. Recommendations for VoiS

There are three main recommendations.

1). Improvement of utterance:

The most frequent recommendation concerned improving Alexa’s ability to understand natural, layperson language. Participants expressed frustration with needing to use specific phrases to communicate effectively: “It doesn’t understand me. Maybe it can be improved by understanding the human language better (P027)” and “And I must say certain words to communicate with her (P022).” A few participants also reported difficulty opening VoiS because Alexa understood VoiS as one of the voice recording apps. For instance, “There are hundreds of applications out there that use the word voice. So when you ask her, you know, to open her a voice, app or voice skill (P007)” and “I think yeah is gonna change the name VoiS a little bit. Health voice or voice health or so. It seems like there’s a lot of apps out there that contain the word voice (P024).”

2). Improvement of the set-up process:

While all participants were guided through the setup process on Day 0, some later found it difficult to modify their reminder schedules independently. They recommended making reminder customization more flexible and user-friendly: “Set-up process. And set-up the reminder. I want it in a different time (P022)” and “I want to be able to schedule at different times to remind you. Do it a little differently. Reset it at a different time. Make the reminder smarter like I can reset it easily (P027).” This feedback highlights a need for a more intuitive and accessible setup interface, particularly for rescheduling tasks during real-world use.

3). Connection with healthcare providers:

Participants valued VoiS’s data representation features and expressed a desire for integration with healthcare systems to facilitate data sharing with providers. This suggestion echoed findings from the previous lab-based usability test: “I would like to see it in the future connected to like my doctor, like MyChart (P024).” Enabling connectivity with clinical platforms would enhance VoiS’s utility for chronic condition management and align with users’ expectations for comprehensive care support.

V. Discussion

The VoiS system was developed to support routine self-management of DM and HTN in individuals with MCC, guided by the individual and family self-management theory [9] and prior needs assessments [12, 13]. This field-based usability test, aligned with Nielsen’s usability standards [26] and Level 4 of Henson’s evaluation framework [23], was conducted to assess ease of use, user satisfaction, accessibility, and real-world functionality. Field testing allowed for the collection of ecological data in naturalistic settings over an extended period, revealing insights beyond those observed in lab environments.

A. Principal Findings

Nine individuals participated in the field test. Overall, participants responded positively to the VoiS system, particularly its voice-based interaction for entering and reviewing self-monitoring data. Most reported that VoiS was easy to use, useful, and satisfying, with strong intentions to continue use. Some even expressed interest in purchasing the system. These findings are consistent with other smart speaker-based interventions that have demonstrated high levels of user satisfaction and feasibility [29, 30]. For instance, Arem et al. [29] and Quinn et al. [30] reported similar user enthusiasm in sleep and physical activity interventions, respectively.

Participants particularly appreciated the voice interaction and data representation features for blood pressure and glucose tracking. These were seen as essential and effective, aligning with findings from our previous lab-based usability test [16]. However, responses to the smart reminder feature were mixed. While participants acknowledged its usefulness, they also found it inconvenient. In the lab test, where reminders were used under controlled conditions, feedback was uniformly positive. In contrast, in the home setting, mismatches between reminder schedules and actual usage led to repeated, sometimes unnecessary notifications.

This issue likely stems from the rule-based AI algorithm used for reminders, which triggers alerts when data is not received at the scheduled time. Since participants typically used VoiS only twice a day, despite setting 2–4 reminders, the system perceived missing data and issued redundant reminders. Additionally, participants found it difficult to adjust reminder times independently once the setup was complete. These challenges are similar to those identified by Shade et al. [31], where older users required more support to modify voice assistant reminders.

B. Usability in Real-World Settings

A key goal of this field test was to identify operational issues in real-life contexts. Early in the study, participants reported challenges related to device interaction—such as initiating and completing tasks, volume control, and Wi-Fi connectivity. These difficulties, mostly reported during the Day 2 interview, are consistent with barriers noted in similar smart speaker-based studies [30–32]. For example, Quinn et al. [30] and Cheng et al. [32] highlighted issues with voice recognition and the steep learning curve for new users.

Despite these initial challenges, most technical and usability issues diminished over time. By the final interview, only concerns about smart reminders persisted. Participants who were unfamiliar with smart speakers initially required more time and support to learn the system but generally became proficient after continued use. This underscores the importance of providing sufficient hands-on training during system onboarding. Greater familiarity with technology could help reduce device-related issues and allow participants to focus more on evaluating content and features.

C. Limitations

This study has several limitations. First, the field test duration was limited to eight days, including the setup day. While this timeframe was sufficient to assess initial usability and gather early feedback, it may not fully capture longer-term behaviors such as sustained integration into daily routines, personalized app adjustments, or evolving user preferences. Second, the study included only nine participants. Although this aligns with recommended sample sizes for usability testing, it limits the generalizability of the findings to broader populations. Third, many participants were unfamiliar with Alexa, yet the time allocated for training was limited. Familiarity with smart speaker technology is essential for effective system use and meaningful feedback. With additional training and practice, participants may have provided more detailed insights, especially regarding advanced features such as reminder customization and system navigation.

VI. Conclusion

This study evaluated the user acceptance and satisfaction of VoiS, a smart speaker–based system designed to support self-management of diabetes and hypertension. Using a mixed-method field test, nine participants engaged with the system in their homes for over one week and shared their experiences through surveys and interviews.

The results demonstrated high user satisfaction and strong intent to continue using VoiS. Participants found the system acceptable, easy to use, and beneficial for monitoring their health conditions. Despite limitations in training and test duration, the findings support the potential of voice-assisted technology in chronic disease management. Future research should explore long-term use and clinical outcomes in a larger, more diverse population to further validate the system’s effectiveness.