Usability of a wearable device for home-based upper limb telerehabilitation in persons with stroke: A mixed-methods study

Abstract

Background

The use of wearable technology offers a promising home-based self-directed option for upper limb training. Although product usability is a crucial aspect of users’ acceptance of a wearable device, usability studies in wearable devices are rare, with most studies focusing primarily on clinical validity.

Objective

This study aimed to explore the usability of a wristwatch device called “Smart reminder” for home-based upper limb telerehabilitation for persons with stroke.

Methods

Eleven stroke participants used the proposed wristwatch for at least two weeks and underwent a home-based telerehabilitation program. A mixed-methods design was used to explore the usability of the wristwatch. Quantitative data were collected through the System Usability Scale (SUS) questionnaire, and the participants’ rate of therapy compliance (gathered from the device) was reported descriptively. In addition, qualitative data were collected through semi-structured interviews with the participants and were analyzed using thematic analysis.

Results

The results demonstrated that the usability of the proposed wristwatch and telerehabilitation system was rated highly by the participants, with a high SUS mean score of 84.3 (12.3) and high therapy compliance rate (mean  =  91%). Qualitatively, all participants reported positive experiences with the wristwatch and indicated keenness to use it again. Participants reported physical improvements and felt motivated to exercise after using the wristwatch. They found the device easy and convenient and appreciated the remote monitoring function. Meanwhile, they highlighted critical considerations for the design of the device and program, including technical support, a wearable design of the device, graded exercise content according to ability, and flexibility in exercise schedules. Finally, they suggested that an interim review with the therapist on their progress might help them continue using the wristwatch.

Conclusions

This study's results supported the proposed wearable device's usability and showed strong acceptance by the participants for using it as a home-based upper limb telerehabilitation intervention.

Background

Upper limb disability in stroke survivors poses a significant challenge to rehabilitation practitioners,¹ and further research on rehabilitation to improve the performance of paretic upper limbs is recognized as a top priority of stroke survivors, carers, and healthcare practitioners.² Intensive and frequent task-specific practice is known to improve upper limb recovery after stroke,^3–5 but it can be resource-demanding when carried out in person. Therefore, there is an increasing emphasis on self-directed upper limb training through empowering patients and caregivers in the home.^6,7 Home-based training provides context-dependent learning and uses objects of daily relevance to the patients.^8,9 The use of wearable technology offers a promising option as a form of home-based self-directed upper limb training while keeping costs low.¹⁰

Using wearable devices in upper limb intervention provides several advantages over conventional rehabilitation by being portable, flexible, and of low cost.^11–13 In such cases, wearable devices are electronic gadgets worn by users to capture or track biometric information related to health and fitness.¹⁴ In addition, the wearable device offers an opportunity for independent training by providing the end-user with augmented feedback, which is crucial for motor retraining.¹¹ Although the clinical application of wearable technology in upper limb rehabilitation among the stroke population is relatively new, emerging research has shown promising results.^15–18 In previous studies, a wearable device was used as an accelerometer to monitor arm movement,^15–17 providing external cues to prompt individuals to use their impaired arms in their daily routines.^15,16 A systematic review by Wang and colleagues¹¹ highlighted that wearable devices should support interactive therapy, which can be delivered through virtual reality and telerehabilitation, with the users receiving interactive stimulation through real-time visual and auditory feedback, thus enhancing their enjoyment during training.^19,20 However, this feature of interactive therapy was generally absent in the studies mentioned earlier.^15–17

In addition to having an accelerometer and providing external cues, this study's proposed wristwatch (wearable device) had two enhanced functions: the addition of a gyroscope sensor and the integration of that sensor with an interactive telerehabilitation application. Indeed, inertial sensors such as accelerometers and gyroscopes are commonly used in wearable devices to capture human motion.²¹ An accelerometer is a sensor that measures linear acceleration along one or several directions by turning kinetic movement into a digital measurement.^22,23 In contrast, a gyroscope measures angular acceleration exclusively and is particularly useful for measuring orientation and projection involving angles, such as range of motion (ROM).^23,24 Argoanam and colleagues²³ highlighted that combining an accelerometer and a gyroscope helps to filter errors and increase accuracy in measuring angles. Unlike previous studies,^15–17 this study's proposed wristwatch featured gyroscope and accelerometer sensors to accurately measure the arm's ROM.

Another enhancement of the proposed wristwatch was that it could be integrated with a telerehabilitation application (app) to offer interactive therapy. The wristwatch was linked to an app downloaded onto the participants’ mobile devices, enabling the participants to view and perform the prescribed exercises while wearing the wristwatch. The sensors in the wristwatch captured the wearer's arm movements, while the mobile device, such as a smartphone, displayed a moving bar indicating the real-time range of motion of the participant's arm. This ROM angle display from the mobile device gave the user concurrent feedback on his or her performance and showed the targeted angle, motivating the user to aim for it. At the therapist's side, the therapist could remotely monitor the number of completed exercise sessions once the participants have done. The proposed wristwatch provided multimodal augmented feedback through its reminder (auditory and vibrotactile) and telerehabilitation (visual and auditory) features. Multimodal augmented feedback has been considered especially effective in motor learning because the human brain processes information better and quicker if feedback is presented from different modalities simultaneously.²⁵

Before evaluating the clinical application of the proposed wristwatch, it was paramount to investigate its usability from the user's perspective. Redström²⁶ highlighted that the lack of end-user involvement in the design process of such items as interactive systems might risk losing users’ acceptance and approval. For a wearable device to be successfully incorporated into clinical trials, the users must be willing to wear and engage with it over a sustained period.²⁷ Previous studies have confirmed that product usability is crucial to users’ acceptance of a device.^28,29 According to the International Organization for Standardization,³⁰ usability refers to a product's effectiveness, efficiency, and user satisfaction rating in a specific environment for a particular purpose. Effectiveness is defined as the accuracy and completeness of a goal achieved by the product; efficiency refers to the effort required by the user to complete a specific task with the product, and satisfaction is the comfort and acceptability of the product.²⁷

To our knowledge, usability studies in wearable devices are rare; previous research on wearable technology has focused primarily on clinical validity.^27,31 Keogh and colleagues³² highlighted that it is essential to investigate the users’ experiences on wearable devices. An understanding of users’ perspectives will aid in the design of a wearable device that is useful and readily accepted by the users. Therefore, this study explored the usability of a wearable device as a wristwatch for home-based upper limb telerehabilitation for persons with stroke. Uncovering participants’ experiences using this device should give valuable insights into improving its features for future clinical trials. This study sought to identify three main aspects of the proposed wristwatch: (a) its functions and features—specifically, which aspects the users valued and which ones they disliked; (b) its usability and acceptability, according to the participants; and (c) its features that required further modification.

Methods

A mixed-methods exploratory study was conducted. Such an approach takes advantage of both quantitative and qualitative data to provide a complete panorama of the usability of the device's application as well as the users’ attitudes and perceptions.³³ The COREQ (consolidated criteria for reporting qualitative research) checklist³⁴ was used to guide the qualitative approach in this study (see Supplementary S1). Figure 1 presents a summary of the overall study design.

Figure 1.

Summary of the overall study design.

Recruitment

This study was conducted in Hong Kong. The research team contacted potential stroke participants from the community via phone and asked if they were interested in participating in the study. These participants were from a contact pool who joined previous research with the research team and had consented to be contacted again for future research. During a screening visit, they were screened using the selection criteria listed in Table 1. Then they underwent an initial assessment by a licensed occupational therapist to evaluate their upper limb function. The study's details were explained to the participants using study information sheets. Written consent was obtained before recruitment. Participants were informed that they could withdraw from the study at any time. Furthermore, if a participant withdrew from the study, any data collected concerning the participant would not be analyzed unless the participant had given consent.

Table 1.

Selection criteria of the study.

Inclusion criteria

1. age ≥ 18 years

2. unilateral hemiparesis

3. time of stroke onset ≥3 months

4. able to understand verbal instructions and follow 2 step commands

5. hemiplegic upper limb with FTHUE-HK ≥ score of 3

6. score on modified Ashworth Scale (MAS) ≤ 2

7. no complaints of excessive pain and swelling over the hemiplegic arm

Exclusion criteria

1. participant in an experimental drug study

2. cognitive or communication difficulties which would inhibit the person in the interview or use of the device

3. had botulinum toxin (Botox) injection in the previous three months with reference from previous studies (16, 69), so that the treatment effect would not be influenced by the effect of Botox.

4. other significant upper limb impairments, i.e., fixed contractures, frozen shoulder

5. a diagnosis which would interfere with the use of the device, i.e., visual impairment

A purposive sampling method was used to identify and select individuals who were knowledgeable about or experienced with the phenomena of interest³⁵ The participants were stratified according to their upper limb severity level, as determined by the Functional Test for the Hemiplegic Upper extremity (Hong Kong version) (FTHUE-HK),³⁶ and their age to gain richer insight into their experience with various characteristics. This study referenced the age classification (young: <48, middle-aged: 48–63, and older persons: >63) described by Lin et al.³⁷

As this was a usability study, a power calculation for the sample size was not conducted. Nonetheless, this study aimed to recruit 12 participants based on the recommendation by Julious³⁸ for a pilot study. This study (HSEARS20220204001) was approved by the ethics committee of the Hong Kong Polytechnic University before its commencement, and the principles of non-maleficence, autonomy, and confidentiality were strictly followed.

Study procedures

The participants were each given a wristwatch (i.e. the study's wearable device) and then underwent home-based telerehabilitation for at least two weeks. The 2-week's trial was to ensure that the participants had sufficient exposure to using the device and telerehabilitation application to share their user experience. An occupational therapist trained the participants to use the watch and connect it to a telerehabilitation application (app) that was downloaded onto their mobile devices. During the training, the therapist prescribed and taught the participants the appropriate exercises, based on the severity of their hemiplegic arm, using the telerehabilitation app. The therapist then discussed with the participants their daily schedules and recommended an exercise schedule for each individual. Finally, the therapist set a reminder interval provided by the watch, according to the agreed-upon exercise schedule. The participants were instructed to wear the watch for at least three hours a day and to follow the exercise schedule for at least three days a week. A typical exercise schedule for the participants was 30 min per session, two to three times a day. With reference to previous studies,^16,39 the recommended 3-h wearing duration was chosen to be intensive but did not demand too much commitment from the participants.

The wearable device

The proposed wristwatch, 5 cm × 3.5 cm × 1.5 cm, weighs 70 g and has an accelerometer and gyroscope sensors. It has a rechargeable battery which allows 72 h of continuous use. In addition, it had a Bluetooth function to allow a connection with a mobile device, such as a smartphone or tablet, to access the telerehabilitation app that had been downloaded onto the mobile device. Two web pages are designed, one for therapists to set the exercise parameters and remotely monitor the participant's movement data and another for patient use.

The wearable device (Figures 2 and 3(a) and 3(b)) provided augmented feedback for training and activity tracking through three mechanisms:

1. Reminder function: The device emits a vibration and sound signal to remind the wearer to do the prescribed exercises. This emitted signal continued until the wearer pressed the acknowledgement button and performed the prescribed exercises. The reminder interval was predetermined and adjusted by the therapist

2. Videos on prescribed exercises: Once the device was connected to the telerehabilitation app on the participants’ mobile devices, they could watch videos of prescribed exercises and perform them. The wristwatch sensors detected and recorded the degree of range of motion as the participants moved their arms. From the telerehabilitation app, the participants received concurrent and terminal feedback on their movements during their exercise sessions.

3. Therapists’ remote access to and evaluation of the participants’ arm movement data: Data captured by the wristwatch for a participant's ROM angles and exercise sessions were uploaded to an encrypted cloud server, from which the therapist could remotely access and evaluate the patient's data.

Figure 2.

The wearable device and the telerehabilitation app.

Figure 3.

(a) Participant's view. (b) Therapist's view.

Data collection

After the trial period, quantitative and qualitative data were collected to understand the participants’ experiences using the wearable device.

Quantitative data. Brief demographics of the participants were collected in terms of their age, time of stroke onset, gender, functioning level of their hemiplegic upper limb, and previous experience in using a wearable device. The participants were asked to complete the System Usability Scale (SUS) questionnaire (Supplementary S3), which is a scale that measures the usability aspect of a device.⁴⁰ It consists of a 10-item questionnaire with five response options for the respondents, ranging from 1  =  strongly disagree to 5  =  strongly agree, and resulted in a possible minimum score of 0 and a maximum score of 100. A score of 68 and above was considered to represent above-average usability.⁴⁰ In addition, this study assessed the level of acceptability of the device, using the SUS score as described by Bangor et al.⁴¹ An SUS score below 50 was considered not acceptable, a score of 50–70 was considered marginally acceptable, and a score above 70 was considered acceptable. Furthermore, to calculate the therapy compliance rate of each participant, the researcher reviewed the wristwatch-collected data on the person's number of completed exercise sessions. The compliance rate was defined as “high compliance” if the participant completed at least 80% of the prescribed exercise sessions and “low compliance” if the participant did less than 80%.⁴²

Qualitative data. The first author (SFMT) and a research assistant conducted semi-structured interviews with the study participants. This method allowed the interviews to be organized with predetermined, open-ended questions guided by the research objectives and other questions that emerged from the dialogue between the interviewers and interviewees.⁴³ The advantage of this method was that it allowed the interviews to be focused on addressing research questions while also providing the flexibility to examine deeper issues raised.⁴⁴ Each interview lasted for 30 min to an hour. All the interviews were conducted face-to-face at a place of the participant's preference (i.e. the participant's home or the university lab). Smith et al.⁴⁵ emphasized the importance of choosing an appropriate environment for an interview where participants feel most comfortable and familiar; hence, two choices for the interview location were given. The interviews were conducted with the participants alone or accompanied by their caregivers. With each participant's consent, the interviews were audio-recorded. The interview guide consisted of core questions on the participant's history of stroke, treatment, motivation for participating in the study, experience in using the wearable device, any changes he or she recommended, and intention to use the device in the future. See Supplementary S4 for the sample interview questions.

Data analyses

Descriptive statistics were calculated to determine the mean SUS score of the participants and the mean rate of their compliance (total completed exercise sessions/prescribed sessions). The SUS score for each participant was computed using the standard scoring methodology.⁴⁰

All the interviews were transcribed verbatim. Two researchers (SFMT and PCG) analyzed the transcripts using the six phases of thematic analysis proposed by Braun and Clark.⁴⁶ Data were arranged according to a thematic framework built upon the constructs of the unified theory of acceptance and use of technology (UTAUT) model, which was used because it is robust and parsimonious in understanding the drivers of a user's intention to accept information communication technology (ICT).⁴⁷ One key strength is that it can account for 70% of the variance in usage intention, in contrast to other models, which routinely only explain approximately 40% of the variance.⁴⁷ This model described four constructs that could influence the user's attitude and behavioral intention to use the device: performance expectancy, effort expectancy, social influence, and facilitating conditions.⁴⁷ In each construct of the UTAUT, subthemes arising from the data were analyzed using an inductive process.⁴⁸

Participant identities were protected by coded identifiers to ensure confidentiality.^49,50 Documents with the participants’ names and code numbers were kept in different locations under lock and key. All research data was to be discarded three years after study completion, with any documents that could reveal the participants’ identities to be shredded.

Rigor and trustworthiness

Specific strategies were applied to ensure the trustworthiness of the qualitative data. First, investigator triangulation was used, with each interview being coded and analyzed separately by two researchers.^51,52 After that, a team meeting was conducted to compare and identify the overlapping themes and subthemes. Any differences were resolved with a third researcher, as needed. Researcher reflexivity was encouraged by maintaining a reflexive journal.⁵³ In addition to the reflexive journal, a researcher positionality map (see Supplementary S4) was also drawn to reflect how the researchers’ positions impacted the analysis, knowledge production, and transparency.^54,55

Results

Twelve participants from the community in Hong Kong were screened for eligibility to participate in the study. Eleven participants were recruited for the study. One participant did not meet the inclusion criteria due to suspected underlying cognitive difficulties influencing the person's learning ability to use the device. This study stratified the recruited participants into different age groups and levels of hemiplegic upper-limb functioning. The participants’ ages ranged from 31 to 68 years, with a mean age of 56.5. 42% of the participants had a higher-functioning upper limb and 58% had a lower-functioning upper limb. As the participants were recruited from the community, they all had experienced chronic stroke onset more than a year previously. All except one participant had no prior experience using a wearable device. Most participants (i.e. 9 out of 11) were no longer receiving outpatient rehabilitation during the study period. Table 2 (and Supplementary S5) outline the demographic characteristics of the participants.

Table 2.

Characteristics of the participants (n  =  11).

Demographics
Gender (n)
Female	7
Male	4
Age (years)
Years (mean  ±  SD)	55.9 (11.1)
Young (<48 years old)	2
Middle aged (48–63 years old)	5
Older (>63 years old)	4
Time of stroke onset (years)
Years (mean  ±  SD)	6.3 (3.1)
>1 year to 5 years	4
>5 years	7
Upper limb severity, FTHUE-HK (levels)
FTHUE-HK: 3 to 5 (low functioning)	7
FTHUE-HK: 6 to 7 (high functioning)	4
Experience with wearable device(s)
Yes	1
No	10
Accompanied during interview
Yes	4
No	7

Note: FTHUE-HK  =  Functional Test for the Hemiplegic Upper extremity (Hong Kong version).

Quantitative results

Figure 4 illustrates the acceptability and adjective rating ranges of the participants’ mean system usability scale score. The participants’ mean SUS score for the wristwatch was 84.3 (12.3) out of a total score of 100, indicating an “above average” usability score⁴⁰ and an “excellent” acceptability level.⁴¹ Regarding the treatment compliance rate, the mean compliance rate among the participants was 91%, indicating a “high compliance” rate as defined by Alexandre et al.⁴²

Figure 4.

The acceptability and adjective rating ranges of the participants’ mean system usability scale score.

Qualitative results

The themes and subthemes arising from the interviews were structured according to the constructs that form the thematic framework of the UTAUT model: (a) performance expectancy, (b) effort expectancy, (c) social influence, and (d) facilitating conditions, to which we added a fifth theme: (e) Intention to use. An overview of all the themes and subthemes is presented in Figure 5. Most participants related their experiences using the proposed wearable device to the constructs of performance expectancy, effort expectancy, and facilitating conditions. Fewer subthemes emerged from the construct of social influence.

Figure 5.

The themes and subthemes of qualitative findings.

(a) Performance expectancy. Performance expectancy refers to the degree to which the participants believed using the device would improve their health outcomes.^44,47 All the participants expressed confidence in the proposed device's performance expectancy (theme/construct) through their positive perceptions of its usefulness and the physical improvements they experienced (subthemes).

Perceived usefulness. Participants found the wristwatch useful for creating health benefits and establishing an exercise routine. “Every night I want to do [it], as I think it will be useful for my hand, and I can control my hand as my hand will tremor,” one female participant reported (P11). Some participants commented that the reminder function of the device established an exercise routine for them. A female participant (P5) shared, “It is like a reminder to me, as people are lazy like you paid to do exercise…. like attending a class… it gives the motivation for me to do [it].” Another female participant (P9) also mentioned that she could establish an exercise routine using the proposed device during the trial and had the autonomy to exercise independently without waiting for her weekly physiotherapy visit.

Perceived physical improvements. Most participants reported physical improvements in their hemiplegic limbs after using the wristwatch, such as increased upper and lower limb strength and reduced upper limb pain. One participant (P7) reported improvement in his shoulder strength, whereas another female participant (P11) indicated that after several exercises, she experienced reduced pain when she lifted her arm, “in the beginning, when I lifted my arm, I felt a bit [of] pain like the tendon stretched. Then after that, it is better.”

(b) Effort expectancy. The effort expectancy theme/construct is defined as the degree of ease the user associates with using the device.^44,47 All participants unanimously shared that the device was user-friendly and easy to use. In addition, some participants found it convenient to use because it allowed them flexibility in the location and time when they used it. Nevertheless, for this construct, the participants highlighted two main challenges: technical issues and the wearability of the device.

Ease and convenience of use. The study participants viewed the wristwatch's user interface design as simple and easy to learn. According to the participants, the exercise videos were easy to follow, and operating the watch and telerehabilitation system was simple. Several participants used terms such as “easy to use,” “user friendly,” “easy to handle,” and “not very complicated” to describe their experiences in using the device. Furthermore, some participants appreciated the device's convenience in terms of the flexibility to use it at a preferred location and time. Given the high ownership of smartphones globally,⁵⁶ one unique feature of this device is its ability to integrate with the smartphone to offer a telerehabilitation experience. The portability of the phone allowed the participants to use the device at their preferred location with minimum set-up required. A male participant (P7) highlighted this aspect, saying that the “phone is very convenient, everyone has a phone, the phone is easy to operate.”

Wearability issues. Gemperle and his colleagues⁵⁷ described wearability as the interaction between a wearable device and the human body. All participants stressed that the straps of the proposed wristwatch could be improved by changing the straps’ material and attachment methods. Donning the watch was difficult for most participants, especially one-handedly, and two participants required assistance from their caregivers. For instance, female participants P3 & P5 commented that: “the strap is too thick, it is hard to wear, single hand it was difficult to wear” (P3); “for the strap, I think the hole too small, need to thread it. Basically, we all are one-handed ….if one of my hands has [a] problem, threading it will be difficult.” Some participants also reported discomfort in wearing the watch for very long in the hot and humid weather due to the nonbreathable strap material. Another wearability issue the participants raised was the wristwatch size, which several felt was too bulky. “It is very bulky if you go out, others will notice it and wonder what it is?” commented a participant (P11). Participants expressed that the watch's size affected their intention to wear it outdoors, as it might appear too obtrusive and attract unnecessary attention.

Technical issues. Nearly all participants reported facing minor technical issues when using the proposed watch. Common technical problems included system hangs, short battery lifespan, and slow response time from the system. One male participant (P1) reported a delayed response from the telerehabilitation screen as he moved his arm with the wearable sensors. Nevertheless, all participants could troubleshoot and resolve these issues independently or with assistance from others (i.e. their caregivers and the researchers). None required a replacement of the device during the trial period.

(c) Social influence. The social influence construct refers to the degree to which an individual perceives that their significant others’ views on the device will influence their usage.⁴⁷ The study participants held differing opinions on the importance of their families’ and friends’ views in influencing their use of the device. Slightly more than half of the participants did not consider the opinions of their families and friends to be significant in affecting their intention to use the proposed wristwatch. One male participant (P10) said: “…not really (refers to family members’ view), you want to try on your own. So no difference.” In contrast, other participants valued the opinions of their family members and felt that family support was essential to encourage them to use it.

(d) Facilitating conditions. The facilitating conditions construct embodied three different constructs: perceived behavioral control, facilitating conditions (i.e. objective factors in the environment), and compatibility.^44,47 These constructs captured the users’ perceptions of their ability to perform the behavior and to measure the degree to which the system aligned with their existing values, previous experience, and current needs.⁴⁷ Four subthemes emerged from this construct: physical location, motivational factor, flexibility in exercise schedule, and exercise content.

Physical location. All participants preferred to use the proposed wristwatch at home, which they perceived as a conducive environment because it provided them with privacy and physical space to perform their exercises. Some expressed hesitancy to wear the device outdoors, as they were concerned about others’ perceptions, primarily due to its obtrusiveness and aesthetics and the lack of a suitable place. For instance, this reservation was expressed by a female participant (P3): “if I bring outside and do [it], I [am] scared it will scare my friends and relatives… it will beep … then others may not understand what happened.” While the home environment offered a conducive place for the participants, their use of the device and exercise compliance were limited by their duration at home. Participants who had several outdoor activities in their daily routine expressed difficulty finding time to do the exercises.

Motivation to exercise. Several participants highlighted a “laziness tendency” within themselves regarding the construct of perceived behavioral control, and they perhaps were not self-disciplined enough to perform the exercises if left alone. However, most participants stressed that the wristwatch's remote monitoring and reminder features motivated them to exercise. The telerehabilitation system connected to the watch allowed the therapist to remotely monitor the participants’ progress. This function was highly valued by the participants and created a source of motivation for them. One female participant (P2) pointed out, “you can check from your computer my progress. If I am too lazy, you can call me, ‘why you didn’t do?’… this gives the patient encouragement….” Another male participant (P7) also emphasized: “I think it is good to us patients, it is useful, it motivates me and monitors because people are lazy… sometime we don’t want to do ….” Moreover, some participants appreciated the device's reminder function and were motivated to exercise. As one female participant (P4) mentioned, she felt compelled to do the exercises when reminded by the wristwatch: “initially I don’t bother [to do] it…. but it is very persistent in calling me to do so I do …If there is no watch, my motivation to do exercise will not be great….”

Despite these features, some participants still lacked the motivation to exercise. One participant (P6) attributed his lack of motivation to his character and recommended a reward system using music or a point system as an external motivator to celebrate a user's success if he had achieved the desired goal.

Flexibility in exercise schedule. Some participants perceived flexibility in their exercise schedule to be a necessity. They preferred to exercise at their preferred time, which might not always coincide with their agreed-upon schedule with the therapist Although the proposed device allowed such flexibility, the device's reminder system created some restrictions because the alarm would ring at the previously scheduled time set by the therapist Unfortunately, the participants did not have access to adjust the reminder times, so instead, they did not use the reminder function and switched on the device only when they wanted to use it.

Exercise content. Some participants requested more variety of exercise content in the telerehabilitation system in order to sustain their long-term engagement. Whenever they made physical improvements, they expected the difficulty level of the exercises to be adapted to their new abilities to provide the “right amount of challenge.”

(e) Intention to use. When asked about their intention to use the device in the future, all participants unanimously indicated their keenness to use it. Nevertheless, to facilitate long-term usage, some participants requested an interim review with the therapist on their progress and the opportunity to adapt the prescribed exercises throughout the intervention. In their reviews with the therapist, they wanted to discuss adjusting the device's external prompts and adapting the exercises’ difficulty level according to their upper limb's progress. In addition, some wished to see their progress and self-monitor it from the telerehabilitation system. With that ability, they could feel even more motivated to pursue continuous improvement.

Suggested changes to the device

This study adopted a human-centered design approach by focusing on the user's needs and requirements.⁵⁸ Besides exploring the device's usability and the users’ experiences with the device, we sought to identify the features of the proposed device that required modification. During the interviews, the participants were asked to suggest any changes they would like to recommend for the device. The top five changes proposed by the participants are: (a) changing to the strap, (b) reducing the size of the device, (c) more exercise variety, (d) reducing the weight of the device, and (e) resolving technical issues.

The suggested changes were consistent with the challenges highlighted by the participants. The participants emphasized the need to improve the wearability of the device in terms of the straps and size. Participants wanted the proposed wristwatch to be easy to wear and more comfortable (i.e. with a more breathable strap material). Some referred to their existing watches, which used silicone straps, as an example. In addition, the participants desired a more unobtrusive and smaller device to wear outdoors. Finally, to prevent boredom and encourage long-term use, they felt that the exercise content needed to be varied and graded in terms of the level of challenge.

Discussion

This study explored the usability of a wristwatch to provide home-based upper limb telerehabilitation for persons with stroke. The study's quantitative and qualitative results demonstrated a positive experience using the proposed wearable device. The participant's high SUS scores and compliance rates implied that the proposed wearable device and telerehabilitation program were well received. In addition, the quantitative results were congruent with the participants’ qualitative accounts, as highlighted in the subthemes, such as the perceived usefulness and ease of use of the device and the participants’ perceived physical improvements and enhanced motivation to exercise. Furthermore, despite the technical and wearability challenges, all participants unanimously indicated their intention to use the device again. This finding echoed the observation made by Keogh et al.³² that participants were willing to accept minor annoyances in a device that they perceived to be useful and beneficial.

This study proposes three possible explanations for the excellent usability scores and positive user experiences with the proposed device. First, the actual experience of using the wearable device appeared to debunk any preconceived fears concerning the device's efficacy, thus underscoring Kairy and colleagues’⁵⁹ emphasis on the importance of allowing users to use the technology when evaluating it, on the assumption that users who have yet to experience an innovative technology might have an inaccurate view of it.⁶⁰ Indeed, several studies have shown that participants changed their perception of using a home-based technology such as telemedicine more positively after using it.^60–62

Second, despite having faced technical issues when using the device, participants could resolve the problems independently or receive timely support from their carers or the researcher. For instance, some participants contacted the researcher during the trial regarding technical issues, and the researcher advised them accordingly through videos and visuals via text messaging. Ultimately, those issues did not persist in affecting their user experience. Hansen and her colleagues⁶³ highlighted the importance of maintaining a positive user experience in the early adoption phase of a technology system to avoid disappointments with its efficacy and efficiency. One solution was providing timely and personalized support and troubleshooting to resolve potential technical issues that arose with the new system.

Lastly, one reason was that the positive experience was influenced by the effect of Gartner hype cycle. The hype cycle introduced by Gartner Inc., a technology research and consulting firm, explains the general path of user expectations when new technology is introduced.⁶⁴ In the initial process of introducing a new technology, the hype cycle indicates a peak of “inflated expectations,” followed by a “trough of disillusionment” when unrealistic expectations are not met.⁶⁵ This phenomenon might be reflected in a previous study by Hansen, Calhoun, and Smith,⁶³ who explored the views of first-time and experienced users of a telemedicine service. Those researchers found differing patterns of fluctuating high expectations and disillusionment among two groups of participants. First-time users were more likely to shift their expectations of the telemedicine service positively, whereas the experienced users changed their attitudes in a more negative direction.⁶³ Most participants in this study were first-time users of this device, and their positive experience might have reflected a degree of “overenthusiasm,” as indicated in the Gartner hype cycle.

Figure 6 summarizes the participants’ suggested design considerations for the proposed device and program. This study identified four considerations concerning the design of the wearable device: wearable factors, user interface, system performance consistency, and graded exercise content. Wearable factors—those that make the device easy to wear, unobtrusive, and comfortable—are emphasized and supported by previous studies.^11,66 Most persons with stroke have unilateral hemiparesis, so the attachment method of the wearable device needs to support single-handed attachment and wearing, and the strap material should be comfortable enough to encourage wearing the device for a long duration. Furthermore, the device should be unobtrusive to avoid attracting unnecessary public attention and support outdoor use. Finally, the device's user interface should be simple and easy to learn to reduce the cognitive load of persons with stroke and support independent use. Our findings were in accord with those in previous studies—persons with stroke feel that home-based assistive technologies should be simple and easy to use to support self-management.^67,68

Figure 6.

Summary of the participants’ suggested design considerations for the device and program.

One critical challenge faced by our study participants was encountering technical issues when using the proposed device, a frequently highlighted problem in previous studies on using technology-based interventions.^44,59,66,69 Unlike in the clinical setting, where the environment offers all the necessary technical support from professionals,⁶⁹ home-conducted therapy requires the patients and their caregivers to navigate technical challenges firsthand. Thus, designing a device that performs consistently is imperative, as it minimizes the risk that users will lose confidence in its efficacy and abandon the device. The reliability and durability aspects of an assistive technology's performance are also considered significant predictors of its acceptance or abandonment by users.⁷⁰ Repeated functional testing and adequate training in device use are methods to minimize technical issues during device implementation. In addition, timely technical support from the caregiver or researcher is pivotal to maintaining a positive user experience when using home-based technologies.⁶⁹

Last, when reporting on the design of the telerehabilitation system, our study participants underscored the importance of having a variety of exercises with a graded level of difficulty to sustain their engagement in the therapy. Like the participants in a previous study,⁶⁹ our participants expected the prescribed exercises to change according to their hemiplegic upper limb's progress over time.

The study identified three considerations that should be followed when designing the therapy program: having an interim review, including therapist-client interactions, and ensuring flexibility in the exercise schedule. The study participants emphasized the importance of an interim progress review with the therapist because they desired an individualized treatment plan adapted to their own therapy needs. Previous studies also argued that customization of clinical intervention is crucial.^52,71 In addition, therapist-client interactions and communication are critical elements for maintaining the participants’ motivation and engagement in a therapy program, especially in an unsupervised home setting. Our study participants appreciated the remote monitoring from the therapist because it motivated them to continue in the therapy program. This finding echoed the participants in the study by Bernal-Utrera et al.,⁵² who indicated that having someone to care for them and monitor their clinical status was a positive factor in their telerehabilitation program.

Finally, an important advantage of the proposed device over traditional scheduled therapy was its flexibility, with the option to use it at various times and locations. Other studies have also documented the value of such an advantage with a telerehabilitation or telemedicine system.^44,59,69 Nonetheless, one caveat was that the reminder function from the proposed device required a fixed schedule which the participants could not modify on their own. Thus, to mitigate the study's issues, flexibility should be enhanced through regular communication between the therapist and client and should include the client's ability to adjust the reminder setting.

Strengths and limitations of the study

This study adopted a mixed-methods design to obtain a robust understanding of the usability and user experience of a wearable device with telerehabilitation features in persons with stroke. Participants were given at least two weeks to use the proposed device, thereby addressing a limitation from previous usability studies which failed to test their devices beyond 24 h.³² Furthermore, this study gathered end-user feedback that included recommendations for further modifications to the proposed device before conducting extensive clinical testing. This study also included a wide age range of participants (i.e. 31 to 68 years old) with varying levels of upper limb function to broaden our insight into the usability of the proposed device, especially because a previous study had indicated that differences in expectations arose between young users and older ones.⁷² An older-person-focused design might not meet the functionality requirements desired by young users⁷² and balancing the needs and interests of a broader range of users when designing such a device is paramount.⁷³

There were some limitations to this study. Firstly, this was a usability study, further investigation of its effectiveness with a larger sample size using the wearable device for telerehabilitation is recommended. Secondly, all participants were in the chronic phase of stroke, and we did not explore the user experiences of persons with acute or subacute phases. A previous study⁷⁴ indicated that subacute users might respond differently to a proposed treatment than chronic users. In addition, persons in the subacute stage of stroke might have relatively more untried therapy options available than those in the chronic phase and may prefer other interventions over the proposed home-based therapy using wearable technology.⁷⁴ Moreover, subacute and acute stroke patients might have multiple medical and rehabilitation appointments that would increase their busyness and affect their commitment to and compliance with such research.

In addition, it was unclear whether the participants’ positive reports about their experience were influenced by the Gartner hype cycle's effect or were authentic physical improvements the participants experienced. Further research can be conducted to evaluate the clinical effectiveness of this proposed device and objectively assess the participants’ physical progress. Lastly, this study focused on the user experience of the wearable device (i.e. stroke survivors), the viewpoint of the healthcare providers was beyond its scope. Healthcare providers play a crucial role as advocates, hence future studies can be considered to explore their viewpoints.

Conclusions

This study's findings supported the usability of the proposed wearable device and showed that participants were satisfied with using it as a home-based upper limb telerehabilitation intervention. In addition, this study highlighted several fundamental considerations for designing such a wearable device and its accompanying telerehabilitation program for individuals with chronic stroke. These findings will aid developers in considering real challenges from the user's perspective and making improvements before conducting more extensive usability testing.