Usability Testing of a Sensor-Controlled Digital Game to Engage Older Adults with Heart Failure in Physical Activity and Weight Monitoring

Abstract

Background  Poor self-management of heart failure (HF) has contributed to poor health outcomes. Sensor-controlled digital games (SCDGs) integrates data from behavior-tracking sensors to trigger progress, rewards, content, and positive feedback in a digital game to motivate real-time behaviors.

Objectives  To assess the usability of an SCDG prototype over a week of game-playing among 10 older adults with HF in their homes.

Methods  During initial play, participants' SCDG experiences were observed in their homes using a checklist based on the seven-item Serious Game User Evaluator (SeGUE) instrument. After a week of game-playing, participants completed a survey guided by the Intrinsic Motivation Inventory, to provide their perceptions of the SCDG's usability. Qualitative analysis via semistructured interview-derived themes on experiences playing the SCDG, perceptions regarding engaging with the SCDG, and any usability issues encountered.

Results  Ten HF participants (50% women and 50% White) played the SCDG for an average of 6 out of 7 days. Nine found the SCDG to be interesting, satisfying, and easy to play. The average step count over a week was 4,117 steps (range: 967–9,892). Average adherence with weight monitoring was 5.9 days in a week. Qualitative analysis yielded outcomes regarding attitudes toward SCDG, and barriers and facilitators that influenced participants' engagement with the SCDG.

Conclusion  To the best of the authors' knowledge, this usability and feasibility study is the first to report an SCDG designed to improve HF self-management behaviors of older adults in their homes. Future research should consider several issues, such as user profiles, prior game-playing experiences, and network conditions most suitable for connected health interventions for older adults living in the community.

Background and Significance

Heart failure (HF) affects 6 million Americans (2.4%) at an estimated annual cost of $32 billion; its proportion rises to 11% in those who are 80 years or older. 1 2 Poor outcomes of HF are associated with the inability of persons with HF to self-manage. 3 4 A promising approach to HF self-management is offered by enjoyable and easy-to-use digital games, which have the potential to engage individuals in important HF self-management behaviors such as weight monitoring or physical activity. 5 6 7 8 Within digital games on mobile phones, advances in network connectivity (as in the “internet of things”) 9 enable seamless integration of real-time behavior data from behavior-tracking sensor devices. These data provide contextually relevant feedback and activate appealing game features to persuade and reinforce behavior changes 10 related to HF self-management (e.g., by prompting the user to increase steps by 250 or to call the doctor because of weight gain). Kinect sensors and home computer have been employed to capture participation and provide feedback to the game about participants' exercise performance. 11 Also, fitness tracker sensors are increasingly employed to help older adults track physical activity behaviors. 12 13 However, combining commercially available behavior-tracking sensors within a digital game format to leverage the growing usage of smartphones 14 and the current increase in digital game playing among older adults 15 remains a gap. The SCDG's development was informed by a preliminary study where a low-fidelity prototype was used to obtain the perceptions of 15 HF older adults on using digital games and sensors for HF self-management. 16 The objective of this study is to assess the usability of a fully developed SCDG prototype over a week of game playing among older adults with HF living in their homes.

Methods

Game Development Infrastructure

Over 8 months, a team comprising of nurse scientists, game developer, psychologist, and mobile computing researchers conceptualized and developed the SCDG called “Heart Mountain” that could be played on Android or iOS mobile platforms. The goal of the SCDG is to help an older adult avatar climb a mountain while avoiding hospitalization. Within the SCDG, our team built an application programming interface that integrated data from Withings (Withings; Issy-les-Moulineaux, France) Go activity tracker, 17 body + smart weight scale 18 sensors and HealthMate application 19 to trigger progress, rewards, messages, and changes in avatar's health status based on players' real-time physical activity and weight-monitoring behaviors ( Fig. 1 ).

Fig. 1.

Game message based on actual behaviors.

The Go activity tracker 17 was selected owing to its robustness; it can work for 9 months without battery recharging, and it is waterproof. The tracker utilizes a clock schema, which is a familiar interface for older adults, to represent physical activity accomplished by the user daily ( Fig. 2 ).

Fig. 2.

Withings go activity tracker.

Physical activity goals were individualized to the participant by consulting with the participant on their preference, as well as assessment of their walking ability by the nursing research assistant. As the avatar climbs the heart mountain, each step triggers information, problem-solving challenge (e.g., quizzes), mini-game (word puzzle or slot game), or bonus wheel spin for extra points ( Fig. 3 ).

Fig. 3.

Progress in the sensor-controlled digital games (SCDG).

HF self-management educational content from the “Living Well with Heart Failure” booklet 20 and Heart Failure Society of American Guidelines 21 was provided within the SCDG through “bite-sized” chunks to aid those with low literacy levels. Game coins earned by solving in-game challenges helped purchase healthy recipes or accessories for the game avatar. Real-time behaviors were weighted higher than in-game challenges. The avatar can be tailored by gender and skin tone. During SCDG development, initial testing by research team members revealed glitches such as unsuccessful or lengthy login attempts, delayed syncing of behavior data within the game, or not saving the player's progress which were then resolved.

Participants

Research staff provided information on the study to English-speaking adults who were age 55 years or older, diagnosed with HF, and were referred by clinical case managers at cardiac centers in central Texas during Fall 2019. Other eligibility criteria included owning a smartphone, walking unassisted, scoring 4 or above on the Mini-cog, 22 a cognitive screen. Exclusion criteria included severe visual or tactile impairments (e.g., legal blindness or severe arthritis) that would adversely prevent the use of a smartphone. Eligible participants received $35 as an incentive for participating in the study. The University of Texas at Austin Institution Review Board approved this study.

Usability Testing Procedures and Measures

We used convergent parallel mixed methods design 23 to assess the SCDG's usability as is consistent with usability assessment studies. 24 All the research assistants were provided a 3-hour training session comprising of application and sensor device installation, and observing and noting usability observations. Two research assistants visited the participants at their homes, asked them to complete a demographic survey, and installed the SCDG and the Health Mate application 19 on their smartphones. All participants were given the two sensor devices 17 18 to use in their homes. Usability of the game was first assessed with participants playing the SCDG during a 30-minute in-home testing session. During this session, participants were encouraged to think-aloud while demonstrating 37 tasks on playing the SCDG and using the different features. A second research assistant observed and took notes on participant's reactions playing the SCDG on a Microsoft Excel checklist based on the seven-item Serious Game User Evaluator (SeGUE) 25 instrument. SeGUE has two dimensions (system related and user related) created for annotation purposes. 25

Then participants were asked to play the SCDG daily in their homes. A testing period of 1-week enabled participants to become familiar with the SCDG, and uncover challenges and errors in playing the SCDG over multiple days. At the end of the week, participants completed a usability survey, based on the four-item interest/enjoyment subscale of the Intrinsic Motivation Inventory ( α  = 0.8) 26 that assessed interest and inherent pleasure in performing a specific activity. At the same meeting, doctoral nursing research assistants (ASD or CA) with prior qualitative experiences conducted a semistructured interview to elicit elaborate responses on the SCDG's usability. The interview included questions on participants' experiences playing the SCDG, perceptions on most or least helpful SCDG features, barriers faced while playing the SCDG and using the sensors, as well as any suggestions for improvement. Depending on their responses, the participants were asked additional follow-up questions. Usability was assessed in an iterative manner. Problems discovered during assessment with the first five participants were improved and then the improved version was assessed with the remaining five participants.

The SCDG transmitted participants' game playing and sensor data to the research team via cellular data service or home Wi-Fi. To protect participants' privacy, a six-digit unique identification number and dummy e-mail addresses were generated for the profile information required by the HealthMate app. A Google cloud console was used to store behavioral data captured from participants' sensors, as well as data on their engagement with the SCDG and its' features.

Usability Analysis

Descriptive statistics were used to analyze participants' demographics, health, and game-playing behaviors, and responses on the usability assessment surveys. All interviews were digitally audio recorded using Rev software and transcribed. To identify key issues regarding the participants' SCDG usage, thematic analysis was used. 27 Two of the authors (K.R. and C.A.) used Microsoft Word to inductively identify codes from all the transcripts. Initial interrater reliability was 0.875 (63/72). Whenever the two reviewers disagreed on selection of phrases and code assignments, transcripts were reviewed again and discussed until consensus was reached. After consensus discussion, 72 codes were finalized which were then organized using Microsoft Excel into themes and subthemes depending on the context in which the responses were elicited ( Table 1 ). Quantitative data from the usability surveys and qualitative data from the interviews were analyzed separately and mixed at the stage of interpretation to comprehend the SCDG's usability.

Table 1. Participants' interview responses with four themes and 15 subthemes.

Themes	Subthemes	Codes
Match with reality	Bird sounds	9
	Avatar imagery	6
	Icon imagery	2
	Reward system	4
Prior game-playing experiences	Need for instruction sheet	3
	Expectations on game features	3
	Content flow	1
Missed features	Healthy recipes	5
	Change in avatar health status	2
Behavior sensor use	Sensor user interface	9
	Syncing issues	8
	Mismatch with own sensor devices Workaround for sensor use	3 2
Health behaviors	Behavior changes	5
	Own behavior insights	4
	Barriers to engage in behaviors	6
Total		72

Results

Of the 40 potential participants who were approached for the study, 18 (45%) consented to participate of which 8 dropped out of the study before enrollment. Reasons for declined participation included being not interested, moving to a different location, caregiver responsibilities, or lack of time. Usability of the SCDG was tested with 10 participants (50% females and 50% White; Table 2 ). Eight used iPhones, and two used Android phones. Six and five participants reported baseline exercise and weight-monitoring, respectively, according to below recommended guidelines which consists of exercising 4 to 6 times in a week and weighing 7 days in a week. 4 28

Table 2. Participants' characteristics.

Demographics ( n  = 10)	n	%
Age (y)
55–64	4	40
65–74	5	50
75–84	1	10
Sex
Male	5	50
Female	5	50
Race and ethnicity
Hispanic or Latino	1	10
Asian	1	10
African American	4	40
White	4	40
Other	1	10
Highest education level
Some college credit, no degree	5	50
Bachelor's degree or higher	5	50
Marital status
Married or partnered	5	50
Single, widowed, or divorced	5	50
I live in a
Single family home	8	80
Apartment	2	20
How many times have you been hospitalized for heart failure related reasons in the past 12 months?
0	6	60
1–2	4	40
What other health conditions have you been diagnosed with?
Hypertension	4	30
Arthritis	1	10
Diabetes	3	30
Other cardiac conditions	2	20
Neurological conditions	1	10
Depression	2	20
Respiratory conditions	1	10
Autoimmune disorders	1	10
How long have you been diagnosed with heart failure?
Less than a year	2	20
1–4 years	2	20
5–9 years	3	30
10+ years	3	30
Have you played digital games before?
Yes	8	80
No	2	20
If yes, what kind of digital games do you like to play?
Card games	6	60
Word games	4	40
Matching or puzzle games	4	40
Casino games	3	30
Simulation games	2	20
Typically, how often do you exercise in a week?
Never	2	20
Once	2	20
2–3 times	2	20
4–6 times	2	20
Daily	2	20
Typically, how often do you weigh yourself in a week?
0 days	1	10
1–2 days	1	10
3–4 days	3	30
Daily	7	50

In-Home Usability Testing

The most common usability issues identified out of a total of 52 issues during the in-home testing with the first five participants were related to the user interface (UI; n  = 15) and functionality ( n  = 9; Table 3 ). Issues included inability to locate the back button to return back to the main game from screens of other game features ( n  = 4) or missing healthy recipes and other incentives in the SCDG shop ( n  = 5). The font size of the word game was too small for the participants to maneuver the letters easily ( n  = 2). Some participants were unable to install the SCDG application as their phones were not updated with the latest software updates ( n  = 2).

Table 3. In-home usability testing observations noted on the SeGUE instrument.

		Interface			Design		N/A	Total
		Content	Layout/UI	Technical error	Game flow	Functionality
Negative	Annoyed	1	2	1	0	0	0	4	27
	Confused	0	8	0	3	3	1	15
	Frustrated	0	2	3	0	0	3	8
	Unable to continue (fatal)	0	0	0	0	0	0	0
Positive	Pleasantly frustrated	0	0	0	1	0	0	1	19
	Reflecting	0	0	0	1	2	0	3
	Satisfied/excited	3	3	0	1	2	3	12
	Learning	3	0	0	0	0	0	3
Neutral	N/A	0	0	1	0	1	1	3	6
	Suggestion/comment	0	0	0	0	1	2	3
Total		7	15	5	6	9	10	52
		27			15		10

Abbreviations: N/A, not available; SeGUE, seven-item Serious Game User Evaluator; UI, user interface.

In response, we redesigned the back button to be more prominent at all game screens. Healthy recipes and local restaurant suggestions were featured more prominently in the SCDG's Shop. The mini games were adapted to be more age friendly by increasing the font size of the heart health relevant word game. Before beginning application installs, phones were checked for latest software updates. The improved version of the SCDG did not reveal any more errors during in-home testing with the remaining five participants.

1-Week Usability Assessment Results

We will now report on the findings from our mixed methods analysis which includes qualitative themes that influenced participants' engagement with the SCDG as well as the quantitative usability survey results on participants' perceptions of the game and the various game features ( Table 4 ).

Table 4. Usability assessment survey results.

Usability parameters	Agreed n (%)
Satisfied	9 (90)
I found the game Interesting	9 (90)
I found the game easy to play	9 (90)
I found the game enjoyable	9 (90)
Satisfied with game goal of climbing a mountain	8 (80)
Satisfied with avatar's look	4 (40)
Satisfied with game sounds	9 (90)
Satisfied with using sensor devices to earn game points and progress in game	8 (80)
Satisfied with content/information	9 (90)
Satisfied with quiz questions	8 (80)
Satisfied with mini games	6 (60)
Satisfied with recipes in and local restaurant suggestions	3 (30)
Satisfied with buying accessories at the game shop	7 (70)
Satisfied with game messages in response to behaviors	8 (80)
Satisfied with activity tracker	10 (100)
Satisfied with weight scale	9 (90)
This game will help me learn about taking care of my heart.	10 (100)
This game will motivate me to exercise more	10 (100)
This game will motivate me to weight myself daily	10 (100)
I will recommend this game to others with heart failure I prefer playing game to other medium on heart failure	10 (100) 7 (70)

Overall Attitudes toward the SCDG

Overall, 9 of the 10 participants reported that they would recommend the game to others with HF. According to patient 1 (P1), “it was very positive. It took me out of my element. I'm not used to playing games, so I found that comical and enjoyable.” According to P3, “I thought the game was silly in that it was really too basic. But, I got into it and got motivated to walk.”

Match with Reality

Connecting game features with reality influenced participants' engagement with the SCDG. Nine participants liked the sound of birds in the background which reminded them of time they spent by a lake or in a forest. According to P4, “isn't it pleasant? When I'm went to the lake, I just listen to the birds.” On the other hand, the avatar's appearance did not reflect participants' perceptions of themselves. Only 4 of the 10 participants were “satisfied” with the appearance of the game's avatar, that of an older adult. According to P2, “I might be old but I do have energy and whatever, the person looks old.” The casino slot game's familiar imagery seemed insufficient to P2, who said that “I like the slot machine. I found that fun, but I didn't see how it tied in the health.” She suggested that this might be improved with different imagery, “so probably if it had been apples and oranges and bananas.” The word scramble mini game included words related to heart self-management. According to P7, “I liked the word scramble because it made me think, maybe use my mind.”

Participants also suggested awarding points instead of deducting points for selecting healthy recipes in the shop. P3 “didn't like that you had to pay points for recipes and restaurant suggestions—I think downloading those would, you should get points.” Another participant suggested that rewards in real points and dollars or gift coupons would be a better incentive to keep participants interested.

Prior Game-Playing Experience

Nine of the 10 participants found the SCDG interesting, satisfying, and easy and enjoyable to play, but 1 participant (P10) who had not played digital games before reported that she did not understand the game. This participant was among the participants who reported the lack of an instruction sheet as a barrier to engage with the SCDG. P10, recommended “something for me to go by, because when you told me I thought I understood then, but afterwards, my mind went blank.”

Of the seven participants who bought clothes in the game's shop to dress up their avatar, participants who had played other games wanted more options. As P7 said, “Need more choice for dressing—Pokemon Go has 50 or 60 options.” Other participants enjoyed the shop option as according to P3, “I did enjoy dressing my avatar, putting some glasses and hats and stuff. I want to make my avatar look good.”

Most of the participants were satisfied with the SCDG's HF educational content and with quiz questions that the game provided. According to P5, “I liked the questions and the information that I got because it was short and to the point, and I didn't have to read a lot, but it still gave me information, so I like that.” However, according to P1 who had played an HF educational game before, “I think there needed to be more and faster pace of information and questions.”

Missed Features

Only three participants expressed satisfaction with the healthy recipes given as incentives in the game's shop or the game's local restaurant suggestions; a majority of the participants simply missed these game features. Failure to notice the recipes in the game were more common among the first five participants and prompted the participants to suggest that the recipes should be featured more prominently. Also, although the SCDG avatar's health status was designed to reflect each participant's behaviors, participants reported that they did not notice much difference in their health status within the week of playing the game. According to P3, “I was lazy and walked 10% of my steps, my avatar should have blown up like a fat slob the next day. But I didn't even know it changed at all. And then I think on days where I did 300% of my steps, I should have been like a muscled god.”

Behavior Sensor Use

Sensor devices were popular with 9 of the 10 participants expressing satisfaction with the tracker and the weight device. According to P4, “I walk with my arms, I'm swinging them, and I started seeing notches. That's why I said we need to exercise more. I wish I could keep this little tracker.” The participants who disliked the sensor devices were the ones who found mismatches in the readings between the devices that they already owned and the sensor devices. For example, according to P6, “your scale is less friendly than mine. I'm about two pounds heavier, than mine.” According to P7, “There were a couple of times where I thought my steps weren't being recorded when I thought I'd easily hit my 3,000 and also my Fitbit would tell me that I had over 3,000.” However, some participants, on their own initiative, found ways to make the tracker work accurately. According to P3, “if I wore the tracker on my shoe, it recorded all activity including bicycling, all the steps, everything.” One participant who already owned a digital scale that paired with the participant's phones preferred to manually enter the weight data into the game.

Eight participants experienced issues with syncing weight with the game. Similarly five participants reported issues with consistently syncing the tracker with the game, and this adversely impacted their engagement. According to P9, “I couldn't become as engaged with it as possible. I'm very goal oriented to get those three stars. But some days there was no way for me to get the three stars even though I exercise, even though I weighed myself because the application did not report some of the information.”

Health and Game-Playing Behaviors

Participants who reported that they weighed themselves for 5 or more days per week increased from five at baseline to 7 at the end of the week of SCDG playing. Average adherence with weight monitoring was 5.9 days in a week. The 10 participants played the SCDG for an average of 6 out of 7 days. Further information on participants' health and game-playing behaviors are provided in Table 5 .

Table 5. Health and game-playing behaviors ( n  = 10) .

Behavior	Mean (range)	Participants n (%)
Days with weight monitoring	5.9 (1–7)
Attained weight-monitoring goal	–	7 (70)
Physical activity step goal	4,050 (3,000–4,500
Physical activity steps	4,117 (967–9,892)
Days with physical activity	6.2 (4–7)
Days with attaining physical activity goals	3.4 (0–7)
Attained physical activity goals more than 3 days in the week	–	5 (50)
Days with game playing	7
Steps progressed in the game	19 (7–30)
Game points earned	784 (530–1,000)
Used game features	–
Slot game		3 (30)
Word game		6 (60)
Shop		6 (60)
Items bought in the shop	4

According to P8, “I felt compelled to meet those goals every day. I would suspect that would draw attention that my weight suddenly went up quite a bit over a 2- or 3-day period or went down.” There was no change in participants who exercised below recommended guidelines after a week of SCDG game playing. Barriers to engage in behaviors included pain, onset of illness (three participants), and the short duration of 1 week for the usability assessment. According to P2, “I wish it could have been longer because it takes a habit, it really took a couple of days to get used to it. I think 2 weeks or a month would be great. And then you could kind of see yourself if you're working to get somebody excited about changing their habits.” Some participants found that the SCDG revealed their behavior patterns. For example, for P2, “the greatest takeaway was that I don't do enough exercise. I don't walk enough.”

Discussion

Adding to the growing literature on digital game use by older adults, 29 30 this study demonstrated the feasibility of playing a digital game that linked with sensors to provide feedback on real-time HF self-management behaviors by diverse older adults with HF. Regardless of the smartphone platform (i.e., iOS or Android) or of internet connectivity supported by home Wi-Fi or a smartphone hotspot, participants were able to play the SCDG on the basis of their real-time behaviors. For this study, we recruited only participants who already had smartphones, having found in our preliminary study 16 that 85% of participants owned smartphones and did not want to use multiple phones. Therefore, the present study demonstrates that a complex technology such as the SCDG may be ready to be scaled to a large number of older adults living in the community for disease self-management.

Although the participants liked the behavior-tracking sensors, the sensors' effectiveness was limited by disruptions in syncing the sensors' behavior data with the SCDG. For five participants, the steps that they earned in real time did not sync with the game fast enough to motivate their continued engagement in physical activity. The activity tracker could be resynced with the SCDG by asking the participant to open the Healthmate application that the activity tracker was linked to. However for the remaining five participants, syncing their behavior data within the game was not problematic. The weight scale presented connectivity issues too: to improve connectivity, the device company recommends placing the scale close to the user's router. Further studies are needed to identify the home environment and network conditions most suitable for connected health interventions for older adults living in the community.

While participants in prior studies 16 30 have preferred more realistic game characters that reflect their active selves, the participants in our study were not satisfied with the SCDG's active older avatar. Perhaps, game developers need to present a balance in covering the wide spectrum of how aging is perceived; the full crop of white hair for the avatar in our study did not meet that balance! Also, participants' prior experiences with other commercial games influenced their expectations about accessories available within the game. Small chunks of information on HF available in the SCDG's mini games were a popular feature among the older adults, as was found in an earlier study with a casino slot game. 31 However, participants with higher education level were dissatisfied with the pace and level of information provided in the first week of playing the SCDG. While the pace and level of the content in the SCDG was tailored to participants' performance on the quiz questions, content tailoring based on knowledge level could occur before game playing begins to sustain interest of older adults with an SCDG. Together, these observations suggest that user profiles can contribute significantly to tailor SCDG features that best appeal to individual users such as provision of instruction sheet, game accessories, and pace and level of knowledge content.

Limitations

Limitations of this study include small sample size and possibly higher motivation levels of this self-selected sample of participants, than might be found in the general HF population. Short testing duration is a limitation; however, the goal of the study was to assess usability of the SCDG and feasibility of its usage in a home environment, and not to assess the effectiveness of the SCDG on behavior improvement. In a future study, we will conduct a randomized controlled trial of the SCDG versus sensors only over 24 weeks to tease out the effect of playing the SCDG as opposed to the effect of the sensors on critical behavior changes for HF self-management.

Conclusion

To the best of the authors' knowledge, this usability and feasibility study is the first to report an SCDG designed to improve HF self-management behaviors of older adults. Future research should consider several issues, such as user profiles, prior game-playing experiences, and network conditions, suitable for connected health interventions for older adults living in the community.

Clinical Relevance Statements

1. Sensor-controlled digital games (SCDHs) that can integrate real-time behavior data from behavior-tracking sensors into mobile game applications are acceptable to older adults with heart failure living in their homes.

2. The goal of the SCDG is to enable older adult game players to make meaningful connections between game events and real-time HF self-management behaviors, increasing the likelihood that they may retain and apply their newly acquired knowledge, skills, and habits for HF self-management behaviors outside the game and improve their health outcomes.

3. Clinicians can leverage technologies, such as SCDG, to empower older adults with HF to take charge of their health by enabling tracking and monitoring of their healthy behaviors and establishing routines and by providing content on strategies to manage heart failure in their homes.

Multiple Choice Questions

1. How can sensor-controlled digital game help patients with heart failure?

   1. Encourage healthy behaviors

   2. Improve knowledge by providing bite-sized information

   3. Monitor and track behavior routines

   4. All of the above

   Correct Answer: The correct answer is option d.

2. What were the barriers to engage in the sensor-controlled digital game?

   1. Syncing of the sensor devices with the game

   2. Pain

   3. Onset of illness

   4. All of the above

   Correct Answer: The correct answer is option d.

Funding Statement

Funding Research reported here was supported by the National Institute of Nursing Research of the National Institutes of Health under award number: R21NR018229. The content is the sole responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.