Older Adults' Engagement During an Intervention Involving Off-the-Shelf Videogame

Abstract

Objective: The overall goal of our current study was to examine older adults' experience of Flow (i.e., subjective engagement) during the course of a home-based cognitive training program.

Materials and Methods: In this study, participants took part in a home-based training program. They were randomized to one of the two training groups. One group played an off-the-shelf videogame (i.e., Crazy Taxi), and the other group played a brain training game (i.e., Insight). Training consisted of 60 training sessions of 1 hour each, which were completed in 3 months (5 hours a week). After each training session, participants completed a Flow questionnaire to measure their engagement with the training.

Results: The analysis was performed with a linear growth curve model. The results indicate that on average, there was no change in flow for the Insight group between time points. There was no difference between the initial flow status of the Insight group and the Crazy Taxi group. However, the interaction between group membership and time was statistically significant, indicating that the participants in the Crazy Taxi group increased their scores at each week at a rate that was 0.99 larger than those in the Insight group.

Conclusion: The analyses revealed that both groups experienced increase in Flow over the period, but only participants in the Crazy Taxi group significantly improved in Flow. This has long-term implications since we would expect participation to go beyond 12 weeks in a real-world scenario.

Introduction

The use of off-the-shelf videogames to train cognitive skills in older adults has become popular in the past decade. Several studies have investigated the impact of these games on a range of cognitive abilities. For the most part, results are promising and show that older adults can benefit from this type of intervention.^1–3 However, other studies did not find any effect,⁴ suggesting caution when recommending videogame interventions. Along with off-the-shelf videogames, brain training games have also gained popularity among this population, and some studies have started to show their positive effect.^5,6

In the literature, the terms “video games” and “brain training” games are often used interchangeably. They have some important distinctions, however. Specifically, off-the-shelf videogames are often designed for entertainment purposes and usually target the younger generation. In these types of games, players are usually thrust into different scenarios and story lines, which make them quite complex. Off-the-shelf videogame, in this article, refers to games that are designed to the general population and widely available in stores. In contrast, brain training games are usually designed for older adults to train specific cognitive skills. They also use a minitask approach in which the same task or exercise is played repeatedly several times, and there is usually no story line. Brain training, in this article, refers to games that are designed with the intent to train a specific skill and target a specific population. Despite their increasing popularity to train cognitive skills, few studies have investigated older adults' engagement in these two forms of cognitive training; however, if gamers do not enjoy the game, they will not play the game. This is an important variable to be considered in cognitive training studies as more and more game-based interventions have been used in rehabilitation settings and the community.

While many different models have been used to explain media enjoyment,^7,8 in this study, we define engagement using the concept of Flow. Flow is a well-established theory to describe individuals' game experience and has been used in many studies in this area.^9–11 The concept of Flow experience during videogame has also been validated in brain imaging studies.¹²

The Flow theory was proposed by Csikszentmihalyi.¹³ It is a popular theory used to examine engagement in the game. Csikszentmihalyi defines the experience of Flow as a state of being “in the zone.” Specifically, Flow is an optimal psychological state said to occur when people are able to meet the challenges of a given task or activity with appropriate skills and accordingly feel a sense of well-being, mastery, and heightened self-esteem.^14,15 He proposed the following eight necessary conditions for Flow to arise: (1) a task to accomplish, (2) the ability to concentrate on the task, (3) clear task goals, (4) immediate feedback, (5) a sense of control over actions, (6) deep but effortless involvement, (7) loss of concern for self, and (8) an altered sense of time.

There are several characteristics of games that promote Flow experience: They provide both concrete goals and novel challenges; they can be adjusted to the player's skill level and are designed to be adaptive, thereby increasing in difficulty as the player advances. Furthermore, they provide the player with control over the experience: They provide concrete consistent feedback regarding performance and create an immersive experience in a virtual environment.¹⁶

We found two studies in the literature in this area. The first study conducted by Belchior et al.¹⁷ examined older adults' engagement in different forms of cognitive training. Participants were asked to play different games: Medal of Honor (MoH) (i.e., first-person shooter, action game), Tetris (i.e., puzzle game), and the Useful Field of View (i.e., a traditional computer-based visual attention training). The training consisted of playing the game for six sessions of 1.5 hours each. It was found that participants in both videogame conditions experienced higher levels of engagement than the more traditional computer-based intervention.

In another study, Boot et al.⁴ investigated older adults' compliance and engagement with two forms of training platforms: Mario Kart (i.e., a racing game) and a brain training, where players were asked to play different activities emphasizing memory, reaction time, language, and mathematical abilities. Participants were required to play their games five times a week for 3 months for a total of 60 hours of game experience. Participants were given a questionnaire at the end of the study and asked to rate their enjoyment with the game. To measure enjoyment, participants were asked to rate their agreement with the following statements: (1) I found the game I was given to play enjoyable, (2) I found the game I was given to play challenging, (3) I found the game I was given to play frustrating, and (4) I was motivated to perform well on the game I was given to play. Boot et al. found that older adults rated the brain training games as more enjoyable. In this study, participants reported more anticipated benefits from brain training, and their compliance with training was related to perceived benefits.

The difference in the findings between both studies may be due to the mode of training (i.e., laboratory based vs. home based), the amount of hours played (i.e., 9 hours vs. 60 hours), and the manner in which engagement was assessed. Specifically, while the study by Boot used a questionnaire to assess engagement in the game at the end of the study, Belchior et al. assessed engagement at the end of each training session, so each participant had 6 scores. Specifically, the Flow theory was used to measure engagement.

The overall goal of our current study was to examine older adults' experience of Flow (i.e., subjective engagement) during the course of a home-based cognitive training program. The study design was similar to the study by Boot⁴ in that participants were asked to play two games: a driving game (Crazy Taxi), which is similar to the Mario Kart, and a brain training program (Insight). In addition, participants were required to play their games five times a week for 3 months for a total of 60 hours of game experience.

This study was part of a larger randomized clinical trial investigating older adults' performance across a number of cognitive measures subsequent to a home-based cognitive training protocol. We hypothesized that participants who played the off-the-shelf videogame would report higher Flow scores compared to those who played the brain training game. The Crazy Taxi game was explicitly designed to fulfill a leisure and amusement motive, whereas the formal training program might be perceived as more serious and rigorous and thus less enjoyable. One important difference between these two groups is that the Insight group has very different short games (exercises), which are the characteristics of brain training games, while Crazy Taxi has only one story line, which is common in off-the-shelf games, we hypothesized that they would be more enjoyable.

Materials and Methods

Overall procedure

Participants were recruited by means of classified advertisement in the local newspaper, solicitation mailings to members of an institutional review board (IRB)-approved registry of older adults interested in research and to commercially available mailing lists, flyer distribution to local retirement communities, and additional word of mouth from other participants. Participants were then randomized to one of the two training groups (Crazy Taxi or Insight). Training consisted of 60 training sessions of 1 hour each, which were completed in 3 months (5 hours a week). After each training session, participants completed a Flow questionnaire to measure their engagement with the training.

Sample

Thirty-five community-dwelling older adults completed this study (Crazy Taxi, n = 16; Insight, n = 19). These were a subset of the REVIVA (Research to Examine Videogame Intervention for Visual Attention) study (N = 54). Participants not included were control participants who received no contact between pre- and posttests and thus could not provide regular Flow assessments. Others also not included were participants who dropped out and one participant who had missing Flow data. Participants were generally young-old, aged 65–86 years (mean age = 73.2, standard deviation = 5.5), and highly educated (46% had a master or doctorate degree and 17% had a bachelor degree); almost 86% of the sample was white, and 66% was female.

Ethics

All research was approved by the IRB of the University of Florida. Informed consent was obtained for all participants, and the investigation was conducted according to the principles expressed in the Declaration of Helsinki.

Measure

The Flow State Scale (FSS) was used. This scale was developed by Jackson and Marsh¹⁸ using the Csikszentmihalyi¹⁴ concept of Flow. The FSS conceptualizes Flow in nine dimensions: challenge (skill balance), action (awareness merging), clear goals, unambiguous feedback, concentration on task at hand, sense of control, loss of self-consciousness, transformation of time, and autotelic experience. The questionnaire was completed at the end of each of the 60 training sessions and took about 5 minutes to complete. Thus, each participant had a total of 60 Flow scores. The questionnaire used a Likert scale in which participants rated their level of agreement (strongly agree through strongly disagree) with statements, such as “I was very challenged but I believed my skill would allow me to meet the challenge,” “my attention was focused entirely on what I was doing,” “I loved the feeling of that performance and want to capture it again,” and “at times, it almost seemed like things were happening in slow motion.” Scores (for each session) could range from 36 to 180, and higher scores were indicative of greater Flow.

Home-based protocol

Participants were instructed to play their respective games for a total of 60 hours for a period of 3 months (12 weeks). To monitor compliance with the training protocol, participants were asked to complete training logs each time they engaged in gameplay and send the logs weekly to the study office in postage-paid envelopes.

Crazy Taxi is an off-the-shelf driving game. Thus, the key features of the game are rapid navigation through an urban environment and attending to speed and roadway features. The narrative scenario of the game is quite simple and involves picking up passengers and dropping them off at their destination. Each passenger comes with specific task features (i.e., to get to a particular location in a particular amount of time). Points are assessed via summing up cab fare following completed pickup–drop-off sequences, as well as by any additional tips. Fare is determined by level of difficulty—passengers with farther destinations from the pickup location pay a higher rate, while tips are assessed through an estimate of driving skill (e.g., avoidance of collision and speediness of delivery). In this game, higher scores represent improvement in the game.

Insight is a series of computer-based brain training exercises adjusted to each individual's level of performance. There are five exercises that build on one another to improve visual processing. Each subgame begins with an introduction to orient participants to the exercise: (1) Sweep Seeker is designed to speed up visual processing of moving contrast gradients. Participants are required to collapse tiles in a variety of environments via rapid and accurate detection of direction of gradient motion; (2) Bird Safari exercises visual precision by requiring participants to locate a target bird in their peripheral vision, among a set of similar birds, located at various eccentricities of central fixation. Birds are presented briefly, with latency of presentation decreasing to increase level of challenge; (3) Jewel Diver is designed to improve divided attention and is a multiple object tracking game. Participants are required to track moving jewels in the presence of distracters and other visual clutter; (4) Master Gardener is a visuospatial working memory task, which required participants to match sets of briefly exposed butterflies. The task requires fast and accurate visual perception, as well as storage of visual information in working memory; and (5) Road Tour is designed to improve useful field of view and analogous to the traditional useful field of view (UFOV) training paradigm. It requires participants to determine whether a centrally presented vehicle is a car or truck and simultaneously locate a Route 66 sign in the periphery in the presence of other distracting peripheral stimuli. We should note that Insight is now superseded by “Brain HQ” (www.brainhq.com). For four of the subtasks (Sweep Seeker, Master Gardener, Bird Safari, and Road Tour), lower scores, representing faster times, represented improvement in the game. For Jewel Diver, higher scores were better.

Orientation session

Each participant took part in three to four 1-hour individual instructions on gameplay (e.g., use of the requisite equipment), which was administered via a manualized procedure by trained research assistants. Manuals with step-by-step instructions on gameplay were developed for both training conditions. An in-home visit was scheduled once the trainers determined that minimum competency had been reached.

Training apparatus

The videogame training conditions (Crazy Taxi) used a Sony Playstation^® 2 (console model 97060) and a dual shock 2 analog controller (model 97026). It was played on a TV. Insight was played on a computer. Monitor size for both TV and computer varied by subject, but the viewing angle was held constant.

Participants' learning

We also evaluated if participants would improve game performance throughout the training. For Crazy Taxi, measures of learning included number of passengers dropped off and money earned. Participants were asked to play on arcade mode for 3 minutes, and gameplay was recorded for later scoring. For Insight, measures of learning included scores on each of the mini games. For Insight, scores were automatically recorded. Mean group scores were calculated before and after the training.

Analysis

The analysis was performed with a linear growth curve model,¹⁹ which is a multilevel model^20,21 for longitudinal data. In this model, Flow is affected linearly by the passing of time, whether the participant was in the Crazy Taxi or Insight group, and the interaction between time and group assignment. The model also contains a random intercept and a random slope of time for each participant. Eq. (1) shows the model used:

In this model, y_ti is the Flow status of individual i in time t, T_i is a time indicator, with zero indicating the first measurement, G_i is a dummy indicator of group assignment, with zero indicating the Insight group and one indicating the Crazy Taxi group. The interpretation of the fixed effects is as follows: the intercept β₀ is the mean of the initial Flow status for the Insight group, the slope β₁ is the mean change in Flow with each week for the Insight group, the main effect β₂ is the mean difference in Flow status between the Insight group and the Crazy Taxi group, and the interaction effect β₃ is the mean difference in Flow change per week between the two groups. The random individual intercept is r_0i, and the random individual slope is r_1i. The residual error is e_it. The covariance matrix for r_0i and r_1i was diagonal, meaning that they were specified as uncorrelated. The covariance matrix for e_it was first-order autoregressive (i.e., AR1).²²

Results

Participants' learning

Crazy Taxi

On average, participants dropped off four more passengers at posttest and earned about $1453 more than pretest.

Insight

In this training, lower scores represent faster times, except for Jewel Diver exercise. On average, participants dropped 21 points on Sweep Seeker, 157 points on Bird Safari, 139 points on Master Gardener, and 466 points on Road Tour and improved 1 point for Jewel Diver.

Flow State Scale

The estimates for the fixed effects are shown in Table 1. The results indicate that the mean Flow for participants in the Insight group was β₀ = 119.17 (standard error [SE] = 3.61, P < 0.05) at the beginning of the study, but time was not significant (β₁ = 0.46, SE = 0.26, P = 0.085), indicating that on average, there was no change in Flow for the Insight group between time points. There was no difference between the initial Flow status of the Insight group and the Crazy Taxi group (β₂ = −9.85, SE = 5.35, P = 0.07). However, the interaction between group membership and time was statistically significant (β₃ = 0.99, SE = 0.38, P = 0.014), indicating that the participants in the Crazy Taxi group increased their scores at each week at a rate that was 0.99 larger than those in the Insight group. Given that for the Insight group there was no significant change over time, we can conclude that the Crazy Taxi group experienced a mean increase in Flow of 0.99 per week, which is illustrated in Figure 1.

Table 1.

Fixed-Effect Parameter Estimates

						95% confidence interval
Parameter	Estimate	SE	df	t	Significance	Lower bound	Upper bound
Intercept	119.17	3.61	35.09	33.02	0.00	111.84	126.49
Group	−9.84	5.35	35.35	−1.84	0.07	−20.70	1.01
Time	0.46	0.26	32.71	1.78	0.09	−0.07	0.98
Time × Group	0.99	0.38	32.66	2.60	0.01	0.22	1.76

SE, standard error.

FIG. 1.

Change in Flow scores throughout the 12 weeks.

The estimates of variances and covariances of random effects are shown in Table 2. The estimated variance of the intercept var(r_0i) = 220.19 indicates that there was substantial variability of the initial Flow status between participants, but the small variance of the slope var(r_1i) = 0.59 indicates that there was little variability in change in Flow with each week of participation. The existence of group differences in rate of Flow improvement, coupled with no remaining random variance in rate of change, suggests that the group difference explained essentially all the reliable variances in Flow improvement. The first-order autoregressive parameter estimate ρ = 0.51 (SE = 0.08, P < 0.05) indicates that there was a substantial correlation between residuals of adjacent time points, which can be interpreted as showing that participants who had higher Flow at one measurement also tended to have higher Flow at the next measurements.

Table 2.

Estimates of Random Effects

					95% confidence interval
Parameter	Estimate	SE	Wald Z	Significance	Lower bound	Upper bound
var(eit)	43.06	6.80	6.33	0.00	31.59	58.69
AR1 ρ	0.51	0.08	6.48	0.00	0.34	0.64
var(r0i)	220.19	58.92	3.74	0.00	130.33	372.00
var(r1i)	0.59	0.31	1.89	0.06	0.21	1.68

Discussion

The aim of this study was to assess whether there were any group differences in the pattern of change in Flow throughout the 12 weeks of training. The analyses revealed that both groups experienced increase in Flow over the period, but only participants in the Crazy Taxi group experienced significant engagement improvement.

The group difference in the pattern of Flow may be a reflection of differences in the training experience. Insight had very clear goals and rules from the beginning. There were five minigames that changed in difficulty level from trial to trial, but the complexity remained the same throughout the training. In this group, participants maintained their level of Flow throughout the training. For Crazy Taxi, the significant difference between the first session and later sessions may have reflected the initial difficulty of learning the game. For instance, Crazy Taxi requires more demanding motor skills, which were not required in Insight. Participants experienced some frustration in the beginning with the controller used to steer the car. In addition, the game was a more complex game than Insight, and it had a story line that participants needed to follow.

We can explain the significant improvement in Flow score for Crazy Taxi based on Csikszentmihalyi's Flow theory. Despite the initial challenges with the game, participants in the Crazy Taxi group experienced an average of 1-point increase in Flow each week, meaning that the gains were gradual and incremental. This suggests a growing sense of mastery of the game: As soon as participants mastered one level, the game became more challenging, demanding greater skills, which is one of the characteristics of Flow. We believe that our training manual also contributed to the successes in gameplay. Specifically, our training manual for Crazy Taxi had step-by-step instructions on gameplay, and the challenges of the game increased week by week. This was clearly demonstrated in the “Results” section.

The results are also encouraging regarding older adults' continued motivation for, and subjective experience with, ongoing mental exercises. Here, we demonstrated that while both groups experienced engagement with the training approaches, only the Crazy Taxi group demonstrated significant engagement improvement over the course of each week. This has long-term implications since we would expect participation to go beyond 12 weeks in a real-world scenario. Enjoyment in the game is an important aspect of cognitive enhancement in older adults due to the abilities of older adults to sustain these activities indefinitely.²³ Additionally, enjoyment in the game might also predict better compliance, adherence, and retention. Even though these issues were not investigated in this study, it is an important topic to be considered in future studies.

Last, this study has important implications for the design of games used for intervention with older adults. It demonstrates that even complex videogames, in this case a game that was developed for the younger generation using a PlayStation device, can elicit engagement in older adults. However, it is important to provide participants with very detailed step-by-step instructions from the very beginning of the gameplay, which were accomplished in this study by the detailed orientation and training manual developed for this specific purpose. During the orientation session, participants were asked to practice very specific maneuvers and only progressed through the game once a specific maneuver had been mastered. For instance, they were not taught how to brake the car until they felt a certain level of confidence maneuvering the car through an open virtual space. Anecdotally, it was noted that the level of confidence of participants increased after each orientation session.

Limitation

This study was designed to analyze Flow as a unified construct; thus, our results are based on the Flow composite score. Future studies should investigate if specific Flow dimensions may have a greater impact on engagement in training programs. Last, as a caveat, although we have suggested that the increases in flow/engagement are positive, it is impossible to rule out that increases represent regression to the mean for persons who started out with low engagement.

Acknowledgments

This research was supported by the Robert Wood Johnson Foundation (RWJF) grant 64441 to P.B. P.B. work on the article was also supported by a Supplement to Enhance Diversity grant U01-AG-014276-S1 to M.M. K.T. work on the article was supported by the National Institute on Aging under grant T32 AG020499. Additional study support in the form of software licenses was provided by PostiScience (San Francisco, CA; www.brainhq.com). The funding sources had no involvement in research design, data collection, or data analysis.

Author Disclosure Statement

No competing financial interests exist.