Selling points: What cognitive abilities are tapped by casual video games?

Abstract

The idea that video games or computer-based applications can improve cognitive function has led to a proliferation of programs claiming to “train the brain.” However, there is often little scientific basis in the development of commercial training programs, and many research-based programs yield inconsistent or weak results. In this study, we sought to better understand the nature of cognitive abilities tapped by casual video games and thus reflect on their potential as a training tool. A moderately large sample of participants (n=209) played 20 web-based casual games and performed a battery of cognitive tasks. We used cognitive task analysis and multivariate statistical techniques to characterize the relationships between performance metrics. We validated the cognitive abilities measured in the task battery, examined a task analysis-based categorization of the casual games, and then characterized the relationship between game and task performance. We found that games categorized to tap working memory and reasoning were robustly related to performance on working memory and fluid intelligence tasks, with fluid intelligence best predicting scores on working memory and reasoning games. We discuss these results in the context of overlap in cognitive processes engaged by the cognitive tasks and casual games, and within the context of assessing near and far transfer. While this is not a training study, these findings provide a methodology to assess the validity of using certain games as training and assessment devices for specific cognitive abilities, and shed light on the mixed transfer results in the computer-based training literature. Moreover, the results can inform design of a more theoretically-driven and methodologically-sound cognitive training program.

1. Introduction

During the last decade, the idea that video games can provide a cognitive benefit to those who play them has gained traction and led to a rapid proliferation of applications designed to “train the brain” and attract non-traditional gamers to the gaming community (see http://www.sharpbrains.com). However, many of these commercial programs are not based on reliable scientific research. Research showing that cognitive training protocols can improve visual attention, inhibition or conflict-related attention, working memory and reasoning occasionally show improvements limited to the trained tasks but rarely to broader abilities (Ackerman, Kanfer, & Calderwood, 2010; Ball et al., 2002; Boot et al., 2010; Boot, Kramer, Simons, Fabiani, & Gratton, 2008; Green & Bavelier, 2003, 2006, 2007; Lee et al., 2012; Mackey, Hill, Stone, & Bunge, 2011; Owen et al., 2010; Willis & Schaie, 1986; Willis et al., 2006). Many such programs also suffer from methodological problems (see Boot, Blakely, & Simons, 2011) and replication failures (Chooi & Thompson, 2012; Redick et al., 2012; Shipstead, Redick, & Engle, 2012). In addition, the finite set of training programs often employed in training studies limits continued progress or efficacy; computer programs or games employed are often built from scratch by the researchers, a process that can be time-consuming and resource-expensive. A less professional or visually appealing interface also limits the ability of some games to engage and potentially motivate users, especially in younger generations and a technology-savvy society that is heavily exposed to the rich visual stimuli used in commercial video games. Scientists can partner with professional game developers to create more research-informed games (e.g. Brain Fitness Program, Posit Science, San Francisco, CA; Cogmed Working Memory Training, Cogmed Systems; Lumosity, Lumos Labs, 2012), although funding and resource concerns often make this approach impractical. As one possible means of overcoming these issues, in the current work we propose an alternative approach that uses existing, widely available games on the web as a toolbox for developing training protocols. The aim of this study was not to test the training and transfer efficacy of these games per se, but to first evaluate the cognitive processes that they recruit. Specifically, we sought to systematically examine the possible overlap in the cognitive processes required for successful game play and successful completion of laboratory-based cognitive assessment tasks. Unlike previous training studies that build “games” from laboratory tasks known to measure specific cognitive abilities, or studies that take off-the-shelf games and conduct intuitive task analyses to assess a game’s validity for training particular cognitive abilities, we employed statistical techniques to validate the cognitive abilities related to game performance.

We selected from a wide variety of “casual games,” which are games that are often catered to non-gamers and involve simple rules that allow for game completion in reasonably short periods of time (e.g., Bejeweled, Solitaire, Minesweeper, etc.). Casual games range in genre and are platform-agnostic, such that they can be played on the Internet, and on most operating systems, game consoles and mobile devices. They are widely available and are typically available at no cost. The Casual Games Association estimates that 200 million people worldwide play casual games via the Internet, with many players over age 30 and female (http://www.casualgamesassociation.org). Although relatively simple, casual games can involve multiple cognitive skills and increasingly challenging levels of performance (i.e. adaptive difficulty based on performance), an important aspect in enhancing training (Brehmer, Westerberg, & Backman, 2012; Holmes, Gathercole, & Dunning, 2009). While intensive action video games have been shown to improve aspects of attention and perception (Green & Bavelier, 2003), little is known about the effect of casual or “mini-games” on these functions and others. Mini-games range from casual video games to games adapted from psychological experiments, and there is a need to better identify useful games for training, as well as tests to assess transfer of training. Mini-games have been developed based on neuropsychological tests of working memory and attention (Owen et al., 2010; Lumosity, Lumos Labs; Jaeggi, Buschkuehl, Jonides, & Shah, 2011). However, since neuropsychological assessments served as templates for game development, outcome measures closely mirrored the games or structure used for training, and as such limit the assessment of “true” transfer to an underlying ability. Similarly, studies that specifically train working memory (WM) and interference control, attention, reasoning and speed of processing, show limited transfer beyond very similar measures and tasks (Ball et al., 2002; Boot et al., 2010; Lee et al., 2012; Owen et al., 2010; Willis & Schaie, 1986; Willis et al., 2006). Many of these training programs attempt to train the same set of processes — which is likely to lead to task specialization. Moreover, training protocols that lack an adaptive component can lead to automatization in task performance.

Recent studies provide motivation for examining mini-games as a means to implement a variety of training via short games in a given training session. A study that trained reasoning and processing speed in children using a variety of games in each training session (computerized, Nintendo-based, individual and group non-computerized games) showed promise in improving the targeted ability (Mackey et al., 2011). Similar to Mackey and colleagues, we hypothesize that maintaining challenge and motivation via “cross-training” will produce maximal gains in the targeted abilities. We also believe that the more diverse nature of the processes tapped and the integration of such processes in relatively more applied situations can engender broader improvement in cognitive skills, and perhaps even to executive function skills crucial to performance in daily life, school and the workplace (Diehl et al., 2005). In one study, Schmiedek, Lövdén, and Lindenberger (2010) found that training on a variety of perceptual speed, working memory and episodic memory tasks resulted in gains not only in the trained cognitive abilities, but also in a latent factor for fluid intelligence. Additionally, compared to games based on psychological tasks, casual video games are more likely to engage individuals, which has implications for efficacy and adherence to cognitive training programs.

In this study, we did not attempt to test the effectiveness of the casual games for training different constructs, but to first examine how the games relate to abilities that are often targeted for training, such as executive function. As executive function relates to a broad set of abilities (Miyake et al., 2000; Salthouse, 2005), in this study we examine cognitive constructs of fluid intelligence, reasoning, working memory and various types of attention. The breadth of relationships examined in this study can be useful in evaluating results from a training program derived from this set of games or similar paradigms. Because the training games and assessments differ substantially in context and task-specific characteristics, one can better infer transfer to the targeted ability. Moreover, insight into other cognitive abilities related to game performance (in addition to the primary cognitive ability targeted at initial game play) provides a framework in which to interpret the breadth of transfer.

In the current project, we administered casual games in a controlled setting. In this first validation phase, we used factor analytic and correlation techniques to shed light on the nature of the abilities that are emphasized in each game. In order to measure perceptual and cognitive performance, we chose well-normed laboratory tasks that measure fluid intelligence, perceptual speed, episodic memory and vocabulary (Salthouse, 2004, 2005, 2010; Salthouse & Ferrer-Caja, 2003), and cognitive tasks that measure additional executive control abilities such as various aspects of attention, inhibition, working memory and task switching. We then selected casual games from categories on the Cognitive Media website (www.cognitiveme.com): executive function and reasoning, working memory, attention and perceptual speed. These groupings were informed by a cognitive task analysis that mapped the specific tasks required for game play to the cognitive abilities that these tasks presumably engage (see Militello & Hutton, 1998 for a review of such an approach). We measured game and task performance from 219 subjects in order to provide sufficient power for us to examine the abilities tapped by these games and assess the validity of using repeated game play to exercise certain cognitive abilities. Importantly, our results highlight the relative importance of different cognitive abilities in the games and as such help shed light on the mechanisms that may develop over training.

2. Methods

2.1. Participants

219 participants (ages 18–30) were recruited from the Champaign–Urbana community and were paid $10/h for all sessions. To encourage completion of the study, participants were informed that if they discontinued participation before the last session, payment would be $5/h instead. Three subjects were disqualified after the first testing session due to participation in a game training study (Space Fortress) that used a subset of similar assessment tasks. Seven subjects dropped out at different points of the study and their data was included in the separate analyses of tasks and games. However, data from these seven subjects was not included in the combined task and game analyses, resulting in a final combined sample of 209 participants (33% male; mean age=21.68, SD=2.9; mean years of education=14.91, SD=1.92). Game data for one or two sessions was not collected for 33 individuals due to technical recording errors or experimental error. Listwise and pairwise exclusion analyses were performed accordingly to account for the missing data. Descriptive statistics of all measures can be found in Appendix A.

Recruitment was conducted through flyers posted in campus buildings and businesses, and through advertisements posted to online bulletin boards and community newspapers. Study requirements were stated as completion of paper–pencil and computer-based games and tests. Individuals responding to these postings were then asked to complete a demographics form and a survey of their video game habits, and to return this information via e-mail. Active video game players were excluded from the study to minimize the influence of previous game experience or expertise on performance metrics. If individuals reported playing more than 10 h of games per week (any game type: card, video, computer) and reported major medical or psychological illnesses, they were excluded from the study. The effect of gaming experience was not of primary interest in this paper, but will be analyzed in a separate study along with other demographic factors. Pre-screened participants were phoned and interviewed regarding medical conditions and medication. Qualified individuals were then invited to the lab to complete an interview. The interview assessed vision status and detailed the requirements for the study. Follow-up questions regarding game habits, illness and medication were conducted as necessary. All participants were fluent English speakers, had normal or corrected-to-normal vision, normal color vision and were right-handed. All participants provided informed consent. The University of Illinois Institutional Review Board approved the study.

2.2. Apparatus

All computer-based cognitive tests were programmed in E-prime (Psychology Software Tools, Pittsburgh, PA) and administered using PC computers with 17″CRT monitors. Game data was collected on networked PC computers and game inputs were made using the computer mouse or a keyboard. All games were played on color 19″LCD monitors. Games were displayed using the Mozilla Firefox browser and were played from a research portal (http://research.cognitiveme.com) designed for this study. Only researchers had access to subject login information, and participants were not informed of the games to be played in succeeding sessions.

2.3. Procedure

Fig. 1 summarizes the experiment protocol. After qualifying for the study, participants returned to the lab for three sessions of cognitive testing and five sessions of game play, with each session lasting 1 to 2 h each. The average time elapsed between the first testing session and the last game session is 17.56 days, with a median of 16 days. Although we are unable to determine whether participants played the study games outside the laboratory, several factors make this less of a concern. Game play is not introduced until the fourth session and there is a relatively short interval between this first gaming session and the last game session. Moreover, each training session included games from different categories.

Fig. 1.

Phase 1 study protocol. Participants completed a battery of tasks from different cognitive domains, followed by 5 sessions of game play using games from a subset of similar cognitive domains.

Participants played 20 casual games, each of which were freely available via the web, over the course of five sessions that took place on different days no more than a week apart. Each session consisted of four games completed in a fixed order, with 20 min of playing time devoted to each game. Participants were allowed to take breaks as needed. All games except for Memotri contained varying levels of difficulty that were adjusted adaptively based upon performance. None of the games contained violent content. After each game session, subjects were asked to rate how much they enjoyed playing the games on a scale of 1–10 (1 least liked to 10 most liked) and to provide a short explanation for their rating.

2.3.1. Cognitive task battery

Tasks from the first cognitive testing session were taken directly from the Virginia Cognitive Aging Project and were designed to test the following abilities: fluid intelligence, spatial reasoning, perceptual speed, episodic memory, and vocabulary (Table 1; Salthouse, 2004, 2005, 2010; Salthouse & Ferrer-Caja, 2003). Only the Paired Associates task was modified, such that participants typed their responses instead of verbally issuing them to the experimenter. Listed below are the details of tasks completed during the second cognitive testing session, as different versions of these tasks have been used in other studies. These included tasks designed to measure aspects of executive control not addressed in the first task battery. They include shifting (task switching, trail making), working and short-term memory (visual short-term memory, n-back, spatial working memory, forward and backward digit span), and various tasks of attentional control (attention network test, Stroop, attentional blink).

Table 1.

Tasks used in the first session of cognitive testing.

Task	Construct	Description	Administration	Source
Matrix reasoning	Fluid intelligence	Select pattern that best completes the missing cell in a matrix	Computer-based	Raven (1962)
Shipley abstraction	Fluid intelligence	Determine the letters, words, or numbers that best complete a progressive sequence	Paper–pencil	Zachary (1986)
Letter sets	Fluid intelligence	Identify which of five groups of letters is different from the others	Computer-based	Ekstrom, French, Harman and Dermen (1976)
Spatial relations	Spatial reasoning	Determine which three dimensional object could be constructed by folding the two dimensional object	Computer-based	Bennett, Seashore and Wesman (1997)
Paper folding	Spatial reasoning	Determine the pattern of holes that would result from a sequence of folds and a punch through folded paper	Computer-based	Ekstrom et al. (1976)
Form boards	Spatial reasoning	Determine shapes needed to fill in a space	Computer-based	Ekstrom et al. (1976)
Digit symbol	Perceptual speed	Use a code table to write the correct symbol below each digit	Paper–pencil	Wechsler (1997a)
Letter & pattern comparison	Perceptual speed	Same or different comparison of pairs of letter strings/patterns	Paper–pencil	Salthouse and Babcock (1991)
Logical memory	Episodic memory	Recall as many idea units as possible from three stories	Computer-based/paper–pencil	Wechsler (1997b)
Free recall	Episodic memory	Recall as many words as possible across four word trial lists	Computer-based/paper–pencil	Wechsler (1997b)
Paired associates	Episodic memory	Recall the second words from word pairs	Computer-based/paper–pencil	Salthouse, Fristoe and Rhee (1996)
WAIS vocabulary	Vocabulary	Define words out loud	Experimenter/paper–pencil	Wechsler (1997a)
Picture vocabulary	Vocabulary	Name the objects presented	Experimenter/paper–pencil	Woodcock and Johnson (1990)
Synonym & antonym	Vocabulary	Choose the word most similar/opposite in meaning to the target	Computer-based	Salthouse (1993)

2.3.1.1. Task switching

Participants completed a task that required them to switch between judging whether a number (1, 2, 3, 4, 6, 7, 8, or 9) was odd or even, and judging whether it was low or high (i.e., smaller or larger than 5). Numbers were presented individually for 1500 ms against a pink or blue background at the center of the screen, with the constraint that the same number did not appear twice in succession. If the background was blue, participants were instructed to report as quickly as possible whether the letter was high (by pressing the X key using the left index finger) or low (by pressing the Z key using the left middle finger). If the background was pink, they were to indicate whether the number was odd (by pressing the N key using the right index finger) or even (by pressing the M key using the right middle finger). Participants completed four single task blocks (2 blocks of odd–even and 2 blocks of high–low) of 30 trials each. They then completed a practice dual-task block in which they switched from one task to the other every five trials for 30 trials. Finally, they completed a dual-task block of 160 trials, during which the task for each trial was chosen randomly. This task is similar to that of Kramer, Hahn, and Gopher (1999) and Pashler (2000). The primary measure in this task is switch cost during the dual-task blocks, calculated by subtracting the reaction times for repeat trials from the reaction times for switch trials. We also used global switch cost, taken by subtracting reaction times during the single-task blocks from reaction times in the mixed or switching task blocks. Accuracy measures of both switch costs were also taken and composite measures taking into account both accuracy and reaction time were used in the final analyses.

2.3.1.2. Trail making (Reitan, 1958)

Participants were presented with 25 numbers distributed over a sheet of paper. The task was to connect the number targets or draw a line from one number to the next number in an ascending pattern, without lifting the pencil from the paper. In the second part, the sheet contained both digits and letter targets, and participants were required to connect the targets in a similar ascending manner, alternating between digits and letters (1-A-2-B-3-C and so on). Subjects were instructed to complete both tasks as quickly as possible. Switch cost was the primary measure, taken by subtracting part A completion time from part B completion time.

2.3.1.3. Visual short-term memory

In each trial, participants were briefly shown (250 ms) an array of two or four colored shapes. After a 900 ms delay, they were presented with a colored shape and asked to indicate whether this stimulus exactly matched one of the previously presented stimuli in that trial. Color, shape or both varied at each presentation. Accuracy during trials when both color and shape varied was used as the primary measure in analyses. This task was derived from paradigms described by Luck and Vogel (1997).

2.3.1.4. n-Back memory task

As in Kirchner (1958), participants viewed a sequence of centrally presented letters. They were instructed to press a specific key if the letter was the same as the previous letter (1-back task) or the letter presented two items back (two-back task). Another key was used to indicate whether the letter differed from the previous letter or the letter two items back. Each letter appeared for 500 ms with an inter-stimulus interval of 2000 ms. Participants first completed a practice block of 13 one-back trials with feedback, then five blocks of 20 trials without feedback. Participants then completed a practice block of 13 two-back trials with feedback, then five blocks of 20 trials without feedback. Of primary interest was accuracy in the 2-back condition. We also computed the memory load cost: the difference in response time between the 2-back and 1-back conditions. Memory load cost and 2-back accuracy were highly correlated.

2.3.1.5. Spatial working memory

On each trial, participants viewed an arrangement of 1, 2 or 3 dots presented on the screen for 500 ms and are told to remember this array. After a 3000 ms delay, a red probe dot is presented for 2000 ms and participants are asked to indicate whether the location of this probe matched the location of one of the dots previously shown for that trial. The primary measure taken was the sum of accuracy from all the trial conditions.

2.3.1.6. Forward and backward digit span

In each trial, participants were read aloud a list of numbers at a rate of 1 number per second and were asked to immediately repeat the numbers in order. If they did this correctly for the first two trials (beginning with a list of 3 numbers), they moved on to a higher span of 4 digits presented at the same rate of 1 number per second. The maximum list administered contained 9 digits. In the second part, they were asked to repeat the numbers in the reverse order. Each participant’s forward and backward span was the highest list length they can remember completely in forward order and reverse order, respectively.

2.3.1.7. Attention network test (Fan, McCandliss, Sommer, Raz, & Posner, 2002)

Participants were asked to respond to a target arrow on the center of the screen that is sometimes flanked by arrows pointing in the same direction (congruent), arrows pointing in a different direction (incongruent), or by dashes (neutral). Participants were also informed that a warning cue may appear prior to the target, indicating that the target is appearing shortly. Sometimes there was no cue, a center cue, or a spatial cue indicating the location of the target. This task measures several aspects of attention: alerting: responses to center-cued trials vs. no-cue trials, orienting: responses to spatially-cued trials vs. center-cued trials, and conflict or executive attention: responses to incongruent trials vs. congruent or neutral trials. Reaction time metrics of the above measures were used.

2.3.1.8. Color Stroop (Stroop, 1992)

Participants viewed a sequence of words and were tasked to indicate the color of each printed word using an appropriate key press. There were 3 word types: neutral, congruent and incongruent. Neutral words did not spell out a color name, congruent words referred to color words whose ink matched the printed word (the word “red” in red ink), and incongruent words were color words written in an ink of a different color (the word “red” in green ink). Participants were encouraged to respond as quickly and as accurately as possible. The primary metric from this task was a measure of the ability to attend to the relevant dimension (word ink) and override the automatic reading response. The “Stroop effect” was taken by subtracting the reaction times of the congruent condition from that of the incongruent condition.

2.3.1.9. Attention blink task (Raymond, Shapiro, & Arnell, 1992)

Participants viewed a rapidly presented sequence of letters (approximately 1° high) on a gray background at the center of the screen and reported two things about each letter sequence: (1) the identity of the one white letter in the sequence of black letters and (2) whether or not an X was present sometime after the white letter (50% of trials). Each letter appeared for 12 ms, followed by an 84 ms blank interval before the next letter. Letter sequences varied in length from 16 to 22 letters. The white letter appeared unpredictably after either the 7th, 10th, or 13th letter. The X could occur 2, 4, 6, or 8 letters after the first target. Participants often failed to report the X when it appeared soon after the first target (referred to as the “attentional blink”). Participants completed one practice block of 20 trials in which they only had to detect the white letter, and another practice block of 20 trials in which they only had to detect whether or not an X was present. Finally, participants completed 144 test trials in which they had to detect both the white letter and whether or not an X occurred after the white letter. The primary measure was the difference in performance between when the X was the second letter after the white target (when detection is typically worst) and when it was the eighth letter (when detection is typically high).

2.3.2. Task analysis of games

Table 2 provides a brief description of each game and the primary cognitive construct presumably tapped by each. The casual games were grouped into different categories using a cognitive task analysis (Militello & Hutton, 1998). Initially performed by cognitive psychologists at Cognitive Media, the task analysis was re-evaluated and validated by several of the study authors.

Table 2.

Games used in the study, categorized using an expert task analysis.

Game	Group	Description	Measure	Source
Silversphere	Reas	The goal is to enter the blue vortex in each level by moving a silver sphere around a maze. Players plan how to use blocks with different features to create paths to the vortex, while avoiding obstacles.	Max level	miniclip.com
Bloxorz	Reas	The aim is to get a block to fall into a square hole at the end of each stage by rotating and moving the block across platforms of different configurations and features, while avoiding falling off from a platform.	Last level	miniclip.com
Sushi-Go-Round	Reas	Players pretend to be a sushi chef. The goal is to learn the different recipes, serve a certain amount of customers with the correct recipes, clean the tables, order ingredients and appease the customers.	Max score	miniclip.com
Blobs	Reas	The aim of the game is to keep jumping blobs until only one remains. A blob can only be jumped in certain directions and a blob that was jumped over is removed from the board.	Last level	miniclip.com
TwoThree	Reas	The aim of the game is to shoot down rapidly presented numbers by subtracting them exactly down to zero using only units of 2 or 3 and sometimes switching between target numbers to shoot.	Mean points	Armor Games
Memotri	WM	Participants uncovered three cards at a time and had to remember the specific items associated with each, with the goal of identifying all matching sets by uncovering each set in a single trial.	Max points	Platina Games
Simon Says	WM	The aim is to replicate the whole sequence of light and sound conjunction patterns played in each level.	Mean score	neave.com
Memocubes	WM	Players are presented with nine cubes with forms on each surface. The aim of the game is to match forms of the same color and complementary shape by rotating and remembering the location of matching cubes.	Mean score	Platina Games
Round Table	WM	A table is divided in marked sections that each hide a number of marbles. The table rotates at each turn. The aim is to get more marbles than the opponent by remembering which segments that still have marbles left.	Mean score	Platina Games
Oddball	WM	In each trial, the aim is to identify the new ball in the display before time runs out. The display gets increasingly complex as all previous balls remain on the screen.	Mean score	Armor Games
Filler	ATT	Player has to fill 2/3 of the screen by creating filler balls of different size while avoiding bouncing balls.	Max score	kongregate.com
Enigmata	ATT	Players navigate a ship through space. The aim of the game is to gather objects that provide power or armor, destroy opponents using the collected fire or armor, and avoiding enemy fire and debris.	Max score	maxgames.com
Dodge	ATT	The aim of the game is to avoid enemy missiles that are actively chasing the player’s ship and destroy enemies by navigating around the enemies so that their missiles destroy each other.	Max level	Armor Games
Cathode	ATT	Players navigate around a space to trace different forms while avoiding colliding with flickers.	Mean score	Armor Games
Music Catch 2	ATT	The aim of the game is to catch certain shapes appearing on the screen while avoiding red shapes.	Max points	reflexive.com
Digital Switch	PS	Players switch digibot positions to correspond to falling targets and collect coins matching the bot color.	Max points	miniclip.com
Crashdown	PS	Players prevent the wall from reaching the top of the display by clicking on three or more adjacent same-colored bricks to remove them.	Max level	miniclip.com
25 Boxes	PS	Two sets of matrices are presented side by side. Players search for a character in the first matrix and indicate its location on the blank matrix.	Max score	Platina Games
Phage Wars	PS	Players spread their parasites and overtake all other parasites to become the dominant species.	Last level	Armor Games
Alphattack	PS	Players prevent bombs from landing by pressing the characters specified on the approaching bombs.	Max points	miniclip.com

2.3.2.1. Reasoning games

The games in this category, originally named “executive function” on the website, varied in format but primarily contained two types: a) puzzle-type games that involved reasoning and problem-solving and b) those that placed emphasis on time-limited strategizing, task-switching or multi-tasking.

In the puzzle-type games such as Bloxorz and Silverphere, participants navigated around maze-type landscapes to get an object to an endpoint; the configurations and obstacles became increasingly complex as players advanced levels. Participants encountered different types of objects that must be used to solve each level. Identifying the relationships between objects around the landscape and planning moves ahead of time were essential. Such spatial and planning demand was also evident in Blobs, where participants had to figure out in advance the order in which to jump blobs in different locations. In these games, motor control and inhibition also played an important role such that mispressed keys led to collision with enemy pieces or falling off the game space.

In the time-limited switching tasks, participants alternated between responding to different stimuli or demands as they saw fit. Participants had to switch their focus of attention from one task or to another as necessary, to deal with the event that carried the highest value or risk. Participants had to keep in mind a considerable amount of information, such as the recipes in Sushi-Go-Round, and the subtractions in TwoThree. Errors were penalized.

Despite the variability in these games, common to them was an emphasis on reasoning and strategizing which subtasks to perform, how to perform them, and the order in which to perform them. Working memory, inhibition, perceptual speed and attention components were also present in the games, but were secondary elements relative to the reasoning and problem solving components.

2.3.2.2. Working memory games

Working memory games entailed maintaining and updating an increasing load of items in memory while avoiding interfering information.

In Memotri, Memocubes and Roundtable, participants had to hold in mind information as stimuli disappeared from the display, and then manipulate that information to achieve the goal in each game. Like the first three games, Oddball and Simon Says required maintaining an increasingly complex array of information, although these games did not heavily demand manipulation between items.

2.3.2.3. Attention games

Overall, attention games emphasized multiple object tracking and divided attention to time, space or objects. These games also included working memory and reasoning elements, but to a less degree as relevant events remained on-screen.

In all five games, participants navigated around a display containing multiple moving objects. The goal involved obtaining or creating objects at specific locations while avoiding certain objects such as bouncing balls (Filler), enemy ships or fire (Enigmata and Dodge), flickers (Cathode), and red shapes (Music Catch). Because of the proximity of targets to enemy objects in time and space, participants had to divide their attention across the display and learn to respond quickly to certain events.

2.3.2.4. Perceptual speed games

Perceptual speed games contained some aspects of the abilities in the above domains, but did not require a high demand from each. Emphasis was on rapid visual processing and speeded responses to relatively simple stimuli. In all these games, the speed of presentation or complexity of stimuli increased in each level.

2.3.3. Statistical analyses

To better inform our analysis of the relationship between task and game performance, we first validated the task structure in the reference cognitive battery and then examined the categorizations of the games separately. Principal component analysis (PCA) was performed using IBM SPSS Statistics 19.0, while confirmatory factor analysis (CFA) and structural equation modeling (SEM) were performed using Mplus (Muthén & Muthén, 1998–2011). We conducted PCA with varimax rotation and evaluated the resulting components using a scree plot, eigenvalues and prior research. We only present PCA components with eigenvalues greater than 1 (Kaiser, 1958). CFA and SEM models were examined using the following metrics: chi-square goodness-of-fit statistic (χ²; Muthén, du Toit, & Spisic, 1997), comparative fit index (CFI; Bentler, 1990), Tucker–Lewis index (TLI; Tucker & Lewis, 1973), and root-mean-square error of approximation (RMSEA; Steiger, 1990).

2.3.3.1. Total and individual correlations

Scores were normalized and used to create one composite measure for each game component (based on the game PCA) and each cognitive ability factor (based on the cognitive task CFA). Normalized scores were summed with equal weight to create the composite measures. Game composite scores were created from games that had loadings greater than .30 on each component, so there was some overlap in the games used to create the scores. To obtain the general pattern of relationships, we first computed correlations between the cognitive factor scores and the game component scores. Because of the variety of the games that comprised each component, we then took a closer look and computed individual correlations between each primary game and task measure.

2.3.3.2. Contextual analysis

Using the most robust results from the total and individual correlations, we then investigated the unique relations of the different cognitive abilities to the game component scores. This analysis is similar to the method used in Salthouse, Pink, and Tucker-Drob (2008) (also see Salthouse & Ferrer-Caja, 2003) where several cognitive constructs are simultaneously used as predictors for target variables, as relations may be overestimated if other constructs are not included in the analysis. To represent the cognitive abilities as latent constructs and thus also account for measurement error, we used SEM to perform the simultaneous regressions.

2.4. Cognitive battery validation

While a CFA using the pre-defined cognitive constructs did not converge on a solution, PCA with varimax rotation on all the cognitive tasks revealed an 8-component structure. We replicated the findings of the Salthouse studies (Salthouse, 2004, 2005, 2010; Salthouse & Ferrer-Caja, 2003) and found segregation of tasks into the same fluid intelligence, perceptual speed, episodic memory, and vocabulary components. Because the vocabulary tasks were of little interest in this study as none of the games queried verbal knowledge, we did not include these tasks in the subsequent task analyses, leaving us with seven components after re-running the PCA. In addition to the three components already identified in the Salthouse studies (fluid reasoning, speed and episodic memory) we found a coherent working memory component composed of the n-back, spatial working memory and visual short-term memory measures. Despite the relative heterogeneity of the working and short-term memory tasks, the three tasks were highly correlated and loaded highly onto a single component. Global switch cost also loaded highly on this working memory component. Forward or backward digit span did not load highly on this component, perhaps due to the verbal nature of the task as opposed to the other visuospatial memory tasks, or the insufficiency of the digit span (as administered in this study) as a measure of working memory (Unsworth & Engle, 2006, 2007). We re-ran the PCA without the span measures and found a similar 7-factor solution that overall explained about 60% of the variance (Appendix B).

In addition to the four well-defined components, we identified a general visual attention component (alerting, orienting and conflict effects in the ANT), a shifting component (local switch cost, attentional blink effect, trail-making B–A cost) and an inhibition-related component (Stroop effect and ANT conflict effect). In summary, our exploratory analyses revealed four reliable cognitive components consistent with our pre-defined task groupings: fluid reasoning, perceptual speed, episodic memory, and working memory and three other components related to attention: inhibition, shifting and visual attention.

Specifying the PCA 7-factor solution for a CFA using maximum likelihood estimation (with robust standard errors) did not converge, likely due to the lack of coherence in the last three attention-related components. As the term “attention” encompasses a variety of processes including selective attention, inhibition, orienting, engaging, disengaging, shifting and divided attention, with each measure or task designed to extract a slightly different component, we first excluded the attention measures in the CFA. A simplified model using only the tasks with the highest loadings on the first four components of fluid reasoning, perceptual speed, episodic memory and working memory provided an excellent fit (χ² (84)=119.977, p=0.0061; RMSEA=0.045; CFI=0.953; TLI=0.941), with all measures loading significantly to their respective latent factors. The CFA also revealed a strong association between working memory and reasoning (standardized estimate=0.498, p<0.001) and modest relationships between perceptual speed and working memory (standardized estimate=0.296, p=0.002), speed and reasoning (standardized estimate=0.190, p=0.013), and reasoning with episodic memory (standardized estimate=0.193, p=0.021). Not surprisingly, modification indices also suggest cross-loadings between variables across factors.

Building on this 4-solution model, we conducted another CFA with the attention tasks. As a 7-factor and 6-factor model with the visual attention, shifting and inhibition components did not converge, we specified only one general attention factor composed of the ANT measures. This 5-factor model produced a decent fit (χ² (125)=194.325, p=0.0001; RMSEA=0.051; CFI=0.915; TLI=0.896, Fig. 2), albeit a weaker fit compared to the 4-factor model. The pattern of results was similar to the 4-factor model and no latent factors were significantly associated with the visual attention component.

Fig. 2.

Confirmatory factor analysis results for the cognitive task battery. Only significant paths are drawn and only the standardized estimates of the factor relationships are shown. ***, ** and * denote significance at the p<.001, p<.01 and p<.05 levels, respectively. All indicator variables loaded highly onto their respective factors and were significant at the p<.001 level.

3. Results

3.1. Games

CFA using the four pre-defined game groupings did not converge on a solution. While the games did not organize according to the task analysis-based categorizations, an exploratory PCA identified five interpretable game groups.

PCA with varimax rotation on the 20 games revealed a 5-component solution that explained about 54% of the variance (Table 3). The first component contained high loadings from all working memory games and reasoning games. The second component contained high loadings from all reasoning games and some attention games that were spatial in nature, hinting at a spatial reasoning component. The games with the highest loadings on the third component were from attention games that all required quick tracking and responding to multiple objects on the screen. The last two components contained high loadings from Alphattack, Crashdown, 25 Boxes, and Dodge, all games that emphasize perceptual or visuo-motor speed. It is important to note that the games selected for the reasoning category were not homogenous and can be categorized into more working memory or spatial reasoning domains. Indeed, all of the reasoning games loaded highly onto the first two components, with some loading more strongly on the working memory or the spatial reasoning component. The perceptual speed games were the most heterogeneous in nature, and did not form a single component overall. Instead, the pattern for these games were distributed across the other components, with DigiSwitch loading more onto a working memory and shifting component, Phage Wars into the spatial component, and Alphattack and Crashdown into a more visuo-motor speed group. Given the multifaceted nature of casual video games, it is not surprising that they did not fit neatly under single psychological constructs. However, as we suspected, different games do somewhat selectively emphasize different constructs. These results indicate that intuitive, task-based analyses of games may not always be sufficient when selecting games for possible training interventions, and underscore the importance of validating game selection with objective approaches such as those used here.

Table 3.

Game battery: principal components analysis.

	Component					Domain
	1	2	3	4	5
Oddball	.786					Working memory, reasoning
Simon Says	.739					Working memory, reasoning
DigiSwitch	.603		.438			Working memory, reasoning
Roundtable	.568					Working memory, reasoning
Bloxorz	.508	.439				Working memory, reasoning
Memocubes	.477					Working memory, reasoning
Memotri	.468					Working memory, reasoning
Sushi-Go-Round	.440	.399			.335	Working memory, reasoning
Blobs	.422	.383				Working memory, reasoning
Enigmata		.717				Spatial reasoning
Silversphere	.341	.619	.348			Spatial reasoning
Phage Wars		.572				Spatial reasoning
Cathode		.555	.479			Spatial reasoning
Music Catch			.789			Attention: multiple object tracking
Filler			.603			Attention: multiple object tracking
TwoThree	.379	.355	.535			Attention: multiple object tracking
Alphattack				.730		Visuo-motor speed
Crashdown				.601		Visuo-motor speed
25 Boxes					.802	Perceptual speed
Dodge		.341	.406		.487	Perceptual speed

Note. Standardized component loadings from a PCA 5-factor solution. For clarity, only loadings above .30 are displayed. Rotation method: varimax with Kaiser normalization. Rotation converged in 8 iterations. Pairwise exclusion was performed.

3.2. Correlation between tasks and games

Table 4 shows the general pattern of relationships while Fig. 3 illustrates a heatmap summarizing the individual game-by-task correlations. Appendix C shows the values and significance of the individual correlations. Most of the working memory, reasoning, and fluid intelligence correlations were robust at p<.001, and would remain significant after multiple comparison correction. The strongest finding for both analyses was a high correlation between participants’ scores in games that emphasized working memory and reasoning abilities and performance on working memory and fluid intelligence tasks.

Table 4.

Total correlations: game component scores vs cognitive abilities.

		Game components
		Working memory		Spatial reasoning		Attention/object tracking		Visuo-motor speed		Perceptual speed
Working memory	r		.548***		.441***		.413***		.174*		.240**
	n	172		167		181		203		193
Fluid intelligence	r		.650***		.566***		.458***		.270***		.363***
	n	167		161		175		197		186
Perceptual speed	r		.356***		.182*		.283***		.230**		.240**
	n	169		164		178		200		189
Episodic memory	r		0.117		0.004		−0.03		0.027		0.015
	n	167		162		176		199		188
Attention (ANT)	r		0.055		0.126		.185*		0.076		0.058
	n	173		168		182		204		194

Note. Correlations between game component scores and task factor scores.

^***Denotes significance at the p<.001 level.

^**Denotes significance at the p<.01 level.

^*Denotes significance at the p<.05 level.

Fig. 3.

Heatmap showing the absolute value of the correlation between the task analysis-based categorization of games (x axis) and tasks (y axis), uncorrected for multiple comparisons. Warmer colors indicate higher correlations. Correlation coefficients and significant values are shown in Appendix C. TASKS legend: speed (SP), working memory (WM), reasoning (GF), switching (SHIFT), attention-inhibition (ATT), and episodic memory (MEM). GAMES legend: speed (SPEED), working memory and short-term memory (WM/SHORT-TERM), reasoning and executive control-switching (REASONING), and attention.

3.2.1. Construct correlations

As shown in Table 4, all game component scores were highly related to working memory, fluid intelligence and perceptual speed. The working memory and spatial reasoning games correlated most robustly with the working memory and fluid intelligence abilities. None of the game component scores were reliably associated with episodic memory and attention network measures.

3.2.2. Perceptual and visuo-motor speed games

Overall, performance on the pre-selected speed games was not highly related to the speed measures, except for DigiSwitch which also displayed high correlations with working memory and fluid intelligence measures. The perceptual speed tasks were all paper–pencil tasks and it is possible that visuo-motor speed measured in the tasks did not correlate well with responses made through a computer. The heterogeneity of the games may have also dampened correlations with the more “process-pure” perceptual speed tasks. Indeed, DigiSwitch and Phage Wars, both of which incorporate spatial attention and reasoning, significantly correlated with fluid intelligence tasks. The mini-games built into DigiSwitch also included working memory type components (such as one similar to the Simon Says game), which likely explains the high correlations with the working memory tasks.

3.2.3. Working memory and reasoning games

Overall, performance on the memory games correlated highly with working memory and fluid intelligence measures, with the highest relations from Memocubes, Oddball and Simon Says. The working memory games also correlated significantly with some perceptual speed measures. Similarly, reasoning games correlated highly with working memory and fluid intelligence tasks, with the most reliable relations from Silversphere, Sushi-Go-Round and Two Three. Sushi-Go-Round and Two Three also correlated strongly with the perceptual speed measures, not surprising given the highly time-limited nature of these two games compared to Bloxorz, Blobs and Silversphere.

3.2.4. Attention-multiple object tracking games

Overall, the attention games did not selectively correlate with the attention measures or any other cognitive domain. Significant correlations were distributed across different types of tasks. Most of the attention games correlated signicantly with one or few measures of working memory and fluid intelligence, although the patterns were not as robust as the working memory and reasoning games.

3.3. Contextual analysis of game scores

While the construct correlations revealed that the three main game groups (working memory-reasoning, spatial reasoning, and attention-multiple object tracking) were all highly correlated with working memory, fluid intelligence and perceptual speed, the contextual analysis better revealed the unique contributions of each cognitive ability (Fig. 4).

Fig. 4.

Contextual analysis to examine the unique relations of different cognitive abilities to the game component scores. Only significant paths are drawn. Standardized estimates are displayed above. Model fit: χ² (78)=129.342, p=0.0002; RMSEA=0.056 [.038, .073]; CFI=0.961; TLI=0.948.

Fluid intelligence accounted for most of the variance (27%) in the working memory-reasoning games, compared to the 14% and 3% accounted for by working memory and perceptual speed, respectively. Fluid intelligence also accounted for majority of the variance in the spatial reasoning games (25%), while the contribution of working memory to this game group was only 5% (Fig. 4). For the attention game group, only 14% of the variance was related to fluid intelligence, and much less was accounted for by working memory and perceptual speed.

4. Discussion

To our knowledge, this is the first attempt to both qualitatively and quantitatively evaluate the cognitive abilities tapped by a variety of video games. We used both cognitive task analysis and factor analytic techniques to understand the structure of the cognitive tasks and casual games. Game analyses revealed five interpretable game groups with close correspondence to the four pre-defined categorizations: working memory and reasoning games, spatial integration/reasoning games, attention/multiple object tracking games, and a mix of perceptual speed games. Importantly, examining the relationship between performance on the tasks and games revealed that working memory and fluid intelligence abilities were highly correlated with performance in the working memory and reasoning games. Furthermore, contextual analyses showed that fluid intelligence best predicted scores on working memory and reasoning games. While this is not a training study, demonstrating the relationship between games and cognitive abilities is an important first step if games are to be used for training. In the same way that working memory tasks are used to train working memory ability, and that a high overlap or relationship between working memory and fluid intelligence formed the logic behind the working memory-to-improve-broad-cognition training studies (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008; for a review see Klingberg, 2010), games must first be evaluated for their relevance to the targeted abilities. The next step is to then use these games in a training program and evaluate their efficacy.

In addition to providing support for the use of certain games to train specific abilities, this study provides an objective way to measure transfer at different levels. Transfer to a cognitive ability can be better evaluated by the use of assessment tasks that although different in stimuli and context as the training tasks, experimentally engage similar fundamental cognitive abilities. Moreover, the contextual analyses shed light on other abilities that may be developed by the training games; correlations between ability and performance have been shown to change as a function of practice (Ackerman, 1988), so it is helpful to know whether game training is likely to develop skills other than or in addition to the targeted ability. Thus, although a training program using these casual games has yet to be conducted, we provide a theoretical framework from which to design such a protocol and consequently interpret the resulting effects.

Working memory and fluid intelligence have been shown to be highly correlated, both in the psychometric and cognitive training literature (Colom, Rebollo, Palacios, Juan-Espinosa, & Kyllonen, 2004; Conway & Getz, 2010; Jaeggi et al., 2008; Jaušovec & Jaušovec, 2012; Kane et al., 2004; Klingberg, 2010; Morrison & Chein, 2011; Salthouse & Pink, 2008; Schweizer, Hampshire, & Dalgleish, 2011; Unsworth & Engle, 2006). Kane et al. (2004) administered a battery of tests designed to evaluate working memory, short-term memory, verbal and spatial reasoning and fluid intelligence. Using factor analytic methods, they found that working memory capacity accounted for 30–40% of the variance in fluid intelligence. Strong relationships between working memory and reasoning, fluid intelligence (gF) or general intelligence (g) have also been reported in other studies (Colom et al., 2004; Gray, Chabris, & Braver, 2003; Halford, Cowan, & Andrews, 2007) and such findings motivated programs that aimed to improve fluid intelligence by training working memory (for a review see Morrison & Chein, 2011). It has been suggested that “working memory constrains intelligent behavior” (Conway & Getz, 2010; Klingberg, 2010), as the act of actively maintaining and manipulating information is crucial to complex tasks such as reasoning and problem solving. Thus, increasing working memory capacity might improve performance on complex tasks. However, the initial study demonstrating gains in fluid intelligence (as indexed by a timed version of the Raven’s Matrices) after improving working memory capacity on a demanding dual n-back task (Jaeggi et al., 2008) has not always been replicated in other labs (Chooi & Thompson, 2012, Redick et al., 2012; Shipstead, Redick, & Engle, 2010; Shipstead et al., 2012). This could be due to several reasons, but a fundamental issue is whether what is honed during dual n-back training is a skill important for fluid intelligence. Participants may have developed a strategy specific to the training task, and not a skill that can transfer to other contexts. As high fluid intelligence is characterized by the ability to solve problems in novel situations (Cattell, 1987), training on a single paradigm and not on a variety of challenging situations may not be a suitable approach. In addition, superior performance on complex tasks has repeatedly shown to be best achieved under conditions of variable priority training, where skills are practiced in an integrated context (see Gopher, 2007 for a review). This is in contrast to isolated or full emphasis learning as in the use of training tasks originally designed to measure a specific cognitive ability. The importance of varied and holistic training underscores the potential of game-based training, given the diversity and complexity of casual video games. Interestingly, a recent study that demonstrated training-related gains in multiple measures of fluid intelligence employed a variety of working memory tasks in their training protocol (Jaušovec & Jaušovec, 2012).

In the current study, the fluid intelligence factor had a greater relationship than the working memory factor to performance on the working memory and reasoning games. This is not surprising given the added complexity and problem-solving inherent in the game environment. Another possible reason is that the fluid intelligence factor, unlike the working memory factor, was based on a broader selection of both verbal and nonverbal reasoning and visualization measures. While this also calls into question the categorization of the “working memory” games, it is also possible that the working memory cognitive factor used in this study was inadequate. Only one of the three indicator variables was from a task that required updating or manipulation of items in memory (n-back). The inconsistencies in previous studies and in the current study suggest re-examination of working memory as a construct (see Kane, 2002; Kane et al., 2004). Distinct sub-processes within working memory such as maintenance, updating, and inhibition may all differentially influence various measures (e.g. capacity in short-term memory tasks vs capacity in updating or span tasks, capacity in simple span vs. complex span tasks). Thus, it is also important for future studies to include comprehensive measures of working memory capacity to account for differences in maintenance-only tasks vs. maintenance-plus-updating tasks, for example.

Other research shows that the relationship between fluid intelligence and working memory is related to the amount of information that can be held online simultaneously (Fukuda, Vogel, Mayr, & Awh, 2010), speculating that “filtering efficiency” may factor into the relationship. As filtering efficiency is the ability to selectively process relevant information while inhibiting irrelevant information, one can think of the working memory–fluid intelligence link as related to the “quality” of maintaining and manipulating a certain quantity of representations. Unsworth and Engle (2006) found that the extent to which working memory predicts higher-order cognition is greater for complex working memory tasks than simple span tasks. They speculate that the critical overlap is controlled or executive attention, an ability akin to filtering efficiency (Heitz et al., 2006). In the context of this study, it can be assumed that the reasoning and working memory games required the most control of attention, given the degree of advance planning required and the amount of potentially interfering information or stimuli. While the working memory games did not explicitly entail the planning complexity of the reasoning games such as the move sequences in Bloxorz or the task prioritizing in Sushi-Go-Round, it is possible that participants adopted reasoning strategies to help them maintain the increasing load of items in memory, such as grouping items in Oddball or sequences in Simon Says. Such representations could then have been linked together to solve the problem, an act that requires high filtering efficiency in order to keep representations distinct from each other and separate from other distracting information.

The attention or multiple object tracking games did not strongly or preferentially tap any cognitive ability measured in this study. Nonetheless, they were also significantly correlated with fluid intelligence and working memory, although to a smaller degree than the reasoning and working memory games. While we did not find high correlations with process-specific attention measures presumably due to the comprehensive nature of the game scores, a variety of attention skills must be essential in playing the games. It is likely that while demand on higher-level abilities grew with each level, demand on lower-level visual attention remained fairly constant, perhaps with a shift in balance from stimulus-driven to goal-driven behavior. Furthermore, whereas basic attention tasks may demand little working memory and reasoning skills, working memory and reasoning tasks typically demand attention skills. Redick and Engle (2006) characterized the overlap between working memory and different types of attention, as indexed by the attention network test (ANT) and found that working memory capacity was significantly associated with the “executive” component of the ANT. The executive component is measured by comparing performance on incongruent versus neutral or congruent flankers and is thus a measure similar to filtering efficiency or controlled attention. Indeed, the attention games (Cathode, Dodge, Enigmata, Filler) that placed higher demand on attentional control because of the need to represent multiple objects and events, showed modest correlations with fluid intelligence and working memory. Although planning and problem solving were important components in the multiple object tracking games, the relevant events remained on the screen and as such did not demand the degree of internal controlled processing of the reasoning and working memory games.

In addition, majority of the games that placed the least demand on controlled attention and were categorized as perceptual speed or visuo-motor speed games (25 Boxes, Alphattack, Crashdown) were not robustly correlated with working memory and fluid intelligence tasks overall. There was also only a weak correspondence between perceptual speed games and perceptual speed tasks, which may indicate that the speed games likely involve significantly more processes than perceptual speed. Indeed, these games had weak but slightly higher correlations with working memory, reasoning and shifting tasks, than with the perceptual speed measures.

Due to the integrative nature of games, they cannot be expected to be “process-pure” like many of the cognitive task measures.¹ While there are sub-scores that can be extracted in some games, of interest was the most important ability needed to perform well on each game, which is reflected in the overall score or level. We do not exclude the possibility that a game can be used to train more than the ability it emphasizes. For example, a reasoning game may be hypothesized to produce “top-down” training-induced transfer, for example to attention skills and perceptual speed that while integrated into the reasoning game, are not the abilities stressed for optimal performance. On the other hand, increased familiarity with the reasoning game may instead lead to a reduction in reasoning demands and an increased emphasis on spatial attention or processing speed. This has practical implications for long-term training, as abilities that predict initial performance in a game may not be the most useful later in training. Indeed, it has long been shown that task manipulations that enhance skill acquisition during training may not support post-training and long-term performance (Schmidt & Bjork, 1992). Playing multiple games can help avoid this issue, as participants may be less inclined to develop task-specific strategies. Moreover, different types of games may be differentially susceptible to such decline in effectiveness. Preliminary data from a multi-session study in our lab indicate that fluid intelligence predicts performance in reasoning games throughout the duration of training, with the strongest constant correlation seen for games that present new challenges or demand new strategies at each level.

4.1. Limitations and future directions

This study provides an important first step in examining the utility of casual video games for research and training. The knowledge derived from our study concerning the relationship between casual game performance and psychometric constructs of different aspects of cognition is critical for designing a game training protocol, with theoretical basis for the training games, inclusion of a proper active control group and adequate assessment measures. However, the study design poses some limitations. The attention tasks employed in the study were few and the extracted measures were not as internally coherent as those used to build the perceptual speed, working memory and reasoning factors. Addition of multiple measures designed to measure specific types of working memory and attention may be more informative. Inclusion of dual-task, multi-task or multiple object tracking measures may also shed light on important skills tapped by the strategic fast-paced attention games. It also remains to be examined whether and to what degree individual differences such as gaming experience, gender, age and education contribute to the relationship between different games and cognitive abilities.

Games were played only once for 20 min and it is possible that the abilities recruited and developed by each game change through the course of playing two or more sessions (Ackerman, 1988; Quiroga et al., 2009). Insight into this issue can be gleaned from assessing the degree of transfer in training programs based on the games in this study, or from follow-up studies that include more sessions of game play. Only 20 games from the Cognitive Media website were examined in this study, a small sample compared to the number of cognitively interesting games available on the web and on mobile applications. While studies such as these cannot be conducted in a similar fashion for all games, programs that endeavor to use different video games for training may choose to include a battery of laboratory-style tests on their website to assess the correlation between cognitive abilities and game performance. Computer-based training protocols are commonly used in the hopes of improving cognitive function relevant to daily life, school or the workplace (Holmes et al., 2009) or prevent age-related cognitive decline (Ball et al., 2002; Basak, Boot, Voss, & Kramer, 2008). However, in addition to mixed or non-replicable results, many programs face methodological problems such as the lack of an appropriate active control group, the use of single tasks to measure training improvements or transfer and the ability to engage and motivate users. As a next step, we plan to use the findings from this study to develop a more theoretically informed program to improve working memory and potentially fluid intelligence, using the games found to be highly related to the targeted abilities. In addition, we will design an active control group composed of games that did not preferentially or strongly involve these abilities. To better interpret and generalize any training-related change, we will also employ multiple assessment tasks such as those used in the factor and structural analyses of the current study. Ultimately, the goal is to demonstrate the relationship of game training not only to performance on laboratory tasks, but also to everyday skills and tasks.

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.actpsy.2012.11.009.