How to Assess Gaming-Induced Benefits on Attention and Working Memory

Abstract

Our daily actions are driven by our goals in the moment, constantly forcing us to choose among various options. Attention and working memory are key enablers of that process. Attention allows for selective processing of goal-relevant information and rejecting task-irrelevant information. Working memory functions to maintain goal-relevant information in memory for brief periods of time for subsequent recall and/or manipulation. Efficient attention and working memory thus support the best extraction and retention of environmental information for optimal task performance. Recent studies have evidenced that attention and working memory abilities can be enhanced by cognitive training games as well as entertainment videogames. Here we review key cognitive paradigms that have been used to evaluate the impact of game-based training on various aspects of attention and working memory. Common use of such methodology within the scientific community will enable direct comparison of the efficacy of different games across age groups and clinical populations. The availability of common assessment tools will ultimately facilitate development of the most effective forms of game-based training for cognitive rehabilitation and education.

Introduction

Attention defines our ability to focus on goal-relevant information and simultaneously ignore distracting, goal-irrelevant information. It thus serves as a gateway for selective information processing.^1–4 Working memory (WM), in turn, is closely linked to attention, in that information that survives the attentional filter is accessible for maintenance over brief periods of time to be retrieved or manipulated for purposes of guiding subsequent goal-directed behavior.^5–7 As such, efficient attention and WM functions allow us to navigate complex and dynamic environments. Given the fundamental importance of these cognitive faculties, many recent neuroscientific investigations are focused on studying ways in which we can improve our attentional or executive control abilities. In recent years, regular action videogame play in young adults has emerged as an activity that has consistently been shown to be associated with superior attention capacities.^8–15 In parallel, other investigators have developed and evaluated novel, computer-based games designed specifically to ameliorate neurophysiological deficits in cognitive control, such as those observed in normal aging.^16–18 Of note is that although most of the studies are behavioral investigations, some have combined neurophysiological measurements with cognitive behavioral assessments. These studies provide important insights into the brain plasticity mechanisms that engender the performance enhancements.^18–20

This mini-review outlines some key experimental paradigms used by neuroscientists to evaluate the impact of videogames and game-based training programs on attention and WM function in humans. It describes cognitive probes used to assess different aspects of attention, followed by those that evaluate WM. Aspects of selective attention include spatial attention (focusing attention on a specific location in the visual field), temporal attention (selective focus on stimuli appearing at specific points in time), object-based attention (focus on organized groups of information), and sustained attention (focus on a simple task for prolonged time periods). In the WM paradigms, the goals are to encode and maintain task-relevant information, which is subsequently probed after varying time delays. WM is evaluated in both visual and verbal contexts.

Attention

Spatial attention

Many gaming-related influences on spatial attention have been evaluated using the Useful Field of View (UFOV) task.²¹ This computerized assessment was originally developed to gauge safe driving performance in older adults and has been used to assess the impact of explicit speed of processing training developed for older adults.^22–24 The task is divided into three subtests (Fig. 1):

1. In the first subtest the participant is required to identify a target of varying duration, either a car or truck, presented in a fixation box. This simple subtest measures the speed of visual processing.

2. In the second subtest, the participant is simultaneously presented with a central and a peripheral target (car) located along any one of eight radial spokes (four cardinal and four oblique) at any visual eccentricities of 10°, 20°, or 30°. The participant reports both the identity of the central target and the location of the peripheral target. This subtest assesses the ability to distribute attention throughout the visual field. The duration of stimulus display is varied to measure speed of processing for this divided attention task.

3. The third subtest is the same as the second subtask with the exception that the peripheral target is embedded in distracters, and this subtest assesses selective attention to spatial targets amidst distractions.

FIG. 1.

Experimental layout for the three subtests of the Useful Field of View task (see Dye and Bavelier³⁰ for an adapted version of this task using fewer distractors and a higher-acuity central task).

Performance on the UFOV task is expressed in terms of the minimum duration required to complete each of the three task subtests. Green and Bavelier^9,25 showed that action videogaming experience improves performance on all subtests of the UFOV, but especially on the third subtest, which requires selection of a target among distractors. It is notable that the investigators showed benefits to peripheral visuospatial attention at far lateral eccentricities that were outside the visual field of typical gameplay. Furthermore, action videogame players (AVGPs), unlike non-game players (NVGPs), did not compromise performance on the primary (central) task while also performing the peripheral tasks in the second and third subtests, revealing efficient multitasking abilities. Of note is that these UFOV benefits were not replicated in two recent studies,^26,27 which compared videogame experts (and also lab-trained gamers²⁶) with NVGPs. These studies, however, included participants with a combination of action and non–action game playing experience, so it remains possible that the impacts observed by Bavelier and colleagues are specific to AVGPs.

Another recently developed task that probes spatial attention is the “Swimmer task.”¹¹ Here participants view a wide-field array of swimmers (15 or 30 swimmers in low and high perceptual conditions, respectively) moving in a random trajectory around the visual array. Participants detect non-swimmer targets displayed on half the trials (one non-swimmer appearing per non-swimmer target trial) at 10°, 20°, or 30° visual eccentricity. AVGPs consistently demonstrated better task performance than NVGPs on this task as well, suggesting superior capacities for distributed spatial attention.

Temporal attention

Assays of temporal attention focus on the ability to allocate attention to targets appearing successively in time. Outcome measures evaluate the detection accuracy at various lag time intervals between two successive targets. The main paradigm used to assess this skill is the attentional blink task.^9,28–30 This task consists of rapid serial visual presentation of items at the center of the screen. One or two target letter/number items (T1 and T2) are embedded sequentially amid distracter letters. The T2 target is presented at varying delays post-T1. T1 and in some cases T2³¹ are usually in white font, whereas distractors are in black font. Accurate detection of T2, following correct identification of T1, is observed at various T1–T2 time lags. Attentional blink investigations have previously shown better performance in AVGPs^9,30,32 but not in players of mixed action and non-action videogames.²⁶ The underlying mechanism, however, remains uncertain as to whether these abilities in AVGPs stem from faster sensory processing,³³ superior attention in time, or both of these processes.¹⁵

Of note is that spatial and temporal attention can also be studied in combination using rapid serial visual presentation streams at different visual field locations.¹⁹

Participants detect rapidly streaming occasional targets at a specific attended location (infrequent numbers interspersed among letters). AVGPS again outperformed NVGPs on this task. Furthermore, electroencephalographic (EEG) recordings revealed that AVGPS suppressed neural processing of unattended peripheral visual stimuli to a much greater extent than control participants. These findings suggested that the superior target detection capabilities of AVGPs can be attributed to enhanced suppression of distracting irrelevant information. Such results highlight how a combination of behavioral and neurophysiological assays provide greater insights into training-induced impacts on attention.

Object-based attention

Motion Object Tracking serves as an ideal task to assay attention to objects. It requires continuous attention to several targets in the presence of non-target distractors. Participants track a precued subset of moving objects (one to seven circles) among many identical items (16 circles) (Fig. 2). At the end of each trial, participants indicate whether a probed item (colored white) belonged to the original target or distractor set. Note that in this Motion Object Tracking design, unlike others, participants are not asked to recall each initially cued target, thus minimizing the role of WM and providing a more objective measure of object tracking. Overall, the ability to track moving objects is enhanced by action videogame play in adults^25,26,32 and in children.^30,34

FIG. 2.

An example trial of the Motion Object Tracking task, where three items (cued white) are required to be tracked. At the end of the trial, participants respond whether probed item (labeled “?”) belonged to the initial cued subset.

Sustained attention

Sustained attention is the ability to consistently maintain attention on an elementary, even boring, task over long periods of time. The Test of Variables of Attention (T.O.V.A.^®, The TOVA Company, Los Alamitas, CA) is a standardized task used for this purpose.³⁵ The T.O.V.A. is a two-segment task; targets are presented rarely in the first segment (one per 3.5 non-target presentations) and appear often in the second segment (3.5 targets per non-target). The two T.O.V.A. segments respectively measure sustained attention (to stay on task and respond speedily to infrequent targets) and impulsivity (to withhold responses to non-targets when most stimuli are targets). The T.O.V.A. can be administered in either the visual or the auditory domain. The visual version uses squares presented above or below fixation as targets and non-targets, respectively, whereas the auditory T.O.V.A. uses 392-Hz target tones amid 262-Hz non-targets. Dye et al.³⁶ used the visual T.O.V.A. to demonstrate that AVGPs exhibit faster response times than non-gamers on both segments of the T.O.V.A. and also retain equivalently high accuracies as the control group. The extent of generalization of these sustained attention effects to the auditory modality is unknown and would benefit from further exploration.

Many other sustained attention tests have been used in the literature, such as the Continuous Performance Test in the MATRICS Schizophrenia Assessment Battery,³⁷ the Sustained Attention to Response Task,³⁸ and the card identification task in the Cogstate Battery.³⁹ These tests could be considered to assess the impact of games on attention. However, all of these tests use more complex stimuli than the T.O.V.A., and the MATRICS Continuous Performance Test also includes WM demands, so it may not provide the purest measure of sustained attention. It is recommended that researchers consider these issues before incorporating such tests into their experimental batteries.

WM

Visual WM

Berry et al.¹⁸ recently reported that game-based, visual perceptual discrimination training improves visual WM in older adults. The study evaluated outcomes in a training versus untrained control group using a delayed-recognition WM paradigm. Participants encoded and maintained a briefly presented motion cue and determined if a motion probe presented after a delay of several seconds matched the original cue. The task consisted of three main WM manipulations: (1) NI, no interference stimulus displayed during the intervening delay; (2) DS, a distracting stimulus (circular dot swirl) presented within the delay that participants ignored; and (3) IS, an interrupting stimulus (circular dot swirl) presented in the delay that participants attended and discriminated (to be a fast or slow swirl) (Fig. 3). The three task conditions—NI, DS, and IS—were incrementally more difficult. Results showed significant WM benefits on the NI condition in trained participants. Furthermore, EEG recordings indicated more efficient early visual processing to encoded stimuli exclusively in trained participants that were directly correlated to the training-related WM improvements.

FIG. 3.

Experimental layout for the delayed-recognition paradigm used to assess visual working memory. The NI, DS, and IS conditions are presented in separate blocks. Stimuli presented in the DS and IS conditions are identical; however, in the latter condition participants respond to the interrupting stimulus with a button press (depicted as an encircled gray button in IS, DELAY 2) when the speed of the distracting swirling dots was fast.

Training-related impacts on visuospatial WM in the face of distractions can be probed using the Filter task.⁴⁰ Participants briefly view a spatial array of colored objects (red and blue rectangles) and are instructed to exclusively attend to targets (red) and ignore distractors (blue). After a delay, participants indicate in a probe array if any target changed from the cue array (in orientation). Participants perform the task at varying target distractor ratios while keeping the total size of the array constant. Preliminary data on this task from the Bavelier laboratory suggest greater visuospatial WM in AVGPs compared with NVGPs.⁴¹ However, much future research is required to establish if AVGPs indeed have enhanced visual WM functions.

Verbal WM

Tests of verbal WM have been primarily derived from the neuropsychological literature. The Digit Span and Letter Number Sequencing (LNS) tests⁴² require participants to repeat back a sequence of numbers, or a random mix of letters and numbers (LNS), spoken by the experimenter in either the order they were presented (forward span), reverse order (backward span), or ascending order of numbers and alphabetical order of letters. These tests have been incorporated in various verbal WM test batteries showing training related benefits in children (forward span⁴³) and older adults (backward span and LNS¹⁷) but may not be suited for healthy young adults who perform near ceiling on these measures. Alternatively, the Auditory Consonant Trigrams test⁴⁴ has been characterized as a sensitive outcome measure in both young and older adults.⁴⁵ In brief, the Auditory Consonant Trigrams test is a delayed recall test with an additional secondary task; participants recall a set of three shuffled letters after a 0/9/18/36-second delay and additionally perform a backward count from a specific number during the delay.

The Operation Span (OSPAN)⁴⁶ and N-back tasks are more sensitive tasks for a young participant cohort. The OSPAN is a dual-task test, requiring participants to maintain a sequence of words in memory while checking the validity of simple math equations. At the end of each trial, participants are probed with a word sequence and indicate if the probe sequence matches the input word order. Basak et al.⁴⁷ showed that real-time strategy videogame training in older adults improved performance on this task. However, no performance benefits were observed on the OSPAN in a study by the same group on young videogame experts or lab-trained gamers who played a combination of action and non-action games.²⁶ It remains to be tested whether AVGPs exclusively show performance benefits on the OSPAN. Alternatively, different training regimens may be needed to enhance verbal WM compared with visuospatial WM.⁴¹

The N-back task is a popular WM assessment in which a sequence of stimuli is continually presented and participants discriminate whether the current stimulus matched a stimulus presented N items previously. The task is complex requiring encoding, temporary maintenance and rehearsal, tracking of serial order, updating, and comparison and response processes for continuously presented stimuli. Two-back and three-back tasks are commonly used to challenge WM in young adults. Boot et al.²⁶ found a positive but nonsignificant trend for better WM performance in videogaming experts (playing both action and non-action) using a visuospatial version of the two-back task in which participants indicated if the location of the current stimulus matched that presented two items back (Fig. 4).

FIG. 4.

Example of a visuospatial two-back working memory task. Participants sequentially view each display and press a button when the square appears at the same location as in two displays back. Two-back displays where participants should respond are indicated with the label “visual target.”

It is notable that N-back tasks have themselves been used as training tasks (for review, see Klingberg⁴⁸) and have shown significant transfer of benefit on assessments of fluid intelligence⁴⁹ and of interference resolution⁵⁰ in young adults. Jaeggi et al.⁴⁹ trained participants on a demanding dual N-back task, where participants simultaneously tracked the location of visual squares and the identity of spoken auditory consonants. Significant training benefits were hypothesized to result from the challenging dual-task nature of the task as well as constant engagement of visuospatial and verbal executive processes. The study of Persson and Reuter-Lorenz⁵⁰ used a verbal three-back training task that contained many interfering non-target trials, where stimulus identity could match two/four/five-back stimuli. Of note is that significant training gains were only observed for participants who performed the three-back task with interference, but not for participants who performed three- or one-back task versions with no interference.

Conclusions

Here we have described several task paradigms that have been and are currently in use by cognitive neuroscientists to assess game-based training-related influences on attention and WM processes. These paradigms are predominantly designed by independent cognitive neuroscience laboratories with open-source sharing among researchers. Some probes, especially those for sustained attention and for verbal WM abilities, are modules in standardized neuropsychology compendiums, providing the advantage of comparisons with age-normalized scores. Comparisons with age-normed data inform whether study results conform to or deviate from the behavior of the general population at a specified age. Despite the caveat that the cognitive neuroscience-based paradigms are non-standardized, they are internally controlled in rigorous studies because they usually compare pre- versus post-training measures and additionally incorporate control-training groups. Control groups are necessary as retesting on a previously taken test usually generates better performance because of prior test familiarity and practice (termed “practice effects”). Training group performance must significantly differ from practice effects in the control group to infer a significant impact of training. Performance training gain can be calculated as equal to ([post-training – post-control] – [pre-training – pre-control]), and the effect size as equal to (training gain/intra-subject standard deviation).⁵¹ As more studies in the budding field of human cognitive training utilize these paradigms consistently, meta-analyses regarding the comparative efficacies of different training regimens will become possible.

A recent commentary on videogaming research suggested that some of these studies have inherent methodological flaws.⁵² In reality, design choices for all training studies, be it gaming or other behavioral interventions such as physical exercise,⁵³ music training,⁵⁴ or WM training,^48,49 require careful thought to include the most sensitive assessment measures, assign appropriate matched training to the control group, and follow the best alternate practices when participants cannot be blinded. Green and Bavelier¹⁰ have discussed best practices in the field in light of these various challenges and highlight how prior gaming studies best addressed these issues.

We emphasize that the tasks listed here in no way form a complete assessment battery and are exclusively focused on testing specific aspects of attention and WM in game/training studies. In this mini-review we also have not discussed paradigms that probe other nuances of attention such as interactions between top-down/voluntary attention and bottom-up/stimulus-driven attention^12,14 or paradigms that study how attention resources are differentially allocated between gamers and non-gamers.^9,36 Other cognitive functions relevant to game/training research include speed of processing, long-term memory, task switching, multitasking, reasoning ability, and, importantly, training generalization to daily life function, which have not been addressed here. While formulating a new test battery with the multitude of cognitive tests available, researchers are encouraged to consider test sensitivity in specific participant populations, as well as total administration time to minimize participant fatigue.

As is evident in this review, action videogame training has been shown to benefit many aspects of attention, although its impacts on WM remain to be rigorously evidenced. It is interesting that recent neuroscientific findings have shown that modulation of early attention processes can strongly impact subsequent WM (for reviews, see Gazzaley⁶ and Gazzaley and Nobre⁷). Training at the perceptual level has been evidenced to result in WM benefits in older adults.¹⁸ Thus, it can be hypothesized that AVGPs would show WM benefits. Investigations of comprehensive cognitive-gaming induced outcomes on WM measures in aging are currently under way in the Gazzaley lab.

As more studies document the robustness of gaming-induced benefits on attention and WM, this may in turn guide game developers to generate games that specifically target these abilities and that can then be applied to various real-world situations. Of note, but not discussed in this review, is that cognitive control has also been shown to be significantly enhanced by physical exercise in adults (for reviews, see Kramer and Erickson⁵³ and Hillman et al.⁵⁵) and children,⁵⁶ by musical training,^57–59 and by meditation-based approaches.^60–62 Perhaps in the future a combined approach utilizing both game-based training and these alternative training procedures may prove most useful for training older populations and children and adults with attention deficits and workforce training such as for military personnel and even surgeons,⁶³ as well as other application to various neurological and psychiatric disorders. At the same time, research studies, especially those that are performed in combination with neuroimaging measures (EEG and/or functional magnetic resonance imaging), are expected to enhance our understanding of the underlying neural plasticity of the involved brain systems. By characterizing these plasticity mechanisms, we aim to gain insights into both the neural benefits and limitations of various training strategies and to maximize the learning potential of the human brain.

Acknowledgments

This project was funded by the Sandler Foundation and National Institutes of Health Fogarty International Center grants to J.M., by the Office of Naval Research, the National Eye Institute, and James S. McDonnell Foundation grants to D.B., and by American Federation for Aging Research, The Ellison Medical Foundation, and Posit Science grants to A.G.

Author Disclosure Statement

J.M. is a part-time senior research fellow at the Brain Plasticity Institute, San Francisco, a company that develops cognitive training software. A.G. and D.B. are consultants for Akili Interactive Labs, a company that develops approaches for various health areas, including cognition. J.M. has a patent pending for methods of suppressing irrelevant stimuli; D.B. has two patents pending, one for action-videogame-based vision training and one for mathematics training; and A.G. has a patent pending for a game-based cognitive training intervention: “Enhancing cognition in the presence of distraction and/or interruption.”