Divergent Research Methods Limit Understanding of Working Memory Training

Abstract

Working memory training has been a hot topic over the last decade. Although studies show benefits in trained and untrained tasks as a function of training, there is an ongoing debate on the efficacy of working memory training. There have been numerous meta-analyses put forth to the field, some finding overall broad transfer effects while others do not. However, discussion of this research typically overlooks specific qualities of the training and transfer tasks. As such, there has been next to no discussion in the literature on what training and transfer tasks features are likely to mediate training outcomes. To address this gap, here, we characterized the broad diversity of features employed in N-back training tasks and outcome measures in published working memory training studies. Extant meta-analyses have not taken into account the diversity of methodology at this level, primarily because there are too few studies using common methods to allow for a robust meta-analysis. We suggest that these limitations preclude strong conclusions from published data. In order to advance research on working memory training, and in particular, N-back training, more studies are needed that systematically compare training features and use common outcome measures to assess transfer effects.

Introduction

A longstanding debate has regarded the extent to which training can improve our basic cognitive functions (Katz et al. 2018). Here, we address this issue in reference to working memory (WM), defined as a limited-capacity system responsible for temporary storage and manipulation of relevant information (Baddeley 2003, 2012).WM isimportant for a wide range of complex cognitive activities, such as reading or problem solving (Shah and Miyake 1999). In the last decade, there has been a considerable amount of literature focused on WM training (Jaeggi et al. 2008; Von Bastian and Oberauer 2014; Morrison and Chein 2011; Klingberg 2012). For example, WM training on a given task can transfer to improvements in untrained working memory tasks (Blacker et al. 2017; Lilienthal et al. 2013; Chein and Morrison 2010; Borella et al. 2010), as well as tasks pertaining to other cognitive domains such as fluid intelligence (Jaeggi et al. 2008; Heinzel et al. 2017; Chein and Morrison 2010; Borella et al. 2010). While there are numerous reports of transfer in the literature, there is also substantial evidence for failure of transfer (Thompson et al. 2013; Jackson et al. 2012). The field has reached a point in which there is a battle of meta-analyses lingering with roughly half of them finding evidence of transfer while the others do not (see Table 1 for variety of individual studies upon which these meta-analyses are based). The lack of explanation regarding this variability not only casts a shadow on WM training research but also poses a significant hurdle when evaluating the effectiveness of WM training.

Table 1.

Summary of training features from the 56 studies selected: transfer effect tests, N-back type, N-back modality, first/last-session N-back level, blocks, feedback, inter-stimulus interval (ISI), intervention length/training sessions (short < 10 sessions ≤ long), adaptivity, control group, years of education, single/double blind, strategies, motivation/expectation, payment, main results, and transfer tasks

Studies	Transfer effects tests	Single/dual	N-back modality	First/last-session N-back level (group average)	Blocks	Feedback	ISI	Training sessions	Adaptivity
1. Anguera et al. (2012)	Within and beyond WM	Dual	Audio/spatial	3-back to 5-back	9 blocks (20 trials/block)	Yes	Long	Long	Non-forgiving
2. Beavon (2012)	Within and beyond WM	Single	Spatial	3-back to 5-back	15 blocks (20 trials/block)	N/A	Long	Long	Non-forgiving
3. Blacker et al. (2017)	Within and beyond WM	Dual	Audio/spatial	1-back to 2-back	20 blocks (20 trials/block)	Yes	Long	Long	Forgiving
4. Bürki et al. (2014)	Within and beyond WM	Single	Visual	3-back to 5-back	15 block (30 trials/block)	N/A	Long	Long	N/A
5. Buschkuehl et al. (2014)	Within WM	Single	Spatial	4-back to 6-back	15 blocks (20 + n trials/block)	No	Long	Short	Non-forgiving
6. Chooi and Thompson (2012)	Beyond WM	Dual	Audio/spatial	2-back to 4-back	N/A	N/A	Long	Long	Non-forgiving
7. Clark et al. 2017a, b)	Beyond WM	Dual	Audio/spatial	2-back to 6-back	4 blocks (20 trials/block)	Yes	Long	Long	N/A
8. Clouter (2013)	Within and beyond WM	Dual	Audio/spatial	2-back to 3-back	20 blocks (20 trials/block)	Yes	Long	Long	Forgiving
9. Colom et al. (2013)	Beyond WM	8 single sessions followed by 16 dual sessions	Audio/spatial	Dual sessions: 2-back to 5-back	N/A	Yes	Long	Long	N/A
10. Qiu et al. (2009)	Beyond WM	Dual	Audio/visual	2-back to 3-back	20 blocks (20 + n trails/block)	N/A	Long	Long	Non-forgiving
11. Heinzel et al. (2014)	Within and beyond WM	Single	Visual	N/A	27 blocks (20–28 trials/block)	No	Short	Long	Non-forgiving
12. Heinzel et al. (2016)	Within and beyond WM	Single	Visual	N/A	36 blocks	No	Short	Long	Forgiving
13. Heinzel et al. (2017)	Within WM	Single	Visual	N/A	36 blocks	No	Short	Long	Forgiving
14. Hogrefe et al. (2017)	Within WM	Single	Spatial	N/A	10 blocks (20 trials/each)	Yes	Short	Long	Non-forgiving
15. Hussey et al. (2017)	Within WM	Single	Visual	N/A	2 blocks (15 trials/block)	Yes	Long	Long	Non-forgiving
16. Jaeggi et al. (2008)	Beyond WM	Dual	Audio/spatial	3-back to 5-back	20 blocks (20 trials/block)	No	Long	Long	Non-forgiving
17. Jaeggi et al. 2010a, b); study 2	Within and beyond WM	Single	Spatial	3-back to 7-back	9 blocks (20 trials/block)	Yes	Long	Long	Non-forgiving
18. Jaeggi et al. 2010a, b); study 2	Within and beyond WM	Dual	Audio/spatial	2-back to 4-back	9 blocks (20 trials/block)	Yes	Long	Long	Non-forgiving
19. Jaeggi et al. (2014)	Within and beyond WM	Single	Audio	3-back to 6-back	15 blocks (20 trials/block)	No	Long	Long	Non-forgiving
20. Jaeggi et al. (2014)	Within and beyond WM	Dual	Audio/spatial	2-back to 4-back	15 blocks (20 trials/block)	No	Long	Long	Non-forgiving
21. Jonasson (2014)	Within WM	Dual	Audio/spatial	N/A	15 rounds	Yes	Long	Short	Non-forgiving
22. Katz et al. (2018)	Within and beyond WM	Dual	Audio/spatial	2-back to 4-back	15 blocks (20 trials/block)	No	Long	Long	Non-forgiving
23. Kühn et al. (2013)	Within WM	Single	Visual/spatial (numerical updating and spatial N-back)	N/A	8 blocks (39 trials/block)	No	Long	Long	N/A
24. Kundu et al. (2013)	Within WM	dual	Visual/spatial	N/A	25 blocks (25 trials/block)	Yes	Long	Long	Forgiving
25. Küper and Karbach (2016)	Within and beyond WM	Single	Visual	3-back to 5-back	15 blocks (20 trials/block)	No	Short	Short	Forgiving
26. Lawlor-Savage and Goghari (2016)	Within and beyond WM	Dual	Audio/spatial	1-back to 4-back	15 blocks (20 trials/block)	Yes	Long	Long	Forgiving
27. Li et al. (2008)	Within WM	Single	Spatial (mental spatial shifting and updating)	N/A	4 blocks (22 trials/block)	Yes	Long	Long	N/A
28. Lilienthal et al. (2013)	Within WM	Dual	Audio/spatial	2-back to 4-back	20 blocks (20 trials/block)	No	Long	short	Non-forgiving
29. Loosli et al. (2016)	Within and beyond WM	Single	Visual	2-back	(100 trials/block)	No	Long	Long	N/A
30. Maraver et al. (2016)	Within and beyond WM	Single	Audio/spatial (N-back, WM search, WM updating)	1-back to 3-back	Not fixed (18 trials/block)	Yes	Short	Short	Non-forgiving
31. Marček (2015)	Beyond WM	Single	Spatial	N/A	20 blocks (20 trials/block)	No	Long	Long	N/A
32. Minear et al. (2016)	Within and beyond WM	Single	Spatial	4-back to 5-backa	15 blocks (20 trials/block)	No	Long	Long	Non-forgiving
33. Mohammed et al. (2017); game N-back	Within and beyond WM	Game	Visual/spatial gaming task	3-back to 4-backa	8–15 blocks (20–40 trials/block)	Yes	N/A	Long	Forgiving
34. Mohammed et al. (2017); standard N-back;	Within and beyond WM	Single	Visual	3-back to 5-backa	8–15 blocks (20–40 trials/block)	Yes	N/A	Long	Forgiving
35. Nagle et al. (2015)	Within and beyond WM	Game	Visual/spatial gaming task	2-Back to 3-Back	4 blocks (15 trials/block)	Yes	Long	Long	Forgiving
36. Preece (2012)	Beyond WM	Single	Spatial	3-back to 5-back	15 rounds	Yes	Long	Long	Non-forgiving
37. Redick et al. (2013)	Within and beyond WM	Dual	Audio/spatial	2-back to 4-back	20 blocks (20 trials for each)	No	Long	Long	Non-forgiving
38. Rudebeck et al. (2012)	Within and beyond WM	Dual	Visual/spatial	LG: 2-back to 2-back; HG: 2-back to 3-back	12 blocks (30 trials for each)	No	Long	Long	Non-forgiving
39. Salminen et al. (2012)	Within and beyond WM	Dual	Audio/spatial	2-back to 5-back	20 blocks (22 trials for each)	Yes	Long	Long	Non-forgiving
40. Salminen et al. (2015)	Within WM	Dual	Audio/spatial	2-back to 5-back	20 blocks (22 trials for each)	Yes	Long	Long	Non-forgiving
41. Schwarb et al. (2015)	Within and beyond WM	Single	Visual/spatial	4-back to 6-back	18 blocks (20 trials/block)	N/A	Long	Short	Non-forgiving
42. Schweizer et al. (2011)	Beyond WM	Dual	Audio/visual	4-back to 7-back	20 blocks (20 trials/block)	N/A	Long	Long	Non-forgiving
43. Shahar and Meiran (2015)	Beyond WM	Single	Visual/spatial	1-back to 3-back	10.7 blocks (64 trials/block)	Yes	Short	Long	Forgiving
44. Smith et al. (2013)	Beyond WM	Dual	Audio/spatial	3-back- 5-back	(20 trials/block)	N/A	Long	Long	N/A
45. Soveri et al. 2017a, b)	Within WM	Dual	Audio/spatial	3-back to 4-back	20 blocks (20 trials/block)	No	Long	Long	Non-forgiving
46. Stepankova et al. (2013); long intervention	Within and beyond WM	Single	Visual	2-back to 3-back	20 blocks (20 trials/block)	Yes	Short	Long	Non-forgiving
47. Stepankova et al. (2013); short intervention	Within and beyond WM	Single	Visual	2-back to 4-back	20 blocks (20 trials/block)	Yes	Short	Short	Non-forgiving
48. Stephenson and Halpem (2013); single visual N-back	Beyond WM	Single	Visual	N/A	(20 trials/block)	No	Long	Long	Non-forgiving
49. Stephenson and Halpem (2013); single audio N-back	Beyond WM	Single	Audio	N/A	(20 trials/block)	No	Long	Long	Non-forgiving
50. Stephenson and Halpem (2013); dual N-back	Beyond WM	dual	Audio/spatial	N/A	(20 trials/block)	No	Long	Long	Non-forgiving
51. Thompson et al. (2013)	Within and beyond WM	Dual	Audio/spatial	3-back to 5-back	30 blocks (20 trials/block)	No	Long	Long	Non-forgiving
52. Urbänek and Marcek (2015)	Beyond WM	Single	Spatial	2-back to 4-back	15 blocks (20 trials/block)	No	Long	short	Forgiving
53. Vartanian et al. (2013)	Beyond WM	Single	Visual	N/A	4	No	Long	Short	N/A
54. Waris et al. (2015)	Within and beyond WM	Dual	Audio/spatial (selective updating task, moving figures, dual N-back)	2-back to 4-back	(20 trials/block)	No	Long	Long	Non-forgiving
55. Zajac-Lamparska and Trempala (2016)	Beyond WM	Single	Visual	N/A	(20 trials/block)	No	Long	Short	N/A
56. Zhao et al. (2018); HAM group	Within and beyond WM	Single	Spatial	3-back to 6-back	15 blocks (20 trials/block)	Yes	Long	Long	Non-forgiving
57. Zhao et al. (2018); LAM group	Within and beyond WM	Single	Spatial	2-back to 4-back	15 blocks (20 trials/block)	Yes	Long	Long	Non-forgiving

Control group	Years of education info	Single/double-blind info	Strategies info	Motivation/expectation info	Payment	Main results (transfer effects)	Transfer tasks
Active (knowledge training)	No	No	No	Yes	Low	Transfer to the 3-back task and visuospatial tasks. No transfer to visuomotor task.	N-back, operation span (OSPAN), card rotation, digit symbol substitution (WAIS-R), visuomotor adaptation
Active (knowledge training)	No	No	No	No	N/A	No transfer effects to STM and WM.	Numbers reversed (WJ-III), auditory WM (WJ-III)
Active (adaptive non-WM task called permuted rule operations)	Yes	No	No	Yes	High	Transfer to near WM task (objects N-back). No far-transfer to fluid intelligence.	N-back, symmetry span, spatial locations, and relations, BOMAT
Active (implicit sequence learning training), passive	Yes	No	Yes	No	N/A	Near-transfer effects to a similar spatial N-back task.	N-back, numerical updating, reading span, RSPM, RAPM, Stroop, letter and number comparison (pattern comparison), SRT
Active (knowledge training)	No	No	No	No	Low	Near transfer to N-back task with different stimuli.	N-back
Active (dual 1-back training), passive	Yes	No	No	No	Low	No transfer effects.	OSPAN, vocabulary (Mill-Hill, PMA), word beginning and ending, Colorado perceptual speed, identical pictures, finding A’s, card rotation (ETS), paper folding (ETS), mental rotation (Shepard-Metzler), RAPM
Active (processing speed training), passive	Yes	Yes (single blindi	No	Yes	High	No transfer effects.	RAPM, WAIS-IV, CCFT, lexical decision, OSPAN, spatial delayed response task
Active (dual 1-back training)	No	Yes (single blindi	No	Yes	N/A	Transfer effects to fluid intelligence, WM and conflict resolution.	CFIT, Stroop, Monty Hall Problem, OSPAN, symmetry span
Passive	No	No	No	Yes	N/A	No transfer to fluid intelligence, crystallized intelligence, WM and attention control.	RAPM, abstract reasoning (DAT-AR), inductive reasoning (PMA-R), WM: reading span, computation span, dot matrix
Passive	No	No	No	No	N/A	Transfer effects to fluid intelligence, larger for the training group.	RSPM
Passive	Yes	No	No	No	N/A	Transfer to verbal fluency and digit symbol, digit span Fwd, digit symbol, and CERAD Del Recall.	Digit span Fwd (WAIS), Digit span Bwd (WAIS), Recall (CERAD) (immediate and delayed), digit symbol (WAIS), verbal fluency (COWAT), RSPM, figural relations (LPS)
Passive	Yes	No	No	No	N/A	Transfer to Sternberg task, processing speed, executive functions and fluid intelligence.	Digit span Fwd (Weschler), digit span Bwd (Weschler), digit symbol substitution (Weschler), D2, Stroop, verbal fluency (COWAT), RSPM, figural relations (LPS)
Passive	Yes	No	No	No	N/A	Near-transfer to N-back.	Visual and auditory single tasks, dual task
Active (N-back training with no immediate feedback), passive	No	No	No	No	N/A	Transfer effects to the N-back task and to numerical memory updating.	N-back, task switching, Flanker, Stroop, numerical updating, RAPM
Active (adaptive N-back training without lures; non-adaptive 3-Back training without lures)	Yes	Yes (double blindi	No	No	High	Transfer to untrained memory and language conditions.	N-back, recognition memory, Verb generation, Stroop, garden-path recovery, relative clause processing
Passive	No	No	No	No	N/A	Transfer to fluid intelligence based on training amount.	RAPM, BOMAT
Passive	No	No	No	No	High	Transfer effects to fluid intelligence.	N-back, OSPAN, RAPM, BOMAT
Passive	No	No	No	No	High	Transfer effects to fluid intelligence.	N-back, OSPAN, RAPM, BOMAT
Active (knowledge training)	No	No	No	Yes	High	Transfer to fluid intelligence.	RAPM, CFIT, BOMAT, surface development test (ETS), space relations (DAT), form board test (ETS), interference test (ETS), reading comprehension (AFOQT), verbal analogies, digit symbol (WAIS)
Active (knowledge training)	No	No	No	Yes	N/A	Transfer to fluid intelligence.	RAPM, CFIT, BOMAT, surface development test (ETS), space relations (DAT), form board test (ETS), interference test (ETS), reading comprehension (AFOQT), verbal analogies, digit symbol (WAIS)
Active (face-name recall training)	No	No	No	No	N/A	No transfer effects.	N-back, OSPAN, addition, trail making (TMT), dual task
Active (knowledge training)	No	No	No	Yes	High	Transfer to visuospatial composite.	RAPM, CFIT, BOMAT, surface development test (ETS), space relations (DAT), form board test (ETS), interference test (ETS), reading comprehension (AFOQT), verbal analogies, digit symbol (WAIS)
Active (numerical updating and N-back training with fixed difficulty level)	No	No	No	No	High	Improvements on untrained working-memory tasks.	N-back, spatial updating, figurai and numerical reasoning (Berlin Intelligence Structure Test)
Active (adaptive Tetris)	No	No	No	No	N/A	Transfer to stimulus processing, STM and visual search performance	Visual-array comparison task, visual search, OSPAN, Stroop, RAPM
Passive	Yes	No	No	No	Low	Near-transfer to a WM updating task. No far-transfer to switch costs, Stroop and matrix reasoning tasks.	N-back, task switching, Stroop, RAPM, digit symbol substitution, Spot-a-word
Active (1-Back training)	No	Yes (double blind)	No	No	N/A	No transfer to fluid intelligence.	Digit span (WAIS), symbol search (WAIS), coding (WAIS), OSPAN, RAPM, CFIT
Passive	No	No	No	No	High	Transfer to a more complex spatial N-back task and numerical N-back task. No far transfer to complex span.	N-back, OSPAN, rotation span, decision speed
Active (non-adaptive dual N-back), passive	No	No	No	No	N/A	Transfer effects from adaptive training to a running span task (focus of attention).	Cued recall, focus-switching, grid span, operation span, running span
Active (recent-probes and N-back training with low proactive interference)	Yes	Yes (double blindi	No	No	Low	No transfer effects to untrained tasks.	Verb generation, paired associates, Stroop, digit symbol substitution (WAIS-R), TONI
Active (processing speed training), passive	No	No	No	Yes	Low	Transfer effects to reasoning for the inhibitory control training group.	N-back, Stroop, OSPAN, stop signal, AX-CPT, RAPM
Active (non-adaptive Sudoku)	No	No	Yes	Yes	Low	No transfer for the single N-back group, transfer to RAPM for the control group (triple N-back task).	RAPM, BOMAT
Active (non-adaptive N-back training, real time strategy video game)	No	No	Yes	Yes	High	Near-transfer effects to a different N-back task for both adaptive and non-adaptive N-back training group.	N-back, Speed of processing, dot judgment, array matching, (letter-digit-arrow-circle-reading-operation-letter-number-rotation-alignment) span, attention network, Simon, nonsense Syllogisms (ETS), inference tests (ETS), RPM, CFIT, mathematical aptitude (ETS)
None	No	No	No	Yes	Low	Transfer to untrained N-back, Far transfer to DRM free recall (falsely remembered), DRM (recognition), Space relations, Surface development, Form board, Delay discounting	N-back, AX-CPT, DRM, space relations (DAT), surface development (ETS), form board test (ETS), BOMAT, learning from lectures, Math, delay discounting
None	No	No	No	Yes	Low	Transfer to untrained N-back, far transfer to DRM free recall (falsely remembered), space relations, surface development, form board, Math, delay discounting	N-back, AX-CPT, DRM, space relations (DAT), surface development (ETS), form board test (ETS), BOMAT, learning from lectures, Math, delay discounting
None	No	No	No	Yes	N/A	No transfer effects.	RAPM, digit span Fwd, digit span Bwd, N-back
Active (vocabulary and knowledge training)	No	No	No	No	N/A	No transfer effects to fluid intelligence compared with the control group.	Figure weights (WAIS), RAPM
Active (visual search training), passive	No	No	Yes	Yes	High	No transfer effects in fluid intelligence, WM, crystallized intelligence and perceptual speed tasks.	RAPM, RSPM, CFIT, paper folding, letter sets, number series, inference, verbal analogies, SynWin, control tower, ATClab, symmetry span, running letter span, vocabulary, general knowledge, letter and number comparison
Passive	Yes	No	No	No	N/A	Transfer effects to episodic memory and fluid intelligence.	BOMAT, recognition memory
Passive	No	No	No	No	N/A	Transfer to WM updating task, switching situation task and attentional processing. No transfer to reasoning or dual N-back task.	Updating (AV numbers, VS black bars that was shown in 4 different locations), dual task, Task switching, attentional blink, RAPM
Passive	Yes	No	No	No	Low	Transfer effects to a WM updating task for both young and older adults.	Updating (AV numbers, VS black bars, one block with VS another block with AV), task switching, attentional blink
Passive	No	No	No	No	Low	Transfer effects to visual short-term memory capacity.	OSPAN, Symmetry span, RAPM, motion interference, rapid decision-making, change detection, short-term recall
Active (feature matching training)	Yes	No	/	No	N/A	Transfer to fluid intelligence. Transfer to emotional Stroop task only for affective training group.	RPM, Stroop, digit span
Passive	No	Yes (double blind)	No	No	Low	Transfer to processing speed compared with the control group.	Digits updating task, shape/digit classification task, Stroop, stop-signal, RAPM
Active (strategy video game training), passive	No	No	No	No	N/A	No transfer effect to fluid intelligence.	RAPM
Active (non-adaptive game Bejeweled 2)	Yes	No	No	Yes	Low	Transfer effects to different single N-back task and to a WM updating task. No transfer effects for active or passive groups.	N-back, verbal running span, visuospatial running span, number substitution, digit span Fwd, digit span Bwd, Corsi Block
Passive	Yes	No	No	No	N/A	Transfer effects to WM and visuospatial skills.	Digit span (WMS), letter number sequencing (WMS-III), block design (WAIS), matrix reasoning (WAIS)
Passive	Yes	No	No	No	N/A	Transfer effects to WM and visuospatial skills.	Digit span (WMS), letter number sequencing (WMS-III), block design (WAIS), matrix reasoning (WAIS)
Active (spatial matrix span training), passive	Yes	No	No	No	N/A	Transfer to 4 fluid intelligence tests (APM, Cattell, WASI, BETA-III).	RAPM, CFIT, WASI, BETA, mental rotation, paper folding, vocabulary, lexical decision
Active (spatial matrix span training), passive	Yes	No	No	No	N/A	Transfer to 3 fluid intelligence tests (APM, Cattell, WASI).	RAPM, CFIT, WASI, BETA, mental rotation, Paper folding, Vocabulary, Lexical decision
Active (spatial matrix span training), passive	Yes	No	No	No	N/A	Transfer to 4 fluid intelligence tests (APM, Cattell, WASI, BETA-III).	RAPM, CFIT, WASI, BETA, mental rotation, paper folding, vocabulary, lexical decision
Active (multiple object tracking training), passive	No	No	Yes	Yes	High	No transfer effects to fluid intelligence and other cognitive tasks.	Operation span, reading span, RAPM, block design (WASI), WAIS-III, reading comprehension (Nelson Denny), digit-symbol coding (WAIS-III), visual matching (Woodcock-Johnson III), pair cancelation (Woodcock-Johnson III)
Active (non-adaptive Sudoku)	No	No	No	No	Low	No transfer effects to fluid intelligence.	RAPM, BOMAT
Active (reaction time training)	No	No	No	No	N/A	Transfer effects to fluid intelligence.	RAPM, alternate uses task (AUT)—test of divergent thinking
Active (Angry Birds, Bejeweled 2, Peggie)	No	No	No	Yes	Low	Near-transfer effects in a different N-back task, WM updating and in a WM task.	Verbal running span, digit span, Corsi block, set shifting, visuospatial running span, CFIT, Simon, number substitution, numerical updating, N-back
Active (attentional control training), passive	Yes	No	No	No	N/A	Weak transfer effects to fluid intelligence.	Attentional control, RSPM, N-back
None	No	No	No	Yes	N/A	Near-transfer effects to WM. No far-transfer effects to executive functions and fluid intelligence.	N-back task, running digit span, Go/no-go, Stroop, task switching, RAPM.
None	No	No	No	Yes	N/A	Near-transfer effects to WM. No far-transfer effects to executive functions and fluid intelligence.	N-back task, running digit span, Go/no-go, Stroop, task switching, RAPM.

Legend: HAM, high achievement motivated group; LAM, low achievement motivated group; WM, working memory; STM, short-term memory; COWAT, Controlled Oral Word Association Test; DAT, Differential Aptitude Test; WAIS, Wechsler Adult Intelligence Scale; WMS, Wechsler Memory Scale; WASI, Wechsler Abbreviated Scale of Intelligence; WJ, Woodcock-Johnson; ETS, Educational Testing Service Kit; RSPM, Raven’s Standard Progressive Matrices; RAPM, Raven’s Advanced Progressive Matrices; CFIT, Culture Fair Intelligence Test; LPS, Leistungspriifsystem; PMA-R, Primary Mental Abilities Battery; TONI, Test of Non-verbal Intelligence; WASI, Wechsler Abbreviated Scales of Intelligence; BOMAT, Bochumer Matrizen Test; AFOQT, Air Force Officer Qualifying Test; BIST, Berlin Intelligence Structure Test; CERAD, Consortium to Establish a Registry for Alzheimer’s Disease; SRT, Simple Reaction Time; AX-CPT, AX-continuous performance task

^aGroup average maximum N-back level

One of the most common measures of WM is the N-back task, an updating task that requires multiple processes (storage, maintenance, and manipulation of information) and is predictive of inter-individual differences in higher cognitive functions (Jaeggi et al. 2010a, b). Since the N-back task is also one of the most prominent tasks used in WM training studies, here, we limit our discussion on WM training to interventions using N-back tasks.

However, with as many studies using the N-back task, there are as many variants in methodology. These range from the adopted training approaches (e.g., varying in terms of task timing, types of stimuli, number of stimulus streams, adaptive algorithms, feedback provided, number of training sessions, blind/not blind; see Fig. 1 for illustration; Table 1) to the transfer tasks that are rarely consistent from one study to the next with over 120 different transfer tasks used across the 57 experiments reviewed in 51 studies (see Fig. 1 for illustration and Table 1 for details). For example, across these experiments, 31 different tasks assess aspects of WM and short-term memory (STM), including N-back and other updating tasks, simple span tasks, and various complex WM tasks. Another 29 tasks assess aspects of fluid intelligence, the content of which is predominantly visuospatial (matrix reasoning, block design, figure weights, paper folding, form board, surface development, space relations, abstract reasoning, mental rotation, card rotation, TONI, etc.) followed by verbal (letter sets, inference test, nonsense syllogisms, inductive reasoning PMA-R, verbal analogies, reading comprehension), and quantitative (number series) (cf. Table 2). With many unique combinations of training methodologies and transfer tasks, and no model to interpret these differences (Katz et al. 2018), we are left with the difficulty of understanding what approaches might give rise to which cognitive outcomes and what features might determine the boundary conditions of N-back training.

Fig. 1.

Diversity of training and transfer procedures. Each circle contains 57 sectors, each one corresponding to an N-back training group included in this review (see Table 1). The six outer circles reflect training task features whereas the four inner circles reflect transfer task features. Starting from the outer circle, each sector is colored in terms of N-back type (1) stimulus modality, (2) inter-stimulus interval (ISI), (3) adaptivity (forgiving vs. non-forgiving), (4) feedback, (5) intervention length (short < 10 sessions ≤ long), (6) WM (transfer) task type, (7) WM (transfer) task domain, (8) fluid intelligence (transfer) task type, (9) and fluid intelligence (transfer) task domain (10)

Table 2.

Transfer tasks categorized by cognitive domain

COGNITIVE DOMAIN	NUMBER OF STUDIES
WORKING MEMORY
Simple Span
Corsi block	2
Digit span (forward, backward)	14
Grid span	1
Arrow/circle span	2
Complex span
Operation span (OSPAN)	13
Symmetry span	4
Reading span	4
Dot matrix	1
Computation span	1
Rotation span	2
Alignment span	1
Updating
N-Back	19
Numerical updating	6
Spatial updating	3
Running digit span	1
Running letter span	2
Verbal running span	2
Visuo-spatial running span	2
Visuospatial and auditory-verbal updating	1
Number substitution	2
Other
Letter number sequencing (WAIS)	1
Letter number sequencing (WMS-III)	1
Auditory WM (WJ-III)	1
Spatial delayed response task	1
Visual array comparison task	2
Change detection task	1
Short term recall task	1
Cued recall span task	1
Focus-switching task	1
Delayed match to sample (single and dual)	2
Spatial locations and relations	1
LTM
Recall (CERAD) (delayed, immediate)	2
Recognition memory	2
Paired associates	1
Learning from lectures	2
FALSE MEMORY
Deese-Roediger-McDermott	2
VISUAL SEARCH
Visual search	1
Symbol search	1
Finding A’s	1
Identical pictures	1
FLUID INTELLIGENCE*
Letter sets	1
Inference	1
Space relations (DAT)	5
Abstract reasoning (DAT-AR)	1
Matrix resoning (BETA-III)	1
Matrix reasoning (WAIS)	1
Block design (WAIS/WASI)	2
Figure weights (WAIS)	1
Nonsense syllogisms (ETS)	1
Inference tests (ETS)	1
Paper folding (ETS)	3
Surface development test (ETS)	5
Form board test (ETS)	5
Interference test (ETS)	4
RSPM	8
RAPM	27
BOMAT	11
Space relations	1
Figural relations (LPS)	2
Inductive reasoning (PMA-R)	1
CFIT	10
TONI	1
Number series	1
Mental rotation (Shepard-Metzler)	2
Figural and numerical reasoning (BIST)	1
Verbal analogies	4
Reading comprehension (AFOQT)	4
Card rotation	2
CRYSTALLIZED / GENERAL INTELLIGENCE
Verbal fluency (COWAT)	2
Lexical decision	2
Word beginning and ending	1
Verb generation	2
Vocabulary (Mill-Hill, PMA)	4
General knowledge	1
WAIS-IV	1
Spot a word	1
Similarities (WASI)	1
Vocabulary (WASI)	1
Extended Range Vocabulary Test (ETS)	1
READING
Nelson-Denny Comprehension	1
Lexical Decision Test	1
Nelson-Denny Reading Rate	1
MATH
Mathematical aptitude (ETS)	1
Arithmetic aptitude test (ETS)	1
Addition	1
Math	2
PROCESSING SPEED
Letter and number comparison (pattern comparison)	4
Simon	2
Coding (WAIS)	2
Visual matching (WJ- III)	1
Colorado Perceptual Speed Test	1
Shape/Digit Classification	1
SRT	2
Decision speed	1
Dot judgement	1
Digit symbol substitution (WAIS-R)	8
DECISION MAKING / PROBLEM-SOLVING
Monty Hall problem	1
Rapid decision making	1
Delay discounting	2
Relative clause processing	1
ATTENTION / COGNITIVE CONTROL
Garden path recovery	1
Set shifting	2
Trail making test	1
Stroop	12
Task switching	5
Focus switching	1
Attentional blink	2
Pair cancellation (WJ-III)	1
Stop-signal	2
Go/no go	1
Flanker	1
Attention network	1
Motion interference	1
AX-CPT	3
D2	1
Attentional control	1
Visuomotor adaptation	1
MOTOR LEARNING
Control tower	1
MULTITASKING
Synwin	1
Atclab	1
DIVERGENT THINKING
Alternate Uses Task	1

^*Fluid intelligence classification was based on Au et al. (2015), Table S3.

Legend: WM = working memory; LTM = long-term memory; COWAT = Controlled Oral Word Association Test; DAT = Differential Aptitude Test; WAIS = Wechsler Adult Intelligence Scale; WMS = Wechsler Memory Scale; WASI = Wechsler Abbreviated Scale of Intelligence; WJ = Woodcock-Johnson; ETS = Educational Testing Service Kit; RSPM = Raven’s Standard Progressive Matrices; RAPM = Raven’s Advanced Progressive Matrices; CFIT = Culture Fair Intelligence Test; LPS = Leistungsprüfsystem; PMA-R = Primary Mental Abilities Battery; TONI = Test of Nonverbal Intelligence; WASI = Wechsler Abbreviated Scales of Intelligence; BOMAT = Bochumer Matrizen test; AFOQT = Air Force Officer Qualifying Test; BIST = Berlin Intelligence Structure Test; CERAD = Consortium to Establish a Registry for Alzheimer’s Disease; SRT = Simple Reaction Time; Ax-CPT = Ax-continuous performance task

To date, discrepant findings regarding transfer effects reported by meta-analytic studies, focusing primarily on healthy adults, have been discussed in regard to important moderators such as population demographics, training dose, training type (e.g., single task, multiple tasks), training task (e.g., single N-back, dual N-back), training modality (visual, auditory, both), stimulus content (verbal, nonverbal), type of transfer tasks, design type (randomized/not randomized), type of control group (active/passive), attrition rate, training location, supervision, instructional support, feedback, and publication bias (Au et al. 2015; Soveri et al. 2017a, b; Melby-Lervåg and Hulme 2013; Melby-Lervåg et al. 2016; Schwaighofer et al. 2015). While these moderators are certainly relevant, the details of procedures employed in each training study, such as trained and transfer tasks features, which may mediate learning, have been largely ignored.

In this qualitative review, we examine a variety of design factors previously overlooked in N-back training that bear potential to affect learning and transfer, such as task timing and adaptive procedures, types of stimuli, and sensory modality. A summary of all training features can be found in Table 1. Interestingly, only 8 experiments relied on the same training method, whereas 49 experiments had unique training conditions (Fig. 1). In addition, we discuss issues pertaining to the size of the transfer battery and the inconsistency in transfer tasks across studies, and how these factors can affect the findings and their interpretation. The novelty of this review is to highlight the fact that different training protocols and transfer tasks might differentially affect training efficacy and transfer results.

Training Task Features

We highlight six training task attributes (types of N-back task and stimulus modality, task timing, adaptive threshold, feedback, and intervention length) that commonly vary across implementations of N-back training studies. In addition to these, numerous other factors varied across studies within training tasks, such as the number of blocks for each training session, response types (e.g., requiring participants to respond to targets only or also to non-targets), and how feedback was provided (visual/auditory). Within participants, there are additional factors that might determine training outcome, such as N-back levels achieved, used strategies, or motivation. Note that in many cases, details of the procedures that might be important are simply not reported (see Table S1, Supplemental Material). Another source of variation is the inclusion of training procedures that go beyond the N-back task, thereby targeting additional cognitive processes. For example, Li et al. (2008) incorporated mental spatial shifting in the N-back training procedure and Mohammed et al. (2017) used a 2D game version of the N-back task that required navigational skills. In four studies, participants trained on other types of updating WM tasks in addition to the N-back, which precludes understanding of the individual contributions of these training tasks to transfer (Maraver et al. 2016; Waris et al. 2015; Kühn et al. 2013; Loosli et al. 2016).

N-back Task Type—Single vs. Dual

A main area of variation is the use of single or dual N-back training. Conducting multiple N-back tasks simultaneously places different demands on attentional and WM resources as compared with a single N-back. For example, Jaeggi et al. (2003) showed that single and dual N-back tasks differ at the behavioral level with longer reaction times and more errors on dual N-back tasks compared with single N-back. On the other hand, no differentiation between single and dual N-back tasks was observed at the neural level: prefrontal activation increased with higher load irrespective of task type. This may explain why single N-back training seems to be as effective as dual N-back training (Jaeggi et al. 2008; Jaeggi et al. 2010a). In the current sample, 30 out of the 57 experiments adopted single N-back training (13 reporting transfer within WM, 11 reporting transfer beyond WM, and 6 reporting no transfer¹) and 27 experiments employed dual N-back training (8 reporting transfer within WM, 9 reporting transfer beyond WM, and 10 reporting no transfer). While this may suggest that dual N-back training is more likely to yield transfer within and beyond WM, as compared with single N-back, which seems more likely to show transfer within WM, it should be noted that not all studies assessed both types of transfer. Within the single N-back studies, 2 experiments tested untrained WM tasks, 10 experiments tested for far transfer (6 experiments focusing on fluid intelligence), and 18 experiments tested both. Within the dual N-back studies, 1 experiment tested untrained WM tasks, 9 tested for far transfer (4 experiments using fluid intelligence), and 17 experiments tested both. Even though the single vs dual N-back dichotomy is the most powered of available comparisons, the differences between study methodologies, as described below, largely preclude strong meta-analytic conclusions.

Stimulus Modalities

While WM is often discussed as a domain-general process (Kane et al. 2004), there is substantial evidence that stimuli presented in different modalities (i.e. visual, spatial or auditory stimuli) are processed differently in WM. Owen et al. (2005) showed changes in brain activation between different N-back modalities, specifically for location and for non-verbal stimuli. Similarly, Crottaz-Herbette et al. (2004) found differences in neural activation for auditory and non-spatial WM tasks. The authors used, in a randomized order, a visual and an auditory N-back task. The stimuli were either single-digit numbers (0–9) presented visually at the center of the screen, or binaurally in case of the auditory version. The results showed bilateral suppression of the superior and middle temporal (auditory) cortex during visual (non-spatial) WM, and changes in the occipital (visual) cortex during auditory WM, suggesting that although similar prefrontal and parietal regions are involved in both auditory and visual WM, there are important modality differences in the way neural signals are generated and processed.

For the current review, we define modalities used to categorize the N-back stimuli as follows: (1) “spatial N-back” is a single N-back task that requires the processing of spatial locations of visual stimuli; (2) “visual N-back” describes a single N-back task that requires the processing of visual stimuli (objects, colors, or letters) irrespective of their spatial location; and (3) “audio N-back” describes a single N-back in which stimuli are presented in the auditory domain (e.g., letters, numbers, or other sounds). Dual N-back stimulus modalities are categorized as combinations of the three types of modalities described above: (1) “audio-spatial N-back” involves concurrent processing of auditory stimuli and spatial locations of visual stimuli; (2) “audio-visual N-back” requires simultaneous processing of auditory stimuli and visual stimuli irrespective of their spatial location; and (3) “visual-spatial N-back” requires the processing of both visual stimuli and the spatial locations of these stimuli. In addition, “visual/spatial gaming N-back” refers to a gamified (dual) N-back task that involves processing of different types of visual stimuli presented at different locations.

In our sample, we find that training task modalities vary widely, with 26 using auditory stimuli (7 reporting transfer within WM, 11 reporting transfer beyond WM, and 8 reporting no transfer), 13 using visual stimuli (non-spatial) (5 reporting transfer within WM, 6 reporting transfer beyond WM, and 2 reporting no transfer), and 18 using spatial stimuli (9 reporting transfer within WM, 3 reporting transfer beyond WM, and 6 reporting no transfer). Within those using auditory stimuli, 2 experiments employed a single audio N-back, 22 used dual audio/spatial N-back, and 2 used audio/visual N-back for training. The variety of the auditory stimuli is further highlighted by some studies using letters or syllables for the audio/spatial sub-group, others using words or other type of sounds for the audio/visual sub-group. Overall, N-back training tasks implement a variety of stimuli (shapes, objects, letters, numbers, etc.) in different modalities (visual, auditory, with or without a spatial component) (see Fig. 1), which can be problematic for cross-study comparisons of transfer effects.

Task Timing

Another training feature rarely considered as a relevant factor impacting WM training is the timing between stimuli in the N-back tasks. Inter-stimulus intervals (ISI) can have an important impact on the time available to process each stimulus and to engage in strategies such as rehearsal or grouping and comparison. The use of these strategies can modify performance levels, give rise to very different experiences during training, and thus likely impact learning outcomes (Laine et al. 2018). Strüber and Polich (2002) showed that during an oddball task, in which participants needed to press a button every time the visual target stimulus appeared, shorter ISIs were associated with smaller P300 amplitudes. They suggested that long ISIs enable a “recovery cycle” that can reduce task difficulty. To date, ISI has not been considered a factor relevant to WM training.

In the papers that we reviewed, we screened 57 experiments across single and dual N-back training and found 46 experiments that reported long ISIs (between 1800 and 2500 ms; 18 reporting transfer within WM, 14 reporting transfer beyond WM and 14reporting no transfer), 9 that used short ISIs (between 500 and 1800 ms; 8 reporting transfer within WM, and 1 reporting transfer beyond WM), while 2 experiments did not report ISI information (and did not report any transfer either).

Adaptive Threshold

The extent to which training adapts to participants’ abilities is another factor that can have a substantial impact on learning and transfer. For example, in the case of perceptual learning, transfer is greatly impacted by task difficulty with more difficult/precise tasks giving rise to more specificity of learning than found through training involving easier/less-precise stimulus judgements (Hung and Seitz 2014; Ahissar and Hochstein 1997). Most N-back training studies utilize adaptive training by adjusting the level of task difficulty based on individual performance, and it has been shown that adapting the difficulty level of the task is engaging for the participant (Jaeggi et al. 2014). Moreover, Holmes et al. (2009) showed that WM training gains were significantly greater for an adaptive training group compared with a non-adaptive training group, although others have failed to observe any effects of adaptivity on learning outcome (von Bastian and Eschen 2016).

In the papers that we reviewed, we distinguished experiments based on the adaptive threshold used to pass to the next difficulty level: most experiments used a threshold of 90% correct responses (non-forgiving), whereas others used a threshold of 65% or 80% (forgiving). Of 46 experiments, 12 adopted a threshold lower than 90% to achieve the next level (7 reporting transfer within WM, 1 reporting transfer beyond WM, and 4 reporting no transfer), while 34 adopted a threshold of 90% correct (16 reporting transfer within WM, 10 reporting transfer beyond WM, and 8 reporting no transfer). Finally, 3 experiments adapted task difficulty by changing the ISI length (not considered here).

Feedback

Feedback plays an important role in the process of learning, particularly in complex cognitive tasks and in monitoring goal progress (West et al. 2001). Feedback is usually delivered based on participants’ accuracy and/or response speed and is typically designed to encourage participants to optimize their performance to achieve better learning and/or greater reward (Simen et al. 2009). Feedback can indeed facilitate learning, as demonstrated by cognitive training and perceptual learning research (Abe et al. 2011; Seitz et al. 2006).

Out of the 57 experiments reviewed, 25 experiments employed some type of feedback (11 reporting transfer to untrained WM tasks, 6 reporting transfer beyond WM, and 8 reporting no transfer) while 32 experiments either did not provide feedback or did not explicitly report the use of feedback (16 reporting transfer within WM, 9 reporting transfer beyond WM, and 7 reporting no transfer). Of those experiments employing feedback, 22 gave information about when the feedback was provided: at the end of each block (N = 9), at the end of each session (N = 9), after each trial (N = 4). Thus, despite the critical role of feedback in motivation and learning (Burgers et al. 2015), the majority of studies (N = 32) do not describe whether or what type of feedback was employed.

Intervention Length

There is evidence that longer training leads to more learning in terms of more pronounced changes in brain regions involved in WM function (Dahlin et al. 2008; Lövdén et al. 2010). Hempel et al. (2004) highlighted the role of visual spatial N-back training length, showing specific brain activation increases with improved performance after 2 weeks of training, and conversely, activation decreases at the time of consolidation of performance gains after 4 weeks. These results are consistent with the hypothesis that WM training duration affects training results (Jaeggi et al. 2008; Stepankova et al. 2013), although the appropriate amount of training for a given procedure for a given participant is not well established.

In our sample, of the 57 experiments that measured both transfer to WM and beyond WM, 47 used training equal or longer than 10 sessions (29 reporting transfer within WM, 12 reporting transfer beyond WM, and 6 reporting no transfer), and 10 experiments used fewer than 10 sessions (5 reporting transfer within WM, 2 reporting transfer beyond WM, and 3 reporting no transfer).

Transfer Task Features

In addition to the parameters of the training tasks, it is important to consider the details of the outcome measures. Across 57 experiments, 122 different transfer tasks were employed (see Table 2), which speaks to the issue of variability in transfer tasks. The number of outcome measures per study ranged from 1 to as many as 20. Using large test batteries can give rise to participant fatigue and decreased participant engagement (Ackerman and Kanfer 2009), and it can also lead to issues with multiple comparison. In addition, unexpected cognitive benefits may occur as a function of assessing multiple tasks at once, wherein the transfer battery could act as a form of training (Salthouse and Tucker-Drob 2008; see also Green et al. 2019; Morrison and Chein 2011). However, using only one or a few outcome measures can limit opportunity to estimate latent factors. Most of the studies investigated transfer effects using a large variety of tests designed to measure more than one cognitive ability, within and beyond WM. In particular, across all the experiments, 9 focused on just one cognitive function (or task type), 11 experiments focused on two, 9 on three, and 28 on four or more cognitive functions. As follows, we give an overview of how these outcome measures varied across experiments:

Transfer within the domain of WM was assessed with 31 different tasks, including various simple span measures (Corsi block, digit span, grid span) and complex span tasks (operation span, symmetry span, etc.), updating tasks (N-back, running span, numerical updating, etc.), and other types of WM tasks such as delayed match to sample tasks and sequencing tasks. Fourteen experiments did not assess WM according to our classification (denoted as N/A in Fig. 1), 21 experiments reported using WM measures that fall under one of the four categories mentioned above, and 22 experiments reported using WM tasks that include at least two of these categories (denoted as multiple in Fig. 1). Out of the experiments that used only one WM task type, 3 experiments used simple span tasks, another 4 used complex span tasks, 13 used updating tasks, and 1 experiment used a WM task classified as “other” (for details, see “WM task type” in Fig. 1). Out of the 43 experiments that measured WM, 13 experiments reported using only verbal/numerical WM tasks and 3 reported using only visual/spatial WM tasks; however, most used WM tasks that covered both verbal/numerical and visual/spatial domains (N = 27; see “WM task domain” in Fig. 1).

In sum, even though they all measure some aspects of WM, these 31 different tasks are likely to measure a number of cognitive skills, a fact often overlooked by extant meta-analyses. While some distinctions have been made in terms of task type (untrained N-back vs. WM tasks in Soveri et al. 2017a, b) and task domain (verbal vs. visuospatial WM in Melby-Lervåg and Hulme 2013; Melby-Lervåg et al. 2016; Schwaighofer et al. 2015), such categorization does not capture the full range of cognitive demands imposed by different WM tasks and may even mask improvements in a subgroup of tasks. Performance on N-back tasks only correlates weakly with performance on complex span tasks (Redick and Lindsey 2013) therefore it makes sense to consider updating and span tasks separately. Furthermore, even if two research groups use the same task with similar types of stimuli, the tasks may still differ in the choice of timing parameters, instructions, feedback, etc., as is often the case with custom-built tasks.

Transfer beyond WM, in particular to fluid intelligence, was assessed with 27 different tasks. Forty-eight out of fifty-seven experiments reported assessing fluid intelligence. These tasks were categorized as: matrix reasoning tests (including any type of Raven’s matrices or Bochum Matrices Test Advanced (BOMAT)), spatial visualization tests (paper folding, mental rotation, card rotation, surface development test, form board, block design, spatial relations), deduction tests (nonsense syllogisms, inferences), induction tests (number series, inductive reasoning PMA-R, letter sets, abstract reasoning DAT, verbal analogies), and other tests (reading comprehension, figure weights). Approximately half of the experiments reported the use of batteries that contain multiple tests (e.g., WASI) or the use of multiple tests that include at least two of the categories described above (e.g., matrix reasoning and deduction), which were classified as Multiple (N = 26). The remaining experiments included matrix reasoning tests (21 experiments) and spatial visualization tests (1 experiment) (see “Fluid intelligence task type” in Fig. 1, and Table 2). Moreover, in terms of “task domain,” fluid intelligence tests were categorized as: figural, verbal, or numerical (Beauducel et al. 2001). Most experiments (N = 39) reported using tests with figural content, and even though no experiments used only verbal or only numerical tests, 9 experiments reported using a combination of figural/verbal or figural/numerical tests. While matrix reasoning was the most common type of test used to assess fluid intelligence, which allows for a certain level of comparison across experiments, using just one type of test is not sufficient to estimate fluid intelligence at the latent level. When combined with other fluid intelligence tasks, which vary substantially in terms of the cognitive processes that are required to solve the task (i.e., visuospatial transformation, induction, deduction, attention, working memory), and the degree to which these overlap with the cognitive processes targeted during training, estimating training-related changes in the construct of fluid intelligence across studies becomes challenging.

In addition to the two cognitive domains described above, studies also used other transfer measures representing a wide range of cognitive functions (not reported in Fig. 1; for further details see Table 2). Specifically, 4 different tasks were used to assess long-term memory (LTM), 1 task to assess false memory, 4 different tasks to assess visual search, 11 to assess crystallized/general intelligence, 3 different tasks for reading, 4 for math, 10 different tasks for processing speed, 4 for decision making/problem solving, 17 different tasks for attention/cognitive control, 1 for motor learning, 2 for multitasking, and 1 for divergent thinking (for further details see Table 2).

Overall, this diversity of transfer tasks measured across studies raises serious issues of the extent to which the same underlying cognitive outcomes are assessed across studies and thus, limits the interpretation of the extant literature.

Test Reliability

An important factor that might impact transfer is task reliability, especially test-retest reliability (Jaeggi et al. 2014). However, for most of the 122 of tasks used, no reliability measures are reported, and it is unclear whether standard forms or custom forms of the tasks are employed, making it difficult to find information on the reliability in the extant literature. It is not uncommon for WM measures to show weak or inconsistent test-retest reliability (e.g., Jaeggi et al. 2010a, b), which could mask transfer effects: the lower the reliability, the lower the chances for transfer (Jaeggi et al. 2014). Comparing transfer effects on tasks that differ substantially in their reliability may be misleading if this factor is not taken into account. Unfortunately, only a few fluid intelligence tasks have reliable parallel test versions and the commonly used method of splitting tests in half can reduce the reliability and validity of the tests (Jaeggi et al. 2014). Recent efforts to develop multiple parallel reasoning tests may mitigate these types of problems in future intervention studies (Pahor et al. 2018; Kyllonen et al. 2018).

Overall, the diversity of transfer tests and batteries used across studies poses a challenge as these outcome measures vary in their degree of similarity with the trained task, and furthermore, their reliability and their validity in measuring the factor of interest are often unclear.

Control Group

It has also been argued that the type of control group plays a significant role in whether transfer is observed. The impact of control groups is related to the degree of similarity between the N-back training and the control interventions, and/or to the differential participant engagement and motivation, and/or participant expectations (Green et al. 2019). For example, Tsai et al. (2018) suggested that placebo effects might represent an additional factor that contributes to improvements achieved during cognitive training due to alterations in participant expectations. However, literature on WM training is mixed both in regard to what control conditions are employed, some using active controls and others passive controls, and also the extent to which the control type seems to alter the magnitude of observed transfer (Au et al. 2015). A simple reason for this is that the features and the effects of the control condition are likely to be more nuanced than what can be captured by simple distinction into active or passive controls. Participant recruitment and population, as well as other factors like engagement and self-perceived improvements might considerably contribute to the extent to which expectations may impact training outcomes.

In our sample, 52 experiments included at least one control group: 22 experiments included only an active control group, 17 experiments included only a passive control group, and 13 experiments included both. Among the 35 experiments that included an active control group, 7 experiments used vocabulary or knowledge-based training, 8 used commercial games such as Tetris, Angry Birds, and Bejeweled, 9 used a variant of N-back training (typically non-adaptive and/or low-difficulty), 8 non-WM training (e.g., processing speed training), and 3 experiments, all belonging to one study, employed alternative WM training (spatial STM). These active control conditions differ in their cognitive and perceptual demands and similarity to the experimental condition, as well as most likely in the induced expectations about performance improvement due to training, again making it difficult to compare results across studies.

Discussion and Future Directions

Although reports on N-back training are steadily increasing, the mechanisms of transfer and the factors that might impact them are still unclear. We suggest that this lack of clarity is due to the variety of training procedures implemented and the selection of transfer measures gauging training outcomes. Despite numerous meta-analyses aimed to understand the effectiveness of N-back training (Au et al. 2015; Soveri et al. 2017a, b; Melby-Lervåg and Hulme 2013; Melby-Lervåg et al. 2016; Schwaighofer et al. 2015), there is still disagreement about the extent of transfer after N-back training. Here we show that N-back training studies, while seemingly similar, employ a wide variety of training features, and in addition, they assess transfer effects with a large and diverse selection of outcome measures. To highlight this variety, we characterized some of the factors that might be important for learning, such as type of N-back, stimulus modalities, task timing, adaptive threshold, feedback and intervention length (see Table 1). Given the small sample size of certain training task features and the extensive variability of methods in the literature, we can only speculate whether these factors are meaningful mediators and moderators. The sheer number of transfer tasks used to assess working memory and other cognitive functions further complicates the matter. At this point, in order to achieve a better understanding of the factors that might interfere with transfer outcomes, we suggest that further training studies and meta-analyses should evaluate more carefully the choice of training features (type of stimuli, ISI, intervention length, etc.), transfer measures (for WM, fluid intelligence, LTM, etc.), the type of control groups, and characteristics within the individuals (educational background, strategies, expectation, etc.) before making inferences. Furthermore, training features, transfer tasks, and individual differences need to be systematically addressed, as the large variability represents a severe issue that limits quantitative conclusions.

We suggest that there are several factors that are leading to this diversity of methods, which we argue limit progress in the field. First, there is the conceptual understanding of WM or fluid intelligence as domain-general processes. This view presumably leads researchers to overlook the importance of domain and task specificity, assuming that it does not matter how a specific exercise or test on WM is given (type, modality, etc.), as all approaches would impact the same cognitive process. Although there is still an ongoing debate about the relationship between specificity of cognitive functions and domain-general processes, emphasis should be given to the fact that all the tests used to investigate these constructs are only partially correlated with the underlying construct. Thus, different training approaches, even if related to the underlying construct, may lead to distinct transfer outcomes due to task specific learning. The second factor is related to the relative nascence of the field. With any new discovery, it makes sense to conduct studies to address the validity of the results and thus using a variety of methods can be vital to explore the space of possibilities. However, this variance of methods produces the inferential problems in making comparisons across studies.

As a first step to address these issues, researchers should both align training and outcome measures across studies and also conduct large-scale comparative studies. As a field, we need to reach some consensus about the training features that may be most conducive to learning, and thus, worth further study. Moreover, a core set of pre-post-measures should be defined both within the WM domain and beyond the WM domain. While studies should necessarily differ in attributes, some uniformity across studies with common tests that have known reliability and stability will allow for comparison with other studies, and researches will still have the option to expand the test battery based on their particular study goals. This would give more power to meta-analyses to address the question whether WM training is worthwhile (and more importantly, for whom it might work and under which circumstances). We recognize that unifying training and transfer task features may be difficult to achieve in practice and so another approach is to conduct larger scale comparative studies with sample sizes sufficient to directly examine unique combinations of training and transfer. Addressing these issues will elevate our understanding about what approaches do or do not lead to improvements in untrained tasks, as well as the specific domains that are most susceptible to the effects of WM training.

Another important step is bridging the gap between lab tests of cognitive functions and tests that reflect the use of cognitive functions in daily life. To enter the next stage of maturity in the field, new approaches that facilitate comparisons of different training approaches and outcomes are needed, to address issues of robustness, reproducibility and broader generality of findings outside of a limited set of laboratory conditions. To accomplish this, we need to become aware of which WM processes are differently required in daily life activities, and which training condition would be hypothesized to transfer to these conditions. To whatever extent existing tests of cognitive functions predict cognitive functions in daily life, this relationship may not hold after training on task structures that specifically resemble the cognitive tests. For example, if performance on two tasks is correlated, but they do not rely upon the exact same mechanisms, then a change in one may not predict a change in the other.

In conclusion, we suggest that it is time for WM training research to retool. Methods employed to date have been valuable to identify a broad set of issues that need to be considered in order to understand the true benefits and limitations of WM training. However, to move the field forward, it will be necessary to conduct large-scale studies that are targeted to uncover how particular training features and transfer measures may lead to differential learning and generalization of that learning. Furthermore, individual differences that may moderate these training effects need to be considered, together with a standard set of reliable outcome measures to better understand the profiles of transfer, and how these are reflected in daily-life activities, going beyond the simple question of whether or not near or far transfer occurs.

Method

To identify candidate papers, we searched Google Scholar, Google, and PubMed for relevant research reports in the last decade, between 2008 and 2018. The search terms used were “N-back training” and “updating training”/”N-back training game” and “updating training game.” In Google, citation marks were used to reduce noise in the research. The first run resulted in 12,100 hits in Google Scholar for N-back training, 675,000 for updating training, 2730 for N-back training game and 127,000 for updating training game. We found 219 hits in PubMed for N-back training, 1501 for updating training, 6 hits for N-back training game and 9 for updating training game. In Google, the hits were 46,300 for N-back training, 71,400 for updating training, 2170 for N-back training game, and no results found for updating training game. We screened all hits in the databases (Google Scholar, PubMed and Google) thereby limiting ourselves to the first 150 ranked ones. For a study to be included at this stage, it needed to meet the following criteria:

1. Cognitive training that included game or no-game version of single or dual N-back task

2. Studies with at least one training group

3. Sample of healthy adults (mean age range 19–69 years old)

4. N-back training equal to or longer than 3 sessions

5. Focused on transfer to WM and/or other cognitive domains

Search hits were screened in the mentioned ranking, and papers already evaluated in previous databases were not considered in the following screening. Our inclusion criteria decreased the number of the studies to 45 on Google Scholar, 6 on PubMed and 0 on Google for N-back training, updating training, N-back training game, and updating training game. In total, our research resulted in 51 studies (excluding the number of overlapping studies) (Fig. 2). Of these 51 studies, 5 studies included more than one N-back training group, which we considered separately, giving rise to a total of 57 experiments.

Fig. 2.

Search for literature and screening process