Direct-current stimulation does little to improve the outcome of working-memory training in older adults

Abstract

The promise of transcranial direct current stimulation (tDCS) as a modulator of cognition has appealed to researchers, media, and general public. Researchers have suggested that tDCS may increase effects of cognitive training. We report results from a study (n=123; age=65-75 years) of the interactive effects of twenty sessions of anodal tDCS over the left prefrontal cortex (vs. sham stimulation) and simultaneous working-memory training (vs. control training) on change in cognitive abilities. Stimulation did not modulate gains from pre to posttest on latent factors of either trained or untrained tasks in a statistically significant manner. A supporting meta-analysis (n=266), also including younger samples, showed that, when combined with training, tDCS was not much more effective than sham stimulation at changing working memory performance (g=0.07[-0.21-0.34]) and global cognition (g=-0.01[-0.29-0.26]) assessed off stimulation. These results question the general usefulness of current tDCS protocols for enhancing the effects of cognitive training on cognitive ability.

Introduction

Working memory (WM), a central component of general cognition, has a close relationship to fluid intelligence (Conway & Kovacs, 2013). This close relationship suggests that broad cognitive improvement may be possible through WM training. The controversial promise of long-term generalized cognitive enhancement from relatively limited practice on a narrow set of tasks has inspired a wealth of research and numerous commercial brain training tools that promise fundamental improvements. The empirical evidence amassed to date shows improvements in WM tasks that are similar to the trained tasks. However, evidence on the transfer of improvements to untrained tasks and broad cognitive abilities is more limited and the credibility and size of these effects remain debated (Au, Buschkuehl, Duncan, & Jaeggi, 2016; Au et al., 2015; Dougherty, Hamovitz, & Tidwell, 2016; Karbach & Verhaeghen, 2014; Melby-Lervag, Redick, & Hulme, 2016; Simons et al., 2016).

An absence of transfer could simply reflect a lack of a causal within-person relationship between WM and fluid intelligence or a failure of the training to engage the processes that the constructs share (Harrison et al., 2013). An alternative view is that an intrinsic limitation of the adult brain’s capacity for change prevents transfer from occurring (Lövdén, Bäckman, Lindenberger, Schaefer, & Schmiedek, 2010). Such a limitation could be restricting training gains to task-specific knowledge and strategies, and preventing modulation of task-general processing capacity relevant to broader cognition. This intrinsic limitation could be expected to vary between individuals and be stronger in older age (Kühn & Lindenberger, 2016). The primary question posed in the present work is whether the potential for plastic change can be increased to allow for larger transfer of improvements from WM training to broad cognitive abilities in older age.

Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique with potential effects on brain plasticity (Dayan, Censor, Buch, Sandrini, & Cohen, 2013). Although the weak direct current that is passed through the brain via electrodes on the scalp is not sufficient to induce an action potential, it is claimed to modulate resting membrane potential and thereby increase spontaneous neuronal activity underneath the anodal electrode and decrease it under the cathodal electrode (Creutzfeldt, Fromm, & Kapp, 1962; Nitsche et al., 2003). Some effects of tDCS have been shown to persist for up to 90 minutes after end of the stimulation (Nitsche & Paulus, 2001). Pharmacological manipulations have implicated neuroplastic mechanisms that may relate to long-term potentiation (LTP) in these long-lasting effects (Nitsche et al., 2003). Other studies have suggested that neurotrophic factors (Fritsch et al., 2010) increase and γ-aminobutyric acid (Stagg et al., 2009) decreases during anodal stimulation.

The potential of tDCS as a tool for modulating cognitive, motor, and behavioral functions has resulted in a fast-paced accumulation of research, broad media coverage, and more than 20 patents for commercial applications (Dubljevic, Saigle, & Racine, 2014; Martins, Fregni, Simis, & Almeida, 2016). Early work showed mixed results, but the authors of recent meta-analyses of studies on healthy populations conclude that anodal tDCS may have concurrent effects on some aspects of cognitive performance (e.g., Hill, Fitzgerald, & Hoy, 2016; Mancuso, Ilieva, Hamilton, & Farah, 2016; Summers, Kang, & Cauraugh, 2016). Numerous researchers have taken promising past results together with the potentially plasticity-enhancing effects of anodal tDCS to suggest that combining stimulation with cognitive training may be a particularly useful application of the technique (e.g., Mancuso et al., 2016; Martins et al., 2016). In the present empirical work, the primary objective was to investigate whether simultaneous anodal tDCS and WM training in older adults improve the key outcomes of training: transfer of improvements to broad cognitive abilities, measured when participants are not receiving tDCS (i.e., offline; that is, sufficiently long after stimulation to exclude direct physiological effects of stimulation).

We used a full factorial design to test the effect of an interaction between tDCS and WM training on the transfer of training gains. Anodal tDCS or sham stimulation was therefore combined with WM training or control training over twenty sessions in a between-subject design. The target of stimulation, the left dorsolateral prefrontal cortex (dlPFC), was selected because of its central role in WM (D'Esposito, Postle, & Rypma, 2000). The empirical investigation was supplemented with a meta-analysis of previous studies that have also reported on the effects of anodal tDCS on change in WM performance and general cognition from pretest to the posttest (i.e., measured offline) immediately following training. Whilst the empirical work focused on an older age group, the meta-analysis also included younger adult samples. Since previous studies have only manipulated anodal tDCS vs. sham tDCS in active training groups, the meta-analysis did not allow for the estimation of the interaction effect but offered a cumulative scientific approach, greater statistical power, and explicit contextualization of the results.

Methods

Participants

We recruited healthy participants between 65 and 75 years of age with no contraindications for tDCS through local newspaper advertisements (see SOM1 for full inclusion and exclusion criteria). Eligible participants (n = 142) provided their informed consent and entered the study, which was approved by the Regional Ethical Review Board in Stockholm (2014/2188-31/1) and conducted in accordance with the Declaration of Helsinki. Participants were randomly allocated to the four experimental groups, using age, sex, and Ravens Progressive Matrices pretest score as stratifiers. Two participants were excluded shortly after entry into the study because they no longer met the study criteria. Seventeen participants dropped out during the study because they could not make the time commitment (n=5), had incidental MR findings (n=4), experienced mild adverse events (mainly skin irritation; n=4), developed an unrelated illness (n=3), or for unknown reasons (n=1). The drop-out rate was similar in the four experimental groups (Table 1). Consequently, 123 participants completed the study and were included in analyses (Table 1). Doing power calculation for the planned structural equation modeling is complicated, and we therefore roughly determined the targeted sample size for detecting a between-within interaction with a traditional ANOVA. The targeted sample size (n = 120) should enable detecting a true interaction effect of the hypothesized kind of 0.4SD with a power of 0.86 (assuming an alpha level of 0.05). A true main effect of stimulation on change of 0.3SD (Mancuso et al., 2016) should be detectable with a power of .90. We deemed these power estimates as satisfactory. To further increase statistical power, we supplemented the empirical study with a meta-analysis.

Table 1.

Demographic information for the four experimental groups.

	tDCS + WM (n=32)		tDCS + control (n=30)		sham + WM (n=33)		sham + control (n=28)
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Age (years)	69.31	2.73	69.87	2.91	69.64	2.97	69.82	2.62
Sex (f/m)	16/16		17/13		22/11		16/12
Education (years)	15.05	3.19	14.29	2.29	14.68	2.86	15.84	4.09
Physical activity (score)1	2.13	0.71	2.21	0.78	2.30	0.64	2.29	0.66
Reasoning ability (score)2	7.06	2.72	6.97	3.02	6.88	2.36	6.93	2.26
Drop-out (n subjects)3	4		5		4		4

¹1<150 min/week, 2>150 min/week, 3>200 min/week.

²Ravens Progressive Matrices score (max = 18).

³Number of drop-outs after entry into the study, reported in addition to final sample size (n)

Experimental design and procedures

In keeping with current recommendations in the field of WM training, we designed a study in which the training was adaptive in nature, targeted theoretically motivated constructs (updating and switching), and promoted process-based over strategy-based improvements by including several training tasks and stimuli sets. The design also included an active control group that received training of equivalent scope but with a different target domain and we evaluated effects of the intervention on change in several cognitive abilities that were statistically represented as latent (i.e., unobserved) variables (or factors) of multiple cognitive tasks (Noack, Lövdén, & Schmiedek, 2014; Shipstead, Redick, & Engle, 2012).

The study employed a 2 (cognitive training; WM training vs. control training) X 2 (stimulation; tDCS vs. sham tDCS) X 2 (time; pretest vs. posttest) mixed factorial design. An average of 19 sessions of adaptive WM training (M=19.29, SD=1.01) or control training (M=19.07, SD=1.32) were completed over four weeks. In these sessions, stimulation was administered while participants were engaged in cognitive training.

The cognitive effects of the intervention were assessed with an extensive test battery that included multiple trained and untrained tasks of each cognitive ability that we aimed to assess (SOM2 for overview, SOM3 for detailed test descriptions). These tasks formed factors of trained updating and switching (indexed by tasks used during training, but with identical difficulty level for all individuals at pre and posttest), updating and switching with untrained stimuli, updating and switching with untrained task paradigms, verbal and spatial reasoning, episodic memory, and perceptual matching. The test battery was identical at pretest and posttest and was completed over four sessions, each lasting for 150-180 minutes including breaks. Pretest took place two weeks before the intervention started, and posttest was completed in the week that followed the last week of the intervention.

Cognitive training

The cognitive test battery and cognitive training programs were developed in JAVA and pilot-tested prior to being used in the present study. To allow for conclusions regarding the specificity of WM training, the training programs were designed to be equivalent with the one exception of the cognitive domain being trained. One program trained WM and the other trained perceptual matching speed. Participants were blind to the hypotheses about the two training protocols. The WM training focused on two facets of WM: (1) the ability to continuously maintain and update mental representations (updating) and (2) the ability to flexibly switch between different rules/tasks (switching). Switching was trained with task switching and rule switching tasks and updating with n-back and running span tasks (see SOM4 for detailed task descriptions). To prevent stimuli-specific strategies and instead promote improvements in processing efficiency, each WM task alternated between four different stimuli sets. The control training focused on perceptual speed, using four versions of the same perceptual matching test. In both training programs, participants spent approximately 10 minutes on each of the four training tests, which varied in order between training sessions. This resulted in 40 minutes of active training per session. Each training task consisted of a set of runs, which allowed performance to be regularly evaluated against a set criterion and the difficulty level to be increased as participants’ performance improved to meet the criterion (see SOM5 for details on difficulty levels). To ensure a maximal training load, participants always trained at the highest level reached. The average level reached by the end of the intervention, averaged over the four respective tasks, was equivalent for the two training protocols (M_WM=10.877, SD_WM=4.218; M_control=11.289, SD_control=1.923; t(121)=0.682, p=0.496).

To increase motivation, participants’ performance relative to the criterion was presented after each run, and a figural progress indicator informed participants of their current level. Every fifth training session, participants were given a printout of their progress to date. Motivation levels were assessed on a 5-point Likert scale (“How motivated do you feel to solve the tasks today?” 1=not motivated at all, 5=very motivated) and were generally high across the intervention period with no differences between training groups (Mdn_WM=4.474, Mdn_control= 4.416, Mann-Whitney U=1851, p= 0.862).

Brain stimulation

Direct current was delivered using the DC-STIMULATOR PLUS (neuroConn GmbH, Ilmenau, Germany) and was transferred by two saline-soaked surface electrodes placed on the scalp. The anode (7x5cm) was positioned horizontally to target the left dlPFC, which corresponds to F3 in the 10-20 international system for EEG placement. The anode was shifted slightly laterally, towards F5, and slightly posteriorly, such that its superior-anterior quarter section and not the center was positioned over F3. The lateral shift aimed to maximize peak current density underneath F3, and the posterior shift minimized the risk of shunting by ensuring the recommended minimum interelectrode distance of 8cm for all participants (Seibt, 2015). The cathode (7x5cm) was positioned over the contralateral supraorbital area. Electrode placements were based on measurements using the 10/20 BraiNet Placement Cap (Jordan NeuroScience, Inc, California, USA). Before fixing the electrodes with rubber straps, the scalp was prepared by parting any hair, cleaning the skin with disinfectant and saline solution, and subsequently ensuring that the scalp was completely dry with the exception of the electrode areas. Impedance was confirmed to be below 20 kΩ before any stimulation was initiated.

For active tDCS, a constant current of 2mA was delivered for 25 minutes, with an additional 8-second ramp-up and a 5-second ramp-down period. For sham stimulation, the same ramping procedure and stimulation intensity was used, but the stimulation lasted for 30 seconds only. The procedure for the active tDCS and sham stimulation was otherwise identical, and both participants and experimenters were blind to stimulation assignment. To avoid distraction caused by starting the stimulation, the training program was initiated five minutes after the stimulation, which left 20 minutes of the stimulation to directly coincide with the training. These 20 minutes correspond to two out of the four tasks. The order of tasks varied from session to session. An assessment of the participants’ stimulation blinding after the last intervention session revealed that it had been successful (52% incorrect guesses, 48% correct guesses; p=0.72 by binomial test).

Side effects of tDCS were evaluated four times during the intervention period with ratings on a 5-point Likert Scale (0= “I did not experience the side effect at all,” 5= “The side effect was so severe that I considered terminating or had to terminate the stimulation”) for the time period before, during, and after the training session. Five direct side effects of tDCS were evaluated: itching, pain, burning, heating, and pinching underneath the electrodes. Ratings averaged over the four evaluations were generally very low; the maximum average was 1.265 (SD=1.200) for burning underneath the electrodes for the stimulation period before the training started. Collapsing the scores across time periods, we found no difference between the direct side effect ratings of participants who received tDCS and those who received sham stimulation (all ps > 0.136 by Mann-Whitney U test).

Data analysis

A latent change score modeling approach (McArdle & Nesselroade, 1994) was adopted to test the effect of training, stimulation, and the interaction of training and stimulation on change in cognitive performance from pretest to posttest (Figure 1). Ability factors that represented the shared variance among multiple tests measuring the construct were formed of the pre and posttest data, and a latent change score, which represented the difference between pre and posttest performance, was estimated. This allowed for change to be estimated as a latent (i.e., unobserved) variable (i.e., factor), attenuating reliability problems of change scores and allowing for task-specific variance to be reduced in favor of task-general (ability) variance. The pretest and change factors were regressed on the predictors (stimulation, training, stimulation x training) as shown in the graphical representation in Figure 1. Active tDCS was coded as 1 and sham as -1. WM training was coded as 1 and control training as coded as -1. We estimated a separate model for each of the considered cognitive abilities. See SOM6 for means and standard deviations for each separate task as a function of group.

Figure 1.

Graphical representation of the latent change score model used to assess effects of training, stimulation, and their interaction on cognitive performance. Observed variables are represented by squares, latent variables by ellipses, and residuals by circles. e1-e6 represent the residual of the observed variables (error terms), and d1-d2 represent the residual of the latent variables for pretest and change (disturbance terms). Regression weights are represented by 1-headed arrows and covariances by 2-headed arrows. Regression weights marked with ones were restricted to 1. For variables marked with zeros, intercepts were restricted to equal 0. All other regression weights, covariances, and intercepts were estimated.

Before estimation, we screened all variables for univariate outliers using the outlier labelling rule (with a G-factor of 2.2). Detected outliers were deleted using pairwise deletion (see SOM6 for effective sample size for all variables). The resulting scattered missing values were accommodated under the missing-at-random assumption using full information maximum likelihood (FIML) estimation in AMOS 23.0.0 (Arbuckle, 2014).

Measurement invariance over time is important for the interpretability of results, as it ensures that the same latent variables are represented on each measurement occasion (Meredith & Teresi, 2006). Weak, strong, and strict levels of measurement invariance were assessed by sequentially restricting the factor loadings, the intercepts of the observed variables, and the residuals of the observed variables to be equal at pretest and posttest. Results were reported for the highest level of measurement invariance admissible, and models were screened for Heywood cases. Updating with untrained stimuli, updating with untrained tasks, trained updating, switching with untrained tasks, and sustained attention all met the criteria for strict invariance (all χ² weak vs. free ≤ 2.196, ps ≥ 0.138; all χ² strong vs. weak ≤ 1.209, ps ≥ 0.272; all χ² strict vs. strong ≤ 3.386, ps ≥ 0.184). For updating with untrained tasks, the free and weak invariance models failed to converge, which means that some caution must be used in the interpretation of this variable at the strict level of invariance. For trained updating, the residual variance was estimated to be zero for two of the observed variables (e2, e4), and results were therefore reported for a model in which the covariance between the residuals was fixed to zero. The switching with untrained tasks was not considered further because of substantial negative residual variance in the observed variables in the strict, strong, and weak invariance model (all e1 and e3 ≤ -10.209). Similarly, sustained attention was not considered because of zero or negative estimates of residual change in all invariance models, with and without the predictors included in the model (all estimates ≤ 0). Verbal reasoning and episodic memory met the criteria for strong invariance (all χ² weak vs. free ≤ 4.597, ps ≥ 0.059; all χ² strong vs. weak ≤ 1.76, ps ≥ 0.185). For episodic memory, the weak invariance model was nevertheless selected because of negative residual variance estimates for the observed variables in the strong invariance model (e2=-0.5220, e4=-2.200) that were not present in the weak invariance model (e1-e4 ≥ 6.745). Spatial reasoning, trained switching, and perceptual matching speed met the criteria for weak invariance (all χ² weak vs. free ≤ 1.537, ps ≥ 0.272). Switching with untrained stimuli did not meet the criteria for weak invariance and was therefore not considered (χ²=4.71, p=0.030). Thus, for evaluating the effects of training, stimulation, and their interaction we considered the ability factors of trained updating, trained switching, updating with untrained stimuli, updating with untrained tasks, spatial reasoning, verbal reasoning, episodic memory, and perceptual speed. All these models had good fit (RMSEA <0.06; CFI >0.95; see SOM7) and the loadings of the tasks on the latent factors were generally high (all standardized loadings > .48) and significant.

Statistical significance of the training, stimulation, and training X stimulation effects was assessed using χ² difference test, contrasting a model in which the relevant effect was restricted to zero with a model in which the effect was estimated freely. To deal with multiple comparisons, statistical significance was reported relative to a Bonferroni corrected α-level (α=0.00625 given from eight finally considered models). For the sake of completeness, any effects below the traditional α-level of 0.05 are mentioned in the text. Standardized βs (β_std), which can be interpreted as correlations, are reported as effect sizes.

Meta-analysis

The meta-analysis was designed in accordance with the statement for systematic reviews developed by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (www.prisma-statement.org; see SOM8 for literature search flow). The online databases Web of Science and PubMed were searched on 21 April 2016 using the key words “transcranial direct current stimulation” or “tdcs” combined with each of the following: “training,” “memory,” “cognit*,” “practice,” “longitudinal,” and “learning.” The reference sections, relevant reviews, and reports were also searched for eligible studies.

Empirical investigations in any report format published in English were eligible. Eligible samples were healthy adults aged 18 or older. Research on non-human subjects and clinical conditions, qualitative studies, and non-empirical publications were excluded. Randomized sham-controlled studies using anodal tDCS in combination with cognitive training over a minimum of two sessions and pre-post testing without stimulation were included. Since learning is a continuous process with unknown carryover and interaction effects with stimulation, studies employing within-subject designs were excluded.

Eligible outcome measures had to assess cognitive performance before and within two weeks after the training period without tDCS. We focused on obtaining one average measure of WM performance (summarizing performance on all tasks measuring WM performance) and one measure of global cognition (summarizing performance on all cognitive measures reported) per study. Since the types and number of outcomes measures varied across tasks and reports, a priori criteria were used to select dependent variables: Accuracy measures were favored over reaction time measures unless performance was near or at ceiling, which was defined as the maximum mean at posttest being within 1 SD of the maximum of the measurement scale. For studies that reported multiple accuracy-based measures, measures such as d’, which combine hit rates with false alarm rates, were preferred. See SOM9 for a complete list of the selected outcome measures for the analysis on WM and global cognition. To arrive at a single effect size per study, effect sizes from multiple task conditions were averaged before averaging effect sizes across tasks. When studies included multiple groups that received anodal stimulation of different brain regions, we selected the group with stimulation sites most similar to the reported empirical study (e.g., left rather than right dlPFC and dlPFC rather the parietal stimulation) for primary analysis. Secondary analysis was conducted on results collapsed across all available groups receiving anodal stimulation.

The analyses were conducted using the “metafor” package (Viechtbauer, 2010) installed in the R 3.3.0 environment (R Core Team, 2016). As described and recommended by Becker (1988), the difference in standardized mean change from pretest to posttest for the tDCS group and the sham group was calculated for all selected outcome measures using raw score standardization with

$$g=g_{tDCS}-g_{sham}$$

where

$$g_{tDCS}=\left(c\left(n_{tDCS}-1\right)\frac{{\overline{x}}_{post,tDCS}-{\overline{x}}_{pre,tDCS}}{S{D}_{pre,tDCS}}\right)$$

$$g_{sham}=\left(c\left(n_{sham}-1\right)\frac{{\overline{x}}_{post,sham}-{\overline{x}}_{pre,sham}}{S{D}_{pre,sham}}\right)$$

where x̄_post,tDCS and x̄_pre,tDCS are the means at posttest and pretest for the tDCS group, SD_pre,tDCS is the standard deviation of the pretest scores, c(n −1) is a bias-correction factor (equation 5 in Morris, 2000), n_tDCS is the sample size of the tDCS group, and x̄_pre,sham, x̄_post,sham, SD_pre,sham and n_sham are the analogous values for the sham group. The sign for g_tDCS and g_sham were assigned so that a high value represented an improvement in performance in all outcome measures. A positive value for g therefore indicated greater gains from pretest to posttest in the tDCS group. All of the analyses were also repeated with an alternative effect size standardized based on the pooled pretest SD (d_ppc2 in Morris, 2008). This analysis resulted in similar conclusions (see SOM10).

Sampling variance was estimated with equation 13 in Becker (1988). Since all necessary pretest-posttest correlations could not be obtained from the individual studies, a pretest-posttest correlation of 0.5 was assumed. Analyses were performed with correlation coefficients of 0.2 and 0.9 to assess the dependence on this assumption. Since the decision regarding statistical significance did not change, results were reported for r = 0.5 only. The standard inverse-variance method for random-effects models was used to weight the effect-sizes when estimating the final outcome. Heterogeneity was evaluated with an extension of the Cochran’s Q test, Tau² and I², in order to assess significance, between-study variance, and the ratio of true heterogeneity to total variation in the observed effects. Publication bias was tested in a mixed-effects meta-regression model for funnel plot asymmetry using standard error as a predictor.

Results

Empirical Investigation

The results revealed a statistically significant main effect of training (WM training vs. control training) on change in cognitive performance from pretest to posttest for the latent cognitive factors of trained updating, trained switching, updating with untrained stimuli, and perceptual matching speed (all β_std > 0.52; ps < 0.001; see Table 2 for all individual effects). Participants who received WM training improved more in the tasks that they had trained on (trained updating and switching) and in similar tasks with new stimuli (updating with untrained stimuli). Participants who received the control training improved more in tasks that they had trained on (perceptual matching speed).

Table 2.

Effects of predictors (Stimulation, Training, Training x Stimulation) on cognitive performance at pretest and on cognitive change derived from the eight considered models.

		Unstandardized effects (SE) Standardized effects
Variable	Overall Mean (SE)	Stimulation	Training	Training x Stimulation
Updating Trained 1
Pretest	1.274 (0.012)*	-0.002 (0.010)	0.007 (0.010)	-0.017 (0.104)
		-0.017	0.068	-0.161
Change	0.074 (0.008)*	-0.011 (0.007)	0.034 (0.007)*	0.008 (0.243)
		-0.169	0.526	0.1130
Switching Trained 3
Pretest	30.613 (0.737)*	0.093 (0.565)	0.433 (0.568)	-0.264 (0.565)
		0.016	0.073	-0.044
Change	15.011 (0.532)*	-0.487 (0.370)	6.486 (0.502)*	0.317 (0.370)
		-0.068	0.904	0.044
Updating Untrained Stimuli 1
Pretest	1.292 (0.013)*	0.004 (0.001)	0.009 (0.001)	-0.005 (0.001)
		0.040	0.084	-0.050
Change	0.107 (0.009)*	-0.015 (0.008)	0.059 (0.008)*	-0.007 (0.008)
		-0.198	0.761	-0.096
Updating Untrained Task 1
Pretest	1.105 (0.047)*	-0.005 (0.036)	0.030 (0.036)	-0.005 (0.036)
		-0.015	0.097	-0.015
Change	0.159 (0.032)*	0.007 (0.020)	0.004 (0.020)	-0.012 (0.020)
		0.129	0.078	-0.223
Spatial Reasoning 3
Pretest	6.961 (0.232)*	-0.091 (0.194)	0.019 (0.194)	0.091 (0.194)
		-0.046	0.010	0.046
Change	0.916 (0.916)*	-0.018 (0.103)	0.113 (0.103)	-0.200 (0.103)
		-0.050	0.322	-0.570
Verbal Reasoning 2
Pretest	5.223 (0.199)*	-0.002 (0.173)	0.196 (0.174)	0.093 (0.173)
		-0.001	0.117	0.056
Change	0.589 (0.097)*	0.057 (0.083)	-0.165 (0.084)	-0.016 (0.083)
		0.154	-0.448	-0.045
Episodic Memory 3
Pretest	16.842 (0.419)*	0.079 (0.270)	-0.012 (0.265)	-0.495 (0.337)
		0.038	-0.006	-0.235
Change	0.011 (0.349)	0.097 (0.238)	0.023 (0.234)	0.524 (0.304)
		0.072	0.017	0.390
Perceptual Speed 3
Pretest	19.439 (0.556)*	-0.252 (0.406)	0.324 (0.406)	0.119 (0.405)
		-0.064	0.082	0.030
Change	9.531 (0.486)*	-0.032 (0.317)	-4.484 (0.482)*	-0.149 (0.317)
		-0.006	-0.864	-0.029

^*p_corrected < 0.00625, derived from χ² difference tests contrasting a model with the relevant effect restricted to zero with a model with the effect being freely estimated. Results were reported at the strict ¹, strong ² or weak ³ level of measurement invariance. Active tDCS was coded as 1 and sham as -1. WM training was coded as 1 and control training as coded as -1.

Importantly, the effect of the critical interaction between training (WM training vs. control training) and stimulation (tDCS vs. sham tDCS) on change in cognitive performance from pretest to posttest was not statistically significant for any of the latent cognitive factors considered, trained or untrained (see Table 2 for all statistics). Thus, the results provided no evidence of the hypothesized greater cognitive improvement from pretest to posttest (i.e., performance measured without stimulation) following tDCS in combination with WMtraining relative to either intervention alone. Furthermore, no main effect of stimulation(tDCS vs. sham tDCS) was detected for any of the latent cognitive abilities, so the experiment provided no evidence of a beneficial effect of multiple sessions of tDCS across training types. Figure 2 illustrates the main outcome of the analyses, demonstrating an example of the effect of training on trained tasks (Figure 2a, which depicts the scores for the factor of the trained switching tasks) but no interaction between training and stimulation for trained (Figure 2a) or untrained tasks (Figure 2b, which depicts the scores for the factor of the spatial reasoning tasks).

Figure 2.

Factor scores extracted from the latent change score models for trained switching (tasks practiced during working memory training) and spatial reasoning (untrained tasks in the domain of spatial reasoning). Error bars represent one standard deviation.

Using an uncorrected α-level of 0.05, a statistical significant effect of training was also detected for verbal reasoning in the unexpected direction of greater improvements for the control training (β_std=-0.448, p=0.049). Similarly, the interaction of training and stimulation on episodic memory was statistically significant at the uncorrected α-level (β_std=0.390, p=0.034), which seemingly reflected improvements in the control group with sham tDCS and the WM group with tDCS, no change in the control group with tDCS, and a worsening of performance in the WM group with sham tDCS. We note that these effects were not predicted and would not persist under our α-level corrected for multiple comparisons (α=0.00625). We consequently refrain from further interpretation of these effects.

There were also no statistical differences between the stimulation groups in the progress through the levels of difficulty in the four training tasks during training/stimulation (see SOM11). This data should be carefully interpreted, however, because the training tasks were designed primarily for training purposes and not for reliably assessing performance and learning curves. For example, the difficulty manipulation is of different magnitude and quality between different levels and the data for the two training paradigms are not comparable.

Meta-analysis

We followed up the empirical investigation with a meta-analysis. Six previous studies met inclusion criteria for the meta-analysis. All of them contrasted the effects of anodal tDCS over the dlPFC with sham stimulation under the same cognitive training conditions (see SOM12 for detailed description and reference list). When we added the results of the WM training arm of the present empirical investigation, a total of seven independent studies were available for analysis, six of which implemented training protocols that targeted WM. Two separate analyses were conducted, one for WM and one for global cognition as the outcome (all cognitive measures, including WM). Total sample size was 131 in the tDCS category and 135 in the sham tDCS category.

There was no evidence of greater change in WM performance from pretest to posttest when cognitive training was combined with tDCS than when it was combined with sham tDCS (g = 0.07, SEM=0.14, p=0.64; 95% CI=[-0.21, 0.34]; Figure 3a). The analysis revealed no statistically significant heterogeneity (Q₇=2.53, p=0.87; I²=0.00%; Tau²<0.01, [0.08]) and no evidence of publication bias (z = -0.39, p = 0.69). Similarly, there were no statistically significant effects on global cognition (g=-0.01, SEM=0.14, p=0.92; 95% CI=[-0.29, 0.26]), and there was no evidence of heterogeneity (Q₇=1.93, p=0.93; I²=0.00%; Tau²<0.01, [0.07]) or publication bias (z = -0.15, p = 0.88). Furthermore, when we restricted the analysis to the five studies that tested young participants, there was no statistically significant difference between the effect of tDCS and sham tDCS on WM (g=0.12, SEM=0.17, p=0.49; 95% CI=[-0.22, 0.46]) or global cognition (g=0.03, SEM=0.17, p=0.86; 95% CI=[-0.30, 0.36]). There was also no evidence of heterogeneity for WM (Q₄=1.32, p=0.86; I²=0.00%; Tau²<0.01, [0.1]) or global cognition (Q₄=1.47, p=0.83; I²=0.00%; Tau²=<0.01 [SE=0.10]).

Figure 3.

Forest plots showing the individual observed effect sizes with corresponding confidence intervals for working memory performance (a) and global cognitive performance (b). The column with percentages contains the inverse variance weights. The polygon shows the summary estimate based on the random effects model; its outer edges indicate the confidence interval limits.

Collapsing across available groups receiving anodal stimulation (two studies, Au, Katz, et al., 2016; Jones, Stephens, Alam, Bikson, & Berryhill, 2015, included multiple groups), rather than selecting one group per study with most similar tDCS parameters to our empirical study, did not change this picture much (g=-0.05, 95% CI=[-0.31, 0.21] for WM and g = -0.05, 95% CI=[-0.30, 0.20] for global cognition). Results were also not substantially different when restricting studies to those employing traditional WM training (i.e., excluding Looi et al., 2016): g=0.03, 95% CI=[-0.26, 0.32] for WM and g=-0.06, 95% CI=[-0.34, 0.22] for global cognition.

Discussion

The empirical study reported here allowed us to dissect the effects of stimulation (tDCS, sham tDCS), cognitive training (WM training, control training) and, critically, their interactive effect on change in cognitive performance (assessed off stimulation) in older individuals. The cognitive test battery included more than one measure per construct of interest, which enabled us to model cognitive change at the ability level (i.e., as a latent factor) and therefore to reduce task-specific influences in favor of task-general effects (Noack et al., 2014; Shipstead et al., 2012). The analyses provided no statistical evidence that stimulation and cognitive training interacted to affect any of the cognitive domains we considered. The stimulation thus failed to modulate either training gains (assessed off stimulation at pretest and posttest) or the transfer of gains to untrained tasks or domains after WM training. Moreover, stimulation did not provide an advantage over sham stimulation in any type of training (WM or control training), which calls into question the overall usefulness of dlPFC tDCS in combination with cognitive engagement for causing improvement in cognitive performance in older age that last beyond possible acute stimulation effects.

When we aggregated the results of the empirical investigation with the results of previous studies (n_total=266; n_tdcs=131, n_sham=135), tDCS combined with cognitive training did not improve, in a statistically significant way, either WM or global cognition (assessed offline) more than sham tDCS combined with cognitive training. It should be noted, though, that because of the small number of studies available, the meta-analysis had limited power to detect effects, evidence of heterogeneity, and publication bias. However, effect-size estimates, and their confidence intervals, for differential change in WM performance (g = 0.07, 95% CI=[-0.21, 0.34]) and global cognition (g=-0.01, 95% CI=[-0.29, 0.26]) suggest that a positive and general effect of current tDCS protocols on offline cognitive performance measured immediately after cognitive training is not very likely, or is at least likely to be small.

The lack of an effect in the meta-analysis appears inconsistent with the results of a recent meta-analysis of ten studies, which provided support for the hypothesis that left dlPFC stimulation coupled with WM training over several sessions has a small but significant effect on subsequent WM performance (g=0.29, 95% CI [0.06, 0.52]; Mancuso et al., 2016). The authors, however, noted that relatively few additional studies with non-significant findings would have rendered their finding non-significant. Here we added the current empirical report and a few recently published studies (Au, 2016; Jones et al., 2015; Looi et al., 2016), which may explain the inconsistency. Other than the inclusion of studies published after the analysis by Mancuso and colleagues, other discrepancies in study inclusion criteria are likely to contribute to the inconsistency. Because of the importance of repeated practice for training gains, we included only studies that had a minimum of two tDCS sessions combined with cognitive training. Furthermore, the analysis was restricted to studies with between-subject designs. The method used to calculate effect sizes may also have contributed to the inconsistency. Here, effect sizes reflected differential change in performance from pretest to posttest in the tDCS group relative to the sham group (Becker, 1988; Morris, 2000).

Although we found little support for the hypothesis that WM training combined with tDCS is superior to training combined with sham tDCS, the empirical study did demonstrate that WM training resulted in greater gains in trained tasks across stimulation protocols. Participants who trained switching and updating during the intervention period demonstrated greater gains in the trained tasks, for trained and untrained stimuli sets alike. Participants in the control group demonstrated greater gains in the trained perceptual matching tasks. However, we found no statistical evidence that gains from WM training generalized to broad cognitive abilities, as evidenced by a lack of an effect of training on the factors of untrained tasks. Available meta-analyses on cognitive training also converge on finding smaller effects on transfer tasks than on trained tasks, and the evidence on transfer effects is heavily debated (Au, Buschkuehl, et al., 2016; Au et al., 2015; Karbach & Verhaeghen, 2014; Melby-Lervag & Hulme, 2015; Melby-Lervag et al., 2016; Shipstead et al., 2012; Simons et al., 2016). In the present empirical work, statistically differential gains from WM training relative to control training did not even extend to untrained tasks that nevertheless tapped the trained abilities. This suggests that the training gains in this study were mostly restricted to task-specific knowledge and strategies, and that there were limited effects on processing efficiency (Lövdén et al., 2010).

The reported results should not be generalized beyond the specific conditions and designs of the considered studies. For example, we did not address effects of anodal tDCS on performance and learning rate during stimulation. Furthermore, although our empirical study showed no statistically significant effects on trained tasks (assessed before and after training without concurrent stimulation), the meta-analytic outcomes mixed transfer and training tasks and are therefore not informative of effects on trained tasks per se. We also note that several of the studies included in the meta-analysis have reported beneficial effects of anodal tDCS combined with cognitive training on select cognitive tasks and time points (e.g., at maintenance assessments). Here we focused on what is arguably the primary outcome of WM training: improvements to broad cognitive abilities and global cognitive performance immediately after the intervention. Future confirmatory work should address whether effects are limited to certain cognitive abilities or tasks and whether they materialize at time points other than immediately after the intervention period.

We also note that a major challenge in the interpretation of the results reported here and in the tDCS field at large is the incomplete knowledge of the mechanism that may underlie effects of tDCS and how tDCS can be optimized to modulate behavior and cognition (Fertonani & Miniussi, 2016). At a basic level, it is possible that the amount of current entering the target region was insufficient to produce the intended effects in our empirical study. Although we were careful to follow current recommendations on how to apply tDCS to optimally target the dlPFC, we cannot exclude the possibility that shunting or electrode drift prevented a sufficient current dose from entering the target region (Miranda, Lomarev, & Hallett, 2006). It is also possible that other parameters, such as intensity or duration, may have been suboptimal. For example, in our empirical study, ethical considerations limited the stimulation period to 20 minutes, which left another 20 minutes of training without concurrent stimulation. Although there has been evidence to suggest that effects can outlast the stimulation period itself (Nitsche & Paulus, 2001), the impact of this procedure versus continuous stimulation during training is unknown.

Inter-individual differences are another important consideration (e.g., Wiethoff, Hamada, & Rothwell, 2014). Gross anatomical features and microarchitectural features influence tDCS current distribution and vary between individuals (Kim et al., 2014). It is particularly relevant to the present empirical study, which investigated effects in an older sample, that tDCS response may differ in older and younger adults (Heise et al., 2014). Although we arrived at unchanged statistical decisions when the meta-analysis was restricted to studies with younger adults, and we must conclude that any true effect in younger adults is also likely to be small, the point estimates were slightly larger in this sub-analysis (for working memory: g = 0.12, 95% CI=[-0.22, 0.46]; for global cognition: g=0.03, 95% CI=[-0.30, 0.36]). We therefore underscore that he true influence of age on tDCS effects remains unknown and that the results of our empirical study should not be generalized beyond the target population of older adults.

The contribution of the present work to the field of tDCS is both timely and needed. The attractiveness of the technique as a safe and effective modulator of cognitive function has been as seductive to the research community as it has been to the media (Dubljevic et al., 2014; Fertonani & Miniussi, 2016). A growing number of people in the general public, presumably inspired by such uninhibited optimism, are now using tDCS to perform better at work or in online gaming, and online communities offer advice on the purchase, fabrication, and use of tDCS devices (Batuman, 2015). Unsurprisingly, fast-paced commercial exploitation is currently underway to meet this new public demand for cognitive enhancement via tDCS, often without a single human trial to support the sellers or manufacturers’ claims (Malavera, Vasquez, & Fregni, 2015). Although tDCS may be beneficial in some contexts, we conclude that current frontal anodal tDCS protocols do little to improve the primary outcomes of working memory training. These results lead us to call for a more cautious appraisal of the potential applications of tDCS.