Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

Aleksandra Petrovskaya; Bogdan Kirillov; Anastasiya Asmolova; Giulia Galli; Matteo Feurra; Angela Medvedeva

doi:10.1371/journal.pone.0235179

. 2020 Dec 9;15(12):e0235179. doi: 10.1371/journal.pone.0235179

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

Aleksandra Petrovskaya ^1,^2,^¤,^#, Bogdan Kirillov ^3,^*,^#, Anastasiya Asmolova ^1,², Giulia Galli ⁴, Matteo Feurra ^1,^2,^‡, Angela Medvedeva ^5,^‡

Editor: Stephen D Ginsberg⁶

PMCID: PMC7725363 PMID: 33296363

Abstract

We aimed to replicate a published effect of transcranial direct-current stimulation (tDCS)-induced recognition enhancement over the human ventrolateral prefrontal cortex (VLPFC) and analyse the data with machine learning. We investigated effects over an adjacent region, the dorsolateral prefrontal cortex (DLPFC). In total, we analyzed data from 97 participants after exclusions. We found weak or absent effects over the VLPFC and DLPFC. We conducted machine learning studies to examine the effects of semantic and phonetic features on memorization, which revealed no effect of VLPFC tDCS on the original dataset or the current data. The highest contributing factor to memory performance was individual differences in memory not explained by word features, tDCS group, or sample size, while semantic, phonetic, and orthographic word characteristics did not contribute significantly. To our knowledge, this is the first tDCS study to investigate cognitive effects with machine learning, and future studies may benefit from studying physiological as well as cognitive effects with data-driven approaches and computational models.

1 Introduction

Transcranial direct-current stimulation (tDCS), a method of safely and non-invasively delivering a weak electric current through the cortex, has been gaining increasing attention [1] as a tool for studying and possibly enhancing episodic memory [2]. Anodal tDCS (atDCS) involves increasing cortical excitability in a target region such as the dorsolateral prefrontal cortex (DLPFC), which may in turn facilitate or enhance memory performance by decreasing reaction time and/or increasing memory accuracy [3, 4]. However, there remains well-founded skepticism about widespread applications of tDCS, partly because of the lack of knowledge about mechanisms of action [5] and difficulty in replicating results [6].

Most tDCS studies in episodic memory have targeted the DLPFC (see [7] for a review) because of its role in selective attention [8], strategic retrieval [9], and other executive functions involved in episodic memory [10]. In line with the hemispheric encoding/retrieval asymmetry model [11], previous studies with tDCS show functional asymmetry in the DLPFC, with stimulation over the left DLPFC affecting encoding and stimulation over the right DLPFC affecting retrieval [12–14]. However, memory enhancements due to atDCS have been inconsistent over the DLPFC, in both individual studies [4, 10, 12, 13, 15] and meta-analyses [16–18]. Most studies over the DLPFC examined effects when atDCS was delivered predominantly during encoding. At least two studies examined purely offline encoding effects: [19] delivered atDCS over F3 for 15 minutes before encoding and found no effects on accuracy or reaction time, although atDCS over the contralateral hemisphere (F4) led to faster reaction times. Lu and colleagues [20] delivered atDCS over a more lateral site (FC5) for 20 minutes before encoding and found an increase in correct memory responses for previously-presented items.

To the authors’ knowledge, no study to date has systematically examined effects of timing of administration on the DLPFC, comparing online vs offline effects at encoding or retrieval. However, a previous study by the authors showed significant differences in online and offline effects over an adjacent site, the left VLPFC [20]. The authors administered atDCS before the study phase (offline encoding) or during the study task (online encoding) that involved intentionally memorizing each presented word (Experiment 1). As expected given support from fMRI and TMS studies, the left VLPFC seemed to be strongly modulated by atDCS at encoding, specifically online but not offline encoding.

The VLPFC and DLPFC are thought to play functionally distinct roles in long-term verbal memory, and the left VLPFC appears selectively engaged in verbal but not non-verbal material. Specifically, the VLPFC may be more involved in encoding for individual items, while the DLPFC is more engaged in associative or relational encoding [21–24]. Moreover, activation in the DLPFC may predict long-term memory success through DLPFC involvement in domain-general working-memory or executive processes such as mental manipulation of information (e.g. visualizing rotating objects; [25]) and applying a specific strategy (e.g. for retrieval of information; [26]). In a tDCS study conducted by [27], tDCS was applied over the left DLPFC in a cued-word-completion task to modulate memory performance. The results showed that cathodal stimulation inhibited memory accuracy after error learning in comparison with sham: the errors were evoked through guessing, or the incorrect completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect). They concluded that the DLPFC was only modulated when its processing demands were higher for conflicting information. In contrast, the VLPFC appears to be engaged in maintaining information in working memory [28] and processing semantic information including accessing lexical identity and connecting words to other words in the semantic network [29].

The current study involved administering atDCS over the left DLPFC during offline or online encoding to identify the effective time of administration and confirm the causal role of the DLPFC in encoding processes. Participants were assigned to a Sham group (over the DLPFC) or one of three tDCS groups: DLPFC Online, DLPFC Offline, and VLPFC Online. Previous research suggests that atDCS could potentially enhance memory performance when delivered online or offline encoding over the DLPFC [30–34], so we predicted higher memory performance for either condition (DLPFC Online or DLPFC Offline). In addition, the study provides a novel comparison of atDCS effects during encoding over the VLPFC vs DLPFC. We predicted a successful replication of Experiment 1 from [20] with higher memory accuracy for the VLPFC Online group compared to Sham, but we also predicted a larger effect size for VLPFC Online than DLPFC Online or DLPFC Offline based on previous rTMS studies comparing memory disruption in the VLPFC vs DLPFC. For example, [35] found that stimulation over the VLPFC during encoding led to a greater disruption in memory performance than over the DLPFC, suggesting that the VLPFC may play a more important role in encoding processes. Finally, there remains a lack of knowledge about the mechanisms of atDCS on verbal memory and whether atDCS effects can be influenced by word characteristics, specifically semantic (meaning), orthographic (letters), and phonological (sounds). We also aimed to examine how the tDCS effects interacted with language and linguistic factors, since the replication study was conducted in Russian and the original study was conducted in English. Thus, data were applied from a previously-published experiment ([20]; Experiment 1) and the current experiment from the Online Encoding (VLPFC) group to an AUROC analysis. After comparisons of group means in reaction time and accuracy on the recognition test, a regression algorithm was applied to infer the quality of recognition using semantic and phonological features of words. Semantic and phonological features were added to the model and predictive capability was assessed. We predicted that the model would be able to distinguish between words that are hard and easy to remember in Russian and English speakers. Thus, machine learning algorithms were applied in order to reveal the impact of factors including semantics, phonetics and individual variance to episodic memory performance as well as the interaction with tDCS.

2 Methods and materials

In line with a replication study, we followed the same procedure as in Experiment 1 of [20] with few exceptions (translation of materials to Russian and comparison of DLPFC and VLPFC as stimulation sites). Broadly, participants memorized words presented individually on a screen while undergoing tDCS (before or during this phase), and 24 hours later they performed a recognition test. The study and test stimuli were translated based on the first word meaning in the vocabulary entry, and the translation achieved relatively matched frequency (M_English = 24.47; M_Russian = 40.38; [36]) and number of letters (M_English = 6.17; M_Russian = 6.19). Specifically, we applied the same tDCS settings (2 mA active tDCS and 30 s Sham tDCS) over the DLPFC (all conditions: offline, sham, and online) in addition to the VLPFC (online encoding only). The electrodes were placed at both sessions (study and test 24 hours later), but stimulation was only delivered before or during the encoding task in the study phase. The anode was placed over the left VLPFC (F7) or the left DLPFC (F3), and the cathode was placed over the ipsilateral (left) shoulder. See Section 2.1 below for the experimental conditions. Stimulation was delivered via a battery-powered, constant-current stimulator (Brainstim, EMS, Bologna, Italy) through 5 × 7cm² electrodes. In addition, participants were asked to describe the sensations that they felt during the stimulation and indicate whether they believed they received real or placebo stimulation. Participants were single-blinded, and the questionnaire indicated that blinding was successful: 80% believed that they received real stimulation. Data were analyzed in SPSS (version 24; IBM, Armonk, New York) and machine learning studies were conducted in Python (version 3.7).

2.1 Participants

Participants were randomly assigned to one out of four groups: VLPFC Online, DLPFC Online, DLPFC Offline and Sham. Based on a power analysis to detect a large effect size [20], d = 1.29;α = 0.05, 1 − β = .95), we aimed to recruit 31 participants per group (124 total), but we analyzed data from 97 participants because of exclusions (see below). Participants (female = 63, male = 49, M_age = 20.51, SD = 2.89) were native Russian speakers with normal or corrected-to-normal vision and no history of neurological or psychiatric illness. Bonferroni-corrected pairwise comparisons of ae between pairs of groups showed a significant difference in age between VLPFC Online (M = 21.76, SD = 3.54) and DLPFC Online (M = 19.21, SD = 1.10), p = .014. However, there were no significant differences in age between the other groups, p_s > .062.

Data from 12 participants were excluded from analysis because 1) the participant was feeling unwell (2 participants: DLPFC Sham and VLPFC Online) 2) there were technical issues (6 participants: two in DLPFC Online, two in VLPFC Online, and two in DLPFC Sham) or 3) the participant was left-handed (one participant: DLPFC Online) and 4) there was an experimenter error (3 participants: one in DLPFC Sham and two in VLPFC Online). The exclusions resulted in 26 in the VLPFC Online group, 31 in the DLPFC Offline group, 28 in the DLPFC Online group, and 27 in the Sham group. Moreover, fifteen outliers were excluded (see data analysis for criteria), leaving a final sample size of 25 for VLPFC Online (one outlier), 25 for DLPFC Offline (six outliers), 23 for DLPFC Online (five outliers), and 23 for DLPFC Sham (four outliers). For all Machine Learning analysis all DLPFC Offline participants except one were used (30 participants).

The study was approved by the ethics committee of National Research University Higher School of Economics (Moscow, Russia) and followed the corresponding ethical guidelines. All participants provided written informed consent and were given monetary compensation (500 rubles) for their time.

2.2 Procedure and experimental design

Each participant, regardless of group assignment, came to the laboratory twice within a 24 hour-interval: on the first day participants memorized the verbal stimuli one word at a time through pleasantness judgements (pleasant or unpleasant), and on the second day they performed an old/new recognition memory task. For DLPFC Sham, stimulation was delivered for 30 seconds before the stimulator was turned off, and the start of the stimulation corresponded to the start of the reading task for half the Sham participants and the start of the memory task (study phase) for the other half. Stimulation was applied for the entire duration (10 minutes) of the reading task for the DLPFC offline group and for the entire duration (10 minutes) of the study memory task that immediately followed for the VLPFC Online and DLPFC Online groups. In summary, the stimulation was delivered for approximately 10 minutes to cover the entire duration of the reading and memory tasks.

2.3 Data analysis

One-way ANOVAs were conducted for each measure of recognition accuracy [37]: discrimination index (Pr: combined index involving hits and false alarms), proportion of hits (correctly identifying an “old” item as studied), and proportion of false alarms (incorrectly identifying a new item as studied). In addition, one-way ANOVAs were conducted for average reaction times and reaction times for hits and false alarms separately. Finally, a one-way ANOVA was conducted on response bias, an index involving hits and false alarms (higher values indicate a more conservative pattern of responding to old items). Significant effects were followed up with Bonferroni-corrected planned contrasts between each stimulation group and Sham (one-tailed). Outliers were excluded based on two standard deviations from the mean on any of the following dependent measures: discrimination index Pr, proportion of hits, proportion of false alarms, and reaction times for hits or false alarms.

2.4 Area under the receiver operating characteristic curve as a measure of predictive performance for episodic memory changes

For the second analysis including data from the [20], we included all 17 participants from the Sham group and 17 from the Online tDCS group, resulting in the same mean age with no significant differences between the groups. From the current study, we only included data from the DLPFC Sham (N = 23) and VLPFC Online groups (N = 25) with no significant differences in age between groups (Bonferroni-corrected paired comparisons), p_s > .852. We implemented a 2 × 2 design with the first factor as language (English or Russian) and the second factor as tDCS group (VLPFC or Sham).

Applying Area Under the Receiver Operating Characteristic Curve (AUROC) was appropriate given that the task involved binary classification (coded as 1 if the word was presented at study and 0 if unpresented), which fits any ML approach to measuring predictive performance of classification models [38]. In ML Classification models, true positives represent the model predicting the positive class correctly and false positives represent incorrect prediction of positive class.

AUROC 1) provides information about true and false positives in a single measure 2) shows the source of model error, with larger values indicating that the model predicts better than chance (greater true positives) and 0.5 indicating the opposite and 3) does not assume a normal distribution and is robust to unequal sample sizes (whereas average accuracy measures would overestimate performance). Thus, it is the most suitable measure for the current data.

We included a trial-based approach, computing AUROC for each word based on the entire sample’s responses for that word, and a participant-based approach, computing AUROC for each participant based on responses (old or new) for all words presented to that participant.

AUROC values can show whether tDCS is effective in enhancing memory performance, and the subsequent ML analyses using AUROC can show the effects of various factors (semantic, phonetic, orthographic word characteristics) on recognition of individual words. We attributed any differences in memory performance not explained by differences in sample size, linguistic characteristics, or group (tDCS vs Sham) to individual differences in memory function.

If stimulation enhances episodic memory performance, we expect a significant increase or decrease in AUROC (both over words and over participants in both VLPFC groups compared to Sham). If tDCS impairs memory performance, we expect values close to 0.5, which reflects that participant response decisions (old or new) approached chance.

If tDCS has no effect, there would be no difference in AUROC values across words or AUROC distributions between VLPFC and Sham. Since tDCS enhanced episodic memory in the original experiment and we expect a successful replication, we expected AUROC values for VLPFC to be significantly different from chance, with significantly different AUROC values and distributions compared to Sham. Moreover, tDCS significantly increased reaction time in [20], so we predicted a significant difference between VLPFC and Sham in reaction time. Since we did not assume a normal distribution, we applied the two-sample Kolmogorov-Smirnov test for accuracy and reaction time. We have also computed median test for differences in median AUROC.

2.5 Extraction of semantic, phonological and orthographic determinants of episodic memory performance

To investigate the effect of semantic and phonetic features on memorization and the interaction with tDCS, we extracted words with significantly more true positive rates (reflecting hits) and false positive rates (reflecting false alarms) in each group (VLPFC and Sham). We examined the top ten words from each category (see S1 Appendix). A preliminary descriptive analysis of the highest and lowest AUROC-words showed no association between individual variance and memory accuracy in VLPFC and Sham groups. Therefore, we do not report these results further.

Moreover, we used word embeddings, which approximated the semantic similarities between words. In line with the trial-based and participant-based approaches, we conducted two kinds of ML studies: 1) Participant-independent (trial-based) analysis tests the success of a model that predicts the AUROC for each word using either word embeddings or letter-based one-hot encoding (each letter i is replaced by a vector of N_alphabet components with the i-th component being equal to 1 while every other component is zero.) and 2) Participant-specific (participant-based) analysis tests the success of a model that predicts whether a participant was able to recall the word given either word embeddings or letter-based one-hot encoding for each word for individual participants. Word embeddings were applied from FastText [39], a library developed in Facebook that incorporates semantic information from each word as well as subwords contained within to embed vectors. The pretrained embedding was aligned following [40] so that English and Russian vectors could lie in the same vector space and we could use it to train a model that uses both English and Russian vectors to check whether there are any language-independent semantic determinants. To artificially enlarge the sample size for our models, we use the augmentation approach described in [41] (see S2 Appendix for details).

For prediction, we applied TPOT [42], an automated Machine Learning library that enables searching for the best classification and regression model using Evolutionary Algorithms, and AutoPyTorch [43], an automating Machine Learning library for the PyTorch [44] deep learning framework. Pipelines found by TPOT or AutoPyTorch are usually better than those found by grid search or manual construction. TPOT was used for experiments with word vectors, and AutoPyTorch was used for experiments with one-hot encoded word images.

We applied the following parameters: for TPOT, default ones for evolutionary algorithm; 20 for number of generations and 5 for population size; 10 folds for cross-validation; mean absolute error as the scoring function, and “TPOT light” as the configuration dictionary, for AutoPyTorch, default validation setup, “tiny-cs” config for participant-independent trials and “medium-cs” config for participant-dependent trials.

Prior to training, for each experiment, we have randomly extracted 10% of the dataset and used it as a hold-out set for test. We trained our AutoML on the rest 90% with the respective cross-validation schemes and we examine the generalization ability of a model by assessing the difference between the quality metrics on training and hold-out sets. The model could perform very well on training set, but show lack of predictive ability on the hold-out. Such an outcome would be a sign of poor generalization ability which would show that based on this data we can not infer the existence of learnable connection between features and a label.

3 Results

3.1 ANOVA

Accuracy was significantly above chance for all groups, t_s > 3.88, p_s < .005. Levene’s test (F_s > 2.84, p_s < .042) and frequency distributions suggested that most data did not follow a normal distribution. Although non-parametric statistics may be more appropriate than an ANOVA, to enable comparison with the original effect sizes, corrected parametric statistics (Brown-Forsythe’s F statistic) are reported in text for corresponding dependent measures. There were no significant differences between groups in recognition accuracy $F (3, 79.78) = 2.37, p = . 076, η_{p}^{2} = . 082$ , response bias $F (3, 93) = 2.15, p = . 100, η_{p}^{2} = . 065$ , or average reaction time $F (3, 74.26) = 0.70, p = . 556, η_{p}^{2} = . 028$ . To enable comparison with the original paper, we report the results of multiple ANOVAs. However, to control for multiple tests and the difference in age between groups, we analyzed all dependent variables including AUROC with a MANOVA, Pillai’s trace (12, 294) = .166, p = .148, and recognition accuracy and average reaction time with age as a covariate, F(3, 98) = .097 and F(3, 98) = .557, respectively. There was no significant difference between groups with hits, false alarms, recognition accuracy, and average reaction time as dependent variables. There was also no significant difference in the proportion of hits $F (3, 76.88) = 1.37, p = . 259, η_{p}^{2} = . 051$ , or associated reaction times $F (3, 75.56) = 0.49, p = . 694, η_{p}^{2} = . 019$ . There was a significant difference in proportion of false alarms F(3, 93) = 2.85, p = .042 but not associated reaction times $F (3, 70.82) = 0.91, p = . 439, η_{p}^{2} = . 037$ . Planned contrasts for false alarms revealed significant differences betzween VLPFC tDCS and Sham t(93) = 2.78 and between Offline DLPFC tDCS and Sham t(93) = 2.14, p = .018, with higher false alarm rates for VLPFC tDCS and Offline DLPFC tDCS. There were no significant differences between Online tDCS and Sham t(93) = 1.36, p = .178. See Table 1 for means and standard deviation for the mean accuracy for each group and Table 2 for means and standard deviation for reaction times.

Table 1. Means and standard deviations for memory accuracy across groups.

	N	Discrimination Pr	Br response bias	Pr hits	Pr false alarms
Online DLPFC mean	24	0.15	0.61	0.68	0.53
Online DLPFC SD	24	0.16	0.14	0.10	0.18
Offline DLPFC mean	25	0.07	0.60	0.63	0.57
Offline DLPFC SD	25	0.12	0.12	0.12	0.14
VLPFC mean	23	0.16	0.56	0.63	0.47
VLPFC SD	23	0.18	0.17	0.16	0.18
Sham mean	25	0.09	0.66	0.69	0.59
Sham SD	25	0.11	0.12	0.11	0.12

Open in a new tab

Table 2. Means and standard deviations for reaction time accuracy across groups.

	N	Average RT	RT hits	RT false alarms
Online DLPFC mean	24	501.49	504.82	498.17
Online DLPFC SD	24	144.91	151.20	139.84
Offline DLPFC mean	25	503.00	505.85	500.16
Offline DLPFC SD	25	103.62	104.71	103.66
VLPFC mean	23	535.72	532.52	538.91
VLPFC SD	23	156.16	142.72	171.43
Sham mean	25	484.00	477.82	484.00
Sham SD	25	82.58	86.10	82.58

Open in a new tab

3.2 AUROC analysis and ML models

The range of performance as measured by AUROC (0.38-0.62; see Fig 1) indicates that participant decisions were close to random choice, since an AUC coefficient of 0.75 or higher reliably reflects accurate performance. The VLPFC group shows a similar distribution of individual AUROC coefficients to the Sham group, indicating that there was no effect of tDCS on memorization and perhaps other factors (word characteristics, individual differences, error) contributed more highly.

Although the distributions of AUROC coefficients for individual words appeared to vary depending on the sample (English or Russian) and group (VLPFC or Sham; see Fig 2), with the English VLPFC group showing more variance than either of the Sham groups, there was no significant difference in AUROC distributions between English or Russian words or experimental groups, suggesting that the greater variance in the English group reflected smaller sample size.

Fig 2 — A—all groups (p-value of median test is 0.004), B—sham (p-value of median test is 0.274), C—VLPFC online (p-value of median test is 0.001).

We found significant differences in distributions for reaction time between Russian and English words as well as significant differences in distributions between sham and VLPFC online across both samples, p < .001, but no significant difference in median AUROC (p-values are either way larger than 0.001 or borderline—about 0.001-0.004), suggesting that as for the accuracy data, differences in distribution reflected differences in sample size (see Fig 3).

Finally, we found no significant contributions of semantic, orthographic, or phonological characteristics of words in terms of predicting the success of participant in recalling the word for participant-independent (see S3 Appendix) or participant-dependent (see S4 Appendix) models. The Spearman correlations for predictions and real labels for hold-out set in prediction of word AUROC were close to zero, the AUROCs of participant-independent models for hold-out set were close to 0.5 with small standard deviation, suggesting that the model did not learn anything useful for prediction of experiment outcome.

4 Discussion

The aim of the current study was to replicate and extend the findings of [20](Experiment 1) with the VLPFC and DLPFC. However, the effect of tDCS over the left VLPFC (decrease in false recognition) was not replicated, and there were weak, negative effects over the DLPFC in the Offline group. In fact, there was a trend toward increased false alarms after online tDCS over the VLPFC and offline tDCS over the DLPFC compared to Sham. The trend suggests that an increase in false alarms obscured group differences, and the increase in false alarms could be driven by an increase in semantic elaboration that leads to better memory for features common to multiple items [45, 46]. While the results do not support effects over the DLPFC and VLPFC, this speaks to the lack of tDCS effect rather than the lack of involvement of these regions in episodic memory. Although it is clear from TMS studies that the VLPFC is necessary for episodic memory, TMS disruption of the DLPFC does not consistently impair episodic memory and the specific roles of both regions remain to be clarified. While meta-analyses and studies aimed at replication of tDCS cognitive effects do not support the effectiveness of tDCS generally [47, 48], the majority of tDCS studies lack deeper explorations of non-linear effects and individual differences through modelling and ML [49]. In our case, a limitation was not estimating whether maximum current flow was underneath the anode electrode and at the region of interest (VLPFC or DLPFC). However, the aim of the current study was to clarify the consistency of the effect found in the original study using an identical method, including montage. Future studies could test a montage with greatest likelihood of current flow at the region of interest by modeling current density maxima. More insight is needed into the relationship between biological and cognitive effects. For example, future studies could attempt to classify groups by using concurrent EEG activity during tDCS administration. However, even tDCS-EEG studies alone may not be beneficial because EEG reflects the synchronized firing of large populations of neurons across the brain (not just the stimulated region), and the neural signature is not always modulated by changes in cognitive performance. Thus, more sensitive measures of tDCS cognitive and neural effects can be implemented, such as changes in resting-state connectivity [50], GABA and glutamate concentrations [51], and cerebral blood flow [52]. It remains unclear whether the significant tDCS modulation of long-term verbal encoding in the original study occurred through modulation of related processes such as lexical retrieval, working memory, or attention. For example, tDCS has been shown to modulate the speed of vocabulary learning but not final vocabulary recall over multiple sessions [53]. Moreover, atDCS over the VLPFC led to improved semantic processing in language tasks [32] such as naming common objects [54] and working memory tasks [55]. Thus, tDCS could modulate a semantic process or working memory maintenance that contributes to long-term memory rather than the formation of the memory trace, which is thought to rely more on the hippocampus [56, 57]. Individual differences in baseline memory performance, correlated cognitive functions, and neuroanatomy could obscure the effect of tDCS on measures of memory performance, especially if the effect is subtle. It is clear that effects of tDCS on physiology may not correspond with cognitive effects, and future studies can better explore this relationship with physiological as well as cognitive measures and by estimating the current density maxima. It is notable that using AUROC we could not discriminate hard and easy to remember words in Experiment 1 of [20]: the AUC curve revealed no significant effect of VLPFC tDCS on English or Russian speakers. Although average memory accuracy was above chance, AUC suggested that when considering individual trials, participants were guessing. Future studies can test the reliability of the recognition test as an assessment for tDCS-induced cognitive effects and try to increase participant motivation and concentration to attain optimal trial-by-trial performance for examining tDCS-induced changes. The ML analysis showed that phonological, semantic, and orthographic features did not influence the episodic memory. Although it is likely that these features contributed to memory formation, their influence was minimal and overshadowed by the large interaction between atDCS and individual differences. This individual variability has been examined in other studies that found differences between high and low performers (e.g. [10]). Individual differences in baseline memory performance and encoding ability appear to be an influential factor, more than language or word characteristics. The results indicate that individual differences may be one of the most important parameters not only in the current study and tDCS field but also in replication studies. A limitation was that we did not conduct a within-subjects study in which we examined differences in individual performance. A replication of a within-subjects study could reveal that the effect of a between-subjects study was due to differences between participants and when participants are compared to their own performance, there is little effect of tDCS. Indeed, studies including [20] have found varying effects when comparing between-subjects to within-subjects tDCS effects over the same location, with potentially smaller or absent effects for within-subjects studies (e.g. compare between-subjects results in [30, 32, 58, 59] and Experiment 1 in [20] to within-subjects results in Experiment 3 in [20]; Experiment 2 in [60] and [14]. Furthermore, it is important to identify the reliability of tDCS effects with direct replications within the same lab as well as other labs [61] with sufficiently large sample sizes. However, continued future examinations of tDCS cognitive effects may have less value if not supplemented with a measure of biological effects. Future data-driven studies should aim to predict when physiological effects such as increases in BOLD activation lead to cognitive outcomes such as higher performance and whether duration and frequency of tDCS are involved. Although the sample size of the current study should have had sufficient statistical power to detect the original effect size ([20], Experiment 1), the true effect size could still be eluded because of a statistical phenomenon known as the “winner’s curse”. The winner’s curse posits that the first studies to find a significant and novel effect will be published, and the reported effect will be exaggerated because these studies tend to be exploratory and include smaller sample sizes. If several small-sample studies are investigating the same effect, random error and sampling variation may lead to one but not all of the studies finding an effect that crosses significance threshold because under-powered studies can only find large effects. The true effect is likely to be smaller, so it would not otherwise emerge in under-powered studies. This phenomenon is illustrated well in the meta-analysis by [2], in which most included articles reported at least one significant effect, but the average effect size was close to zero when all studies were included in the analysis. For example, [62] found an effect of atDCS in their first experiment but not in a subsequent replication in the same paper. The conclusion of [63] is in line with the “winner’s curse” effect, while the authors suggest a smaller sample size (between d = .40 and d = .50 according to Cohen, 1988) as more appropriate for tDCS studies. We did not successfully replicate the results of the previous work, although we used a larger sample size and nearly identical method. It is worth mentioning that the results of the replicated study are not significant based on the ANOVA (and this statistical model is appropriate for the original study) or the AUROC analysis applied to both datasets. It does not seem that there was a cultural or linguistic component involved, in line with previous tDCS experiments that found similar effects on verbal memory performance between countries (Italy: [32]; England: [20]; USA: [59]). However, culture-dependent tDCS should be tested directly by comparing individuals of different cultures in the same language. We would expect culture-dependent effects in social cognition (e.g. [53]) but not in processes such as memory encoding that are thought to rely on the same neural architecture across people. In the current study, tDCS did not lead to significant differences in memory performance or reaction time. While it seems unlikely that the significant effect in the original study was due to sample size alone, we conclude based on the lack of replication that the effect of tDCS on long-term verbal memory when the anode is placed over the VLPFC or DLPFC may be subtle. Moreover, the effect may be completely obscured during interactions with other factors that may be difficult to measure or quantify, such as the state of the network. Regression approaches could be successful for future studies that model interactions between biological and cognitive effects. In spite of the significant tDCS effects found in previous memory studies, we suggest caution in interpreting these effects and applying tDCS as a neuromodulator until replications are conducted with biomarkers.

Supporting information

S1 Appendix. Significantly different words.

Russian and English words that differ significantly in the number of hits, false alarms, AUROC and reaction time.

(PDF)

Click here for additional data file.^{(93.6KB, pdf)}

S2 Appendix. Data augmentations.

The procedures used to augment the training set according to the method proused in [41].

(PDF)

Click here for additional data file.^{(127.8KB, pdf)}

S3 Appendix. Participant-independent ML trials.

The performance of participant-independent TPOT and AutoPytorch models.

(PDF)

Click here for additional data file.^{(175.7KB, pdf)}

S4 Appendix. Participant-dependent ML trials.

The performance of participant-independent TPOT and AutoPytorch models.

(PDF)

Click here for additional data file.^{(74.7KB, pdf)}

Acknowledgments

The study used the HSE Synchronous Eye-tracking, Brain Signal Recording and Non-Invasive Brain Stimulation System. We are grateful for the assistance of Anastasiya Bogdanova and Daniil Krivenok in data collection.

Data Availability

All related files are available from the Zenodo database (DOI: 10.5281/zenodo.3837111).

Funding Statement

The article was prepared within the framework of the HSE University Basic Research Program and funded by the Russian Academic Excellence Project ‘5-100’. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1. Polania R, Nitsche MA, Ruff CC. Studying and modifying brain function with non-invasive brain stimulation. Nature neuroscience. 2018;21(2):174–187. 10.1038/s41593-017-0054-4 [DOI] [PubMed] [Google Scholar]
2. Galli G, Vadillo MA, Sirota M, Feurra M, Medvedeva A. A systematic review and meta-analysis of the effects of transcranial direct current stimulation (tDCS) on episodic memory. Brain stimulation. 2019;12(2):231–241. 10.1016/j.brs.2018.11.008 [DOI] [PubMed] [Google Scholar]
3. Teo F, Hoy KE, Daskalakis ZJ, Fitzgerald PB. Investigating the role of current strength in tDCS modulation of working memory performance in healthy controls. Frontiers in psychiatry. 2011;2:45 10.3389/fpsyt.2011.00045 [DOI] [PMC free article] [PubMed] [Google Scholar]
4. Zwissler B, Sperber C, Aigeldinger S, Schindler S, Kissler J, Plewnia C. Shaping memory accuracy by left prefrontal transcranial direct current stimulation. Journal of Neuroscience. 2014;34(11):4022–4026. 10.1523/JNEUROSCI.5407-13.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
5. Giordano J, Bikson M, Kappenman ES, Clark VP, Coslett HB, Hamblin MR, et al. Mechanisms and effects of transcranial direct current stimulation. Dose-Response. 2017;15(1):1559325816685467 10.1177/1559325816685467 [DOI] [PMC free article] [PubMed] [Google Scholar]
6. Heroux ME, Loo CK, Taylor JL, Gandevia SC. Questionable science and reproducibility in electrical brain stimulation research. PloS one. 2017;12(4). 10.1371/journal.pone.0175635 [DOI] [PMC free article] [PubMed] [Google Scholar]
7. Manenti R, Cotelli M, Robertson IH, Miniussi C. Transcranial brain stimulation studies of episodic memory in young adults, elderly adults and individuals with memory dysfunction: a review. Brain Stimulation. 2012;5(2):103–109. 10.1016/j.brs.2012.03.004 [DOI] [PubMed] [Google Scholar]
8. Szczepanski SM, Knight RT. Insights into human behavior from lesions to the prefrontal cortex. Neuron. 2014;83(5):1002–1018. 10.1016/j.neuron.2014.08.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Fuster JM. Executive frontal functions. Experimental brain research. 2000;133(1):66–70. 10.1007/s002210000401 [DOI] [PubMed] [Google Scholar]
10. Habich A, Klöppel S, Abdulkadir A, Scheller E, Nissen C, Peter J. Anodal tDCS enhances verbal episodic memory in initially low performers. Frontiers in human neuroscience. 2017;11:542 10.3389/fnhum.2017.00542 [DOI] [PMC free article] [PubMed] [Google Scholar]
11. Nyberg L, Cabeza R, Tulving E. PET studies of encoding and retrieval: The HERA model. Psychonomic Bulletin & Review. 1996;3(2):135–148. 10.3758/BF03212412 [DOI] [PubMed] [Google Scholar]
12. Javadi AH, Cheng P. Transcranial direct current stimulation (tDCS) enhances reconsolidation of long-term memory. Brain stimulation. 2013;6(4):668–674. 10.1016/j.brs.2012.10.007 [DOI] [PubMed] [Google Scholar]
13. Javadi AH, Walsh V. Transcranial direct current stimulation (tDCS) of the left dorsolateral prefrontal cortex modulates declarative memory. Brain stimulation. 2012;5(3):231–241. 10.1016/j.brs.2011.06.007 [DOI] [PubMed] [Google Scholar]
14. Manenti R, Brambilla M, Petesi M, Ferrari C, Cotelli M. Enhancing verbal episodic memory in older and young subjects after non-invasive brain stimulation. Frontiers in aging neuroscience. 2013;5:49 10.3389/fnagi.2013.00049 [DOI] [PMC free article] [PubMed] [Google Scholar]
15. Lara GAd, Knechtges PN, Paulus W, Antal A. Anodal tDCS over the left DLPFC did not affect the encoding and retrieval of verbal declarative information. Frontiers in neuroscience. 2017;11:452 10.3389/fnins.2017.00452 [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Brunoni AR, Vanderhasselt MA. Working memory improvement with non-invasive brain stimulation of the dorsolateral prefrontal cortex: a systematic review and meta-analysis. Brain and cognition. 2014;86:1–9. 10.1016/j.bandc.2014.01.008 [DOI] [PubMed] [Google Scholar]
17. Dedoncker J, Brunoni AR, Baeken C, Vanderhasselt MA. A systematic review and meta-analysis of the effects of transcranial direct current stimulation (tDCS) over the dorsolateral prefrontal cortex in healthy and neuropsychiatric samples: influence of stimulation parameters. Brain stimulation. 2016;9(4):501–517. 10.1016/j.brs.2016.04.006 [DOI] [PubMed] [Google Scholar]
18. Dedoncker J, Brunoni AR, Baeken C, Vanderhasselt MA. The effect of the interval-between-sessions on prefrontal transcranial direct current stimulation (tDCS) on cognitive outcomes: a systematic review and meta-analysis. Journal of Neural Transmission. 2016;123(10):1159–1172. 10.1007/s00702-016-1558-x [DOI] [PubMed] [Google Scholar]
19. Lafontaine MP, Theoret H, Gosselin F, Lippe S. Transcranial direct current stimulation of the dorsolateral prefrontal cortex modulates repetition suppression to unfamiliar faces: an ERP study. PloS one. 2013;8(12). 10.1371/journal.pone.0081721 [DOI] [PMC free article] [PubMed] [Google Scholar]
20. Medvedeva A, Materassi M, Neacsu V, Beresford-Webb J, Hussin A, Khan N, et al. Effects of anodal transcranial direct current stimulation over the ventrolateral prefrontal cortex on episodic memory formation and retrieval. Cerebral Cortex. 2019;29(2):657–665. 10.1093/cercor/bhx347 [DOI] [PubMed] [Google Scholar]
21. Murray LJ, Ranganath C. The dorsolateral prefrontal cortex contributes to successful relational memory encoding. Journal of Neuroscience. 2007;27(20):5515–5522. 10.1523/JNEUROSCI.0406-07.2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
22. Blumenfeld RS, Ranganath C. Prefrontal cortex and long-term memory encoding: an integrative review of findings from neuropsychology and neuroimaging. The Neuroscientist. 2007;13(3):280–291. 10.1177/1073858407299290 [DOI] [PubMed] [Google Scholar]
23. Blumenfeld RS, Parks CM, Yonelinas AP, Ranganath C. Putting the pieces together: the role of dorsolateral prefrontal cortex in relational memory encoding. Journal of cognitive neuroscience. 2011;23(1):257–265. 10.1162/jocn.2010.21459 [DOI] [PMC free article] [PubMed] [Google Scholar]
24. Blumenfeld RS, Nomura EM, Gratton C, D’Esposito M. Lateral prefrontal cortex is organized into parallel dorsal and ventral streams along the rostro-caudal axis. Cerebral Cortex. 2013;23(10):2457–2466. 10.1093/cercor/bhs223 [DOI] [PMC free article] [PubMed] [Google Scholar]
25. Blumenfeld RS, Ranganath C. Dorsolateral prefrontal cortex promotes long-term memory formation through its role in working memory organization. Journal of Neuroscience. 2006;26(3):916–925. 10.1523/JNEUROSCI.2353-05.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
26. Hawco C, Berlim MT, Lepage M. The dorsolateral prefrontal cortex plays a role in self-initiated elaborative cognitive processing during episodic memory encoding: rTMS evidence. PLoS One. 2013;8(9). 10.1371/journal.pone.0073789 [DOI] [PMC free article] [PubMed] [Google Scholar]
27. Hammer A, Mohammadi B, Schmicker M, Saliger S, Münte TF. Errorless and errorful learning modulated by transcranial direct current stimulation. BMC neuroscience. 2011;12(1):72 10.1186/1471-2202-12-72 [DOI] [PMC free article] [PubMed] [Google Scholar]
28. Takahashi E, Ohki K, Kim DS. Diffusion tensor studies dissociated two fronto-temporal pathways in the human memory system. Neuroimage. 2007;34(2):827–838. 10.1016/j.neuroimage.2006.10.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
29. Raposo A, Han S, Dobbins IG. Ventrolateral prefrontal cortex and self-initiated semantic elaboration during memory retrieval. Neuropsychologia. 2009;47(11):2261–2271. 10.1016/j.neuropsychologia.2008.10.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Sandrini M, Brambilla M, Manenti R, Rosini S, Cohen LG, Cotelli M. Noninvasive stimulation of prefrontal cortex strengthens existing episodic memories and reduces forgetting in the elderly. Frontiers in aging neuroscience. 2014;6:289 10.3389/fnagi.2014.00289 [DOI] [PMC free article] [PubMed] [Google Scholar]
31. Lu Y, Wang C, Chen C, Xue G. Spatiotemporal neural pattern similarity supports episodic memory. Current Biology. 2015;25(6):780–785. 10.1016/j.cub.2015.01.055 [DOI] [PubMed] [Google Scholar]
32. Pisoni A, Vernice M, Iasevoli L, Cattaneo Z, Papagno C. Guess who? Investigating the proper name processing network by means of tDCS. Neuropsychologia. 2015;66:267–278. 10.1016/j.neuropsychologia.2014.11.025 [DOI] [PubMed] [Google Scholar]
33. Balzarotti S, Colombo B. Effects of unilateral transcranial direct current stimulation of left prefrontal cortex on processing and memory of emotional visual stimuli. PloS one. 2016;11(7). 10.1371/journal.pone.0159555 [DOI] [PMC free article] [PubMed] [Google Scholar]
34. Pergolizzi D, Chua EF. Transcranial direct current stimulation over the parietal cortex alters bias in item and source memory tasks. Brain and cognition. 2016;108:56–65. 10.1016/j.bandc.2016.06.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
35. Blumenfeld RS, Lee TG, D’Esposito M. The effects of lateral prefrontal transcranial magnetic stimulation on item memory encoding. Neuropsychologia. 2014;53:197–202. 10.1016/j.neuropsychologia.2013.11.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Lyashevskaya ON, Sharov SA. Frequency dictionary of the modern Russian language (the Russian National Corpus). M: Azbukovnik (in Russian). 2009.
37. Snodgrass JG, Corwin J. Pragmatics of measuring recognition memory: applications to dementia and amnesia. Journal of experimental psychology: General. 1988;117(1):34 10.1037/0096-3445.117.1.34 [DOI] [PubMed] [Google Scholar]
38. Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition. 1997;30(7):1145–1159. 10.1016/S0031-3203(96)00142-2 [DOI] [Google Scholar]
39. Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics. 2017;5:135–146. 10.1162/tacl_a_00051 [DOI] [Google Scholar]
40.Joulin A, Bojanowski P, Mikolov T, Jégou H, Grave E. Loss in translation: Learning bilingual word mapping with a retrieval criterion. arXiv preprint arXiv:180407745. 2018.
41.Zhang D, Yang Z. Word embedding perturbation for sentence classification. arXiv preprint arXiv:180408166. 2018.
42. Olson RS, Moore JH. TPOT: A tree-based pipeline optimization tool for automating machine learning In: Automated Machine Learning. Springer; 2019. p. 151–160. [Google Scholar]
43. Mendoza H, Klein A, Feurer M, Springenberg JT, Urban M, Burkart M, et al. Towards automatically-tuned deep neural networks In: Automated Machine Learning. Springer; 2019. p. 135–149. [Google Scholar]
44.Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, et al. Automatic differentiation in pytorch. In: Proceedings of Neural Information Processing Systems; 2017.
45. Einstein GO, Hunt RR. Levels of processing and organization: Additive effects of individual-item and relational processing. Journal of experimental Psychology: Human learning and Memory. 1980;6(5):588. [Google Scholar]
46. Staresina BP, Gray JC, Davachi L. Event congruency enhances episodic memory encoding through semantic elaboration and relational binding. Cerebral Cortex. 2009;19(5):1198–1207. 10.1093/cercor/bhn165 [DOI] [PMC free article] [PubMed] [Google Scholar]
47. Horvath JC, Forte JD, Carter O. Quantitative review finds no evidence of cognitive effects in healthy populations from single-session transcranial direct current stimulation (tDCS). Brain stimulation. 2015;8(3):535–550. 10.1016/j.brs.2015.01.400 [DOI] [PubMed] [Google Scholar]
48. Vannorsdall TD, Van Steenburgh JJ, Schretlen DJ, Jayatillake R, Skolasky RL, Gordon B. Reproducibility of tDCS results in a randomized trial: failure to replicate findings of tDCS-induced enhancement of verbal fluency. Cognitive and Behavioral Neurology. 2016;29(1):11–17. 10.1097/WNN.0000000000000086 [DOI] [PubMed] [Google Scholar]
49. Peterchev AV. Neuromodulation: Transcranial electric stimulation seen from within the brain. Elife. 2017;6:e25812 10.7554/eLife.25812 [DOI] [PMC free article] [PubMed] [Google Scholar]
50. Stagg CJ, Bachtiar V, Amadi U, Gudberg CA, Ilie AS, Sampaio-Baptista C, et al. Local GABA concentration is related to network-level resting functional connectivity. Elife. 2014;3:e01465 10.7554/eLife.01465 [DOI] [PMC free article] [PubMed] [Google Scholar]
51. Bachtiar V, Near J, Johansen-Berg H, Stagg CJ. Modulation of GABA and resting state functional connectivity by transcranial direct current stimulation. Elife. 2015;4:e08789 10.7554/eLife.08789 [DOI] [PMC free article] [PubMed] [Google Scholar]
52. Paquette C, Sidel M, Radinska BA, Soucy JP, Thiel A. Bilateral transcranial direct current stimulation modulates activation-induced regional blood flow changes during voluntary movement. Journal of Cerebral Blood Flow & Metabolism. 2011;31(10):2086–2095. 10.1038/jcbfm.2011.72 [DOI] [PMC free article] [PubMed] [Google Scholar]
53. Meinzer M, Jähnigen S, Copland DA, Darkow R, Grittner U, Avirame K, et al. Transcranial direct current stimulation over multiple days improves learning and maintenance of a novel vocabulary. Cortex. 2014;50:137–147. 10.1016/j.cortex.2013.07.013 [DOI] [PubMed] [Google Scholar]
54. Fertonani A, Rosini S, Cotelli M, Rossini PM, Miniussi C. Naming facilitation induced by transcranial direct current stimulation. Behavioural brain research. 2010;208(2):311–318. 10.1016/j.bbr.2009.10.030 [DOI] [PubMed] [Google Scholar]
55. Fregni F, Boggio PS, Nitsche M, Bermpohl F, Antal A, Feredoes E, et al. Anodal transcranial direct current stimulation of prefrontal cortex enhances working memory. Experimental brain research. 2005;166(1):23–30. 10.1007/s00221-005-2334-6 [DOI] [PubMed] [Google Scholar]
56. Olsen RK, Moses SN, Riggs L, Ryan JD. The hippocampus supports multiple cognitive processes through relational binding and comparison. Frontiers in human neuroscience. 2012;6:146 10.3389/fnhum.2012.00146 [DOI] [PMC free article] [PubMed] [Google Scholar]
57. Staresina BP, Davachi L. Mind the gap: binding experiences across space and time in the human hippocampus. Neuron. 2009;63(2):267–276. 10.1016/j.neuron.2009.06.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
58. Gray SJ, Brookshire G, Casasanto D, Gallo DA. Electrically stimulating prefrontal cortex at retrieval improves recollection accuracy. Cortex. 2015;73:188–194. 10.1016/j.cortex.2015.09.003 [DOI] [PubMed] [Google Scholar]
59. Matzen LE, Trumbo MC, Leach RC, Leshikar ED. Effects of non-invasive brain stimulation on associative memory. Brain research. 2015;1624:286–296. 10.1016/j.brainres.2015.07.036 [DOI] [PubMed] [Google Scholar]
60. Smirni D, Turriziani P, Mangano GR, Cipolotti L, Oliveri M. Modulating memory performance in healthy subjects with transcranial direct current stimulation over the right dorsolateral prefrontal cortex. PloS one. 2015;10(12). 10.1371/journal.pone.0144838 [DOI] [PMC free article] [PubMed] [Google Scholar]
61. Cesario J. Priming, replication, and the hardest science. Perspectives on psychological science. 2014;9(1):40–48. 10.1177/1745691613513470 [DOI] [PubMed] [Google Scholar]
62. Jones KT, Gözenman F, Berryhill ME. Enhanced long-term memory encoding after parietal neurostimulation. Experimental brain research. 2014;232(12):4043–4054. 10.1007/s00221-014-4090-y [DOI] [PMC free article] [PubMed] [Google Scholar]
63. Minarik T, Berger B, Althaus L, Bader V, Biebl B, Brotzeller F, et al. The importance of sample size for reproducibility of tDCS effects. Frontiers in Human Neuroscience. 2016;10:453 10.3389/fnhum.2016.00453 [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0235179.r001

Decision Letter 0

Stephen D Ginsberg

20 Jul 2020

PONE-D-20-16976

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

PLOS ONE

Dear Dr. Kirillov,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration by 2 Reviewers and an Academic Editor, all of the critiques of both Reviewers must be addressed in detail in a revision to determine publication status. If you are prepared to undertake the work required, I would be pleased to reconsider my decision, but revision of the original submission without directly addressing the critiques of the two Reviewers does not guarantee acceptance for publication in PLOS ONE. If the authors do not feel that the queries can be addressed, please consider submitting to another publication medium. A revised submission will be sent out for re-review. The authors are urged to have the manuscript given a hard copyedit for syntax and grammar.

==============================

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The aim of the this study conducted by Petrovskaya et al. was to use tDCS to replicate and extend the findings of Medvedeva et al. (2019)](Experiment 1) with the anode over VLPFC and DLPFC. However, the effect of tDCS with the anode over the left VLPFC (decrease in false recognition) was not replicated, and there were weak, negative effects with the anode over the DLPFC in the Offline group. In fact, there was a trend toward increased false 309 alarms after online active tDCS over the VLPFC and offline active tDCS over the DLPFC compared to Sham. The trend suggests that an increase in false alarms obscured group differences, and the increase in false alarms could be driven by an increase in semantic elaboration that leads to better memory for features common to multiple items. While the results do not support effects of tDCS with the anode over the DLPFC and VLPFC, this speaks to the lack of tDCS effect rather than the lack of involvement of these regions in episodic memory.

The manuscript is well written and I like the approach to investigate cognitive effects with machine learning, but there are some issues that need to be addressed.

The authors stated that tDCS was applied over VLPFC and DLPFC but only the anode was placed over these brain areas. I do not see any evidence that this electrode montage targeted the VLPFC and DLPFC. The current density maxima seems to be between the electrodes. Please provide evidence that the montage targets VLPFC and DLPFC. For instance, HD-target software (Soterix medical) use to determine the optimal electrode configuration for your chosen brain target shows that in order to target the Left DLPFC the anode should be over C3 and the cathode over AF3 according to 10-20 EEG electrode positioning system. If you do not have evidence you should mention this issue as a limitation.

Please state that the anode was applied over VLPFC and DLPFC instead of anodal tDCS.

Reviewer #2: This is a well-written replication study of the effect of tDCS on memory encoding. Such studies are important to verify the claims in the brain stimulation literature, particularly as there have been some criticisms of the field in recent years (e.g. Horvath et al. meta-analysis cited by the authors of the present study). My only major concern is regarding the interpretation of results:

Page 10, Line 330 – “Nonetheless, the results of the current study support the conclusion that tDCS does not modulate memory in the VLPFC and DLPFC directly but rather a subprocess leading to successful memory formation and retrieval.”

It's not clear that this can be claimed on the basis of the present findings. There was no direct examination of subprocesses leading to successful memory formation and retrieval. Rather, results appear to largely conform with the null hypothesis. Indeed, the significant ANOVA findings for false alarms could be somewhat questioned, given multiple ANOVAs were conducted (7 in total), which can inflate the false positive rate in much the same way that conducting multiple pairwise comparisons can. Correcting for multiple tests, by using a MANOVA for example, would likely leave the false alarm results non-significant. If this is indeed the case, one would have to conclude the the current study failed to replicate the previous findings, that the stimulation conditions did not significantly differ on any of the outcome variables, and that tDCS to the DLPFC or VLPFC was not observed to improve memory encoding.

Page 11, Line 400 – “We conclude that tDCS may exert a subtle modulation that also interacts strongly with individual differences, particularly in baseline activation and neuroanatomy.”

Similar to previous point, it’s not clear what empirical data obtained from the current study leads to the conclusion that individual differences, including baseline activity and neuroanatomy, may alter findings for tDCS and memory encoding. While this can be presented speculatively as an important variable/s to consider and investigate in future research, it’s not clear that these factors are at all relevant to the current findings. It may well be possible, at least theoretically, that they are irrelevant and tDCS exerts little to no reliable effect on memory encoding in healthy individuals.

Minor issues

Abstract – sample size would be useful information to include in the Abstract.

Abstract – Acronyms for VLPFC and DLPFC not explicitly defined in the abstract.

Page 1, Line 6 – a citation is needed to support the claim that tDCS can decrease reaction time or increase memory accuracy.

Page 2, Line 35 – “…and the left hemisphere in the VLPFC appears selectively engaged…” Somewhat oddly worded, could be rephrased to the left VLPFC

Page 3, Line 42 – “In one tDCS study, [27] found that cathodal stimulation over the left DLPFC (F3) in a cued-word-completion learning task where errors were evoked through guessing the wrong completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect).”

This sentence is difficult to parse, and may need to be presented in two sentences to better convey the meaning.

Page 3, Line 51 – “The current study involved administering atDCS over the left DLPFC offline or online encoding”. I believe there is a missing word, ‘during’, between DLPFC and offline.

Methods – A few additional details should be provided in the Methods. What was the specific timing of the stimulation relative to the task in the offline condition (e.g. did one immediately follow the other)? What was the duration of stimulation for active tDCS? This is described in citation [1], but could be repeated in the present manuscript.

Page 4, Line 113 – Minor typo of ‘ae’ instead of ‘age in “Bonferroni-corrected pairwise comparisons of ae…”

Considering the significant difference in age between two of the groups (VLPFC online and DLPFC online), the authors could run one-way ANCOVAs, incorporating Age as a covariate, to ensure that this variable doesn’t confound the results.

There is some repetition in the Methods that could be minimised:

Page 5, Line 96 – “However, stimulation was delivered only on the first day and at different sites (DLPFC or VLPFC) and times (offline or online encoding) depending on the group.”

Page 4, Line 137 – “Stimulation was only delivered on the first day, but electrodes were placed on the head on the second day without stimulation (to replicate the method of [1] exactly).”

Page 4, Line 139 – “For all groups, stimulation was delivered on the first day only.”

Page 6, Line 218 – A brief description of what is meant by ‘letter-based one-hot encoding’ would be useful.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Stevan Nikolin

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

==============================

Please submit your revised manuscript by January, 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Stephen D. Ginsberg, Ph.D.

Section Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Your abstract cannot contain citations. Please only include citations in the body text of the manuscript, and ensure that they remain in ascending numerical order on first mention.

3. Please ensure that you refer to Figure 3 in your text as, if accepted, production will need this reference to link the reader to the figure.

4. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 in your text; if accepted, production will need this reference to link the reader to the Table.

PLoS One. 2020 Dec 9;15(12):e0235179. doi: 10.1371/journal.pone.0235179.r002

Author response to Decision Letter 0

18 Nov 2020

Responses to reviewer #1:

The manuscript is well written and I like the approach to investigate cognitive effects with machine learning, but there are some issues that need to be addressed.

Please state that the anode was applied over VLPFC and DLPFC instead of anodal tDCS.

Response: We are glad the reviewer finds the approach and manuscript suitable and thank the reviewer for pointing this out as an important consideration. Modelling approaches are valuable for confirming that the stimulation corresponds to the expected montage. We have stated a lack of evidence for our montage targeting the VLPFC and DLPFC as a limitation. We have also corrected the description of the montage to state “the anode was applied” rather than “anodal tDCS.”

Throughout the manuscript, atDCS was replaced with tDCS in reference to the current study.

Page 4 lines 95-97

Specifically, we applied the same tDCS settings (2 mA anodal active tDCS and 30 s Sham tDCS) over the DLPFC (all conditions: offline, sham, and online) in addition to the VLPFC (online encoding only).

Page 10 lines 342-348

In our case, a limitation was not estimating whether maximum current flow was underneath the anode electrode and at the region of interest (VLPFC or DLPFC). However, the aim of the current study was to clarify the consistency of the effect found in the original study using an identical method, including montage. Future studies could test a montage with greatest likelihood of current flow at the region of interest by modeling current density maxima.

Page 11, Line 400 – “We conclude that tDCS may exert a subtle modulation that also interacts strongly with individual differences, particularly in baseline activation and neuroanatomy.”

Response: We thank the reviewer for the thoughtful consideration of the manuscript and helpful suggestions for clarifying and resolving certain issues. We added a correction for multiple ANOVAs using a MANOVA and tested group differences in recognition accuracy and reaction time with age as a covariate. In line with these results that show no differences between groups, we clarify in the discussion that there may be no significant effects of tDCS on memory performance in the current study. We clarify that while individual differences may interact with tDCS to influence cognitive effects, further work is needed to explore this relationship.

Page 7 Lines 270-277

To enable comparison with the original paper, we report the results of multiple ANOVAs. However, to control for multiple tests and the difference in age between groups, we analyzed all dependent variables including AUROC with a MANOVA, Pillai’s trace (12, 294) = .166, p = .148, and recognition accuracy and average reaction time with age as a covariate, F(3, 98) = .097 and F(3, 98) = .557, respectively. There was no significant difference between groups with hits, false alarms, discrimination ability, and average reaction time as dependent variables.

Page 11

Lines 362-364

It remains unclear whether the significant tDCS modulation of long-term verbal encoding in the original study occurred through modulation of related processes such as lexical retrieval, working memory, or attention.

Lines 370-376

Individual differences in baseline memory performance, correlated cognitive functions, and neuroanatomy could obscure the effect of tDCS on measures of performance, especially if the effect is subtle. It is clear that effects of tDCS on physiology may not correspond with cognitive effects, and future studies can better explore this relationship with physiological as well as cognitive measures and by estimating the current density maxima.

Page 12

Lines 437-443

In the current study, tDCS did not lead to significant differences in memory performance or reaction time. While it seems unlikely that the significant effect in the original study was due to sample size alone, we conclude based on the lack of replication that the effect of tDCS on long-term verbal memory when the anode is placed over the VLPFC or DLPFC may be subtle. Moreover, the effect may be completely obscured during interactions with other factors that may be difficult to measure or quantify, such as the state of the network.

Minor issues

We present parts of the manuscript that address each comment.

Abstract – sample size would be useful information to include in the Abstract.

Abstract – Acronyms for VLPFC and DLPFC not explicitly defined in the abstract.

We aimed to replicate a published effect of transcranial direct-current stimulation (tDCS)-induced recognition enhancement over the human ventrolateral prefrontal cortex (VLPFC) [1] and analyse the data with machine learning. We investigated effects over an adjacent region, the dorsolateral prefrontal cortex (DLPFC). In total, we analyzed data from 97 participants after exclusions. We found weak or absent effects over the VLPFC and DLPFC. We conducted machine learning studies to examine the effects of semantic and phonetic features on memorization, which revealed no effect of VLPFC tDCS on the original dataset or the current data. The highest contributing factor to memory performance was individual differences in memory not explained by word features, tDCS group, or sample size, while semantic, phonetic, and orthographic word characteristics did not contribute significantly. To our knowledge, this is the first tDCS study to investigate cognitive effects with machine learning, and future studies may benefit from studying physiological as well as cognitive effects with data-driven approaches and computational models.

Page 2, Line 6 – a citation is needed to support the claim that tDCS can decrease reaction time or increase memory accuracy.

Transcranial direct-current stimulation (tDCS), a method of safely and non-invasively delivering a weak electric current through the cortex, has been gaining increasing attention [2] as a tool for studying and possibly enhancing episodic memory [3]. Anodal tDCS (atDCS) involves increasing cortical excitability in a target region such as the dorsolateral prefrontal cortex (DLPFC), which may in turn facilitate or enhance memory performance by decreasing reaction time and/or increasing memory accuracy. (3,4)

3. Investigating the Role of Current Strength in tDCS Modulation of Working Memory Performance in Healthy Controls

Florence Teo,1 Kate E. Hoy,1,* Zafiris J. Daskalakis,2 and Paul B. Fitzgerald1

Front Psychiatry. 2011; 2: 45. Published online 2011 Jul 18. doi: 10.3389/fpsyt.2011.00045

PMCID: PMC3141358

PMID: 21811474

4. Shaping Memory Accuracy by Left Prefrontal Transcranial Direct Current Stimulation

Bastian Zwissler,1 Christoph Sperber,1 Sina Aigeldinger,1 Sebastian Schindler,2 Johanna Kissler,2 and Christian Plewnia,3

J Neurosci. 2014 Mar 12; 34(11): 4022–4026.

doi: 10.1523/JNEUROSCI.5407-13.2014

Page 2, Line 35 – “…and the left hemisphere in the VLPFC appears selectively engaged…” Somewhat oddly worded, could be rephrased to the left VLPFC

The VLPFC and DLPFC are thought to play functionally distinct roles in long-term verbal memory, and the left VLPFC appears selectively engaged in verbal but not non-verbal material.

This sentence is difficult to parse, and may need to be presented in two sentences to better convey the meaning.

In a tDCS study conducted by [27], tDCS was applied over the left DLPFC in a cued-word-completion task to modulate memory performance. The results showed that cathodal stimulation inhibited memory accuracy after error learning in comparison with sham. The errors were evoked through guessing, or incorrect completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect).

Page 3, Line 51 – “The current study involved administering atDCS over the left DLPFC offline or online encoding”. I believe there is a missing word, ‘during’, between DLPFC and offline.

The current study involved administering atDCS over the left DLPFC during offline or online encoding

Page 5

Lines 149-154

Stimulation was applied for the entire duration (10 minutes) of the reading task for the DLPFC offline group and for the entire duration (10 minutes) of the study memory task that immediately followed for the VLPFC Online and DLPFC Online groups. In summary, the stimulation was delivered for approximately 10 minutes to cover the entire duration of the reading and memory tasks.

Page 4, Line 113 – Minor typo of ‘ae’ instead of ‘age in “Bonferroni-corrected pairwise comparisons of ae…”

Bonferroni-corrected pairwise comparisons of age

As cited in an earlier response, we conducted an ANCOVA in addition to a MANOVA:

Page 7 Lines 270-277

To enable comparison with the original paper, we report the results of multiple ANOVAs. However, to control for multiple tests and the difference in age between groups, we analyzed all dependent variables including AUROC with a MANOVA, Pillai’s trace (12, 294) = .166, p = .148, and recognition accuracy and average reaction time with age as a covariate (ANCOVA), F(3, 98) = .097 and F(3, 98) = .557, respectively. There was no significant difference between groups with hits, false alarms, discrimination ability, and average reaction time as dependent variables.

There is some repetition in the Methods that could be minimised:

Page 5, Line 96 – “However, stimulation was delivered only on the first day and at different sites (DLPFC or VLPFC) and times (offline or online encoding) depending on the group.”

Page 4, Line 137 – “Stimulation was only delivered on the first day, but electrodes were placed on the head on the second day without stimulation (to replicate the method of [1] exactly).”

Page 4, Line 139 – “For all groups, stimulation was delivered on the first day only.”

We minimized repetition and increased conciseness in the method:

Page 4 Lines 97-101

The electrodes were placed at both sessions (study and test 24 hours later), but stimulation was only delivered before or during the encoding task in the study phase. The anode was placed over the left VLPFC (F7) or the left DLPFC (F3), and the cathode was placed over the ipsilateral (left) shoulder. See section 2.2 below for the experimental conditions.

Page 6, Line 218 – A brief description of what is meant by ‘letter-based one-hot encoding’ would be useful.

Lines 227-229

In line with the trial-based and participant-based approaches, we conducted two kinds of ML studies: 1) Participant-independent (trial-based) analysis tests the success of a model that predicts the AUROC for each word using either word embeddings or letter-based one-hot encoding (each letter i is replaced by a vector of Nalphabet components with the i-th component being equal to 1 while every other component is zero) and 2) Participant-specific (participant-based) analysis tests the success of a model that predicts whether a participant was able to recall the word given either word embeddings or letter-based one-hot encoding for each word for individual participants.

Attachment

Submitted filename: Petrovskaya_et_al_rebuttal_17_11_20.pdf

Click here for additional data file.^{(78KB, pdf)}

PLoS One. doi: 10.1371/journal.pone.0235179.r003

Decision Letter 1

Stephen D Ginsberg

23 Nov 2020

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

PONE-D-20-16976R1

Dear Dr. Kirillov,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Stephen D. Ginsberg, Ph.D.

Section Editor

PLOS ONE

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

**********

6. Review Comments to the Author

Reviewer #1: The authors addressed all my concerns. Their response yo my questions was hood. I think the paper is ready for publication

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

PLoS One. doi: 10.1371/journal.pone.0235179.r004

Acceptance letter

Stephen D Ginsberg

26 Nov 2020

PONE-D-20-16976R1

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

Dear Dr. Kirillov:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Stephen D. Ginsberg

Section Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Appendix. Significantly different words.

Russian and English words that differ significantly in the number of hits, false alarms, AUROC and reaction time.

(PDF)

Click here for additional data file.^{(93.6KB, pdf)}

S2 Appendix. Data augmentations.

The procedures used to augment the training set according to the method proused in [41].

(PDF)

Click here for additional data file.^{(127.8KB, pdf)}

S3 Appendix. Participant-independent ML trials.

The performance of participant-independent TPOT and AutoPytorch models.

(PDF)

Click here for additional data file.^{(175.7KB, pdf)}

S4 Appendix. Participant-dependent ML trials.

The performance of participant-independent TPOT and AutoPytorch models.

(PDF)

Click here for additional data file.^{(74.7KB, pdf)}

Attachment

Submitted filename: Petrovskaya_et_al_rebuttal_17_11_20.pdf

Click here for additional data file.^{(78KB, pdf)}

Data Availability Statement

All related files are available from the Zenodo database (DOI: 10.5281/zenodo.3837111).

[pone.0235179.ref001] 1. Polania R, Nitsche MA, Ruff CC. Studying and modifying brain function with non-invasive brain stimulation. Nature neuroscience. 2018;21(2):174–187. 10.1038/s41593-017-0054-4 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref002] 2. Galli G, Vadillo MA, Sirota M, Feurra M, Medvedeva A. A systematic review and meta-analysis of the effects of transcranial direct current stimulation (tDCS) on episodic memory. Brain stimulation. 2019;12(2):231–241. 10.1016/j.brs.2018.11.008 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref003] 3. Teo F, Hoy KE, Daskalakis ZJ, Fitzgerald PB. Investigating the role of current strength in tDCS modulation of working memory performance in healthy controls. Frontiers in psychiatry. 2011;2:45 10.3389/fpsyt.2011.00045 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref004] 4. Zwissler B, Sperber C, Aigeldinger S, Schindler S, Kissler J, Plewnia C. Shaping memory accuracy by left prefrontal transcranial direct current stimulation. Journal of Neuroscience. 2014;34(11):4022–4026. 10.1523/JNEUROSCI.5407-13.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref005] 5. Giordano J, Bikson M, Kappenman ES, Clark VP, Coslett HB, Hamblin MR, et al. Mechanisms and effects of transcranial direct current stimulation. Dose-Response. 2017;15(1):1559325816685467 10.1177/1559325816685467 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref006] 6. Heroux ME, Loo CK, Taylor JL, Gandevia SC. Questionable science and reproducibility in electrical brain stimulation research. PloS one. 2017;12(4). 10.1371/journal.pone.0175635 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref007] 7. Manenti R, Cotelli M, Robertson IH, Miniussi C. Transcranial brain stimulation studies of episodic memory in young adults, elderly adults and individuals with memory dysfunction: a review. Brain Stimulation. 2012;5(2):103–109. 10.1016/j.brs.2012.03.004 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref008] 8. Szczepanski SM, Knight RT. Insights into human behavior from lesions to the prefrontal cortex. Neuron. 2014;83(5):1002–1018. 10.1016/j.neuron.2014.08.011 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref009] 9. Fuster JM. Executive frontal functions. Experimental brain research. 2000;133(1):66–70. 10.1007/s002210000401 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref010] 10. Habich A, Klöppel S, Abdulkadir A, Scheller E, Nissen C, Peter J. Anodal tDCS enhances verbal episodic memory in initially low performers. Frontiers in human neuroscience. 2017;11:542 10.3389/fnhum.2017.00542 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref011] 11. Nyberg L, Cabeza R, Tulving E. PET studies of encoding and retrieval: The HERA model. Psychonomic Bulletin & Review. 1996;3(2):135–148. 10.3758/BF03212412 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref012] 12. Javadi AH, Cheng P. Transcranial direct current stimulation (tDCS) enhances reconsolidation of long-term memory. Brain stimulation. 2013;6(4):668–674. 10.1016/j.brs.2012.10.007 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref013] 13. Javadi AH, Walsh V. Transcranial direct current stimulation (tDCS) of the left dorsolateral prefrontal cortex modulates declarative memory. Brain stimulation. 2012;5(3):231–241. 10.1016/j.brs.2011.06.007 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref014] 14. Manenti R, Brambilla M, Petesi M, Ferrari C, Cotelli M. Enhancing verbal episodic memory in older and young subjects after non-invasive brain stimulation. Frontiers in aging neuroscience. 2013;5:49 10.3389/fnagi.2013.00049 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref015] 15. Lara GAd, Knechtges PN, Paulus W, Antal A. Anodal tDCS over the left DLPFC did not affect the encoding and retrieval of verbal declarative information. Frontiers in neuroscience. 2017;11:452 10.3389/fnins.2017.00452 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref016] 16. Brunoni AR, Vanderhasselt MA. Working memory improvement with non-invasive brain stimulation of the dorsolateral prefrontal cortex: a systematic review and meta-analysis. Brain and cognition. 2014;86:1–9. 10.1016/j.bandc.2014.01.008 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref017] 17. Dedoncker J, Brunoni AR, Baeken C, Vanderhasselt MA. A systematic review and meta-analysis of the effects of transcranial direct current stimulation (tDCS) over the dorsolateral prefrontal cortex in healthy and neuropsychiatric samples: influence of stimulation parameters. Brain stimulation. 2016;9(4):501–517. 10.1016/j.brs.2016.04.006 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref018] 18. Dedoncker J, Brunoni AR, Baeken C, Vanderhasselt MA. The effect of the interval-between-sessions on prefrontal transcranial direct current stimulation (tDCS) on cognitive outcomes: a systematic review and meta-analysis. Journal of Neural Transmission. 2016;123(10):1159–1172. 10.1007/s00702-016-1558-x [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref019] 19. Lafontaine MP, Theoret H, Gosselin F, Lippe S. Transcranial direct current stimulation of the dorsolateral prefrontal cortex modulates repetition suppression to unfamiliar faces: an ERP study. PloS one. 2013;8(12). 10.1371/journal.pone.0081721 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref020] 20. Medvedeva A, Materassi M, Neacsu V, Beresford-Webb J, Hussin A, Khan N, et al. Effects of anodal transcranial direct current stimulation over the ventrolateral prefrontal cortex on episodic memory formation and retrieval. Cerebral Cortex. 2019;29(2):657–665. 10.1093/cercor/bhx347 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref021] 21. Murray LJ, Ranganath C. The dorsolateral prefrontal cortex contributes to successful relational memory encoding. Journal of Neuroscience. 2007;27(20):5515–5522. 10.1523/JNEUROSCI.0406-07.2007 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref022] 22. Blumenfeld RS, Ranganath C. Prefrontal cortex and long-term memory encoding: an integrative review of findings from neuropsychology and neuroimaging. The Neuroscientist. 2007;13(3):280–291. 10.1177/1073858407299290 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref023] 23. Blumenfeld RS, Parks CM, Yonelinas AP, Ranganath C. Putting the pieces together: the role of dorsolateral prefrontal cortex in relational memory encoding. Journal of cognitive neuroscience. 2011;23(1):257–265. 10.1162/jocn.2010.21459 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref024] 24. Blumenfeld RS, Nomura EM, Gratton C, D’Esposito M. Lateral prefrontal cortex is organized into parallel dorsal and ventral streams along the rostro-caudal axis. Cerebral Cortex. 2013;23(10):2457–2466. 10.1093/cercor/bhs223 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref025] 25. Blumenfeld RS, Ranganath C. Dorsolateral prefrontal cortex promotes long-term memory formation through its role in working memory organization. Journal of Neuroscience. 2006;26(3):916–925. 10.1523/JNEUROSCI.2353-05.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref026] 26. Hawco C, Berlim MT, Lepage M. The dorsolateral prefrontal cortex plays a role in self-initiated elaborative cognitive processing during episodic memory encoding: rTMS evidence. PLoS One. 2013;8(9). 10.1371/journal.pone.0073789 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref027] 27. Hammer A, Mohammadi B, Schmicker M, Saliger S, Münte TF. Errorless and errorful learning modulated by transcranial direct current stimulation. BMC neuroscience. 2011;12(1):72 10.1186/1471-2202-12-72 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref028] 28. Takahashi E, Ohki K, Kim DS. Diffusion tensor studies dissociated two fronto-temporal pathways in the human memory system. Neuroimage. 2007;34(2):827–838. 10.1016/j.neuroimage.2006.10.009 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref029] 29. Raposo A, Han S, Dobbins IG. Ventrolateral prefrontal cortex and self-initiated semantic elaboration during memory retrieval. Neuropsychologia. 2009;47(11):2261–2271. 10.1016/j.neuropsychologia.2008.10.024 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref030] 30. Sandrini M, Brambilla M, Manenti R, Rosini S, Cohen LG, Cotelli M. Noninvasive stimulation of prefrontal cortex strengthens existing episodic memories and reduces forgetting in the elderly. Frontiers in aging neuroscience. 2014;6:289 10.3389/fnagi.2014.00289 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref031] 31. Lu Y, Wang C, Chen C, Xue G. Spatiotemporal neural pattern similarity supports episodic memory. Current Biology. 2015;25(6):780–785. 10.1016/j.cub.2015.01.055 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref032] 32. Pisoni A, Vernice M, Iasevoli L, Cattaneo Z, Papagno C. Guess who? Investigating the proper name processing network by means of tDCS. Neuropsychologia. 2015;66:267–278. 10.1016/j.neuropsychologia.2014.11.025 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref033] 33. Balzarotti S, Colombo B. Effects of unilateral transcranial direct current stimulation of left prefrontal cortex on processing and memory of emotional visual stimuli. PloS one. 2016;11(7). 10.1371/journal.pone.0159555 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref034] 34. Pergolizzi D, Chua EF. Transcranial direct current stimulation over the parietal cortex alters bias in item and source memory tasks. Brain and cognition. 2016;108:56–65. 10.1016/j.bandc.2016.06.009 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref035] 35. Blumenfeld RS, Lee TG, D’Esposito M. The effects of lateral prefrontal transcranial magnetic stimulation on item memory encoding. Neuropsychologia. 2014;53:197–202. 10.1016/j.neuropsychologia.2013.11.021 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref036] 36.Lyashevskaya ON, Sharov SA. Frequency dictionary of the modern Russian language (the Russian National Corpus). M: Azbukovnik (in Russian). 2009.

[pone.0235179.ref037] 37. Snodgrass JG, Corwin J. Pragmatics of measuring recognition memory: applications to dementia and amnesia. Journal of experimental psychology: General. 1988;117(1):34 10.1037/0096-3445.117.1.34 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref038] 38. Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition. 1997;30(7):1145–1159. 10.1016/S0031-3203(96)00142-2 [DOI] [Google Scholar]

[pone.0235179.ref039] 39. Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics. 2017;5:135–146. 10.1162/tacl_a_00051 [DOI] [Google Scholar]

[pone.0235179.ref040] 40.Joulin A, Bojanowski P, Mikolov T, Jégou H, Grave E. Loss in translation: Learning bilingual word mapping with a retrieval criterion. arXiv preprint arXiv:180407745. 2018.

[pone.0235179.ref041] 41.Zhang D, Yang Z. Word embedding perturbation for sentence classification. arXiv preprint arXiv:180408166. 2018.

[pone.0235179.ref042] 42. Olson RS, Moore JH. TPOT: A tree-based pipeline optimization tool for automating machine learning In: Automated Machine Learning. Springer; 2019. p. 151–160. [Google Scholar]

[pone.0235179.ref043] 43. Mendoza H, Klein A, Feurer M, Springenberg JT, Urban M, Burkart M, et al. Towards automatically-tuned deep neural networks In: Automated Machine Learning. Springer; 2019. p. 135–149. [Google Scholar]

[pone.0235179.ref044] 44.Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, et al. Automatic differentiation in pytorch. In: Proceedings of Neural Information Processing Systems; 2017.

[pone.0235179.ref045] 45. Einstein GO, Hunt RR. Levels of processing and organization: Additive effects of individual-item and relational processing. Journal of experimental Psychology: Human learning and Memory. 1980;6(5):588. [Google Scholar]

[pone.0235179.ref046] 46. Staresina BP, Gray JC, Davachi L. Event congruency enhances episodic memory encoding through semantic elaboration and relational binding. Cerebral Cortex. 2009;19(5):1198–1207. 10.1093/cercor/bhn165 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref047] 47. Horvath JC, Forte JD, Carter O. Quantitative review finds no evidence of cognitive effects in healthy populations from single-session transcranial direct current stimulation (tDCS). Brain stimulation. 2015;8(3):535–550. 10.1016/j.brs.2015.01.400 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref048] 48. Vannorsdall TD, Van Steenburgh JJ, Schretlen DJ, Jayatillake R, Skolasky RL, Gordon B. Reproducibility of tDCS results in a randomized trial: failure to replicate findings of tDCS-induced enhancement of verbal fluency. Cognitive and Behavioral Neurology. 2016;29(1):11–17. 10.1097/WNN.0000000000000086 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref049] 49. Peterchev AV. Neuromodulation: Transcranial electric stimulation seen from within the brain. Elife. 2017;6:e25812 10.7554/eLife.25812 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref050] 50. Stagg CJ, Bachtiar V, Amadi U, Gudberg CA, Ilie AS, Sampaio-Baptista C, et al. Local GABA concentration is related to network-level resting functional connectivity. Elife. 2014;3:e01465 10.7554/eLife.01465 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref051] 51. Bachtiar V, Near J, Johansen-Berg H, Stagg CJ. Modulation of GABA and resting state functional connectivity by transcranial direct current stimulation. Elife. 2015;4:e08789 10.7554/eLife.08789 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref052] 52. Paquette C, Sidel M, Radinska BA, Soucy JP, Thiel A. Bilateral transcranial direct current stimulation modulates activation-induced regional blood flow changes during voluntary movement. Journal of Cerebral Blood Flow & Metabolism. 2011;31(10):2086–2095. 10.1038/jcbfm.2011.72 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref053] 53. Meinzer M, Jähnigen S, Copland DA, Darkow R, Grittner U, Avirame K, et al. Transcranial direct current stimulation over multiple days improves learning and maintenance of a novel vocabulary. Cortex. 2014;50:137–147. 10.1016/j.cortex.2013.07.013 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref054] 54. Fertonani A, Rosini S, Cotelli M, Rossini PM, Miniussi C. Naming facilitation induced by transcranial direct current stimulation. Behavioural brain research. 2010;208(2):311–318. 10.1016/j.bbr.2009.10.030 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref055] 55. Fregni F, Boggio PS, Nitsche M, Bermpohl F, Antal A, Feredoes E, et al. Anodal transcranial direct current stimulation of prefrontal cortex enhances working memory. Experimental brain research. 2005;166(1):23–30. 10.1007/s00221-005-2334-6 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref056] 56. Olsen RK, Moses SN, Riggs L, Ryan JD. The hippocampus supports multiple cognitive processes through relational binding and comparison. Frontiers in human neuroscience. 2012;6:146 10.3389/fnhum.2012.00146 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref057] 57. Staresina BP, Davachi L. Mind the gap: binding experiences across space and time in the human hippocampus. Neuron. 2009;63(2):267–276. 10.1016/j.neuron.2009.06.024 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref058] 58. Gray SJ, Brookshire G, Casasanto D, Gallo DA. Electrically stimulating prefrontal cortex at retrieval improves recollection accuracy. Cortex. 2015;73:188–194. 10.1016/j.cortex.2015.09.003 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref059] 59. Matzen LE, Trumbo MC, Leach RC, Leshikar ED. Effects of non-invasive brain stimulation on associative memory. Brain research. 2015;1624:286–296. 10.1016/j.brainres.2015.07.036 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref060] 60. Smirni D, Turriziani P, Mangano GR, Cipolotti L, Oliveri M. Modulating memory performance in healthy subjects with transcranial direct current stimulation over the right dorsolateral prefrontal cortex. PloS one. 2015;10(12). 10.1371/journal.pone.0144838 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref061] 61. Cesario J. Priming, replication, and the hardest science. Perspectives on psychological science. 2014;9(1):40–48. 10.1177/1745691613513470 [DOI] [PubMed] [Google Scholar]

[pone.0235179.ref062] 62. Jones KT, Gözenman F, Berryhill ME. Enhanced long-term memory encoding after parietal neurostimulation. Experimental brain research. 2014;232(12):4043–4054. 10.1007/s00221-014-4090-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0235179.ref063] 63. Minarik T, Berger B, Althaus L, Bader V, Biebl B, Brotzeller F, et al. The importance of sample size for reproducibility of tDCS effects. Frontiers in Human Neuroscience. 2016;10:453 10.3389/fnhum.2016.00453 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

Aleksandra Petrovskaya

Bogdan Kirillov

Anastasiya Asmolova

Giulia Galli

Matteo Feurra

Angela Medvedeva

Roles

Abstract

1 Introduction

2 Methods and materials

2.1 Participants

2.2 Procedure and experimental design

2.3 Data analysis

2.4 Area under the receiver operating characteristic curve as a measure of predictive performance for episodic memory changes

2.5 Extraction of semantic, phonological and orthographic determinants of episodic memory performance

3 Results

3.1 ANOVA

Table 1. Means and standard deviations for memory accuracy across groups.

Table 2. Means and standard deviations for reaction time accuracy across groups.

3.2 AUROC analysis and ML models

Fig 1. Violin plots of AUC distributions for Russian vs English participants in each group.

Fig 2. Violin plots showing AUROC distributions for Russian vs English words in each group.

Fig 3. Reaction time distribution for English and Russian words.

4 Discussion

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Stephen D Ginsberg

Roles

Author response to Decision Letter 0

Decision Letter 1

Stephen D Ginsberg

Roles

Acceptance letter

Stephen D Ginsberg

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases