Effects of Online Repetitive Transcranial Magnetic Stimulation (rTMS) on Cognitive Processing: A Meta-Analysis and Recommendations for Future Studies

Abstract

Online repetitive transcranial magnetic stimulation (rTMS), applied while subjects are performing a task, is widely used to disrupt brain regions underlying cognition. However, online rTMS has also induced “paradoxical enhancement”. Given the rapid proliferation of this approach, it is crucial to develop a better understanding of how online stimulation influences cognition, and the optimal parameters to achieve desired effects. To accomplish this goal, a quantitative meta-analysis was performed with random-effects models fitted to reaction time (RT) and accuracy data. The final dataset included 126 studies published between 1998 and 2016, with 244 total effects for reaction times, and 202 for accuracy. Meta-analytically, rTMS at 10 Hz and 20 Hz disrupted accuracy for attention, executive, language, memory, motor, and perception domains, while no effects were found with 1 Hz or 5 Hz. Stimulation applied at and 10 and 20 Hz slowed down RTs in attention and perception tasks. No performance enhancement was found. Our meta-regression analysis showed that fMRI-guided targeting and short inter-trial interval durations are associated with increased degree of disruption with rTMS.

I. Introduction

Repetitive transcranial magnetic stimulation (rTMS) is a non-invasive brain stimulation technique that uses brief, high intensity magnetic field pulses to modify neural activity. Over the last decades, rTMS has become a widely used research tool for studying the involvement of brain regions underlying cognitive processes such as attention, memory, language, and perception. A large variety of studies have shown that when applied over a central node of a brain network hypothesized to support a targeted cognitive function, rTMS can affect performance on the cognitive task. rTMS can be administered using two different paradigms, “online” and “offline”.

With offline rTMS, task performance is assessed before and after rTMS administration, during which series of pulse trains are applied over a period typically lasting 10 to 20 minutes. The cumulative effect of rTMS is a temporary modulation of cortical excitability in the targeted cortical region and its associated networks, which affects post-rTMS task performance as compared to that at pre-rTMS baseline. One typical aspect of offline stimulation is that rTMS-induced effects are frequency-dependent. As a rule of thumb, low frequencies (≤ 1 Hz) are associated with decreased cortical excitability, while higher frequencies (≤ 5 Hz) generally lead to increased cortical excitability (for a review, see [1]). Offline rTMS therefore provides evidence that a stimulated brain region may be involved in a cognitive process and a general heuristic for the frequency dependence of rTMS effects.

With online rTMS, the stimulation is applied at discrete time points while subjects are engaged in a cognitive task; and rather than looking for cumulative effects, the immediate effect on behavior is assessed. This provides a better insight into the timing of neural processing that underlies behavior. For example, seminal work by Amassian and colleagues [2] used an online TMS protocol based on single pulses to the occipital cortex while subjects performed a letter judgment task. They demonstrated that subjects failed to identify the letters only when TMS was applied between 80 and 100 ms after the presentation of the visual stimuli. Such experiments inspired the notion that TMS may cause a “virtual lesion”, in which online stimulation induces random firing of a neuronal population, causing a temporary disruption in ongoing cognitive processing. On the other hand, while less prevalent, performance enhancement in a variety of tasks involving perceptual, motor, and executive processing has also been report with online TMS (for a review, see [3]). It is unclear whether the frequency-dependent inhibitory–excitatory heuristics used in offline rTMS paradigms relate to online disruption or facilitation of cognitive processing.

The neuromodulatory effects of online rTMS depends heavily on the stimulation conditions, which are comprised of a vast composition of spatial and temporal parameters. Spatial parameters that affect the location and focality of stimulation include: coil geometry, stimulation target, targeting strategy, and stimulation intensity. Temporal parameters include: pulse waveform, pulse train frequency, number of pulses, inter-pulse train interval (or inter-trial interval, ITI), and timing of the pulses relative to cognitive processes that unfold during the concurrent behavioral task. Given this very large parameter space and the substantial heterogeneity across studies, it is important to evaluate how these stimulation parameters impact rTMS effects on human behavior. Thus, the current study reports the results of a quantitative meta-analysis of online rTMS effects in the cognitive domains of attention, executive functions, imagery, language, memory, motor task, and perception. rTMS effects we assessed on two main task performance outcomes—accuracy and reaction time. In a second step, a meta-regression was performed to further explore the moderator effects of relevant stimulation parameters. To orient the reader, a mini-review of the stimulation parameters is provided below.

A. Spatial Parameters

1). Coil Geometry:

The coil geometry determines the spatial extent and depth of the electric field induced in the brain, and consequently the neuronal effect of TMS and behavioral outcomes. Two main coil types are presently used when applying TMS: circular and figure-8. Circular coils were used in early TMS studies due to ease of construction; they induce a non-focal electric field distribution under the perimeter of the coil, and therefore are less capable of selective targeting of brain regions. Some of the so-called deep TMS coils also induce circular electric field patterns and are also classified as circular type coils [4]. Figure-8 coils, on the other hand, induce a more focal electric field beneath the central junction of the two windings. Not only do these two coil types differ in the spatial distribution of the induced electric field, but they also differ in their depth–focality profile [5]. To prevent confounds due to the coil shape, this meta-analysis will only consider studies using figure-8 coils, which are more prevalent in modern TMS studies.

2). Amplitude Dosing Approach:

In addition to coil geometry, stimulation amplitude also affects the focality as well as the intensity of stimulation. Amplitude dosing can either be set to a fixed stimulator output or individualized for each participant according to a physiological response induced by stimulation. When TMS is applied over the hand representation of the motor cortex, a muscle response is elicited in the contralateral hand, typically in the first dorsal interosseous or the abductor pollicis brevis, measured as a motor evoked potential (MEP) using electromyography (EMG). The resting motor threshold (rMT) is then defined as the lowest stimulation intensity to induce a MEP larger than 50 μV, at least 50% of the time in a finite number of trials (typically 10 trials). rMT is measured when the hand is at rest, but thresholds can also be assessed while the subject is voluntarily contracting their finger muscles. In that case, the active motor threshold (aMT) is typically defined as the lowest intensity required to reliably induce a MEP of at least 200 μV when the subject is exerting a voluntary contraction at 20% of the maximum strength of the target muscle. Similarly, when applied over the visual cortex, TMS induces a subjective perception of brief flashes of light, called phosphenes. These phosphenes can be either stationary, when TMS is applied over the primary visual cortex (V1), or moving, when TMS is applied over the motion-sensitive cortex (V5). Analogous to the motor threshold, phosphene threshold (PT) is defined as the lowest TMS intensity inducing phosphenes in at least 5 out of 10 pulses. Both motor and phosphene thresholding methods reflect a stimulation intensity necessary to induce effective neuronal depolarization, accounting for factors such as coil-to-cortex distance and individual cortical excitability. Subsequent application of TMS is then dosed in reference to rMT, aMT, or PT. However, while these methods provide a means to individualize dosing that has physiological relevance, they have been criticized since some studies have failed to find a correlation between rMT and PT [6, 7], suggesting that MT and PT may be inappropriate to guide amplitude selection for non-motor or non-visual areas of the brain. Consequently, some practitioners do not attempt to individualize intensity, but instead, use a fixed intensity across subjects, expressed as a percentage of the maximum stimulator output (MSO). Given the importance that amplitude plays in modulating cortical and cognitive function, the impact of this factor on behavioral outcomes will be assessed.

3). Targeting Method:

The ability to accurately localize brain function depends on the targeting method that is used. Early TMS studies localized the stimulated site using scalp measurements. For example, the standardized EEG coordinate system was used, which provides a fast and inexpensive way of localizing the stimulation target that somewhat accounts for individual head size. However, this scalp-based approach does not take into account inter-individual differences in brain anatomy, and has been shown not to be accurate for fine-grained targeting [8]. An alternative scalp-based approach relies on defining a functional hotspot, i.e, the optimal spot on the subject’s head that induces either muscle movement by stimulating the motor cortex; or visual phosphenes or degraded performance in visual task, by stimulating the visual cortex. While this method is subject-specific, it is limited to these two cortical areas that have readily observable physiological response and therefore cannot be used to define other stimulation sites. In the last two decades, the use of frameless stereotactic neuronavigation, coupled with neuroimaging have enabled more accurate and precise targeting of the TMS coil in relation to the individual anatomy or functional physiology. Neuronavigation-guided targeting can use either the subject’s anatomical magnetic resonance imaging (MRI) scans, individual functional MRI data, or functional neuroimaging data from group fMRI analysis, so-called “probabilistic targeting”. A study comparing the behavioral effects of four targeting methods—EEG-based scalp measurement, probabilistic approach, individual anatomical MRI, and individual functional MRI—revealed that individualized functionally guided rTMS was associated with the largest effects, while scalp measurement localization led to the smallest [9]. However, while this result is appealing, it was found for only a specific task (number comparison), and used a between-subject trial design with a small sample size (n = 5). As such, a more thorough evaluation of targeting approaches is still needed to better understand how this important factor influences TMS effects.

B. Temporal Parameters

1). Pulse Waveform:

Pulse waveform has been shown to influence TMS effects [10, 11]. Conventional TMS devices have an oscillator circuit topology that produce monophasic or biphasic sinuoidal current waveforms. Recent developments have enabled more flexible and efficient pulse waveforms to be generated (see [12] for review). However, such technology has not been widely adopted yet; for the most part all rTMS studies covered by the present meta-analysis used a standard biphasic waveform, and thus waveform will not be considered here.

2). Stimulation Frequency:

While it is widely accepted that the effects of offline rTMS are frequency-dependent, little is known about the effects of frequency when using online rTMS. Very few studies have used low-frequency (≤ 1 Hz) online rTMS because the slow pace of this stimulation does not usually match well with the temporal dynamics of most cognitive tasks. Consequently, short bursts or trains at higher frequencies of stimulation have been typically used, mainly as a means to maximize disruption within different task phases. In addition to disruption effects, it has been shown that online rTMS can induce local entrainment of endogenous brain oscillations occurring during a task, sometimes leading to performance enhancement [13]. This has been demonstrated in studies showing, for example, that only 5 Hz rTMS, and not 1 Hz or 20 Hz enhanced performance during a verbal maintenance task that is known to involved theta oscillations [14]. It has also been shown that cognitive performance in a mental rotation task could be enhanced when rTMS was applied at individualized alpha frequency, but not at 20 Hz or at 3 Hz less than the alpha frequency [15].

3). Number of Pulses Per Trial:

While it is typically assumed that applying a greater number of pulses per trial leads to stronger rTMS effects on behavior, this assumption has yet to be directly tested. The theoretical impact of the number of pulses has been evaluated in a handful of studies applying stimulation over the motor cortex, with offline TMS. Previous work with offline rTMS has shown that the number of pulses could drastically influence the cumulative effects. For example, increasing the number of pulses over the motor cortex either induced a stronger rTMS effect [16]; or totally reversed the expected effect [17]. However, with online rTMS, the number of pulses is often determined by the timing of the stimulus train relative to the phases of the task, by the duration of the cognitive process (or by the duration of the task phase), and by the stimulation frequency. Therefore, the number of pulses is generally not an independent parameter and is, instead, dependent on the specifics of the task used.

4). Inter-Trial Interval:

Another important temporal factor influencing TMS effects is the intertrial interval (ITI) that occurs between trains of stimulation pulses. In an effort to limit coil heating and reduce the risk of seizure, early TMS applications typically utilized long ITIs. However, with technological advances that improved the cooling and heat tolerance of the devices, and more optimistic safety data [18], shorter ITIs are being implemented more frequently. Few studies have explicitly tested how ITI influences offline rTMS effects. For example, one study compared the effects of applying 1200 pulses of 5 Hz rTMS over the motor cortex in either six blocks of 200 pulses with an ITI of 60 s between blocks, or in a continuous manner. The results showed that while the expected facilitation of the MEP was found with the 60 s ITI, the absence of ITI reversed the effect, leading to a trend toward inhibition of the MEP [19]. Interestingly, the same pattern of results has been found with continuous- and intermittent-theta burst stimulation (cTBS, iTBS, respectively), in which three-pulse 50 Hz bursts with an interburst interval of 200 ms or 5 Hz applied continuously (cTBS) or in a 2-seconds on/8-seconds off pattern (iTBS) [20]. Here, the continuous application of pulses (cTBS) led to an inhibitory effect on cortical excitability, while the introduction of an 8-second break between bursts in iTBS led to a facilitatory effect on the MEP. As such, there is some preliminary evidence from offline protocols that indicates the importance of ITI on rTMS motor system effects, but it is yet unknown how ITI influences online rTMS effects.

5). Stimulus Timing Relative to Cognitive Process:

Because online rTMS is intended to engage specific stages of neurocognitive processing that unfold as part of a designed behavioral task, the timing of pulses within an experimental design is an important factor that may modulate effects. For example, by applying online rTMS during a working memory task, a study showed performance enhancement associated with active stimulation only when rTMS was applied during the retention period, and not the probe period of the task, suggesting that the effects of online rTMS are timing-dependent [14, 21]. Other explanations have been proposed to explain time-specific rTMS effects. For example, when applied immediately before a cognitive process, it has been suggested that the auditory and/or the somatosensory sensations caused by the clicking sound or mechanical vibrations induced by rTMS could alter reaction time though inter-sensory facilitation mechanisms [22].

C. Control Conditions

In addition to direct neuronal stimulation effects, TMS produces a number of ancillary effects, including auditory and somatosensory responses. Depending on the coil location, some cranial nerves can also be activated during the stimulation, resulting in activation of afferent inputs to the brain. For the purpose of blinding, it is important for research studies to include a sham stimulation condition that mimics the ancillary effects of TMS without inducing neuronal depolarization. Some studies have used no-stimulation as a control condition. This is a weak control that does not attempt to match sensory confounds that are likely to contribute very strongly to observed differences. Thus, these studies will be excluded from the current meta-analysis. An alternative approach is to stimulate a control site, often the vertex of the head. As such, while this type of “active control” reproduces both the auditory and somatosensory sensations of active stimulation, it is based on the assumptions that (i) stimulating the control location has no impact on the cognitive process of interest, and (ii) that the stimulation does not spread to other adjacent brain regions involved in the cognitive process under consideration, both of which are not necessarily true. Possibly the most widely used approach to experimental control consists of comparison with some form of sham stimulation. In early TMS studies, sham stimulation was commonly implemented by tilting an active coil at an angle from its tangential placement, such that the strongest magnetic field is oriented away from the head, while the subject can still hear the coil clicking noise and receive some scalp stimulation from the coil periphery. However, the coil-tilt sham condition was shown to be about half as potent as active TMS in eliciting MEPs [23] and can induce observable changes in cerebral glucose metabolism [24]. More involved sham manipulations involve the use of a magnetic shield or spacer to reduce field strength reaching the head, in addition to electrical scalp stimulation to reproduce the scalp sensation of TMS. For this meta-analysis, the type of sham conditions used in each study is recorded, as it reflects the methodological quality and interpretation of the findings.

In summary, the number of stimulation parameters and range of possible settings within each parameter, are extremely large across TMS studies. Moreover, these parameters may interact with each other, and with the underlying state-dependent physiology that is expressed during the execution of behavioral tasks. As such, the field is still grappling with the optimal parameters that will induce selective modulation of brain function and behavior. The actual sampling of selected parameters in the published studies using online rTMS represents only a small fraction of the entire parameter space, and further, as is demonstrated in the present study, there is a large variability between studies in the choice of parameters. The sparse sampling of the parameter space, and the sampling variability, makes it challenging to draw conclusions on the optimal stimulation parameters for induction of rTMS effects. The aim of the present work is therefore to provide a quantitative meta-analysis to test the effectiveness of online rTMS in modulating performance, as defined by reaction time and accuracy, on cognitive tasks in healthy subjects, and to test the relevance of six factors as potential predictors of rTMS effects, with the hope of providing guidance towards such optimization.

II. Methods

A quantitative meta-analysis was performed according to the recommendations of the Cochrane group, involving the search, screening, and selection of eligible articles according to inclusion and exclusion criteria, quality and bias assessments, data extraction, and quantitative synthesis. The search protocol for this meta-analysis was registered on the PROSPERO international prospective register for systematic reviews (DOI: 10.15124/CRD42016038981 [25]).

A. Literature Search

Three authors (LB, WL, NC) searched in PubMed, the Cochrane Central Register of Controlled Trials, EMBASE, Web of Science, Scopus and PsycInfo databases through July 1, 2016 to identify and to compile all English-language studies that conducted a study exploring the effects of online rTMS on cognitive processing. The following keywords were used: “cognitive task”, “cognitive process”, “cognition”, “cognitive”, “memory”, “working memory”, “attention”, “visual task”, “vision”, “language”, “decision making”, “decision-making”, “perception”, “reasoning”, “executive function”, “cognitive function”, “judgment”, or “number”. These search terms were combined with keywords describing brain stimulation, including: “online transcranial magnetic stimulation”, “online TMS”, “concurrent transcranial magnetic stimulation”, “concurrent TMS”, “event-related transcranial magnetic stimulation”, “event-related TMS”, “transcranial magnetic stimulation”, “TMS”, “repetitive transcranial magnetic stimulation”, or “rTMS”.

B. Study Screening

Each paper from the initial electronic search were submitted to two screening steps for inclusion; the criteria are listed in Table I. The first step, consisting in screening each title and abstract, was performed by two investigators (LB and WL), independently. Inter-rater reliability was assessed using a randomly chosen sample of 108 articles. Only one paper led to a discrepancy between the two screeners. Articles that passed the title/abstract screening were then subjected to full-text screening. Two investigators (LB and CC) independently reviewed the full text of each article to decide further inclusion for data extraction. To assess inter-rater reliability, 101 articles were randomly chosen across the 751 remaining papers, and a disagreement was found on three papers only, leading to a Cohen’s kappa coefficient of 0.896. Discrepancies between investigators were resolved through group discussion among authors.

Table I:

Inclusion and exclusion criteria for study selection

Study characteristics	Inclusion criteria	Exclusion criteria
Population	Healthy controls; age ≥ 18; male and female	Presence of any psychiatric or neurological conditions
Interventions	Online rTMS; using a figure-8 coil; applied over any brain region	Offline rTMS; non-figure-8 coil; single pulse TMS
Comparators	Active control site; sham configurations including: dedicated sham coil, electrical cutaneous stimulation, or coil tilt	No stimulation; no control site
Outcomes	Reaction time and accuracy in cognitive task	Outcomes normalized as a function of results obtained during a no-stimulation block
Timing	Task performed during the rTMS session	Task performed after the rTMS session
Study design	Parallel groups, cross-over	Case study

C. Risk of Bias Assessment

Following the Cochrane guidelines [26], the methodological quality of the included studies was assessed by rating the risk of bias in the following domains: 1) randomization, 2) allocation concealment, 3) blinding of participants, 4) blinding of personnel, 5) blinding of outcomes assessor, 6) incomplete outcome data, 7) selective reporting, and 8) other bias. For each category of risk, two investigators (LB and DN) assigned one of the following ratings: “Low Risk”, “High Risk”, or “Unclear”. Inter-rater reliability was calculated on a sample of 83 papers out of 143. Disagreement were found on 10 papers for randomization, 6 papers for allocation concealment, and 7 papers for blinding of participants. No disagreements were found on the four other categories. On average, 93% agreement between both investigators was found in the inter-rated sample. Discrepancies between the two investigators were resolved through group discussion with a third investigator (CC). Finally, funnel plots and Harbord’s regression were used to assess for possible publication bias.

D. Data Extraction

Two investigators (LB and ZD) extracted the reaction time and accuracy data. For each article, information was extracted on study design, population characteristics, cognitive domains, a priori hypothesized directionality of rTMS effect, targeted brain regions, and type of sham method. Stimulation parameters such as intensity and frequency of stimulation, number of TMS pulses applied in each trial, timing of pulses relative to the trial, as well as technical information about the equipment used were extracted. Then, the principal outcomes measures of cognitive task performance were extracted: 1) mean reaction time and the corresponding standard error; 2) mean accuracy and the corresponding standard error. When data were only presented in graphic format, “Engauge Digitizer” [27] was used to digitize the plots. Finally, when variances were not reported, missing data were replaced by the average of the standard deviation across all other studies.

E. Quantitative Synthesis

Using the Mantel–Haenszel method, random effects models were fitted to each outcome to synthesize a pooled odds ratio with 95% confidence intervals. Effect sizes are presented as standardized mean difference (Hedge’s g). Analyses were conducted separately for each cognitive domains and stratified by frequency of stimulation (1 Hz, 5 Hz, 10–19 Hz, ≥20 Hz). Outcome heterogeneity was assessed using the Cochran’s Q, τ², and I² statistics. A meta-regression was then performed using four categorical covariates: 1) stated study intent (disrupt, enhance, not specified), 2) targeting strategy (scalp measurement, hotspot, anatomical MRI, functional MRI), 3) frequency of stimulation (1 Hz, 5 Hz, 10–19 Hz, ≥20 Hz), and 4) amplitude dosing approach (individual threshold-based, fixed device output); and two continuous variables: 1) inter-trial interval, and 2) number of pulses per trial.

III. Results

A. Search Results

Search criteria across the six different databases yielded 9134 citations. Of those, 8383 articles were excluded at the abstract and title screening stage because they were performed on patients, rTMS was applied offline, they used single pulse TMS, they used a circular coil, they did not use a control condition, or they used blocks without an appropriate stimulation control condition. The full text of the remaining 751 articles were then examined, and 608 additional articles were excluded because they also did not match the inclusion criteria or because the outcomes were expressed as a percentage of change between active rTMS and control conditions. Figure 1 summarizes the flow of articles through the search and screening steps. The meta-analytic dataset therefore included 143 studies published between 1998 and 2016. Among this sample, 17 articles from Balconi et al. were excluded as they reported inconsistent results within study. The final data set included 126 studies.

Figure 1:

Consort diagram.

B. Descriptive Statistics

In the following sections, a general overview of the three main parameters used with online rTMS (cognitive domains and expected effect, spatial parameters, and temporal parameters) are presented using descriptive statistics. Each of the following result is expressed as a percentage of the sum of the adjusted number of subjects across all studies (N = 3401).

1). Cognitive Domains, a priori Directionality of Effect, and Sample Size:

Across all cognitive domains (Figure 2A), memory and perception were the two most studied (31 and 26%, respectively), while far fewer studies addressed the areas of imagery (3%), motor (3%), executive function (11%) and attention (10%). Although a recent review has shown that online rTMS can enhance performance [3], the large majority of studies (79%) planned to disrupt cognitive functions (Figure 2B), suggesting that the virtual lesion concept still dominates the field. The median number of subjects across studies is 5 (Figure 2C).

Figure 2:

Studies summary. Percentage experiments addressing: A) cognitive domains, B) a priori directionality of effect, C) histogram of sample size included across studies (adjusted for number of experiments), D) targeting approach, E) stimulated brain target, F) use of neuronavigation, G) amplitude dosing approach, H) stimulation frequency, I) inter-trial interval (ITI), J) number of pulses per trial, and K) stimulation timing relative to the cognitive process.

2). Spatial Parameters: Targeting Method, Cortical Target, and Amplitude Dosing:

To localize the stimulated target (Figure 2D), the majority of studies used anatomical MRI guidance (52%), about a third of the studies relied on scalp measurement (27%), followed by fMRI (15%) and hotspot targeting (6%). The most commonly stimulated area is the occipital cortex (52%), followed by the frontal cortex (36%), and relatively few studies investigated the effect of rTMS over other cortical regions (Figure 2E). The majority of studies (77%) used neuronavigation to guide coil positioning during stimulation (Figure 2F). To define the intensity of stimulation, the majority of studies (73%) used individualized thresholds, such as motor or phosphenes threshold, while 27% applied stimulation using a fixed intensity referenced to the maximum stimulator output (MSO) (Figure 2G).

3). Temporal Parameters: Stimulation Frequency, Number of Pulses per Trial, Inter-Trial Interval, and Stimulation Timing:

Unlike offline rTMS studies, which commonly utilize low frequency stimulation, only 1% of the online studies used 1 Hz stimulation. Rather, the vast majority of studies used faster frequencies, with 14% of studies using 5 Hz, 67% using stimulation between 10 and 19 Hz, and 18% using more than 20 Hz (Figure 2H). Substantial variability was observed for the range of ITIs used, with durations ranging from 300 ms to 37400 ms between burst of pulses (Figure 2I). For the number of pulses administered on each trial, a bimodal distribution appeared (Figure 2J), with a first group of studies using less than 10 pulses per trial, and another group using more than 20 pulses per trial. For the stimulation timing, although it has been suggested that stimulation applied before a cognitive process might induce performance enhancement, only 5% of the studies applied rTMS at this timing, while 90% stimulated the brain concurrent to the expected cognitive process (Figure 2K).

C. Meta-Analysis

1). Accuracy:

As shown in Figure 3A, no studies investigated the effects of 1 Hz rTMS, and 5 Hz rTMS has only been applied only for memory and motor cognitive domains, but did not lead to any significant effects when compared to sham. Although 10 Hz rTMS significantly disrupted performance when applied during language, memory, motor and perception tasks, it did not have a significant meta-analytic effect on attention, executive function or imagery. Finally, 20 Hz rTMS significantly disrupted accuracy for tasks involving attention, executive functions, memory and perception, but did not significantly affect imagery, language and motor tasks. When looking at the effects across all frequencies, it appears that online rTMS consistently disrupted accuracy, when compared to sham rTMS, for all domains except imagery. Significant accuracy enhancement was not found for any of these domains or frequencies.

Figure 3:

Summary of the standardized mean differences and 95% confidence intervals between active and sham rTMS obtained on A) accuracy, and B) reaction times, stratified by cognitive domains and frequency of stimulation. Negative values (blue) represent lower accuracy or slower reaction time associated with active rTMS, while positive values (red) represent higher accuracy or faster reaction time. Light grey indicates that no data were found, while dark grey indicates that the rTMS effect did not reach significance. These tables are summarizing the forest plots that can be found in the supplementary material.

2). Reaction Time:

Studies investigating the effects of 1 Hz and 5 Hz rTMS did not produce a significant meta-analytic effect of active over sham on attention or memory domains. rTMS applied at 10 and 20 Hz both increased reaction times when applied during tasks involving attention and perception, while no significant effects were found for the five other cognitive domains. This overall analysis of rTMS effects across all frequencies of stimulation revealed performance disruption for attention and perception domains only (Figure 3B).

Collectively, these results revealed that online rTMS significantly disrupted accuracy and reaction time performance when applied at 10 or 20 Hz on specific cognitive domains. While these effects were consistently disruptive, they were also associated with a large amount of variability. For example, when applied at 20 Hz, active rTMS disrupted performance for both accuracy and reaction times in the attention domain, but only affected accuracy when applied during tasks involving executive function. Therefore, a crucial remaining issue is to explore factors that predict the rTMS effects. This is the goal of the following analysis.

D. Meta-Regression

Eight factors (stated study intent, brain target, targeting approach, frequency, amplitude dosing, ITI, number of pulses, and type of control) were entered into a multivariate analysis to assess the potential predictors of rTMS effects. The stimulation timing relative to the cognitive task was not included due to uneven distribution of the data (90% applied rTMS during the cognitive process vs. 5% before and 1% after). The continuous variables (ITI, and number of pulses) were not normally distributed, and therefore were log-transformed. The effect of each of the predictors was compared to a reference point, defined as the modality leading to the smallest change in performance between active rTMS and the control condition, as defined by the Hedges’ g value. Finally, we performed a backward stepwise regression, in which covariates were iteratively removed until the model with the lowest Akaike information criterion (AIC) estimate was obtained.

1). Accuracy:

The optimal (lowest AIC) regression model explained 11.8% of the variance in rTMS effects (F(6, 156) = 4.60, p < 0.001). Three out of eight factors predicted a significant difference between active and sham rTMS: stated study intent, targeting approach, and ITI (Table II). The analyses revealed that studies that did not state intent showed stronger disruptive effects on accuracy compared to studies intended to enhance performance (t = −2.34, p = 0.02). The analysis of the targeting approach showed that compared to studies using a functional hotspot, studies using fMRI-guidance produced stronger disruptive effects on accuracy (t = −2.68, p < 0.01), a trend in the same direction was observed for studies using anatomical MRI-guided targeting (t = −1.77, p = 0.08). For the continuous predictors, the ITI negatively predicted the rTMS effect with shorter ITI leading to stronger disruptive effects, or longer ITI leading to stronger performance enhancement (t = 2.22, p = 0.03).

Table II:

Optimal regression model for accuracy, which includes three covariates: stated study intent, targeting approach, and ITI.

Covariate	Factor	Ref. level	β	SE	t-value	p-value
Stated study intent	Disrupt	Enhance	0.04	0.16	0.28	0.78
	Unspecified		−0.46	0.20	−2.34	0.02*
Targeting	MRI	Hotspot	−0.36	0.21	−1.77	0.08
	fMRI		−0.61	0.23	−2.68	< 0.01*
	Scalp measurement		−0.32	0.22	−1.45	0.15
ITI (log)	−	−	0.27	0.12	2.22	0.03*

2). Reaction Time:

The optimal regression model for reaction time significantly explained 8.0% of the variance in rTMS effects (F (5, 238) = 5.25, p < 0.001). Among the eight predictors included, stated study intent and stimulation frequency led to significant differences between active and sham rTMS. Compared to studies intended to enhance performance, studies intended to disrupt performance significantly slowed down reaction time (t = 3.67, p < 0.001). Regarding the effect of the stimulation frequency, 5 Hz rTMS significantly slowed down reaction time compared to rTMS applied between 10 and 19 Hz (t = 2.89, p < 0.01).

E. Risk of Bias Assessment

Risk of bias assessment is summarized in Figure 4. Risk of bias for randomization was low as 93% of the studies reported using a random or counter-balanced assignment for allocating participants to the different stimulation conditions.

Figure 4:

Assessment of risk of bias, presented as percentages across all included studies.

Concerning allocation concealment, only 8% of the studies reported allocation schemes. This information was not reported in the other studies, leading to an unclear risk for allocation concealment. Regarding the blinding of participants, and more precisely the sham procedure employed, different procedures were reported. The large majority of studies used a control site (57.7%) and a minority used an electrically-matched sham coil (8%). These two procedures mimic both the auditory and the somatosensory sensations induced by the active stimulation, and thus lead to a low risk of bias for the blinding of participants, as they are less likely to report a difference between both stimulations. On the other hand, in 24% of the studies, the coil was held at a particular angle from the head (generally 45°), while other studies placed a non-discharging coil (8.1%), or a piece of plywood (0.7%), between the head and the active coil, with the added distance attenuating the magnetic field before it reached the brain. While these latter methods still produce approximately the same auditory sensation, the somatosensory sensation tends to be quite different. While getting these kinds of sham stimulation, subjects might be able to tell the difference between active and sham stimulation, which could therefore lead to a potentially high risk of bias. A high risk of bias was also attributed to studies contrasting active rTMS with rTMS applied at a very low intensity of stimulation (1.3%), which creates less intense somatosensory and auditory sensations.

No information was provided regarding the blinding of personnel, suggesting that all of the studies were single-blinded. As such, this leads to the conclusion that there was a high risk of bias as the experimenters could have unconsciously influenced the subject’s performance. Finally, for the parameters of Blinding of Outcome Assessment, Incompleteness of Outcome Data, Selective Reporting and Others, no information was found in the articles leading to an unclear or low risk of bias. In summary, while the majority of studies used standard procedures for the randomization and implementation of sham and therefore appear to be of good quality, the blinding of personnel appears to be largely ignored by all the studies.

F. Publication Bias and Heterogeneity

Funnel plots of the standardized mean difference between active and sham rTMS against study precision, as defined by standard error, were used to quantify potential publication bias for each of the cognitive domains. A symmetric inverted funnel shape indicates that publication bias is unlikely [28]. Heterogeneity of effects was low for both accuracy (Figure 5A; τ² = 0.0958; I² = 0.0%; Q = 151.69, p > 0.99) and reaction time (Figure 5A; τ² = 0.06188; I² = 0.0%; Q = 126.78, p > 0.99) and reaction time.

Figure 5:

Funnel plot assessing potential publication bias in the A) accuracy and B) reaction time outcomes. The outer dashed lines indicate the triangular region within which 95% of studies are expected to lie in the absence of biases and heterogeneity.

IV. Discussion

The goal of this quantitative meta-analysis was to gain a better understanding of the effects of online rTMS on cognitive functions. Across the 126 studies analyzed, only about half found significant disruptive effect of active rTMS over sham (53% and 50% of studies for accuracy and reaction time, respectively). Among the studies showing a significant rTMS effect, online rTMS was more likely to impair performance (88% and 73% of the studies for accuracy and reaction time, respectively), rather than to enhance it (12% and 27%). This meta-analysis also demonstrated a range of effect sizes across distinct domains of cognitive functioning (Hedge’s g of −0.39–−0.08 and 0.02–0.26 for accuracy and reaction time, respectively), suggesting that even modest effects are highly dependent on the underlying cognition being tested.

In order to more fully characterize the factors that contributed to rTMS efficacy, a meta-regression was performed. We included 6 categorical and 2 continuous variables as predictors of the rTMS effects, expressed as the standardized mean difference between active and sham, on accuracy and reaction time. These factors were selected as they have been shown to interact with rTMS effects when used in an offline fashion. When considering rTMS effects on accuracy, significant predictors are stated study intent, targeting approach, and ITI. First, studies with no stated intent showed stronger disruptive effect compared to the effect obtained for studies that stated the intent to disrupt performance. Second, when compared to studies using hotspot targeting, studies relying on fMRI-guidance to define the stimulation site led to the strongest effect. Studies defining the stimulation target by using anatomical landmarks led to stronger effect than the ones using functional hotspot, but the difference was not significant. Third, shorter ITIs induced stronger disruptive rTMS effects (or longer ITIs lead to stronger enhancement). Similar results were found when investigating the rTMS effects on reaction time, such that studies intending to disrupt performance significantly slowed down reaction time compared to studies trying to enhance performance. This analysis also found that when applied at 5 Hz, rTMS disrupted performance compared to studies applying rTMS between 10–19 Hz. This result indicated a frequency-dependent heuristic, but one that is different than one typically reported in offline studies. Lastly, in this meta-analysis, several factors failed to influence rTMS outcomes: brain target, amplitude dosing, number of pulses per trial, and the type of control. These factors are discussed in greater detail in the following sections.

A. Predictors of rTMS Effect

1). Stated Study Intent:

An unique feature of this meta-analysis is the inclusion of “stated study intent” as a predictor of potential intervention effects. Because of the strong frequency-dependent heuristic known from offline stimulation, many of the online studies analyzed here reported planned effects with prespecified directionality. Regression results from this meta analysis suggest that stated study intent significantly predicted rTMS effects on both accuracy and reaction time performance.

The analysis performed on reaction time showed that, as expected, studies intended to disrupt performance significantly slowed down reaction time compared to those intended to enhance performance. However, the analysis of the accuracy measure suggested that when the stated study intent was not specified, the disruption was stronger compared to the studies that tried to enhance performance, while no differences were found between disruptive compared to enhancement intent. Two biases might explain this result. First, the repartition of studies within each of these categories in not evenly distributed: only 13% of studies tried to enhance performance, while 79% tried to disrupt performance. Sample size inequality can create heterogeneous variance that affects the model fit. Moreover, one can wonder how often the “stated” study intent was based on a priori hypothesis or if it was driven by the results. Initially, online rTMS appeared to show promise as a tool to study neurocognitive function by disrupting ongoing processing (e.g., visual “virtual lesions” [2]; speech arrest [29]). However, a number of studies have shown that rTMS can also cause performance enhancements (for a review, see [3]). While the mechanisms of such enhancements are not as yet well-understood, the ability to cause both disruption and enhancement of performance via rTMS provides a useful tool, both in terms of experimental design and control, and as a way to explore neurocognitive processing. As such, although the conclusion from this meta-analysis pointed primarily towards a disruptive effect of rTMS, it is important to consider that as of yet, only a few online studies reported intent to enhance performance. Thus, while the perturb-and-measure approach is still largely predominant in the field, the present analysis points to a large gap in the use of this potentially powerful tool to enhance cognitive performance. Therefore, one recommendation for future studies is to pre-register the hypothesis of each study before performing them.

2). Targeting Approach:

rTMS effects were also predicted by the targeting approach. As compared to studies using hotspot targeting, studies relying on functional image-guided rTMS led to stronger rTMS disruptive effects. This result is in agreement with Sack and colleagues who conducted a head-to-head comparison of different targeting methods and found the greatest effects for individualized fMRI-guidance, followed by structural MRI-guidance, then use of Talairach coordinates from group fMRI, and lastly use of scalp-based 10–20 EEG coordinates for targeting produced the smallest effects. Strikingly, this demonstrates that individualized fMRI guidance increased targeting efficacy by a factor of 10 over scalp-based targeting methods [9]. The current meta-analytic regression results suggest that the superiority found for imaging-based targeting, especially if used on an individualized basis, can be generalized across the field of rTMS studies. Only 15% of the included studies used fMRI-guidance, while 27% used scalp measurement. The distribution of targeting methods observed in this meta-analysis was thought to be due to grouping studies across time, assuming that scalp measurement was used in the older studies and, as targeting technology improved, that more recent studies would rely on MRI- and fMRI-guidance. However, while 100% of the studies published between 1998 and 2002 used scalp measurements for targeting, this approach was gradually replaced by MR-guidance such that between 2007 and 2013 70% of studies used MR-guidance. Interestingly, this trend has reversed somewhat in the latest studies, such that in 2016, a majority of studies used scalp measurement (44%), followed by fMRI-guidance (25%), MRI-guidance (18%), and hotspot targeting (13%). This could be responsible, at least in a part, for the lack of efficacy of online rTMS in facilitating cognitive performance.

3). Stimulation Frequency:

While many factors determine how a particular type of cognition is affected by rTMS, stimulation frequency is often considered one of the dominant factors. Here, rTMS at 5 Hz significantly slowed down reaction times, compared to rTMS applied in the 10–19 Hz range, while 1 Hz rTMS did not induce any changes. One possible reason for this is that when using online stimulation one can impose a burst of high frequency rTMS concurrently with a task, but it is harder to do so for low frequency stimulation. The descriptive results bear this out, as almost no studies have applied 1 Hz rTMS on-line (1%). The frequency-dependence of online rTMS deviates from the offline low-frequency-inhibitory/high-frequency-excitatory heuristic. When investigating the superiority of active over sham rTMS across each cognitive domain and stimulation frequency, it was found that active rTMS applied at 10 Hz and 20 Hz disrupted accuracy for attention, executive, language, memory, motor, and perception domains, while no effects were found with 1 Hz or 5 Hz. Stimulation applied at 10 and 20 Hz slowed down RTs in attention and perception task. Indeed, it has been suggested that applying rTMS frequencies similar to ongoing endogenous brain oscillations could cause entrainment that impacts cognitive performance, depending on the function of the oscillation [30–32]. Online comparison of various frequencies has shown frequency-specific performance effects in visual detection [33, 34] and working memory [14]. Notably, in Chanes et al. (2013), while short trains applied at 30 Hz affected behavior, trains of the same length but with pulses applied at irregular intervals did not, providing evidence that the frequency itself was important [33]. While the distribution of studies between cognitive domain and stimulation frequency prevented a direct test of this interaction, it appears that the ongoing brain state—and its concomitant oscillatory patterns—is a likely determining factor in the efficacy of rTMS applications.

4). Inter-Trial Interval:

The inter-trial interval (ITI) is a more subtle design element that is not frequently the focus of rTMS studies other than those targeting the motor cortex [35]. Here, however, it was found to provide a significant prediction of rTMS effects across studies with shorter ITIs inducing stronger disruptive rTMS effects (or longer ITIs lead to stronger performance enhancement). There are plausible psychological and physiological explanations. Psychologically, one could assume that having stimulus trains too close to each other might simply be too distracting for the subjects, and therefore disrupt performance. Physiologically, ITI can influence both second- and millisecond-scale dependencies in cortical excitability. For example, applying single pulse every 10 seconds versus one pulse every second, leads to different effects on cortical excitability [36]. On the millisecond scale, the classic paired pulse paradigm shows that a preceding conditioning pulse can inhibit or facilitate the effect of the subsequent probing pulse depending on the inter-stimulus interval [37]. Similar mechanisms could be in play for patterned stimulation. As in the case of theta burst stimulation, inter-burst interval affects the inhibitory–excitatory balance of the underlying cortex [20, 38]. It may be that the interaction of ITI with homeostatic processes responding acutely to the excitatory or inhibitory effects of individual rTMS trains explains the effects found in the present analysis.

5). Factors that Did Not Predict rTMS Effect:

Neither the number of pulses per trial, nor the amplitude dosing method predicted rTMS effects. Studies using a fixed intensity in reference to percent stimulator output, and studies using individualized intensities based on motor or phosphene threshold, did not differ in terms of rTMS effects. The use of these two different approaches reflects current disagreement in the field about how to define this parameter. On one hand, since there is large variation among individuals in head anatomy and the induced electric field strength in the brain, dosing techniques that accounts for these inter-individual differences seem appropriate. On the other hand, even accounting for local tissue variations, physiological thresholds derived from the motor or visual cortices may not be relevant for other cortical regions [39]. The lack of difference observed in the present analyses bears out this concern, and certainly does not provide evidence for preferring one method over the other. It is clear from the physics of TMS that using a fixed intensity in a study introduces a great deal of variability in field strength and distribution, and that a method of equalizing dose across individuals is preferable. Such a method, perhaps based on local reactivity using TMS evoked EEG potentials and/or computational electric field models, standardized across the field, is urgently required.

Our results did not show significant effects of brain target on reaction time or on accuracy outcomes. Our parcellation was coarse; we performed our analysis at the level of lobes of the brain, which are expected to interact with the type of cognitive process. However, much more data is needed to conduct a more granular analysis and with interactions. Finally, different type of control conditions were compared. While a high risk of bias was attributed to control condition that induced highly different somatosensory sensations compared to the active stimulation (spacer, low intensity of stimulation), no differences were found on reaction time and accuracy effects.

B. Limitations

There are a number of limitations of this meta analysis that deserve consideration. For example, the choice and division of the cognitive domains were imperfect. For example, working memory was included in the memory domain, but could have also been included in the executive domain. Given that the results showed disruptive rTMS effects across all cognitive domains, it is unlikely that reclassification of certain cognitive function would alter the outcomes, but it is important to note that other domains or assignments may have led to different findings.

In this meta-analysis, several factors could not be tested because of insufficient detail in the original report. For example, the stimulation timing relative to the cognitive task was not evaluated. Only a handful of studies have been mindful of such “chronometric” parameters [40] or the interaction of the cortical target with the stimulation timing [41]. Nonetheless, this chronometric information can provide evidence for the selective involvement of a cortical region/network in processing at a certain time; this has been largely ignored in the studies included in this meta-analysis. Indeed, the descriptive statistics showed 90% of the studies applied stimulation online during the cognitive process, while only 5% and 1% of the studies applied it immediately before or immediately after the cognitive process. While this distribution prevented including this factor in the regression analysis, it could explain why the overall effect of rTMS was disruptive, with little sign of performance enhancement, which often requires that TMS is not applied while cortical processing is ongoing, but, for example, immediately prior to that processing.

The actual stimulation intensity relative to threshold was not evaluated even though it is widely appreciated that intensity can have an impact on TMS outcomes, for example on motor evoked potential size. This variable could not have been included in the current analysis, primarily because of the disagreement in dosing method, and also because of the large variability in studies using stimulation intensity ranging from 80% to 120% rMT, aMT or PT. Another spatial factor that was not tested was TMS coil orientation. It has, for instance, been demonstrated that orienting the coil such that the induced current is perpendicular to the target gyrus produces more profound effects than when the current is induced parallel to it [42]. However, no method of orienting the coil relative to target cortex has been conventionally adopted outside of the motor cortex, and moreover the vast majority of studies included in this analysis did not report any information on coil orientation.

Finally, the available data is insufficient to examine interactions between the different predictors presented in Tables II and III. For example, both logical interactions (e.g., the number of pulses and their dependence on the duration of each phase of the task) and more qualitative interactions (e.g., the type of cognitive task is dependence on stimulation frequency given the role of endogenous brain oscillations) could have been considered, however, adding these interactions would have substantially reduce the statistical power of the analysis, and therefore remain an interesting avenue for future research.

Table III:

Optimal regression model for reaction time, which includes two covariates: stated study intent and stimulation frequency.

Covariate	Factor	Ref. level	β	SE	t-value	p-value
Stated study intent	Disrupt	Enhance	0.60	0.17	3.67	< 0.001*
	Unspecified		0.12	0.20	0.58	0.56
Frequency	1 Hz	10–19 Hz	0.40	0.31	1.31	0.19
	5 Hz		0.46	0.16	2.89	< 0.01*
	≥ 20 Hz		0.13	0.08	1.57	0.12

V. Conclusions and Recommendations for Future Studies

Based on the results of this meta-analysis, we arrived at a number of recommendations for future online rTMS studies seeking to achieve reliable effects on cognition. Methodologically, future studies should strive for individualized, functional imaging guidance to localize the stimulation target. Clearly the first step to successfully engage the target is the reliable identification of the neural substrate underlying the particular cognitive process of interest. Neuronavigation technology is becoming widely accessible, and furthermore electric field modeling based on individualized cortical anatomy may help to both normalize the dosage of rTMS, as well as provide valuable post-hoc information that accounts for individual differences in rTMS response. Lastly, studies must find reliable techniques for demonstrating engagement with regions that do not have a readily measurable response, such as the DLPFC. The rise in use of pre–post treatment neuroimaging, as well as concurrent applications of TMS–fMRI or TMS–EEG, may help serve to address this need.

On the level of design, a number of specific proscriptions are recommended. Shorter ITI duration might be used to disrupt accuracy (or longer duration to enhance it). More studies should address the influence of the ITI on behavioral outcomes for the online rTMS protocol, and all studies should report this duration for a better understanding of its effect. Frequency-specificity was also a clear trend in the effect sizes reported here. Stimulation should be applied at 5 Hz to disrupt reaction time. While the superiority of this frequency remains unclear, studying relationship of online rTMS frequencies and endogenous brain oscillations might help answering that question. For the stated study intent, a way to potentially improve knowledge about rTMS effects on behavioral outcomes would be to pre-register all study hypotheses before starting the experiments. Indeed, being able to know if online rTMS will up-regulate and/or down-regulate the network function during a task, i.e., enhance and disrupt cognitive performance, could largely enhance the use of rTMS as a reproducible experimental tool.

Therapeutically, there is great interest in combining rTMS with concurrent behavioral interventions for the treatment of neuropsychiatric disorders, including major depressive disorder, post-traumatic stress disorder, obsessive compulsive disorder, schizophrenia, and autism (see [43] for review). In these combinatorial treatment studies, the whole rTMS sequence is considered as an inhibitory or excitatory stimulation paradigm, which augments the concurrent behavioral intervention (cognitive behavioral therapy, exposure therapy, etc.) that is considered as a way of functionally maintaining a certain “brain state”. There is a gap between this framework and online rTMS in which stimuli are time-locked to a particular phase of a cognitive process at the millisecond timescale. Nevertheless, in designing treatment studies, one should consider the effects of stimulation parameters, such as ones we found to be significant—stimulation frequency, targeting approach, and ITI.

More recently, computational electric field modeling has been proposed as a means to optimize targeting to the stimulated site. Indeed, while all of the formerly mentioned targeting methods assume that the peak of the TMS-induced electric field stimulates neurons located immediately underneath the TMS coil, electric field modeling actually estimates the distribution of the electric field within the cerebral tissue. Such electric field models take into account differences in tissue-specific impedance, such as cerebral spinal fluid, grey matter, and white matter, as well as the interface between these tissues that induce distortions of the electric field. Therefore, electric field modeling can provide highly individualized information to estimate the optimal coil position and orientation in order to induce the desired stimulation intensity and focality at the targeted location. However, this technique is far from being established as a standard tool, and only starting to be used in some studies [21].

Finally, many of the studies with online rTMS were underpowered. This was exacerbated by multiple experimental conditions with the same group of subjects within each study. The median sample size across studies—when adjusted for the number of experimental conditions—was 5. Larger sample sizes are critical for obtaining reliable and valid results.