Recovery of reward function in problematic substance users using a combination of robotics, electrophysiology, and TMS

Abstract

Background:

Theoretical and empirical work suggest that addictive drugs potentiate dopaminergic reinforcement learning signals and disrupt the reward function of its neural targets, including the anterior midcingulate cortex (aMCC) and the basal ganglia. Here, we aim to use prefrontal 10-Hz TMS to enhance aMCC reward activity and reward learning by the basal ganglia in problematic substance users.

Methods:

22 problematic substance users were randomized into an Active and SHAM (coil flipped) TMS group. We recorded the reward positivity—an electrophysiological signal believed to index sensitivity of the aMCC to rewards—while participants engaged in 4 blocks (100 trials per block) of a reward-based choice task. A robotic arm positioned a TMS coil over a prefrontal cortex target, and 50 pulses were delivered at 10-Hz before every 10 trials of blocks 2–4 (1500 pulses, 400 trials). Participants then completed a decision-making task that is diagnostic of striatal dopamine dysfunction.

Results:

The present study revealed three main findings. First, both groups failed to elicit a reward positivity during the first two task blocks. Second, applying robot-assisted TMS enhanced the amplitude of the reward positivity in the Active group, but not the SHAM group, across the last two task blocks. Third, the Active group performed relatively better at reward-based learning than the SHAM group.

Conclusion:

These results demonstrate that 10-Hz TMS is successful in modulating the reward function of the aMCC and basal ganglia in problematic substance users, which may have utility in the treatment of reward-related neural dysfunction commonly associated with substance use disorders.

1. Introduction

A cardinal feature of substance use disorders–continued drug consumption despite long-term adverse consequences–is indicative of abnormal cognitive control and decision-making (Redish et al., 2008), processes that are mediated in part by two key neural targets of the mesocorticolimbic reward system. First, the anterior midcingulate cortex (aMCC) is said to utilize dopaminergic reward prediction error signals (RPEs)¹ to learn the value of rewards for the purpose of selecting and motivating the execution of goal-directed behavior (Holroyd and Coles, 2002; Holroyd and Yeung, 2012). Second, dopaminergic RPE signaling in the basal ganglia plays a role in the facilitation and suppression of action representations during decision-making processes (Frank et al., 2004; Cohen and Frank, 2009). Theoretical and empirical work suggest that the potentiation of dopaminergic RPE signals by addictive drugs upsets the reward function of the aMCC and the basal ganglia, and precipitates the abnormal motivation, cognitive control, and decision making processes characteristic of SUD (Redish et al., 2008; Baker et al., 2011; Maia and Frank, 2011; Baker et al., 2013; Holroyd and Umemoto, 2016). Yet, despite the progress made in the identification of neural substrates involved in the development and maintenance of drug-seeking behavior, including those mediating drug reward and reinforcement, the neuroscience of addiction has yet to be translated into significant advances in the treatment of addiction (Ekhtiari et al., 2019; Verdejo-Garcia et al., 2019). For example, pharmacological interventions have shown limited success in reversing drug-induced behavior by acting on mesolimbic dopamine signaling, namely in the nucleus accumbens (Potenza et al., 2011; Chung et al., 2016). Here, rather than targeting hedonic hotspots (i.e., the affective enjoyment or ‘liking’ of the drug reward when consumed), we aimed to modify dopaminergic RPE signals and their neural targets, which appear to increase the ‘wanting’ of rewards, that is, the motivation to work for the reward in a given behavioral context (Redish et al., 2008). To do so, we conducted a proof-of-concept study using a combination of robotics, non-invasive brain stimulation, and RPE-related biomarkers to modify abnormal neural and behavioral reward responses in problematic substance users.

Recent neuroimaging data have revealed that the reward processing function associated with the aMCC is impaired across various SUDs (Peoples, 2002; Goldstein et al., 2007; Baker et al., 2011; Holroyd and Umemoto, 2016). Notably, excessive drug use is associated with aMCC hypoactivity to natural rewards and hyperactivity to drug-related rewards and cues (Peoples, 2002; Goldstein et al., 2009; Baker et al., 2016c, 2017; Moeller et al., 2018). We have argued that the distortion of RPE signals by addictive drugs would upset the normal function of aMCC by associating the pursuit of the drug-related goals with enhanced reward value, while de-potentiating the value of other, non-drug-related activities (Baker et al., 2011, 2016a, 2016c). Support for this proposal comes from observations of a component of the event-related brain potential (ERP) called the reward positivity. The reward positivity is observed as a differential response in the ERP to positive and negative feedback received during goal-orientated decision-making tasks (Holroyd et al., 2008; Baker and Holroyd, 2011; Proudfit, 2015). It is believed that the impact of positive (increase in dopamine) and negative (decrease in dopamine) RPEs on the aMCC following goal-directed feedback modulates the amplitude of the reward positivity (Holroyd and Coles, 2002; Baker and Holroyd, 2009; Holroyd and Yeung, 2012). Although hotly debated, converging evidence across multiple methodologies indicate that the reward positivity reflects an RPE signal and is generated by aMCC (Holroyd and Yeung, 2012; Sambrook and Goslin, 2015; Holroyd and Umemoto, 2016). Importantly, a series of ERP experiments have shown that problematic substance users produce a severely blunted reward positivity to monetary feedback (Baker et al., 2011, 2016a), but following an overnight period of abstinence, the size of the reward positivity in smokers is normalized by feedback stimuli that signify a chance to puff on a cigarette (Baker et al., 2016c, 2017). Our findings of reduced reward positivity amplitude in individuals who tend to misuse a range of drugs of abuse suggest abnormal goal-directed behavior in this population, which dovetail previous neuroimaging studies on SUD (Peoples, 2002; Goldstein et al., 2007).

In addition, numerous studies have focused on the contributions of the basal ganglia to impaired decision making in SUD (Redish et al., 2008), as well as the development of stereotypical behaviors involved in drug consumption (Everitt and Robbins, 2005; Redish et al., 2008; Everitt and Robbins, 2016). A biological-based model of reinforcement learning holds that positive RPEs facilitate approach learning by reinforcing dorsal striatal D1 receptors, whereas negative RPE facilitate avoidance learning by reinforcing dorsal striatal D2 receptors (Frank et al., 2004; Cohen and Frank, 2009). Empirically, the model predictions are tested with the probabilistic selection task (PST), a trial-and-error learning task that is believed to be sensitive to dopamine dysfunction (Maia and Frank, 2011). Extensive psychiatric and neurological research indicate that a disruption in positive and negative RPE signaling in the dorsal striatum can selectively impair approach and avoidance learning, respectively (Maia and Frank, 2011). For example, people with Parkinson’s disease are more accurate at avoidance than approach learning while off medication (low dopamine), and more accurate on approach than avoidance learning trials while on medication (high dopamine), as predicted by the model (Frank et al., 2004, 2007b; Wiecki and Frank, 2010). We recently hypothesized that chronic drug use might similarly affect PST performance (Baker et al., 2011, 2013), as most classes of drugs of abuse effectively increase the magnitude of the positive RPEs by raising extracellular dopamine levels either directly or indirectly (Dichiara and Imperato, 1988). Indeed, tobacco smokers show deficits in learning from positive RPEs during smoking abstinence (low dopamine), but enhanced reward learning following acute cigarette consumption (high dopamine) (Baker et al., 2020b). Problematic substance users display a global deficit in PST performance (Baker et al., 2011, 2013), which may have actually preceded drug use (Baker et al., 2019). Such abnormalities in striatal functioning may present a core susceptibility to decision-making deficits and contribute to several substance-related problems (e.g. treatment dropout, poor physical and mental health, impaired social functioning, and high unemployment rates) (Biernacki et al., 2016).

Counteracting such drug-induced neurocognitive deficits thus relies on identifying non-invasive brain stimulation methods capable of modulating RPE-related activity in the aMCC and striatum. In fact, an FDA approved treatment for depression, 10-Hz repetitive transcranial magnetic stimulation (rTMS) to the left dorsal lateral prefrontal cortex (DLPFC), has been shown to enhance dopamine release, neuronal activity, and cerebral blow flow in the cingulate cortex of healthy individuals (Hayward et al., 2007; Cho and Strafella, 2009; Paus et al., 2001; Strafella et al., 2001). While aMCC is too deep for TMS to directly impact its activity, TMS neuroimaging studies have demonstrated that TMS exerts distant effects localized in networks connected to the site of stimulation (Sandrini et al., 2011; Fox et al., 2012). Thus, given the dense anatomical connections between the DLPFC and the cingulate cortex (Koski and Paus, 2000), it has been suggested that TMS effects on aMCC activity might be mediated by local activation of the neuron terminals in the DLPFC and aMCC (Paus et al., 2001). Furthermore, others suggest that stimulation of the DLPFC, and subsequent activation of the ventral tegmental area, can modulate dopaminergic activity in the cingulate through its connectivity with the ventral tegmental area, an argument supported by a recent rTMS study (Cho and Strafella, 2009). In fact, some argue that the observed modulation of dopamine release and frontal-cingulate connectivity might mediate, at least in part, the therapeutic effects of rTMS seen in patients with depression (Paus and Barrett, 2004; Fox et al., 2012). By extension, we have shown that 10-Hz rTMS applied to the left DLPFC enhanced the reward positivity to monetary rewards in smokers, bolstering the utility of TMS in modulating the RPE-related aMCC activity in problematic substance users (Baker et al., 2017). Further, TMS-PET imaging studies have also revealed that 10-Hz rTMS to the left DLPFC can induce dopamine release in the basal ganglia, namely in the region of the ipsilateral dorsal striatum known to receive most of the projections of the prefrontal area that was stimulated (Strafella et al., 2001). Together, previous studies combining TMS with neuroimaging exemplify nicely how TMS can be used to modulate dopamine and neural activity in the cingulate cortex and striatum in healthy individuals, thus presenting a potential therapeutic strategy for restoring the aMCC and striatal reward function in SUD.

In this proof of concept study, we aimed to measure the feasibility of applying prefrontal 10-Hz rTMS for the purpose of recovering the reward function associated with the aMCC (as evaluated by the reward positivity) and the decision-making function associated with the basal-ganglia (as evaluated by the PST) in problematic substance users. To note, conventional TMS systems are subject to human error in coil positioning and maintenance of coil position within (and between) testing sessions, as well as compromises in the participants’ comfort and their muscle tone over time, thus corrupting stimulation and imaging results (Lancaster et al., 2004; Matthaus et al., 2006). To overcome such limitations, robot-assisted TMS (R-TMS) has recently emerged as an superior alternative because it allows real-time tracking of head movements and automatic adjustment of the coil position in tandem with the subject’s head movements, does not require rigid fixation of the subject’s head, and coil positions can be easily stored and retrieved, ensuring consistency within and between testing sessions (Matthaus et al., 2006). Thus, R-TMS is expected to improve effect sizes, permit investigators to test hypotheses with smaller samples, and maintain target consistency, particularly during long experimental sessions and across multiple sessions (De Goede et al., 2018; Finke et al., 2008; Ginhoux et al., 2013; Kantelhardt et al., 2010; Lancaster et al., 2004; Matthaus et al., 2006). Combined with the use of the reward positivity to monitor aMCC function, R-TMS thus provides an unprecedented opportunity to systematically study the impact of 10-rTMS on aMCC activity in SUD during task performance, with high precision and reliability.

Assuming that 10-Hz rTMS over the left DLPFC would enhance dopamine and neural activity in the aMCC, we predicted a positive shift in the aMCC valuation of monetary rewards in problematic substance users. To test this prediction, we recorded the reward positivity from two groups of problematic substance users (Active TMS and SHAM) as they navigated a ‘virtual T-maze’ to find rewards (Baker and Holroyd, 2009; Baker et al., 2017). After the first block of the task, the Active group received 10-Hz rTMS over the left DLPFC using a robot-assisted TMS system, which continuously tracks the coil position in real-time to ensure targeting precision during task performance (Matthaus et al., 2006). The coil in the SHAM group was flipped to guarantee that no brain stimulation occurred but participants received the same protocol and auditory sensation. We specifically examined three questions. First, we tested whether, consistent with our previous findings, feedback stimuli indicating small monetary rewards would elicit an attenuated reward positivity in both groups of problematic substance users prior to rTMS. Second, we asked whether rTMS would normalize this impairment in the Active TMS group. Third, if striatal dopamine activity is amplified following the rTMS protocol (assuming an after-effect of 5–20 min), we asked whether individuals in the Active group would perform better at approach learning but worse at avoidance learning when compared to the SHAM group.

It remains an open question of whether a systematically managed TMS session could reliably influence neural signals over time and result in long-lasting changes in behavior. While future longitudinal experiments could potentially answer this question, a critical first step would be to identify whether TMS can impact RPE-related neural activity and behavior in one experimental session. By including both ERP and behavioral experiments in a single session, we are able to conduct between–subject analyses of the relationships between rTMS and the integrity of the reward processing function of the aMCC (as revealed by the reward positivity) on the one hand, and the decision-making function of the basal ganglia (as revealed by the PST) on the other. Given that previous research has not assessed the impact of TMS on reward processing in substance users, the results presented in this proof-of-concept study would provide the first step toward the use of TMS in the recovery of reward-related neural dysfunction in SUD.

2. Methods

2.1. Participant recruitment and assessment

Participants were recruited through Rutgers Alcohol and Drug Assistance Program and local advertisements placed in the Newark community (e.g. social networking groups, recovery houses) using an ‘expression of interest’ flyer. Those interested were directed to contact the research team via email or phone to determine whether they meet initial inclusion/exclusion criteria (e.g., substance use within the last 3 months; TMS contraindications; older than 18 years of age). Eligible participants were then scheduled to complete one experimental EEG-TMS session. On the day of testing, participants were asked to provide informed consent and were then randomly assigned to either a TMS or SHAM session. Participants were screened for TMS contraindications (e. g., pregnancy, metal implants, history of seizures or seizure medication), un-correctable visual impairment, uninterruptable central nervous system medication, and severe brain injury (traumatic or acquired). Participants were excluded for severe mental health issues (e.g., schizophrenia, bipolar) but not mood disorders such as depression or anxiety. Personality risk factors for substance dependence were assessed using the Substance Use Risk Profile Scale (SURPS) (Woicik et al., 2009; see Supplementary Information for more details).

In line with our previous work, we defined the level of substance misuse as frequency of substance use behaviors that impose a significant cost on the individual, are difficult to interrupt, and are likely to recur after interruption (Baker et al., 2011). Substance misuse by this definition was measured using the Global Continuum of Substance Risk (GCR) score of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) to assess problematic substance use (Humeniuk et al., 2008; Newcombe et al., 2005). The ASSIST is a validated screening test using DSM-specific criteria for identifying the degree of problematic substance use (i.e., tobacco, alcohol, cannabis, cocaine, amphetamine-type stimulants, sedatives, hallucinogens, inhalants, opioids, and “other drugs”), especially in individuals who consume a variety of different substances, for example, as occurs with polydrug use. According to cut-offs scores established in validation studies of the ASSIST, mid-range GCR scores between 16 and 39 are an indication of hazardous or harmful substance use. GCR scores of 39.5 and higher suggests that the individual is at high risk of substance dependence (Humeniuk et al., 2008, Newcombe et al., 2005). Participants meeting these criteria (GCR >16) were included in the study. Individuals with GCR scores less than 16 are likely not experiencing any substance-related problems and were excluded from the study. Participants also self-reported their smoking behavior using the Fagerström Test for Nicotine Dependence (Heatherton et al., 1991; Pomerleau et al., 1994). Participants were financially reimbursed for their time and all data obtained was kept strictly confidential. The study was approved by the local research ethics committee and was conducted in accordance with the ethical standards prescribed in the 1964 Declaration of Helsinki.

2.2. Electrophysiological task: virtual T-maze

The virtual T-maze is a reward-based choice task that elicits robust reward positivities (Baker and Holroyd, 2009; Baker et al., 2011; Baker et al., 2016a; Baker et al., 2016c). Participants navigated the virtual T-maze by pressing left and right buttons corresponding to images of a left and right alley presented on a computer screen. After each response, an image of the chosen alley appeared, followed by a feedback stimulus (apple or orange) indicating whether the participant received 0 or 5 cents on that trial; unbeknown to the participants, the feedback was random and equiprobable. The experiment consisted of four blocks of 100 trials each separated by rest periods.

2.3. EEG data acquisition

EEG was recorded using a montage of 23 electrodes placed according to the extended international 10–20 system (Jasper, 1958). Signals were acquired using Ag–AgCl ring electrodes mounted in a nylon electrode cap with a conductive gel (Falk Minow Services, Herrsching, Germany). Signals were amplified by low-noise electrode differential amplifiers with a frequency response of DC 0.017–67.5 Hz (90-dB octave roll-off) and digitized at a rate of 1000 samples per second. Impedances were reduced to less than 10 kΩ. Digitized signals were recorded to disk using Brain Vision Recorder software (Brain Products GmbH, Munich, Germany). Two electrodes were also placed on the left and right mastoids. The EEG was recorded using the average reference. For the purpose of artifact correction, the horizontal EOG was recorded from the external canthi of both eyes, and vertical EOG was recorded from the suborbit of the right eye and electrode channel Fp2.

2.4. Reward positivity analysis

Post-processing and data visualization were performed using Brain Vision Analyzer software (Brain Products GmbH). The digitized signals were filtered using a fourth-order digital Butterworth filter with a bandpass of 0.10–0.20 Hz. A 1000-msec epoch of data extending from 200 msec before to 800 msec after the onset of each feedback stimulus was extracted from the continuous data file for analysis. Ocular artifacts were corrected using the eye movement correction algorithm described by Gratton et al. (1983). By convention, the EEG data were re-referenced to linked mastoids electrodes and baseline-corrected by subtracting from each sample the mean voltage associated with that electrode during the 200-msec interval preceding stimulus onset (Baker et al., 2011, 2013, 2016a, 2016b, 2016c). Muscular and other artifacts were removed using a ±150-μV level threshold and a ±35-μV step threshold as rejection criteria. ERPs were then created for each electrode and participant by averaging the single-trial EEG according to feedback type (Reward, No-reward). Reward positivity amplitude was determined by identifying the maximum absolute amplitude of the difference wave within a 200- to 400-ms window following feedback onset (Baker et al., 2011). The difference wave method, which was recommended in a recent meta-analysis of reward positivity studies, isolates the reward positivity from other ERP components (Sambrook and Goslin, 2015). The reward positivity was evaluated at Cz (Baker et al., 2017). In addition, the P2, N2, and P3 components were also measured for the purpose of comparison. These the mean amplitude of the P2 and N2 components were measured at channel Cz, and at channel Pz for the P3. We used a window extending from 150 to 250 msec for the P2, 250–350 msec for the N2, and 350–450 msec for the P3 following the presentation of the feedback stimulus.

2.5. Task performance analysis

T-maze performance measures included: (a) Post-feedback RT (in milliseconds); and (b) win-stay and lose-shift (WSLS) behavior. We excluded trials with RTs faster than 100 msec and slower than 5% of the higher bound (Ratcliff, 1993). An Ex-Gaussian model was fitted to the post-feedback RT and three model parameters (mu, sigma, and tau) were estimated using Matlab DISTRIB tool box (Lacouture and Cousineau, 2008). The ex-Gaussian method provides an opportunity to examine both fast and slow RTs, and intra-trial variability (Heathcote et al., 2002, Lin et al., 2015, Van Belle et al., 2015). Sigma represents the variability of RT of the Gaussian component and tau is sensitive to intra-individual RT variability in terms of extremely slow but infrequent responses. An analysis of WSLS behavior and their corresponding RTs were also conducted (see Fig. 3 and Tables S5 and S6).

Fig. 3.

Reward Positivity results. ERPs elicited by reward feedback (blue), no-reward feedback (red) and difference wave (black-reward positivity) averaged across all blocks for the SHAM group (A) and Active TMS Group (B). For purpose of comparison, topoplots denote the amplitude of the reward positivity at 300 msec for the SHAM and Active group averaged across TMS blocks. (C) Reward positivity amplitude across blocks for the SHAM (solid line, triangle) and Active (dashed line, circle) TMS group. Red box denotes the TMS blocks. Significant effects are shown as follows: *p < .05, **p < .01, ***p < .005, (two-tailed). Error bars denote standard error. Data are associated with channel Cz and negative is plotted up by convention.

2.6. Probabilistic selection task

Immediately following the T-Maze, participants were asked to complete the Probabilistic Selection Task (PST) (Frank et al., 2004). Briefly, during an initial Learning phase, participants were presented with three pairs of stimuli (AB, CD, and EF), with the response mappings probabilistically allocated such that one stimulus in each of the three pairs is rewarded on 80%, 70%, and 60% of the trials, respectively, with the remaining stimulus in each pair rewarded on the complementary percentage of trials. Participants learned by trial-and-error to choose the more frequently rewarded stimulus over the alternative in each pair. Critically, they could do so either by learning that particular stimuli were associated with relatively more reward (A, C and E), by learning that particular stimuli were associated with relatively more punishment (B, D and F), or both. During the Test Phase, participants were exposed to all possible combinations of these stimuli in a random order and were required to select the symbol in each pair that they believed to be correct, but without receiving any feedback about their choices, to determine whether subjects learn more from postivie or negative feedback. If participants learned more from positive feedback during the Learning Phase, then they should reliably choose the good stimulus “A” in all novel test pairs in which it is present. On the other hand, if they learned more from negative feedback during the Learning Phase, then they should reliably avoid the Bad Stimulus “B” in all novel test pairs in which it is present. Consistent with standard practice, participants who did not perform better than chance on Test Phase trials consisting of the easiest stimulus pair (AB) were eliminated from further analysis (Frank et al., 2007a; Frank et al., 2004; Baker et al., 2011, 2013).

2.7. Transcranial magnetic stimulation

The stimulator device was a MagPro X100 with the Cool-B70 figure-of-eight coil (MagVenture, Falun, Denmark). Following EEG-cap set-up, resting Motor Threshold (rMT) measurements was performed via visual twitch in the contralateral (right) hand. The coil was positioned over the supposed left motor cortex area (electrode location C3) and the coil was moved in a grid like search pattern until the location at which a reproducible abductor pollicis brevis response was detected. rMT was then defined as the lowest stimulation intensity, expressed as a percentage of max output of the Magstim equipment, that reliably yielded a visible muscle twitch in the hand when stimulating the hand area of the contralateral motor cortex with a single pulse (3–5 times). Stimulation intensity for each experiment was set at 110% of maximal stimulator output. Average rMT stimulation intensity was 62.14% (range 50%–80%) and maximum stimulation output was 88% (range: 55%–88%).

At the start of the experiment a 3D model of the subjects’ head surface was created, and the robotic arm, head model, and the tracking system were registered to a common coordinate system (SmartMove, ANT Neuro, Enschede, The Netherlands). The TMS target was placed on the x y z coordinate corresponding to electrode location F3, and the corresponding point on the surface is selected and the orientation of the coil is calculated tangentially to cortical surface and 45° to a sagittal plane based on the head model. The virtual coil position was then transformed to robot coordinates, which define the movement of the robot to the corresponding target position relative to the subject’s head. The position of the cranium is continuously tracked, and the trajectory of the coil’s path is adapted to head movements in real-time. This guarantees a precise <1 mm coil-to-target position and allows free head movements during the experiment.

2.8. Experimental design

All consenting procedures and questionnaires were administered at the beginning of the experiment. Following the electrode cap set-up and rMT measurements, the robotic arm positioned the TMS coil over the electrode location F3, and maintained the coil position 10 mm from the scalp at a 45°angle (Fig. 1). The EEG was recorded while participants freely navigated the virtual T-maze to find monetary rewards (Fig. 1C and D) and were asked to respond in a way that maximized their rewards. The experiment consisted of four blocks of 100 trials separated by rest periods (400 trials total; Fig. 1C). Following the first block, 50 rTMS pulses were delivered at 110% of participants’ rMT at 10-Hz continuously over the predefined left DLFPC target (duration: 5 s) immediately before each of the next 10 blocks of 10 trials (duration: 30 s per block). The duration between the last pulse of the train and first trial exceeded 100 msec, allowing EEG data to be collected without TMS artifacts. This sequence was repeated for Blocks 3 and 4, with self-timed rest breaks between blocks. Each TMS block consisted of 500 pulses (total: 1500 pulses) and 100 trials (total: 150 reward and 150 no-reward trials). The total duration of the task was approx. 20 min. The SHAM condition replicated this protocol, however the coil was flipped 180° to ensure that participants did not receive active simulation but received the same auditory sensation. After the task was completed, participants engaged in the PST task within 5 min of the last pulse sequence. At the end of the experiment, the EEG cap was removed and participants were debriefed and compensated for their participation.

Fig. 1.

Experimental Protocol. (A) Ri-TMS system (B) Block sequence (C) Single trial sequence. (D) ERPs associated with Reward (blue), No-reward (red), and reward positivity (black line-difference wave). Reward positivity occurs over frontal-central areas of the scalp about 250 to 350 ms post-feedback. Negative is plotted up by convention. The representative dataset of heathy adults is from Baker et al., 2020a.

2.9. Statistical analysis strategy

We conducted separate two-factor analysis of variance (ANOVA) on reward positivity and T-maze performance (mu, sigma, and tau; WSLS) with block (Blk1–4) as a within-subject factor, and TBS group (Active, SHAM) as a between-subject factor, followed by polynomial contrasts to test the type of trend (linear or quadratic). Sex and age were included as co-variates and Type 1 errors were controlled following Benjamini and Hochberg (1995) with a corrected significance level of α = 0.05. Given the small sample size, a sensitivity analysis was also conducted to determine the minimum effect size we were adequately powered to detect.

3. Results

3.1. Participants

A total of 38 participants were recruited and screened for the present study. Of these 38 participants, 10 participants met exclusion criteria due to a previous brain injury [n = 1], a pre-existing mental illness [n = 1] and TMS contraindications (e.g., history of seizures [n = 1], metal spinal implant [n = 1], CNS medication [n = 2], sleepiness/sleep deprivation [n = 3])(Rossi et al., 2009). Although the exclusion rates in this study were not ideal, it is not uncommon for TMS studies of SUD populations (Ekhtiari et al., 2019). Further, we could not perform TMS on one subject due to technological problems and the data of one participant was excluded because of excessive EEG artifacts. Four participants who failed to learn the PST task were excluded from the TMS analysis (this portion was comparable with previous work, Baker et al., 2013). In total, the data of 22 substances users (GCR score: M = 104, ±12 | age: M = 44, ±3) were included in this study, with 11 subjects per TMS group (Active and SHAM; see Table 1 and Fig. 2 for subject/group information).

Table 1.

Sample characteristics (n = 22).

	Active		SHAM			Effect size (Cohen’s d)
	M	(SE)	M	(SE)	t (20) = t, p-value
Sample (N)	n = 11		n = 11
Sex (male/female)	4/7		10/1
Age (years)	39.5	(4)	49.7	(4)	−1.9, p > .05	0.81
GCR	88	(12)	120	(21)	−1.3, p > .05	0.58
Lifetime use	18	(3)	17	(2)	0.19, p > .05	0.08
SSI
Nicotine	18	(2)	23	(2)	−1.9, p > .05	0.82
Alcohol	10	(2)	15	(3)	− 1.4, p > .05	0.62
Cannabis	6	(2)	9	(3)	− 0.72, p > .05	0.31
Cocaine	8	(2)	9	(4)	− 0.22, p > .05	0.11
Opioids	12	(4)	30	(2)	− 4.4, p < .001*	1.91
Amphetamines	30	(1)	5	(3)	− 0.88, p > .05	0.37
Sedatives	4	(2)	4	(3)	0.08, p > .05	0.03
Hallucinogens	4	(2)	3	(3)	0.26, p > .05	0.11
Inhalants	2	(2)	3	(3)	− 0.22, p > .05	0.09
FSRQ	3.8	(0.6)	3.5	(0.6)	0.37, p > .05	0.14
SURPS
Hopelessness	13	(1)	11	(1)	1.1, p > .05	0.46
Anxiety sensitivity	13	(0.6)	13	(0.9)	0.00, p > .05	0.00
Sensation seeking	15	(1)	15	(1)	0.00, p > .05	0.00
Impulsivity	13	(0.8)	10	(0.5)	2.9, p < .01*	1.23

Numbers indicate means and standard errors of the mean, except for Sample, which indicates sample size (N), and Sex, which indicates participant number (male/female).SSI-Specific Substance Involvement Score of the ASSIST. Midrange SSI scores between 4 and 26 are an indication of hazardous or harmful substance use, and SSI score of 27+ are indication of dependence on that substance (Humeniuk et al., 2008; Newcombe et al., 2005).

^*denotes signficant difference.

Fig. 2.

Substance preference. (A) SHAM (white bars) or Active (dashed bars) TMS group according to their scores on the Global Continuum of Substance Risk (GCR) scale (Left panel) of the ASSIST. Red line denotes cut-offs scores established in previous validation studies of the ASSIST for substance dependence (score > 39.5) (Newcombe et al., 2005). Right panel: Substance preferences for SHAM (white bars) and Active (dashed bars) TMS groups as measured by the Specific Substance Involvement Score of the ASSIST. Red line denotes the bottom end of SSI midrange scores (score = 4; 11 for alcohol) for any substance.

3.2. Virtual T-maze results

3.2.1. . Reward positivity

Fig. 3A and B illustrates the ERPs elicited by the Reward and No-reward feedback and the reward positivity (difference wave), averaged across blocks separately for the SHAM and Active TMS Groups (see also Fig. S1). The ERPs for the Active Group revealed a reward positivity with a relatively small amplitude (M = −1.12 μV, ±0.5) occurring at about 250–300 ms following feedback presentation (Fig. 3B), whereas the SHAM Group failed to produce a reward positivity (M = −0.23 μV, ±0.5). Visual inspection of Fig. 3A show that the condition-specific ERPs for the SHAM Group were nearly identical, exhibiting little difference between conditions (Fig. 1A). Fig. 3C presents the reward positivity across blocks, revealing a truncated reward positivity for both groups at Block 1 (Pre-TMS) and Block 2 (first block of TMS), but an elevation in reward positivity amplitude for the Active Group at Block 3 and Block 4 (see Fig. S1).

This effect was confirmed by a repeated measures ANOVA on reward positivity amplitude, which revealed a significant Block × TMS group interaction, F_{3, 54} = 2.98, p < .05, ${\mathrm{\eta }}_{\mathrm{p}}^2=0.14$. Post-hoc analysis revealed that while no differences between groups were observed for Block 1 (t₍₂₀₎ − 1.2, p = .215, Cohen’s d = 0.55) and Block 2 (t₍₂₀₎ −1.0, p = .318, Cohen’s d = 0.43), the amplitude of the reward positivity was larger for the Active group at Block 3 (t₍₂₀₎ −3.2, p < .005, Cohen’s d = 1.42) and Block 4 (t₍₂₀₎ −2.78, p < .01, Cohen’s d = 1.24), with a corrected significance level of α = 0.05 (Benjamini and Hochberg, 1995). This interaction remained statistically significant when variability associated with substance use scores (e.g. GCR, Opioid SSI score) and personality-related risk factors for substance use (i.e., depression-proneness, anxiety, impulsivity and sensation seeking), as measured by the SURPS, were controlled for (p < .05). Importantly, with the current sample size of n = 22, we could reasonably detect a minimum detectable effect size of d = 0.25 or ${\mathrm{\eta }}_{\mathrm{p}}^2=0.08$. Thus, our observed effect size (${\mathrm{\eta }}_{\mathrm{p}}^2=0.14$, large effect by convention) was greater than the predicted minimal detectable effect size $\left({\mathrm{\eta }}_{\mathrm{p}}^2=0.08\right)$, suggesting the null hypothesis could be rejected. Thus, the difference between groups appears to reflect the impact of rTMS on reward positivity amplitude, and likely not due to the small sample size.

Further, as a check, the reward positivity for the SHAM group was not significantly different than zero for any Block (all p’s > .05, all Cohen’s d’s < 0.25). By contrast, the reward positivity for the Active group was not significantly different from zero at Block 1 (p > .05, Cohen’s d = 0.62) and Block 2 (p > .05, Cohen’s d = 0.47), but was for Block 3 (p < .01, Cohen’s d = 1.05) and Block 4 (p < .01, Cohen’s d = 1.02). An analysis on other ERP components (P200, N200, P300) indicated that their amplitudes were about the same for the two groups (p > .05), confirming that the effect of interest was isolated to the predicted ERP component, the reward positivity (see Tables S1 to S4). Unexpectedly, the N100 was relatively larger for the Active TMS group (p < .05) (see Table S4). However, this component did not change as a function of TMS (p > .05), and the Block × TMS interaction remained significant when the N100 amplitude was controlled for (p < .05).

3.2.2. T-maze task performance

Repeated measures ANOVAs on ex-Gaussian parameters found no main effect of post-feedback responses (post-reward/post-noreward), group or interaction for mu, sigma and tau parameters (all p’s > .05, all ƞ2 < 0.07). In regards to win-stay and lose-shift behavior, no significant main effects of block or group, nor any interaction between block and group were detected (all p’s > .05, all ƞ2 < 0.02).

3.3. Probabilistic selection task results

3.3.1. PST testing phase results

A repeated-measures ANOVA on test phase accuracy with stimulus condition (approach and avoidance) as a within-subject factor and TMS group (SHAM vs. Active) as a between-subject factor did not reveal a main effect of stimulus, F_{1, 18} = 0.01, p > .05, ${\mathrm{\eta }}_{\mathrm{p}}^2=0.01$, but a trend toward a main effect of TMS group, F_{1, 18} = 2.52, p = .13, ${\mathrm{\eta }}_{\mathrm{p}}^2=0.12$. An interaction was not observed (p > .05). Given the moderate effect size of the main effect of group, which exceeded that of the effect size predicted by sensitivity analyses $\left({\mathrm{\eta }}_{\mathrm{p}}^2=0.08\right)$, we decided to explore this result a little further by testing our specific predictions of TMS separately on approach and avoidance accuracy. Interestingly, there was a marginal difference of approach accuracy between group means as determined by one-way ANOVA (F_1,20 = 3.1, p = .09, ${\mathrm{\eta }}_{\mathrm{p}}^2=0.13$), indicating that the Active Group were more accurate at approach performance (M = 74%, SE = 7) than the SHAM group (M = 57%, SE = 0.5), Cohen’s d = 0.78. Performance on avoidance accuracy were about the same between groups (F1,20 = 0.95, p = .34, ${\mathrm{\eta }}_{\mathrm{p}}^2=0.04$), (see Fig. 4c).

Fig. 4.

Behavior results. (A) Top Panels. Ex-Gaussian parameters (Mu [left], sigma [middle], and tau [right]) associated with post-Reward and post-NoReward RTs for the SHAM (solid line, triangle) and Active (dashed line, circle) group. (B) Percentage of win-stay (blue lines) and lose-shift (red-lines) choice behavior across blocks for SHAM and Active TMS groups. (C) Performance on the Probabilistic Selection Task (PST). Accuracy (left panel) and RT (Right panel) data in the Test Phase of the PST for the SHAM and Active TMS Groups, separately for the Approach and Avoidance conditions. (D) An exploratory correlation analysis between the reward positivity amplitude and approach accuracy revealed a marginal correlation at Block 4 (r = −0.389, p = .07, two-tailed), but not any other block, indicating that the larger the reward positivity at the end of the task, the more accurate participants were at learning from positive feedback. Error bars denote standard error.

4. Discussion

An extensive literature has detailed the potentiating effects of addictive substances on the mesocorticolimbic reward system (Dichiara and Imperato, 1988), a maladaptive process thought to dysregulate reward-related processes of the aMCC and basal ganglia (Redish et al., 2008; Baker et al., 2011). Despite efforts to identify methods to counteract such drug-induced neural alterations, brain-based interventions for this disorder remain underdeveloped (Ferenczi and Deisseroth, 2016). In the present proof-of-concept study, we made the first attempt to use robot-assisted 10-Hz rTMS for the purpose of recovering the reward function associated with the aMCC (as evaluated by the reward positivity) and the decision-making function associated with the basal ganglia (as evaluated by the PST) in problematic substance users.

Foremost, by applying 10-Hz rTMS to the left DLPFC using a robot-assisted TMS system, we were successful at recovering the aMCC reward response in problematic substance users. To be specific, across the first two blocks of the task, feedback stimuli indicating monetary rewards failed to elicit a reward positivity in both groups of problematic substance users, a finding consistent with previous work (Baker et al., 2011, 2016b). It is believed that the reward positivity is produced by the impact of RPEs carried by the midbrain dopamine system onto aMCC, where they are utilized to learn the value of behavior directed toward particular goals (Holroyd and Yeung, 2012; Holroyd and Umemoto, 2016). Evaluated in this context, the absence of the reward positivity amplitude in individuals who tend to misuse drugs of abuse suggest abnormal goal-directed behavior in this population (Baker et al., 2016b). More importantly, unlike the case for the SHAM group in which the absence of the reward positivity persisted across the remainder of the task, we found a quantifiable reward positivity in the Active group in the last two blocks of the task. In other words, our results clearly demonstrate that reward positivity amplitude was boosted by rTMS, suggesting that the aMCC assigned a value to the pursuit of small monetary rewards in this sample. These findings are consistent with our previous TMS study on smokers, in which TMS enhanced the reward positivity to monetary rewards relative to drug-related rewards (Baker et al., 2017). Further, the effect of TMS was isolated to the predicted ERP component, the reward positivity, and was independent of group differences related to substance use and personality factors. Together, these findings indicate that by potentiating dopaminergic RPE signaling in the aMCC (as indicated by the reward positivity), rTMS corrected the reward system in problematic substance users by enhancing the value of monetary rewards. This idea dovetails previous research suggesting that DLPFC stimulation indirectly increases activity of the ventral tegmental area, and subsequently its dopamine signaling to its projected targets, such as the cingulate and striatum (Cho and Strafella, 2009; Ferenczi and Deisseroth, 2016; Ferenczi et al., 2016; Strafella et al., 2001). It is worth noting that the reward positivity amplitude is predictably altered by pharmaceutical manipulations of the dopamine system and is highly correlated with mesocorticolimbic reward activation (Holroyd and Yeung, 2012; Holroyd and Umemoto, 2016; Proudfit, 2015). Taken together, a closer examination of the role of rTMS in modulating aMCC reward activity should greatly advance clinical and therapeutic research in SUD.

To be specific, these observations elucidate an important yet under investigated role of rTMS in enhancing the reward valuation function of the aMCC in SUD, and underscore the utility of the reward positivity as a highly sensitive biomarker of SUD severity and TMS treatment efficacy. While the clinical utility of TMS has been investigated in SUD for over a decade (e.g., nicotine, alcohol, cocaine), the majority of these studies have evaluated clinical endpoints (e.g. craving, consumption, and relapse) (Ekhtiari et al., 2019; Enokibara et al., 2016; Stein et al., 2018), and have often produced mixed results (Gorelick et al., 2014; Stein et al., 2018). Although clinical outcomes are highly relevant, studies designed to target the aberrant reward and cognitive processes that sustain SUD may better illuminate how TMS may improve neurocognitive functioning in these disorders (Ferenczi and Deisseroth, 2016; Baker et al., 2017; Stein et al., 2018). Surprisingly, according to a recent meta-analysis of TMS studies on SUD, only 7 studies used brain activity (e. g. fMRI, EEG, fNIRS) as an outcome measure (Ekhtiari et al., 2019). More so, there are currently no clinically useful biomarkers for SUD or TMS efficacy in the domain of reward-related dysfunction (Ekhtiari et al., 2019; Stein et al., 2018). Without such biomarkers, it is impossible to predict an individual’s vulnerability to SUDs, the severity of an individual’s level of dependence, and TMS treatment effectiveness. In fact, event-related EEG activities are experiencing a resurgence in popularity as promising biomarkers for assessing canonical neural operations that underlie emergent psychological operations, and for revealing neurocognitive processes that can be altered in clinical disorders (Cavanagh, 2019). Because the reward positivity has been shown to be a highly sensitive biomarker of aMCC reward sensitivity (Proudfit, 2015; Holroyd and Umemoto, 2016), SUD severity (Baker et al., 2016b) and TMS efficacy (Baker et al., 2017; Baker et al., 2020a), this ERP component should be considered as a candidate surrogate clinical end-point, where it could be rapidly applied and efficiently used to screen SUD patients and evaluate the impact of future TMS treatments for SUD. Because TMS has been shown to produce long-term potentiation following several sessions (Sandrini et al., 2011; Paus and Barrett, 2004), it is tempting to believe that modulating the aMCC’s putative function in one session could extend beyond the laboratory if the exact same target can be consistently targeted across multiple sessions. Thus, understanding the long-term effects of robot-assisted TMS on the aMCC is critical if this method is to be effectively implemented in clinical research.

Our second aim of this study was to use a decision-making task (PST) to investigate the impact of 10-Hz rTMS to the left DLPFC on approach and avoidance learning. As predicted, individuals in the Active TMS group performed relatively better at approach learning compared to the SHAM group. In the interpretation of this result, we should keep in mind that while the effect size for this contrast was large by convention, the anticipated interaction between PST condition and Group failed to reach significance (see discussion below). Nevertheless, it deserves some further consideration since it constitutes a novel finding and has potential clinical implications. It has been proposed that phasic bursts of dopamine activity (positive RPEs) facilitate approach learning by reinforcing dorsal striatal connections that express D1 receptors (Cohen and Frank, 2009; Frank et al., 2004). By extension, as rTMS to the DLPFC can induce dopamine release in the dorsal striatum (Strafella et al., 2001), it is reasonable to assume that the enhancement of approach learning in the Active TMS group was mediated by dopaminergic mechanisms. In fact, an fMRI study demonstrated that left DLPFC stimulation (i.e., theta burst stimulation) led to a significant bias toward approach learning in the PST, and enhanced RPE coding in the striatum (Ott et al., 2011). Moreover, low-dose D2 receptor antagonism (presumed to increase striatal dopamine) (Jocham et al., 2009; Jocham et al., 2011; Jocham et al., 2014), medication status in Parkinson’s disease (Wiecki and Frank, 2010; Frank et al., 2004), and cigarette consumption (Baker et al., 2020b) have also been shown to enhance positive RPE signaling, and subsequently approach learning. Further, the observed correlation between approach learning and reward positivity amplitude (block 4) further suggests an increased sensitivity to reinforcement, which has been previously shown to enhance PST performance in depression patients (Chase et al., 2010). However, given the exploratory nature of this analysis and the small sample size, this correlation should be replicated in a larger sample. Taken together, we propose that rTMS to the left DLPFC enhanced positive RPE coding in the striatum, and as a functional consequence, enhanced approach learning in the Active group. Because decision-making is consistently and severely affected in substance-dependent individuals (Redish et al., 2008), these initial findings underscore such a need for continued research of the utility of TMS in the treatment of decision-making deficits in this population.

Although this research presents some of the first data regarding the effect of TMS on reward-related neural outcomes in SUD, future research may address some of the study’s limitations. First, a key limitation was the sample size. While small, the current sample size was sufficient for a preclinical trial (Phase 0 or 1), consistent with other noninvasive brain stimulation studies (Ekhtiari et al., 2019), and was adequately powered to detect a moderate-to-large effect size (see sensitivity analysis). Nonetheless, future research should replicate this work using a larger sample of SUD individuals, and healthy controls for the purpose of comparison. Second, it is unclear why the reward positivity did not change in amplitude following the first block of TMS pulses, and further, the overall amplitude of the reward positivity in the later post-TMS blocks was still relatively small compared to healthy controls using the same task (Baker and Holroyd, 2009; Baker et al., 2011, Baker et al., 2016b). In regards to the former, this result likely suggests that the initial TMS block (Block 2: 500 pulses) was not sufficient enough to modulate aMCC reward activity, which is not surprising given that the duration of rTMS is presumed to cause more of an effect by virtue of temporal summation of the effects of the stimulation (Sandrini et al., 2011). In regards to latter, conventional targeting approach used in the current study (scalp-landmark) frequently miss the DLPFC (Sandrini et al., 2011; Fox et al., 2012). The DLPFC spans a large anatomical region (Brodmann’s areas [BA] 8, 9 and 46) and is highly variable across individuals (Petrides et al., 2002, Petrides et al., 2012), thus simply placing the TMS coil on a DLPFC-based scalp landmark would be less than ideal for precision medicine. These considerations highlight the need to optimize targeting techniques for DLPFC-aMCC precision (e.g., individualized targets based on DLPFC structure, function, or connectivity with aMCC) in order to increase the efficacy of TMS to normalize aMCC activity in SUD (Baker et al., 2017; Baker et al., 2020a). Third, given the stark difference in tactile sensation between sham and active TMS, future studies should consider adopting a double-blind procedure (e.g. a placebo coil) to reduce participant and experimenter bias.

Finally, we note that the effects of rTMS on PST behavior were inconsistent with the predicted Group by Condition interaction. Previous work support the idea that disruption in RPE signaling in the striatum can selectively impair approach and avoidance performance (Maia and Frank, 2011). For example, Parkinson’s patients on medication have sufficient dopamine to learn from positive feedback (positive RPEs), but are impaired at learning from negative feedback because the medication blocks the effects of normal dopamine dips (negative RPEs) (Frank et al., 2004; Wiecki and Frank, 2010). Because rTMS can enhance striatal dopamine levels, we expected that individuals in the Active group, relative to the SHAM group, would perform better at approach learning but worse at avoidance learning. However, while the Active group were more accurate at approach learning, both groups performed poorly at avoidance learning, which likely contributed to the absence of the predicted interaction. One possible explanation is the deficit in avoidance learning observed in substance dependent individuals may play out more strongly via an alternative mechanism (Baker et al., 2013). Another explanation could be the time-lag between rTMS and the PST. While the time-course of “off-line” prefrontal rTMS effects on dopamine signaling is unknown, a PET study found that cerebral blood flow in the prefrontal cortex returned to baseline within 9 min (i.e., 60% the stimulation duration) after a 15-minute rTMS protocol (Eisenegger et al., 2008). Accordingly, our 17.5-minute protocol would have had an estimated after-effect of 10 min, thus the variability in PST start time may have reduced the potency of its dopaminergic effects, and thereby its effects on reinforcement learning. Thus, future investigations should extend these findings by adopting a within-subjects design (SHAM and rTMS on separate sessions), and to deliver rTMS on-line rather than off-line during PST performance. Lastly, adopting a computational modeling approach using a large sample (e.g. Q-learning; Burnside et al., 2019; Fischer and Ullsperger, 2013; Baker et al., 2020b) may provide a better quantitative and precise understanding of the effects of TMS on decision-making deficits in SUD that would otherwise be missed by standard behavioral measures.

In conclusion, our findings indicate that 10-Hz rTMS to the left DLPFC potentiated dopamine-dependent RPE signals to rewarding outcomes, thereby recovering the reward function of the aMCC (as revealed by the reward positivity) and the decision-making function of the striatum (as revealed by approach learning in the PST) in problematic substance users. Taken together, these results suggest that rTMS could be a potent brain-based intervention if systematically managed TMS sessions could reliably modify RPE processes in the aMCC and striatum over time, and perhaps result in long-lasting changes in behavior in SUD individuals. For instance, if long-term interventions (multiple sessions) require brain system changes that last hours, days, and (ideally) longer, then any possible indication that TMS modulated reward processes in a single session likely possess high clinical value. Future longitudinal experiments could potentially answer this question.