The Role of the Dorsal–Lateral Prefrontal Cortex in Reward Sensitivity During Approach–Avoidance Conflict

Abstract

Approach–Avoidance conflict (AAC) arises from decisions with embedded positive and negative outcomes, such that approaching leads to reward and punishment and avoiding to neither. Despite its importance, the field lacks a mechanistic understanding of which regions are driving avoidance behavior during conflict. In the current task, we utilized transcranial magnetic stimulation (TMS) and drift-diffusion modeling to investigate the role of one of the most prominent regions relevant to AAC—the dorsolateral prefrontal cortex (dlPFC). The first experiment uses in-task disruption to examine the right dlPFC’s (r-dlPFC) causal role in avoidance behavior. The second uses single TMS pulses to probe the excitability of the r-dlPFC, and downstream cortical activations, during avoidance behavior. Disrupting r-dlPFC during conflict decision-making reduced reward sensitivity. Further, r-dlPFC was engaged with a network of regions within the lateral and medial prefrontal, cingulate, and temporal cortices that associate with behavior during conflict. Together, these studies use TMS to demonstrate a role for the dlPFC in reward sensitivity during conflict and elucidate the r-dlPFC’s network of cortical regions associated with avoidance behavior. By identifying r-dlPFC’s mechanistic role in AAC behavior, contextualized within its conflict-specific downstream neural connectivity, we advance dlPFC as a potential neural target for psychiatric therapeutics.

Introduction

Humans constantly engage in decision-making throughout daily life. Many of these decisions are motivated by the drive to “avoid” negative outcomes (e.g., stop at the red light to avoid a traffic accident) or “approach” positive outcomes (e.g., arrive early to work to prepare for a meeting). However, individuals often encounter circumstances where a particular decision has embedded positive and negative outcomes (e.g., stopping for a red light to avoid a traffic accident and being late to work), which creates a conflict between approach and avoidance motivation (approach–avoidance conflict [AAC]) (Champion 1961; Elliot and Thrash 2002). While the general population maintains a balance in their AAC decision-making, an imbalance between approaching and avoiding such conflict decisions is a hallmark of a number of psychiatric disorders (Gray 1991). For instance, anxiety disorders (Muris et al. 2001); (Aupperle and Paulus 2010), posttraumatic stress disorder (Casada and Roache 2005), depression (Kasch et al. 2002), and obsessive compulsive disorder (Figee et al. 2011) have all been characterized by enhanced avoidance-motivated behavior, shown to associate with anxiety and neuroticism (Gray and McNaughton 2000; Corr 2004). In contrast, disproportionate approach-motivated behavior is associated with impulsivity and extraversion (Gray 1991) and a hallmark feature of addiction (Bijttebier et al. 2009), attention deficit hyperactivity disorder (Mitchell and Nelson-Gray 2006), bipolar disorder (Chandler et al. 2009), and psychopathy (Newman et al. 2005). Imbalances in motivated decision-making can potentiate cyclic patterns of behavior leading to sustained, and even more severe, disordered symptomatology (Mogg and Bradley 2005). Accordingly, elucidating the neural circuitry that drives variability in AAC behavior would advance our understanding of the network-level patterns that underlie symptomatic imbalances in motivated behavior and provide critical insight into potential biomarkers for treatment.

A brain region thought to be important for AAC decision-making is the dorsolateral prefrontal cortex (dlPFC) (Kirlic et al. 2017). The dlPFC has been heavily implicated in executive function, specifically cognitive control and decision-making (Yuan and Raz 2014). The dlPFC has also been well established in emotion-regulating processes (Banks et al. 2007; Golkar et al. 2012; Etkin et al. 2015), and it has been proposed this region acts to indirectly inhibit the amygdala through its anatomical projections (Ray and Zald 2012). The dlPFC has also been found relevant to AAC, where the dlPFC has been found to activate specific to conflict processing, the extent to which positively associates with the amount of reward responding (Aupperle and Paulus 2010; Ballard et al. 2011; Spielberg et al. 2013; Aupperle et al. 2015; Chrysikou et al. 2017). However, in the absence of causal interrogation, the direct role the dlPFC in regulating emotional processes in humans, specifically AAC, remains unclear.

In order to understand the causal role of the prefrontal cortex (PFC), an experimental approach is needed where its activity can be directly manipulated during AAC. Transcranial magnetic stimulation (TMS) provides a noninvasive tool for the causal investigation of cortical regions in various cognitive processes (Pascual-Leone et al. 2000; Walsh and Cowey 2000; Calvo-Merino and Haggard 2004; Sandrini et al. 2011). TMS activates targeted brain regions through the pulsing of a strong electromagnet over that region, which induces an electric field in the underlying brain tissue and thus depolarizes neurons (Siebner et al. 2009). While TMS has been used extensively in the last decade to investigate the role of cortical regions in motor and cognitive processes (Chail et al. 2018), it has yet to be used in the context of AAC behavior. TMS has great potential, when applied in the context of task performance, to elucidate mechanisms underlying the roles of key brain regions in approach–avoidance behavior, thus moving the field beyond correlational measures common to neuroimaging approaches.

TMS can furthermore be utilized in various ways for investigating the role of neural regions within a task. Delivering suprathreshold (120% of resting motor threshold [rMT]) TMS pulses during performance of a task will overwhelm precise local signaling (Harris et al. 2008) with unstructured noise (Schwarzkopf et al. 2011) due to activation of large numbers of neurons by the magnetic pulse. This is often referred to as “online disruption” (Bolognini and Ro 2010) and is a powerful tool to investigate the causal role of a particular region in a distinct phase of information processing. Importantly, online TMS disruption allows the perturbation of cortical activity during specific phases of processing within a task, without interfering with other phases of processing, such as motor responses (Bolognini and Ro 2010). Whereas online disruption utilizes multiple suprathreshold TMS pulses, “online probe” methods utilize a single subthreshold (80% rMT) pulse of TMS during a task, in combination with concurrent neuroimaging techniques such as electroencephalography (TMS-EEG), to evaluate the local and downstream cortical excitability during a distinct phase of processing, without perturbing the target’s function (Pascual-Leone et al. 2000; Walsh and Cowey 2000; Calvo-Merino and Haggard 2004; Sandrini et al. 2011; Chail et al. 2018). In this way, TMS-EEG can be used as an “excitability probe,” with the EEG output reflecting the excitability of the underlying cortex at the time the TMS pulse was delivered. Task-locked single TMS pulses are often delivered at subthreshold intensities since pulse intensities low as 60% rMT produce reliable EEG-evoked responses (Kahkonen et al. 2005; Komssi et al. 2007; Saari et al. 2018), comparable global signal information (Komssi et al. 2004) and attenuated pain (Holmes and Meteyard 2018; Meteyard and Holmes 2018), distraction (Robertson et al. 2003), and muscle artifact (Komssi et al. 2004) as compared with suprathreshold intensities.

The current study was designed to interrogate the role of the dlPFC in AAC through 2 specific aims: 1) investigate the causal role of the dlPFC in AAC through examining avoidance behavior following dlPFC online disruption TMS and 2) map the dlPFC connectivity patterns that differentiate choice behavior during AAC through examining TMS-evoked cortical spectral response following dlPFC online probe TMS. Given the known role of the dlPFC in decision-processing in the context of AAC (McNaughton et al. 2016; Kirlic et al. 2017), we hypothesized that disrupting the dlPFC during decision-processing would cause greater avoidance behavior as compared with both ventrolateral prefrontal cortex (vlPFC) and a vertex control disruption. We further hypothesized that dlPFC would exhibit greater excitability, as indexed by the evoked spectral TMS response, and that this excitability would differ as a function of decision choice (approach vs. avoid) at the trial level. Given the known functional connectivity between the dlPFC and other conflict-provoked medial PFC regions—particularly the cingulate and orbitofrontal cortex (OFC) (Zorowitz et al. 2019), we anticipated TMS-evoked spectral response within these regions to associate with conflict-specific behavior.

Materials and Methods

Participants

Participants were recruited and randomized into one of 2 studies: Disrupt or Probe (Fig. 1). Twenty-five healthy adults (12 female; mean age = 29.62 ± 10.27 years) participated in the Disrupt experiment and 19 healthy adults (10 female; mean age = 33.47 ± 8.90 years) in the Probe experiment. Written informed consent was obtained from each participant under Institutional Review Board-approved protocols at Stanford University.

Figure 1 .

Experimental design. (a) Participants were recruited and randomized to one of 2 studies—Disrupt (N = 25) or Probe (N = 19). (b) Visualization of hypothetical targeted brain regions: r-dlPFC (blue), r-vlPFC (yellow), vertex (blue).

Task Design

The task, modeled after a task originating from primate and rodent work (Amemori and Graybiel 2012; Amemori et al. 2015; Amemori et al. 2018, 2020; Ironside et al. 2020), is designed to assay decision-making during conflict (Fig. 2). AAC is created by pairing each trial with a reward and punishment of varying intensities and asking participants to approach (and receive both punishment and reward) or avoid (and receive neither punishment nor reward) each trial. The task begins with a fixation (0.5–10 s), in which participants are instructed to fixate on the center circle. Following this fixation, the target (4 s) is presented, which offers the participant 2 options (approach or avoid) in the face of a given combination of reward and punishment values. Participants select their choice using one of keyboard buttons corresponding to their selected choice. The punishment is indicated by the size of the red bar, scaled from 1 to 5 in punishment intensity, and corresponds to the normative negative valence rating for a picture from the International Affective Picture Series (Lang et al. 1999). The reward is indicated by the size of the purple bar, scaled from 1 to 5 in reward intensity, and corresponds to the number of points given. If the participant choses to approach by choosing the “plus” object, they will receive a punishment (i.e., presentation of picture) and reward (i.e., gain of points). If they choose the “square” object, they will receive neither a punishment (i.e., presentation of picture) nor reward (i.e., gain of points). The length of the 2 bars was parametrically varied (for further details, see Ironside et al. 2020).

Figure 2 .

Approach–Avoidance task design: Probe TMS consisted of a single pulse, delivered at 80% rMT at the onset of the decision screen. Disrupt TMS consisted of 5 single pulses, delivered at 10 Hz and 120% rMT at the onset of the decision screen.

During the task, time-locked to the target onset (see Fig. 1), TMS was delivered to investigate 1) whether disrupting the function of targeted region during decision-processing altered behavior (Disrupt experiment) and 2) the cortical excitability and downstream connectivity of the targeted region during the decision-processing phase of AAC.

TMS Targeting

For either experiment, the target regions included the right dlPFC (r-dlPFC), right vlPFC (r-vlPFC), and a vertex control. The r-dlPFC, the primary region of investigation for both studies, was chosen due to its corroborated role in approach–avoidance decision-making (Aupperle and Paulus 2010; Spielberg et al. 2013; Aupperle et al. 2015; Chrysikou et al. 2017). The vlPFC, a region anatomically proximal but functionally distinct (Arsalidou et al. 2013; Clarke et al. 2015; He et al. 2018; Kim et al. 2019) from the dlPFC, was chosen as the functional control. The vertex location, with no known relevance to emotion regulation or AAC, was chosen as an anatomical control for both studies. The Disrupt experiment was designed to address whether disrupting the function of targeted region during decision-processing altered behavior, and if this alteration was specific to the dlPFC. The Probe experiment was designed to assess the cortical excitability and downstream connectivity of the targeted region during the decision-processing phase of AAC, and how this relates to in-task behavior at the trial level.

Following an anatomical magnetic resonance imaging (MRI; T₁-weighted, 3 T) to determine MRI-guided TMS targets, subjects received TMS using a Cool-B B65 butterfly coil and a MagPro X100 TMS stimulator (MagVenture). Stimulations were delivered to 2 (Probe) or 3 (Disrupt) cortical targets, including right posterior dorsolateral prefrontal cortices (right posterior midfrontal gyrus; r-dlPFC), right ventral lateral prefrontal cortices (right inferior frontal gyrus; r-vlPFC), and vertex. Given some studies have suggested laterality in the dlPFC, we chose to target only the right hemisphere prefrontal targets (Pizzagalli et al. 2005; Berkman and Lieberman 2010). For r-dlPFC and r-vlPFC, the stimulation sites were identified as the peak coordinates in clusters derived from brain networks parcellated from a separate group of subjects’ resting-state functional magnetic resonance imaging data (Sridharan et al. 2008; Shen et al. 2013) using independent component analyses (ICA). These targets were then transformed to individual subject native space using nonlinear spatial normalization with FSL (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki) and used for TMS targeting. The vertex stimulation site was targeted using the digitized location of the central electrode (Cz).

The rMT was determined as the minimum stimulation intensity that produced visible finger movement of the right hand at least 50% of the times when the subject’s left M1 is stimulated. TMS coil placement was guided by Visor2 LT 3D neuronavigation system (ANT Neuro) based on coregistration of the functionally defined target to each participant’s structural MRI (T₁ weighted, slice distance 1 mm, slice thickness 1 mm, sagittal orientation, acquisition matrix 256 3256) acquired with a 3 T GE DISCOVERY MR750 scanner. The TMS coil was placed tangentially to the scalp with the handle pointing backward and laterally at an angle of 45° to the sagittal plane.

Disrupt Experiment

To assess the cortical excitability and downstream connectivity of the targeted region during the decision-processing phase of AAC, online probe TMS was used by applying a single subthreshold (80% rMT) pulse of TMS during a task, in combination with concurrent EEG (TMS-EEG), to evaluate the local and downstream cortical excitability during a distinct phase of processing. For each trial, at the onset of the target, 5 suprathreshold TMS pulse were delivered at 120% intensity for 500 ms (10 Hz) (Chick et al. 2020). TMS stimulation site was randomized across blocks within participant, with 35 trials per stimulation site condition.

Probe Experiment

To address whether disrupting the function of targeted region during decision-processing altered behavior, online disruption TMS was used by delivering suprathreshold (120% of rMT) TMS pulses during the AAC task. For each trial, at the onset of the target, 1 subthreshold TMS pulse was delivered at 80% intensity. TMS stimulation only during the target presentation phase allows the perturbation of cortical activity of the target only during the processing of the decision, without interfering with motor response or feedback processing. TMS stimulation site was randomized across blocks within participant, with 35 trials per stimulation site condition.

E‌EG Data Acquisition

EEG recordings were acquired with a BrainAmp DC amplifier (sampling rate: 5 kHz; measurement range: ± 16.384 mV; cut-off frequencies of the analog high-pass and low-pass filters: 0 and 1 kHz) and the Easy EEG cap with 64 extra-flat, freely rotatable, sintered Ag–AgCl electrodes (Brain Products GmbH). The electrode montage followed an equidistant arrangement extending from below the cheekbone back to below the inion. Electrode locations were digitized for each participant within the Visor2 LT 3D neuronavigation system (ANT Neuro). Electrode impedances were kept below 5 kΩ. An electrode attached to the tip of the nose was used as the reference. Participants were seated on a comfortable reclining chair.

Cap positioning was standardized for each participant by positioning the central electrode (Cz) of the 64-channel cap over the intersection of the midway point between the nasion to inion and the left to right tragus.

E‌EG Data Processing

All EEG data analyses were performed in MATLAB (R2014b, The Mathworks Inc.) using custom scripts built upon the EEGLAB (Delorme and Makeig 2004) and ARTIST (Wu et al. 2018) toolboxes.

The recorded single-pulse TMS (spTMS)-EEG data were cleaned offline with ARTIST, which is a fully automated artifact rejection algorithm for spTMS-EEG (Wu et al. 2018). The following steps were implemented: 1) The initial 10-ms data segment following TMS pulses were discarded to remove the large stimulation-induced electric artifact. 2) The EEG data were downsampled to 1 kHz. 3) Big decay artifacts were automatically removed using ICA based on thresholding. 4) The 60 Hz AC line noise artifact was removed by a notch filter. 5) Nonphysiological slow drifts in the EEG recordings were removed using a 0.01 Hz high-pass filter, and high-frequency noise was removed by using a 100 Hz low-pass filter. 6) The spectrally filtered EEG data were then re-referenced to the common average and epoched with respect to the TMS pulse (−500 ~ 1500 ms). 7) Bad trials were rejected by thresholding the magnitude of each trial. Trials with signal amplitudes outside 3 standard deviations (SDs) ± the trial average within each block were excluded. Corrupted channels were rejected based on the spatial correlations among channels. The rejected bad channels were then interpolated from the EEG of adjacent channels. 8) Remaining artifacts were automatically removed using ICA. Independent components (ICs) related to the scalp muscle artifact, ocular artifact, ECG artifact were rejected using a pattern classifier trained on expert-labeled ICs from other TMS/EEG data sets.

Following preprocessing, source localization of the data was performed in MATLAB using custom code for nonparametric minimum norm estimates (Hamalainen and Ilmoniemi 1994). Source localization to the cortical surface was performed using custom script. A 3-layer symmetric boundary element model of the head was computed based on a Montreal Neurological Institute (MNI) brain template (regularization parameter = 0.1, depth-weighting component = 0.5, noise-covariance estimation procedure = identity matrix). Rotating dipoles at 3003 vertices were generated on the cortical surface. The lead-field matrix was obtained by projecting the standard electrode positions onto the scalp. For each subject, an imaging kernel that maps from the channel space EEG to the source space current density was then estimated by the minimum norm estimation approach with depth weighting and regularization. For each vertex, the current density time series were reduced from their 3 orthogonal axes to a single principal direction by principal component analyses (PCA), then bandpass filtered into canonical frequency bands associated with different neurophysiological processes. The current density time series at each canonical frequency band were Hilbert transformed to yield source space analytical time series.

E‌EG Spectral Analyses

All spectral analyses were computed at the vertex level using 3003 vertices in MNI template space. Data underwent Morlet wavelet time frequency transformation, and then the power was averaged within the 2 canonical frequencies: theta (4–7.5 Hz) and alpha (8–12 Hz) frequency bands. Finally, power estimates for each individual were normalized across the 3003 vertices, using a Z-transformation statistic (Tong and Thakor 2009). We specifically focused on alpha and theta narrow spectral bands due to the abundant evidence for their relevance to AAC above and beyond higher-frequency bands (Neo and McNaughton 2011; Neo et al. 2011; McNaughton et al. 2013; Kelley et al. 2017; Neal and Gable 2017; De Pascalis et al. 2018; Neo et al. 2020).

Statistical Analyses

Disrupt Experiment: Drift-Diffusion Model

To gain insight into the mechanistic alterations of disruption through their influence on choice and the response time distribution of choices, we modeled AAC decisions with a drift-diffusion model (DDM) (Ratcliff 1978) a computational model describing the dynamics of decision processes. To examine the behavioral effect of cortical disruption of our key regions of interest (ROI) (r-dlPFC, r-vlPFC, vertex), we used a DDM (Ratcliff and McKoon 2008). The DDM is a cognitive model describing the process of 2 alternative forced choice decision-making (Fig. 3). The DDM captures choice and reaction time through (at least) 4 parameters describing separable underlying mechanisms of a decision. The drift rate describes the rate of evidence accumulation toward 1 of 2 decision boundaries. At each time step evidence is drawn from a normal distribution with a mean reflecting the drift rate and SD reflecting noise in the accumulation process. High drift rates lead to fast and deterministic choices, whereas low drift rates lead to slower and more random choices (due to stronger relative influence of noise). The distance between decision boundaries is described by the boundary separation (sometimes called decision threshold) parameter, which captures the speed-accuracy tradeoff. Wide boundaries will lead to slow and accurate choices, whereas narrow boundaries will lead to fast and less accurate choices as noise can have a stronger influence on which boundary is reached. The nondecision time captures the time spent on stimulus perception and motor preparation and thus only affects reaction time. Lastly, the starting point parameter describes potential bias toward one of the decision alternatives and will result in faster response times (RTs) and a higher proportion choice for the preferred option. The DDM has proven successful in accounting for choice and reaction time on perceptual and memory tasks (Ratcliff 1978; Gold and Shadlen 2007) and has more recently been applied to value- and reinforcement-based decision-making (Basten et al. 2010; Hare et al. 2011; Pedersen et al. 2017).

Figure 3 .

Illustration of an approach–avoidance decision as modeled by the DDM. The DDM assumes evidence is accumulated toward decision boundaries to approach (upper decision boundary) or avoid (lower decision boundary) an offer of reward and aversiveness. In the current model drift rate on each trial was modeled as the combination of offered reward and aversiveness. The boundary separation parameter describes the distance between decision boundaries, the starting point parameter describes the initial bias in favor of options and the nondecision time (NDT) describes time spent on stimulus encoding and motor response.

Approach-Avoidance DDM

Here we model approach–avoidance decision-making, according to the method described by Pedersen et al. (2021), by modeling approach and avoid responses as evidence accumulation processes terminating at upper and lower decision boundaries, respectively. Such an approach allows insight into the cognitive mechanisms underlying decision-making under AAC by capturing both the choice and response time distributions of decisions. In the context of AAC, in which there is no accurate response, the boundary separation parameter reflects the consistency of choices, that is, how consistently a decision maker would give the same response for the same offers of reward and aversiveness. In order to distinguish the influence of offered reward and aversiveness on decisions (and their response time), we captured trial-by-trial drift rates, that is, the rate of evidence accumulation, as the combined influence of offered reward and aversiveness:

Where reward_t and aversiveness_t were the offered reward and aversiveness on trial t, respectively, and positive (negative) drift rate on trial t would on average lead to approach (avoid) decisions. Such a model allowed us to estimate sensitivity to offered reward and aversiveness in how they influenced the drift rate, for example, the reward sensitivity onto drift rate as reflected in . For the remaining parameters (boundary separation, nondecision time and bias) a single coefficient was estimated, resulting in the following model used to capture observed choice and its reaction time with the wiener first passage time (wfpt) likelihood function of the DDM:

All parameters were estimated separately for conditions for each subject.

D‌DM Analyses

We analyzed data with the DDM using the open-source python software package HDDM (Wiecki et al. 2013). The HDDM estimates parameters hierarchically in a Bayesian framework allowing for robust estimation of subject and group parameters (Kruschke et al. 2012) and uses Markov-Chain Monte-Carlo sampling to identify the range of most likely parameters, given the data and prior probability. Bayesian hierarchical estimation has been shown to recover the parameters of the DDM better than other methods of analysis (Wiecki et al. 2013). We used weakly informed prior parameter values identified from previous reports in the literature (Wiecki et al. 2013). The model estimated the within-effect of condition on all parameters to capture the effect of disruption of each ROI.

Previous work utilizing the DDM to model AAC (Pedersen et al. 2021) has shown that transformation of the predictors can improve model fit. Therefore, we ran all permutations of models allowing offered reward and aversiveness to be log-transformed or not, based on the assumption that the impact of offers might be nonlinear. All models (Supplementary Table S5) were run with 5 chains of 5000 samples each, of which the first 2500 were discarded as burn-in. Model fit was measured using the deviance information criterion (Spiegelhalter et al. 2002). The best-fitting model used a log-transformation of reward. Maximum rhat was 1.09, indicating that the chains for all parameters converged to similar estimates (Gelman and Rubin 1992). Posterior predictive checks also indicated that the model captured both response and RT patterns across conditions and subjects (Supplementary Figs S1 and S2).

Probe Experiment: Linear Mixed-Effects Modeling

To examine the relationship between approach–avoidance behavior and cortical excitability of our key regions (r-dlPFC, vertex), linear mixed-effects (LME) models were applied to each vertex band power estimate (Lindstrom and Bates 1990). Analyses for the probe experiment were restricted to the r-dlPFC and vertex target ROIs due to the large muscle artifact contamination of the vlPFC TMS-evoked potential. Analyses were conducted using the NLME package in R (Pinheiro et al. 2007). Two models were run to examine 1) how behavior varies with EEG estimates, specific to r-dlPFC stimulation condition and 2) how behavior varies with EEG estimates, specific to r-dlPFC stimulation condition and conflict. The outcome variable for both models was the 3003 vertex-level EEG spectral estimates for alpha and theta bands, across 4500 ms time bins following time-locked to the online TMS probe.

The first model aimed to examine how behavior varies with EEG estimates, specific to r-dlPFC stimulation condition. Modeled effects included a random intercept and fixed effect of stimulation condition (r-dlPFC, vertex) and trial-level behavior [approach (1), avoid (0)]. These effects were modeled by an interaction with the source-level spectral estimates (EEG) examined to identify which EEG estimates were associated with behavior, varied by stimulation condition. The primary term of interest is the behavior × stimulation condition interaction, which assessed trial-level variability in EEG spectral band as a function of behavior and stimulation condition.

The second aimed to examine how behavior varies with EEG estimates and conflict, specific to r-dlPFC stimulation condition. Modeled effects included a random intercept and fixed effect of stimulation condition (r-dlPFC, vertex), trial-level behavior [approach (1), avoid (0)] and conflict (−5 to 5). Conflict intensity was calculated to reflect the net value on a given trial, indexed as the sum of punishment and reward intensities (conflict = −pun + rew), both of which ranged from 0 to 5. Therefore, 0 represented the least conflict, and the further from 0, the less conflict—biasing either toward reward (5) or punishment (−5). These effects were modeled by an interaction with the source-level spectral estimates (EEG) examined to identify which EEG estimates were associated with behavior, varied by both level of conflict and stimulation condition. The primary term of interest is the behavior × stimulation condition × conflict intensity interaction, which assessed trial-level variability in EEG spectral band as a function of behavior, stimulation condition, and conflict intensity.

All P values were false discovery rate (FDR)-corrected for multiple comparisons across all vertices, time bins, and frequency bands (3003 vertices × 2 bands × 4 time bins) to control for type I errors.

Vertices found to associate with behavior differentially depending on 1) stimulation condition and 2) stimulation condition and conflict intensity were then grouped into clusters, with boundaries defined by spatial contiguity. Those clusters whose maximal voxel-to-voxel Euclidian distance was >5% of the total Euclidian distance across the cortex were divided into spatially defined subclusters using k-means. The number of subclusters were defined as the ratio of the cluster maximal voxel-to-voxel Euclidian distance to the squared cortical maximal voxel-to-voxel Euclidian distance. Finally, the vertex with peak activation was extracted from each cluster as representative of the greater cluster and the Brodmann area (Brodmann 1909) within which it fell was identified.

Results

Disrupt Experiment: DDM

We used the DDM to model choice and reaction time as an accumulation-to-bound process where amount of offered reward and aversiveness influence evidence accumulation, captured by the drift rate parameter, toward choosing to approach or avoid offers. This framework allows insight into the cognitive mechanisms underlying approach–avoidance decision-making by estimating changes in decision parameters and the effect of reward and aversiveness onto decision parameters. Data were analyzed with the HDDM package (Wiecki et al. 2013), which uses Bayesian hierarchical estimation to simultaneously capture group and individual parameters. We tested several variations of models in an attempt to best describe observed data and report results from the best-fitting model identified by comparing model fit (see Materials and Methods). Effects of condition across parameters are reported as the probability that 2 conditions are different, given the posterior distribution.

The results of the best-fitting model indicated that disruption of r-dlPFC, compared with vertex, resulted in decreased reward sensitivity onto drift rate (vertex = 0.81, r-dlPFC = 0.64, P_{vertex > r-dlPFC} = 0.97) and a reduced boundary separation (vertex = 2.39, r-dlPFC = 2.08, P_{vertex > r-r-dlPFC} = 0.99). These parameters were also reduced in r-dlPFC compared with r-vlPFC-disruption, but to a lesser extent for reward sensitivity onto drift rate (reward: r-vlPFC = 0.74, r-dlPFC = 0.64, P_{r-vlPFC > r-dlPFC} = 0.91, boundary separation: r-vlPFC = 2.25, r-dlPFC = 2.11, P_{r-vlPFC > r-dlPFC} = 0.98). None of the other parameters differed across conditions (Supplementary Fig. S3 and Supplementary Table S6).

Probe Experiment: LME Modeling

For each LME effect (stimulation condition × behavior; stimulation condition × behavior × conflict intensity), we first report on the significant EEG clusters that associate with choice behavior at the trial-level, differentially by 1) stimulation condition and 2) stimulation condition and conflict intensity. We then run the analyses again, but separately for each stimulation condition to elucidate the EEG clusters that were unique to r-dlPFC TMS.

For the first model, a significant effect of stimulation condition × behavior for a given EEG estimate suggests that the trial-level power at that vertex significantly associates with trial-level behavior (e.g., more theta power at vertex B associates with trials where the subject avoided), but that this association is not consistent between stimulation conditions (e.g., more r-dlPFC TMS-evoked theta power at vertex B associates with trials where the subject avoided, whereas more vertex-evoked theta power at vertex B does not associate with trials were the subject avoided). However, the directionality of this interaction effect is difficult to interpret as it specifically relates to stimulation condition. When this model is run separately for each stimulation condition, main effects of behavior are identified for any EEG vertex whose power associates with behavior and directionality of this effect can be evaluated for each stimulation condition.

For the second model, a significant effect of stimulation condition × behavior × conflict intensity for a given EEG estimate suggests that the trial-level power at the vertex significantly associates with trial-level behavior (e.g., more theta power at vertex B associates with trials where the subject avoided, but only during high conflict), but that this association is not consistent between conflict intensities nor stimulation conditions (e.g., more r-dlPFC TMS-evoked theta power at vertex B associates with trials where the subject avoided, whereas more vertex-evoked theta power at vertex B does not associate with trials were the subject avoided). However, the directionality of this interaction effect is difficult to interpret as it specifically relates to stimulation condition and conflict intensity. When this model is run separately for each stimulation condition, interaction effects of behavior × conflict intensity are identified for any EEG vertex whose power associates with behavior and conflict intensity, and directionality of this effect can be evaluated for each stimulation condition.

Stimulation Condition × Behavior

To identify whether choice behavior during conflict was associated with spectral EEG differentially across stimulation conditions, we examined the stimulation condition × behavior effect from the LME. After FDR correcting for multiple comparisons across all 24 024 (3003 vertices, 2 spectral bands, 4 time bins) EEG features assessed, 117 distinct clusters were found to associate with trial-level variability in in-task avoidance behavior differentially by stimulation condition, as measured by choice response (Supplementary Table S1; Fig. 4).

Figure 4 .

Neural features associated with behavior differentially by stimulation condition. Z-score of condition × behavior LME analyses with dlPFC TMS single pulse (a) theta (4–7.5 Hz) and (b) alpha (8–12 Hz) spectral response, with TMS subthreshold pulse occurring at t = 0 s. Only those voxels with FDR-corrected P < 0.05 are plotted.

The prominent clusters that associated with choice behavior differentially by stimulation condition were found within bilateral:

1. Lateral prefrontal—<1 s post-TMS, alpha/theta

2. Ventromedial prefrontal/orbitofrontal—early (<0.5 s post-TMS), alpha/theta

3. Cingulate—early (<0.5 s post-TMS), alpha/theta

4. Temporal—superior: early (<0.5 s post-TMS), theta; medial: late(>0.5 s post-TMS), alpha/theta; pole: late(>0.5 s post-TMS), alpha

5. Sparce clusters in parietal and visual/somatosensory/motor cortices

Cingulate and ventromedial prefrontal/orbitofrontal activation was most pronounced early, peaking between 0 and 0.25 s post-TMS. Superior temporal activation was most pronounced within the theta band, peaking between 0.25 and 0.5 s post-TMS. However, medial temporal activation, prominent in both alpha and theta bands, emerged after 0.5 s, peaking at 0.75–1 s post-TMS. Temporal pole activation was isolated to alpha, peaking at 0.75–1 s post-TMS. Finally, a number of effects were found in the superior parietal lobe, specifically within theta band.

We next sought to identify which of the significant EEG clusters associated with choice behavior from the stimulation condition × behavior effect were specific to the r-dlPFC TMS condition (Supplementary Table S2; Fig. 5). We ran mixed models with EEG and behavior for the 2 stimulation conditions separately, further constraining analyses to only those EEG features identified as significant in the full analysis using a P value threshold of 0.05. As this split analysis was performed to interrogate what was driving the stimulation condition × behavior interaction found in the full analysis, we constrained this split analysis to only those FDR-significant connections identified as significant in the full analysis. As such, we did not again correct for multiple comparisons for this split analyses, which was done for visualization purposes only.

Figure 5 .

Neural features associated with behavior specific to the dlPFC stimulation condition. Z-score of behavior LME analyses with dlPFC TMS single pulse (a) theta (4–7.5 Hz) and (b) alpha (8–12 Hz) spectral response, with TMS subthreshold pulse occurring at t = 0 s. The voxels visualized are masked by those included in full stimulation condition × behavior analyses. Only those voxels with P < 0.05 are plotted.

The prominent clusters that “negatively” associated with avoidance choice behavior specific to r-dlPFC TMS were found in the:

1. dlPFC—< 0.5 s post-TMS, alpha/theta

2. Ventromedial prefrontal/orbitofrontal—early (>0.5 s post-TMS), alpha/theta

3. Cingulate—early (<0.5 s post-TMS), alpha

4. Temporal—early (<0.5 s post-TMS), theta/alpha

Our results found that the TMS-evoked spectral power within the dlPFC was negatively associated with avoidance behavior differentially by stimulation condition, specific to early (<0.5 s post-TMS) theta and alpha. Regions within the temporal cortex (early theta and late alpha), ventromedial prefrontal cortex (vmPFC)/OFC (late theta and alpha), and cingulate (early alpha) were also negatively associated with avoidance.

The prominent clusters that “positively” associated with avoidance choice behavior specific to r-dlPFC TMS were found in the:

1. Left anterior PFC—< 0.5 s post-TMS, alpha/theta

2. Dorsal OFC—early (>0.5 s post-TMS), alpha/theta

3. Temporal—early (>0.5 s post-TMS), theta

4. Parietal—early (>0.5 s post-TMS), theta

Regions found to positively associate with avoidance behavior included early left PFC, a bit more anterior to the dlPFC, and early medial prefrontal cortex (mPFC) (dorsal to OFC), medial temporal cortex (late theta), and superior parietal cortex (early theta).

Stimulation Condition × Behavior × Conflict Intensity

To identify whether choice behavior during conflict was associated with spectral EEG differentially across conflict intensity and stimulation conditions, we examined the stimulation condition × behavior × conflict intensity effect from the LME. After FDR correcting for multiple comparisons across all 24 024 (3003 vertices, 2 spectral bands, 4 time bins) EEG features assessed, 253 distinct clusters were found to associate with trial-level variability in in-task avoidance behavior differentially by task condition and conflict intensity, as measured by choice response (Supplementary Table S3; Fig. 6).

Figure 6 .

Neural features associated with behavior differentially by stimulation condition and conflict intensity. Z-score of condition × behavior × conflict LME analyses with dlPFC TMS single pulse (a) theta and (b) alpha spectral response, with TMS subthreshold pulse occurring at t = 0 s. Only those voxels with FDR-corrected P < 0.05 are plotted.

The prominent clusters that associated with choice behavior × conflict intensity differentially by stimulation condition were found within bilateral:

1. Lateral prefrontal—> 0.5 s post-TMS, alpha

2. Ventromedial prefrontal/orbitofrontal—early (<0.5 s post-TMS), alpha

3. Cingulate—(<1 s post-TMS), alpha/theta

4. Temporal (left)—>0.5 s post-TMS, alpha/theta

5. Sparce clusters in parietal and visual/somatosensory/motor cortices

The majority of these clusters were found within bilateral prefrontal, orbitofrontal, cingulate, and left temporal, with the rest falling within the parietal, visual, and somatosensory/motor cortices. While the majority of effects were broad in both frequency and post-TMS time bin, ventromedial prefrontal/orbitofrontal, and dlPFC activation was most pronounced within the alpha band, peaking between 0.25–0.5 s and 0.5–0.75 s post-TMS, respectively. Temporal activation was most pronounced within the left hemisphere, peaking between 0.5 and 0.75 s post-TMS.

We next sought to identify which of the significant EEG clusters associated with choice behavior and conflict intensity from the stimulation condition × behavior × conflict intensity effect were specific to the r-dlPFC TMS condition (Supplementary Table S4; Fig. 7). We ran mixed models for the 2 stimulation conditions separately, further constraining analyses to only those EEG features identified as significant in the full analysis using a P value threshold of 0.05. Our results found that the TMS-evoked spectral power within a number of key regions was associated with avoidance behavior differentially by stimulation condition and conflict. Patterns of activation were investigated specific to 1) trials in which the participant avoided versus approached the decision and 2) trials offering a range of decision outcomes, from punishment-biased (offered punishment > offered reward) to reward-biased (offered punishment < offered reward) trials.

Figure 7 .

Neural features associated with behavior, differentially by conflict intensity, specific to the dlPFC stimulation condition. Z-score of behavior × conflict LME analyses with dlPFC TMS single pulse (a) theta and (b) alpha spectral response, with TMS subthreshold pulse occurring at t = 0 s. The voxels visualized are masked by those included in full stimulation condition × behavior × conflict analyses. Only those voxels with P < 0.05 are plotted.

The prominent clusters that positively associated with avoidance choice behavior on reward-biased trials, specific to r-dlPFC TMS were found in the:

1. dlPFC—<1 s post-TMS, alpha

2. Ventromedial prefrontal/orbitofrontal—early (>0.5 s post-TMS), alpha

3. Cingulate—<1 s post-TMS, theta

4. Temporal—<1 s post-TMS, theta/alpha

Results found that vmPFC/OFC early (0–0.25 s) alpha, dlPFC alpha, superior temporal, and cingulate theta all showed more power during avoidance with greater reward-bias and less power during approach with greater reward-bias.

The prominent clusters that negatively associated with avoidance choice behavior on reward-biased trials, specific to dlPFC TMS were found in the:

1. Ventromedial prefrontal/orbitofrontal—late (>0.5 s post-TMS), theta

2. Temporal—late (>0.5 s post-TMS), alpha

3. Parietal—late (>0.5 s post-TMS), alpha

vmPFC/OFC late (0.5–0.75 s) theta, superior temporal alpha (0.75–1 s), and superior parietal alpha (0.5–0.75 s) all showed less power during avoidance with greater reward bias and more power during approach with greater reward bias.

Discussion

The current study utilized TMS to accomplish 2 aims in elucidating the neural circuitry underlying AAC. The first experiment combined trial-locked disruptive TMS with DDM modeling of behavior to identify the functional role of the r-dlPFC in AAC decision-making. The second experiment utilized trial-locked subthreshold spTMS with LME modeling of TMS-evoked EEG spectral response to identify how r-dlPFC-probed local and distributed cortical activity interacts with AAC decision-making. Together, the 2 studies used TMS and TMS-EEG to both characterize and alter behavioral patterns during conflict. Our results suggest that the r-dlPFC subserves approaching reward during conflict by 1) enhancing reward sensitivity and 2) coactivating with or signaling a network of regions (including the cingulate, medial/orbital/lateral PFC, temporal, and superior parietal).

r-dlPFC Disruption Blunts Reward Sensitivity During Choice Conflict

By disrupting different regions at the decision point during conflict and capturing the effect of this disruption on choice behavior through modeling the behavioral choices at the trial-level for each stimulation condition, we were able to identify the pattern of behavior uniquely resulting from r-dlPFC disruption. Our first experiment identified a role for the r-dlPFC in reward sensitivity during conflict. Applying disruptive TMS to the r-dlPFC resulted in decreased reward sensitivity, compared with disruptive TMS when applied to the vlPFC or the vertex. The literature has consistently found associations with the dlPFC activation and conflict—however, to our knowledge, this is the first study causally tying the dlPFC’s role in conflict to reward sensitivity. This finding is well supported by the literature in goal-pursuit and motivation, in which the dlPFC has been shown to associate with the representation and integration of goals with reward information (Miller and Cohen 2001; Watanabe and Sakagami 2007; Spielberg et al. 2013; Aupperle et al. 2015) thought to do so via its projections to the ventral striatum and ventral tegmental area (Ballard et al. 2011). Our finding that disrupting the dlPFC at the onset of a conflict decision blunts sensitivity to reward lends critical insight into the functional role the dlPFC plays in AAC decision-making.

dlPFC-Evoked Cortical Response Relates to Conflict Decision-Making

By probing different regions at the decision point during conflict and capturing the spatial, temporal, and spectral characteristics of those regions that associated with whether someone approached or avoided at the trial-level, we were able to characterize the patterns of dlPFC connectivity relevant to choice behavior during varying levels of conflict. Our second experiment identified a network of regions coactivating with the dlPFC specifically when approaching conflict decisions. Applying subthreshold r-dlPFC (compared with vertex) single pulses identified a number of cortical regions whose TMS-evoked spectral response associated with conflict-specific decision-making. We specifically focused on alpha and theta narrow spectral bands due to the abundant evidence for their relevance to AAC above and beyond higher-frequency bands (Neo and McNaughton 2011; Neo et al. 2011; McNaughton et al. 2013; Kelley et al. 2017; Neal and Gable 2017; De Pascalis et al. 2018; Neo et al. 2020). Some of the most prominent effects were in the dlPFC and medial frontal cortex—specifically the OFC, vmPFC, and anterior cingulate. Interestingly, previous literature has highlighted the importance of a cortical network including the dlPFC and vmPFC, anterior cingulate cortex (ACC), and OFC along with the amygdala and ventral striatum, to AAC decision-making (Aupperle et al. 2015; McNaughton et al. 2016; Kirlic et al. 2017). Further, greater anxiety has been shown to be associated with less mPFC, OFC, ACC, and dlPFC activation during AAC—purported to relate to difficulties in integrating signals from other brain regions concerning the various characteristics of a decision-making situation (Aupperle and Paulus 2010). We found the majority of these regional activations to negatively associate with avoidance behavior, specifically to the r-dlPFC TMS stimulation condition. Interestingly, during avoidance, we found reward-biased conflict to evoke greater power in these same regions, specific to certain bands and latencies. However, some more dorsal parts of the mPFC, superior parietal, and medial temporal were found to positively associate with avoidance behavior, showing less power during reward-biased conflict during avoidance decisions, specific to certain bands and latencies.

The dlPFC is one of the most prominent regions associated with AAC (Aupperle et al. 2015; McNaughton et al. 2016; Kirlic et al. 2017). The dlPFC has consistently been shown to activate with greater conflict during AAC paradigms, and excitatory stimulation to the r-dlPFC has been shown to elicit greater avoidance behavior (Chrysikou et al. 2017). The specific role of dlPFC in regard to behavior itself has been less clear—which could be a product of the proposed lateralization sometimes found in the dlPFC—with left dlPFC activation relating to approach tendencies and r-dlPFC activation relating to behavioral inhibition and avoidance tendencies (Kirlic et al. 2017). However, this lateralization is far from consistent across studies, which could indicate some degree of complexity in dlPFC function as it relates to AAC behavior that is not accounted for in traditional analyses.

There is abundant literature spanning rodent, nonhuman primate, and human studies implicating the vmPFC in value-based decision-making, particularly AAC (Hiser and Koenigs 2018). This is thought to be influenced by the vmPFC’s connectivity with the ventral striatum and amygdala (Hiser and Koenigs 2018). The rodent literature has elucidated a causal role of the vmPFC in AAC—however, the nature of this role (i.e., inactivation leading to greater or less avoidance) is unclear due to conflicting results in the literature (for review see Kirlic et al. 2017; Hiser and Koenigs 2018; Roberts and Clarke 2019). This could in part be due to the vmPFC’s role in both processing negative valence (i.e., anxiety) and reward processing, which is thought to relate to its connectivity with the amygdala and ventral striatum, respectively (Di Martino et al. 2008; Cauda et al. 2011; Choi et al. 2012; Jung et al. 2014). The OFC, also strongly implicated in AAC (Spielberg et al. 2013), and often termed vmPFC, is linked to the maintenance of the current and expected motivational value of stimuli (O’Doherty and Dolan 2006). The OFC has also been hypothesized to relay stimulus value to regions such as the dlPFC (O’Doherty and Dolan 2006; Szatkowska et al. 2008), particularly when these valuations are relevant to goal-directed behavior (as opposed to habit). Finally, the ACC has a well-established role in AAC (Spielberg et al. 2013), thought to encode predicted value associated with actions (Rushworth and Behrens 2008), updating and relaying this information to the dlPFC (Spielberg et al. 2013).

Additionally, there were a number of effects specific to the temporal cortex—both lateral and medial. The temporal cortex, though to a lesser extent than the aforementioned cortices, has been implicated in AAC response—with greater temporal activation positively associated with approach responses (O'Neil et al. 2015). The temporal cortex has known projections to the prefrontal cortex in nonhuman primates (Ungerleider et al. 1989) and humans (Jung et al. 2017), as well as connectivity with subcortical temporal regions such as the amygdala (Jiang et al. 2019) and ventral striatum (Choi et al. 2017). Therefore, while not often elucidated, it is not surprising that we saw temporal activation in response to dlPFC-TMS in the context of this task.

Finally, we saw a significant cluster within the right superior parietal lobe negatively and positively associate with avoidance behavior following dlPFC TMS in early theta and late alpha—respectively. While not a key region within the AAC literature—the right superior parietal lobe has been implicated often in emotional arousal and regulation (Lang et al. 1998; Beauregard et al. 2001; Goldin et al. 2008).

Interpreting Spectral Variations in dlPFC-Evoked Effects

Our analyses specifically focused on lower frequency bands (theta, alpha) as they have been implicated in AAC above and beyond higher-frequency bands (Neo and McNaughton 2011; Neo et al. 2011; McNaughton et al. 2013; Kelley et al. 2017; Neal and Gable 2017; De Pascalis et al. 2018; Neo et al. 2020). However, despite this evidence, the functional interpretation of spectral band power at the EEG level remains inconsistent (Klimesch 1999; Klimesch et al. 2005; Newson and Thiagarajan 2018). Though TMS-evoked spectral propagation has strong groundings in the literature, the neural mechanism underlying spectral response is not well understood. TMS-evoked spectral response, both broadband and band specific, has been found to index cortical excitability, but that literature is predominantly within primary motor cortex—and therefore, the generalizability of such findings to other cortical areas is less well understood (Van Der Werf and Paus 2006; Rosanova et al. 2009; Ferrarelli et al. 2012; Johnson et al. 2012; Fecchio et al. 2017). While the majority of findings were consistent between alpha and theta bands, both of which have known relevance to AAC, the interpretation of the variation in results between alpha and theta band remains unclear without further mechanistic investigation.

Interpreting Latency Variations in dlPFC-Evoked Effects

While TMS-EEG is a useful tool to probe the cortical excitability of local and downstream activation during specific cognitive states and processes, the mechanism underlying the different spectral propagations, and their latencies, is not well understood. However, TMS-evoked potentials have been shown to primarily reflect fluctuations in cortical excitability resulting from excitatory and inhibitory neurotransmission at the site of stimulation, as well as the propagation of activation through cortical networks following TMS (Rogasch and Fitzgerald 2013; Premoli et al. 2014; Darmani et al. 2016). Further, early (<250 ms) post-TMS activation has been shown to reflect site-specific activations, whereas later potentials are less site specific (Rogasch et al. 2020). This is particularly relevant as we find a number of our results to be specific to particular latency bands, with few effects spanning the entire post-TMS temporal trajectory. For instance, the early TMS-evoked mPFC, dlPFC, and cingulate activation found to associate with avoidance behavior may be more reflective of dlPFC-specific activation, whereas the later OFC and temporal activations may be more reflective of general trial-specific processing.

Limitations

The results from the current study necessitate caution in interpretation due to a handful of experimental and analytic limitations. First, regarding experimental design, the task itself, though well-grounded in both nonhuman (Amemori and Graybiel 2012; Amemori et al. 2015; Amemori et al. 2018, 2020) and human work (Ironside et al. 2020), is limited in that we cannot state with certainty that each participant was equally conflicted by the choice to approach points or avoid pictures. Second, the 2 studies presented are conducted on separate groups of people, and therefore inherently a between-subject design, reducing the interpretability of the 2 sets of results taken together. Third, stimulation in both experiments were lateralized, with only right hemispheric regions probed—therefore, we are not able to speak to the laterality of these findings, and how they would or could relate to left hemispheric results. Fourth, the study was not strongly powered, neither in trials nor sample size, and therefore, we call for future work with more power to replicate the presented results. Fifth, while source localization is well founded in its accuracy in solving the inverse problem of mapping sensor-level activation to its cortical source, there is a lack of consensus as the best method for its implementation (Grech et al. 2008; Asadzadeh et al. 2020). Lastly, TMS-evoked spectral power is an exciting measure for causal probing of cortical excitability, but the interpretation of its neural relevance is still largely unknown, and therefore, we are limited in our interpretation of the TMS-evoked spectral results presented in this article (Pellicciari et al. 2017).

Conclusions and Future Directions

The current 2 studies found that the dlPFC motivates approaching reward during conflict by 1) enhancing reward sensitivity and 2) coactivating with or signaling a network of regions (including the cingulate, medial/orbital/lateral PFC, temporal, and superior parietal). These findings corroborate many existing theories only previously supported by correlational imaging in humans but are novel to the field as they provide mechanistic understanding of the dlPFC’s role in conflict behavior. The study highlights the importance of the prefrontal, orbitofrontal, and medial cortices in avoidance behavior during conflict and calls for further causal interrogation of all nodes within this network to better understand the directionality and behavioral relevance of their interactions.

Funding

National Institute of Neurological Disorders and Stoke (R01 NS025529 to A.M.G.); Army Research Office (W911NF-16-1-0474 to A.M.G.) and the Saks Kavanaugh Foundation (to A.M.G.).

Notes

Thank you to Callie McGrath for early contributions to this work. Conflict of Interest: None declared.