Neural substrates of approach‐avoidance conflict decision‐making

Abstract

Animal approach‐avoidance conflict paradigms have been used extensively to operationalize anxiety, quantify the effects of anxiolytic agents, and probe the neural basis of fear and anxiety. Results from human neuroimaging studies support that a frontal–striatal–amygdala neural circuitry is important for approach‐avoidance learning. However, the neural basis of decision‐making is much less clear in this context. Thus, we combined a recently developed human approach‐avoidance paradigm with functional magnetic resonance imaging (fMRI) to identify neural substrates underlying approach‐avoidance conflict decision‐making. Fifteen healthy adults completed the approach‐avoidance conflict (AAC) paradigm during fMRI. Analyses of variance were used to compare conflict to nonconflict (avoid‐threat and approach‐reward) conditions and to compare level of reward points offered during the decision phase. Trial‐by‐trial amplitude modulation analyses were used to delineate brain areas underlying decision‐making in the context of approach/avoidance behavior. Conflict trials as compared to the nonconflict trials elicited greater activation within bilateral anterior cingulate cortex, anterior insula, and caudate, as well as right dorsolateral prefrontal cortex (PFC). Right caudate and lateral PFC activation was modulated by level of reward offered. Individuals who showed greater caudate activation exhibited less approach behavior. On a trial‐by‐trial basis, greater right lateral PFC activation related to less approach behavior. Taken together, results suggest that the degree of activation within prefrontal‐striatal‐insula circuitry determines the degree of approach versus avoidance decision‐making. Moreover, the degree of caudate and lateral PFC activation related to individual differences in approach‐avoidance decision‐making. Therefore, the approach‐avoidance conflict paradigm is ideally suited to probe anxiety‐related processing differences during approach‐avoidance decision‐making. Hum Brain Mapp 36:449–462, 2015. © 2014 Wiley Periodicals, Inc.

INTRODUCTION

Approach behavior occurs in presence of reward or stimuli that further ensure the integrity of the individual, whereas avoidance behavior is often related to impending or experienced punishments, which threaten the integrity of the individual [Gray, 1981; Gray and McNaughton, 2000; Lang et al., 1998]. In daily life, we are often faced with difficult decisions in which the same choice could lead to both rewarding and threatening outcomes (e.g., giving a public speech for work could lead to anxiety or embarrassment but also job promotion), creating an “approach‐avoidance conflict.” Approach‐avoidance conflict situations pose a unique challenge for comparing the value of available options because individuals must integrate information concerning the value of potential rewards and punishments, as well as the likelihood and magnitude of those potential outcomes [Aupperle and Paulus, 2010; Quartz, 2009; Rolls and Grabenhorst, 2008]. Such conflict situations may be of particular interest when considering psychiatric conditions. For example, anxiety disorders often involve avoidance of emotional discomfort even when this requires sacrifice of longer‐term gains. Exposure therapies for anxiety disorders require that a patient agree to experience emotionally provoking situations for the purpose of correcting maladaptive cognitions and experiencing longer‐term benefits [Barlow, 2002; Foa and Kozak, 1986]. Further understanding of how individuals make decisions in situations involving approach‐avoidance conflict may contribute to our understanding of emotional decision‐making as well as have implications for psychiatric treatment.

Animal models of approach‐avoidance conflict have been used extensively to model anxiety related behaviors [Millan, 2003; Millan and Brocco, 2003]. The basic model involves the same behavior being associated with both a reward (e.g., water or food) and a punishment (e.g., mild electric shock). In this way, a conflict between approaching the reward and avoiding the negative stimulus is established. Anxiolytic agents consistently increase approach behavior during these conflict paradigms [Millan, 2003; Millan and Brocco, 2003]. Moreover, lesioning of the amygdala and medial prefrontal cortex (PFC; infralimbic or prelimbic) has also been shown to increase approach behavior during conflict [Kopchia et al., 1992; Millan, 2003; Moller et al., 1997; Resstel et al., 2008; Yamashita et al., 1989].

Human neuroimaging research has provided a wealth of information related to (a) processing of emotional or threat‐related stimuli and avoidance motivation, (b) reward processing and approach motivation, and (c) decision‐making. Processing of emotional or threat‐related stimuli is thought to rely primarily on a prefrontal–insula–amygdala network [Quirk and Mueller, 2008; Shin and Liberzon, 2010]. The amygdala has been implicated in the processing of fear, or salient stimuli in general, particularly in relation to Pavlovian conditioning [Davis and Whalen, 2001; LeDoux, 2000; Morrison and Salzman, 2010]. The insula has been implicated in monitoring internal bodily state (i.e., “interoceptive” processing), anticipating potential changes in that state, and integrating this information for the purposes of homeostasis, emotional processing, or cognitive control [Craig, 2009; Critchley, 2005; Critchley et al., 2004]. Medial PFC and anterior cingulate cortex (ACC) regions have been implicated in the inhibition and regulation of responses to emotional stimuli [Compton, 2003; Etkin et al., 2011; Ochsner and Gross, 2005; Salzman and Fusi, 2010].

Reward processing and decision‐making research has highlighted the importance of a cortico‐striatal network [Haber and Knutson, 2010; Hare et al., 2008; O'Doherty, 2004; Rolls and Grabenhorst, 2008; Schultz, 2000; Spielberg et al., 2008]. The striatum, its ventral aspects specifically, has been implicated in signaling the value of a reward as well as in anticipating or predicting reward [Diekhof et al., 2012; Haber and Knutson, 2010; O'Doherty, 2004]. Orbitofrontal regions have also been implicated in signaling the value of reinforcers or rewards and guiding decision‐making [Grabenhorst and Rolls, 2011; Hare et al., 2008; Rolls and Grabenhorst, 2008; Spielberg et al., 2008; Wallis, 2007], while dorsolateral PFC (dlPFC) regions have been implicated in incorporating such value signals when directing attention and planning for the execution of a decision or response [Lee and Seo, 2007; Rorie and Newsome, 2005; Rosenbloom et al., 2012]. The ACC has been implicated in monitoring errors or conflicts in the environment, inhibiting responses to conflicting stimuli, and guiding decision‐making [Botvinick, 2007; Rosenbloom et al., 2012; Rushworth and Behrens, 2008; Shackman et al., 2011; Spielberg et al., 2008].

Recently, different neuroimaging approaches (i.e., electroencephalography [EEG], fMRI) have been used to identify that a combined neural circuitry involving PFC (ACC, orbitofrontal cortex [OFC], and dlPFC), amygdala, and basal ganglia underlie approach‐avoidance learning and action motivations, as well as the influence of trait approach and avoidance motivations [Prevost et al., 2011; Schlund et al., 2011; Berkman and Lieberman, 2010; Spielberg et al., 2008, 2011, 2012]. In particular, the right PFC has been associated with trait avoidance motivations [Spielberg et al., 2012] while ventral striatal, medial OFC [Simon et al., 2010], and left PFC has been more associated with trait approach motivations [Spielberg et al., 2012]. Etkin et al. have developed a Stroop‐like paradigm to reveal neural responses associated with processing emotionally conflicting stimuli (e.g., negatively valenced face stimuli paired with a positively valenced word) [Etkin et al., 2006]. This work supports the role of frontal–amygdala networks in processing emotional conflict and left rostral ACC regions specifically in adapting efficiently to this emotional conflict (reflected by decreased response times). Most relevant to the current study, Talmi et al. [2009] investigated neural responses during a learning paradigm that involved making decisions with conflicting outcomes of pain and reward. They found activation within the right rostral ACC and ventral striatum to reward prediction errors was attenuated by pain. In addition, orbitofrontal–insula connectivity related to greater pain avoidance behavior. This study provides additional support that an overlapping circuitry involving insula, striatum, and PFC regions are most likely involved in decision‐making during conflict.

We developed the approach‐avoidance conflict (AAC) paradigm aimed at quantifying decision‐making behavior during situations that involve conflicting outcomes that could motivate approach and/or avoidance behaviors [Aupperle et al., 2011]. We aim to add to the current approach‐avoidance literature by examining decision‐making behaviors during conflict—rather than focusing on trait motivations or approach versus avoidance learning. This paradigm differs from that used by Talmi et al. [2009] in that it purposely does not involve a learning component, focusing more specifically on conflict aspects of decision‐making. In addition, the AAC includes emotional “punishment” (negative affective images) rather than painful stimuli, and therefore may have implications for understanding avoidance behavior related specifically to affect—such as that observed in anxiety disorders [American Psychiatric Association, 2000; Barlow, 2002]. Lastly, this paradigm measures behavior as a continuous measure, allowing investigation of what regions may be contributing to the level of approach/avoidance behavior exhibited by an individual in response to a conflict situation.

In this study, we administered the AAC task [Aupperle et al., 2011] in conjunction with functional magnetic resonance imaging (fMRI) to further previous research related to approach‐avoidance conflict. Specifically, our goals were to help elucidate brain regions that (a) are specifically recruited during conflict (vs. nonconflict) decision‐making situations and (b) signal the level of potential reward during conflict situations (thus reflecting approach motivation). By measuring approach behavior in response to conflict decisions, we were additionally able to investigate whether (and how) neural responses specific to conflict or level of reward influenced individuals' decisions. We hypothesized that conflict (as compared to nonconflict) decisions would involve greater recruitment of amygdala, insula, striatum, medial PFC (OFC, ACC), and dlPFC. In addition, we hypothesized that striatal and left PFC regions would be modulated by level of reward during conflict and that activation in these regions would relate to greater approach behavior, while greater amygdala, insula, and right PFC regions would relate to greater avoidance behavior during conflict.

METHOD

Participants

Fifteen healthy volunteers (eight male; mean age = 23.27 ± 1.91; mean education = 16.13 ± 1.81) completed one testing session involving the AAC paradigm during fMRI and self‐report measures. Exclusion criteria included self‐report of current or lifetime diagnosis of Axis I psychiatric, substance abuse, or neurological disorders. The University of California, San Diego human research protection program approved the study and all subjects gave written, informed consent prior to completion of study procedures.

Measures

AAC task

The ACC task was programmed using Adobe Flash Professional CS5©. Prior to scanning, subjects were trained on the AAC and completed three practice trials to ensure full understanding of the task. The AAC was conducted similar to previously described [Aupperle et al., 2011] but with the addition of approach‐reward trials. For each trial, participants were shown a runway with pictures on each side to represent two potential outcomes (Fig. 1). Each potential outcome included an affective stimulus and certain level of reward points. A picture of a sun indicated a positively valenced affective stimulus, while a cloud indicated a negatively valenced affective stimulus. Level of reward points was represented by the amount of red fill in rectangles. The subject used button presses to move an avatar on the runway to indicate their relative preference for the potential outcomes. The location of the avatar at the end of the decision phase (the end position that the subject moves the avatar to) corresponded to the probability of the two outcomes occurring. If the avatar was moved to the middle of the runway, there was a 50% chance of each outcome; if all the way to one side, there was a 90% chance of the nearest outcome and 10% chance of the further outcome; and so on. Subjects, therefore, controlled the likelihood of the outcomes, but were unable to determine one or the other outcome for certain. The starting position of the avatar influences both initial response time [F(8,112) = 3.10, P = 0.003] as well as the end position of the avatar [F(8,112) = 2.87, P = 0.006]. However, the starting position of the avatar was counterbalanced across trials (for each condition type) to control for these effects and the effort required for any individual across the task.

Figure 1.

Decisional conditions included within the AAC paradigm. Avoid‐threat conditions (Part A) involve no point‐reward incentives but only the possibility of viewing a negative (indicated by a cloud) or positive (indicated by a sun) affective stimulus. Approach‐reward conditions (Part B) involve no threat of negative affective stimuli but only the possibility of obtaining reward points or no reward points (both paired with positive affective stimuli). During conflict conditions (Parts C–E), reward points (2, 4, or 6 point levels) are given only for the outcomes associated with a negative affective stimulus while the competing choice includes no points but a positive affective stimulus. The avatar starts out at different locations on the runway, counterbalanced within each of the condition types. The subject is asked to move the avatar (by pressing arrow keys on a keyboard) to a position that accurately reflects their preference between the two potential outcomes. The position in which they move the avatar determines the relative probability of each of the two outcomes occurring (Part E; 10/90%, 20/80%, 30/70%, 40/60%, 50/50%, and vice versa probabilities, corresponding to the nine potential avatar positions ranging from −4 to +4). Therefore, if they move their avatar to the middle, there is a 50% chance of each outcome occurring; if they moved all the way to one side, there is a 90% chance of the nearest outcome occurring, but still a 10% chance of the furthest outcome occurring, and so on. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

The affective stimuli included image and sound combinations collected from the International Affective Picture System (IAPS) [Lang et al., 2008], International Affective Digitized Sounds (IADS) [Bradley and Lang, 1999] and other freely available audio files. The “reward” included 0, 2, 4, or 6 points presented along with a trumpet sound. There were three trial types (see Fig. 1), named in reference to the behavioral motivation presumably elicited by the negative affective stimulus and/or the reward points: (1) “Avoid‐threat” (AV), in which 0 points were offered for both outcomes and thus, the only explicit motivation was to avoid the negative affective outcome. (2) “Approach‐reward” (APP), in which 2 versus 0 points were offered but with positive affective stimuli associated with both outcomes. For these conditions, the only explicit motivation was to approach the rewarded outcome. (3) Three levels of “Conflict” in which 2 (CONF2), 4 (CONF4), or 6 (CONF6) points were offered for the outcome involving a negative affective stimulus while 0 points were offered for the outcome involving a positive affective stimulus. These conditions were designed to produce approach‐avoidance conflict, as the same behavior was associated both with reward and punishment. Notably, subjects had no way of knowing the valence/arousal level of the potential negative affective outcome and that outcome was not proportionate to the level of reward points. Thus, the increasing reward level was meant to increase motivation to approach the negative affective stimulus. There were a total of 90 trials, with 18 of each trial type (AV, APP, and three levels of conflict), administered via an event‐related design over three scans (each scan 30 trials, 544 s duration). The task was divided into three scans to allow participants time to rest and to help ensure they remained alert throughout the task. At the end of each scan, a screen appeared displaying total points received and an award ribbon. The points did not translate into monetary reward and thus, subjects were playing for points only. Points were used as reward rather than money to ensure relative balance in the salience of reward and punishment on this task. Notably, previous research indicates that paradigms involving either nonmonetary or monetary reward elicit similar neural activation patterns [Peters et al., 2011; Peters and Buchel, 2010]. The timing of each AAC trial is displayed in Figure 2. Descriptive statistics of the valence and arousal ratings of stimuli included in the AAC, as well as the number of trials in which individuals experienced negative versus positive image/sounds outcomes and no reward versus 2, 4, or 6 points rewards is provided in Supporting Information.

Figure 2.

Sequence of screens presented during one trial of the AAC task. A decisional phase is first presented for a maximum of 4 s. The affective stimulus phase consists of either a negative or positive affective image (from IAPS) [Lang et al., 2008] and a matched affective sound (from free source websites such as http://freesound.org and the IADS) [Bradley and Lang, 1999]. The affective stimulus phase lasts a total of 6 s. The reward phase consists of a screen displaying points earned on the current trial as well as the total points collected thus far on the task in combination with a reward‐related trumpet sound. The reward phase lasts a total of 2 s. An intertrial fixation averaging 6 s is displayed between trials. The AAC task consists of 18 trials of each condition type (displayed in Fig. 1), for a total of 90 trials. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

The main dependent variables for AAC behavioral analyses included the following, averaged for each trial type: (1) approach behavior, or the avatar's end position on the runway in relation to the negative affective outcome. This ranged from −4 (avoidance all the way away from the negative affective stimulus/reward) to +4 (approach all the way towards the negative affective stimulus/reward) and (2) Response time for initial button press.

Self‐report measures

After completing the task, subjects completed a questionnaire which involved rating on a 1–7 Likert scale (1) how difficult it was for them to make decisions during the task, (2) how motivated they were to get reward points, (3) how motivated they were to avoid negative affective stimuli (question left blank by two subjects), (4) how enjoyable they found the positive pictures, and (5) how anxious or uncomfortable they felt in response to the negative pictures. For further description and histograms displaying the ratings on these questions, please see Supporting Information. The State‐Trait Anxiety Inventory (STAI) State and Trait subscales [Spielberger et al., 1983] were completed prior to completion of the fMRI scan (within the same session) and were available for 14 of the subjects.

fMRI acquisition

The AAC task was conducted during three fMRI scans sensitive to blood oxygenation level‐dependent (BOLD) contrast using a Signa Excite (GE Healthcare) 3.0 Tesla scanner (T2*‐weighted echoplanar (EPI) imaging, TR = 2000 ms, TE = 30 ms, field of view [FOV] = 24 cm, 64×64 matrix, forty 3.0 mm axial slices, 272 scans). During each scan session, a high‐resolution T1‐weighted image [spoiled gradient recalled, TR = 8 ms, TE = 3 ms, FOV = 25 cm, 172 sagittal slices with approximately 1 mm³ voxels] was obtained for anatomical reference.

Behavioral data analyses

To characterize behavioral differences between task conditions, we used within‐subjects analyses of variance (ANOVA). Post hoc two‐tailed t‐tests were conducted to further characterize differences. Spearman's rho correlations were used to investigate relationships between behavioral measures during AAC conflict trials and self‐report measures and post‐task questionnaire ratings. Correlational results were corrected for multiple comparisons using Bonferroni adjustment, and thus considered significant at P < 0.004.

fMRI data analyses

Data were preprocessed and analyzed using analysis of functional neuroimages (AFNI) [Cox, 1996]. EPI images were aligned to high‐resolution anatomical images. Voxel data points representing outliers relative to surrounding data points were eliminated and interpolated. Voxel time series were interpolated to correct for nonsimultaneous slice acquisition and corrected for three‐dimensional motion. Data were spatially blurred (to 6 mm full width at half maximum [FWHM]) and normalized to Talairach space. Individual time‐series data were analyzed using a multiple regression model with a BOLD hemodynamic response function (4–6 s peak). Regressors of interest included (1–5) APP, AV, and CONF2, CONF4, CONF6 decision phases, (6–7) outcome image phases involving negative images (NI) and positive images (PI), (8–9) outcome reward phases involving 2, 4 or 6 points (REWpos) and no points (REW0). Regressors of no interest included: (1) baseline regressor, (2–4) motion‐related regressors (roll, pitch, and yaw), and (5) linear trend used to eliminate slow signal drifts. A secondary linear contrast was computed by averaging BOLD activation for all conflict trials (CONFall).

A separate multiple regression model utilized an “amplitude‐modulated” regressor, created for conflict trials. The regressor values for conflict trials were modulated by the avatar's end position (ranging from −4 to +4) on each individual trial to identify regions for which BOLD response varied with the level of approach behavior. The amplitude‐modulated regressor (CONFamp) was entered into the model as a regressor of interest along with the following regressors of no interest: (1) baseline regressor, (2–4) motion‐related regressors (roll, pitch, and yaw), (5) linear trend to eliminate slow signal drifts, (6–7) APP and AV decision phases, (8–9) REWpos and REW0, and (10–11) NI and PI.

Percent signal change (PSC) was calculated by dividing the regressor of interest by the control regressors. Repeated measures ANOVA were used to examine differences in PSC between CONF, APP, and AV decision trials and between CONF2, CONF4, and CONF6 decision trials. PSC was extracted from each cluster of activation and post hoc t‐tests were conducted to determine directionality of findings. Another paired‐samples t‐test was conducted in AFNI comparing PSC for the CONFamp as compared to a base of zero to determine brain regions for which activation related to level of approach behavior on a trial‐by‐trial basis. The AAC paradigm is optimized to be sensitive to neural responses during the decision phase (with jittered interstimulus interval prior to this phase). However, to supplement analyses of the decision phase, paired‐samples t‐tests were used to examine differences in PSC for NI compared to PI outcome phases and for REWpos compared to REW0 outcome phases. These results are included in Supporting Information.

Analyses were conducted voxel‐wise within regions of interest (ROI) of the bilateral amygdala, insula, striatum, medial PFC (including anterior and dorsal cingulate), and middle frontal gyrus (for dlPFC). ROI masks were constructed from 43 T1‐weighted images of healthy control participants using data‐driven methods combining Talairach stereotactic definitions and anatomical gray matter probabilities (full description and figure of ROI masks provided in Supporting Information). Whole‐brain analyses were also conducted and results included in Supporting Information. Results were considered significant at P < 0.01, Monte Carlo corrected for multiple comparisons, resulting in a minimum cluster extent (and adjusted P threshold) of 768 mm³ (P < 0.00002) for whole‐brain, 384 mm³ (P < 0.0002) for medial PFC/cingulate, 320 mm³ (P < 0.0003) for middle frontal gyrus, 256 mm³ (P < 0.0005) for insula, 192 mm³ (P < 0.001) for striatum, and 128 mm³ (P < 0.002) for amygdala regions. Average PSC was extracted from ROI clusters of activation and Spearman's rho (ρ) correlation analyses were used to examine relationships to self‐report measures and AAC task behavior. The correlational results with each ROI cluster were corrected separately for multiple comparisons, using Bonferroni correction for AAC task behavior (response time and approach behavior; 0.05/2 = adjusted P‐threshold of 0.025) and self‐report measures (five questions relating to the AAC task + 2 STAI subscales = 0.05/7 = adjusted P ‐threshold of 0.007).

RESULTS

Behavioral

Approach behavior differed between task conditions [F(4,56) = 60.75, P < 0.001]. Post hoc tests revealed that approach behavior during conflict (across 2, 4, and 6‐point trials; M = 2.05, SD = 1.22) differed from both AV [t(14) = −8.60, P < 0.001; M = −3.05, SD = 0.86] and APP [t(14) = 5.45, P < 0.001; M = 3.96, SD = 0.09] conditions and that approach behavior increased significantly from 2‐ to 4‐point [t(14) = −2.46, P = 0.028] but not from 4‐point to 6‐point [t(14) = −.83, P = 0.419] conditions see Figure 3.

Figure 3.

Approach behavior during the AAC task. A +4 indicates the subjects moved the avatar all the way towards the NI and/or reward points. Approach behavior significantly differed between task conditions [F(4,56) = 60.75, P < 0.001). Post hoc tests revealed that approach behavior during conflict (2, 4, and 6‐point trials) significantly differed from avoid‐threat [t(14) = −8.60, P < 0.001] and approach‐reward [t(14) = 5.454, P < 0.001] conditions and that approach behavior during conflict increased significantly from 2‐point to 4‐point conditions [t(14) = −2.455, P = 0.028] but not from 4‐point to 6‐point conditions [t(14) = −0.833, P = 0.419].

Response time also differed between conditions [F(4,56) = 2.57, P = 0.048]. Post hoc tests revealed slower response times during conflict as compared to APP conditions [t(14) = −3.14, P = 0.007]. There was a nonsignificant trend for slowed response during CONF4 conditions compared to CONF6 [t(14) = 1.97, P = 0.069]. There was no difference in response time between AV and conflict conditions [t(14) = −0.85, P = 0.411] or between CONF2 conditions and either CONF6 [t(14) = 0.66, P = 0.517] or CONF4 conditions [t(14) = −1.03, P = 0.319].

There were trend correlations suggesting that individuals who showed greater approach behavior (ρ = −0.54, P = 0.040) and rated themselves as being more motivated to obtain a reward (ρ = 0.66, P = 0.007) also exhibited faster response times during conflict. There were no significant or trend correlations between AAC behavior and STAI scores. See Supporting Information Table 1 for full list of correlations between self‐report and AAC behavioral measures.

Table 1.

Regions of interest exhibiting activation differences between conflict, approach‐reward, and avoid‐threat decision trials of the AAC

Side	Region	BA	Cluster size (mm3)	x a	y	z	F b	Direction
Right	Dorsal ACCc	32	4032	5	30	27	8.73	CONF > AV > APP
Right	Middle frontal/precentral gyrus	6	896	40	−3	47	7.75	[CONF = AV] > APP
Right	dlPFC	9,6	448	51	6	37	7.58	CONF > AV > APP
Right	dlPFC	9	384	43	33	33	6.98	CONF > APP > AV
Right	Anterior insula, inferior frontal gyrus	13, 47	2752	37	18	3	9.10	CONF > AV > APP
Left	Anterior insula, inferior frontal gyrus	13, 47	832	−35	20	2	8.21	CONF > [AV = APP]
Right	Caudate body		320	10	5	11	7.48	[CONF = AV] > APP
Left	Caudate head		192	−9	7	3	9.33	CONF > [AV = APP]
Left	Posterior insula, superior temporal	13	512	−42	−21	19	7.45	AV > CONF > APP
Left	Dorsal cingulate, medial frontal	24, 6	448	−8	−5	48	7.18	[AV = APP] > CONF
Right	Posterior insula, superior temporal	13	384	44	−16	8	11.66	AV > CONF > APP
Right	Posterior insula	13	384	39	−13	18	6.69	[AV = APP] > CONF
Left	Middle frontal/precentral gyrus	6	448	−24	19	53	7.07	[AV = APP] > CONF

^aAll coordinates are Talairach coordinates (x,y,z) based on Talairach Daemon software (Lancaster, et al., 2000).

^bResults shown from ANOVA analysis examining effect of condition (conflict, approach‐reward, avoid‐threat) on percent signal change during decision trials of the AAC task (for N = 15 healthy controls). Clusters listed met significance cutoff of P < 0.01, Monte Carlo adjusted for multiple comparisons within small volume regions of interest. Direction of findings was identified using post‐hoc paired samples t‐tests.

^cAbbreviations: BA = Brodmann area; ACC = anterior cingulate cortex; CONF = conflict decision trials; AV = avoid‐threat decision trials; APP = approach‐reward decision trials; mm³ = millimeters cubed.

There was one subject who exhibited less approach behavior compared to the other subjects (>1 SD below the mean for remainder of the group). All correlations conducted for this study were therefore repeated with the outlier removed. If the outlier was removed from the behavioral analyses, the listed correlations remained consistent.

fMRI

Conflict versus avoid‐threat and approach‐reward decisions

ROI analyses revealed that activation within the right rostral/dorsal ACC (BA 32), right dlPFC (BA 6, 9), bilateral anterior insula (BA 13, 47), and bilateral caudate was greatest for conflict conditions. Activation within the bilateral posterior insula (BA 13), dorsal mid cingulate (BA 6, 24), and left lateral PFC (BA 6) was greater for both AV and APP conditions than the conflict conditions see Figure 4 and Table 1. All of these ROI clusters remained significant for whole‐brain analyses except for the caudate activations (see Supporting Information). PSC was extracted for ROI clusters identified as significant for conflict conditions and correlations with task behavior and self‐report measures were examined.

Figure 4.

Regions exhibiting greater activation for conflict than nonconflict conditions of the AAC. Conflict decision trials of the AAC were associated with greater activation than both approach‐reward and avoid‐threat decision trials within the (A) right ACC (BA 32; shown at x = 5), (B) right caudate (shown at y = 6), (C) right anterior insula (BA 13; shown at z=3), and (D) right dorsolateral prefrontal cortex (dlPFC; BA 9; shown at y = 8). As shown in the scatterplots, greater PSC to conflict conditions in the right caudate body (ρ = −0.62, P = .014) related to less average approach behavior for conflict trials of the AAC. Right dlPFC PSC related to self‐report of greater difficulty making decisions on the task (ρ = 0.77, P = 0.001). [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Individuals with greater PSC to conflict conditions in the caudate body (ρ = −0.618, P = 0.014) exhibited less approach behavior during conflict. There was also a trend‐level correlation suggesting that individuals with greater ACC activation also exhibited less approach behavior during conflict (BA 32; ρ = −0.539, P = 0.038). Individuals with greater PSC within the left caudate head exhibited slower average response time during conflict (ρ = 0.639, P = 0.010) see Figure 4. When the outlier was removed, these correlations remained consistent. There was also an additional correlation between right (ρ = −0.63, P = 0.017) anterior insula and approach behavior.

Individuals with greater right dlPFC (BA 9, 6) PSC reported having greater difficult making decisions during the task (ρ = 0.77, P = 0.001). There were trend correlations indicating that individuals with greater ACC PSC may also have had greater difficult making decisions during the task (ρ = 0.590, P = 0.021), been less motivated to get reward points (ρ = −0.565, P = 0.028), and found the positive pictures less enjoyable (ρ = −0.538, P = 0.038). Additional trend relationships were identified between both right anterior insula PSC (ρ = −0.574, P = 0.025) and right caudate PSC (ρ = −0.567, P = 0.028) and self report of being less motivated to obtain reward points. There were no significant or trend correlations with STAI. If the outlier was removed from analyses, correlations described remained consistent.

BOLD activation modulated by level of potential reward

Repeated measures ANOVA examined activation differences between CONF2, CONF4, and CONF6 conditions. Post hoc t‐tests were used to determine the directional difference between the conditions. Activation within right dorsal caudate was greatest for CONF6 decisions (post hoc tests revealed CONF6 > CONF2 > CONF4). There were also several middle frontal regions exhibiting greatest activation for the CONF2 conditions. See Table 2.

Table 2.

Regions exhibiting activation differences between conflict decision trials of the AAC involving 2, 4, or 6 potential points

Side	Region	BA	Cluster size (mm3)	x a	y	z	F b	Direction
Right	Caudate head		192	10	16	2	7.20	6 > 2 >4
Right	Middle frontal/precentral gyrus	6	2752	35	14	51	7.56	2 > [4 = 6]
Left	dlPFCc	10,46	1152	−41	52	10	8.30	2 > [4 = 6]
Right	dlPFC	10,46	704	40	49	8	7.52	2 > [4 = 6]
Right	Middle/superior frontal gyrus	8	640	30	28	46	6.85	2 > [4 = 6]
Left	Dorsolateral prefrontal cortex	9,8	576	−43	24	38	7.51	2 > 6 > 4
Left	Middle frontal/precentral gyrus	6	576	−31	12	48	8.60	2 > [4 = 6]
Left	Middle frontal/precentral gyrus	6	320	−43	6	47	5.83	[2 = 6] > 4

^aAll coordinates are Talairach coordinates (x,y,z) based on Talairach Daemon software (Lancaster, et al., 2000).

^bResults shown from ANOVA analysis examining effect of level of reward (2‐point, 4‐point, 6‐point) on percent signal change during conflict decision trials of the AAC task (for N = 15 healthy controls). Clusters listed met significance cutoff of P < 0.01, Monte Carlo adjusted for multiple comparisons within small volume regions of interest. Direction of findings was identified using post‐hoc paired samples t‐tests.

^cAbbreviations: BA = Brodmann area; mm³ = millimeters cubed.

PSC was extracted from the right caudate and correlations between CONF6 PSC and conflict behavior and self‐report were examined. Individuals with greater right caudate PSC exhibited slower response time (ρ = 0.675; P = 0.006) and less approach behavior (ρ = −0.671, P = 0.006) during conflict. This correlation remained significant when the outlier was removed. There was also a trend correlation between right caudate PSC and self‐report of motivation to obtain reward (ρ = −0.645, P = 0.009). This correlation remained consistent when excluding the outlier.

Amplitude‐modulated BOLD activation during conflict

Activation within two regions of the right lateral PFC related to less approach behavior on a trial‐by‐trial basis during conflict. This included right lateral frontal (BA 6; 1216 mm³; t(14) = −3.53; x,y,z = 30, 6, 57) and right middle/superior frontal (BA 10, 46; 576 mm³; t(14) = −3.88; x,y,z = 39, 50, 15) see Figure 5.

Figure 5.

Regions exhibiting a relationship with approach behavior exhibited on individual trials. An amplitude‐modulated regressor (modulated by level of approach behavior on each trial) was used to identify regions relating to trial‐by‐trial approach behavior during conflict. Greater activation within (A) right lateral frontal [BA 6; 1216 mm³; t(14) = −3.53; x,y,z = 30, 6, 57; shown at x = 30] and (B) right middle/superior frontal (BA 10,46; 576 mm³; t(14) = −3.88; x,y,z = 39, 50, 15; shown at x = 38) related to less approach behavior. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

DISCUSSION

This study aimed to identify neural substrates of approach‐avoidance conflict decision‐making. The AAC paradigm allowed us to (1) identify brain regions involved in making conflict decisions (vs. the nonconflict decisions, i.e., “approach‐reward” and “avoid‐threat”) and (2) examine how activation within these regions relates to level of reward offered as well as the decisional responses made (i.e., level of approach behavior). Conflict decisions involved greater activation within bilateral anterior insula, bilateral caudate, and primarily right ACC, and dlPFC regions—partially supporting our hypothesis regarding circuitry underlying AAC (with the exception of the amygdala). Activation within right caudate and lateral frontal regions related to level of approach behavior during conflict, while right dorsal caudate activation related to level of reward. These results extend previous research related to neural substrates of approach and avoidance motivations and support the use of the AAC paradigm in probing AAC neural circuitry.

The right ACC cluster identified for conflict versus “approach‐reward” and “avoid‐threat” decisions lies in dorsal aspects of pregenual ACC and the anterior midcingulate cortex [Vogt, 2009], often also referred to as dorsal ACC. The ACC in general has been associated with conflict and error monitoring, inhibition, and emotion regulation [Botvinick, 2007; Etkin et al., 2011; Ochsner and Gross, 2005] as well as reward/value prediction, action selection, and decision‐making [Rosenbloom et al., 2012; Rushworth and Behrens, 2008; Wallis and Kennerley, 2011]. Dorsal ACC has been proposed to play a primary role in cognitive control or appraisal of emotion while ventral/rostral aspects play a more primary role in emotional processing or automatic emotional regulation [Bush et al., 2000; Etkin et al., 2011; Mohanty et al., 2007; Shackman et al., 2011; Steele and Lawrie, 2004]. Using an emotional Stroop‐like conflict task, Etkin and colleagues provided evidence that the dorsal ACC may be involved in processing of conflicting stimuli whereas rostral ACC and lateral PFC regions may be involved in resolving emotional and nonemotional conflict, respectively [Egner et al., 2008; Etkin et al., 2006]. The resolving of emotional conflict in these Stroop‐like tasks involves focusing attention away from task‐irrelevant aspects to respond quicker to the salient aspects. The conflict paradigm used in the current study involves more explicit decision‐making that instead involves conflicting outcomes of actions (rewarding and punishing aspects) and is presumably under more conscious cognitive control. Current results suggest that resolution of emotional conflict for the purposes of explicit decision‐making requires involvement of dorsal ACC and its connections with lateral PFC regions [Koski and Paus, 2000]. Notably, right lateral PFC (BA 6 and BA 10, 46) activation related to the level of avoidance behavior exhibited on a trial‐by‐trial basis. We propose that the ACC may be involved in monitoring and processing the level of emotional conflict experienced by an individual, signaling a representation of approach/avoidance drives or goals to the lateral PFC. The dorsal and lateral PFC regions are then potentially recruited to exert attentional control, maintain goal pursuit, and implement final decisions and motor responses [Chouinard and Paus, 2006; Hoshi, 2006; Hoshi and Tanji, 2007; Spielberg et al., 2012]. Of note, the regions identified as relating to increased avoidance behavior were right lateralized—supporting propositions that right PFC regions are more involved in avoidance motivations as opposed to approach motivations or to processing negative valence in general [Harmon‐Jones et al., 2010]. These results also indicate that these higher‐order, cortical regions may play a more prominent role in determining explicit decisions during emotional conflict than subcortical structures such as the amygdala and insula. This proposition is perhaps further supported by dlPFC PSC relating to self‐reported difficulty making decisions.

Previous research suggests the posterior insula may be involved in processing exteroceptive environmental/sensory information (e.g., touch, pain) and interoceptive information (e.g., heart rate, body orientation) [Cauda et al., 2012; Craig, 2003, 2011]. The anterior insula has been implicated in integrating interoceptive information for the purposes of awareness, emotional processing, and cognitive control [Craig, 2009, 2011; Critchley, 2005; Menon and Uddin, 2010]. In the current study, dorsal anterior insula regions activated more during conflict than either of the nonconflict decision trials—perhaps reflecting increased awareness of interoceptive signals and integration with emotional/reward valuations to inform the decision‐making process. In contrast, the posterior insula exhibited greater activation during both of the nonconflict (particularly “avoid‐threat” trials) conditions. When a decision involves low levels of conflict and thus requires less cognitive control, somatosensory afferents potentially signaling aversive outcomes may be processed primarily by the posterior insula. In comparison, the anterior insula may play a more integrative role (e.g., with PFC regions) during high levels of conflict. An important aspect of these findings is that anterior insula activation was not simply signaling greater anticipation of negative affective outcomes. Instead, results indicate that greater anterior insula activation related to greater avoidance behavior (when the outlier was removed from analyses), which is consistent with previous findings that anterior insula activation relates to risk aversion [Rudorf et al., 2012]. Rather than signaling the likelihood of an outcome, the insula may signal the potential changes in emotional or interoceptive state of the individual if the negative affective outcome were to occur–thus, relating to increased avoidance behavior. Alternatively, the increased activation with greater avoidance behavior could signal uncertainty of outcomes [Singer et al., 2009], as subjects' behavior in the current study mostly ranged between approach (e.g., moved the avatar to the +4 position) and staying in the middle (moving the avatar to the 0 or 1 position). Further research is needed that experimentally manipulates the level of uncertainty involved in AAC situations to disentangle the role of the insula in signaling homeostatic changes in response to potential threat versus uncertainty.

Dorsomedial striatal activation was also observed during the AAC task. Specifically, bilateral caudate regions exhibited greater activation during conflict compared to nonconflict decisions, and right caudate exhibited greater activation during 6‐point conflict conditions compared to 2‐point and 4‐point conditions. In addition, greater caudate activation during decisions related to slower response times and greater avoidance behavior. There has been much work focused on understanding subregions of the striatum and their relative roles in reward processing, reward learning, action selection, and motor initiation and control [Atallah et al., 2007; O'Doherty et al., 2004; Voorn et al., 2004]. It has been proposed that ventral regions serve as “critic,” predicting and evaluating the summed value of potential outcomes, while the dorsal regions serve as “actor,” using predictive and evaluative signals to guide motoric and action functions [Atallah et al., 2007; Mattfeld et al., 2011; O'Doherty et al., 2004]. This theory has been expanded by more recent evidence that the dorsomedial striatum is involved in more deliberative (goal‐directed) action selection as opposed to the dorsolateral striatum, which may be more involved in habit learning or the ventral striatum, which may be more involved in conditioned responding [van der Meer et al., 2012]. The AAC task presumably involves explicit decision‐making rather than instrumental learning or conditioning. The involvement of caudate regions more during conflict decisions (as opposed to “approach‐reward” or “avoid‐threat” decisions) and the relationship with level of avoidance behavior supports the dorsal striatum's role in deliberative action selection (as opposed to habitual or automatic responses that may be more likely during nonconflict decisions). Caudate regions were also modulated by level of reward in the current study, indicating these regions may be involved in weighing the relative values of outcomes. This supports recent primate findings that the dorsal striatum may signal the difference between two values more reliably than ventral striatal regions [Cai et al., 2011].

Interestingly, there was a large cluster within the dlPFC that exhibited greater activation for 2‐point compared to 4 and 6 point conflict conditions. Considering the correlation between dlPFC activation during conflict and self‐reported difficulty making decisions, this may reflect that 2‐point conflict conditions were the more difficult decisions (with more balance between approach and avoidance drives). This is supported by the approach behavior being the lowest for 2‐point conflict conditions. Notably, previous research with the AAC task has reported greater approach behavior with each point level increase during conflict, while participants in the current study increased approach behavior between 2‐ and 4‐point but not between 4‐ and 6‐points. This may have limited our ability to identify neural substrates for conflict reward modulation.

We did not identify amygdala activation during our analyses of decision trials on the AAC. This is somewhat surprising, particularly given the significant role of the amygdala in emotional processing and the effect of amygdala lesions on animal conflict behavior [Davis and Whalen, 2001; LeDoux, 2000; Millan, 2003]. It is important to note that our lack of results does not preclude the amygdala being involved in emotional decision‐making, but indicates it may have been (a) involved relatively equally across conflict and nonconflict trials, and/or (b) primarily involved in other phases of conflict decision making. Concerning the former, there is evidence that the amygdala is involved in processing positively valenced (e.g., appetitive or rewarding) stimuli in addition to negatively valenced (e.g., threatening or fearful) stimuli [Baxter and Murray, 2002; Morrison and Salzman, 2010; Murray, 2007] and therefore may have been involved in signaling the salience of outcomes for conflict, “approach‐reward,” and “avoid‐threat” trials. There is a plethora of evidence concerning the amygdala's specific role in Pavlovian conditioning [Davis and Whalen, 2001; LeDoux, 2000], with more recent research suggesting it may also be involved in signaling the learned value or salience of stimuli to guide actions [Seymour and Dolan, 2008]. It is possible the amygdala is more involved in the learning of conflicting stimulus‐outcome associations than in signaling the difference in conflicting outcomes at the time of decision‐making. Although the AAC paradigm is optimized for decision‐trial analyses, we included outcome phase analyses in Supporting Information. Greater amygdala activation was identified when processing negative versus positive affective outcomes, the extent of which correlated with prefrontal, insula, and caudate activation during decision‐making (see Supporting Information). It is possible that the amygdala played a role in signaling the salience of outcomes—which was then incorporated into future decisions via these other regions.

LIMITATIONS

This study was limited by a relatively small sample size (N = 15), which may have limited our power. This may have been one reason we did not identify any significant correlations with measures of anxiety, as was found in a previous behavioral study with the AAC paradigm [Aupperle et al., 2011]. Future studies with a larger study population would also be useful in examining how self‐reported trait reward and avoidance motivations may relate to conflict behavior. The design of the AAC task is useful in that it allows for understanding of how neural activations may relate to behavior during conflict. However, because of this, there were unequal experiences between subjects on the task (i.e., with some subjects exhibiting greater approach behavior and thus, experiencing greater number of negatively valenced images and greater number of reward points; see Supporting Information). This could alter activations in ways that cannot be completely accounted for by our analyses. In addition, we defined “reward” in terms of the points offered on any given trial and punishment in terms of the negatively valenced emotional images. The positively valenced emotional images could have been rewarding as well, which could mean that “avoidance” of negatively valenced stimuli was, in fact, “approach” toward positively valenced emotional stimuli. However, task behavior was highly correlated with how much subjects reported being motivated to obtain reward points but not with how enjoyable they found the positive pictures (see Supporting Information), providing some evidence that reward points were the primary motivator for approach behavior. With this decision‐making task, it is possible that individual variability of the subject's expectations could have influenced results in a way that limits generalizability with animal conflict paradigms, which presumably would not be influenced by such factors. Further translational work, identifying whether the same pharmacologic agents influence approach behavior for both animal conflict paradigms and the AAC, could help in alleviating this concern. Lastly, the AAC task was not designed to distinguish neural underpinnings for the proposed hierarchical levels of approach‐avoidance motivations (e.g., goals vs. strategies vs. tactics; Spielberg et al., 2013), which would be a fruitful endeavor for future research.

CONCLUSIONS

The AAC paradigm demonstrated usefulness for probing brain regions involved in AAC decision‐making. Results suggest that a prefrontal–insula–striatal circuitry is recruited when making explicit decisions during approach‐avoidance situations. In addition, we provide evidence that right lateralized ACC, caudate, and lateral PFC regions may be instrumental in determining approach‐avoidance decisions. The AAC task may be helpful in further understanding neural substrates of psychiatric disorders characterized by imbalances in approach‐avoidance drives, including anxiety and depressive disorders, and substance abuse disorders.

Supporting information

Supplementary Information