Emotional arousal amplifies the effects of biased competition in the brain

Abstract

The arousal-biased competition model predicts that arousal increases the gain on neural competition between stimuli representations. Thus, the model predicts that arousal simultaneously enhances processing of salient stimuli and impairs processing of relatively less-salient stimuli. We tested this model with a simple dot-probe task. On each trial, participants were simultaneously exposed to one face image as a salient cue stimulus and one place image as a non-salient stimulus. A border around the face cue location further increased its bottom-up saliency. Before these visual stimuli were shown, one of two tones played: one that predicted a shock (increasing arousal) or one that did not. An arousal-by-saliency interaction in category-specific brain regions (fusiform face area for salient faces and parahippocampal place area for non-salient places) indicated that brain activation associated with processing the salient stimulus was enhanced under arousal whereas activation associated with processing the non-salient stimulus was suppressed under arousal. This is the first functional magnetic resonance imaging study to demonstrate that arousal can enhance information processing for prioritized stimuli while simultaneously impairing processing of non-prioritized stimuli. Thus, it goes beyond previous research to show that arousal does not uniformly enhance perceptual processing, but instead does so selectively in ways that optimizes attention to highly salient stimuli.

INTRODUCTION

In the last decade, much progress has been made to further our understanding of the effects of emotional arousal on cognition, demonstrating that arousal influences everything from the early stages of processing such as perception and attention to higher order cognitive processes such as memory and decision-making (Bechara, 2004; Phelps, 2006; Mather, 2007; Levine and Edelstein, 2009; Pessoa, 2009). However, there are also a growing number of apparently contradictory findings; for example, some studies reveal emotion-induced enhancement in visual perception (Phelps et al., 2006; Padmala and Pessoa, 2008; Lim et al., 2009), whereas other studies reveal impairment in perception due to emotion (Most et al., 2005, 2006; Ciesielski et al., 2010). Similarly, there also is contradictory evidence in the memory literature, with so-called emotion-induced retrograde amnesia on the one hand (Strange et al., 2003, 2010; Hurlemann et al., 2005) and emotion-induced retrograde enhancement on the other (Anderson et al., 2006; Knight and Mather, 2009). It is not clear from these previous studies how and why emotion can produce these opposing effects.

Arousal-biased competition theory (ABC theory; Mather and Sutherland, 2011) is motivated by these contradictory findings and provides an account of how and why emotional arousal (whether elicited by external stimuli, internal motivations, or stress hormones) can sometimes enhance and sometimes impair perception and memory. ABC theory is based on models of biased competition (Bundesen, 1990; Vecera and Farah, 1994; Desimone and Duncan, 1995; Itti et al., 1998; Miller and Cohen, 2001; Deco and Rolls, 2005). Although there are several computational accounts for biased competition in attention, such as an object-based (e.g. Vecera and Farah, 1994), a location-based (e.g. Desimone and Duncan, 1995) or an interactive model (Deco and Lee, 2002), the important aspect of biased-competition processes is that they allow a particular stimulus (or its location) to be prioritized in attention and garner more neural resources for its representation. For instance, the biased competition models of attention posit that visual attention involves competitive processes that are biased in favor of high-priority stimuli at the expense of low-priority stimuli (Itti et al., 1998; Itti and Koch, 2000). Both bottom-up saliency and top-down relevancy help determine priority (e.g. Beck and Kastner, 2009; Baluch and Itti, 2011). For instance, stimuli that move suddenly or are brighter than their surroundings attract attention (i.e. bottom-up saliency). Also task goals or internal expectations help determine priority (i.e. top-down relevancy).

ABC theory proposes that these biased-competition processes are amplified by emotional arousal, such that when there is one stimulus with high priority, that stimulus will gain even more resources under arousing conditions than it would otherwise. Thus, according to the ABC theory, arousal can lead to both more enhanced processing of salient stimuli (winner-take-more) and more impaired processing of non-salient stimuli (loser-take-less). Initial behavioral tests support this ABC hypothesis in perception and short-term memory (Lee et al., 2012; Sutherland and Mather, 2012). Lee et al. (2012) demonstrated that when participants were exposed to intermittent arousing images during a visual search task, perceptual learning was enhanced for a salient target among non-salient distractors (a 80°-tilted target line among 55°-tilted distractor lines), compared with when they were exposed to intermittent neutral images. In contrast, the same arousal manipulation impaired perceptual learning for a non-salient target (50°-tilted target line among 55°-tilted distractor lines). Thus, whether learning was enhanced or impaired by arousal was determined by the salience of the target. Sutherland and Mather (2012) found that when participants were asked to report as many letters as they could from a briefly flashed array of letters, if they had just heard an arousing sound they were more likely to report perceptually salient letters (those presented in dark grey against the white background) and less likely to report non-salient letters (those presented in light grey) than if they had just heard a neutral sound, indicating that arousal enhances processing of salient stimuli at the cost of processing non-salient stimuli.

This study followed up on these previous behavioral findings to investigate the neural underpinnings of the interactions between emotional arousal and priority on visual processing. ABC theory predicts that enhancement in brain activation seen under arousal should be specific to high-priority stimuli and their locations, with concurrent diminished processing to non-priority stimuli. Thus, our hypothesis is that emotional arousal does not enhance visual processing indiscriminately. Instead, emotional arousal should modulate visual processing differently depending on whether those stimuli are dominating the current competition among stimuli or not.

We know from previous functional magnetic resonance imaging (fMRI) studies that emotional arousal can enhance activity in visual processing regions (Phan et al., 2002). These studies typically manipulated arousal via pictures that were emotional or neutral (e.g. Lang et al., 1998; Mather et al., 2006). Even subliminal presentation of arousing stimuli can lead to greater activity in primary visual cortex, which has led researchers to posit that ‘core neuronal arousal in the brain…involves a network incorporating primary visual areas, somatosensory, implicit memory and conflict monitoring regions’ (Brooks et al., 2012, p. 2962). Similarly enhanced activity has been found in primary auditory cortex in response to emotional sounds, leading to the idea that, under arousal, ‘increased activation within primary areas might contribute to efficient processing of behaviorally relevant information across different sensory modalities’ (Ethofer et al., 2012, p. 196). As represented by these quotes, current thinking in the field is that arousal potentiates activity in perceptual processing regions, which in turns enhances processing of the stimuli eliciting the arousal. This ‘enhancement-only’ perspective on the effects of arousal on perception does not consider how arousal might also impair processing, despite much behavioral evidence that arousal does sometimes impair perceptual processing (Mather and Sutherland, 2011).

A limitation of most previous fMRI studies of how emotional arousal influences information processing is that the perceptual qualities of the arousing vs non-arousing stimuli are not fully controlled. Features such as color, luminance and object salience may vary on average across the two types of stimuli, and it might be those perceptual qualities that enhance perceptual processing of arousing stimuli, rather than the arousal per se. For instance, arousing sounds may be louder and arousing pictures may have brighter colors than neutral pictures. One way to eliminate this concern is to use fear conditioning to endow a previously neutral tone or image with affective meaning. Pessoa et al. have shown that visual stimuli that were previously conditioned to predict a shock elicit greater amygdala and visual cortex activity, even when no shock occurs on that particular trial (Padmala and Pessoa, 2008; Lim et al., 2009). Neutral targets (pictures of buildings or houses) that were previously associated with shock were detected better than those not associated with shock (Lim et al., 2009). In addition, playing a tone that predicts shock increases participants’ ability to detect a visual target shown 1 s later (Padmala and Pessoa, 2008). These findings rule out perceptual confounding factors for emotion-potentiated visual processing. Furthermore, they indicate that arousal induced by one modality (audition) can increase perceptual processing in another modality (vision).

In addition, these findings are consistent with the ‘enhancement-only’ perspective described earlier that posits that arousal leads to a global increase in activation in sensory regions (e.g. Brooks et al., 2012; Ethofer et al., 2012). But the paradigms used that showed enhanced perceptual processing under arousal all examined activation in response to high-priority visual stimuli that were either the target of a task goal or were likely to attract attention due to their emotional content. ABC theory predicts that arousal should also ‘diminish’ perceptual processing of low-priority stimuli presented with high-priority stimuli.

In this study, to test the hypothesis that emotion leads to simultaneous enhancements and impairments in visual processing, we simultaneously presented two task-irrelevant visual cues with different saliency levels. We used face and place images as the two visual cues based on previous research showing that the fusiform face area (FFA) responds selectively to faces (Kanwisher et al., 1997) and parahippocampal place area (PPA) responds selectively to spatial place images (Epstein and Kanwisher, 1998), allowing us to differentiate brain activation in response to each of these cues. To differentiate the priority of the two cues, there was always a brief luminance increase (i.e. a yellow colored frame; Figure 1B) in the salient cue’s location. In addition, in this study, we always used face images as the more salient cues given their intrinsic evolutionary value (for a review, see Palermo and Rhodes, 2007). During the task, participants had to identify the location of a green dot target, shown on the same (salient-location target) or opposite (non-salient-location target) side as the salient cue. As the dot appeared in the salient and non-salient locations equally often, the salience of the visual cues was not predictive of the dot location. We used a fear-conditioned tone (i.e. CS+) to manipulate participants’ arousal levels on a trial-by-trial basis during the dot-probe task.

Fig. 1.

Schematic illustration of one trial for the (A) fear-conditioning and (B) dot-probe tasks. In the dot-probe task, participants were asked to detect the dot’s location. In salient-location-target trials, the target dot appeared in the location of the salient cue, whereas in non-salient-location-target trials, the target dot appeared in the location of the non-salient cue. Note that the trial in this figure shows a CS+ trial (A) and non-salient-location target type (B).

Based on the ABC model, we hypothesized that arousal induced by the CS+ would lead to stronger perceptual/attentional processing for the salient face cue, as indicated by increased FFA activation, and simultaneously reduce processing for the non-salient place cue, as indicated by decreased PPA activation. Additionally, we predicted that responses in intraparietal sulcus (IPS) region would also show a saliency–arousal interaction, because of its well-known role in attentional orienting to uninformative but salient stimuli (Corbetta et al., 2008) and its hemispheric lateralization depending on the location of stimuli presentation (Geng et al., 2006; Konen and Kastner, 2008; Schenkluhn et al., 2008; Geng and Mangun, 2009; Szczepanski et al., 2010; Chica et al., 2011). Because the IPS responds to external salient stimuli regardless of the stimulus content (Talmi et al., 2008), it permitted us to examine the arousal–saliency interaction independently from the category-specific brain responses (i.e. FFA and PPA). Specifically, we hypothesized that if the salient cue was presented on the left side, then the right IPS response would be greater on CS+ trials than on CS− trials; in contrast, if the salient cue was on the right side, then the left IPS response would be greater on CS+ trials than on CS− trials.

One tenet of biased competition theory is that when an object gains dominance within one part of the network, aspects of its representation will be strengthened elsewhere (Duncan, 2006). Therefore, when an object dominates competition, it will bias processing to favor other information from the same location (Szczepanski et al., 2010). In our study, this means that the perceptual salience of a face stimulus attracting attention to the right side of the screen should bias processing to favor information appearing in that location and lead to faster detection of the target when it appeared behind the salient cue than the non-salient cue. Of particular importance given our hypotheses, this detection advantage should be greater for trials following CS+ than for trials following CS− tones. We tested this in an initial behavioral experiment in addition to an fMRI experiment.

METHODS

Participants

The behavioral experiment involved 52 healthy participants (14 Males, 38 Females; mean age = 20.50, range = 18–30) and the fMRI experiment involved 20 healthy subjects (9 Males, 11 Females; mean age = 21.95, range = 18–35). All subjects gave informed consent in accordance with University of Southern California Institutional Review Board guidelines.

Stimuli and apparatus

Two tones (500 and 1500 Hz) were adopted as conditioned stimuli (i.e. CSs) to avoid possible confounding effects of using stimuli in the same sensory modality to induce emotional arousal and to measure perceptual processing (e.g. Zeelenberg and Bocanegra, 2010). To separate the blood-oxygen-level dependent (BOLD) fMRI response for salient vs non-salient stimuli, we used faces (known to elicit selective responses in the FFA; 140 female and 140 male) and places (known to elicit selective responses in the parahippocampal gyrus; 139 buildings and 139 houses) as cue stimuli. The face and place stimuli were selected from multiple stimuli libraries (Lundqvist et al., 1998; Tottenham et al., 2002; Lee et al., 2006; Ebner et al., 2010; Konkle et al., 2010). All stimuli were gray-scaled and normalized to the mean luminance of all images. In the main attention task session, 64 face and 64 place stimuli were randomly selected from the larger pool of stimuli and assigned to the conditions for each participant. The schedule of stimulus presentation and data collection was controlled by the PsychToolbox extensions (Brainard, 1997; Pelli, 1997) based on Matlab 2010b (The MathWorks Corp. Natrick, MA). The mild electric shock used as an unconditioned stimulus was delivered to the third and fourth fingers of the left hand via a shock stimulator (E13–22; Coulbourn Instruments, Allentown, PA), which included a grounded RF filter.

Procedure

In this study, we report on two experiments; the first one was conducted in the lab (behavioral experiment), and the other one was conducted in the scanner (fMRI experiment). Each participant completed a fear-conditioning session (one run with 30 trials) and a dot-probe session (two runs for the behavioral experiment; three runs for the fMRI experiment; 64 trials per run). An additional localizer scan was administered during the fMRI experiment.

Fear conditioning

An initial fear-conditioning session established the emotionally arousing nature of the CS+ tone with a trace-conditioning paradigm. In this session, either the low- or high-pitched tone was paired with electric shock. Which tone was paired with shock was counterbalanced across participants. Each trial in the conditioning session began with onset of two placeholders (3.8° × 3.8°; 7° eccentricity) against a gray background to match contextual information between conditioning session and subsequent dot-probe session. Participants were then presented with one of the CS tones for 0.7 s, followed by a 1.2 s inter-stimulus interval. After this interval, a shock was delivered for 0.5 s if the tone was assigned to the CS+ condition (Figure 1A) and followed by a fixation jittered to appear for 10, 11 or 12 s. On the CS− tone trials, there was no shock. The 1.2 s interval before the shock was chosen to allow participants’ arousal level induced by the CS+ tone to increase before the face-scene cues appeared in the main dot-probe task. In order to ensure that participants attended to the tones, they were asked to indicate the type of tone (i.e. low- or high pitched) with a button press immediately after they were presented with a tone. A total of 30 trials were presented in a random order: 10 CS+ with shock, 10 CS+ without shock and 10 CS− tones. Thus, CS+ tones were followed by a shock with a 50% partial reinforcement schedule. Prior to the experiment, participants were informed which tone was predictive of the electric shock, but they were not informed about the probability of shock on each trial. The intensity of ‘highly unpleasant but not painful’ electric shock was determined individually (behavioral experiment: mean intensity = 2.26 mA, range 1.1–4.0 mA; fMRI experiment: mean intensity = 2.30 mA, range 1.4–4.0 mA). Trials that included shocks were excluded in subsequent analyses.

Dot-probe task

After the fear-conditioning task, participants performed the dot-probe task. A total of 128 trials were presented over two runs in the behavioral experiment, and a total of 192 trials were presented over three runs in the fMRI experiment; each run consisted of 32 CS+ trials (16 salient-location target and 16 non-salient-location target trials) and 32 CS− trials, and thus a total of 64 trials per run were presented in a random order. A trial began with onset of two placeholder outlines (3.8° × 3.8°; 7° eccentricity), followed by either the CS+ or CS− tone playing for 0.7 s, and a 1 s blank screen to maximize the effect of the CS tones in eliciting emotional arousal (Bocanegra and Zeelenberg, 2009). Then, a face–place image pair was presented in the two placeholder frames simultaneously for 0.1 s. Finally, a dot target was shown 0.1 s after offset of the face–place pair (stimulus onset asynchony = 0.2 s) for 1.0 s on the same or opposite side as the salient face cue. Participants were asked to identify the location of the dot target (0.5° × 0.5°) by pressing a left or right button. A fixation cross (randomly jittered; 2–8 s) was presented between trials (Figure 1B). Each face was randomly paired with one of the place images assigned to the same condition; the location of each stimulus was also randomly determined for each participant.

To enhance the saliency of cue stimuli, there was always a brief luminance increase consisting of a yellow colored frame in the salient cue’s location (Figure 1B). In addition, we always used a face image as the salient cue given its own intrinsic evolutionary value (for a review, see Palermo and Rhodes, 2007). To minimize extinction of conditioned responses, three additional CS+ trials with shock were presented randomly in each run. Other than the shock, these booster trials were identical to the main trials, and were excluded from further analysis. The booster trials were always followed by a 10 s blank interval. The face and scene cue stimuli in the booster trials were selected from images not used in the main task.

Localizer session

An additional face/scene localizer run followed the dot-probe task. The localizer consisted of 24 blocks (12 face-task blocks and 12 scene-task blocks). Each block contained eight trials that lasted 11.6 s and were separated from each other by a 10 s blank screen. Each block began with a 2 s task cue to indicate which task to perform, followed by a series of face or scene images; each stimulus was shown for 1.2 s. Participants were asked to indicate the sex of faces in the face-task blocks and the type (building or house) of places in the scene-task blocks. Participants were explicitly informed that no shocks would be administered during the run.

Psychophysiology data

During both experiments, individual skin conductance responses (SCRs) were also acquired to confirm the success of the emotional arousal manipulation (Lim et al., 2009) with MRI-compatible electrodes placed on the index and middle finger of the left hand. All physiological data were recorded at 1 kHz sampling rates through the MP-150 system (BIOPAC system, Goleta, CA), connected to a grounded RF filter, and MR-compatible leads and electrodes. For SCRs, the data were detrended, smoothed with a median filter over 50 samples to filter out MRI-induced noise. SCR data epochs were extracted from a time window between 0 and 8 s after CS tone onset, and baseline-corrected between 0 and 1 s. The peak SCR amplitude was taken between 1 – 8 s from the trial-by-trial average SCR epoch as a function of CS tone. Due to a technical failure, recording could not be completed for one participant in the fMRI experiment.

MRI data

Acquisition

All MRI data were acquired on a Siemens 3T Magnetom Trio with stimuli presented on a liquid crystal display monitor (1024 × 768 pixels at 60 Hz) positioned behind the head of participants and viewed using a mirror attached to a 32-channel matrix head coil at the University of Southern California Dana & David Dornsife Cognitive Neuroscience Imaging Center. High-resolution (T1-MPRAGE) structural images were acquired first (repetition time or TR = 1950 ms; echo time or TE = 2.26 ms; flip angle or FA = 7°; 1 mm isotropic voxel; 256 mm field of view). Next, functional images were acquired with gradient-echo echo-planar T2*-weighted imaging. Each functional volume consisted of 40 interleaved (no skip) 2.5 mm axial T2*-weighted slices (TR = 2000 ms; TE = 25 ms; FA = 90°; matrix size = 64 × 64; field of view = 192 mm).

Preprocessing

The first eight volumes were discarded to allow for T1 equilibration. Standard preprocessing was conducted using FSL FMRIBs Software Library (FSL v5.0; www.fmrib.ox.ac.uk/fsl); slice-time correction, motion correction with MCFLIRT, spatial smoothing with a Gaussian kernel of full width at half maximum (FWHM) 6 mm, high-pass temporal filtering with a filter width of 100 s and skull stripping of structural images with FSL's Brain Extraction Tool (BET), and registering each functional image to both the participant’s high-resolution structural image and the standard Montreal Neurological Institute (MNI) 2 mm brain. MELODIC ICA2 (Beckmann and Smith, 2004) was applied to remove noise components.

Fear-conditioning data analysis

For fear-conditioning data, a standard two-stage mixed-effects analysis was performed. The general linear model (GLM) of the BOLD signal for each CS tone type including trace-interval period was estimated at the first (fixed) level with a double-gamma hemodynamic response function. We added six motion parameters to the design matrix, following the example of numerous previous fear-conditioning studies (Büchel et al., 1999; Dunsmoor et al., 2008; Lim et al., 2008, 2009), including those that localized ‘fear-network’ regions (for a review, see Sehlmeyer et al., 2009) in using motion parameters. One limitation to this approach that should be noted is that adding motion regressors to the design matrix as covariates of no interest may lead to under-estimates of the cluster activity insofar as participants’ motion is correlated with anticipating a shock on CS+ trials (e.g. Johnstone et al., 2006). In addition, a timeline demarcating trials involving an electrical shock was added as a covariate of no interest. The participants’ data were then inputted into a random-effect model for group analysis (Beckmann and Smith, 2004; Woolrich et al., 2004). Group level analysis was thresholded using cluster detection statistics, with a height threshold of Z > 2.3 and a cluster probability of P < 0.05 (one-tailed) (Worsley, 2001), corrected for whole-brain multiple comparisons using Gaussian Random Field Theory unless otherwise noted.

Dot-probe task analysis for category-specific regions of interest

For the main dot-probe task data, an ROI analysis was performed. To do so, stimulus-dependent changes in BOLD signal were modeled at the first (fixed) level with regressors for cue stimulus presentation (a face–house pair) and their respective temporal derivatives for each arousal condition (i.e. CS+ and CS−) separately. Trials were collapsed across those with dots in the salient cue location and the non-salient cue location. Motion parameters and booster shock trials were also included in the design matrix as covariates of no interest. The effects of each regressor were estimated over three runs, except for two participants who each had one run excluded due to extensive movement.

Using FSL Featquery, percent signal change values were extracted from the FFA and PPA region of each hemisphere separately for the arousal and non-arousal conditions as a weighted average of the surrounding voxels, with weights determined by a 4 mm FWHM Gaussian kernel mask. The ROI mask for each FFA and PPA region were individually defined from the localizer session as the peak voxel in ventral temporal cortex that was most selective for faces (face block > scene block; Z = 2.57, uncorrected) and for scenes (scene > face) in each hemisphere, respectively. In the left hemisphere, ROIs could be defined for all participants for both FFA (mean peak MNI voxel coordinates: [−42 −54 −24]) and PPA ([−26 −44 −14]). In the right hemisphere, ROIs could be defined for all participants for the PPA (mean peak [26 −40 −14]) and for all but one for the FFA (mean peak [40 −54 −22]).

Although the main goal of this study was to determine the effects of saliency–arousal interactions within ROIs, a group-level analysis (random-effects) was also performed to model general task-related activation at a group level (Supplementary Figures S1 and S2).

Dot-probe task analysis for IPS ROI

To examine responses in IPS as a function of emotional arousal and the location of salient cue presentation, another GLM was estimated; stimulus-dependent changes in BOLD signal were modeled at the first (fixed) level with regressors for cue stimulus presentation and their respective temporal derivatives as a function of salient cue location (left, right) and arousal condition (i.e. CS+ and CS−). The effects of each regressor were first estimated for each participant over three runs.

To define the IPS ROI region, regressors only for target dot and their respective temporal derivatives collapsed across arousal conditions were modeled separately as a function of dot location (left or right). The right IPS mask was then defined for each individual, based on a contrast between left vs right target dot location. Specifically, we determined the peak voxel (3 mm, Gaussian sphere mask) in the contrast (left > right target location; Z = 1.64, uncorrected) within a standard anatomical brain mask of IPS (as provided by FSL; Jülich histological atlas) that was most selective in the right hemisphere. In turn, the left IPS mask was defined individually based on a reversed contrast (right > left target location). A right IPS mask (mean peak voxel coordinates: [−40 −56 46]) was identified in 19 out of 20 participants for the left > right target contrast and a left IPS mask was identified in 14 out of 20 participants for the right > left contrast (mean peak [32 −48 44]).

Functional connectivity analysis

To characterize dynamic interregional interactions, a beta series correlation analysis (Rissman et al., 2004; Gazzaley et al., 2007) was applied. To do so, a new design matrix was created where a visual cue event per each trial was coded as a unique covariate, resulting in 192 independent variables (i.e. 96 cues with CS+ and 96 with CS−). The global mean signal level over all brain voxels was calculated for each time point and was included to reduce the confounding effects of the global signal change. Motion parameters and booster shock trials were also included in the design matrix as covariates of no interest. Finally, extracted mean activities (i.e. mean parameter estimates) of each trial from a seed region (peak voxel of each individual functional mask) were used to compute correlations between the seed’s signal and signal of all other voxels in the whole brain, thus generating condition-specific seed correlation maps. Correlation magnitudes were converted into z-scores using the Fisher’s r-to-z transformation. Condition-dependent changes in functional connectivity were assessed using random effects analyses, which were thresholded at the whole-brain level using clusters determined by Z > 2.3 and a cluster significance threshold of P = 0.05 (corrected; one-tailed).

RESULTS

Behavioral experiment

Fear-conditioning results

Fear conditioning successfully modulated arousal as indicated by greater SCRs in response to CS+ tones than in response to CS− tones, t(51) = 9.19, P < 0.001 (Figure 2A).

Fig. 2.

(A) SCR results and (B) dot-probe task results in the behavioral experiment. (C) Whole-brain analysis results from the fear-conditioning session. Voxels that showed stronger activation during CS+ trials than CS− trials during the fear-conditioning task. The regions overlap with both the salience network (Shirer et al., 2012) and the fear network (Sehlmeyer et al., 2009). (D) SCR results and (E) dot-probe task results from the fMRI experiment. Error bars denote the standard within-subject error term (Loftus and Masson, 1994). *P < 0.05, **P < 0.01, ***P < 0.001.

Dot-probe task results

The reaction times (RTs) from error trials (0.01%) or those with more than 2.5 s.d. above or below each participant’s mean were removed (0.03%) before obtaining the mean RTs for each condition for each participant. A repeated-measures analysis of variance (ANOVA), Arousal Condition (CS+, CS−) × Trial Type (salient-location target, non-salient-location target) was conducted. The manipulation of salience worked, as there were faster RTs in salient-location-target (343.39 ms) than non-salient-location-target trials (354.73 ms), as indicated by a main effect of Trial Type, F(1,51) = 29.04, P < 0.001. There was also a significant arousal-by-trial type interaction, F(1,51) = 7.79, P < 0.01 (Figure 2B). Subsequent pairwise comparisons revealed CS+ tones marginally significantly facilitated RTs during the salient-location-target trials (P = 0.084), but impaired RTs during the non-salient-location-target trials (P = 0.05). Thus, as predicted by the ABC model, participants were faster to respond to the target dot when it appeared in the location of the more salient cue, and simultaneously slower to respond to the target when it appeared in the location of the non-salient cue in the arousing than in the non-arousing trials.

fMRI experiment

Fear-conditioning results

As expected, the fear-conditioned tone (i.e. CS+) compared with the other tone (CS−) elicited more brain activity (Figure 2C; see Table 1 for local maxima regions in the clusters), in ‘fear-network’ regions (see Sehlmeyer et al., 2009) including bilateral insular (R: [34 24 4], Z = 4.93, Cluster 1; L: [−30 22 −6], Z = 5.03, Cluster 2), bilateral frontal operculum cortex/inferior frontal gyrus (R: [48 18 −2], Z = 5.01, Cluster 1; inferior frontal gyrus (IFG); L: [−44 20 0], Z = 3.99, Cluster 2) and bilateral caudate (L: [−8 8 2], Z = 4.16; R: [10 12 4], Z = 4.71; both Cluster 1). Increased activation in the right amygdala ([28 0 −16], Z = 2.36; Cluster 1), and anterior cingulate gyrus ([4 6 38], Z = 4.40; Cluster 1) were observed as well. Due to the auditory nature of the CSs in this study, greater activation in bilateral Heschl’s gyrus (i.e. cortical center of primary auditory cortex; L: [−40 −22 8], Z = 3.17, Cluster 3; R: [46 −22 12], Z = 4.23, Cluster 2) was found for the CS+ tone than for the CS− tone. Confirming the success of the arousal manipulation via fear conditioning in this study, CS+ trials yielded greater SCRs than that of CS− trials, t(18) = 2.20, P < 0.05 (Figure 2D).

Table 1.

Whole-brain significant clusters and locations of ‘local maxima’ during the fear-conditioning session

					MNI
k	Cluster	Regions of local maxima peak	Z	H	x	y	z
CS+ > CS−
21 111	1	Frontal orbital cortex	5.34	R	36	22	−8
		Frontal operculum cortex/IFG	5.01	R	48	18	−2
			4.80	R	44	20	4
		Insular	4.93	R	34	24	4
		Caudate	4.71	R	10	12	4
		Superior parietal lobule	4.52	R	30	−44	68
1587	2	Insular	5.03	L	−30	22	−6
			3.68	L	−36	8	4
			3.26	L	−38	10	−6
		Frontal operculum cortex	3.99	L	−44	20	0
		Temporal pole	3.57	L	−58	6	−2
		Precentral gyrus	3.27	L	−58	−2	10
899	3	Supra marginal gyrus, anterior	3.54	L	−68	−28	18
		Supra marginal gyrus, posterior	3.49	L	−64	−44	18
			3.13	L	−66	−44	24
		Heschl's gyrus	3.17	L	−40	−22	8
		Parietal operculum cortex	3.11	L	−48	−32	18
		Planum polare	2.92	L	−40	−20	−4
714	4	Lingual gyrus	3.18	L	0	−72	−4
			3.14	L	−4	−72	−4
			3.11	L	−12	−78	−8
			3.04	R	10	−76	−8
			3.01	R	8	−70	−6
		Cerebellum	3.13	L	−4	−64	−14
CS− > CS+
1391	1	Superior frontal gyrus	4.1	L	−26	24	56
			3.65	L	−22	22	46
			3.53	L	−22	32	52
			3.31	L	−16	34	44
		Cerebral white matter	3.25	L	−20	26	6
		Frontal pole	3.07	L	−20	38	56
1099	2	Temporal fusiform cortex, posterior	3.98	L	−34	−40	−14
		Sub-gyral	3.53	L	−30	−40	2
		Cerebral white matter	3.21	L	−22	−28	24
			3.1	L	−18	−44	12
		Inferior temporal gyrus	3.09	L	−46	−52	−12
		Lingual gyrus	3.04	L	−28	−50	6

k, number of voxels; L, left; R, right; H, hemisphere.

Dot-probe task results

Reaction times

The RTs from error trials (0.04%) or those with more than 2.5 s.d. above or below each participant’s mean were removed (0.02%) before obtaining the mean RTs for each condition for each participant. A repeated-measures ANOVA (2 Arousal Condition × 2 Trial Type) revealed a main effect of Trial Type, F(1,19) = 18.98, P < 0.001, reflecting faster RTs in salient-location-target (370.49 ms) than non-salient-location-target trials (388.35 ms), and also a main effect of Arousal Condition, F(1,19) = 5.34, P < 0.05, reflecting faster RTs in arousing (i.e. CS+; 375.84 ms) than non-arousing trials (i.e. CS−; 383.00 ms) (Figure 2E). A planned pairwise comparison revealed that the facilitation in RTs during the CS+ salient-location-target trials was significant (P < 0.05). Overall, participants were faster to respond to the target dot when it appeared in the location of the more salient face cue and this detection advantage was greater for trials following CS+ than for trials following CS− tones. There was no significant difference between CS+ and CS− trials when the target appeared in the location of the less-salient place cue. However, the arousal-by-trial type interaction did not reach statistical significance in the fMRI experiment.

ROI analysis results

To probe how emotional arousal interacted with stimulus saliency, brain activity during the CS+ and CS− trials was quantified within a set of ROIs in the PPA and FFA. These ROIs were individually defined based on localizer run results. Percent signal changes extracted from these ROI masks in the left and right hemispheres indicated that CS+ trials led to stronger FFA activation (i.e. response to the salient face cue) than did the CS− trials. Simultaneously, CS+ trials led to decreased PPA activation (i.e. response to the non-salient place cue) compared with CS− trials. This pattern was confirmed by a 2 Arousal Condition (CS+, CS−) × 2 Region (FFA, PPA) × 2 Hemisphere (left, right) repeated-measures ANOVA, which revealed a significant Arousal Condition × Region × Hemisphere, F(1,18) = 5.20, P < 0.05, and Arousal Condition × Region interaction effect, F(1,18) = 10.36, P < 0.005, and a main effect of Region, F(1,18) = 81.74, P < 0.001. To further examine the three-way interaction, we conducted a repeated-measure ANOVA with Arousal Condition (CS+, CS−) × 2 Region (FFA, PPA) separately for each hemisphere. In the left hemisphere, it revealed a main effect of Region, F(1,19) = 52.50, P < 0.001, and a significant cross-over interaction, F(1,19) = 11.24, P < 0.005, indicating that the effect of the arousal-by-trial type interaction differed for salient and non-salient stimuli; subsequent pairwise comparisons showed that there was increased activation in left FFA in the CS+ compared with the CS− trials (P < 0.05), and decreased activation in left PPA in the CS+ compared with the CS− trials (P < 0.05). That is, the results showed that emotional arousal both increases brain activity associated with the salient stimuli (i.e. face stimuli in this case) and decreases brain activity associated with the non-salient stimuli (i.e. place stimuli; Figure 3A). Brain responses from the right hemisphere also showed a main effect of region, F(1,18) = 33.99, P < 0.001. A similar cross-over pattern of interaction was observed that was not quite significant (P = 0.087). Therefore, our follow-up analyses below focused on the left hemisphere.

Fig. 3.

(A) ROI results in both FFA as a response area for salient face cues and PPA for non-salient place cues. (B) Functional connectivity results. The amygdala region (red arrow) showed greater positive functional connectivity with FFA during CS+ trials than during CS− trials. The lowered threshold map showed greater positive functional connectivity with FFA during CS+ than CS− trials in brainstem regions including in the region of LC (green arrow) and a greater negative functional connectivity in left PPA region (blue arrows). *P < 0.05, **P < 0.005, ^†P = 0.087.

Functional connectivity and trial-by-trial relationship between brain response and RTs

The whole-brain connectivity analysis comparing the CS+ trials and CS− trials revealed that the left FFA had greater positive functional connectivity with the right amygdala in the CS+ trials than in the CS− trials (Figure 3B; Table 2). To provide additional information about overall connectivity, we examined a differential correlation map (CS+ > CS−) of the left FFA with a lowered threshold (i.e. Z = 2.3, uncorrected). Although this low-threshold map should be interpreted with caution, we found several interesting patterns. First, in the CS+ trials compared with the CS− trials, the left FFA showed a greater positive functional connectivity with brainstem regions including a region consistent with the location of the locus coeruleus (LC), known for its broad range of modulatory role in emotion, memory and attention processing, as well as its role in modulating arousal (for a review, see Sara, 2009). We also found greater negative functional connectivity between the left FFA and left PPA in CS+ than in CS− trials (Z = 1.64, uncorrected). Thus, enhanced processing for salient face cues under arousal was associated with increased activity of the amygdala and a region in the approximate location of LC as well as stronger inhibition of non-salient place cues.

Table 2.

Brain regions showing connectivity with the FFA seed region during the dot-probe session

CS+ > CS−			MNI
Regions	Z	H	x	y	z
Thalamus	3.83	R	16	−26	−2
Hippocampus	3.49	R	32	−16	−14
Amygdala	3.35	R	26	−12	−12
Putamen	2.79	R	30	−10	−8
Heschl’s gyrus	2.31	R	38	−24	12

L, left; R, right; H, hemisphere.

To examine the relationship between brain responses in category-specific regions (i.e. increased FFA and decreased PPA by emotional arousal) and speed of processing stimuli in the location of the corresponding face or place, a hierarchical linear modeling (HLM) analysis was conducted, treating each trial as a level-1 unit and each participant as a level-2 unit. A separate analysis was carried out for salient-location-target and non-salient-location-target trials. In both analyses, predictor variables included percent signal changes extracted from left FFA and left PPA signals, respectively, arousal condition (CS+ or CS−), an interaction between FFA and the arousal condition, and an interaction between PPA and the arousal condition for each trial. Trial RTs were used as the dependent variable. This HLM analysis on salient-location-target trials revealed significant effects of FFA, indicating that as FFA activity increased, RTs for the targets shown in the face location speeded up. Furthermore, there was a significant interaction between the arousal condition and FFA activity (Table 3), reflecting that greater activation in FFA was more strongly associated with faster RTs for the face-location targets in CS+ trials than in CS− trials. With a marginal significance level (P = 0.09), the analysis also revealed an interaction between the arousal condition and PPA activity, indicating that the stronger PPA activity led to slower RTs for the face-location targets under arousal. A similar analysis on non-salient-location-target trials did not reveal any significant results.

Table 3.

Results from hierarchical linear regression analyses; signal estimates in each region (FFA and PPA) and arousal condition (CS+ and CS−) were the predictors; RTs for salient-location-target trials were the dependent variable

Model (predictor)	Beta	s.e.	t
FFA	−9.55	4.50	−2.12*
PPA	6.12	8.94	0.69
Arousal Condition (CS−, CS+)	−3.54	4.68	−0.76
FFA × Arousal Condition	−13.39	3.58	−3.74**
PPA × Arousal Condition	9.51	5.38	1.77†

**P < 0.005; *P < 0.05; ^†P = 0.09. Note that arousal conditions were coded as 1 (CS+) and −1 (CS−), and thus the negative beta value indicates that RTs are faster during the CS+ than CS− and vice versa.

Arousal amplified weighted attention to saliency

To probe how attentional weighting to the salient cue interacts with emotional arousal, brain activity during the CS+ and CS− trials was examined within right and left IPS ROIs, regions associated with attentional orienting to contralateral salient stimuli. A repeated-measure 2 Arousal Condition (CS+, CS−) × 2 Saliency Location (left, right) ANOVA on the percent signal changes from the right IPS ROI mask revealed a significant cross-over interaction, F(1,18) = 4.78, P < 0.05, on the right IPS; subsequent pairwise comparisons showed significantly greater right IPS responses in the CS+ condition than in the CS− condition when the salient cue was presented on the left side (P < 0.05). A similar analysis on the left IPS revealed a main effect of Arousal Condition, F(1,13) = 7.55, P < 0.05; and a marginally significant interaction, F(1,13) = 3.50, P = 0.084; a subsequent pairwise comparison revealed significantly greater left IPS response in the CS+ condition than in the CS− condition when the salient cue was presented on the right side (P < 0.05; Figure 4A). These results indicated that arousal amplifies the effects of saliency even beyond category-specific regions such as PPA and FFA.

Fig. 4.

(A) ROI results in IPS in response to right vs left salient cues in each hemisphere. (B) Scatter plot of difference values in percentage signal change (CS+ minus CS−) illustrating the relationship between IPS and FFA/PPA regions (right). Gray-shaded area indicates 95% bootstrapped CIs. Outlier data (rectangles) were corrected using the robust method (Wilcox, 2012) when the correlation was calculated. *P < 0.05, ^††P = 0.084.

Additional correlation analyses were conducted to explore how the weighted attentional processing to salient stimuli can influence processing in both FFA and PPA regions. In this analysis, signal change values in IPS to salient cues (i.e. estimates in left IPS when the salient cue was presented on the right side and in right IPS when the salient cue was on the left side) were combined across right and left IPS, and signal change differences were calculated for FFA, PPA and IPS regions by subtracting the percentage signal change value for CS− from that of CS+. The robust method was used to correct for outliers (Wilcox, 2012) via the robust correlation toolbox (Pernet et al., 2012). A significant positive correlation between IPS and FFA regions was identified, r = 0.48, P < 0.05, indicating that increased attentional processing of salient cues compared with non-salient cues in IPS was associated with increased FFA activation. Simultaneously, a significant negative correlation between the IPS and PPA regions, r = −0.61, P < 0.05, indicated that increased attentional prioritization of salient cues is associated with reduced processing of non-salient cues in their associated representational region (Figure 4B).

DISCUSSION

Previous neuroimaging research on how arousal influences perceptual processing demonstrated ways in which arousal enhances sensory processing and attention (Lim et al., 2009; Brooks et al., 2012; Ethofer et al., 2012) but ignored the possibility that arousal can also impair perceptual processing, despite evidence from behavioral research showing that arousal has both enhancing and impairing effects (Mather and Sutherland, 2011). In the current fMRI study, we used a simple dot-probe task to test the hypothesis that arousal amplifies the effects of competition among salient and non-salient stimuli in perception, enhancing processing of salient stimuli while impairing processing of non-salient stimuli. During the task, on each trial we presented one face and one place image simultaneously as cue stimuli, and gave a brief luminance increase in the face cue’s location to enhance its perceptual saliency. As predicted by ABC theory, there was an arousal-by-saliency interaction in the FFA and PPA. On arousing compared with non-arousing trials (i.e. fear-conditioned tone vs neutral tone trials), responses in FFA (i.e. responses to the salient face cue) were enhanced and responses in PPA (i.e. responses to non-salient place cues) were reduced (Figure 3A). These findings indicate that arousal and saliency interact to determine the strength of visual processing, an advance on previous studies, which typically had an ‘enhancement-only’ perspective on the effects of arousal on perception and did not consider how arousal might also impair processing.

We also found that responses in IPS were more enhanced for the salient cue (or its location) on arousing trials than non-arousing ones, indicating that attentional engagement to the salient cue location was also augmented by the arousal–saliency interaction (Figure 4A). Furthermore, IPS activity showed a significant positive relationship with the FFA under arousal, whereas there was a negative relationship with the PPA (Figure 4B). IPS plays a key role in attentional weighting toward prioritized external stimuli (Geng et al., 2006; Konen and Kastner, 2008; Szczepanski et al., 2010; Chica et al., 2011) regardless of stimulus content (Talmi et al., 2008). Our findings suggest that IPS’s attentional weighting role increased during arousing trials, enhancing prioritized stimuli at the cost of non-prioritized stimuli. This is consistent with previous research indicating a synergistic role of attention and arousal in enhancing perception of salient stimuli (Bermpohl et al., 2006; Phelps et al., 2006; Lim et al., 2009; Mohanty et al., 2009). Indeed, Lim et al. (2009) found that the amygdala influences successive visual perception by mediating the frontoparietal attention network. Similarly, we found that CS+ trials induced greater attentional network activity than did CS− trials in the group-level analysis of our dot-probe task (see Supplementary Figures S1 and S2). Collectively, these findings from our study as well as previous research indicate that emotionally arousing stimuli enhance subsequent perceptual processing of prioritized external stimuli.

At the behavioral level, participants were faster to respond to the target dot when it appeared in the same location of salient face cue and slower to respond to the target when it appeared in the difference location of salient cue. As predicted by the ABC model, in our initial behavioral study, both salient-location targets were responded to faster and non-salient-location targets were responded to slower on arousing trials. However, in the fMRI experiment, the arousal-by-salience interaction seen in the behavioral study (see Figure 2B) did not achieve significance although we did see a significant speeding of responses to the salient-location trials and not to the non-salient-location trials (Figure 2E). The lack of replication of the interaction may be due to the smaller number of participants in the fMRI experiment or to some other difference resulting from the scanning environment. However, the HLM analyses indicate that both increased FFA and decreased PPA activities during CS+ trials relative to CS− trials (i.e. the brain-based arousal–saliency interaction effect) were associated with faster RTs for detecting the target in the salient cue location during the fMRI experiment. In sum, these relationships further support the idea that arousal–saliency interactions subserve the final behavioral outcome.

Although this study only tested the situation where there is a strong salience difference between stimuli, previous research on the effects of salience on attention suggests that the degree of difference in salience matters (Itti and Koch, 2000). In the scenario tested in this study, there were large differences between salient and non-salient stimuli, leading the salient stimulus representation to suppress other representations and gain further advantage during the competition among representations. In contrast, when there are multiple stimuli with similar salience competing for representation, saliency map model of Itti and Koch (2000) predicts that they will mutually suppress each other. Thus, when multiple stimuli with similar salience compete, the competition should decrease activation for all of the stimuli and arousal should further suppress activity of those representations. Although we only tested the situation where there is a strong salience difference between stimuli in this study, in a previous behavioral study we examined arousal effects on perceptual learning of both salient and non-salient targets (Lee et al., 2012). There we found that emotional arousal enhanced perceptual learning of salient targets but suppressed perceptual learning of targets that were perceptually very similar to distractors. Thus, as predicted by the ABC model, arousal amplified the effects of competition in both cases.

Consistent with a large body of research implicating the amygdala in emotional processing (Anderson et al., 2003; Mather et al., 2004; LaBar and Cabeza, 2006; Phelps, 2006), the amygdala showed greater functional connectivity with the FFA during CS+ trials than during CS− trials, suggesting that emotional processing evoked by fear-conditioned tones was involved in the emotional–saliency interactive processes in visual perception. Although it should be interpreted with caution because of the limited resolution (slice thickness = 2.5 mm) of the current scan protocol and a lenient threshold (Z= 1.64, uncorrected), we also identified that a region consistent with the location of the LC, known for its modulating role in arousal and attention (Sara, 2009), also showed more coordinated activity with the FFA during arousing than during non-arousing trials (Figure 3B). These results indicate that emotional arousal influences subsequent visual processing to amplify competitive processes that are biased in favor of high-priority stimuli at the expense of low-priority stimuli. The cell bodies of noradrenergic neurons within the LC have widely distributed, ascending projections to the forebrain regions including the amygdala (Luppi et al., 1995; Berntson et al., 2003). Thus, LC may help trigger involvement of the amygdala in shaping successive visual processing. Alternatively, released norepinephrine from LC may directly influence visual cortex. According to previous studies (Berridge and Waterhouse, 2003), norepinephrine can simultaneously increase both excitatory and inhibitory components of visual cortex neuronal responses. In particular, previous animal research demonstrates that norepinephrine can change visual perception by altering receptive field properties such as direction selectivity, velocity tuning and response threshold (McLean and Waterhouse, 1994). Thus, it is also possible that norepinephrine release elicited by an arousing sound modulates visual processing as a function of stimulus priority.

Although we induced arousal using negatively arousing stimuli as negative stimuli generally induce stronger arousal responses than those of positive stimuli (e.g. Lang et al., 1998), previous research reveals that highly arousing positive and negative stimuli affect perception in similar ways; for instance, like negative arousing pictures, erotic pictures impair perception of visual targets (Most et al., 2007). Recent findings also reveal similar ABC effects on long-term memory when arousal is induced by intense positive pictures as when it is induced by intense negative pictures (Sakaki et al., 2013). However, future research is needed to test whether positive arousing stimuli play the same role as negative arousing stimuli in the biased-competition processes seen in this dot-probe paradigm.

Although there has been little focus on the notion that emotional arousal can impair, as well as enhance, perceptual processing in the brain, in the behavioral literature the idea that emotional arousal leads to trade-offs in attention and memory has received much more investigation. Here, previous research has focused on the trade-off between emotionally arousing foreground objects and neutral background information (Christianson et al., 1991; Waring and Kensinger, 2011). For instance, better memory for a foreground gun comes at the cost of background details of the scene (Fawcett et al., 2013). However, the use of paradigms where arousal is induced by the same stimulus that is used to assess attention or memory suffers from the potential for confounding factors. Because guns and other arousing objects differ perceptually from the comparison neutral objects, it is impossible to know whether there are some other perceptual or conceptual features of the emotionally arousing objects that lead to the trade-off effects, rather than the emotional arousal in itself. Also, there is a long-standing debate about whether all of the memory trade-off effects can be accounted for by the attention-grabbing nature of the emotionally arousing objects or not (Christianson et al., 1991; Mitchell et al., 1998; Steinmetz and Kensinger, 2013). The advantage of our approach is that we separate out the source of the arousal and the target items. The salient items are perceptually identical on arousing and non-arousing trials, allowing us to attribute any differences to the arousal rather than to the perceptual qualities of the salient item itself. Our results indicate that emotional arousal induced by one stimulus can influence the competitive processes engaged by other stimuli, such that processing impairments are seen as well as enhancements.

SUPPLEMENTARY DATA

Supplementary data are available at SCAN online.