An Affective Circumplex Model of Neural Systems Subserving Valence, Arousal, & Cognitive Overlay During the Appraisal of Emotional Faces

Abstract

Increasing evidence supports the existence of distinct neural systems that subserve two dimensions of affect – arousal and valence. Ten adult participants underwent functional magnetic resonance imaging during which they were presented a range of standardized faces and then asked, during the scan, to rate the emotional expressions of the faces along the dimensions of arousal and valence. Lower ratings of arousal accompanied greater activity in the amygdala complex, cerebellum, dorsal pons, and right medial prefrontal cortex. More negative ratings of valence accompanied greater activity in the dorsal anterior cingulate and parietal cortices. Extreme ratings of valence (highly positive and highly negative ratings) accompanied greater activity in the temporal cortex and fusiform gyrus. Building on an empirical literature which suggests that the amygdala serves as a salience and ambiguity detector, we interpret our findings as showing that a face rated has having low arousal is more ambiguous and a face rated as having extreme valence is more personally salient. This explains how both low arousal and extreme valence lead to greater activation of an ambiguity/salience system subserved by the amygdala, cerebellum, and dorsal pons. In addition, the right medial prefrontal cortex appears to down-regulate individual ratings of arousal, whereas the fusiform and related temporal cortices seem to up-regulate individual assessments of extreme valence when individual ratings are studied relative to group reference ratings for each stimulus. The simultaneous assessment of the effects of arousal and valence proved essential for the identification of neural systems contributing to the processing of emotional faces.

INTRODUCTION

The vast majority of imaging studies of affect thus far have compared brain responses to a wide range of emotional stimuli with responses to more “neutral” stimuli. Although these studies have tended to confirm hypotheses about the regions that compose the “emotional brain” (including the amygdala, anterior cingulate [ACC], medial prefrontal cortex [mPFC], and fusiform gyrus), those studies have failed to provide a more comprehensive model for how those regions represent, process, and generate different emotions from among the full range of possible emotions. The findings from these studies have instead been interpreted as identifying distinct and dissociable circuits subserving one of several basic emotions, such as the amygdala for fear (Davis & Whalen, 2001; LeDoux, 2003), the subgenual ACC for sadness (Beauregard et al., 1998; Reiman et al., 1997), the insula for disgust (Lane, Reiman, Ahern, Schwartz, & Davidson, 1997; Sprengelmeyer, Rausch, Eysel, & Przuntek, 1998), and the ventral striatum for happiness (Davidson & Irwin, 1999; Lane, Chua, & Dolan, 1999; Whalen, Bush et al., 1998). Moreover, the brain regions identified in studies of these basic emotions have been far from consistent (Barrett & Wager, 2006), with even the most reproducible finding, that of fear activating the amygdala, appearing in only 60% of studies in one meta-analysis (Phan, Wager, Taylor, & Liberzon, 2002), and in fewer than 40% of studies in another (Murphy, Nimmo-Smith, & Lawrence, 2003).

Meanwhile, a separate model of affect emerged principally from studies of participant ratings of the degree of similarity between emotions. In this circumplex model of core affects, emotions were thought to be classified along two independent dimensions -- that of arousal (the extent to which an emotion is associated with an individual sensation of energy) and valence (the extent to which an emotion reflects a negative or positive state of mind, which has been variously framed as reflecting either approach or withdrawal, or attraction or aversion, respectively -- see Fig. 1) (Barrett, Mesquita, Ochsner, & Gross, 2007; Russell, 1980, 2003; Russell & Bullock, 1985). This model suggests that distinct neural pathways subserve not distinct emotions, but each of these two underlying neurophysiological dimensions of the affective circumplex (J. Posner, Russell, & Peterson, 2005). The circumplex model further proposes that our own subjective experience of emotion is a cognitive interpretation of the neurophysiological experience of valence and arousal in a given situational context (Russell, 2003).

Figure 1.

The affective circumplex depicts each emotion along continuous dimensions of arousal (y-axis) and valence (x-axis) (Feldman, Barrett, & Russell, 1998).

Neuroimaging studies have suggested that activation in the amygdala, cerebellum, pons, insula, thalamus, and medial and inferior temporal cortex (especially in the fusiform gyrus) correlates positively with intensity of the emotion depicted in the stimulus (Anders, Lotze, Erb, Grodd, & Birbaumer, 2004; Anderson, Christoff, Stappen et al., 2003; Junghofer et al., 2006; Lewis, Critchley, Rotshtein, & Dolan, 2007; Small et al., 2003; Surguladze et al., 2003). Some have interpreted this as meaning that these regions encode the “arousal” of the emotion, as suggested by the circumplex model (J. Posner et al., 2005). However, these studies fail to make clear whether intensity corresponds most closely with the “arousal” attributed to the emotional stimulus or with the extremes of positive and negative valence attributed to the stimulus.

Much research has also aimed to distinguish brain regions that subserve emotions with predominantly positive versus negative emotional valence. Lesion studies in animals have implicated a role for the amygdala in learned fear and possibly in other negatively-valenced affects (Anglada-Figueroa & Quirk, 2005; Davis, 1997; LeDoux, 2003). Meta-analyses of early imaging studies have suggested that the amygdala is more active during the processing of unpleasant emotions, although frequently negatively valenced emotion and “salience” (i.e., the degree to which a stimulus attracts or biases attentional resources) were confounded in these studies (Kastner & Ungerleider, 2000; Murphy et al., 2003; Sarter, Givens, & Bruno, 2001; Wager et al., in press). More recent studies have shown that an intact amygdala is not necessary for the experience of, or subjective feelings that attend, negatively valenced emotions, including fear (Davis & Whalen, 2001; Walker & Davis, 1997; Wallace & Rosen, 2001); increasing evidence has suggested instead that the amygdala subserves the representation of salience for emotional stimuli, be they either positively or negatively valenced (Adolphs et al., 2005; Amaral, 2003; Anderson, 2005, 2007; Barrett & Niedenthal, 2004; Davis & Whalen, 2001; Kim, Somerville, Johnstone, Alexander, & Whalen, 2003; Liberzon, Phan, Decker, & Taylor, 2003; Vuilleumier & Pourtois, 2007; Whalen, 1998). Indeed, single or small groups of neurons that encode the perception of only positively or negatively valenced stimuli have been identified within the amygdalae of non-human primates (Paton, Belova, Morrison, & Salzman, 2006).

Only a handful of studies to date have studied the simultaneous influence of arousal and valence on brain activity, and they have produced inconsistent results (Anders et al., 2004; Kensinger & Corkin, 2004; Small et al., 2003). These studies agree that separate circuits subserve arousal and valence, but they have all been limited by the use of a small range of emotional stimuli, thereby failing to identify reliably a consistent set of brain regions underlying these circuits. Although assessed less frequently, the extremes of positive or negative emotional valence (which can be computed as the absolute value of valence ratings) have produced the most consistent activation of brain regions in imaging studies, including the lateral OFC, ACC, operculum, and temporal and visual cortices (Lewis et al., 2007; Phan et al., 2002).

Several studies have demonstrated that the attentional demands of the emotion processing task influence the pattern of regional brain activity, particularly the extent to which the amygdala and fusiform gyrus activate (Anderson, Christoff, Panitz, De Rosa, & Gabrieli, 2003; Hopfinger, Buchel, Holmes, & Friston, 2000; Pessoa, McKenna, Gutierrez, & Ungerleider, 2002; Vuilleumier, Armony, Driver, & Dolan, 2001; Wager et al., in press). There has been a trend in neuroimaging and electrophysiological studies of emotion towards using “orthogonal” or “indirect” tasks in which subjects are not asked to perform an on-line task relevant to the emotional content of specific stimuli in an attempt to avoid a confound with attention (Carretie, Hinojosa, Lopez-Martin, & Tapia, 2007; Carretie, Iglesias, Garcia, & Ballesteros, 1997). While this experimental design is useful for studying the neurobiological correlates of passive viewing, particularly when used with reference or off-line ratings of these stimuli, it does not provide information about the brain bases of on-line subjective experiencing and processing of emotion, as we have set out to do in this study. Including an on-line rating task allows us to study real-time experiencing and rating of emotion. Attention is an important consideration in both designs, as the nature of the task affects the extent to which subjects are motivated (in or out of awareness) to attend to different types of stimuli. In keeping with the guiding principle of the affective circumplex, we conceptualize the affective response as being as much a product of the viewer and his/her cognitive overlay as it is a reflection of the “true” nature of the affective stimulus. Therefore, we think of attention as being part of the affective response, as opposed to a confound that must be eliminated.

Interestingly, prior research has shown that tasks requiring a substantial allocation of attentional resources can suppress rather than enhance activity in the amygdala and fusiform gyrus (Pessoa et al., 2002), even when the task involves the explicit labeling of affect (Critchley et al., 2000; Lange et al., 2003; Surguladze et al., 2003). Such tasks instead typically increase activity in the middle temporal cortex and PFC. We believe that these tasks encourage more “top-down” control of subcortical limbic responses, leading to reduced activity in these areas. In addition, we hypothesize that increased attention to ambiguous (i.e., low arousal) stimuli counterbalances the absence of strong positive or negative affect (i.e., extreme valence) in most of these stimuli, preventing the relationship between subcortical activity and affective rating from reaching significance. We hope to better describe the nature of this phenomenon by capturing on-line ratings of both arousal and valence, and disentangling their simultaneous influence on brain activity.

Herein we explore the neural processes required for the evaluation and classification of facial affects. Participants classified affects depicted in facial stimuli along the independent dimensions of valence and arousal. We identified brain regions in which imaging-based indices of neural activity varied systematically and parametrically with ratings of valence and arousal that were provided by each participant on-line during the scan, as well as with average ratings of valence and arousal of the same stimuli provided by a large reference group. These two analyses were intended to help disentangle the neurophysiological systems that subserve the universal experiences of valence and arousal from the neural systems that caused participant ratings to deviate from the reference ratings, systems that we presume contributed to the regulation and modification of the subjective experiences of valence and arousal during the labeling of facial affects.

METHODS

Subjects

The procedures of this study were approved by the Institutional Review Board of the New York State Psychiatric Institute. Subjects were recruited through advertisements in New York City. All subjects provided informed written consent and were paid for their participation. The Structured Clinical Interview for DSM-IV (Endicott & Spitzer, 1978) was administered by trained, reliable raters to exclude participants who met DSM-IV criteria for current Axis I disorder or who had a lifetime history of psychotic or substance abuse disorder (APA, 1994). Additional exclusion criteria included a history of head trauma, seizure disorder, or other neurological disorders.

The 10 participants were 5 men and 5 women, ages 19 to 34 (mean = 25, SD = 4.5). All were Caucasian, native English speakers, and right-handed. Reliable raters administered to each subject the Wechsler Abbreviated Scale of Intelligence. Subjects were of average to above-average intelligence (mean full scale IQ = 112.4, SD = 13.7) and were from households of middle-to-high socioeconomic status. None were taking psychotropic medications.

Task Construction

This experiment made use of a parametric experimental design to identify brain regions in which neural activity correlated with ratings of stimuli along the two dimensions, arousal and valence. Neural activity was indexed using the magnitude of fMRI blood oxygen level dependent (BOLD) response as participants viewed photographs of emotional faces. After viewing each face, subjects simultaneously rated arousal and valence by selecting an individual box on a 9 by 9 two-dimensional grid. Location on the x-axis indicated the participants rating of valence (left = negative valence, right = positive valence), and location on the y-axis indicated the rating of arousal (top = high arousal, bottom = low arousal). We recorded the selected box as two integer scores, each ranging from −4 to +4, representing valence and arousal.

Stimuli

Each trial consisted of 3 components presented in succession: (1) Visual presentation of a photograph of a human face for 18 seconds. The photographs were copied, with permission, from the 20 distinct stimuli used by Russell and Bullock (1985) for their studies of the affective circumplex. Thirteen of these 20 images were taken from Ekman’s (1976) Pictures of Facial Affect and depicted expressions of a number of emotions (two pictures each of emotional faces commonly classified as expressing happiness, surprise, fear, anger, disgust, or sadness, and one commonly classified as neutral). Russell and Bullock supplemented this set to represent better portions of the circumplex that the Ekman series under-sampled (emotions associated with low arousal but positive or neutral valence). These included two photographs each of actors and actresses expressing boredom, contentment, and sleepiness, as well as one expressing excitement. (2) Visual presentation of a two-dimensional grid on which participants indicated their ratings of arousal and valence for each stimulus by moving an arrow controlled by an MRI-compatible computer mouse. This screen remained visible until the subject clicked the mouse button, up to a maximum of 20 seconds. (3) Visual presentation of a fixation point (+) at the center of the subject’s visual field. The fixation point was displayed immediately following the rating of valence and arousal. The durations of rating and gaze fixation were each variable, but when summed together always equaled 20 seconds. One imaging run consisted of 20 trials presented in a pseudorandom order (but uniform from subject to subject).

Visual stimuli were presented to the subject via MRI-compatible LCD goggles (Resonance Technology, Northridge, CA) using E-Prime software, version 1.1 (Schneider, Eschman, & Zuccolotto, 2002) running on a Dell IBM-compatible computer. Measures of stimulus durations and reaction times were accurate to 20 milliseconds. Stimuli were presented at the center of the subject’s visual field, subtending 19 degrees of vertical and 15 degrees of horizontal visual field.

Instructions

Prior to the scanning session, each subject was instructed in the performance of the task and use of the computer mouse by practicing a shortened version outside of the scanning environment on a desktop computer with verbal instruction from the experimenter. Each subject was told, “You will be shown a face that expresses a certain feeling. You will be asked to assess the feeling on the chart shown below… On the chart, the vertical dimension represents degree of arousal. Arousal has to do with how awake, alert, or energetic a person is… The right half of the chart represents pleasant feelings -- the farther to the right, the more pleasant. The left half represents unpleasant feelings -- the farther to the left, the more unpleasant… During the experiment, you will first be shown a face. This will appear on the screen for 15 seconds. Then you will be shown the grid. When the grid appears, you will click on the area you think best describes the face… Try to think about the feeling expressed by the face during the 15 seconds that it is shown. It will not be on the screen when you are shown the grid.” At the time of instruction and during the experiment itself, the words “High Pleasure” appeared to the right of the grid, and “High Energy” above the grid. The shortened practice version consisted of three faces – one each expressing sadness, happiness, and anger. To minimize the possibility of habituation, none of the practice faces were identical to actual experimental stimuli.

Reference Versus Individual Ratings

In order to explore both the neurobiological basis of automatic emotional reaction, as well as cognitive-emotional “overlay” (i.e., emotional processing and modulation) we consider the properties of the emotional stimulus from two perspectives. First, because automatic emotional reactions are rapid and universal, we capture this response (the “reference” response) using average ratings of emotional arousal and valence from a large group of participants as reported in Russell and Bullock (Fig. 2, 1985). Second, since emotional modulation takes place in a highly individual and subjective way, we use the individual’s on-line rating of a stimulus’s arousal and valence to study this part of the emotion system (Ochsner, Bunge, Gross, & Gabrieli, 2002).

Figure 2.

Mean individual and reference circumplex ratings are highly, though not completely, correlated. Subjective ratings (diamonds) are mean values from the 10 subjects in the current experiment. Error bars represent standard errors. Reference ratings (stars) are mean values using the same stimuli as reported by Russell & Bullock, 1985 on a sample of 300 adults (15 per photograph).

Image Acquisition

Imaging was performed on a GE Signa 3T whole body scanner (Milwaukee, WI) using a GE single channel quadrature head coil for both transmitting and receiving radiofrequency signal. Head positioning in the magnet was standardized using the canthomeatal line. A T1-weighted sagittal localizing image was used to position axial functional images parallel to the anterior commissure – posterior commissure (AC-PC) line. In all subjects, a 3D spoiled gradient recall (SPGR) image was acquired for coregistration with axial functional images and for coregistration with a standard reference image (Montreal Neurological Institute). Functional images were acquired using a single shot gradient echo planar (EPI) pulse sequence in groups of 43 axial slices per volume and 273 volumes per run (preceded by 6 “dummy” volumes to ensure scanner stability). Parameters for the EPI images were: Repetition Time (T_R) =2800 ms, Echo Time (T_E) =25 ms, flip angle=90°, acquisition matrix= 64×64, field of view (FOV)=24×24 cm, slice thickness=3 mm, skip=0.5 mm, receiver bandwidth=62.5 kHz, in-plane resolution=3.75 mm ×3.75 mm. Each run lasted 13:01 min:sec, for a total EPI scan time of 39:03 min:sec.

Image Preprocessing

Pre-processing was performed using functions from SPM2 (Wellcome, 2003) running under a Windows version of Matlab 6.5 (Mathworks, Inc., Natick, MA) on a Pentium III based Dell computer. Images were visually inspected at intermediate stages of preprocessing and at its conclusion to ensure the absence of ghosting or significant signal dropout. Pre-processing included: (1) Slice timing correction based on a linear interpolation algorithm applied to all functional images referenced to the middle slice (22 of 43) of each imaging volume. (2) Motion correction using an affine rigid-body alignment referenced to the middle imaging volume (138 of 273) of the run. (3) Spatial normalization of motion-corrected images to the standard template, which included (a) warping the high resolution, T1-weighted SPGR image of each subject to the MNI template (avg152T1, resolution=2×2×2 mm per voxel) with reslicing and then (b) warping each functional image to the subject’s own SPGR image. Warping was accomplished using a hybrid algorithm of affine transformation and nonlinear warping, following automated segmentation of the SPGR image. (4) Gaussian spatial filtering with a full-width, half maximum (FWHM) of 8 mm. (5) Band-pass temporal filtering of the fMRI time series that removed frequencies above 0.15 Hz (i.e., below a period of 6.67 sec) and below 0.078 Hz (i.e., above a period of 128 sec). We believe that subjects who differ in their mean behavioral ratings are likely to differ in the extent of BOLD activation in the areas we are studying. Therefore, grand mean scaling and intensity normalization were not applied to the image data because these would have eliminated such effects.

First-Level Parametric fMRI Analysis

We carried out first level parametric analyses individually for each subject using the general linear model in SPM2. We modeled preprocessed BOLD time series data at each voxel, concatenated from all three runs, using a core of 6 independent functions, in addition to either 3 or 6 additional “variables of interest” that were specific to individual analyses. The core functions consisted of (1) a boxcar function representing the 18 seconds when a facial stimulus was presented, convolved with a canonical hemodynamic response function (HRF), (2) a boxcar function representing a period when subjects rated the stimuli, convolved with the HRF, (3) a boxcar function representing a period of gaze fixation, convolved with the HRF, (4) a “dummy variable” marking run 1 (1 if run 1, otherwise 0), and (5) a dummy variable marking run 2 (1 if run 2, otherwise 0), and (6) a constant.

Variables of interest consisted of the stimulus boxcar/HRF function multiplied by (1) the subject’s arousal rating for each stimulus (henceforth, called the “individual” rating), (2) the individual valence rating for each stimulus, (3) the absolute value of the individual valence rating for each stimulus, (4) a reference arousal rating for each stimulus, (5) a reference valence rating for the stimulus, and (6) the absolute value of the reference rating for the valence of each stimulus. Associations between arousal and absolute valence (both for individual and reference ratings), valence and absolute valence, and the corresponding individual and reference ratings (for arousal, valence, and absolute valence) were predicted and confirmed in the experimental data (Table 1).

Table 1.

Pearson correlation coefficients between individual and reference ratings of arousal, valence, and absolute valence across entire sample show a significant association between arousal and absolute valence (individual and reference), and between corresponding individual and reference ratings. Each correlation summarizes 600 data points (10 subjects × 20 trials/run × 3 runs)

	Individual arousal	Individual valence	Individual absolute valence	Reference arousal	Reference valence	Reference absolute valence
Individual arousal	1	−0.03	0.34*	0.77*	0.10	0.33*
Individual valence		1	−0.22*	−0.25*	0.86*	−0.04
Individual absolute valence			1	0.36*	−0.02	0.57*
Reference arousal				1	−0.12	0.43*
Reference valence					1	0.06
Reference absolute valence						1

^*= p < 0.0001.

We discerned the independent contributions of arousal, valence, and absolute valence to the observed fMRI time series by including all three in a single first-level analysis – once each for the individual and reference ratings to detect effects common to models of both individual and reference ratings. We then included all six variables of interest in a single model to assess the independent contributions of arousal, valence, and absolute valence from the individual and reference ratings.

This design of these analyses followed the principles of parametric analyses previously described (Acton & Friston, 1998; Buchel, Holmes, Rees, & Friston, 1998; Frackowiak et al., 2003). We did not perform Euclidean normalization of the parametric variables (an SPM default), because doing so would have eliminated the inter-subject differences in mean ratings that we think should be reflected in the functional image data. We also did not orthogonalize the parametric variables (another SPM default), because the correlation between parametric variables was too small to threaten the stability of the GLM analysis (Table 1) and doing so would have made it hard to interpret the influence of the orthogonalized variable. For every subject, we used ordinary least squares (OLS) regression to estimate parameters for each of the predictor variables. We verified the assumptions of OLS regression for each of the first-level analyses (Luo & Nichols, 2003). Correlations between predictors were not a problem for interpretation of regression results because the variance inflation factor (VIF) did not exceed 3.9 even for a regression that included both individual and reference valence as predictors (a VIF over 10 is usually considered problematic; Kleinbaum, Kupper, & Muller, 1988). Given the large number of observations per subject (273 × 3 = 819 scans), the power to detect even small effect sizes (d = 0.2) for first level analyses was over 80%. Thus our within subject power is high, giving us an excellent chance of capturing important correlations between different dimensions of affect and neural activity and minimizing Type II error at the individual subject level.

Second-Level Analyses

Aggregating contrast images voxel-wise across subjects was accomplished using second-level analyses of random effects. We set a voxel-wise p-value threshold of p < 0.001 (i.e., T statistic > 4.3, for a sample of 10 subjects) conjointly with a minimum cluster size of 30 voxels to control for multiple comparisons and minimize Type I error (Forman et al., 1995). The sample size of 10 subjects yielded a power of 56% for detecting large effects (d = 1.70). Therefore, although our power to detect correlations within subjects is high, our power to detect significant correlations across subjects is only moderate and, consequently, we will be careful not to interpret the absence of findings as proving the null hypothesis in any second-level analyses.. Given that our approach to study affect is new and exploratory, we chose to maximize power at the individual subject level, even at the risk of failing to capture all of the significant group effects. We kept Type I error low, allowing us to conclude that any significant group level findings do reflect real and generalizable associations between affective properties and brain response.

RESULTS

Behavior

Between-subjects mean values for ratings of arousal and valence of each stimulus are presented in Figure 2. Correlations between individual and reference ratings of arousal, valence, and absolute valence are summarized in Table 1. Ratings for arousal and absolute valence correlated positively for both individual and reference ratings. Individual ratings of valence and absolute valence correlated inversely as a result of a slight overall negative bias for valence ratings in this sample (Fig. 2). No correlation was detected between reference ratings of valence and absolute valence, reflecting the more even distribution of valence ratings in the larger reference sample (Fig. 2). Corresponding individual and reference ratings all correlated positively with one another.

Imaging

Both individual and reference ratings of arousal correlated inversely with activation of the amygdala complex, thalamus, cerebellum, and a portion of the dorsal pons (Fig. 3a–b, Tables 1 and S1). Significant voxels in the cerebellum and dorsal pons appeared to be in the same cluster in second level analyses, but on visual inspection of the results of first level analyses, they separated into two distinct regions, both of whose activity were inversely associated with arousal. Individual arousal alone correlated inversely with activity in the right medial prefrontal cortex (mPFC, BA 10/32), including the anterior cingulate cortex (ACC; Fig. 3a–b).

Figure 3.

Brain regions showing significant BOLD correlations with: (a) Individual & reference arousal. Amygdala: Tal −24, −18, −14 & 22, −16, −14, 42 & 43 voxels, T=5.35 & 6.17. Thalamus: Tal 0, −17,6, 45 voxels, T=5.34. Dorsal pons: Tal 2, −42, −20, 435 voxels, T=7.11. (b) Individual arousal. R mPFC/ACC: Tal 18,54,1, BA 10/32, 31 voxels, T=5.18 (uncorrected voxelwise p < 0.001, cluster size > 30 voxels). (c) Individual & reference valence. Dorsal ACC: Tal −8,12,40, BA 24/6, 116 voxels, T=6.69. Inf parietal: Tal −44, −54,45, BA 40/7/39, 110 voxels, T=7.63. (d) Reference valence: L lat PFC: Tal −46,9,16, BA 44/45, 132 voxels, T=9.34 (uncorrected voxelwise p < 0.001, cluster size > 30 voxels). (e) Individual & reference absolute valence. Amygdala: Tal 28, −14, −8, 32 voxels, T=5.11. Dorsal pons: Tal −2, −42, −15, 149 voxels, T=7.07. (f) Individual absolute valence. Temporal: Tal −57, −39, −5, BA 19/21/37, 185 voxels, T=6.44 (uncorrected voxelwise p < 0.001, cluster size > 30 voxels).

Individual and reference ratings of valence correlated inversely with activity in the left parietal (BA 40/7/39) and dorsal anterior cingulate (dACC, BA 24/6) cortices. Reference valence ratings correlated inversely with activity in the left inferior frontal cortex (BA 44/45) (Fig. 3c–d).

Individual and reference ratings of absolute valence correlated positively with activity in the amygdala complex, cerebellum, and dorsal pons, in nearly the same locations as those that correlated inversely with arousal ratings. Activity in the fusiform gyrus and occipito-temporal junction correlated positively with individual ratings of absolute valence (Fig. 3e–f).

DISCUSSION

The findings of this study support our central hypothesis that ratings of arousal, valence, and absolute valence (i.e., the absolute value of the valence rating, reflecting extremes of valence in either direction) when viewing a facial expression are associated with activity in demonstrably distinct brain regions, and thus different functional pathways. Activity in the amygdala complex, cerebellum, and dorsal pons correlates positively with ratings of absolute valence and inversely with ratings of arousal. Thus for stimuli rated with high absolute valence together with high arousal, or else low absolute valence together with low arousal, the two effects tend to cancel one another, producing minimal activation in these regions. Activity in the thalamus is associated linearly with ratings of arousal, whereas activity in the dorsal ACC and parietal cortices are associated linearly with ratings of valence. Correlations of activity in these regions with both individual ratings (recorded by participants during the scan) and reference ratings (mean values drawn from a larger sample of subjects who viewed the same stimuli) of arousal and valence, suggest that these regions subserve core emotional processes, independent of our individual or subjective modifications of them.

Arousal

Our finding of an inverse correlation of arousal with activity in the amygdala complex, thalamus, cerebellum, and dorsal pons appears to contradict several studies that have reported opposite correlations in these regions (Anders et al., 2004; Anderson, Christoff, Stappen et al., 2003; Lewis et al., 2007; Small et al., 2003). In each of these prior studies, however, subjects were not asked to rate or describe the stimuli on-line during the scanning procedure. This suggests that the nature of the task (as it determines the cognitive function that the task demands from the subject) is an important determinant of the observed association between affective properties of a stimulus and brain activity. Moreover, a recent meta-analysis of prior imaging studies reports that activity in no region consistently correlates with arousal ratings (Phan et al., 2002). We believe that the absence of consistent correlations with arousal is attributable to (1) the failure of prior studies to dissociate the effects of arousal, valence, and absolute valence ratings on those correlations and (2) the meta-analytic approach combines results across studies that do and do not ask the subject to rate affect during the scan.

Human imaging studies and animal models increasingly suggest that activity in the set of brain regions in which we detected correlations of neural activity with arousal ratings is best understood as subserving not only individual judgments of the arousing qualities of an emotional stimulus, but also as a system that subserves the monitoring and assessment of the salience of the stimulus (i.e., that monitors the need for allocating attentional resources to processing of the stimulus) during the processing of stimuli that are ambiguous and yet biologically relevant (Reiman et al., 1997; Silvestri & Kapp, 1998; Whalen, 1998). A wide range of stimuli, including those that are unexpected, faces expressing fear or surprise, and subliminal images, lead participants to seek more information that will improve classification of ambiguous stimuli as potentially harmful or not, and thus they activate these salience-related structures (Adolphs, Tranel, Damasio, & Damasio, 1994; Buchel, Morris, Dolan, & Friston, 1998; LaBar, Gatenby, Gore, LeDoux, & Phelps, 1998; Whalen, 1998; Whalen, Rauch et al., 1998; Whalen et al., 2001). This interpretation of our findings for the correlates of arousal accords well with strong prior evidence that the amygdala plays an essential role in fear conditioning, in which an animal must process and learn classification of sensory stimuli that are important for survival. Under any circumstances, a face that displays low arousal is ambiguous and thus may engage this system (Whalen, 1998). This effect is enhanced, however, when a subject makes emotional judgments about a face, because classifying emotional valence is difficult for faces that express emotions of low arousal (to appreciate the validity of this claim, simply note that the regions of the circumplex representing low arousal and either high negative or high positive valence are the least populated and the most difficult to probe effectively with emotional stimuli).

Considerable experimental evidence supports a role for this network of brain regions in subserving salience detection. According to this model, the amygdala helps to detect and resolve stimulus ambiguities (Buchel, Morris et al., 1998; LaBar et al., 1998; Sander, Grafman, & Zalla, 2003; Silvestri & Kapp, 1998; Whalen, 1998; Whalen, Rauch et al., 1998; Whalen et al., 2001; Wright & Liu, 2006). The brainstem, long known to contribute to the experience of affect, has bidirectional connections with the amygdala and other limbic regions. The dorsal pons, where increasing activity accompanied ratings of low arousal, contains the locus ceruleus and parabrachial nucleus, brainstem nuclei that coordinate physiological and behavioral responses to threat (Amaral, Price, Pitkanen, & Carmichael, 1992; Barbas, Saha, Rempel-Clower, & Ghashghaei, 2003; Liddell et al., 2005; McDonald, 1998; Ongur, An, & Price, 1998; Ongur & Price, 2000). Similarly, evidence increasingly suggests that the cerebellum contributes to a salience-detecting system (Schmahmann & Sherman, 1998; Turner et al., 2007; Zhu, Yung, Kwok-Chong Chow, Chan, & Wang, 2006). Finally, the thalamus has reciprocal connections with both the amygdala and prefrontal cortex and plays a role in modulating sensory input as it is processed by other limbic structures (Martin, 1989; Wager et al., in press). Our results suggest that, together with the cerebellum, dorsal pons, and amygdala, the thalamus is more active when processing faces that express ambiguous emotions.

We believe that the response of the cerebellum, dorsal pons, and amygdala to rating ambiguous emotion is inherently tied up with what is described elsewhere as the attention system. In a face rating task (or probably in any situation in which one is motivated to learn from emotion) ambiguous, low arousal affective stimuli motivate the subject to attend more closely and activate circuits needed for affective learning. Our finding that activity in the amygdala, dorsal pons, and cerebellum is inversely associated independently with both individual and reference ratings of arousal suggests that this affect happens both in and out of awareness.

Valence

We detected fewer associations of brain activity with valence ratings than have typically been reported in prior studies (Murphy et al., 2003; Phan et al., 2002; Wager et al., in press), likely because of our more careful parsing of the contributions of arousal and absolute valence to these neural responses. Indeed, many of the regions commonly associated with negative valence (amygdala, brainstem, and PFC) were instead associated with ratings of arousal or absolute valence. More sophisticated study designs are slowly dispelling the notion that the amygdala subserves only negative emotions (Davis & Whalen, 2001; Paton et al., 2006; Walker & Davis, 1997; Wallace & Rosen, 2001).

Activity in the dorsal ACC (BA 6/24) is associated with the processing of negatively valenced stimuli (Damasio et al., 2000), likely because of its role in supporting executive functions (Bush, Luu, & Posner, 2000; M. I. Posner & Raichle, 1997). Executive processes are necessary for the appropriate processing of negatively valenced stimuli in order to identify such stimuli as external rather than as internal signs of danger, as suggested by the association of executive dysfunction and mood dysregulation in schizophrenia (Green, Waldron, & Coltheart, 2007; Shamay-Tsoory, Shur, Harari, & Levkovitz, 2007). The inferior parietal region plays a similar role in attending and regulating perception of negative stimuli as part of the dorsal visual pathway (Adolphs, Tranel, & Damasio, 2003). In our study, activity in both regions correlated inversely with individual and reference ratings of valence. Thus, the more negatively valenced the stimulus, the more active was the dorsal ACC and inferior parietal cortices, supporting findings from prior studies.

Absolute Valence

In three of the four structures where we detected activity inversely correlated with arousal, neural activity correlated positively with absolute ratings of valence, suggesting that these regions subserve not only the detection and resolution of ambiguity, but also the shining of an “emotional spotlight” on salient stimuli that lie at the extremes of positive and negative valence (Adolphs et al., 2005; Anderson, 2005). Numerous prior studies have identified the amygdala as a key structure in representing the intensity of an observed or experienced emotion, independent of whether it is positively or negatively valenced (Anderson, 2007; Davis & Whalen, 2001; Lewis et al., 2007; Liberzon et al., 2003; Murphy et al., 2003; Paton et al., 2006; Phan et al., 2002; Phelps & LeDoux, 2005; Small et al., 2003). Neuroanatomical data from primates suggests that distinct populations of neurons in the amygdala subserve positively and negatively valenced emotions. And although the evidence is more scant, the cerebellum and dorsal pons are thought to contribute to the processing of strong affects of either valence (Damasio et al., 2000; Schmahmann & Sherman, 1998; Small et al., 2003; Turner et al., 2007; Zhu et al., 2006).

Task Demands

Previous studies have reported that when subjects perform a task that demands explicit labeling of supraliminally presented emotions, the correlation of affective ratings with neural activity in limbic regions is lower than when no such labeling is required (Critchley et al., 2000; Lange et al., 2003; Phan et al., 2002; Surguladze et al., 2003). Our data suggest that this labeling-dependent activity may be the product of two countervailing effects: (1) the inverse correlation of limbic activity with arousal, and (2) the positive correlation of limbic activity with absolute valence. Because ratings of arousal and absolute valence correlated positively (in our sample, the Pearson’s r is 0.3 to 0.4), these effects tended to counteract one another for stimuli expressing affects at the extremes of valence (which tended to be rated as having high arousal), and would make limbic activity appear not to be associated with either arousal or valence, at least in these regions within the affective circumplex, unless the correlates of absolute valence are dissociated statistically from those of arousal. The labeling task enhances the inverse relationship between arousal and limbic activity by encouraging subjects to attend more to ambiguous stimuli and less to unambiguous ones. Thus it is particularly with such a task that this effect is sufficiently strong to counterbalance the limbic-absolute valence relationship, thus hiding an overall relationship between limbic activity and affective arousal when arousal and valence are not considered simultaneously.

We believe that if our experiment had not included on-line rating, we would likely have only detected the positive relationship between activity in subcortical limbic structures and absolute valence that has been seen in previous experiments (and often falsely labeled as an effect of arousal because of the correlation between arousal and absolute valence). Using such a design we would have been less likely to detect a relationship between limbic system activity and low arousal/ambiguity because off-line or reference ratings are a less accurate representation of an individual’s subjective emotional experience during the scan and thus provide less power for detection of this effect. Prior studies that have attempted to look at the effects of arousal and valence on limbic activity using a passive task have produced inconsistent results, likely due to this difficulty and to the failure to include absolute valence and arousal in the same regression equation (Anders et al., 2004; Kensinger, 2004; Kensinger & Corkin, 2004; Kensinger & Schacter, 2006; Small et al., 2003). As we argued earlier in the discussion, we believe that attention is an inherent part of the affective response and our task is intentionally an “active” task of emotional processing and rating, in contrast with passive, indirect/orthogonal tasks that have been studied elsewhere. However, we would expect the relationship between absolute valence and brain activity to be independent of the nature of the task.

This study is the first to use individual and reference ratings of emotional arousal and valence to explore the simultaneous and overlapping influence of these two different aspects of how affect is processed by the brain. We believe that mean ratings of the experimental stimuli taken from a large population (Russell & Bullock, 1985) capture the universal properties of the stimuli and that the brain regions whose activity is associated with these properties subserve a corresponding universal affective processing system. In contrast, we believe that individual ratings of arousal and valence, as captured during our experiment, reflect individual cognitive overlay added to the universal properties. In the reference population these individual/cognitive overlay effects cancel each other out when the mean is calculated. Thus, individual ratings will be associated selectively with activity in brain regions which subserve this cognitive overlay system. When both individual and reference ratings were added into the same regression equation, we identified brain regions whose activity is associated more with one affective system than the other.

In prior research on the perception and regulation of affects, the mPFC has activated most consistently when a subject perceives or tries to modulate the effect of an emotionally evocative (as compared to emotionally neutral) stimulus (Lane, Fink, Chau, & Dolan, 1997; Lane, Reiman, Ahern et al., 1997; Ochsner et al., 2002; Phan et al., 2002; Reiman et al., 1997). We found that activity in the mPFC (BA 10/32) correlated inversely with individual ratings of arousal. Reference ratings of arousal were positively associated with activity in the mPFC, though just below the level of statistical significance. Therefore, when the mPFC was more active, individual participants perceived the face as showing less arousal than did a reference sample viewing the same face. This finding suggests that the mPFC may help to down-regulate subjective responses to the viewing of faces that are emotionally aroused and arousing. This interpretation is consistent with findings from previous animal models and human imaging studies suggesting that activity in the mPFC contributes to the extinction of fear (Quirk & Gehlert, 2003), the allocation of attention to internal states (Lane et al., 1998; Lane, Reiman, Bradley et al., 1997), and the modulation of affect (Beer, Heerey, Keltner, Scabini, & Knight, 2003; Berlin, Rolls, & Kischka, 2004; Bush et al., 2000; Devinsky, Morrell, & Vogt, 1995; Pezawas et al., 2005; Simmons, Stein, Matthews, Feinstein, & Paulus, 2006; Vogt, Nimchinsky, Vogt, & Hof, 1995; Wager et al., in press).

The association of activity in left lateral PFC (BA 44/45) with reference ratings of negative valence is consistent with increasing evidence that this region serves to regulate negative affect, particularly during the explicit identification or naming of affects (Gorno-Tempini et al., 2001; Hamzei et al., 2003; Hariri, Bookheimer, & Mazziotta, 2000; Lewis et al., 2007; Matsuo et al., 2003; Simmons et al., 2006; Wright & Liu, 2006). The lateral PFC activates more for a negatively valenced stimulus, as rated by the reference group. Activity in the lateral PFC was not associated with the individual ratings of valence, suggesting that modulation of the lateral PFC may keep the individual from becoming aware of how negative the stimulus actually is.

Research has long implicated the temporal cortex, including the fusiform gyrus and occipito-temporal junction, in mediating strong and vivid emotional experiences, as first demonstrated during electrical stimulation of this region in awake humans (Sem-Jacobsen, 1968). Functionally, these regions are tightly coupled with activity in the amygdala during the viewing of emotionally charged faces (Amaral, Behniea, & Kelly, 2003; Catani, Jones, Donato, & Ffytche, 2003; Chao, Martin, & Haxby, 1999; Kanwisher, McDermott, & Chun, 1997; Morris et al., 1998; Surguladze et al., 2003; Vuilleumier & Pourtois, 2007; Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004). Our finding that activity in this region correlates with individual absolute ratings of valence suggests either that activity in this region enhances subjective ratings of stimuli at the extremes of emotional valence, or that activity in prefrontal cortices that encode absolute valence upregulates activity in temporal cortices (Hopfinger et al., 2000; Morris et al., 1998; Pessoa et al., 2002; Sprengelmeyer et al., 1998). The direction of causality in this association is unclear.

Behavioral Findings

Behavioral ratings of arousal and valence were consistent and stable both within and across participants for all 20 emotional stimuli. Variability in the ratings was small, and the means were similar to those previously reported for a larger sample (Russell & Bullock, 1985, Fig. 2). Differences between individual and reference ratings were greatest for affects having low valence and high arousal (anger, scared, surprise, disgust), in accord with prior reports that negative, high arousal affects are difficult to distinguish from one another and that their ratings depend more heavily on context and cognitive overlay than do ratings of other affects (Ochsner et al., 2006; Phillips et al., 2004; Stark et al., 2003; Whalen, 1998; Whalen et al., 2001). In both our sample and the reference, arousal ratings correlated significantly with ratings of absolute valence, confirming our expectation that the constructs of arousal and extremes in valence can be difficult to distinguish from one another, and supporting our contention that they have often been conflated in prior studies of emotional processing.

Limitations

The detailed behavioral data collected from participants during the scanning process constituted this study’s greatest strength. The large quantity of data collected at the level of individual subjects ensured the accuracy and statistical power of the first-level correlations of neural activity with emotional ratings. The statistically significant correlation of arousal with absolute valence complicated the analyses and possibly could have confounded them. Their intercorrelation was a consequence of using a naturalistic set of emotional faces, one that is not unique to this study. However, we were careful to demonstrate through the use of regression diagnostics that the correlation between predictor variables did not cause instability in the calculation of regression coefficients. We turned this intercorrelation among individual ratings to our advantage, by uncovering findings that would have remain hidden had both variables not been simultaneously considered in the regression model. As described above, this study was designed to explore the active processing and rating of emotional stimuli and thus is different from studies that have focused on passive viewing or the use of an indirect/orthogonal task. Perhaps a future study could analyze both active and passive affective processes in the same individuals as a way to most efficiently disentangle the differences in response to task demands.

Conclusions

The findings of this study support the validity of the circumplex model of affect and its utility in dissociating the roles of the amygdala, brainstem, cerebellum, thalamus, and association cortices in representing and modulating the affective evaluation of human faces. The simultaneous analysis of arousal, valence, and absolute valence, in both subjective and reference ratings of valence and arousal, was essential for clarifying the roles of these structures in mediating arousal, valence, or their cognitive overlay when evaluating emotional stimuli. The amygdala, cerebellum, and dorsal pons constitute an affective system that becomes more active when ambiguous, low-arousal affective stimuli are presented, particularly if the stimulus represents a strong positively- or negatively-valenced affect. We propose that the salience system described in previous studies can only be understood adequately when both ambiguity and absolute valence are simultaneously considered. Furthermore, we can best understand the role of medial and prefrontal cortices in modulating affect by considering individual ratings in comparison with ratings from a larger reference group. Further work using tasks that incorporate these ratings and statistical analyses that include these variables will yield a better understanding of the neurobiological bases of emotion.

Table 2.

Summary of significant second-level analyses shows association of BOLD response in distinct regions with concordant (individual and reference), individual only, and reference only ratings of arousal, valence, and absolute valence. Regions listed in this table show a significant cluster (voxelwise p < 0.001) larger than 30 voxels on second-level analyses and appear consistently, by inspection, on first-level analyses. “+” signifies a positive association between behavioral rating score and BOLD response, while “−” signifies an inverse association. BA = Brodmann’s area. PFC = prefrontal cortex. ACC = anterior cingulate cortex.

	Arousal	Valence	Absolute Valence
Concordant	− Amygdala complex	− Dorsal ACC (BA 6/24)	+ Amygdala complex
	− Thalamus	− Parietal (BA 40/7/39)	+ Dorsal pons
	− Cerebellum		+ Cerebellum
	− Dorsal pons
Individual only	− R medial PFC/ACC (BA 10/32)		+ Temporal/fusiform/occipitotemporal (BA 19/21/37)
Reference only		− L Inferior frontal/lateral PFC (BA 44/45)