Novelty as a Dimension in the Affective Brain

Abstract

Many neuroscience studies have demonstrated that the human amygdala is a central element in the neural workspace that computes affective value. Emerging evidence suggests that novelty is an affective dimension that engages the amygdala independently of other affective properties. This current study is the first in which novelty, valence, and arousal were systematically examined for their relative contributions to amygdala activation during affective processing. Healthy young adults viewed International Affective Picture System (IAPS) images that varied along the dimensions of valence (positive, negative, neutral), arousal (high, mid, low), and novelty (novel, familiar). The results demonstrate that, in comparison to negative (vs. positive) and high (vs. low) arousal stimuli, the amygdala has higher peak responses and a selectively longer timecourse of activation to novel (vs. familiar) stimuli. In addition, novelty differentially engaged other affective brain areas including those involved in controlling and regulating amygdala responses (e.g. orbitofrontal cortex), as well as those transmitting sensory signals that the amygdala modulates (e.g. occipitotemporal visual cortex). Taken together with other findings, these results support the idea that an essential amygdala function is signaling stimulus importance or salience. The results also suggest that novelty is a critical stimulus dimension for amygdala engagement (in addition to valence and arousal).

Introduction

“Affect” refers to the ability of an object to influence internal physical state in a way that is experienced as part of a mental state (c.f., Barrett & Bliss-Moreau, in press). Hundreds of studies show that affect can be described in terms of two properties: valence (pleasure or displeasure) and arousal (activation) (e.g., Barrett, 2006^{a, b;} Barrett & Bliss-Moreau, 2009^; Russell & Barrett, 1999; see Figure 1). Objects in the world are said to be “positive” or “negative” or “arousing” by virtue of their capacity to influence a person’s affective state. For example, if the perception of a snake involves unpleasant, high arousal affect, then the snake is said to be negative and arousing. Neuroscientists have made significant progress in understanding how valence and arousal are realized in the human brain. In this paper, we present clear evidence that affective responses to novelty are realized within the same neural workspace as valence and arousal. A “neural workspace” refers to the brain areas that are routinely included in the variety of neural assemblies that correspond to a class of mental events such as affect (Edelman, 1987). Consequently, novelty might be considered a fundamental stimulus dimension that evokes affective responses.

Figure 1.

Circumplex model of affect comprised of the bipolar dimensions of valence and arousal. Adapted from Barrett & Bar, in press.

The amygdala is the centerpiece of the affective workspace in both human and animal neuroscience studies. Amygdala activity increases during high arousal (e.g., Phan et al., 2003), and in response to positively and negatively valenced faces (e,g., Zald, 2003), pictures (e.g., Anders, Eippert, Weiskopf, & Veit, 2008), words (e.g., Posner et al., in press), and scents (e.g., Anderson et al., 2003), even when controlling for arousal (Anders et al., 2008). The amygdala is also responsive to stimulus intensity (i.e., the absolute value of valence; e.g., Gerber et al., 2008) and the state of the individual (Belova, Paton, & Salzman, 2008).

Several studies now show that stimulus novelty also is affectively significant. Novelty and uncertainty engage the same cardiovascular systems as valence and arousal (Mendes, Blascovich, Hunter, Lickel, & Jost, 2007). The amygdala is reliably responsive to novel objects (e.g., Breiter et al., 1996^; Schwartz et al., 2003^; Wilson & Rolls, 1993^; Wright et al., 2006) and novel (neutral) faces across the lifespan (Wright at al., 2008). Amygdala activity is associated with orienting responses (e.g., Holland & Gallagher, 1999) and amygdala lesions disrupt normal responses to novelty in primates (e.g. Prather et al., 2001).

This report details the first fMRI study examining the relationship between novelty, valence, and arousal in amygdala response within one experiment. Participants viewed familiar and novel pictures that were pleasant, unpleasant, or neutral, with some degree of arousal. We hypothesized that amygdala activation would be greater for novel versus familiar pictures, high versus lower arousal, and valenced versus neutral pictures. For exploratory purposes, we also examined activation in other areas relevant to affective picture processing, including frontal and occipitotemporal cortical areas. We hypothesized that similar areas would be active for novelty, valence, and arousal, and that the effect of novelty would be additive or interactive.

Method

Subjects

Fifteen healthy young adults (8 females; age M = 22.2, SD = 2.37, range = 19-27 years) participated in the study. Data from three additional subjects were excluded due to excessive head motion (i.e., total motion vector > 5mm) or scanner/image-related difficulties (e.g., spiking, amygdala artifact). We administered Structured Clinical Interview for DSM-IV (SCID; First et al., 1995) to confirm the absence of DSM-IV Axis I diagnoses (American Psychiatric Association, 1994). All subjects were right-handed, as determined by the Edinburgh Handedness Inventory (Oldfield, 1971), and free of psychoactive medications.

Materials

Twenty-one full-color images from the International Affective Picture System (IAPS; Lang, Bradley, & Cuthbert, 1997) were selected for each of six combinations of arousal and valence (i.e., high arousal negative, high arousal positive, mid arousal negative, mid arousal positive, mid arousal neutral, and low arousal neutral). We were unable to fully cross arousal and valence, because the IAPS stimulus set does not include high arousal, neutral images.

Participants were familiarized to one image in each stimulus category, and the remaining images were used as novel images (image valence and arousal norms available from authors on request). During the scan, participants rated each image for how aroused it made them feel using a three point scale (1 = low, 2 = mid, 3 = high). Participant ratings were consistent with published norms. During the test runs, participants rated negative pictures (M = 2.21, SD = .37) as more arousing than positive (M = 1.58, SD = .24; t (14) = 6.51, p < .001, and neutral (M = 1.45, SD = .23; t (14) = 11.14, p < .001) pictures. In addition, during the test runs subjects rated novel stimuli (M = 1.89, SD = .18) as more arousing than familiar stimuli (M = 1.61, SD = .29; t (14) = 6.07, p < .001). Note, however, that initial arousal ratings (M = 1.85) for to-be-familiarized images during the familiarization run did not differ from arousal ratings for novel stimuli during the test runs, thus subjective participant ratings of arousal for novel and familiar stimuli did not differ prior to familiarization.

Procedure

The paradigm consisted of five event-related fMRI runs. During each run, participants viewed IAPS images and rated each as low, mid, or high arousal using a three-button box. The first run was the familiarization run. The six IAPS images were each shown ten times. Participants then completed four test runs within they viewed each familiarized image five times, and each of thirty novel images once and only once. Each run was 340 seconds in length and each image was presented for 3.5 seconds with a jittered ISI that varied from .5 seconds to 12 seconds.

Prior to scanning, each participant completed a brief practice run outside the scanner to become familiar with the experimental task; the images used for practice varied in valence and arousal, and were not used in the experimental runs. The task was run using E-Prime experimental software (Psychology Software Tools, Pittsburgh, PA) on a PC, from which images were projected onto a screen in the magnet bore. Participants viewed images via a mirror mounted on the head coil.

Image Acquisition

We used a Siemens Magnetom Trio Tim 3T whole body high-speed imaging device equipped for echo planar imaging (EPI) (Siemens Medical Systems, Iselin NJ) with a 12-channel gradient head coil. Expandable foam cushions restricted head movement. After an automated scout image was acquired and shimming procedures were performed to optimize field homogeneity, high-resolution 3D MPRAGE sequences (TR/TE/flip angle = 2.53s/3.39ms/7°) with an in-plane resolution of 1.3 × 1.0 mm, and 1.3 mm slice thickness were collected for spatial normalization and for positioning the slice prescription of the subsequent sequences. Then a T1- EPI (TR/TE/flip angle = 10.0s/39ms/90°) and a T2-weighted (TR/TE/flip angle = 5.21s/94ms/150°) sequences were collected to assist in registration of the functional data to the high-resolution anatomical scan. Functional MRI images (blood oxygenation level dependent or BOLD) (Kwong et al., 1992) were acquired using a gradient echo T2*-weighted sequence (TR/TE/flip angle = 2.0s/30ms/90°). Prior to each scan, four time points were acquired and discarded to allow longitudinal magnetization to reach equilibrium. The T1, T2, and gradient-echo functional images were collected in the same plane (33 coronal slices angled perpendicular to the AC/PC line) with the same slice thickness (5mm; voxel size 3.12 × 3.12 × 5 mm), excitation order (interleaved) and phase encoding (foot-to-head). We used these parameters based on earlier work that suggested that the parameters helped minimize susceptibility in medial temporal lobe regions (Wright et al., 2001).

Image Pre-preprocessing

Functional and structural MRI data were analyzed using the standard processing stream of the Martinos Center for Biomedical Imaging (http://surfer.nmr.mgh.harvard.edu). BOLD data were motion-corrected and inspected for gross motion. Slices were discarded if the total motion vector exceeded 5mm. Data in each functional run were intensity normalized and spatially smoothed (full-width half-maximum = 8mm) using a 3D Gaussian filter. To remove temporal autocorrelation noise, we also included polynomial drift correction with 2 nuisance regressors to account for low-frequency drift and whitening based on a single autocorrelation function estimated across all brain voxels (Burock and Dale, 2000).

Following preprocessing, functional images for each participant were registered to an average 3D structural image created from that participant’s two high-resolution 3D MPRAGE images. We estimated the duration of the hemodynamic response to be 20 seconds. Functional data for each condition were modeled using a finite impulse response (FIR) model beginning at 4 seconds pre-stimulus, and utilizing 2 second bins (see Figure 2). Thus, for example, BOLD data at timepoint 6 corresponded to activation averaged across 6-8 sec post-stimulus. The use of an FIR model allows differentiation of neural response across timepoints in the HDR; whereas additive effects at some periods within the HDR might be masked by a canonical gamma fit model.

Figure 2.

Timepoints in FIR model of the hemodynamic response. Each timepoint represents a two-second time window, beginning at 4 seconds pre-stimulus and ending at 16 seconds post-stimulus onset.

Functional data then were visualized over the averaged 3D image for each individual to ensure that the fMRI signal in the amygdala was not obscured by susceptibility artifact. Data from one participant were excluded on this basis.

Anatomical ROI Analyses

We used an anatomically-based approach to conduct region of interest (ROI) analyses of functional data from the amygdala. We applied automated subcortical segmentation methods to the native 3D MPRAGE structural images for each subject to create anatomically-defined amygdala ROIs (Fischl et al., 2002). We manually verified these amygdala ROIs according to our previously published protocols (Wedig et al., 2005; Wright et al., 2006; Wright et al., 2008). The anatomically-defined amygdala ROIs were registered to fMRI data and BOLD signal was extracted for each participant. Percent signal change for combinations of valence, arousal, and novelty versus baseline (fixation) was calculated.

We conducted two different ANOVAs each for the left and right amygdalae to investigate all effects of interest. The first set of ANOVAs allowed us to examine the interactive effects of novelty, arousal, as well as positive and negative valence. The second set allowed us to examine the effect of novelty in hedonically neutral pictures.

Whole Brain Cluster Analyses

Whole-brain cluster analyses were used to investigate other brain areas that might give a fuller picture of the brain states for novelty, valence, and arousal. Data were spatially normalized into Talairach space (Talairach and Tournoux, 1988) and a cortical surface-based spherical coordinate system (Dale et al., 1999; Fischl et al., 1999). For Talairach spatial normalization, procedures developed and distributed by Montreal Neurological Institute were used to compute a transformation matrix from the high-resolution MPRAGE volumes (Collins et al., 1994). For spherical spatial normalization (software and documentation is available at http://www.nmr.mgh.harvard.edu/freesurfer), the averaged high-resolution 3D MPRAGE volume was used to create a segmentation of the gray/white matter boundary and outer cortical surface for each subject using a semi-automated procedure. This surface was then smoothed using a topology-preserving deformable surface algorithm, allowing specification of which voxels in the original volume correspond to the cortical surface.

To create group statistical maps for volume and surface-based analyses, each participant’s data were selectively averaged for each condition and re-sampled into Talairach space or spherical space for weighted random-effects analyses. The main contrasts of interest for the whole-brain analyses (Talairach and spherical) were (1) novel versus familiar, (2) negative versus neutral, (3) positive versus neutral, and (4) high arousal versus mid arousal conditions. We conducted these analyses for the time window corresponding to 6-8 seconds post-stimulus (i.e., timepoint 6) as our amygdala ROI analyses revealed this timepoint as initial peak activation in the hemodynamic response. For these exploratory analyses we set alpha at p < .00001 for peak voxels, and specified a lower limit of ten contiguous active voxels to constitute a cluster. In Table 1 we report significant clusters that are of particular relevance; comprehensive lists of clusters are available from the authors on request.

Table 1.

Significant whole brain cluster activations by contrast and function.

Contrasts and Regions
Novel > Familiar	BA	Tal x	Taly	Tal z	p-value for peak voxel in cluster
Affective Areas
L amygdala	na	−21.8	−7.4	−12.3	< 10−5
L thalamus	na	−15.8	−24.2	1.2	< 10−5
R thalamus	na	10.9	−22.3	0.7	< 10−7
L caudate	na	−8.9	6.4	1.5	< 10−6
R caudate	na	10.9	8.3	1.0	< 10−5
L putamen	na	−21.8	−0.7	4.6	< 10−6
L pallidum	na	−20.8	−2.7	3.4	< 10−5
L insula	13	−29.7	22.5	3.5	< 10−5
R insula	13	33.7	16.4	−1.7	< 10−5
L parahippocampal gyrus	36	−23.8	−34.8	−16.8	< 10−7
R parahippocampal gyrus	36	24.8	−39.4	−11.5	< 10−7
R inferior temporal pole	38	35.6	20.8	−30.5	< 10−5
L inferior temporal pole	38	−34.9	7.5	−24.1	< 10−5
L medial temporal gyrus	38	−39.6	14.4	−22.8	< 10−5
R superior temporal gyrus	38	23.8	13.6	−20.0	< 10−8
Memory Areas
L hippocampus	na	−21.8	−32.5	−9.3	< 10−5
R hippocampus	na	24.8	−28.7	−10.3	< 10−5
Control Areas
R inferior frontal gyrus	44	35.6	8.1	26.3	< 10−10
R inferior frontal gyrus	45	43.6	34.3	6.6	< 10−8
L middle frontal gyrus	45	−39.6	25.2	18.1	< 10−8
L inferior frontal gyrus	45	−46.5	34.1	2.4	< 10−5
Visual Areas
L striate area	17	−11.9	−73.9	17.5	< 10−5
R area 17	17	10.9	−85.9	11.7	< 10−6
L area 17	17	−15.8	−86.9	9.9	< 10−6
R occipital gyrus	18	34.7	−86.2	5.2	< 10−7
L occipital gyrus	18	−37.6	−85.6	16.3	< 10−12
R occipital gyrus	19	−48.5	−76.8	17.7	< 10−9
L occipital gyrus	19	46.5	75.9	16.7	< 10−9
L fusiform gyrus	37	−26.7	−50.1	−11.8	< 10−8
R fusiform gyrus	37	25.7	−50.0	−10.1	< 10−9
L lateral geniculate nucleus	na	−20.8	−26.5	−5.4	< 10−6

Negative > Neutral	BA	x	y	z	p-value for peak voxel in cluster
Affective Areas
L inferior frontal gyrus	47	−45.5	23.6	−13.8	< 10−5
L inferior frontal gyrus	47	−37.6	22.3	−0.2	< 10−5
Control Areas
none
Visual Areas
L area17	17	−7.9	−83.7	−3.4	< 10−5
R area17	17	11.9	−83.8	−5.1	< 10−7
L occipital gyrus	18	−29.7	−83.5	0.0	< 10−5
R occipital gyrus	18	25.7	−73.2	−7.3	< 10−9
L angular gyrus	19	−27.7	−80.2	27.0	< 10−5
R parieto−occipital gyrus	19	21.8	−83.1	27.2	< 10−5
L fusiform	37	−45.5	−59.6	−8.0	< 10−5
R fusiform	37	47.5	−59.1	3.9	< 10−5

Positive > Neutral	BA	x	y	z	p-value for peak voxel in cluster
Affective Areas
L middle frontal gyrus	10	−4.0	49.6	2.1	< 10−6
Control Areas
R middle frontal gyrus	9	47.5	25.5	23.6	< 10−6
L superior frontopolar gyrus	9	−9.9	46.4	15.2	< 10−5
L inferior frontal gyrus	45	−41.6	25.0	14.4	< 10−5
L middle frontal gyrus	46	−49.5	46.1	9.7	< 10−8
Visual Areas
L fusiform gyrus	19	−29.7	−67.2	−4.2	< 10−7
L middle temporal gyrus	19	−37.6	−76.0	13.9	< 10−8
R medial occipital gyrus	19	35.6	−76.5	4.8	< 10−5
R angular gyrus	39	41.6	−69.2	33.9	< 10−7

High Arousal > Mid Arousal	BA	x	y	z	p-value for peak voxel in cluster
Affective Areas
none
Control Areas
none
Visual Areas
R occipital gyrus	17	9.9	−72.61	4.56	< 10−6
L occipital gyrus	18	−39.6	−87.65	14.52	< 10−8
R middle temporal gyrus	18	45.54	−57.71	−8.05	< 10−6
R inferior temporal gyrus	19	37.62	−79.63	19.65	< 10−6
L inferior temporal gyrus	37	−37.62	−59.98	−14.66	< 10−5

Note: Cluster requirements were (1) minimum ten contiguous voxel extent, and (2) minimum peak voxel significance threshold at p <.00001. Brodmann areas were localized using the Talairach & Tournoux (1988) atlas, and anatomic areas were identified using the Mai, Paxinos, & Voss (2008) atlas. Significant clusters in functional areas beyond the scope of the current paper are not reported in this table.

Results

Anatomical ROI Results

Novelty, Valence, and Arousal

First, we conducted Novelty (familiar, novel) × Arousal (high, mid) × Valence (positive, negative) × Timepoint (1-10) ANOVAs for right and left amygdalae. Unless otherwise specified, we set α at p <.05 for all analyses. These analyses confirmed the independent effects of novelty, valence, and arousal in the amygdala.

As predicted, participants showed significant increases in left and right amygdala response to novel versus familiar stimuli across all timepoints (left: F (1,14) = 22.38, p <.001, ηp² = .62; right: F (1,14) = 29.92, p <.000, ηp² = .68), and to negative versus positive stimuli across all time points (left: F (1,14) = 6.74, p = .021 ηp² = .33; right: F (1,14) = 8.78, p = .010, ηp² = .39). Further, there was a significant interaction of Novelty and Valence (left: F (1,14) = 6.45, p = .024 ηp² = .32; right: F (1,14) = 4.61, p = .050, ηp² = .25; see Figure 3), showing significantly greater bilateral amygdala activation to novel negative versus novel positive pictures (left: t (14) = 3.49, p = .004; right: t (14) = 3.59, p = .003); similar differences for familiar pictures were not observed. There was no main effect of arousal (i.e., high versus mid arousal) across all timepoints.

Figure 3.

Significant Novelty by Valence interaction for left and right amygdala ROI activation.

More importantly, a significant Timepoint by Novelty interaction (left: F (9,126) = 12.68, p <.001, ηp² = .48; right: F (9,126) = 9.70, p <.001, ηp² = .41) indicated significantly different timecourses for amygdala responses for novel versus familiar pictures (see Figure 4a). For both the left and right amygdalae, follow up t-tests using a Bonferroni correction revealed significantly greater BOLD activation to novel versus familiar stimuli at timepoints 6-8 (6-12 seconds post-stimulus; ts >3.66; ps <= .003). This pattern of differential activation persisted into timepoint 9 (12-14 seconds post-stimulus onset) in the left amygdala (t (14) = 3.20, p = .006). In the right amygdala only, a significant Valence by Timepoint interaction (F (9,126) = 2.86, p = .004, ηp² = .17) was driven by persistent activation for negative versus positive stimuli at timepoints 7 (t (14) = 3.52, p = .003) and 8 (t (14) = 2.56, p = .023), or 8-12 seconds post-stimulus onset (see Figure 5b). In addition, a significant Timepoint by Arousal interaction (F (9,126) = 3.18, p =.002, ηp² = .19) indicated different timecourses for high versus mid arousal images in the left amygdala (see Figure 4c). Specifically, at timepoint 8 (10-12 seconds post-stimulus onset), the left amygdala was still active for high arousal stimuli but not for mid arousal stimuli (t (14) = 2.46, p = .027).

Figure 4.

Finite impulse response (FIR) models of the hemodynamic response across ten timepoints in the left and right amygdala ROIs. Volume images represent amygdala activation for each contrast at the timepoint of greatest differentiation. Panels illustrate (a) novel versus familiar timecourses and amygdala contrast at timepoint 6, (b) negative versus positive timecourses and amygdala contrast at timepoint 7 (c) high arousal versus mid arousal images and amygdala contrast at timepoint 8. Significant interactions of timepoint and affective property are denoted as follows: † p < .05; * p < .01.

Figure 5.

Significant Novelty by Valence (including neutral) interaction in the left amygdala ROI.

Finally, a four-way interaction confirmed the combinatory effects of the affective properties of interest, but only in the right amygdala. A significant four way Novelty by Valence by Arousal by Timepoint interaction (F (9,126) = 2.50, p =.011, ηp² = .15) indicated that, as predicted, novel, negative, high arousal stimuli showed the greatest amygdala response at timepoint 8, or 10-12 seconds post-stimulus onset. Amygdala activation persisted at this timepoint for novel, negative, high arousal stimuli (13% signal change compared to baseline) but not for other stimulus categories (all zero activation or deactivation).

Novelty and Neutral Valence

We next conducted Novelty (familiar, novel) − Valence (positive, negative, neutral) × Timepoint (1-10) ANOVAs on the BOLD response within the amygdala ROIs using mid-arousal stimuli only. This analysis allowed us to examine any amygdala response to novel neutral stimuli, and to compare amygdala responses to positive and negative images versus neutral images. Again, for this analysis we used only data for mid-arousal images; this procedure eliminated the potential confounding of neutral valence and low arousal as well as the confounding of positive and negative valence with high arousal.

In the left amygdala, there was a significant effect of Valence (F (2,28) = 3.39, p = .048, ηp² = .19) and a significant Novelty × Valence interaction (F (2,28) = 5.56, p = .009, ηp² = .29). The latter effect was due to greater activation for novel negative stimuli compared to novel positive (t (14) = 3.01, p = .009) and novel neutral stimuli (t (14) = 3.91, p = .002). Amygdala response did not differ between novel positive and novel neutral stimuli, and there were no differences by valence for familiar stimuli. In the right amygdala, in addition to a significant effect of Novelty (F (1,14) = 5.43, $p=.03\underset{.}{5}$,ηp² = .28), there was also a significant Novelty × Valence × Timepoint interaction in the right amygdala (F (18,252) = 2.66, $p<.00\underset{.}{1}$,ηp² = .16), indicating greater amygdala activation for novel negative versus novel positive pictures at timepoint 6 (t (14) = 2.81, p = .014). Amygdala activation for novel neutral stimuli did not differ from novel positive or novel negative stimuli. For familiar stimuli, there were no differences by valence across time. Finally, consistent with prior findings (Schwartz et al., 2003), although left amygdala activation to novel neutral pictures did not differ from that of familiar neutral pictures, right amygdala activation was significantly greater for novel neutral compared to familiar neutral pictures (t (14) = 2.95, p = .011).

Influence of Arousal

One possible reason for the novelty effect is that novel stimuli might be more arousing than familiar stimuli; as noted earlier, participants rated novel stimuli as subjectively more arousing than familiarized stimuli (though only after familiarization). We conducted a follow-up analysis to rule out the potential confound of arousal with the novelty response. We chose the images with the highest arousal ratings across the test runs in order to maximize the potential effect of arousal. We thus compared familiar and novel high arousal (HA) negative images. These categories did not differ in participant arousal ratings (novel HA negative M = 2.69, SD = .23; familiar HA negative M = 2.54, SD = .62; t = 1.11 , p = .29). However, there was significantly greater amygdala response to novel HA negative images compared to familiar HA negative images (novel M = .07, SD = .06; familiar M = −.01, SD = .06; t = 3.97 , p < .001).

Given the differences in amygdala activation for novel negative compared to neutral and positive images and the generally greater arousal ratings for negative stimuli, we also conducted a follow-up analysis to rule out arousal as an explanation for the valence effect. We thus compared two categories of novel stimuli that had similar arousal ratings. Specifically, arousal ratings for novel mid-arousal (MA) negative images (M = 2.08; SD = .32) and novel HA positive images (M = 2.11; SD = .30) were not significantly different from each other (t = .319; p = .755). We compared amygdala activation across the hemodynamic response for these two categories. Although these means were not significantly different, (t = 1.51; p = .152), the effect size was medium (Cohen’s d = .54), showing that controlling for arousal and novelty resulted in somewhat greater amygdala response to negative (M = .067; SD = .08) compared to positive (M = .03; SD = .06) images. Given the relatively small number of images in each of the two categories, these findings must be considered preliminary as the present study may be underpowered to adequately assess this effect.

Whole Brain Results

We were most interested in the identification of brain areas that are implicated in (1) affective processing, (2) memory, (3) control, or regulation, of affective processing, and (4) visual processing. For affective processing, in addition to the amygdala, we expected to see activation in areas such as the temporal pole (Olson, Plotzker, & Ezzyat, 2007), orbitofrontal cortex (e.g., Bechara, Damasio, & Damasio, 2000), and subcortical structures such as the thalamus and hypothalamus (e.g., Dalgleish, 2004). We also expected differential activation in control areas that regulate the amygdala including medial and lateral prefrontal areas (e.g., Ochsner & Gross, 2005) and the anterior cingulate cortex. In addition, given the role of visual areas in the rapid extraction of salient information (e.g., Fecteau & Munoz, 2006) and the fact that the amygdala projects all along the ventral visual stream back to V1 (Amaral, 2003), we expected to see differential activation in visual areas for affective stimuli.

Consistent with our expectations, there were several sets of brain areas that were active across and differentiated between the dimensions of affective processing. Novel versus familiar stimuli, in addition to engaging the amygdala (consistent with the ROI analyses), produced increased BOLD responses in paralimbic cortical areas as well as selected subcortical regions (see Table 1). Specifically, there were significant clusters of activation in anterior cingulate, orbitofrontal cortex, temporal pole, and insula, as well as clusters in subcortical areas including the thalamus and striatum. Novelty also activated areas involved in memory (i.e., the hippocampus) as well as areas that serve to control affective responding, including ventrolateral and dorsolateral prefrontal areas. Finally, novelty activated significant clusters of activity in early visual areas including V1, V2, and the fusiform gyrus.

As predicted, valence also significantly activated clusters generally associated with affective processing. In addition to activation in the amygdala, positive versus neutral stimuli activated temporal areas and the precuneus, whereas negative versus neutral revealed activation in the orbitofrontal region. In addition, we observed areas implicated in the control of affective responding, specifically the dorsolateral and ventrolateral prefrontal cortex, for positive versus neutral, but not for negative versus neutral. Finally, in contrast to the early visual activation for novelty, negative versus neutral valence was differentiated in visual areas V2 and V3, and positive versus neutral valence was differentiated in visual area V3. High arousal versus mid-arousal revealed that only clusters in visual areas differentiated between high and mid arousal. Similar to novel stimuli, high versus mid arousal stimuli activated significant clusters throughout the ventral visual stream (i.e., V1-V3 and fusiform cortex).

Thus, within the window in the HDR corresponding to initial peak activation, differentiation of neural activation within and between affective dimensions suggests that novelty is processed earliest and most efficiently throughout the ventral stream, followed by valence, and last by arousal.

Discussion

Our results demonstrate that novelty is a dissociable stimulus property with affective significance, both in peak magnitude and duration of activation in the amygdala. Prior studies addressing the properties of affect generally focused only on arousal and valence (e.g., Anders et al., 2008), combined valence and novelty (e.g., Wright, Wedig, Williams, Rauch, & Albert, 2006), or studied novel neutral stimuli (e.g., Wright, Negreira, Gold, Britton, Williams, & Barrett, 2008). Our study is unique in examining the additive and interactive effects of affective dimensions in amygdala response. As with negative (vs. positive) and high arousal (vs. mid-arousal) stimuli, amygdala responses were most intense and persisted longest for novel (vs. familiar) stimuli. The idea of novelty as an independent affective property is also supported by studies in which the amygdala habituates even to very evocative stimuli (e.g., Fischer et al., 2000^; Wright et al., 2001).

In hundreds of published studies on affect, there is a strong correspondence between the valenced property of affective feelings and the valenced perception of objects in the world. Objects are said to be “positive” by virtue of their capacity to create a pleasant affective feeling, or “negative” by virtue of their ability to create an unpleasant affective feeling. The same can be said about arousal. What, then, is the mental counterpart of novelty? Although we cannot say for certain, our behavioral results are suggestive. In the current study, participants rated stronger feelings of arousal in response to novel stimuli than in response to familiar stimuli (although novel and familiar pictures were equivalent in their normative arousal ratings). These findings suggest that novelty is experienced as arousing, such that arousal, as a subjective property of one’s reaction to the world, is multiply determined. Despite this close relationship, novelty and arousal are independent properties; we demonstrated that arousal does not account for the effects of novelty (or valence) in the amygdala.

In addition to robust amygdala activation, novelty also engaged other affective areas involved in visceromotor responding, including orbitofrontal, ventral anterior cingulate, and dorsal anterior cingulate, providing further evidence of novelty as a significant affective dimension. Novelty also produced increased activity in early visual areas V1 and V2 in a manner that is similar to what has been observed for valence and arousal in other studies (e.g., Lang et al., 1997^; Mourao-Miranda et al., 2003). In the present study, early activation for novelty was differentiated from activation in later visual areas for valence and arousal. Novelty also uniquely recruited areas involved in the control of affective responding, including ventrolateral and dorsolateral prefrontal cortex.

The present findings not only reveal the affective significance of novelty, but they also extend our prior understanding of amygdala function. Evidence indicates that the amygdala’s function is to help direct attention (Holland & Gallagher, 1999) towards a source of sensory stimulation when the predictive value of that stimulation for well-being and survival is unknown or uncertain, or when the appropriate response to a stimulus is uncertain (cf. Barrett, Lindquist et al., 2007^; Davis & Whalen, 2001^; Kim, Somerville, Johnstone, Alexander, & Whalen, 2003^; Whalen, 1998). When uncertainty increases, so too does the amygdala’s response (Herry et al., 2007), and in the animal literature, amygdala lesions result in an absence of differentiation in responses to uncertain compared to familiar information (e.g., Burns, Annett, Kelley, Everitt, & Robbins, 1996^; Mason, Capitiano, Machado, Mendoza, & Amaral, 2006). Several human neuroimaging studies support the idea that amygdala activation is related to the salience or potential information value of visual stimuli (Amaral, 2003; Liberzon, Phan, Decker, & Taylor, 2003; Whalen et al., 2004), and amygdala engagement has been associated with enhanced memory (e.g., Kensinger & Schachter, 2006) and even enhanced vision (e.g., Padmala & Pessoa, 2008^; Vuilleumier & Driver, 2007). It will be of interest to examine in future studies how the persistence or duration of amygdala responses may relate to these observed enhancements of cognitive functions.

There were several potential limitations to the current study. First, our sample size was relatively small given our factorial design; however, the effects of interest are robust enough to detect a significant four-way interaction in the amygdala. Second, participants rated their arousal response during the task; some evidence suggests that such an explicit rating during scanning might limit the magnitude of amygdala response (Lieberman et al., 2007), suggesting that our findings may represent the lower limit of the amygdala novelty effect. Third, although use of an FIR model allows greater temporal specificity than the commonly-used gamma fit models, each of our timepoints encompasses two seconds of the hemodynamic response. It is possible that with a shorter window we might distinguish even finer-grained temporal distinctions between novelty, valence, and arousal.

The current work may have implications for understanding certain aspects of personality variations and specific psychopathological conditions. For example, individuals with inhibited (vs. uninhibited) temperament have overactive amygdala responses to novelty (Schwartz et al., 2003), but that study did not examine novelty response peak or duration differences that might underlie the temperament effects. Likewise, prior work has identified amygdala dysfunction in a variety of psychiatric conditions such as posttraumatic stress disorder (PTSD) (e.g., Bryant et al., 2007; Shin, Rauch, & Pitman, 2006). Novel stimuli routinely evoke a strong fear-like response in PTSD, but novel stimuli have not been examined for their amygdala effects in PTSD. Overall, our findings suggest that maladaptive responses to novelty are another manifestation of potentially problematic affective processing, and may importantly contribute to the psychopathological conditions that are typically associated with abnormal fear or arousal processing. Further research should examine whether novelty operates independently or interactively with other affective properties in psychopathological states.

Moreover, the affective value of novelty suggests a new dimension for understanding the links between affect and memory. Studies of memory routinely compare novel and familiar stimuli. The fact that novel stimuli engage affective circuitry independently of the more traditional affective dimensions of valence and arousal suggests that affective significance may play a role in memory formation regardless of the overt affective content of the stimuli. In the current study, this is supported by greater hippocampal activation for novel versus familiar stimuli. Our findings suggest the possibility of a more basic framework for understanding affective and memory processing that centers around stimulus novelty.

Figure 6.

Whole brain analyses with Bonferroni correction for multiple comparisons. Panels illustrate (a) left lateral and (b) left medial whole brain activation for each contrast of interest; for purposes of comprehensive whole-brain visualization, these images include activation that did not meet the stricter cluster criteria (i.e., 10 contiguous voxel extent) as reported in Table 1. Note that the high versus mid arousal images reflect a lower exploratory threshold. Right hemisphere activation as shown in Table 1 was similar or lesser for each contrast. The third panel (c) shows active subcortical regions for the novel versus familiar contrast; in addition to bilateral amygdala activation, there were significant clusters of activation in thalamus and striatum.