The Neural Temporal Dynamics of the Intensity of Emotional Experience

Abstract

Despite the fact that emotions involve multiple time-varying components, little is known about the underlying neural basis of these temporal dynamics. In this paper, we assess these temporal dynamics by using time-varying hemodynamic response functions (HRF) to model BOLD responses to emotional stimuli. We show that these time-varying HRFs lead to a better fit to the BOLD data and yield larger areas of significant activation than do conventional gamma-based canonical HRFs. We also report for the first time that intensity of emotional experience is associated with both magnitude and duration of brain activation. Specifically, greater negative emotional intensity was associated with greater magnitude of activation in the occipital cortex and with longer duration of activation in regions along the cortical midline associated with self-referent processing: the anterior medial prefrontal cortex and the posterior cingulate cortex. These data significantly advance our understanding of how the brain processes emotion and suggest that the intensity of a negative emotional experience is due in part to elaborative self-referent processing that is captured by the duration of neural activity in cortical midline structures. These data also underscore the importance of using modeling techniques that will help elucidate the chronometry of both normal and psychopathological emotional processes.

Emotional processing involves multiple components, each unfolding in its own time scale (Davidson, 1998; Gross, 2001). For example, whereas the initial appraisal of the salience or valence of an emotional event can occur within hundreds of milliseconds (Schupp et al., 2000), rumination about an emotional event can last weeks (McCullough et al., 2007). Despite the fact that temporal dynamics have long been a part of emotion theories, little is known about their neural underpinnings. To date, only a few functional magnetic resonance imaging (fMRI) studies have addressed the possibility that the neural processes that underlie emotional functioning may vary differentially across time. Indeed, the results of these few studies underscore the importance of assessing neural temporal dynamics for understanding both normal and psychopathological emotional processes, including recovery from threat in resilience (Waugh et al., 2008) and rumination in depression (Siegle et al., 2002). More typically, however, fMRI studies of emotional processing have not examined neural bases of temporal dynamics. In this paper, we argue that assessing neural temporal dynamics will not only lead to a better characterization of the underlying blood-oxygen level-dependent (BOLD) response to emotional stimuli, but will also better elucidate how the brain fundamentally processes emotion.

In typical fMRI studies of emotion, a gamma-based (or similar) canonical HRF is used to model the BOLD response to emotional stimuli. These canonical HRFs are sensitive when the underlying BOLD response follows a gamma-shaped curve and when BOLD differences between conditions of interest are a function primarily of the height of activation. When the BOLD response does not follow this pre-defined shape, however, or when differences between conditions are due to other parameters of the BOLD response (e.g., time-to-peak, width), using these canonical HRFs can lead to mis-modeling (Lindquist and Wager, 2007). For example, true underlying differences in the width of BOLD activation can be mis-modeled as differences in the height of activation (Lindquist and Wager, 2007).

Our primary premise is that because of the time-varying nature of emotions, modeling the underlying BOLD response associated with emotional processing requires flexible estimation of HRFs that can also vary across time. In the current study, therefore, we use inverse logit (IL) modeling to separately estimate the height, time-to-peak, and width parameters of BOLD responses (Lindquist and Wager, 2007). Recent simulation studies have demonstrated that when true differences in these parameters exist in the BOLD data, IL modeling is more accurate and incurs less cross-parameter bias than do other modeling approaches, including the gamma-based canonical HRF and its temporal derivative (Lindquist and Wager, 2007). Although these separate parameters are only loosely indicative of the underlying neural processes hypothesized to be associated with them (i.e., height -> neural firing rate, width -> duration of neuronal firing), data from both simulation (Lindquist and Wager, 2007) and functional imaging (Bellgowan et al., 2003) studies suggest that these associations can be inferred given certain fMRI design restrictions (see Materials and methods).

One aspect of emotion in particular that would likely benefit from assessing its underlying neural temporal dynamics is the intensity of emotional experience. People report experiencing more intense emotional responses when they elaborate on the self-relevance and importance of the emotional events (Larsen and Diener, 1987). Consistent with this formulation, neuroimaging studies have shown that the brain regions involved when participants rate emotional intensity (Heinzel et al., 2005), or when emotional intensity is manipulated (Ochsner et al., 2004), fall along the cortical midline – the area of the brain associated most consistently with self-referential thought (Northoff et al., 2006). In these and other fMRI studies examining emotional intensity, researchers have operated under the reasonable assumption that the elaboration of self-relevance underlying these intense emotional experiences is reflected by the height of the BOLD response in these midline regions. There is evidence, however, that the elaboration of self-relevance that leads to the experience of greater emotional intensity may be better captured by examining the duration rather than the magnitude of emotional processing. Investigators have found that emotion duration is a significant predictor of emotional intensity (Sonnemans and Frijda, 1994), and that both duration and intensity of emotion are predicted by the frequency of rumination about an event, a principal form of emotional elaboration (Luminet et al., 2000; Nolen-Hoeksema and Morrow, 1993). Thus, we should gain important information about the neural processes underlying the experience of emotion by assessing the width of BOLD activation, a potential proxy for the duration of emotional processing.

In the current study, we compare two HRF modeling approaches in examining whether modeling emotional intensity with flexible time-varying HRFs provides novel information about the neural processes underlying the experience of emotion. Participants viewed emotional and non-emotional images and rated the intensity of their emotional response to the images. We use IL modeling to estimate separately the height, time-to-peak, and width of the BOLD response, although we focus mainly on height and width given our hypotheses below, and we compare this modeling approach with the more standard gamma-based canonical HRF approach (GAM) in which only height of BOLD activation is estimated. We hypothesize that because the IL model estimates time-varying HRFs, it will result in a better fit than the GAM model to the BOLD responses to the emotional stimuli. We further hypothesize that increased emotional intensity will be reflected by greater width of BOLD activation in cortical midline regions associated with elaborating on the self-relevance of emotional stimuli (Heinzel et al., 2005; Northoff et al., 2006; Ochsner et al., 2004).

Materials and methods

Participants

Twenty-four participants (13 females) were recruited through flyers around Stanford University’s campus as well as online community postings. Inclusion criteria required that all participants: (1) were between the ages of 18 and 50 years (M = 33.6, SE= 1.99); (2) had no reported history of brain injury, social phobia, mania, or post-traumatic stress disorder; (3) did not meet diagnostic criteria for current generalized anxiety disorder or current major depressive disorder; (4) had no reported substance abuse within past six months; and (5) had no physical limitations that prevented them from entering the MRI machine. A limited portion of the data from 12 of these participants have been presented elsewhere (Hamilton and Gotlib, 2008), but does not overlap with the data presented in this report. Informed consent was obtained from all participants, and all aspects of the study complied with ethical standards as outlined by the Stanford University IRB.

Picture Viewing Task

Participants viewed stimuli during scanning through a mirror directed at a video projector. The stimuli were selected from the International Affective Picture System (IAPS; Lang et al., 1997). Each trial consisted of four parts: 1) picture presentation (2s); 2) self-paced picture affective intensity rating (1 – not intense, 2 – somewhat intense, 3 – quite intense, 4 – extremely intense); and 3) self-paced affective valence rating of the picture (1 – negative, 2 – neutral, 3 – positive); 4) fixation cross viewing for the remainder of the trial. The trials were 14 seconds long to provide enough time between pictures for relatively bias-free estimation of width using the IL model (Lindquist and Wager, 2007). For the affective intensity ratings, participants were instructed to focus on and rate their emotional response to the pictures.

Each participant viewed 70 normed negative (mean valence: 2.60; range: 1.3 – 3.9), 70 neutral (mean valence: 5.05; range: 4.3 – 5.8), and 70 positive (mean valence: 7.30; range: 6.7–8.3) pictures over the course of five 588-second scanning runs. Each participant viewed the stimuli in random order.

FMRI Data Acquisition

BOLD data were acquired with a 1.5 T General Electric Signa MR scanner. Following scout scanning, high order shimming was performed over the whole brain until diminishing returns on image distortion correction were met. Next, BOLD data were acquired with a single channel, whole-head imaging coil from 24 axial slices using a spiral pulse sequence (Glover and Law, 2001) [repetition time (TR) = 83 ms/slice, echo time (TE) 40 ms, flip angle = 70°, field of view (FOV) = 24 cm, acquisition time = 2000 ms per frame, number of frames = 299 per run]. Axial slices had 3.75 mm² in-plane and 4 mm through-plane resolution (with 1 mm between-slice distance). A high resolution structural scan (115 slices, 1 mm² in-plane and 1.5 mm through-plane resolution, TE = min, flip angle = 15°, FOV = 22cm) was performed following BOLD scanning runs. Head movement was minimized by using a bite-bar formed with each participant’s dental impression.

BOLD Data Analysis: Preprocessing

BOLD images were slice-time corrected using the thirteenth axial slice as the reference slice. Images were then motion corrected using a Fourier interpolation procedure (3dvolreg) from the AFNI imaging analysis suite (Cox, 1996). Data for which sudden movement did not exceed 1mm were not corrected further. Scans for which sudden movement was greater than 1mm were corrected with AFNI’s 3dDespike, an algorithm that replaces data from individual high motion acquisitions with data estimates from an outlier insensitive multi-parameter model of the fMRI time series. Data were then spatially smoothed with a Gaussian kernel (full width at half maximum = 4 mm) and high-pass filtered with a frequency criterion of one cycle per minute. Next, the data were converted to units of percent signal change and temporally smoothed with a 6s exponential kernal. Finally, the BOLD data were warped to a common template space (Talairach and Tournoux, 1988) and converted to SPM analyze format for the IL analysis.

Functional analysis

The trials were categorized according to each participant’s idiographic responses to the pictures. For pictures that the participants identified as positive or negative, we categorized their reactions as being high in intensity (intensity rating of 3 or 4; HiNeg, HiPos) or low in intensity (intensity rating of 1 or 2; LoNeg, LoPos). The final category consisted of pictures that participants identified as neutral. Eight participants were excluded from the positive emotion intensity analyses because they had an insufficient number of HiPos trials (< 5). When examining the influence of affect intensity on brain activation, researchers often use parametric modulation as a regression approach (e.g., Phan et al., 2004) in which intensity values are convolved with an HRF and regressed against the data. This approach, however, relies on the use of a canonical HRF for which affect intensity values vary along a single parameter: height. In the IL model, we estimated multiple parameters for each HRF and did not want to impose constraints on which parameter could covary with affect intensity. Therefore, we separately estimated the parameters of each HRF for each picture type (e.g., HiNeg) and used post-estimation statistical analyses to examine the relations between these parameters and affect intensity.

IL modeling

SPM2 (Wellcome Department of Cognitive Neurology) was used, along with custom routines, to estimate the HRF to each picture type in each voxel. For the IL modeling approach, three parameters (height, time to inflection, and slope of inflection) were estimated for each of the IL functions using an optimization algorithm that minimized sum of squares error (SSE). For this optimization algorithm, we chose to use a fast deterministic solution (instead of stochastic solution, which is slower but less prone to finding local minima) with convergence after 2000 iterations. To balance flexibility with parameter interpretability, we chose the initial parameters (V₀ = 2.5, 2.5, .2, 1.3, 5, .7, 8) to match the shape of a 12 TR (24s) canonical gamma-based HRF, and the boundary limits of the parameters (V_lb:ub = .05:6, 1:5, 0:2, .01:2, 2.5:7.5, .05:3, 3:10) wide enough to flexibly capture time-varying components of the HRF, but narrow enough to avoid fitting the HRF to noise. The resulting IL functions were then summed to create the HRF from which the height (H), time-to-peak, and width at half-height (W) of the BOLD responses were estimated.

Gamma modeling

For the gamma modeling approach, we used SPM2’s canonical HRF, consisting of a mixture of two gamma functions. We used the same optimization algorithm described above, but allowed only a single parameter – height – to vary.

We adopted a two-step statistical approach to classify significant voxels. First, to identify voxels that were responsive to intense affect, we created linear contrasts (1 0 -1) in H and W for positive and negative pictures: [HiNeg vs. LoNeg vs. neutral], [HiPos vs. LoPos vs. neutral]. ¹ For the group analysis, we used robust regression at the 2nd level (Wager et al., 2005) to conduct random effects analyses on the linear contrasts. Robust regression minimizes the influence of outliers at a small cost in power relative to ordinary least squares when statistical assumptions are met. To constrain our analyses to potential sites of interest, we used a large ‘emotional brain’ mask derived from a meta-analysis of emotion fMRI studies (Wager et al., in press). Results for the initial linear contrasts were thresholded using a combined cluster size and per-voxel threshold of .005 to render a cluster-level corrected p-value of .05 (based on Monte Carlo simulations; Ward, 2000). This led to a minimum cluster size of 8 voxels for the ‘emotional brain’ mask.

In the second step, we conducted follow-up pairwise t-tests between trial-types (e.g., HiNeg vs. LoNeg, LoNeg vs. neutral) for the clusters that showed a significant linear contrast effect. Together with the initial linear contrasts, these follow-up tests were necessary to determine whether these clusters were responsive to the intensity of the affect (e.g. HiNeg > LoNeg) or to any level of affect (e.g. HiNeg = LoNeg). For these follow-up t-tests, results were thresholded at a bonferroni-corrected p < .01 (α = .05 / 5 tests).

To compare the gamma model (GAM) to the inverse-logit model (IL), we fitted each participant’s estimated HRF for each model to his or her time-series data and calculated the sum of squares error (SSE). Next, at the group level, we performed a pairwise t-test between the SSE from the GAM and IL models and thresholded the results at the same level reported above (p < .005, k = 8).

Results

Behavioral data

Participants rated a greater percentage of pictures as HiNeg (M = 19.1%, SE = 1.4%) than LoNeg (M = 13.4%, SE = 1.4%), t(23) = 2.17, p = .04, but a lower percentage of pictures as HiPos (M = 7.8%, SE = 1.4%) than LoPos (M = 25.5%, SE = 1.8%), t(23) = 6.57, p < .001. Participants rated LoNeg pictures (M = 1.81, SE = .03) as more intense than they did LoPos pictures (M = 1.61, SE = .06), t(23) = 3.33, p = .003, both of which were rated as more intense than were neutral pictures (M = 1.36, SE = .05), ts(23) = 7.30, 3.80, ps < .001 for LoNeg and LoPos, respectively.²

Neuroimaging data: IL model

Negative emotion intensity

Several brain regions that have been implicated in emotion processing showed significantly greater height of activation to the high intensity negative (HiNeg) vs. neutral pictures, including dorsal MPFC (dMPFC), insula, thalamus/mid-brain, posterior cingulate cortex (pCC), and occipital cortex (Table 1; Figure 1). Pairwise t-tests revealed, however, that most of these regions showed greater height of activation to any level of affect. Only a region of the inferior occipital cortex exhibited greater height of activation to HiNeg than to LoNeg pictures.

Table 1.

Regions shown by the IL model to have significantly greater height and/or width of activation to high intense negative than neutral pictures

	Cluster		Height in % signal change (t)			Width in TRs (t)
Region	Center (x,y,z)	Voxels	HiNeg > Neutral	HiNeg > LoNeg	LoNeg > Neutral	HiNeg > Neutral	HiNeg > LoNeg	LoNeg > Neutral
Clusters significant for height
dMPFC	2,48,23	16	5.54***	0.14	4.47**	3.34*	1.08	2.83*
STG	42,13,−20	9	4.62**	0.06	5.52***	1.64	1.75	0.1
Insula	−41,11,−3	160	7.23***	0.51	4.56**	4.73***	1.39	2.96*
IFG	47,8,18	12	4.93***	0.57	5.76***	0.94	0.77	1.76
Insula	40,2,−8	39	6.83***	1.13	4.39**	1.04	1.32	0.06
Thal.	−4,−21,3	275	8.71***	1.45	6.56***	3.64*	3.22*	0.51
PCC	−5,−45,35	12	4.38**	0.72	2.65	5.02***	2.82*	0.78
Fus.G/ITG	−45,−56,−10	34	4.96***	1.77	2.43	3.05*	1.41	1.96
IOcc	−52,−67,12	133	12.39***	1.87	4.64**	4.82***	1.28	1.91
MOcc	43,−71,19	43	7.03***	1.05	3.28*	2.35	1.02	1.57
IOcc	39,−75,−2	19	5.07***	2.91*	2.11	3.03*	1.83	0.51
Clusters significant for width
aMPFC	2,47,13	13	2.36	0.54	2.08	7.35***	4.48**	1.57
Ins./IFG	−45,14,−7	21	4.49**	0.47	4.14**	9.04***	2.3	4.6**
Thal.	−4,−31,2	9	5.77***	1.11	5.31***	3.35*	2.96*	0.8
PCC	−4,−47,31	14	3.62*	0.26	3.28*	7.63***	2.97*	3.11*
MTG/STG	−55,−68,17	10	7.03***	1.75	3.65*	5.17***	1.12	2.23

Note. IL = inverse logit; HiNeg = high intensity negative pictures; LoNeg = low intensity negative pictures. dMPFC = dorsal medial prefrontal cortex; STG = superior temporal gyrus; IFG = inferior frontal gyrus; Thal. = thalamus; PCC = posterior cingulate cortex; Fus.G. = fusiform gyrus; ITG = inferior temporal gyrus; IOcc = inferior occipital cortex; MOcc = middle occipital cortex; aMPFC = anterior medial prefrontal cortex; Ins. = insula; MTG = middle temporal gyrus; STG = superior temporal gyrus.

^***p< .0001,

^**p < .001,

^*p < .01.

Figure 1.

Brain regions showing significantly greater height of activation (red), width of activation (yellow), and both (orange) to the HiNeg (vs. neutral) pictures. The line graphs are the estimated HRFs from the aMPFC and dMPFC, which show non-overlapping differences in activation width and height, respectively (shown in the bar graphs). All error bars are standard error of the mean. Solid lines above bars represent significant differences at p < .01.

Regions that showed significantly longer width of activation to the HiNeg (vs. neutral) pictures included the anterior MPFC (aMPFC), anterior insula, thalamus/mid-brain, pCC and middle/superior temporal gyrus (Table 1; Figure 1). Consistent with our hypothesis, the cortical mid-line regions (aMPFC, pCC) discriminated HiNeg from LoNeg pictures by exhibiting longer width of activation to the more intensely emotional negative pictures. The thalamus also showed this pattern.

There was substantial overlap between the regions that showed greater height and longer width of activation to the HiNeg than to the neutral pictures. For example, a conjunction analysis indicated that 30 of the 67 voxels showing longer width of activation to the HiNeg pictures also exhibited greater height of activation. There was a dissociation, however, between height and width in the MPFC. There was no overlap between the dMPFC, which showed greater height of activation, and the aMPFC, which showed greater width of activation. Even at the lower threshold of the pairwise t-tests, the aMPFC is the only region from both the height and width analyses that did not exhibit significantly greater height of activation to the HiNeg pictures (Figure 1).

Positive emotion intensity

Greater height of activation to the high intensity positive pictures (HiPos) vs. neutral pictures was found in the ventral anterior cingulate cortex, inferior frontal gyrus, mid-insula, thalamus, superior temporal gyrus, hippocampus, and middle occipital gyrus (Table 2). In contrast to the results obtained with negative emotion intensity, most of these regions (except superior temporal gyrus and middle occipital gyrus) discriminated emotional intensity by exhibiting greater height of activation to the HiPos than to the low intensity positive (LoPos) pictures.

Table 2.

Regions shown by the IL model to have significantly greater height and/or width of activation to high intense positive than neutral pictures

	Voxels		Height in % signal change (t)			Width in TRs (t)
Region	Center (x,y,z)	Number	HiPos > Neutral	HiPos > Neutral	LoPos > Neutral	HiPos > Neutral	HiPos > LoNeg	LoPos > Neutral
Clusters significant for height
vACC	8,31,−2	15	5.10**	3.83*	2.08	1.22	0.31	1.26
IFG	47,5,16	21	5.12**	4.49**	1.04	1.46	2.88	1.33
STG	41,5,−24	11	4.51**	2.80	3.17*	0.69	0.74	0.31
Ins./IFG	−44,5,8	24	6.26***	5.39***	1.88	0.44	0.64	0.21
Thal.	−8,−20,4	20	5.56***	3.78*	2.45	0.47	0.37	0.86
Hipp.	26,−20,−8	8	5.40***	4.30**	0.19	1.04	0.94	0.07
MOcc	−54,−73,8	8	4.26**	2.41	2.01	2.54	0.83	2.14
Clusters significant for width
No significant clusters

Note. IL = inverse logit; HiPos = high intense positive pictures; LoPos = low intense positive pictures. vACC = ventral anterior cingulate cortex; IFG = inferior frontal gyrus; STG = superior temporal gyrus; Ins. = insula; Thal. = thalamus; Hipp. = hippocampus; MOcc = medial occipital gyrus.

^***p< .0001,

^**p < .001,

^*p < .01.

There were no regions that exhibited a longer width of activation to the HiPos than to the LoPos pictures.

Gamma model

Negative emotion intensity

There was extensive overlap among the regions that exhibited greater height of activation to the HiNeg pictures in the GAM model and those in the IL model (Table 3). The only differences were the addition of the precuneus and the omission of superior temporal gyrus in the GAM model. Also similar to the IL model results, the only regions that showed differences in height of activation to the HiNeg and LoNeg pictures were two posterior perceptual regions (fusiform gyrus, inferior occipital gyrus).

Table 3.

Regions with significant activation using the gamma HRF to high intensity negative and positive vs. neutral pictures

	Voxels		Height in % signal change (t)
Region	Center (x,y,z)	Number	HiEmo > Neutral	HiEmo> LoEmo	LoEmo > Neutral
Negative v. Neutral
Anterior MPFC	−1,49,20	22	5.07***	1.02	2.06
Insula	37,14,−6	10	3.63*	1.57	1.93
Insula	−39,10,−4	91	6.40***	0.82	2.49
Inferior frontal gyrus	47,4,19	8	3.00*	0.99	1.47
Thalamus/Mid-brain	−2,−24,4	149	10.04***	1.76	5.56***
Posterior Cingulate	−4,−43,36	21	5.15***	1.21	1.36
Fusiform/Inferior temporal gyrus	−45,−54,−9	33	4.73***	3.69*	2.18
Precuneus	−8,−57,31	8	3.81**	0.15	1.92
Middle Temporal/Inferior Occipital	41,−68,1	12	5.14***	3.12*	1.31
Middle Temporal/Middle Occipital	−52,−68,11	74	10.56***	2.54	2.74
Middle Occipital	42,−72,18	36	5.60***	1.61	1.98
Positive v. Neutral
Inferior Frontal Gyrus	47,4,17	12	4.17**	4.51**	0.21

Note. HiEmo = high intensity negative or positive pictures depending on the contrast; LoEmo = high intensity negative or positive pictures depending on the contrast.

^***p< .0001,

^**p < .001,

^*p < .01.

Positive emotion intensity

Unlike the results from the IL model, there was only one region – inferior frontal gyrus – that showed significantly greater height of activation to the HiPos pictures in the GAM model. Similar to the IL model results, however, this region also exhibited greater height of activation to HiPos than to LoPos pictures.

IL vs. GAM

Consistent with our hypothesis that the IL model would fit the data better than would the GAM model, all but 3 of 8215 voxels showed less sum of squares error (SSE) with the IL model, t(8214) = 50.04, p < .0001. Even when submitting the SSE data to statistical thresholding, 5,363 voxels showed significantly better fit for the IL model than for the GAM model, and no voxels showed the reverse pattern (Figure 2).

Figure 2.

Brain regions showing significantly better fit (less SSE) of emotional responding to the BOLD data using the inverse-logit model (IL) vs. the gamma-based canonical HRF model (GAM). No regions showed poorer fit with the IL model. The line graph provides an example of the better fit associated with the IL model than the GAM model by highlighting responses to the high intense negative images in the left insula.

Finally, when comparing the regions that exhibited greater height of activation to the HiNeg pictures in either the IL or GAM model, there was a substantial number of voxels (355) that showed significant effects in both models. It is important to note, however, that the IL model yielded many more unique voxels (397) than did the GAM model (109). Indeed, compared with the GAM model, the IL model yielded larger clusters of significant voxels in 16 of the 19 total regions from the HiNeg and HiPos analyses (Table 4).

Table 4.

Comparison of the regions found significant using the GAM and IL models (only height is presented).

Region	Coordinates: IL (GAM)			Cluster size (voxels)
	x	y	z	IL	GAM	IL vs. GAM
Negative v. Neutral
Anterior MPFC	2 (−1)	48 (49)	23 (20)	16	22	GAM
Insula	40 (37)	2 (14)	−8 (−6)	39	10	IL
Insula	−41 (−39)	11 (10)	−3 (−4)	160	91	IL
Inferior frontal gyrus	47 (47)	8 (4)	18 (19)	12	8	IL
Thalamus/Mid-brain	−4 (−2)	−21 (−24)	3 (4)	275	149	IL
Posterior Cingulate	−5 (−4)	−45 (−43)	35 (36)	12	21	GAM
Fusiform/Inferior temporal gyrus	−45 (−45)	−56 (−54)	−10 (−9)	34	33	IL
Middle Temporal/Inferior Occipital	39 (41)	−75 (−68)	−2 (1)	19	12	IL
Middle Temporal/Middle Occipital	−52 (−52)	−67 (−68)	12 (11)	133	74	IL
Middle Occipital	43 (42)	−71 (−72)	19 (18)	43	36	IL
Superior temporal gyrus	42	13	−20	9		IL
Precuneus	(−8)	(−57)	(31)		8	GAM
Positive v. Neutral
Inferior frontal gyrus	47 (47)	5 (4)	16 (17)	21	12	IL
Ventral ACC	8	31	−2	15		IL
Superior Temporal	41	5	−24	11		IL
Mid-insula/Inferior Frontal Gyrus	−44	5	8	24		IL
Thalamus	−8	−20	4	20		IL
Hippocampus	26	−20	−8	8		IL
Middle Occipital gyrus	−54	−73	8	8		IL

Note. IL = inverse logit; GAM = gamma.

Discussion

A primary goal of affective neuroscience is to understand how the brain processes emotion; importantly, however, much of affective neuroscience has not addressed the temporal features inherent in emotional processing. The data from the current study suggest that, relative to traditional canonical HRF approaches, modeling fMRI data with flexible, multiple-parameter models that characterize independent temporal features of the BOLD response may better inform us about the chronometry of affective processing. We compared the canonical gamma-based HRF, which models only the height of BOLD activation, and the IL model (Lindquist and Wager, 2007), a flexible time-varying HRF able to capture differences in the width and time-to-peak in addition to differences in the height of the BOLD response. When modeling the BOLD response to emotionally intense stimuli, the IL model showed a remarkably better fit than the gamma-based HRF model. The IL model also revealed a more extensive network of brain regions associated with processing emotionally intense stimuli by identifying regions that responded with longer width of activation, a temporal feature the canonical HRF model is unable to assess. Even when estimating height of activation, the IL model detected a greater number of significant voxels, possibly due to increased sensitivity for detecting peaks whose onset varied from voxel to voxel.

Beyond additional information about where emotional intensity is processed in the brain, assessing neural temporal dynamics also revealed information about how emotional intensity is processed in the brain. When participants viewed negative emotional images, most of the brain regions (except for a portion of the occipital cortex) that showed greater height of activation did not differentiate between high- and low-intensity emotional reactions to those images. Instead, this discrimination between high and low emotional intensity was found in regions along the cortical midline (aMPFC, PCC) that showed longer activation width. These cortical midline regions are associated with self-relevant processing (Northoff et al., 2006; Ochsner et al., 2004), supporting our hypothesis that processing emotional intensity is due in part to the elaboration of self-relevant processing (Larsen and Diener, 1987). Importantly, there was a dissociation between height and width of the activation in the medial prefrontal cortex, with a more dorsal region exhibiting increased height of activation and a more ventral/anterior region showing longer width of activation. These adjacent regions are theorized to be involved similarly in self-referent processing (Amodio and Frith, 2006). Our results suggest the possibility that within the broader region associated with self-mentalizing, the more dorsal regions are involved in the identification that a stimulus is self-relevant, whereas the more ventral regions are involved in elaborating on this self-relevance.

We did not directly assess the self-relevance of stimuli in this study; our interpretation of the data, therefore, requires further examination. For example, functional neuroimaging studies are, by their nature, correlational. Consequently, to elucidate the direction of the relation between self-referent processing and emotional intensity, behavioral studies are needed in which these variables are manipulated and their reciprocal effects are assessed. In addition, it is important to note that the cortical midline structures that were found to show longer activation width in the current study (especially the aMPFC) are involved in a broader social cognition network including thinking or reflecting about others and as well as about oneself (Amodio and Frith, 2006). It is possible, therefore, that the longer width of activation in these structures may have been a result of elaborating on the emotions and thoughts of people depicted in the stimuli. Another possibility is that these cortical midline structures were not associated with elaboration of self- or other-referent processing, but instead, reflected elaborated processing of other facets of the emotional images, such as their visual salience and/or complexity. Although we consider the self-referent processing interpretation more parsimonious and consistent with previous findings, these alternative explanations should be considered until future experimental studies are conducted in which the self-relevance of emotional stimuli is manipulated and both emotional intensity and neural temporal dynamics are assessed.

Assessing neural temporal dynamics also revealed potentially integral information about how the brain processes positive emotion. First, the IL model detected nine times more significant voxels in six additional regions than did the gamma-based model. This suggests that canonical HRFs mis-model the underlying BOLD response to positive emotional stimuli, and that modeling these responses with time-varying HRFs is critical to understanding how the brain processes positive emotions. Second, unlike the case with negative emotion, there were no regions that showed longer width of activation to positive emotional images (vs. neutral images). Consequently, increasing positive emotional intensity was associated only with greater height of activation. This lack of longer activation width to positive emotion may be because these images induced less elaborative processing than did the negative emotional images. This formulation is consistent with findings that positive emotions are typically weaker and less immediately impactful than are negative emotions (Baumeister et al., 2001), and that positive emotions induce broader and less self-focused thinking (Fredrickson, 1998). Indeed, participants in the current study rated a smaller percentage of pictures as being high in intense positive emotion than they did any other category, and rated positive pictures as less intense than they did negative pictures. Future investigations should induce stronger positive emotions that are more likely to induce elaborative self-relevant processing (e.g., showing pictures of babies to their mothers; Nitschke et al., 2004) to elucidate neural temporal dynamics that may underlie positive emotions.

These results have implications for a wide range of studies examining both normal and psychopathological emotional processes. A particularly salient example is emotion regulation. In fMRI studies of emotion regulation, successful down-regulation is typically measured as decreases in the intensity of emotional experience coupled with regions in which height of activation also decreases (e.g. Ochsner et al., 2002). Recent data suggest, however, that emotion regulation can also influence the timing of neural activity (Goldin et al., 2008). Our data complement these findings and suggest that studies of emotion regulation should also assess potential decreases in the width of activation as a result of successful decreases in emotional intensity. This is especially important given that gamma-based HRF models may erroneously interpret true decreases in width of activation as decreases in height of activation (Lindquist and Wager, 2007).

One field of inquiry that has shown significant advances in assessing neural temporal dynamics is the examination of the temporal dynamics of emotional experience underlying psychopathology (Larson et al., 2006; Siegle et al., 2002). For example, phobia is associated with the faster onset of amygdala activity to threatening images (Larson et al., 2006), and depression is characterized by prolonged amygdala response (Siegle et al., 2002). The current study can make significant contributions to this literature by providing a flexible modeling technique that can be easily applied to studies examining the neural temporal dynamics of emotional processing in psychopathology.

The relation between neural temporal dynamics and BOLD temporal dynamics is still unclear because there is only an indirect relation between neuronal firing and BOLD response (Logothetis and Wandell, 2004). The coupling between neuronal firing and BOLD relies on a wide range of factors, including aspects of stimulus type and duration (Logothetis and Wandell, 2004), inter-neuron coupled firing within a population of neurons (Nir et al., 2007), and the saturation of the vascular response over time (Vazquez et al., 2006). Recent studies have found that a high-band gamma local field potential (LFP) is highly coupled with BOLD response (Nir et al., 2007), and that this LFP can be assessed at a much higher temporal resolution than BOLD (Edwards et al., 2005), thus offering a more finely tuned assessment of neural temporal dynamics. It is important to note, however, that the coupling between neuronal firing rate and LFP is subject to many of the same factors as the coupling between neuronal firing rate and BOLD (e.g., inter-neuron correlations; Nir et al., 2007).

Although the relation between neural temporal dynamics and BOLD temporal dynamics remains unclear, there is evidence that temporal characteristics of the BOLD response maps onto temporal properties of psychological processes. For example, Bellgowan et al. (Bellgowan et al., 2003) found that the onset and width of BOLD activation corresponded to when and for how long a word was processed. Therefore, although we may not yet be able to map the activation width differences in the current study onto underlying differences in neuronal firing duration, the results of Bellgowan et al.’s study allow us to propose mapping these width differences onto psychological processes – in this case, elaboration of self-relevant processing.

In summary, researchers have long acknowledged that emotions evolve over time. We hypothesized that time-varying HRFs more accurately capture these temporal dynamics of emotional processing than do canonical HRFs. The present data supported this hypothesis and showed that compared with the canonical gamma-based HRF, the IL model (one such time-varying HRF) modeled variations in emotional intensity with less error and with more robust detection of significantly active brain regions. In addition, only the IL model was able to detect regions with differing widths of activation. Assessing this temporal component of the BOLD responses provided novel insight into how the brain processes emotional intensity by showing that greater negative emotional intensity was associated with longer width of activation in regions along the cortical midline. These data have implications for future examinations of the neural temporal dynamics of both normal and psychopathological emotional processes.

Abbreviations

BOLD

blood-oxygen level-dependent

fMRI

functional magnetic resonance imaging

HRF

hemodynamic response function

IL

inverse logit