Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2008 Mar 12;28(11):2793–2803. doi: 10.1523/JNEUROSCI.3476-07.2008

Rapid Interactions between the Ventral Visual Stream and Emotion-Related Structures Rely on a Two-Pathway Architecture

David Rudrauf 1,2,✉, Olivier David 3,7, Jean-Philippe Lachaux 6, Christopher K Kovach 2, Jacques Martinerie 1, Bernard Renault 1,5, Antonio Damasio 4
PMCID: PMC6670659  PMID: 18337409

Abstract

Visual attention can be driven by the affective significance of visual stimuli before full-fledged processing of the stimuli. Two kinds of models have been proposed to explain this phenomenon: models involving sequential processing along the ventral visual stream, with secondary feedback from emotion-related structures (“two-stage models”); and models including additional short-cut pathways directly reaching the emotion-related structures (“two-pathway models”). We tested which type of model would best predict real magnetoencephalographic responses in subjects presented with arousing visual stimuli, using realistic models of large-scale cerebral architecture and neural biophysics. The results strongly support a “two-pathway” hypothesis. Both standard models including the retinotectal pathway and nonstandard models including cortical–cortical long-range fasciculi appear plausible.

Keywords: attention, emotion, limbic, magnetoencephalography, network, visual

Introduction

Visual attention and awareness are modulated by multiple top-down influences (Kastner and Ungerleider, 2000). Emotion-related processing plays an important role in these influences, and can modulate visual attention in a highly dynamic manner through the rapid extraction of relevant information present in the stimuli before full-fledged visual processing (Anderson, 2005; Vuilleumier, 2005).

The amygdala, temporal pole, and orbitofrontal cortex, which we will refer to as the “anterior affective system” (AAS), are likely to be central in this process: the AAS is critical for emotion processing, receives visual afferents, and can modulate visual processing through feedback to the ventral visual stream (VVS) (Damasio, 1994; LeDoux, 1998; Rolls, 2000; Miller and Cohen, 2001; Kringelbach and Rolls, 2004; Vuilleumier et al., 2004; Adolphs et al., 2005; Vuilleumier, 2005). The AAS also exhibits early responses to visual stimuli, which are modulated by the emotional significance of the stimuli (Kawasaki et al., 2001; Eimer and Holmes, 2002; Pizzagalli et al., 2002; Streit et al., 2003), and there is indirect evidence that it interacts concurrently with the VVS (Bar et al., 2006).

The mechanisms and routes by which visual information rapidly reaches the AAS to allow for early modulatory feedback of the VVS remain an open question. Two main classes of models have been proposed (Vuilleumier, 2005): (i) “two-stage models,” involving rapid sequential processing along the VVS, followed by integration in the AAS and secondary feedback from the AAS to the VVS; and (ii) “two-pathway models,” which include additional parallel short-cut pathways reaching the AAS more directly.

Studies on covert emotional visual attention have provided indirect and restricted support for the two-pathway hypothesis, mainly by using negatively valenced faces presented under masking conditions (Morris et al., 1999; Pourtois et al., 2004; Liddell et al., 2005). However, whether or not these findings apply to visual processing of emotion-related information in general remains to be established (Vuilleumier, 2005).

Here, building on new methodological developments (David et al., 2006), we tested which type of architecture most likely operates in the human brain. We generated realistic models of neural biophysics and alternative large-scale cerebral architectures using dynamic causal modeling (DCM) applied to magnetoencephalography (MEG) (David et al., 2006). Simulations of neural dynamics based on these models were used to predict real MEG responses in subjects presented with a broad range of unmasked arousing visual stimuli. The ability of the models to reproduce the measured signals with parsimony was compared using Bayesian inference.

The results strongly and reliably support a “two-pathway” hypothesis. Two-pathway models relying on cortical–cortical long-range fasciculi predicted MEG data comparably with more standard two-pathway models, which included a “retinotectal” subcortical pathway from the retina to the amygdala (Weiskrantz et al., 1974; Morris et al., 1999; Liddell et al., 2005). These long-range fasciculi are anatomically well established (Catani et al., 2003) and are equipped to convey large amounts of visual information, yet have not previously been proposed to play a role in rapid emotional visual attention.

Materials and Methods

Subjects

Fifteen normal right-handed male volunteers, mean age 26 (range: 20–45), participated in the experiment. The subjects had no history of neurological or psychiatric disease and had normal or corrected-to-normal vision. All the subjects gave written consent to participate in the study and were paid for their time. The design and procedures of the experiment, which took place in Paris, were approved by the French national ethical committee.

Stimuli and procedure

Sixty pleasant, 60 unpleasant, and 60 neutral movies depicting complex visual scenes, each with a duration of 10 s, were presented in random order on a screen placed in front of the subjects in the MEG-shielded room. We chose the classes of stimuli for the pleasant and unpleasant conditions that elicit the strongest and most reliable emotional responses in normal adult male subjects (Bradley et al., 2001). These classes of stimuli have also been found to be involved in early emotional modulation of visual attention (Anderson, 2005). The pleasant movies depicted erotic content (see note in supplemental material, available at www.jneurosci.org), the unpleasant movies depicted fearful or disgusting content, and the neutral movies included natural scenes, landscapes, neutral faces, and objects.

Each movie panned across a static color photograph in a smooth and continuous manner (e.g., as in close-up footage from a camera panning slowly across a painting) (Fig. 1a). The movie window was centered on the presentation screen and encompassed ∼10° of visual angle (larger aspect; aspect ratio height/width = 0.79). The amount of displacement of the camera along the background static picture was overall counterbalanced across conditions: there was no significant difference across conditions in the frequency at which the center of the “camera” fell within any of the four quadrants of the underlying static picture (χ(3)2 = 3.26; p = 0.35). Each movie was built using Matlab (The MathWorks, Natick, MA). Brightness and contrast were normalized by equalizing image histograms using Matlab scripts. The frame rate was 30 fps.

Figure 1.

Figure 1.

Open in a new tab

Protocol and stimulus emotional competence. a, Protocol: randomized presentation of neutral, unpleasant, or pleasant movies based on static pictures (n = 3 × 60) during MEG recordings, followed after each trial by subjective ratings, including arousal and valence. b, Left, Pattern of average ratings of arousal and valence across subjects for each movie. Top right, Modulation of heart rate change by stimulus emotional competence. Bottom right, Correlation of valence with heart rate change. All results follow an expected pattern.

The movies were designed so that they would start with frames that unambiguously conveyed the emotional content of the whole sequence (e.g., if the movie panned along a mutilated body, the initial field of view clearly depicted a mutilated body). The amount of displacement of the camera during each movie was voluntarily kept small. In general, most of the salient/central/meaningful elements of the underlying picture were always apparent in the field of view, and the most salient information was never fully outside of the field of view. This design was aimed at limiting the effect of anticipation/surprise for elements that came into view from outside of the initial field of view, and subjects were made aware of this feature. The experiment was designed to be able to study both: (1) early processing in response to movie onset (during the 800 ms of the time window that constitutes the bulk of the evoked response and that is the focus of this report because of the hypothesis tested); and (2) processing involving a longer time scale, with a special interest in the dynamics of emotional experience, for which the use of movies was critical, but which is beyond the scope of the current article (see supplemental material, available at www.jneurosci.org).

Half of the movies used pictures from the international affective picture system (IAPS) (Lang et al., 2005), selected according to their standard ratings for arousal and valence. We chose the most arousing and most pleasant or unpleasant pictures from the IAPS. The other half used pictures freely available on the Internet. These pictures were selected according to their categorical similarities with those in the IAPS.

A radial grating of contrast with vanishing spatial frequencies and concentric circles was presented before each movie for 5 s and served as a “baseline” stimulus (Fig. 1a).

To minimize eye movements, the outline of an ellipsoid (three pixels of width, ∼5° of visual angle along its largest axis and same aspect ratio as the movie window) was overlaid on the movie window during the entire presentation of the stimuli (Fig. 1a). The ellipsoid was semitransparent so that the movie could still be seen, and was centered with respect to the movie window and the screen. The subjects were instructed to fixate the center of the ellipsoid and to use it as a reference frame to maintain their gaze in a fixed direction. Based on pilot experiments, this semitransparent ellipsoid provided the best solution for preventing subjects' eye movements [measured with electrooculograms (EOGs)], while at the same time allowing them to watch the movie comfortably and with optimal attention. With this solution, subjects tended to report that maintaining their gaze fixed while watching the movie was easy and quasieffortless, and indeed the data were virtually free of eye movement artifacts during the period of interest.

Subjects were instructed at the beginning of each run to stay still, fixate the ellipsoid, pay careful attention to the stimuli, and freely experience any emotional feelings that might occur during the movies. They were also instructed to rate their feelings after each trial using different scales and interfaces that would appear on the screen after each movie presentation. Each subject was trained how to use these scales during a previous session (for more information, see supplemental material, available at www.jneurosci.org).

After each movie presentation, subjects rated the feelings induced by the movie on several scales, using a potentiometer (Fig. 1a). These included scales of valence and arousal, which we use in this report, reproducing the standard self-assessment Manikin (SAM) scales (Bradley and Lang, 1994) (for information about the other subjective ratings, see supplemental material, available at www.jneurosci.org).

The onset of the next trial started 2 s after completion of the ratings. The total rating time after each movie presentation was variable with a duration of at least 30 s, thus allowing for the emotional state to subside (Garrett and Maddock, 2001). The 180 trials were distributed into 12 runs of 15 trials each. All stimuli and scales were presented on a screen in the MEG-shielded room with a video projector outside of the room, via a series of lenses and mirrors.

Recordings

The subjects were tested at the Paris MEG facility (http://web.ccr.jussieu.fr/meg-center) (Université Pierre et Marie Curie, Centre National de la Recherche Scientifique, Inserm, Commissariat à l'Energie Atomique, Hôpital de la Pitié-Salpêtrière), using a 151-axial-gradiometer whole-head MEG array CTF Omega system (VSM MedTech, Coquitlam, British Columbia, Canada). The MEG data sampling frequency was 625 Hz, with an acquisition high-pass filter threshold of 0.65 Hz. Before each acquisition run, the position of the subject's head was located using three fiducial coils placed next to the left and right ears and on the nasion.

An electrocardiogram (EKG) was recorded using two silver chloride electrodes, one at the bottom of the neck on the right, the other on the left lateral costal margin. For artifact correction purposes, vertical and horizontal EOGs were recorded using pairs of silver chloride differential electrodes with bitemporal and subocular-frontal placements, respectively.

T1-weighted magnetic resonance images (MRIs) of the subjects' brains were acquired at the Centre de Neuroradiologie of the Hôpital de la Pitié-Salpêtrière using a 1.5 T GE Signa scanner with a sagittal acquisition (three-dimensional spoiled gradient-recalled acquisition in the steady state; repetition time, 24 ms; echo time, 8 ms; 124 1.3 mm joint slices; 256 × 256; field of view, 25 cm). Fiducial markers clearly visible on structural MRIs were placed at the same locations as the fiducial coils during MEG recording to facilitate coregistration between the MRI and MEG reference systems.

Artifact correction and other MEG preprocessing

Ocular and cardiac artifacts were removed from the MEG data. Blinks were visually identified on the MEG signals with the help of the EOG, and a window was manually defined around the blinks to define their onset and offset. A Savitsky-Golay filter was used to fit polynomials on the blink waveforms within the windows of interest. The blinks were then removed from the signal by local linear decorrelation. Cardiac artifacts were detected using the EKG as a temporal reference. MEG signals were averaged using the peak of the R-wave as a time reference for each MEG channel independently, to extract the typical artifact waveform associated with a given channel, within a given run and a given subject. The channel-specific cardiac waveform was then fitted to each individual cardiac artifact and subtracted from the MEG signals. MEG signals were corrected for interrun head movements using procedures implemented in the BrainStorm toolbox for Matlab (http://neuroimage.usc.edu/brainstorm).

For each subject, artifact corrected MEG signals were averaged across trials within each experimental condition (neutral, pleasant, unpleasant), taking movie onset as the temporal reference, thus generating evoked responses for each subject and each condition. These evoked responses were bandpass filtered between 1 and 15 Hz.

MRI postprocessing and coregistration

The subjects' T1-weighted MRIs were processed for extraction of the individual cortical surfaces using BrainVisa (http://brainvisa.info/), except for two brains that had been previously processed with Freesurfer (http://surfer.nmr.mgh.harvard.edu/). After automatic tissue segmentation, each brain's cortical surface (gray/pial interface) was subsequently triangulated. The resulting dense tessellations were reduced to ∼10,000 vertices for subsequent estimation of the local cortical currents. In the perspective of computing group statistics across the individual activation patterns, spatial normalization of the individual tessellations was achieved following a procedure of landmark-based pseudoelastic warping derived from Drury et al. (1996) and Fischl et al. (1999). Thirty-six anatomical landmarks, consisting of points defined along major sulci and gyri, were manually delineated on each hemisphere for every subject's brain. After projection into Talairach's reference system, each hemisphere was inflated to a sphere. Landmarks of each individual brain were moved toward homologous landmarks from a reference brain (one brain selected from the cohort of subjects). The displacement of the landmarks generated pseudo-body forces, inducing tension in the network and driving the displacement of the other vertices. Registration was achieved when the derivative of the displacement field was below a certain threshold.

EKG processing

To assess the emotional competence of our stimuli, we computed heart rate changes as a function of time and stimulus category. Heart rate was extracted from the EKG using an in-house Matlab program. EKGs were corrected for linear trends. The square of the derivative of the signal was thresholded above two SDs to identify windows of occurrence of the R-waves. The maxima of the detrended EKG within these windows were used to identify the peaks of the R-waves. All the trials for all the subjects were visually checked for correct detection of the peaks. The inverse of the interpeak intervals was computed and expressed in beats per minute, providing a measure of heart rate (for supplemental method, see supplemental material, available at www.jneurosci.org).

Group-level correlations between heart rate change and subjective ratings of valence and arousal were calculated following a standard procedure (Bradley et al., 2001). At each level of valence and arousal of the SAM scales, the corresponding average heart rate change across subjects was calculated. A Pearson correlation was then computed between the average heart rate change and the subjective scale levels for each time point of the trial.

Source reconstruction

We used standard source reconstruction algorithms for selection and basic ordering of regions of interest (ROIs) to inform the basic design of our models. We also used source reconstruction as a first approximation to characterize the effects of emotional competence on cortical responses.

Source reconstruction was applied to the evoked fields obtained for each condition and each subject. The forward and inverse source imaging problems were solved using anatomical priors based on the assumption that the main contributions to the MEG signals originate from cortical currents oriented perpendicularly to the cortical surface (Baillet et al., 2001). The head volume conductor was modeled with three concentric spherical shells, which in MEG versus EEG usually provides a sufficiently accurate approximation (Yvert et al., 1997; Leahy et al., 1998; Darvas et al., 2004). Elementary current-dipole sources were distributed at the nodes of each individual cortical envelope, normal to the cortical surfaces. A gain matrix was computed for each subject and each condition using the quasistatic approximation of Maxwell's equations, using procedures implemented in the BrainStorm toolbox for Matlab (http://neuroimage.usc.edu/brainstorm). The time courses of the current-dipole moments over the cortical surface were estimated using Tikhonov-regularized weighted linear minimum-norm solutions to the inverse problem, also from Brainstorm (Dale and Sereno, 1993; Baillet et al., 2001). The Tikhonov-regularized weighted minimum-norm approach provides an optimal solution when a minimum of a priori information has to be used. It gives the solution X* (the estimated distribution of current densities) corresponding to the distribution of current densities with the maximum likelihood, max(p(X|M)), based on M, the measured MEG signal, the equations of electromagnetism, the possible space of sources, and assumptions on the noise (Baillet et al., 2001). We used the same adaptive values for the Tikhonov regularization parameter across all datasets, that is, 10% of the maximum singular value of the associated lead-field matrix. The weighted minimum-norm involves a reweighting of the gain matrix, aimed at compensating for known bias favoring proximal sources in the original method. For each dipole, the modulus of the current-dipole moment was noise normalized through a z-score procedure using a 500 ms baseline before stimulus onset, providing a measure of normalized dipole strength.

Statistical analyses for source reconstruction results

The general linear model (Myers, 1990) was used to compute statistical parametric maps (SPMs), implementing one-way ANOVAs for repeated-measure design and testing for one-tail specific contrasts such as pleasant minus neutral (P − N) and unpleasant minus neutral (U − N) stimuli, based on the hypothesis of higher MEG activity in response to emotional versus neutral stimuli. We thresholded SPMs based not only on the p values but also on the spatial extent of the activated vertices (i.e., vertices associated with effects reaching the uncorrected threshold of p < 0.05). Only activated vertices belonging to clusters of at least 100 adjacent activated vertices were conserved. We chose 100 vertices because, on average, this encompassed approximately one-half of the cortical surface covered by each ROI involved in our hypotheses, so that sub-ROI resolution could still be achieved. On average, 100 activated vertices corresponded to a mean cortical surface of size of ∼9 cm2, with 6.9 cm2 SD. The mean cortical surface per ROI was ∼19.8 cm2 (∼194 vertices on average), with 9.7 cm2 SD (approximately corresponding to 98 vertices). No further assumption was made about the shape of the clusters because of the high variability in shape of anatomical-functional cortical areas, which, as suggested by Brodmann's cytoarchitectural parcellation, led to the expectation of oblong (e.g., BA 1,2,3) as well as square-like (e.g., BA 40) possible activations if specific anatomic-functional areas were fully activated.

DCM analysis

We assessed which of the tested models best explained the first 800 ms of MEG data evoked by the stimuli. This time window was chosen because it corresponded to the bulk of the evoked response that served as a basis for the DCM analysis. Most of the non-DC components of evoked responses in both MEG and EEG typically occur during the first second after stimulus onset.

Basic principles of DCM

DCM for MEG/EEG has been developed as a generic tool to analyze evoked potentials or fields obtained at the scalp level for any kind of neuropsychological or cognitive experiment. DCM models evoked brain responses as deterministic responses to perturbations induced by inputs (i.e., stimuli), which propagate in neural networks with connected cortical regions capable of context-dependent coupling. DCM for MEG/EEG relies on a neuronally plausible model with parameters that must be estimated to adjust the data. Three main components are involved in this process (see also Fig. 2a):

Figure 2.

Figure 2.

Open in a new tab

Tested models. a, Basic components of the generic model, including all the possible types of connections used in this report, within and between two connected regions. Top, Cortical regions are modeled as three layered columns with three types of neuronal populations (pyramidal, excitatory spiny, and inhibitory interneurons), connected through intrinsic and extrinsic (feedforward and backward) connections. Bottom, The dynamics is mathematically expressed at the level of neural populations and is defined by nonlinear differential equations in which the change of state of each unit dxi/dt depends on its current state xi(t); thalamic inputs ui(t); average firing rate of afferents S(xj(t − δij)); transmission delays δij; forward, backward, and intrinsic effective connectivity matrices CF, CB, Ci, and other parameters. The MEG signal M is assumed to be related to the local average current density x generated by pyramidal populations through a linear forward model M = GX (David et al., 2006). b, Lateral, mesial, and ventral views of the mapping of the regions of interest common to all models on a reference cortical tessellation [for color code, see text and c (top row)]. c, Schematic representation of the architecture of the tested models. All the models share the same basic layout (see text). Null model, Simple feedforward model. Model 1, Adjunction of connectivity modulation. Model 2 (2-stage model), Adjunction of local feedbacks. Model 3 (2-stage model), Adjunction of long-range feedbacks from structures of the AAS. Model 4 (2-pathway model), Adjunction of a direct subcortical retinotectal short-cut pathway to the AAS. Model 5 (2-pathway model), Alternative short-cut pathways to the AAS via the inferior longitudinal and frontal–occipital fasciculi. Model 6 (2-pathway model), Combination of models 4 and 5. Orange circles, “Synapses” at which modulation by emotional competence of the stimuli is implemented.

(1) Biophysically realistic models of neural networks that explicitly simulate causal interactions between populations of cortical neurons (David et al., 2005).

Because MEG/EEG signals are the macroscopic resultant of the activity of millions of neurons, MEG/EEG modeling uses neural-mass models (Lopes da Silva et al., 1974; Jansen and Rit, 1995; Robinson et al., 2001; David et al., 2007). The neural-mass models consist of a set of differential equations that describes different inhibitory and excitatory neuronal populations and their local interactions. The equations also include terms for modeling large scale interactions among cortical–cortical networks, with forward, backward, and lateral connections (for a generic description of the model, see Fig. 2a). The generative model of DCM for MEG/EEG (David et al., 2005) combines the Jansen model (Jansen and Rit, 1995), a neural-mass model originally developed for explaining visual responses, with rules of cortical–cortical connectivity derived from the analysis of connections between the different cortical layers in the visual cortex of the monkey (Felleman and Van Essen, 1991). In the Jansen model, a cortical area is modeled by a population of excitatory pyramidal cells, receiving (i) inhibitory and excitatory feedback from local (i.e., intrinsic) interneurons and (ii) excitatory input from neighboring or remote (i.e., extrinsic) areas.

(2) Forward modeling of electromagnetic signals that relates the modeled neural activity with MEG/EEG scalp signals.

We used DCM for EEG/MEG with lead-fields a priori fixed (David et al., 2006) (for a description of the definition of the gain matrix, see above, Source reconstruction, the forward modeling procedure). The lead-fields for each cortical ROI modeled (see below) were obtained by averaging the lead-fields of each current dipole of the ROI. The coordinates of the ROI were defined as the center of mass of cortical patches manually delineated on each individual brain based on anatomical criteria (Fig. 2b), using cortical surfaces extracted from T1 MRIs and coregistered to the MEG referential. We assumed a homogenous current source density within each ROI. In the DCM analysis, forward modeling was applied to the local current densities generated by the pyramidal populations, which are believed to be the main generators of the MEG signals.

(3) Use of an iterative Bayesian inversion scheme to estimate the neural parameters of the neural-mass models based on the measured evoked responses.

This approach enables the imposition of biologically grounded constrains on the values of the parameters. At each iteration, (i) simulated time courses of the response of the DCM network to the input stimulus are calculated for the current parameters, and (ii) predicted MEG measures are calculated by applying forward modeling to these time courses. The degree of fit between the observed and predicted MEG time courses drives the Bayesian estimation of the parameters of the DCM network. Inferences about particular connections are made using their posterior or conditional density. The full set of equations for DCM specification and Bayesian parameter estimation can be found in the original papers (Friston et al., 2003; David et al., 2005, 2006; Kiebel et al., 2006). The observed and predicted MEG signals are represented by the first modes of their singular value decomposition (David et al., 2006). Practically, we chose to only include four modes because, in most subjects, a visual analysis revealed that higher modes did not contain obvious evoked components in the 100–800 ms postonset time interval. Higher modes were thus considered as representing mainly residuals of experimental noise or ongoing activity (which are of no interest for DCM applied to event-related fields). The first four modes explained on average 88% of the variance of the signal (±4%), and up to 95% in one subject, and we considered that restricting the analysis to those four modes was a good trade-off between accuracy, noise rejection, and computational time.

Specification of priors

In our study, the input of the model was a unitary current pulse convolved with a gamma function (of which the parameters were estimated). The gamma function is assumed to model early subcortical processing (thalamic relay before V1–V2 and/or the retinotectal pathway depending on the models). Priors on model parameters were identical to those previously described (David et al., 2005, 2006). Parameters were initialized at their prior values. Note that the prior values of parameters were not highly critical for our study because we assumed large prior variances. In other words, estimated parameter distributions were usually significantly different from prior assumptions, i.e., estimated cortical activity was very different from the one first generated by prior parameters. To minimize the effects of local minima on the optimization criterion, the parameter estimation was repeated several times (up to 30 times depending on the models) using a new random starting point close to prior parameter values. The adjustment between measured and predicted MEG time series was visually checked for each model and each subject.

In any case, in a DCM study, the critical user-dependent factor is not so much the prior values of the neural model but rather the architecture of the model itself, i.e., how the different nodes of the network (cortical region) are chosen and interconnected. The neural architecture at the macroscopic level must thus be chosen according to clearly defined anatomical-functional hypotheses that are suitable for comparisons.

Definition of ROIs and null model for the DCM analysis

All the different models that we compared differed from one another by the pattern of connections between a set of seven predefined bilateral ROIs (Fig. 2b,c). ROIs were essentially defined based on anatomic-functional criteria, and were partially informed by the apparent signal propagation along the cerebral cortex observed in MEG source reconstruction results. The ROIs approximated the relevant neural systems, i.e., the VVS and AAS. The first five regions were used as an approximate model of the VVS, which runs through the ventral occipital lobe and the temporal lobe (ventral and lateral aspects) up to the temporal pole. The VVS includes both parvocellular and magnocellular projections (Shipp and Zeki, 1995), and presents a relative functional specialization in complex detections, categorization, recognition, and analysis of visual objects (Mishkin and Ungerleider, 1982; Goodale et al., 1991a,b; Goodale and Milner, 1992). To simplify the models, clusters of visual areas were defined based on their anatomical proximity (Van Essen, 2003) and knowledge about their connectivity (Felleman and Van Essen, 1991; Jouve et al., 1998). We grouped V1 and V2 because those areas are closely connected and essentially located on the medial wall of the occipital lobe (V1–V2). Two other clusters of occipital areas were defined: one cluster of areas in the lateral occipital cortex (LOC), which would correspond to V3, V3a, and LO, and another cluster in the ventral occipital cortex (VOC), a part of the early VVS mainly corresponding to V8. We then defined the fusiform gyrus (Fg), restricted to its posterior aspect, as an intermediate area along the VVS. This was based on numerous studies demonstrating that the fusiform gyrus is a high-order visual cortex critically involved in face and object processing (Kanwisher et al., 1997), which has also been shown to be modulated by emotionally competent stimuli in a manner dependent on emotion-related structures (Vuilleumier et al., 2004). Inferotemporal cortex (IT) (a large region involved in higher-order visual processing) was defined as the middle and inferior temporal gyri. The AAS was modeled by including (1) an anterior temporal cluster grouping together the uncus (cortical fold beneath which the amygdala resides) (Amg) and the temporal pole (TP) and (2) the orbitofrontal cortex (OFC).

The amygdala was not directly modeled as a subcortical structure, because we used a cortical tessellation to define the putative space of sources generating the signal. However, as mentioned above, the ROI that we call Amg/TP included the cortical shell just around the amygdala (the uncus). From a forward modeling standpoint, the uncus, which is round shaped and right around the amygdala, appears to us as a reasonable approximation for possible sources in the amygdala, given the resolution of MEG. There is also a controversy as to whether MEG can record amygdala signals, because this structure is composed of multiple nuclei that may constitute a magnetically closed structure. We chose to pull the “amygdala” and temporal pole together, precisely because we thought that MEG would not necessarily permit us to disentangle them, and we wanted to account for this uncertainty.

Although many more areas are involved in the visual system and subdivisions of the AAS, we chose a limited number of regions that appeared to constitute a good trade-off between modeling the complexity of the organization of the these systems and the ability to robustly identify model parameters. Our purpose was to reveal fundamental differences in neurodynamic behaviors, resulting from radical changes in the neural architecture, not to provide a full account of the detailed dynamics of the visual system and AAS (for which the modeling approach and the experimental technique are not well suited). Of note, although the exact number of areas and patterns of connectivity influence the delays and dynamics that are involved in neural processing, the Bayesian framework used in our modeling approach allows for flexibility in model specification: part of the unmodeled delays are integrated in the time constants and dynamical parameters of the areas explicitly modeled. To limit the number of ROIs contributing to the dynamics in the model, we created bilateral clusters of areas that were considered as one coherent functional region. In other words, estimated dynamics of homologous regions on both sides of the brain were identical.

A simplified scheme of connection, or null model, was used as a baseline for the other DCM models (the log evidence of the null model served as a reference for the log evidences of the other models) (see below and Fig. 2c). The null model was motivated both by anatomical-functional rationale and results from source reconstruction (see Fig. 3b,c; supplemental movie, available at www.jneurosci.org as supplemental material), which suggested an overall sequence of activation unfolding from the mesial occipital cortex through the lateral occipital cortex, posterior ventral occipital cortex, fusiform gyrus, lateral temporal cortex, and temporal pole to the OFC. The null model connected the VVS ROIs sequentially: V1–V2 > LOC > VOC > Fg > IT, simulating the U-shaped fiber system, which connects areas along the VVS. IT then projected to the Amg/TP cluster, which then connected to the OFC through the equivalent of the uncinate fasciculus.

Figure 3.

Figure 3.

Open in a new tab

Basic MEG results. a, MEG evoked response averaged across stimulus and across subjects (n = 15) per category. b, Cortical noise normalized minimum-norm source reconstruction results averaged across subjects and thresholded above three SDs above baseline for each category. The results show an overall propagation of activity from early visual cortices to frontal regions through the temporal lobes, with indication of early responses in the OFC for unpleasant and pleasant stimuli. c, Thresholded minimum-norm results for the contrast between pleasant and neutral stimuli on slightly unfolded cortical surfaces. Effects are reconstructed in occipital and temporal regions, in the temporal pole and the OFC (early and late). Similar but less extended effects were found with unpleasant stimuli (see supplemental Fig. 1, available at www.jneurosci.org as supplemental material).

We did not include explicit modeling of regions of the dorsal visual stream per se in our analysis, for several reasons. First, we did not have specific network hypotheses regarding the causal implication of the dorsal stream in the type of emotional modulation under study. Second, it appears that modulation of the dorsal stream by emotional competence is weak (Fichtenholtz et al., 2004). Accordingly, we did not find strong evidence of modulation of activity by the emotional competence of the stimuli in high-order areas associated with the dorsal visual stream using distributed source reconstruction. Also, given the distribution of the estimated power in the source reconstruction results [most important in ventral regions (see Fig. 3)], it is unlikely that the possible unspecification of any dorsal region in the DCMs had a significant effect on the estimated dynamics of DCMs restricted to the ventral visual stream. Third, we wanted to minimize the model complexity to facilitate the interpretation of the results.

Alternative models for rapid processing in the anterior affective system

In addition to the null model, we tested six alternative models (Fig. 2c) that included the null model as a common network. The tested models were obtained by adding connections between areas with variable feedforward and backward patterns of projections.

The first model of connectivity (model 1) was purely hierarchical, and served as a control. Models 2 and 3 corresponded to two-stage architectures, introducing local feedback or long-range feedback, respectively. The feedback in these models originated from the Amg/TP cluster and from the OFC and projected to the ROIs of the VVS. Models 4 and 5, which also included the long-range feedback, introduced additional subcortical pathways or long-range forward cortical–cortical pathways. Model 4 modeled the retinotectal pathway and model 5 the inferior longitudinal and frontal–occipital fasciculi. The inferior frontal–occipital fasciculus (IFOF) connects the early visual cortices with the OFC, and the inferior longitudinal fasciculus (ILF) connects the early visual cortices with the amygdala and temporal pole. The existence of these fasciculi in humans is supported by recent fiber tract analyses based on diffusion tensor imaging (Catani et al., 2002, 2003). The existence of the IFOF in the monkey is still controversial, but recent results of invasive tracing in the monkey based on histochemical techniques have provided clear support for the ILF and shown that many of the ILF fibers are bidirectional (Schmahmann and Pandya, 2006). Finally, model 6 combined together models 4 and 5.

Modulation of cortical activity by emotional competence

Stimuli conditions (neutral, unpleasant, and pleasant) were allowed to modulate, in a context-dependent manner, the strength of the postsynaptic gain at the level of the input layers of the regions of the AAS (Amg/TP, OFC), i.e., where the feedforward afferents project. This context-dependent modulation was aimed at simulating the effect of emotion-related information processing within the AAS. We assumed that the amount of emotion-related processing within the AAS would depend on the emotional information carried by the visual afferents. By design, this information was dependent on the category of stimuli. We also assumed that the amount of emotion-related processing would be reflected in the level of activation of the AAS regions in response to the category-dependent inputs from the visual afferents. In the modeling framework, the level of activation in response to inputs is directly controlled by the postsynaptic gain of the input layers. Of note, the resulting modulation of the AAS activation could generate secondary causal top-down modulatory effects on the VVS via the feedback projections.

We did not allow the postsynaptic gains of the feedback projections themselves (from the AAS to the VVS) to be directly modulated by emotional information in any of the models tested, because this was not compatible with our basic anatomical-functional hypotheses. Indeed, this would have implied that basic processing of emotional information actually takes place within the visual regions themselves rather than upstream, at the level of the emotion-related regions. However, following a recommendation from a reviewer, we added to the supplemental material (available at www.jneurosci.org) results from a model in which modulation occurred at the level of the feedback projections.

The significance of the effects of modulation by emotional competence on the DCM time courses were tested using one-way ANOVAs for repeated-measures design, testing for a main effect of conditions: neutral versus unpleasant versus pleasant. Tests were applied during the first increase of activity in the time courses after stimulus onset, to detect when effects would first become significant, and at the level of each minimum and maximum on the time courses corresponding to the neutral condition (per our hypothesis, this comparison was only performed when the neutral condition had a lower amplitude than the emotional conditions). The results were thresholded at p < 0.05 (one-tail, uncorrected).

Model comparisons

After estimating the parameters of each competing model for each subject, the models were compared to determine the most plausible model, i.e., anatomical-functional hypothesis. The comparison was done using Bayesian model selection in which the “evidence” of each model is used to quantify the model plausibility (Penny et al., 2004). The evidence can be decomposed into two components: an accuracy term, which quantifies the data fit, and a complexity term, which penalizes models with a large number of parameters. The most likely model is the one with the largest log evidence. Conventionally, strong support in favor of one model requires the difference in log evidence to be three or more when compared with other models. Assuming that each dataset is independent of the others, the best model at the group level is obtained by computing the evidence at the group level by multiplying the marginal likelihoods, or equivalently, by adding the log evidences from each subject (Garrido et al., 2008). Note that the evidence can only be approximated under some assumptions. We have chosen to use the Bayesian information criterion (BIC) because it does penalize complex models more than the Akaike information criterion, which is an alternative choice (Penny et al., 2004).

Results

Behavioral and psychophysiological results

The behavioral and psychophysiological results demonstrated the emotional competence of the stimuli on the subjects in the experiment (Fig. 1b). Several hypotheses were tested using one-way ANOVAs, planned comparisons and correlations: (1) that pleasant stimuli were associated with self reports of higher arousal and more positive valence than neutral stimuli; (2) that unpleasant stimuli were associated with self reports of higher arousal and more negative valence than neutral stimuli; (3) that heart rate changes averaged across subjects for each stimulus were correlated with subjective ratings of valence during the secondary acceleration phase of the heart response (Bradley and Lang, 2000). There was a significant difference in the ratings of arousal (F(2,28) = 29.74; p = 1.18 × 10−7) and valence (F(2,28) = 28.728; p = 1.643 × 10−7) between the three categories of stimuli. Neutral stimuli were associated with weak arousal (mean = 1.049; SD = 0.82) and close to zero valence (mean = 0.22; SD = 0.39). Unpleasant stimuli were associated with a significantly higher arousal (mean = 3.05; SD = 1.53) than neutral (F(1,14) = 43.8; p = 5.75 × 10−6) and a significantly more negative valence (mean = −1.44; SD = 1.09) than neutral (F(1,14) = 20.78; p = 2.24 × 10−4). Likewise, pleasant stimuli provoked a significantly higher arousal (mean = 3.68; SD = 1.8) than neutral (F(1,14) = 32.64; p = 2.68 × 10−5) and a significantly more positive valence (mean = 1.82; SD = 1.29) than neutral (F(1,14) = 31.29; p = 3.31 × 10−5). On average, the arousing effect of the pleasant stimuli was higher than that of the unpleasant ones (F(1,14) = 5.12; p = 0.02).

As expected, heart rate showed a triphasic pattern in response to stimulus onset. During the second phase of this response, heart rate change was significantly correlated with subjective ratings of valence. The average time course of heart rate change associated with neutral stimuli tended to show a pronounced pattern of alternating increases and decreases from one 500 ms time window to the other (Fig. 1b, right, top chart). As shown in Figure 1b (top right chart, inset), this pattern was predicted by a simulation of two interleaved mean random walks of cumulated heart rate changes when the probability of increase or decrease with respect to the central tendency is equal (this was expected with neutral stimuli).

Source reconstruction results

Figure 3a–c shows basic MEG results, including source reconstruction results. These suggest a general pattern of propagation of brain currents from the occipital lobes to the frontal lobes through the temporal lobes. The pattern of propagation appears to be modulated by emotional competence, with some indications of early effects of modulation at the level of the TP and OFC. These results, which do not constitute the core result of this study, are presented in a heuristic perspective. We emphasize that they have to be taken with the caution necessary for source reconstruction results, and should be essentially viewed as providing a useful way of representing the data that can serve as a basis for partly informing the modeling framework.

According to the source reconstruction results, three main chronological periods can be distinguished (Fig. 3b), as follows.

(1) At ∼70 ms, an activation of the primary visual cortex [Brodmann area (BA) 17, 18] is observed, followed by an early activation of the TP and OFC at ∼100 ms after stimulus onset. From 100 to 170 ms, activity spreads from the primary visual cortex over the rest of the occipital cortex [to include the ventral (BA 19, 37) and lateral (BA 18, 19) occipital cortices]. During this same epoch (100–170 ms), activity spreads over the parietal cortex to include the depth of the parietooccipital sulcus, the posteromedial cortex [BA 7, 23, 32 and retrosplenial cortex (RSC)], and the lateral parietal cortex (BA 7, 39, 40). Between 170 and 200 ms, a brief and widespread drop in activity occurs (see supplemental movie, available at www.jneurosci.org as supplemental material).

(2) Immediately before the onset of the main activity in the VVS (from 200 to 350 ms), an intense response in the RSC can be noted at ∼200 ms. The VVS activity propagates slowly along the ventral visual cortices, from the occipital cortex to the TP. The ventral occipitotemporal cortex, including BA 18, 19, and 37, also shows a sustained increase of activity, peaking at ∼240 ms. During the same period, an increase of signal is observed within the Sylvian region, including the insular cortices.

(3) From 350 to 500 ms, an extensive increase of activity appears in the OFC, lateral temporal cortex, the frontal and parietal opercula (including part of SII), and the anterior cingulate cortex (ACC). It must be noted that the sequence of activity described here does not capture the fine dynamical behavior shown by the movies presented in the supplemental material (available at www.jneurosci.org).

Figure 3c shows SPMs contrasting pleasant and neutral stimuli, which showed the most obvious effects. An early significant effect (∼100 ms) is reconstructed in the TP and OFC for the contrast between pleasant and neutral stimuli. The cortex corresponding to the VVS, along with the TP and OFC, also shows sustained significant effects for pleasant minus neutral stimuli during the period of the main VVS activation (200–350 ms). Significant effects are observed between 350 and 440 ms in the OFC, ventromedial prefrontal cortex, dorsal ACC, SI, SII, and insula. The contrast between unpleasant and neutral resulted in similar patterns of activation, although less extensive, less significant, and more irregular in time (see supplemental results and supplemental Fig. 1, available at www.jneurosci.org as supplemental material).

DCM results: comparing network architectures

Figure 4, a and b, shows the model log evidences, from which the evidence of the null model was subtracted (Garrido et al., 2008). For almost all subjects (13/15), the two-pathway models (models 4–6) were associated with much higher log evidence than the purely feedforward (model 1) and two-stage (models 2–3) models, with differences largely above standard criteria for significance (Penny et al., 2004) (Fig. 4b). Plotting the group log evidence confirms the much higher plausibility of the two-pathway hypothesis (Fig. 4a). Figure 4 clearly shows that the two-pathway models constitute a class of model much more plausible than two-stage models. The main reason for this higher plausibility was that two-pathway models allowed earlier responses in the AAS than the other models (average latency of first peak: 148 ms vs 362 ms; 356 ms for the two-stage models) and earlier modulation of the VVS by stimulus emotional competence (average onset of first significant effect in VVS regions: 194 ms for two-pathway models vs 454 ms for two-stage models) (Fig. 4c).

Figure 4.

Figure 4.

Open in a new tab

Dynamic causal modeling results. a, Log evidence (BIC) of the different models (log evidence of the null model subtracted). Left, Group level. The log evidence is significantly different between two-stage and two-pathway models according to Bayesian inference (Penny et al., 2004). Right, Individual level. For 13 of the 15 subjects, the two-pathway models have better log evidence than the other models. b, Time courses of the average (n = 15) absolute values of the evoked dynamics estimated at the level of the populations of pyramidal neurons, for each region of interest and model. c, Region color codes correspond to the color codes in Figure 2, b and c. The main traces (underlined by darker surfaces) correspond to the neutral condition. The secondary traces (underlined by lighter surfaces) correspond to the maximum average response to either pleasant or unpleasant stimuli. Black vertical bars indicate significant effects of modulation by emotional competence (p < 0.05 uncorrected) (see Materials and Methods).

Differences among the different two-pathway models were more subtle and less reproducible across subjects. The model combining the long-range fasciculi and the retinotectal pathway (model 6) showed the highest evidence, despite the higher penalization because of higher complexity. This suggests that both pathways (retinotectal and long-range fasciculi) may coexist and probably convey different and/or redundant types of information about visual stimuli.

Discussion

Our results strongly and reliably support the hypothesis that the human brain uses a two-pathway architecture for rapid top-down modulation of visual attention, notably in the context of the processing of emotion-related stimuli. Only a two-pathway architecture can account for early responses in the anterior affective system and early modulation of ventral visual processing by emotional competence of the stimuli under the control of this system. Early responses in the anterior affective system clearly appeared in the time courses predicted by the most plausible DCM models, i.e., the two-pathway versus two-step models.

Early processing in the anterior affective system had been previously suggested by several electrophysiological studies (Kawasaki et al., 2001; Eimer and Holmes, 2002; Pizzagalli et al., 2002; Streit et al., 2003; Bar et al., 2006), and also appeared plausible in our source reconstruction results (see Fig. 3b,c). Kawasaki et al. (2001) demonstrated more directly a modulation of the rate of spiking of individual neurons in the OFC as early as 120–160 ms, in a patient with intractable epilepsy presented with faces expressing negative emotions. This result is bolstered by the fact that the investigators used tetrodes implanted in the OFC, a methodology that is not confounded by the uncertainty of localization affecting source reconstruction analyses. Our DCM findings show how such emotional information can rapidly propagate to the anterior affective system.

Our study also shows that, at least in normal conditions, short-cut pathways relying on long-range association fiber tracts (connecting early visual cortices to the anterior affective system) can provide an architectural basis for rapid emotion-related processing. The model including such short-cut pathways had a degree of plausibility comparable with the model including the subcortical retinotectal pathway, while providing comparable speeds of information transmission. Such association fiber tracts can convey a substantial amount of visual information from V1–V2, whereas the retinotectal pathway can only rely on the superior colliculus, which supports a much coarser representation of visual information. Both types of pathways may operate in parallel and might play a partially distinct role in the processing of different classes of stimuli or conditions, with perhaps a specialization of the retinotectal pathway in processing structurally stereotyped stimuli, such as faces. Thus, according to our results, combined architectures appear highly plausible (model 6). It would be interesting to test whether brain damaged subjects with probable disconnection of the inferior frontal–occipital fasciculus and/or inferior longitudinal fasciculus would show impairments in processing visual emotional information, in particular in the recognition of the facial expression of emotion. Indeed, such a finding would shed light on the relative role of long-range association fiber tracts versus the retinotectal pathway in this process.

It is important to note that our study does not suggest that early visual cortices themselves contain neurons capable of explicitly extracting emotional information. It suggests, according to models 5 and 6, that early visual cortices can act as relays for the rapid transmission of this implicit information to the anterior affective system. Only at the level of the anterior affective system would this information be explicitly extracted. Our study also suggests that neuronal activity in early visual cortices can be modulated in return by feedback from the anterior affective system.

Although our study focuses on emotional visual attention, it is possible that the anterior affective system and in particular the OFC play a general role in the rapid top-down modulation of visual processing, i.e., not restricted to processing explicitly involving interactions between emotion and perception. This is supported by a recent MEG/functional MRI study (Bar et al., 2006) that suggested that early responses in the OFC related to the process of visual object recognition occur before correlative responses in areas of the ventral visual stream. The authors hypothesized the involvement of dorsal magnocellular pathways from early cortical visual areas to the OFC in the rapid top-down modulation of visual processing. The authors motivated their specific anatomical hypothesis by referring to the lack of evidence for direct connections between the two sets of regions, and found effects of the spatial frequency of the stimuli in their results, in a way compatible with the involvement of magnocellular pathways. However, as discussed above, there is substantial evidence in humans (Catani et al., 2002, 2003) and evidence in the monkey (Schmahmann and Pandya, 2006) for the existence of such direct connections, through long-range association fiber tracts. Also, the type of delays observed between occipital and OFC responses suggests the involvement of direct connections, and the phase synchrony analyses in Bar et al. (2006) are compatible with this idea, because one might expect phase synchrony to be particularly high between regions directly interacting. It would be interesting to know whether the interior frontal–occipital fasciculus (or the inferior longitudinal fasciculus if one allows for disynaptic connections), or perhaps other plausible long-range fasciculi, contain a substantial proportion of magnocellular fibers, because this would be compatible with their model and findings. As mentioned above, Bar et al. (2006) use phase synchrony as an index of causal interactions between the visual occipital cortex, the OFC, and the fusiform gyrus. Phase synchrony is an indirect way of quantifying interactions between regions, which relies on several assumptions with potential confounds: certain coupled oscillators tend to synchronize with a fixed phase lag when they interact (Rudrauf et al., 2006), but this does not imply that phase synchronization necessarily reflects interactions, because noninteracting parallel systems with a common driver will generally show phase synchronization between systems that are not coupled. Because the DCM technique explicitly models causal interactions, it goes a step further. It would be very interesting to see if DCM applied to the Bar et al. (2006) data, with the same architectures as tested here, would also lead to the selection of two pathway models, and if this selection would depend on the spatial frequency of the stimuli as their model would predict.

Although in constructing the models we have aimed for a certain degree of biophysical and anatomical-functional realism, it is important to acknowledge that the models are a dramatic simplification with respect to the true architecture and dynamics of the brain in general, and of the visual system in particular. To preserve stability and avoid overfitting, DCM requires the use of relatively simple models. The use of DCM is thus clearly limited by the intrinsic complexity of neural networks. DCM is also limited by the small dimension and poor spatial resolution of noninvasive electrophysiological recordings such as MEG. The large number of estimated parameters and the fact that all the parameters interact make it very difficult to isolate the contribution of each parameter or determine how critical the different parameters are in the modeling of the dynamical behavior observed in the data. This is why inferences are usually restricted to interregional effective connectivity, which fortunately matches the scale at which most hypotheses are formulated in cognitive neuroscience.

Despite its limitations, DCM provides a valuable approach to testing hypotheses in cognitive neuroscience. Although DCM cannot determine when the true model is not among the compared models, the plausibility of well grounded biophysical models with alternative architectures of connectivity can be directly compared vis-à-vis measured physiological responses. It is important to emphasize that results depend much less on initial parameter settings than on the type of architecture defining the network (see Materials and Methods, Specification of priors). As shown in our results, the differences in model evidence within an architectural class of models (e.g., two-step vs two-pathway models) are much smaller than the differences between architectural classes of models.

In our study, the compared classes of models represent the extremes of two types of architectural principles: sequential processing and parallel processing. Although intermediate alternatives representing some gradation between purely parallel and purely sequential architecture cannot be ruled out, the general conclusion presented in this report is that the true model is likely to be strongly parallel, resembling the two-pathway models more than the two-stage models.

In summary, our study provides strong and reliable evidence in support of the implementation of a two-pathway architecture in the human brain. This architecture is involved in the rapid processing of complex visual scenes and in the rapid modulation of visual attention by emotion-related information. In the future, further avenues of research may clarify the differential roles of long-range association fiber tracts and retinotectal transmission in this process.

Footnotes

D.R. was supported by a grant from the Ministère de la Recherche, the Unité Propre de Recherche 640–Laboratoire d'Electroencephalographie et de Neurophysiologie Appliquée (LENA) Centre National de la Recherche Scientifique, and by a grant from the Mathers Foundation (to A.D.), as well as grants from the National Institutes of Health to Thomas Grabowski. J.-P.L. was supported by a grant from the Fyssen Foundation. This paper is in memory of Francisco Varela, with whom this work had been started. We thank Antoine Ducorps and Denis Schwartz (MEG Center, Paris, France), Laurent Hugueville (LENA, Paris, France), Sylvain Baillet (LENA, Paris, France), and Sonya Mehta [Laboratory of Computational Neuroimaging (LCN), University of Iowa, Iowa City, IA] for important support concerning technical and methodological issues. We thank Thomas Grabowski, Daniel Tranel, Ralph Adolphs, Gary Van Hoesen, Steven Luck, Hanna Damasio, Sahib Khalsa, and Nicolas Lori, who offered many helpful suggestions regarding methods and interpretations. We thank Sahib Khalsa for his careful rereading of this manuscript. We particularly thank Thomas Grabowski for his enthusiastic support as the head of the LCN at the University of Iowa during the last stage of completion of this study.

References

  1. Adolphs R, Gosselin F, Buchanan TW, Tranel D, Schyns P, Damasio AR. A mechanism for impaired fear recognition after amygdala damage. Nature. 2005;433:68–72. doi: 10.1038/nature03086. [DOI] [PubMed] [Google Scholar]
  2. Anderson AK. Affective influences on the attentional dynamics supporting awareness. J Exp Psychol Gen. 2005;134:258–281. doi: 10.1037/0096-3445.134.2.258. [DOI] [PubMed] [Google Scholar]
  3. Baillet S, Mosher JC, Leahy RM. Electromagnetic brain mapping. IEEE Sig Proc Mag. 2001;18:14–30. [Google Scholar]
  4. Bar M, Kassam KS, Ghuman AS, Boshyan J, Schmid AM, Dale AM, Hamalainen MS, Marinkovic K, Schacter DL, Rosen BR, Halgren E. Top-down facilitation of visual recognition. Proc Natl Acad Sci USA. 2006;103:449–454. doi: 10.1073/pnas.0507062103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bradley MM, Lang PJ. Measuring emotion: the self-assessment Manikin and the semantic differential. J Behav Ther Exp Psychiatry. 1994;25:49–59. doi: 10.1016/0005-7916(94)90063-9. [DOI] [PubMed] [Google Scholar]
  6. Bradley MM, Lang PJ. Measuring emotion: behavior, feeling, and physiology. In: Lane RD, Nadel L, editors. Cognitive neuroscience of emotion. New York: Oxford UP; 2000. pp. 242–276. [Google Scholar]
  7. Bradley MM, Codispoti M, Cuthbert BN, Lang PJ. Emotion and motivation I: defensive and appetitive reactions in picture processing. Emotion. 2001;1:276–298. [PubMed] [Google Scholar]
  8. Catani M, Howard RJ, Pajevic S, Jones DK. Virtual in vivo interactive dissection of white matter fasciculi in the human brain. NeuroImage. 2002;17:77–94. doi: 10.1006/nimg.2002.1136. [DOI] [PubMed] [Google Scholar]
  9. Catani M, Jones DK, Donato R, Ffytche DH. Occipito-temporal connections in the human brain. Brain. 2003;126:2093–2107. doi: 10.1093/brain/awg203. [DOI] [PubMed] [Google Scholar]
  10. Dale AM, Sereno MI. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: a linear approach. J Cogn Neurosci. 1993;5:162–167. doi: 10.1162/jocn.1993.5.2.162. [DOI] [PubMed] [Google Scholar]
  11. Damasio AR. Descartes' error: emotion, reason, and the human brain. New York: Grosset/Putnam; 1994. [Google Scholar]
  12. Darvas F, Pantazis D, Kucukaltun-Yildirim E, Leahy RM. Mapping human brain function with MEG and EEG: methods and validation. NeuroImage. 2004;23(Suppl 1):S289–S299. doi: 10.1016/j.neuroimage.2004.07.014. [DOI] [PubMed] [Google Scholar]
  13. David O, Harrison L, Friston KJ. Modelling event-related responses in the brain. NeuroImage. 2005;25:756–770. doi: 10.1016/j.neuroimage.2004.12.030. [DOI] [PubMed] [Google Scholar]
  14. David O, Kiebel SJ, Harrison LM, Mattout J, Kilner JM, Friston KJ. Dynamic causal modeling of evoked responses in EEG and MEG. NeuroImage. 2006;30:1255–1272. doi: 10.1016/j.neuroimage.2005.10.045. [DOI] [PubMed] [Google Scholar]
  15. David O, Harrison L, Friston K. Neuronal models of EEG and MEG. In: Friston KJ, Ashburner JT, Kiebel SJ, Nichols TE, Penny WD, editors. Statistical parametric mapping: the analysis of functional brain images. London: Academic; 2007. pp. 414–440. [Google Scholar]
  16. Drury HA, Van Essen DC, Anderson CH, Lee CW, Coogan TA, Lewis JW. Computerized mappings of the cerebral cortex: a multiresolution flattening method and a surface-based coordinate system. J Cogn Neurosci. 1996;8:1–28. doi: 10.1162/jocn.1996.8.1.1. [DOI] [PubMed] [Google Scholar]
  17. Eimer M, Holmes A. An ERP study on the time course of emotional face processing. NeuroReport. 2002;13:427–431. doi: 10.1097/00001756-200203250-00013. [DOI] [PubMed] [Google Scholar]
  18. Felleman DJ, Van Essen DC. Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex. 1991;1:1–47. doi: 10.1093/cercor/1.1.1-a. [DOI] [PubMed] [Google Scholar]
  19. Fichtenholtz HM, Dean HL, Dillon DG, Yamasaki H, McCarthy G, LaBar KS. Emotion-attention network interactions during a visual oddball task. Brain Res Cogn Brain Res. 2004;20:67–80. doi: 10.1016/j.cogbrainres.2004.01.006. [DOI] [PubMed] [Google Scholar]
  20. Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. NeuroImage. 1999;9:195–207. doi: 10.1006/nimg.1998.0396. [DOI] [PubMed] [Google Scholar]
  21. Friston KJ, Harrison L, Penny W. Dynamic causal modelling. NeuroImage. 2003;19:1273–1302. doi: 10.1016/s1053-8119(03)00202-7. [DOI] [PubMed] [Google Scholar]
  22. Garrett AS, Maddock RJ. Time course of the subjective emotional response to aversive pictures: relevance to fMRI studies. Psychiatry Res. 2001;108:39–48. doi: 10.1016/s0925-4927(01)00110-x. [DOI] [PubMed] [Google Scholar]
  23. Garrido MI, Kilner JM, Kiebel SJ, Stephan KE, Friston KJ. Dynamic causal modelling of evoked potentials: a reproducibility study. NeuroImage. 2007;36:571–580. doi: 10.1016/j.neuroimage.2007.03.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends Neurosci. 1992;15:20–25. doi: 10.1016/0166-2236(92)90344-8. [DOI] [PubMed] [Google Scholar]
  25. Goodale MA, Milner AD, Jakobson LS, Carey DP. A neurological dissociation between perceiving objects and grasping them. Nature. 1991a;349:154–156. doi: 10.1038/349154a0. [DOI] [PubMed] [Google Scholar]
  26. Goodale MA, Milner AD, Jakobson LS, Carey DP. Object awareness. Nature. 1991b;352:202. doi: 10.1038/352202b0. [DOI] [PubMed] [Google Scholar]
  27. Jansen BH, Rit VG. Electroencephalogram visual evoked potential generation in a mathematical model of coupled cortical columns. Biol Cybern. 1995;73:357–366. doi: 10.1007/BF00199471. [DOI] [PubMed] [Google Scholar]
  28. Jouve B, Rosenstiehl P, Imbert M. A mathematical approach to the connectivity between the cortical visual areas of the macaque monkey. Cereb Cortex. 1998;8:28–39. doi: 10.1093/cercor/8.1.28. [DOI] [PubMed] [Google Scholar]
  29. Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17:4302–4311. doi: 10.1523/JNEUROSCI.17-11-04302.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kastner S, Ungerleider LG. Mechanisms of visual attention in the human cortex. Annu Rev Neurosci. 2000;23:315–341. doi: 10.1146/annurev.neuro.23.1.315. [DOI] [PubMed] [Google Scholar]
  31. Kawasaki H, Kaufman O, Damasio H, Damasio AR, Granner M, Bakken H, Hori T, Howard MA, III, Adolphs R. Single-neuron responses to emotional visual stimuli recorded in human ventral prefrontal cortex. Nat Neurosci. 2001;4:15–16. doi: 10.1038/82850. [DOI] [PubMed] [Google Scholar]
  32. Kiebel SJ, David O, Friston KJ. Dynamic causal modelling of evoked responses in EEG/MEG with lead field parameterization. NeuroImage. 2006;30:1273–1284. doi: 10.1016/j.neuroimage.2005.12.055. [DOI] [PubMed] [Google Scholar]
  33. Kringelbach ML, Rolls ET. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog Neurobiol. 2004;72:341–372. doi: 10.1016/j.pneurobio.2004.03.006. [DOI] [PubMed] [Google Scholar]
  34. Lang PJ, Bradley MM, Cuthbert BN. Technical report A-6. Gainesville, FL: University of Florida; 2005. Internation affective picture system (IAPS): affective ratings of pictures and instruction manual. [Google Scholar]
  35. Leahy RM, Mosher JC, Spencer ME, Huang MX, Lewine JD. A study of dipole localization accuracy for MEG and EEG using a human skull phantom. Electroencephalogr Clin Neurophysiol. 1998;107:159–173. doi: 10.1016/s0013-4694(98)00057-1. [DOI] [PubMed] [Google Scholar]
  36. LeDoux J. Fear and the brain: where have we been, and where are we going? Biol Psychiatry. 1998;44:1229–1238. doi: 10.1016/s0006-3223(98)00282-0. [DOI] [PubMed] [Google Scholar]
  37. Liddell BJ, Brown KJ, Kemp AH, Barton MJ, Das P, Peduto A, Gordon E, Williams LM. A direct brainstem-amygdala-cortical “alarm” system for subliminal signals of fear. NeuroImage. 2005;24:235–243. doi: 10.1016/j.neuroimage.2004.08.016. [DOI] [PubMed] [Google Scholar]
  38. Lopes da Silva FH, Hoeks A, Smits H, Zetterberg LH. Model of brain rhythmic activity. The alpha-rhythm of the thalamus. Kybernetik. 1974;15:27–37. doi: 10.1007/BF00270757. [DOI] [PubMed] [Google Scholar]
  39. Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Annu Rev Neurosci. 2001;24:167–202. doi: 10.1146/annurev.neuro.24.1.167. [DOI] [PubMed] [Google Scholar]
  40. Mishkin M, Ungerleider LG. Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behav Brain Res. 1982;6:57–77. doi: 10.1016/0166-4328(82)90081-x. [DOI] [PubMed] [Google Scholar]
  41. Morris JS, Ohman A, Dolan RJ. A subcortical pathway to the right amygdala mediating “unseen” fear. Proc Natl Acad Sci USA. 1999;96:1680–1685. doi: 10.1073/pnas.96.4.1680. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Myers RH. Classical and modern regression with applications. Ed 2. Pacific Grove, CA: Duxbury; 1990. [Google Scholar]
  43. Penny WD, Stephan KE, Mechelli A, Friston KJ. Comparing dynamic causal models. NeuroImage. 2004;22:1157–1172. doi: 10.1016/j.neuroimage.2004.03.026. [DOI] [PubMed] [Google Scholar]
  44. Pizzagalli DA, Lehmann D, Hendrick AM, Regard M, Pascual-Marqui RD, Davidson RJ. Affective judgments of faces modulate early activity (approximately 160 ms) within the fusiform gyri. NeuroImage. 2002;16:663–677. doi: 10.1006/nimg.2002.1126. [DOI] [PubMed] [Google Scholar]
  45. Pourtois G, Grandjean D, Sander D, Vuilleumier P. Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cereb Cortex. 2004;14:619–633. doi: 10.1093/cercor/bhh023. [DOI] [PubMed] [Google Scholar]
  46. Robinson PA, Rennie CJ, Wright JJ, Bahramali H, Gordon E, Rowe DL. Prediction of electroencephalographic spectra from neurophysiology. Phys Rev E Stat Nonlin Soft Matter Phys. 2001;63:021903. doi: 10.1103/PhysRevE.63.021903. [DOI] [PubMed] [Google Scholar]
  47. Rolls ET. Précis of the brain and emotion. Behav Brain Sci. 2000;23:177–191. doi: 10.1017/s0140525x00002429. discussion 192–233. [DOI] [PubMed] [Google Scholar]
  48. Rudrauf D, Douiri A, Kovach C, Lachaux JP, Cosmelli D, Chavez M, Adam C, Renault B, Martinerie J, Le Van Quyen M. Frequency flows and the time-frequency dynamics of multivariate phase synchronization in brain signals. NeuroImage. 2006;31:209–227. doi: 10.1016/j.neuroimage.2005.11.021. [DOI] [PubMed] [Google Scholar]
  49. Schmahmann JD, Pandya DN. Fiber pathways of the brain. Oxford: Oxford UP; 2006. [Google Scholar]
  50. Shipp S, Zeki S. Segregation and convergence of specialised pathways in macaque monkey visual cortex. J Anat. 1995;187:547–562. [PMC free article] [PubMed] [Google Scholar]
  51. Streit M, Dammers J, Simsek-Kraues S, Brinkmeyer J, Wolwer W, Ioannides A. Time course of regional brain activations during facial emotion recognition in humans. Neurosci Lett. 2003;342:101–104. doi: 10.1016/s0304-3940(03)00274-x. [DOI] [PubMed] [Google Scholar]
  52. Van Essen D. Organization of visual areas in macaque and human cerebral cortex. In: Chalupa L, Werner J, editors. Visual neurosciences. Cambridge, MA: MIT; 2003. p. 842. [Google Scholar]
  53. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends Cogn Sci. 2005;9:585–594. doi: 10.1016/j.tics.2005.10.011. [DOI] [PubMed] [Google Scholar]
  54. Vuilleumier P, Richardson MP, Armony JL, Driver J, Dolan RJ. Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nat Neurosci. 2004;7:1271–1278. doi: 10.1038/nn1341. [DOI] [PubMed] [Google Scholar]
  55. Weiskrantz L, Warrington EK, Sanders MD, Marshall J. Visual capacity in the hemianopic field following a restricted occipital ablation. Brain. 1974;97:709–728. doi: 10.1093/brain/97.1.709. [DOI] [PubMed] [Google Scholar]
  56. Yvert B, Bertrand O, Thevenet M, Echallier JF, Pernier J. A systematic evaluation of the spherical model accuracy in EEG dipole localization. Electroencephalogr Clin Neurophysiol. 1997;102:452–459. doi: 10.1016/s0921-884x(97)96611-x. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES