Emotion and the motivational brain

Abstract

Psychophysiological and neuroscience studies of emotional processing undertaken by investigators at the University of Florida Laboratory of the Center for the Study of Emotion and Attention (CSEA) are reviewed, with a focus on reflex reactions, neural structures and functional circuits that mediate emotional expression. The theoretical view shared among the investigators is that expressed emotions are founded on motivational circuits in the brain that developed early in evolutionary history to ensure the survival of individuals and their progeny. These circuits react to appetitive and aversive environmental and memorial cues, mediating appetitive and defensive reflexes that tune sensory systems and mobilize the organism for action and underly negative and positive affects. The research reviewed here assesses the reflex physiology of emotion, both autonomic and somatic, studying affects evoked in picture perception, memory imagery, and in the context of tangible reward and punishment, and using the electroencephalograph (EEG) and functional magnetic resonance imaging (fMRI), explores the brain’s motivational circuits that determine human emotion.

The general thesis examined here is that experienced emotions are founded on the activation of neural circuits that evolved in the mammalian brain to ensure the survival of individuals and their progeny. Primitively, these motive circuits were engaged by external stimuli that are appetitive and potentially life sustaining, or alternatively, represent threats to the organism’s survival. The psychobiological consequences of this neural firing are potentially twofold: On the one hand, they engage sensory systems that increase attention and facilitate perceptual processing, and on the other, they initiate reflex responses that mobilize the organism and prompt motor action.

Research with animals (e.g., Davis, 2000; Fanselow & Poulos, 2005; Kapp, Supple & Whalen, 1994; LeDoux, 2003) has defined the key neural structures in this survival network (see Figure 1: Lang & Davis, 2006; see also, Davis & Lang, 2003): The bilateral amygdalae-- two small, almond shaped bundles of nuclei in the temporal lobe-- plays a central role. Each amygdala receives input from cortex and thalamus (sensory) and hippocampus (memory), subsequently engaging through the central nucleus and extended amygdala (basal nucleus of the stria terminalis) a range of other brain centers that modulate sensory processing (vigilance), increase related information processing, and activate autonomic and somatic structures that mediate defensive or appetitive actions.

Figure 1.

Schematic diagram of the outputs of the basolateral nucleus of the amygdala to various target structures, and the subsequent outputs and targets of the amygdala’s central nucleus (CeA) and the lateral basal nucleus of the stria terminalis (BNST: the “extended” amygdala). Known and possible functions of these connections are briefly described (Davis & Lang, 2003).

It is convenient to consider this survival circuit as organized into two motivational systems, one defensive and associated with reports of unpleasant affect and the other appetitive, associated with pleasant affect. Konorski (1967) early conceived such a motivational typology, keyed to the survival role of the body’s many unconditioned reflexes. Exteroceptive reflexes were either preservative (e.g., ingestion, copulation, nurture of progeny) or protective (e.g., withdrawal from or rejection of noxious agents). He further suggested that affective states were consistent with this preservative/protective grouping: Preservative emotions include such affects as sexual passion, joy, and nurturance; fear and anger are protective affects. Dickinson and Dearing (1979) developed Konorski’s distinction, renaming the two motivational systems, aversive and attractive, with each again activated by a different, but equally wide range of unconditioned stimuli, determining perceptual-motor patterns and the course of learning. In this general view, affective valence is determined by the dominant motive system: the appetitive system (preservative/attractive) prompts positive affect; the defense system (protective/aversive) is the source of negative affect. Affective arousal reflects the “intensity” of motivational mobilization, determined originally by degree of survival need and the imminence or probability of nociception or appetitive consummation. In this regard, it is pertinent that factor analyses of emotional/evaluative language (since Osgood, Suci, & Tannenbaum, 1957; see also, Russell & Mehrabian, 1977; Bradley & Lang, 1994) have consistently found two main factors accounting for the most variance among affect descriptors: Emotional valence (positive/pleasant/appetitive vs. negative/aversive/defensive) and arousal (intensity of activation). Thus, it would appear that, despite the great number and diversity of emotional words, the underlying structure of affective language appears to reflect the dual motive system model-- appetitive and defensive neural circuits that vary in the vigor of their activation.

The defense system is ultimately a fight or flight circuit, but in response to danger cues it also mediates behavioral “freezing” (hiding), increased vigilance, and counter-threat displays in animal subjects. The appetitive system is activated variously in alimentation, sex, and nurturance of progeny. However, as will be seen, although some reactions are uniquely appetitive or defensive, many physiological and behavioral patterns are similar in both contexts of arousal, and are mediated by the same neural structures.

The view to be elaborated here is that these survival circuits in primitive cortex and the limbic brain are the stuff of human motivation, and that motivational arousal is the foundation of emotion. From this neuroscience perspective, human emotions can be described as dispositions to action (e.g., anger>attack; sexual desire>approach; fear>escape-- see also Frijda, 1986). That is, expressed emotions are grounded in motivational circuits that feed back to sensory systems, heightening vigilance and information gathering, and importantly, prompting reflexive autonomic and motor responses that in evolutionary history acted directly to counter threats and escape punishment, or achieve needed rewards.

In humans, affective percepts, thoughts, and images act as motive cues that automatically engage the older limbic circuits and reflex patterns. Of course, with evolution of the cerebrum came our human metamorphic capacity to delay, modulate or inhibit overt actions, providing for more flexible and adaptive responses to achieve survival’s aims. Nevertheless, the primitive physiology of action preparation persists, mobilizing muscles and glands and placing sensory systems on high alert-- functions that are the physiological manifestations of emotional expression.

This essay considers research undertaken from this perspective, first examining bodily, behavioral, and brain reactivity associated with threatening or appetitive cues, as they vary in their proximity to, and imminence of, punishment or reward. Subsequently, we describe research with emotional pictures, assessing EEG event related potentials, eye tracking, and changes in pupil dilation that co-vary with emotional arousal. Finally, we consider recent fMRI research that delineates structural and network activation differences for appetitive and aversive cues, and show a common activation of the brain’s motivational circuits in emotional perception and mental imagery.

Predator and Prey: The defense cascade

In considering the mortal confrontations between predator and prey, ethologists and comparative psychologists have defined characteristic response patterns that change systematically with the proximity and probability of an encounter (e.g., Blanchard & Blanchard, 1989; Campbell, Wood, & McBride, 1997; Timberlake, 1993; Tinbergen, 1951). When one antagonist perceives the other, responses are sequentially deployed that vary in function and emphasis with the imminence of a physical confrontation: For prey, the sequence enhances the probability that a threat will be survived; for the predator, it optimizes the chance of a final strike and capture. In many ways the required responses (and their mediating brain circuits) are broadly similar, consisting of augmented vigilance (a rising demand for attentional resources), accompanied by increasing physiological mobilization and a readiness for action (fight/flight or strike/capture). In the natural environment, of course, the same organism often plays both roles, foraging for rewards, but wary of being attacked: A small bird fishing for modest game risks the notice of an eagle, a larger, more voracious species that views the fisher and its catch as its proper prey.

In a recent experiment (Löw, Lang, Smith, & Bradley, 2008), we examined anticipatory brain and reflex reactions in human beings in the laboratory using a simulation that was modeled on features of the predator-prey survival scenario, in which the participant acted both parts. The research was prompted by previous data indicating that viewing unpleasant, emotionally evocative pictures elicits patterns of physiological reflex reactions that vary markedly (in direction and amplitude) with evaluative reports of the intensity of affective arousal (e.g., Lang, Bradley, & Cuthbert, 1997). Thus, some autonomic responses, such as skin conductance or pupil dilation, show a monotonic increase with rated emotional arousal. Heart rate, however, decreases with greater arousal, except when viewing aversive pictures that prompt the reports of highest arousal (e.g., in phobic reactions), which then show a dramatic tachycardia (Hamm, Cuthbert, Globisch, & Vaitl, 1997). The probe startle reflex also shows directional changes, initially inhibited and then increasingly potentiated with greater rated arousal of individual aversive picture cues (Bradley, Codispoti, & Lang, 2006; Cuthbert, Bradley, & Lang, 1996; Lang, 1995).

This directional variability within and between measures as emotional intensity increases suggests that different defensive functions are being served along the arousal continuum. One hypothesis is that reports of aversive arousal reflect an atavistic distance parameter, and that the changes in physiology when viewing unpleasant pictures reinstantiate reactions that in prey animals changed as threatened confrontation became increasingly imminent (Lang et al., 1997). From this perspective it is assumed that a human participant looking at an unpleasant/aversive picture is viewing threat at a distance, analogous to a prey animal (e.g., gazelle) apprehending a distant predator (e.g., a lion) from afar. Increases in rated arousal might be associated with progressive changes in reflex physiology that are similar to those observed if a threatening event is increasingly imminent—the lion approaches. That is, reports of emotional arousal indicate, however imperfectly, the intensity of defense system activation which then deploys different reflex responses for remote threats (where confrontation is uncertain) than for an increasingly imminent encounter. At a distance, perceptual processing and information gathering dominate, but as the distance from a threatening stimulus diminishes, organism mobilize increasingly for coping action—the fight/flight imperative.

The aim of the first research to be described here is to examine changes in reaction patterns as motive cues appear to loom progressively closer, as well as to consider possible differences (and similarities) associated with cues that signal pleasant or alternatively, unpleasant consequences (reward or punishment). In a series of classic studies, Neal Miller (1959) assessed behavioral responses (latency and strength of response) in rodents confined to an alley maze, and defined motivational gradients of approach to food reward vs. avoidance of electric shock. His work showed greater relative approach motivation when the animal was distant from the reward, but that the strength of avoidance rapidly superseded approach when the goal was proximal. More recently Kahneman and Tversky (2000) have reported a similar advantage for aversive motivation in human beings, showing that motivation to avoid loss greatly exceeds motivation to gain new resources.

A computer simulation

The predator/prey research environment took the form of a computer simulation: Participants observed a continuous stream of briefly presented pictorial images (see Figure 2). Most of these images depicted motivationally neutral objects, animals, or people, each replacing the other at a constant rate in a continuous flow, mimicking the serial humdrum of daily life. Occasionally, however, a picture appeared that had motivational significance: An offered fist-full of money or alternatively, an attacker’s hand pointing a gun at the viewer. These motivationally relevant images were sometimes repeated several times in a sequence, each time larger, as if the gun or money were moving closer to the viewer. This “looming” sequence could continue for a few “foil” trials, ending with naught but a return to the neutral image stream.

Figure 2.

Diagram of the experimental design: A continuous stream of picture stimuli was presented throughout the experiment. Most of these pictures were emotionally neutral in content (A). However, a fist full of money (B) or a gun aimed at the viewer (C) occasionally appeared in the stream. On response trials (D), the gun, the money, or one of the neutral images loomed increasingly larger for 6 or 7 repetitions. Participants were instructed to press a key at the end of this picture sequence, when the picture background changed color (E). On gun trials a slow response resulted in a loss of $1; a fast response on money trials added $1 to the participant’s stake; all responses to neutral cues were acknowledged without regard to latency. Money and gun pictures could also appear briefly in the neutral stream-- singly or in a short looming sequence of three images. For these brief presentations, there was no subsequent response cue (Löw et al., 2008).

If, however, the image continued its approach, the player was metaphorically in the predator-prey strike zone. At some point in this region, the background of the picture (gun or money) changed color, signaling the participant to press a reaction-time-key as rapidly as possible. On money trials, a fast response resulted in a reward of 1 dollar; a slow response produced no gain and an image of the money burning up. If reactions were slow on a threatening gun trial, the gun appeared to fire and 1 dollar was deducted from the participant’s stake; with a timely response, however, she/he escaped the potential loss and a steel door slammed shut, blocking the gunner. As a control procedure, neutral pictures also sometimes appeared to loom, but, when their background color changed, a key press (under no time pressure) only served to continue the sequence of pictures.

The experimental task permitted the evaluation of both reflex reactions and brain potentials as they are modulated by increasing temporal, proximity/probability of reward or of a punishing loss. In humans, these measures are held to reflect modulation of three coincident anticipatory processes: heightened attention to motivationally significant cues; mobilization of the body for action, preparatory to avoiding a theft or achieving a reward; and emotional arousal, evoked by the context of threat and appetite.

Autonomic responses

As anticipated, skin conductance increased progressively during both reward and punishment sequences. Its amplitude was already somewhat heightened, compared to neutral cues, early in the sequence when a final confrontation was still uncertain. Then, as shown for the gun cue in Figure 3-- but also seen for reward-- it rose dramatically during the final steps before the goal, when a fast response would be required. Sweat gland activity is a well-known indicator of general sympathetic activation (e.g., Wallin, 1981), and associated with increased metabolism, visceral changes, and modulation of sensory systems, as in pupil dilation (e.g., Janisse, 1977). The other autonomic measure examined here, heart rate, showed a modest initial increase (perhaps attributable to vagal release); however, as a possible reward or the risk of loss became more certain (Figure 3), heart rate decreased steadily in frequency to a nadir on the penultimate approach cue. This deceleration was then followed by an abrupt acceleration just prior to the final response signal for escape or capture.

Figure 3.

Heart rate and skin conductance change from pre-stimulus baseline values (samples every 2 sec.) for the last three looming pictures prior to the final gun response signal (Löw et al., 2008).

In preparing to seize a reward or escape a punishment, the two branches of the autonomic nervous system are co-active: The increase in sweat gland activity indicates progressively augmented sympathetic arousal; in contrast, heart rate is mainly under parasympathetic control, and decreases progressively with closer proximity to the goal. This latter phenomenon has been widely noted in biological studies of prey animals confronting a predator (Campbell et al., 1997), and is coincident with relative immobility (freezing) and an increasingly focused vigilance. In the context of predator approach, it is described as “fear bradycardia” (and for the predator/prey task, the deceleration was predictably, somewhat greater for the gun cue). Nevertheless, within the strike zone, of course, there is anticipatory parasympathetic release and a sympathetic increase in rate accompanies the final mobilization for goal directed action. Interestingly, brief anticipatory increases in rate can also be seen increasing over the immediately preceding picture cues.

The probe startle reflex

In less complex animals, the startle response can be considered an automatic escape reflex as with an abrupt movement overhead, the sudden flight of a fly. Although this functional significance has been lost in humans, it is clear that startle reflex amplitude is modulated by at least two factors: Hedonic content (most clearly, a potentiated response in unpleasant perception) and focused attention (prompting relative reflex inhibition).

During the predator-prey simulation, startle reactions were tested at various stages of the approach/avoidance sequence, including when viewing the initial cue, midway through the sequence, and just prior to the response cue. As illustrated in Figure 4, the initial exposure to a motivational cues (i.e., a distant threat or reward) yielded the pattern observed in many studies—significant reflex potentiation when viewing the threat cue and a relative inhibition for appetitive cues (e.g., Bradley et al., 2006; Lang, Bradley, & Cuthbert, 1990). However, as the distance to confrontation closed, reflex magnitude is increasingly inhibited, reflecting effects of task-focused attention and suppression of irrelevant action, facilitating a final functional response to the motive cue. From this perspective, fear potentiated startle appears to be a premature action trigger, subsequently suppressed when the exigencies of a more efficient survival response dominate.

Figure 4.

Average magnitude of the startle probe reflex for money, gun, and neutral cues over the looming sequence. Probes were presented 1100 ms after picture onset during pictures 1 and 3 (“maybe” stage) and during either picture 5 or picture 6 (“imminent” stage). Mean t scores and their standard errors are shown (Löw et al., 2008).

The poetry of the predator-prey interaction can be described as an orchestration of fear and desire. For a natural science of emotion, however, it highlights the impossibility of marrying, in the Jamesian sense, emotional language (fear, anxiety; longing, joy) directly to the underlying dynamic flow of behavior that changes differently across different physiological systems as an anticipated reward or fear of loss passes from the possible to the imminent.

The brain in threat and reward

In addition to autonomic and somatic reflex measures, the electroencephalogram was recorded throughout the predator-prey simulation, and event related potentials were assessed at each stage along the approach/avoidance gradients. Figure 5 shows the electroencephalographic event related potential (ERP) responses to each motivational cue (i.e., gun or money), at their first appearance in the continuing sequence of neutral pictures (upper right panel). Emotional modulation of the ERP is essentially the same as that found when pictures are presented for free viewing (e.g. Cuthbert, Schupp, Bradley, Birbaumer ,& Lang, 2000): A slow positive waveform is apparent beginning around 300 ms and extending past 700 ms that is reliably enhanced over centro-parietal sensors for emotional pictures - either pleasant or unpleasant - that increases in amplitude with increases in rated arousal. As illustrated in the left panel, this late positive potential is dramatically enhanced for all cues at the penultimate (presumably most activating) cue which occurs just before a motor response is required.

Figure 5.

Upper panel: Event-related potentials (average waveform across 13 central-parietal sensors) for the initial money, gun, and neutral cue and for the penultimate looming picture preceding the response signal. The voltage map illustrates the scalp distribution of differences in the late positive potential (300–600 ms after picture onset) between motivationally significant and neutral pictures (i.e., gun pictures minus neutral pictures, money pictures minus neutral pictures). A 0.5-mV change is represented by a contour line; darker shading indicates greater positive voltage. Lower panel: The bar graph shows late positive potential (300–600 ms after picture onset) amplitude differences (means and standard errors) between gun cues and neutral cues and between money cues and neutral cues as a function of picture position in the looming sequence (Löw et al., 2008).

Differences in the late positive potential for the looming gun and money cues, compared to the neutral cue, are shown at the bottom of Figure 5 for each step in the looming sequence. It is of interest that the initial heightened potential to motivational cues shows some decrease early in the sequence, when the continuation of the sequence is uncertain (1-3). Subsequently, however, the amplitude of the late positive potential increases progressively, peaking just before the cue that initiates action. Interestingly, the late positive potential is significantly larger for the gun cues (threat of loss) than for money (a possible gain) at this final stage, consistent with Miller’s (1959) hypothesis that avoidance gradients are steeper than approach (relatively higher close to the goal, cf. Cacioppo & Berntson, 1994), as well as with Kahneman and Tversky’s (2000) view that motivation to avoid a loss is greater than that to achieve a gain.

The late positive potential and motivational relevance

The late positive potential evoked by picture stimuli is a reliable, replicable index of their motivational relevance. Recording from scalp electrodes and with the electroencephalograph (EEG) amplifiers set for a long time constant (allowing one to see sustained brain potentials), Cuthbert et al. (2000) observed an enhanced centroparietal positive signal that began around 300 ms after the onset of an emotional picture and persisted for almost the entire duration of a 6 s viewing interval (see Figure 6). Its relationship to emotional arousal was confirmed in a second study in which we found that this positive potential was most enhanced for the pictures rated highest in emotional arousal, regardless of whether they depicted appetitive (e.g., erotica) or aversive (e.g., mutilated bodies) hedonic contents (Schupp et al., 2004). The amplitude of the late positive potential (LPP) not only correlated highly with reports of rated emotional arousal but also co-varied with sympathetic arousal, as measured by electrodermal response amplitude.

Figure 6.

Event-related potentials over Fz, Cz, and Pz sensors during a 6 s viewing period illustrates a slow positive potential that is enhanced when viewing pleasant or unpleasant, compared to neutral, picture. (Cuthbert et al., 2000).

Complexity or arousal

To confirm that the late positive potential varies with motivational significance, rather than with other physical characteristics of these relatively complex visual stimuli, we varied the perceptual composition of emotional and neutral pictures and presented either simple figure-ground compositions or more complex scenes in a free viewing paradigm (Bradley, Hamby, Löw, & Lang, 2007). Not surprisingly, perceptual composition had strong effects on the event-related potential; however, these effects appeared earlier in time (150-250 ms), were maximal over occipital-temporal sensors, and did not differ for emotional and neutral pictures. The late positive potential, on the other hand, continued to be enhanced when viewing emotional (pleasant or unpleasant), compared to neutral pictures, regardless of perceptual composition. Interestingly, effects of emotion on the late positive potential were somewhat larger for simple figure-ground compositions, contrary to the view that the late positive potential reflects the complexity of information processing. Rather, hedonic content was communicated more effectively by these simple figure-ground compositions, consistent with the finding of stronger affective modulation of the startle reflex for simple, compared to complex, scenes (Bradley et al., 2007).

Attention and habituation

We originally entertained the hypothesis that sustained positive potentials during affective perceptual processing simply reflected enhanced attention allocation to the motivationally relevant pictures. Previous findings using secondary probes were consistent with this view as the evoked brain response (i.e. probe P3 amplitude) is smaller when viewing emotionally arousing pictures (e.g., Schupp, Cuthbert, Bradley, Birbaumer & Lang, 1997) and probe reaction times are slower (Bradley, Cuthbert, & Lang, 1996). Furthermore, memory is enhanced for emotional, compared to neutral, pictures (Bradley, Greenwald, Petry, & Lang, 1992). However, a more recent series of studies conducted in conjunction with Maurizio Codispoti and Vera Ferrari argue that this simple, attention-capture interpretation may be too narrow (Codispoti, Ferrari, & Bradley, 2006, 2007). In these studies, we presented the same picture for as many as 90 trials in the same experimental session. Although the overall amplitude of the late positive potential did diminish somewhat with repetition, significant modulation of the late positive potential by emotion persisted to the last presentation (see Figure 7). Moreover, an enhanced late positive potential is found even when pictures are presented very briefly (25 ms): Given that the late potential persists for seconds after the picture itself is no longer on the screen in this case, the hypothesis that it reflects a continuing perusal of the sensory array would seem to be ruled out (Codispoti, Mazzeti, & Bradley, 2009; see Figure 7).

Figure 7.

Event-related potentials measured over centro-parietal sensors illustrate an enhanced late positive potential for emotional (either pleasant or unpleasant), compared to neutral pictures when pictures are novel (left panel), repeated (middle panel; Codispoti et al., 2007) or presented for only 25 ms (right panel; Codispoti et al., 2009).

On the basis of these and other autonomic data involving picture repetition, it was suggested (Bradley, 2009) that the late positive potential is a broad index of motivational “significance”. That is, the LPP is a persistent sign that the evoking stimulus is motivationally relevant, reliably different from neutral cues. Furthermore, of course, it varies in amplitude with increasing motive system activation (appetitive or defensive). Late positive potentials are enhanced at initial orienting, extended with sustained perceptual processing, associated with preferential memory storage, and in considering the predator/prey findings, perhaps further augmented when motivated actions are imminent.

What brain structures underlie the ERP late positive potential?

Determining the neural structures that are primary contributors to any specific component of the event-related potential is a difficult task: Most meaningful features of the ERP waveform are less than 200 ms in duration, and these bioelectric signals are determined by volume conduction across the whole brain at each scalp electrode (albeit from different trajectories). Considering the relatively large amplitude of the late positive potential, however, and its sustained duration (which may persist over several seconds), it is perhaps the strongest candidate-ERP event for which the spatial location of primary voltage sources in the brain might be uncovered. Consistent with the view, Keil et. al (2002) used the minimum norm method to determine dipole sources contributing to the LPP signal, based on the analysis of a 129-electrode array. The most reliable differences in dipole strength that showed enhanced activity when viewing emotional, compared to neutral, pictures, were located in the dorsal and ventral posterior brain. While lacking structural specificity, the findings are consistent with fMRI findings (Bradley et. al, 2003; Lang et. al, 1998) showing enhanced activation for emotional pictures in secondary visual processing sites in lateral occipital cortex, extending up the dorsal stream to parietal cortex.

A more recent study (Sabatinelli, Lang, Keil, & Bradley, 2007) provided a more precise answer to the question of the neural determinants of the late positive potential evoked during picture viewing. The experiment was conducted in two sessions that repeated the same procedure on two different days. Participants were seen either first in the scanner at the MRI facility where functional changes in specific brain regions were measured, or first in an MRI simulator that exactly duplicated the scanner environment, and where dense sensor EEG was safely and accurately recorded. Subjects were simply instructed to look at each picture while it was on the screen, and at a fixation cross between trials. Pictures from the International Affective Picture System (IAPS, Lang, Bradley, & Cutlhbert, 2008; Bradley & Lang, 2007) included equal numbers of images rated pleasant, unpleasant and neutral (using SAM, Bradley & Lang, 1994; Lang, 1980), presented in an intermixed order. The same pictures were seen in both sessions. As previously discussed, research has shown that modulation of the late positive potential by emotion is highly stable over repeated trials, even when the same picture is presented multiple times (e.g., Codispoti et. al, 2007). We hypothesized that the determining underlying neural structures would show a similar consistency over the two experimental sessions.

Figure 8 (left panel) shows the scalp voltage maps for the late positive potential and the fMRI activation sites observed when participants viewed pleasant, neutral, and unpleasant pictures. From this posterior view, it is apparent that, compared to neutral pictures, pleasant and unpleasant emotional images similarly prompt greater blood oxygen level-dependent (BOLD) increases in the occipital and parietal brain regions that support perceptual processing (lower left panel). The topographical maps of the late positive potential (upper left) show that emotional pictures (both pleasant and unpleasant) prompt parallel stronger voltage foci for the LPP over the anterior brain than is found for neutral scenes. The right panel compares event related response waveforms from the EEG (positive voltage up) and fMRI sessions: Similar enhanced responses are apparent for emotionally arousing pictures (pleasant and unpleasant), relative to neutral, despite the great difference in the latency of EEG and BOLD signals.

Figure 8.

Left: Comparison of ERP scalp topography (above) and fMRI BOLD contrast overlay (below) for pleasant, neutral, and unpleasant pictures. The map shows average microvolt change for the late positive potential (400 to 900 ms after picture onset) with red indicating positivity and blue indicating negativity. The BOLD contrast overlay (coronal slice y = –68) represents a random-effects analysis of picture elicited activity, peaking roughly 8 s after picture onset, with red indicating more reliable increases in oxygenated blood flow and yellow indicating a threshold of P < 0.000000001 (Sabatinelli et al., 2007). Right: Time course development of the late positive potential (a) and visual cortical BOLD (b) for pleasant, neutral, and unpleasant picture contents. The late positive potential waveform represents 27 centroparietal sensors (Fig. 2b), whereas the BOLD waveform represents average activity in inferotemporal, lateral occipital, and parietal ROIs. Picture onset is indicated by the arrow at time zero.

In subsequent analyses, we assessed the relationship between the late positive potential (measured over centro-parietal sensors) and BOLD activity in specific brain regions implicated in secondary visual processing: Significant correlations were obtained between the amplitude of the LPP and activation in lateral occipital cortex (.65), parietal cortex (.61), and in inferotemporal cortex (.55). Importantly, the amplitude of the late positive potential did not correlate significantly with activation in primary (pericalcarine) visual cortex, where the sensory stimulus is first mapped prior to secondary (semantic/affective) elaboration. Significant correlations of the late positive potential with deeper limbic structures (e.g., the amygdala) were also not found, perhaps because the latter, small neural structures generate a weaker and less reliable BOLD response, and being distant from the scalp, their relatively minute neural signal is negligible in the volume conduction recorded by surface electrodes. Overall, these data show a dramatic co-variation between the late positive potential of the ERP and BOLD activity measured using fMRI, suggesting that the late positive potential, evoked during non-task-related picture viewing, is generated predominantly by elaborated perceptual processing in the extra-striate visual system of the cerebral cortex.

Amygdala/visual cortex connections during picture viewing

One interpretation of these ERP and fMRI findings is that the higher activation of visual processing areas for emotional stimuli is attributable to re-entrant projections to the sensory system from the brain’s motivational circuits. In detailed primate studies (Amaral, Price, Pitkänen, & Carmichael, 1992), it has been shown that visual information is processed in a hierarchical fashion along the ventral temporal stream from primary striate cortex to higher level processing areas and proceeding finally from the inferotemporal (TE) area to the lateral nucleus of the amygdala. Subsequently, the amygdala’s basal nucleus back-projects “to virtually all levels of the visual cortex’ (p.44: Amaral et al., 1992). Thus, it is held that emotional cues receive sustained processing that is mediated through a positive feedback loop that is specifically enhanced for “significant” stimuli.

Presumptive evidence for this hypothesis was obtained in a fMRI experiment in which we presented fear-related pictures to highly fearful individuals (Sabatinelli, Bradley, Fitzsimmons & Lang, 2005). Emotional and neutral pictures were organized into categories that represented a monotonic increase in rated emotional arousal sequentially, from neutral scenes to neutral people, to non-threatening animals, to snakes, to erotica, to mutilated human bodies. The regions of specific interest were the inferotemporal cortex and the amygdala. Inferotemporal cortex has been implicated in the processing color, faces, and semantic categories (Bartels & Zeki, 2000; Chao, Haxby & Martin, 1999; Gauthier, Curran, Curby & Collins, 2003; Grill-Spector, Knouf, & Kanwisher, 2004) and, as noted previously, we have consistently found greater activation of secondary visual processing centers when people view emotional, compared to neutral, pictures (Bradley et al., 2003; Lang et al., 1998). Furthermore, diffusion tensor imaging research has revealed a direct white matter tract in humans between secondary visual and anterior temporal cortex (Catani, Jones, Donato, & 2003), complementing the findings from primates (cited above).

In figure 9 (right panel), overall differences in the BOLD response between neutral picture contents and the most emotionally arousing pictures (erotica and mutilations) are shown for the bilateral amygdala and inferotemporal cortex. Consistent with hypothesis, both high and low fear individuals showed substantially larger responses when viewing emotionally arousing contents (both pleasant and unpleasant). Also as predicted, there was a parallel significant linear increase in BOLD signal over categories, almost identical in the two bilateral structures (see Figure 9, upper left panel).

Figure 9.

Effects of emotional intensity and fear relevance on amygdala and inferior temporal regions of interest (ROIs) as measured with functional magnetic resonance imaging (fMRI; Sabatineli et al., 2005). Areas of functional activity during picture processing (superimposed on the average brain) are present in the right panel. Average standardized blood oxygen level dependent (BOLD) signal change in the control group (left panel, top) and the snake fearful group (left, bottom) are presented for the amygdala and inferotemporal cortex, as these signals varied over different picture content categories. Error bars represent standard errors of the mean. The picture types are ordered on the abscissa (left to right) based on average ratings of emotional arousal from the International Affective Picture System (IAPS; Lang et al., 2008).

The bottom left panel illustrates the findings for the set of participants who reported high fear of snakes on a prescreening questionnaire. These fearful individuals (N=9) showed a significantly enhanced response when viewing pictures of snakes when compared with the other (unselected) participants. The co-variation between rated arousal of the picture and BOLD activity is impressive, both generally and considering individual differences in fear. Furthermore, these data are consistent not only with the hypothesis that these structures are interconnected, but that re-entrant input from the amygdala prompts enhanced perceptual processing of emotionally arousing cues.

More recently, using a novel fMRI fast-slice methodology, Sabatinelli, Lang, Bradley, Costa and Keil (2009) showed clear differences in the onset latency of BOLD effects in secondary visual processing areas further supporting Amaral’s primate model (Amaral et al., 1992). That is, emotionally arousing pictures prompted significant activation first in structures more closely connected to the amygdala (i.e., inferotemporal “fusiform” cortex) and palpably latter in more distant, posterior regions of the visual system (i.e., lateral occipital cortex). Other findings based on a Granger causality analysis of steady-state EEG potentials visually evoked by emotional stimuli (Keil et. al, 2009) suggest there may be an even broader re-entrant connectivity to the visual processing system from anterior neural structures, both subcortical and cortical.

The brain begins at the eye

For visually oriented mammals, particularly primates, the eye is the principal organ for gathering information from the environment concerning threat and safety, sexual opportunity, and for successful foraging. The brain begins with the neuroectodermal photoreceptors at the back of the eye, and the optical system as whole-- lenses, receptor bed, and pupillary and orienting musculature—is under fine nervous control. Thus, motivationally relevant signals from the environment prompt rapid adjustment in the brain’ primary receptor organ.

In a recent series of experiments we have tracked the eye as it looks at visual cues depicting appetitive and aversive events, measuring both scanning behavior (e.g., number and duration of fixations; Bradley, Miccoli, & Lang, 2009) and changes in pupil diameter (Bradley, Miccoli, Escrig, & Lang, 2008). Although physical properties of visual stimuli such as contrast, edges, luminance, etc. are known to assist in guiding the eye as it traverses a visual stimulus (e.g. Marr, 1985), it has also long been known that the eye is also more likely to fixate on interesting or salient information in the environment and that the gaze remains longer on interesting or informative events (e.g., Loftus & Mackworth, 1978).

In our research (Bradley et. al, 2009), participants viewed emotional and neutral pictures that were either simple figure-ground compositions, or more complex scenes. Eye tracking behavior was significantly affected by the motivational relevance of pictures -- regardless of their perceptual complexity. Thus, emotional pictures - either pleasant or unpleasant - were associated with a significantly greater number of fixations than neutral images, and this pattern was found regardless of whether the pictures depicted simple figure-ground compositions or more complex scenes. Moreover, affectively engaging cues were scanned much more broadly, as evidenced by significantly greater average distances between successive fixations than for neutral pictures, which again was found for both simple figure-ground compositons as well as more complex scenes. Taken together, the data suggest that picture cues that contain emotionally arousing content generate increased scanning that sweeps more broadly across the image when motivationally relevant information is detected.

Affective images also prompt significantly larger pupil diameter during picture viewing. Hess (Hess & Polt, 1960) famously proposed that the pupil dilated for positive cues (e.g., acceptance) and constricted for negative cues (e.g. rejection), but there were numerous methodological and measurement issues in those early studies, and the results proved difficult to replicate. Because the pupil is extremely sensitive to changes in light, it is quite important to equate affective cues in terms of their brightness. Fortunately, the pupil itself proves to be a good index of brightness differences, as the initial light reflex (pupil constriction) following presentation of a visual cue can be easily measured. In our study (Bradley et. al, 2008), we initially matched emotional and neutral pictures in terms of contrast and brightness, and measured the initial light reflex. As illustrated in Figure 10, the initial pupil constriction following picture onset was equivalent across pleasant, neutral, and unpleasant pictures, allowing an assessment of pupillary changes during picture viewing that may be specifically associated with hedonic content.

Figure 10.

Pupil diameter shows an initial light reflex following picture onset, with a subsequent increase in pupil diameter that is enhanced when viewing pleasant or unpleasant, compared to neutral pictures. Inset: The same pattern of increased pupil diameter is found regardless of whether pictures depict simple figure-ground compositions or more complex scenes. (Bradley et al., 2008.)

As illustrated in Figure 10 (inset), viewing either pleasant or unpleasant pictures prompted relative pupil dilation, compared to neutral pictures, and the same pattern was found regardless of whether pictures had been judged to be simple in perceptual composition (e.g., figure-ground) or were more complex scenes, showing a within-study replication across different affectively engaging pictures. Pupil dilation also covaried strongly with changes in skin conductance activity. Thus, for both pupil diameter and skin conductance change, reactions were largest for the pleasant and unpleasant pictures rated as most arousing. The close relationship between pupil diameter and skin conductance reactivity was expected, considering that increases in both responses are mediated by the sympathetic nervous system.

In the view presented here, these autonomic reactions begin when a motivational cue is processed in visual cortex and activates the basolateral and then the central nucleus of the amygdala, which subsequently projects to the lateral hypothalamus, prompting broad engagement along the sympathetic chain, including neural connections to the rapidly dilating pupil, as well as to the more slowly responding palmar sweat glands (see Figure 1). The evolutionary significance of these responses has prompted much speculation (e.g., pupil dilation increased light gathering in dark environments; palmar sweating reduced abrasion), but there is little support for a specific view. As for the potentiated startle response, these autonomic reflexes appear to be vestigal remainders from evolutionary history. They are still automatically deployed in motivational arousal, indexing emotional intensity; however, their specific survival value is no longer apparent.

Differences in brain circuit activation for pleasant and unpleasant cues

Looking at pictures

Initial exposure to a novel unpleasant picture is associated with sustained cardiac deceleration, facial muscle action (corrugator “frown” muscle), and potentiation of startle probe reflexes, whereas these responses differ dramatically in direction and magnitude during exposure to a novel pleasant cue (e.g., Bradley & Lang, 2007; Lang, Greenwald, Bradley, and Hamm, 1993). Thus, these physiological measures indicate clearly which motivational system is active -- appetitive or defensive. In contrast, skin conductance, pupil dilation, and the brain’s late positive potential increase in amplitude as both pleasant and unpleasant pictures are rated more arousing, indicating the intensity or vigor of motivation, but not its goal direction (see Bradley and Lang, 2007.

These findings show, on the one hand, that much of the brain is similarly engaged by motivationally relevant cues, whether they signal threat or reward; on the other hand, it is apparent that some brain circuits must be specific to the hedonic valence of an initiating cue, deploying different responses depending on the goal. We have already seen that the amygdala fits the former case and is highly responsive to the intensity of both appetitive and aversive emotional stimulation, and that it is central in elevating sympathetic reactivity and in enhancing visual processing-- reflexes that are clearly appropriate both under conditions of threat and appetitive arousal. We will next consider research exploring other brain structures and circuit connections—sites in the cortex and limbic brain specific to appetitive motivational processing and positive affect.

Research with animals suggests that reward-seeking behavior is controlled by neural circuits that include the basolateral nucleus of amygdala, medial prefrontal cortex, and the nucleus accumbens (e.g., Ishikawa, Ambroggi, Nicola & Fields, 2008) that function as an interface with motor areas (Mogenson, 1977). This appetitive circuit might also subserve pleasure-specific emotional reflexes. Neural imaging research with human participants has indeed implicated these neural structures in the perception of motivationally positive stimuli (e.g., pictures of loved ones: Aron et al., 2005, Bartels and Zeki, 2004; addictive drug cues; David et al., 2005); however, other research has suggested that the striatum (which includes the nucleus accumbens) may be active for any salient stimulus that arouses and prompts attention (Zink, Pagnoni, Chappelow, Martin-Skurski, & Berns, 2006).

To assess whether these mesolimbic circuits are specifically activated for pleasurable cues, or whether their activation instead varies with emotional arousal, we presented highly arousing pleasant (erotic) and unpleasant (mutilated bodies) pictures together with pictures of neutral people in a brain imaging (fMRI) study (Sabatinelli, Bradley, Lang, Costa, & Versace, 2007). As anticipated, the BOLD response in the bilateral amygdala was significantly greater when participants viewed erotic or mutilation pictures than for quotidian pictures of neutral people, again indicating that the amygdala is equivalently activated by emotionally arousing stimuli, whether appetitive or aversive. In contrast, it was only erotic stimuli that activated medial prefrontal cortex and the nucleus accumbens, consistent with the view that these structures are specifically implicated in the brain’s appetitive motivational circuit.

A second, replication experiment expanded the picture content to include clothed romantic couples (e.g, kissing, holding hands) and unpleasant scenes in a dental operatory, snakes (coiled or attacking), and threatening people (with guns or knives). As illustrated in Figure 11, viewing appetitive pictures of either explicit erotica or scenes with romantic couples prompted significant BOLD increases in the nucleus accumbens. An almost identical result was obtained for the medial prefrontal cortex. However, neither neutral nor unpleasant pictures activated these structures. In fact, BOLD activity when viewing these non-appetitive stimuli appears to actually be below baseline, particularly in the medial prefrontal cortex (a phenomenon also seen in the first experiment). Again, amygdala was significantly activated when viewing either arousing pleasant or unpleasant pictures, compared to the neutral content.

Figure 11.

Functional activity when viewing erotic and romantic pictures, relative to when viewing neutral people and unpleasant contents. Random-effects analyses reveal bilateral nucleus accumbens activation (Nac) that is greater when viewing pleasant (erotic and romantic) relative to neutral people and unpleasant pictures. Furthermore, event-related time courses of percentage BOLD signal change in nucleus accumbens reveal that, in contrast to the response to erotic and romantic pictures, snake and human threat contents did not prompt a signal increase. (Sabatinelli et al., 2007)

The results are clear in finding that arousing appetitive pictures selectively activate the same neural structures observed in animals anticipating rewards and as seen in human substance users exposed to tangible drug cues. The data do not support the hypothesis that accumbens and prefrontal cortex sites respond to all salient cues that arouse or capture attention. Rather, in contrast to activation patterns found in the amygdala and inferotemporal cortex, appetitive neural structures are silent, perhaps even deactivated in the context of arousing unpleasant picture stimuli.¹

Emotional imagery and appetitive motivation

Text-driven mental imagery of traumatic memories or phobic confrontations prompts psychophysiological reactions similar to those observed in humans anticipating an actual aversive experience (Lang, Kozak, Miller, Levin, & McLean, 1980; Miller et. al, 1987). The bio-informational theory of imagery (Lang, 1979, 1994) originally proposed that text cues engage an emotional network of representations in the brain that includes sensory, semantic, and response information. The response information (procedural knowledge) is of particular significance for the action mobilization that characterizes emotional engagement, and it is these efferent memories that are most directly connected to the brain’s motivational system.

The above conception suggests that motor control systems in the brain should be active during affective imagery, mediating the efferent output represented in emotional scenarios. Brain activation patterns consistent with this bio-informational view were obtained in a recent fMRI study of pleasant and unpleasant imagery: Greater activation was observed for arousing emotional scenarios in supplementary motor area, prefrontal cortex, and cerebellum (areas involved in planning and executing motor action) than for neutral imagery (Sabatinelli, Lang, Bradley, & Flaisch, 2006). This activation pattern is, of course, consistent with psychophysiological studies showing reflex mobilization in arousing emotions-- increased facial muscle action, heart rate and skin conductance increases, and potentiated startle when imaging arousing aversive content (e.g., Lang & McTeague, 2009; McTeague et. al, 2009).

Perhaps because the evocation of vivid fear imagery in desensitization (e.g., Wolpe, 1961) and exposure (e.g., Foa & Kozak, 1986) have been important methods in the treatment of anxiety disorders, fMRI and PET studies of limbic circuits have focused almost exclusively on imagery activation during aversive emotional imagery (e.g., Shin et al., 2004). In a recent study, however, we included both pleasant and unpleasant emotional contents to determine if the same neural circuits implicated in appetite and defense are activated in mental imagery as in picture perception. Costa, Lang, Sabatinelli, Bradley & Versace (2009) assessed imagery of erotic content (e.g., sexual encounters), as well as scenes of joy (e.g., as winning a lottery), quotidian neutral scenes (e.g., doing laundry), and scenes describing animal attacks, contamination (e.g, a drunk vomits on your hand), survival threats (e.g., an auto accident) and dental exams.

Figure 12 shows the location and extent of BOLD activation in the motive relevant structures and the temporal course of the imagery-evoked waveforms for pleasant, neutral, and unpleasant imagery contents. As for emotional pictures, both pleasant and unpleasant arousing imagery strongly activated the amygdala. In contrast, only imagining pleasant scenes engaged the nucleus accumbens and the medial prefrontal cortex. Furthermore, the reduction in regional blood flow during imagery of unpleasant scenes in accumbens and prefrontal cortex is even more dramatic than it was during picture perception. Because hemodynamic measures are thought to reflect both excitatory and inhibitory processes (Logothetis, 2008), this mesolimbic BOLD decrease could indicate inhibition of the appetitive system in aversive arousal, permitting the reciprocal activation of defense circuits.

Figure 12.

Increased activation of nucleus accumbens (Nac—A) and medial prefrontal cortex (mPFC--B) is obtained during pleasant, compared to neutral, imagery, whereas amygdala (C) activity is increased both during pleasant and unpleasant, compared to neutral, imagery. Event-related signal change is plotted for pleasant, neutral, and unpleasant scenes beginning with text presentation and continuing through subsequent imagery in accumbens (D), prefrontal cortex (E), and the amygdala (F). Talairach coordinates are based on coronal slice selection. Error bars represent 95% confidence intervals (Costa et al., 2009)

High correlations were obtained between the participant’s pleasure ratings of the imagined scene and BOLD activity in both the nucleus accumbens (.75) and the medial prefrontal cortex (.82: see Figure 13). In addition, valence-dependent functional connectivity analyses (Friston et al., 1997) revealed a reliable time series co-variation between the accumbens and prefrontal cortex that was significant over all valence categories. However, connectivity between these sites and the amygdala was present only during pleasant imagery. In effect, although amygdala reacts to any motive cue-- and more strongly with increasing arousal, activation of the appetitive system and a consequent positive emotional states, depends on an active connection with the mesostriatal circuit.

Figure 13.

Pleasure ratings correlate with mesolimbic BOLD activity during imagery. Scenes were rank ordered by each participant’s pleasure ratings and plotted against the mean signal change in medial prefrontal cortex (mPFC) at each rank. Pie charts depict the proportion of the a priori (based on ANET standardized ratings; Bradley and Lang, 2007) pleasant, neutral, and unpleasant texts at each rank (Costa et al., 2009).

Summary and Conclusions

The research presented in this review is driven by an animal model that defines appetitve and defensive motivational centers-- deep in the older limbic brain and primitive cortex-- that are the foundation of human emotion. In this view, emotions reflect sequenced, somatic and autonomic reflexes, organized by brain circuits that developed over evolutionary history to ensure individual survival and the propagation of genes across generations. The research has shown that in emotional perception some of these reflexes are prompted by appetitive cues, others by aversive cues, and many more are active in response to motivationally relevant stimuli, both appetitive and aversive. It has been further found that these reflex patterns are dynamic, and that individual responses are differently modulated in direction and amplitude along gradients of appetitive approach and aversive arousal.

Reflex reactions to emotional cues that signal rewards or threaten danger are associated with two primary functions: (1) They capture attention preferentially and prompt perceptual processing and information gathering (e.g. Lang et al., 1997; Bradley, 2009); (2) they occasion metabolic arousal and mobilize the organism for coping actions. In this review we considered how emotional cues, both pleasant and unpleasant, evoke an extended late positive potential in the EEG that is—as shown by fMRI evidence-- highly correlated with the activation of visual processing centers in the posterior brain. This event related potential is a persistent, specific reaction to cues of emotional significance, whether perceptually simple or complex in composition. Functional MRI research demonstrates a high correlation between activation of the amygdala and posterior cortical sites that are involved in elaborative picture processing. Furthermore, activation in both sites vary systematically with reports of emotional arousal, supporting the hypothesis that increases in the visual processing of emotional cues are attributable to re-entrant processing from anterior motivational circuits.

Studies of gaze direction indicate that the eye preferentially fixates first on pictures with emotional content over competing neutral images. Furthermore, both complex and simple emotional pictures are associated with significantly more fixations than neutral content, and are scanned more broadly. The pupil also responds to emotional content, dilating more for pleasant and unpleasant images than for neutral pictures, and similarly mediated by the sympathetic nervous system, shows a close correlation with skin conductance and with arousal ratings.

Finally, human fMRI research is consistent with animal studies that implicate the mesostriatum in reward processing. Both in picture perception and during narrative imagery, the nucleus accumbens and medial prefrontal cortex are selectively activated by pleasant stimuli, whereas the amygdala shows increasing activation with greater rated arousal for both pleasant and unpleasant content. Connectivity analyses reveal an appetitive circuit connecting all three structures, uniquely during pleasant picture processing.

In conclusion, both psychophysiological and neuroscience research supports the motivational view of human emotional expression, delineating the mediating neural structures and their autonomic and somatic reflex output . Emotions occur when appetitive or aversive cues activate primitive circuitry in the limbic brain that earlier evolved to promote the survival of organisms and their genetic inheritance.