Emergent processes in cognitive-emotional interactions

Abstract

Emotion and cognition have been viewed as largely separate entities in the brain. Within this framework, significant progress has been made in understanding specific aspects of behavior. Research in the past two decades, however, has started to paint a different picture of brain organization, one in which network interactions are key to understanding complex behaviors. From both basic and clinical perspectives, the characterization of cognitive-emotional interactions constitutes a fundamental issue in the investigation of the mind and brain. This review will highlight the interactive and integrative potential that exists in the brain to bring together the cognitive and emotional domains. First, anatomical evidence will be provided, focusing on structures such as hypothalamus, basal forebrain, amygdala, cingulate cortex, orbitofrontal cortex, and insula. Data on functional interactions will then be discussed, followed by a discussion of a dual competition framework, which describes cognitive-emotional interactions in terms of perceptual and cognitive competition mechanisms.

A century of neuroscience research has yielded evolving views of the organization of the brain in general, and of how emotion and cognition are instantiated in gray matter in particular. Proposals highlighting the importance of specific regions, including the hypothalamus and the amygdala, as well as proposals describing elaborate circuits, such as those by Papez and MacLean, have been advanced. It is undeniable that certain brain regions play an important role in emotion. Yet, it is also apparent that they do not work in isolation and, instead, participate in distributed networks of regions that, collectively, carry out important functions. From both a basic and clinical perspective, an especially challenging problem is to understand the relationship between brain networks that are important for perception and cognition, and those that determine the affective value of stimuli and contexts. In this review, I will highlight the interactive and integrative potential that exists in the brain to bring together the cognitive and emotional domains. Because the backbone for these interactions is anatomical, the first section will describe several examples of how the transfer of information takes place. The second section illustrates some examples of the interaction between perception and emotion, and between cognition and emotion. The final section presents considerations of how to conceptualize cognitive-emotional interactions in terms of perceptual and cognitive competition mechanisms.

Anatomical substrates for cognitive-emotional interactions

This section describes how the architecture of the brain includes multiple avenues for information integration. As described, the substrates for information interaction and integration are plentiful and provide the potential for the coordinated flow of information that characterizes complex behaviors.

Hypothalamus

The importance of the hypothalamus in certain aspects of emotion is well known, as highlighted by the work of Cannon and Bard; the latter showed via “decortication” experiments that emotional expression effects were abolished when the hypothalamus was eliminated, but not when only the neocortex was compromised. Since the 20s and 30s our knowledge of hypothalamic function has been greatly extended and refined, and current understanding concurs with the earlier notion that the hypothalamus is involved in several important survival-related functions. To coordinate these functions, the hypothalamus works in association with a multitude of other sites in the brain stem and spinal cord.

Historically, the role of the hypothalamus has often been conceptualized as “descending”, a view that is summarized in the designation of the hypothalamus as the “head ganglion” of the autonomic nervous system. The importance of the hypothalamus for descending control notwithstanding, a recently recognized fact is the recognition that the cerebral cortex and hypothalamus share massive bidirectional connections. In the rat, which constitutes the best studied case, there are four major routes from the hypothalamus to the cerebral cortex¹ (Fig. 1). These include a major direct projection to all parts of the cortical mantle, and three indirect routes by way of the thalamus, basal nuclei (specifically, magnocellular basal forebrain and amygdala), and brainstem (for discussion of the indirect routes, please see ref.¹).

Figure 1.

Hypothalamic ascending connectivity. Summary of the four major pathways from the hypothalamus to the cerebral cortex schematized on a flattened representation of the rat brain. The basal ganglia here refer to the magnocellular basal forebrain and the amygdala complex. Note that one of the indirect connections first “descends” to the brainstem. Abbreviations: BG: basal ganglia; BS: brainstem; CTX: cortex; HY: hypothalamus; TH: thalamus. Reprinted from ref.¹ with permission from Elsevier.

The direct input to the cortical mantle appears to be the largest source of non-thalamic input to the cortex^{1, 2}. In the rat, some important targets include infralimbic, prelimbic, anterior cingulate, and insular cortices. Interestingly, projections to the lateral prefrontal cortex are also found, and even to primary sensory areas (though both are less prominent). An important indirect system connects the hypothalamus to the cortex via the magnocellular basal forebrain system. Another noteworthy route to the cortex involves several amygdala nuclei, including projections via the basolateral nucleus that reach cingulate, motor, and visual areas. The organization of the connections between prefrontal cortex and hypothalamus has been investigated in non-human primates, too, and are in close concordance with the findings in rats³. Notably, all prefrontal areas investigated received projections from the hypothalamus. In addition to the systems linking the hypothalamus to cortex, conversely, major telencephalic projections to the hypothalamus also exist, including those from the hippocampal formation, amygdala, insular cortex, and prefrontal cortex.

In summary, whereas the hypothalamus is involved in a host of basic control functions, it is part of an extensive bidirectional connective system with cortex and many other subcortical structures in a manner that allows for extensive integration of cognitive and emotional information. Critically, the hypothalamus is linked to other structures that have themselves widespread connectivity, including the magnocellular basal forebrain and the amygdala.

Basal forebrain

The basal forebrain is a heterogeneous set of structures close to the medial and ventral surfaces of the cerebral hemispheres. The magnocellular basal forebrain system is a prominent feature of the primate basal forebrain, involving a continuous collection of large neurons that involve the basal nucleus of Meynert (sometimes called “substantia innominata”), and cell groups within the septum and the horizontal limb of the diagonal band. The magnocellular basal forebrain system originates an “ascending” (i.e., corticopetal) cholinergic and GABAergic projection system that innervates throughout the cortical mantle. Major projections reach several cortical areas, including peristriate, inferotemporal, superior temporal, parahippocampal, temporopolar, posterior parietal, cingulate, frontoparietal opercular, lateral prefrontal, and orbitoinsular regions⁴. Extensive projections are found to both the hippocampus and amygdala⁵, too. An important pattern of this projection system is that a connectivity gradient can be identified, such that the densest projections from the basal forebrain to cortex are found for non-isocortical components of temporopolar, insular, and orbitofrontal cortices⁵ (where isocortical typically refers to cortex with six identifiable layers). This pattern is consistent with the dense innervations (stronger than cortex) observed for both the hippocampus and amygdala, two regions with simplified cytoarchitecture (i.e., pattern of laminar structure).

Given its overall connectivity pattern, the magnocellular basal forebrain system is in a favorable position to influence cortical sites across the brain, including sensory cortex, and thus to influence the flow of information processing. These distributed effects result in increased vigilance, alertness, and attention, and more generally have the potential for widespread impact on cognitive function both in health and mental illness^{6, 7}.

As with other neurotransmitter systems in the brain, the effects of the magnocellular system are at times described as relatively global, or at least unspecific. However, specific effects have also been documented. For instance, visual responses that are conveyed to prefrontal cortex engage the basal forebrain in a polysynaptic way, which then further enhances visual responding⁸. Direct stimulation of the basal forebrain also enhances the cortical coding of natural scenes in visual cortex by markedly improving the reliability of cell responses⁹.

Whereas the magnocellular system projects in a widespread, distributed fashion to cortical and subcortical regions, it is noteworthy that afferent fibers originate from a much more circumscribed set of regions. Cortically, inputs originate largely from non-isocortical areas^{5, 10}. Given that these are exactly the regions that receive the densest inputs from the basal forebrain, potent basal forebrain-cortical circuits can be established.

Amygdala

A remarkable property of the primate amygdala is its massive interconnection with cortex. Based on the available data at the time, analysis of amygdala connectivity revealed that this structure was connected to all but 8 of the cortical areas included in the study¹¹ (see also refs.^{12, 13}). These connections involved multiple region clusters, suggesting that the amygdala¹⁴ is not only one of the most highly connected regions of the brain, but that its connectivity topology is consistent with that of a “connector” hub¹⁵ (where a hub is a region with high degree of connectivity) that links multiple “provincial” hubs¹⁵ – where the latter refer to regions of dense connectivity more closely associated with a specific functional group, such as area V4 in visual cortex¹⁶. In this manner, the amygdala has strong potential for integrating cognitive and emotional information¹⁷.

When whole-brain connectivity data are analyzed, prefrontal areas are among those most distant from the sensory periphery – based on the average number of connections¹¹. Thus, on average, the prefrontal cortex receives highly-processed and integrated sensory information. This structural feature is thought to be important because it provides the prefrontal cortex with relative insulation from the periphery. Indeed, this organization has been proposed to be a key anatomical feature of this region that may confer the primate brain with a greater degree of flexibility⁴. Highly processed information may also be important in supporting more abstract processing that is required for cognition. It is thus noteworthy that the amygdala (and other regions, such as the hippocampus and entorhinal cortex) was found to be also removed from the sensory periphery¹¹, indicating that this region is well situated to integrate and distribute information, not unlike certain prefrontal cortex territories.

Connections from the sensory periphery to the amygdala that bypass the cortex have been documented, too. For instance, in rodents, the medial geniculate body in the thalamus conveys auditory information to the amygdala and provides a “low road” (i.e., subcortical pathway) for auditory information¹⁸. The potential role of subcortical pathways conveying emotional information is discussed at length elsewhere. As described, in primates, it is unlikely that fast, subcortical pathways play a prominent role in affective visual processing¹⁹. Instead, it was suggested that fast visual processing of affective stimuli relies on multiple, parallel cortical pathways that rapidly convey information to the amygdala and other evaluative sites, such as the orbitofrontal cortex^{19, 20}.

The pattern of connectivity between the amygdala and prefrontal cortex²¹ is of particular interest given the latter’s role in cognitive functions. In addition to substantial connections between the amygdala and both medial and orbital aspects of the prefrontal cortex, recent findings indicate that the interconnection between the amygdala and lateral prefrontal cortex extends throughout the lateral surface²². Considered together, the connectivity of the amygdala reveals a substrate for diverse cognitive-emotional interactions that involves the main sectors of the prefrontal cortex – though the anatomical connectivity strength is markedly weaker in the case of the lateral prefrontal cortex.

A further aspect of amygdala connectivity relates to the visual cortex, an aspect that is critical in understanding how amygdala signals modulate visual processing according to an item’s affective significance. Information from visual cortex reaches the amygdala from regions in the anterior ventral visual system, specifically, responses in inferior temporal cortex are conveyed to the lateral and accessory basal nuclei²³. In contrast, efferent projections from the amygdala are organized in a completely distinct manner and connect the basal nucleus of the amygdala with nearly all levels of the ventral visual pathway, including primary visual cortex²⁴. Projections from the amygdala to visual cortex terminate preferentially in cortical layers I-II and V-VI (i.e., not in layer IV), a pattern that is typical of feedback-type connections²⁴ (e.g., from V2 to V1). Typically, these connections are unable to drive neuronal activity²⁵ (i.e., independently generate spiking outputs) but have the ability to modulate information processing by enhancing (or decreasing) neural responses²⁶.

I have discussed patterns of amygdala connectivity without closely examining the different components of the amygdala complex. Yet, the connectivity pattern of the central nucleus is quite distinct from the one observed for regions such as the anterior basolateral and lateral nuclei of the amygdala. The latter have been suggested to be part of a frontotemporal association system, in contrast to the central nucleus, which is more directly linked to autonomic structures²⁷. More generally, when discussing the functions of the amygdala, it is thus important to consider how distinct subregions of this structure are anatomically connected.

Prefrontal monitoring and control of visceral and other bodily functions

The idea that the prefrontal cortex is involved in the control of the autonomic nervous system is not new, dating to the turn of the twentieth century (see the historical account by Neafsey²⁸). More recently, the tight interrelation between prefrontal cortex and bodily functions was refined by the work of Damasio, Bechara, and colleagues on the somatic marker hypothesis (i.e., the idea that bodily states function as “marker” signals that influence reasoning and decision making), especially with respect to the orbitofrontal and ventromedial prefrontal cortices²⁹. Likewise, the notion that the anterior insula, a region that is here discussed in conjunction with prefrontal sites, is involved in complex bodily representations has gained visibility^30-32.

Cingulate cortex

The functions of the cingulate cortex, which may comprise more than 30-40 sub-areas, are complex³³. The anterior sector of the cingulate gyrus is involved in a broad array of functions, including willed action, executive functions, and emotion. A remarkable property of this cortical tissue is that it probably has a more extensive descending projection system than any other cortical region³⁴, including major projections to autonomic regulatory structures, notably the lateral hypothalamus, periaqueductal gray, parabrachial nucleus, and the nucleus of the solitary tract³⁵. This connectivity is consistent with stimulation studies that have documented effects of cingulate electrical stimulation on virtually all autonomic and many endocrine functions³³. Conversely, a range of brainstem projections influence cingulate responses³⁶. These include projections from the locus coeruleus to sites throughout the cingulate cortex, as well as from the nucleus of the solitary tract. Several nociceptive circuits also reach anterior- and mid-cingulate areas indirectly via thalamic nuclei. These findings therefore emphasize the notion that the cingulate gyrus is involved in the bidirectional integration of body-related signals – this is true not only for more anterior regions, but also for the posterior cingulate cortex. Given the well-described roles of the cingulate cortex in cognitive functions, this arrangement provides exceptional opportunities for cognitive-emotional interaction and integration.

Orbitofrontal cortex

Based on its connectivity pattern, the orbitofrontal cortex can be divided into “orbital” and “medial” subcomponents³⁷. The orbital network receives extensive sensory information and appears to integrate it, particularly in relation to the assessment of food and reward. The medial network exhibits a distinctive connectivity pattern, and is heavily connected with areas of the medial wall of the brain, including those surrounding the cingulate gyrus, as well as Brodmann areas 9 and 10 medially. Again in contrast to the orbital network, the medial network receives few sensory inputs (with the exception of auditory association areas). Importantly, it projects to the hypothalamus and other visceral-control areas, leading to the suggestion that it is involved in “visceral modulation of emotion”³⁸. Via the hypothalamus, descending medial orbitofrontal influence appears to extend as far as spinal autonomic centers³⁹. In contrast, there are relatively few projections to the hypothalamus from the orbital network.

Anterior insula

The anterior insula is another structure that is critically involved in the processing of bodily signals as it contains a visceral sensory cortex that maps the internal state of the body in a precise fashion^{31, 32}. It has been suggested³¹ that the anterior insula is more involved in the “afferent representation of ‘feelings’ from the body” (including representation of sensations such as temperature, pain, and visceral ones; see also ref.³⁰), and the cingulate, for instance, is instead involved in the initiation of behaviors (thus more “motor” in function).

More generally, when considering the connectivity of the prefrontal cortex, more differentiated (in terms of laminar structure) regions appear to have restricted connections, whereas the least-differentiated regions have widespread intrinsic connections⁴⁰. For example, the highly differentiated area 8 on the lateral surface has connections that are more likely to target neighboring regions on the lateral surface of the hemisphere. In contrast, both orbital and medial non-isocortical areas (i.e., areas with poor lamination structure, such as a conjoined layer II/III and/or layer V/VI⁴¹) have extensive connections that span the orbital, medial, and lateral surfaces of the hemisphere. Thus, it has been suggested⁴⁰ that, on the one hand, the widespread connectivity of the less differentiated regions is consistent with a more “global role” in neural processing; on the other hand, the more differentiated regions may have more specific roles in information processing.

Summary on anatomy

Historically, subcortical structures such as the hypothalamus and the amygdala have been implicated in emotion. It is becoming increasingly clear, however, that their connectivity affords them with great potential to interact with many other cortical and subcortical structures that are involved in cognitive functions. As noted in the particularly prescient words by Amaral and Price in the context of the amygdala²¹: “As our knowledge of the connections of the amygdala has expanded, it has become apparent that the earlier view that it is primarily involved in the control of visceral and autonomic function is incomplete… These widespread interconnections with diverse parts of the brain simply do not fit with a narrow functional role for the amygdaloid complex. They support, rather, the behavioral and clinical observations which suggest that the amygdaloid complex should be included among the structures which are responsible for the elaboration of higher cognitive functions” (pp. 492-493).

The understanding of the anatomy of the prefrontal cortex has also evolved considerably. As described, large sectors of the prefrontal cortex are strongly interconnected with brainstem nuclei that are responsible for controlling autonomic and endocrine function in the service of supporting survival and bodily integrity via homeostasis. The prefrontal and related sectors comprising the cingulate, orbitofrontal, and insula cortices are also strongly interconnected. In addition, they are also strongly interconnected with the amygdala. In all, the vertical integration of information, both ascending and descending, is implemented in an extensive manner. Accordingly, in conceptualizing the function of the prefrontal cortex, not only is horizontal communication (e.g., links between parietal and prefrontal cortices) important, but also vertical communication is of paramount relevance. Finally, given that several prefrontal and insular areas contain less differentiated gray matter, their widespread connectivity amplifies the potential for cognitive-emotional interactions.

Functional interactions between emotion and cognition

Having discussed anatomical substrates for communication, I now turn to functional studies that, when combined with anatomical evidence, further illustrate the interactions between emotion and cognition. The examples will focus on interactions between emotion and i) perception and attention; and ii) executive functions (see also refs. ^{20, 30, 42-45}).

Perception and attention

Viewing emotion-laden visual stimuli is linked to heightened and more extensive visual system activation^{46, 47}. For instance, viewing faces with emotional expressions evokes increased responses relative to viewing neutral faces throughout ventral occipitotemporal visual cortex. Visual responses are also stronger when subjects view emotional scenes (e.g., a war scene) compared do neutral scenes (e.g., a lake scene). Increased visual activation is observed in both “late” visual areas, such as the fusiform gyrus and superior temporal sulcus, and “early” visual cortex in the occipital lobe. Recent studies have shown that, in humans, even retinotopically organized visual cortex, including visual areas V1 and V2 along the calcarine fissure, are modulated by the affective significance of a stimulus^{48, 49}.

Enhanced visual activation when viewing emotional stimuli is consistent with observed improvements in behavioral performance across several tasks. For instance, there is some evidence that angry and happy faces are detected faster in visual search tasks⁵⁰, and possibly other emotional stimuli, too, such as a snake or spider⁵¹ (but see ref.⁵²). Stronger evidence comes from studies of the attentional blink paradigm, in which subjects are asked to report the occurrence of two targets (T1 and T2) among a rapid stream of visual stimuli. When T2 follows T1 by a brief delay, participants are more likely to miss it, as if they had blinked (hence the name). The attentional blink, which is believed to reflect a capacity-limited processing stage, has been shown to be modulated by emotional stimuli, as subjects are significantly better at detecting T2 when it is, for instance, an emotion-laden word (e.g., “rape”) than when it is a neutral word⁵³.

Converging evidence for a link between perception, attention, and emotion comes from additional studies. For example, patients who present with unilateral inattention due to spatial hemineglect (often as a result of right hemisphere parietal lesions) are better at detecting happy or angry faces compared to neutral ones⁵⁴. These findings are consistent with the notion that emotional faces may direct the allocation of attention. For instance, in one study, emotional faces were flashed at spatial locations that subsequently displayed low-contrast visual stimuli⁵⁵. Subjects exhibited improved performance for detecting targets shown at those locations, suggesting that attention was deployed to them, thereby facilitating visual detection (see also ref.⁴⁸).

What are the mechanisms subserving the increase in perceptual processing and attentional capture that are observed during the perception of affective stimuli? Some evidence links the amygdala with these effects. For instance, patients with amygdala lesions do not exhibit improved detection of T2 emotional targets during the attentional blink (i.e., do not show a decrease in the magnitude of the blink)⁵⁶, and may not exhibit increased responses in visual cortex during the viewing of fearful faces⁵⁷ (but see ref.⁵⁸ for evidence that the amygdala is not required for at least some effects). Consistent with the involvement of the amygdala, in a recent study of the attentional blink, we observed that trial-by-trial fluctuations of responses in the amygdala were predictive of behavioral performance in the task – the greater the evoked response, the higher the likelihood that the subject would correctly detect an emotional T2 stimulus⁵⁹. Thus, it appears that the amygdala may underlie a form of emotional modulation of information that in many ways parallels attentional effects that are observed with non-emotional information^{47, 60} – the latter is thought to depend on fronto-parietal regions. As discussed in the previous section, given that the amygdala sends projections across nearly all levels of the visual system, it is well situated to modulate sensory processing according to the affective significance of a visual object (see also next section).

Is the perception of emotion-laden stimuli “automatic”, namely independent of attention and awareness? This question has received considerable attention because specific answers (“no” or “yes”) suggest potentially different relationships between emotion and cognition (more or less independence between the two, respectively). Evidence both for and against automaticity has been presented. For instance, emotional faces evoke responses in the amygdala when attention is diverted to other stimuli^{61, 62}. Perhaps even more strikingly, amygdala responses are sometimes observed for emotional faces of which subjects are presumably not conscious^63-65. Furthermore, cases of so-called affective blindsight have been reported⁶⁶. These and other related findings suggest that at least some types of emotional perception occur outside of “cognitive” processing. Other findings have suggested, however, that the perception of emotion-laden items requires attention, as revealed by attentional manipulations that were designed to more strongly consume processing resources, leaving relatively few for the processing of unattended emotional items^67-73. It also appears that amygdala responses evoked by “unaware” stimuli depend on the manner by which awareness is operationally defined⁷⁴, such that unaware responses are not observed when awareness is defined, for instance, via signal detection theory methods⁷⁵. Overall, the automaticity debate remains unresolved and controversial^{47, 76-79}.

Executive functions

The impact of emotion on cognition is rich and varied and has been documented in a range of tasks. This section will briefly illustrate interactions involving two executive functions. The first examples come from an important dimension of cognitive function that includes inhibiting and controlling behavior. Response inhibition, namely the processes required to cancel an intended action, is believed to involve control regions in medial and lateral prefrontal cortex, including pre-supplementary motor cortex and inferior frontal gyrus^80-82. Response inhibition is at times investigated by using so-called go/no-go tasks in which subjects are asked to execute a motor response when shown the “go” stimulus (e.g., “press a key as fast as possible when you see a letter stimulus”), but to withhold the response when shown the “no-go” stimulus (e.g., “do not respond when you see the letter Y”). Typically, the go and no-go stimuli are shown as part of a rapid stream of stimuli (e.g., a sequence of letters). A recent study investigated the interaction between the processing of emotional words and response inhibition⁸³. Response inhibition following negative words (e.g., “worthless”) engaged the dorsolateral prefrontal cortex (although behavioral effects of emotional content were modest, further evidence indicates that response inhibition behavior is affected by stronger emotional stimuli⁸⁴). Interestingly, this region was not recruited by negative valence or inhibitory task demands per se; instead, the dorsolateral prefrontal cortex was sensitive to the interaction between behavioral inhibition and the processing of negatively valenced words, namely a cognitive-emotional interaction.

Working memory, another important cognitive function, involves the maintenance and updating of information in mind when the information is no longer available to sensory systems. Evidence for cognitive-emotional interaction comes from working memory studies, too. For instance, when participants were asked to keep in mind neutral or emotional pictures, maintenance-related activity in dorsolateral prefrontal cortex was modulated by the valence of the picture, with pleasant pictures enhancing activity and unpleasant pictures decreasing activity relative to neutral ones⁸⁵. Interestingly, emotional pictures did not affect dorsolateral responses during a second experimental condition during which participants were not required to keep information in mind, indicating that the modulation of sustained activity by emotional valence was particular to the experimental context requiring active maintenance. In another study, participants watched short videos intended to induce emotional states (e.g., clips from uplifting or sad movies), after which they performed challenging working memory tasks⁸⁶. Lateral prefrontal cortex activity on both hemispheres equally reflected the emotional and working memory task components. In other words, prefrontal activity did not stem from the working memory task alone or by the mood ensuing from the viewing of the video, but resulted from an interaction between emotion and cognition.

In summary, these examples highlight the notion that many of the effects of emotion on cognition are best viewed as interactions between the two such that the resulting processes and signals are neither purely cognitive nor emotional. Instead, the “cognitive” or “emotional” nature of the processes is blurred in a way that highlights the integration of the two domains in the brain.

Dual competition framework

The last two sections described both anatomical and functional evidence for the interaction between emotion and cognition. How do these interactions influence the flow of information processing in the brain^{14, 43, 87, 88}? Several proposals have been advanced in the literature, focusing either on perceptual or cognitive processing. Here, the discussion of the previous sections is extended to further delineate how some of the brain regions discussed may contribute to cognitive-emotional interactions. The presentation refines and extends a conceptual framework described recently⁸⁹. It was suggested that both emotion and motivation signals are integrated with perception and cognition so as to effectively incorporate value into the unfolding of behavior. The proposed framework was called the dual competition model to reflect the suggestion that affective significance influences competition at both the perceptual and executive levels (Fig. 2) – and because the impact is due to both emotion and motivation, although the latter is not discussed here (but see ref.⁹⁰).

Figure 2.

Cognitive-emotional interactions. Affective value interacts with both perceptual and executive processes.

Objects compete for limited perceptual processing capacity and control of behavior^{91, 92}. Because processing capacity is limited, selective attention to one part of the visual field comes at the cost of neglecting other parts. Thus, a popular notion is that there is competition for neural resources^{91, 93}. As described below, to understand the flow of information processing more generally, it is necessary to go beyond the role of perceptual competition, and explicitly incorporate the impact of executive control functions on processing. Behavioral research supports the notion that executive control is not unitary and that different mechanisms may have their own limited processing capacities, or resources^{94, 95}. Neuropsychological research also supports the dissociation of cognitive functions, consistent with the “fractionation” of the central executive^{96, 97}. Yet, ample evidence suggests some unity of executive functions, specifically that certain mechanisms are shared across them^{98, 99}. This capacity sharing has important implications for the understanding of human information processing because it leads to executive competition: subcomponents of executive control are mutually interacting, such that resources devoted to one component will not be available to other functions.

Perceptual competition

Perceptual competition, which takes place in visual cortex, is affected by emotional content. As discussed, the amygdala is well positioned to implement the enhancement of visual activity given that its efferents reach multiple levels of the visual cortex, including primary visual cortex²³. Although the role of the amygdala in the modulation of visual processing is often emphasized in the literature, several other mechanisms likely play important roles, too¹⁰⁰. A second modulatory source may involve the orbitofrontal cortex²⁰ (Fig. 3A), a structure that has important roles in the evaluation of sensory stimuli¹⁰¹. The orbitofrontal cortex is reciprocally interconnected with visual cortex, especially the more anterior portions of the ventral stream^{12, 102} and is thus capable of influencing evoked responses in visual cortex based on affective value.

Figure 3.

Interactions between emotion and perception. (A) Visual processing is suggested to be modulated by affective value via several mechanisms, including those involving projections from the amygdala, basal forebrain, orbitofrontal cortex, and possibly hypothalamus (not shown). (B) Another class of modulatory mechanisms relies on interactions between “evaluative” sites (shown in color) and “control” sites (shown in white), the latter located in fronto-parietal cortex and known to exert top-down influences on visual processing. Dotted lines indicate possibly indirect connections. All locations are approximate, and the positions of the amygdala, basal forebrain, hypothalamus, and pulvinar are shown on the lateral surface for schematic purposes. Abbreviations: A, amygdala; BF, basal forebrain; FEF, frontal eye field; Hyp, hypothalamus; IPS, intraparietal sulcus; OFC, orbitofrontal cortex; Pul, pulvinar.

A third important mechanism involves the basal forebrain (Fig. 3A). The central nucleus of the amygdala has significant projections to several basal forebrain structures, and one mechanism by which the central nucleus influences cortical processing is by engaging magnocellular basal forebrain neurons (see ref.^{103, 104}), whose terminals release acetylcholine onto cortical sensory neurons (GABAergic processes have also been described). Lesions of the basal forebrain have been shown to impair a host of attentional tasks, and together with physiological studies, reveal the importance of the basal forebrain not only for sustained attention, but also for selective aspects of stimulus processing, including the filtering of irrelevant information^{6, 7}.

A final class of modulatory mechanisms relies on the fronto-parietal attentional network (Fig. 3B), including lateral prefrontal cortex, frontal eye field, and parietal cortex, which modulate visual processing according to an item’s behavioral relevance. These regions are believed to be “control sites” that provide the source of top-down attentional signals^{105, 106}. Importantly, both frontal eye field and parietal cortex appear to contain a “priority map”, namely a representation of spatial locations containing information that is rich in terms of salience (e.g., high-contrast stimuli) and/or relevance (e.g., stimuli connected to current goals)^{107, 108}. It is suggested here that the fronto-parietal network works closely with several “evaluative” sites discussed in the first section, such as hypothalamus, amygdala, cingulate cortex, orbitofrontal cortex, and anterior insula, to prioritize processing based on the affective significance of a sensory stimulus (for a related discussion in the case of motivation, see ref.⁹⁰). In some of these cases, the direct connections between “evaluative” and “control” regions may be relatively weak, and indirect routes involving one or more intermediate steps are probably involved.

An additional modulatory role is proposed for the pulvinar complex of the thalamus (Fig. 3B). Based on anatomical and physiological considerations, it was suggested that the importance of the pulvinar for affective processing is not due to its putative role as part of a subcortical pathway, as often assumed in the literature, but instead because of its connectivity with other cortical regions¹⁹. Briefly, the medial nucleus of the pulvinar, which projects to the amygdala, is part of several thalamo-cortical loops that include orbitofrontal, cingulate, and insular cortices (in addition to frontal and parietal sites). Given this broad connectivity pattern, the medial nucleus may be involved in two general functions that directly impact emotional processing: determining behavioral relevance and/or value. Therefore, the role of the pulvinar may extend beyond the well-established roles in attention¹⁰⁹ and contribute to affective processing^110-112.

In summary, during the past decade, an important role for the amygdala in the emotional modulation of vision has been highlighted in the literature. Yet, as described here, the amygdala is but one of the sources of modulation of visual responses that take into consideration the behavioral and affective significance of sensory stimuli. Future research is needed to establish how these multiple modulatory sources influence visual processing in particular, and other sensory modalities more generally.

Executive control and competition

How does emotional content impact executive function? Because emotion can either enhance or impair performance of executive functions, answering this question has been challenging. At least part of the answer may be related to the level of threat posed by an emotional item. When threat content is relatively low, processing is biased in favor of the emotional item and although emotional items are prioritized, the impact on behavior may be modest. Importantly, emotional content enhances task-relevant processing with relatively minor effects on irrelevant stimuli and other executive functions that may be concurrently needed.

A more dramatic effect of emotional content on behavior is expected when the level of threat is high. In this situation, processing resources are diverted toward the processing of the item at hand and because the mobilization of resources is more extreme, the effects on behavior are considerably more dramatic^{113, 114}. In particular, the impact on behavior may come from the recruitment of attentional/effortful control that is required to prioritize the processing of high-threat information. Attentional/effortful control involves processing resources that are shared across executive functions and because high-threat is expected to recruit some of these resources (see also ^{78, 115, 116}), it will impair other executive functions that are reliant on them (Fig. 4). Consistent with this idea, performance during response inhibition was compromised when participants viewed high- vs. low-arousing pictures⁸⁴.

Figure 4.

Executive competition and threat. Executive functions can be viewed as relying on multiple mechanisms, also referred to as resources (R1 through R4; e.g., “shifting”, “updating”), that are partly independent but, critically, are also shared (indicated by the orange circle). When threat content is high, these shared resources (“SR”) are engaged, thus detracting from the ability to recruit specific mechanisms at optimal levels. Consequently, behavioral performance relying on those mechanisms will be impaired.

In the past, the notion of resources has been employed in order to account for the limits of human information processing. A potential approach to understanding resource consumption by threat may be to probe the correspondence of brain sites that are sensitive to specific experimental conditions. It is particularly instructive, for instance, to examine the overlap between manipulations of threat level and those involving attention – given that attentional manipulations are sensitive to changes in the distribution of processing resources. The “attentional network” has been extensively researched and is believed to involve fronto-parietal regions, including the middle frontal gyrus, inferior frontal gyrus, anterior cingulate cortex, and anterior insula^{105, 106}. To assess brain regions that are sensitive to high levels of threat, the activation sites of the contrast of CS+ (i.e., aversively conditioned) vs. CS− (i.e., neutral) of 34 aversive conditioning studies were reviewed. Although great emphasis is put on the involvement of the amygdala in the processing of threat, this summary revealed that several frontal activation sites were consistently reported, including middle frontal gyrus, inferior frontal gyrus, anterior cingulate cortex, and anterior insula⁸⁹. This evaluation thus suggests that processing high-threat items engages key nodes of the attentional network, suggesting that it consumes processing resources.

What are some of the neural substrates of the interactions between emotion and cognition? When items are high in threat, robust interactions between affective processing and executive functions are proposed to take place via several neural mechanisms. First, it is hypothesized that threat processing engages attentional/effortful control mechanisms in several fronto-parietal sites, including lateral prefrontal cortex, anterior cingulate cortex, and parietal cortex. The role of the anterior cingulate cortex may be particularly important because of its role in integrating inputs from multiple sources, including cognitive, affective and motivational inputs¹¹⁷ (Fig. 5). In cognitive studies, the anterior cingulate has been suggested to be involved in conflict detection, error likelihood processing, and error monitoring, among other functions. Anterior cingulate engagement during threat may impair executive function because shared resources required to prioritize threat processing are recruited. In other words, anterior cingulate sites engaged by high-threat are at the intersection of the resources needed for several executive functions (as indicated by the orange region in Fig. 4). Notably, the anterior cingulate engagement includes the dorsal sector, in contrast to the idea that the dorsal anterior cingulate is involved in cognitive function, in opposition to the more rostral, “emotional” sector¹¹⁸.

Figure 5.

Interactions between emotion and cognition. The anterior insula and the anterior cingulate cortex are important sites involved in determining affective significance and value. In so doing, they are closely affiliated with an extended set of regions, some of which are shown here in the orange ellipse. The anterior insula and anterior cingulate cortex interact closely with the lateral prefrontal cortex, a region involved in several cognitive functions. In this manner, cognitive-emotional interactions occur during complex behaviors. Notably, these interactions can be either beneficial or detrimental to behavior. In the latter case, for instance, when threat level is high, resources required for cognitive operations are partly consumed, thereby compromising behavioral performance. Abbreviations: ACC: anterior cingulate cortex; Amyg, amygdala; Ant. Insula, anterior insula; BF, basal forebrain; Hyp, hypothalamus; LPFC, lateral prefrontal cortex.

As discussed, the anterior insula is critical for interoception, which involves monitoring the sensations that are important for the integrity of the internal body state, and interacting with systems that are important for evaluating context, allocating attention, and planning actions¹¹⁹. Threat, uncertainty, and risk are all potent factors that engage the anterior insula¹²⁰. Remarkably, the anterior insula also was found to be activated in most cognitive tasks for which Van Snellenberg and Wager¹²¹ had meta-analytic data. The anterior insula is thus a site that is engaged during both cognitive and emotional contexts (Fig. 5). Accordingly, recruitment of the anterior insula during high-threat conditions will detract from its ability to assist in executive functions; a concomitant impairment in performance is thus expected. Note that this argument assumes that the engagement of the anterior insula during high-threat conditions substantially intersects with cortical territories that are required for cognitive processes (see “SR” in Fig. 4). Naturally, these and other aspects of the dual competition framework need to be validated by experimental data.

A second effect of threat is to trigger specific executive functions to handle on-going challenges to the organism. For instance, “updating” might be needed to refresh the contents of working memory, “shifting” might be recruited to switch the current task set, and “inhibition” could be called to cancel previously planned actions. Again, this recruitment is suggested to depend, at least in part, on the anterior cingulate cortex and the anterior insula – the former is known to influence activity in other brain regions and to modulate cognitive, motor, and visceral responses¹¹⁷. For instance, the anterior cingulate may work in close cooperation with lateral prefrontal cortex (see also ref.¹²²), a region that is important for the manipulation of information, among other functions. In this manner, additional specific processing resources are coordinated in the service of threat processing (Fig. 5). Affective information conveyed by other brain regions, including the hypothalamus, amygdala, basal forebrain, and orbitofrontal cortex is conveyed (possibly indirectly) to lateral prefrontal cortex and parietal sites, too, further engaging executive power in the function of handling the threat to the organism. In finalizing the discussion of the involvement of fronto-parietal regions in interactions between emotion and executive function, note that these are some of the same regions that were implicated as having an important effect on perceptual competition (Fig. 3B), highlighting the interdependence of perceptual and executive processes – in other words, the sharp distinction between bottom-up and top-down in Fig. 2 is artificial.

A third effect of threat on executive functions involves state changes that are implemented via ascending systems^{7, 123}. The basal forebrain, hypothalamus, and reticular formation have the ability to influence both cortical and subcortical processing via widespread projections. In particular, the overall anatomical arrangement of the basal forebrain (here, more broadly construed) might involve multiple functional-anatomical macrosystems^{124, 125} with wide-ranging effects on brain computations and important clinical implications^{6, 14, 124}. More generally, the three structures may be viewed as key components of the “behavioral state system”, which has been suggested to be one of the major functional subsystems of the vertebrate nervous system¹²⁶ (together with “cognitive”, “sensory”, and “motor” systems).

Conclusions

Historically, emotion and cognition have been viewed as largely separate entities. One way in which emotion has been contrasted to cognition has been to link the former with “irrational” or “suboptimal” processes¹²⁷ that are more “basic”, namely more linked to survival, than cognitive ones. Although much has changed in the past two decades, versions of this viewpoint still are quite frequent in the literature (even if, at times, implicitly). Research in the past decades suggests, however, that such view is likely erroneous and that, in order to understand how complex behaviors are carried out in the brain, an understanding of the interactions between the two is indispensable. Interestingly, neuroimaging in humans may have been one factor contributing to the change in this viewpoint. Because neuroimaging techniques afford whole-brain investigations, it has become increasingly evident that large portions of both cortex and subcortex are engaged during emotional information analyses¹²⁸.

In many current formulations of how emotion is organized in the brain, a heavy emphasis is found on “special” regions, most notably, the amygdala. In particular, it could be argued that the amygdala is “primitive” (in the sense of being derived from ancestral form), and that it may be better viewed as tied to fear-related functions and as an effective “alarm system” – one that has been evolutionarily conserved for good reasons. Yet, even in rodents important roles for the amygdala in “cognitive” operations, such as attention and decision making, have been documented^{129, 130}. And in primates, as pointed out by Sander and colleagues, the amygdala may have evolved into a less specialized system in order to cope with new environmental problems¹³¹. One way in which this may have occurred may be related to an expansion of the connectivity of the amygdala with a wider range of cortical territories¹³². This may involve new direct connections, such as the connectivity documented between the amygdala and lateral prefrontal cortex²² and, more extensively, indirect connections via other important cortical hubs, such as those involving the anterior cingulate, orbitofrontal, and insular cortices. Altered and enhanced connectivity may be one way in which a system expands the repertoire of functions it is involved in. Although the evolution of the brain is highly constrained, dramatic changes in the pattern of connectivity have been documented – such as those involving the somatosensory cortex and thalamus in several mammals ^{133, 134}. Furthermore, Whereas mice have about 10 cortical fields, and macaque monkeys have more than 50 fields, humans may have more than a hundred fields¹³⁴. The combinatorial nature of connectivity is such that, in humans, the amygdala, which is extremely highly interconnected, as reviewed here, may be in a position to be an important player in an impressive array of cognitive-emotional functions.

Generally speaking, given the combinatorial connectivity of the brain, it will be important to go beyond simply describing interactions between emotion and cognition, some of which are suggested to be mutually antagonistic¹³⁵. Instead, future advances will be made by the mechanistic description of how cognition and emotion are effectively integrated in the brain. This is especially pertinent in light of the suggestion that in many cases functional specialization is lost, and emotion and cognition conjointly and equally contribute to the control of mental activities and behavior⁸⁶. For instance, the affective dimensions of a visual item are reflected at multiple processing stages, from early visual areas to prefrontal sites¹³⁶. In addition, visual cortical responses reflecting an item’s significance will be a result of simultaneous top-down modulation from fronto-parietal attentional regions and emotional modulation from the amygdala, basal forebrain, orbitofrontal cortex, and other regions. This perspective can also be adopted in the context of executive functioning, such that cognitive and emotional contributions to executive control are difficult to separate. For example, lateral prefrontal cortex signals involved in inhibitory processes may reflect both cognitive variables (e.g., an inhibitory response is required) and affective information (e.g., negative stimuli are viewed before being required to inhibit a response). A key implication of the integration viewpoint is that, in general, it may be simply counterproductive to attempt to separate emotion and cognition. Instead, their interdependence challenges a simple division into separate “cognitive” and “emotional” domains⁸⁸.