Natural selective attention: Orienting and emotion

Abstract

The foundations of orienting and attention are hypothesized to stem from activation of defensive and appetitive motivational systems that evolved to protect and sustain the life of the individual. Motivational activation initiates a cascade of perceptual and motor processes that facilitate the selection of appropriate behavior. Among these are detection of significance, indexed by a late centro-parietal positivity in the event-related potential, enhanced perceptual processing, indexed by a initial cardiac deceleration, and preparation for action, indexed by electrodermal changes. Data exploring the role of stimulus novelty and significance in orienting are presented that indicate different components of the orienting response habituate at different rates. Taken together, it is suggested that orienting is mediated by activation of fundamental motivational systems that have evolved to support survival.

Attention and emotion are often conceptualized as disparate processes, perhaps even opposite in nature, with attention considered rational, dispassionate, and “cold” and emotion as irrational, passionate, and “hot.” Attention is typically used to characterize activities of the head and the mind, whereas emotion is the provenance of the heart and the body. And, although attention and emotion have been the focus of many investigations in the history of psychophysiology, their databases remain aloof, seldom speaking to one another. Here, an integration is considered that suggests that many of the physiological responses held to be indices of orienting and attention are mediated by neurally organized defensive and appetitive motivational systems that have evolved to protect and sustain the life of the individual. Activation of motivational systems initiates a cascade of sensory and motor processes, including mobilization of resources, enhanced perceptual processing, and preparation for action, that have evolved to assist in selecting appropriate survival behaviors. It is proposed here that orienting engages these same motivational circuits, which is why there is so much attention in emotion and why attention and emotion are inextricably linked.

The breadth of the theoretical and empirical research devoted to the psychophysiological study of orienting, attention, and emotion clearly cannot be embraced here. Rather, it will be necessary to skate somewhat on the surface of each of these areas, highlighting communalities among the bodily processes modulated when people process “novel” events—that is, those stimuli that have provided the foundation for studies of orienting and attention, and when people process “significant” events—those stimuli that are the focus in studies of emotion. To begin, I will focus on classical studies of orienting and attention, a field initiated and carried forward by an impressive group of investigators, including Pavlov (1927), Sokolov (1963), Graham (1979), Näätänen (1979), Donchin (1981; Donchin, Ritter, & McCallum, 1978), Rohrbaugh (1984), Bernstein (1979), Maltzman (1979), Öhman (1979), Siddle (e.g., Siddle & Spinks, 1979), Roth (1983), Barry (1987), and many others.

Following a discussion of the role of stimulus significance in orienting, I review data from a number of studies in our laboratories that assess psychophysiological responses as they vary with both stimulus novelty and significance. Based on these data, it will be suggested that orienting is best conceptualized as a set of functional responses, with different rates of habituation, that index specific central, perceptual, and motor processes. The foundations of orienting in motivation are explored, and it will be concluded that orienting in emotional, novel, and task-relevant contexts reflects engagement of motivational systems that have evolved to support perception and action.

Novelty and Significance

Stimulus Novelty

The story begins with an often-told tale in which Pavlov’s attempt to demonstrate conditioning to visiting colleagues was thwarted by the experimental animal’s failure to “attend to” the relevant conditioned stimulus, instead directing attention to the novel visitor. Pavlov (1927) defined this “investigatory reaction” as a set of behavioral changes in which the animal’s receptor organs (ears, eyes, etc.) were oriented toward the novel stimulus. Later called the “novelty reflex,” the “exploratory reflex,” or the “orienting reflex” (or reaction), it is often more colloquially labeled the “What is it” response. Whereas Pavlov focused primarily on behavioral manifestations of the orienting reflex, Sokolov (1963) later noted that novel stimulation induced a variety of physiological changes in the body that he collectively called the orienting response (OR). According to Sokolov, at any instant, a mental (or “neuronal,” using his term) model was active that represented events and objects in the current environment. Novel stimuli—those that did not match elements in the current mental model—elicited an orienting response.

Classical tests of orienting therefore often employed a repetition– change paradigm in which one stimulus, usually a tone of low to moderate intensity, was repeatedly presented. At certain intervals, a different tone was presented, and orienting responses to this novel stimulus were measured, including changes in vascular, electrodermal, pupillary, central, and, later, cardiac responses. Whereas Sokolov (1963) examined vasomotor changes, as well as changes in alpha blocking of the electroencephalogram (EEG), the bulk of early experimentation focused on skin conductance changes. Among the most important defining features of the OR that resulted from this work were that orienting responses were elicited in the context of stimulus change, and that orienting habituated with stimulus repetition.

Stimulus Significance

In the late 1960s and 1970s, psychophysiologists began to discuss whether stimulus change alone determined orienting responses, a debate that is documented in an interesting and influential set of articles published in Psychophysiology (Bernstein, 1979; Maltzman, 1979; O’Gorman, 1979). In an article entitled “To What Does the Orienting Response Respond?”, Bernstein (1969) questioned whether stimulus change was necessary in eliciting the orienting response, basing his speculations on data indicating that some stimulus changes did not elicit orienting and that some individuals did not show orienting, despite reports of detecting stimulus change. In addition, it was clear that “set”—defined as instructions or task-relevance—affected both the magnitude of orienting and the rate of habituation. Apparently, not all stimulus changes are equal, with the magnitude (and probability) of electrodermal responding affected not only by novelty, but also by variables related to “significance.” Although Sokolov (1963) had noted that stimuli that require a response (“signal” stimuli) elicit ORs, assimilating these set effects into a neuronal mismatch model poses significant problems. Nonetheless, many of the Russian investigators routinely included stimulus significance or “relevance” in the list of variables that elicit or modulate the OR, together with a number of factors more clearly related to stimulus novelty, including surprise, unexpectedness, and unfamiliarity.

Among the responses to the issue of stimulus significance were a general call for a satisfactory operational definition of “significance,” independent of the magnitude of the OR, as well as for a neuronal mechanism that was as clearly stated as Sokolov’s mismatch model for novelty. In terms of defining stimulus significance, Maltzman (1979) noted that “the problem is similar to the one Thorndike faced: how to define satisfiers and annoyers. His solution was to define rein forcers in terms of approach and avoidance behavior. Both kinds of situations are significant” (p. 277, emphasis added). Based on this definition, Maltzman proposed that “significance” could be operationally defined on the basis of the semantic differential (Osgood, Suci, & Tannenbaum, 1957), a series of judgments that can be made about stimuli as diverse as language and sonar signals (Mehrabian & Russell, 1974). When submitted to factor analysis, these semantic judgments consistently identify two dimensions of pleasure and arousal as the primary factors that organize human experience (as originally noted by Wundt, 1896). And, although Maltzman suggested that the neural substrate mediating significance may be hemispheric specialization, Bernstein (1979), responding to the same critiques, proposed instead that cortical-limbic interactions are important in mediating orienting responses to significant stimuli.

A Motivational Model of Emotion

Defining “stimulus significance” in terms of pleasure and arousal provides the first link between emotion and orienting. We have suggested that emotion is fundamentally organized around two motivational systems, one defensive and one appetitive, that have evolved to mediate transactions in the environment that either threaten or promote survival of the individual and the species (Bradley, 2000; Bradley, Codispoti, Cuthbert, & Lang, 2001; Lang, Bradley, & Cuthbert, 1997). These systems are implemented by limbic circuits in the brain that are old (phylogenetically) and broadly similar across the mammalian species (see Davis, 2000; Lang & Davis, 2006; Fanselow, 1994; LeDoux, 1990).

Emotion is considered fundamentally a disposition to act (Frijda, 1986, 1987; Lang, 1979, 1985)—to respond effectively to events that threaten or sustain life. The defense system is activated in contexts involving threat, with a behavioral repertoire built on withdrawal, escape, and attack. Conversely, the appetitive system is activated in contexts that promote survival, including sustenance, procreation, and nurturance, with a basic behavioral repertoire of ingestion, copulation, and caregiving. The primacy of pleasure and arousal in organizing human experience derives from the fact that these variables serve as parameters of motivational activation, with judgments of pleasure indexing which motivational system is engaged (appetitive, defensive) and judgments of arousal indexing the intensity of its activation (Bradley & Lang, 2007).

Stimulus Significance: Pleasure and Arousal

When pictures that illustrate a variety of the objects and events that constitute human experience are rated in terms of pleasure and arousal, a two-dimensional space such as that illustrated in Figure 1 results (e.g., Lang, Greenwald, Bradley, & Hamm, 1993). This distribution is consistent with the idea that judgments of pleasure and arousal reflect varying activation in two underlying appetitive and defensive motivational systems. When activation in either system is minimal, rated arousal is low and events are usually labeled “unemotional” or “neutral.” From a motivational perspective, this suggests a weak action tendency and little energy mobilization. As defensive or appetitive motivation increases, however, ratings of arousal increase as well, indexing the metabolic requirements of current and anticipated action.

Figure 1.

The distribution of pictures in a two-dimensional space defined by ratings of pleasure and arousal are consistent with the hypothesis that evaluative judgments reflect the level of activation in underlying appetitive or defensive motivational systems.

Orienting in Picture Viewing: Novelty and Significance

Defining stimulus “significance” in terms of pleasure and arousal allows an empirical evaluation of the effects of stimulus novelty and stimulus significance on the orienting response. In the studies reviewed here, traditional physiological indices of attention and orienting, including skin conductance change, cardiac deceleration, and event-related potentials (ERPs), are measured as participants view significant (arousing pleasant and unpleasant) pictures that are novel or repeated. Among the questions that will be evaluated are the following: (a) Does orienting occur for all novel stimuli? (b) Does stimulus significance affect the magnitude of orienting? (c) Does orienting habituate with stimulus repetition?

In the picture viewing paradigm, all stimuli can be “novel” (in the Sokolovian sense) if specific pictures vary from trial to trial and have not been seen or presented before. Thus, if orienting is engaged by novel stimulation, ORs are expected for all pictures during their initial presentation—even those that are not significant. Moreover, in a passive viewing context, all stimuli can be considered “non signal” in the sense that no task or instructional set targets specific pictures as differentially relevant or significant. If stimulus significance naturally modulates orienting, its magnitude should be heightened when viewing unpleasant and pleasant, compared to neutral, pictures. Finally, if orienting is a unitary response, as some have maintained, orienting should be apparent (or not) in all of the measured systems and should diminish similarly with picture repetition.

Electrodermal Activity

Skin conductance change is a classic component of the orienting response and, as detailed elsewhere (e.g., Critchley, 2002; Venables & Christie, 1980), is a sympathetically mediated response that has been associated with physiological arousal. In general, skin conductance change is heightened for novel stimuli in the repetition–change paradigm and reliably decreases with stimulus repetition. Figure 2 (left panel) illustrates skin conductance responses when participants view novel pleasant, neutral, and unpleasant pictures. Consistent with its interpretation as an orienting response, all novel pictures prompt measurable skin conductance change, as even neutral pictures elicit significant skin conductance changes (i.e., from zero) during initial viewing. On the other hand, electrodermal responses are significantly larger when viewing novel pleasant or unpleasant, compared to neutral, pictures, indicating heightened orienting as a function of stimulus significance.

Figure 2.

Skin conductance changes when viewing novel pictures (left panel), repeated pictures (middle panel), and novel pictures presented for a brief duration (right panel) are larger for emotional (pleasant or unpleasant) compared to neutral pictures.

When the same pictures are presented in a second session (conducted 1 week later), the pattern of affective modulation is the same: Both pleasant and unpleasant pictures continue to elicit larger orienting responses than neutral pictures, as illustrated in Figure 2 (middle panel). For neutral pictures, however, repetition eliminates significant changes in electrodermal activity, suggesting that novelty is the primary factor prompting orienting to neutral pictures during initial processing. When affective pictures are repeatedly presented within the same session, measurable skin conductance changes eventually diminish (Bradley, Lang, & Cuthbert, 1993), although at a slower rate of habituation than for neutral pictures.

These findings indicate that both novelty and significance affect skin conductance changes during picture viewing. Whereas a single repetition a week later eliminates this component of the orienting response for neutral pictures, significant pleasant and unpleasant pictures continued to elicit measurable orienting. Moreover, it is instructive that affective modulation of skin conductance is found even when sensory information is available for only a brief period. Figure 2 (right panel) illustrates that skin conductance changes are larger for novel emotional, compared to neutral, pictures when pictures are presented for 500 ms and then removed from the screen (Codispoti, Bradley, Cuthbert, & Lang, 2001), and more recent data indicate clear electrodermal discrimination between emotional and neutral pictures when the sensory information is only available for 80 ms (Codispoti, Mazzetti, & Bradley, in press). Thus, because the conductance component of the orienting response does not depend on the sustained presence of a sensory foreground, it is probably not related to perceptual processing subsequent to initial attention capture. One interpretation is that this component of the orienting response reflects action preparation, which is consistent with the role of the sympathetic system in arousing and engaging behavior in response to salient stimulation, including both novel and motivationally significant events.

Heart Rate Change

In a series of studies in the late 1960s, both Lacey (1967) and Graham & Clifton (1966) reported prolonged cardiac deceleration during perception, which was subsequently found to be predominantly parasympathetically mediated, with increased vagal control resulting in decreased cardiac rate (Campbell, Wood, & McBride, 1997). Lacey (1967; Lacey & Lacey, 1970) noted that HR deceleration occurred in contexts involving perception (“environmental detection”), whereas HR acceleration was more pronounced in contexts that emphasized mentation (“environmental rejection”). Noting similarities between these differences and Sokolov’s distinction between orienting and defense responses, Graham (1979; Graham & Clifton, 1966) suggested that heart rate deceleraton indexes sensory intake (orienting) whereas cardiac acceleration indexes sensory rejection (defense). According to this scheme, perceptual processing involves an openness to sensory information that is indexed by cardiac deceleration.

Consistent with this, when novel pictures are viewed for the first time, a relatively prolonged cardiac deceleration is apparent during the initial 2 s of presentation for all picture contents, including those that are neutral, as illustrated in Figure 3 (left panel). Cardiac deceleration, however, is significantly larger for unpleasant pictures, compared to those that are either pleasant or neutral in content. In fact, the same pattern of heightened cardiac orienting for unpleasant stimuli was previously reported by Hare, Wood, Britain, and Shadman (1970), Libby, Lacey, and Lacey (1973), and Winton, Putnam, and Krauss (1984). In most instances, this pattern of results was somewhat surprising, as unpleasant pictures were initially hypothesized to prompt a defensive reflex, which predicted cardiac acceleration. Moreover, rather than evidence of rejecting unpleasant stimuli, the increased cardiac deceleration suggests enhanced sensory intake, with a pronounced perceptual focus for these aversive cues.

Figure 3.

Heart rate changes when subjects view novel pictures (left panel), repeated pictures (middle panel), and novel pictures presented briefly (right panel) show an initial deceleration that is enhanced when subjects view novel unpleasant pictures and that is absent when pictures are either repeated or presented for a brief duration.

As Figure 3 (middle panel) illustrates, however, repetition of the same pictures a week later results in a quite different pattern of heart rate change, with no evidence of prolonged cardiac orienting for any of the pictures and no differences in heart rate as a function of stimulus significance. Similarly, when pictures are repeatedly presented within the same session, (Bradley et al., 1993), differential cardiac orienting as a function of affective content disappears. Rather, following repetition, picture viewing elicits a brief, small deceleration at picture onset that is not affected by stimulus significance.

Graham (1987) labeled this brief decelerative response a “transient detecting response” (TDR) and did not consider it to be a component of the cardiac orienting response. She instead proposed it was a reflexive index of “stimulus registration,” occurring following the presentation of any low- to moderate intensity stimulus, regardless of novelty, repetition, or significance. Much of the discussion in the literature regarding whether cardiac orienting habituates (e.g., Barry& Maltzman, 1985)may simply reflect confusion between this brief decelerative response and the sustained decelerative response that is indicative of orienting: Because novel stimuli in the repetition–change paradigm are typically repeated within the session, sustained cardiac orienting may diminish, leaving only a TDR, as in the current data.

In terms of cardiac changes indicative of orienting, however, all novel pictures elicit significant deceleration, consistent with the view that an initial prolonged deceleration is a component of the orienting response to novel stimulation. Unlike skin conductance change, however, larger cardiac deceleration, indicative of enhanced orienting, is primarily evident for aversive cues. This result is reminiscent of the “fear bradycardia” that is apparent in most mammals, and even reptiles, in which heart rate slows when the organism is initially confronted with a threatening cue (Campbell et al., 1997) and suggests that this component of the orienting response is reflexively enhanced primarily following defensive activation. Moreover, picture repetition, either within- or between-session, not only attenuates this cardiac component, but also eliminates differences as a function of motivational significance. Importantly, cardiac orienting was absent in the same individuals (see Figure 3, middle panel) for whom orienting was still apparent when measured by skin conductance changes (see Figure 2, middle panel), indicating that these components are differentially affected by stimulus repetition.

Also unlike skin conductance change, modulation of the cardiac component of the orienting response is highly dependent on the presence of a sensory foreground. As illustrated in Figure 3 (right panel), when novel pictures are only presented for 500 ms, a brief TDR is apparent in the cardiac waveform for all pictures, but prolonged cardiac deceleration disappears when picture presentation is terminated. Moreover, heart rate change does not reflect differences as a function of stimulus significance when the picture is presented briefly and then removed from view. Taken together, these cardiac data support Graham’s (1979) hypothesis that cardiac deceleration is related to sensory intake, reflecting processes involved in extracting information from the sensory array. Heightened perceptual processing as a function of stimulus significance, however, is most apparent during defensive activation, and this perceptually focused index of orienting habituates rapidly.

Event-Related Potentials

A number of different ERPs appear to be sensitive to different aspects of stimulus novelty, including an early mismatch negativity (Näätänen, 1979) as well as later components including N2b and P3a (e.g., Courchesne, Hillyard, & Galambos, 1975; Rohrbaugh, 1984). A number of ERP components are also held to vary with stimulus significance, particularly when defined by task-relevance, including P300 (P3b), processing negativity, and a late positive complex (Ruchkin & Sutton, 1983; see the interesting discussion of P300 and orienting in Donchin et al., 1984). Early on, an ERP that showed enhanced positivity over central-parietal sensors (and often negativity over frontal sensors) that accompanied or followed the P300 was proposed as an index of orienting and subsequently called the “orienting” wave or “O-wave” (Connor & Lang, 1969; Loveless & Sanford, 1974; Rohrbaugh, 1984; Rohrbaugh & Gaillard, 1983; Weerts & Lang, 1973).

A similar slow potential that shows enhanced positivity over centro-parietal sensors (and often negativity over frontal sensors) is the most reliable component modulated by stimulus significance in the passive picture viewing context and is often called the “late positive potential” (LPP; Cacioppo, Crites, Gardner, & Berntson, 1994). As illustrated in Figure 4 (left panel), during initial presentation, significant pleasant or unpleasant pictures, compared to neutral cues, elicit an enhanced positivity over centro-parietal sensors in a window beginning around 300–400 ms after picture onset. This modulatory effect is quite reliable and has been reported in numerous studies (e.g., Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Keil et al., 2002; Palomba, Angrilli, & Mini, 1997), with the largest modulatory differences elicited when viewing the most arousing pleasant or unpleasant pictures (Schupp et al., 2004).

Figure 4.

Event-related potentials measured over centro-parietal sensors show enhanced positivity when viewing emotional (pleasant or unpleasant), compared to neutral, pictures regardless of whether pictures are novel (left panel), repeated (middle panel), or briefly presented (right panel).

When the same pictures are repeated within the same session, there is little change its modulation by emotion, as illustrated in Figure 4 (middle panel). Even when pictures are presented up to 90 times in the same session (Codispoti, Ferrari, & Bradley, 2007), significant affective pictures continue to prompt a larger late positive potential, with a pronounced centro-parietal distribution, compared to neutral pictures, as illustrated in Figure 5. Taken together, although the amplitude of the late positive potential diminishes somewhat with picture repetition, its modulation by stimulus significance is fairly resistant to repetition, showing significant differences even after multiple repetitions of the same picture within a single session.

Figure 5.

The magnitude of a centro-parietal late positive potential is enhanced when subjects view emotional (either pleasant or unpleasant) compared to neutral pictures, despite multiple within-session repetition. Based on data from Codispoti, Ferrari, and Bradley (2007).

Interestingly, like skin conductance, modulation of the late positive potential does not rely on the sustained presence of a sensory foreground. When pictures are presented briefly (i.e., 25 ms) and not masked by a subsequent visual stimulus, a robust difference in the late positive potential is found between significant emotional and neutral contents, as illustrated in Figure 4 (right panel; Codispoti et al., in press). Moreover, affective modulation of the late positive potential is observed regardless of perceptual differences in complexity, brightness, contrast, or spatial frequency: When simple figure-ground compositions and more complex scenes were presented, complexity prompted an early ERP difference, maximal over occipital sensors, that was not affected by stimulus significance (see Figure 6; Bradley, Hamby, Loew, & Lang, 2007). The late positive potential, on the other hand, was reliably enhanced for emotional, compared to neutral, pictures, regardless of picture composition. Taken together, whereas perceptual differences modulate earlier ERPs for sensors over visual cortex, modulation by stimulus significance is most reliable for a late slow potential whose centro-parietal scalp distribution is consistent with that often attributed to the O-wave. This late positive potential is fairly resistant to habituation.

Figure 6.

Event-related potentials measured when subjects view simple figure-ground compositions, compared to more complex scenes, show a relative negative deflection beginning around 150–250 ms following picture onset that is modulated by perceptual complexity, but not emotion. Data from Bradley et al. (2007).

Summary: Orienting in Picture Viewing

Returning to the three basic questions regarding novelty and significance, some preliminary answers are possible:

1. Can skin conductance increase, heart rate deceleration, and late positive potential be considered indices of orienting? Yes, these indices are typically apparent when novel pictures are presented during passive viewing, regardless of stimulus significance.

2. Does stimulus significance affect orienting? Yes, significant stimuli modulate the magnitude of orienting responses, with both pleasant and unpleasant pictures prompting heightened skin conductance changes and larger late positive potentials. Enhanced cardiac deceleration, on the other hand, is most pronounced when viewing novel aversive stimuli.

3. Does habituation occur with repetition? Not necessarily. Rather, different components of the orienting response drop out at different rates. Sustained cardiac deceleration is eliminated the most rapidly, disappearing even after a single picture repetition a week later. Skin conductance, on the other hand, shows slower habituation, with significant differences between affective and neutral pictures still apparent following a single repetition a week later, but reduced and then eliminated following multiple within-session repetitions. Compared to these peripheral responses, modulation of the late positive potential is the most resistant to habituation, showing effects of significance even following multiple repetitions.

As these different patterns of habituation suggest, it is unlikely that orienting should be considered a unitary process. Although some have maintained that a key feature of the orienting response is that all of its components co-occur and show the same rate of habituation, in fact, as Graham (1987) notes, “Sokolov did not usually employ the term ‘unitary’ OR. Rather, he emphasized that the OR was a functional system which consisted of a ‘complex combination’ of responses having in common response to …change in stimulation” (p. 223). Barry (1987), in his preliminary process theory, in fact, has linked different components of the orienting response to different processes. Consistent with this type of approach, the data reviewed here not only suggest that different components of the orienting response reflect different facets of orienting, but that they are also differentially modulated by stimulus significance and repetition.

Understanding Orienting

The Function of Orienting

Understanding the components of the orienting response and their rate of habituation requires reconsidering the biological function of orienting to the individual. As Biriukov (1965), among others, emphasized, “Pavlov stresses… [that] the OR is necessary in terms of the organism’s survival. If it were absent, the life of the animal would be in constant danger” (p. 23, emphasis added). Cognitive psychophysiologists also appeal to survival when discussing the presence of ORs obtained in rather neutral experimental tasks: “It must be recalled… that the S is always performing a task of paramount importance—namely, the monitoring of the environment for significant events.… When a very rare event occurs, it may well be relevant to the important task of ensuring survival” (Donchin, 1984, p. 384, emphasis added).

Throughout the orienting literature, this emphasis on the OR as a process that has evolved to support survival, as critical for survival, and which somehow assists the organism in selecting responses that facilitate survival is a consistent, resonating drum-beat. In the animal laboratory, in which novel stimulation almost always consists of threatening or appetitive cues, orienting clearly retains its link to motivation. For humans in the context of a repetition/change tone paradigm, however, it has become more difficult to appreciate the survival function of the orienting response, as the novel stimuli are not clearly motivationally engaging, or, even within the experimental session, completely novel. Nonetheless, understanding orienting requires a reconsideration of how the responses elicited by novel stimulation in the laboratory may index processes that once facilitated survival.

Characterizing Novelty

As noted earlier, Sokolov (1963) defined “stimulus novelty” as any change in stimulation from the “currently active neuronal model,” which, in more modern terminology might be called “short-term memory” (STM). Sensory stimuli that do not match representations that are currently active in STM could be considered “novel” in the Sokolovian sense. As Öhman (1979) earlier noted, however, “It is clear that most of the stimuli used in OR experiments are relatively familiar to the subject—that is, in Sokolov’s (1963) terms, they … have … neural models available in memory.” That is, even if not represented currently in STM, most, if not all, experimental stimuli share features with objects and events previously experienced by an individual and represented in long-term memory (LTM).

When “novelty” is defined on the basis of feature matches to either short- or long-term memory representations, different degrees of stimulus novelty can be characterized, as illustrated in Figure 7. In this scheme, the least novel stimulus in an oddball (or repetition–change) paradigmis the “background” or “standard” stimulus, whose repeated contiguous presentation results in a strong match to both STM and LTM representations. When averaged across an experimental session, these repeated (non-signal) stimuli show little evidence of orienting in skin conductance, heart rate, or in the late positive potential (e.g., Simons et al., 1987; Simons, Graham, Miles, & Balaban, 1998), and, of course, reactions to these standard stimuli provide the baseline for assessing orienting to the infrequently presented “rare” stimuli. On the other hand, rare stimuli in the oddball paradigm are infrequently presented throughout the experiment, decreasing the probability of a strong STM match. To the extent that orienting is related to a mismatch with currently active STM representations (as Sokolov first proposed), variables that degrade STM representations (such as time since last presentation) should increase orienting, and data are consistent with this for both autonomic (e.g., Gatchel & Lang, 1974) and ERP measures (Gonsalvez, Barry, Rushby, & Polich, 2007).

Figure 7.

A scheme for characterizing different degrees of stimulus novelty is based on the the extent to which a current cue matches short-term memory (STM) representations and long-term memory (LTM) representations.

Novel stimuli, by definition, do not share features with STM representations. On the other hand, as Öhman (1979) noted, in the laboratory, novel cues most likely are similar to (share features with) existing LTM representations, even though, prior to initial presentation in the experiment, the identical stimulus has not been encountered before. Thus, for example, novel pictures of shoes, babies, or accident scenes; tones of varying frequencies and intensities; even abstract objects and strange geometric figures typically share features with existing LTM representations, rendering these stimuli what might be termed “familiar,” rather than truly novel. On the other hand, one can conceive of a “truly novel” stimulus that is not similar to (i.e., shares no features with) any event or object in either short-term or long-term memory. Not only have truly novel objects and events never been experienced before, these stimuli bear extremely little, or no, featural similarity, to anything that has ever been experienced previously. For adult humans, very few, if any, stimuli will be truly novel in this sense. Nonetheless, understanding the functional role of different OR components requires understanding reactions to truly novel stimuli—those that do not match anything in the individual’s experience.

Truly Novel Stimuli

Whereas reactions to truly novel stimuli are difficult to study in the adult human laboratory, stimuli are more likely to be truly novel for children, who still lack extensive perceptual and conceptual experience, and in laboratory animals who live in controlled environments. In fact, a group of Russian investigators (Dolin, Zborovskaya, & Zamakhovev, 1965) observed a cascade of behaviors when chimps and other primates were presented with a variety of novel animate and inanimate objects. Initially, these novel stimuli elicited a set of behaviors that Dolin et al. called the “general inhibitory reaction” and that Pavlov (1927) more lyrically termed the “primary reflex of biological caution.” This initial response involved a cautious cessation of activity and freezing, together with attentive visual tracking: The animal stopped, looked, and listened.

In a second stage, which Dolin et al. (1965) called “motor activity” and which Pavlov (1927) termed the “positive orienting response,” the responses were anything but positive, in a motivational sense. Rather, the animal displayed a defensive posture, accompanied by threatening facial displays, signs of preparation to attack, and other protective behaviors, as illustrated in Figure 8. Following a period of time in which the novel stimulus did not actually attack (or “the environment did not change”; Dolin et al., 1965), a third stage of “exploratory” activity ensued, consisting of slow approach, visual inspection from all angles, and, finally, sniffing and handling. This approach or exploratory behavior, often considered synonymous with orienting in the human and animal literature, was initiated only following a series of reactions that were clearly defensive in nature.

Figure 8.

Primates exposed to novel stimuli react with a variety of defensive postures, consistent with the idea that novel stimulation elicits defensive activation. Reprinted from Dolin et al. (1965).

Thus, when confronted by a truly novel stimulus, the initial reactions are defensive and protective. Rather than blithely approaching a truly novel object to sniff, lick, or handle it, the animal is cautious, defensive, fearful. Initial stages of defense prompt focused perceptual intake and attention, closely followed by preparation to attack and other defensive displays. Only after information accumulates over time suggesting a lack of threat do more traditional “orienting” behaviors of approach and exploration ensue. Truly novel events are presumably terrifying because of their potential to threaten life—what you don’t know can hurt you. Orienting to novel stimulation is important for survival primarily because a truly novel stimulus may be dangerous. In the absence of any stored information (i.e., past experience) regarding hedonic valence, these novel stimuli reflexively engage the defense system.

According to this perspective, the different components of the orienting response index specific processes that evolved to serve defensive behavior—that is, processes that assist in defending the organism from a potentially life-threatening event. When confronted with a possible threat, the first step is to figure out “What is it?” or, given the highly motivated nature of this activity in the context of truly novel stimuli, it is perhaps more appropriately rendered as “What is THAT???” Initial processing focuses on acquiring more information about the current environment— seeking sensory and perceptual cues regarding the identity, capability, and intent (if animate) of the novel stimulus. Perceptual processing, indexed by cardiac deceleration, is an important first stage in defensive behavior and comprises the foundation for this component of the orienting response. This initial stage of orienting, variously labeled as “information-gathering” (Näätänen, 1992), “analyzing” (Douglas, 1972); “modeling” (Sokolov, 1963), or simply “sensory” (Graham, 1979; Loveless, 1979), focuses on the sensory-perceptual processes in which information is extracted from the current environment. Its behavioral manifestation, as noted by Pavlov (1927), is to direct sensory receptors (i.e., eyes, ears, nose, etc.,) toward the instigating cue; its physiological correlate is a relatively prolonged cardiac deceleration.

Because the outcome of the information-gathering stage could very well confirm danger, a corollary component of the orienting response is to prepare for action. More precisely, a parallel question for the individual, which Germana (1968) conceptualized as “What’s to be done?”, can again be rendered with more urgency as “What should I DO???” This stage of orienting has been labeled the “energizing” aspect (Näätänen, 1992), the “motivational” system (Douglas, 1972), the “amplifying” system (Sokolov, 1963), or, simply, the “motor” system (Graham, 1979). The role of the sympathetic nervous system in supporting behavioral arousal and action has long been noted, and skin conductance can be considered an index of this component of the orienting response. Thus, when truly novel stimuli activate the defense system, preparation for action ensues, indexed in the orienting component of a sympathetically mediated electrodermal increase.

Beyond Truly Novel Stimulation

Based on their evolutionary relationship to truly novel stimuli, reflexive orienting responses are still elicited, albeit to a reduced degree, when current stimuli do not exactly match a STM or LTM representation. The extent of initial orienting for novel stimuli that are familiar or rare will initially be a function of their similarity to existing STM and/or LTM representations. For instance, in their model, Gati and Ben-Shakhar (1990) define stimulus novelty on the basis of featural matches to currently active STM representations. The closer the featural match, the less novel the stimulus. A similar continuum, in terms of feature matches, can be proposed for LTM representations, ranging from a very low match for truly novel stimuli to substantial overlap for extremely familiar stimuli. Based on a feature similarity model, orienting to novel stimuli will be greatest for cues with a poor featural match to either STM or LTM representations and will be least for cues with high featural overlap. Moreover, this model suggests that orienting to familiar stimuli can decrease across the course of a study, as features of new experimental exemplars become represented in STM (and active in LTM or “primed”) on the basis of experience with previous exemplars.

How does stimulus significance, defined in terms of pleasure and arousal, affect the magnitude of orienting to novel pictures? These familiar visual cues match, more or less well, existing representations in long-termmemory. Thus, a picture of a baby that has never been seen before matches LTM representations of previously seen babies, and new pictures of attacking dogs similarly activate related LTM representations. For unpleasant pictures, the LTM representations include existing associations to the subcortical systems mediating defense—in fact, it is these associations (and their links to the neural structures mediating enhanced perception, arousal, and action) that render these cues unpleasant and significant (Lang, 1984). As for the defensive response that is reflexively engaged by truly novel stimuli, activation of the defensive system through existing associations initiates heightened perceptual processing (cardiac deceleration) and preparation for action (skin conductance change) whose intensity varies with degree of defensive activation.

Similarly, novel pleasant pictures are presumed to match, more or less well, existing LTM representations, but these now include associations to the appetitive, rather than the defensive, motivational system. In this case, a stimulus, though novel, has a high probability of being nonthreatening, and perhaps even rewarding, based on its similarity to pleasurable events and objects from past experience. These existing associations to the appetitive system decrease the need for intense sensory/ perceptual processing in the service of protection, reflected in reduced cardiac orienting. Rather, because past experience identifies this novel stimulus as safe and potentially pleasurable, preparation for action (Go for it!) immediately ensues, indexed by heightened skin conductance changes. In this model appetitive motivation (and positive emotion) is considered to be as action-oriented as is defensive motivation.

The magnitude of the late positive potential is interpreted here as an index that motivational activation has occurred, that is, that significance has been detected. As noted above, stimuli that do not exactly match a STM or LTM representation result in (low level) activation of the defensive system, leading to differences in the late positive potential for rare compared to standard stimuli. Pictures of never seen but familiar pleasant and unpleasant pictures naturally activate appetitive and defensive motivational systems through existing associations, with this activation similarly indexed by an enhanced late positive potential. Whereas this component of the ERP has sometimes been linked to resource allocation, its resistance to habituation with multiple repetitions is not completely consistent with this type of interpretation.¹ The conclusion that the late positive potential is an index of stimulus significance is consistent with Ritter and Vaughan (1969), who concluded, “The LPC [late positive complex], then, can be elicited in different situations… and appears to be a central correlate for cognitive evaluation of stimulus significance” (p. 328). In the current view, stimulus “significance” is defined as activation of cortico-limbic appetitive and defensive systems that mediate the sensory and motor processes that support perception and action.

Orienting and Habituation

If the different components of the orienting response reflect the operation of different adaptive reactions to novel or significant stimuli, their rates of habituation need not be constant. In this view, habituation is not considered a defining feature of the orienting response. Rather, specific components will fall out as the processes they index are no longer engaged. For instance, cardiac deceleration apparently terminates when a stimulus is well represented in either STM or LTM, resulting in relatively rapid habituation of this component of the orienting response. Skin conductance change, on the other hand, shows a slower rate of habituation, indicating continued preparation for action despite stimulus familiarity. The magnitude of the late positive potential appears to be the most resilient index that significance has been detected, that is, that activation of basic motivational systems has occurred. Changes in the significance of pleasant and unpleasant stimuli will only occur when these cues lose their associative links to the subcortical systems mediating appetitive and defensive motivation. Simple repetition of an unpleasant (or pleasant) picture does not appear to eliminate these associative connections, consistent with learning studies that suggest that extinguishing associative links to appetitive or defensive systems (using extinction procedures) is rarely complete (e.g., Rescorla, 2001).

Passive, Active, and Natural Selective Attention

Thus far, we have considered orienting to novel or significant stimuli in the absence of a task context. In fact, two classes of orienting responses have often been distinguished: involuntary (or passive) ORs and voluntary (or active) ORs (Corbetta & Shulman, 2002; Graham & Hackley, 1991; Maltzman, 1979; Näätänen, 1979; Roth, 1983). Methodologically, ORs studied in passive contexts typically involve manipulations of stimulus novelty, whereas ORs in active contexts involve manipulations of task-relevance or instructions (i.e., Sokolov’s “signal” stimuli). A thorny issue, previously raised by a number of theorists (Öhman, 1979; Roth, 1983) concerns the unsuitability of neuronal mismatch as a mechanism able to mediate both involuntary and voluntary ORs. The particular difficulty concerns the inability of a mismatch model to explain why task-relevant targets, which presumably have an exact match in the current mental model, should ever elicit an OR. As noted by Roth (1983): “Paradoxically, P300 has been said to occur when the current stimulus matches the neuronal model…or when it fails to match the template” (p. 191).

In their computational model, Gati and Ben-Shakhar (1990) account for these dual effects by simply proposing that skin conductance orienting is elicited by two independent processes, one based on novelty (STM matches) and one based on matching significant representations. Relatedly, Öhman (1979) proposed that orienting reflects a call to initiate processing, with stimuli that do not match the current neuronal model initiating such a call, as well as stimuli that match “a memory representation that has been primed as significant” (p. 445).

In the current view, on the other hand, task-relevant stimuli are held to elicit orienting because they utilize the same processes of enhanced perceptual processing and preparation for action that are initiated by unconditioned appetitive and aversive cues. And, like these biologically relevant cues, stimuli that are task relevant elicit an enhanced late positivity in the ERP that indexes their detection as significant events. For the simple auditory and visual cues used in many investigations of active orienting, this centro-parietal positivity occurs somewhat earlier in time, can be more punctate, and is typically labeled a P300 or P3 (Donchin et al., 1984). In fact, P3 amplitude to task-relevant cues has often been hypothesized to reflect stimulus “significance” (Donchin, 1981), “meaning” (Johnson, 1986), or “information value” (Sutton, Braren, Zubin, & John, 1965), and its sensitivity to motivational relevance has not gone unnoticed by researchers. For instance, noting that extremely loud and extremely rare stimuli both elicit larger P3 components than expected, Johnson (1986) argued that these contexts created a situation “in which the stimuli became biologically relevant. This would result in their being endowed with additional value” (p. 380). Taken together, the hypothesis that a late positive potential in the ERP reflects the motivational relevance of a cue in the current context is quite consistent with a variety of theoretical conceptualizations of P3 modulation (e.g., Donchin, 1981; Johnson, 1986).

Summary and Conclusions

Through existing (emotional) or new (task-relevant) associations to the motivational systems that evolved to support perception and action in living organisms, “significant” stimuli prompt an enhanced LPP that may be accompanied by cardiac deceleration, if extensive perceptual processing is required, and/or skin conductance increases, when action is imminent. This is why there is so much attention in emotion—and why motivation remains an important building block in perception and learning. Whereas emotional cues naturally engage the appetitive and defensive motivational circuits that activate adaptive perceptual and motor processes, experimental tasks, salient contexts, and verbal instructions are now able to flexibly engage the neural circuits that were once dedicated exclusively to survival. In this view, orienting is fundamentally a multifaceted reaction to an event’s significance, engaging sensory-motor processes that support not only passive and active attention, but what is viewed here as its foundation—natural selective attention.