ORIENTING OF ATTENTION: THEN AND NOW

Abstract

It is nearly 35 years since I gave the 7th Sir Frederick Bartlett lecture at Oxford University. This was published as a paper entitled Orienting of Attention in the Quarterly Journal (1980, 32: 3–25). The topic was then primarily in Psychology, but now equally often in Neuroscience. This paper summarizes the background of the reaction time methods used in the original paper and findings that emerged later on the sensory consequences of orienting, mainly in the visual system. It then discusses the brain network which is the source of the sensory amplification and other brain networks that are involved in attention. Next, it reviews studies of the development of attentional networks in early life. Finally, it indicates how the new tools available to explore the human brain can lead to further progress.

I was delighted to be invited to give the 7^th Sir Frederick Bartlett lecture at a meeting of the Experimental Psychology Society in July of 1979 (Posner, 1980). My pleasure partly reflected having met Bartlett in 1968 when I was a visitor at the Applied Psychology Lab (now Brain and Cognitive Science) of the MRC in Cambridge, U.K. Even in his 80s Bartlett was an imposing figure and a person who had greatly influenced the development of cognitive psychology in general and me in particular. The second source of the delight was kept mainly to myself, but I felt I had something important to say. Unfortunately this is not always the case when invited to talk. My students and I had measured the time course of attention shifts without any eye movements. I felt it was an important model for the likely integration of cognitive with neurophysiological approaches to cognition. I was right about that, and the article Orienting of Attention (O of A) resulting from the lecture has been cited more than 5,000 times according to Google Scholar. A recent book with the same name summarizes much this work in considerable detail (Wright & Ward, 2011).

In this paper I have reviewed more recent developments some of which were directly stimulated by O of A. These include use of the method to explore orienting, findings concerning how the network of brain areas that are the source of orienting influence sensory systems, and the relation of orienting to eye movements. I also consider extensions that regarded orienting as only one of several functions of attention and based on the use of neuroimaging to understand the anatomy of brain networks related to attention. Understanding the anatomy of attention has helped foster studies of the development of attentional networks both in childhood and through training studies in adults. Finally I consider how the combined cognitive and neuroscience approach to attention may influence future findings.

The method

Probably the largest number of citations to my Bartlett lecture arose from the cueing method employed to observe the movement of attention to the target. I did not originate the method nor was this my first use of it. To my knowledge the method began with the effort of J.A. Leonard (1953) at the time a researcher in Cambridge, to discover the length of time needed to assimilate one bit of information. He wanted to separate the one bit of knowledge from the time to perceive the stimulus or produce the response. To do this he presented subjects with six lights, the participants were to respond as quickly as possible when one light was turned off. In some conditions prior to extinguishing the target light he turned off three of the lights thus reducing the possible S-R combinations by one bit (from six to three alternatives). The time required to reduce reaction time from that obtained with six alternatives to that obtained with only three was the desired time for assimilating one bit of information. This was a brilliant study, but unfortunately because the use of information theory did not solve all the problems of psychology as had been hoped it is largely forgotten. Leonard was a student of Sir Frederick Bartlett and later did research in the United States with Prof. Paul M. Fitts then at Ohio State University. Later, after Fitts had moved to the University of Michigan, I studied under him and took my PhD in 1962. This history perhaps explains my later postdoc at Cambridge with Robert Wilkinson and the close links my work has always had with the Cambridge unit.

I first applied Leonard’s idea to letter matching where we (Posner, Boies, Eichelman & Taylor, 1969) were able to measure the time necessary to derive the name of the letter by presenting for example upper case (A) after a varying interval with a different case (e.g. a). When the letters were simultaneous or nearly so the cross case match took about 80 milllisec longer, but eventually identical and cross case matches produced the same reaction time. I believed this yielded the time to name the letter. In 1978 (Posner,1978) I called this general method of using reaction time to measure entirely covert cognitive processed Encoding Functions, since they could be used to measure any internal operation free from stimulus and response factors.

In O of A I was reporting on our adaptation of this method to the study of attention in an empty visual field. The subject looked at a central stimulus, flanked on each side by a box, after an interval the box would change in luminance and when a target asterisk appeared the subject had to response by pressing a single key. A change in the luminance of the box was the cue for attention to move to the target and thus the time needed to shift attention to the cued location could be measured. Various control conditions were used to eliminate alternative explanations such as forward masking or inhibiting a response to the cue. In these early experiments I also used probability to make sure that participants oriented to the cue. If the cue indicated that the target would occur at the cued location with probability .8, the target was facilitated in the first 200 millisec following the cue in comparison with other locations of similar eccentricity and the facilitation remained as though attention remained at the cue. However, if the probability of the target being at the cued location was only .2 while .8 of the time in occurred at another location one found facilitation of reaction time at the cued location for the first 200 millisec followed by facilitation at the most probably location. I believed that attention had been summoned to the cue exogenously, but was then voluntarily (endogenously) moved from the cued location to where the target was most likely. Within .5 sec we seemed to have trapped a movement of attention from fixation to the cue and then from the cue to the most likely target location. It was this beautiful time locked shifts of attention that I thought would open the way for a detailed physiology of attention.

Another aspect of the cueing method was the peripheral cues that summoned attention to a location could be compared with central cues (arrows) that had a merely symbolic relationship to where one was to look. I called these methods exogenous and endogenous cueing. Subsequent studies have shown that the arrow heads might produce a form of orienting that is neiteher purely exogenous or endogenous (Ristic & Kingstone, 2012).

At the time O of A was written many psychologists did not believe attention involved internal physical mechanisms but instead viewed it as a resource or general skill (Kahneman, 1973; Neisser, 1976). However, the discoveries of Mountcastle (1978) and Wurtz, et al (1980) of the involvement of neurons in the superior colliculus and the parietal lobe persuaded me to attempt to examine the neural basis of orienting. The cueing method survived the development of neuroimaging and has been applied to separating the neural systems used by the cue from those related to the target (Corbetta & Shulman, 2002).

As a cognitive psychologist, my goal was to understand the attention system of the human brain. Because of this goal, I was interested in the common source of attentional effects. Researchers who examined attention from the psychophysical tradition concentrated on the effects of attention on sensory systems, without worrying much about the source of these effects. Both the psychophysical and cognitive approaches have made substantial progress and fit together to describe attention and its influence on even the early stages of sensory processing.

Sensory Consequences of Orienting

In the 25 years since O of A, most research has been directed to the consequences of orienting particularly within the visual system. The exciting psychophysical results have been summarized recently by Carrasco (2012). While our work demonstrated that orienting attention prior to a target produced faster reaction times to the target, giving it priority, work by Yeshurun & Carrasco (1998) using the cueing method I described above, coupled with sinusoidal grating targets, showed attention actually improved visibility for high spatial frequency information.

In a brilliant experiment, Carrasco used a segmentation task and found that in the fovea where spatial frequency resolution was higher than optimal for segmentation, attention actually impaired performance while at the periphery where spatial resolution was low attention improved performance. Models that thought of attention as a response bias or a skill designed to improve performance could not handle these results.

Also important were results using electrical recording which support amplitude gain models of attention to visual information by showing an ampliction of the P1 and N1 components of the event related electrical potential (Hillyard, Di Russo & Martinez, 2004). These results fit well with those of Carrasco. However, in the auditory system the effects of attention occurred later in time and were found not to amplify the early event related components but superimposed an additional negative response (Hillyard, Hink, Schwent, & Picton,1973).

At the time of O of A there was a controversy about whether attention was helpful in the accuracy and speed of perceiving a target in an empty visual field. There was no doubt of the importance of attention when the field was cluttered with distractors (Engle, 1971). Knowing where to attend allowed you to go directly to the target location and save a large amount of time. It was controversial whether knowledge about where the target was to occur actually improved performance when the field was empty. We learned, using the cueing method, that the onset of a stimulus in an otherwise empty field was such a good cue for orienting, there was only a small benefit of having a cue in advance of the target. However, once engaged at a location, reorienting attention had a large effect on the time to detect a target at an uncued location. I summarized findings on orienting in an empty field by arguing that the cost of disengaging from attending is larger than the benefits of attending. Thus when not attending there is little advantage to a cue, once orienting somewhere the cost of disengaging makes the cue quite important.

This principle can be applied more generally. Shortly before the Bartlett lecture Richard Shiffrin (Shiffrin, McKay & Shaffer, 1976) showed that one could attend to 49 locations as well as to one. Was attention really so unlimited? Duncan (1980) showed that it mattered very little whether you knew which of several targets was going to occur, but if you detected one target your performance was greatly diminished for a second one. In other words, once attending to something there is a powerful cost of switchin attendtion. Duncan (1980) result was important in showing one could monitor in parallel with relatively little or no loss, but attending in the sense of conscious detection was limited indeed. These findings became the basis for distinguishing between an orienting system involved in monitoring the sensory world and a second attention system more related to detection and conscious control.

One of the most striking demonstrations of the importance of attention in vision, called change blindness (Rensink, O'Regan, & Clark, 1997), was a further extension of this principle.. This work presented participants with a complex scene. A change was produced somewhere in the scene, but without either luminance nor motion cues that are normally effective in reorienting attention. They found even dramatic changes like substituting a horses head for a human head at the dinner table went unreported. The dramatic nature of this demonstration often leads people to forget that with luminance cues or motion cues present as happens most often, re-orienting occurs and changes can be easily detected.

The Orienting Network

My goal was to understand the source of the orienting effect. At the time Orienting of Attention (O of A) was written it seemed important to me to show that attention actually moved across the visual field in a way analogous to a saccade. I felt this would contribute to making covert attention seem more concrete like an eye movement. A paper by Shulman, Remington & McLean (1979) showed that intermediate locations between fixation and target were facilitated during the time of the shift. However, this behavioral evidence was challenged by subsequent reports (Gololmb et al 2011). In retrospect it proved not to be crucial. At the time, the idea of an attention movement meant that we had to regard orienting as a physical event with a real time consequence in the nervous system. However, when Georgopoulis et al (1989) showed how that changing set of receptive field orientations in the motor system could produce a covert analogue of mental rotation in the case of monkey arm movements, it no longer seemed necessary to have something actually moving in order to consider it as a real time event in the human brain.

A more persistent issue has been the relation between covert shifts of attention and eye movements. This issue was fundamental to me because I hoped to use orienting of attention as a model for probing areas of attention that were not at all close to sensory systems (e.g. attending to the meaning of a word). If orienting was the same as preparing a saccade, knowledge of its properties would be less useful as a model for types of attention which had nothing to do with sensory systems, but involved emotions, memories or thoughts. To capture this idea, I now distinguish between the site at which attention can operate and the sources of that influence in the orienting and executive attention networks (Petersen & Posner, 2012).

In O of A I did establish that orienting of attention could take place without an eye movement. I also presented evidence in the same paper (Posner, 1980, Fig. 11 page 18) that attention shifts could occur in one direction while preparing to move the eyes in a different direction, a result that I thought fatal to various efference theories based on the preparation of saccades that were not executed. I was certainly wrong about that. Rizzolatti et al (1987) argued that premotor cortex especially the frontal eye fields was the source of the orienting effects which involved programming, though not always making an eye movement. Moreover some behavioral results did not show the independence between eye movements and attention shifts that were reported in O of A, but favored the Rizolatti’s argument. Somewhat later there was also a clear imaging result (Corbetta, 1998) showing a very strong overlap, approaching identity, between brain areas involved in generation of saccades and those involved in covert orienting of attention.

For this reason I began to think that orienting of attention was not a good model for a separate attention system, but was instead very closely related to saccadic eye movements. However, studies using cellular recording in the frontal eye fields, which was a part of the overlapping networks for both saccades and attention shifts, showed there were separate populations of cells that were either active before saccades or before covert eye movements, but not both (Thompson, Biscoe & Sato, 2005). Some recent reviewers of the behavioral work also concluded that covert attention were not as dependent upon eye movement programming as required by the premotor theory (Smith & Schenk, 2012). Important to the relationship between covert and overt attention is the idea may be that transient shifts of attention are more dependent on saccade preparation than is the maintenance of attention once a shift has occurred (Belopolsky & Theeuwes, 2009).

One place where dependence between covert attention and eye movements is strongest is when stimuli lie between the fovea and a peripheral target so that the perception of the target is diminished (Bouma, 1970). This phenomenon is often called crowding. When people are asked to make an eye movement toward the target the crowding effect is reduced, even before the eyes begin to move. An instruction to attend covertly to the target has no similar effect (Harrison, Mattingley & Remington, 2013). This finding shows making an eye movement can amplify attention effects and produce results not obtained by a covert attention shift. Moore et al (2012) argue that the populations of sensory and movement cells in the frontal eye fields are not distinct and most cells have both motor and sensory functions. These authors also indicate that covert shifts and saccadic preparation interact and that in some circumstances, the attention shifts appear to control saccadic trajectories, and in other situation, the reverse.

Although the premotor theory was certainly correct that both attention and eye movements are influenced by the same prefrontal structure, it appears that there is an important separations and interaction between the two at both the cellular and behavioral levels. Although even now this issue is not settled, it is a very good example of the importance of considering all levels of analysis when attempting to develop a strong theoretical account. For the time being, I still think O of A, which illustrates the various theories of the relation between saccades and eye movements, may be about right in proposing an intermediate level of dependence that may reflext early experience leading to their close coordination. It has been observed that infants often make multiple saccades when attempting to foveate targets (Aslin & Salapatek, 1975), thus providing an opportunity to learn to coordinate attention and eye movements.

The cueing method and the distinction between exogenous and endogenous cueing had a further significance when neuroimaging began to be used to study orienting of attention (Corbetta & Shulman, 2002). I had often used an arrow at fixation to direct attention to locations in the visual field. Since the cueing method allows separation of the influence of the cue from that of the target it is possible to examine the parts of the brain activated by the cue separate from those activated by targets. In a very influential series of experiments (summarized in Corbetta & Shulman, 2002) it was found that the arrow cue influenced a dorsal network of brain areas including the superior parietal lobe and frontal eye fields that seemed most important for voluntary orienting of attention. Following an invalid target, a more ventral set of brain areas were activated that included the temporal–parietal junction.

At the time of O of A I did not imagine that neuroimaging would provide evidence clearly suggesting a ventral brain network involved in more automatic processes and a dorsal network in more voluntary top down control (Corbetta & Shulman 2002). The finding that the brain systems of orienting separate voluntary from automatic control into distinct but interacting brain networks is, to me, one of the best openings for the study of the physical basis for volition or what some call “will” that I know about.

Other attention networks

At the time I was writing O of A, I did not think about there being separate brain networks for different functions of attention. In fact almost nothing was known about the neural system underlying orienting much less other networks of attention. However, within a decade the advent of neuroimaging had made a dramatic change (Posner & Petersen, 1990). In our earliest neuroimaging studies of language (Petersen, Fox, Posner, Mintun & Raichle,1989) we had shown that making a simple word association, in comparison to merely reading a word aloud, activated an areas of the frontal midline called the anterior cingulate. Jose Pardo (Pardo, Pardo, Janer & Raichle, 1990), who had worked with us on these studies asked me to say what task he could use to see if the cingulate activation was due to attention. I said try the Stroop effect, he did and his study became the first of many revealing that Stroop and other conflict related tasks activate the dorsal anterior cingulate (Bush, Luu & Posner, 2000).

These studies led me to update three functions of attention I had earlier postulated (Posner, 1978) by arguing for three different brain network supporting the functions of orienting, alerting and executive control (Posner & Petersen, 1990; Petersen & Posner, 2012). Each of these networks involved multiple brain areas and their connections. Imaging data support the argument for separable brain networks (Fan et al 2005). At first, imaging was very restricted in the ability to deal with individual brains because of limits to the amount of radiation one could use, but with the advent of MRI that restriction was reduced and it became possible to consider individual differences as resulting from the efficiency of brain networks that were common to everyone. I believe that the ideas concerning brain networks that arose with imaging studies provides a very good way of relating common psychological functions, studied by cognitive psychologists, with individual differences as they have been studied by researchers in development and personality.

There are individual differences in the efficiency of each of the three attention networks. The Attention Network Test (ANT) was devised as a means of measuring these differences (Fan et al 2002). The task requires the person to press one key if a central arrow points to the left and another if it points to the right. Conflict is introduced by having surrounding flanker arrows point in either the same (congruent) or the opposite (incongruent) direction. Cues presented prior to the target provide information on where or when the target will occur. Three scores are computed, that relate to the performance of each individual in alerting, orienting and executive control. In our work we have used the Attention Network Test (ANT) to examine the efficiency of brain networks underlying attention (Fan et al 2002). A children’s version of this test is very similar to the adult test, but replaces the arrows with fish (Rueda et al 2004).

Studies have shown moderate reliability of conflict scores, but much lower reliability for the orienting and alerting scores (MacLeod et al., 2010) and recent revisions of the ANT provide better measures of orienting and alerting that may improve these results, but usually at the cost of additional trials (Fan et al., 2009). The attentional networks involve different cortical brain areas (Fan et al 2005), and scores on the ANT are related to distinct white matter pathways (Niogi & McCandliss, 2009) and structural differences in cortical thickness (grey matter) (Westlye, Grydland, Walhove, Fjell, 2011). Although there is considerable independence between the networks, revisions of the ANT show significant interaction between network (Callejas, Lupianez & Tudela, 2004; Fan et al., 2009). It is clear that the networks communicate and work together in many situations, even though their anatomy is mostly distinct.

The network view arising from imaging of attention seems to me to bring together the cognitive approach with its emphasis on functions common to most or all of the people studied with the individual differences approach. Attention networks are common to everyone, but their efficiency differs. These differences may in part reflect genetic variation between people and in part reflect life experiences.

Development

An important consequence of imaging brain networks is to raise the issue of how attention networks become organized in early life. We have been examining issues of how genes and experience shape the three attention networks (Posner, Rothbart, Sheese, & Voelker, 2012). We conducted a longitudinal study on the development of the executive attention network which is closely related to self regulation The testing began when the infants were 7 months old. We had thought that this was your enough for us to observe the earliest part of the development of the executive network. However, even at 7 month infants detect errors by activating the anterior cingulate just as adults do (Berger, Tzur & Posner, 2006).

Because infants are not able to carry out voluntary attention tasks, we used a visual task in which a series of attractive stimuli are put on the screen in a repetitive sequence (Clohessy, Posner & Rothbart, 2001; Haith, Hazan & Goodman, 1988). Infants orient to them by moving their eyes (and head) to the location. On some trials infants showed they anticipated what was coming by orienting prior to the stimulus. We found (Sheese et al 2008) that infants who made the most anticipatory eye movements also exhibited a pattern of cautious reaching toward novel objects that predicts effortful control in older children (Rothbart, 2011). In addition, infants with more anticipatory looks showed more spontaneous attempts at regulation of emotional distress when presented with somewhat frightening objects.

We had originally thought that the relation of anticipatory eye movements to self regulation was evidence of early control by the executive networks. However, this was a longitudinal study so at age 4 we were able to run the same infants in the ANT and that indicated that anticipatory eye movement in infancy were more related to the orienting scores at age 4.

These findings led us to the view that the orienting network provides the primary regulatory function during infancy. The orienting network continues to serve as a control system, but starting in childhood the executive attention appears to dominate in regulating emotions and thoughts (Isaacowitz, 2012; Posner et al, 2012; Rothbart et al, 2011). The executive network is present in infancy but it is not yet connected in a way that produces control over behavior. For example, even though infants at 7 months detect errors we observed the ability to slow down behavior following an error to develop between 3and 4 years (Jones, Rothbart & Posner, 2003)

Changes in connectivity in infancy and early childhood have been supported by resting state MRI studies (Fair et al 2009) and by MRI during conflict tasks (Fjell et al, 2012). Also this parallel use of the two networks fits with the findings of Dosenbach et al (2007) that in adults the frontal-parietal network (orienting) controls task behavior at short time intervals whereas the cingulo-opercular (executive) network exercises strategic control over long intervals.

There are very important consequences for the developing child in these internal changes. The executive network is involved in resolving competing actions in tasks where there is conflict. This is done both by enhancing activity in networks related to our goals and inhibiting activity in conflicting networks, these controls are effected by long connections between the nodes of the executive network and cognitive and emotional areas of the frontal and posterior brain. In this way the executive network is important for voluntary control and self regulation (Bush, Luu & Posner, 2000; Sheth et al 2012). Effortful control is a higher order temperament factor assessing self regulation that is obtained from parent report questionnaires (Rothbart, 2011). In childhood, performance on conflict related cognitive tasks is positively related to measures of children’s effortful control (Rothbart, 2011). During childhood and in adulthood effortful control is correlated with school performance and with indices of life success, including health, income and successful human relationships (Checa & Rueda, 2011; Moffitt et al 2011).

The changes in connectivity reported by Fair et al, 2009 during development using resting state MRI studies involve functional connectivity based upon correlations between BOLD activity in separated brain areas. During development there are large physical changes in connections between brain areas. The number of axons connecting brain areas increases followed by an increase in the myelin sheath that surrounds the axon and provides insulation. Together these changes result in more efficient connections (Lebel et al 2012). Fractional anisotropy (FA) is the main index for measuring the integrity of white matter fibers when using DTI.

In our work we studied FA in college students before and after a form of mindfulness meditation called Integrated Body Mind Training (IBMT) in comparison to a control group given the same amount of relaxation training. Using the ANT we found clear improvement in executive attention after only five days of training. After two to four weeks of training we found significantly greater change in FA following meditation training than following the relaxation training control in all areas of connectivity of the anterior cingulate, but not in other brain areas (Tang et al 2010).

These alterations in FA could originate from several factors such as changes in myelination, or factors related to axon density. Several DTI studies have examined axial diffusivity (AD) and radial diffusivity (RD), the most important indices associated with FA, to understand the mechanisms of FA change (Bennett et al 2010; Burzynska et al 2010). Changes in AD are associated with axon morphological changes, with lower AD value indicating higher axonal density. In contrast, RD implicates the character of the myelin. Decreases in RD imply increased myelination, while increases represents demyelination.

In our study (Tang et al 2012), we investigated AD and RD where FA indicated that integrity of white matter fibers was enhanced in the IBMT group more than control group. We found that after two weeks there were changes in axonal density but not in myelination. In some areas these changes in axonal density were correlated with improved mood and affect as measured by self report. After 4 weeks of training we found evidence of myelination changes. Since the developmental changes in childhood first involve changes in axonal density and only later myelination, our training may provide changes that are somewhat similar to those found in development. If so, it might be possible to use training to study how physical changes in connectivity alter aspects of control including reaction time, control of affect, stress reduction and other changes found with meditation training. In fact at the time of changes from the orienting to the executive network children are undergoing changes in behavior that are consonant with the development of self control.

Environmental factors help to shape development of the brain network related to attention. Several lines of research converge to argue that training can influence these networks. In childhood exposing the infant to novel objects may help develop the executive network (Posner, et al, 2012; Shulman et al 2009). In addition specific training at age 4–6 appear to produce changes in the executive network that make it more adult like in response to conflict related challenges (Diamond & Lee, 2011; Rueda et al 2005; 2012). Even adults can show change in white matter pathways due to training as discussed above. Thus the general environment together with genes provide important means for shaping the efficiency of executive attention all through life.

The Future

The Bartlett lecture was one of the most memorable events of my career in psychology. O of A was a purely behavioral paper. At the time I could not have imagined writing the paragraphs above in which changes in control pass from the orienting network, involving one set of brain areas, to the executive network due to changes in connectivity that can be mapped in the developing human brain. The advent of neuroimaging made this possible.

Further changes in the technology for studies of the brain may be expected. For example, current studies of rodents and primates (Diester, 2011) are using light (optogenetics) methods to manipulate cells of particular types within brain networks. This method could help to solve the problem of relating large scale neural networks more directly to specific neural activity. The connectome project may allow tracing of large number of white matter pathways in the human brain at varying ages to provide a detailed pattern of development (Sporns, 2011).

The pace of technological advance in mapping brain systems is likely to increase over the coming years. It may be daunting for psychologists to understand and keep up with these advances. However, it does seem to me the lesson of O of A is that psychological studies at the behavioral level will continue to be needed in order to be able to relate the myriad of brain changes to their significance for human thought and action. Even at this current moment, we have a rough picture of how brain activation, functional connectivity and white matter efficiency change with age, but only the most primitive ideas of how these changes actually work to produce the dramatic differences between infancy and childhood.

It is certainly true that not all of the ideas described in this paper came directly from O of A. However, at least eight years before imaging was to usher in the era of cognitive neuroscience, I was already convinced that we had opened a small but important window on how cognition and neuroscience could work together to solve the many issues of brain research. Over thirty years later I realize how far we have come and how distant the goal remains, but am still pleased to have had this small role in its history.