CNTRICS Final Task Selection

Abstract

The construct of attention has many facets that have been examined in human and animal research and in healthy and psychiatrically disordered conditions. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) group concluded that control of attention -- the processes that guide selection of task-relevant inputs -- is particularly impaired in schizophrenia and could profit from further work with refined measurement tools. Thus, nominations for cognitive tasks that provide discrete measures of control of attention were sought and were then evaluated at the third CNTRICS meeting for their promise for future use in treatment development. This article describes the five nominated measures and their strengths and weaknesses for cognitive neuroscience work relevant to treatment development. Two paradigms, Guided Search and the Distractor Condition Sustained Attention Task, were viewed as having the greatest immediate promise for development into tools for treatment research in schizophrenia and are described in more detail by their nominators.

Introduction

Abnormalities in attention were described in the early clinical accounts of schizophrenia ^1,2 and are one of the frequently studied of the cognitive deficits in schizophrenic patients ^3-6. Furthermore, attentional deficits appear to be core elements of schizophrenia, as they endure across periods of psychosis and remission ^7-9. Their presence in attenuated form among the first degree relatives of schizophrenic patients suggests their promise as components of genetic susceptibility to schizophrenia and related disorders ^10,11. Attentional deficits among children with a schizophrenic parent are found by late childhood and adolescence ^12,13 and are among the cognitive predictors of a later schizophrenia spectrum outcome in such “high-risk” children ^14,15. Abnormalities in attention in schizophrenia also show relationships to everyday functioning, as measured by sustained, focused attention, reaction time, dichotic listening, and digit span tasks ¹⁶. Indeed, deficits in attention emerged from an evaluation of factor analytic studies of cognition in schizophrenia as one of the key separable dimensions of cognition in this disorder ¹⁷. Based on these many sources of evidence, attention was included among the core cognitive domains in the recent Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative ^18,19 that preceded the current CNTRICS process ²⁰.

One might have thought that the prominence of research on attentional abnormalities in schizophrenia would mean that the cognitive mechanisms that underlie these deficits would be well understood. However, agreement on the precise nature of attentional deficits in schizophrenia has remained elusive ^5,6,21-24. Part of this situation is likely due to the fact that the term “attention” has many meanings in the scientific literature and refers to a set of cognitive processes rather than any single process. However, another limiting factor has been the rarity with which basic cognitive psychologists and cognitive neuroscientists have come together with clinical investigators to bring the more differentiated constructs and paradigms of the former to clinical research. The CNTRICS initiative and other recent collaborations among basic and clinical researchers have begun to address this limitation ^25-27.

Control of Attention as the CNTRICS Attention Construct

At the first CNTRICS meeting, considerable discussion focused on delineating the aspects of attention that would be the most productive focus for CNTRICS purposes. Given the conceptual overlap between attention, working memory, and executive control systems in the basic cognitive and cognitive neuroscience literature, a decision was made to emphasize input selection processes under the heading of attention ²¹. Some concepts and tasks that might otherwise have been viewed as reflecting attention can be found in the articles in this issue concerning working memory and executive control processes. Furthermore, given recent studies which suggest that impairment in control but not implementation of input selection is deficient in schizophrenia ^25,28,29, the CNTRICS group decided to emphasize control of attention as the core attention construct in its evaluation of promising cognitive paradigms. Control of attention was defined as “the ability to guide and/or change the focus of attention in response to internal representations”. Luck and Gold ²¹ have described the CNTRICS conceptual distinctions in more detail and have discussed the interfaces among attention, executive control, and working memory.

For the third CNTRICS meeting, the focus was on evaluating the promise of current paradigms measuring control of attention for further development in clinical research on schizophrenia. Paralleling the nomination process for tasks relevant to other cognitive constructs described in this series of CNTRICS articles, scientists were asked to nominate tasks and to provide information on construct validity, linkage to neural circuits, clarity of the contributing cognitive processes, availability of versions of the task for animal research, support for neuropharmacological linkages to neural systems, amenability for use in neuroimaging studies, evidence of task impairment in schizophrenia, and psychometric characteristics favoring use in clinical trials. At the third CNTRICS meeting, a working group on attention met to evaluate the evidence for each task that was nominated as a measure of the control of attention.

Evaluation of Tasks Considered for Measurement of Control of Attention

Five tasks were nominated by scientists as promising measures of control of attention for the purposes of CNTRICS: 1) an Attention Capture Task³⁰, 2) an Attention Networks Task^31,32, 3) a Distractor Condition Sustained Attention Task^33,34, 4) a Guided Search Task^25,35, and 5) a Spatial Cueing Task^36,37.

After initial discussion of the five nominations, the CNTRICS working group on control of attention concluded that the Attention Networks Task included potentially valuable measures of alerting, orienting, and conflict aspects of attention, but did not include a measure of control of attention in response to an internal representation. Thus, the measures of that task were excluded from further consideration as an index of control of attention.

Of the remaining four nominated tasks, two paradigms for measuring control of attention were judged as the most promising for further development for work on schizophrenia at this time - the Guided Search paradigm and the Distractor Condition Sustained Attention Task. These two paradigms and their supporting research will be described in detail in the following sections of this article. It is noteworthy that they were viewed by the CNTRICS working group as equally promising, but with strengths in differing areas.

The Guided Search paradigm involves search for a target in a visual array using some feature of a subset of stimuli to limit search to only the stimuli that include that feature^25,38. It was judged to have very strong construct validity as a measure of the ability to guide the focus of attention using an internal representation, has clear links to specific cognitive mechanisms, is easily adaptable for human neuroimaging studies, and is known to detect a noteworthy top-down processing deficit in schizophrenia. Several studies also indicate a link of guided search processes to specific neural regions. On the other hand, the Guided Search task used in humans has not been applied in the same form in animals, although the group believed that it could be successfully adapted for application for monkeys and perhaps in an adapted form for non-primate animals.

The Distractor Condition Sustained Attention Task involves detection of a signal (e.g., a brief focal light illumination) in a series of discrete trials under distractor conditions (e.g., changing overall lighting intensity) as compared to nondistractor conditions³³. It was judged to have a particular advantage in applicability to several species of animals as well as to humans, as it was initially developed as a sustained attention paradigm for rat research. Its links to neural circuits were viewed as well documented, including through studies involving neuropharmacological manipulations, thereby strengthening the task’s applicability for new drug development. The Distractor Condition Sustained Attention Task was also viewed as easily amenable to human neuroimaging studies and as having construct validity as a measure of control of attention that was nearly as strong as the Guided Search paradigm. In contrast, the specific cognitive mechanisms that underlie performance on the Distractor Condition Sustained Attention Task were believed to be only partially understood. Furthermore, the task is only beginning to be applied to schizophrenia patients.

The Attention Capture Task was viewed as having strong construct validity as a measure of the control of attention, as it examines the response cost of introducing salient but task-irrelevant stimuli within a search for other relevant stimuli. In a Fan et al. (1992) ³¹ version of this task, for example, subjects are exposed to a rapid-onset set of small circles in a cue array and then search a visual array for a target that is a rapid onset “x” or “=” character³¹. To the extent that attention is captured by the irrelevant stimulus in the initial cue array, reaction times are slowed. However, the links of the measures in the Attention Capture paradigm to neural circuits were judged to be only moderately understood, the cognitive mechanisms only partially explicated, and its ability to detect impairment in schizophrenia unknown. While the working group believed that an animal version could be developed for this task, it was not aware of any current version.

The Spatial Cueing task developed by Posner³⁷ involves spatial cues in the periphery of the visual field that precede the detection of targets on the same or opposite side of the visual field. The reaction time to valid (same side) cues and invalid (opposite side) cues are typically the primary indices of the benefits and costs of spatial orienting of attention. Another aspect of performance is slowing of reaction times to valid cues if the delay between cue and target is more than 250 msec, believed to represent a relatively automatic inhibitory mechanism that protects one against redirection of attention to previously scanned locations that were insignificant³⁹. The CNTRICS working group on control of attention considered this task to be of considerable interest as a spatial orienting of attention measure, but thought that it was only moderately related to control of attention by an internal representation. While the cognitive mechanisms and neural circuits underlying this task are well understood, the group noted that the available literature is inconsistent regarding the task’s sensitivity to a key attentional impairment in schizophrenia. Its sensitivity to neuropharmacological manipulations was also felt to be only moderate. Thus, the other nominated tasks were judged to be more promising measures of deficits in control of attention in schizophrenia.

The two most highly regarded CNTRICS tasks for measuring control of attention are described and reviewed in much more detail in the sections that follow.

The Guided Search Task

Description

The guided search paradigm derives from Wolfe’s Guided Search theory of attention ^35,38, which is in turn related to Treisman’s highly influential Feature Integration Theory ^40,41. The general idea is simple: When an individual is looking for an object that contains certain features, top-down control mechanisms are used to highlight items containing those features so that attention is directed to those items. For example, if an individual is looking for a blue pen, a target template will be activated that specifies the features of the pen (its size, shape, color, etc.). Items containing those features will tend to attract attention more than other items, making it possible to find the pen relatively quickly.

Figure 1 illustrates a typical laboratory-based guided search task. The subject is told to search for a target, defined as a red square with a gap on the top, and is then presented with a series of stimulus arrays that either do or do not contain this target. The subject presses one button if the target is present in a given array and a different button if the target is absent. Equal emphasis is placed on speed and accuracy, but reaction time (RT) is the primary dependent variable.

Figure 1.

Typical stimuli (A) and idealized results (B) from a guided search task. Subjects look for a red square with a gap at the top amongst red and blue distractors (indicated by gray and black in the print version of the figure). Set size is manipulated by varying the number of quadrants in which a cluster of items is presented, thus controlling stimulus density. Reaction times increase linearly as the set size increases. If a subject can limit search to the items of the relevant color, the slope of this function should be half as great when half of the items are drawn in this color than when all of them are drawn in this color.

The nontarget items in the array are called distractors, and the total number of items in the array (distractors plus target, when present) is called the set size. Half of the distractors match the target in terms of a highly discriminable feature, in this case color. This allows the subject to restrict attention to the items containing this feature (the red items in Figure 1). However, the search arrays are designed so that all items have equal bottom-up salience. That is, attention could be drawn just as easily to the blue squares instead of the red squares in Figure 1 if red were the color of the target. Thus, a subject will be able to limit search to items containing the relevant feature only if that subject possesses intact top-down attentional control mechanisms.

Overall RTs in this task do not provide a very precise measure of top-down attentional control, because they are influenced by many other factors as well, such as the speed of sensory and motor processes ^21,26. To isolate the efficiency of the search process, basic science studies usually manipulate the set size of the search arrays (e.g., 4, 8, 12, or 16 items per array). RTs typically increase linearly as the set size increases ⁴², and the slope of the function relating RT to set size provides a measure of how much time is spent searching each item in the array²⁶ (see Figure 1B). If a subject can limit search to the half of the items in an array that contain the target color, then the slope of the RT function should decrease by 50%. Thus, the slope of this function provides a good measure of the ability of an individual to use top-down attentional control mechanisms to guide attention to task-relevant objects.

Task Design and Analysis Considerations

Because RT is the primary dependent variable in this paradigm, it is important to ensure that speed-accuracy tradeoffs do not distort the results. In visual search tasks, subjects can be certain that they have found the target once they have focused attention onto it, but they are never really certain that the target is absent. Thus, subjects tend to search until they either find the target or “give up.” Subjects who give up early exhibit shallow RT slopes, but their error rates increase as the set size increases. This pattern has been observed in studies of healthy young adults ⁴³ and in studies of schizophrenia patients ^44,45. To solve this problem, the task can be modified so that two targets are possible and every array contains a target; the subject’s response indicates which of the two targets was present rather than whether a target was present or absent. For example, subjects could search for a red item with a gap on either the top or the bottom, pressing one of two buttons to indicate which of the two targets was present in the current stimulus array. This minimizes speed-accuracy tradeoffs in both healthy young adults ⁴⁶ and schizophrenia patients ^25,28.

RT distributions tend to be skewed, and outlier values are relatively common, and this makes it difficult to reliably measure an individual’s mean RT. Consequently, greater statistical power can be obtained by measuring median RTs rather than mean RTs ²⁶. Alternatively, outlier trials can be removed by means of an automated algorithm⁴⁷. To measure the slope of the function relating RT to set size, RT is typically measured from at least three set sizes. This makes it possible to ensure that the functions are actually linear. To avoid confounding set size with stimulus density, set size can be varied by presenting items in clusters and varying the number of clusters. For example, set sizes 4, 8, and 12 can be created by placing 4 items per quadrant and presenting stimuli in 1, 2, or 3 quadrants (see Figure 1A). The center-to-center distance between items is usually 150-200% of the diameter of each item, which avoids overlap between the stimuli and minimizes low-level lateral inhibition. In the absence of speed-accuracy tradeoffs and density confounds, the RT functions are usually highly linear. Once this has been demonstrated for a given group of subjects, it would be justifiable to test only two set sizes, which can reduce the duration of testing.

The resulting slope value will reflect both the ability of the subjects to restrict search to the items containing the appropriate feature and the speed with which the remaining items can be searched. To separately measure these factors, it is useful to include an unguided search condition in which all of the items contain the relevant features and must be searched. In the example shown in Figure 1, this could be done by including trials in which all items are red. Because top-down control cannot be used to limit search to a subset of items in the unguided search condition, the slope of the RT function in this condition provides a relatively pure measure of the speed of search. Perfect guidance of attention should lead to a 50% reduction in the RT slope for the guided search condition compared to the unguided search condition, and a smaller reduction provides evidence of specific impairment in top-down control.

Studies of visual search have found reliable differences between schizophrenia patients and healthy control subjects in experiments using three set sizes, approximately 50 trials per set size in each subject, and sample sizes of approximately 20 subjects per group^25,28. Each stimulus array is usually presented until the subject responds, followed by a brief intertrial interval (e.g., 500 ms). Short rest breaks may be given after every 10-20 trials. Given that patient RTs in this task are usually between 1 and 2 seconds, each trial requires an average of approximately 2 seconds, and it is possible to administer 50 trials at each of three set sizes in approximately 5 minutes (excluding rest breaks). More trials would be needed to obtain highly stable single-subject estimates of performance when the goal is to correlate visual search performance with other variables, such as outcome measures. More testing time would also be necessary if additional conditions are required (e.g., an unguided search baseline condition in which all items are the same color).

In addition, some time must be devoted to training each subject before usable data can be collected. This training typically begins by showing the subject the stimuli and pointing out the target, followed by a few trials in which the subject points out the target with no time pressure. Once the subject understands the task, some practice with making a speeded response to the target is necessary. Fifty trials of training (divided among the set sizes) is usually sufficient for RTs to stabilize. Accuracy should be very high, and any subjects who fail to achieve an accuracy of at least 80% correct should receive additional instruction and training before the start of data collection. Visual search tasks are usually very easy for patients to understand, so virtually all patients should be able to perform the task with this accuracy level after a few minutes of instruction and practice.

Guided search tasks can be programmed relatively easily in standard commercial and pen-source software packages, such as Presentation (http://www.neurobs.com/), E-Prime (http://www.pstnet.com/), and PsychToolbox (http://psychtoolbox.org/). The biggest challenge is randomizing the stimulus locations from trial to trial, while maintaining a minimized inter-item distance and avoiding density confounds (as described above). Some software packages make this relatively easy, but others do not. In some cases, it is necessary to write a program in a general-purpose programming language (e.g., Basic, Matlab) that creates bitmap images for each trial, and these images can then be displayed by the stimulus presentation package.

Cognitive Processes and Neural Systems

Guided search appears to rely on a target template that represents the features of the to-be-detected target. Psychophysical studies have shown that the target template is created rapidly when the subject is initially told the identity of the to-be-detected target ^48,49. Single-unit recordings from monkeys have provided evidence that this template is stored, at least in part, in inferotemporal cortex ^50,51 and that prefrontal cortex also plays a key role⁵². Neuroimaging studies in humans have also shown that prefrontal cortex is activated when subjects change the target template⁵³. These studies have examined the creation of the target template by changing the identity of the target from trial to trial. Under these conditions, the template is clearly stored in visual working memory⁵⁴. Under more typical conditions, in which the identity of the target (or targets) remains constant across trials, the template does not interfere with the storage of other information in working memory and is presumably maintained in a longer-term storage system⁴⁶.

Various visual search tasks have been studied in monkeys using both neurophysiological and behavioral methods^50,51,55-57, and it should be relatively easy to adapt the guided search task described above for use in monkeys. Visual search has also been studied extensively using event-related potentials in humans^58-60 and more recently in monkeys⁶¹, providing an opportunity for direct translation between human and animal models. Visual search has also been studied fairly extensively in birds^62,63, including pharmacological manipulations⁶⁴. However, given that the main visual pathway in birds is homologous to the mammalian superior colliculus rather than the visual cortex, pharmacological studies in pigeons may be of limited value in drug development efforts. Unfortunately, rodent research using tasks involving an element of visual search have been very different from the human visual search task⁶⁵, presumably because rodent visual systems are quite different from those of humans. However, it may be possible to develop a rodent version of the task that is more similar to the human task.

Deficits in Schizophrenia

Early studies of visual search in schizophrenia found that performance was relatively intact, as measured from the RT slope^44,45. As discussed above, however, these findings may have been distorted by speed-accuracy tradeoffs. More recent studies have found evidence of substantial increases in patient search slopes under conditions that emphasize top-down control of attention^25,28. One of these studies²⁵ used a guided search task much like that shown in Figure 1 and found a 70% slowing of search in schizophrenia patients compared with demographically matched control subjects, corresponding to an effect size of 0.81 (Cohen’s d). No significant impairment was observed in a task that was nearly identical but in which bottom-up sensory salience could be used to guide attention, making top-down guidance unnecessary. Thus, evidence exists for a patient deficit in the guided search task.

Future Directions

Although one study²⁵ has found clear evidence that schizophrenia patients are impaired in guided search, this result must be replicated and generalized to new stimuli and patient populations before it can be considered a robust finding. In addition, it is important for future studies to rule out the possibility that the previously observed impairment is a result of a deficit in top-down guidance rather than reflecting a general slowing of visual search. It would also be useful for future studies to examine the effects of varying the identity of the to-be-detected target from trial to trial, which should force subjects to rely more heavily on storing the target template in working memory. In addition, it would be useful for future patient studies to examine eye movements during visual search, which could make it possible to determine more directly the extent to which patients are unable to focus attention onto objects that possess the appropriate features.

The psychometric properties of guided search tasks are relatively unknown and deserve further study. Almost nothing is currently known about factors such as the stability of performance, the effects of repeated testing, the number of trials necessary to obtain good estimates of individual-subject performance, etc. We also know very little about the effects of pharmacological manipulations on guided search, and this would be a good avenue for future research. Finally, it would be useful to develop rodent analogs of visual search in general and guided search in particular. These analogs could potentially use other stimulus modalities as long as they involve the same top-down control over the selection of sensory inputs, because the control processes are likely to be similar across modalities⁶⁶.

The Distractor Condition Sustained Attention Task (dSAT)

Description

The guided search task clearly requires top-down control, because the subject must restrict attention to one of two sets of equally salient stimuli. However, more stress may be placed on the top-down control of attention by requiring subjects to attend to one stimulus set in the face of competition from an even more salient stimulus set. This is accomplished in the dSAT by requiring subjects to detect a weak target signal in the presence of a salient distractor signal and requiring continued discrimination between target signals and non-signal events.

The control of attention refers to the ability to guide and/or change the focus of attention in response to internal representations or goals. Thus, a task designed to test the control of attention would be expected to consist of two main components. First, a base task that assesses attentional performance based primarily on “bottom-up” processes, meaning that the salience of the signal primarily determines detection and discrimination performance. Second, a manipulation or addition to the base task that recruits cognitive or top-down control processes to maintain and/or recover performance in response to such a manipulation. Traditionally, such manipulations included additional and competing task demands, distractors, and/or demands on prolonged performance over time. The distractor-condition Sustained Attention Task (dSAT) incorporates these two conditions: It consists of a base sustained attention task (SAT) that entails some limited demands on top-down attention because it requires performance over prolonged periods of time, but which otherwise relies largely on bottom-up attentional processes (signal-driven attention to a sudden-onset signal). In the distractor condition (dSAT), a changing background reduces the discriminability of the signal, and instead subjects must exert increased top-down control to maintain performance. Such top-down control mechanisms include the filtering of the distractor, amplification of the representation of signals, augmentation of the processes associated with the discrimination between signal and non-signal events in the presence of a distractor that is presented in the same modality as the signal and, generally, sustaining motivated task performance under challenging conditions and decreased reward rates⁶⁷.

The SAT evolved primarily from attempts to design rodent analogues of human continuous performance tasks (CPT) ⁶⁸. Its advantages over “pure” signal-detection tasks included the inclusion of nonsignal trials to allow a differentiation between omissions and misses (and measurement of correct rejections and false alarms), a time-restricted response period more similar to human studies, and a randomized series of trial types to increase demands on sustained attention. (See discussion in ⁶⁹). The addition of a distraction condition (dSAT) was a further step towards the assessment of cognitive control³⁴.

The non-signal trials have had unexpectedly fortunate implications for research concerning the role of the cortical cholinergic input system in mediating task performance. In particular, when performing the task under standard (no-distraction) conditions, subjects appear to operate in a “default” or internally-directed mode biased towards the nonsignal trials, due in part to the fact that signal trials can be detected via largely bottom-up processing. When a signal appears, if it triggers a transient cholinergic response, it is detected and leads to a shift out of this mode. This shift is mediated by the cholinergic system; lesions of that system attenuate the detection of signals but spares non-signal trial performance⁷⁰, and recent evidence from experiments using enzyme-selective microelectrodes to record cholinergic activity at a subsecond resolution indicate that transient (on the scale of seconds) increases in cholinergic activity occur after successful signal detection. Thus, cholinergic lesions reduce the number of hits because they interfere primarily with such processing mode shifts⁷¹.

Rats

As the task was developed for rats and as a substantial amount of neurobiological and pharmacological evidence is based on research using rats, the rat version of the SAT/dSAT will be described in some detail, allowing a more restricted description of parametric and procedural differences for use in humans. Figure 2 illustrates the main components of the task, the main measures of performance and the effects of a distractor on these measures. Data shown were taken from Demeter, Sarter, & Lustig³³.

Figure 2.

Illustration of the main events (a), response categories and outcome (b) of the SAT as used in rats and illustration of performance data from SAT and dSAT sessions (c-h; base task performance: black lines, squares; distractor condition: orange lines, triangles; data from Wistar rats; the presence of the distractor in blocks 2 and 3 is indicated by the oranges block on the abscissa). The performance in signal and non-signal trials is collapsed into one measure of performance, the SAT/dSAT scores (Figure 2h). SAT/dSAT scores are calculated for each signal duration (e.g., 500, 50, 25 ms; SAT/dSAT_500,50,25) on the basis of the relative number of hits (h) and false alarms (f), in accordance with this formula: (SAT/dSAT=[(h-f)/2(h+f)-(h+f)²]) and then averaged over all signal durations, yielding a single overall score (Figure 2h). The formula is a variation of the non-parametric calculation of signal sensitivity (SI; Frey & Colliver, 1973). SI is based on the probabilities for hits and false alarms. In contrast, the calculation of SAT/dSAT scores is based on the relative number of hits and false alarms, thereby removing the confounding effects of omissions from this performance measure. SAT/dSAT scores range from -1 to +1. Values of 0 indicate randomized lever section, +1 indicates perfect response accuracy in signal and non-signal trials, and -1 indicates complete inaccuracy. The scores shown in Figure 2h are averaged over all signal durations and depicted by block to visualize the contrast between SAT and dSAT scores.

The task consists of a random sequence of signal and non-signal events that occur unpredictably following a variable intertrial interval (ITI). Following a signal (center panel light illumination for 500, 50 or 25 ms) or a non-signal event, levers are extended into the chambers 2 s later and remain active for 4 s. In the version illustrated in Figure 2, panel a, following a signal, a left lever press is counted as a hit and rewarded (water port situated between the levers; not shown). Following a non-signal event, a right lever press indicates a “correct rejection” and is also rewarded. Incorrect responses are misses and false alarms, respectively, and trigger an ITI. An error of omission is defined as the failure to operate a lever within 4 s. Note that arrows indicating the 4 responses are color-coded to match the arrows in the outcome matrix in b. Top-down control of attention is tested by the presentation of a distractor, requiring the recruitment of top-down effects designed to stabilize residual attentional performance. The performance data shown in c-h depict performance over blocks of trials as, in this example, the distractor (chamber houselights flashing on/off at 0.5 Hz) was presented during the 2nd and 3rd of a total of five 8-min blocks of trials. As illustrated in c and d, the distractor reduced the relative number of hits to 500 and 50 ms signals both acutely and during subsequent blocks. This suggests that as a result of distractor presentation, animals were not able to sustain levels of signal detection during the remainder of the task. For hits to shortest signals (e) floor effects limited the manifestation of even more robust detrimental effects of the distractor. In contrast to the effects of the distractor on hits, the ability to respond correctly in non-signal trials was acutely disrupted by the distractor but completely recovered thereafter (f). Errors of omission were not robustly affected by the distractor and generally remained below 10% of all trials, indicating the persistent motivation of the animals to stay on task and to regain performance in response to distractor challenges. The SAT/dSAT scores shown in h illustrates that this distractor robustly impaired overall performance and continued to impair performance during trial blocks that followed the period of distractor presentation.

Humans

Validation experiments comparing human and rat performance are reported in Demeter et al. (2008) ³³. Human experiments are conducted on a standard PC, and the signal consists of a small (3.5 mm²) dark grey square against a lighter grey background. (E-prime software (Psychology Software Tools) was used in the Demeter et al. studies³³, with the standard “grey” color used for the signal, and the standard “silver” color used for the background.) For the standard condition, the background is static; for the distractor condition, it rapidly alternates (10 Hz) between silver and black for the distractor condition. Headphones are used to deliver auditory cues and feedback, and the standard keyboard is used to collect responses. Participants are familiarized with task instructions and trained on the sustained attention task under standard conditions for 30 s and under distractor conditions for 30 s. Practice is repeated until the participant reaches ≥ 60% accuracy in the standard condition practice. (Note that actual performance is much higher, Figure 3).

Figure 3.

SAT/dSAT performance of healthy human volunteers. Except for different signal durations and longer trial blocks, this figure allows direct comparisons with the data from rats illustrated in Figure 2 (data taken from Demeter et al., 2008; see legend of Figure 2 for more details).

Each trial requires participants to detect the presence or absence of a signal (small grey square) of varying durations (17, 50, or 100 ms). One hundred milliseconds after the occurrence of a signal or nonsignal event, the response period is cued by a 75 ms low-frequency buzzer. Parallel to the rat task, participants are reinforced for pressing one key for signal trials and the other key for nonsignal trials (‘z’ key for left-hand responses; ‘/’ key for right-hand responses, left-right assignments to signal or nonsignal trials counterbalanced across participants). In our experiments, participants received one cent for every percentage point of overall accuracy for each run ($1 maximum per run). A 75 ms high-frequency feedback tone follows correct responses. No feedback is given following incorrect trials or omissions (failures to respond within 1 s after the response buzzer). Within each run, intertrial interval, trial type (signal or nonsignal), and signal duration are varied in a pseudo-random order with an equal distribution across trials.

At the parameters tested, human performance levels were superior to those of rats, exhibited more limited impairments to the effects of the distractor, and recovered more effectively during post-distractor periods. The pattern of distractor effects on false alarms paralleled that seen in rats (Figure 2). The dSAT scores (Figure 3f; see Figure 2 legend for calculation) showed a large effect of distraction but also showed strong post-distraction recovery, the latter of which remained incomplete in rats.

Signal detection measures (d’ and response bias) were calculated to determine the degree to which impairments in performance during the distraction condition were related to changes in perceptual sensitivity (d’) and to examine potential differences in response bias between the two species. For rodents, the perceptual sensitivity measure was near zero for shortest signals, forming a “floor” that limited the effects of the distractor on the detection of shortest signals. Sensitivity in the other two durations was low but above zero. Humans showed a distraction-related drop in sensitivity at all signal durations but maintained reasonable sensitivity even for the shortest duration.

The response bias measure indicated that both species were more conservative at shorter signal durations. Rodents were more conservative than humans overall, and showed a shift towards becoming more liberal under distraction, whereas humans showed a shift toward a more conservative criterion. The latter effect was amenable to changing reward contingencies: A manipulation of the payoff matrix to penalize misses resulted in humans showing a liberal shift in response to distraction, similar to the rats. This pattern substantiates the assumption that performance under distraction is influenced by top-down control. Measures of perceptual sensitivity did not change as a result of the manipulation in reward contingencies.

Recommendations for future use in studies with patients and normal controls

As humans did not show a significant post-distractor impairment, the task may be shortened to include only one post-distractor block, rather than the 2-3 blocks used by Demeter et al. (2008)³³. (It is still desirable to include at least one post-distractor block to demonstrate such performance recovery, as impaired top-down control may manifest by retarding post-distractor performance recovery.) Event rate (ITI) did not affect human performance on the standard task and thus the slow event-rate run may be eliminated as well. However, these statements come with the caveat that we do not yet know whether these factors would have stronger effects in a patient population. As noted below, experiments to address this issue and other issues directly related to patient testing are just beginning.

For certain (patient) populations it may be necessary to titrate the stimulus durations so that performance in the standard condition during the 30 s practice blocks is similar to that of controls, and then apply those same duration parameters to the distraction blocks. This strategy may allow a better estimate of potential differences in the distractor effect across groups, reducing potential contamination from baseline differences in controlled-attention demands in the base task. However, it is important to continue to use a range of stimulus durations, not just one. The use of variable stimulus durations increases uncertainty and helps prevent the possibility that subjects will “tune” to a particular stimulus duration and thus enhance the ability to detect it (see also the section on validity). An important safety concern is that potential participants should be carefully screened for seizure disorders or migraines, as the strobe-like nature of the distractor may trigger the symptoms of these disorders.

Construct Validity

The SAT may meet our initial definition of a base task (i.e., largely driven by bottom-up processes and signal salience) better for humans than for rodents. Humans show good (though duration-dependent, as would expected if performance is dependent on signal salience) performance in this condition, maintain strong performance over time, and recover performance in the SAT quickly after exposure to distraction. There are some demands on top-down control related to task engagement, remembering the correct responses for each trial type, and so on, but they are minimal due to the environmental constraints (i.e., subjects are seated directly in front of the screen with their fingers on the response keys). For rodents, the task may require relatively more top-down processes even in the standard condition, as they must inhibit other behaviors (e.g., grooming) or environmental distractions. Also, rodents presumably have much reduced top-down control ability compared to humans, making even small demands relatively greater (i.e., the same demand may require a greater percentage of top-down control abilities for rodents than for humans). These relatively greater demands on top-down control for rodents may explain why they show time-on-task-related decreases in performance and have greater difficulty recovering performance after the distractor.

Both species show performance declines in response to the distractor condition (dSAT). Humans are less severely affected than are rats, consistent with the idea that the task makes demands on top-down control and that humans have greater top-down control capabilities. The idea that performance decrements are due to increased demands on top-down attention rather than some other reason (e.g., disruptions in bottom-up processing or general disruption and loss of task set) is supported by evidence on several fronts, as described below.

Given that the distractor acts by increasing perceptual difficulty, the first question is whether the performance decrements reflect increased demands on top-down control or whether they simply make the signal relatively less salient and therefore affect primarily bottom-up processes. There are several reasons to discount this alternative. First, signal-detection analyses showed that the distraction condition influenced both response-bias (β) and perceptual sensitivity measures (d’) in both species³³. Furthermore, for humans, changes in the reward contingencies (to penalize misses) resulted in a shift in top-down bias, but not in the measure of perceptual sensitivity. These shifts were isolated to the dSAT condition, again consistent with the idea that human SAT performance is driven primarily by bottom-up processing whereas the dSAT makes demands on top-down control. Finally, performance in the dSAT condition is associated with frontal-parietal attention networks in both species, as described in more detail below. Taken together, these results argue for the idea that dSAT performance is related to top-down attention, and we continue to perform further experiments to validate this idea (e.g., examining the effects of imposing other demands on top-down attention).

The lower performance overall of rodents versus humans, and the greater vulnerability of rodent performance to distraction effects, raises the question of possible differences in motivation or engagement with the task. In particular, one might be concerned that distraction generally disrupts task engagement for the rodents, given that performance is lower even after the distractor. However, Demeter et al.³³ found that omissions (failures to respond) were low for both species (2.5 % of trials for rats; 1.6 % of trials for humans) and did not differ across conditions. Again, although the SAT likely imposes greater relative demands on rodents than on humans, both species continue to perform in a motivated fashion. A trend towards recovery in the last post-distractor block is also consistent with the idea that poor performance by rats is due to an exhaustion of attentional resources during the distractor period (followed by recovery over time) rather than an overall loss of the task set. Finally, as described below, distraction and post-distraction recovery appear to rely heavily on brain systems associated with attention.

Neuronal Systems

Evidence from research in animals

There is extensive evidence that the cortical cholinergic input system (including its connections to frontal and parietal regions) is central for both SAT and dSAT performance. This evidence accrues from experiments that selectively lesion of the cortical cholinergic input system, from systemic or intracranial administration of drugs that modulate cholinergic neurotransmission, from microdialysis measurements of cortical acetylcholine (ACh) release in task-performing animals, and most recently from enzyme-selective microelectrodes that monitor ACh release at a subsecond resolution (for reviews, see67,71,72). Cholinergic contributions may occur both at a “state” or task-set level and at an “item” or individual-trial level. At the state level, lesions of the prefrontal output circuitry completely randomize lever selection in the base task regardless of trial type, suggesting that these regions are important for response-rule processing and/or execution. On the individualtrial level, recent evidence from electrochemical recordings suggests that transient cholinergic activity in prefrontal regions following a signal mediates processing mode switches from internally-directed processing modes, which seem to be dominant during non-signal trials, to the signal-evoked implementation of detection processes. In other words, if the appearance of a signal triggers the cholinergic transient, it sets in motion conscious perception, pre-attentional and attentional processing of the signal, and the response. Accordingly, cholinergic lesions produce misses predominantly in signal trials that follow non-signal trials, indicating lesion-induced disruption of processing mode switches. Relevant to drug development, although we know that removal of the noradrenergic projection system has little effect on base task performance, there is a paucity of information about the role of interactions between these different neuromodulators, particularly in prefrontal regions for SAT and dSAT performance⁷³.

Evidence from research in humans

Neuroimaging data collected from humans while performing the SAT and dSAT are currently being analyzed. Each task condition was contrasted with a baseline fixation condition that attempts to control for the visual-input differences between the SAT and dSAT (i.e., the baseline condition for the SAT condition uses a static black background, the baseline condition for the dSAT condition uses a “flashing” background that alternates between dark and light at the same frequency as during the dSAT task)

Preliminary analyses indicate that performance in the SAT is associated with activations in sensory and motor cortices related to the detection and response to the signals and task cues (auditory cues are used to indicate the response period and to give feedback following correct responses), along with small activations in right medial frontal gyrus, medial prefrontal cortex, and left putamen. The dSAT additionally shows increased visual activations, right parietal activations, and extensive right prefrontal activations. The dSAT-related right frontal regions remain active even after controlling for both the basic task demands imposed by the SAT and the visual stimulation provided by the flashing screen in the dSAT condition. Overall these patterns fit quite well with the predictions made from the rat work, particularly with regard to increased recruitment of prefrontal regions in the dSAT condition.

Animal Models

The long-term attentional consequences of prior exposure to an escalating dosing regimen of amphetamine has been used in rats to model the key cognitive symptoms and neurobiological symptoms of schizophrenia^74,75. Following this treatment regimen, SAT performance remains unaffected. However, subsequent administrations of small doses of amphetamine that do not affect the performance of control animals, and that are though to model acute disease periods, severely disrupt SAT performance of these pre-treated animals⁷⁶. Moreover, performance-associated cholinergic activity is severely attenuated⁷⁷. Importantly, these reductions in cholinergic activity in response to a small amphetamine challenge are not observed in non-performing animals, supporting the view that research on the neuronal mechanisms underlying the cognitive symptoms of schizophrenia requires the measurement of neuronal mechanisms in the presence of appropriate cognitive demands⁷⁸. We also found that the seemingly intact performance of amphetamine-pretreated animals in the absence of challenges requires abnormally high levels of cholinergic activity and that the performance of these animals is extremely vulnerable to distractors. Thus, this animal model reproduces the vulnerability of attentional performance of schizophrenic patients to demands on top-down control (for details see79).

Although clinically-effective cognitive enhancers are not available for schizophrenia, low-dose treatments with haloperidol and second-generation antipsychotic drugs produce moderate benefits that can be reproduced in the amphetamine pre-treated rat model (for references to relevant clinical studies and definition of “low dose”, see80). Such treatment with haloperidol, and with slightly greater efficacy, clozapine, attenuates the attentional disruption caused by amphetamine challenges in this animal model.

Performance in Schizophrenia

An earlier version of the base SAT task was assessed in a small trial; results indicated that in the absence of a distractor, medicated patients produced more hits than controls, consistent with suggestions that in appropriate contexts, patients may exhibit aspects of hyperattention or hypervigilance⁸¹. We have begun to collect data from schizophrenic outpatients using a SAT and dSAT procedure similar to that used to validate the task in healthy adults³³. Preliminary data suggest a severe disruption of dSAT performance in schizophrenia outpatients (Guthrie, Taylor, Sarter & Lustig, unpublished).

Psychometric Data

The effects of signal duration, event rate, intertrial interval variability, and variation of outcome for errors in signal trials on performance of the base task and the dSAT are described above. Internal consistency is generally quite good, with the exception of rat performance in the distractor condition. Cronbach’s alpha values for the data presented in Demeter et al.³³ are: Rat SAT = 0.83, rat dSAT = 0.24; human equal-reward contingencies SAT = 0.93, human equal-reward contingencies dSAT = 0.88; human penalize-misses reward contingencies SAT = 0.86, human penalize-misses dSAT = 0.93. The low reliability of the rat distractor condition likely reflects the floor effects in that condition (see Figure 2). Of interest, when the rat pre- and post-distractor SAT blocks are examined separately, they show similar levels of internal consistency (0.86 vs 0.84). Additional studies will be needed to establish test-retest reliability, convergent and divergent validity with other tests, and to examine psychometric properties in patient populations.

Future Directions

As mentioned above, the assessment of dSAT performance in schizophrenic patients is paramount and will guide additional experiments in human volunteers in order to optimize task parameters. Measures of reliability and validity with this patient population will be obtained to expand upon those already obtained with our normal control subjects. Additional studies with both humans and rats are planned to more precisely characterize the role of bottom-up versus top-down attention in the SAT versus dSAT in these species. Of particular interest is the question of whether the SAT task is proportionally more demanding on top-down attention in rats than in humans, and whether this drives the species difference in post-distractor recovery.

Summary and Future Developments

The tasks nominated as measures of control of attention were judged to vary in their construct validity for this aspect of attention, in the clarity of the underlying cognitive mechanisms and relevant neural circuitry, in their applicability in animal and human research, and in the strength of current evidence for detecting key impairments in schizophrenia. All were viewed as useful attempts to differentiate more fine-grained aspects of attention, with the Guided Search paradigm and the Distractor Condition Sustained Attention Task emerging as showing strong construct validity for control of attention and the best balance of other desirable characteristics for further cognitive neuroscience work that would develop their potential for treatment research.

Both of the selected paradigms have strong construct validity and are clearly amenable to neuroimaging and psychophysiological research to further delineate the relevant mediating neural circuits. The Guided Search paradigm is more fully developed as a human research tool at this time and has already shown clear evidence of detecting and differentiating a key cognitive deficit in top-down control of attention in schizophrenia. It would profit from use in neuropharmacological research to examine its sensitivity to drug effects and from development of an animal analogue. The Distractor Condition Sustained Attention Task is very well developed for animal research and has clearly been shown to be sensitive to inputs of cortical cholinergic systems. A human task version has recently been developed but needs more extensive testing. An initial study to demonstrate that it detects a deficit in schizophrenia patients has just been completed.

In addition to the task-specific areas for further development, all tasks nominated in this cognitive domain were viewed as having relatively unexplored psychometric characteristics for clinical treatment research, particularly test-retest reliability, and none of the measures had yet been examined for the strength of their predictive association with functional outcome in schizophrenia. Strong psychometric characteristics and demonstrated connection to functional outcome in schizophrenia research were primary considerations in the recent selection of cognitive measures for the MATRICS Consensus Cognitive Battery for clinical trials^18,19, and are critical areas for future development for the more differentiated cognitive paradigms arising from cognitive psychology and cognitive neuroscience. As is typical of paradigms drawn from cognitive psychology and cognitive neuroscience, initial development focuses on task properties that allow fine-grained and valid distinctions among cognitive processes at the group contrast level. Adaptation for use in clinical trials requires consideration of ways to enhance test-retest stability, avoid ceiling effects with repeated administrations, and ensure relationships to real-life functional outcome, while at the same time maintaining the high level of construct validity for measuring discrete cognitive processes that is one of the initial virtues of such paradigms.

As the Guided Search and dSAT paradigms are developed further, it would also be useful to examine the extent to which their performance is intercorrelated, as this would shed light on the extent to which there is commonality in the top-down attentional control mechanisms that they measure. Similarly, both of these paradigms emphasize top-down control of attention at the input selection level. Control processes in later components of processing were considered in the CNTRICS initiative primarily under the rubric of executive control processes⁸². Given that distinctions between the nature of attentional control at the input level and attentional control in later response selection and response production phases of processing have sometimes been prominent in other paradigms from cognitive psychology^27,83, it would also be very helpful for future research to address whether deficits in schizophrenia at these two levels are discrete or share common cognitive and neurobiological mechanisms.

Though there is a substantial amount of work to be done to develop these cognitive neuroscience measures of attention into measures for human and animal research to aid development and evaluation of new treatments for deficits in schizophrenia, the CNTRICS working group on attention was enthusiastic about the promise of using paradigms from cognitive psychology and cognitive neuroscience to delineate more precisely the core cognitive mechanisms underlying attentional deficits in schizophrenia and to link these deficits more clearly to particular neural mediators.