CNTRICS final animal model task selection: Control of attention

Abstract

Schizophrenia is associated with impaired attention. The top-down control of attention, defined as the ability to guide and refocus attention in accordance with internal goals and representations, was identified by the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) initiative as an important construct for task development and research. A recent CNTRICS meeting identified three tasks commonly used with rodent models as having high construct validity and promise for further development: The 5-choice serial reaction time task, the 5-choice continuous performance task, and the distractor condition sustained attention task. Here we describe their current status, including data on their neural substrates, evidence for sensitivity to neuropharmacological manipulations and genetic influences, and data from animal models of the cognitive deficits of schizophrenia. A common strength is the development of parallel human tasks to facilitate connections to the neural circuitry and drug development research done in these animal models. We conclude with recommendations for the steps needed to improve testing so that it better represents the complex biological and behavioral picture presented by schizophrenia.

Every moment, we are bombarded with stimuli competing for our attention. As you read these words, your eyes may also take in other items on your screen as your ears funnel in the background hum of your computer and the conversation of someone walking by. Perhaps your stomach sends you a reminder that it’s been a while since lunch, or you absentmindedly scratch an errant itch on the back of your neck. Keeping your focus on this paper, and shifting your focus as appropriate (e.g., if you have a meeting in the next few minutes, putting the paper down to prepare for that meeting), both require the goal-driven, top-down control of attention.

This process of input selection – the ability to guide and/or change the focus of attention in response to internal representations – was identified by the CNTRICS (Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia) group as a core cognitive domain representing the control of attention (Luck & Gold, 2008; Luck et al., in press; Nuechterlein et al., 2009). The general term attention has been used to describe such a wide array of processes that almost any performance deficit in schizophrenia could be ascribed to a problem with attention. CNTRICS chose to focus on the control of attention, specifically input selection, a well-defined construct that is impaired in schizophrenia (Gold et al., 2007) and somewhat distinct from other domains. Although they certainly interact at the task level, at the construct level input selection is distinguished from rule selection - the process of choosing which rules to use in responding to selected inputs. Rule selection is instead included under the domains of executive function and working memory.

Within the construct of input selection, the CNTRICS emphasis is on the control processes that choose which inputs should be in the focus of attention. These control processes are distinct both behaviorally and neurally, localized mainly within prefrontal and parietal regions, from the implementation processes that amplify or suppress the representations of those inputs and that are localized in more posterior regions (see discussion by Luck et al., in press). Importantly, control processes are impaired in schizophrenia while implementation processes are relatively spared: Patients show poor performance in antisaccade tasks, are differentially disrupted by invalid cues, and have more difficulty than controls in conjunction-search tasks in which a top-down representation is required to distinguish the target from distractors. However, they frequently perform as well as controls in prosaccade tasks, in response to valid cues, and when the target is distinguished by salient bottom-up features (see discussion by Luck et al, in press).

The fourth meeting of CNTRICS II aimed to identify animal cognitive neuroscience paradigms to be developed for research on the treatment of attention and other cognitive deficits in schizophrenia. In choosing these tasks, the highest priority was given to construct validity, both in terms of the ability of the paradigm to specifically measure the process of interest and evidence that it recruited the neural systems thought to be critical for that process and impaired in schizophrenia. Reliability and the ability to standardize the paradigm across laboratories were also major concerns. Given the strong emphasis on construct validity, it is perhaps unsurprising that the three paradigms chosen for this construct – the 5-choice serial reaction time task (5CSRTT), the 5-choice continuous performance task (5C-CPT), and the distractor condition sustained attention task (dSAT) - all have their roots in continuous performance tasks that have long been associated with behavioral and neural impairments in schizophrenia (Cornblatt & Keilp, 1994; Volz et al., 1999). Before describing these paradigms, we turn briefly to the other nominated tasks.

Other nominated tasks

Tasks considered but not chosen for further development in the attention construct were Multiple Choice Reaction Time tasks, Stroop (Stroop, 1935), and the R+A−/R+A+ variants of the active place avoidance task (Cimadevilla et al., 2000). The nomination for the Multiple Choice Reaction Time Task was seen as a broader recommendation for the 5CSRTT (Robbins, 2002; see Bushnell & Strupp, 2009); which was selected. Stroop was designated in earlier CNTRICS meetings as a prototypical measure of rule selection rather than input selection (Luck & Gold, 2008), and concerns were raised about its alternate-forms reliability (Shilling et al., 2002). The R+A−/R+A+ was considered an interesting paradigm for the investigation of hippocampal function. However, its construct validity for assessing input processes was unclear, and no evidence was presented for its recruitment of the fronto-parietal systems typically associated with input selection.

Tasks selected for further development

As stated previously, the three tasks selected for further development were the 5CSRTT (see Robbins 2002 for a review); the 5C-CPT (Young, Light, Sharp, & Geyer, 2009), an elaboration of the 5CSRTT; and dSAT (McGaughy & Sarter, 1995; Nuechterlein et al., 2009). All of these have their roots in traditional continuous performance tasks (CPT) that emphasize identifying and responding to targets. Alternate CPTs that place the major burden on context processing and/or the inhibition of responses to nontargets (e.g., Cohen et al., 1999) may be more relevant to the working memory and executive control constructs.

It was observed during the discussion that the three tasks had common features. In particular, although all can be used to assess a variety of attentional functions, they share the requirement to select a target input in the face of competing inputs. In the two 5-choice tasks, this is operationalized quite literally in the number of “choices” or possible spatial locations where the target signal could appear. Varying the number of possible signal locations impacts attentional load and performance; the 1-choice variant of the 5CSRTT is faster to acquire and yields better performance than the 5-choice version (see also Passetti et al., 2000). In the dSAT, signal location typically remains constant (though see below), and demands on input selection arise to some degree from unpredictability in signal occurrence, timing, and duration. However, in this task demands on the input selection process may be most effectively isolated by contrasting the standard (SAT) condition with the distractor (dSAT) condition, which requires participants to detect the target input (“signal”) against a very salient distracting input (rapidly-changing background illumination).

It was suggested that in the future, the three tasks might be further integrated into one task. There have been some preliminary steps in this direction; for example, some variants of the 5CSRTT further increase demands on input selection by making target presentation temporally unpredictable or by adding distraction (Amitai & Markou, 2011; Carli et al., 1983; see also Bari et al., 2008). Conversely, a version of the dSAT has been developed that adds unpredictability to signal location (Demeter, 2011). This convergence in methods and in the behavioural outcomes of these manipulations substantiates the strong construct validity of the tasks, but should be further tested by exploring the convergence (or lack thereof) of results across the tasks from pharmacologic manipulations, lesion studies, and neuroimaging. Therefore the general agreement was that currently, each task has both strengths and areas in need of further development and independent testing. Below, we focus primarily on research with rodents; data from other species including healthy humans and patients with schizophrenia are described if available.

5-choice serial reaction time task (5CSRTT)

Task description

For rats (Carli et al., 1983) and mice (Humby et al., 1999), the requirement is to detect a brief visual target (visual cue) presented unpredictably in a 5 choice array. The rodent ‘nose-pokes’ the locations of presentation within apertures monitored by an infra-red photocell beam. Correct responses are reinforced by food presented at the rear of the apparatus. Incorrect responses receive a time-out from positive reinforcement and a period of darkness. Performance is measured in terms of accuracy (% correct), number of correct targets,, and response latencies. Omissions and magazine latency (latency to collect earned food pellets) act as internal control for motivation level and ability to perform the task. Premature responses (i.e. prior to target presentation) are defined as impulsive responses. Repetitive responses in the nose-poke holes after a trial are measured as perseverative (‘compulsive’) responses. Detailed descriptions of the task, protocols and variations are provided in two papers (Bari et al., 2008; Robbins, 2002; see also Figure 1, Video 1).

Figure 1.

The 5 Choice Serial Reaction Time Task (5CSRTT). Adapted from Dalley et al. (2001) and Robbins (2002). The animal’s task is to respond to a brief visual cue with a nose-poke at the location of the cue; reward is collected from a separate food magazine at the rear of the cage. See text for details.

The 5CSRTT, already widely used in drug development research, is based on a test of sustained attention or continuous performance of the same name originally developed for humans (Wilkinson, 1963). It has been used with both healthy and clinical populations as part of the CANTAB battery (e.g., Alexander et al., 2005; Sahakian et al., 1993; Cambridge Cognition, camcog.com). In addition to the rat and human tasks, versions have been developed for mice (Humby et al., 1999) and nonhuman primates (Weed et al., 1999).

Neural substrates

There is a very extensive literature on the neural substrates of 5CSRTT performance in rats (see reviews by Chudasama & Robbins, 2004a; Robbins, 2002). Briefly, different parameters of task performance are affected dissociably by excitotoxic lesions of different sectors of the rat prefrontal cortex. For example, accuracy is quite selectively affected by lesions of the dorsal cingulate cortex, whereas more ventral lesions tend to affect measures of response control such as premature responding; orbitofrontal lesion increase perseverations (Chudasama et al., 2003). There are also parallel data on effects of lesions of different sectors of the striatum, as well as asymmetrical lesions of fronto-striatal systems to define the relevant neural systems circuitry underlying performance. There are also published data on effects of lesions of the subthalamic nucleus, pedunculopontine nucleus and hippocampus. In humans, the 5CSRTT has been used to characterize regional effects of prefrontal cortex damage (Alexander et al., 2005).

There are extensive data on neurochemical manipulations of the dopamine, noradrenaline serotonin and acetylcholine neurotransmitter systems (see Robbins 2002), as well as interfering with glutamatergic transmission via intra-cerebral infusions (e.g. Murphy et al 2005) and chronic PCP administration. Some studies have used intra-cerebral microdialysis (Dalley et al., 2001) and microPET (Dalley et al., 2007) to obtain neurochemical correlates of performance. Performance parameters have been linked to biochemical and structural magnetic resonance variables (Caprioli et al 2010). New unpublished data use neuroprobes in freely-moving rats to obtain electrophysiological biomarkers of task performance (e.g. impulsive responses). Deep brain stimulation of the subthalamic nucleus has been shown to influence performance (Baunez et al 2007).

Task parameters and variants

Given the wide use of the 5CSRTT, there are many manipulations and variants. As mentioned earlier, these include variations of target duration and brightness, reduced temporal predictability (varying the normal fixed 5s ITI), introduction of white noise distracters interpolated unpredictably in the ITI, and changes to the rate of presentation of visual targets (sustained attention), number of trials (vigilance paradigm), or number of response options (‘one choice task’, Passetti et al., 2000). Recently, Robinson (2012) introduced training under unpredictable inter reaction time interval (IRTIs) to enhance sensitivity to accuracy improvement. The task has also been adapted to include a working memory variant (Chudasama and Robbins 2004b). Furthermore, it is currently being modified for use on touch-sensitive screens (e.g. (Romberg et al., 2011) to implement a direct analogue of the human AX-CPT continuous performance test (Mar and Robbins, personal communication).

Sensitivity to pharmacologic manipulations and genetic influences

This task has been widely used in pharmaceutical settings. Approximately 200 papers using the 5CSRTT have been published since Robbins (2002) exhaustively reviewed the literature published to that point; most of these employed manipulations of pharmacologic, genetic, or other parameters. Virtually every major pharmacological class of drugs has been tested, either systemically or intra-cerebrally, including standard reference compounds such as scopolamine (muscarinic antagonist), anticholinesterases, nicotine and mecamylamine (nicotine antagonist), amphetamine, methylphenidate and modafinil, monoamine reuptake blockers (NA- and 5-HT-), dopamine receptor antagonists, NMDA receptor antagonists (MK-801, PCP), 5-HT receptor agents and benzodiazepines (Robbins & Mar, review in preparation).

A brief survey found 17 post-2002 reports of drugs producing performance enhancement, primarily on accuracy measures. The baseline or comparison condition varied across studies, including normal performance by healthy animals, poorly-performing animals, or following degradation of performance by task parameter manipulation or brain lesion, pharmacological treatment or sleep deprivation, Drugs producing performance enhancement, though often under specific conditions, have included the following: D1 dopamine receptor agonists (Granon et al 2000; Chudasama and Robbins 2004, but see also Passetii et al 2003), donepezil (Balducci et al 2003; Lindner et al, 2006; Romberg et al 2011), M2 muscarinic, agonist (Terry et al 2010), alpha-7 and alpha-4, beta4 subunit selective nicotinic receptor agonists (Grottick et al 2003; Mohler et a 2010; Semenova et al 2007), ciproxifan (Ligneau et al 1998), sertindole (Carli et al 2011b), aripiprazole (Carli et al 2010b), clozapine (Amitai and Markou 2007), methylphenidate (Paine et al 2010), adrenergic agents (Jakala et al 1992; Puumala et al 1997) reboxetine (Liu et al 2009) and atomoxetine (Robinson 2012). The 5CSRTT has also been used to study performance in genetically modified mice including a genetic model of attention deficit disorder (Davies et al., 2009) or other impulse-control dysfunction (Isles et al., 2004; Lambourne et al., 2007; Oliver et al., 2009; Yan et al., 2011); a triple-transgenic 3×Tg-AD mouse model of Alzheimer’s Disease (Romberg et al., 2011) and alpha 7 nAChR mutant mice (Hoyle et al., 2006; Young et al., 2004). It has also been used with these genetic models to detect cognitive enhancing effects of drugs (e.g., Romberg et al 2011).

Sensitivity to manipulations designed to model schizophrenia

The 5CSRTT is a behavioral test that captures some of the attentional impairments that may be relevant to schizophrenia. So far it has predominantly been employed in pharmacological or neurodevelopmental rat models. For example, the 5CSRTT has been used to evaluate the MAM model, and the effects of intracerebral or subchronic NMDA receptor blockers such as PCP in rats and mice (e.g. Amitai & Markou, 2010; Carli et al 2011; Paine and Carlezon 2009; Pozzi et al., 2010; see also Chudasama and Robbins 2004a).

Psychometrics

An informal survey of the numerous articles on the 5CSRTT shows that it appears to have excellent psychometric properties in well-trained animals. Within-animal variance is very low (often about 1%) in most of the dependent variables,, so that test can be used in multiple experiments where the baseline levels of performance need to be re-established. Between-animal variance is also quite low (usually < 5%), although it is possible to form groups of “poorly-performing” or “high impulsive rats” within the natural variations in performance. It is, despite claims to the contrary, entirely feasible to employ signal detection analysis of performance, with only a few assumptions, which may provide enhanced sensitivity, especially for accuracy improvements following drug treatment (Mar & Robbins 2011) The measures generally have a wide dynamic range (as chance accuracy performance is 20%). Ceiling effects in very well-trained animals can be offset by manipulations of visual target parameters such as reducing stimulus duration, adding distraction or increasing the ITI.

Suggestions for further optimization and development

The 5CSRTT is already widely used in industry and might be considered a “mature” task. However, innovation and development continue. The speed of training, currently reduced to 3–4 weeks, might be further optimized; optimization of other training parameters is also being explored. The distractibility manipulation (interpolation of bursts of loud white noise in the ITI; see Carli et al., 1983) suffers from rapid habituation effects and this may be addressed by the development of a touch-screen version, ‘continuous performance (AX-CPT) version, which will also be able to employ targets with distinct identities, with the aid of computer-generated stimuli. A computerized version is being developed for humans that simulates the animal 5CSRTT.

Best stage for use in drug discovery

The 5CSRTT is best employed in the characterization of mouse or rat phenotypes and in the evaluation of drug effects relevant to improving sustained attention and controlling impulsivity in preclinical studies. It translates to tests of continuous performance frequently employed to evaluate executive performance in patients with schizophrenia in experimental medicine studies.

The sensitivity of the 5CSRTT to a variety of pharmacologic manipulations and to manipulations designed to model schizophrenia may lead to the question of why research with this task has not yet led to development of effective pro-cognitive treatments in clinical trials. While impatience on this front is understandable, given the current state of research the question is somewhat premature. In the final section of this paper we discuss the issues that must be considered in translating from pre-clinical to clinical research with any of the CNTRICS (or other) tasks. Here we describe the evidence most relevant to the 5CSRTT in particular.

A first consideration is whether the pro-cognitive effects of a manipulation have been observed in the appropriate animal model of schizophrenia-related dysfunction, not just in healthy animals. The MAM and chronic PCP models produce schizophrenia-like deficits on the task but positive effects of drug treatment on these deficits are relatively rare (Featherstone et al., 2007; Le Pen et al.., 2003). Anti-psychotic drugs such as clozapine and aripiprazole have been shown to reverse some effects of (sometimes intra-cerebrally-administered, Baviera et al 2008) NMDA antagonists on measures of attentional function (Amitai and Markou 2010) in the 5CSRTT, and thus, given that these drugs are held not to enhance cognition in schizophrenia, these could be counted as a ‘false positives’. However, in fact, effects of neuroleptics on cognitive function in schizophrenia are controversial. Some studies have shown small beneficial effects of such drugs on attentional performance, reviewed by Mortimer (1997), quite possibly arising from their anti-psychotic actions. Cognitive performance is not likely to show much benefit from the presence of distracting positive symptoms, and so small improvements, especially in tests of continuous performance could be expected to occur. Whether the chronic administration of neuroleptics in schizophrenia has deleterious effects on cognitive performance seems plausible; however, this interesting hypothesis has not yet been tested using the appropriate model of schizophrenia for the 5CSRTT.

One also has to distinguish between those studies that use the 5CSRTT (or any task) to understand the functioning of local neural circuits versus those designed to have predictive clinical validity. For example, to our knowledge, dopamine D1 agonists have not been tested in schizophrenia (although in fact in our hands such drugs only improve performance when infused centrally (Granon et al 2000), and not systemically (Passetti et al 2003) as would be the case for human treatment). These studies are valuable for understanding in principle whether it might eventually be feasible to use a D1 receptor agonist strategy if the appropriate receptors can be selectively targeted by medication.

Promising signs for the 5CRTT’s predictive validity in schizophrenia comes indirectly from investigations of other disorders where some of the requirements for making connections between preclinical and clinical research may be better fulfilled. One example of the 5CSRTT’s clinical predictive validity comes from the fact that it has correctly detected attentional effects of cholinergic agonists (such as donezepil) in patients with Alzheimer’s disease, involving a directly translatable CANTAB version of the test in human patients (Sahakian et al 1993), using both rat models with lesions of the cholinergic nucleus basalis (e.g. Muir et al 1995) and transgenic mouse models (Romberg et al 2011). Another test of validity comes from effects of drugs such as atomoxetine, reboxetine, methylphenidate and modafinil which do appear to reduce impulsivity and enhance attention in healthy humans and patients with adult ADHD, as well as in rats tested on the 5CSRTT (e.g. Crockett et al 2010; Robinson et al 2008). In summary, although the 5CSRTT has promise for predictive validity in preclinical to clinical translation, this promise has not yet been appropriately tested. Conducting those tests will be an important part of the CNTRICS development process.

5 Choice Continuous Performance Test (5C-CPT)

Task description

The 5-choice continuous performance test (5C-CPT) is a version of the 5CSRTT paradigm specifically designed to add nontargets to which the subject must inhibit responding, thereby mimicking human CPT paradigms (Young et al., 2009). Utilizing 5-/9-hole chambers, the 5C-CPT requires visual scanning of a 5 hole array. As in the standard 5CSRTT, illumination of a single lit hole (target trial) requires a response in that hole. However, the 5C-CPT also includes trials in which all five lights are illuminated; on these nontarget trials, the animal must inhibit responding. (Figure 2, Video 2). There are 5 target trials for every nontarget trial; this trial elicits a prepotent response which “must be overridden in a top-down manner … (in order) to activate a task-appropriate rule” (Luck and Gold, 2008, pp. 38). Moreover, consistent with human CPT paradigms, performance can be analyzed using signal detection theory measures of vigilance because correct rejections and false alarms to nontargets are generated as well as hits and misses to targets. The extent to which the subject can appropriately respond to the prepotent target stimuli as well as inhibit that prepotent rule when presented with nontarget stimuli and differentiate between these two trial types provides a measurement of vigilance controlled attention, while the rate of responding (or not) measures the subject’s bias to respond (or not).

Figure 2.

The 5 Choice Continuous Performance Task (5C-CPT). As with the 5CSRTT, the animal monitors five locations and responds via nosepoke to the location where the visual cue occurs. However, the 5C-CPT also includes nontarget trials, in which all locations are illuminated. The animal is to inhibit responding on these trials.

Accuracy of responding, premature responding, omission levels, latencies of correct, incorrect, premature responding, and reward collection are recorded, as well as perseverative and timeout responses. The addition of nontarget stimuli to the 5C-CPT also provides a measurement of response inhibition as well as the latency by which the subject performs such a response. Importantly, response inhibition and premature responding do not correlate with one another and can be differentiated via genetic and pharmacological means (Young et al, 2011).

The 5C-CPT was originally developed for mice but has also been developed for rats and humans (Barnes et al., 2011a, b; Eyler et al., 2011; Young et al., 2009, 2011a, b). Both rodent 5C-CPTs utilize 5-/9-hole chambers. The human 5C-CPT utilizes a similarly presented array of circles on a monitor to which the subject uses a joystick to respond in the direction 1 circle appears or must inhibit responding when 5 circles appear.

Neural substrates

Functional magnetic resonance imaging (fMRI) is available in the human 5C-CPT. A frontostriatal and parietal cerebral network is activated during task performance of the 5C-CPT (Eyler et al, 2011), consistent with other human CPTs (Serra-Grabulosa et al., 2010; Schneider et al., 2011). Importantly, differential activation/deactivation patterns are observed depending on whether a target or nontarget trial type is presented (Eyler et al, 2011).

Task parameters and variants

The development of the 5C-CPT in 5-choice chambers readily allows the same manipulations as used with the standard 5CSRTT. For example, target duration and brightness could be altered to assess visual attentional load, temporal predictability could be challenged (varying the ITI beyond the current 3–7 s), and white noise and bright light distractors could be used to degrade performance. Consistent with the dSAT (described below) 5C-CPT performance can be degraded in mice and rats by flashing the houselight during performance of the task. Thus, top-down control of attention is challenged beyond the standard task. Moreover, impaired performance with increasing trial numbers is observed in mice, rats, and humans (Barnes et al., 2011b; Young et al., 2009, 2011a) indicative of a vigilance decrement consistent with other CPTs (see Riccio et al., 2002). Finally, as evidence that the subjects respond appropriately to the respective trial types, a greater proportion of correct rejections (appropriate lack of response to a nontarget) is observed when compared with proportion of omissions (inappropriate lack of response to target) across the three species (Barnes et al., 2011b; Young et al., 2009; Young et al., 2011a). These data are consistent with what one would expect from CPT data in humans (Parasuraman, 1988, Riccio et al., 2002).

Sensitivity to pharmacologic manipulations and genetic influences

Limited data has been published to date regarding the pharmacological sensitivity of the 5C-CPT. Post-withdrawal subchronic phencyclidine treatment in rats has been used to model various aspects of schizophrenia (Neill et al., 2010). Mice treated acutely with scopolamine exhibit impaired performance of an extended session 5C-CPT, primarily indexed by a reduced hit rate and modestly increased false alarm rate (Young et al, 2010).

The only genetically modified-induced 5C-CPT abnormality published to date was the increased false alarm responding and impaired vigilance exhibited by mice with reduced dopamine D4 receptor expression (Young et al, 2011b). Reduced expression of the dopamine D4 receptor in mice results in increased false alarm responding to nontargets without affecting responding to target stimuli (Young et al, 2011b). Moreover, this response disinhibition to irrelevant (nontarget) stimuli was not accompanied by increased impulsivity as measured by premature responding (with no stimuli present). Separable mechanisms of these two forms of impulsivity was further supported when the 5-HT2C antagonist SB242084 increased premature responding in these mice, consistent with mice and rats in the 5CSRTT (Fletcher et al., 2007; Robinson et al., 2008), but did not increase false alarm responding in the 5C-CPT (Young et al, 2011b).

Sensitivity to manipulations designed to model schizophrenia

Rats that have received only low levels of vitamin D during development – a putative animal model for neurodevelopmental abnormalities in schizophrenia (Eyles et al, 2009) - exhibit impaired 5C-CPT performance as measured by increased false alarm responding (Burne et al, 2011). Rats treated with subchronic PCP (5 mg/kg bi daily for 7 days) exhibit impaired 5C-CPT performance 14 days later when challenged in an extended session (256 compared to the normal 120 trials), reflected primarily by increased false alarm responding (Barnes et al, in 2011b). Scopolamine administration also impaired accuracy of responding in mice (Young et al, 2010) although normal accuracy was observed in patients with schizophrenia (Young et al, 2011a). Given the limited data published on this matter and that some of these data are awaiting acceptance for publication in more than abstract form, more work is required using the 5C-CPT.

Psychometrics

Only limited data are available on the psychometric properties of the 5CCP, but indicate good test-retest reliability in mice. In a recent series of experiments, mice were tested with various compounds over the course of 4 months, with two-week washouts and return to baseline between studies (Young et al, in submission). Test-retest reliability for the primary measure of attention (sensitivity index) ranged from 0.69–.85 from test to test. The correlation of baseline performance from study 1 to study 6 was 0.78. Correlations on other measures were also strong (e.g. for bias, correlations ranged from 0.56 – 0.78, for mean correct latency correlations ranged from 0.55 – 0.83, for accuracy correlations ranged from 0.51 – 0.67).

Suggestions for further optimization and development

The 5C-CPT is a newly-developed paradigm and as such requires further research and development. The innovative inclusion of nontarget stimuli necessitates the differentiation of stimuli and inhibiting a prepotent rule to activate task appropriate rule. Thus, the top-down control of attention is required in the 5C-CPT because the type of response required varies with the stimulus type. More work is required in understanding the neural contributions to performance of the task, especially with regard to differentiating between target and nontarget responding. Moreover, the data collected to date on challenging performance with visually distracting stimuli require further investigation.

Best stage for use in drug discovery

The ready use of this task in mice as well as rats, and the development of a fMRI-compatible human version, makes the 5C-CPT attractive for cross-species translational testing. This cross-species translational validity for assessing top-down control of attention means that the 5C-CPT is likely to be useful in the drug-discovery process just prior to taking a compound into Phase-II trials to assess as a putative cognition enhancer.

Distractor condition sustained attention task (dSAT)

Task description

The dSAT differs from the five-choice tasks above in that it requires detection of a single stimulus rather than a choice between stimuli, and a designated response for nonsignal trials, so that omissions can be differentiated from misses. The base task (SAT) requires the animal to monitor a central panel for the presence or absence of a briefly-presented central illumination. After a short delay following the signal (or nonsignal) event, response levers are extended and the animal presses one lever to indicate that a signal was present on that trial, another to indicate that no signal was presented. The task is typically performed with the houselight on so that the animal must attend to the center panel to detect the illumination of the center signal light. In the distractor (dSAT) condition, the houselight continuously flashes (turned on/off at .5 Hz rate), increasing the demands on top-down attention and typically reducing performance. (Figure 3 outlines the human and rat versions; video of a mouse performing the task can be found at http://sitemaker.umich.edu/martin.sarter/files/2011 mouse711 dsat.wmv).

Figure 3.

The Distractor Condition Sustained Attention Task (dSAT) for humans and rats. (Courtesy of Anne Berry.) The subject monitors for the presentation of a brief, centrally-presented visual signal which occurs on 50% of trials. Shortly after the signal or nonsignal event, the subject is cued to respond (audio cue for humans, extension of response levers for rats). The subject is to press one response option if the signal did appear, the other response option if it did not. Correct trials are rewarded (audio tone signaling increase in monetary reward for humans, water reward for rats). Signal duration and the amount of time the subject must monitor for the appearance of the signal are varied unpredictably across trials to increase demands on top-down control. The distractor condition further increases demands on top-down control by rapidly changing the background illumination (flashing computer-screen background for humans, flashing houselight for rats).

A computerized version has been developed for human use in laboratory settings and the fMRI environment (Demeter et al., 2008; 2011). In this version, the distractor is implemented by having the screen rapidly alternate (10Hz) between light grey and black. It has been used successfully with populations including healthy adults, children, and patients with schizophrenia (Demeter, 2011). Humans perform more accurately and conservatively than rodents, but other qualitative aspects of performance are similar across species. A mouse version using an extendable/retractable nose-poke device (Michigan Controlled Access Response Port, MICARP) has been developed and is described, along with evidence in support of its validity, in a recent publication (St. Peters et al. 2011a).

Neural substrates

The SAT and dSAT have been extensively used to characterize how basal forebrain cholinergic innervation of front-parietal systems acts to modulate top-down and bottom-up attention (see Hasselmo & Sarter, 2011 and Sarter, Parikh, & Howe, 2009 for recent reviews). New methodological developments have made it possible to observe cholinergic effects operating at different timescales (Parikh et al., 2007). At least three timescales seem to be important: tonic, transient, and intermediate. Traditionally-studied tonic (minutes-range) increases in acetylcholine under challenging conditions may be especially important for basal forebrain projections modulating primarily mediodorsal thalamic projections to the the PFC, via α4β2* nAChR expressed by the terminals of these neurons. These tonic increases amplify the size of transient, signal-evoked thalamic glutamatergic inputs to fronto-parietal structures. These glutamatergic transients in turn are necessary but not sufficient for the generation of a transient (1–3 seconds) cholinergic response thought to activate the appropriate task sets and behavioural responses (Howe et al., 2010; Parikh et al., 2007, 2010; see Hasselmo & Sarter, 2011 for a review). These transients precede and are temporally associated with the initiation of the response to the signal, not to the signal itself, indicating that they are involved in the cognitive processing of the signal and its translation to behaviour rather than simple sensory processing. At an intermediate range, decreasing trends in the amount of cholinergic activity over the 20 seconds prior to signal presentation are associated with better detection; these are not well-understood but may reflect decreased noise from other, irrelevant cognitive activity (c.f, Weissman et al., 2006).

In humans, task performance and vulnerability to the distractor are associated with activation in right middle frontal gyrus (Demeter et al., 2011); this region has also been implicated as a major site of cognitive dysfunction in schizophrenia (Minzenberg et al., 2009). Ongoing studies suggest that, as in the rat studies, tonic activations associated with performance and attentional challenge may be separable from those more transiently associated with signal detection (Berry et al., 2011.).

Task parameters and variants

The major manipulation is the presence or absence of the background distractor. Signal duration and monitoring time (the time in which the signal may occur) are varied unpredictably across trials to increase demands on top-down control. Performance typically varies with signal duration (shorter signal durations associated with worse performance) but not monitoring time. Other manipulations that can be used to influence human performance include changes in reward contingencies (Demeter et al., 2008), using unpredictable stimulus locations (Mar et al., 1996; Demeter, 2011) and the addition of trials with long signal durations to relax response criteria.

Sensitivity to pharmacologic manipulations and genetic influences

Older studies using the SAT have been used to characterize the role of acetylcholine and catecholamine systems and other variables (e.g., gonadal steroids) in signal detection and vigilance (Briand et al., 2008; McGaughy & Sarter 1995, 1998, 1999; McGaughy et al., 1996; 1997,1999, 2000; Turchi et al.,1995, 1996; Turchi & Sarter 2001a,b; Himmelheber et al. 2000a,b, 2001; Holley et al., 1995; Miner et al., 1997, 1999; Gill et al., 2000; Burk et al., 2001, 2002; Nelson et al., 2002; Arnold et al. 2002; Broussard et al. 2009; Gritton et al., 2009; St. Peters et al. 2011b.

Attempts to increase SAT performance using nicotine have been mixed (e.g., Bushnell et al., 1997; Turchi et al., 1995). The failure of nicotine to improve performance may at first seem surprising in light of the high numbers of schizophrenic patients who smoke, a proclivity often interpreted as an attempt at cognitive repair. However, nicotine robustly benefits dSAT performance if its effects at alpha7 receptors are blocked (Howe et al., 2010). The neurobiological explanation of this effect concerns the ability of nicotine to increase the amplitude of brief, cue-evoked increases in cholinergic neurotransmission that mediate the detection of cues in this task. Nicotine increases the amplitude of these transients but, via effects at the alpha7 nAChR, nicotine, drastically lengthens the release event beyond the duration of the individual trial. Blocking the alpha 7 receptor restricts the beneficial effects of nicotine to increasing the amplitude of these cholinergic transients, similar to effects of selective alpha4beta2 nAChR agonists (Howe et al. 2010). Sstimulating alpha4beta2* nAChR subtypes robustly enhances attentional performance (e.g., McGaughy et al., 1999; Howe et al., 2010; Parikh et al., 2008, 2010).

Collectively, these results suggest that α7 nAChR blockade should increase the procognitive benefits of nicotine use in patients. Second, development of an effective α4β2* nAChR agonist could provide many of the pro-cognitive benefits of nicotine without the adverse effects associated with smoking and other methods of tobacco use. Given that such compounds also benefit the attentional control of patients with adult ADHD (Apostol et al., 2011), the foundations for a test of such compounds in schizophrenic patients, administered as an adjunctive, appear to be in place. Preliminary preclinical evidence suggest that co-administraiton of an α4β2* nAChR agonist with chronic nicotine and an antipsychotic compound (risperidol) does not limit the efficacy of selective nAChR agonist (Taylor et al., 2011).

Genetic influences on dSAT have remained limited to studies in mice hemizygous for the choline transporter. These mice exhibit dSAT impairments and these impairments are related to attenuated CHT exocytosis (from intracellular sites to the synaptosomal plasma membrane). Such CHT trafficking is observed in wild type mice in the right prefrontal cortex following completion of a dSAT session. In mutant mice, such trafficking is not evoked by dSAT performance, and thus dSAT performance is robustly lower compared with wildtype mice (Parikh, St. Peters, Blakely & Sarter, in preparation). Mice hemizygous for the CHT are considered models of deficient top-down control and are relevant for studying such control deficits in schizophrenia and adult ADHD.

Sensitivity to manipulations designed to model schizophrenia

The dSAT has been used with an amphetamine pretreatment regimen to model cognitive impairments both during active illness periods and during remission (see review by Sarter et al., 2009). In the active-illness model, performance was at floor even in the SAT condition. This poor performance was associated with a failure to increase tonic ACh levels and partially restored by haloperidol. In the remission model, SAT performance was normal but associated with abnormally high ACh levels similar to those seen in normal animals dSAT conditions. The performance of model animals was differentially impaired by the distractor, with performance deficits approximately as severe as those shown by animals with prefrontal cholinergic deafferentation. These deficits were not rescued by haloperidol. In short, these results suggest that in the active-illness model, animals failed to engage top-down control mechanisms. In contrast, remission-model animals appeared to fully engage these mechanisms to maintain performance in the SAT condition and to lack further capacity to increase their engagement in response to the increased demands on control imposed by the distractor.

Psychometrics

For rodents, internal consistency is quite good in the SAT condition (Cronbach’s alpha = .83) but poor in the dSAT condition due to near-floor performance at the short durations (Nuechterlein et al., 2009). In humans, internal consistency is good (>.80) in both conditions for college students, children, schizophrenic patients, and age-matched controls (Demeter, 2011). Other psychometric dimensions (e.g, test-retest reliability) require further testing.

Suggestions for further optimization and development

The dSAT condition has been less extensively used than the SAT, but as described above the distractor may be important for assessing behavioral and neural effects of challenges to the control of attention. It would therefore be useful to test whether different versions of the distractor (e.g., different flash rates, different levels of contrast) could be used to more finely adjust those demands. Different distractor modalities and presentation methods are being tested in humans, although the neuroimaging data already strongly suggest that the attentional demands of the distractor can be dissociated from bottom-up perceptual effects (Demeter et al., 2011). Although work with the SAT has typically used relatively long testing durations to get at the sustained attention component, the internal-reliability data suggest that this may not be necessary and that further streamlining may be feasible, especially for questions involving the transient effects associated with signal detection.

Best stage for use in drug discovery

In intact animals, the d/SAT appears to discriminate well between truly effective compounds and those that have occasionally but not consistently been associated with enhanced performance. Numerous compounds that have occasionally been classified as “cognition enhancers” did not benefit d/SAT performance. These compounds include atomoxetine, nicotine (see above), donepezil, guanfacine, methylphenidate, thioperamide, d-cycloserine, modafinil (Kozak et al., 2007; Kozak & Sarter, unpublished results). As described above, it also discriminates between models of active illness and remission when used in conjunction with the escalating dose of amphetamine pre-treatment regimen (Sarter et al., 2009), and between actions at different receptor types. This suggests that the animal version may have value at early preclinical stages for assessing specifically cognitive effects (rather than effects on disorganized behavior associated with the illness state) and targeting particular neural substrates. The neuroimaging results and initial patient data suggest it may ultimately prove valuable for Phase II trials and beyond, but this step awaits further validation.

Summary and suggestions for increasing predictive validity

As described above, the three tasks all require that subjects attend and appropriately respond to designated signal inputs, but vary in their methods of increasing the challenges to attention. The two 5-choice tasks challenge input selection via demands on detection capacity (i.e., 5 possible locations must be monitored); the dSAT constrains location and instead challenges detection via the flickering background. The dSAT also includes nonsignal trials to discriminate omissions indicating loss of task set from failures to detect the signal; the 5C-CPT presents two stimulus types to which the subject should either respond or inhibit responding.

The consideration of these tasks brings up several issues that potentially apply not only to the control of attention but to all CNTRICS tasks. The first is the tension between the need for construct validity and the need for tasks that can efficiently assess a range of cognitive functions. That is, the behavioral and neural work already done especially with the 5CSRTT and dSAT provide clear reminders that no task is “process pure” – instead it will be important to clearly define which aspects of behavior and neural activity can be specifically assigned to input selection processes and which may be more closely related to other cognitive/executive control processes such as inhibition. The control of attention is itself a multi-dimensional construct; correlating behavioral patterns to specific neuronal patterns and populations will increase our understanding of how different task variants load on different dimensions of attention and improve predictive validity and diagnostic specificity. Ideally, there would be some dimensions on which patients and animal models of the disease perform normally (without ceiling/floor effects) and some on which they show clear deficits. This would allow discrimination of specific cognitive deficits from generalized impairments due to, e.g., motivation failures, fatigue, or failures to understand instructions.

The panel also recognized the challenges inherent in assessing higher cognitive functions such as the control of attention, and that implementing these tasks for drug development, where there is a desire for short training periods to facilitate rapid throughput that allow the advancement of putative drugs, posed additional challenges. One solution suggested by the panel was to recognize that variance in cognitive control makes a large contribution to variance in the rate of task acquisition, and to use this data rather than regarding it only as a nuisance variable. However, it was generally agreed that manipulations (drug, lesions, etc.) should be applied during steady state.

In addition, the current strategy of testing primarily intact animals, as opposed to an animal model of the cognitive symptoms of schizophrenia, may not be an optimal screening method. There is also an urgent need for valid genetic and neurodevelopmental models (e.g., Murray & Lewis, 1987). Finally, given the high probability of administrating putative cognitive enhancers as an adjunct therapy to patients, preclinical tests of candidate drugs should demonstrate preserved efficacy in the presence of co-treatment with antipsychotics; the high rate of nicotine use among patients must also be considered. In short, in this field, it is doubtful that attempts to use simplified screening methods will increase the success rate of preclinical drug discovery. Rather, such screening may need to be carried out using rather advanced and effortful methods, models, and psychopharmacological approaches (see also Sarter, 2006).

With these and other concerns in mind, we suggest the following questions for evaluating whether precognitive effects demonstrated in preclinical studies have predictive validity – or, at a more “meta” level, whether predictive validity has been appropriately tested:

1. Does the drug in question reliably reduce or reverse deficits on the task in a valid model of the schizophrenia-related cognitive deficits (rather than just improving performance in intact animals)? Note that this question implies two assumptions that must also be evaluated: First, that the model of the schizophrenia-related dysfunction is valid, and second, that the task was appropriately implemented. These assumptions are often not met, even in published studies. Ideally the drug should produce these pro-cognitive effects in more than one appropriate model of the disease-related deficits.

2. Are the clinical trials in humans conducted using a validated analogous or directly translatable task? Face validity is not sufficient here. It is important to show “back translation” in terms of similar cross-species performance changes in response to the manipulation of experimental variables and to the extent possible evidence for parallel neural substrates. Of course, the same caveats regarding appropriate administration of the task mentioned above hold here.

3. Are methods of drug administration, dosage, etc. appropriately parallel across preclinical vs clinical studies? For example, a drug producing beneficial effects when infused centrally in preclinical trials may not work the same when administered systemically in human patients.

4. Are side effects, toxicity, or other adverse actions of the drug parallel across the preclinical and clinical studies? Further, have issues of co-treatment and nicotine consumption been appropriately addressed?

Although these critiera are clear and obvious in principle, they are not easy to fulfill in practice. Nonetheless, we feel they are worth aspiring to when conducting such studies, and useful to keep in mind when evaluating them.

Conclusions

To summarize, the current state of work on the CNTRICS construct of control of attention is promising, as there are three tasks with high construct validity and surface similarities that suggest that they might ultimately be combined into a single paradigm that would allow efficient evaluation of the input selection dimension of attentional control as well as others. However, further development work is needed with all of these tasks to better define the assignment of behavioral and neural variables to attentional constructs. More generally, to create effective cognitive enhancers and predictive validity, screening procedures need to be developed that better take into account the complicated genetic, developmental, and environmental milieu presented by schizophrenia.

Supplementary Material

01

Video 1. Rat performing the 5CSRTT. (Courtesy of Andrea Bari). The animal scans the five locations, waiting for one to be illuminated. It nose-pokes to the illuminated location, then collects food reward at the other end of the apparatus.

02

Video 2. Video of a mouse performing the 5C-CPT: The task is performed in the dark and the infra-red camera picks up the infra-red beams located across each of the apertures (5 on the right of the screen) and from the main reward delivery area (on the left of the screen). During the first trial, only 1 aperture is illuminated (bottom) and the mouse responds at that aperture thus gaining a reward delivered at the left. On the next trial however, 5 apertures are illuminated, indicating the mouse is to refrain from responding which the subject does while performing a lateral head movement scanning all 5 apertures. Correct inhibition of responding results in a reward being delivered. By the next trial a single aperture is lit again and the mouse makes the appropriate response, gaining a reward. Hence the subject is required to respond when appropriate (target stimuli) and inhibit from responding when appropriate (nontarget stimuli) indicative that they are asserting top-down control of attentional responding.

03

Video 3. Human version of the Distractor Condition Sustained Attention Task (dSAT). In the standard condition, the stimulus is presented against a static grey background. The signal only occurs on 50% of trials, and varies in duration when it does occur. Responses are made via right or left keypress (left/right key assignment to positive/negative response counterbalanced across subjects). The low ringing tone cues the participant to respond as to whether or not the signal occurred; the higher-pitched reward tone indicates that a correct response was made. The distractor condition is identical other than the “strobing” alternation of the background between light and dark. Note that when signal trials occur, they always occur on the lighter background, as they do in the standard condition, so that signal/background contrast does not differ across these two conditions.