Selecting Paradigms From Cognitive Neuroscience for Translation into Use in Clinical Trials: Proceedings of the Third CNTRICS Meeting

Abstract

This overview describes the goals and objectives of the third conference conducted as part of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) initiative. This third conference was focused on selecting specific paradigms from cognitive neuroscience that measured the constructs identified in the first CNTRICS meeting, with the goal of facilitating the translation of these paradigms into use in clinical trials contexts. To identify such paradigms, we had an open nomination process in which the field was asked to nominate potentially relevant paradigms and to provide information on several domains relevant to selecting the most promising tasks for each construct (eg, construct validity, neural bases, psychometrics, availability of animal models). Our goal was to identify 1–2 promising tasks for each of the 11 constructs identified at the first CNTRICS meeting. In this overview article, we describe the on-line survey used to generate nominations for promising tasks, the criteria that were used to select the tasks, the rationale behind the criteria, and the ways in which breakout groups worked together to identify the most promising tasks from among those nominated. This article serves as an introduction to the set of 6 articles included in this special issue that provide information about the specific tasks discussed and selected for the constructs from each of 6 broad domains (working memory, executive control, attention, long-term memory, perception, and social cognition).

Cognitive impairments in schizophrenia are present at the onset of the illness, persist throughout the lifespan, are strongly associated with functional disability, and are largely treatment refractory. Hence, the development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the 21st century.^1,2 Importantly, the past decade has seen a rapidly growing understanding of the neurobiology and neuropharmacology of cognition in the domains of human and animal cognitive neuroscience, and recent research has identified many molecular and psychological targets with the potential to enhance cognitive processing in schizophrenia and other psychiatric disorders.^3–9 However, despite this growing knowledge, until recently there was no established mechanism for developing cognitive enhancing drugs or psychosocial interventions for schizophrenia. The MATRICS^10,11 process brought together academia, the pharmaceutical industry, and the Food and Drug Administration to (1) identify cognitive targets in schizophrenia, (2) identify promising molecular targets that could enhance cognition, and (3) develop a process by which new therapeutic agents could be approved for the treatment of schizophrenia.

One of the challenges measurement and treatment research to improve cognition in schizophrenia (MATRICS) faced was a need to produce a consensus-based set of cognitive measures in a rapid time frame. Thus, MATRICS focused upon tasks with well-known and strong measurement properties (test-retest reliability, low practice effects, etc), although considerations of construct and neural validity were also of high importance. Measures derived from cognitive neuroscience were considered, but many were not included primarily because they did not have already established measurement properties. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project grew out of the final MATRICS meeting,¹ where the potential benefits of using tasks and tools from cognitive neuroscience were broadly acknowledged. These benefits include (1) the use of fine grained tasks that measure discrete cognitive processes, (2) the ability to design tasks that distinguish between specific cognitive deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills etc, and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. Measuring the function of specific cognitive systems that are linked to specific neural systems using a cognitive neuroscience approach offers unique advantages, especially for translational research. One of the key advantages is the ability to use the results of animal as well as human studies to identify molecular targets that modulate specific cognitive systems. Many such targetable systems, such as long-term and working memory, attention, perceptual processing, and cognitive control, are conserved across many mammalian species and measurable using parallel versions of experimental cognitive tasks.

At its first meeting, CNTRICS identified a set of constructs across 6 cognitive systems to be targeted, and at the second meeting the measurement issues were laid out together with strategies for addressing them in future research. The third CNTRICS meeting was focused on selecting promising paradigms from cognitive neuroscience that measured the constructs identified in the first CNTRICS meeting in order to facilitate the process of translation for use in clinical trials in a way that would address the challenges and goals identified in the second CNTRICS meeting. Table 1 lists the 11 constructs identified in the first CNTRICS meeting that were the focus of task selection in the third CNTRICS meeting. The third CNTRICS meeting as held in Sacramento, California, on March 19th and 20th and was attended by 58 basic and clinical scientists from government, academia, and industry.

Table 1.

Constructs, Definitions, and Nominated Tasks

Perception

Gain control: The processes whereby neurons adapt their response levels to take into account their immediate context in order to make best use of a limited dynamic signaling-range.

Nominated tasks:

Contrast-contrast effect task

Contrast sensitivity

Mismatch negativity

Prepulse inhibition of startle

Steady state visual evoked potentials to magnocellular vs parvocellular biased stimuli.

Integration: The processes linking the output of neurons—that individually code local (typically, small) attributes of a scene—into global (typically, larger) complex structure, more suitable for the guidance of behavior.

Nominated tasks:

Babble task

Coherent motion detection task

Contour integration task

Working memory

Goal maintenance: The processes involved in activating task related goals or rules based on endogenous or exogenous cues, actively representing them in a highly accessible form, and maintaining this information over an interval during which that information is needed to bias and constrain attention and response selection.

Nominated tasks:

AX-CPT/Dot pattern expectancy task

Operation span/symmetry span

Probabilistic reversal learning

Interference control: The processes involved in protecting the contents of working memory from interference from either other competing internal representations or external stimuli.

Nominated tasks:

Ignore suppress task

Inhibition of currently irrelevant memories task

Recent probes task

Attention

Control of attention: The ability to guide and/or change the focus of attention in response to internal representations.

Nominated tasks:

Attention networks task

Attention capture task

Guided search

McGaughy and Sarter sustained attention task

Posner spatial cueing

Executive control

Rule generation and selection: The processes involved in activating task related goals or rules based on endogenous or exogenous cues, actively representing them in a highly accessible form, and maintaining this information over an interval during which that information is needed to bias and constrain attention and response selection.

Nominated tasks:

1-2 AX-CPT

Groton Maze Learning Test

Switching Stroop (task switching asymmetric or symmetric)

Intradimensional/extradimensional shifting task

Dynamic adjustments of control: The processes involved in detecting the occurrence of conflict or errors in ongoing processing, identifying the type of control adjustments needed, and recruiting additional control processes.

Nominated tasks:

Attention networks task

Simon task

Stop signal task

Stroop task

Long-term memory

Relational encoding and retrieval: The processes involved in memory for stimuli/elements and how they were associated with coincident context, stimuli, or events.

Nominated tasks:

Associative inference

Relational encoding and retrieval (REaR) task

Transitive inference

Item encoding and retrieval: The processes involved in memory for individual stimuli or elements irrespective of contemporaneously presented context or elements.

Nominated tasks:

Inhibition of currently irrelevant memories task

REaR task

Reinforcement learning: Acquired behavior as a function of both positive and negative reinforcers, including the ability to (a) associate previously neutral stimuli with value, as in Pavlovian conditioning; (b) rapidly modify behavior as a function of changing reinforcement contingencies; and (c) slowly integrate over multiple reinforcement experiences to determine probabilistically optimal behaviors in the long run.

Nominated tasks:

PIzzagalli reward task

Probabilistic reversal learning

Probabilistic selection task

Weather prediction task

Social/emotional processing

Affective recognition and evaluation: The ability to detect, recognize and judge the affective value of both linguistic (eg, seen or spoken words and their prosodic contour) and nonlinguistic (eg, images of people, facial expressions, eye gaze, scenes) stimuli.

Nominated tasks:

Facial affect recognition and the effects of situational context

Multimorph task

Penn emotion recognition task

Perceiving emotion using point light walkers

Reading the mind in the eyes task

Note: AX-CPT, AX continuous performance test.

Soliciting Task Nominations

Many of the members of the CNTRICS executive committee were aware of promising paradigms from cognitive neuroscience that were potential measures of the cognitive constructs identified at the first CNTRICS meeting. However, the CNTRICS executive committee also knew that there were many paradigms about which we were unaware and we wanted to solicit nominations as broadly as possible. Hence, we advertised in diverse venues as a way to solicit nominations from basic and clinical scientists who might have relevant information about promising tasks. To do so, we sent an e-mail notification to anyone who had ever attended a prior CNTRICS meeting and to anyone identified as a potential respondent in any prior CNTRICS survey. In addition, we posted advertisements in venues such as the Neuroscience Nexus (the Society for Neuroscience newsletter), the Cognitive Neuroscience Society newsletter, and the listserves of relevant societies. In addition, we identified authors who had published data with new paradigms in major cognitive neuroscience journals and sent them e-mail notifications as well.

We asked nominators to provide us with initial information about the paradigms in several different domains. First, we asked them to tell us which of the 11 CNTRICS constructs they felt the paradigm measured and asked them to provide us with an overview of the data supporting the construct validity of the paradigm as a measure of the selected construct. Second, we asked them to tell us about the data identifying the neural mechanisms that supported performance on the task. Third, we asked them to tell us if there were any published or unpublished data on psychometric characteristics such as test-retest reliability, practice effects, floor/ceiling effects, etc. Fourth, we asked them to tell us if there were any available homologous animal models of the task. Lastly, we asked them to answer 3 questions that would help identify where the task was in a translational research pathway.

Question 1 (choose one option):

• 1. There is evidence that this specific task elicits deficits in schizophrenia.

• 2. This specific task needs to be studied in individuals with schizophrenia.

Question 2 (choose one option):

• 3. Data already exist on the psychometric characteristics of this task, such as test-retest reliability, practice effects, ceiling/floor effects.

• 4. We need to assess psychometric characteristics such as test-retest reliability, practice effects, ceiling/floor effects.

Question 3 (choose one option):

• 5. There is evidence that performance on this task can improve in response to pharmacological or psychological interventions.

• 6. We need to study whether performance on this task can improve in response to pharmacological or psychological interventions.

CNTRICS received a total of 48 task nominations. Of these 48 nominations, 7 were not considered at the third meeting either because they were already established neuropsychological tasks (eg, the Wisconsin Card Sorting Test, Complex Figure of Rey, Tower of London) or because the nomination was so general that a specific task could not be identified (eg, “eye tracking”). Of the 41 remaining nominations, several were nominated as measures of several different constructs. In order to make the process of task discussion and selection at the in-person meeting as efficient as possible, the CNTRICS executive committee decided that each task should be considered for only 1 or 2 constructs. Thus, for tasks nominated for many different constructs, the CNTRICS executive committee selected 1 or 2 of the constructs that they felt were the best fit for the task. The tasks that were evaluated for each of the 11 constructs are shown in table 2. The CNTRICS staff then gathered as many published references as possible for each task to include with the materials that would be provided to the participants at that the third meeting. This information was collated into packets that described the nominated tasks in for each construct along with a brief summary of the supporting information for their nomination. In addition, 2 primary articles were identified for each paradigm and made available to all conference attendees prior to the meeting (they could request the full set of articles relevant to each task if they so desired).

Table 2.

Criteria Used To Evaluate Task Nominations

Construct validity

Clarity of link to neural circuit

Clarity of link to cognitive mechanisms

Availability of animal model

Link to neural systems through neuropsychopharmacolog

Amenable for use in human neuroimaging studies

Evidence of impairment in schizophrenia

Linked to functional outcome in schizophrenia

Good psychometric characteristics

Selecting The Most Promising Tasks

The process of selecting the most promising tasks in the third CNTRICS meeting was similar to the process used in the first CNTRICS meeting to identify the most promising constructs. The first morning of the meeting involved overview presentations by 6 scientists that introduced the constructs identified in the first CNTRICS meeting. These scientists were (1) Todd Braver (executive control), (2) Ed Smith (working memory), (3) Steve Luck (attention), (4) Charan Ranganath (long-term memory), (5) Steve Dakin (perception), and (6) Ann Kring (social and emotional processing). These overview talks were followed by a series of 3 breakout sessions, during which the tasks for the constructs from each of 2 broad domains were considered. Thus, each conference attendee participated in 3 breakout sessions to consider nominations for constructs in 3 different domains. On the morning of the second day, all the conference attendees met again and the breakout groups presented their suggested nominations. These presentations were followed by group discussion of the pros and cons of each task and ideas for future direction (eg, psychometric development, potential animal models, etc).

In each of the breakout sessions, the participants were asked to consider the various task nominations using the criteria that the field selected as the most relevant as part of the surveys conducted for the first and second CNTRICS conferences (see table 2). The breakout group members were asked to treat some of the criteria differently than others. Specifically, conference attendees were told that if psychometric characteristics were not yet known for a task, if a task had not yet been studied in patients with schizophrenia or if a task had not yet been studied in relationship to functional outcome in schizophrenia, this should not necessarily lead to a lower score for that task. If participants automatically rated tasks lower if they had not yet been studied in schizophrenia or if their psychometric properties were unknown, this would defeat the purpose of trying to facilitate translation of promising tasks that had not yet been incorporated into clinical research. However, participants were told that all else being equal, positive evidence on these criteria should be considered an advantage for a task. The following 6 articles in this special issue describe the consideration process that each breakout group went through in deciding among the task nominations using the criteria listed in table 2. In addition, these 6 articles provide an overview of the information relevant to the criteria used to select the most promising tasks for translation for each of the 11 constructs.

Summary and Conclusions

One thing that became clear as part of third CNTRICS meeting is that there was no “perfect” single task for any particular construct. All nominated tasks had strengths and weaknesses, and all tasks are in need in subsequent psychometric and clinical development work. As noted above, the second CNTRICS meeting was focused on the very real challenges that accompany the translation of paradigms from basic science to clinical science, and the results of this meeting were published in a prior special issue in Schizophrenia Bulletin.^12–17 These challenges include facing the practical realities inherent in clinical trial contexts, including the need for tasks that are of a reasonable length (unlike the very long tasks often used in basic science studies), paradigms that are well standardized (unlike the fluid and dynamic use of tasks that often occurs in basic science studies), and the need for good psychometric properties (an aspect of task development that is often not considered in basic science studies). Further, it may be the case that there are some difficulties associated with the use of paradigms from cognitive neuroscience, such as enhanced technological requirements, that are not present with traditional “pen-and-paper” type tasks. However, such difficulties may be more of a challenge for large-scale “phase III” type clinical trials and less of a challenge for earlier phase trials for which such cognitive neuroscience paradigms may be particularly appropriate. Further, the potential advantages to using paradigms derived from cognitive neuroscience—outlined in the introduction to this overview—make attempts to overcome these challenges worth the effort. We will not know whether the advances made in paradigm development in cognitive neuroscience will facilitate cognition enhancing drug discovery or the development of novel psychosocial interventions in schizophrenia until we try. Further, we will not learn what approaches to translation are effective until we go through this process with paradigms of different forms and content and learn from what works and what does not.

This third CNTRICS conference was the last of the 3 conferences planned as part of this initial initiative. However, in many ways, this is just the beginning of the real work. Now that promising paradigms have been identified, a number of concrete steps need to occur to start the translation processes. What these next steps are will differ as a function of where the paradigm is in the translational research pathway. As described in the subsequent articles, some very promising measures of specific constructs have never been studied in patients with schizophrenia. Thus, the first critical steps for such paradigms will be to determine whether they are sensitive to cognitive deficits in schizophrenia and to determine whether they show any evidence of relationship to functional outcome in this illness. Other paradigms have been studied in schizophrenia but have either unknown or problematic practical and psychometric characteristics. For these tasks, the next critical steps are to identify ways in which to enhance their psychometric characteristics and to make them more practical for use in clinical trials but without sacrificing the construct validity that made them attractive candidates in the first place. This research may include determining how many trials of critical conditions are needed to achieve acceptable reliability, reducing floor and ceiling effects, evaluating practice effects and determining ways to reduce them, and creating paradigm “packages” that are of practical use in diverse settings. These types of research pathways would be facilitated by the development of research teams that combine expertise in multiple areas (clinical science, basic cognitive neuroscience, cognitive rehabilitation, psychometrics, etc) who can work together to balance the competing demands that arise during the translation process.

Another important next step is the explicit use of measures of the underlying neural systems in addition to measures of behavior (eg, biomarkers) as a means of more directly assessing the mechanisms by which cognitive enhancing agents or therapies are working. Although changes in behavior are by themselves important, the use of simultaneous measures of brain function may be able to help determine why new drugs or psychosocial interventions are or are not working. Further, it is possible that such “biomarker” measures will be sensitive leading indicators of eventual change in cognition that will be apparent via behavioral measures, providing early information about the potential utility of different pharmacological or psychosocial approaches. In addition, it is possible that such biomarker measures will provide important information about individual differences in neural function that may determine who will respond in what way to which type of medication or psychosocial therapy. Further, a highly critical direction is also to work with animal cognitive neuroscientist to test or generate homologous animal models of specific paradigms in different species (monkey, rodent, etc) to facilitate the important interplay between human and animal work in the drug discovery process. It would be foolish to deny the hard work needed to face and overcome these challenges. However, if we are to be successful in overcoming one of the biggest challenges facing 21st century psychiatry—the need to enhance impaired cognition in schizophrenia as a means to improving life function—we need to use every potential tool and research approach available to us via the major advances that have occurred in basic science of human cognition and brain function.