Thinking About the Future Cognitive Remediation Therapy—What Works and Could We Do Better?

Abstract

This article reviews progress in the development of effective cognitive remediation therapy (CRT) and its translational process. There is now enough evidence that cognitive difficulties experienced by people with schizophrenia can change and that the agenda for the next generation of studies is to increase these effects systematically through cognitive remediation. We examine the necessary steps and challenges of moving CRT to treatment dissemination. Theories which have been designed to explain the effects of cognitive remediation, are important but we conclude that they are not essential for dissemination which could progress in an empirical fashion. One apparent barrier is that cognitive remediation therapies look different on the surface. However, they still tend to use many of the same training procedures. The only important marker for outcome identified in the current studies seems to be the training emphasis. Some therapies concentrate on massed practice of cognitive functions, whereas others also use direct training of strategies. These may produce differing effects as noted in the most recent meta-analyses. We recommend attention to several critical issues in the next generation of empirical studies. These include developing more complex models of the therapy effects that take into account participant characteristics, specific and broad cognitive outcomes, the study design, as well as the specific and nonspecific effects of treatment, which have rarely been investigated in this empirical programme.

We naturally think of treatment development as proceeding in a linear fashion, from understanding the causes of a specific disorder to designing treatment interventions with targets identified by the etiology. It does not necessarily work that way, especially for psychological treatments. Psychological treatments are often based more on theories about cognitive and behavioral functioning than theories of etiology, and analyses of treatment effects contribute to our understanding of the disorder as much as vice versa. Because a treatment is developed from a prototype to general use, critical questions are addressed in a cyclic, iterative process: Can the treatment be clearly and reliably defined? What are its active ingredients? Is it acceptable to recipients and providers? Are the treatment procedures consistent with theory? Are the treatment effects consistent with theory (ie, do we understand how it works)? Do the risks and costs of the treatment outweigh the benefits? Can it be efficiently provided in real world service systems?

The progress from basic science to standard treatment is getting increasing attention in the research community. There is new appreciation that it does not just happen automatically. “Translational research” is a concept that reflects this realization. Each step in the translational pipeline serves a distinct purpose and has implications for treatment validation. However, conventional understandings of evidence-based practice are not necessarily sensitive to the cyclic nature of treatment development. Translation into practice is not simply the final destination in a linear stepwise process. The pipeline loops and meanders (illustrated in figure 1).

Fig. 1.

Possible pathways in the meandering pipeline of treatment development

This discussion considers the dissemination of cognitive remediation therapy (CRT), a rapidly developing treatment approach that targets the cognitive impairments of schizophrenia. Examining the nature of rate-limiting factors, in terms of conceptual, theoretical, and clinical validation, clarifies the directions that research needs to take to overcome the barriers in the translational pipeline and allows this new treatment tool to be disseminated.

Conceptual Validation: The Role and Malleability of Cognitive Impairments in Schizophrenia

In the 1990s, it became clear that in chronic schizophrenia, more cognitive impairment leads to poorer outcome, even in the context of high quality treatment and rehabilitation.^1–3 For some, these findings energized the development of psychological treatment techniques that target impaired cognition in schizophrenia, techniques whose roots were in the 1960s.⁴ Others were skeptical that cognitive impairments could be remediated, or if they could, whether this would produce meaningful benefits.⁵

Pessimism about the malleability of cognition has been given a further recent boost (at least in the United Kingdom) by more recent findings that computerized cognitive exercises do not improve cognitive performance in 11 430 normal adults.⁶ Of course, that finding is not necessarily relevant at all to cognitive remediation in people with schizophrenia who we know already have impairments that interfere with everyday life.

The lingering skepticism about the malleability of cognitive impairments in schizophrenia may account for editorial policies that encourage publication of negative findings about CRT effects.⁷ This inevitably encounters the methodological problem of “proving the null.” There are too many explanations for why any experiment did not work, and this normally leads to conservatism in publishing negative results. Meta-analytic techniques use corrections that adjust for the likelihood of “desk drawer” negative results, but they are based on the assumption of normally conservative editorial policies toward publication. Applying meta-analyses when negative results are more liberally published therefore can mislead conclusions.

Since the mid-1990s, against the tide of pessimism, voluminous evidence has accumulated to support the malleability of cognitive impairment in schizophrenia. Much of this comes from clinical trials of CRT and related treatments (eg, reviewed by^8–10). Ironically, criticism of these trials has strongly gravitated to the question of whether observed cognitive changes produce meaningful benefits, leaving behind the original doubts that impairments can be changed at all. Evidence from longitudinal analyses shows that cognitive impairments appear to be remarkably static in “treatment as usual,”¹¹ but there is mounting evidence of recovery, at least in some individuals, in rehabilitation settings of sufficient quality and intensity.^12–14 Cognitive improvements may appear in the absence of explicit cognitively targeted treatments like CRT. In the terminology of psychotherapy research, these are understood to be “nonspecific treatment effects” of rehabilitation that may closely resemble the “specific treatment effects” of CRT.

The appearance of nonspecific treatment effects creates a new question: Do the benefits of CRT exceed those of a rich psychosocial rehabilitation environment? Studies that segregate specific from nonspecific effects, ie, cognitive vs noncognitive treatment, generally support an additive contribution of cognitive treatment.^10,15

If the translation of CRT is being held back, it is not by lack of evidence that cognitive impairments in schizophrenia are relevant to course and outcome or that the impairments can be changed.

Where are the Theories and What are Their Implications?

It is rare for a complete understanding of a therapy to be explicated prior to its use, in fact had we waited for one for antipsychotic medications they would still not be in general use. In the case of CRT, theoretical validation means identification of mechanisms of action that could account for cognitive impairments and, even in the absence of understanding etiology, how theories might account for cognitive treatment effects and where usefully they may help to increase efficacy. Not surprisingly, mechanisms have been proposed that operate at both neurophysiological and cognitive levels of functioning.

At the neurophysiological level, interest focuses on “neural plasticity,” the ability of the brain to modify its neural connections and networks. Trauma, genetic anomalies, medications, environmental conditions, and as yet unidentified etiological processes may affect these connections and networks. It has been suggested that medication and therapeutic environmental conditions may stimulate neuroplastic processes toward recovery but most research is in the form of animal models and their relevance to human learning is not yet clear. However, human studies do have implications. For example, Nemeth and Janacsek¹⁶ find that poor plasticity is related to poor consolidation which is overcome if the time between learning and recall is shortened with the clear implication for CRT to provide intensive massed practice. This study also showed that elements of learning responded differentially suggesting that CRT should include some explicit learning to overcome neural plasticity problems. Learning and related cognitive functions are the targets of CRT and therefore of relevance to an integrated neurophysiological and neuropsychological theory of treatment mechanisms. However, there remains a big gap in our understanding of molecular vs molar processes, eg, between synaptic activity, neuroplasticity, and learning, and even a partial understanding will be inordinately complex.¹⁷ A more complete understanding will happen eventually and for the purposes of translation, it will be helped by the development of new outcomes.¹⁸ However, this search for the ultimate theory of everything is probably less important than clinical-level validation of the unique and specific benefits of CRT.

Bridging biological and psychological levels of analysis, Spaulding and colleagues¹⁹ propose different mechanisms for cognitive recovery in general and CRT effects in particular. Here, we concentrate on those that have specific relevance to CRT development. The first mechanism involves a dopaminergic response organization system, which in the normal brain adjusts the activation thresholds of cognitive processes in accordance with changing environmental demands. This allows optimal cognitive performance in a variety of different environmental conditions by prioritizing the accessibility of processes or skills best suited to particular conditions. The hierarchical organization created by differential activation thresholds is disrupted by the severe dopamine dysregulation associated with acute psychosis. After neurophysiological stabilization of the response organization system, CRT is hypothesized to help reestablish hierarchical organization of adaptive activation thresholds by creating a high demand for the most critical processes, in a manner analogous to the organization of motor skills through rehearsal and practice. For example, musicians must create an artificial demand for certain motor skills, those in involved in performing, because those skills are not commonly used in routine daily functioning. They create this demand by “practicing.” CRT lowers the activation thresholds of key skills whose accessibility was lost in acute psychosis, eg, focusing attention, selecting and sequencing behavioral responses, analyzing task demands, processing interpersonal information. The other proposed mechanism is compensatory learning, ie, learning new cognitive skills to compensate for those lost. This intersects with most other psychological models of CRT effects.

Integrated Psychological Therapy (IPT)^20,21 was based on a model positing a hierarchical arrangement of cognitive, social cognitive, and behavioral functioning with behavioral deficits resulting from cascades of more molecular deficits. IPT follows a progression of exercises from molecular to molar levels, starting with procedures common in CRT modalities and ending with social skills training. However, hierarchical systemic models have recently become less hierarchical and evidence of linear stepwise progression through treatment is equivocal, at best.²² Causal influences are thought to operate in all directions between all possible levels of systemic organization. This does not disallow discovery of specific causal relationships that can inform treatment,²³ but it does suggest that in the bigger picture of treatment, problems at different levels of functioning can be flexibly addressed by separate treatments. Flexibility should enhance personalization of treatment, which is generally assumed to enhance outcome.

A theoretical model to guide treatment and to explain treatment effects was formulated by Hogarty and Flesher.^24,25 Focusing primarily on cognition and social cognition, they understand the principle cognitive failure in a patients’ social functioning to be in rapid and highly efficient processing of social context. Adults take for granted their ability to instantly get the “gist” of what is going on in a social situation, and automatically access relevant contextual and procedural information in memory. Acquiring this ability is a developmental learning process that extends from childhood through adulthood. Disruption of the process by mental illness (or even by subclinical etiological factors or vulnerabilities) produces the cognitive deficit in adulthood. The resulting treatment modality, cognitive enhancement therapy (CET), was designed to strengthen “gistful” processing, via cognitive, social cognitive, and social behavioral exercises. Controlled clinical trials suggest CET is effective in improving personal and social functioning but of course that does not validate the theory on which it is based.

Wykes and Reeder's comprehensive theoretical model for CRT²⁶ also has considerable overlap with CET, in the domain of “metacognition” (cognition concerning one’s own cognition). The emphasis—like that in CET—is on developing an understanding of the abstract principles that underlie tasks, which is valued more than ultimate task proficiency as emphasized in CRTs with a more biological basis.

The appearance of similar treatment procedures derived from different theoretical models is a recapitulation of psychotherapy research in the 1950s and 1960s. It is another reflection of the cyclic, iterative interactions between basic science, treatment development and outcome research that gravitate toward dissemination. In psychotherapy research, there came a stage where the crucial validation question became, “what specific components (or ingredients) of therapy affect what specific problems in what individuals under what circumstances?” We are quickly approaching that stage in CRT research. Our service users present individually unique constellations of cognitive problems. Different treatment procedures presumably affect these problems differently, for reasons that may involve several different neurophysiological, cognitive, and behavioral mechanisms. Our theoretical models will eventually have to account for this complexity. However, for the time being, the barrier to dissemination is the need for specific procedures that target specific deficits in personalized treatment protocols, not the lack of a theory that explains the multiple deficits or the respective treatment effects.

This is not to say that more aggressive investment in basic science relevant to CRT effects would not accelerate the translation process (see Adcock and colleagues²⁷). Nevertheless, we are at a stage of treatment development where the dissemination to practice is more expeditiously resolved by empirical studies of the critical ingredients of treatment and how these relate to specific treatment outcomes. In the language of psychotherapy research, we need to distinguish specific from nonspecific treatment effects. We also need to identify mediators and moderators of treatment effects such as extrinsic and intrinsic motivation,²⁸ mental effort,²⁹ and self-perceptions.³⁰ Then, we must use that information to design treatment arrays that optimally combine the effects to maximize personalization of treatment and cost-effective outcome. This is the research task more pertinent to dissemination. Theory can certainly inform that work, but theoretical validation is not what will drive translation into practice.

What Should be the Outcome of Cognitive Remediation?

We naturally expect that the treatments that get disseminated are the ones that are validated by outcome. However, even our brief theoretical review reveals that outcome of CRT is complex and multidimensional.

Changing Cognition

It may seem obvious to say that CRT should do what it says on the tin and improve cognition. A clear driver is that people experiencing the disorder say that they would like to improve their thinking. But if cognition is the sole remediation outcome then it needs to be a noticeable, durable improvement. This requires improvement measureable on cognitive tests, and these tests are designed to withstand practice effects and related methodological biases that make detection of change very conservative. This yields “modest” treatment effects of 0.41–0.45 in meta-analyses,^10,31 but “modest” is a misnomer if the change is salient and desirable to our service users.

Learning Skills

The general aim for cognitive remediation has been durable cognitive change, but this is not essential if the aim of remediation is to aid the learning of a specific skill. Transient cognitive improvements can successfully support learning of a new skill. When the new skill has become relatively automatic then the cognitive improvement may not be needed. For such supported learning, it is not clear how large such a cognitive improvement might need to be. It is possible that small changes that have an impact on learning may not be detectable in remediation experiments with small samples (median sample size 43; range 10–145; ¹⁰). One study which shows this effect was carried out by Silverstein and colleagues.³² In their study of attention training, there was no evidence on cognitive tests that attention had improved despite behavioral measures showing these improvements. The treatment and control groups were then given skill training in a community reintegration programme. The group that received remediation showed significant advantages in the knowledge and skills learnt in this latter programme suggesting that remediation can improve skill learning even when cognitive improvements are not even apparent, probably because they were too small. This may also explain how CRT effects on occupational functioning continue to mount for months after the end of treatment.³³

Changing Functioning

A third possibility for cognitive remediation is to concentrate solely on functional outcome. Again, it is not clear how large a cognitive improvement would be necessary to support functional improvements, but it is clear that for continued functional improvement, cognitive improvements would need some durability. Functional improvements might also have feedback effects on cognitive skill and help maintain the effects of limited therapy. So, for instance, if self-esteem was an outcome or effects on symptoms then improvement in either measure is likely to be beneficial for cognition. If cognitive remediation therapies are designed to improve functioning then we need a model of how such transfer of skills takes place. This transfer is vital if we are to provide our patients with the tools to cope with everyday life.

This endeavor may require us to look at rehabilitation practices as a whole. For instance, Wykes and Reeder²⁶ specify variables such as task context, motivation, and personal goals as likely conditions under which transfer or generalization of cognitive skills to the real world is likely to occur. Community opportunity and the expectations of self and others were added by Bellack and colleagues.³⁴ These additional variables may also be targeted in remediation therapies through both specific and nonspecific effects of the therapy. Few studies have directly measured these variables and so we do not know if they change with therapy. But even if they did, we would not know whether the effects were due to specific but weak effects which may be counterbalanced by strong nonspecific effects in the same therapy.

In summary, the outcome from cognitive remediation must not confuse statistical with clinical significance. Improvements, even very small ones, may have an impact on learning skills or future functioning. Small improvements in the right cognitive domain may never have been detectable in the small samples used in most cognitive remediation studies. Conversely, large effect sizes may be noticeable in some remediation studies but may not be clinically significant because they target a specific cognitive skill that might not be relevant for functional outcome or learning a skill if that was the main goal of treatment.

What is Cognitive Remediation?

Any treatment can be disseminated only to the degree that it can be defined and distinguished from other treatments. We do not usually think of this as a type of validation, but as shown in our consideration of specific vs nonspecific treatment effects, clinical validity rests in part on our ability to distinguish between treatments and their respective contributions to recovery.

Despite being developed from the early 1990s, it is only relatively recently that a clear definition of CRT was developed. The Cognitive Remediation Experts Workshop represents the leading developers and evaluators of cognitive remediation. In 2010, the following was finally ratified. CRT for schizophrenia is “a behavioral– training based intervention that aims to improve cognitive processes (attention, memory, executive function, social cognition, or metacognition) with the goal of durability and generalization” where social cognition is defined as “the mental operations that underlie social interactions, including perceiving, interpreting, and generating responses to the intentions, dispositions, and behaviors of others.”

One of the first challenges for CRT evaluation is the number of treatment labels that purport to be cognitive remediation. This suggests that differences between programmes are important for their beneficial effects, but it is our contention that these apparent surface differences hide a multitude of similarities that probably contribute to overall benefits. The programmes differ in their name, their mode of operation, the presence of a therapist, and whether they are provided to individuals, in a group or both. But do these differences really have an impact on the benefits of cognitive remediation? The evidence is mixed. One meta-analysis³¹ provided no evidence of a lack of homogeneity of effects in global cognition between different studies, suggesting that most were equivalent. A more recent meta-analysis¹⁰ did find some heterogeneity in global cognition that allowed the investigation of therapy differences but found no therapy characteristics to explain the variation in cognitive outcome. These results suggest that specific programme differences are not important to the benefit, although, of course, they may be important in other ways such as in the acceptability of treatment, cost-effectiveness, specificity to certain deficits, etc.

We believe that remediation therapies are developed from the same behavioral training template with variations in dimensions such as duration, intensity, and therapist support. If we can agree on these commonalities then the next level of empirical investigation is the threshold levels on the differing dimensions that predict overall improvement. In other words, we need to extract the effective ingredients.

How are Cognitive Remediation Therapies the Same?

In general, the training protocols use 3 or 4 techniques supplemented by extended practice in carrying out tasks to improve the automatization of the information processing necessary for each task. Each cognitive remediation paradigm usually picks from the training techniques and presents them in a tailored way both for the specific programme and for the individual receiving the training.

The first technique adopted by most remediation programmes is “errorless learning” where individuals are taught to carry out cognitive tasks by reducing the opportunity for making errors. This means that the task or therapist provides support for all the steps required to complete the task. Sometimes, this even means that the task is built up backwards from the end result to the beginning,³⁵ sometimes tasks are presented very slowly initially and then speeded up,¹³ or in most programmes, very simple tasks are provided first and then this leads to more difficult ones^36,37. Gradually, the supports can be removed as the participant becomes more skilled at the component processes, more confident in their ability, and therefore, more independent of the therapist or task support. In general, cognitive remediation programmes that depend on this type of training try to ensure that the participant scores at a high level (usually above 80%) throughout the cognitive remediation programme.

The second type of training technique is “self-monitoring” which allows an individual to develop a technique for rehearsal of both the task instructions as well as being of use for task completion. This is implemented in a variety of ways. For instance, some programmes carry out this monitoring by reminders or hints, which are embedded within a computer programme, whereas in other regimes, the therapist provides the structure for the reminders and the therapeutic rational for the use of self-monitoring. This may involve verbalization of the task instructions, overtly initially and then covertly. This has recently been shown to improve performance on some executive tasks although it is detrimental to easier tasks, which depend more on speed than accuracy.³⁸ It may therefore be especially effective in normalizing dysfunctional language dependent cognitive systems in schizophrenia or on the most complex tasks.

Another prevalent training technique is to provide “scaffolding.” In this method, task complexity is tied to the participants’ level of competence such that they are able to carry out the task but it requires some effort. This technique differs from errorless learning in that the task components have generally been learnt but the complexity of the task can be varied. This is essential as new information processing strategies will need to be adopted when the task complexity exceeds current demands. So for instance, if the task is to learn 3 numbers, a simple visual technique could be used but when the number exceeds 5 or 6 digits, new strategies need to be implemented. This technique is introduced in most programmes and is dependent on achieving a particular performance level on a task so that moving from one level to another is automatic in a computer task and is generated by therapists in other programmes. This technique ensures that the participant can be engaged in the therapy whilst not being either bored or overwhelmed.

A further specific technique adopted by many is to suggest alternative learning supports that allow the processing of more information. So the term “chunking” may be used to produce smaller subgoals or pieces of information that can be rehearsed or monitored. This type of learning is more often used in those cognitive remediation programmes that have been described as strategy learning to differentiate them from those techniques where practice is more prevalent.

Do Differences Between Remediation Programmes Matter?

All therapies include practice on tasks in order to build confidence and skill. However, some programmes build practice on a specific skill set,³⁷ whereas others build practice around different tasks where the strategies might be the same but are generalized across tasks with different formats within as well as between sessions.^39,40 The intensity of practice is also usually agreed as needing to be high, either using a psychological approach because it prevents forgetting between sessions and/or in a neuroscience approach because it is essential to building neural plasticity as the most neural reorganization in motor learning occurs at the flat part of the learning curve ie, when learning is relatively maximal for a specific task.⁴¹ Practice to achieve successful performance within remediation therapies therefore provides no clear differentiation between the 2 approaches rather they differ in their theoretical foundations which lead to different approaches to therapy design.

Extended practice on a single task is purported to have an impact on neural plasticity because it relies on delete implicit and procedural memory, which is relatively unimpaired in people with schizophrenia.⁴² In contrast, strategy-based training also uses practice but relies on the premise that generalizations of strategy use across tasks is likely to have a larger effect on future learning and the transfer of training. This assumption is supported by evidence from computational neuroscience models that specifically identify mechanisms underlying cognitive flexibility, which are essential for novel learning. For instance, Rougier and colleagues show that better generalization takes place when initial learning is carried out in more task contexts even when the number of trials is equivalent.⁴³ The authors of that study suggest that this is because more task contexts compel the use of the same representation across tasks, whereas with few tasks, it was possible for the network to use different representations for different tasks. Using the same representation clearly has the advantage when switching to novel situations, as far fewer errors were made in the networks trained to develop a strategic approach to all tasks. This facilitation of transfer has also been found for the teaching of mathematical skills where providing tasks that encouraged the building of broad cognitive schemas led to better transfer of mathematical skill to new problems than practice on very similar tasks with the same descriptions of the problem structure.⁴⁴ If remediation therapies are concerned with transfer of training (and not all are) then the breadth of tasks included in the programme needs to be considered. Practice on the same task might allow for near transfer (to tasks similar in style) but would be less likely to lead to far transfer.

Although cognitive remediation therapies differ in their intensity of training and their length, these variables had no effects in the most recent meta-analysis and neither did the use of a computer in therapy.¹⁰ Despite practice having little effect on learning in the large scale normal population study,⁶ there were effects for people with a diagnosis of schizophrenia on generalized cognitive improvement for both practise and strategy-plus-practice remediation approaches with no significant differences in the size of the effect, even when methodological rigor had been controlled in the analyses. So for a cognitive outcome, there is no difference between the approaches. However, there is a significant difference for functional outcomes. Only strategic training produced a significant effect on functioning. The most recent meta-analysis involved a large number of studies and took account of methodological differences between studies and so is our best estimate of effects. What it seems to show is that for specific outcomes, different approaches might be important—where the concentration is on cognition alone, there is nothing to choose between the 2 approaches, but when functioning is a clear outcome, then there is currently an advantage for strategic approaches.

Potential Moderators of Treatment Success

Studies that show how treatment does not work also provide useful information by revealing factors that impede or prevent treatment effects. This is different from studies that simply show “negative results,” ie, failure to replicate positive results. Studies that identify specific moderators of a positive treatment effect are especially informative. For instance self-esteem is affected by whether individuals within therapy perceived that they had improved.⁴⁵ Those who had improved cognitively also improved their self-esteem, whereas those that showed no improvement actually decreased their self-esteem. This suggests that therapy should proceed in small increments (errorless learning and scaffolding) so participants can experience self-efficacy following improvement, which in turn may affect the likelihood of improvements translating into everyday life. These small steps and perceived self-efficacy are also important for therapists to consider when reviewing progress through therapy.

Choi and colleagues³⁰ found that if a task was perceived as useful or worthwhile there were greater expectations of success for the task and these greater expectations were, in turn, related to greater persistence of learning even when the task did not seem to be related to a person’s life. The lesson learned here is that cognitive remediation has to be presented within a framework of personal goals and the tasks within the programme should have clear links to these goals. In other words, the participant needs a narrative for therapy in order to engage and persist in therapy. It may be that most cognitive remediation therapies do provide such obvious links but where they do not a clear therapeutic rational must be produced.

Therapist expertise is likely to have an effect as those with lower levels of training produce less beneficial effects.⁴⁶ However, it is not clear why this effect might occur. The CRT in that study was computer presentation of educational software. The therapists may be having their effect because of their nonspecific expertise (instilling hope, increasing motivation, etc) or through their specific expertise (eg, in choosing the level and type of computer software for the trainee). The presence of a therapist also throws up the possibility that positive links with the therapist are beneficial to outcomes. A recent study in our group provides some evidence that higher levels of therapeutic alliance were associated with larger improvements in client rated outcomes following CRT, though there was no association with cognitive outcomes.

Participant characteristics can make a difference to the success of therapy. Participant age is associated with lower cognitive effects,^47,48 although these smaller effects were not detrimental in terms of transfer to functional outcome. Those studies with higher proportions of people with clear cognitive impairment were more likely to make large changes.¹⁰ Lower levels of symptoms were weakly associated with larger effects so symptom stability might predict those who would benefit the most. In other words, the people most likely to benefit are younger, have stable symptoms and who have clear cognitive impairment.

Research Design

Negative studies may solely have implications for research design rather than for therapy. For instance, when a study shows a failure in randomization (unbalanced groups), this is likely to affect whether change can be measured and more worryingly the reverse—increasing the likelihood of showing a main effect of therapy.¹⁰ Although these issues may be partly overcome by controlling for baseline values, this may not completely eradicate the problem. Consideration of the internal and external validity of the study is essential and has not always been well investigated in this field.

Studies which evaluate the specific effects of cognitive remediation by comparing this with a group receiving a control for the amount of contact (with a computer, a therapist, etc) may show no differences between the treatment groups.⁷ This is because it is well known that some of the therapies chosen as a control, eg, learning computer skills⁴⁹ have effects on cognition. However, this may not mean that there is no difference between the treatment groups but only that the differences are small and may only be apparent when compared with a Treatment-as-Usual Group where significant and cost-effective approaches may become apparent. A clear recommendation is therefore to have a 3-group design with controls for specific effects in addition to a no treatment control.

Study Outcome Measures

If outcomes are more global (ie, cover a number of different cognitive domains), they are likely to provide a general view of how effective cognitive remediation is for the majority of people. This is because these outcomes incorporate many different cognitive problems each of which might benefit from remediation and which may differ from individual to individual. So, for instance, a person who has difficulty with sustained attention and a person who has difficulty with stereotypical responses may both improve on the Wisconsin Card Sort test following remediation of general cognitive ability. But if the cognitive remediation is specific to a particular domain and/or the outcomes are specific, then the effect size will be driven down because some individuals will not benefit (see Wykes and Huddy⁵⁰). The more specific approach clearly allows cognitive mechanisms to be highlighted and this is part of the CNTRICS initiative,¹⁸ but it may affect the overall effect sizes of cognitive remediation trials.

Cognitive outcomes are also measured by neuropsychological tasks that have components that may be detrimentally affected by remediation. Many remediation approaches teach the participant to slow down, to increase the time to plan a task, to chunk information in order to increase recall or reduce memory load for other parts of a task, and to monitor potential responses for correction prior to their production. All these behaviors are likely to improve success in everyday tasks but on timed neuropsychological tasks are likely to slow down performance and so reduce the overall score. Remediation may therefore not necessarily lead to improved scores because although errors may decrease; fewer tasks are completed or correct performance is produced outside the time limits. An agenda for cognitive remediation might also include a closer observation of the data we have already collected to understand, not whether our participants fail, but whether there is any indication of change in style or pattern of performance. These changes in style might be crucial for identifying remediation-based changes.

But if we are to explore the models that might promote the largest change, we also need to consider other outcomes that might mediate or moderate the effects of remediation. These measures might allow us to assess the specific or nonspecific effects of therapy, which impinge on outcome. The outcome from different therapies might depend on different levels of each of these variables as well as their direct effects on cognition. For instance, figure 2 shows how measures not ordinarily defined at the outset of an efficacy study may have large effects on outcome. In this exemplar, we chose motivation which we know has an effect on cognitive and functioning outcome.⁵¹ Motivation may be improved by therapy perhaps through the experience of high levels of success. Recent evidence on the effects of cognitive deficits on future thinking (the ability to represent future rewards and events) suggests that motivation may also be directly affected by cognitive improvements⁵².

Fig. 2.

Effects of mediator and moderator variables on behavior change.

Cognitive remediation, in addition to the direct effect on functioning, might also act through a mediating variable. In the figure 2, we provide coping skills as one exemplar which might be boosted by cognitive improvements and the improved ability to cope with environmental stressors which would then improve functioning.

These mediators (coping skill), moderators (age), and third factors (motivation) may all contribute in unequal measure in different forms of cognitive remediation, which may also be affected by therapist factors such as the relationship between therapist and participant. More complex figures, which also take into account the environmental context, are given in Wykes and Reeder.²⁶

Conclusion and Recommendations

We need to be mindful, rather than mindless, in our approach to cognitive remediation therapies. Because, it is a therapy we must pay attention to therapist effects even if the therapist’s role is just to switch on the computer and provide basic advice. Even, these brief relationships may have an effect on outcome. Scattered throughout this essay has been references to the importance of these nonspecific effects, which might have a role in affecting moderating, mediating, or third factor variables, which are vital for improvement in the cognitive, social, and/or functional domain. These have rarely been investigated but may hold the key to increased efficacy.

Table 1 shows the agenda that may move the field on in terms of implementation into health services and wider access. Health service providers need to know much more about what might be expected to be the costs and benefits of cognitive remediation and how they might implement it to get the best effects. It is up to the clinical research community to provide this information which might not even take that long or require too much resource.²⁷ If the community pooled data, it might be possible within a year to answer some of the crucial questions and reduce the wait for an effective therapy.

Table 1.

An Agenda for Improving What Works in Cognitive Remediation

Category	Issue	Recommendation
Participant  characteristics	Do different cognitive remediation therapy  (CRT) affect different age groups	1. Recruit across the age span2. Secondary analyses of age effects
	Cognitive impairment	1. Consider cognitive reserve of participants and stratify for this2. Identify specific and general effects (see outcomes below)
	Participant approach to therapy	1. Measure therapy engagement (attendance, estimation of worth, understanding of)2. Measure clinical alliance with therapist
Therapy characteristics	What are the key components	1. Description in terms of errorless learning, scaffolding2. Session intensity and length3. Measure reward schedule and negative reinforcement (ie, errors)
	Therapist skills	1. Define therapist basic skills2. Measure skills and fidelity3. Define and measure nonspecific effects thought to moderate effect of this therapy (eg, schedule of reward)
Outcomes	Specific vs General cognitive measures	Include both specific cognitive measures as well as general ones to test:1. General cognitive improvement (efficacy)2. Mechanisms of improvement3. Investigations of patterns of performance within and across tests to identify the mechanisms of change in each CRT
	Alternative translational outcomes	Agreement on what key outcomes to include:1. Process measures2. Functional outcome or coprimary3. Self-efficacy or self-esteem4. Improved motivation
	Mediating and moderating factors	See above
Design	Implementation of designs to improve  the understanding of effects	Three group (Experimental Treatment [ET], Control Treatment [CT] and Treatment-as-Usual [TAU]) to test:1. Efficacy and effectiveness (ET vs TAU)2. Specific efficacy, process and mechanism effects (ET vs CT vs TAU)3. Studies should be designed to test therapies against each other for specific participant groups4. Separating specific from nonspecific treatment effects and measuring their contribution to outcome
Theory	Therapeutic implications not clear	1. Define specific therapeutic implications of a theory2. Define expected differences between 2 models
	Not clear what improvements into  everyday life requires	1. Simple model building including process measures, moderating, and mediating factors2. Testing models within and across datasets

Funding

National Institute for Health Research NIHR Biomedical Research Centre for Mental Health at the South London; Maudsley NHS Foundation Trust; Institute of Psychiatry Kings College London (to T.W.)