Abstract
The National Institute of Mental Health (NIMH)-Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Project and related efforts have stimulated the initiation of several studies of pharmacologic treatments for cognitive impairment in schizophrenia. Cognitive remediation may provide an excellent platform for the provision of new learning opportunities and the acquisition of new skills for patients who are engaged in pharmacologic trials to improve cognition. However, it is not clear how a cognitive remediation intervention would be employed in multisite clinical trials. A meeting of experts on cognitive remediation and related methodological topics was convened to address the feasibility and study design issues for the development of a multisite trial of cognitive remediation in schizophrenia called the Cognitive Remediation in the Schizophrenia Trials Network study. This report details the findings from this meeting, which included the following 4 conclusions. (1) A multisite trial of a cognitive remediation intervention using a network of diverse research sites would be of great scientific value. (2) Various interventions could be employed for this multisite trial. (3) Programs that do not address key motivational and interpersonal aspects of cognitive remediation may benefit from supplementation with “bridging groups” that allows patients to meet with others and to apply their newly acquired cognitive skills to everyday life. (4) Before a multisite efficacy trial is initiated, a pilot study could demonstrate the feasibility of conducting a trial using a cognitive remediation intervention.
Keywords: cognition, cognitive remediation, functional outcomes, rehabilitation, neuropsychology, motivation
Introduction
Neurocognitive impairment, a core component of schizophrenia, is increasingly under investigation as a potential treatment target. Such impairment, which affects almost all patients with schizophrenia,1 ranges from moderate to severe2–4 and is strongly correlated with functional outcomes.5 Antipsychotics provide minimal neurocognitive improvement6 consistent with practice effects7 in chronic patients treated with conventional or second-generation antipsychotics. Treatment intervention is sorely needed. The National Institute of Mental Health (NIMH)-Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Project and related efforts have stimulated interest from government and industry, and several studies are underway to explore new pharmacologic treatments for cognitive impairment in schizophrenia (www.clinicialtrials.gov, accessed November 19, 2009), although no pharmacologic approaches to improve cognition have yet received regulatory approval.
While broad efforts are underway to refine and harness pharmacologic mechanisms that could contribute to enhanced cognitive functioning in schizophrenia, one unaddressed area of work is the relatively impoverished cognitive lives of patients who enroll in these pharmacologic enhancement studies. It is possible that the cognitive benefit of these experimental pharmacologic interventions is minimized when patients are studied in the context of the low level of cognitive, behavioral, and environmental stimulation that is typical in patients with schizophrenia. Thus, analogous to the need for physical exercise in an individual who takes steroids to increase muscle mass, schizophrenia patients in cognitive enhancement trials may require learning contexts sufficient to “exercise” any newfound cognitive potential that they may have acquired from the drug under study.
Cognitive remediation may provide an excellent platform for the provision of new learning opportunities and the acquisition of new skills for patients who are engaged in pharmacologic trials to improve cognition. As defined by McGurk et al,8 cognitive remediation programs developed for schizophrenia seek to address cognitive impairment through a variety of methods such as drill and practice exercises, compensatory strategies, and group discussions. Such programs may be computer based, may rely on interactions with trained instructors, and/or be classroom based.
Recent work on the effects of cognitive remediation suggests that this approach may demonstrate moderate efficacy in improving cognition in schizophrenia. A meta-analysis of 26 randomized controlled trials involving a total of 1151 patients concluded that cognitive remediation produces moderate improvements in cognitive performance and, when combined with psychiatric rehabilitation, also improves functional outcomes.8 Additionally, these programs are quite popular with patients and have even been linked with increases in participant self-esteem.9 Ongoing treatment with cognitive remediation may thus provide schizophrenia patients with the necessary cognitive enrichment and motivation to demonstrate the true potential of effective cognitive enhancement from pharmacologic agents.
However, there are clear challenges to progress. First, results from individual studies remain mixed.10,11 Remediation programs vary in terms of underlying conceptual foundations and intervention modalities, and the field has yet to reach consensus about the essential elements of the intervention. Second, methodological challenges are considerable. It is not clear how a cognitive remediation intervention would be employed in multisite clinical trials, especially in industry trials that may include a number of nonacademic sites with little cognitive remediation experience. Most of the cognitive remediation trials in patients with schizophrenia have been implemented at single sites with highly trained academic research personnel and methods developed at those sites; thus, the generalizability of these methods is not well known. Furthermore, as with drugs in the pharmaceutical industry, the ability of the developers of cognitive remediation programs to evaluate the efficacy of their own programs without bias may be questioned.
The feasibility of completing a study with both pharmacologic and behavioral interventions in schizophrenia may be particularly challenging. It is not clear what percentage of patients would be able to meet medical screening criteria for an experimental drug and would also be able to devote the time necessary to complete a behavioral regimen. Furthermore, because pharmaceutical company trials are increasingly conducted outside of North America, the feasibility of these interventions to be conducted internationally will also need to be determined. One of the crucial next steps is to determine the feasibility of conducting a multisite trial of cognitive remediation in patients with schizophrenia in a circumscribed geographical region that may facilitate maximal benefit.
We convened a meeting of North American–based experts on cognitive remediation and related topics to address several study design issues for the development of a multisite trial of cognitive remediation in schizophrenia (see table 1). The eventual goal for this project will be to test the efficacy of a combined pharmacologic and cognitive remediation treatment program. The immediate goal is to determine the feasibility of implementing a cognitive remediation program in a network of sites that do not specialize in this area of research. This study, called the Cognitive Remediation in the Schizophrenia Trials Network study, will determine the feasibility of multisite cognitive remediation projects both as solo behavioral interventions and as platforms for pharmacologic cognitive enhancement trials. This article is a report of the methodological issues that were addressed during the course of this working group conference.
Table 1.
Design Issues for Multisite Trials of Cognitive Remediation in Schizophrenia
Choice of intervention |
Characteristics of intervention |
Duration and frequency of sessions |
Duration of trial |
Degree of therapist involvement |
Individual vs group administration |
Computerized vs noncomputerized methods |
Control condition |
Treatment as usual |
Computer games |
Control procedures for frequency of interpersonal contact |
Blindness to treatment group |
Primary outcome measures |
Cognitive performance |
Functional outcomes |
Functional capacity |
Interview-based assessments of cognition |
Potential mediating outcomes |
Symptoms |
Self-esteem |
Motivation |
Attendance |
Site selection |
Training |
Patient population |
Data analysis |
Discussion of Programs
Several potential cognitive remediation interventions that could be utilized in multisite trials were reviewed at the conference based upon the experience of the attendees. Our aim was not to choose the “best” cognitive remediation strategy but rather to discuss cognitive remediation interventions that could best be implemented in a multisite trial, with the intention that these interventions may eventually be useful for trials that integrate cognitive remediation and pharmacologic interventions. Because a review of available programs is beyond the scope of this article, and has been covered recently in the literature,12 we will not devote space in this journal to describe the programs that were under consideration. There are many programs that could serve as potential interventions for multisite trials.
The following features of a cognitive remediation program were identified as desirable for a multisite trial in schizophrenia:
multiple sessions to learn, practice, and begin to automatize new cognitive skills
emphasis on increasing self-efficacy and intrinsic motivation
training manuals for clinicians who administer the intervention, video demonstrations, and establishment of intervention fidelity across sites at trial initiation and midtrial
demonstrated efficacy on key outcome measures of cognition and/or functional outcomes
Key Design Issues for Multisite Trials of Cognitive Remediation
Duration and frequency of sessions. The various cognitive remediation programs include a wide variety of frequencies and durations, from 1 to 10 hours per week. Maintenance phases in some programs are less frequent. Session duration for most programs is 60 minutes, although 90 minutes seemed reasonable if patients were able to stay engaged in the material. As in pharmacologic studies, it is not sufficient to compare doses from different studies because a variety of other factors may vary across studies. Two studies have addressed “dosing” of a specific cognitive remediation intervention. The duration of integrated psychological therapy per session, the number of therapy sessions, the length of treatment, or the frequency of therapy sessions has been reported not to correlate with global therapy outcome.13 Studies using the Neuropsychological Educational Approach to Remediation (NEAR)14 indicate that at least 2 sessions a week are necessary for cognitive benefit.15,16 However, clinical benefit has been reported in studies using cognitive remediation in one 2-hour group session per week.17 It is unknown the extent to which more frequent intervention may increase the persistence of benefit over time after treatment has been completed. In addition, different interventions may require/allow different frequencies. More research is clearly needed on this crucial question.
Duration of trial. The length of programs vary, but 30–40 hours of training and 3 months of trial duration was viewed as a minimum. Most interventions are based upon number of hours of treatment (ranging from 30 to 90) and not the number of weeks. However, because a longer study duration with a reduced frequency of sessions may reduce the “dose strength” of the intervention, it is important to attend to both treatment duration and frequency of interventions. Because multisite trials need to balance rigor with practicality, some flexibility in study design may be preferred. The group consensus was that for a brief trial 40 hours of treatment over the course of 2–3 months would be acceptable.
Degree of therapist involvement. Each program differs in the amount of effort required from patients and therapists. The PositScience auditory training program10,11 consists of a heavy schedule of computerized training that places implicit, increasing demands on auditory perception and accurate aural speech reception. Programs such as PositScience place the least demand on clinical services as in some cases patients can complete the training by themselves. Severely ill patients may be unlikely to be able to complete such an independent task. Programs that require more time from therapists, such as Attention Shaping Procedures (ASP), a behavioral intervention designed to increase attentiveness, and ultimately learning and skill acquisition, in psychosocial skills training or other groups, require greater resources. However, they may be able to be applied to patients with more severe illness. The amount of therapist involvement may determine whether a clinical service can provide certain interventions; it is possible that clinics with minimal staff resources may not have the personnel to provide treatment with a high degree of therapist involvement.
Individual vs group administration. While group administration has clear benefits for reducing variability and resource demands in multisite trials and allows for the potential positive effects of peer support, individual administration allows flexibility within sessions. Individual sessions allow participants to gain maximally from each session, especially when they include highly personalized treatment for people who may rely on individualized attention or experience the interpersonal stimulation of groups as too stressful. There is no research to definitively guide the use of group vs individual treatment, and both have been used successfully. There is a question of whether individualized programs have the capacity to be generalized to larger systems of mental health care. Combinations of individualized and group interventions are feasible, such as the NEAR model in which participants are not only entered into the program on a rolling basis for individualized intervention within the context of a nominal “group” but also attend “bridging groups” that meet once weekly to teach participants as a group how to apply newly acquired cognitive skill to everyday tasks, promote group identity, and promote socialization.14
Computerized versus noncomputerized methodology. This distinction was viewed as less important than the strategies reflected in the method. For instance, the PositScience computerized programs target cognitive ability hierarchically, with multiple exercises aimed at lower cognitive levels (eg, increasing the efficiency of sensory processing) before advancing to higher order functions. The NEAR program uses computerized tasks also, but the tasks are chosen on the basis of 4 dimensions, only one of which is the cognitive skill addressed.
Location. Clinic-based training creates a learning center with social reinforcers and better control for fidelity. Home-based training is self-directed and more convenient. Interventions should take place in a clinical/educational setting but can incorporate homework assignments and availability of learning paradigms so that actively engaged participants can accelerate skills on an individual basis.
Control Condition
The group agreed that an active control condition is important for the control for nonspecific elements of treatment.10,11,18 The 2 key factors were controlling for time spent with a trainer or in-group interaction and controlling for total training time. The nature of the control condition should be governed by the hypothesized mechanism of action and what are considered to be nonspecific effects.
“Treatment as usual” and passive control conditions (eg, TV watching) were viewed as least rigorous among the control conditions.
Elements of active control conditions in recent trials have included matched supportive interactions with trainers, matched experience with computers and computer activities, and monetary payments.10,11 Computer-based games were viewed as a rigorous control condition component for computer-assisted remediation programs. The use of computer games controls for the general engagement of attentional systems, executive functions, and motivation but can be selected and implemented so as not to incorporate the critical training in the remediation programs. In a trial of computerized cognitive training, control subjects should play enjoyable computer games for the same number of hours as active training subjects and receive the same amount of contact with personnel as the experimental group. A rotating set of computerized games (eg, visuospatial puzzle games, clue-gathering mystery games, pinball-style games, and target-aiming games) has been successfully implemented.10 Computer games, however, do not control for the nonspecific effects of interaction with a therapist, which in itself may benefit cognitive skill and self-esteem. It was recommended by a blind reviewer of this article that ideally a multisite trial would have 2 control conditions: a passive one in which subjects engage in computer games on their own and an active one involving increased interaction with a therapist.
Controls for the group interpersonal aspects of the training are also important. If a cognitive remediation program includes interpersonal activities such as bridging groups, it is possible that patients may benefit solely from increased social activity. Thus, a rigorous control treatment arm should include similar social activities such as a healthy lifestyles group.
Blinding. A high level of rigor is essential to adequately test the efficacy of a cognitive remediation intervention, as has been discussed in detail in previously published articles.19,20 Cognitive remediation strategies with computerized control conditions may have special challenges. A recent study by Dickinson et al11 approached these challenges with a “quasi-double-blind” design. Raters were kept fully blind to condition, and participants were not informed that they were assigned to “treatment” or “control” conditions; rather, individuals in both groups were told that the study aimed to determine whether participation in a “computer activities program” improved thinking skills. Study coordinators and cognitive remediation coaches will need to know which treatment group the patient is assigned to, but the patients and cognitive testers should be blinded to the random assignment. The following steps can be taken to preserve the blinding: Study staff refer to the groups as cognitive training 1 and cognitive training 2 (rather than “cognitive remediation” and “control”) to decrease the likelihood of accidental unblindings. Sites should have adequate space so that potentially unblinding activities (eg, training sessions, bridging groups) can take place in a separate place from where the cognitive testers carry out their daily activities. Posttreatment assessment of the success of the blind for each patient and each cognitive tester should be completed.
Outcome Measures
Outcome assessors/raters should be separate from training staff and should be kept fully blind to treatment condition. The optimal trial design practice would call for raters to be queried to assure that the blind is maintained in practice.
Training outcomes. Some remediation programs include metrics that are internal to the training exercises.10,11 Such “trained outcomes” provide a useful, proximal gauge of training response.
Cognitive outcomes. Because cognitive performance is the direct target of cognitive remediation, it is viewed as the most sensitive untrained treatment outcome measure. The MATRICS Consensus Cognitive Battery (MCCB) is a reasonable endpoint. A unanimous view of participants was that the outcome measures should not allow “training to the task.” Additional measures of social cognition under consideration have been refined since the MATRICS Neurocognition Committee convened to choose social cognition tests. Possibilities include Bell Lysaker Emotion Recognition Task,21 Social Attribution Test—Multiple Choice,11 and Hinting Task.22 It must be kept in mind, however, that, in both the traumatic brain injury23 and schizophrenia24 cognitive rehabilitation literature change in behavioral performance (ie, disability) is often independent of change on cognitive tests (ie, impairment). Therefore, improvement on cognitive tests may not indicate improvement in real-world abilities, and lack of change in one domain may be accompanied by positive or negative change in the other.
Functional outcomes. Much of the emphasis on the treatment of cognition in schizophrenia stems from the notion that improving cognition may lead to improvements in the functional outcomes that they predict.5 Thus, functional outcomes are key treatment targets. However, because functioning in patients with schizophrenia is determined by multiple factors, some of which are immutable and related to societal (eg, the availability of social services) rather than individual variables,25 changes in functional outcomes were viewed as too stringent a criterion for treatment success in the context of a short-term (12 wk) trial. Functional change may be possible in very long trials, but these will have additional implementation challenges.
Functional capacity. As recommended by the conclusions of the MATRICS project in the context of pharmacologic trials, because the ultimate goal of cognitive remediation is to make learning and the acquisition of new functional skills easier, measures of functional capacity should also be included. The NIMH-MATRICS Validation of Intermediate Measures initiative results, presented on October 27, 2009 (www.matrics.ucla.edu/matrics-ct/), suggested that the University of California San Diego Performance-based Skills Assessment, 2nd edition (UPSA-2) has the best psychometric characteristics among existing instruments of longer duration, while the Brief version of the UPSA and the Test of Adaptive Behavior in Schizophrenia26 were adequate short-form measures. The UPSA-2 and UPSA-B are currently being utilized in several large multisite pharmaceutical company trials; thus, comprehensive data from actual clinical trials will soon be available.27,28 However, alternative measures have demonstrated promise. Virtual-reality assessment of real-world skills or simulated real-world training, which can be completed in a laboratory in a single session, has considerable face validity and can be reliably scored. Improvements in functional capacity would not be required for a cognitive remediation intervention to be considered a success.
Interview-based measures were viewed as clinically relevant. Improvement on these measures would suggest that the cognitive improvement indeed had generalized to everyday life skills.
Secondary outcomes
Symptoms. Positive, negative, and general symptoms can be measured with standard assessments such as the PANSS.
Self-esteem can be measured using the Personal Mastery Scale, a 7-item, 5-point Likert scale developed by Pearlin and Schooler.29 The content of this scale overlaps with the concept of locus of control, but it is more focused on perceived ability to change. Alternatively, it might also be useful to use the Self-efficacy Scale,30 an 83-item scale that was specifically developed for use with schizophrenia-spectrum patients. The scale has 3 subscales, one of which specifically addresses perceived self-efficacy in interpersonal situations. It demonstrated good reliability and validity in 2 samples of schizophrenia patients.30 Recently, the Perceived Competency Scale has been developed and has great promise for use in cognitive remediation trials as it measures competency in learning situations.31
Motivation. The Intrinsic Motivation Scale,31 Work Preference Inventory, Motivation Traits Inventory, and Situational Motivation Scale were all viewed as acceptable measures of motivation in multisite trials.
Program attendance as an outcome measure. While attendance may be determined by multiple complex factors, if one assumes that an intervention is beneficial, a measure of treatment continuation can be viewed as an important composite variable of effectiveness.32
Criteria for Site Selection
Sites and testers should be evaluated for inclusion in the study with questionnaires describing their relevant experience with cognitive assessments, cognitive remediation, schizophrenia trials, facilities, and relevant schizophrenia population. A key feasibility determination is whether patients have the time and financial resources to travel regularly to the site for the intervention and assessments. Payment to patients for transportation may facilitate attendance yet is not without controversy. On the one hand, if it is necessary to pay patients for treatment, it calls into question whether such a treatment could be used as a regular intervention in clinical practice. On the other hand, patients may expect to be reimbursed for travel costs related to participation in a clinical trial, and thus modest payment may be necessary.
Necessary personnel for administering the intervention. Some clinical experience with persons with schizophrenia is necessary. Although some programs require that trainers have at least a masters degree, we began with the premise that a clinical degree is preferable but not essential and that trainers will need to demonstrate a minimum level of skill in administering the interventions. The PositScience programs administered alone do not require specialized clinical experience. In a study using the NEAR model, schizophrenia clients trained by clinicians with more specialized training in mental health were found to have better response to treatment.15
Necessary equipment, materials, and facilities. Most cognitive remediation interventions can be administered using standard computers and software platforms. Large (17″ or larger) screens are preferred. Laptop computers will enable greater flexibility of location, but desktop computers may be preferred at sites where theft is a potential problem. Because both types of computer will use similar screens, it will be acceptable to have different machines across sites. Laptops will require an external mouse. It is likely that all patients in the study will be outpatients. Office space for individual and group interventions will likely be necessary. Dedicated office space where participants can get additional training spontaneously is preferred. Sites with day hospital programs will be advantageous.
Procedures for Training Remediation Therapists Across Sites
The previous experience of the meeting attendees suggested that a variety of training strategies are acceptable depending upon the complexities of the intervention, especially the degree of interpersonal interactions between patient and therapist. Some of the more interpersonally intensive interactions, such as the NEAR program, have been found to benefit from an intensive face-to-face training session using a clinician's training manual19 followed by individual study of the manualized training procedures, pencil-and-paper and role-play certification examinations, review of gold standard videotaped sessions, shadowing of actual training sessions, and supervised live sessions.10 It is best if trials can include an ongoing process of assessing whether the patient-therapist interaction continues to reflect the clinical procedures taught at the beginning of the study, although it will be challenging to implement these so-called “fidelity checks” in multisite trials.33,34
Interventions that rely less upon patient-therapist interaction will obviously require less training. The standard PositScience training and certification system has involved a half-day session, but this approach has not been implemented with trainers conducting multisite schizophrenia trials, and there was some question about whether this would be sufficient for a multisite trial.
The consensus opinion was that an investigators meeting could have multiple elements for the personnel involved and that web-based training for certain staff members such as PI's and raters of secondary outcomes would be possible. More intensive face-to-face training is necessary for cognitive testers, cognitive remediation trainers, and clinicians leading bridging groups. Central oversight of cognitive data collection is essential, and a consistent quality of the intervention must be maintained.
Target Patient Population and Inclusion/Exclusion Criteria
Age. There is little reason to exclude participants based upon age although older adults may be less likely to be able to learn new material and demonstrate cognitive adaptability.8,35 Those with greater duration of illness may be less responsive, and older patients may have greater difficulty with computerized methods. Because the level of familiarity that the schizophrenia population has with computerized tasks is changing over time as computers become more commonplace, it was recommended that a multisite trial could inform this area of work by collecting data on age and previous computer experience as a baseline measure.
Medications. Because serum anticholinergic activity is negatively associated with cognitive improvement after 50 hours of the PositScience auditory training program36 and because the addition of anticholinergic treatment to an antipsychotic regimen is associated with poorer cognitive performance,6,37 cognitive remediation trials should consider whether to prohibit anticholinergic medications (eg, benztropine), as has been suggested by the updated MATRICS recommendations (R.W. Buchanan, MD, R.S.E. Keefe, PhD, D. Umbricht, MD, M.F. Green, PhD, T. Laughren, MD, and S.R. Marder, MD, unpublished data, 2009) or antipsychotics with high anticholinergic activity (eg, clozapine and olanzapine). These trials may also consider excluding patients who require regular treatment (more than 3 times per wk) with medications such as benzodiazapines that may cause sedation during cognitive training.
IQ and education. Study subjects should have IQ scores greater than 75 or 80 and at least a fifth-grade reading level.
Comorbid conditions. Screening methods for substance abuse and dependence are standard; however, additional screening may be necessary. A recent 10-week industry trial of a cognitive enhancing drug that did not exclude patients with current substance abuse suggested that 31% of patients who completed the trial tested positive for illicit drugs (R.W. Buchanan, MD, R.S.E. Keefe, PhD, D. Umbricht, MD, M.F. Green, PhD, T. Laughren, MD, and S.R. Marder, MD, unpublished data, 2009). These figures suggest that neglecting to screen patients for current substance use or abuse may include a significant portion of patients whose substance use may interfere with their ability to benefit from cognitive remediation.
Symptoms. The MATRICS criteria for maximal positive, disorganized, and negative symptom criteria38 are appropriate for cognitive remediation trials as well, although recent updates of the MATRICS recommendations allow inclusion of patients with greater than or equal to 5 (moderately severe) on positive symptom PANSS items and allow inclusion of patients regardless of the severity of their negative symptoms (R.W. Buchanan, MD, R.S.E. Keefe, PhD, D. Umbricht, MD, M.F. Green, PhD, T. Laughren, MD, and S.R. Marder, MD, unpublished data, 2009).
Functioning level. There are very few data to address whether patients with better or poorer baseline levels of functioning will benefit equally from cognitive remediation. However, it is reasonable that patients with very low levels of functioning, such as Global Assessment of Functioning (Scale) scores of less than 30, may not be able to benefit from computerized cognitive remediation interventions. For such patients, however, ASP may be a good option, as this intervention was developed for, and has been studied primarily with low functioning, “treatment-refractory” patients.
Motivation. An assessment of a patient's motivation to benefit from cognitive remediation may help exclude those patients without interest in engaging in the program. Requiring patients to be able to state goals that may benefit from the program can achieve this aim.8
Other Study Design Issues
Expected effect size. A large range of effect sizes has been reported in single-site cognitive remediation studies. Following NEAR, effect sizes range between d = 0.23 (set-shifting) and d = 0.68 (problem solving) at end of treatment and between d = 0.17 (set-shifting) and d = 0.39 (problem solving) 15 weeks after treatment discontinuation.39 Effect sizes of 0.86 for MATRICS-based general cognition composite score change and 0.89 for verbal memory improvement have been reported following the auditory training program of PositScience.11 In a small (N = 22) group of subjects followed for 6 months after the intervention was discontinued, the effects persisted over time.40 The effect size for improvement in number of minutes of attentiveness per group in a controlled study of ASP by Silverstein et al41—based on serial correlation–corrected slopes of observed attentiveness ratings across all treatment sessions over 4 months—was d = 1.51. In an earlier, smaller controlled study,24 the effect size was d = 1.19. The effect size for the group (ASP vs Control) × time (pre-post treatment) interaction effect demonstrating that patients receiving ASP acquired more skills in the social skills training group than patients who received the same group without ASP, was d = .72. As noted, however, results of remediation trials have been mixed, with some studies showing little or no remediation advantage on primary outcome measures.1,11 Thus, effects of the magnitudes noted may be difficult to attain in multisite trials. A cognitive benefit of d = 1.0–1.5 as assessed by global cognitive measures such as the MCCB in schizophrenia would have a significant clinical impact on many patients.
Assessment of persistence of effect. Treatment effects persist39 and may even increase42 over time after treatment has been discontinued. Thus, it is highly recommended that a posttreatment follow-up assessment is included in the trial.
Data Analysis Issues
Problematic data analyses such as failure to control for multiple comparisons, post hoc data analyses, and “completer-only” data analyses exaggerate the efficacy of an intervention and should be avoided.
Rigorous, blinded data analyses should be employed in cognitive remediation trials. These analyses should be similar to those applied in phase II clinical trials, in which a company needs to determine whether a drug has a likely chance of demonstrating sufficient efficacy in larger trials to meet Food and Drug Administration approval.
Such a trial should have a predefined data analysis plan.
The plan should specify a limited number of neuropsychological and functional outcomes and provide an explicit rationale for the number of planned comparisons.
The plan should also address the issue of intent-to-treat style analysis, providing a clear rationale for variation from this standard.
The trial should include as many testing points as feasible to allow for detection of nonlinear changes in cognitive functioning and adequate characterization of both change (slope) and variability around the slope (root-mean-square error).43
Conclusions
The group consensus was that a multisite trial of a cognitive remediation intervention using a network of diverse research sites would be of great scientific value.
While specific design issues may need further deliberation, a trial involving approximately 10 sites and 200 patients is likely to be feasible and provide adequate statistical power to test the efficacy of a cognitive remediation intervention vs a rigorous control group.
Clinician/trainers and cognitive testers will need to be trained and certified at all sites.
Before the multisite efficacy trial is initiated, it will be important to conduct a pilot study that could demonstrate the feasibility of conducting a trial using a cognitive remediation intervention.
Funding
National Institute of Mental Health (N01 MH90001).
Acknowledgments
Additional attendees: Alison Adcock, Duke University; Jeffrey Baker, Abbott Laboratories; Wendy Granberry, GlaxoSmithKline; Matcheri Keshavan, BIDMC and Wayne State University; Eva Kohegyi, Sanofi-aventis; Michael Kraus, Duke University; Martin Lowy, GlaxoSmithKline; Henry Mahncke, PositScience; Wen-Chen Ouyang, Duke University Medical Center; Donna Palumbo, Pfizer Inc; Thomas Patterson, University of California San Diego; David Penn, The University of North Carolina; Diana Perkins, The University of North Carolina; Ingrid Rojas, The University of North Carolina; Jaskaran Singh, Johnson & Johnson PRD, LLC; Joyce Tsai, Forest Research Institute; Claire Vilain, Lundbeck SAS; Trina Walker, Duke University; and Nancy Zucker, Duke University. R.S.E.K. reports that he currently or in the past 12 months has received investigator-initiated research funding support from the National Institute of Mental Health, Allon, GlaxoSmithKline, Novartis, and the Singapore National Medical Research Council and an unrestricted educational grant from Astra-Zeneca. He currently or in the past 12 months has received honoraria, served as a consultant, or advisory board member for Abbott, BiolineRx, BrainCells, CHDI, Dainippon Sumitomo Pharma, Eli Lilly, Lundbeck, Memory Pharmaceuticals, Merck, Neurosearch, Novartis, Orion, Otsuka, Pfizer, Roche, Sanofi/Aventis, Shire, Solvay, Takeda, and Wyeth. In the past, he has received honoraria, served as a consultant, or advisory board member from Acadia, AstraZeneca, Bristol-Myers Squibb, Cortex, Cephalon, Eli Lilly, Johnson & Johnson, Orexigen, Organon, Pfizer, and Xenoport. Also in the past, he has received research funding from AstraZeneca, Eli Lilly, Janssen, Organon, and Pfizer. R.S.E.K. receives royalties from the Brief Assessment of Cognition in Schizophrenia (BACS) testing battery and the MATRICS Battery (BACS Symbol Coding).
References
- 1.Keefe RSE, Eesley CE, Poe M. Defining a cognitive function decrement in schizophrenia. Biol Psychiatry. 2005;57:688–691. doi: 10.1016/j.biopsych.2005.01.003. [DOI] [PubMed] [Google Scholar]
- 2.Saykin AJ, Gur RC, Gur RE, et al. Neuropsychological function in schizophrenia. Selective impairment in memory and learning. Arch Gen Psychiatry. 1991;48:618–624. doi: 10.1001/archpsyc.1991.01810310036007. [DOI] [PubMed] [Google Scholar]
- 3.Harvey PD, Keefe RSE. Cognitive impairment in schizophrenia and implications of atypical neuroleptic treatment. CNS Spectr. 1997;2:1–11. [Google Scholar]
- 4.Heinrichs RW, Zankanis KK. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology. 1998;12:426–444. doi: 10.1037//0894-4105.12.3.426. [DOI] [PubMed] [Google Scholar]
- 5.Green MF. What are the functional consequences of neurocognitive deficits in schizophrenia? Am J Psychiatry. 1996;153:321–330. doi: 10.1176/ajp.153.3.321. [DOI] [PubMed] [Google Scholar]
- 6.Keefe RSE, Bilder RM, Davis SM, et al. Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the CATIE trial. Arch Gen Psychiatry. 2007;64:633–647. doi: 10.1001/archpsyc.64.6.633. [DOI] [PubMed] [Google Scholar]
- 7.Goldberg TE, Goldman RS, Burdick KE, et al. Cognitive improvement after treatment with second-generation antipsychotic medications in first-episode schizophrenia: is it a practice effect? Arch Gen Psychiatry. 2007;64:1115–1122. doi: 10.1001/archpsyc.64.10.1115. [DOI] [PubMed] [Google Scholar]
- 8.McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT. A meta-analysis of cognitive remediation in schizophrenia. Am J Psychiatry. 2007;164:1791–1802. doi: 10.1176/appi.ajp.2007.07060906. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wykes T, Reeder C, Corner J, Williams C, Everitt B. The effects of neurocognitive remediation on executive processing in patients with schizophrenia. Schizophr Bull. 1999;25:291–307. doi: 10.1093/oxfordjournals.schbul.a033379. [DOI] [PubMed] [Google Scholar]
- 10.Fisher M, Holland C, Merzenich MM, Vinogradov S. Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. Am J Psychiatry. 2009;166:805–811. doi: 10.1176/appi.ajp.2009.08050757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dickinson D, Tenhula W, Morris S, et al. A randomized, controlled trial of computer-assisted cognitive remediation for schizophrenia. Am J Psychiatry. 2010;167:170–180. doi: 10.1176/appi.ajp.2009.09020264. [DOI] [PubMed] [Google Scholar]
- 12.Kern RS, Glynn SM, Horan WP, Marder SR. Psychosocial treatments to promote functional recovery in schizophrenia. Schizophr Bull. 2009;35:347–361. doi: 10.1093/schbul/sbn177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Roder V, Mueller DR, Mueser KT, Brenner HD. Integrated psychological therapy (IPT) for schizophrenia: is it effective? Schizophr Bull. 2006;32:S81–S93. doi: 10.1093/schbul/sbl021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Medalia A, Revheim N, Herlands T. Cognitive remediation for psychological disorders: therapist guide. New York, NY: Oxford University Press; 2009. [Google Scholar]
- 15.Medalia A, Richardson R. What predicts a good response to cognitive remediation interventions? Schizophr Bull. 2005;31:942–953. doi: 10.1093/schbul/sbi045. [DOI] [PubMed] [Google Scholar]
- 16.Choi J, Medalia A. Factors associated with a positive response to cognitive remediation in a community psychiatric sample. Psychiatr Serv. 2005;56:602–604. doi: 10.1176/appi.ps.56.5.602. [DOI] [PubMed] [Google Scholar]
- 17.Twamley EW, Savla GN, Zurhellen CH, Heaton RK, Jeste DV. Development and pilot testing of a novel compensatory cognitive training intervention for people with psychosis. Am J Psychiatr Rehabil. 2008;11:144–163. doi: 10.1080/15487760801963678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kurtz MM, Seltzer JC, Shagan DS, Thime WR, Wexler BE. Computer-assisted cognitive remediation in schizophrenia: what is the active ingredient? Schizophr Res. 2007;82:251–260. doi: 10.1016/j.schres.2006.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Tarrier N, Wykes T. Is there evidence that cognitive behaviour therapy is an effective treatment for schizophrenia? A cautious or cautionary tale? Behav Res Ther. 2004;42:1377–1401. doi: 10.1016/j.brat.2004.06.020. [DOI] [PubMed] [Google Scholar]
- 20.Wykes T, Steel C, Everitt B, Tarrier N. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr Bull. 2008;34:523–537. doi: 10.1093/schbul/sbm114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bell MD, Lysaker PH, Bryson GJ. Positive and negative affect recognition is schizophrenia: a comparison with substance abuse and normal control subjects. Psychiatr Res. 1997;73:73–82. doi: 10.1016/s0165-1781(97)00111-x. [DOI] [PubMed] [Google Scholar]
- 22.Greig TC, Bryson GJ, Bell MD. Theory of mind performance in schizophrenia: diagnostic, symptom, and neuropsychological correlates. J Nerv Men Dis. 2004;192:12–18. doi: 10.1097/01.nmd.0000105995.67947.fc. [DOI] [PubMed] [Google Scholar]
- 23.Wilson BA. Cognitive rehabilitation: how it is and how it might be. J Int Neuropsychol Soc. 1997;3:487–496. [PubMed] [Google Scholar]
- 24.Silverstein SM, Hatashita-Wong M, Solak BA, et al. Effectiveness of a two-phase cognitive rehabilitation intervention for severely impaired schizophrenia patients. Psychol Med. 2005;35:829–837. doi: 10.1017/s0033291704003356. [DOI] [PubMed] [Google Scholar]
- 25.Rosenheck R, Leslie D, Keefe RSE, et al. Barriers to employment for people with schizophrenia. Am J Psychiatry. 2006;163:411–417. doi: 10.1176/appi.ajp.163.3.411. [DOI] [PubMed] [Google Scholar]
- 26.Velligan DI, Diamond P, Glahn DC, et al. The reliability and validity of the Test of Adaptive Behavior in Schizophrenia (TABS) Psychiatry Res. 2007;151:55–66. doi: 10.1016/j.psychres.2006.10.007. [DOI] [PubMed] [Google Scholar]
- 27.Fox KH, Keefe RSE, Harvey PD, Cucchiaro J, Siu C, Loebel A. Presented at: Annual Meeting of the International Society for CNS Clinical Trials Methodology. San Diego, CA: October, 2009. Test-retest characteristics of the MATRICS Consensus Cognitive Battery in a 29-site schizophrenia clinical trial of Lurasidone HCl Versus Risperidone. [Google Scholar]
- 28.Keefe R, Siu C, Fox K, et al. Test-retest characteristics of the MATRICS Consensus Cognitive Battery in a 20-site schizophrenia clinical trial of R3487/MEM3454 versus placebo. Schizophr Bull. 2009;35:358–359. [Google Scholar]
- 29.Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978;19:2–21. [PubMed] [Google Scholar]
- 30.McDermott BE. Development of an instrument for assessing self-efficacy in schizophrenic spectrum disorders. J Clin Psychol. 1995;51:320–331. [PubMed] [Google Scholar]
- 31.Choi J, Mogami T, Medalia A. Intrinsic Motivation Inventory (IMI): an adapted measure for schizophrenia research Schizophr Bull. 2010;36:966–976. doi: 10.1093/schbul/sbp030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Lieberman JA, Stroup TS, McEvoy JP, Swartz MS, et al. Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Investigators. Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. N Eng J Med. 2005;353:1209–1223. doi: 10.1056/NEJMoa051688. [DOI] [PubMed] [Google Scholar]
- 33.Hodge MA, Siciliano D, Withey P, et al. A randomized controlled trial of cognitive remediation in schizophrenia. Schizophr Bull. 2010;36:419–427. doi: 10.1093/schbul/sbn102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Wykes T, Reeder C, Landau S, et al. Cognitive remediation therapy in schizophrenia: randomised controlled trial. Br J Psychiatry. 2007;190:421–427. doi: 10.1192/bjp.bp.106.026575. [DOI] [PubMed] [Google Scholar]
- 35.Wykes T, Huddy V. Cognitive remediation for schizophrenia: it is even more complicated. Curr Opin Psychiatry. 2009;22:161–167. doi: 10.1097/YCO.0b013e328322fbf4. [DOI] [PubMed] [Google Scholar]
- 36.Vinogradov S, Fisher M, Warm H, Holland C, Kirshner MA, Pollock BG. Am J Psychiatry. The cognitive cost of anticholinergic burden: decreased response to cognitive training in schizophrenia [published online ahead of print July 01, 2009] doi:10.1176/appi.ajp.2009.09010017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.McGurk SR, Green MF, Wirshing WC, et al. Antipsychotic and anticholinergic effects on two types of spatial memory in schizophrenia. Schizophr Res. 2004;68:225–233. doi: 10.1016/S0920-9964(03)00123-3. [DOI] [PubMed] [Google Scholar]
- 38.Buchanan RW, Davis M, Goff D, et al. A summary of the FDA-NIMH-MATRICS workshop on clinical trial designs for neurocognitive drugs for schizophrenia. Schizophr Bull. 2005;31:5–19. doi: 10.1093/schbul/sbi020. [DOI] [PubMed] [Google Scholar]
- 39.Medalia A, Revheim N, Casey M. The remediation problem solving skills in schizophrenia: evidence of a persistent effect. Schizophr Res. 2002;57:165–171. doi: 10.1016/s0920-9964(01)00293-6. [DOI] [PubMed] [Google Scholar]
- 40.Fisher M, Holland C, Subramaniam K, Vinogradov S. Neuroplasticity-based cognitive training in schizophrenia: an interim report on the effects 6 months later. Schizophr Bull. 2010;36:869–879. doi: 10.1093/schbul/sbn170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Silverstein SM, Spaulding WD, Menditto AA, et al. Attention shaping: a reward-based learning method to enhance skills training outcomes in schizophrenia. Schizophr Bull. 2009;35:222–232. doi: 10.1093/schbul/sbm150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Bell MD, Bryson GJ, Greig TC, Fiszdon JM, Wexler BW. Neurocognitive enhancement therapy with work therapy: Productivity outcomes at 6- and 12-month follow ups. J Rehabil Res Dev. 2005;42:829–838. doi: 10.1682/jrrd.2005.03.0061. [DOI] [PubMed] [Google Scholar]
- 43.Silverstein SM. Measuring specific, rather than generalized, cognitive deficits and maximizing between-group effect size in studies of cognition and cognitive change. Schizophr Bull. 2008;34:645–655. doi: 10.1093/schbul/sbn032. [DOI] [PMC free article] [PubMed] [Google Scholar]