Skip to main content
PMC home page
JAMA Network logoLink to JAMA Network
. 2017 Aug 2;74(8):807–814. doi: 10.1001/jamapsychiatry.2017.1574

Placebo Response and Practice Effects in Schizophrenia Cognition Trials

Richard S E Keefe 1,2,✉, Vicki G Davis 2, Philip D Harvey 3, Alexandra S Atkins 2, George M Haig 4, Owen Hagino 5, Stephen Marder 6, Dana C Hilt 7, Daniel Umbricht 8
PMCID: PMC5710557  PMID: 28636694

Abstract

Importance

Patients’ previous experience with performance-based cognitive tests in clinical trials for cognitive impairment associated with schizophrenia can create practice-related improvements. Placebo-controlled trials for cognitive impairment associated with schizophrenia are at risk for these practice effects, which can be difficult to distinguish from placebo effects.

Objectives

To conduct a systematic evaluation of the magnitude of practice effects on the Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB) in cognitive impairment associated with schizophrenia and to examine which demographic, clinical, and cognitive characteristics were associated with improvement in placebo conditions.

Design, Setting, and Participants

A blinded review was conducted of data from 813 patients with schizophrenia who were treated with placebo in 12 randomized placebo-controlled clinical trials conducted mostly in outpatient clinics in North America, Europe, Asia, and Latin America from February 22, 2007, to March 1, 2014. A total of 779 patients provided data for the primary outcome measure at baseline and at least 1 follow-up. Seven trials had prebaseline assessments wherein the patients knew that they were not receiving treatment, allowing a comparison of practice and placebo effects in the same patients.

Interventions

Placebo compared with various experimental drug treatments.

Main Outcomes and Measures

Composite score on the MCCB.

Results

Of the 813 patients in the study (260 women and 553 men; mean [SD] age, 41.2 [11.5] years), the mean MCCB composite score at baseline was 22.8 points below the normative mean, and the mean (SEM) total change in the MCCB during receipt of placebo was 1.8 (0.2) T-score points (95% CI, 1.40-2.18), equivalent to a change of 0.18 SD. Practice effects in the 7 studies in which there was a prebaseline assessment were essentially identical to the postbaseline placebo changes. Baseline factors associated with greater improvements in the MCCB during receipt of placebo included more depression/anxiety (F1,438 = 5.41; P = .02), more motivation (F1,272 = 4.63; P = .03), and less improvement from screening to baseline (F1,421 = 59.32; P < .001).

Conclusions and Relevance

Placebo effects were minimal and associated with the number of postbaseline assessments and several patient characteristics. Given that the patients performed 2.28 SDs below normative standards on average at baseline, a mean placebo-associated improvement of less than 0.2 SD provides evidence that ceiling effects do not occur in these trials. These minimal changes in the MCCB could not be responsible for effective active treatments failing to separate from placebo.

This study evaluates the magnitude of practice effects on the Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery in cognitive impairment associated with schizophrenia and examines which characteristics were associated with improvement in placebo conditions.

Key Points

Questions

What is the magnitude of learning effects in patients with schizophrenia receiving placebo on the criterion standard Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB), and can these learning effects be anticipated in individual patients?

Findings

This blinded review of 12 trials including 813 patients with schizophrenia suggested that the mean change in the MCCB during receipt of placebo was less than 0.2 SDs, a small effect. Factors associated with greater improvements on the MCCB included more motivation, more depression/anxiety, and less improvement from screening to baseline.

Meaning

These changes in the MCCB are too small to be responsible for effective active treatments failing to separate from placebo.

Introduction

Cognitive impairment associated with schizophrenia is severe and a primary cause of poor functional outcomes. Extensive resources have been devoted to developing pharmacologic, psychosocial, and cognitive remediation treatments to reduce cognitive and functional impairment in patients with schizophrenia. To our knowledge, no treatments to date have had sufficient benefit to warrant regulatory approval. Because there are no effective medical treatments for cognitive impairment associated with schizophrenia, it is challenging to determine whether the lack of success of previous treatment development programs has been due to inadequacies of the treatments being tested or to weaknesses in the methods used.

Cognitive enhancement studies have typically used a regulatory pathway that was developed during the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative. Central to this method has been the assessment of cognition with a set of performance-based tests referred to as the MATRICS Consensus Cognitive Battery (MCCB), which was selected on the basis of several criteria, including usefulness as a repeated measure, defined as high test-retest reliability combined with minimal practice effects. However, the MCCB validation study used a single test-retest assessment, which does not reflect the complexity of standard clinical trials that have more than 2 assessments performed with the MCCB and multiple treatment conditions.

Patients with schizophrenia manifest consistent cognitive performance over time with a variety of different tests and may generate practice effects when they are reassessed. However, most clinical trials have used parallel group designs with a placebo control; thus, it is difficult to determine if improved performance over time in patients receiving placebo is due to simple learning or practice effects or to the expectation bias inherent in a placebo-controlled design. A comparison of the improvement in patients receiving placebo with those who were aware that they were not receiving any treatment would help to determine if expectation bias increases the known improvement due to practice effects.

Regardless of their source, improvements on cognitive tests in patients receiving placebo may be so large that treatment effects are impossible to detect. The opposite speculation has equivalent face validity and has been supported by single studies; if an individual is familiar with being assessed and with the assessment materials, additional assessments are less likely to lead to gains in performance based solely on test exposure. Furthermore, the magnitude of expected practice and placebo response of the MCCB, which is used in most large multisite clinical trials for cognitive impairment associated with schizophrenia, is unknown. Finally, while certain patients with schizophrenia and depression who are receiving placebo demonstrate large symptom responses that can be anticipated based on their baseline characteristics, it has not been determined if patients receiving placebo who respond in schizophrenia cognition trials can be similarly identified.

There are clear implications in these questions for the interpretation of previous results and the design of future clinical trials. If individual differences among patients, including differences in their performance change from screening to baseline, are associated with postbaseline retest changes, these measures could be used as factors to be considered in statistical analysis or in patient randomization and stratification.

In this study, we merged data from 12 separate clinical trials, including 813 patients who were randomized to receive placebo during the course of a double-blind pharmacologic intervention study, and investigated the characteristics of retest-associated improvements in performance on the MCCB. We compared prerandomization improvements with postrandomization improvements in patients receiving placebo, and we assessed the effect of a number of measures on the magnitude of postbaseline improvement, including the number of retest assessments, the spacing of the retest assessments, the time elapsed since baseline while receiving placebo, and the number of treatment arms of the study. Finally, we examined a variety of patient demographic and clinical characteristics to determine if they are associated with cognitive placebo response.

Methods

Sample Characteristics

This review included 813 patients with schizophrenia who were treated with placebo in randomized clinical trials from February 22, 2007, to March 1, 2014. The Table describes the demographic and baseline characteristics of the merged data set, with mean (SD) values for continuous variables and numbers (percentages) for categorical variables. As is typical of clinical trial data on patients with schizophrenia, approximately two-thirds of the patients (553 [68.0%]) were men. Most of the patients were evaluated in North America, with samples also collected in Europe, Asia, and Latin America. eTable 1 in the Supplement describes the characteristics of the 12 trials from which the data were merged, including ClinicalTrials.gov identification number, total number of individuals randomized and assigned to receive placebo, dates of the trial, the number of treatment groups, and the number of MCCB assessments performed. Each clinical trial was approved by central and/or local institutional review boards, and all study participants provided written informed consent.

Table. Demographic and Baseline Characteristics.

Characteristic Valuea
Age, mean (SD), y (n = 813) 41.2 (11.45)
Sex (n = 813)
Male 553 (68.0)
Female 260 (32.0)
Race (n = 813)
White 406 (49.9)
Black 290 (35.7)
Other 117 (14.4)
Geographical region (n = 813)
North America 536 (65.9)
Eastern Europe 118 (14.5)
Asia 63 (7.7)
Latin America 50 (6.2)
Western Europe 46 (5.7)
Postsecondary education (n = 552) 151 (27.4)
Current smoker (n = 603) 258 (42.8)
Age at onset of illness, mean (SD), y (n = 233) 23.3 (8.34)
Duration of illness <10 y (n = 626) 234 (37.4)
Baseline antipsychotic (n = 488)
Risperidone/paliperidone 159 (32.6)
Olanzapine 118 (24.2)
Other 89 (18.2)
Quetiapine 66 (13.5)
Aripiprazole 56 (11.5)
Baseline MCCB composite T score, mean (SD) (n = 782) 27.2 (12.78)
Baseline UPSA-2 total, mean (SD) (n = 225) 87.8 (16.10)
Baseline SCoRS total, mean (SD) (n = 158) 38.7 (9.85)
Baseline PANSS total, mean (SD) (n = 669) 59.7 (15.01)
Baseline NSA-16 total, mean (SD) (n = 489) 56.5 (11.96)
Open in a new tab

Abbreviations: MCCB, Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery; NSA-16, Negative Symptom Assessment; PANSS, Positive and Negative Syndrome Scale; SCoRS, Schizophrenia Cognition Rating Scale; UPSA-2, University of California at San Diego Performance-based Skills Assessment, 2nd edition.

a

Data are presented as number (percentage) of patients unless otherwise indicated.

Thirty-one patients had 1 or more missing MCCB domain values at the baseline visit, resulting in 782 patients with a cognitive composite score derived from all 7 MCCB domains at baseline. Three patients did not have at least 1 follow-up score, resulting in a final sample size of 779. An additional 54 patients had test results missing in a domain that had additional tests, so a domain score was able to be calculated. For all of the trials, patients were treated with a stable dose of antipsychotic medication, and most patients were receiving only 1 antipsychotic. The mean (SD) MCCB cognitive composite score at first assessment was 27.2 (12.78), consistent with similarly timed assessment scores from a larger data set of more than 4000 patients. This score reflects performance that is more than 2 SDs below normative standards. An improvement of 1 point on the MCCB cognitive composite score reflects a 0.1-SD improvement in normative scores, which means that the patients were a mean (SD) 23.0 (2.3) points below the normative population mean and at least a mean (SD) of 73.0 (7.3) points below a maximum score on the MCCB composite.

MCCB Assessments

The MCCB subtests are organized into the following 7 domains: (1) Speed of Processing: Trailmaking Test, Brief Assessment of Cognition Symbol Coding, and Category Fluency; (2) Attention-Vigilance: Continuous Performance Test-Identical Pairs; (3) Working memory: Wechsler Memory Scale–III Spatial Span and Letter-Number Span; (4) Verbal learning: Hopkins Verbal Learning Test-Revised; (5) Visual learning: Brief Visuospatial Memory Test-Revised; (6) Reasoning and Problem Solving: Neuropsychological Assessment Battery Mazes; and (7) Social Cognition: Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) Managing Emotions.

Data Quality Assurance

All testers who interacted with patients were first trained on the administration and scoring of the MCCB using video and group training sessions and were individually certified by an MCCB expert. All MCCB data were scored locally or sent to a central site, where they were scored or rescored. Training, data collection, and data quality assurance were implemented or supervised by an experienced psychologist (R.S.E.K. or A.S.A.) as per the guidelines outlined in the MCCB manual.

MCCB Composite Scores

For the 10 subtests, 7 domains, and 2 composite scores of the MCCB, the scoring program yields T scores that are standardized and corrected for age and sex, based on previously collected normative data matched for age, sex, and race/ethnicity to the US Census. Each standardized measure has a mean (SD) of 50 (10). The cognitive composite is the standardized total of the 7 domains. The neurocognitive composite is calculated similarly but does not include Social Cognition.

Symptom Assessments

Positive and Negative Syndrome Scale

Data from the Positive and Negative Syndrome Scale (PANSS) were available in 7 studies. The PANSS symptoms were grouped into 5 factors (positive symptoms, negative symptoms, disorganized thought, hostility/excitement, and depression/anxiety), based on a previously established model.

Negative Symptom Assessment

In 5 studies, the Negative Symptom Assessment was used to examine negative symptoms in 4 factors: communication, emotion, motivation, and sociality.

Statistical Analysis

Data were combined from the intent-to-treat populations of 12 MCCB clinical trials. Visit schedules varied considerably among the studies, ranging from 2 to 6 MCCB assessments (1-4 postbaseline reassessments) during a period of 4 to 24 weeks. Change from baseline in the MCCB cognitive composite score was investigated using a basic linear mixed model for repeated measures, controlling for baseline value and week of assessment nested within a random study effect. The ability of patient-level and study-level variables to estimate the magnitude of the placebo effect was analyzed with separate mixed models for each predictor. Change in MCCB composite score between screening and baseline (“pure” practice effect) was compared between studies with analysis of covariance adjusting for the screening value. This analysis included the 7 studies (n = 558) that administered the MCCB prior to and again at patients’ baseline assessment, during a period in which patients knew that they were not receiving treatment. P < .05 was considered significant. Complete information on the statistical plan is in the eAppendix in the Supplement.

Results

The mean MCCB composite score at baseline was 22.8 points below the normative mean (Table). Figure 1 describes the mean MCCB cognitive composite score over time in all 12 studies. The full 56-week data for 3 studies are available in the eFigure in the Supplement. The overall mean (SEM) change in the MCCB cognitive composite during receipt of placebo, after adjusting for baseline score, week of assessment, and study, was 1.8 (0.2) T-score points (95% CI, 1.40-2.18), with a range across studies of 0.6 to 3.3 points (Figure 2). The effects of baseline score (F1,642 = 7.52), week of assessment (F11,181 = 3.43), and study (F11,184 = 2.82) were all statistically significant (P < .01). Mean (SEM) change in scores for the 10 subtests comprising the MCCB were all small, ranging from 0.2 (0.3) (MSCEIT managing emotions) to 2.2 (0.3) (Trail Making A) T-score points (eTable 2 in the Supplement).

Figure 1. Mean Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB) Cognitive Composite T Scores by Study and Visit for Patients Receiving Placebo.

Figure 1.

Open in a new tab

Unadjusted mean MCCB cognitive composite scores are plotted by study and week of assessment through week 24. The line color denotes studies that assessed cognition at a screening visit (“Yes”) and those that began assessments at baseline (“No”). The line thickness is proportional to the study sample size, and assessment time points for each study are denoted by markers.

Figure 2. Change in Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery Cognitive Composite T Score by Study for Patients Receiving Placebo.

Figure 2.

Open in a new tab

Least squares mean change, 95% CIs, sample size, and number of postbaseline visits are presented for each study and overall. Individual study results are from separate mixed-effects repeated measures models of the change from baseline through week 24, with a fixed categorical effect for visit and a continuous fixed covariate for the baseline score, assuming an unstructured covariance matrix. In the overall model, visit was nested within a random study effect.

Prerandomization Practice Effects

Seven of the studies had a prebaseline screening assessment (n = 558). The changes from screening to baseline assessment for the cognitive composite score varied from 0.7 to 3.1 MCCB T-score points (mean [SEM], 2.2 [0.25] T-score points). The analysis of covariance for the practice effects found a significant effect of retesting (F1,502 = 10.98; P = .001). The effect of the study was nonsignificant (F6,502 = 1.69; P = .12), indicating that the retest effects were not significantly different across studies.

Postbaseline Changes in Patients Receiving Placebo

To examine the effects of a prebaseline screening assessment on postbaseline placebo effects, the postbaseline changes in the 7 studies with a screening assessment were compared with the changes for the 5 studies without such an assessment. When a dichotomous variable indicating the presence or absence of a screening MCCB assessment was added to the model as a covariate for the postbaseline analyses, its effect was not statistically significant (F1,695 = 1.64; P = .20), indicating that performing prebaseline screening assessments was not associated with differences in postbaseline changes while the patients were receiving placebo.

We also explored the association between postbaseline changes and the numbers of weeks and assessments that had occurred since baseline, adjusting for baseline scores and whether the study had a screening visit. The effect of time since baseline was statistically significant (F1,507 = 15.28; P < .001), as was the number of retest assessments (F1,463 = 16.39; P < .001). A screening visit did not exert a statistically significant influence on postbaseline change (F1,649 = 3.03; P = .08). The mean (SEM) change in MCCB composite scores per week, after controlling for the other factors, was 0.08 (0.02) T-score points per week of placebo exposure and 0.73 (0.18) T-score points per follow-up assessment. The increases by assessment were 1.2 MCCB T-score points for 1 postbaseline assessments, 1.9 points for 2 postbaseline assessments, 2.2 points for 3 postbaseline assessments, and 3.6 points for 4 postbaseline assessments. Thus, the points gained per retest assessment during receipt of placebo generally decreased with an increased number of assessments, and the retesting effects were larger than the effects of time of placebo exposure alone.

Factors Associated With Placebo Response

The following variables were associated with placebo response: the Negative Symptoms Assessment motivation factor score (F1,272 = 4.63; P = .03), the PANSS Depression/Anxiety Marder Factor score (F1,438 = 5.41; P = .02), the practice effect between screening and baseline MCCB (F1,421 = 59.32; P < .001), 4 vs 3 treatment groups (F1,652 = 5.14; P = .02), and the study sample size (F1,725 = 4.65; P = .03). A larger placebo response was associated with more motivation, greater depression/anxiety, lower prebaseline practice effects, 4 vs 3 treatment groups, and a larger study sample.

Given the robustness of the screening to baseline MCCB composite score change as a factor associated with response, we further investigated the effect of classifying patients into the following 3 groups based on the magnitude of the change in their screening to baseline practice effect: negative (≤−5-point change), small or neutral (>−5- to 5-point change), and positive (>5-point change). As demonstrated in Figure 3, there were robust differences between the groups. Patients who manifested negative practice effects (mean [SEM], 4.8 [0.68] T-score points) demonstrated larger placebo effects (t425 = 3.43; P < .001) than those with small practice effects (mean [SEM], 2.2 [0.26] T-score points), who in turn demonstrated larger placebo effects (t410 = 4.13; P < .001) than those who had positive practice effects (mean [SEM], −0.1 [0.51] T-score points).

Figure 3. Change in Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery Cognitive Composite T Score by Level of Practice Effect in Studies With a Screening Assessment.

Figure 3.

Open in a new tab

Least squares mean change from baseline through week 24 is plotted by level of practice effect as calculated by the T-score change in the cognitive composite from screening to baseline. Results are from a mixed-effect repeated measures model with a continuous fixed covariate for the baseline score and a categorical effect for visit nested within a random study effect, assuming an unstructured covariance matrix. This analysis was limited to the 7 studies that assessed cognition at screening (N = 558) and revealed a significantly increasing placebo effect with decreased practice effect between screening and baseline (P < .001 overall and for all pairwise comparisons). Patients were classified into 3 groups based on the magnitude of the change in their screening to baseline practice effect: negative (change of ≤–5 points), small or neutral (change of >–5 to 5 points), and positive (>5-point change). Forty-two patients without composite scores at screening, baseline, and at least 1 follow-up were excluded from the analysis (n = 516). Test for practice effect-by-week interaction was nonsignificant (P = .67).

Discussion

The data from this study were generated from the placebo condition in 12 separate randomized, placebo-controlled clinical trials using the MCCB as the primary end point to assess response to treatment with a pharmacologic intervention in patients with schizophrenia. The mean amount of total improvement during treatment in patients receiving placebo was approximately 2 T-score points on the MCCB composite score, consistent with a Cohen d effect size of 0.18, which is slightly smaller than the Cohen definition of small. The amount of change due to the practice effect from prebaseline to baseline was almost identical to the mean amount of change reported in placebo conditions. Studies using a prebaseline assessment did not have smaller postbaseline placebo treatment effects than did studies with the first MCCB assessment at baseline. The size of the placebo effect was more influenced by the number of postbaseline retest assessments than by the duration of placebo treatment, with the number of reassessments associated with an increase in scores. Key additional factors associated with the magnitude of placebo response in these trials included smaller improvement from prebaseline to baseline assessment with the MCCB, less severe negative symptoms, more severe depression/anxiety symptoms, 4 vs 3 treatment groups in the trial, and larger sample size.

There are several important implications of these findings. The most obvious conclusion from these data is that the concept of a substantial placebo effect in schizophrenia clinical trials using the MCCB—one that may obscure the detection of meaningful clinical change—is clearly not substantiated. The data in this report suggest that cognitive performance assessed with the MCCB is 2.28 SDs below the mean of the healthy general population at baseline. Thus, the 0.18-SD change reported with placebo is exceeded by the size of the baseline cognitive impairment in patients with schizophrenia by a factor of 12. This amount of postbaseline improvement is almost identical to the amount of prebaseline improvement in these trials and is consistent with the amount of prebaseline improvement reported in a larger sample of 1814 patients from 10 trials. Thus, the amount of improvement in the placebo group in cognition clinical trials is consistent with the expected amount of improvement due to learning effects and is small enough to be easily overwhelmed by a legitimate treatment effect. The notion that this magnitude of retest improvement accounts for the negative results in previous cognitive enhancement trials seems unlikely. For the cognitive changes associated with a pharmacologic agent to be of real benefit to patients with schizophrenia, they must exceed the small learning effect represented in the placebo condition in these trials, which is also a part of the cognitive response in patients receiving active treatment.

This amount of improvement with placebo in performance-based cognitive assessments is clearly less than that seen in clinical trials across different neuropsychiatric conditions, including symptom rating scales such as the PANSS and the Hamilton Depression Rating Scale. Because the amount of cognitive improvement in patients with cognitive impairment associated with schizophrenia receiving placebo is almost identical to the learning effects that are seen prior to baseline, the increases are almost certainly due to practice, comfort, or learning effects and not to the expectation biases that can have a tremendous effect on rating scales.

Our results can be interpreted to provide recommendations regarding the strategy of testing patients once prior to baseline to expose them to the testing process or to minimize a learning effect. A simple analysis comparing patients who were tested prior to baseline with patients who were not suggests that they had similar improvements in placebo conditions and that this strategy does not systematically lead to reduced postbaseline changes. Also, a larger number of assessments after baseline was associated with slightly larger MCCB improvements, although the magnitude of between-assessment improvement may diminish as the number of postbaseline assessments increases.

These results have implications for study design. First, researchers will need to weigh the value of additional assessments during clinical trials against the potential for an additional learning effect in patients receiving placebo and active treatment, although this effect is small. Second, important information may be gained from a prebaseline assessment. Patients whose cognitive performance worsened from screening to baseline had the largest placebo effect, and those with the largest improvements from screening to baseline had no placebo effect at all. These results, consistent with the principle of regression to the mean, suggest that, for schizophrenia cognition studies, the concept of eliminating “placebo responders” who would then go on to have large placebo effects during a trial is not a viable approach. However, a screening assessment may produce information that can facilitate study design, such as stratification of patients based on potential placebo effects. It may also allow statistical approaches that adjust for screening values (along with the customary baseline values) in the primary efficacy model as a method for reducing the noise associated with large fluctuations in patients who may have had exceptionally good or bad days at the baseline visit. Preliminary analyses using this method suggest that it may increase precision of assessing treatment effects in cognitive treatment trials. Other individual difference characteristics found in our analyses to be associated with placebo group improvements, including more severe depression/anxiety and more motivation, could be combined with prebaseline changes to equalize factors that appear to increase the placebo response.

The present results raise a more general question of whether cognitive impairments measured by neuropsychological tests are indeed mutable with pharmacologic intervention in schizophrenia, and whether other measures of brain function are better candidates to serve this purpose. First, neuropsychological tests have demonstrated sensitivity to changes under a variety of different conditions in schizophrenia. They are highly sensitive to the effects of alcohol, nicotine, anticholinergic medications, and stimulants, as well as other drugs. Second, because they have consistently demonstrated a clinically significant magnitude of improvement in response to cognitive remediation strategies in schizophrenia, the possibility that neuropsychological tests could be a plastic attribute in response to behavioral intervention but a stable trait in response to pharmacologic intervention does not seem reasonable. Third, although it is often assumed that the effects of drugs on other, more biologically proximal measures of brain function, such as brain imaging or electrophysiology, are more powerful than on neuropsychological tests, there is little evidence to support this assumption. Finally, several early-phase studies have demonstrated improvements on the MCCB with pharmacologic intervention that were not replicated in larger programs. It is not at all clear whether the inability of later studies to replicate these findings were due to inefficacy of the pharmacologic intervention, insensitivity of the cognitive tests, or other weaknesses of study design and implementation. Only one phase 3 program has used the MCCB as a primary end point, and the results of that program were inconclusive.

Conclusions

In 12 double-blind, placebo-controlled, clinical trials for cognitive impairment associated with schizophrenia, the magnitude of placebo effects on the MCCB was small and not different from the retesting effects during the pretreatment screening period when patients knew that they were not receiving treatment. The magnitude of the placebo effect was not large enough to obscure a treatment effect with a small to medium effect size. There are patient and study characteristics that have a modest influence on postbaseline retest changes, and the number of reassessments is minimally associated with changes in performance on the MCCB.

Supplement.

eAppendix. Additional Statistical Analysis

eFigure. Mean MCCB Cognitive Composite T-Score

eTable 1. Study Characteristics

eTable 2. LS Mean Change from Baseline to Week 24 for MCCB Subtests, Domains, and Composites

Click here for additional data file. (257.2KB, pdf)

References

  • 1.Keefe RS, Meltzer HA, Dgetluck N, et al. . Randomized, double-blind, placebo-controlled study of encenicline, an α7 nicotinic acetylcholine receptor agonist, as a treatment for cognitive impairment in schizophrenia. Neuropsychopharmacology. 2015;40(13):3053-3060. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Haig GM, Bain EE, Robieson WZ, Baker JD, Othman AA. A randomized trial to assess the efficacy and safety of ABT-126, a selective α7 nicotinic acetylcholine receptor agonist, in the treatment of cognitive impairment in schizophrenia. Am J Psychiatry. 2016;173(8):827-835. [DOI] [PubMed] [Google Scholar]
  • 3.Bugarski-Kirola D, Blaettler T, Arango C, et al. . Bitopertin in negative symptoms of schizophrenia—results from the phase III FlashLyte and DayLyte studies [published online December 15, 2016]. Biol Psychiatry. 2016. doi: 10.1016/j.biopsych.2016.11.014 [DOI] [PubMed] [Google Scholar]
  • 4.Umbricht D, Alberati D, Martin-Facklam M, et al. . Effect of bitopertin, a glycine reuptake inhibitor, on negative symptoms of schizophrenia: a randomized, double-blind, proof-of-concept study. JAMA Psychiatry. 2014;71(6):637-646. [DOI] [PubMed] [Google Scholar]
  • 5.Keefe RS, Buchanan RW, Marder SR, et al. . Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? Schizophr Bull. 2013;39(2):417-435. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Schooler NR, Buchanan RW, Laughren T, et al. . Defining therapeutic benefit for people with schizophrenia: focus on negative symptoms. Schizophr Res. 2015;162(1-3):169-174. [DOI] [PubMed] [Google Scholar]
  • 7.Hogarty GE, Greenwald D, Ulrich RF, et al. . Three-year trials of personal therapy among schizophrenic patients living with or independent of family, II: effects on adjustment of patients. Am J Psychiatry. 1997;154(11):1514-1524. [DOI] [PubMed] [Google Scholar]
  • 8.Eack SM, Hogarty GE, Greenwald DP, Hogarty SS, Keshavan MS. Effects of cognitive enhancement therapy on employment outcomes in early schizophrenia: results from a two-year randomized trial. Res Soc Work Pract. 2011;21(1):32-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Fisher M, Holland C, Merzenich MM, Vinogradov S. Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. Am J Psychiatry. 2009;166(7):805-811. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Fisher M, Mellon SH, Wolkowitz O, Vinogradov S. Neuroscience-informed auditory training in schizophrenia: a final report of the effects on cognition and serum brain-derived neurotrophic factor. Schizophr Res Cogn. 2016;3:1-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Mahncke H, Kim SJ, Stasio C, et al. Results of an FDA device clearance trial for Plasticity-Based Adaptive Cognitive Remediation (PACR). Paper presented at: 5th Schizophrenia International Research Society Conference; April 5, 2016; Florence, Italy. [Google Scholar]
  • 12.Kantrowitz JT, Sharif Z, Medalia A, et al. . A multicenter, rater-blinded, randomized controlled study of auditory processing–focused cognitive remediation combined with open-label lurasidone in patients with schizophrenia and schizoaffective disorder. J Clin Psychiatry. 2016;77(6):799-806. [DOI] [PubMed] [Google Scholar]
  • 13.Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am J Psychiatry. 2011;168(5):472-485. [DOI] [PubMed] [Google Scholar]
  • 14.Buchanan RW, Davis M, Goff D, et al. . A summary of the FDA-NIMH-MATRICS workshop on clinical trial design for neurocognitive drugs for schizophrenia. Schizophr Bull. 2005;31(1):5-19. [DOI] [PubMed] [Google Scholar]
  • 15.Buchanan RW, Keefe RSE, Umbricht D, Green MF, Laughren T, Marder SR. The FDA-NIMH-MATRICS guidelines for clinical trial design of cognitive-enhancing drugs: what do we know 5 years later? Schizophr Bull. 2011;37(6):1209-1217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Green MF, Nuechterlein KH, Kern RS, et al. . Functional co-primary measures for clinical trials in schizophrenia: results from the MATRICS Psychometric and Standardization Study. Am J Psychiatry. 2008;165(2):221-228. [DOI] [PubMed] [Google Scholar]
  • 17.Nuechterlein KH, Green MF, Kern RS, et al. . The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am J Psychiatry. 2008;165(2):203-213. [DOI] [PubMed] [Google Scholar]
  • 18.Kern RS, Nuechterlein KH, Green MF, et al. . The MATRICS Consensus Cognitive Battery, part 2: co-norming and standardization. Am J Psychiatry. 2008;165(2):214-220. [DOI] [PubMed] [Google Scholar]
  • 19.Umbricht D, Keefe RS, Murray S, et al. . A randomized, placebo-controlled study investigating the nicotinic α7 agonist, RG3487, for cognitive deficits in schizophrenia. Neuropsychopharmacology. 2014;39(7):1568-1577. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Javitt DC, Buchanan RW, Keefe RSE, et al. . Effect of the neuroprotective peptide davunetide (AL-108) on cognition and functional capacity in schizophrenia. Schizophr Res. 2012;136(1-3):25-31. [DOI] [PubMed] [Google Scholar]
  • 21.Goldberg TE, Keefe RS, Goldman RS, Robinson DG, Harvey PD. Circumstances under which practice does not make perfect: a review of the practice effect literature in schizophrenia and its relevance to clinical treatment studies. Neuropsychopharmacology. 2010;35(5):1053-1062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, Coughenour L. The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr Res. 2004;68(2-3):283-297. [DOI] [PubMed] [Google Scholar]
  • 23.Papakostas GI, Østergaard SD, Iovieno N. The nature of placebo response in clinical studies of major depressive disorder. J Clin Psychiatry. 2015;76(4):456-466. [DOI] [PubMed] [Google Scholar]
  • 24.Haig GM, Bain E, Robieson W, Othman AA, Baker J, Lenz RA. A randomized trial of the efficacy and safety of the H3 antagonist ABT-288 in cognitive impairment associated with schizophrenia. Schizophr Bull. 2014;40(6):1433-1442. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Clinicaltrials.gov A phase 2 study to evaluate ABT-126 for the treatment of cognitive deficits in schizophrenia. NCT01678755. https://clinicaltrials.gov/ct2/show/NCT01678755. Accessed April 4, 2015.
  • 26.Haig G, Wang D, Othman AA, Zhao J. The α7 nicotinic agonist ABT-126 in the treatment of cognitive impairment associated with schizophrenia in nonsmokers: results from a randomized controlled phase 2b study. Neuropsychopharmacology. 2016;41(12):2893-2902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Buchanan RW, Keefe RS, Lieberman JA, et al. . A randomized clinical trial of MK-0777 for the treatment of cognitive impairments in people with schizophrenia. Biol Psychiatry. 2011;69(5):442-449. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Clinicaltrials.gov Efficacy and safety of AVE1625 as a co-treatment with antipsychotic therapy in schizophrenia (CONNECT). NCT00439634. https://clinicaltrials.gov/ct2/show/NCT00439634. Accessed January 10, 2015.
  • 29.Clinicaltrials.gov A study of LY2140023 in schizophrenia patients with prominent negative symptoms. NCT01052103. https://clinicaltrials.gov/ct2/show/NCT01052103. Accessed June 21, 2015.
  • 30.Patterson TL, Goldman S, McKibbin CL, Hughs T, Jeste DV. UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr Bull. 2001;27(2):235-245. [DOI] [PubMed] [Google Scholar]
  • 31.Georgiades A, Davis VG, Atkins A, et al. . Psychometric characteristics of the MATRICS Consensus Cognitive Battery in a large pooled cohort of stable schizophrenia patients [published online April 19, 2017]. Schizophr Res. doi: 10.1016/j.schres.2017.03.040 [DOI] [PubMed] [Google Scholar]
  • 32.Green MF, Harris JG, Nuechterlein KH. The MATRICS consensus cognitive battery: what we know 6 years later. Am J Psychiatry. 2014;171(11):1151-1154. [DOI] [PubMed] [Google Scholar]
  • 33.Kay SR, Fiszbein A, Opler LA. The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13(2):261-276. [DOI] [PubMed] [Google Scholar]
  • 34.Marder SR, Davis JM, Chouinard G. The effects of risperidone on the five dimensions of schizophrenia derived by factor analysis: combined results of the North American trials. J Clin Psychiatry. 1997;58(12):538-546. [DOI] [PubMed] [Google Scholar]
  • 35.Alphs LD, Summerfelt A, Lann H, Muller RJ. The negative symptom assessment: a new instrument to assess negative symptoms of schizophrenia. Psychopharmacol Bull. 1989;25(2):159-163. [PubMed] [Google Scholar]
  • 36.Axelrod BN, Goldman RS, Alphs LD. Validation of the 16-item negative symptom assessment. J Psychiatr Res. 1993;27(3):253-258. [DOI] [PubMed] [Google Scholar]
  • 37.Cohen J, ed. Statistical Power Analysis for the Behavioral Sciences. Orlando, FL: Academic Press, Inc; 1977. [Google Scholar]
  • 38.Ivanova A, Qaqish B, Schoenfeld DA. Optimality, sample size, and power calculations for the sequential parallel comparison design. Stat Med. 2011;30(23):2793-2803. [DOI] [PubMed] [Google Scholar]
  • 39.Doros G, Pencina M, Rybin D, Meisner A, Fava M. A repeated measures model for analysis of continuous outcomes in sequential parallel comparison design studies. Stat Med. 2013;32(16):2767-2789. [DOI] [PubMed] [Google Scholar]
  • 40.Brannan S. The difficulties in going from P2 to P3 in CNS trials: red flags from a recent CIS program. Paper presented at: 13th Annual Scientific Meeting of the International Society of CNS Clinical Trials; February 23, 2017; Washington, DC. [Google Scholar]
  • 41.Peterson JB, Rothfleisch J, Zelazo PD, Pihl RO. Acute alcohol intoxication and cognitive functioning. J Stud Alcohol. 1990;51(2):114-122. [DOI] [PubMed] [Google Scholar]
  • 42.Smith RC, Warner-Cohen J, Matute M, et al. . Effects of nicotine nasal spray on cognitive function in schizophrenia. Neuropsychopharmacology. 2006;31(3):637-643. [DOI] [PubMed] [Google Scholar]
  • 43.Keefe RS, Bilder RM, Davis SM, et al. ; CATIE Investigators; Neurocognitive Working Group . Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the CATIE Trial. Arch Gen Psychiatry. 2007;64(6):633-647. [DOI] [PubMed] [Google Scholar]
  • 44.Szeszko PR, Bilder RM, Dunlop JA, Walder DJ, Lieberman JA. Longitudinal assessment of methylphenidate effects on oral word production and symptoms in first-episode schizophrenia at acute and stabilized phases. Biol Psychiatry. 1999;45(6):680-686. [DOI] [PubMed] [Google Scholar]
  • 45.Kantrowitz JT, Epstein ML, Beggel O, et al. . Neurophysiological mechanisms of cortical plasticity impairments in schizophrenia and modulation by the NMDA receptor agonist D-serine. Brain. 2016;139(pt 12):3281-3295. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement.

eAppendix. Additional Statistical Analysis

eFigure. Mean MCCB Cognitive Composite T-Score

eTable 1. Study Characteristics

eTable 2. LS Mean Change from Baseline to Week 24 for MCCB Subtests, Domains, and Composites

Click here for additional data file. (257.2KB, pdf)

Articles from JAMA Psychiatry are provided here courtesy of American Medical Association

RESOURCES