Skip to main content
PMC home page
JAMA Network logoLink to JAMA Network
. 2019 Jun 3;76(10):1063–1073. doi: 10.1001/jamapsychiatry.2019.1530

Evaluation of Differences in Individual Treatment Response in Schizophrenia Spectrum Disorders

A Meta-analysis

Stephanie Winkelbeiner 1,2,3,4,✉, Stefan Leucht 5, John M Kane 1,2,3, Philipp Homan 1,2,3,✉
PMCID: PMC6547253  PMID: 31158853

Key Points

Question

Is there evidence from randomized clinical trials that patients respond differently to antipsychotic drugs?

Findings

In this meta-analysis of 52 randomized clinical trials involving 15 360 patients with a schizophrenia or schizoaffective diagnosis, the outcome variability in the antipsychotic drug treatment group was not higher but slightly lower than that in the placebo control group.

Meaning

This study cannot rule out that individual differences in drug response might still exist, but it does question the assumption of a personal element of response to antipsychotic treatment.

Abstract

Importance

An assumption among clinicians and researchers is that patients with schizophrenia vary considerably in their response to antipsychotic drugs in randomized clinical trials (RCTs).

Objective

To evaluate the overall variation in individual treatment response from random variation by comparing the variability between treatment and control groups.

Data Sources

Cochrane Schizophrenia, MEDLINE/PubMed, Embase, PsycINFO, Cochrane CENTRAL, BIOSIS Previews, ClinicalTrials.gov, and World Health Organization International Clinical Trials Registry Platform from January 1, 1955, to December 31, 2016.

Study Selection

Double-blind, placebo-controlled, RCTs of adults with a diagnosis of schizophrenia spectrum disorders and prescription for licensed antipsychotic drugs.

Data Extraction and Synthesis

Means and SDs of the Positive and Negative Syndrome Scale pretreatment and posttreatment outcome difference scores were extracted. Data quality and validity were ensured by following the PRISMA guidelines.

Main Outcomes and Measures

The outcome measure was the overall variability ratio of treatment to control in a meta-analysis across RCTs. Individual variability ratios were weighted by the inverse-variance method and entered into a random-effects model. A personal element of response was hypothesized to be reflected by a substantial overall increase in variability in the treatment group compared with the control group.

Results

An RCT was simulated, comprising 30 patients with schizophrenia randomized to either the treatment or the control group. The different components of variation in RCTs were illustrated with simulated data. In addition, we assessed the variability ratio in 52 RCTs involving 15 360 patients with a schizophrenia or schizoaffective diagnosis. The variability was slightly lower in the treatment compared with the control group (variability ratio = 0.97; 95% CI, 0.95-0.99; P = .01).

Conclusions and Relevance

In this study, no evidence was found in RCTs that antipsychotic drugs increased the outcome variance, suggesting no personal element of response to treatment but instead indicating that the variance was slightly lower in the treatment group than in the control group; although the study cannot rule out that subsets of patients respond differently to treatment, it suggests that the average treatment effect is a reasonable assumption for the individual patient.

This meta-analysis assesses individualized response to treatment and reviews treatment-by-patient interaction among patients with schizophrenia.

Introduction

Personalized medicine is based on a widely held assumption that patients differ substantially in their response to treatments. The goal of personalized medicine is to find the right treatment for the right patient. Psychiatry is no exception. An assumption among clinicians and researchers alike is that the response to antipsychotic drugs by patients with psychosis differs considerably between individuals.1

We report that this assumption may be ungrounded. Although variation in the observed treatment responses obviously exists, it is crucial to distinguish between observed and true treatment response: observed response consists of true response plus regression to the mean, some placebo effects, and random terms such as (but not restricted to) measurement error. First, we exemplify why confusing observed with true treatment response is so common, and we use simulated data to show how variation that is purely random and unrelated to permanent differences in treatment response may suggest the need for personalized treatment. Next, we review the evidence of the differences in treatment response by conducting a meta-analysis of the variation in antipsychotic treatment trials. Although this issue is brought up by statisticians regularly,2 it deserves more attention from a general psychiatric audience.

Where does the assumption of individual differences in treatment response come from? In general, antipsychotic drugs are assessed in randomized clinical trials (RCTs), the criterion standard for identifying the efficacy of a treatment. In RCTs, patients are assessed at baseline (eg, with the Positive and Negative Syndrome Scale [PANSS]) and randomized to either a treatment or a control group. What RCTs can ultimately provide is an answer to whether a treatment works in general. This average treatment effect is derived from the direct comparison of the response between the treatment and the control groups, which is imperative in an RCT.3 Understandably, this answer may leave clinicians unsatisfied; after all, they are treating individual, and not typical, patients. From a clinical perspective, patients vary considerably in their response to antipsychotic drugs, and the general response may seem almost like an uninformed guess for the individual patient. Furthermore, clinicians seem to prefer categories such as normal or abnormal and responders or nonresponders to inform diagnostic and therapeutic decisions. A consequence is that many investigators now try to personalize medicine by aiming to tailor treatments to individual patients. They agree that response to treatment varies from patient to patient.

However, estimating individual response to treatment, known as the treatment-by-patient interaction, is more complex than often appreciated and depends on laborious study designs, such as repeated crossover trials.2 However, as we illustrate with simulated data, such study designs are needed to distinguish individual response to treatment from other components of variation that are unrelated to permanent differences in treatment response.4,5,6,7

By design, RCTs cannot estimate the treatment-by-patient interaction, the index of individual response. Although RCTs do not tell anything about individual response, they might indicate something about the presence of individual response. As recognized early by Fisher,8 an increase in variance in the treatment group compared with the control group could indicate the presence of variation in response to treatment.2 The strength of this increase would then quantify the size of the personal element of response and provide evidence for the presence of a treatment-by-patient interaction.9 A method has been developed to compare variances between groups across studies10 and has been adopted by a meta-analysis package.11 In psychiatry, this method has been applied to compare variances in brain structure12 and inflammatory parameters in psychosis.13 This method compares the variance of treatment and control by computing their ratio: a ratio of 1 means equal variances, a ratio greater than 1 means more variability in the treatment group, and a ratio smaller than 1 means less variability in the treatment group compared with the control group.10,12,13

This study is organized in 2 parts. The first part illustrates the different components of variation in RCTs with simulated data, showing the importance of recognizing the treatment-by-patient interaction (which reflects individual treatment response) as the component of interest. The second part shows the results of a meta-analysis, which tested for the presence of treatment-by-patient interaction in empirical data from antipsychotic drug RCTs. We compared the overall variability in the treatment group with the overall variability in the control group, using data from a recently published meta-analysis,14 summarizing 24 years of placebo-controlled, antipsychotic RCTs in schizophrenia. We hypothesized that compared with control, the often-highlighted heterogeneity in patients with schizophrenia would be reflected by a clinically relevant increase in overall variance of treatment, outcome, which is compatible with a personal element of response that deviates from the estimated average treatment effects.

Methods

Trial Simulation

To illustrate the different components of variation in RCTs, we simulated data from patients with schizophrenia who were randomized to either the antipsychotic treatment or control group and assessed with the PANSS and a positive effect of treatment (Cohen d = 1.32; t27.5 = 3.46; P = .002). First, we added a single crossover condition with either a constant or a varying treatment effect, and then we added a double crossover to this simulated trial. With these additions, we show how the variability between and within patients has to be distinguished from the treatment-by-patient interaction, the component reflecting the individual differences in treatment response.

Meta-analysis

To ensure data quality and validity, this meta-analysis was conducted in accordance with the PRISMA guidelines. We searched Cochrane Schizophrenia, MEDLINE/PubMed, Embase, PsycINFO, Cochrane CENTRAL, BIOSIS Previews, ClinicalTrials.gov, and World Health Organization International Clinical Trials Registry Platform from January 1, 1955, to December 31, 2016.

Using the meta-analysis of Leucht et al15 as a basis, we included published and unpublished double-blind, placebo-controlled RCTs of at least 3 weeks’ duration. These studies investigated adults with a diagnosis of schizophrenia spectrum disorders and prescription for licensed antipsychotic medications, except clozapine. Studies were excluded if they investigated relapse prevention, patients with predominant negative symptoms, patients with major concomitant somatic or psychiatric illness, or intramuscular formulations of antipsychotic treatment, or if they were Chinese research. We included only studies that reported the necessary information (mean, SD, and sample size) of the PANSS pretreatment and posttreatment outcome difference scores.

In studies that combined comparisons of multiple antipsychotic drugs with placebo, we calculated an aggregated SD across all comparisons, leaving only 1 SD per study. We extracted the PANSS means and SDs of the pretreatment and posttreatment outcome difference scores as well as the sample sizes for the treatment and the control groups. Further information on the search strategy is published elsewhere.15

Statistical Analysis

The SDs of the pretreatment and posttreatment outcome difference scores in the treatment and control groups consist of the same variance components, including the within-patient variation. The treatment group, however, may also include the additional treatment-by-patient interaction, which could indicate the presence of individual response differences. Thus, in the case of a variable treatment effect, an increase of the variance in the treatment group, compared with the control group, should be observable. To assess this variation, we calculated for each comparison between antipsychotic and placebo drugs the relative variability of treatment and control as the log variability ratio (log VR)16 with

graphic file with name jamapsychiatry-76-1063-e001.jpg,

in which SDTx was the reported sample SD for treatment, SDCt was the reported sample SD for control, nTx was the treatment sample size, and nCt the control sample size.10 The corresponding sampling variance (SD2logVR)for each comparison between antipsychotic and placebo drugs can be expressed as follows:

graphic file with name jamapsychiatry-76-1063-e002.jpg.

We did not find an association between the pretreatment and posttreatment outcome difference scores and their respective SDs in the data for the control group (β = 0.16; P = .15; eFigure 1A in the Supplement) or the treatment group (β = –0.05; P = .63; eFigure 1B in the Supplement). For this reason, we did not consider the log coefficient of variation ratio (log CVR) as an additional index for comparing variabilities.10

We weighted each log VR with the inverse of this sampling variance11 and entered it into a random-effects model. This approach allows for the quantification of the true individual response, after adjusting for within-patient variability and regression to the mean.5,9 Results were back-transformed from the log scale for better interpretability, with a variability ratio higher than 1, indicating greater variability under treatment compared with control, and a ratio lower than 1, indicating less variability under treatment compared with control.

The analysis was performed from October 31, 2018, to March 29, 2019, with the R package metafor, version 2.0.0,11 and the manuscript was produced with the R package knitr, version 1.20, in RStudio (R Foundation for Statistical Computing). All the data and code we used are freely available online to ensure reproducibility (https://doi.org/10.17605/OSF.IO/QARVS).

Results

Simulation

We simulated an RCT of 30 patients with schizophrenia randomized to either the treatment or the control group. The individual pretreatment and posttreatment outcome differences (Figure 1A) might tempt us to infer that some patients in the treatment group responded better than others. We might then rank these patients according to their outcome and classify them as either responders or nonresponders. However, such ranking and classification can be misleading.

Figure 1. Observed Response Suggests Heterogeneity in Treatment Response.

Figure 1.

Open in a new tab

A, Thirty patients with schizophrenia were randomized to either the antipsychotic treatment or the control group. Despite the trial being simulated, with a main effect of treatment (Cohen d = 1.32; t27.5 = 3.46; P = .002), it may be tempting to infer from the Positive and Negative Syndrome Scale (PANSS) pretreatment and posttreatment outcome difference scores that some patients in the treatment group responded better than others. This observed pretreatment and posttreatment outcome difference can be misleading given that individual differences might be merely some unexplained components of variance. B, The treatment group patients are ranked according to the observed pretreatment and posttreatment outcome differences and classified as responders or nonresponders based on an arbitrary threshold (dashed light blue line). Although the ranking is not necessary for the classification, it increases the perception of individual differences in response to treatment.

Although seemingly different (Figure 2A), adding a simulated crossover condition to the initial parallel trial may reveal that the apparent differences in improvement among patients in the treatment group vanish (Figure 2B) and the treatment effect may actually be constant across patients (Figure 2C). Such a scenario cannot be ruled out from the results of a parallel group trial. In addition, the same ranking (Figure 2A) may reflect yet another scenario, in which differences in improvement as calculated from a crossover condition may reverse the ranking (Figure 2D), such that patients who appeared to have improved the most had actually the smallest net improvement (Figure 2E). Apparent outcome differences among patients in an RCT may still be compatible with a constant treatment effect.

Figure 2. Consequences of Between-Patient Variation.

Figure 2.

Open in a new tab

Two simulated scenarios are shown: one with a constant treatment effect across patients (A-C), and one with a reversed ranking after a control condition is taken into account (D, E). A, Using the same ranking from the simulated parallel trial, we show that ranking is a flawed approach to quantifying symptom improvement. B, When adding a control condition to the initial parallel trial, the seeming differences in improvement from patients in the treatment group vanish. C, The crossover trial simulation eliminates spurious differences in the outcome and reveals no between-patient differences in response. Although seemingly unlikely, such a scenario cannot be ruled out from the results of a parallel group trial. In another scenario, a variable treatment effect is added to a parallel trial (A). D, Differences in improvement may reverse the ranking if patients varied in their response to the treatment compared with controls. E, The patient who appeared to have improved the most in D had actually the smallest net improvement. PANSS indicates Positive and Negative Syndrome Scale.

Next, outcome differences may also be found within patients. Assessing patients repeatedly over time might reveal that symptoms fluctuate randomly around the same mean score (Figure 3). This fluctuation shows that within-patient variability alone may suggest differences in treatment response that are a mere reflection of random fluctuation.

Figure 3. Random Within-Patient Fluctuations.

Figure 3.

Open in a new tab

Four patients from the simulated trial were measured repeatedly over time using the Positive and Negative Syndrome Scale (PANSS). A, The differences in a patient’s symptom severity scores from time point to time point are independent of any intervention. B, Measuring repeatedly and calculating the means over all time points account for random within-patient fluctuations and reveal that patients had the same mean PANSS score. What differed was the amount of random fluctuation, which might be a highly unlikely scenario but, until tested, cannot be assumed is not true.

Again, we can add a simple crossover condition to the simulated parallel group trial (Figure 4A), in which each patient received both the treatment (antipsychotic drug) and control (placebo). Only by running the crossover trial once again (Figure 4B) can we determine whether the differences observed in the first crossover trial are indeed stable features of the patients. The net improvement from crossover trial 1 may not replicate in crossover trial 2, which indicates that the response differences are still not stable features of the patients (Figure 4C). For that stability to be the case, we would have to see a similar outcome in crossover trial 1 (Figure 4A) as in crossover trial 2 (Figure 4D), in which case we have identified a substantial treatment-by-patient interaction (Figure 4E).

Figure 4. Estimating Treatment-by-Patient Interaction With Repeated Crossover Trials.

Figure 4.

Open in a new tab

The same simulation is shown of 30 patients with schizophrenia in a parallel trial assessed with the Positive and Negative Syndrome Scale (PANSS). A, Patients received both the antipsychotic and placebo drugs in a crossover trial. B, Only by running a crossover trial more than once can we identify whether individual response is a trait or a permanent feature of the patient. C, Associating the net improvement from crossover trials 1 and 2 (solid dots) shows that advantages from the first trial do not replicate in the second (r = 0.02; 95% CI). In this scenario, response to treatment is not a permanent feature of the patient. D, Another scenario using the same crossover trial 1 (A) and repeating it, we might observe that the response was almost identical to the response in crossover trial 1. E, Patients responded similarly in both trials (r = 0.77; 95% CI). In this scenario, we can consider response to treatment as a trait in these patients. Repeatability of the association from one crossover trial to the next in the same sample is the only way to separate treatment-by-patient interaction from random within-patient variation and to determine whether response is a trait or a state.

A careful distinction of the sources of variation in a simulated RCT has shown that it is not trivial to distinguish the source of primary interest (treatment-by-patient interaction) from components that tell nothing about individual response. In the meta-analysis, we assessed whether evidence exists for such treatment-by-patient interaction across antipsychotic drug trials.

Meta-analysis

We investigated 75 comparisons of antipsychotic drug with placebo in 52 RCTs.17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68 None of these studies used a design such as repeated crossovers that would have allowed for a direct estimate of individual responses. Overall, a total of 15 360 patients with a schizophrenia or schizoaffective diagnosis were included, of whom 8550 (55.7%) had been randomized to the treatment group and 6810 (44.3%) to the control group (more details can be found in the eResults in the Supplement).

We found an overall lower variability in treatment compared with control (variability ratio = 0.97; 95% CI, 0.95-0.99; P = .01; Figure 517,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68). This finding indicates that the overall variability across treatment groups was 3% lower compared with that in the control groups. Furthermore, we compared the variances in individual antipsychotic drug outcome and found the same pattern, with lower variability across treatment compared with control (variability ratio = 0.97; 95% CI, 0.95-1.00; P = .02; eFigure 2 in the Supplement).

Figure 5. Lower Variability in the Treatment Group Compared With the Control Group Across Antipsychotic Drug Parallel Trials.

Figure 5.

Open in a new tab

The forest plot shows the VR together with its 95% CI for treatment vs control across 52 studies. Each study is listed with its respective citation number. The overall VR was lower for treatment compared with control group.

No evidence was found that antipsychotic treatment increased the outcome variance compared with the control. Instead, the outcome variance was slightly lower in the treatment than in the control group.

Discussion

A widespread belief among clinicians and researchers is that patients differ substantially in their antipsychotic treatment response, but finding evidence for this assumption is complex. A likely explanation, supported by the simulations conducted for this study, is that taking an observed treatment response as the true treatment response is tempting, compelling us to ignore the components of variation most likely encountered: random variation within patients and differences between patients. The existing empirical evidence for such individual differences is weaker than expected: No evidence was found that the antipsychotic drug increased the outcome variance compared with the placebo. Instead, the outcome variance was slightly lower in the treatment group. With this finding, we still cannot rule out that subsets of patients responded differently to treatment, but the overall small difference in variances suggests that the average treatment effect is a reasonable assumption for the individual patient. By assuming heterogeneity in treatment outcomes, we might ultimately introduce noise into clinical practice by refusing to go with the best available evidence, the average treatment effect derived from RCTs.2

Although RCTs are questioned regularly, sometimes using questionable arguments,69 they remain the criterion standard in clinical research. They provide unbiased estimates of the relative efficacy of an intervention, which even the largest observational studies cannot provide.70 In addition, appreciating the role of randomization in RCTs is important. Randomization is not compatible with the notion that specific features, such as placebo response, increase in one but not the other group in an RCT. If evidence existed of an enhanced placebo response over time, as has been suggested repeatedly in the past years,14 this response would have been apparent in both the control and the treatment groups because of randomization and thus would have canceled out. Furthermore, the concept of placebo response, although regularly investigated,71 cannot be studied by looking at the observed responses in control groups,72 for the same reasons that this approach does not work for the treatment groups, as this study has shown.

Comparing the variabilities between treatment and control groups may provide valuable insight into the presence of individual response and the scope of personalized medicine. Recently, other groups have taken a similar approach to assess the presence of individual differences in brain structure12 and immunological parameters in psychosis.13 We assumed that, in the presence of a personal element of response to treatment, the variance in the treatment group should be higher compared with the control group, which in turn would require further investigation (eg, with n-of-1 trials).73,74,75 However, our results indicate that overall variability in the treatment groups was slightly lower, if only by a modest amount (1% to 5%). One explanation might be that the treatment had a stabilizing quality9 that reduced the variability in the treatment group. An example for such variance stabilization might be the floor effect, in which the assessment instrument is too coarse to capture patient improvements over a certain level.

Nevertheless, given the slightly lower variability under treatment compared with control found in this study, we cannot rule out that individual differences in response to antipsychotic drugs might still exist. A subset of ill patients may have responded well to treatment, whereas less affected patients may not have improved, resulting in an overall decreased variance under treatment than under control.9 Yet, the finding of a narrow CI around an overall only slightly lower variability suggests that substantial differences in drug response are rather unlikely. Thus, analyses aimed at estimating individual response might be premature until these differences have been shown to exist and to be clinically relevant.

As the simulations have shown, labeling patients as responders might be misleading. The label suggests that true response has been established as a permanent feature of the patient, even though the label is a mere reflection of the observed response, which includes true response plus regression to the mean, some placebo effect, and random terms such as measurement error. Thus, response rates that are calculated in RCTs reflect observed but not true response. We suggest that biomarker research aimed at identifying response to treatment of individuals or subgroups should consider the possibility that treatment outcome is less heterogeneous than anticipated and might even be close to constant across individuals.

Limitations

This meta-analysis has some limitations. First, the calculation of the pretreatment and posttreatment outcome difference scores varied between studies. Although some RCTs calculated the differences between outcome and baseline PANSS scores, others used analysis of covariances with the baseline PANSS scores and additional variables as covariates. Thus, some of the included SDs of change were adjusted for covariates but others were not. Second, the use of pretreatment and posttreatment outcome difference scores might lead to a loss of information and might not be sensitive enough to capture differences in response to treatment.76 Third, we assumed that individual responses to treatment were reflected by increased variance in the treatment group. Yet, this increased variance could have also indicated the presence of subgroups who responded differently to the treatment.9 Such a case would argue for stratified medicine rather than personalized medicine, in which subgroups of patients receive varying treatments. As any interaction, a treatment-by-patient interaction is ultimately scale dependent, which means it can be removed by transformation of the scale.77

Conclusions

Until the differences in individual response to treatment have been demonstrated with careful designs, the overall small differences in outcome variance suggest that the average treatment effect is a reasonable assumption for the individual patient.

Supplement.

eResults

eFigure 1. No Associations Between Mean and SDs

eFigure 2. Variability for Treatment vs Control Across All Investigated Antipsychotic Drugs

Click here for additional data file. (616.7KB, pdf)

References

  • 1.Garver DL, Holcomb JA, Christensen JD. Heterogeneity of response to antipsychotics from multiple disorders in the schizophrenia spectrum. J Clin Psychiatry. 2000;61(12):964-972. doi: 10.4088/JCP.v61n1213 [DOI] [PubMed] [Google Scholar]
  • 2.Senn S. Mastering variation: variance components and personalised medicine. Stat Med. 2016;35(7):966-977. doi: 10.1002/sim.6739 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Senn S. Trying to be precise about vagueness. Stat Med. 2007;26(7):1417-1430. doi: 10.1002/sim.2639 [DOI] [PubMed] [Google Scholar]
  • 4.Homan P, Kane JM. Clozapine as an early-stage treatment. Acta Psychiatr Scand. 2018;138(4):279-280. doi: 10.1111/acps.12965 [DOI] [PubMed] [Google Scholar]
  • 5.Hecksteden A, Kraushaar J, Scharhag-Rosenberger F, Theisen D, Senn S, Meyer T. Individual response to exercise training - a statistical perspective. J Appl Physiol (1985). 2015;118(12):1450-1459. doi: 10.1152/japplphysiol.00714.2014 [DOI] [PubMed] [Google Scholar]
  • 6.Hecksteden A, Pitsch W, Rosenberger F, Meyer T. Repeated testing for the assessment of individual response to exercise training. J Appl Physiol (1985). 2018;124(6):1567-1579. doi: 10.1152/japplphysiol.00896.2017 [DOI] [PubMed] [Google Scholar]
  • 7.Dworkin RH, McDermott MP, Farrar JT, O’Connor AB, Senn S. Interpreting patient treatment response in analgesic clinical trials: implications for genotyping, phenotyping, and personalized pain treatment. Pain. 2014;155(3):457-460. doi: 10.1016/j.pain.2013.09.019 [DOI] [PubMed] [Google Scholar]
  • 8.Bennett JH, ed. Statistical Inference and Analysis: Selected Correspondence of RA Fisher. Oxford, UK: Clarendon Press; 1990. Cited by: Senn S. Seven myths of randomisation in clinical trials. Stat Med. 2013;32(9): 1439-1450. doi: 10.1002/sim.5713 [DOI] [PubMed] [Google Scholar]
  • 9.Cortés J, González JA, Medina MN, et al. . Does evidence support the high expectations placed in precision medicine? A bibliographic review. F1000 Res. 2019;7:30. doi: 10.12688/f1000research.13490.4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Nakagawa S, Poulin R, Mengersen K, et al. . Meta-analysis of variation: ecological and evolutionary applications and beyond. Methods Ecol Evol. 2015;6(2):143-152. doi: 10.1111/2041-210X.12309 [DOI] [Google Scholar]
  • 11.Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1-48. doi: 10.18637/jss.v036.i03 [DOI] [Google Scholar]
  • 12.Brugger SP, Howes OD. Heterogeneity and homogeneity of regional brain structure in schizophrenia: a meta-analysis. JAMA Psychiatry. 2017;74(11):1104-1111. doi: 10.1001/jamapsychiatry.2017.2663 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Pillinger T, Osimo EF, Brugger S, Mondelli V, McCutcheon RA, Howes OD. A meta-analysis of immune parameters, variability, and assessment of modal distribution in psychosis and test of the immune subgroup hypothesis[published November 8, 2018]. Schizophr Bull. 2018. doi: 10.1093/schbul/sby160 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 15.Leucht S, Davis JM. Do antipsychotic drugs lose their efficacy for relapse prevention over time? Br J Psychiatry. 2017;211(3):127-129. doi: 10.1192/bjp.bp.117.201103 [DOI] [PubMed] [Google Scholar]
  • 16.Hedges LV, Nowell A. Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science. 1995;269(5220):41-45. doi: 10.1126/science.7604277 [DOI] [PubMed] [Google Scholar]
  • 17.Litman RE, Smith MA, Doherty JJ, et al. . AZD8529, a positive allosteric modulator at the mGluR2 receptor, does not improve symptoms in schizophrenia: a proof of principle study. Schizophr Res. 2016;172(1-3):152-157. doi: 10.1016/j.schres.2016.02.001 [DOI] [PubMed] [Google Scholar]
  • 18.StudyRIS Office of Clinical Pharmacology and Biopharmacy Review. NDA number: 20272. Janssen-Cilag; data on file 1996. Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed]
  • 19.Daniel DG, Zimbroff DL, Potkin SG, Reeves KR, Harrigan EP, Lakshminarayanan M; Ziprasidone Study Group . Ziprasidone 80 mg/day and 160 mg/day in the acute exacerbation of schizophrenia and schizoaffective disorder: a 6-week placebo-controlled trial. Neuropsychopharmacology. 1999;20(5):491-505. doi: 10.1016/S0893-133X(98)00090-6 [DOI] [PubMed] [Google Scholar]
  • 20.Potkin SG, Litman RE, Torres R, Wolfgang CD. Efficacy of iloperidone in the treatment of schizophrenia: initial phase 3 studies. J Clin Psychopharmacol. 2008;28(2)(suppl 1):S4-S11. doi: 10.1097/JCP.0b013e3181692787 [DOI] [PubMed] [Google Scholar]
  • 21.Zborowski J, Schmitz P, Staser J, et al. . Efficacy and safety of sertindole in a trial of schizophrenic patients. Biol Psychiatry. 1995;9(37):661-662. doi: 10.1016/0006-3223(95)94656-H [DOI] [Google Scholar]
  • 22.Marder SR, Meibach RC. Risperidone in the treatment of schizophrenia. Am J Psychiatry. 1994;151(6):825-835. doi: 10.1176/ajp.151.6.825 [DOI] [PubMed] [Google Scholar]
  • 23.Kane J, Canas F, Kramer M, et al. . Treatment of schizophrenia with paliperidone extended-release tablets: a 6-week placebo-controlled trial. Schizophr Res. 2007;90(1-3):147-161. doi: 10.1016/j.schres.2006.09.012 [DOI] [PubMed] [Google Scholar]
  • 24.Marder SR, Kramer M, Ford L, et al. . Efficacy and safety of paliperidone extended-release tablets: results of a 6-week, randomized, placebo-controlled study. Biol Psychiatry. 2007;62(12):1363-1370. doi: 10.1016/j.biopsych.2007.01.017 [DOI] [PubMed] [Google Scholar]
  • 25.Durgam S, Starace A, Li D, et al. . An evaluation of the safety and efficacy of cariprazine in patients with acute exacerbation of schizophrenia: a phase II, randomized clinical trial. Schizophr Res. 2014;152(2-3):450-457. doi: 10.1016/j.schres.2013.11.041 [DOI] [PubMed] [Google Scholar]
  • 26.Casey DE, Sands EE, Heisterberg J, Yang H-M. Efficacy and safety of bifeprunox in patients with an acute exacerbation of schizophrenia: results from a randomized, double-blind, placebo-controlled, multicenter, dose-finding study. Psychopharmacology (Berl). 2008;200(3):317-331. doi: 10.1007/s00213-008-1207-7 [DOI] [PubMed] [Google Scholar]
  • 27.Potkin SG, Cohen M, Panagides J. Efficacy and tolerability of asenapine in acute schizophrenia: a placebo- and risperidone-controlled trial. J Clin Psychiatry. 2007;68(10):1492-1500. doi: 10.4088/JCP.v68n1004 [DOI] [PubMed] [Google Scholar]
  • 28.Durgam S, Cutler AJ, Lu K, et al. . Cariprazine in acute exacerbation of schizophrenia: a fixed-dose, phase 3, randomized, double-blind, placebo- and active-controlled trial. J Clin Psychiatry. 2015;76(12):e1574-e1582. doi: 10.4088/JCP.15m09997 [DOI] [PubMed] [Google Scholar]
  • 29.Hirayasu Y, Tomioka M, Iizumi M, Kikuchi H. A double-blind, placebo-controlled, comparative study of paliperidone extended release (ER) tablets in patients with schizophrenia. Jpn J Clin Psychopharmacol. 2010;13:2077-2103. [Google Scholar]
  • 30.Hera021, 041-021SH. A multicenter, randomized, double-blind, fixed-dose, 6-week trial of the efficacy and safety of asenapine compared with placebo using olanzapine positive control in subjects with an acute exacerbation of schizophrenia. Center for Drug Evaluation and Research. Medical review(s); 2009. Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 31.Ogasa M, Kimura T, Nakamura M, Guarino J. Lurasidone in the treatment of schizophrenia: a 6-week, placebo-controlled study. Psychopharmacology (Berl). 2013;225(3):519-530. doi: 10.1007/s00213-012-2838-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.McEvoy JP, Daniel DG, Carson WH Jr, McQuade RD, Marcus RN. A randomized, double-blind, placebo-controlled, study of the efficacy and safety of aripiprazole 10, 15 or 20 mg/day for the treatment of patients with acute exacerbations of schizophrenia. J Psychiatr Res. 2007;41(11):895-905. doi: 10.1016/j.jpsychires.2007.05.002 [DOI] [PubMed] [Google Scholar]
  • 33.Tzimos A, Samokhvalov V, Kramer M, et al. . Safety and tolerability of oral paliperidone extended-release tablets in elderly patients with schizophrenia: a double-blind, placebo-controlled study with six-month open-label extension. Am J Geriatr Psychiatry. 2008;16(1):31-43. doi: 10.1097/JGP.0b013e31815a3e7a [DOI] [PubMed] [Google Scholar]
  • 34.ClinicalTrials.gov A study of the efficacy and safety of asenapine in participants with an acute exacerbation of schizophrenia. Identifier: NCT01617187. https://clinicaltrials.gov/ct2/show/NCT01617187?cond=A+study+of+the+efficacy+and+safety+of+asenapine+in+participants+with+an+acute+exacerbation+of+schizophrenia&rank=1. Accessed October 31, 2016.
  • 35.Nasrallah HA, Silva R, Phillips D, et al. . Lurasidone for the treatment of acutely psychotic patients with schizophrenia: a 6-week, randomized, placebo-controlled study. J Psychiatr Res. 2013;47(5):670-677. doi: 10.1016/j.jpsychires.2013.01.020 [DOI] [PubMed] [Google Scholar]
  • 36.Correll CU, Skuban A, Hobart M, et al. . Efficacy of brexpiprazole in patients with acute schizophrenia: review of three randomized, double-blind, placebo-controlled studies. Schizophr Res. 2016;174(1-3):82-92. doi: 10.1016/j.schres.2016.04.012 [DOI] [PubMed] [Google Scholar]
  • 37.Canuso CM, Schooler N, Carothers J, et al. . Paliperidone extended-release in schizoaffective disorder: a randomized, controlled study comparing a flexible dose with placebo in patients treated with and without antidepressants and/or mood stabilizers. J Clin Psychopharmacol. 2010;30(5):487-495. doi: 10.1097/JCP.0b013e3181eeb600 [DOI] [PubMed] [Google Scholar]
  • 38.Shen JH, Zhao Y, Rosenzweig-Lipson S, et al. . A 6-week randomized, double-blind, placebo-controlled, comparator referenced trial of vabicaserin in acute schizophrenia. J Psychiatr Res. 2014;53:14-22. doi: 10.1016/j.jpsychires.2014.02.012 [DOI] [PubMed] [Google Scholar]
  • 39.Kane JM, Cohen M, Zhao J, Alphs L, Panagides J. Efficacy and safety of asenapine in a placebo- and haloperidol-controlled trial in patients with acute exacerbation of schizophrenia. J Clin Psychopharmacol. 2010;30(2):106-115. doi: 10.1097/JCP.0b013e3181d35d6b [DOI] [PubMed] [Google Scholar]
  • 40.Study93202. Center for Drug Evaluation and Research. Application number 21-436. Medical review(s); 2002. Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 41.Lieberman JA, Davis RE, Correll CU, et al. . ITI-007 for the treatment of schizophrenia: A 4-week randomized, double-blind, controlled trial. Biol Psychiatry. 2016;79(12):952-961. doi: 10.1016/j.biopsych.2015.08.026 [DOI] [PubMed] [Google Scholar]
  • 42.Bugarski-Kirola D, Wang A, Abi-Saab D, Blättler T. A phase II/III trial of bitopertin monotherapy compared with placebo in patients with an acute exacerbation of schizophrenia: results from the CandleLyte study. Eur Neuropsychopharmacol. 2014;24(7):1024-1036. doi: 10.1016/j.euroneuro.2014.03.007 [DOI] [PubMed] [Google Scholar]
  • 43.van Kammen DP, McEvoy JP, Targum SD, Kardatzke D, Sebree TB. A randomized, controlled, dose-ranging trial of sertindole in patients with schizophrenia. Psychopharmacology (Berl). 1996;124(1-2):168-175. doi: 10.1007/BF02245618 [DOI] [PubMed] [Google Scholar]
  • 44.Study94202. Center for Drug Evaluation and Research. Application number 21-436. Medical review(s); 2002. Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 45.Potkin SG, Saha AR, Kujawa MJ, et al. . Aripiprazole, an antipsychotic with a novel mechanism of action, and risperidone vs placebo in patients with schizophrenia and schizoaffective disorder. Arch Gen Psychiatry. 2003;60(7):681-690. doi: 10.1001/archpsyc.60.7.681 [DOI] [PubMed] [Google Scholar]
  • 46.Litman RE, Smith MA, Desai DG, Simpson T, Sweitzer D, Kanes SJ. The selective neurokinin 3 antagonist AZD2624 does not improve symptoms or cognition in schizophrenia: a proof-of-principle study. J Clin Psychopharmacol. 2014;34(2):199-204. doi: 10.1097/JCP.0000000000000071 [DOI] [PubMed] [Google Scholar]
  • 47.Kinon BJ, Zhang L, Millen BA, et al. ; HBBI Study Group . A multicenter, inpatient, phase 2, double-blind, placebo-controlled dose-ranging study of LY2140023 monohydrate in patients with DSM-IV schizophrenia. J Clin Psychopharmacol. 2011;31(3):349-355. doi: 10.1097/JCP.0b013e318218dcd5 [DOI] [PubMed] [Google Scholar]
  • 48.Kane JM, Carson WH, Saha AR, et al. . Efficacy and safety of aripiprazole and haloperidol versus placebo in patients with schizophrenia and schizoaffective disorder. J Clin Psychiatry. 2002;63(9):763-771. doi: 10.4088/JCP.v63n0903 [DOI] [PubMed] [Google Scholar]
  • 49.Egan MF, Zhao X, Smith A, et al. . Randomized controlled study of the T-type calcium channel antagonist MK-8998 for the treatment of acute psychosis in patients with schizophrenia. Hum Psychopharmacol. 2013;28(2):124-133. doi: 10.1002/hup.2289 [DOI] [PubMed] [Google Scholar]
  • 50.Durgam S, Litman RE, Papadakis K, Li D, Németh G, Laszlovszky I. Cariprazine in the treatment of schizophrenia: a proof-of-concept trial. Int Clin Psychopharmacol. 2016;31(2):61-68. doi: 10.1097/YIC.0000000000000110 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Barbato L, Newcomer J, Heisterberg J, Yeung P, Shapira N. Efficacy and metabolic profile of bifeprunox in patients with schizophrenia. Paper presented at: 11th International Congress on Schizophrenia Research; March 28 to April 1, 2007; Colorado Springs, CO. [Google Scholar]
  • 52.Study049. A 6-week, double-blind, randomized, fixed dose, parallel-group study of the efficacy and safety of three dose levels of SM-13496 (lurasidone) compared to placebo and haloperidol in patients with schizophrenia who are experiencing an acute exacerbation of symptoms. Center for Drug Evaluation and Research; Medical review(s). 2010. Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 53.Meltzer HY, Cucchiaro J, Silva R, et al. . Lurasidone in the treatment of schizophrenia: a randomized, double-blind, placebo-and olanzapine-controlled study. Am J Psychiatry. 2011;168(9):957-967. doi: 10.1176/appi.ajp.2011.10060907 [DOI] [PubMed] [Google Scholar]
  • 54.Meltzer HY, Barbato L, Heisterberg J, Yeung P, Shapira N. A randomized, doubleblind, placebo-controlled efficacy and safety study of bifeprunox as treatment for patients with acutely exacerbated schizophrenia. Schizophr Bull. 2007;33(2):446. [Google Scholar]
  • 55.Loebel A, Cucchiaro J, Sarma K, et al. . Efficacy and safety of lurasidone 80 mg/day and 160 mg/day in the treatment of schizophrenia: a randomized, double-blind, placebo- and active-controlled trial. Schizophr Res. 2013;145(1-3):101-109. doi: 10.1016/j.schres.2013.01.009 [DOI] [PubMed] [Google Scholar]
  • 56.Kane JM, Zukin S, Wang Y, et al. . Efficacy and safety of cariprazine in acute exacerbation of schizophrenia: results from an international, phase III clinical trial. J Clin Psychopharmacol. 2015;35(4):367-373. [DOI] [PubMed] [Google Scholar]
  • 57.Garcia E, Robert M, Peris F, Nakamura H, Sato N, Terazawa Y. The efficacy and safety of blonanserin compared with haloperidol in acute-phase schizophrenia: a randomized, double-blind, placebo-controlled, multicentre study. CNS Drugs. 2009;23(7):615-625. doi: 10.2165/00023210-200923070-00006 [DOI] [PubMed] [Google Scholar]
  • 58.Davidson M, Emsley R, Kramer M, et al. . Efficacy, safety and early response of paliperidone extended-release tablets (paliperidone ER): results of a 6-week, randomized, placebo-controlled study. Schizophr Res. 2007;93(1-3):117-130. doi: 10.1016/j.schres.2007.03.003 [DOI] [PubMed] [Google Scholar]
  • 59.Canuso CM, Lindenmayer J-P, Kosik-Gonzalez C, et al. . A randomized, double-blind, placebo-controlled study of 2 dose ranges of paliperidone extended-release in the treatment of subjects with schizoaffective disorder. J Clin Psychiatry. 2010;71(5):587-598. doi: 10.4088/JCP.09m05564yel [DOI] [PubMed] [Google Scholar]
  • 60.Hera022, SH H 041-022. A multicenter, randomized, double-blind, fixed-dose, 6-week trial of the efficacy and safety of asenapine compared with placebo using olanzapine positive control in subjects with an acute exacerbation of schizophrenia. Center for Drug Evaluation and Research; Medical review(s); 2009.Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 61.Nakamura M, Ogasa M, Guarino J, et al. . Lurasidone in the treatment of acute schizophrenia: a double-blind, placebo-controlled trial. J Clin Psychiatry. 2009;70(6):829-836. doi: 10.4088/JCP.08m04905 [DOI] [PubMed] [Google Scholar]
  • 62.Study115. Center for Drug Evaluation and Research. Approval package for application number 20-825. Medical review(s); 2000. Cited by: Leucht S, Leucht C, Huhn M, et al. . Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. doi: 10.1176/appi.ajp.2017.16121358 [DOI] [PubMed] [Google Scholar]
  • 63.Beasley CM Jr, Sanger T, Satterlee W, Tollefson G, Tran P, Hamilton S. Olanzapine versus placebo: results of a double-blind, fixed-dose olanzapine trial. Psychopharmacology (Berl). 1996;124(1-2):159-167. doi: 10.1007/BF02245617 [DOI] [PubMed] [Google Scholar]
  • 64.Lindenmayer J-P, Brown D, Liu S, Brecher M, Meulien D. The efficacy and tolerability of once-daily extended release quetiapine fumarate in hospitalized patients with acute schizophrenia: a 6-week randomized, double-blind, placebo-controlled study. Psychopharmacol Bull. 2008;41(3):11-35. [PubMed] [Google Scholar]
  • 65.Schmidt ME, Kent JM, Daly E, et al. . A double-blind, randomized, placebo-controlled study with JNJ-37822681, a novel, highly selective, fast dissociating D2 receptor antagonist in the treatment of acute exacerbation of schizophrenia. Eur Neuropsychopharmacol. 2012;22(10):721-733. doi: 10.1016/j.euroneuro.2012.02.007 [DOI] [PubMed] [Google Scholar]
  • 66.Meltzer HY, Arvanitis L, Bauer D, Rein W; Meta-Trial Study Group . Placebo-controlled evaluation of four novel compounds for the treatment of schizophrenia and schizoaffective disorder. Am J Psychiatry. 2004;161(6):975-984. doi: 10.1176/appi.ajp.161.6.975 [DOI] [PubMed] [Google Scholar]
  • 67.Coppola D, Melkote R, Lannie C, et al. . Efficacy and safety of paliperidone extended release 1.5 mg/day-a double-blind, placebo- and active-controlled, study in the treatment of patients with schizophrenia. Psychopharmacol Bull. 2011;44(2):54-72. [PMC free article] [PubMed] [Google Scholar]
  • 68.Chouinard G, Jones B, Remington G, et al. . A Canadian multicenter placebo-controlled study of fixed doses of risperidone and haloperidol in the treatment of chronic schizophrenic patients. J Clin Psychopharmacol. 1993;13(1):25-40. doi: 10.1097/00004714-199302000-00004 [DOI] [PubMed] [Google Scholar]
  • 69.Senn S. Seven myths of randomisation in clinical trials. Stat Med. 2013;32(9):1439-1450. doi: 10.1002/sim.5713 [DOI] [PubMed] [Google Scholar]
  • 70.Leucht S, Davis JM. Enthusiasm and skepticism about using national registers to analyze psychotropic drug outcomes. JAMA Psychiatry. 2018;75(4):314-315. doi: 10.1001/jamapsychiatry.2017.4704 [DOI] [PubMed] [Google Scholar]
  • 71.Kubo K, Fleischhacker WW, Suzuki T, Yasui-Furukori N, Mimura M, Uchida H. Placebo effects in adult and adolescent patients with schizophrenia: combined analysis of nine RCTs. Acta Psychiatr Scand. 2019;139(2):108-116. doi: 10.1111/acps.12960 [DOI] [PubMed] [Google Scholar]
  • 72.Hróbjartsson A, Gøtzsche PC. Is the placebo powerless? an analysis of clinical trials comparing placebo with no treatment. N Engl J Med. 2001;344(21):1594-1602. doi: 10.1056/NEJM200105243442106 [DOI] [PubMed] [Google Scholar]
  • 73.Senn S. Applying results of randomised trials to patients. N of 1 trials are needed. BMJ. 1998;317(7157):537-538. doi: 10.1136/bmj.317.7157.537 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine–reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357(21):2189-2194. doi: 10.1056/NEJMsr077003 [DOI] [PubMed] [Google Scholar]
  • 75.Araujo A, Julious S, Senn S. Understanding variation in sets of N-of-1 trials. PLoS One. 2016;11(12):e0167167. doi: 10.1371/journal.pone.0167167 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Joyce DW, Tracy DK, Shergill SS. Are we failing clinical trials? a case for strong aggregate outcomes. Psychol Med. 2018;48(2):177-186. doi: 10.1017/S0033291717001726 [DOI] [PubMed] [Google Scholar]
  • 77.González AB, Cox DR. Interpretation of interaction: a review. Ann Appl Stat. 2007;1(2):371-385. doi: 10.1214/07-AOAS124 [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement.

eResults

eFigure 1. No Associations Between Mean and SDs

eFigure 2. Variability for Treatment vs Control Across All Investigated Antipsychotic Drugs

Click here for additional data file. (616.7KB, pdf)

Articles from JAMA Psychiatry are provided here courtesy of American Medical Association

RESOURCES