Randomized controlled trials in schizophrenia: opportunities, limitations, and trial design alternatives

Abstract

State-of-the art clinical trial design and methodology are enormously important for the advancement of the field. In contrast, the critical relevance of trial conduct and implementation have only more recently been the focus of discussion and research. Although randomized controlled trials are generally considered the gold standard for the assessment of pharmacologic and nonpharmacologic interventions in medicine, trials are vulnerable to complications and influences that can seriously compromise their success, Like interventions, trial design and conduct are also contextual. They need to be individualized and adapted to a number of relevant variables, such as setting, population, illness phase, interventions, patient and rater expectations and biases, and the overall aims of the investigation. While this means that there is no unified approach possible, certain general principles and guidelines require careful consideration. Knowledge of basic solutions and alternatives, and the recognition of the complex challenges that need to be addressed proactively can help to minimize unwanted outcomes, including trial failure and uninformative or falsely negative outcomes. Moreover, novel design alternatives need to be explored that target sample enrichment according to the study question and enhancement of precision in the measurement of relevant outcomes. We propose two novel design strategies that take advantage of the recently validated early antipsychotic response paradigm (that has also been observed with antidepressants and mood stabilizers). In the “early responder randomized discontinuation design” all patients are assigned to the active drug, and only those who had at least a minimal response at 2 weeks are enrolled in a double-blind, placebo-controlled discontinuation trial, enriching the placebo controlled trial portion with true drug responders. In the mirror image “early nonresponder randomized dose increase or augmentation design,” early nonresponders at 2 weeks are assigned to staying on the medication or going either to a higher dose or an augmentation agent. It is hoped that through increased attention to the issues raised in this article and further refinement of trial methodology and conduct, the field will make much needed additional progress in the prevention and treatment of schizophrenia.

Randomized controlled trials (RCTs) have become a cornerstone of evidence-based medicine, and therefore have an important impact on clinical decision-making and clinical practice. The clinical trial can be a much more precise and accurate assessment of therapeutic potential than the anecdotal report or uncontrolled case series. However, clinical trials have important limitations in terms of feasibility and generalizability and can also fail or prove to be erroneous in their conclusions. The process of patient selection in clinical trials further highlights the strengths and weaknesses of the current nosology, and the prevalence of comorbid conditions and other factors can also influence treatment response. Moreover, the clinical trial serves to highlight the ethical and scientific tension between striving for the common good and the treatment of the individual person. When and to what extent the use of placebos is appropriate when proven effective treatments are available is an important and complex issue about which reasonable people may disagree. In order for RCTs to serve the common good in an optimal fashion, clinicians, health care policy makers and other individuals with a stake in influencing and evaluating clinical care must be informed consumers of clinical trial data. Similarly, for clinical trials to be informative, those involved must carefully consider the opportunities and challenges of trial design, methodology, conduct, implementation, and interpretation.

In designing and conducting clinical trials, there is a constant tension between the “perfect” and the “feasible,” the desirable and doable, and between striving for scientific excellence and clinical impact. This dichotomy needs to be resolved proactively, consciously realizing and weighing the cost and benefit of each decision aimed at resolving this unavoidable conflict. It is in that spirit that we provide the following overview of controlled clinical trials in schizophrenia. We will first discuss the changing clinical and scientific context in which RCTs are taking place, followed by a discussion of specific trial components and their importance.

Historical developments

The somewhat serendipitous observation that chlorpromazine had a pronounced “calming” activity that extended to benefits for psychotic signs and symptoms was one of the great advances in 20th-century medicine. This effect was observed without the benefit of an RCT. Chlorpromazine was subsequently approved by the Food and Drug Administration in 1954, and by 1964, approximately 50 million people around the world had been treated with this medication.

In 1949, the World Health Organization published the sixth revision of the International Statistical Classification of Diseases (ICD), which for the first time included a section on mental disorders.¹ The first official Diagnostic and Statistical Manual of Mental Disorders (DSM) was published in 1952 by the American Psychiatric Association.² Diagnostic criteria were not really specified for discrete disorders until the third edition of DSM (III),³ which attempted to improve the validity and reliability of psychiatric diagnosis. This, in turn, had enormous implications for clinical practice, clinical research and drug development.

In 1969, Klein and Davis published a seminal work entitled Diagnosis and Drug Treatment of Psychiatric Disorders. ⁴ In the introduction, they wrote,

“We may be fortunate to be entering a period in which rational comparative study will become standard for therapeutic decision. Although clinical hunches and results of clinical experience are important factors in the determination of proper treatment, the findings of research studies, particularly those which are done with controlled double-blind technique, provide the behavioral scientific data for informed decision.”

By 1969, Klein and Davis identified 126 controlled studies comparing antipsychotic drugs and placebo in which the medications were found to be more effective and 26 comparisons in which they were not.“ They also examined the role of dose adequacy and found that most of those studies that found chlorpromazine to be ineffective used very small doses, and all 23 studies that employed doses over 500 mg/day were positive. Similarly, in all studies, which were judged to be methodologically rigorous, the phenothiazine derivatives (and reserpine) were shown to be more effective than the control conditions. These data led to an enormous shift in clinical practice, with antipsychotic drugs becoming the critical component in the treatment of schizophrenia.

Treatment-refractory illness and clozapine

With the development and initial evaluation of clozapine in Europe, early observations suggested a novel compound had been developed with a qualitatively different clinical profile that included the relative absence of drug-induced extrapyramidal effects and potentially superior efficacy in patients who had failed other antidopaminergic agents. However, a series of cases of agranulocytosis⁵ led to a delay in the further development of clozapine in the US. Based on a large RCT with prospective validation of treatment refractoriness demonstrating clozapine's superiority over chlorpromazine in refractory schizophrenia,⁶ the FDA approved clozapine with the narrow indication for treatment resistant patients in 1990. Since then, clozapine's singular role in treatment-refractory patients with schizophrenia has been confirmed⁷ and its role in the management of suicidality has also been established.⁸ Nevertheless, recent meta-analyses“ did not uniformly confirm clozapine's superiority over other antipsychotics in schizophrenia. Again, several design issues need to be considered when evaluating this inconsistency, including inappropriately low doses of clozapine⁹, as well as the lack of selection for truly resistant patients.

Attention to first-episode schizophrenia

Beginning in the mid 1980s increased attention to first episode patients seemed warranted to evaluate treatment outcomes that are unconfounded by the effects of prior treatment, multiple relapses, and chronic illness.^11-13 Studies revealed cognitive and psychosocial deficits that were present at illness onset,¹⁴ a long duration of untreated psychosis prior to first mental health contact,¹⁵ increased sensitivity to medication side effects,¹⁶ but also a better treatment response compared with more chronically ill patients.¹⁷ Exploring biological heterogeneity and treatment response at this phase has become an important focus. In addition, as part of the move toward the early treatment of schizophrenia, and the response to new FDA incentives, the efficacy of antipsychotics has also demonstrated in adolescents with schizophrenia.¹⁸ In adolescents, appropriate selection criteria and trial design considerations are also critical.

Comparative efficacy and effectiveness first-generation and second-generation antipsychotics

With the introduction of second-generation antipsychotics, there were observations of lower extrapyramidal side-effect burden and tardive dyskinesia risk and expectations of superior efficacy for positive, negative, and cognitive symptoms.¹⁹ Initial efficacy studies seemed to confirm the superiority of second-generation antipsychotics, but the comparator consisted predominantly of haloperidol, used at moderate to high doses and often without anticholinergic cotreatment, which made early treatment discontinuation and secondary negative symptoms more likely in haloperidol treated patients. Since then, a series of acute phase and longer-term studies have been completed,^20-24 including large efficacy-effectiveness hybrid trials that compared first- and second generation antipsychotics. These data have led to varying conclusions. Interpretations include that there is no difference between first- and second-generation antipsychotics, that second-generation antipsychotics are superior to first-generation antipsychotics, that some second generation antipsychotics are superior to either all or some first-generation antipsychotics, in general, or in certain efficacy and/or side effect domains, or in patient subgroups that are not yet easily identified prior to choosing a specific agent. Since such a number of divergent interpretations have been offered, this indicates that blanket statements do not do justice to the complex data base. Moreover, in comparative trials, design issues are highly relevant to interpretation of the data,²⁵ including sample size, choice of dose of the study drug and active comparator, prior treatment, blinding, duration of treatment, patient and rater expectations and biases, choice of outcomes, handling of dropouts and interpretation of the data, all of which we will discuss in detail below.

Shifting adverse event focus to physical health

Following the predominant use of second -generation antipsychotics, there has been a shift in side effect concerns from Parkinsonism and tardive dyskinesia, to physical health risks and outcomes that are associated with decreased longevity.^26-30 Even more so than the study of tardive dyskinesia, the study of adverse effect risks that are either rare or distal outcomes that occur after many years of illness and antipsychotic exposure pose formidable challenges. This applies to study of sudden cardiac death as a potential consequence or QTc prolongation or other arrhythmogenic properties of antipsychotics,³¹ as well as to diabetes and cardiovascular and cerebrovascular morbidity and mortality. Since these outcomes occur generally after many years or represent premature onset of disorders that also occur in the general population, RCTs might not be the best way to assess the comparative safety of antipsychotics.³² In fact, RCTs have largely focused on the assessment of risk factors (such as weight gain, lipid and glucose abnormalities) for cardiovascular and cerebrovascular illness, rather than on the development of such illnesses themselves. An exception is the assessment of death and cerebrovascular events associated with antipsychotics in the elderly, which, by definition, is an enriched, high-risk cohort. Even here, however, the increased risk with antipsychotics was only uncovered after pooling all data from placebo controlled RCTs in meta-analyses.³³ These examples highlight the fact that for some outcomes, such as rare and temporally distal events, meta-analyses,³⁴ pharmacoepidemiologic studies,^31,35 and cohort studies³⁶ or registries are more useful than RCTs. This is true, despite the considerable limitations of meta-analyses and pharmacoepidemiologic studies and registries. Thus, these methods are merely hypothesis-generating, requiring follow-up with mechanistic studies and RCTs in enriched samples that were informed by meta-analytic, pharmacoepidemiologic, and registry data.

Raising the bar for outcomes

In addition to a broadened focus on physical health, outcomes other than symptomatic improvement have become standard in the field including more systematic and operationalized approaches to measuring response, remission and recovery,^37-40 subjective well-being and quality of life,^41,42 cognition^43-45 and psychosocial/vocational performance.^46-48 These have become end points of increasing importance in routine practice and in clinical trials. However, focusing on such outcomes also requires specific considerations, including treatment modalities, trial duration, assessments, end points, etc.

Targeting individualized treatment

In addition to raising the bar for outcomes, the ways to measure and predict them have also become topics of increasing interest. Efforts at increasing the predictability of outcomes for individual patients (ie, personalized medicine) have included clinically driven nosological and phenomenological approaches, but, so far, these have not really succeeded.⁴⁹ Current approaches that do not yet have consistent clinical applicability include the use of genetics, neuroimaging, neurocognition, and blood- or tissue-based biomarkers and sets of biomarkers, also called biosignatures.⁵⁰ Similarly, developments are underway to define biomarkers as surrogate end points in drug development.⁵¹ To achieve personalized psychiatric treatments, specific design considerations are needed. These include ways of decreasing the heterogeneity- of the study population and the parsing of clinical and biological variables that are relevant for specific mechanisms and treatment effects. In addition, relevant treatment mediators and moderators can serve as selection criteria and randomization or stratification variables. However, obviously, these approaches will depend on the identification of markers that predict treatment outcome to specific interventions and that are not “just” markers of general illness severity and responsiveness (although even such general markers could, for example, help to facilitate early identification of patients who should have a trial of clozapine).^52,53 One such potentially useful “biomarker” or endophenotype that we will discuss subsequently is the presence or absence of early minimal clinical response, which might be useful to enrich samples for specific studies.

Commercialization and globalization of clinical trials

Another historical element in any discussion of RCTs is the “commercialization” and “globalization” of research. In the early days of clinical trials in psychopharmacology, there were a relatively small number of largely academic sites which participated in the design and conduct of such investigations. The development of a specialized clinical trials network was greatly facilitated by the establishment of the “Early Clinical Drug Evaluation Units” by the Psychopharmacology Research Branch at NIMH. Over the last few decades, there has been an enormous shift in the locus of clinical trials from academia to more commercial sites, from the US and Europe to many other countries, and a much greater involvement of a variety of vendors and middlemen in the management and conduct of such trials. The reasons for and consequences of this shift are complex and varied, and a detailed discussion is beyond the scope of this paper. This phenomenon will be discussed subsequently in relation to patient recruitment as well as study implementation and management.

Designing RCTs in schizophrenia

The essential first step in designing any trial, however, is to determine 'what is the question?' All too often investigators attempt to address more than one question in the same clinical trial. Although there is often an opportunity to collect meaningful data on several primary and secondary outcome measures simultaneously, in some cases (eg, the efficacy of a putative therapeutic agent for cognition on negative symptoms or agitation), a specific and distinct type of sample and trial design is needed. Once the primary question is established, patient selection, randomization strategy, treatment selection and controls/comparator (s), trial duration, assessment measures, power analysis, and statistical plan will be the focus of attention. The degree to which appropriate decisions are made regarding these issues will be critical in the success of the trial. We will return to these issues in the subsequent sections of this paper.

Types of trials

Like any other scientific method, RCTs have specific strengths and weaknesses (Table I). These need to be considered and adapted to the specific aim of the investigation. One important decision is the degree to which real world characteristics of populations, treatments, and procedures are systematically restricted and standardized. There are a number of broad categories in which RCTs can be placed. Efficacy studies involve clearly defined and often narrow populations of patients who can be studied with some frequency and intensity with a variety of measures, which would not likely be used in routine clinical practice. Primary outcomes of interest ordinarily include symptom reduction on a validated and reasonably comprehensive scale that is rarely used in routine treatment. While this procedure increases the chance of finding specific efficacy or tolerability signals, the sample and settings in which this signal is detected might become so restricted that as few as 10% to 20% of individuals with a given diagnosis are enrolled,^54,55 affecting the generalizability of the findings.

TABLE I. Randomized controlled trials: strengths and weaknesses.

Selected trial characteristics	Strengths	Weaknesses
Restricted inclusion criteria	High internal validity	Limited external validity/generalizability
	Specific signal detection capacity in carefully selected target population	Difficulty assessing optimal dosing in unrestricted, more heterogeneous or seriously ill sample
	Usable for regulatory and registration purposes	Decreased knowledge about response and side effect patterns in patients with comorbid psychiatric and/or medical conditions
		Slow enrolment
Randomization	Controlling for measured and, especially, unmeasured group differences	Selection bias towards a less generalizable sample (less severely ill, more chronically ill patients; patients with prior stabilization or treatment phase)
	Challenges associated with placebo controls, maintaining blind in the face of specific adverse effects	Selection bias towards a less generalizable sample (less severely ill, more chronically ill patients; patients with prior stabilization or treatment phase)
Small-to-medium sized samples	More homogeneous and carefully characterized samples	Reduced generalizability
		Need for multiple research-oriented sites
		Low signal detection for rare outcomes
Specialized settings	Greater potential for careful selection and diagnostic/assessment	Reduced generalizability
	More control over study procedures	Fewer potential sites
	Greater comfort using placebo controls	Greater likelihood of professional patients
	Study conduct by well-trained personnel with allocated research time	Lower enrolment rates
		Potentially less access to patients of interests (eg, acute exacerbations, drug-naïve)
Frequent visits	Controlled treatment and assessment	Reduced generalizability
	Better assurance of patient safety	Reduced enrolment
	More systematic quantitative assessments for therapeutic and adverse effects	Increased burden and dropouts
	Increased opportunity to facilitate/monitor adherence	Frequent quantitative assessments not done in clinical practice
	Greater ability to use informative laboratory tests
	Better assessment of use of ancillary services or use of other medications	Potential influence on specific, investigated effect by increased contact
Comprehensive assessments	Specific assessment of measurable outcomes (including safety and tolerability) using validated and reliable scales administered by well-trained personnel	Primary/secondary outcomes rarely assessed in clinical practice.
	Usable for regulatory and registration purposes	Patient/caregiver rated outcomes, quality of life and functional capacity rarely assessed in clinical practice
		Use of quantitative measures unlikely in clinical practice and clinicians not trained in their use
		Need for careful training and ongoing supervision of raters
		Increased burden and dropouts
Cost	Usable for regulatory and registration purposes	Increased per-patient costs
	Usable for potential marketing

Effectiveness trials, on the other hand, involve a broader patient population intended to facilitate greater (although still imperfect) generalizability of the trial results to “real-world” patients and less intensive and frequent assessments with more objective and easily determined outcome measures, such as all-cause discontinuation, hospitalization, or death. Effectiveness studies are more likely to include issues such as adherence, tolerability, and quality of life as well as evaluation of the overall impact of a specific treatment. Large “simple” trials or pragmatic/practical trials ideally should include much larger numbers of patients than would be feasible to include in an efficacy study; however, large, multicenter, recruitment efforts can certainly result in sizeable numbers of subjects participating in efficacy studies as well. Other types of trials include crossover and adaptive or sequential designs, which are described in more detail below.

Patient characteristics and selection

Sponsors and funding agencies usually conceptualize acute treatment as that involving recently relapsed or exacerbated patients (often newly admitted to a hospital). However, some investigators include patients who have been chronically symptomatic, but with a symptom severity level great enough to qualify for the study. This alternative approach can lead to the inclusion of patients with varying degrees of potential drug responsiveness. In addition, even if patients are recently relapsed and admitted to hospital, if they have been treated for 2 or more weeks prior to going into the study, their partial treatment (or established poor response) might limit the determination of the full therapeutic potential of the treatments to which they are assigned. That patients have already been partially treated does not necessarily mean that a significant drug effect will not be evident, but the magnitude and time course of that effect and, ultimately, the statistical power can be affected.

Age is another important consideration not only because of the relative prevalence of comorbid medical conditions with increasing age, but also because of potential differences in patterns of response depending upon the phase of illness. Consideration should be given to both age and duration of illness in selecting patient populations for clinical trials. Response patterns, dosage requirements, and vulnerability to side effects are, for example, quite different in first-episode vs more chronically ill patients.^16,56

The duration of the current episode is a factor which has been given insufficient attention in most clinical trials. Recent work^57-61 has suggested that a substantial proportion of the response to antipsychotic agents (at least for ”positive“ symptoms) is likely to occur in the first 1 or 2 weeks of treatment. Therefore, as suggested previously, if a patient has been treated for more than a week or 2 prior to entering the trial it is likely that considerable drug response has already taken place. In some cases, patients need to be at least partially treated before they will be available to participate in a clinical trial, but documentation of length and type of prior treatment in the current episode would help in interpreting results, particularly in failed trials. This also could help to eliminate patients who have been chronically symptomatic and have not had a recent exacerbation.

In this context, it is also important to consider the presence and potential duration of a washout period (which may or may not be placebo-controlled, double- or single-blind). In some cases, investigators might use this not only to eliminate the adverse effects of other drugs, but also to see the severity of untreated symptoms. It is important from an ethical and scientific standpoint that it is clear what the expectations from a washout period really are. In addition, as we will discuss subsequently, we do not have good data on the time course of potential symptom worsening among those patients who are partially treated and who then have their medication discontinued. There are a number of practical constraints on drug washout, which include increased length of trial duration, hospital stay, staff requirements, etc. Without an adequate washout, however, it can be difficult to establish a true baseline for both psychopathology and adverse effects. In addition, there might be withdrawal effects from the prior agent,⁶² which could complicate interpretation of data on a newly instituted medication. The use of a concurrent placebo group can help to mitigate some of these concerns, and this is one important argument for the use of placebos in such trials

Documentation of response to previous treatment in prior episodes can be difficult to obtain on a retrospective basis; however, this can also be important in eliminating potential poor or partial responders. Many current trials stipulate this exclusion criterion, but little is done to validate these assumptions and, given the pressure to enter subjects, there is likely some slippage in this domain.

There is increasing focus on studying patients who have poor or partially responsive symptoms in specific domains of psychopathology, such as negative, positive, or cognitive dysfunction. The selection of such patients is also a challenge, as the retrospective documentation of the stability- and persistence of symptoms despite adequate trials (and with some evidence of treatment adherence) can also be difficult and requires special effort in both the design and execution of appropriate RCTs. In addition, cross-sectionally, negative symptoms can be difficult to differentiate from drug-induced Parkinsonism, depression, demoralization or guardedness. Negative symptoms may also be a secondary (possibly adaptive) response to positive symptoms. Therefore, a longitudinal, prospective approach might be necessary to identify eligible patients for such studies

The presence of comorbid substance abuse and/or other psychiatric conditions is often an exclusion criterion in efficacy trials, but not in practical trials. Suicidality has usually been an exclusion criterion for clinical trials, but it has been shown to be a feasible target for RCTs it its own right with appropriate safeguards.⁶³ The presence of comorbid medical conditions or abnormal laboratory tests has also become an increasing concern given the potentially high rates of metabolic syndrome, smoking, etc. among people with schizophrenia.

Randomization strategies

Most RCTs use even randomization strategies, which assign equal numbers of patients to each treatment arm. This makes sense since the smallest arm will determine the statistical power involved in the comparison between treatments. However, there are some situations where equal randomization might not be appropriate. Equipoise randomization⁶⁴ has been used in some trials and can provide patient and physician input into the potential options involved in the treatment assignment without doing away with randomization altogether. Adaptive design is another strategy which entails the use of data already collected to influence subsequent randomization of treatment groups.⁶⁵ This can be particularly useful in studies involving dose-finding; however, it can also lead to premature closure of a study arm or an entire study based on relatively small amounts of data. Sequential design allows pairing of individuals who are assigned to alternate treatments with the results of each subsequent pair contributing to an ongoing analysis of the odds of reaching a significant effect or not.⁶⁵ This can provide statistical significance with a small number of patients if the treatment effect is substantial, or it can provide an initial confirmation of the null hypothesis. Such a design has rarely been used in RCTs in schizophrenia,⁶⁶ but should be considered.

Stratified randomization is an important tool to ensure that the treatment arms are balanced on a small number of potentially important mediating or moderating variables. The number of such variables will be determined by the overall sample size. Crossover studies are also conducted in some situations, but given potential medication carryover effects and uncertainty as to how vulnerable patients are to returning to their baseline state make placebo-controlled crossover trials less informative in psychiatry than they might be in some medical conditions. So-called “switch” studies are often done in psychiatry, but unless there is a control group they cannot be considered RCTs. Switch studies are obviously much more informative if patients are randomized to be switched or to stay on the original treatment, in a double-blind fashion.

Blinding

While blinding/masking is an important feature to minimize expectation biases, blinding patients and providers to the treatment does not match clinical practice. Moreover, this feature can reduce enrolment and retention and blinding can be undone by known or expected adverse effects, pill characteristics (overcoating, taste) or administrative/procedural issues, such as different treatment providers for different interventions or availability- of laboratory test results that are differentially affected by treatments. A compromise that is sometimes used is randomized, open treatment with utilization of masked raters, but care needs to be taken to maintain the masking.

Treatment selection

The choice of treatment and control(s) will, of course, be heavily influenced by the basic question that the RCT is intended to address. The choice of active control and the target dose(s) of both the investigational medicine and the control agent are important. Estimates of therapeutic equivalence and comparative adverse effect profiles are affected by these choices. If a dose is too low, efficacy may be suboptimal, but if a dose is too high it might inflate the incidence of adverse effects. Titration schedules can also be important for some drugs as well as bioavailability issues related to food ingestion, or metabolic issues related to smoking, body weight, concomitant medications, etc. The side effect profiles of the experimental drug and comparator can also lead to functional unblinding and should be considered from that standpoint as well, or methods can be used to reduce the likelihood of such effects by using an ineffective low dose of the experimental drug as a pseudoplacebo, or separating the ratings of efficacy from those of tolerability, or using centralized raters who do not follow the same patient through a trial.

An important and potentially difficult issue is the extent to which and what kind of “rescue” medication should be made available to those individuals who might otherwise drop out of the trial due to lack of efficacy- and need for further treatment. This possibility can complicate the assessment of the therapeutic agent. However, in some settings it is difficult to conduct a controlled trial without such a provision. As will be discussed subsequently, the possibility of treating all patients initially with active agents, identifying those with a clear early response and then enrolling only the latter subjects in a double -blind, placebo controlled discontinuation study could be a powerful strategy to detect a true drug effect while exposing a minimal number of patients to placebo.

Comedications

The permission, timing, and dosing of comedications also requires consideration. Comedications are useful to limit adverse effect burden and dropouts, but can obscure true treatment effects. Moreover, differential washout of comedications in treatment groups prior to randomization can create confounds, whereas overly limited use of comedications might limit the feasibility of the trial and not match clinical reality.

Placebo controls

Recent discussion regarding placebo controlled clinical trials in schizophrenia has largely focused on ethical issues. The World Medical Association's Declaration of Helsinki⁶⁷ stipulates that “The use of placebo is acceptable in studies where no current proven intervention exists; or where for compelling and scientifically sound methodological reasons the use of placebo is necessary to determine efficacy or safety of an intervention and the patients who receive placebo or no treatment will not be subject to any risk of serious or irreversible harm. Extreme care must be taken to avoid abuse of this option.” Many investigators and, very importantly-, regulatory agencies, such as the Food and Drug Administration in the US and the European Medicines Agency, have taken the position that a valid evaluation of a treatment for schizophrenia (in terms of both efficacy and safety) is not possible without a placebo-controlled design, unless the goal is to demonstrate superiority of the experimental agent over existing treatments. As a result, every antipsychotic that has been approved for the treatment of schizophrenia in either the US or Europe in the past 20 years has been assessed for acute efficacy in placebo-controlled clinical trials.

However, such designs have been challenged.^68-70 In addition, ethical committees in many settings are implementing stricter standards, making it increasingly difficult to conduct placebo controlled clinical trials in schizophrenia. Furthermore, high dropout rates have been reported in clinical trials utilizing placebo controls,⁷¹ and there has also been a decrease in the drug effect observed in clinical trials comparing both experimental and approved antipsychotics with placebo.^72-74

There are a number of potential factors which contribute to these findings ranging from protocol design to patient selection and assessment procedures. Moreover, unexpectedly high placebo response is also seen in patients enrolled in augmentation studies who were supposed to have stable, unresponsive residual symptoms.⁷⁵ Taken together, all of these factors underscore the importance of carefully- considering the benefits and risks of placebo controlled trials, evaluating alternative strategies to achieve needed goals in drug development and ensuring that when placebos are involved that trials are implemented and conducted in such a way as to not inflate or exaggerate the placebo response. It is also important to distinguish between different types of trials, since acute treatment and maintenance treatment trials, or studies of treatment resistant patients, etc. might provide varying challenges in this context.

Trial duration

Both feasibility and scientific considerations influence the length of a trial. Though the full therapeutic benefit of antipsychotics might not be seen for weeks or months, the greatest proportion of response occurs within the first few weeks,^57,58 although this pattern is somewhat less clear for first-episode patients.^76,77 Improvement in positive symptoms can even be seen in a matter of hours or days.⁷⁸ The potential use of placebo controls in short-term, acute treatment trials argues for as short a duration as possible, in that those patients who are assigned to placebo are more likely to experience further exacerbation or lack of response and, therefore, terminate prematurely. Those patients who respond to placebo will be more likely to remain in the study, thereby complicating statistical analysis, diminishing the drug placebo difference and reducing generalizability. In many studies, a significant drug effect can be seen after 1 or 2 weeks. However, if drugs with novel mechanisms are the focus of investigation then assumptions about time course of response might be less reliable. There is also a subgroup of patients who are slower to respond and if the ultimate goal is to compare the full therapeutic potential of alternative treatments, then a longer trial might be desirable. Issues related to trial duration in maintenance of effect/relapse prevention studies will be discussed subsequently.

Outcome and assessment measures

The selection of assessment measures and instruments will be largely driven by the choice (s) of the primary and secondary outcome measures as well as by feasibility and rater/patient burden. Often, too many scales are included in a clinical trial and some of the data are never analyzed or published. Attention to scale validity and reliability is also important, and in regulatory trials there is a particular emphasis on instruments, which have been demonstrated on a broad scale to have the desired characteristics. If a new scale is introduced, it is often recommended to have an existing and widely utilized scale for the same domain included as a reference point. As there is increasing emphasis on patient reported outcomes, however, there is also some concern as to the validity of such measures for those individuals who are lacking in insight or unable to reliably evaluate their own subjective and/or functional state. In the case of schizophrenia, informant information can also be extremely valuable. Patient-reported outcomes in some cases can be im0peded by willful concealment or distrust of the interviewer or interview situation. Clearly, as broader outcome assessments are called for, measures of negative symptoms, cognitive function, social and vocational performance/quality of life, subjective well-being, family burden, etc should be considered.

In contrast to some dimensions of psychopathology, the longer time frame needed to assess the amelioration of negative symptoms and cognitive dysfunction or improvements in overall social and vocational adjustment might require trials of much longer duration. This is especially true if issues of persistence of effect are to be clarified. Finally, it has been recognized that adverse effects are not as carefully and comprehensively measured as efficacy- measures.⁷⁹ This should also be remediated by adding a brief interview based or self-administered adverse effect check list to spontaneous reporting.

Quality of ratings and fidelity of assessments

In order to have a sufficient signal-to-noise detection ratio for diagnostic, symptomatic and side effect assessments, both the utilized tools and the raters performing the interviews and ratings need to be highly reliable. In fact, the intraclass correlation coefficient (ICC) between ratings has a profound impact on sample size requirements. For example, an ICC of 0.9 requires 111 patients compared with 200 patients if the ICC is 0.5 in order to achieve the same statistical power.⁸⁰ The way that raters are trained and the manner in which reliability- is established varies. In fact, true interrater reliability is rarely established in multicenter clinical trials. Specifically, having prospective interviewers only rate videotaped assessments performed by an expert does not establish the kind of reliability that is necessary. Even high ICCs with the expert rater do not in any way establish the ability of the rater to elicit the same symptoms when conducting an independent interview that he/she was able to rate when being fed the patient responses in an idealized training tape. Moreover, the method of rating even taped interviews is not usually standardized, so that it is not clear to what degree ratings occur completely independent in the classroom. In addition, a sufficient number of such assessments to establish statistical correlations is rarely done. Furthermore, even if reliability was established for both the interview and the rating, rater drift needs to be countered by reassessing the reliability of the ratings periodically throughout the trial, as well as training new raters when there is staff turnover. Other methods of increasing precision of ratings include comparing similar outcome dimensions across different assessment scales (ie, convergent validity) or checking rater-assessed outcomes against patient reported outcomes or against the evaluation of quality control by remote expert raters (ie, external consistency). In case of obvious inconsistencies, raters can then be approached and simply be given feedback or they can be retrained. However, even though expert raters can be used to check or adjudicate site based ratings, they have to rely on the interviews that may be less than optimal in obtaining a full clinical picture. Research has shown that many assessments were deficient when site based interviews were audiotaped and randomly assessed by expert raters.⁸¹ Another method, particularly for multisite studies that has shown considerable promise to increase the reliability of ratings and reduce placebo response;⁸² includes the use of remote centralized expert raters who perform the assessments via live, two-way video. This method can be expensive and poses some logistical challenges, but is in keeping with the desire to centralize and standardize assessments whenever possible, as has increasingly been done with cardiology, pathology, radiology, and laboratory tests in multicenter trials.

Relapse prevention

Relapse prevention in schizophrenia remains a major public problem. However, the number of studies focusing on relapse prevention/maintenance treatment is substantially smaller compared with acute phase trials.^83,84 A particular challenge in relapse prevention studies is the definition of relapse, for which no universal criteria exist.⁸³ Given ethical concerns about placebo-controlled trials in relapse prevention, it has become customary to utilize relapse criteria which do not require a full-blown psychotic exacerbation, but rather rely on minimally clinically significant early signs of relapse. Subsequently, relapse rates might be higher than in studies conducted previously, and there are a number of potential false positives.

The use of placebo controls in relapse prevention studies is another source of controversy, and opinions of regulatory authorities also differ on this topic. Some would argue that the demonstration of non-inferiority in comparison to a proven efficacious compound should be sufficient. However, both dropout and response rates vary whether an active or placebo control is used,⁸⁵ and relapse rates vary enormously- across trials. For example, a recent trial comparing depot and oral medications reported rehospitalization rates of 39% and 45%, respectively, in a 2-year study.⁸⁶ By contrast, other trials reported rehospitalization rates as low as 1.3% and 5.8% with depot and oral medications, respectively, at 1 year,⁸⁷ and 9.3% and 15.2% , respectively, at 2 years.⁸⁸ Therefore, it is difficult to be certain if one is dealing with an ineffective medication or with a patient population that is highly vulnerable to relapse regardless of medication status.

Another important issue that needs to be considered in the design of maintenance and relapse prevention studies is the timing of the randomization. In most trials, patients are randomized in the acute treatment phase and then continued into an extension maintenance study. However, if patients are not rerandomized after stabilization, the concern is that by including randomly assigned, acutely exacerbated patients, only those patients at risk for relapse who had responded to and tolerated the specific acute treatment participate in the maintenance portion of the trial. This could lead to a selection bias for patients who experienced less side effects or experienced more improvement on the allocated medication. This concern is particularly relevant when there are unequal proportions of patients in each originally randomized group that enter the maintenance and relapse prevention phase of the study.

The degree to which patients entering the trial are stable and whether this is established retrospectively or prospectively are other important considerations. As for relapse, stability criteria and the required duration of stability or remission are insufficiently standardized. Another important issue is the duration of the trial. Since some long studies suggest different patterns of relapse during the first and second years,^89,86,90 a duration of 2 years or longer is ideal. But, of course, the longer the duration, the higher the dropout rate might be. The dropout rate varies from study to study, but some surpass 50%. This issue is clearly more important in active-active comparisons, as drug placebo differences can usually be seen in shorter time frames.

Finally, the determination of optimum dosing for maintenance/relapse prevention is a particular challenge, given the unpredictable time course of relapse. Even when patients are completely withdrawn from antipsychotics when in a state of remission or stability, the resulting relapse might not occur for weeks or months. Therefore, if a flexible dose is used, it is difficult to determine whether or not a relapse is due to an ineffective medication or if it is due to reducing the dose below an efficacy threshold for that patient. Therefore, fixed-dose studies are valuable in the maintenance phase to evaluate the dose response relationship, which might be quite different from that observed in acute efficacy trials where the goal is reducing acute and severe psychopathology. Importantly, given high potential rates of non adherence, the use of long-acting injectable medications can be very valuable in this context to ensure that nonadherence does not confound the interpretation of dose-response relationships.^91-92

Statistical issues

Several issues of clinical trial design influence sample size estimates and the power to detect a clinically meaningful treatment effect, while maintaining a nominal level of type I error. For example, multiple outcomes can inflate type I error, and unreliable assessment processes and imprecise measurements can introduces biases and reduce statistical power.⁹³ In addition, missing data pose considerable challenges. It is increasingly recognized that last-observation-carried-forward (LOCF) analytic methods are problematic and that mixed models repeated measures (MMRM) analyses for continuous outcomes and Generalized Estimation Equation (GEE) models are a superior way of handling missing data. It took a while for regulatory agencies to agree to this, but nowadays MMRM analyses are also an acceptable analysis method for registration trials. However, there are really no good solutions for dealing with missing data that are almost never missing truly at random. Even methods like MMRM and GEE that adjust the analyses based on results from patients who continued in the trial have their limitations, as their validity is based on the assumption of ignorable attrition,⁹⁴ highlighting the importance of minimizing dropouts and missing data as much as possible. In fact, dropout rates have become an increasing problem like placebo response rates.⁹⁵ Thus, studies need to be designed in ways to minimize dropout rates, for example by not creating incentives for leaving the study early. Incentives for patients may include a rollover in an open long-term extension phase study where treatment is free, while incentives for investigators might include recruiting patients to a subsequent randomized study phase. In addition, comedications to minimize the severity of adverse effects and time limited rescue strategies for inefficacy that do not compromise the primary outcome should also be considered. Another proposed possibility is the measurement of the “intent to attend” the next study visit and to use this as a covariate to decrease the attrition bias.⁹⁴ In addition, identifying patients who are not likely to continue in the study or who find participation burdensome prior to their dropping out can help research personnel to proactively address barriers to trial completion. Finally, allowing for in-person, two-way video or telephone assessments in the patient's home should also be considered to reduce the amount of missing data.

Trial Implementation and conduct

One of the most neglected areas of clinical trials is trial management and oversight. As signal detection has become increasingly difficult and sample sizes have increased, the conduct and quality control of large-scale RCTs has become increasingly complex and difficult.⁶⁵ Subsequently, many companies have outsourced this important aspect of trial implementation and performance. As a result, there is the danger of a loss of control and diffusion or narrowing of responsibility, in that clinical research organizations are mostly in charge of assuring that increasingly tight time lines are kept and quota are met. The enormous time pressures can lead to a problematic disconnect between the desired quantity and the desired quality of enrolled patients and assessments. Moreover, increasing regulatory requirements can also lead to an overburdening of sites and investigators who are not part of professional clinical trials sites and who might drop out of multisite RCTs, thereby narrowing the settings in which patients are studied. Furthermore, the focus on assuring adherence to formal requirements, which has appropriately attracted scrutiny and attention, should not distract from assessing the quality of the trial conduct that is not equivalent to following checklists and increasingly complex documentation.

To overcome some of these problems, recent concern has focused on finding ways to encourage trial management organizations to broaden their responsibility beyond considerations of documentation and fulfilling quota. Rather, methods need to be considered that provide incentives to these organizations to assure adherence to appropriate standards of patient selection and high quality assessments, follow-up, protocol adherence, and retention. However, a high placebo response or lack of separation of the investigational drug or standard comparator from placebo cannot automatically or solely be used as a quality indicator. Moreover, quality adherence should also be measurable and achievable during the conduct of the trial and not only after its conclusion. Therefore, the field needs to develop standards against which sites and clinical trial management organizations can be assessed and which can provide clear guidance to all parties involved in the conduct of the trial.

Novel trial designs

In addition to addressing the issues and challenges previously discussed, there is also the need to explore novel trial designs which can help to facilitate signal detection and overall trial efficiency. There is a powerful clinical tool that uses the patients' own response pattern to predict outcomes. This intraindividual test of early response/nonresponse as a predictor of subsequent response^96,97 or the predictive value of dysphoric response⁹⁸ had been studied briefly in the 1980s. As much as 15 to 20 years later, these findings have been revisited and expanded upon, stimulated by analyses showing that, at least at a group level, the majority of antipsychotic response occurs within the first few weeks^57,58 and, even days⁹⁹ after antipsychotic initiation. Building on these findings, a series of post-hoc analyses^{59,60,100-102} plus a recent prospective study⁶¹ showed that nonresponse at study end point can be predicted with high sensitivity, specificity and predictive power by presence of less than a minimal response, equivalent to less than 20% reduction in the Positive and Negative Syndrome Scale¹⁰³ or Brief Psychiatric Rating Scale104 total score at 2 weeks after antipsychotic initiation. However, having identified this general response pattern, questions remain as to whether such trajectories are similar in the more likely heterogeneous first-episode schizophrenia samples and in treatment-refractory patients.^76,77 In addition, it needs to be determined whether or not a limited set of specific symptom items that could be used in clinical practice are equally valid and reliable¹⁰⁵ and what one can learn from symptom trajectories at an individual patient level.^106-109

This strategy would be very valuable in helping to determine what alternative treatments are likely to be more successful after early nonresponse has been identified.⁶¹ A novel design to help enhance signal-to-noise ratio in an acute trial could take advantage of the response patterns that have been identified (Figure 1), In the “early responder randomized discontinuation design” all patients are assigned to active drug, and then only those who had at least a minimal response at 2 weeks are enrolled in a double-blind, placebo-controlled discontinuation trial. This design could potentially enrich the placebo controlled portion of the trial with true drug responders and thereby expose fewer patients to placebo. A recent report by Marques et al¹¹⁰ suggests that those patients with a robust early response are less likely to include placebo patients than other trajectories of response. Appropriate data should be collected to determine what proportion of early responders would show an exacerbation following placebo substitution and within what time-frame. The ethical implications of such a design should also be considered. Is an exacerbation on placebo substitution for a subgroup of patients more acceptable than a 4- to 6-week exposure to placebo for a larger number of patients? At the same time, early nonresponders at 2 weeks can also be used for rerandomization to test strategies focusing on enhancing response. In the mirror image “early nonresponder randomized dose increase or augmentation design,” early nonresponders at 2 weeks are assigned to staying on the medication or going either to a higher dose or an augmentation agent. The dose increase or augmentation option will likely mostly be studied separately. However, including both options in a three-arm design would also be possible. This might be especially attractive when studying the addition of a second antipsychotic as the augmentation strategy. Having the dose increase arm in this design would allow distinguishing the effect of non-dopaminergic receptor synergies vs mere increased antipaminergic “dose” increase.

Figure 1. Novel drug development design utilizing early response/nonresponse for sample enrichment. randomization time point.

In effect, this proposed design, “the early responder randomized discontinuation design” is an alternative to a previously proposed study design, “the sequential parallel comparison design.”¹¹¹ In contrast to the design that we are proposing which has a 2-week active drug run-in phase, the sequential parallel comparison study consists of two phases of randomized treatment of equal duration. The first phase involves an unbalanced randomization between placebo and active treatment with over-sampling of placebo randomization. The second phase involves re-randomization of placebo nonresponders to active treatment or placebo. As patients in the second phase “failed” placebo before, they are less likely to respond to placebo, which diminishes the placebo response and has the potential of enhancing power.

However, at the same time, drug response rates are also likely to be reduced. The complication with this design is the proposed data analytic technique that does not only use patients from the second phase, ie, in an enhanced sample of placebo nonresponders. Rather, outcomes from both phases are combined in a complicated pooling ratio.¹¹¹ However, were only patients from phase two to be used, this would necessitate a very high number of patients to undergo the first phase.

Conceivable alternatives to this design include two phases of unequal duration with rerandomization of early placebo nonresponders (in schizophrenia: <20% reduction in PANSS total score) after only 2 weeks to either placebo or active treatment (ie, early placebo nonresponder sequential parallel comparison design). Alternatively, a triple-blind, 2-week placebo lead-in phase could precede randomization to drug or placebo in patients with <20% reduction in PANSS total score, rather than randomizing patients without a “full” response (however defined in a given study and disease) at 4 weeks or longer, as proposed in the sequential parallel comparison design.

As can be seen from these examples, the enrichment strategy of focusing on early responders or nonresponders to either drug or placebo could potentially be leveraged to enhance the signal-to-noise ratio, depending on the question that is being asked.

Summary and conclusions

Clinical trials, like everything else man-made, are imperfect. Their specific content and success are context dependent. A number of factors that were outlined in this article need to be considered, controlled, monitored, and improved upon. In addition to a number of standard features, the design of RCTs needs to be tailored to the research question, population, illness phase, setting, active treatment, control condition and outcome under investigation. Patient selection, blinding, ratings, study/site management and adherence are important aspects. Innovative designs should be considered in order to deal with some of the inevitable compromises involved in designing and conducting RCTs. For some research questions, alternative study types might need to be considered, such as cohort, pharmacoepidemiologic database or registry studies. Importantly, measurable quality standards for RCTs need to be developed. Applying these standards along with novel ways to incentivize all of the parties involved in order to achieve increased adherence to quality measures need to be explored. To achieve this, the different stakeholders should share experiences and actual data to come up with appropriate solutions. We need to learn from the past as much as possible and we need to appreciate that failed and uninformative trials, increasing placebo response rates and increased sample size requirements in the context of decreasing effect sizes are a critical and destructive, but shared problem that needs viable solutions. Without this shared responsibility for the design and conduct of high quality trials, the development of new compounds and the broadening of indications for patients in strong need of effective and safe treatment alternatives will become increasingly difficult. In addition, more and more companies will be discouraged from pursuing these therapeutic targets for drug development. Finally, the utility of novel trial designs that decrease placebo response and enrich samples should be tested and their appropriateness for regulatory approval pathways needs to be explored.

Financial Disclosures: Dr Correll has been a consultant and/or advisor to or has received honoraria from: Actelion, AstraZeneca, BoehringerIngelheim, Bristol-Myers Squibb, Cephalon, Eli Lilly, Intracellular Therapies, Ortho-McNeill/Janssen/J&J, Merck, Otsuka, Pfizer, and Sepracor/Sunovion. He has received grant support from the Feinstein Institute for Medical Research, the National Institute of Mental Health (NIMH), and the National Alliance for Research in Schizophrenia and Depression (NARSAD) and Ortho-McNeill/Janssen/J&J. Dr Kishimoto has received speaker's honoraria from Banyu, Eli Lilly, Dainippon Sumitomo, Janssen, Otsuka, Pfizer. He has received grant support from the Byoutaitaisyakenkyukai Fellowship (Fellowship of Astellas Foundation of Research on Metabolic Disorders) and Eli Lilly Fellowship for Clinical Psychopharmacology. Dr Kane has been a consultant to Astra-Zeneca, Janssen, Pfizer, Eli Lilly, Bristol-Myers Squibb, Dainippon Sumitomo/Sepracor/Sunovion, Johnson & Johnson, Otsuka, Vanda, Proteus, Takeda, Targacept, Intracellular Therapies, Merck, Lundbeck, Novartis Roche, Rules Based Medicine, Sunovion and has received honoraria for lectures from Otsuka, Eli Lilly, Esai, Boehringer-lngelheim, Bristol-Myers Squibb, and Janssen. He is a shareholder of MedAvante. He has received grant support from The National Institute of Mental Health.