Clinical trials of novel psychotropic agents: some caveats

In their paper, Correll et al ¹ present proposals for strategies to “de‐risk” trials of nov­el psychotropic agents. However, several of their suggestions may inadvertently increase the risk that clinical trials be uninformative, especially when considering requirements for drug approval. Here we provide our perspective on their advice.

The authors begin their “de‐risking” advice with some foundational concepts related to validity, power, and a priori hypothesis generation. They go on to discuss the importance of “clinical equipoise” in randomized controlled trials. This emphasis is reasonable. Without clinical equipoise, trials are vulnerable to bias and are more difficult to interpret. For example, the enthusiasm for psychedelic drug development from both the lay press and investigators may contribute to difficulties separating drug effect from expectation bias.

Correll et al subsequently offer sugges­tions for modifying trial designs in an at­tempt to avoid failed studies. One recom­mendation is to consider adaptive trial de­signs whereby the beginning of the trial informs its later stages. The US Food and Drug Administration (FDA) has published a guidance for in­dus­try on the use of adaptive trials ² . Compared with a traditional clinical trial, patients enrolling at the start of an adaptive clinical trial may not have the same experience as those enrolling later in the trial (e.g., possible doses). This may lead to challenges in interpreting the trial results. Further, although adaptive trials may be designed to maximize the possibility of quickly detecting efficacy with limited enrollment, more subjects may still be needed to characterize safety. A positive adaptive trial may not translate into approval if there are safety signals that must be explored in larger or longer studies. Sponsors considering adaptive studies in phase 3 should discuss their plans with regulatory authorities before implementation.

Phase 2 is an important part of dose ex­plo­ration. Correll et al suggest using adap­tive trials to determine the maximum tolerated dose (MTD) and prevent “expensive and underpowered multi‐armed studies”. However, as they later acknowledge, there are chal­lenges to using an MTD when a dose range may be required. The MTD as determined early in a study may not translate to the op­timum dose, considering benefit‐risk, as the study progresses.

A Phase 2 program examining several doses based on phase 1 data (e.g., receptor binding, tolerability) need not be adequate­ly powered to demonstrate safety and effi­cacy for each arm. It is meant to inform a phase 3 program. Sponsors sometimes de­sign phase 2 studies with characteristics of adequate and well‐controlled investigations in hopes that a positive trial may be used to support a marketing application. However, if there are dosing, endpoint, population or safety is­sues, this approach may ultimately prove more costly.

The paper's discussion of precision medicine versus generalizability is important. We acknowledge that particular mechanisms of action may have benefits particularly ap­plicable to subpopulations, and that en­riched trials may improve the chance of de­tecting an efficacy signal. However, deve­l­op­ment programs should focus not on ar­ti­ficially narrowed populations, but on a pop­ulation widely inclusive of those likely to re­ceive benefit.

A reasonable starting point for separat­ing promising subgroup effects from post‐hoc artifact is biological plausibility. Although a collection of clinical characteristics could be representative of a biological construct, there is a public health interest in determining what that underlying construct is. The authors suggest that positive studies from an enriched population could lead to an approval for use of the drug in a subpopulation, with studies of a broader population deferred to post‐approval. However, in the absence of a biologically plausible sub­group definition supported by strong scientific understanding, we do not support this approach. Sponsors should explore scientifically justified potential subgroups in phase 2, refer to the appropriate guidance ³ , and discuss plans with regulatory authorities.

Placebo lead‐in studies have often not met expectations in psychiatric disorders. Sequential parallel design remains an unproven alternative to traditional placebo lead‐in strategies. As with adaptive trials in general, there are significant challenges in interpreting the results of such studies. There is not a standard method for analyzing the results of sequential parallel design studies, and employing such a design in phase 3 entails risk on the part of a sponsor. Sponsors considering sequential parallel design should discuss this with regulatory authorities.

Correll et al state that “FDA…[has] taken the position that to assess the efficacy of a new treatment for many mental disorders is not possible without a placebo‐controll­ed design”. This is not accurate4, 5. The Code of Federal Regulations, Title 21 (section 314.126) ⁶ describes the characteristics of an adequate and well‐controlled clinical investigation, and specifically mentions other types of controls – such as active treatment concurrent control and no treatment concurrent control – in addition to placebo concurrent control. Placebo‐controlled trials are often favored and chosen by sponsors because they typically produce the most readily interpretable results.

Regarding generalizability of clinical tri­al results, Correll et al note that many “real world” patients would not qualify for pharmaceutical trials because of comorbidities. Sponsors should be prepared to justify their exclusion criteria, focusing on comorbidities that are expected to complicate interpretation of the study or decrease the likelihood of detecting an effect (e.g., active substance use disorders). The paper suggests requiring post‐marketing studies to examine drug efficacy in “real world” patients; however, the FDA does not have the statutory authority to require such studies ⁷ .

Correll et al describe scenarios in which rapid recruitment may impact study quality. Baseline symptom inflation and diagnostic imprecision may speed recruitment but will also make demonstrating efficacy more difficult. Although small sites may be a source of heterogeneity, they may simply be recruiting judiciously. Therefore, we recommend caution regarding the suggestion to drop poorly recruiting sites early in the study.

We agree that some new technologies might have the potential to improve assessments; however, before incorporating novel assessments (e.g., digital endpoints), we recommend that sponsors submit supportive evidence that the technology is fit‐for‐purpose. For example, a computerized system for assessing patient speech may seem to be an improvement on established subjective clinician ratings. However, it is the subjective clinical ratings which would have been tied to dysfunction and prognosis. Unless the computerized system also reflects dysfunction and prognosis, it may not be fit‐for‐purpose. Additionally, sponsors should ensure that including technology does not discourage or prevent certain groups from enrollment or introduce unanticipated biases.

Sponsors should discuss novel statistical approaches with regulatory authorities prior to starting clinical trials. Regarding the suggestion to use an endpoint that reflects symptom course over time (rather than at discrete time points), this may or may not be acceptable for a given trial. Such averaged endpoints may reflect improvement at the start of a trial that is lost as the trial progresses, leading to questions about the durability of effect.

Before attempting something novel in a development program, sponsors should meet with regulatory authorities, which can often refer companies to pertinent publi­sh­ed guidances, help think through regulato­ry requirements, and use experience from oth­er programs to offer recommendations.