Spinal cord injury neuroprotection and promise of flexible adaptive clinical trials

Abstract

Effective treatments for acute neurological illness and injury are lacking – particularly for spinal cord injury (SCI). The very structure of clinical trials may be contributing to this, as assumptions made during trial planning preclude additional learning within residual important areas of uncertainty, such as dose, timing and duration of treatment.

Adaptive clinical trials offer potential solutions to some of the factors that may be slowing the pace of discovery. Broadly defined one can consider an adaptive clinical trial as any sort of clinical trial that makes use of information from within the trial to make decisions about how the trial is conducted going forward; however it is important to emphasize that regardless of the degree of flexibility or complexity of an adaptive clinical trial design the types of designs being described are only those in which all potential changes to the conduct of the trial are prospectively defined prior to enrollment of the first patient. Within this review, we describe the structure of flexible adaptive clinical trial designs, the process by which they are developed and conducted, and potential opportunities and drawbacks of these approaches.

We must accept that there are some uncertainties that remain when designing both exploratory and confirmatory trials. The process by which teams carefully consider which uncertainties are most important and most likely to potentially compromise the ability to detect an effective treatment can lead to trial designs that are more likely to find the right treatment for the right population of patients.

Introduction

Effective treatments for acute neurological illness and injury are lacking. Given the high morbidity and mortality associated with these conditions, this is an important area for discoveries that will translate into improved patient outcomes. To date, successes are far outnumbered by failures in clinical trials.¹ This lack of translation is particularly frustrating given the number of promising targets that have been identified in preclinical studies.

Within the specific field of spinal cord trauma, we find that the situation is similar.² There is a paucity of available treatments for patients with spinal cord injury (SCI) and serious motor deficits. This is in spite of promising targets from preclinical data. One potential explanation for this is that clinical trials are not conducted to mimic the preclinical scenarios. Assumptions are often made in clinical trial planning that do not address areas of uncertainty, such as dose, timing and duration of treatment.³ Faulty clinical trial designs may lead the conclusion that the treatment is ineffective, when in fact there is a positive treatment effect. Contemporary clinical trials have a high failure rate, are expensive, and are difficult to conduct in fairly rare disease states – particularly where time sensitive enrollment is needed as in emergency conditions such as spinal cord injury.⁴

In order to learn more about potential avenues for the acceleration of the drug discovery pathway, the FDA and NIH jointly funded the Adaptive Designs Accelerating Promising Trials Into Treatments (ADAPT-IT) project.³ Briefly, in this project we are designing four adaptive clinical trials of acute neurological emergencies for the Neurological Emergencies Treatment Trials (NETT) network, including one for acute spinal cord injury, using a formalized process that includes: a) statisticians experienced in Bayesian adaptive trial design, b)statisticians experienced in the design, conduct and reporting of large network based clinical trials; c)clinical trialists focused on acute neurological emergencies, d) statisticians from the FDA, e) program officials from the National Institutes of Neurological Disorders and Stroke (NINDS), f) content experts in SCI, and g) NETT investigators. The two major aims of this project are to create flexible, innovative, adaptive designs that could potentially be moved forward as part of major clinical trial grant applications to the NIH, and to study the process of trial development via direct observation, surveys, and focus groups (mixed methods).

Adaptive clinical trials offer potential solutions to some of the factors that may be slowing the pace of discovery. Broadly defined one can consider an adaptive clinical trial as any sort of clinical trial that makes use of information from within the trial to make decisions about how the trial is conducted going forward.⁵ Under this view, a group sequential design with an interim analysis for futility or overwhelming efficacy is a form of an adaptive design, although the amount of additional learning or flexibility that this sort of design affords is limited. It is important to emphasize that regardless of the degree of flexibility or complexity of an adaptive clinical trial design the types of designs being described are only those in which all potential changes to the conduct of the trial are prospectively defined prior to enrollment of the first patient.⁶ Adherence to this principal is necessary to ensure the statistical validity of an adaptive design. Within this review, we will describe the structure of flexible adaptive clinical trial designs, the process by which they are developed and conducted, and potential opportunities and drawbacks of these approaches.

Adaptive designs: definitions and taxonomy

The most general definition of an adaptive design is a clinical trial in which information from enrolled patients informs what happens to subsequent patients. The simplest example of this would be a group sequential design of a new treatment versus a placebo.⁷ If the efficacy boundary has been crossed at the prespecified interim analysis, then the trial stops and success is declared. Other types of designs can potentially provide more information or potentially more definitively illustrate that a treatment strategy should be abandoned (See Table for overview). As an example, one could consider a drug that is given out to 12 hours after SCI. A relatively inflexible design will carry out this trial design as planned with one treatment window at 12 hours. If the trial is stopped for futility, the investigators may regret that they did not look at a six hour window or considered using a higher dose of the drug that was well tolerated. An adaptive design that takes into account and learns about the residual uncertainty in optimal doses and treatment windows could address both of these areas of uncertainty and render more definitive conclusions at the end of the trial. A number of areas can potentially be modified by an adaptive clinical trial algorithm during the course of the trial. In multi-arm trials (and to a lesser extent two arm trials) the allocation of patients into arms can be driven by statistical modeling. For example, in a dose finding study, information regarding the efficacy and toxicity of a particular dose strategy is available from within that arm but also potentially from other adjacent arms can be informative regarding the true shape and slope of the dose response curve. During randomization, patients can be preferentially assigned into arms that are performing better or in which greater uncertainty exists to better calibrate and fit the model to determine the optimal dose. This also tends to place more patients near the dosages that are more likely to be effective.⁸ Another area that can be adjusted using adaptive design is the sample size of the trial.⁹ This can be adjusted in both directions. However, for planning and budgetary purposes a maximal total size for the trial needs to be defined. Typically, if a conservative estimate for the treatment effect is made and patients respond to the new treatment better than expected, the trial could potentially be ended earlier at an interim analysis similar to the group sequential method described earlier. Group sequential methods typically only have one or two interim analyses; however some alternative designs can take more frequent looks without meaningful loss in overall trial power assuming the operating characteristics of the trial have been carefully simulated. These simulations allow one to balance the “cost” in loss of power versus the potential benefit of ending the trial early if the true treatment effect is greater (or smaller) than initially hypothesized. This means that for treatments that are truly appearing superior, a planned 1000 patient trial may be accomplished with 500 patients. On the other hand, for treatments appearing to be non-efficacious or harmful, termination will likely occur earlier compared to the typically extreme group sequential boundary.¹⁰ Finally, another additional high yield area is in patient selection. Whereas a more traditional trial may employ both pre-specified and post hoc subgroup analyses to inform the design of further trials, it is possible to prospectively define these within the auspices of an adaptive design. In such as design, it may be that the overall population included in the trial is not responding, but an identifiable group of responders exists.¹¹ The trial can then switch its efforts to focus on only enrolling those patients. In certain cases, especially if the trial has been rigorously simulated in advance, one can use patients enrolled from this early learning phase to contribute to the final hypothesis test of efficacy—increasing efficiency and maximizing the contribution of all trial volunteers.

Table.

Types of Adaptations of Potential Use in Spinal Cord Injury Trials

Type of Adaptation	Description	Comment
Dose Response Modeling	Simultaneous evaluation of multiple doses or treatments within a unified statistical model, multiple doses may be evaluated within a trial to learn about shape of dose response curve and to maximize likelihood of discovering effective treatment	While more common in exploratory trials, there is often residual uncertainty regarding optimal dose even in confirmatory trials.
Early stopping for efficacy or futility	The use of Bayesian predictive probabilities or statistical boundaries to make trial decisions at interim analysis points	Common in clinical trials through group sequential designs, most previous trials have had few interim analyses. More aggressive but judicious use may allow patients and resources to be more quickly allocated to new trials when trials can fail elegantly at an early time within accrual.
Response adaptive randomization	Patients are randomized to arms unequally, with preference towards arms with higher predicted probability of success	Can potentially improve learning by assigning more patients to greater area of uncertainty. In addition, in trials with few arms, will usually assign more patients to treatment regimen ultimately shown to be more efficacious if it exists.
Longitudinal modeling	Within individual patients, early responses to treatment are used to predict later responses for the purposes of trial decisions (efficacy stopping or randomization ratios).	Ensures that at any interim analysis, all data that has been collected on each patient contributes to decision making at that point in time. Therefore trial decisions regarding changing randomization ratios, dropping arms, or declaring early success are based on all enrolled patients, not only ones that have completed delayed outcomes. As SCI trials frequently have a primary outcome at one year, the amount of patients within a trial with partial information at any point in time may be significant.
Enrichment	Prospectively searching for the ideal target population and potentially closing future enrollment to subjects outside those parameters	This technique is in distinction to searching a neutral trial for subgroups that appear likely to succeed. One example would be starting a Phase III trial with a 6 hour enrollment window, but if the treatment appears to be working poorly in later patients, trim down the enrollment window to 4 hours based on a pre-defined statistical rule.

In the exploratory phase, adaptive designs can help identify the best population to move forward into a phase 3 trial along with the most promising doses and treatment strategy. In order to maintain flexibility and recognize that there will be residual uncertainty even after a well-constructed and well implemented phase 2 trial (if for no other reason that these are usually relatively small), it is important to consider that the phase 3 trial following it (confirmatory trial) will also have some flexible elements regarding two or three potential doses moving forward and perhaps some question regarding the optimal treatment window, especially in acute trials for central nervous system neuroprotection. This is in contrast to fixed exploratory phase trials in which a small version of the phase 3 trial occurs with a primary outcome of a low probability safety event. Such a design may not provide much information regarding the treatment effect or which dose/ strategy to move forward.

The potential scientific and medical goals that can be accomplished using adaptive designs in the confirmatory phase are similar to those in the exploratory phase; however the type I error rate of the trial needs to be well understood via simulation so that inference regarding whether the new treatment is effective is straightforward to clinicians and regulatory agencies. Scientifically, one can allow for greater uncertainty in the selection of the optimal dose and patient population at the beginning of the phase 3 trial, although the understanding should be greatly improved from what was known prior to the informative phase 2 trial. In certain cases, the learning and confirming phase can be combined into a single registration confirmatory trial. This often depends on how much uncertainty exists at the beginning and how many potential pathways the exploratory phase can take. Separate from the scientific aim of a trial, an important medical goal of particular importance in serious illnesses can be to improve the outcomes of the patients within the trial. This can be accomplished using response adaptive randomization to assign more patients to treatments with a higher likelihood of emerging as superior.⁸

Motivation to being adaptive in spinal cord injury trial designs

From the patient's perspective, certain advantages of adaptive versus more traditional clinical trials may exist. In trials of conditions requiring emergency treatment, it is unlikely that an individual would have the opportunity to choose between two trials with different designs. However, if an adaptation such as response adaptive randomization is assigning more patients into the arms of the trial that have a greater likelihood of either succeeding in a confirmatory phase trial or providing important information about dose response in an exploratory phase trial, then the patients within the trial may benefit. If the interventional treatment is actually harmful, more patients would be assigned to the standard treatment or placebo, improving the overall outcomes of the population of patients in the trial. From the perspective of scientists and sponsors, adaptive designs in spinal cord injury are appealing for a few reasons. First, the overall number of patients enrolled in spinal cord injury trials is relatively limited therefore a great deal of knowledge regarding how patients will respond to the treatments is lacking. Even unbiased, contemporary natural history data is limited. A more flexible adaptive design may answer more questions or least provide more information than a more traditional fixed design. Second, an adaptive trial may better explore the design space of potential uses of the treatment within multiple different and more restrictive patient populations. If the areas that were deemed promising are sufficiently broad to dismiss the treatment, resources can be reallocated to look at the next best treatment which has emerged from ongoing discovery initiatives in the pre-clinical and “first in human” phases. Overall, this should increase the likelihood that that next treatment will be one that ultimately improves outcomes; at the very least the expected time to identifying the next effective treatment should be reduced. In certain cases, the exploratory phase can be done in an ongoing and seamless fashion. The Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging And moLecular Analysis 2 (I-SPY2) trial in which patients with various phenotypes of breast cancer are assigned different potential treatment regimens is a good example of this. The overall goal of the phase 2 trial is to graduate promising treatment regimens into phase 3 trials with an estimation of the expected probability of success.¹² Agents are dropped and replaced by other agents within the ongoing I-SPY2 mechanism. At the present time, specific examples of successful adaptive clinical trials specifically in SCI are lacking, but there have been growing numbers in other fields, especially oncology and medical devices.^{13, 14} Since it is not possible to conduct competing trials using alternative designs in order to gather evidence in favor of one design or the other, carefully conducted simulations over a wide variety of potential truths are necessary to determine the potential benefits of adding adaptive elements to clinical trials.

Adaptive designs do have potential drawbacks.¹⁵ Some are general and some are specific to the type of adaptation. In general, much more planning and work needs to be done prior to initiating the trial. This involves discussions between statisticians and clinicians regarding the general understanding of the clinical problem and the potential scenarios that may emerge during the trial and areas where the clinicians have uncertainty regarding the right dose or the right patient population. This is one potential reason that adaptive designs have been more broadly adopted within the pharmaceutical and device industries versus government funded research. While all for profit and non-profit sponsors are motivated by financial concerns, the direct benefit to for-profit entities investing in this planning has been established. In government-funded research, planning activities and the time needed for careful discussion between clinicians and statisticians along with the review of the simulations of trials is not always an area where funding has been available. For investigator initiated research, these types of designs may be difficult to practically create unless other grant funding is found or a large research network with some flexible funding is willing to invest in this sort of pre-award work. After a trial has started, the inclusion of multiple interim analyses and adjustments to randomization allocation ratios adds a layer of complexity to the trial which is greater than what is typically encountered. For an unblinded trial, there is the additional risk that researchers enrolling patients may gain insight into the outcomes within a trial. For example, if investigators note that more patients are being allocated into one arm, they may conclude that this is the effective treatment even though the trial continues to enroll. Given the expected accrual rates within centers for an SCI trial, this concern is largely theoretical, while the clinicians may think they know which treatment is “winning”, it is highly unlikely that their observations would be truly informative. Another concern is when response adaptive randomization is being used and the overall population of patients being included in the trial changes over time. This potentially introduces bias; however a statistical interaction between the assigned arm and whatever aspect of the population has changed that is associated with the treatment effect would need to be present for this to actually bias the trial. In practice, such interactions are rare. Finally, with more complex designs it is possible that patients will not necessarily understand the trial or will be concerned given the complexities and refuse consent in the trial.

Spinal cord injury is a serious, acute condition with few treatments to reduce disability. With any emergent condition, the difficulty entailed in acute enrollment has likely inhibited discovery of new treatments. Spinal cord injury has been particularly limited. One potential contributory factor has been that trials in the acute phase have different stakes when considered against trials with more leisurely enrollment windows such as primary prevention windows or oncology trials. These allow more discussion between researcher and volunteer and possibly a greater understanding of the potential benefits (or lack thereof) of participation. Using an adaptive design to improve the outcomes of all of the patients within a 20,000 patient primary prevention trial may be less important when you may be reducing the five-year rate of myocardial infarction from 1.2 to 1%. In such a trial, the most important goal of the trial is to precisely estimate that treatment effect relative to adverse events. On the other hand, for acute serious conditions with no available treatments, one could reasonably sacrifice some precision in the estimation of the treatment effect in order to improve patient outcomes and find new treatments more quickly. Of course, such a design must exclude the possibility that the treatment is ineffective with the same degree of rigor and statistical certainty that is afforded by a more traditional design.

Pathway forward to for therapeutic hypothermia in spinal cord injury

The use of therapeutic hypothermia for patients with acute spinal cord injury has been promising in preclinical models and early human trials.^{16, 17} Anecdote has also established this as a potential treatment in the popular media. As with any promising treatment, it is important to learn if it is truly efficacious.

As with any promising treatment that is not yet been proven in a confirmatory phase trial, substantial residual questions remain regarding the use of therapeutic hypothermia in spinal cord injury. From one perspective, hyperthermia causes harm in nearly all acute neurological injury states and it is certainly plausible that enforced normothermia may confer benefit over more uncontrolled treatment. Preclinical data has shown that therapeutic hypothermia, a broad spectrum neuroprotectant, has substantial promise as an acute treatment for SCI. Studies of therapeutic hypothermia in comatose survivors of cardiac arrest has shown benefit, but there is still uncertainty regarding the optimal duration for therapy and the therapeutic window.^{18, 19} These are areas of uncertainty for the treatment of SCI as well.

Learning and confirming simultaneously

In the ADAPT-IT project, various aspects of the clinical treatment protocol and scientific questions in a trial of therapeutic hypothermia for SCI were carefully considered and incorporated into the design.³ Two major questions remained in the design process for a spinal cord injury trial: how long should patients be cooled and within what time window from the onset of injury should treatment be started? Regarding the latter, it is intuitive that at some point it will be too late to influence outcomes, at least within what would be observable in a feasible clinical trial. This trial was iteratively designed using the ADAPT-IT process, starting with an initial face-to-face meeting where the clinical requirements of the trial were discussed, followed by the general presentation of a design concept, followed by several working group meetings by teleconference demonstrating how the design performed under a variety of assumptions. There was a final meeting of all the investigators to discuss the design along with representatives from the NIH and the FDA. From this process the design, protocol, and grant applications were developed. The final design addresses several questions: how long patients should be treated, is therapeutic hypothermia efficacious and what the treatment window is (4 or 6 hours). The trial will start with a 6 hour treatment window but will transition from a six to a four hour inclusion window if the likelihood of observing a positive treatment effect appears sufficiently low. In this case, the trial will transition to being a learning phase trial only. On the other hand, should an effective cooling duration be identified within the 0–6 hour population as superior to enforced normothermia – it will be confirmed in the final stage of the trial. This type of design has great appeal because it enables us to acknowledge areas of uncertainty and incorporate them into the trial design. In the event that there is definitively no (or a very small) treatment effect and the observed dose (duration) response is quite flat from a duration of 0 to 72 hours then the door will be effectively closed on this treatment and efforts in this field can focus on other agents or combinations of agents in future trials. If there is promise only within the 4-hour group – the trial will learn a great deal and better inform us at the end regarding whether a follow up trial would have decent prospects for success.

Conclusions

Substantial challenges exist to the discovery of important new treatments for patients with acute spinal cord injury. Some of these are scientific and some of these are structural. Emerging tools from the field of flexible innovative adaptive designs have the potential to accelerate discovery and maximize the learning from patients who are afflicted with SCI. The major problem inhibiting the uptake of adaptive designs into confirmatory phase, academic trials, is the extensive up-front work that needs to occur. This includes basic scientists, clinical trialists, statisticians, data managers, regulators and others interacting to ensure that a design meets all scientific goals, yet appropriately takes into account real uncertainty that remains when embarking on a Phase 3 trial. This iterative process has been embraced by industry, and has been pilot tested by the NIH within the ADAPT-IT project. It is still unclear how these more flexible designs could be incorporated into future clinical trials. Since all this work occurs before a grant proposal can even be submitted (with no guarantee of funding), there is no current mechanism to fund this pre-planning which could potentially increase the likelihood of trial success. As these trials need to be simulated under a wide variety of assumptions and further adjusted based on feedback from the clinicians and scientists, this process has been infrequently used due to the constrained availability of biostatisticians with expertise in clinical trial simulation.

The high burden of disability for patients with spinal cord injury motivates us to ensure that we apply the best and most appropriate techniques in clinical trial design. Namely we must accept that there are some uncertainties that remain when designing both exploratory and confirmatory trials. While every single uncertainty may not be able to be addressed, the process by which one thinks carefully about which uncertainties are most important and most likely to potentially compromise the ability to detect an effective treatment can lead to trial designs that are more likely to find the right treatment for the right population of patients. Such designs have to be balanced against the potential difficulty in their explanation and the added work that needs to be done prior to the funding of this sort of work within government-sponsored research. The additional logistics required to estimate the models and use them to inform the decisions of the trials is nontrivial and has to be considered beneficial relative to its cost in order to improve scientific and medical treatment of the patients within the trial. Given the current low discovery rate of the existing process for acute neurological injury and spinal cord injury in particular, we believe that these approaches to the trial design process and discovery in general will help accelerate discovery of treatments for spinal cord injury.

Abbreviations

SCI

spinal cord injury

NIH

National Institutes of Health

FDA

Food and Drug Administration

ADAPT-IT

Adaptive Designs Accelerating Promising Trials Into Treatments

NETT

Neurological Emergencies Treatment Trials

NINDS

National Institutes of Neurological Disorders and Stroke

I-SPY2

Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging And moLecular Analysis 2