Time to event clinical trial designs: existing evidence and remaining concerns

Abstract

Well-designed placebo-controlled clinical trials are critical to the development of novel treatments for epilepsy, but their design has not changed for decades. Patients, clinicians, regulators, and innovators all have concerns that recruiting for trials is challenging, in part, due to the static design of maintaining participants for long periods on add-on placebo when there are an increasing number of options for therapy. A traditional trial maintains participants on blinded treatment for a static period (e.g., 12 weeks of maintenance), during which participants on placebo have an elevated risk of sudden unexpected death in epilepsy compared to patients on an active treatment. Time-to-event trials observe participants on blinded treatment until a key event occurs (e.g., post-randomization seizure count matches pre-randomization monthly seizure count). In this manuscript, we review the evidence for these designs based on re-analysis of prior trials, one published trial that used a time-to-2^nd seizure design, and experience from an ongoing blinded trial. We also discuss remaining concerns regarding time-to-event trials. We conclude that, despite potential limitations, time-to-event trials are a potential promising mechanism to make trials more patient-friendly and reduce placebo exposure, which are urgent needs to improve safety and increase recruitment to trials.

1. Introduction

The dual goals of clinical trials for epilepsy are to establish efficacy while maintaining safety. In traditionally designed clinical trials, participants keep a seizure diary during a pre-randomization baseline period of 4- to 8-weeks, followed by a blinded post-randomization phase that can include titration plus 12 or more weeks of maintenance (Figure 1).[1] To be eligible for these trials, participants frequently are required to have a high baseline seizure frequency (at least 4 seizure per month).[2, 3] For placebo-controlled adjunctive efficacy trials of antiseizure medications (ASMs), participants’ baseline ASMs are unchanged during the entire study so that the only change in seizure treatment is the introduction of active drug or placebo at randomization. In the traditional design, patients remain on the same dose of the same ASMs during 1 month before entry, 1–2 months of baseline, and 3 or more months in the treatment phase, for a total of at least 5 months. In clinical practice, medicine changes would be made over this interval. Maintenance of ineffective treatment can cause harm. In a review of mortality for participants enrolled in epilepsy trials, participants in the placebo arm had a 5.8 times greater risk of sudden unexpected death in epilepsy (SUDEP) compared to participants taking effective treatments.[4] In infants, a higher seizure burden has been required until recently to limit trial duration.[5, 6] Investigators and families were reluctant to expose participants with this high seizure burden to placebo when the studies’ ASM usually was available off label. Therefore, there is a need to reduce participant exposure to placebo and other ineffective treatments.

Figure 1:

Typical timeline of a double-blind placebo-controlled parallel trial for a treatment of epilepsy. Abbreviations: antiseizure medications (ASMs), weeks (wks).

Reducing this risk of harm could improve recruitment. The time needed to recruit participants has been increasing over time, in part, because each trial site typically recruits 1 to 3 participants instead of the 7 to 15 recruited into trials years ago.[7–10] Limited participant numbers from each site make it difficult to statistically evaluate for site-level sources of variability. Also, the rate of placebo-response substantially varies by country, which can lead to substantially different effect sizes.[11] Therefore, there is a critical need to improve participant recruitment.

Time-to-event (TTE) design clinical trials require post-randomization observation only until a target “event” has occurred, thereby potentially reducing harm. Withdrawal to monotherapy trials were performed as TTE trials, but these studies carried a high risk of harm since participants were withdrawn to a “pseudoplacebo,” defined as an inadequate ASM dose. The “event” consisted of “escape” criteria that resulted in 85% of participants on pseudoplacebo reaching an endpoint before 112 days: (1) post-randomization doubling of monthly seizure frequency; (2) seizure clusters defined by doubling of the highest consecutive 2-day seizure rate; (3) a new, more severe, seizure type; or (4) clinically significant prolongation of generalized tonic-clonic seizures.[12, 13] More recently, proposals for TTE trials have evaluated the time-to-N^th seizure counts including the 1^st seizure, 2^nd seizure, and the individually determined average monthly pre-randomization seizure count (PSC).[14] After reaching this “event” of interest, participants can switch to the open-label extension or withdraw from the study. In traditionally designed trials, more than 80% of participants opted for open-label extension therefore the risk of withdrawing effective treatment is low.[15–28] Treatment efficacy can be measured using Cox-proportional hazards models, proportion of failures, and the traditional primary efficacy endpoints of percent reduction in seizure frequency and 50% responder rate.[29] These latter primary efficacy endpoints are calculated to the “event” of interest, similar to the last observation carried forward method for addressing missing data that is used within trials commonly.

In this review, we provide an up-to-date overview of the TTE design by summarizing the existing evidence on the TTE design for ASM evaluation, presenting experience from an ongoing TTE designed trial, and reporting a discussion of remaining concerns from multiple stakeholders who attended the 2022 Research Roundtable for Epilepsy (RRE).

2. Existing Evidence about TTE Trials

The evidence supporting time-to-N^th trials focuses on re-analysis of traditionally designed trials for adjunctive ASMs plus the trial of lacosamide that used the endpoint of time-to-2^nd primary generalized tonic-clonic (PGTC) seizure.[14, 29–31] Most retrospective evidence supports the time-to-PSC design where each participant is observed until their seizure count post-randomization exceeds their individual baseline average monthly seizure count (Figure 2). As compared to the time-to-1^st and 2^nd seizure designs, time-to-PSC encourages at least a month of observation post-randomization, which can reduce sensitivity to seizure clusters and monthly seizure cycles. In the re-analysis of fenfluramine for Dravet syndrome a substantial portion of participants on placebo reached time-to-PSC between 30 to 50 days after randomization (Figure 3).[32] Figure 4 uses idealized simulated seizure diaries designed to match adult patients with medication refractory epilepsy to illustrate the expected time-to-PSC based on the degree of response to treatment. Due to a placebo effect, the median time-to-PSC is longer than 28-days for participants with no change (100% of baseline seizure frequency).

Figure 2:

A hypothetical illustration of how the estimate of seizure frequency (seizure freq) stabilizes over the course of the typical 12-week randomized controlled trial (RCT) for a simulated participant randomized to placebo who had 6 seizures (Sz) in a 4-week baseline. The simulation was using the methods of Goldenholz and colleagues (Ann Clin Trans Neurol 2017). The time-to-event endpoint of time-to-prerandomization monthly seizure count (T-PSC) is illustrated at 48 days, with an interim seizure frequency reduction of 44%, compared to a 56% reduction when treatment was continued through 12-weeks. Comparison to other historical escape criteria is illustrated by highlighting seizure clusters.

Figure 3:

Time to pre-randomization monthly seizure count for participants with Dravet Syndrome enrolled in the fenfluramine (FFA) trial. Reproduced from Sullivan et al. Epilepsia 2021.

Figure 4:

The expected time to pre-randomization monthly seizure count (PSC) for participants with simulated seizure frequency (SF) response to an antiseizure medication. The 100% curve reflects no change; 200% reflects doubling of SF; and 50% reflects halving of SF. Simulations based on Goldenholz et al. Ann Clin Trans Neurol 2017.

When this time-to-PSC design was used, sufficient seizures were observed so that the efficacy conclusions were unchanged, despite shortening observation duration. When comparing percent reduction in seizure frequency and 50% responder rate at time-to-PSC to the full-length trial, there was more than 90% and 80% correspondence on an individual participant level, respectively.[29] Using simulated data, the time-to-PSC outcome had similar power and total trial cost to the percent reduction in seizure frequency, and both approaches were superior to 50% responder rate.[33]

The only published prospective time-to-N^th seizure trial evaluated the efficacy of lacosamide for PGTC seizures.[31] After the 6-week titration period, participants were observed until the 2^nd PGTC seizure. Around 25% of participants on lacosamide and 50% of participants on placebo had experienced the 2^nd PGTC seizure before 12 weeks. By 24 weeks, 45% of participants on lacosamide and 67% of participants on placebo had had experienced the 2^nd PGTC seizure.

There are unique challenges and proposals for TTE designs in pediatrics. The seizure frequency in infantile epilepsy can be higher than adult epilepsy. Auvin and colleagues evaluated if “placebo”-observation could be reduced further by applying TTE both in the baseline and post-randomization phase.[34] In their re-analysis of trials for levetiracetam and lacosamide in children, they proposed three baseline duration categories: (1) at least daily seizures observed for 7 days, (2) at least weekly seizures for 14 days, and (3) at least 3 seizures per month for 28 days.[30] After enrollment, participants were evaluated at 7, 14, and 28 days. If the participant had sufficient baseline seizures at day 7, then blinded randomization would start. If baseline seizure frequency was lower, then the participant would continue baseline until either qualifying for baseline category (2), or baseline category (3), or having insufficient baseline seizures for enrollment. After randomization, participants who qualified for these categories were followed until the pre-randomization seizure count matched the post-randomization seizure count. When analyzed using a stratified Cox-proportional hazards model, the efficacy of lacosamide and levetiracetam were re-demonstrated, and the effect sizes were similar to the original trials’ analysis.

3. Experience from an ongoing clinical study

We provide insights from an ongoing TTE study for adults with focal-onset seizures. In a proof-of-concept study, Dr. van der Geyten and colleagues required at least 28 days of post-randomization observation. If baseline seizure count was exceeded during the 1^st 28 days, or any subsequent moving 28-day window, the participant would either complete the trial or enter the open label extension. If this event was not reached, then participants would take blinded treatment for up to 12 weeks.

In their preliminary analysis, they observed substantial variability in baseline seizure frequency, seizure cycling, and unanticipated missing data (Figure 5). They observed cycles of seizures where some participants had 2 to 3 weeks of seizure freedom, followed by a few days with a cluster of seizures. The relationship between each individual participant’s seizure cycle and the phases of the trial could substantially impact the conclusions of the trial. In one example participant, the blinded treatment lengthened the interval between seizures but the total number of seizures within each cluster remained the same. In another example participant, the baseline was variable: the first 4 weeks included 15 seizures whereas the subsequent 4 weeks included 4 seizures. After randomization, the patient had 14 seizures in 12 weeks, representing a 69% improvement from 15 seizures during the first 4-week baseline, but a 17% worsening from 4 seizures during the second 4-week baseline period. At this time the study was blinded, so it was unknown whether the participant received placebo or active treatment. It was theoretically possible that the participant had spontaneously transitioned to a lower seizure frequency state between the first and second month of baseline. When in that lower seizure frequency state, the blinded treatment had no substantial impact upon that participant’s seizure frequency. This natural variability in seizure frequency also has been described in re-analysis of other trials [35] and could substantially impact the conclusions of a trial with a short baseline like that proposed by Auvin and colleagues.[34] This natural variability in seizure frequency could be addressed with a longer baseline, but a longer baseline period would incur a similar increase in SUDEP due to more exposure to a placebo equivalent.[4]

Figure 5:

Individual seizure diaries from blinded adult participants in an ongoing time-to-event clinical trial for focal-onset seizures. (A) Periodicity in seizures with brief periods of seizures and semi-regular seizure free intervals. (B) Variability in the seizure frequency in the first versus second baseline period.

In addition, they observed challenges with missing data, where seizure data was unavailable for up to 45 days for some participants from Ukraine who were impacted by geopolitical events. In one participant, the monthly seizure frequency was lower than baseline prior to the missing data, and higher than baseline after the missing data. Therefore, the time to exceed monthly baseline seizure count was hypothesized to occur sometime within the missing data. Additionally, the moving 28-day window of seizure counts was theoretically and practically concerning when the 28-days spanned the period of missing data. If the missing data were excluded, then either the 28-day windows could include seizure counts more than 1 month apart or participants would need another 28-days of available data before being considered to switch to open label extension or complete participation.

4. Remaining points of discussion

Seizures and seizure frequency are dynamic and can be individually unique. Our current statistical models of seizure counts use averages over longer periods to reduce seizure variability from errors in patient-reported seizure counts, clustering, seizure cycling, and natural variability in seizure frequency.[35–38] These complex patterns in seizure diaries may be enhanced in a TTE trial. For example, seizure clusters likely average out across 12-weeks, but an early seizure cluster in a TTE trial may lead to a false determination of non-response. In addition, the “event” could occur during the titration phase. For this reason, it is advisable to “start the clock” during maintenance. Even in the early maintenance, delayed onset of efficacy could lead to false determinations of non-response. These limitations in a TTE design could be addressed by a composite endpoint requiring a minimum duration of observation followed by a TTE endpoint. However, a long minimum duration of observation would perpetuate the very problems that TTE seeks to address.

In an ideal trial, presumptive participants would have a long-term high-quality seizure diary before enrollment to address this natural variability and avoid the need for a prospective baseline. While seizure diary apps may provide quality diaries, most trials require at least 4 weeks of prospective baseline including at least 1 to 4 seizure(s) to supplement 4 to 12 weeks of historical baseline.[21, 22, 31, 39, 40] This may be due to the poor association between seizure diaries and objectively measured seizures using devices.

The inherent limitation in the reliability of participant-provided seizure diaries applies both to current trial methods and TTE trials. Re-analysis of prior RCTs with a time to pre-randomization seizure count design demonstrated that the TTE design reproduced the group-level efficacy conclusions, effect size, and individual-level efficacy in more than 80% of participants, therefore this TTE design measures the outcome of improvement of seizures similarly to the traditional design. It is important that both trial designs require that seizures be countable. This design would be inappropriate for trials of, for example, generalized absence epilepsies or epileptic spasms in the context of infantile spasm syndrome where video-EEG monitoring may be required to accurately count seizures.[41, 42] An alternative TTE endpoint could be lack of seizure freedom as measured by clinical observation plus video-EEG monitoring at scheduled intervals.

In addition to shorter observation, a TTE design trial observes fewer seizures which, intuitively, could reduce statistical power. However, simulations and re-analyses of prior trials have demonstrated similar or improved statistical power of a TTE design to traditional analyses.[33] In pediatric epilepsies like the application of time-to-PSC used to evaluate the efficacy of fenfluramine for Dravet syndrome, baseline seizure frequencies were high enough that numerous seizures were observed prior to the TTE endpoint (median 26/month).[32] In contrast, the perampanel trial for primary generalized tonic-clonic seizures included seizure frequencies as low as 1 per month (median 2.6/month) and the TTE reanalysis redemonstrated efficacy.[29] These two applications reflect epilepsies that are particularly suitable for a TTE design because of the high risk of SUDEP, clear identifiable clinical correlate of myoclonic or generalized tonic-clonic seizures, and sufficient baseline seizure frequency.

In adult epilepsy, it is becoming increasingly challenging to recruit patients with at least weekly seizures, the traditional minimum seizure frequency for trials. Insufficient seizure frequency of less than 3 per month occurred in at least 55% of patients and was the most common reason for ineligibility for clinical trials (Table 1).[43] Therefore, there’s high interest in lowering these minimum seizure frequency requirements, as has been done in four prior trials.[18, 31, 39, 44] Lowering these minimums requires efficacy to be determined with both fewer pre- and post-randomization seizures.[2, 3] TTE designs reduce the number of post-randomization seizures, thereby potentially less statistical power for low seizure frequencies. For example, there are edge effects where a single seizure that occurs either just before or just after the 12-week maintenance period could substantially change a participant’s percent reduction in seizure frequency and 50% response rate (e.g., 3 baseline seizures over 12 weeks compared to 1 post-randomization seizure in 12 weeks [67% reduction, responder] versus 2 post-randomization seizures in 12 weeks [33% reduction, non-responder]). Increased variability means reduced statistical power. Therefore, expanding trials to patients with fewer seizures both with and without a TTE design could necessitate increased participants or longer observation. It is unclear if the financial costs of that may be outweighed by a shorter recruitment phase due to a profound increase in patient eligibility.

Table 1:

Proportion of patients with focal- or generalized-onset medication resistant epilepsy meeting NINDS 2011 workshop inclusion criteria for clinical trials seen at New York University’s epilepsy clinic in a 3-month period in 2013. Patients with symptomatic and combined focal- and generalized-onset epilepsy were excluded.

Number (%)	Focal	Generalized
Total number	144	29
Eligible for clinical trials	19 (13%)	1 (3%)
Reason for exclusion
Seizure frequency < 3/month	84 (58%)	16 (55%)
Age (< 16 years)	18 (13%)	10 (37%)
Recent ASM change (< 4 weeks)	27 (19%)	4 (14%)
Unstable medical or psychiatric condition	11 (8%)	1 (3%)
Inability to maintain seizure diary	10 (7%)	0
More than 3 concurrent ASMs	8 (6%)	1 (3%)

This uncertainty regarding statistical power of TTE trials may have been why the time-to-2^nd PGTC seizure trial for lacosamide used a 24-week maintenance phase, even though the efficacy results were nearly identical at 12- and 24-weeks.[31] If this uncertainty is not resolved, an overpowered trial may result in more participant-years on placebo by either observing participants on placebo for longer or randomizing more participants to placebo. While an increase of 1,000 participant-years on placebo would be expected to contribute to 6 additional deaths from SUDEP,[4] participants who had not reached the TTE endpoint would be expected to have fewer seizures and thereby a lower risk of SUDEP.[45] The potential for overpowered trials due to increased uncertainties would counteract some of the original goals of TTE trials to reduce total participant-years on placebo and ineffective treatment.

For trials of ultra-rare epilepsies where a critical challenge for the trial is the small number of eligible participants, TTE designs as described here may be inappropriate because, due to limited sample size, trials may need to focus on N-of-1 designs, crossover designs, non-seizure outcomes (e.g., cognitive outcomes), and other methods to obtain as much information as possible from each participant. For urgent situations such as treatment of acute repetitive seizures or status epilepticus, there may be alternative “events” or escape criteria including time to cessation of motor seizures in convulsive status epilepticus, seizure-free withdrawal of general anesthesia in super refractory status epilepticus, or use of a rescue treatment in patients with recurrent repetitive seizures or status epilepticus.

Clinical trials for epilepsy aim to address more than seizures; they also establish safety as well as secondary outcomes.[46] An issue with the TTE design is the potential for reduced observation in the placebo arm of trials that may lead to reduced placebo-controlled adverse effect information. The majority of dose-dependent adverse effects occur early, but idiosyncratic and longer-term adverse effects may take longer to evaluate.[47–51] By having fewer participants on placebo, it’s harder to estimate the background rate of more rare idiosyncratic adverse effects. While these longer-term adverse effects typically are established in open-label extension and phase 4 studies, a minimum duration of blinded observation could observe some of the medium-term adverse effects without substantially increasing placebo-exposure. Further studies of the timing of adverse effects are needed to evaluate this potential limitation of TTE trials.

A broader theme of modern clinical trials for epilepsy includes the exploration of disease modifying treatments for epilepsy. In these treatments, the primary outcome may focus on cognitive function, functional independence, and quality of life. Improvements in non-seizure outcomes could be observed even when seizure frequency is not improved, as suggested by prior work addressing comorbid depression in patients with epilepsy.[52] Changes in these functional outcomes also are slower than changes in seizure frequency, therefore TTE trials focused on seizures alone may be underpowered to observe these additional benefits of treatment. Ideas to address these dual outcomes included an initial TTE trial to evaluate efficacy for seizure, followed by a longer blinded period where patients would be allowed to change non-trial ASMs to optimize seizure control as the non-seizure efficacy outcomes would be evaluated.

5. Conclusion

TTE trials potentially establish the efficacy of novel treatments for epilepsy, while also reducing exposure to ineffective treatments. This design could improve recruitment by reducing participant risk. There is building evidence from re-analysis of prior trials and emerging evidence from trials that implemented this design that the time-to-PSC endpoint may accomplish this without reducing statistical power. However, due to a reliance on a comparatively small number of seizures, these TTE designs may have reduced power if participants with lower seizure frequencies are included in future trials. TTE trials also primarily focus on seizure counts and have reduced placebo-controlled observation for adverse effects or non-seizure efficacy outcomes (e.g., cognition). Additionally, the TTE design does not address known challenges in the analysis of seizure counts within clinical trials including contributions of clustering, seizure cycles, and delayed onset of treatment efficacy. These uncertainties may lead to the initial trials implementing a TTE design to be overpowered by including more participants and lengthening the duration of observation, which counteracts the goals of the TTE design to reduce exposure to placebo and ineffective treatments. Despite these limitations, TTE designed trials may address the urgent needs to increase participant recruitment and improve safety by reducing placebo exposure.

Highlights.

• Time to Event designs can reduce participant risk of exposure to ineffective treatments and thereby improve recruitment.

• Trial re-analyses suggest Time to Event endpoints can demonstrate the efficacy of treatments in substantially less time.

• Time to Event endpoints may be impacted by seizure cycling, clusters, missing data, inaccurate seizure diaries, and low seizure counts.

• Time to Event designs may limit the duration of placebo-controlled observation for adverse events.