Key design elements of successful acute ischemic stroke treatment trials

Abstract

Purpose

We review key design elements of positive randomized controlled trials (RCTs) in acute ischemic stroke (AIS) treatment and summarize their main characteristics.

Method

We searched Medline, Pubmed and Cochrane databases for positive RCTs in AIS treatment. Trials were included if (1) they had a randomized controlled design, with (at least partial) blinding for endpoints, (2) they tested against placebo (or on top of standard therapy in a superiority design) or against approved therapy; (3) the protocol was registered and/or published before trial termination and unblinding (if required at study commencement); (4) the primary endpoint was positive in the intention to treat analysis; and (5) the study findings led to approval of the investigational product and/or high ranked recommendations. A topical approach was used, therefore the findings were summarized as a narrative review.

Findings

Seventeen positive RCTs met the inclusion criteria. The majority of trials included less than 1000 patients (n = 15), had highly selective inclusion criteria (n = 16), used the modified Rankin score as a primary endpoint (n = 15) and had a frequentist design (n = 16). Trials tended to be national (n = 12), investigator-initiated and performed with public funding (n = 11).

Discussion

Smaller but selective trials are useful to identify efficacy in a particular subgroup of stroke patients. It may also be of advantage to limit the number of participating countries and centers to avoid heterogeneity in stroke management and bureaucratic burden.

Conclusion

The key characteristics of positive RCTs in AIS treatment described here may assist in the design of further trials investigating a single intervention with a potentially high effect size.

Introduction

RCTs are needed to prove efficacy of a treatment in any disease, including stroke. RCTs can be big or small and in general trials with larger sample sizes are considered to yield higher success rates. In this article we review and discuss key design elements of positive acute ischemic stroke (AIS) trials and summarize their main characteristics.

Methods

We searched Medline, Pubmed and Cochrane databases using the following search criteria: acute stroke treatment, neuroprotection, thrombolysis, mechanical thrombectomy, hemicraniectomy, and randomized. All papers published in English between 1980 and 2021 were evaluated for inclusion. The search was performed independently by WH, IK and AS.

Trials were considered ‘positive’ if they met the following criteria: (1) a randomized controlled design, with (at least partial) blinding for endpoints, (2) tested against placebo (or on top of standard therapy in a superiority design) or against approved therapy (such as rtPA); (3) protocol was registered and/or published before trial termination and unblinding (if required at study commencement); (4) primary endpoint was positive in the intention to treat analysis; and (5) the study findings led to approval of the investigational product and/or high ranked recommendations in internationally accepted guidelines.

Meta-analyses and pooled analyses of positive RCTs were not included. The few positive trials with a noninferiority design will be reported separately. Early secondary prevention trials were not considered.

As we used a topical approach, focusing on a general discussion of the subject instead of starting from a specific hypothesis, the findings were summarized in a narrative review.

A short history of acute stroke treatment trials

Neuroprotection

Neuroprotection aims to reduce the harmful effects of ischemia at the neuronal, glial, and blood–brain barrier level, and so limiting or even preventing tissue damage [1, 2]. Hundreds of candidate drugs have been tested in stroke models in rats, mice, canines and primates, many of them yielding smaller infarcts in treated animals. However, the positive effects were not confirmed in subsequent multicenter phase 2 and 3 clinical trials [3]. Reasons for the translation failure include: the use of inappropriate animal models (young, male animals), experiments which do not mimic clinical practice (e.g., treatment prior to vessel occlusion), use of different dosages, unexpected adverse events, overestimation of effect sizes leading to underpowered trials and wide eligibility criteria with inclusion of non-informative patients [4].

Only one RCT in neuroprotection showed positive results (SAINT-1 testing free-radical-trapping agent NXY-059), but the results were not confirmed in the SAINT-2 trial [5, 6]. The ESCAPE-NA1 (nerinetide in patients with a small ischemic core undergoing endovascular therapy) and URICO-ICTUS (combined uric acid and rtPA versus rtPA trials were neutral for their primary outcome but found positive signals in secondary endpoints or subgroups; however, a positive signal from a subgroup or a secondary outcome has never been confirmed in a subsequent stroke RCT [7, 8].

The Stroke Therapy Academic Industry Roundtable (STAIR) consortium has recommended several methodological improvements in further studies, including using drugs with multiple mechanisms, combining new interventions with thrombolytics, and using advanced imaging for patient-selection [9, 10]. Nevertheless, all trials following these approaches failed to find benefit, indicating that more rigorous preclinical designs are required before conducting phase 2 or 3 RCTs. Steps to improve the quality of experimental studies have been proposed by the Stroke Preclinical Assessment Network (SPAN) and several new compounds including fasudil (rho-kinase inhibitor), fingolimod (sphingosine-1-phosphate receptor inhibitor), tocilizumab (IL-6 antagonist), uric acid and veliparib (PARP-1 inhibitor) [11]. These should be tested in rigorous multicenter preclinical randomized blinded studies (as in clinical trials) involving more than one species and using clinically relevant outcomes and appropriate time windows [12].

Antiplatelet therapy

Antiplatelet therapy might modulate the thrombotic event causing AIS, by reducing re-thrombosis and peripheral embolization. The Chinese Acute Stroke Trial (CAST) was positive, reporting that aspirin reduced all-cause death within four weeks (aspirin 3.3% vs control 3.9%, p = 0.044) [13]. The small effect size required that more than 20,000 patients would be needed. These results were supported by the just-neutral International Stroke Trial (IST), another aspirin mega-trial [14].

Pharmacological recanalization

From 1994, several randomized, placebo-controlled, double-blind studies were conducted to determine the safety and efficacy of recombinant tissue plasminogen activator (rtPA) and streptokinase [15–24]. The first positive study, NINDS, was published in 1995 and showed that intravenous rtPA given between 0 to 3 h from stroke onset led to a 30% relative (11–15% absolute) increase in the number of patients with minimal or no disability at 90 days, compared to placebo [24]. Other RCTs published during the 1990s failed to reach their primary endpoint or showed an increased risk of intracranial hemorrhage. Methodological differences between the trials included the use of different thrombolytic agents and doses, variable treatment windows (e.g., up to 6 h) and concomitant use of antithrombotics [15, 16, 20–22].

A pooled analysis of these trials indicated efficacy up to 4.5 h without an increased risk of hemorrhage [18]. This prompted the design of the ECASS-3 trial which evaluated efficacy and safety of rtPA between 3 to 4.5 h and found favorable outcomes in patients treated with alteplase compared with placebo [19].

To further extend the treatment window, advanced magnetic resonance imaging was used in DIAS-1/2 and DEDAS trials of another thrombolytic agent, desmoteplase; the trials were neutral due to the presence of very small infarct cores [25–27]. In 2018, the efficacy and safety of alteplase in wake-up stroke patients was demonstrated in the WAKE-UP study [28]. This led to the early termination of the EXTEND and the ECASS-4 trials which had selected patients using CT or MRI perfusion 4.5–9 h after onset or on awakening from stroke [29, 30].

Mechanical recanalization

In addition to intravenous pharmacological reperfusion, intra-arterial revascularization has been explored, an approach originating in the positive PROACT-2 trial [31]. In 2004, the first Mechanical Embolus Removal in Cerebral Ischemia (MERCI) device for mechanical thrombectomy became commercially available, achieving around 50% recanalization rates in single arm open reports [32]. In 2009, the Penumbra system became available, combining aspiration-debulking with mechanical retrieval, which resulted in higher recanalisation rates (82% TIMI 2–3), but only a modest effect on clinical outcome (mRS 0–2 28% at 90 days) in another single arm study [33]. With self-expanding retrievable stents (Solitaire, Trevo) better radiographic and clinical outcomes were reported as compared to MERCI Retrievers [34, 35].

Subsequently, three trials (IMS-3, MR-RESCUE, SYNTHESIS) compared endovascular therapy with standard medical therapy and failed to show a significant difference in functional outcome (mRS 0–2) between the treatment groups [36–38]. However, the trials used first-generation devices and had slow recruitment, delayed times to reperfusion, and in one study, failed to demonstrate large-vessel occlusion before enrollment [39].

The MR-CLEAN trial finally confirmed the utility of mechanical thrombectomy versus standard care [40]. Several ongoing endovascular RCTs then stopped, showing positive results in the interim analyses. These trials used modern devices, achieved higher reperfusion rates, shorter onset-to-treatment time and used rtPA co-treatment in 75% of all cases [41–45]. Recently, the RESILIENT trial confirmed the feasibility and benefit of thrombectomy in a middle-income country [46].

The DAWN and DEFUSE-3 trials then confirmed the efficacy of thrombectomy in longer time windows (DAWN < 24 h, DEFUSE-3 < 16 h) using penumbra-infarct core imaging [47, 48].

Findings

Seventeen stroke studies fulfilled the inclusion criteria for this review (Table 1): one trial in the field of neuroprotection (SAINT-1, although not confirmed in SAINT-2), one study of antiplatelet therapy (CAST); one hemicraniectomy trial (DESTINY-2); four trials of alteplase (NINDS, ECASS-3, WAKE-UP, EXTEND); one on intra-arterial thrombolysis (PROACT-2); and nine of mechanical thrombectomy (MR-CLEAN, SWIFT-PRIME, ESCAPE, EXTEND-IA, REVASCAT, THRACE, DEFUSE 3, DAWN, RESILIENT) [5, 13, 19, 24, 28, 29, 31, 40–49].

Table 1.

Positive trials summary

Study, country, publication year	n	Endpointsa	Effect measureb	Primary sponsor	Termination	Inclusion criteria	Intervention	Control
NINDS [24] USA, 1995	624	Global testc,d	1.7 (1.2–2.6)	Public	Per protocol	Selective (time)	IVT	Placebo
CAST  [13] China, 1997	21,106	28-day death Death or dependence at discharge	p = 0.04e p = 0.08e	Public	Per protocol	Broad	Aspirin	Placebo
PROACT-2 [31] USA, 1999	180	mRS 0–2	p = 0.04f	Commercial	Per protocol	Selective (time, LVO)	IA prourokinase + heparin	Heparin
SAINT-1 [5] Global, 2006	1722	mRS shift	1.20g (1.01–1.42)	Commercial	Per protocol	Selective (time), Broad (clinical)	Neuroprotection (NXY-059)	Placebo
ECASS-3 [19] European, 2008	821	mRS 0–1	1.42 (1.02–1.98)	Commercial	Per protocol	Selective (time)	IVT	Placebo
DESTINY-2 [49] Germany, 2014	112	mRS 0–4 at 6 months	2.91 (1.06–7.49)	Public	Per protocol	Selective (rare subgroup, imaging)	Hemicraniectomy	Conservative treatment
MR-CLEAN [40] Netherlands, 2015	500	mRS shift	1.67 (1.21–2.30)	Public	Per protocol	Selective (time, LVO)	MT + Standard care (90.6% IVT)	Standard care (87.1% IVT)
SWIFT-PRIME [45] Global, 2015	196	mRS shift mRS 0–2 (secondary)	p < 0.001h RR 1.70 (1.23–2.33)	Commercial	Premature	Selective (time, LVO)	MT + IVT	IVT alone
ESCAPE [43] Canada, 2015	316	mRS shift	3.1 (2.0–4.7)	Commercial	Premature	Selective (time, LVO)	MT + Standard care (72.7% IVT)	Standard care (78.7% IVT)
EXTEND-IA [42] Australia, 2015	70	Reperfusioni Early improvementj	4.7 (2.5–9.0) 6.0 (2.0–18.0)	Public	Premature	Selective (time, LVO, imaging)	MT + IVT	IVT alone
REVASCAT [44] Spain, 2015	206	mRS shift	1.7 (1.05–2.8)	Public	Premature	Selective (time, LVO)	MT + Standard care (68.0% IVT)	Standard care (77.7% IVT)
THRACE [41] France, 2016	414	mRS 0–2	1.55 (1.05–2.30)	Public	Premature	Selective (time, LVO)	MT + IVT	IVT alone
DEFUSE-3 [47] USA, 2018	182	mRS shift	Unadjusted OR 2.77 (1.63–4.70)	Public	Premature	Selective (time, LVO, imaging)	MT + Standard care	Standard care
DAWN [48] USA, 2018	206	Utility-weighted mRS % of mRS 0–2	ADk 2.0 (1.1–3.0) ADk 33 (21–44)	Commercial	Premature	Selective (time, LVO, imaging)	MT + Standard care (5.0% IVT)	Standard care (13.0% IVT)
WAKE-UP [28] European, 2018	503	mRS 0–1	1.61 (1.09–2.36)	Public	Premature	Selective (imaging)	IVT	Placebo
EXTEND [29] Australia, 2019	225	mRS 0–1	1.44 (1.01–2.06)	Public	Premature	Selective (time, imaging)	IVT	Placebo
RESILIENT [46] Brazil, 2020	221	mRS shift	2.28 (1.41–3.69)	Public	Premature	Selective (time, LVO)	MT + Standard care (68.5% IVT)	Standard care (71.8% IVT)

AD: adjusted difference; IA: intra-arterial; IVT: intravenous thrombolysis (alteplase); LVO: large vessel occlusion; mRS: modified Rankin Scale; MT: mechanical thrombectomy; OR: odds ratio; RR: risk ratio

^aPrimary or co-primary endpoints (at 90 days)

^bValues presented are adjusted odds ratios with 95% confidence intervals

^cPart 1 was negative, part 2 and combined results were positive

^dGlobal test statistic for four primary outcome measures

^eTwo-sided p-value

^f15% absolute increase in favorable outcome with intra-arterial prourokinase

^gEffect was not reproduced in SAINT-2 (n = 3306, OR 0.94, 95% CI 0.83–1.06)

^hCochran-Mantel–Haenszel test

ⁱDefined as the percentage reduction in the perfusion-lesion volume between initial imaging and 24-h imaging

^jDefined as a reduction of 8 points or more on the National Institutes of Health Stroke Scale or a score of 0 or 1 at 3 days

^kBayesian trial design. For every co-primary outcome, adjusted differences and 95% credible intervals are presented

Study size

Only two trials (SAINT-1 and CAST) included over 1000 patients [5, 13] Four trials included 500–1000 patients (NINDS, ECASS-3, MR-CLEAN, WAKE-UP), two trials included 250–499 patients (THRACE, ESCAPE) and nine trials included fewer than 250 patients (DESTINY-2, SWIFT-PRIME, EXTEND-IA, EXTEND, REVASCAT, DEFUSE-3, DAWN, RESILIENT, PROACT-2) [19, 24, 28, 29, 31, 40–49].

The majority of studies were halted prematurely, mainly due to loss of equipoise or overwhelming efficacy at interim analysis [28, 29, 41–48].

Inclusion criteria: selective versus broad patient selection

While trials with broad inclusion criteria tend to be large and have long time windows, other trials have been defined based on: moderate to severe stroke symptoms, short onset-to-treatment times, localization of vessel occlusion, and/or advanced imaging selection. Except for CAST, all positive trials were highly selective [13].

Study endpoints

Only two trials (EXTEND-IA, CAST) did not include the mRS score in its primary endpoint [13, 42]. The mRS was analyzed as a dichotomy (ECASS-3, DESTINY-2, THRACE, WAKE-UP, EXTEND, PROACT-2) or using its full distribution (SAINT-1, MR-CLEAN, SWIFT-PRIME, ESCAPE, REVASCAT, DEFUSE-3, RESILIENT) [5, 19, 28, 29, 31, 40, 41, 43–47, 49]. DAWN used the utility weighted mRS [48]. NINDS used a global endpoint encompassing the mRS, Barthel index, Glasgow outcome scale and NIHSS [24]. EXTEND-IA used recanalization and early NIHSS improvement as co-primary endpoints [42].

Effect size

All but one trial used a frequentist design; DAWN used a Bayesian design [48]. With the exception of CAST, PROACT-2, SWIFT-PRIME and DAWN, most trials reported the odds ratio as the primary effect measure [13, 31, 45, 48]. The odds ratio was reported unadjusted in DEFUSE-3 and adjusted for covariates in 12 other trials [5, 19, 24, 28, 29, 40–44, 46, 47, 49]. Reported odds ratios ranged from 1.2 for modified Rankin scale shift in SAINT-1 [5] to 6.0 for early improvement in EXTEND-IA [42].

Study origin and funding

Only three of the trials (SWIFT-PRIME, ESCAPE, SAINT-1) involved study sites on different continents [5, 43, 45]. Two trials recruited in several European countries (ECASS-3, WAKE-UP) [19, 28]. All other studies were national studies [13, 24, 29, 31, 40–42, 44, 46–49].

Two thirds of the trials were investigator initiated and publicly sponsored [13, 19, 28, 29, 40–42, 44, 46–49].

Discussion

To the best of our ability, over the past 25 years, we were only able to identify 17 positive acute stroke trials With 888 interventional acute stroke trials registered in this period, the success rate is extremely low [50]; indeed there are more negative (not neutral) trials and interventions [3]. Successful trials share similarities in their design, most were investigator-initiated, publicly-funded, recruited nationally or regionally, and selected patients based on strict time, stroke severity, or imaging criteria. Half of these positive RCTs were ‘small’, with less than 250 patients. The first ever successful acute stroke intervention trial (NINDS) was a selective and small trial [24]. Between 1995 and 2014 only 5 positive trials were published [5, 19, 24, 31]. After the publication of MR-CLEAN and other thrombectomy trials, the number of positive RCTs increased, with 11 published between 2015 and 2021 [28, 29, 40–48].

In the design of clinical trials, there are two main scientific approaches. One advocates a pragmatic design with large sample size, many trial centers, broad patient eligibility criteria with a wide range of stroke severity and partial site monitoring. The other approach advocates for rigid selection criteria, strict data monitoring and standardization of interventions. The rationale for this latter approach is to include only patients likely to benefit from the intervention under investigation. Each approach may be right or wrong for a given scenario and careful consideration should be given to the approach to be used, although the track record of positive trials in acute stroke strongly speaks for the selective approach.

Based on the key characteristics of positive RCTs, several recommendations can be made for the design of acute stroke treatment trials. The effect size should reflect the expected mode of action and whether the intervention is expected to address multiple pathophysiological mechanisms. For example, malignant middle cerebral artery syndrome has a very poor outcome and hemicraniectomy is expected to dramatically reduce the risk of coning and death. Trials of hemicraniectomy are expected to be highly selective and small. In contrast, vessel occlusion by thrombus underlies many ischemic strokes, but has a complex pathophysiology; antithrombotic drugs are only likely to have a mild effect and so trials of aspirin (CAST, IST) had to be very large [13, 14].

Researchers should consider other clinical trials that address a similar research question, as publication of one positive trial can lead to loss of equipoise and premature termination of other trials. This is especially the case when results cause a paradigm shift, as was seen for thrombectomy after the publication of the results of MR CLEAN [40].

Phase III trials should be based on pilot studies that demonstrate proof of principle. Although it is tempting to design a trial on the basis of a positive subgroup in a neutral study, this approach has been unsuccessful in stroke trials. Positive subgroups may be falsely positive due to multiple statistical testing. Examples where a positive subgroup in one trial led to a neutral follow-on study include assessments of clomethiazole and piracetam for AIS, surgery for ICH, and glyceryl trinitrate for lowering blood pressure [51–58]. In contrast, adequately powered studies based on subgroup results from analyses of pooled trial data are more likely to be successful [19].

The mRS was the most common single primary outcome measure. The choice for using an ordinal (shift analysis) or simple dichotomized approach depends on the treatment effect of the intervention. Shift analysis is preferred when treatments yield a small, uniform degree of benefit over all ranges of stroke severity (f.ex. neuroprotection). When a substantial benefit over all ranges of stroke severity is expected, but treatment cannot provide a cure, dichotomization at good outcome is the most efficient statistical technique [59].

Trials tended to be investigator-initiated and performed with public funding or unconditional industry support, with investigators maintaining control of trial processes and publications. It may also be of advantage to limit the number of participating countries and centers so as to avoid heterogeneity in stroke management and bureaucratic burden, being mindful not to compromise the generalizability of the trial results.

However, this is not a one-size-fits-all model and these recommendations refer to hyperacute interventions in acute stroke and cannot be applied to prevention studies with much smaller effect sizes for which global trials involving thousands of patients are needed.

Conclusions

In acute ischemic stroke research, smaller but selective trials are useful to identify efficacy in subgroups of stroke patients. The key characteristics of previous positive RCTs described here may assist in the design of further acute stroke treatment trials investigating a single intervention with a potentially high effect size.

Abbreviations

AD

Adjusted difference

AIS

Acute ischemic stroke

CAST

Chinese Aspirin Stroke Trial

IA

Intra-arterial

IST

International Stroke Trial

IVT

Intravenous thrombolysis (alteplase)

LVO

Large vessel occlusion

MAST-I

Multicentre Acute Stroke Trial—Italy

MERCI

Mechanical Embolus Removal in Cerebral Ischemia

mRS

Modified Rankin Scale

MT

Mechanical thrombectomy

OR

Odds ratio

RCTs

Randomized controlled trials

RR

Risk ratio

rtPA

Recombinant tissue plasminogen activator

SPAN

Stroke Preclinical Assessment Network

Author contributions

The literature search was performed independently by WH, IK and AS, supported by MM. The positive trials summary table was created by IK. LY, IK and AS were responsible for the interpretation of the results of the literature search. LY drafted the first version of the manuscript, together with VYV (neuroprotection section) and SC (pharmacological recanalization section) which was substantively revised by AS, MC, VC, WH and PMB. IK managed the reference list. All authors have read and approved the manuscript and take full responsibility for the information, interpretation, and the conduct of the report.

Authors' information

The submitted manuscript originates from a working group of the World Stroke Organization Future Stroke Leaders Program.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Availability of data and materials

Not applicable.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

WH reports non-significant honoraria for his role as member of Steering Committees of Angels (BI) and Cerenovus as well as minor honoraria for his work in several safety and adjudication committees, all not related to this manuscript. WH was involved in several of the positive trials reported here (ECASS 3, DESTINY II, ESCAPE, and other supporting trials), which may be considered a potential conflict of interest. PB reports no conflict of interest related to this manuscript. He has received grants from British Heart Foundation, National Institute of Health Research. Received honoraria from DiaMedica, Moleac and Phagenesis for Advisory Boards. The other authors declares that there is no conflict of interest.