Importance of proper patient selection and endpoint selection in evaluation of new therapies in acute stroke: further analysis of the SENTIS trial

Abstract

Background

The magnitude of treatment effect in acute stroke depends on several factors, including time from symptom onset (TFSO) to treatment and severity of the initial insult.

Objective

To report further evaluation of NeuroFlo therapy, focusing on the effect of time and stroke severity.

Methods

SENTIS was a prospective randomized trial (N=515) comparing standard medical therapy with/without NeuroFlo therapy. For this analysis, we evaluated outcomes in groups of patients based on TFSO and stroke severity: patients randomized <6 h, 6–10 h, and >10 h with mild (NIHSS<8), moderate (8–14), and severe (>14) symptoms at randomization. 90-Day mRS (modified Rankin Scale) scores and stroke-related death rates were compared between treatment groups.

Results

For patients randomized <6 h TFSO (n=128), the OR for mRS 0–2 was 3.11 (CI 1.30 to 7.46, p=0.011) for treated versus non-treated patients. In patients with disease of moderate severity (NIHSS 8–14, n=214), NeuroFlo-treated patients were more likely to have a good outcome (mRS 0–2; OR=1.84, CI 1.02 to 3.33, p=0.043). The stroke-related death rate was better in the treated group with TFSO >10 h and NIHSS >14 (n=42) (OR=7.10, CI 1.13 to 44.55, p=0.036).

Conclusions

The results of our analysis support the importance of careful selection of outcome measures and the impact that rapid treatment and initial stroke severity have on outcome.

Clinical trial registration

URL://http.clinicaltrials.gov. Unique identifier: NCT00119717.

INTRODUCTION

Tissue plasminogen activator (rt-PA) was approved by the US Food and Drug Administration (FDA) for treatment of acute stroke in 1995 after the successful NINDS (National Institute of Neurological Disorders and Stroke) thrombolysis study.¹ In the 18 years since the introduction of rt-PA, despite over 100 clinical trials with new agents, no other treatments have been approved for acute stroke by regulatory authorities in Europe and North America. This is primarily because no drugs have met the efficacy end points. Recent research suggests that testing of treatments expected to have mild-to-moderate improvement in outcome, especially if evaluated over a long period of time, are best evaluated with transitions from ‘fair to good’ instead of transitions from ‘good to excellent’ as would be expected in very early treatment with reperfusion strategies.² The dismal success record in acute stroke management may in part be related to the way in which treatments are evaluated.

The time from onset to treatment is an important determinant of successful outcome, as demonstrated by the collective experience from thrombolysis trials.³ The NINDS thrombolysis study’s positive result on the ‘global endpoint outcome score’ was possible in a 3 h time window.¹ The ECASS III trial, evaluating the same thrombolytic agent, used a modified Rankin Scale (mRS) of 0–1 as evidence for efficacy for a 3–4.5 h time window.⁴ In contrast, an mRS of 0–2 was used as a measure of success in thrombolysis trials for evaluating treatment in the 3–6 h time window.^3,5 The hemicraniectomy trials are the only stroke trials to show a significant positive treatment effect in a late treatment window (24 h). These trials used an mRS of 0–4 as a measure of efficacy.⁶ There is also evidence from several acute stroke clinical trials that patients with moderate neurological deficits (NIHSS (National Institutes of Health Stroke Scale) scores of about 7–8 up to 15–16 at onset of symptoms) tend to respond best to investigational treatments.^7–10 The plausible explanation for this may be that patients with mild strokes tend to recover even without treatment while patients with severe strokes have extensive injury, making recovery unlikely despite any intervention. Patients with moderately severe strokes are most informative about the effects of a new agent and may thus be ideal candidates for investigational treatments. Therefore, the selection of various trial end points chosen for different treatments reflects the recognition of different expectations of improved outcome based on the severity of the stroke, time to treatment, and the mechanism of the treatment.

The SENTIS trial was a prospective, randomized, multicenter trial to determine the safety and efficacy of NeuroFlo therapy (CoAxia, Inc, Minneapolis, Minnesota, USA) in improving neurological outcome after acute ischemic stroke. The primary results have been recently published, with the trial raising no safety concerns but not achieving its prespecified primary efficacy end point of a return-to-normal.¹¹ In retrospect, the return-to-normal primary end point—the global outcome test (NIHSS, mRS, Glasgow Outcome Scale, and Barthel Index)—might not have been the best measure of efficacy in a trial such as SENTIS where patients were enrolled up to 14 h after symptom onset.

In this report, we present further exploratory analyses of the SENTIS trial data and examine the effects of the time to treatment and stroke severity on long-term outcomes. The main purpose for this further analysis of the SENTIS trial data is to focus attention on the importance of proper endpoint and patient selection when new treatments are being evaluated in acute stroke.

METHODS

The SENTIS trial methods and primary results have been previously published.¹¹ The trial was conducted in accordance with good clinical practices. Institutional review boards/ethics committees approved the study at all sites. Using the SENTIS dataset, we explored the premise that patients receiving earlier treatment (eg, time from symptom onset (TFSO) to enrollment of less than 6 h) and with strokes of moderate severity (NIHSS 8–14) will achieve better outcomes with cerebral blood flow augmentation. SENTIS patients were considered evaluable for analyses if they had 90-day data (or 30-day data carried forward). If neither 30-day nor 90-day data were available, the patient was considered non-evaluable.

First, we compared patient groups based on TFSO to randomization: TFSO <6 h, 6–10 h, and >10 h. We used time to randomization instead of time to treatment to retain a consistent measurement between the patients who received NeuroFlo treatment and those that did not. The average time to randomization for all patients was 8.1 h and the average time between randomization and treatment in the treated group was 1.5 h. We evaluated good outcome (mRS 0–2), moderate outcome (mRS 0–4), and survival rates (freedom from stroke-related mortality) for the various treatment windows.

Second, we compared patient groups stratified by initial stroke severity scores: NIHSS<8, NIHSS 8–14, and NIHSS >14. There is no standard definition of moderate stroke severity and stroke trials have used different definitions based on the selection criteria for each trial. Our stratification, NIHSS <8, 8–14, and >14, was selected based on the overall inclusion range of NIHSS scores in the SENTIS trial (range 5–18, mean 10.9) and is in general agreement with the analyses of previous trials.^7–10

Finally, we evaluated the effect of time combined with stroke severity on all three outcome measures.

Within the subgroups described above, comparisons between treated and non-treated patients were made using logistic regression models. Models were adjusted for baseline NIHSS and age. The OR, 95% CI, and p values from these models were obtained. All analyses were retrospective. Nominal p values were reported, and no adjustments for multiple comparisons were made as this was an exploratory, post hoc analysis. All statistical analyses were conducted in SAS V.9.2 or above (SAS Institute, Cary, North Carolina, USA).

RESULTS

Of the 515 patients enrolled in SENTIS, 28 were excluded from analysis owing to predefined criteria and 12 were lost to follow-up; 475 were evaluable for long-term outcomes. This cohort comprises the modified ‘as treated’ group for this analysis (n=475). The study arms were balanced for demographics, stroke presentation, and medical history; the details have been previously published.¹¹

Effect of time on outcomes

Among the 475 evaluable patients, 128 (64 treated, 64 non-treated) were randomized within 6 h; 217 (94 treated, 123 non-treated) were randomized between 6 and 10 h; and 130 (63 treated, 67 non-treated) were randomized >10 h from symptom onset.

Patients randomized to NeuroFlo treatment within 6 h of symptom onset had better mRS 0–2 outcomes at 90 days than non-treated patients (OR=3.11; CI 1.30 to 7.46; p=0.011) (see table 1). There were no differences in good outcome (mRS 0–2) between treatment groups for patients with TFSO>6 h.

Table 1.

SENTIS trial outcomes: subgroup analysis results

Subgroup	Cohort n (Tx/NTx)	mRS 0–2	mRS 0–4	Freedom from stroke-related mortality
Time from symptom onset to randomization
<6 h	n=128 (64/64)	3.11 (1.30 to 7.46) 0.011	0.99 (0.37 to 2.65) 0.983	1.69 (0.50 to 5.71) 0.396
6–10 h	n=217 (94/123)	1.07 (0.56 to 2.04) 0.849	1.99 (0.88 to 4.49) 0.098	2.27 (0.79 to 6.47) 0.126
10–14 h	n=130 (63/67)	0.90 (0.35 to 2.32) 0.832	2.04 (0.61 to 6.79) 0.247	4.97 (1.05 to 23.59) 0.044
Baseline stroke severity (NIHSS)
NIHSS <8	n=141 (56/85)	1.19 (0.50 to 2.82) 0.690	2.21 (0.43 to 11.41) 0.342	–
NIHSS 8–14	n=214 (108/106)	1.84 (1.02 to 3.33) 0.043	1.44 (0.63 to 3.27) 0.386	2.38 (0.82 to 6.87) 0.110
NIHSS >14	n=120 (57/63)	0.63 (0.19 to 2.09) 0.450	1.67 (0.71 to 3.96) 0.240	2.71 (1.09 to 6.72) 0.031
Selected time×stroke severity cross-cuts
Time <6 h and NIHSS 8–14	n=58 (32/26)	5.24 (1.37 to 20.03) 0.015	0.90 (0.18 to 4.58) 0.895	2.01 (0.35 to 11.51) 0.435
Time 6–10 h and NIHSS 8–14	n=107 (52/55)	1.29 (0.57 to 2.96) 0.540	2.30 (0.72 to 7.34) 0.160	3.68 (0.69 to 19.69) 0.128
Time 10–14 h and NIHSS >14	n=42 (22/20)	0.04 (0.00 to 2.44) 0.122	2.37 (0.46 to 12.31) 0.306	7.10 (1.13 to 44.55) 0.036

Results are presented as OR (95% CI) nominal p value. ORs are adjusted by age and baseline stroke severity (NIHSS).

mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; Tx, treated; NTx, non-treated.

In patients randomized between 6 and 10 h, there was a trend toward achievement of mRS 0–4 (OR=1.99, CI 0.88 to 4.49, p=0.098) and freedom from stroke-related mortality (OR=2.27, CI 0.79 to 6.47, p=0.126). In the latest time window of >10 h after symptom onset, NeuroFlo-treated patients were more likely to be free of stroke-related mortality (OR=4.97, CI 1.05 to 23.59, p=0.044).

Effect of initial stroke severity on outcomes

When categorized by baseline stroke severity, 141 patients (56 treated, 85 non-treated) had an NIHSS score <8; 214 (108 treated, 106 non-treated) had an NIHSS score 8–14; and 120 (57 treated, 63 non-treated) had an NIHSS score >14.

For patients with mild strokes (NIHSS <8), there were no differences between treatment groups for any of the evaluated outcomes (see table 1). In the patients with moderately severe stroke (NIHSS 8–14), NeuroFlo-treated patients were more likely than non-treated patients to have a good outcome (mRS 0–2; OR=1.84, CI 1.02 to 3.33, p=0.043). In the patients with severe stroke (NIHSS >14), the NeuroFlo-treated patients were more likely than the non-treated patients to be free from stroke-related mortality (OR=2.71, CI 1.09 to 6.72, p=0.031).

Effect of time and initial stroke severity on outcomes

To evaluate the combined effect of time and initial stroke severity on the outcomes, we further analyzed the data by placing the patients into subgroups crossed by both time and baseline NIHSS.

There were no significant findings of treatment benefit for any time window at the mildest stroke category (NIHSS <8). The 58 patients (32 treated, 26 non-treated) with moderately severe strokes (NIHSS 8–14) and early time to randomization (<6 h) showed treatment benefit for a good outcome of mRS 0–2 (OR=5.24, CI 1.37 to 20.03, p=0.015). Moderate stroke severity and times to randomization of >6 h did not demonstrate treatment benefit for an outcome of mRS 0–2 (see table 1).

Patients with moderate severity strokes and time to randomization of 6–10 h had a trend towards treatment benefit for an outcome of mRS 0–4 (OR=2.30, CI 0.72 to 7.34, p=0.160) and freedom from stroke-related mortality (OR=3.68, CI 0.69 to 19.69, p=0.128). Among the 42 patients (22 treated, 20 non-treated) with severe strokes (NIHSS >14) and the later times to randomization of 10–14 h, freedom from stroke-related mortality (OR=7.10; CI 1.13 to 44.55; p=0.036) was improved in the treated group.

DISCUSSION

SENTIS is the largest randomized trial of a device therapy in acute ischemic stroke to date. The failure of the study to achieve its primary efficacy end point may be because of the very rigorous criteria for the NINDS excellent global outcome test (NIHSS 0–1, mRS 0–1, Barthel Index 95–100, and Glasgow Outcome Scale 5), a score best suited to evaluate the effects of a reperfusion strategy very early after an ischemic stroke.³ In this exploratory analysis, we analyzed subgroups and dichotomized outcome cut-off points more likely to be informative about treatment effects in a later enrolling study.

Other recent clinical trials report findings similar to the results presented here, identifying stronger signals of potential treatment response in the subgroup of patients with moderate, compared with mild or severe, initial deficits. In the NeuroThera Effective and Safety trial-2 (NEST-2), baseline severity was categorized into NIHSS score groups of 7–10, 11–15, and 16–22. Patients with NIHSS scores of 16–22 had a success rate of 8% on the dichotomized mRS of 0–2 vs mRS 3–6 (n=224, treated 7.0%, sham 9.1%). In patients with moderate to moderately severe stroke at admission (NIHSS of 7–15), post hoc analysis showed a significant effect (n=434, treated 51.6% vs sham 41.7%; p=0.044).⁷ These findings are similar to those of a smaller study of the membrane-activated metal ion chelator DP-b99. Among the 147 patients randomized within 9 h from symptom onset, the best treatment effect was seen in patients with NIHSS scores of 11–15.⁸

Although based on post hoc analyses, findings from SENTIS and these other studies suggest that care must be exercised in determining selection criteria and appropriate outcome measures for acute stroke clinical trials. Patients with moderately severe stroke are likely to provide useful information as the cohort that may best demonstrate treatment effects. Care needs to be taken in designing stroke trials that have broad time and stroke severity enrollment criteria when a fixed dichotomy or return-to-normal measure is employed as the sole criterion of success, because the results from substantial proportions of patients may not be informative about treatment efficacy. Broad enrollment criteria have often been used in interventional stroke trials, but have not always proved to be advantageous for the clearest evaluation of treatment effect.

Observations from SENTIS and other recent trials, showing better treatment effects in patients with moderate stroke symptoms and/or earlier time to intervention, raise the issue of identifying clinically appropriate stroke outcome measures for patient benefit along the continuums of severity and time. Dichotomizing final outcome scales at excellent versus not excellent results (eg, mRS 0–1 vs 2–6), as in the NINDS study, may be informative for interventions applied in the first 3 h after onset when patients have often not yet developed substantial irreversible injury. However, at later times, especially when symptoms are severe, injury accumulated before intervention places a ceiling on potential recovery and makes achieving an excellent final outcome highly unlikely. In intermediate time periods, such as 3–8 h, dichotomizing final outcomes as good versus not good (eg, mRS 0–2 vs 3–6) may be a more informative statistical approach. Correspondingly, at later time periods, such as 8–14 h, dichotomizing final outcomes of fair versus not fair (eg, mRS 0–4 vs 5–6) or survival versus fatal outcome may be more clinically relevant. Such a strategy was used in patients with moderate-to-severe deficits in the hemicraniectomy trials.⁶

When the SENTIS trial was designed, the ‘global outcome test’ and some dichotomization in mRS scores (0–2 vs 3–6) were standard tools for evaluating effects of treatment in studies of acute stroke. Assessment of the effects of treatment in randomized trials has improved in recent years.² In addition to evaluating fixed dichotomous (mRS of 0–2 vs 3–6) outcomes, there is increasing emphasis on shift analysis, especially in patients treated over prolonged periods of time where the effect of intervention may decrease over time. In such situations, freedom from dependency (mRS of 0–3 vs 4–6) or freedom from severe disability (mRS of 0–4 vs 5–6) are being used in determining the efficacy of interventions in patients with acute stroke.² The post hoc subgroup analysis from the SENTIS trial presented here adds to the clinical trial experience in evaluation of treatments of acute stroke where intervention is started several hours after the onset of symptoms. Considering the time window and evaluation of the mRS parameters from our post hoc analysis may be useful in the design of future studies. The authors recognize that additional studies are needed to verify the results of this analysis and guide clinical standards.

The major limitation of these findings is that the data are the result of a post hoc subgroup analysis of the SENTIS trial patients. Thus, the p values presented are nominal, and multiple comparison adjustments were not performed. The limited number of events in multivariable logistic regression models for smaller subgroups may produce difficulties with the statistical estimates. However, this analysis was intended as an exploratory examination of the SENTIS results to gain further understanding of the study population and identify patient subgroups that might have benefited from the treatment despite the failure to meet the primary end point for the overall study. Although the exact subgroup cut-off points were not prespecified, the variables of time and severity were prespecified for subgroup analyses, and these cut-off points were intended to represent trichotomizations of time (early, mid, and late) and stroke severity (mild, moderate, and severe) within the overall SENTIS patient population. These results may be informative for designing future trials of stroke treatments that might have marginal treatment effects in a broad patient population. Despite these limitations, the SENTIS data were collected in a blinded manner and show similar trends to results from other recent stroke trials.

The results of our analysis of the SENTIS trial support the importance of careful selection of outcome measures and the impact that rapid treatment and initial stroke severity have on outcome. When SENTIS outcomes were dichotomized as good versus not good (mRS 0–2 vs 3–6), NeuroFlo-treated patients who had earlier times to treatment and moderately severe initial deficits had better outcomes than similar patients receiving medical management alone.