Redefined Measure of Early Neurological Improvement Shows Treatment Benefit of Alteplase over Placebo

Abstract

Background and Purpose

The first of the 2 NINDS Stroke Study trials did not show a significant increase in early neurological improvement (ENI), defined as NIHSS improvement by ≥ 4, with alteplase treatment. We hypothesized that ENI defined as a percentage change in NIHSS (percent change NIHSS) at 24 hours is superior to other definitions in predicting 3-month functional outcomes and using this definition there would be treatment benefit of alteplase over placebo at 24 hours.

Methods

We analyzed the NINDS rt-PA Stroke Study (Parts 1 and 2) trial data. Percent change NIHSS was defined as [(admission NIHSS score–24-hour NIHSS score)x100/admission NIHSS score] and delta NIHSS as (admission NIHSS score–24-hour NIHSS score). We compared ENI using these definitions between alteplase vs. placebo patients. We also used receiver operating characteristic (ROC) curve to determine the predictive association of ENI with excellent 3-month functional outcomes [Barthel Index (BI) score 95 – 100 and modified Rankin scale (mRS) 0–1], good 3-month functional outcome (mRS 0–2) and 3-month infarct volume.

Results

There was a significantly greater improvement in the 24-hour median percent change NIHSS among patients treated with alteplase compared to the placebo group (28% vs. 15%, p = 0.045) but not median delta NIHSS (3 vs. 2, p = 0.471). ROC curve comparison showed that percent change NIHSS (ROC_percent) was better than delta NIHSS (ROC_delta) and admission NIHSS (ROC_admission) with regards to excellent 3-month BI (ROC_percent 0.83, ROC_delta 0.76, ROC_admission 0.75), excellent 3-month mRS (ROC_percent 0.83, ROC_delta 0.74, ROC_admission 0.78), and good 3-month mRS (ROC_percent 0.83, ROC_delta 0.76, ROC_admission 0.78).

Conclusion

In the NINDS rt-PA trial, alteplase was associated with a significant percent change improvement in NIHSS at 24 hours. Percent change in NIHSS may be a better surrogate marker of thrombolytic activity and 3-month outcomes.

Introduction:

The National Institute of Neurological Disorders and Stroke (NINDS) Recombinant Tissue Plasminogen Activator Stroke Study was a multicenter, prospective, double-blind, placebo-controlled, randomized trial of intravenous alteplase for acute ischemic stroke (AIS) and was conducted in 2 parts. ¹ Part 1 of the study did not show a statistically significant improvement in the percentage of patients with early neurologic improvement (ENI) at 24 hours in between alteplase and placebo group (47% vs 39%; p=0.06). ¹ ENI, in this study, was defined as a complete resolution of the neurologic deficit or an improvement from baseline in the National Institutes of Health stroke scale (NIHSS) by 4 or more points 24 hours after the onset of stroke. After the publication of this trial, several papers criticized the interpretation arguing that there is no substantial benefit of alteplase. ^2–4 Ensuing the negative results of part 1 of the NINDS rt-PA trial, ENI was not used as a treatment outcome measure in other thrombolytic trials. ^5–10

While ENI at 24 hours has been shown to correlate with 90-day outcomes, the definition of ENI has varied across studies. Some studies defined ENI using the 24-hour NIHSS score or absolute change in NIHSS from baseline to 24-hours while others used dichotomized absolute change or percentage change from baseline to 24-hours.^11–17 Absolute changes in NIHSS score without consideration of admission NIHSS may not represent the same degree of improvement or neurological status at 24-hours, whereas using the percent change in NIHSS score accounts for the wide variation in admission scores across patients. We hypothesized that, redefining ENI as a percentage change in NIHSS at 24-hours, accounts for the baseline stroke severity and is a superior definition of ENI as compared to delta NIHSS in predicting 90-day functional outcomes and that using this definition we demonstrate a treatment effect of alteplase vs. placebo at 24 hours.

Methods:

Publicly available, archived, and completely de-identified NINDS rt-PA dataset was obtained from the NINDS. Data from both parts of the study were combined for the analyses reported here to obtain more statistical power. Percent change NIHSS and other definitions of ENI (delta NIHSS of ≥ 4, ≥ 8 points or percent change ≥ 20%, ≥ 30% improvement at 24 h) were used to compare treatment benefit in between alteplase and placebo groups. We used the 30% decrement as used by previous studies ¹⁸. The 30% cut-off in our study appears to be the best predictive value in terms of both specificity (0.759) and sensitivity (0.772) based on the ROC graph. ENI was defined as percent change [(Admission NIHSS – 24hr NIHSS)*100/Admission NIHSS]. We examined the association of long-term stroke outcomes with ENI as defined by us and other previously used definitions. 3-month excellent outcomes were Barthels’s Index (BI) score 95 – 100 and modified Rankin scale (mRS) < 2 and good outcomes were mRS < 3. Furthermore, we analyzed associations between ENI definitions (percent change NIHSS and delta NIHSS) and symptom onset – treatment time and 3-month raw infarct volumes. 3-month raw infarct volumes in this study were acquired from computed tomography scans. ¹⁹ We did not use imputation to calculate 3-month infarct volumes for missing patients.

Statistical analysis:

The Pearson chi-square test was used to compare frequencies. For continuous variables, data distributions were examined for normality using the Shapiro-Wilk test. Provided that the data was not normally distributed, the median and interquartile range (IQR) were calculated and non-parametric tests (Mann–Whitney U test) were applied. Receiver operating characteristics (ROC) curve analysis was used to quantify the discriminative ability of the admission NIHSS, delta NIHSS and percent change NIHSS with predictors of long term stroke outcomes. To control for stroke severity, we performed secondary analysis and stratified patients by levels of admission stroke severity: mild (NIHSS score ≤5), moderate (NIHSS score 6–14), and severe (NIHSS score ≥ 15) as these definitions predict clinical outcomes well.²⁰ P < 0.05 was considered statistically significant for all analyses All data was analyzed using the Statistical Package for Social Science (SPSS) software version 25.0 (IBM Corp., Armonk, NY, USA).

Results:

There were 624 patients enrolled in the NINDS rt-PA Stroke Study (312 randomized to alteplase and 312 randomized to placebo). Baseline characteristics of the patients in the treatment and placebo group have been previously reported. ¹ There was a significantly greater improvement in the 24-hour median percent change NIHSS at 24 hours among patients treated with alteplase compared to the placebo group (28% vs. 15%, p = 0.045) (Table 1). On the other hand, the median delta NIHSS was not significantly different between the two groups (3 vs. 2, p = 0.471).

Table 1.

Treatment benefit in alteplase vs placebo group based on different definitions of Early Neurological Improvement at 24 hours

	Alteplase	Placebo	p-value
Admission NIHSS (median, IQR)	14, 11	15, 11	0.423
Percentage change (median, IQR)	28, 75	15, 50	0.045*
Delta NIHSS (median, IQR)	3, 9	2, 7	0.471
Delta 4 or more	147/312 (47.1%)	122/312 (39.1%)	0.043*
Delta 8 or more	81/312 (25.9%)	49/312 (15.7%)	0.002*
Percentage change 30% or more	155/312 (49.7%)	102/312 (32.7%)	<0.001*
Percentage change 20% or more	169/312 (54.2%)	143/312 (45.8%)	0.037*

NIHSS - National Institutes of Health Stroke Scale; IQR - Interquartile range

Receiver operating characteristic (ROC) curve comparison showed that percent change NIHSS (ROC_percent) had better discriminative ability than delta NIHSS (ROC_delta) and admission NIHSS (ROC_admission) to quantify excellent 3-month BI (ROC_percent 0.83, ROC_delta 0.76, ROC_admission 0.75), excellent 3-month mRS (ROC_percent 0.83, ROC_delta 0.74, ROC_admission 0.78), and good 3-month mRS (ROC_percent 0.83, ROC_delta 0.76, ROC_admission 0.78) (Figure 1).

Figure 1:

Comparision of receiver operating characteristic (ROC) curve for admission NIHSS (ROC_admission), delta NIHSS (ROC_delta) and percent change NIHSS (ROC_percent) demonstrating the better discriminative ability of percent change NIHSS to predict a) 3-month mRS<2 (ROC_percent 0.83 (0.79-0.87), ROC_delta 0.76 (0.70-0.78), ROC_admission 0.75 (0.74-0.82)); b) 3-month mRS<3; c) 3-month BI>95 for all patients. Data is represented as ROC (95% Confidence Interval)

Among patients who received alteplase therapy, median delta NIHSS (4 vs 2; p = 0.039) and median percent change NIHSS (39.1 vs 16.7; p = 0.023) were significantly greater among patients who received treatment within 0–90 mins compared with 91–180 mins (Table 2). 3-month raw infarct volumes were available for 471(76%) patients. They were significantly lower with increasing percentage change categories (percent change NIHSS < 20% = 32.3 ml, percent change NIHSS 20 – 30% = 30.6 ml, percent change NIHSS >30% = 4.2 ml; p <0.001), whereas this dose-response relationship was not as robust when delta NIHSS was used (delta NIHSS < 4 = 23.9 ml, delta NIHSS 4 – 7 = 7.2 ml, delta NIHSS >8 = 7.2 ml; p <0.001). (Figure 2).

Table 2:

Early neurologic improvement in patients receiving alteplase treatment based on time to treatment

	0 – 90 mins	91 – 180 mins	p-value
delta NIHSS, median (IQR)	4 (9)	2 (7)	0.039
percent change NIHSS, median (IQR)	39.1% (75)	16.7% (80.1)	0.023

NIHSS - National Institutes of Health Stroke Scale; IQR - Interquartile range

Figure 2:

90-day infarct volumes in patients with different delta NIHSS and percent change NIHSS.

Sensitivity analysis:

Sensitivity analyses were performed to study the correlation of ENI measures on study outcomes stratified by stroke severity. In this analysis, percent change NIHSS outperformed delta NIHSS and admission NIHSS in patients with moderate and severe strokes (Table 3). In patients with mild stroke (NIHSS ≤ 5), however, both measures of ENI had only modest value in predicting functional outcomes (Table 3).

Table 3:

Area under the curve values for admission National Institutes of Health Stroke Scale (NIHSS), delta NIHSS and percent change NIHSS, quantifying the discriminative ability of these variables to predict 3-month outcomes for all patients and stratified based on stroke severity. Data is presented as Odds Ratio (95% Confidence Interval)

	mRS < 2	mRS < 3	BI 95-100
Mild Stroke: NIHSS 0-5
Percentage change	0.829 (0.698 – 0.959)	0.807 (0.649 – 0.966)	0.876 (0.734 – 1.000)
Delta NIHSS	0.854 (0.740 – 0.969)	0.828 (0.679 – 0.976)	0.890 (0.760 – 1.000)
Admission NIHSS	0.449 (0.246 – 0.653)	0.440 (0.196 – 0.684)	0.471 (0.240 – 0.703)
Moderate Stroke: NIHSS 6-15
Percentage change	0.824 (0.776 – 0.872)	0.832 (0.785 – 0.878)	0.827 (0.779 – 0.874)
Delta NIHSS	0.799 (0.748 – 0.849)	0.813 (0.763 – 0.862)	0.808 (0.757 – 0.858)
Admission NIHSS	0.661 (0.596 – 0.726)	0.669 (0.606 – 0.732)	0.640 (0.576 – 0.704)
Severe Stroke: NIHSS > 15
Percentage change	0.942 (0.908 – 0.975)	0.938 (0.909 – 0.967)	0.903 (0.864 – 0.943)
Delta NIHSS	0.917 (0.880 – 0.954)	0.925 (0.839 – 0.957)	0.893 (0.852 – 0.934)
Admission NIHSS	0.688 (0.601 – 0.774)	0.626 (0.547 – 0.706)	0.607 (0.528 – 0.685)

Discussion:

In this study, we show that in the NINDS rt-PA stroke trial, there was treatment benefit with alteplase over placebo using percentage change in NIHSS at 24 hours and this was not present when delta NIHSS was used. In addition, the percentage change in NIHSS had a stronger predictive ability of functional outcomes at 3 months as compared to delta NIHSS. Therefore it can be used as a surrogate marker of thrombolytic activity. In clinical practice, early neurological improvement is considered a consequence of successful reperfusion, and may be regarded by a clinician as an indicator of effective acute stroke therapy. Furthermore, this may be a useful tool to track patient outcomes at an early time frame after acute therapy.

Part 1 of the NINDS rt-PA trial was the first study to use improvement at 24 hours as a primary outcome measure, but this has been critiqued as it does not account for the effect on stroke recovery and it is not a direct measure of functional outcome, which is considered a more valuable target in clinical trials.²¹ In this trial, 4-point cut-off used for delta NIHSS may not have been specific to thrombolytic activity as a relatively high proportion of stroke patients in the NINDS trial achieved this. In fact, a post-hoc analysis of the NINDS tPA trial revealed that a 5 point decrease in NIHSS produced a statistically significant improvement.²²

The 24-hour post-stroke time point is clinically important because many prognostically relevant clinical and radiological variables become available at or after 24 hours following admission for acute ischemic stroke. Patients who received alteplase typically undergo 24-hour follow-up brain imaging to exclude hemorrhagic transformation. In addition, the half-life of alteplase is short and the lytic activity of alteplase can persist up to 24 hours from the infusion time. ²³ Therefore, reperfusion caused by alteplase is best demonstrated at the 24-hour mark highlighting the importance of using a reliable measure for ENI at 24 hours. In addition, in our study, the effect of alteplase on ENI was more pronounced with faster treatment times.

The NIHSS and delta NIHSS scores are both ordinal and skewed in their functional significance. A major limitation of using absolute changes in NIHSS scores is that while a certain absolute change may reflect a dramatic improvement in patients with lower admission NIHSS score, the same change in a patient with a more severe stroke may have no significant impact on functional outcome.²⁴ For instance: a 4-point improvement from 20 to 16 is a considerably smaller percent change (20%) than the improvement from 6 to 2 (67%). Note that both the final score and the percent change are more revealing than the absolute NIHSS score change. Although the absolute change is 4 in both situations, the final score of 2 and the 67% improvement in the second situation are more likely to represent favorable clinical outcome than the final score of 16 and the 20% improvement in the first situation.

Functional clinical outcomes in ischemic stroke are typically measured using disability scales such as the mRS and BI, and a 3-month assessment is the most frequently used time point for primary outcome assessment in stroke research. An mRS < 2 or BI ≥95 at 3 months is considered an excellent outcome and an mRS < 3 is considered a good clinical outcome that is indicative of functional independence in ischemic stroke patients. ²⁵ Results of our analysis demonstrated that while both delta NIHSS and percent change NIHSS correlated with 3-month functional outcomes, percent change NIHSS had a stronger predictive ability. In the sensitivity analysis stratifying patients according to stroke severity, this was more pronounced in patients with moderate and severe strokes but not mild strokes. This could possibly be due to the fact that in patients with mild strokes, the majority of patients have a good functional outcome. In this cohort of patients, various studies have shown that the type of deficit may be more important than the severity of the deficit in the prediction of 90-day functional outcomes.^26–28 This highlights the limitation of the NIHSS score in this patient population. For instance, distal hand weakness may be an isolated disabling symptom in a number of patients with stroke and is not well captured in the arm drift item of the NIHSS. The NIHSS also does not capture gait or mobility measures, such as gait speed, which have been suggested to be predictors of functional outcome in minor stroke.²⁹ In addition, certain measures of cognition, which may be affected by stroke including visuospatial and executive function ³⁰ are not captured by the stroke scale. Therefore in patients with minor stroke, the admission NIHSS score or changes (absolute or percentage) in the NIHSS at 24 hours may not be extremely useful in predicting the functional outcome as they do not account for the above-mentioned measures.

Secondary analysis done by the NINDS rt-PA Stroke Study group on the trial data demonstrated that after alteplase treatment, there was a trend toward smaller 3-month lesion volume compared with the placebo group (p=0.06), with a reduction of 11% in cumulative lesion volume in the alteplase treated group compared with the placebo-treated group. ¹⁹ Several studies have also assessed the relation between infarct volume and functional outcome. ^31–34 All these studies report that infarct volume is a strong predictor of functional outcome, independent of other known important factors such as age and admission NIHSS score. Our analysis demonstrated that 90-day infarct volume was smaller in patients with a greater change in 24 hour NIHSS (delta and percent change), which correlated with long term functional outcomes. This again, emphasizes the clinical utility of using the 24-hour measures of clinical outcomes as well as the measures construct validity.

Our study has several limitations including its being a post-hoc analysis and therefore the findings are hypothesis-generating. Despite these limitations, our study has some significant strengths which include the fact that we used data from a large randomized clinical trial, where the 24-hour NIHSS and long term functional outcome scores was performed in a prospective and blinded manner and not calculated retrospectively from documented neurological examinations.

Conclusion:

In the NINDS rt-PA trials, there was a treatment benefit at 24 hours in the alteplase group when the data is analyzed through a percentage change in NIHSS. This may be a better surrogate marker of ENI as compared to absolute change in NIHSS at 24 hours that correlates with functional outcome and infarct volume at 3 months. This parameter should further be validated in further prospective larger studies.