Relationship between 3-Month National Institutes of Health Stroke Scale Score and Dependence in Ischemic Stroke Patients

Abstract

Background

The National Institutes of Health Stroke Scale (NIHSS) provides a standardized measure of stroke severity and is frequently captured to assess 3-month outcome. Other outcome measures have been assessed for the relationship to dependence; a clinically relevant outcome. The relationship between NIHSS score and functional dependence is unknown. The purpose of this study was to assess the relationship between NIHSS score and accepted measures of dependence in surviving ischemic stroke patients.

Methods

3-month NIHSS scores were compared to residence and Glasgow Outcome Scale (GOS) scores at 3 months in the Randomized Trial of Tirilazad Mesylate in Patients with Acute Stroke (RANTTAS). For residence, patients who were in a nursing home, chronic hospital or substantially dependent on a caregiver were characterized as ‘dependent’. For GOS, a score of 3 (severely disabled) or 4 (vegetative) was characterized as ‘dependent’. The sensitivity, specificity and positive (PPV) and negative predictive values (NPV) for various NIHSS score cut points compared to dependence were calculated. Logistic regression analysis was used to assess the association between the NIHSS score and dependence.

Results

In 385 subjects from the RANTTAS, an NIHSS score cut point of ≥ 15 resulted in 100% of subjects identified as being dependent by residence, sensitivity = 24%, specificity = 100%, PPV = 100% and NPV = 80%. Using GOS as the measure of dependence, the results were almost identical. NIHSS was strongly related to dependence with an area under the receiver operating characteristic curve (AUC) = 0.86 for residence and an AUC = 0.94 for GOS.

Conclusions

3-month NIHSS score is strongly associated with dependence. An NIHSS score of ≥ 15 at 3 months may be a reasonable estimate of subjects who are highly likely to be dependent at 3 months. These data require validation in an independent data set.

The National Institutes of Health Stroke Scale (NIHSS) is a well-accepted standardized measure of stroke severity [1] . It has traditionally been used in acute ischemic stroke clinical trials to assess baseline stroke severity as well as outcome deficit [2, 3] . Other frequently used clinical outcome measures for acute ischemic stroke trials have clear definitions for alive but dependent including the Barthel Index [4] , the modified Rankin [5] and the Glasgow Outcome Scale (GOS) [6] , but there are no published studies documenting the relationship of NIHSS score at 3 months with other measures of dependence. Identifying a relationship between 3-month NIHSS score, a measure commonly used by clinical researchers, and dependence, a concept well understood by patients and clinicians, may facilitate the understanding of clinical trial results. The purpose of this study was to assess the relationship between 3-month NIHSS score and 3-month dependence, as measured by residence and by GOS, in ischemic stroke patients.

Methods

Study Population

A total of 385 subjects from the Randomized Trial of Tirilazad Mesylate in Patients with Acute Stroke (RANTTAS) [7] were used for this analysis. The RANTTAS population has been described in detail previously [7] . Briefly, this was a multicenter, randomized, double-blinded, vehicle-controlled trial to evaluate the efficacy and safety of tirilazad mesylate in ischemic stroke patients. Outcome measures used in this trial included the Barthel Index, GOS and NIHSS. Place of residence at 3 months was also captured. As there was no demonstrated treatment effect, both treatment groups were combined for this analysis. Four hundred and seventy-four of those subjects were alive at 3 months and could be considered for this analysis. Eighty-nine of the surviving 474 subjects were excluded for this analysis. Prespecified exclusions included missing 3-month residence (n = 15) and missing NIHSS score within the 3-month window (60–125 days) (n = 74). Dependence was defined in 2 different ways: first as residence at 3 months and second as GOS score at 3 months. For residence, subjects that were reported to be in a nursing home, in a chronic hospital or at home but dependent on a family member or nurse were defined as ‘dependent’. Subjects that were reported to be at home and fully independent, at home with supervision, in a minimal care facility or other (3 subjects) were categorized as ‘not dependent’. For GOS score, subjects with a score of 3 (severely disabled) or 4 (vegetative) were categorized as ‘dependent’ and those with a score of 1 (good recovery) or 2 (moderate disability but independent) were categorized as ‘not dependent’.

Analysis

We calculated the percent dependent for all values of the NIHSS. The sensitivity and specificity (and 95% CI) of the NIHSS score cut point to detect dependence (by residence and by GOS) were calculated for several scores. The 95% CI calculations for sensitivity and specificity with rare events (<5) used exact binomial for <5 or for 0 [8] . Positive (PPV) and negative predictive values (NPV) and area under the receiver operating characteristic curve (AUC) were also calculated.

Sensitivity Analyses

Selection bias was assessed by comparing demographic characteristics of those alive at 3 months but excluded for missing data to those included in the analysis. Rates of dependence were also assessed in those missing NIHSS scores. When differences were identified, further analysis of the excluded group was conducted.

A sensitivity analysis excluding the patients with any degree of disability prior to stroke was completed to assess the impact of prestroke disability.

Results

Characteristics of the population are shown in table 1 .Ninety-seven subjects (25%) were dependent at 3 months by residence and 69 (18%) by GOS as shown in table 1 . The distribution of NIHSS scores is shown in table 2 with the proportion of dependence (by residence and GOS) for each level. Subjects with NIHSS scores of 4 or less were unlikely to be dependent at 3 months by both measures of dependence, while subjects with NIHSS scores of 15 or greater were all dependent by both measures.

Table 1.

Patient characteristics (n = 385)

Age (mean ± SD), years	68±13
Race
Caucasian	319	(83)
AA/black	46	(12)
Other	20	(5)
Male	216	(56)
Baseline NIHSS score1 (median; IQR)	8	(5; 13)
Diabetes mellitus	94	(24)
Lacune	97	(25)
Disabled prior to stroke	42	(11)
3-month residence
At home fully independent	222	(57.7)
At home with supervision	55	(14.3)
Minimal care facility	8	(2.0)
Other	3	(0.8)
At home dependent on family/nurse	34	(8.8)
Skilled nursing facility	34	(8.8)
Chronic hospital or Rehab hospital	29	(7.5)
3-month GOS score
Good recovery	212	(55)
Moderate disability (independent)	104	(27)
Severely disabled	68	(18)
Vegetative	1	(0.3)

Figures in parentheses indicate percentages, unless otherwise specified. IQR = Interquartile range.

¹Baseline NIHSS scores (n = 381) due to missing data.

Table 2.

Distribution of dependence (defined by residence or GOS) by NIHSS score

3-month NIHSS score		Residence				GOS
		not dependent		dependent		not dependent		dependent
score	n	n	%	n	%	n	%	n	%
0	130	122	94	8	6	130	100	0	0
1	64	59	92	5	8	62	97	2	3
2	40	36	90	4	10	37	93	3	7
3	37	31	84	6	16	34	92	3	8
4	20	16	80	4	20	19	95	1	5
5	11	4	36	7	64	7	64	4	36
6	10	5	50	5	50	6	60	4	40
7	11	4	36	7	64	6	55	5	45
8	8	2	25	6	75	3	37	5	63
9	8	2	25	6	75	4	50	4	50
10	6	1	17	5	83	2	33	4	67
11	8	3	38	5	63	3	37	5	63
12	7	2	29	5	71	2	29	5	71
13	2	0	0	2	100	1	50	1	50
14	5	1	20	4	80	0	0	5	100
15	1	0	0	1	100	0	0	1	100
16	2	0	0	2	100	0	0	2	100
17	2	0	0	2	100	0	0	2	100
18	4	0	0	4	100	0	0	4	100
19	5	0	0	5	100	0	0	5	100
21	1	0	0	1	100	0	0	1	100
22	1	0	0	1	100	0	0	1	100
24	1	0	0	1	100	0	0	1	100
29	1	0	0	1	100	0	0	1	100

Table 3 shows the sensitivity and specificity for several different cut points for dependence as measured by residence. The same table demonstrates the PPV and NPV for a range of cut points for the NIHSS score. Though a low cut point results in a high NPV (if one says not dependent, very likely not dependent), a higher cut point such as 13 or 15 results in excellent PPV (if one says is dependent, is highly likely to be dependent). Table 3 also demonstrates very similar results using GOS to define dependence.

Table 3.

Sensitivity, specificity, PPV and NPV of NIHSS identifying dependence as measured by residence and GOS

NIHSS score	Sensitivity, %	Specificity, %	PPV, %	NPV, %
By residence
≥5 (94)	72 (62–81)	92 (88–94)	72	92
≥10 (46)	40 (31–51)	98 (95–99)	85	83
≥13 (25)	23 (15–32)	99 (98–100)	96	79
≥15 (18)	24 (16–34)	100 (99–100)	100	80
By GOS
≥5 (94)	87 (79–95)	89 (86–92)	64	97
≥10 (47)	55 (43–67)	97 (96–99)	83	91
≥13 (25)	35 (23–46)	99 (98–100)	96	87
≥15 (18)	26 (16–36)	100 (99–100)	100	86

Figures in parentheses indicate number or 95% CIs.

Using logistic regression, the NIHSS score alone was strongly related to dependence with an AUC of 0.86 for residence and 0.94 for GOS.

Sensitivity Analyses

We also examined the characteristics of the 89 subjects that were alive at 3 months but had missing data to assess for selection bias. There were differences between the subjects with complete data and those with missing data. The missing data subjects were more frequently African American (AA)/black (p < 0.001) and had more frequent small vessel infarct (p < 0.002). No other baseline characteristics were found to be statistically different. The proportion of the 74 subjects excluded for missing NIHSS scores that were dependent by residence at 3 months was 35%, which was similar to that for the included population.

Because of the race and small vessel etiology differences that were identified, we explored the 89 subjects with missing data further. Twenty-eight percent were from a single site. We therefore examined whether the exclusions were more suggestive of a site effect or a selection bias throughout multiple sites. That single site had 83% AA and 83% small vessel etiology for their enrolled population which differed dramatically from the 12% AA and 25% small vessel etiology for this study population. Reanalysis of baseline characteristics without that single site demonstrated no difference in any baseline characteristics between the excluded subjects and the included subjects. Furthermore, analysis of baseline characteristics at the single site suggested that subjects missing data, at that site, were very similar to all enrolled subjects, at that site, with no clinically or statistically significant differences between those missing data and those with complete data.

The sensitivity analysis excluding the 42 patients with any degree of disability prior to stroke resulted in nearly the same results. An NIHSS score of 15 or greater was 100% specific in identifying dependence for both residence and GOS. The AUC for residence was 0.85 and for GOS it was 0.94.

Discussion

The NIHSS is a stroke severity scale and was not designed to measure dependence. Our data, however, suggest a strong relationship between the NIHSS score at 3 months and dependence as determined by residence or GOS at 3 months. The AUC for the relationship is sufficiently strong to suggest that it may be useful in individual assessment [9] . Depending on the intended use, the true-positive rate can be maximized using a cut point of ≥ 15 with 100% true positives. Alternatively, the true-negative rate can be nearly maximized using a cut point of ≥ 5 with 92% when dependence is defined by residence and 97% when dependence is defined by GOS.

Stroke clinical trials frequently capture 3-month NIHSS scores as outcomes. There is no standard accepted dichotomization of poor outcome by the NIHSS score and no published data on the relationship between 3-month NIHSS score and measures of a clinically relevant outcome such as dependence. Other scales have been described in detail to allow for dichotomization of those outcome scales to reflect various levels of disability; these are accepted as very clinically relevant and easily understood by clinicians and patients. The current analysis allows for a detailed characterization of 3-month NIHSS scores as they relate to dependence in a clinical trial population. Better understanding this relationship may be useful in considering appropriate levels of dichotomization of this outcome to reflect poor outcome in future clinical trials. It may also be useful for clinicians and patients to better understand the results of clinical trials.

This study is limited by the small sample size, especially at the higher levels of NIHSS scores. However, the CIs provided in the sensitivity and specificity analysis suggest that our point estimates are representative. The exclusion of 16% subjects for missing data raises the question of selection bias. We opted to exclude those subjects without NIHSS scores in the 3-month time window, rather than impute or carry forward values from earlier measurements, to reduce the likelihood of disparate results between the capture of 3-month residence or GOS scores and NIHSS scores. The analysis of the baseline characteristics of the group that was missing data did show clinically and statistically relevant differences in race and stroke subtype suggesting that a selection bias may have occurred. The additional analysis, however, suggested a single site effect not an overall selection bias was responsible for those missing data. Because of these limitations and because this is the first data set of our knowledge looking at this relationship, these data need to be validated in an independent data set.

The NIHSS score at 3 months does not directly measure dependence, but these data suggest that it may be an excellent estimate of dependence in situations when a clinically meaningful dichotomization is required. An NIHSS score of ≥ 15 in our data set resulted in a 100% PPV for identifying dependence as defined by 2 different well-accepted measures. These data suggest that patients with NIHSS scores of ≥ 15 at 3 months are highly likely to be dependent.