Assessment of neurological function using the National Institute of Health Stroke Scale in patients with gliomas

Abstract

Background

The evaluation of treatment response in patients with gliomas is performed using the Response Assessment in Neuro-Oncology (RANO) criteria. These criteria are based on cerebral magnetic resonance imaging (MRI), steroid use, and neurological function. However, a standardized tool for evaluating neurological function was lacking. We compared changes in the National Institute of Health Stroke Scale (NIHSS) to changes in the RANO categories to determine the relationship between clinical and neuroradiological findings.

Methods

We reviewed data on all adult patients with supratentorial gliomas WHO grade II-IV who were treated at the Cantonal Hospital St. Gallen from 2008 to 2015. The NIHSS was performed prospectively at baseline and at 3-month intervals simultaneously to MRI. Associations between changes in the NIHSS and RANO categories were assessed using the Stuart-Maxwell test.

Results

Our cohort consisted of 61 patients from which 471 observations were analyzed. The most common histological diagnosis was glioblastoma (49.2%). In total, 74% of RANO categories and 81% of the NIHSS scores remained stable on follow-up. Statistically, contemporaneous changes in the RANO category did not correlate with changes in the NIHSS (P < .0001).

Conclusion

The application of the NIHSS is easy and feasible in the heterogeneous population of glioma patients. In our cohort, the RANO categories did not reflect contemporaneous changes in the NIHSS. A validated clinical outcome measure with a well-defined minimal clinically important difference is warranted in neuro-oncological research and clinical practice.

Gliomas are the most common primary brain tumors in adults and were recently reclassified by the World Health Organization (WHO) taking molecular markers into account.¹ Although important progress has been made in the field of neuro-oncology in the last decade, treating high-grade gliomas remains a significant challenge with only few effective treatment options. The introduction of several new agents in various stages of the treatment pipeline of gliomas has so far failed to provide a substantial improvement in overall survival.^2,3 Nevertheless, these advances in drug development have led to an increasing number of clinical trials.⁴ To date, overall survival is considered the gold standard endpoint for oncological studies. However, it is widely acknowledged that the effect on quality of life and neurological function has to be taken into account when evaluating the benefit of palliative therapies in patients suffering from brain tumors. This is even more important as even sophisticated imaging can be ambiguous in the context of surgery, radiotherapy, and systemic treatment including anti-angiogenetic drugs.

Since 2010, the international Response Assessment in Neuro-Oncology (RANO) criteria have been used to assess the therapeutic response in gliomas in clinical practice and research.⁵ These criteria rely mainly on 2-dimensional contrast-enhancing tumor measurements on magnetic resonance imaging (MRI). Since they were first implemented, variations of the RANO criteria have been developed for low-grade glioma,⁶ brain metastases,⁷ and immunotherapy.⁸ These adjustments reflect the need for an ongoing evolution of response assessment along with the advent and usage of new treatment modalities.

Despite this progress, the differentiation between pseudoprogression, pseudoresponse, and non-enhancing tumor progression remains a challenge even with sophisticated anatomical and metabolic imaging techniques.⁹ Both, pseudoresponse and pseudoprogression, underline the fact that contrast enhancement is not a straightforward measurement of tumor activity, but also dependent on changes of vascular permeability caused by different treatment modalities.¹⁰ Consequently, additional information, such as repeated MRI, steroid use, anti-angiogenetic drug treatment, and clinical status, has to be taken into account when assessing therapeutic response. Although the RANO criteria incorporate these aspects, they lack objective measurements for evaluating neurological response. Given the importance of the neurological status in the patient evaluation process and drug development, there is a clear need to robustly quantify the effect of interventions on neurological function both at the level of the individual patient and in the context of clinical trials. To address this clinical challenge, the Neurologic Assessment in Neuro-Oncology (NANO) working group developed a clinical assessment tool specifically for patients with brain tumors, which was first published in 2017.

Given the lack of validated measures of neurological function in neuro-oncological patients at the time of the study (2008-2015), we incorporated the National Institute of Health Stroke Scale (NIHSS) in the standard assessment of glioma patients in our interdisciplinary neuro-oncology clinic at the time of increasing usage of bevacizumab in the treatment of higher-grade gliomas. The NIHSS is a standardized and validated bedside test of neurological function used in the field of stroke.¹¹ Furthermore, this scale has been used to quantify outcomes in glioma patients,^12–15 despite not being systematically validated in this patient group. The current study aimed to determine whether the NIHSS might complement the current RANO classification by adding a quantitative measurement of neurological function. We hypothesized that changes in the NIHSS would correlate with changes in the RANO categories in individual patients in response to various treatments.

Materials and Methods

Study Population

A monocentric, retrospective review was performed of adult patients with de novo or recurrent cerebral glioma WHO grade II-IV treated at the Canton hospital St. Gallen from January 1, 2008 through December 31, 2015. Patients were identified by the local central nervous system tumor database. All glioma patients were prospectively evaluated using the NIHSS and Karnofsky performance scale as a standard of care. These scores were collected prospectively at each patient encounter. Additionally, the dose of steroids (in mg) was documented as part of the RANO assessment. The diagnosis of glioma was confirmed by histopathology in all patients. Subjects were excluded from this study if they were under the age of 18 years or did not have more than 1 complete neurological exam (including the NIHSS) or MRI during follow-up. Furthermore, patients with preexisting neurological comorbidities, such as antecedent brain trauma, stroke, or severe neuropathy, which might interfere with the NIHSS measurement, were excluded. All patients included in this study received treatment according to international standards. The study was approved by the local ethical committee (EKO/2019-02111). The retrospective chart review did not require patient consent.

Measurements

We retrospectively reviewed the electronic medical records of all eligible patients. Patient records and radiological reports were reviewed for demographic and clinical information that included: age, sex, pre- and postoperative MRI scans, surgical pathology, and NIHSS scores. MRI scans were exclusively analyzed by consultants with expertise in neuroradiology. Patients were categorized as either having: (1) complete response (CR), (2) partial response (PR), (3) stable disease (SD), or (4) progressive disease (PD) according to the RANO criteria. The NIHSS was used to quantify neurological function during routine clinical examination in patients at their follow-up appointments, at least every 3 months in parallel with neuroimaging.¹⁶ Over the last decade, the NIHSS has been used as a primary endpoint in several landmark trials in stroke.¹⁷ This quantitative measure of neurological deficits evaluates 15 domains including consciousness, aphasia and dysarthria, hemianopia and extraocular movement, hemiparesis, sensory loss, ataxia, and neglect. The score ranges from a minimum of 0 to a maximum of 42, higher scores indicating worse neurological function. For the purpose of this and other ongoing studies, an increase by 1 point was considered as neurological decline in the statistical analysis. Physicians at our clinic received a web-based teaching course prior to using the NIHSS in clinical practice. Death of patients or loss of follow-up resulted in a different number of follow-up visits across the patient cohort. Patient’s death dates were gathered by either a documented date of death in the electronic medical record or public records documenting patient death.

Statistical Analysis

All statistical analyses were performed using R version 3.2.3. Descriptive methods were used to summarize baseline demographic and clinical data. To determine whether changes in the RANO classification were associated with changes in the NIHSS in individual patients during follow-up, first, the transition rates for the NIHSS and the RANO levels (change from a given NIHSS or RANO score to a follow-up score in counts or categories, respectively) were graphically expressed. Thereafter, the agreement between changes in the NIHSS and in RANO categories was assessed. To examine the primary hypothesis that changes in the RANO categories and changes in the NIHSS go in the same direction, each variable was categorized in a change larger (>0), smaller (<0), or equal to zero (0). A negative value indicated a worsening in the classification of the corresponding factor. A value above zero implied an improvement and a value of zero implied no change. The Stuart-Maxwell test was used to examine the agreement between changes in RANO categories and NIHSS scores.

Results

The initial dataset included 69 patients of which 8 patients were excluded because of incomplete data. After excluding these patients, a dataset of 471 observations from 61 patients was available. However, 106/471 (23%) RANO values and 93/471 (20%) NIHSS scores were not assessable due to disease progression where clinical assessment was not possible (remote encounter) or imaging was not appropriate (deterioration of clinical condition) (Figure 1). These values were excluded from further analysis. Demographic and clinical characteristics are summarized in Table 1. Glioblastoma was the most common histological diagnosis (49.2%). The mean age of the patient population was 52 ± 11.0 years. Patients were followed up for a mean period of 1.4 years (range 0.13-5.6 years) and had an average of 6 assessments (range 2-22).

Figure 1.

Consort diagram of the RANO and NANO data. Abbreviations: NANO, Neurologic Assessment in Neuro-Oncology; RANO, Response Assessment in Neuro-Oncology.

Table 1.

Baseline Demographic and Clinical Characteristics of All Patients

Age
Years, range	52.0 (±11.0), 24-75
Gender
Female	36 (59.0%)
Male	25 (41.0%)
WHO grade
II	10 (16.4%)
III	21 (34.4%)
IV	30 (49.2%)
Total cohort	n = 61 (100%)

Data are presented as count (%) or mean (standard deviation), range.

Table 2 outlines the number of transitions between 2 subsequent observations in RANO categories and NIHSS scores in individual patients. In total, 96 (26.0%) transitions were observed across different RANO categories and 70 (18.5%) transitions across different NIHSS scores.

Table 2.

Transitions Between RANO Categories and NIHSS Scores During Follow-Up Visits

RANO Categories
Follow-Up			CR		PR		SD		PD
Baseline	CR		14		0		1		5
	PR		1		10		11		9
	SD		3		6		201		31
	PD		0		10		19		44
NIHSS Scores
Follow-Up		0	1	2	3	4	5	6	7	8	9
Baseline	0	224	10	6	5	2	0	0	0	0	1
	1	7	26	6	1	1	0	0	0	1	0
	2	3	1	23	3	2	0	0	0	0	0
	3	0	1	5	15	1	1	0	1	0	0
	4	1	0	1	0	17	0	0	0	1	0
	5	0	0	1	0	2	3	2	0	0	0
	6	0	0	0	1	0	1	0	0	0	0
	7	0	0	0	0	0	0	0	0	1	0
	8	0	0	0	0	0	0	1	0	0	0

Abbreviations: CR, complete remission; NIHSS, National Institute of Health Stroke Scale; PD, progressive disease; PR, partial remission; RANO, Response Assessment in Neuro-Oncology; SD, stable disease.

The rows indicate the starting state (from) and columns the state at the next follow-up visit (to). The tables outline the number of transitions between 2 subsequent observations in RANO categories and NIHSS scores in individual patients. In total, 96/365 (26%) transitions were observed from 1 RANO category to another (ie, complete remission to progressive disease) and 269/365 (74%) RANO scores remained stable on follow-up. Only 70/378 (19%) transitions from a given NIHSS score to a different score were observed, and 308/378 (81%) were retained. Therefore, only the minority of the study group changed their RANO category or their NIHSS score, respectively, from a given clinical encounter to a follow-up assessment.

Agreement Between Changes in RANO Categories and NIHSS Scores

Table 3 illustrates the synthesis of RANO criteria changes and NIHSS score changes (n = 339 observations, Figure 2). The agreement and especially the disagreement are the main interest of the study. For the category “no change,” 217/339 (64.0%) of the observations in the 2 methods were agreed. However, a stable RANO category came along with a disagreement in 9/251 (4%) and 25/251 (10%) with the NIHSS declining or improving, respectively. Hence, 14% of stable RANO assessment did not correspond to stable NIHSS (second row in Table 3, n = 251). For negative changes in RANO, only 2/339 (0.6%) of NIHSS worsening was also observed. Hence, a worse RANO category demonstrating imaging criteria for PD came along with a stable NIHSS in 39/53 (74%) patients and an improved NIHSS in 12/53 (23%) patients. In total, a worsening in RANO category did not correspond to the NIHSS in 97% of cases (first row in Table 3, n = 53). For improving RANO assessment, only 1/339 (0.3%) NIHSS score agreed, whereas 27/35 (77%) patients remained stable or 7/35 (20%) improved in the NIHSS, respectively (third row in Table 3, n = 35). Statistically, the Stuart-Maxwell test indicates that there was overall no agreement between the 2 scales (P < .0001). This lack of agreement is exemplified in the trajectories of individual RANO and NIHSS follow-up assessments of 3 selected patients (Figure 2A–C).

Table 3.

Agreement Between Changes in RANO Categories (Rows) and NIHSS Scores (Columns)

NIHSS		<0	0	>0
RANO	<0	2	39	12
	0	9	217	25
	>0	7	27	1

Abbreviations: NIHSS, National Institute of Health Stroke Scale; RANO, Response Assessment in Neuro-Oncology.

The agreement of the 2 variables corresponds to the entries on the diagonal. In total, 339 observations were included. There were 26 observations for which the NIHSS score was missing, which were omitted. The Stuart-Maxwell test shows no agreement between the 2 scales (P < .0001). <0, downgrading; 0, unchanged; >0, upgrading in each assessment method.

Figure 2.

Trajectories of individual MRI and neurological follow-up assessments of three patients. Patient 32 showed a progressive disease in MRI scan, but after an initial worsening in NIHSS, it improved and remained stable afterwards until last visit (i.e. asymptomatic progression). The asterix for Patient 32 (*) represents a survival event. Patient 40 had discordance between progressive disease on MRI and stable NIHSS (i.e. asymptomatic progression. Patient 41 concomitant worsening of NIHSS when disease was progressive on MRI (i.e. symptomatic progression), but clinical function did not fully improve after the disease remained stable. However, further progression on MRI was NIHSS-wise asymptomatic.

Discussion

Functional status and quality of life are increasingly recognized as important factors in the overall outcome assessment of glioma treatment. They are known the be independent predictive factors for survival.^18,19 Yet, until recently, there was no established method to quantify neurological symptoms in glioma patients. The NIHSS has been and is still used to describe neurological status in glioma patients in the literature and in ongoing clinical studies.^13–15,20 However, it has not been formerly validated in this patient cohort so far. This study demonstrated that RANO categories do not correlate with the NIHSS scores on clinical follow-up assessments in glioma patients. Only when RANO remains stable there is a moderate overlap with a stable NIHSS (64%), but improving or deteriorating RANO disagreed in 97% of cases with the NIHSS. The RANO classification represents the standard of care not only in clinical routine but also in clinical studies. Our results do not discourage the usage of the NIHSS in the clinical evaluation of patients with neurological deficits; however, it highlights the need for a standardized and comprehensive clinical evaluation in this patient group. Unfortunately, the NIHSS seems to be too insensitive to correspond to changes in RANO categories. This is explained by the fact that 25% and 50% of contrast enhancement changes are necessary to change a RANO category to progression or partial remission, respectively.⁵ Even small structural changes of the CNS below the thresholds of the RANO classification can lead to significant clinical deterioration without changes in the RANO classification and complete remission of contrast enhancement, ie, in the context of anti-angiogenetic therapy may not improve the clinical status of a patient due to a permanent destruction of neuronal tissue. These discrepancies will persist even with the introduction of any examination score as clinical and imaging categories reflect different aspects of the disease.

In the absence of a threshold for a minimal clinically important difference (MCID) in neuro-oncological patients, we considered an increase or decrease of 1 point in the NIHSS score as a change in clinical performance. The same threshold is currently used in the ongoing, randomized controlled SAFE trial (NCT03861299), which evaluates the impact of awake craniotomy on neurological outcome and extent of resection in glioblastoma. The same threshold had been previously used to assess neurological function after surgery, however, it has been noted that an increase by 1 point in the NIHSS was not associated with a self-reported improvement in quality of life measures in the majority of patients.¹⁵ In contrast, in the randomized controlled trial evaluating 5-aminolevulinic acid in glioma surgery, patients were classified as having a favorable neurological outcome (≤2 points change in the NIHSS) compared to patients with an unfavorable neurological outcome (>2 points change).¹² Similarly, in stroke, a change in the NIHSS score of more than 2 points has been suggested as clinically relevant.²¹ It is important to recognize that the MCID of a scale might differ across diseases. Its condition-specific definition is crucial in the interpretation of observed changes, and, in turn, treatment response.²² The low threshold of 1 point used in our study might not reflect a clinically meaningful change in neurological status in glioma patients, which is known to fluctuate, and, thus might not be sensitive enough to reflect changes in the underlying tumor activity. Hence, the results of our study emphasis that the use of outcome measures that have not been thoroughly validated in a specific patient population may lead to a wrong interpretation of the results. As shown in our study, this also applies to scales like the NIHSS, which have been extensively studied and validated in other conditions.

The motivation to implement a measurable tool to quantify neurological function has arisen as a consequence of the obvious lack of such a measurement in the RANO criteria and the well-known clinical observation that neurological and neuroradiological changes do not always correlate, eg, in cases of asymptomatic progression or treatment-related neurological decline/improvement. It has previously been demonstrated that neurological and neurocognitive decline sometimes precede radiological deterioration.^23–26 This, in turn, might be explained by the development of perifocal brain edema, non-contrast enhancing tumor progression or side effects of drug treatment.²⁷ These scenarios must be taken into account especially in the context of anti-angiogenetic therapies. Bevacizumab, for instance, has been associated with a decline in neurocognitive function despite prolonging progression-free survival.^28,29 Among other aspects, this underlines the importance of clinical outcome assessments in drug development, which should focus not only on progression-free or overall survival but also on neurological function and quality of life.

Limitations of this study have to be addressed, including its retrospective design and small sample size which prohibited a meaningful stratification of patients according to the WHO grading system or treatment modality. Additionally, it is important to note that the average follow-up period of 1.4 years in our heterogenic cohort of glioma patients might be too short to observe clinically relevant neurological changes, especially in patients with lower-grade gliomas. This might be reflected in the low number of NIHSS transitions rates of 19%. The use of corticosteroids is part of the RANO criteria and therefore implemented in the analysis. Therefore, doses and duration of steroid application were not separately analyzed. However, the RANO working group recently proposed new criteria to assess corticosteroid use in clinical trials, which might complement the multimodal assessment of glioma patients in future clinical practice.³⁰

Recently, the NANO working group proposed a novel scale to measure the clinical performance of patients with brain tumors.³¹ A retrospective study recently demonstrated that the initial NANO score correlated with survival in glioblastoma patients.³² The NANO score evaluates 9 domains of neurologic function that are most relevant to patients with gliomas. Since the introduction and validation of the NANO score, we have replaced the NIHSS with the NANO score in our clinical practice. This disease-specific clinical tool certainly represents an important step towards a uniform quantification of neurological outcomes in patients with brain tumors. In future studies, we therefore aim to investigate the relationships between NANO scores with RANO scores in larger patient cohort.

Conclusion

The application of the NIHSS is easy and feasible in the heterogeneous population of glioma patients. In our cohort, the RANO imaging assessment did not correspond to contemporaneous changes of NIHSS scores, which is partly explained by the rough thresholds of the RANO criteria. Comprehensive measures of neurological function, quality of life, seizures, and neurocognition specifically developed for patients with brain tumors are currently being assessed and will require the definition of a well-defined MCID to facilitate the interpretation of treatment response especially in clinical trials of treatment in this complex patient group.

Funding

The authors received no funding.

Conflict of interest statement. The authors declare that they have no conflict of interest.

Ethics approval. All procedures performed in studies involving human participants were in accordance with the ethical standards of the regional research committee (approved by the Local Ethical Committee, No. EKO/2019-02111) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.