Learning ability correlates with brain atrophy and disability progression in RRMS

Abstract

Objective

To assess the prognostic value of practice effect on Paced Auditory Serial Addition Test (PASAT) in multiple sclerosis.

Methods

We compared screening (day −14) and baseline (day 0) PASAT scores of 1009 patients from the FTY720 Research Evaluating Effects of Daily Oral therapy in Multiple Sclerosis (FREEDOMS) trial. We grouped patients into high and low learners if their PASAT score change was above or below the median change in their screening PASAT quartile group. We used Wilcoxon test to compare baseline disease characteristics between high and low learners, and multiple regression models to assess the respective impact of learning ability, baseline normalised brain volume and treatment on brain volume loss and 6-month confirmed disability progression over 2 years.

Results

The mean PASAT score at screening was 45.38, increasing on average by 3.18 from day −14 to day 0. High learners were younger (p=0.003), had lower Expanded Disability Status Scale score (p=0.031), higher brain volume (p<0.001) and lower T2 lesion volume (p=0.009) at baseline. Learning status was not significantly associated with disability progression (HR=0.953, p=0.779), when adjusting for baseline normalised brain volume, screening PASAT score and treatment arm. However, the effect of fingolimod on disability progression was more pronounced in high learners (HR=0.396, p<0.001) than in low learners (HR=0.798, p=0.351; p for interaction=0.05). Brain volume loss at month 24 tended to be higher in low learners (0.17%, p=0.058), after adjusting for the same covariates.

Conclusions

Short-term practice effects on PASAT are related to brain volume, disease severity and age and have clinically meaningful prognostic implications. High learners benefited more from fingolimod treatment.

Introduction

Multiple sclerosis (MS) is a chronic and debilitating disease affecting mainly adults early in their life. It is a progressive disease characterised by the formation of lesions in the brain and spinal cord. Symptoms of the disease are widespread and largely dependent on the location of the lesions and the extent of inflammatory and degenerative pathology within the central nervous system. The majority of patients manifest with a relapsing-remitting disease course but progress to a more progressive and degenerative stage of the disease within a decade.¹ In addition to motor symptoms, cognitive impairment is prevalent in all stages of MS,^{2 3} including early relapsing-remitting disease and clinically isolated syndromes.^4–7 The cognitive functions preferentially affected in MS include attention, information processing efficiency and speed, executive functioning, and long-term memory.³ Over the last two decades, the Paced Auditory Serial Addition Test (PASAT), as part of the Multiple Sclerosis Functional Composite score, has been the most commonly used test to evaluate attentional function in MS clinical trials.^8–10 However, at repeated administrations, a significant practice effect has been reported resulting in increased mean PASAT scores in both the active and control groups over time.^{11 12} This practice effect makes interpretation of true treatment effects difficult^{10 13} and accentuates the ceiling effect in patients with higher baseline scores who anyway have limited room for further improvement in contrast to those with low baseline scores who can further improve at subsequent visits.¹³ Nevertheless, PASAT scores correlate with brain volume (BV) cross-sectionally.¹⁴ Brain volume loss (BVL) is increasingly recognised as a predictor of disability progression,^15–17 but interpretation of the longitudinal association to PASAT scores is complicated by the observed practice effects. Recent studies have suggested that higher BV and cognitive reserve can be buffers to accelerated disability progression.^{18 19} We investigated the relation of the PASAT practice effect as a measure of learning ability and cognitive reserve with other disease characteristics, and determined whether this practice effect—as a dynamic measure—can inform prognosis of patients with MS.

In patients who participated in the FTY720 Research Evaluating Effects of Daily Oral therapy in Multiple Sclerosis (FREEDOMS) trial comparing fingolimod and placebo, we (1) assessed and quantified learning ability as expressed by change in PASAT score after short-term repeated administration accounting for baseline cognitive performance; (2) investigated the correlation of practice effect with baseline MS disease characteristics; and (3) determined whether the practice effect can predict on-study BVL, future disability worsening and treatment effect.

Methods

Study design and patients

This is a post-hoc analysis of data from the FREEDOMS²⁰ study (n=1009, complete data set), a phase III, randomised, placebo-controlled study of fingolimod in patients with relapsing-remitting multiple sclerosis (RRMS). The study design and inclusion/exclusion criteria of the FREEDOMS study have been previously described.²⁰ In brief, the study included patients diagnosed with RRMS (according to the revised McDonald criteria²¹) aged 18–55 years who had a score of 0–5.5 on the Expanded Disability Status Scale (EDSS), and one or more documented relapses in the past year, or two or more relapses in the past 2 years prior to randomisation. Patients were to be neurologically stable with no evidence of relapse or corticosteroid treatment within 30 days prior to randomisation. Eligible patients were randomised (1:1:1) to receive either fingolimod 0.5 or 1.25 mg once daily or placebo for 2 years.²⁰ In all the analyses the two treatment arms are pooled together.

Assessments

Standardised neurological assessments, including determination of EDSS score, were performed at baseline and every 3 months. MRI scans were obtained at baseline and months 6, 12 and 24 or at the end of study in patients prematurely discontinuing the study. MRI lesion activity and BVL were assessed at a central reading site (Medical Image Analysis Center, Basel, Switzerland) that remained blinded for clinical data and randomisation. Normalised brain volume (NBV) was assessed at baseline using ‘Structural Image Evaluation, using Normalization, of Atrophy, Cross-sectional’ (SIENAX). PASAT was administered at screening (day −14; two trials) and at baseline (day 0; one test). Per cent brain volume change (PBVC) between baseline and month 24 was determined using ‘Structural Image Evaluation, using Normalization, of Atrophy’ (SIENA). Disability worsening was defined as a ≥1-point increase in the EDSS score sustained for ≥6 months (1.5 points if baseline EDSS=0).

Statistical analysis

Patients were divided into four groups by quartiles of the PASAT score at screening (day −14; average of two trials), and the median value of PASAT change for each quartile was calculated. In each of these four groups, patients were further defined as ‘high learners’ if they had a PASAT score improvement above their group’s median, or ‘low learners’ if they had a PASAT score change below the median of their quartile group. To mitigate ceiling effects patients with screening PASAT scores of 60 (maximum value) and no change at baseline were counted as ‘high learners’. By defining learning ability within the screening quartile groups, we could take into account the fact that patients with higher screening scores have less room for further improvement (ceiling effect) compared with those who started with a low screening score.

The Wilcoxon rank-sum test was used to compare different baseline characteristics (age, clinical disease burden (EDSS), T2 lesion volume (T2LV) and NBV) between high and low learners. A multivariate logistic regression model with a forward selection was run to select factors independently associated with learning ability. Pearson’s correlation (r) was used to measure the correlations between PASAT score at screening and change in PASAT score with baseline age, EDSS, disease duration, T2 lesion load and NBV. A Cox proportional hazard model was used to assess if there was a difference in the risk of 6-month confirmed disability progression (CDP) over 2 years between high and low learners after adjusting for fingolimod treatment, NBV and screening PASAT score. The Cox model was used to assess if treatment effect differed based on the high versus low learner status, by including a treatment-by-learning status interaction in the model. Finally, a linear regression model was used to determine whether learning ability between screening and baseline is predictive of the rate of PBVC over 2 years, after adjusting for fingolimod treatment, NBV and screening PASAT score.

Results

PASAT practice effects

Overall, a mean improvement of 3.18 (SD=6.44) was noted in the PASAT score from screening (45.38 (11.24), day −14) to baseline (48.56 (11.07), day 0), compatible with the well-known practice effect (figure 1A). The median change in PASAT score from screening to baseline was 5.0 for Q1, 3.5 for Q2 and Q3, and 0.5 for Q4 (figure 1B). A total of 513 patients were included in the ‘high-learners’ group and 496 in the ‘low-learners’ group (figure 1B).

Figure 1.

Paced Auditory Serial Addition Test (PASAT) score change from screening to baseline: practice effect. Patients with baseline PASAT score of 60 (n=12) who did not change between screening and baseline were classified as ’High learners’. Panel A shows mean PASAT change, Panel B shows screening PASAT quartiles and, Panel C shows correlation of PASAT screening score with practice effect.

Correlations of baseline findings and practice effect

The baseline characteristics of the total analysis cohort by learning ability are reported in table 1.

Table 1.

Baseline characteristics

	Total	High learners	Low learners	P values
	N=1009	n=513	n=496
Age, years	37.07 (8.72)	36.27 (8.90)	37.9 (8.45)	0.003
Disease duration since first symptoms	8.1 (6.6)	7.8 (6.3)	8.4 (6.87)	0.256
NBV, cm3	1517.24 (84.02)	1527.45 (81.02)	1506.68 (85.82)	<0.0001
EDSS score	2.35 (1.28)	2.24 (1.21)	2.46 (1.35)	0.031
T2LV, mm3	6253.67 (7653.41)	5568.68 (6987.82)	6962.13 (8233.09)	0.009

Data presented as mean (SD), unless specified otherwise.

EDSS, Expanded Disability Status Scale; NBV, normalised brain volume; T2LV, T2 lesion volume.

The PASAT score at screening was positively correlated with baseline PASAT score (r=0.83, p<0.0001) and NBV (r=0.26, p<0.0001), and was inversely correlated with PASAT change/practice effect (r=−0.31, p<0.0001; figure 1C). No correlation was found between NBV and unadjusted PASAT change (r=0.02, p=0.48).

High learners were younger than low learners (p=0.003) and had higher NBV (p<0.001; table 1). Moreover, high learners had lower clinical (EDSS score, p=0.03) and MRI (T2LV, p=0.009) disease burden at baseline (table 1). The multivariate analysis showed that NBV has the strongest association with learning ability: the OR=1.35 (p<0.001) indicated a 35% increase in the probability to be a high learner for every increase of NBV by 100 mm³.

Disease outcomes and treatment effect by practice effect

The PASAT score at screening, treatment and NBV were all associated with the probability of 6-month CDP in the univariate analysis, but when included in a multivariate model only treatment with fingolimod (HR 0.56, p<0.001 vs placebo) and NBV (HR 0.76, p=0.006) were independently associated with a decreased risk of 6-month CDP (table 2). The risk of 6-month CDP was not significantly affected by the learning ability. However, when a model with a treatment-by-learner status interaction was applied, the interaction term was significant (p=0.05), indicating that the fingolimod treatment effect is stronger in high learners (60% risk reduction, HR=0.40, p<0.001) than in low learners (21% risk reduction, HR=0.79, p=0.35; figure 2).

Table 2.

Univariate and multivariate analyses assessing the effects of baseline variables on 6-month CDP at 2 years

Covariate	Unit	Univariate analysis		Multivariate analysis
		HR (95% CI)	P values	HR (95% CI)	P values
Treatment (fingolimod vs placebo)	–	0.55 (0.39 to 0.76)	<0.001	0.56 (0.40 to 0.78)	0.001
NBV	100 mm3	0.72 (0.60 to 0.87)	0.001	0.76 (0.62 to 0.92)	0.006
Screening PASAT score	1 point	0.98 (0.97 to 0.99)	0.015	0.99 (0.98 to 1.01)	0.15
Learning ability (high vs low)	–	0.88 (0.64 to 1.23)	0.46	0.95 (0.68 to 1.33)	0.78

CDP, confirmed disability progression; NBV, normalised brain volume; PASAT, Paced Auditory Serial Addition Test.

Figure 2.

Differential treatment effect on risk of 6-month confirmed disability progression in low versus high learners. Treatment by learners interaction p=0.05.

PBVC over 2 years was lower in patients treated with fingolimod (PBVC=−0.87%) than those treated with placebo (PBVC=−1.28%; difference −0.39%, p<0.001; table 3). It was also lower in patients with higher NBV values at baseline (p=0.006) and higher screening PASAT scores (p=0.038) (table 3). Patients from the high-learner group experienced less PBVL than those from the low-learner group (p=0.01, table 3). Treatment, NBV, screening PASAT score and learning ability all correlated with PBVL in both the univariate and multivariate analyses. Being in the high PASAT learner group was therefore independently associated with a lower PBVL throughout the study period, and even after adjusting for screening PASAT score and baseline NBV the difference (−0.17%) remained borderline significant (p=0.058; figure 3).

Table 3.

Univariate and multivariate analyses assessing the effects of baseline variables on 2-year PBVC

Covariate	Unit	Univariate analysis		Multivariate analysis
		Beta*	P values	Beta*	P values
Treatment (fingolimod vs placebo)	–	0.41 (0.22–0.59)	<0.001	0.39 (0.21–0.57)	0.001
NBV	100 mm3	0.34 (0.24–0.45)	<0.001	0.30 (0.19–0.40)	0.006
Screening PASAT score	1 point	0.015 (0.007–0.023)	<0.001	0.008 (0.001–0.016)	0.038
Learning ability (high vs low)	–	0.23 (0.06–0.41)	0.01	0.17 (0.006–0.34)	0.058

*Beta: PBVC change associated with the 1-unit change of the independent variable.

NBV, normalised brain volume; PASAT, Paced Auditory Serial Addition Test; PBVC, per cent brain volume change.

Figure 3.

Brain volume loss in low versus high learners (per treatment group). PBVC, per cent brain volume change.

Discussion

The practice effect seen with PASAT is usually considered a limitation of this test, because it can potentially confound the interpretation of a treatment effect on cognitive performance in clinical trials.²² Because this effect is most pronounced in the first repetitions of the test, most studies require to repeat the test at least two to three times before treatment start. However, the improvement in performance over time observed at repeated administrations is reflective of active learning.^{13 23} The driving hypothesis of our study was that this learning effect induced by repeated testing, besides being an interfering variable, would by itself be a valuable measure of cognitive performance. To date the potential clinical relevance and implications of this learning effect and the association of practice effect and learning ability with disease outcomes have been poorly investigated. A study in patients with clinically isolated syndrome on disease-modifying therapy assessed the temporal association between MRI measures and PASAT performance and reported higher PASAT learning effects in patients with lower rates of grey matter atrophy.²⁴

Based on a large, randomised controlled study, we confirm the existence of a significant practice effect on PASAT scores¹³; patients’ cognitive performance improved on average when the test was repeated 14 days after the first PASAT trial. Patients with high PASAT scores at screening exhibited lower PASAT changes compared with those with a low screening PASAT score. This inverse correlation indicates the need to adjust the PASAT change for screening PASAT scores in order to correctly identify the patients who show improvements at baseline. Assessing the change in PASAT score, as compared with the median screening PASAT score within a quartile, allowed to quantify the learning ability as a measure of cognitive reserve adjusting for the impact of baseline performance.

Patient baseline demographic and disease characteristics are also associated with cognitive performance as measured by PASAT. High PASAT scores at baseline corresponded to younger patients who also had high NBV and lower disease burden at baseline. Thus, the learning ability of patients with RRMS, as measured using the practice effect on PASAT, is reflective of age and MS disease severity (EDSS score, BV and T2LV) at baseline. It is known that performance in PASAT is influenced by intelligence and mathematical ability,^{13 25} which could contribute to the ceiling effect. Also, comorbidities such as depression and/or anxiety¹³ and occurrence of a recent MS relapse may affect the cognitive status of the patient. However, it is difficult to measure the impact of these variables, and therefore adjustment for these factors is difficult to achieve. The FREEDOMS trial did not include patients with pre-existing serious psychiatric conditions. Although assessment of neurological disability (changes in functional system and EDSS score) was performed to confirm a relapse, neurocognitive status was not evaluated at the time or immediately after the relapse. Deficits in information processing speed and working memory is the most prominent aspect of cognitive impairment in patients with MS. The association of cognitive status and disability progression as assessed by change in EDSS score is not clearly reported in the literature, and is further complicated by the fact that there is lack of a standard tool to measure cognitive impairment. It has been reported that patients with cognitive impairment at baseline had worsening EDSS score^{26 27} and were at significantly higher risk of reaching EDSS >4 as compared with cognitively preserved patients.²⁸ However, some studies suggest that cognitive decline is better correlated with brain atrophy measures, particularly deep grey matter atrophy, than EDSS progression.^{29 30}

It has been previously reported that treatment effect on brain atrophy correlates with the effect on disability progression over 2 years and predicts future disability progression independent of the effect on active MRI lesions in patients with RRMS.³¹ Furthermore, a correlation between treatment effect of fingolimod on BVL and its corresponding effect on future disability progression has also been reported.³² In patients treated with fingolimod, 2-year disability progression was associated with the rate of BVL over 2 years, and treatment effect of fingolimod on PBVC was predictive of future disability.³² We now report the association of fingolimod treatment effect on disability progression based on the learning ability of patients. In both high and low learners, treatment with fingolimod reduced the risk of disability progression, with stronger effects in high learners. After adjusting for the practice effect, high learners showed better prognosis in terms of both BVL and CDP over 2 years. After accounting for the treatment effect of fingolimod and baseline NBV, PASAT and learning ability had however no additional explanatory value for the prediction of disability progression in patients with RRMS. The lack of correlation between unadjusted PASAT change and NBV was due to the dependence of PASAT change on the screening value. This supports the importance and stronger relevance of NBV (also called as brain reserve) in predicting disability progression in patients with RRMS compared with PASAT. Our data contradict the findings of Raghupathi et al ³³ that stipulate PASAT as an independent predictor of disability progression in an integrated analysis including data from the AFFIRM, DEFINE, CONFIRM and ADVANCE trials. Others have reported that higher brain reserve attenuates the impact of disease on cognitive efficiency, as measured by performance in PASAT.^{34 35} Our results, along with the findings from these previous reports, further suggest that both brain reserve and cognitive reserve protect against disease-related disability and cognitive decline. Interestingly, baseline PASAT score, learning ability and treatment were independent predictors of BVL.

In conclusion, our results suggest that greater cognitive reserve, as indicated by high baseline PASAT score, is a good prognostic indicator and that a greater ability to learn is associated with better treatment response and better disease outcomes in the future.