Abstract
Objective:
To determine whether Ginkgo biloba extract (ginkgo) improves cognitive function in persons with multiple sclerosis (MS).
Methods:
Persons with MS from the Seattle and Portland VA clinics and adjacent communities who scored 1 SD or more below the mean on one of 4 neuropsychological tests (Stroop Test, California Verbal Learning Test II [CVLT-II], Controlled Oral Word Association Test [COWAT], and Paced Auditory Serial Addition Task [PASAT]) were randomly assigned to receive either one 120-mg tablet of ginkgo (EGb-761; Willmar Schwabe GmbH & Co, Germany) or one placebo tablet twice a day for 12 weeks. As the primary outcome, we compared the performance of the 2 groups on the 4 tests at exit after adjusting for baseline performance.
Results:
Fifty-nine subjects received placebo and 61 received ginkgo; 1 participant receiving placebo and 3 receiving ginkgo were lost to follow-up. Two serious adverse events (AEs) (myocardial infarction and severe depression) believed to be unrelated to the treatment occurred in the ginkgo group; otherwise, there were no significant differences in AEs. The differences (ginkgo − placebo) at exit in the z scores for the cognitive tests were as follows: PASAT −0.2 (95% confidence interval [CI] −0.5 to 0.1); Stroop Test −0.5 (95% CI −0.9 to −0.1); COWAT 0.0 (95% CI −0.2 to 0.3); and CVLT-II 0.0 (95% CI −0.3 to 0.3); none was statistically significant.
Conclusions:
Treatment with ginkgo 120 mg twice a day did not improve cognitive performance in persons with MS.
Classification of evidence:
This study provides Class I evidence that treatment with ginkgo 120 mg twice a day for 12 weeks does not improve cognitive performance in people with MS.
Cognitive impairment (CI) affects 40%1 to 60%2 of people with multiple sclerosis (MS), most commonly affecting their processing speed, memory, and executive skills.1–3
The etiology of CI in MS is incompletely understood but appears to be related to the extent of damage to the brain overlaid on each individual's cognitive reserve.4 Dysfunction in neurotransmitter systems may contribute to CI in MS. The noradrenergic system appears to be important because methylphenidate and amphetamine improve attention and memory in MS.5–8 A cholinergic imbalance occurs in the hippocampus of people with MS,9 but treatment with donepezil, a cholinesterase inhibitor, failed to improve memory in subjects with MS.10 People with MS show increased brain glutamate11 and altered expression of glutamate receptors and antiporters.12,13 However, memantine, an NMDA receptor antagonist, failed to improve CI in people with MS14 and was poorly tolerated.14,15
Ginkgo biloba (ginkgo) contains ginkgolides that inhibit platelet-activating factor (PAF). PAF is an inflammatory mediator released by immune cells and also a neurotransmitter released by postsynaptic terminals. PAF modulates presynaptic glutamate release,16 and, thus, PAF antagonists could modulate glutamate excitotoxicity and improve cognition. The results of studies with ginkgo in Alzheimer disease (AD)17 and its frequent use by people with MS18 led us to conduct a pilot study in people with MS and CI, which showed that ginkgo improved their performance on the Stroop Test.19 The pilot study results led us to carry out a trial to more definitively establish whether or not ginkgo could improve CI in subjects with MS.
METHODS
We conducted a randomized placebo-controlled trial comparing the effects of treatment with ginkgo 120 mg twice a day (EGb-761; Willmar Schwabe GmbH & Co, Germany) or placebo for 12 weeks on cognitive performance in people with MS. The full protocol can be accessed at http://www.ohsu.edu/ms/ginkgoprotocol. Our ginkgo tablets contained 29.7 mg of flavoglycosides and 7.3 mg of terpene lactones measured by high-performance liquid chromatography.
Our primary hypothesis was that ginkgo would improve cognitive function in people with MS. We tested this hypothesis, providing Class I evidence by comparing the 2 groups on their performance at exit adjusted for baseline performance on the 4 neuropsychological tests (Stroop Test, California Verbal Learning Test II [CVLT-II; long-delay free-recall score], Controlled Oral Word Association Test [COWAT], and the 2-second version of the Paced Auditory Serial Addition Task [PASAT]). We used multivariate analysis of covariance using the exit z scores on the 4 tests as the dependent variables and baseline z score results on the 4 tests as covariates as the primary outcome; if results of the multivariate test were significant, then individual analysis of covariance was conducted for each of the tests using the Bonferroni correction.
Our secondary hypotheses were that 1) ginkgo would be safe, 2) ginkgo would result in an improvement in cognition that subjects and their families would notice, and 3) ginkgo would be associated with improved social integration. A hierarchical approach was used to consider the secondary outcomes significant (see figure e-1 on the Neurology® Web site at www.neurology.org for details).
Subjects.
Persons with MS by McDonald's criteria20 (Expanded Disability Status Scale [EDSS] score 0−7.5) who were stable for at least 30 days, scored 1 SD or more below the mean of established normative samples on one or more of the 4 tests in the neuropsychological test battery, and did not have contraindications for ginkgo or problems that would interfere with testing were eligible for the study (table e-1). The study was performed at 2 sites, the Portland and Seattle VA Medical Centers. We recruited from both sites' MS clinics, from their affiliated university's MS clinics, and from their communities. Enrollment started on January 2009 and ended in November 2010.
Standard protocol approvals, registrations, and patient consents.
The institutional review board at each site approved the study, all participants provided written informed consent, and we registered the study with clinicaltrials.gov (NCT00841321).
Randomization and masking.
The Portland VA research pharmacist generated the allocation sequence in Excel with alternating random blocks of 4 or 6 subjects stratified by disease duration and study site and a 1:1 ratio. Only the research pharmacists had access to the allocation sequence. They assigned the participants to either ginkgo or matching placebo tablets and dispensed the tablets to them in matching identical containers. One research assistant assessed adverse events and another administered the neuropsychological tests. They were instructed not to discuss adverse events (AEs) between them to ensure that the cognitive test assessments would be masked.
Outcome measures.
Cognitive tests.
The cognitive battery consisted of the Stroop Color-Word Test−Victoria version, the CVLT-II, the COWAT, and the 2-second version of the Paced Auditory Serial Addition Task (PASAT). z Scores based on healthy, age-matched control norms for the time on the Color-Word portion of the Stroop Test,21 COWAT,21 long-delay free-recall portion of the CVLT-II,22 and PASAT1,23 were used for the analysis of the primary outcome (multivariate test).
These tests take less than 30 minutes to administer, measure domains frequently impaired in MS, and are not highly correlated. The 2-second version of the PASAT was used as opposed to the 3-second version to diminish ceiling effects. The protocol (http://www.ohsu.edu/ms/ginkgoprotocol) contains detailed descriptions of the tests and administration method.
Subject's and subject's family member reports.
Secondary outcomes included the following: the Perceived Deficits Questionnaire (PDQ), a standardized questionnaire in which the subject reports on his or her cognitive function24; the Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNSQ),25 in which the family member who was most aware of the participant's cognitive deficits reports on the subject's cognitive deficits; and the Community Integration Questionnaire (CIQ),26 in which the subject reports his or her degree of social integration.
Exploratory outcomes included changes from baseline to exit on the Modified Fatigue Impact Scale (MFIS)27 and the Beck Depression Inventory-II (BDI-II).28
Safety.
We called subjects by phone monthly and evaluated them in the clinic if necessary. We then coded the AEs according to the Common Terminology Criteria for Adverse Events version 4 and compared the AE incidence between the 2 groups.
Relapses.
Relapses were recorded as AEs and were analyzed separately from other AEs as part of the safety analysis. Relapses were defined as the occurrence of a new neurologic symptom or worsening of an old one, with an objective change of at least 1 point in the functional system scale score from the EDSS, which lasted at least 24 hours, without fever, and which followed a period of clinical stability or of improvement of at least 30 days. All patients with relapses were assessed in the study center within 7 days of the relapse for objective confirmation by one of the study physicians.
Statistical analyses and sample size.
Figure e-1 shows the flowchart for the analyses including covariates used as specified before the start of the study. All analyses were intent to treat. For subjects with missing exit values, the baseline measures were carried forward. Based on our pilot study, we estimated needing 110 subjects with complete observations to have 80% power to detect a difference as low as 0.5 on the z scores on any of the tests at both the multivariate level and the univariate levels at an α level of 0.05 (two-sided). Assuming 80% compliance as seen in our pilot study and a dropout rate of 30%, we predicted needing to recruit 158 subjects.
RESULTS
We stopped enrollment after 121 participants had been enrolled because the dropout rate was less than 5% and more than 110 subjects had completed the study. Figure 1 shows the CONSORT flow chart. As shown in table 1, the randomization balanced the groups well except for gender. Pill counts indicated that 88% of subjects took 80% or more of their treatment.
Table 1.
Abbreviations: DMT = disease-modifying therapy; EDSS = Expanded Disability Status Scale; GA = glatiramer acetate; PPMS = primary progressive multiple sclerosis; PRMS = progressive-relapsing multiple sclerosis; RRMS = relapsing-remitting multiple sclerosis; SPMS = secondary progressive multiple sclerosis.
There were 2 serious adverse events and both occurred in the ginkgo group. One subject had a myocardial infarction and another was hospitalized for depression. Neither adverse event was considered to be related to treatment. There were otherwise no significant differences in AEs between the 2 groups (table e-2). Three participants in the ginkgo group and one in the placebo group were lost to follow-up. They were included in the analyses as planned by carrying forward their baseline data to the follow-up time point.
There were no significant differences between the 2 groups at baseline on the 4 neuropsychological tests and no significant differences at exit on the multivariate tests. Table e-3 shows the distribution of the baseline z scores for each of the cognitive tests included in the battery, which were also well balanced between the 2 groups. Table 2 shows the 95% confidence intervals for the differences between groups on each test. On the univariate tests, the placebo group performed better than the ginkgo group on the Stroop Test, but this difference is not considered significant because the results of the multivariate test were not significant.
Table 2.
Abbreviations: COWAT = Controlled Word Association Test; CVLT-II = California Verbal Learning Test II; PASAT = Paced Auditory Serial Addition Test.
Least-squares means adjusted for baseline performance; larger values indicate better performance. Positive values indicate a beneficial effect from ginkgo.
Post hoc subgroup analyses limited to participants scoring less than 1 SD below the mean on each test showed no significant differences between the groups; these are presented in table e-4. Post hoc subgroup analyses comparing different types of MS showed no differential effect of ginkgo (table e-5). Post hoc analysis of the primary outcome limited to the completers and with no Bonferroni correction did not show any significant differences.
Neither the subjects nor the study coordinators administering the tests were correct in determining participants' treatment assignment more than expected by chance (binomial test for proportion >0.5, all p > 0.1).
Both groups had a similar incidence of MS relapses (n = 2 for the ginkgo treatment group; n = 3 for the placebo group) and changes in EDSS scores. There were no significant differences between the 2 groups on perceived cognitive deficits (PDQ and MSNSQ), fatigue (MFIS), depression (BDI-II), or social activities (CIQ) (table e-6).
DISCUSSION
In our study, treatment with ginkgo did not result in any improvement in cognitive performance. Our study had sufficient power to detect effects thought to be clinically significant. Smaller effects could have been missed, but the confidence intervals suggest that they would be relatively small (at most 0.3 SD for the corresponding normative data). We did not observe an interaction between baseline performance and treatment effects that could suggest a benefit restricted to those who were more severely impaired. Ginkgo was well tolerated; the 2 serious AEs were isolated and were not previously seen in trials with ginkgo.
Our study had several methodologic strengths. We used a high-quality product with a standardized concentration of active compounds. Our subjects were very motivated to complete the trial and compliant with the protocol, and we had very few subjects lost to follow-up. Our primary outcome analyses were intent-to-treat, and we included susceptibility analyses limited to the completers, all of which showed no significant differences.
There are some limitations to this study. Minorities were underrepresented, limiting the generalizability of our results mainly to Caucasians. Subjects received ginkgo for only 12 weeks. Although we believe this was long enough to detect symptomatic improvement in cognitive performance, it was not long enough to determine whether ginkgo had any disease-modifying effects. Our subjects had a long disease duration (median 20 years), and, thus, it is possible that Ginkgo may improve cognitive performance early in the disease process. Our sample had mild impairment overall and mainly were impaired on the PASAT and mostly below average on the CVLT-II. Although we did not see interactions with baseline performance or in the post hoc subgroup analyses to suggest so, it is possible we could have seen a positive effect if we had recruited a sample of subjects more impaired on attention and verbal fluency tests or with impairment in verbal learning.
Our current study did not show a significant effect on cognitive performance, whereas our pilot study did show an effect restricted to the Stroop Test. In our pilot study, we enrolled 43 subjects with MS who scored between 0.5 and 2.5 SDs below the mean on the PASAT or the CVLT-II total score. Thus, very impaired subjects in these 2 tests were excluded in our pilot study, whereas they were eligible for our current study. It is unlikely that this change in classification accounted for our findings because we did not see any differences in the post hoc analyses that restricted the sample to only impaired subjects. The cognitive test battery used in the pilot study included the same tests used in this study plus the Symbol Digits Modalities Test and the Useful Field of View Test, neither of which showed a difference in the pilot study, so the difference in the batteries is not a likely explanation for the different outcomes.
Subjects in our pilot study were randomly assigned to receive either placebo or the same dose of ginkgo as used in our current study and the duration of treatment was the same; however, the manufacturer of the products used in our pilot and current studies were different. Both products had similar concentrations of the active compounds, so it is unlikely that a difference in the products was responsible for the different outcomes.
A more likely explanation for the difference in findings between the pilot and the present study is that there is really no effect and the pilot study results were due to chance. With a p level of 0.05, 1 in 20 trials will produce false-positive results. In addition, the sample size of our pilot study was relatively small, and we used parametric statistics in the analyses. The effect in the pilot study was limited to the Stroop Test and was much more prominent in participants with the greatest impairment. The Stroop Test does not have a normal distribution, and, thus, the results of the pilot study might have been more influenced by extreme observations, whereas the effects of outliers are smaller when a larger sample is used, as in the current study.
In our pilot study, we had 2 practice sessions before the baseline assessment to minimize the effects of practice. In the analysis of the pilot data, we noted there was still some residual practice effect between the third (baseline) and fourth (exit) visits because the placebo group continued to improve. In addition, the performance on the CVLT and PASAT in the pilot approached the ceiling for these measures, limiting our ability to see an effect. For these reasons we chose a harder version of the PASAT and omitted the practice visits. It is possible that the elimination of the practice visits could account for the different results.
Our test battery was limited, and it is certainly possible that we did not include a test for which ginkgo would have an effect. However, we assessed the most important cognitive domains that are affected in MS. A larger battery might have been more informative but would have also required a larger sample size.
We included self-report measures that assessed the reports of the subject's cognitive function from both the participant's and caregiver's perspectives and that did not show any treatment effect. It is possible that additional functional assessments that measure performance in real-life situations could have detected an effect that we missed by limiting the outcome measures to cognitive tests and questionnaires.
This study represents the third large double-blind placebo-controlled trial of a therapy thought be effective in AD that has failed to improve cognitive performance in MS. Donepezil, memantine, and now ginkgo all have Class I data indicating that they are ineffective in improving cognitive performance in MS.5,6 In AD, donepezil is believed to improve cognitive impairment by increasing acetylcholine, whereas memantine works by antagonizing glutamate effects on the NMDA receptors in a use-dependent manner. Ginkgo antagonizes PAF and thus may have an effect similar to that of memantine on glutamate neurotransmission. Based on the negative results of these 3 studies, it is tempting to hypothesize that the biochemical underpinnings of cognitive dysfunction in AD and MS differ and neither an acetylcholine deficit nor an excess glutamate may be critical to cognitive dysfunction in MS. Better understanding of the pathophysiology of cognitive dysfunction in MS is needed to rationally design therapeutic strategies for this significant complication of MS.
Supplementary Material
ACKNOWLEDGMENT
The authors thank Ruth Whitham, MD, Cynthia Morris, PhD, MPH, and Joseph Quinn, MD, for their contribution to the monitoring of the study as part of the Data Safety and Monitoring Board, Susanne Kraft for her assistance with the investigational new drug application, and Willmar Schwabe GmbH & Co for providing us with ginkgo and placebo.
GLOSSARY
- AD
Alzheimer disease
- AE
adverse event
- BDI-II
Beck Depression Inventory II
- CI
cognitive impairment
- COWAT
Controlled Word Association Test
- CVLT-II
California Verbal Learning Test II
- EDSS
Expanded Disability Status Scale
- MFIS
Modified Fatigue Impact Scale
- MS
multiple sclerosis
- MSNSQ
Multiple Sclerosis Neuropsychological Screening Questionnaire
- PAF
platelet-activating factor
- PASAT
Paced Auditory Serial Addition Test
- PDQ
Perceived Deficits Questionnaire
Footnotes
Supplemental data at www.neurology.org
AUTHOR CONTRIBUTIONS
J. Lovera: drafting/revising the manuscript for content, including medical writing for content, study concept or design, analysis or interpretation of data, statistical analysis, obtaining funding. E. Kim: drafting/revising the manuscript for content, including medical writing for content, acquisition of data, study supervision. E. Heriza: drafting/revising the manuscript for content, including medical writing for content, acquisition of data, study coordination. M. Fitzpatrick: drafting/revising the manuscript for content, acquisition of data. J. Hunziker: drafting/revising the manuscript for content, acquisition of data, study coordination. A. Turner: drafting/revising the manuscript for content, study concept or design. J. Adams: drafting/revising the manuscript for content, acquisition of data, study coordination. T. Stover: drafting/revising the manuscript for content, acquisition of data, study coordination. A. Sangeorzan: drafting/revising the manuscript for content, acquisition of data, study coordination. A. Sloan: drafting/revising the manuscript for content, acquisition of data, study coordination. D. Howieson: drafting/revising the manuscript for content, study concept or design. K. Wild: drafting/revising the manuscript for content, study concept or design. J. Haselkorn: drafting/revising the manuscript for content, including medical writing for content, obtaining funding. D. Bourdette: drafting/revising the manuscript for content, including medical writing for content, study concept or design, analysis or interpretation of data, obtaining funding.
DISCLOSURE
J. Lovera is funded by NIH NCCAM K23AT004433 and has received compensation from EMD Serono and TEVA Pharmaceuticals for conducting educational activities for physicians. He is on the speakers' bureau for Biogen Idec and EMD Serono. E. Kim has received honoraria from Biogen Idec for conducting educational activities for physicians. E. Heriza, M. Fitzpatrick, and J. Hunziker report no disclosures. A. Turner is funded by a Rehabilitation Research and Development Service Career Development Award (B4927W) and by VA RR&D I01 RX000223–01A1. J. Adams, T. Stover, and A. Sangeorzan report no disclosures. A. Sloan received travel expenses from Paralyzed Veterans of America and Consortium of MS Centers for serving as faculty in conferences. D. Howieson is funded by NSF BCS 0826654 and NIH/NIA P30 AG008017 grants. K. Wild is funded by NIH/NIA grants P30 AG008017 and R01 AG024059. J. Haselkorn received compensation from Paralyzed Veterans of America for travel to their Education Planning Committee and to the MS Summit conference; she is funded by VA Merit Review Grants 11–2705, 05–2706, B4368R, and RDIS 01104 and VA Cooperative Studies 558. D. Bourdette has received honoraria and travel expenses for speaking and serving as faculty at company sponsored meetings from Biogen Idec, Serono, and Teva Neurosciences. He is funded by the following grants: NIH/NINDS 1P30 NS069346–01, NIH/NINDS 1R01 NS057433, Department of Veterans Affairs/BLR&D Merit Review, Department of Veterans Affairs/RR&D B4368R; and National MS Society, CA 1055-A-3. Go to Neurology.org for full disclosures.
REFERENCES
- 1. Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology 1991; 41: 685– 691 [DOI] [PubMed] [Google Scholar]
- 2. Amato MP, Ponziani G, Siracusa G, Sorbi S. Cognitive dysfunction in early-onset multiple sclerosis: a reappraisal after 10 years. Arch Neurol 2001; 58: 1602– 1606 [DOI] [PubMed] [Google Scholar]
- 3. Benedict RH, Cookfair D, Gavett R, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc 2006; 12: 549– 558 [DOI] [PubMed] [Google Scholar]
- 4. Benedict RH, Zivadinov R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nat Rev Neurol 2011; 7: 332– 342 [DOI] [PubMed] [Google Scholar]
- 5. Harel Y, Appleboim N, Lavie M, Achiron A. Single dose of methylphenidate improves cognitive performance in multiple sclerosis patients with impaired attention process. J Neurol Sci 2009; 276: 38– 40 [DOI] [PubMed] [Google Scholar]
- 6. Sumowski JF, Chiaravalloti N, Erlanger D, Kaushik T, Benedict RH, Deluca J. l-Amphetamine improves memory in MS patients with objective memory impairment. Mult Scler 2011; 17: 1141– 1145 [DOI] [PubMed] [Google Scholar]
- 7. Morrow SA, Kaushik T, Zarevics P, et al. The effects of l-amphetamine sulfate on cognition in MS patients: results of a randomized controlled trial. J Neurol 2009; 256: 1095– 1102 [DOI] [PubMed] [Google Scholar]
- 8. Benedict RH, Munschauer F, Zarevics P, et al. Effects of l-amphetamine sulfate on cognitive function in multiple sclerosis patients. J Neurol 2008; 255: 848– 852 [DOI] [PubMed] [Google Scholar]
- 9. Kooi EJ, Prins M, Bajic N, et al. Cholinergic imbalance in the multiple sclerosis hippocampus. Acta Neuropathol 2011; 122: 313– 322 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Krupp LB, Christodoulou C, Melville P, et al. Multicenter randomized clinical trial of donepezil for memory impairment in multiple sclerosis. Neurology 2011; 76: 1500– 1507 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Srinivasan R, Sailasuta N, Hurd R, Nelson S, Pelletier D. Evidence of elevated glutamate in multiple sclerosis using magnetic resonance spectroscopy at 3 T. Brain 2005; 128: 1016– 1025 [DOI] [PubMed] [Google Scholar]
- 12. Newcombe J, Uddin A, Dove R, et al. Glutamate receptor expression in multiple sclerosis lesions. Brain Pathol 2008; 18: 52– 61 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Pampliega O, Domercq M, Soria FN, Villoslada P, Rodriguez-Antiguedad A, Matute C. Increased expression of cystine/glutamate antiporter in multiple sclerosis. J Neuroinflammation 2011; 8: 63 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Lovera JF, Frohman E, Brown TR, et al. Memantine for cognitive impairment in multiple sclerosis: a randomized placebo-controlled trial. Mult Scler 2010; 16: 715– 723 [DOI] [PubMed] [Google Scholar]
- 15. Villoslada P, Arrondo G, Sepulcre J, Alegre M, Artieda J. Memantine induces reversible neurologic impairment in patients with MS. Neurology 2009; 72: 1630– 1633 [DOI] [PubMed] [Google Scholar]
- 16. Bazan NG. The neuromessenger platelet-activating factor in plasticity and neurodegeneration. Prog Brain Res 1998; 118: 281– 291 [DOI] [PubMed] [Google Scholar]
- 17. Birks J, Grimley Evans J. Ginkgo biloba for cognitive impairment and dementia. Cochrane Database Syst Rev 2009: CD003120. [DOI] [PubMed] [Google Scholar]
- 18. Yadav V, Shinto L, Morris C, Senders A, Baldauf-Wagner S, Bourdette D. Use and self-reported benefit of complementary and alternative medicine among multiple sclerosis patients. Int J MS Care 2006; 8: 5– 10 [Google Scholar]
- 19. Lovera J, Bagert B, Smoot K, et al. Ginkgo biloba for the improvement of cognitive performance in multiple sclerosis: a randomized, placebo-controlled trial. Mult Scler 2007; 13: 376– 385 [DOI] [PubMed] [Google Scholar]
- 20. McDonald WI, Compston A, Edan G, et al. Recommended diagnostic criteria for multiple sclerosis: guidelines from the International Panel on the Diagnosis of Multiple Sclerosis. Ann Neurol 2001; 50: 121– 127 [DOI] [PubMed] [Google Scholar]
- 21. Spreen O, Strauss E. A Compendium of Neuropsychological Tests, 1st ed. New York: Oxford University Press; 1991 [Google Scholar]
- 22. Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test Manual: Second Edition, Adult Version. San Antonio, TX: Psychological Corporation; 2000 [Google Scholar]
- 23.PASAT Manual [online]. [Accessed March 16, 2012]. Available at: http://www.pasat.us/PDF/PASAT_Manual.pdf.
- 24. Sullivan J, Edgley K, Dehoux E. A survey of multiple sclerosis: part 1: perceived cognitive problems and compensatory strategy use. Can J Rehab 1990; 4: 99– 105 [Google Scholar]
- 25. Benedict RH, Zivadinov R. Predicting neuropsychological abnormalities in multiple sclerosis. J Neurol Sci 2006; 245: 67– 72 [DOI] [PubMed] [Google Scholar]
- 26. Willer B, Ottenbacher KJ, Coad ML. The community integration questionnaire: a comparative examination. Am J Phys Med Rehabil 1994; 73: 103− 111 [DOI] [PubMed] [Google Scholar]
- 27. Fisk JD, Ritvo PG, Ross L, Haase DA, Marrie TJ, Schlech WF. Measuring the functional impact of fatigue: initial validation of the fatigue impact scale. Clin Infect Dis 1994; 18 (suppl 1): S79− S83 [DOI] [PubMed] [Google Scholar]
- 28. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation; 1996 [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.