Abstract
Background
Prospective data on cognition in prodromal Parkinson’s disease (PD) are limited.
Objectives
To assess in prodromal PD if (1) baseline cognition predicts conversion to clinical PD, (2) baseline dopamine transporter (DAT) binding predicts longitudinal changes in cognition, and (3) impaired olfaction predicts future cognitive decline.
Methods
Prodromal participants were 136 hyposmic individuals enrolled in the Parkinson Associated Risk Study study. We examined baseline neuropsychological test performance in PD converters vs. non-converters and the association between baseline DAT binding and change in cognition. An additional 73 normosmic individuals were included in analyses of the relationship between hyposmia and cognitive decline.
Results
In prodromal participants, baseline cognitive scores did not significantly predict conversion, but converters performed numerically worse on 5 of the 6 cognitive domains assessed, with the greatest differences in executive function/working memory (0.68 SD lower) and global cognition (0.64 SD lower). Lower baseline DAT binding predicted greater future decline in processing speed/attention (p=0.02). Hyposmia predicted greater future decline in language (p=0.005) and memory abilities (p=0.01).
Conclusions
Given hyposmia in the general population predicts cognitive decline, the role of cognition in predicting conversion in prodromal PD needs to be assessed in large cohorts followed long-term. The dopamine system may be associated with changes in processing speed/attention in individuals at-risk for PD.
INTRODUCTION
Parkinson’s disease (PD) has, until recently, been defined based on the presence of specific motor features(1). Several lines of evidence indicate that there is an identifiable and definable phase prior to the time that motor diagnostic criteria for PD are met. These include the recognition that advanced nigral degeneration is already present at the time of motor symptoms(2,3), and that there are several clinical, biomarker, and genetic markers that alone, or in combination, can accurately identify a subset of individuals who eventually meet motor criteria(4). Prodromal PD is the phase during which signs, symptoms, genotype, or biomarker findings suggest the presence of early neurodegeneration, but the individual does not yet exhibit the motor features necessary for the diagnosis of PD. Formal diagnostic criteria for prodromal PD were recently proposed(4).
Among the many clinical markers of the prodromal state, the three associated with the highest likelihood of future PD include REM sleep behavior disorder (RBD), olfactory loss, and reduced dopamine transporter (DAT) binding(4). Consistent with this, in the Parkinson Associated Risk Study (PARS) study cohort, the combination of hyposmia and DAT deficit at baseline was highly predictive of conversion to PD within four years(5). However, impaired cognition occurs in 20–30% of individuals with early PD(6,7), and has been associated with both reduced DAT binding(8) and hyposmia(9), which also predicts greater rate of cognitive decline at this stage(9). The epidemiology of cognitive changes among individuals with clinical markers of the prodromal state, and longitudinal changes at this stage of putative PD neuropathology, has not been well studied.
We recently described, in a cross-sectional analysis, presence of impaired cognition in individuals with hyposmia and DAT deficit in the PARS study cohort(10). We found that incorporating cognition into models improved prediction of DAT binding deficit. Our objectives here were to (1) assess whether baseline cognitive performance in prodromal PD predicts future conversion to PD, (2) assess if baseline DAT binding in prodromal PD predicts future cognitive decline, and (3) assess if hyposmia predicts future cognitive decline.
METHODS
Study description
Methods for participant recruitment and assessments have been previously described(10,11,12). Briefly, individuals with or without a family history of PD were targeted via electronic media outreach, and assessments occurred at 16 movement disorders specialty clinics in the United States. Inclusion criteria were (1) age >50 years (or within 10 years of age of onset of an affected PD relative) and (2) no known reason for abnormal olfaction. Exclusion criteria included (1) established diagnosis of PD or other neurodegenerative disorder, (2) history of dementia or evidence for it on examination (including Mini-Mental State Examination score < 27), and (3) evidence of parkinsonism on examination thought to be consistent with PD (as determined by the investigator). Participant enrollment initiated in 2010, and the study is ongoing. The analyses described here include data from the first four years of follow up.
Participants
A total of 303 individuals completed baseline clinical and imaging evaluations; excluded from this group were those diagnosed with PD by the investigator at the time of baseline assessment (N=1), conversion to non-specified “Lewy body disease” (N=1), no follow-up visits (N=22), no baseline cognitive evaluation (N=25), and baseline cognitive data obtained at or after their second DAT scan (N=39). Of the remaining 215 patients, 136 subjects were hyposmic (i.e., defined here as at-risk or prodromal PD) and were included in the conversion analysis (Figure 1). 105 of the hyposmic subjects had baseline cognitive testing and a baseline DAT SPECT within 1 month of each other, along with a follow-up cognitive assessment, and were included in the analysis examining long-term cognitive change. Finally, cognitive changes in 73 normosmic subjects and 125 hyposmic subjects with at least one follow-up visit were compared.
Figure 1.
Flow diagram of prodomal participants
Assessments
Participants underwent olfactory testing, DAT scan, clinical/motor assessments, and cognitive testing. Cognitive testing (administered annually) and other study assessments have been described in detail elsewhere(10) and are shown in Supplementary Table 1. Twenty cognitive measures were combined and used to assess the following 5 domains: executive function/working memory, verbal memory, processing speed/attention, visuospatial function, and language. Scores for each cognitive test were reversed, if needed, so higher scores indicate better performance. Domain z-scores and a global cognitive score based on the domain z-scores were generated as previously described(10). Normative scores were generated from the cohort of 215 participants (79 normosmics and 136 hyposmics)(10). Depression was defined as a score ≥16 on the Center for Epidemiological Studies Depression Scale (CES-D)(13) at study screening.
Statistical Analysis
Baseline characteristics were described using means, standard deviations and ranges or number and percent, as appropriate.
Conversion was defined as the determination, by a site study investigator (movement disorders neurologist), that a participant met criteria for PD. The associations between baseline cognitive domains and conversion were investigated using separate logistic regression models adjusted for age at baseline cognitive testing, sex, depression, and education. Results are presented as odds ratios (95% confidence intervals), with the odds ratios signed so that an odds ratio >1 implies a greater risk of conversion.
Annualized changes in cognitive domains between the baseline cognitive testing visit and the last testing visit were calculated. Their associations with baseline age-adjusted percent [123I]β-CIT SPECT lowest putamen:cerebellum binding ratio (hereto forth referred to as DAT binding) were investigated using separate multiple regression models adjusted for age at baseline cognitive testing, sex, depression, education, and the baseline value of the cognitive domain. For ease in presentation, these adjusted models were rerun with baseline age-adjusted percent putamen dichotomized at 80%; results are presented as adjusted (least squares) z-score means with 95% confidence intervals. The associations of annualized changes in cognitive domains with olfactory status were investigated using the same method.
RESULTS
Participant characteristics
Baseline characteristics of the 136 hyposmic participants included in these analyses are shown in Supplementary Table 2. Of the 136, 35 (26%) had DAT reduction and 101 (74%) did not.
Association between baseline cognition and conversion to PD
During the mean of 3.8 (SD 0.8; range 1–5) years of follow-up for the 136 individuals included in this analysis, 10 individuals converted to a clinical diagnosis of PD, two of whom were subjects with scans without evidence of dopaminergic deficit (SWEDDs), who were excluded from the subsequent analyses. Baseline global cognition and all cognitive domains, except language, were numerically worse among converters compared with non-converters, with the greatest differences for executive function/working memory (0.68 SD difference) and global cognition (0.64 SD difference) scores (Supplementary Table 3). The odds ratios for predicting conversion were 2.71 (0.97, 7.57) for executive function/working memory and 2.55 (0.92, 7.09) for global cognition.
Association between baseline DAT binding and longitudinal cognitive performance
Hyposmics in the DAT reduction group had numerically greater annualized rate of decline in global cognition, language, memory, and processing speed/attention than those in the higher DAT group (Table 1). In separate regression models that controlled for baseline cognitive performance, age, sex, education and depression, lower baseline DAT binding (as continuous measure) predicted long-term decline in processing speed/attention (p=0.02).
Table 1.
Association between baseline DAT binding and annualized changes in cognitive measures in hyposmics
| Cognitive measurea | DAT status | Entire sample (N=105) p-valuec |
|
|---|---|---|---|
| Low baseline DAT bindingb (N=23) |
High baseline DAT bindingb (N=82) |
||
| Global cognition | −0.12 (−0.21, −0.03) | −0.08 (−0.15, −0.02) | 0.19 |
| Executive function / Working memory | −0.04 (−0.13, 0.05) | −0.05 (−0.11, 0.01) | 0.93 |
| Languaged | −0.18 (−0.38, 0.01) | −0.04 (−0.18, 0.10) | 0.61 |
| Memory | −0.17 (−0.28, −0.05) | −0.13 (−0.21, −0.05) | 0.29 |
| Processing speed / Attention | −0.05 (−0.15, 0.05) | 0.03 (−0.04, 0.10) | 0.02 |
| Visuospatial | −0.07 (−0.21, 0.08) | −0.09 (−0.19, 0.02) | 0.95 |
Adjusted mean (95% CI) cognitive changes by baseline DAT status.
Based on 80% of age-expected lowest putamen of [123I]β-CIT uptake for the hyposmic group.
Significance of baseline DAT binding (continuous) on change in cognition in entire sample from separate multiple regression models adjusted for age at baseline cognitive testing, sex, education, depression, and baseline cognitive measure.
One outlier omitted.
Association between baseline olfaction and longitudinal cognitive performance
In multiple regression models participants with hyposmia at baseline, compared with normosmics, had numerically greater future decline in global cognition and all cognitive domains, except processing speed/attention, with statistically significant decline in language (p=0.005) and memory (p=0.01) (Supplementary Table 4).
DISCUSSION
In this study of individuals considered to be at risk, or prodromal, for PD based on being hyposmic, cognitive performance did not predict which hyposmic individuals in the general population convert to PD. However, processing speed/attention declined more over time among those at greatest risk for PD (i.e., with DAT reduction), and hyposmia was associated with future decline in language and memory abilities. In addition, although baseline cognitive performance was not a significant predictor of conversion to PD, numerical differences (i.e., 0.6–0.7 SD for executive function/working memory and global cognition) suggest that it has the potential to contribute to prediction in a larger sample followed long-term.
That hyposmics with DAT reduction had relative decline in processing speed/attention over time also supports categorizing this group of individuals as prodromal PD. Although there are little data on longitudinal cognitive changes in prodromal PD, several studies have shown multi-domain cognitive changes in idiopathic RBD(14), a disorder with almost universal long-term conversion to PD or dementia with Lewy bodies (DLB), and a study of idiopathic RBD patients found longitudinal changes in delayed verbal memory and visuospatial learning domains(15). In addition, a recent case-control study found that PD patients demonstrated subtle cognitive deficits up to seven years before diagnosis compared with controls(16).
According to Braak’s hypothesis(17), an individual with hyposmia and reduced DAT binding likely has involvement of several brain areas including the olfactory bulb, medulla/pons (including involvement of norepinephrine and serotonin nuclei), and midbrain (dopaminergic neurons). Many of the latter regions and neurotransmitter systems, including dopaminergic(18) and noradrenergic(19) projections to the prefrontal cortex, mediate processing speech/attention(18). This region also mediates executive function/working memory, and we expected, based on this and our previous cross-sectional analysis(10), that executive function/working memory would have also declined more rapidly in hyposmics with DAT reduction.
We recently reported in a cross-sectional analysis that baseline executive function/working memory and global cognition improve prediction of DAT deficit when added to other prodromal symptoms(10). In regards to the predictive value of baseline cognition on risk of conversion to PD, we did not replicate this finding longitudinally. A previous study examining executive function in hyposmics also did not find that it predicted conversion(20). However, in our study the number of PD converters was very small (N=8), hyposmic converters did perform approximately 0.65 SD worse on both executive function/working memory and global cognition at baseline, and hyposmics overall had greater decline over time in language and memory compared with normosmics. The trends we observed suggest that in larger cohorts followed longer (i.e., with more converters) baseline cognitive performance might contribute to prediction of conversion. It would be of particular interest to examine this in individuals with additional prodromal features, such as hyposmics with DAT deficit and RBD. Unfortunately, our dataset does not allow incorporation of RBD, as RBD in PARS was assessed based on bed-partner input and this was not available for a large number of participants(10). An aspect of the cohort that limits generalizability is the high percentage of participants with a family history of PD (35%).
In summary, in individuals at increased risk for PD on the basis of being hyposmic, the dopamine system is implicated in ongoing cognitive decline, particularly in the domain of attention/processing speed. Also, the role of baseline cognitive performance in prodromal PD in predicting future conversion to PD should be assessed in large cohorts followed long-term. Detectable cognitive changes are present in prodromal PD, in part appear to be due to neurodegeneration of the dopamine system, and may have prognostic significance for future development of PD.
Supplementary Material
Acknowledgments
The authors wish to acknowledge additional study team members including Carolyn Cioffi and Maria Abeleo (data management), Shirley Lasch and Donna Miles (project management) and the PARS study participants for their dedication to complete this study.
PARS study Collaborators: David Russell, MD, PhD, Institute for Neurodegenerative Disorders, New Haven, CT; Kapil Sethi, MD, Medical College of Georgia, Augusta, GA; Samuel Frank, MD, Boston University Medical Center, Boston, MA; Tanya Sumuni, MD, Northwestern University, Chicago, IL; Matthew Stern, MD, University of Pennsylvania PDMC, Philadelphia, PA; Robert Hauser, MD, University of South Florida, Tampa, FL; Bernard Ravina and Irene Richards, MD, University of Rochester, Rochester, NY; Grace Liang, MD, Parkinson’s Institute, Sunnyvale, CA; Charles Adler, MD, PhD, May Clinic, Phoenix, AZ; Rachel Saunders-Pullman, MD, Beth Israel Medical Center, New York, NY Marian L Evatt, MD, Emory University; Atlanta, GA; Eugene Lai, MD, Michael E. DeBakey Department of Veteran’s Affairs Medical Center, Houston, TX; Indu Subramanian, MD, UCLA Medical Center, Los Angeles, CA; Penelope Hogarth, MD and Kathryn Chung, MD, Portland VA Medical Center, Portland, OR.
Financial Disclosure:
Daniel Weintraub has received research funding or support from Michael J. Fox Foundation for Parkinson’s Research, National Institutes of Health (NINDS), Novartis Pharmaceuticals, Department of Veterans Affairs, Avid Radiopharmaceuticals, Alzheimer’s Disease Cooperative Study, and the International Parkinson and Movement Disorder Society; honoraria for consultancy from Acadia, Biogen, Biotie (Acorda), Bracket, Clintrex LLC, Eisai Inc., Eli Lilly, Lundbeck, Takeda, UCB, and the CHDI Foundation; license fee payments from the University of Pennsylvania for the QUIP and QUIP-RS; royalties from Wolters Kluweland; and fees for legal consultation for lawsuits related to medication prescribing in patients with Parkinson’s disease.
Danna Jennings is a full time employee of Eli Lilly and Company and owns equity in MNI Holdings.
Andrew Siderowf is a full time employee of Avid Radiopharmaceuticals, a wholly owned subsidiary of Eli Lilly and Company.
Matthew Stern has been a consultant to Neuroderm, Impax, Acorda, Adamas and has Stock options in Adamas.
Shirley Eberly has received grant support from the Huntington Study Group on behalf of Auspex/Teva and from Biogen Idec and Vaccinex.
David Oakes has received research support from NIH, DOD, Michael J. Fox Foundation, Vaccinex Inc. Auspex/TEVA and Prana Biotech. He has received personal compensation from Voyager Therapeutics and the University California for service on DSMB’s, and from Raptor Pharmaceuticals for personal consulting.
Kenneth Marek has been a consultant to Pfizer, GE Healthcare, Lilly, BMS, Piramal, Biogen, Prothena, Roche, Neuropore, US Worldmeds, Neurophage, UCB, Oxford Biomedica, Lysosomal Therapetic, Inc, Neuroderm, Denali and is a co-founder of Molecular NeruoImaging and an owner of inviCRO.
Study funding: Support for this study is provided by the Department of Defense award number W81XWH-06-067. The funding organization exerted no involvement in the following: design and conduct of the study; collection, management, analysis and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Footnotes
Authors' Roles
Daniel Weintraub, MD: 2C, 3A, and 3B
Lana M. Chahine, MD: 2C, 3A, and 3B
Keith A. Hawkins, PsyD: 1A, 2C, 3B
Andrew Siderowf, MD, MSCE: 1A, 1B, 1c, 2C, and 3B
Shirley Eberly, MS: 1A, 2A, 2B, 2C, 3A, and 3B
David Oakes, PhD: 1A, 2C, and 3B
John Seibyl, MD: 1A, 2C, and 3B
Matthew B. Stern, MD: 1A, 1B, 1C, 2C, and 3B
Kenneth Marek, MD: 1A, 1B, 1C, 2C, and 3B,
Danna Jennings, MD: 1A, 1B, 1C, 2C, and 3C
Conflicts of Interest:
The authors have no conflicts of interest to report.
References
- 1.Hughes AJ, Daniel SE, Kilford L, Lees AJ. Accuracy of clinical diagnosis of idiopathic parkinson's disease: A clinico-pathological study of 100 cases. J Neurol Neurosurg Psychiatry. 1992;55(3):181–4. doi: 10.1136/jnnp.55.3.181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Fearnley JM, Lees AJ. Ageing and parkinson's disease: Substantia nigra regional selectivity. Brain. 1991;114(Pt 5):2283–301. doi: 10.1093/brain/114.5.2283. (Pt 5) [DOI] [PubMed] [Google Scholar]
- 3.Ross GW, Petrovitch H, Abbott RD, Nelson J, Markesbery W, Davis D, et al. Parkinsonian signs and substantia nigra neuron density in decendents elders without PD. Ann Neurol. 2004;56(4):532–9. doi: 10.1002/ana.20226. [DOI] [PubMed] [Google Scholar]
- 4.Berg D, Postuma RB, Adler CH, Bloem BR, Chan P, Dubois B, et al. MDS research criteria for prodromal parkinson's disease. Mov Disord. 2015;30(12):1600–11. doi: 10.1002/mds.26431. [DOI] [PubMed] [Google Scholar]
- 5.Jennings D, Siderowf A, Stern M, Seibyl J, Eberly S, Oakes D, et al. Conversion to parkinson disease in the PARS hyposmic and dopamine transporter-deficit prodromal cohort. JAMA Neurol. 2017 doi: 10.1001/jamaneurol.2017.0985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Weintraub D, Simuni T, Caspell-Garcia C, Coffey C, Lasch S, Siderowf A, et al. Cognitive performance and neuropsychiatric symptoms in early, untreated parkinson's disease. Mov Disord. 2015;30(7):919–27. doi: 10.1002/mds.26170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Aarsland D, Bronnick K, Larsen JP, Tysnes OB, Alves G. Norwegian ParkWest Study Group. Cognitive impairment in incident, untreated parkinson disease: The norwegian ParkWest study. Neurology. 2009;72(13):1121–6. doi: 10.1212/01.wnl.0000338632.00552.cb. [DOI] [PubMed] [Google Scholar]
- 8.Siepel FJ, Bronnick KS, Booij J, Ravina BM, Lebedev AV, Pereira JB, et al. Cognitive executive impairment and dopaminergic deficits in de novo parkinson's disease. Mov Disord. 2014;29(14):1802–8. doi: 10.1002/mds.26051. [DOI] [PubMed] [Google Scholar]
- 9.Fullard ME, Tran B, Xie SX, Toledo JB, Scordia C, Linder C, et al. Olfactory impairment predicts cognitive decline in early parkinson's disease. Parkinsonism Relat Disord. 2016 doi: 10.1016/j.parkreldis.2016.02.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chahine LM, Weintraub D, Hawkins KA, Siderowf A, Eberly S, Oakes D, et al. Cognition in individuals at risk for parkinson's: Parkinson associated risk syndrome (PARS) study findings. Mov Disord. 2016;31(1):86–94. doi: 10.1002/mds.26373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Siderowf A, Jennings D, Eberly S, Oakes D, Hawkins KA, Ascherio A, et al. Impaired olfaction and other prodromal features in the parkinson at-risk syndrome study. Mov Disord. 2012;27(3):406–12. doi: 10.1002/mds.24892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Jennings D, Siderowf A, Stern M, Seibyl J, Eberly S, Oakes D, et al. Imaging prodromal parkinson disease:The parkinson associated risk syndrome study. Neurology. 2014;83:1739–46. doi: 10.1212/WNL.0000000000000960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Radloff L. The CES-D scale: A self-report depression scale for research in the general population. New York, NY: West Publishing Co; 1977. [Google Scholar]
- 14.Gagnon JF, Bertrand JA, Genier Marchand D. Cognition in rapid eye movement sleep behavior disorder. Front Neurol. 2012;3:82. doi: 10.3389/fneur.2012.00082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Fantini ML, Farini E, Ortelli P, Zucconi M, Manconi M, Cappa S, et al. Longitudinal study of cognitive function in idiopathic REM sleep behavior disorder. Sleep. 2011;34(5):619–25. [PMC free article] [PubMed] [Google Scholar]
- 16.Darweesh SK, Verlinden VJ, Stricker BH, Hofman A, Koudstaal PJ, Ikram MA. Trajectories of prediagnostic functioning in parkinson's disease. Brain. 2017;140(Pt 2):429–41. doi: 10.1093/brain/aww291. [DOI] [PubMed] [Google Scholar]
- 17.Braak H, Del Tredici K, Rub U, de Vos RA, Jansen Steur EN, Braak E. Staging of brain pathology related to sporadic parkinson's disease. Neurobiol Aging. 2003;24(2):197–211. doi: 10.1016/s0197-4580(02)00065-9. [DOI] [PubMed] [Google Scholar]
- 18.Matsumoto M. Dopamine signals and physiological origin of cognitive dysfunction in parkinson's disease. Mov Disord. 2015;30(4):472–83. doi: 10.1002/mds.26177. [DOI] [PubMed] [Google Scholar]
- 19.Vazey EM, Aston-Jones G. The emerging role of norepinephrine in cognitive dysfunctions of parkinson's disease. Front Behav Neurosci. 2012;6:48. doi: 10.3389/fnbeh.2012.00048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ponsen MM, Stoffers D, Twisk JW, Wolters EC, Berendse HW. Hyposmia and executive dysfunction as predictors of future parkinson's disease: A prospective study. Mov Disord. 2009;24(7):1060–5. doi: 10.1002/mds.22534. [DOI] [PubMed] [Google Scholar]
- 21.Doty RL, Shaman P, Dann M. Development of the university of pennsylvania smell identification test: A standardized microencapsulated test of olfactory function. Physiol Behav. 1984;32(3):489–502. doi: 10.1016/0031-9384(84)90269-5. [DOI] [PubMed] [Google Scholar]
- 22.Seibyl JP, Marek KL, Quinlan D, Sheff K, Zoghbi S, Zea-Ponce Y, et al. Decreased single-photon emission computed tomographic [123I]beta-CIT striatal uptake correlates with symptom severity in parkinson's disease. Ann Neurol. 1995;38(4):589–98. doi: 10.1002/ana.410380407. [DOI] [PubMed] [Google Scholar]
- 23.van Dyck CH, Seibyl JP, Malison RT, Laruelle M, Zoghbi SS, Baldwin RM, et al. Age-related decline in dopamine transporters: Analysis of striatal subregions, nonlinear effects, and hemispheric asymmetries. Am J Geriatr Psychiatry. 2002;10(1):36–43. [PubMed] [Google Scholar]
- 24.Brandt J, Benedict RHB. The hopkins verbal learning test-revised. Odessa, FL: Psychological Assessment Reources; 2001. [Google Scholar]
- 25.Randolph C. Repeatable battery for the assessment of neuropsychological status. San Antonio: Psychological Corporation; 1998. [Google Scholar]
- 26.Wechsler D. Wechsler memory scale. 3. San Antonio, Tx: Psychological Corporation; 1997. [Google Scholar]
- 27.Spreen O, Strauss E. A compendium of neuropsychological tests: Administration, norms and commentary. 2. New York: Oxford University Press; 1998. [Google Scholar]
- 28.Manual of directions and scoring. Washington, DC: War Department, Adjutant General’s Office; 1944. Army Individual Test Battery. [Google Scholar]
- 29.Warrington EK, James M. Visual object and space perception battery (VOSP) Bury St. Edmunds, England: Thames Valley Test Co; 1991. [Google Scholar]
- 30.Kaplan E, Goodglass H, Weintraub S. Boston naming test. Philadelphia: Lea & Febiger; 1983. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.