Brain Bank Questionnaire Helps in Differential Diagnosis of Movement Disorders: An Autopsy Study of 150 Patients

ABSTRACT

Background

As part of the CurePSP brain donation program, a questionnaire was developed to gather basic clinical information on donors; however, its usefulness has not been evaluated.

Objective

To assess the value of information obtained from the questionnaire in differentiating between parkinsonian disorders.

Methods

We reviewed 150 questionnaires, including 50 patients, each with a neuropathologic diagnosis of Lewy body disease (LBD), multiple system atrophy (MSA), or progressive supranuclear palsy. The frequency of clinical features recorded in the questionnaires was compared for the three disorders, and a machine learning algorithm was used to identify features predicting neuropathologic diagnosis.

Results

The information from the questionnaires correlated with core clinical features for each disorder, such as hallucinations for LBD and autonomic dysfunction for MSA. Hallucinations and disorientations were identified as the key variables that contributed most to the prediction of neuropathology.

Conclusion

The questionnaire provides useful clinical information for clinicopathological correlative studies.

The clinical diagnosis of neurodegenerative disorders such as Parkinson's disease (PD), multiple system atrophy (MSA), and progressive supranuclear palsy (PSP), is often challenging because of overlapping clinical features and limited availability of reliable diagnostic biomarkers. ¹ Discrepancy between clinical and postmortem pathological diagnoses for a given patient is not uncommon, ² , ³ highlighting the importance of developing better screening tools for differential diagnosis.

Clinicopathological studies based on postmortem analysis of brain tissue have played a pivotal role in identifying clinical features useful in the differential diagnosis of movement disorders. ⁴ Clinical information gathered from medical records and ancillary documents is crucial for clinicopathological research, but the quality and quantity of medical records vary in retrospective autopsy studies. ² To address this limitation, questionnaires have been developed to collect clinical information from patient informants. These forms can be a valuable source of information that might otherwise be unavailable because of the timeliness and incompleteness of medical records available for clinicopathologic studies of brain bank patients.

In the present study, we assessed the usefulness of a brain bank questionnaire developed by CurePSP, the leading patient support group in the United States for PSP and atypical parkinsonian disorders, in the differential diagnosis of parkinsonian disorders.

Methods

Subjects and Neuropathological Diagnoses

The study consisted of 150 consecutive patients of autopsy‐confirmed Lewy body disease (LBD), MSA, and PSP (50 patients of each disorder) received by the Mayo Clinic brain bank between 2020 and 2022. All neuropathological diagnoses were made by a single, experienced neuropathologist (D.W.D.) as previously described. ⁵ , ⁶ , ⁷ , ⁸ , ⁹ Alzheimer's disease (AD) neuropathological change was evaluated with thioflavin S fluorescence microscopy, and Lewy‐related pathology was assessed with α‐synuclein immunohistochemistry. A subset of patients had concurrent pathology; 18 LBD, 4 MSA, and 7 PSP patients met the neuropathological criteria of AD (ie, Braak neurofibrillary tangle stage of IV or higher and Thal amyloid phase of 3 or higher). Concurrent LBD was observed in 2 MSA and 2 PSP patients.

Data Collection

The following clinical information was collected from the CurePSP questionnaire (Fig. S1): age at onset, family history, symptoms early in the disease course, rapid disease progression, symptoms present at any time, and personality change. The symptoms present at any time were obtained from a checklist on the questionnaire that included the following: disorientation, tremors, wandering, visual problems, agitation, stiffness, violent outburst, weight loss, delusions, hallucinations, difficulty walking, eating disorder, sleep disorder, and falls. The early symptoms, rapid disease progression, and personality change sections of the questionnaire are open‐ended questions filled in by the applicant or family member in their own words. In the early symptoms section, responses were used to determine the frequency of falls (defined as early falls), autonomic dysfunction, dream enactment behavior, and memory loss.

Prediction of the Pathological Diagnosis Using a Machine Learning Algorithm

To investigate whether the information gathered from the questionnaire could predict the pathological diagnosis of neurodegenerative movement disorders, we used the Extreme Gradient Boosting (XGBoost) algorithm. XGBoost is an open‐source implementation of the gradient boosted trees method, which aims to accurately predict a target variable by combining the estimates of a set of simpler, weaker models. ¹⁰ The variables used in the analysis included the presence of symptoms in the checklist and personality change throughout the disease course, rapid disease progression, as well as the presence of falls, memory loss, dream enactment behavior, speech issue, loss of smell, autonomic dysfunction, and word‐finding difficulty as early symptoms.

The dataset was split into training and testing datasets in a 9:1 ratio. We performed Bayesian hyperparameter optimization with 5‐fold cross‐validation on training dataset for the XGBoost, using the Hyperopt library. After obtaining the best hyperparameters, we trained the XGBoost model using 5‐fold cross‐validation. We used Platt's scaling for model calibration, a post‐processing technique, to adjust the predicted probabilities output by the trained model to match the true probabilities of the target label, before evaluating the testing dataset. The testing dataset was evaluated using each model, and the average performance was computed to obtain the results. To identify the most important feature of this task, we analyzed feature importance. We calculated the feature importance score for each fold, and then calculated the mean feature importance score across all folds. The code for the analysis was written in Python, and all computations were performed on the Google Cloud Platform.

Statistical Analysis

All statistical analyses we performed used Rcmdr 2.8‐0. (R., Boca Raton, FL). A χ² test was used for group comparisons of all categorical data, and a one‐way analysis of variance was used for continuous variables. P values <0.05 were considered statistically significant.

Results

Clinical Characteristics

Questionnaires were most frequently completed by the spouse of the donor (42%), followed by their child (20%). LBD patients had longer average disease durations than MSA and PSP. MSA had the earliest average age at onset compared with LBD and PSP. The frequency of a positive family history of neurodegenerative disorders, such as AD and PD, was not significantly different between the three disorders (Table 1).

TABLE 1.

Demographic information on all three groups

	LBD (n = 50)	MSA (n = 50)	PSP (n = 50)	P value
Sex (men:women)	35:15	27:23	30:20	0.25
Age at onset, years	65 ± 7.6	61 ± 8.8	70 ± 7.5	<0.001
Disease duration, years	9.1 ± 4.6	6.3 ± 2.1	6.3 ± 2.6	<0.001
Family history of neurodegenerative disorder	14 (28%)	13 (26%)	17 (34%)	0.65
Family history of AD	8 (16%)	6 (12%)	5 (10%)	0.65
Family history of PD	3 (6%)	4 (8%)	7 (14%)	0.35
Other a	3 (6%)	4 (8%)	5 (10%)	0.76
Concurrent diagnosis of AD	18 (36%)	4 (8%)	7 (14%)	<0.001
Concurrent diagnosis of LBD	NA	2 (4%)	2 (4%)	1.00

Note: Age at onset and disease duration are shown as mean ± standard deviation. Other data are shown as number (%).

Abbreviations: LBD, Lewy body disease; MSA, multiple system atrophy; NA, not applicable; PSP, progressive supranuclear palsy; AD, Alzheimer's disease; PD, Parkinson's disease.

^a Other includes undefined dementia and Cruetzfeld‐Jakob disease in LBD group; undefined dementia and amyotrophic lateral sclerosis in MSA group; undefined dementia, multiple sclerosis, and amyotrophic lateral sclerosis in PSP group.

Although LBD is a pathological diagnosis most frequently associated with PD and dementia with Lewy bodies (DLB) as clinical presentations, the LBD group in the present study had a range of clinical diagnoses: 15 patients had DLB, 12 patients had MSA, 9 patients had PD, 6 patients had PSP, 2 patients had frontotemporal dementia, 1 patient had AD, and 1 patient had atypical parkinsonism. The MSA group had 47 patients diagnosed with MSA, 2 with corticobasal syndrome, and 1 patient with PD. In the PSP group, 46 patients were diagnosed with PSP, and 4 patients were diagnosed with corticobasal syndrome. Regarding the disease progression, the MSA group had the highest frequency of rapid progression, followed by PSP and LBD (MSA: 62%, PSP: 44%, LBD: 28%; P = 0.002).

Clinical Symptoms from the Checklist

In this section, informants checked whether 14 specific symptoms occurred at any point during the disease course. The LBD group had the highest number of positive responses (8.4 ± 3.1), which was significantly higher than the MSA (5.9 ± 2.3) and PSP groups (5.9 ± 2.7) (P < 0.001). The LBD group also had a higher frequency of disorientation, tremors, wandering, agitation, violent outbursts, delusions, and hallucinations compared to the MSA and PSP groups. Other symptoms were similar in frequency between the three disorders (Table 2).

TABLE 2.

Frequencies of symptoms present at any time

	LBD (n = 50)	MSA (n = 50)	PSP (n = 50)	P value
Disease progression
Rapid progression	14 (28%)	31 (62%)	24 (42%)	0.002
Symptoms at any time
Disorientation	35 (70%)	8 (16%)	16 (32%)	<0.001
Tremors	35 (70%)	33 (66%)	11 (22%)	<0.001
Wandering	11 (22%)	0 (0%)	3 (6%)	<0.001
Visual problems	30 (60%)	28 (56%)	36 (72%)	0.22
Agitation	35 (70%)	9 (18%)	25 (50%)	<0.001
Stiffness	37 (74%)	35 (70%)	37 (74%)	0.87
Violent outburst	17 (34%)	3 (6%)	7 (14%)	<0.001
Weight loss	29 (58%)	26 (52%)	17 (34%)	0.04
Delusions	28 (56%)	8 (16%)	8 (16%)	<0.001
Hallucinations	37 (74%)	11 (22%)	7 (14%)	<0.001
Difficulty walking	46 (92%)	49 (98%)	48 (96%)	0.35
Eating disorder	6 (12%)	7 (14%)	8 (16%)	0.84
Sleeping disorder	33 (66%)	33 (66%)	24 (48%)	0.10
Falls	42 (84%)	46 (92%)	48 (96%)	0.11
Symptoms in the early stages
Fall	0 (0%)	9 (18%)	29 (58%)	<0.001
Autonomic dysfunction	5 (10%)	16 (32%)	0 (0%)	<0.001
Dream enactment behavior	6 (12%)	14 (28%)	0 (0%)	<0.001
Memory loss	18 (36%)	2 (4%)	10 (20%)	<0.001
Personality changes
Any symptoms	38 (76%)	26 (52%)	37 (74%)	0.02
Agitation/irritability	13 (26%)	2 (4%)	13 (26%)	0.005
Social withdrawal	10 (20%)	5 (10%)	9 (18%)	0.35
Depression/anxiety	9 (18%)	7 (14%)	6 (12%)	0.69
Apathy	3 (6%)	2 (4%)	10 (20%)	0.01

Note: Data are shown as number (%).

Abbreviations: LBD, Lewy body disease; MSA, multiple system atrophy; PSP, progressive supranuclear palsy.

Clinical Symptoms from Open‐Ended Questions

Family members described clinical features in the early stage of the disease in their own words, such as dizziness or memory problems. MSA had higher frequencies of autonomic dysfunction (eg, urinary dysfunction, hypotension, and erectile dysfunction) and dream enactment behavior. PSP had a higher frequency of falls. Of the PSP patients that reported falls, 29% of patients had falling backward. LBD patients had the highest frequency of memory loss (Table 2).

Personality changes were also reported by family members in their own words, if any. The MSA group less frequently had personality changes compared to the other two disorders (Table 2). The most common personality change was agitation/irritability, followed by social withdrawal. Agitation/irritability was the least frequent in MSA. The frequencies of social withdrawal and depression or anxiety were not different among the three diseases. Apathy was most frequent in PSP (20% in PSP, 6% in LBD, and 4% in MSA; P = 0.01).

Prediction of the Pathological Diagnosis

We examined whether the XGBoost algorithm could predict the pathological diagnosis based on information tabulated from the questionnaire. The resultant model demonstrated a diagnostic accuracy of 68%, an F1‐Score of 68.3%, and an area under the receiver operating characteristic curve of 81.6%. Analysis of the “feature importance” revealed that hallucinations, disorientation, and falls as early symptoms were the most significant variables contributing to the prediction (Fig. S2).

Discussion

In this study, we found that the CurePSP questionnaire provides basic clinical information that can differentiate underlying neuropathology of neurodegenerative disorders. The checklist of symptoms successfully identified key clinical features for LBD, MSA, and PSP. For instance, LBD patients had the highest frequency of hallucinations, a core clinical feature of DLB. ⁶ The checklist was supplemented by the section on symptoms in the early stages, which was formatted as an open‐ended question. A subset of LBD patients had a record of memory loss, a common feature of dementia. Symptoms consistent with autonomic dysfunction were often documented in the MSA group, whereas the PSP group had the highest frequency of falls. ¹¹ Additionally, both MSA and PSP groups had a higher frequency of rapid progression compared to the LBD group. By combining information from these sections, we were able to gather the characteristic clinical features of each disease.

The XGBoost analysis validated the importance of these clinical features, such as hallucinations, disorientation, and early falls, in predicting the underlying pathology. All variables used in XGBoost are binary; however, caution is required when interpreting certain variables, such as autonomic dysfunction, which were obtained from open‐ended questions. These symptoms were not reported as frequently as those in the checklist, which may have led to underestimating the importance of autonomic dysfunction in the XGBoost models.

Although this version of the questionnaire provides standardized clinical information, there is room for improvement. For instance, the checklist of symptoms may be biased toward LBD, as patients with LBD had the highest frequency of symptoms listed. Additionally, some symptoms were not well‐defined, such as “sleep disorder,” which can encompass several different disorders, including insomnia, obstructive sleep apnea, and dream enactment behavior.

To address these issues, CurePSP revised the questionnaire in 2022, introducing an extensive checklist with more than 40 different symptoms. The new checklist includes seven symptoms of autonomic dysfunction and five separate symptoms of sleep changes. The addition of these specific symptoms may help identify previously underreported symptoms, such as autonomic dysfunction and dream enactment behavior. Furthermore, a study has shown that a single question, “Have you ever been told, or suspected yourself, that you seem to act out your dreams” can detect rapid eye movement sleep behavior disorder with high sensitivity and specificity. ¹² Incorporating disease‐ and symptom‐specific questions like these may further enhance the usefulness of brain bank questionnaires in clinicopathologic studies. Additionally, the new questionnaire added a section for age at diagnosis, which can help evaluate disease progression and changes in clinical diagnosis over time.

One limitation of our study was that the XGBoost analysis did not consider comorbid pathologies such as AD pathology as a single category. Instead, for example, PSP + AD and PSP were considered under the same category of PSP. Concurrent pathologies, such as AD and LBD, are common in neurodegenerative disorders, and clinical presentations would be influenced by these mixed pathologies. ¹³ , ¹⁴ In the present study, we focused on assessing the usefulness of questionnaire in the differential diagnosis of movement disorders; therefore, the influence of concurrent pathologies on clinical presentations was beyond the scope of this study. Another limitation is that our inclusion criteria strictly relied on neuropathological diagnosis, and clinical data that could differentiate between PD, PD with dementia (PDD), and DLB were not available. Our study design combined patients with PD, PDD, and DLB into one neuropathological diagnosis of LBD, making it necessary to interpret clinical characteristics with caution. The mean disease duration of our LBD patient group was 9.1 years, which was longer than the ~4 years reported for DLB patients in the literature. ¹⁵ , ¹⁶ , ¹⁷ This difference could be attributed to the fact that our LBD patient group includes not only DLB, but also PD and PDD.

In conclusion, we demonstrated the value of using a lay‐language questionnaire to collect standardized clinical information from family members or those with close contact with the patient. Such questionnaires can provide a valuable source of information for retrospective clinicopathological studies, which improves the quality of clinicopathological research. In addition, questionnaires are a valuable tool for family members to actively contribute to research.

Author Roles

(1) Research project: A. Conception, B. Organization, C. Execution; (2) Statistical Analysis: A. Design, B. Execution, C. Review and Critique; (3) Manuscript: A. Writing of the First Draft, B. Review and Critique.

N.M.: 1A, 1C, 2B, 3A.

H.S.: 1A, 1C, 2C, 3B.

M.K.: 1C, 2B, 2C, 3B.

D.D.: 1B, 1C, 3B.

S.K.: 1A, 1B, 1C, 2A, 2B, 2C, 3B.

Disclosures

Ethical Compliance Statement: Neuropathologic evaluations were performed at the Mayo Clinic brain bank for neurodegenerative disorders. The brain bank operates under procedures approved by the Mayo Clinic Institutional Review Board. Brain autopsies were performed after consent of the legal next‐of‐kin or individuals with legal authority to grant permission for autopsy. De‐identified studies of autopsy samples are considered exempt from human subject research by the Mayo Clinic Institutional Review Board. Therefore, the authors confirm that the approval of Mayo Clinic Institutional Review Board was not required for this work. We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this work is consistent with those guidelines.

Funding Sources and Conflicts of Interests: This study is partially supported by the CurePSP and Rainwater Charitable Foundation. There are no conflicts of interest to report.

Financial Disclosures for the Previous 12 Months: Shunsuke Koga receives Mayo Clinic Alzheimer's Disease Research Center Research Grant and the State of Florida Ed and Ethel Moore Alzheimer's Disease Research Fellowship Grant. The other authors declare that there are no additional disclosures to report.

Supporting information

Figure S1. Main page of the CurePSP brain bank questionnaire. Personal information has been redacted from the top of the page.

Figure S2. Feature importance for predicting the pathological diagnosis. Variables are arranged from high (yellow) to low (dark blue) feature importance score.