Validation Study of the MDS Criteria for the Diagnosis of Multiple System Atrophy in the Mayo Clinic Brain Bank

Abstract

Background and Objective

The second consensus criteria in 2008 have been used in diagnosing multiple system atrophy (MSA). The International Parkinson and Movement Disorder Society (MDS) proposed new diagnostic criteria for MSA in 2022. This study aimed to compare the diagnostic accuracy between these 2 criteria and validate the clinical utility of the newly proposed criteria for MSA.

Methods

We conducted a retrospective autopsy cohort study of consecutive patients with a clinical or pathologic diagnosis of MSA from the Mayo Clinic brain bank between 1998 and 2021. We studied 352 patients (250 pathologically diagnosed MSA and 102 non-MSA); MDS criteria and the second consensus criteria were applied. The sensitivity, specificity, and area under the curve (AUC) of receiver operating characteristic curves were compared between these criteria. Comparison was conducted between clinical subtypes and among clinically challenging cases (those with different clinical diagnoses or those with suspected but undiagnosed MSA before death). We also used machine learning algorithm, eXtreme Gradient Boosting, to identify clinical features contributing diagnostic performance.

Results

The sensitivity and specificity of clinically established and probable MSA by the MDS criteria were 16% and 99% and 64% and 74%, respectively. The sensitivity and specificity of probable MSA and possible MSA by the second consensus criteria were 72% and 52% and 93% and 21%, respectively. The AUC of MDS clinically probable MSA was the highest (0.69). The diagnostic performance did not differ between clinical subtypes. In clinically challenging cases, MDS clinically established MSA maintained high specificity and MDS clinically probable MSA demonstrated the highest AUC (0.62). MRI findings contributed to high specificity. In addition, combining core clinical features with 2 or more from any of the 13 supporting features and the absence of exclusion criteria also yielded high specificity. Among supporting features, rapid progression was most important for predicting MSA pathology.

Discussion

The MDS criteria showed high specificity with clinically established MSA and moderate sensitivity and specificity with clinically probable MSA. The observation that high specificity could be achieved with clinical features alone suggests that MSA diagnosis with high specificity is possible even in areas where MRI is not readily available.

Introduction

Multiple system atrophy (MSA) is a neurodegenerative disease with varying degrees of autonomic failure, parkinsonism, and cerebellar dysfunction.^1,2 MSA is considered an α-synucleinopathy based on the pathologic α-synuclein aggregates in glial cells^3,4 and neurons.^5,6 The second consensus statement⁷ has been widely used to diagnose MSA; however, it has been pointed out that later onset^8-11 and cognitive impairment,^12-14 which are included in the nonsupporting features, are also seen in MSA, and the diagnostic accuracy is suboptimal.^15,16 The International Parkinson and Movement Disorder Society (MDS) MSA Criteria Revision Task Force developed new diagnostic criteria to address these problems.¹⁷ In the new criteria, 4 diagnostic categories were created: neuropathologically established MSA, clinically established MSA, clinically probable MSA, and possible prodromal MSA. Reliable biomarkers for MSA are still under development, and pathologic evaluation of autopsy brains postmortem remains the gold standard.^18-21 Neuropathologically established MSA requires the neuropathologic confirmation of widespread and abundant α-synuclein–positive glial cytoplasmic inclusions with neurodegeneration in striatonigral and/or olivopontocerebellar systems.²² For the clinical diagnosis, essential features, including onset after 30 years of age, a negative family history, and a progressive disease course, are required. Clinically established MSA was designed to have maximum specificity with acceptable sensitivity. This category must fulfill all the following components: the clinical core features, at least 2 supportive features, and MRI findings suggestive of MSA, with none of the exclusion criteria. The clinical core features for this category were stringent requirements and include autonomic dysfunction, such as urge urinary incontinence, voiding difficulty with postvoid residual urine volume greater than 100 mL, and orthostatic hypotension with a drop in systolic blood pressure of 20 mm Hg or more within 3 minutes of standing. They also had poorly l-dopa–responsive parkinsonism or cerebellar syndromes with at least 2 of gait ataxia, limb ataxia, ataxic dysarthria, or oculomotor dysfunction. Clinically probable MSA was then aimed to balance sensitivity and specificity. This requires at least 2 core clinical features and at least 1 supportive feature. The clinical core features for this category are simplified as autonomic dysfunction (urge urinary incontinence, voiding difficulties, or orthostatic hypotension with a drop in systolic blood pressure of 20 mmHg or more within 10 minutes of standing), parkinsonism, or cerebellar syndromes consist of at least one of gait ataxia, limb ataxia, ataxic dysarthria, or oculomotor dysfunction. Possible prodromal MSA was lastly introduced for research purposes. This category was intended to detect patients in the earliest stages of the disease. Furthermore, the age at onset of 30 years or older is an essential feature. Older age limit was no longer included. Regarding cognitive dysfunction, the time of dementia onset was added and only dementia within 3 years of onset was listed as an exclusion criterion.

In this study, we retrospectively examined the sensitivity and specificity of the new MSA criteria using an autopsy cohort from the Mayo Clinic brain bank, which is independent from the criteria revision task force. We also compared the concordance of the MDS criteria and the second consensus statement.

Methods

Study Cohort

We analyzed confirmed cases from Mayo Clinic brain bank to validate the new criteria. We identified consecutive cases between 1998 and 2021 of 422 clinically diagnosed MSA (including cases in which MSA was included in the differential disease) and 333 pathologically confirmed MSA. Pathologically confirmed MSA included atypical cases and mixed cases, such as MSA with Lewy bodies²³ or progressive supranuclear palsy.²⁴ Among 422 patients with clinically diagnosed MSA, 259 cases had MSA and 163 cases had other pathologic diagnoses. Among 333 pathologically confirmed MSA, 259 had an antemortem clinical diagnosis of MSA, and 74 had other clinical diagnoses. We restricted this study to cases with medical records from neurologists or movement disorder specialists to accurately assess neurologic signs and symptoms. After excluding cases with inadequate medical records, we included 102 cases with a clinical diagnosis of MSA but a pathologic diagnosis other than MSA, 213 cases with a clinical and pathologic diagnosis of MSA, and 37 cases with a pathologic diagnosis of MSA and clinical diagnosis other than MSA. Figure 1 schematically presents the study design. We applied the MDS criteria (clinically established MSA and clinically probable MSA) and the second consensus criteria (probable MSA and possible MSA) and compared the sensitivity, specificity, and area under the curve (AUC) of receiver operating characteristic (ROC) curves between these criteria.

Figure 1. Study Cohort.

MSA = multiple system atrophy; Dx = diagnosis.

Standard Protocol Approvals, Registrations, and Patient Consents

All brain autopsies were performed with the consent of the next-of-kin or an individual with legal authority to grant permission. Deidentified studies using these autopsy samples are considered exempt from human subject research by the Mayo Clinic Institutional Review Board.

Clinical Information

We systematically reviewed the medical records of each case and extracted data from brain bank questionnaires completed by family members. The following information was collected: sex, age at onset, age at death, clinical signs and symptoms, clinical subtypes, clinical diagnosis, and MRI findings. Clinical signs and symptoms included autonomic dysfunction, parkinsonism, l-dopa responsiveness, cerebellar symptoms, postural instability, craniocervical dystonia, speech impairment, dysphagia, Babinski sign, jerky myoclonic postural or kinetic tremor, postural deformities, stridor, inspiratory sighs, cold discolored hands and feet, erectile dysfunction, pathologic laughter or crying, anosmia, fluctuations in cognition, visual hallucinations, cognitive impairment, and downgaze supranuclear palsy or slowing of vertical saccades. Clinical features were considered present if they were documented in the medical records and absent if they were specifically noted as absent or if they were not mentioned. Autonomic dysfunction, parkinsonism, and cerebellar symptoms were determined based on the MDS criteria for its evaluation and on the second consensus statement for its evaluation, respectively.

Clinical subtypes were determined based on the predominant features; patients with predominant parkinsonism were classified as MSA-P and patients with predominant cerebellar symptoms were classified as MSA-C. Patients who presented only with autonomic dysfunction without evident motor symptoms such as parkinsonism or cerebellar symptoms were classified as MSA-A. We defined clinically challenging cases as cases in which the clinical diagnosis was not MSA and cases in which MSA was included in the differential diagnosis, but a definitive diagnosis was not made.

Neuropathologic Assessment

All cases were evaluated with a standardized neuropathologic assessment by a board-certificated neuropathologist (D.W.D.). Paraffin-embedded 5-µm thick sections were studied with hematoxylin and eosin staining and immunohistochemistry with an antibody against α-synuclein (NACP, rabbit polyclonal, 1:3,000, with formic acid pretreatment). A pathologic diagnosis of MSA was established with widespread and abundant glial cytoplasmic inclusions with neurodegeneration in the striatonigral or olivopontocerebellar systems or both. The pathologic subtypes of MSA were assigned based on the severity of affected regions: striatonigral degeneration (MSA-SND) had most severe pathology in striatonigral regions, olivopontocerebellar atrophy (MSA-OPCA) had most severe pathology in olivopontocerebellar regions, and MSA-SND/OPCA had these regions equally affected.²⁵ In addition to these subtypes, minimal change MSA was assigned to a case with no or minimal neuronal loss and gliosis in striatonigral and olivopontocerebellar systems despite widespread distribution of glial cytoplasmic inclusions.^26-30

Prediction of Pathologic Diagnosis by Machine Learning

To examine whether the presence or absence of supportive features alone could predict the pathologic diagnosis of MSA, we performed Extreme Gradient Boosting (XGBoost) using the same approach in our previous study.^31,32 Each supportive feature defined by the MDS criteria was used for the variable. The dataset was split into a training and test dataset with a ratio of 9:1. Bayesian hyperparameter optimization with 5-fold cross-validation was performed on the XGBoost training dataset using the Hyperopt library. After obtaining the optimal hyperparameters, the XGBoost model was trained using 5-fold cross-validation. Before evaluating the test dataset, a postprocessing method, model calibration, was used to adjust the predicted probabilities output by the trained models to match the true probabilities of the target labels. The test dataset was then evaluated using each model, and the average performance was calculated and results were obtained. Feature importance was analyzed to identify the most important features of the task. The feature importance score for each fold was calculated, and then the average of the feature importance scores for all folds was calculated. The analysis was also conducted in male patients and female patients separately. The code was prepared in python, and all analyses were performed on Google Colaboratory.

Statistical Analysis

We conducted statistical analysis using R 4.2.2³³ and GraphPad Prism (version 9.2.0, GraphPad software, La Jolla, CA). A 2-sided unpaired t test was used for parametric comparisons of the 2 groups. One-way analysis of variance with the Tukey test was performed for multiple comparisons. A Fisher exact test was performed for group comparisons of categorical data. The McNemar test was used to compare the sensitivity and specificity between the MDS criteria and the second consensus criteria. ROC curves were generated to compare the overall diagnostic accuracy of each criterion. The Delong test was used to compare each AUC. Statistical tests were adjusted for multiple hypothesis testing using Bonferroni correction. p Values <0.05 were considered statistically significant.

Data Availability

Data not provided in the article because of space limitations may be shared (anonymized) at the request of any qualified investigator for purposes of replicating procedures and results.

Results

Characteristics of Patients

Table 1 summarizes the characteristics of patients included in this study. Patients with MSA diagnosed both clinically and pathologically had younger ages at onset than those diagnosed only clinically or pathologically (58 ± 9 years vs 65 ± 9 and 64 ± 11; p < 0.0001 and p = 0.002). Age at death was also younger in patients with MSA diagnosed both clinically and pathologically compared with those diagnosed only clinically or pathologically (65 ± 8 vs 75 ± 9 and 72 ± 10; p < 0.0001 and p < 0.0001). The disease duration was also shorter in patients diagnosed both clinically and pathologically with MSA compared with those with MSA only diagnosed clinically (8 ± 3 vs 10 ± 6; p = 0.0011).

Table 1.

Characteristics of Patients

	All (N = 352)	Clinical Dx:	MSA	MSA	Non-MSA
		Path. Dx:	Non-MSA	MSA	MSA
			(N = 102)	(N = 213)	(N = 37)
Sex, Male	205, 58%		73, 72%	112, 53%	20, 54%
Age at onset (y)	61 ± 10		65 ± 9	58 ± 9	64 ± 11
Age at death (y)	69 ± 9		75 ± 9	65 ± 8	72 ± 10
Disease duration (y)	8 ± 4		10 ± 6	8 ± 3	8 ± 5
Clinical subtypes
MSA-P	272, 77%		91, 89%	147, 69%	34, 92%
MSA-C	79, 22%		10, 10%	66, 31%	3, 8%
MSA-A	1, 0.3%		1, 0.1%	0, 0%	0, 0%

Abbreviations: A, autonomic dysfunction type; C, cerebellar type; Dx, diagnosis; MSA, multiple system atrophy; P, parkinsonian type.

This study included 352 cases with a clinical or pathologic diagnosis of MSA. The clinical subtypes were as follows: 272 MSA-P (77%), 79 MSA-C (22%), and 1 MSA-A (0.3%). The pathologic examination confirmed MSA in 250 cases and other diagnoses in 102 cases. The pathologic subtypes of pathologically confirmed MSA were as follows: 114 MSA-SND (46%), 61 MSA-OPCA (24%), 70 MSA-SND/OPCA (28%), and 5 minimal change MSA (2%). Among those without MSA, 59 (58%) were Lewy body disease, 20 (20%) were progressive supranuclear palsy, 3 (3%) were corticobasal degeneration, 3 (3%) were cerebrovascular disease, and 17 (17%) were others.

Diagnostic Performance of Each Criterion

Table 2 summarizes the sensitivity and specificity of each criterion for each patient group. When we applied the MDS criteria to our cohort, 40 patients (11%) fulfilled the criteria for clinically established MSA, and 186 patients (53%) met the criteria for clinically probable MSA. Among 40 cases with clinically established MSA, 39 cases were pathologically MSA and 1 case was not MSA. Among 186 cases with clinically probable MSA, 159 cases were pathologically confirmed MSA and 27 cases were not MSA. Thus, the sensitivity and specificity of clinically established MSA were 16% and 99%, whereas the sensitivity and specificity of clinically probable MSA were 64% and 74%. Then, we applied the second consensus statement to our cohort. Among 352 patients, 228 patients (65%, pathologic diagnosis: 179 MSA and 49 non-MSA) met the criteria of probable MSA and 313 patients (89%, pathologic diagnosis: 232 MSA and 81 non-MSA) met the criteria of possible MSA. The sensitivity and specificity of probable MSA and possible MSA under the second consensus criteria were 72% and 52% and 93% and 21%, respectively. The specificity of clinically established MSA by the MDS criteria was significantly higher than that of probable and possible MSA by the second consensus criteria (p < 0.0001 and p < 0.0001), although the sensitivity was significantly lower than the 2 categories by the previous criteria (p < 0.0001 and p < 0.0001). When we compare probable MSA by the MDS criteria and by the second consensus criteria, the sensitivity was not significantly different (p = 0.19), but the specificity was significantly higher with the MDS criteria (p = 0.02). In addition, clinically probable MSA by the MDS criteria showed lower sensitivity and higher specificity compared with possible MSA by the second consensus criteria (p < 0.0001 and p < 0.0001).

Table 2.

Sensitivity, Specificity, and Accuracy of Each Criterion

	Sensitivity (%)	Specificity (%)	Accuracy (%)
All patients (N = 352)
MDS criteria
Clinically established MSA	16	99	40
Clinically probable MSA	64	74	67
Second consensus criteria
Probable MSA	72	52	66
Possible MSA	93	21	72
MSA-P (N = 272)
MDS criteria
Clinically established MSA	13	100	42
Clinically probable MSA	55	77	63
Second consensus criteria
Probable MSA	69	53	64
Possible MSA	91	21	67
MSA-C (N = 79)
MDS criteria
Clinically established MSA	23	90	32
Clinically probable MSA	86	40	80
Second consensus criteria
Probable MSA	78	40	73
Possible MSA	99	10	87
Clinically challenging cases (N = 110)
MDS criteria
Clinically established MSA	8	100	44
Clinically probable MSA	43	79	57
Second consensus criteria
Probable MSA	58	47	54
Possible MSA	84	19	58

Abbreviations: C, cerebellar type; MDS, International Parkinson and Movement Disorder Society; MSA, multiple system atrophy; P, parkinsonian type.

MSA has 2 major clinical subtypes based on the predominant motor symptoms: MSA-P and MSA-C. To investigate potential differences in diagnostic performance between these subtypes, we applied each diagnostic criterion to each clinical subtype (272 MSA-P and 79 MSA-C). Among the 272 patients with MSA-P, 23 (pathologic diagnosis: 23 MSA and 0 non-MSA) met the criteria for clinically established MSA and 121 (pathologic diagnosis: 100 MSA and 21 non-MSA) met the criteria for clinically probable MSA using the MDS criteria, whereas 168 (pathologic diagnosis: 125 MSA and 43 non-MSA) met the criteria for probable MSA and 236 (pathologic diagnosis: 164 MSA and 72 non-MSA) met the criteria for possible MSA under the second consensus statement. In MSA-P, the sensitivity and specificity were 13% and 100% with MDS clinically established MSA, 55% and 77% with MDS clinically probable MSA, 69% and 53% with probable MSA under the second consensus criteria, and 91% and 21% with possible MSA under the second consensus criteria, respectively.

Among the 79 patients with MSA-C, 17 (pathologic diagnosis: 16 MSA and 1 non-MSA) met the criteria for clinically established MSA and 65 (pathologic diagnosis: 59 MSA and 6 non-MSA) met the criteria for clinically probable MSA using the MDS criteria, whereas 60 (pathologic diagnosis: 54 MSA and 6 non-MSA) met the criteria for probable MSA and 77 (pathologic diagnosis: 68 MSA and 9 non-MSA) met the criteria for possible MSA under the second consensus criteria. In MSA-C, the sensitivity and specificity were 23% and 90% with MDS clinically established MSA, 86% and 40% with MDS clinically probable MSA, 78% and 40% with probable MSA under the second consensus criteria, and 99% and 10% with possible MSA under the second consensus criteria, respectively.

We generated ROC curves to compare the MDS criteria with the second consensus criteria (Figure 2A). The AUCs of each criterion were as follows: 0.57 with MDS clinically established MSA, 0.69 with MDS clinically probable MSA, 0.62 with probable MSA under the second consensus criteria, and 0.57 with possible MSA under the second consensus criteria. The AUC of the MDS clinically probable MSA was significantly higher than that of probable MSA and possible MSA under the second consensus criteria (p = 0.0022 and p < 0.0001). ROC curves were also generated for each clinical subtype (Figure 2, B and C). The AUC of the MDS clinically probable MSA was significantly higher than that of probable MSA and possible MSA with the second consensus criteria in MSA-P (p = 0.046 and p < 0.001). The AUC of the MDS clinically probable MSA was significantly higher than that of probable MSA with the second consensus criteria in MSA-C (p = 0.02). The AUCs of clinically established MSA and clinically probable MSA with the MDS criteria were not significantly different between those of MSA-P and MSA-C (p = 0.97 and p = 0.71).

Figure 2. ROC Curves of Each Criterion.

(A) ROC curves in all MSA cases, (B) ROC curves in cases with MSA-P, (C) ROC curves in cases with MSA-C, (D) ROC curves in clinically challenging cases. Red represents clinically established MSA by the MDS criteria, yellow represents clinically probable MSA by the MDS criteria, green represents probable MSA by the second consensus criteria, and yellow-green represents possible MSA by the second consensus criteria. AUC = area under the curve; C = cerebellar type; MDS = International Parkinson and Movement Disorder Society; MSA = multiple system atrophy; P = parkinsonian type; ROC = receiver operating characteristic.

Finally, to examine the performance of each diagnostic criterion under more complicated conditions, we applied the MDS criteria and the second consensus statement for the clinically challenging cases. We identified 37 cases in which antemortem clinical diagnoses were not MSA and 73 cases in which MSA was included in the differential diagnosis, but a definitive clinical diagnosis was not made. Among these 110 cases, 67 cases were pathologically confirmed MSA and 43 cases were pathologically not MSA. When we applied the MDS criteria to these cases, 23 (pathologic diagnosis: 23 MSA and 0 non-MSA) met the criteria for clinically established MSA and 120 met the criteria for clinically probable MSA (pathologic diagnosis: 99 MSA and 21 non-MSA). The sensitivity and specificity were 8% and 100% with clinically established MSA and 43% and 79% with clinically probable MSA. Under the second consensus statement, 62 (pathologic diagnosis: 39 MSA and 23 non-MSA) met the criteria for probable MSA and 91 (pathologic diagnosis: 56 MSA and 35 non-MSA) met the criteria for possible MSA. The sensitivity and specificity were 58% and 47% for probable MSA and 84% and 19% for possible MSA. ROC curves were generated for each criterion (Figure 2D). The AUC of clinically probable MSA with the MDS criteria was significantly higher than that of probable and possible MSA with the second consensus criteria (p = 0.04 and 0.04).

Supporting Features and Exclusion Criteria

The frequency of supportive features and exclusion criteria are summarized in Table 3. The average number of positive supportive features was significantly higher in pathologically confirmed MSA than in non-MSA (2.0 ± 1.5 vs 1.4 ± 1.4; p < 0.0001). Among the supportive features, rapid progression within 3 years of motor onset, stridor, erectile dysfunction before the age of 60 years, and pathologic laughter and crying were significantly more frequent in pathologically confirmed MSA than in non-MSA (rapid progression: 33% vs 21%, p = 0.03; stridor: 21% vs 4%, p < 0.0001; erectile dysfunction: 33% vs 11%, p = 0.0004; pseudobulbar affect: 18% vs 6%, p = 0.003).

Table 3.

Supportive Features and Exclusion Criteria

	Pathologic diagnosis		p Value
	MSA (N = 250)	Non-MSA (N = 102)
Supportive features
Rapid progression	82, 33%	21, 21%	0.028*
Postural instability	116, 46%	40, 39%	0.24
Craniocervical dystonia	3, 1%	0, 0%	0.56
Speech impairment	16, 6%	4, 4%	0.45
Dysphagia	16, 6%	4, 4%	0.45
Babinski sign	47, 19%	16, 16%	0.54
Jerky myoclonic tremor	16, 6%	7, 7%	0.82
Postural deformities	49, 20%	21, 21%	0.24
Stridor	52, 21%	4, 4%	<0.0001****
Inspiratory sighs	5, 2%	0, 0%	0.33
Cold discolored hands and feet	21, 8%	9, 9%	1.0
Erectile dysfunctiona	44/132, 33%	8/73, 11%	0.0004***
Pathologic laughter or crying	45, 18%	6, 6%	0.003**
Exclusion criteria
Significant levodopa responsiveness	20, 8%	8, 8%	1.0
Fluctuations in cognition	5, 2%	13, 13%	0.0001***
Repetitive visual hallucinations	3, 1%	7, 7%	0.0078**
Dementia	2, 1%	12, 12%	<0.0001****
Downgaze supranuclear gaze palsy or slowing of vertical saccades	24, 10%	9, 9%	1.0
Brain MRI findings suggestive of alternative diagnosis	7, 3%	4, 4%	0.74

Abbreviation: MSA = multiple system atrophy.

^aErectile dysfunction was assessed only in male patients (132 in MSA and 73 in non-MSA).

* P < 0.05; ** P < 0.01; *** P < 0.001

Regarding exclusion criteria, 49 of pathologically confirmed MSA met exclusion criteria, whereas 42 of non-MSA met exclusion criteria; MSA had significantly lower frequencies of exclusion criteria than non-MSA (20% vs 41%; p < 0.0001). Among each exclusion criterion, the frequencies of fluctuating cognition, repetitive visual hallucinations, and dementia within 3 years of disease onset were significantly lower in pathologically confirmed MSA than in non-MSA (cognitive fluctuation: 2% vs 13%, p = 0.0001; hallucinations: 1% vs 7%, p = 0.008; dementia: 1% vs 12%, p < 0.0001).

Contributing Factors for High Specificity in Established MSA

Clinically established MSA consists of core clinical features, at least 2 supportive features, MRI findings suggestive of MSA, and no exclusion criteria. To determine which item particularly contributed to the high specificity of established MSA, we applied several hypothetical criteria to our cohort and calculated sensitivity and specificity (Table 4).

Table 4.

Hypothetical Criteria

	Sensitivity (%)	Specificity (%)	Accuracy (%)
A+B+C+D (= clinically established MSA)	16	99	40
A	54	59	55
A+B	40	78	51
A+C	23	93	44
A+D	47	78	56
A+B+C	19	95	41
A+B+D	17	97	40
A+C+D	19	98	42

A: Core clinical features; B: Two or more supportive features; C: MRI findings suggestive of MSA; D: Absence of exclusion criteria.

When the second consensus criteria for probable MSA were used instead of core clinical features of the MDS criteria, the sensitivity and specificity were 19% and 99%, which were similar to the original MDS criteria. When we diagnosed cases only with core clinical features, the sensitivity increased to 54%, yet the specificity decreased to 59%. When the condition of at least 2 supportive features was added to the core clinical features, the sensitivity and specificity were 40% and 78%. Then, when clinical core features and MRI findings suggestive of MSA were used, the sensitivity and specificity were 23% and 93%, which were similar to the criteria for established MSA. The sensitivity and specificity were 47% and 78% with core clinical features and absence of exclusion criteria. Therefore, MRI findings were the most specific item.

Finally, we examined the sensitivity and specificity when one of each of the supportive features, MRI findings, and absence of exclusion criteria were removed from established MSA. In each case, the sensitivity was lower than 20%, but the specificity was 95%–98%.

Prediction of Pathologic Diagnosis by Machine Learning

We also examined whether the machine learning algorithm, XGBoost, could predict pathologic diagnosis of MSA based on clinical information of supporting features. We trained the algorithm using the variables of clinical information, and the resultant model demonstrated an average prediction value of 67%, an F1 score of 45.7%, and AUC of 0.61. Analysis of the feature importance revealed that rapid progression, stridor, pseudobulbar affect, and erectile dysfunction before the age of 60 years were the key variables that significantly contributed to the prediction (Figure 3). When the analysis was conducted by sex, erectile dysfunction before the age of 60 years was most important, followed by rapid progression, pseudobulbar affect, and stridor in male patients, whereas postural deformities were most important, followed by rapid progression, stridor, and pseudobulbar affect in female patients (eFigure 1, links.lww.com/WNL/D188).

Figure 3. Prediction of Pathologic Diagnosis by Machine Learning.

Whether machine learning can predict the pathologic diagnosis of MSA is examined. The trained model demonstrates an average prediction accuracy of 67%. Rapid progression has the highest feature importance among supportive features, followed by stridor and pseudobulbar effect. ED = erectile dysfunction; MSA = multiple system atrophy.

Discussion

This study examined the clinical utility of 2 categories (i.e., clinically established MSA and clinically probable MSA) proposed in the new MDS criteria in an autopsy cohort. Clinically established MSA had excellent specificity and fulfilled the purpose of its creation. Future trials of disease-modifying therapies in MSA can be expected to include only patients with genuine MSA by adapting these stringent criteria. Clinically probable MSA with the MDS criteria showed moderate sensitivity and specificity and had the highest AUC. On the contrary, possible MSA by the second consensus criteria showed more than 90% high sensitivity, whereas the MDS criteria, in exchange for higher specificity, had lower sensitivity. Failure to meet possible MSA by the second consensus criteria might be useful when exclusion of MSA is necessary. In addition, it is notable that the MDS criteria showed high specificity even in clinically challenging cases, although the second consensus statement criteria classified many non-MSA cases as probable or possible MSA, resulting in a low specificity. The AUC of the MDS clinically probable MSA was significantly higher than that of the probable and possible MSA with the second consensus criteria in this population.

To meet the criteria for clinically established MSA, 2 or more supportive features, MRI findings suggestive of MSA, and absence of exclusion criteria are required in addition to clinical core features. The combination of these requirements was considered to be the reason why clinically established MSA could achieve a high specificity. From the results of hypothetical criteria that operationally removed some of composite items, MRI findings suggestive of MSA was the single item that contributed most to specificity. On the contrary, even when MRI was not available, a specificity as high as 97% was achieved if 3 core clinical features were met: 2 or more supportive features and no exclusion items. This indicates that a high specificity can be achieved by clinical findings alone including regions of the world where MRI is difficult to access or in patients who cannot undergo MRI because of a pacemaker or other reasons.

Using a machine learning algorithm, we assessed which supporting feature is most informative in predicting pathologic diagnosis of MSA. This approach resulted in an average prediction value of 67% and an AUC of 0.61, demonstrating modest accuracy in predicting MSA pathology. XGBoost provides “feature importance” as one of its outputs, which can help identify the most important variables that contribute to the classification. In a binary classification task of pathologic diagnosis (MSA vs non-MSA), rapid disease progression was the most important supporting feature that predicted pathologic diagnosis of MSA. This feature was not included in the second consensus criteria,⁷ and this addition may help improve the clinical diagnosis of MSA. Furthermore, it is noteworthy that the most important features differed by sex, with erectile dysfunction before the age of 60 years and postural abnormalities being the most important in men and women, respectively.

This study compared diagnostic utility between clinical subtypes and demonstrated that the AUCs of clinically established and clinically probable MSA by the MDS criteria were not significantly different between MSA-P and MSA-C. In general, MSA-P has a wider range of differential diagnoses such as Lewy body disease and progressive supranuclear palsy than MSA-C. In addition, patients with Lewy body disease often show autonomic dysfunction.³⁴ In fact, 77% of the cases in this study with clinically diagnosed MSA but pathologically not MSA were Lewy body disease or progressive supranuclear palsy. Nevertheless, the new criteria were able to differentiate MSA from non-MSA as well in MSA-P as in MSA-C. The frequencies of MSA-P and MSA-C are different between Western and Asian countries.^35-38 Our cohort is an American brain bank with a high frequency of MSA-P; therefore, validation of the MDS criteria is warranted in regions with a high prevalence of MSA-C.

These results are consistent with those of the recent report from the Queen Square Brain Bank in that clinically established MSA had the highest specificity.³⁹ The sensitivity of clinically established MSA and clinically probable MSA of the MDS criteria in this study were lower than those in the previous study. The cohort from the Queen Square Brain Bank consisted of cases with clinical parkinsonism or cerebellar symptoms, whereas our cohort consisted of cases with clinically diagnosed MSA and cases with pathologically confirmed MSA. In actual clinical practice, cases clearly diagnosed with Parkinson disease, progressive supranuclear palsy, or alcoholic cerebellar disorder are less likely to be differentiated from MSA. Therefore, we selected cases to be more relevant to the clinical situation. As a result, our series is more likely to have a higher frequency of cases in which the diagnosis of MSA is more difficult to make. Nevertheless, it is notable that the new MDS criteria showed favorable results.

We recognize the inherent nature of the retrospective design as a potential limitation; therefore, data from this study should be interpreted with caution. Although we limited our study to cases with medical records from neurologists or movement disorder specialists, we cannot rule out the possibility that some clinical signs may have been missed because the medical records were not compiled in a predetermined format for all items included in the diagnostic criteria. Many patients did not undergo longitudinal medical interviews and neurologic examinations at regular time points, making it difficult to determine when patients met each diagnostic criterion. This is also a limitation of this study. In this respect, a future prospective study is warranted to further examine the validity of the diagnostic criteria. Nevertheless, this study provides the insight into the utility of the MDS criteria by reviewing the largest number of cases with pathologically confirmed MSA to date.

We validated the diagnostic utility of new diagnostic criteria for MSA by MDS using a large autopsy cohort from the Mayo Clinic brain bank. The new category, clinically established MSA, showed high specificity, while clinically probable MSA showed moderate sensitivity and specificity. This study demonstrated the validity of the new diagnostic criteria for MSA proposed by the MDS.

Glossary

AUC

area under the curve

MDS

The International Parkinson and Movement Disorder Society

MSA

multiple system atrophy

MSA-A

multiple system atrophy-autonomic dysfunction

MSA-C

multiple system atrophy-cerebellar

MSA-OPCA

multiple system atrophy-olivopontocerebellar atrophy

MSA-P

multiple system atrophy-parkinsonism

MSA-SD

multiple system atrophy-striatonigral degeneration

ROC

receiver operating characteristic

XGBoost

Extreme Gradient Boosting

Appendix. Authors

Name	Location	Contribution
Hiroaki Sekiya, MD, PhD	Department of Neuroscience, Mayo Clinic; Division of Neurology, Kobe University Graduate School of Medicine	Drafting/revision of the article for content, including medical writing for content; major role in the acquisition of data; study concept or design; and analysis or interpretation of data
Shunsuke Koga, MD, PhD	Department of Neuroscience, Mayo Clinic	Drafting/revision of the article for content, including medical writing for content; major role in the acquisition of data; study concept or design; and analysis or interpretation of data
Aya Murakami, MD, PhD	Department of Neuroscience, Mayo Clinic; Department of Neurology, Kansai Medical University	Major role in the acquisition of data; analysis or interpretation of data
Miki Kawazoe, MD, PhD	Department of Neuroscience, Mayo Clinic	Major role in the acquisition of data; analysis or interpretation of data
Minji Kim, MS	Department of Artificial Intelligence and Informatics Research, Mayo Clinic	Analysis or interpretation of data
Nicholas B. Martin, BS	Department of Neuroscience, Mayo Clinic	Major role in the acquisition of data
Ryan J. Uitti, MD	Department of Neurology, Mayo Clinic	Major role in the acquisition of data; analysis or interpretation of data
William P. Cheshire, MD, MA	Department of Neurology, Mayo Clinic	Major role in the acquisition of data; analysis or interpretation of data
Zbigniew K. Wszolek, MD	Department of Neurology, Mayo Clinic	Drafting/revision of the article for content, including medical writing for content; major role in the acquisition of data; and analysis or interpretation of data
Dennis W. Dickson, MD	Department of Neuroscience, Mayo Clinic	Drafting/revision of the article for content, including medical writing for content; major role in the acquisition of data; study concept or design; and analysis or interpretation of data

Study Funding

The authors report no targeted funding.

Disclosure

H. Sekiya reports fellowships from the Japanese Society of Neurology, the Cell Science Research Foundation, and the Uehara Memorial Foundation; S. Koga is partially supported by the State of Florida Ed and Ethel Moore Alzheimer's Disease Research Program, Mayo Clinic Alzheimer's Disease Research Center Research Grant, and CurePSP Research Grant; A. Murakami reports fellowships from the Japanese Society of Neurology; M. Kawazoe, M. Kim, N.B. Martin, R.J. Uitti, and W.P. Cheshire report no disclosures relevant to the manuscript; Z. K. Wszolek is partially supported by the NIH/NIA and NIH/NINDS (1U19AG063911, FAIN: U19AG063911), Mayo Clinic Center for Regenerative Medicine, the gifts from the Donald G. and Jodi P. Heeringa Family, the Haworth Family Professorship in Neurodegenerative Diseases fund, and the Albertson Parkinson's Research Foundation. He serves as PI or Co-PI on Biohaven Pharmaceuticals, Inc. (BHV4157-206), Neuraly, Inc. (NLY01-PD-1), and Vigil Neuroscience, Inc. (VGL101-01.002, VGL101-01.201, PET tracer development protocol, Csf1r biomarker and repository project, and ultra-high field MRI in the diagnosis and management of CSF1R-related adult-onset leukoencephalopathy with axonal spheroids and pigmented glia) projects/grants. He serves as Co-PI of the Mayo Clinic APDA Center for Advanced Research and as an external advisory board member for the Vigil Neuroscience, Inc., and as a consultant on neurodegenerative medical research for Eli Lilli & Company; D.W. Dennis reports no disclosures relevant to the manuscript. Go to Neurology.org/N for full disclosures.