Abstract
Background
The unified multiple system atrophy rating scale was developed to provide a surrogate measure of disease progression in multiple system atrophy. In the present study, the intrarater agreement of the motor examination part of the unified multiple system atrophy rating scale was determined.
Methods
All patients were first examined face-to-face, whilst being video-recorded, by two senior and two junior investigators. The patients’ videotaped examinations were re-evaluated after 3 months. Intrarater reliability for each item was analyzed by kappa statistics.
Results
Overall weighted κ values were at least substantial or excellent for all unified multiple system atrophy rating scale motor examination items except for ocular motor dysfunction which showed only moderate intrarater agreement. Intrarater reliability was comparable between senior and junior raters with all κ differences being ≤ 0.22.
Conclusions
The motor examination part of the unified multiple system atrophy rating scale was found to have satisfactory intrarater reliability in the present cohort.
Keywords: Multiple system atrophy, MSA, UMSARS, rating scale, EMSA-SG
Introduction
Multiple system atrophy (MSA) is a relentlessly progressive and ultimately fatal neurodegenerative disease characterized clinically by autonomic failure accompanied by characteristic motor features.1, 2 According to applicable diagnostic criteria two motor variants may be distinguished; patients with predominant parkinsonism are designated MSA-P, whereas MSA-C applies to patients having a prominent cerebellar phenotype.3 In addition, three diagnostic categories of increasing certainty were specified: possible, probable, and definite MSA. While definite MSA requires pathological confirmation of widespread glial cytoplasmic inclusions accompanied by a distinct pattern of neurodegeneration,4 possible and probable MSA rely on clinical and neuroimaging findings.3 Patient care is currently restricted to symptom-based therapy because of the current absence of disease-modifying agents.5 The European MSA Study Group (EMSA-SG) recognized the need for a disease-specific rating instrument which may serve as an outcome measure in clinical trials, and thus reliably determine the efficacy of given interventions. We therefore developed and validated the Unified MSA Rating Scale (UMSARS).6 During the validation process, the UMSARS was shown to be a reasonably short, multidimensional, reliable, and valid scale for semiquantitative assessment of Caucasian MSA patients with high internal consistency and substantial-to-excellent interrater agreement.6 Subsequently, two independent groups confirmed the instrument’s sensitivity to change7, 8 which allowed the conduct of multicenter clinical trials exploiting UMSARS as surrogate measure of disease progression.9, 10 We here report the intrarater reliability of the motor examination part of UMSARS as determined in a multicenter validation study.
Methods
A total of 40 patients with a clinical diagnosis of MSA according to the initial Gilman criteria11 were recruited in four EMSA-SG centers (Bordeaux, Innsbruck, London, Toulouse) and assessed on their regular medication. Global disease severity was determined by Hoehn & Yahr staging (H&Y) and a 3-point disease severity scale (SS-3).6 Intrarater validation was performed in a two-step approach. First, all patients were examined face-to-face (live rating) by one senior investigator travelling to the centers (GKW) as well as an additional senior and two junior investigators from each local center (Local rater teams were Bordeaux: FT [senior], IG [junior], FY [junior]; Innsbruck: WP [senior], KS [junior], ADZ [junior]; London: NPQ [senior], MB [junior], TS [junior]; Toulouse: OR [senior], MG [junior], FO [junior]). Board-certified neurologists with long-lasting experience in movement disorders were considered senior investigators whereas the term junior investigators applied to residents in neurology. Each face-to-face examination was recorded on videotape. In a second step, three months after the initial examination, the same investigators re-rated the motor part of the UMSARS examination from the original video recordings. In order to obtain independent assessments, the examiners were not allowed to exchange opinions during evaluations. Finally, as rigidity cannot be judged by inspection, item 6 of the UMSARS motor examination subscale (“Increased tone”) was excluded from the present analysis.
Statistical Analysis
Data were analysed using SPSS 20.0 (SPPS Inc., Chicago, IL, USA) and Excel (Microsoft, Redmond, WA, USA). Intrarater reliability among individual junior and senior raters as well as overall agreement for each UMSARS motor examination item were determined by kappa (κ) statistics.12, 13 Weighted κ values were calculated by means of quadratic disagreement weights14 and interpreted as follows according to recommendations published previously:15 0 to 0.20 slight agreement; 0.21 to 0.40 fair agreement; 0.41 to 0.60 moderate agreement; 0.61 to 0.80 substantial agreement; 0.81 to 1.00 excellent agreement. Intrarater agreement of UMSARS motor examination subscore of live examination and video rating was determined using intraclass correlation coefficients (ICC) derived from a one-way random effects analysis of variance model.16
Results
In the present cohort, MSA-P was more frequent than MSA-C (MSA-P:MSA-C 1.8:1) with diagnostic certainty being considered probable in 32 and possible in 8 out of 40 cases. Mean (standard deviation) age at symptom-onset and disease duration were 57.0 (8.5) and 6.0 (4.2) years, respectively. Further demographic and clinical data are given in Table 1.
Table 1.
Demographics and clinical characteristics
| Patients (n) | 40 | |
| Gender (male/female, n) | 16/24 | |
| MSA-P/C (n) | 26/14 | |
| MSA-possible/probable (n) | 8/32 | |
| Age at onset, years (mean ± SD) | 57.0 ± 8.5 | |
| Disease duration, years (mean ± SD) | 6.0 ± 4.2 | |
| H & Y stage, median (range) | 4 (2 – 5) | |
| 3-point severity scale (n) | 5 / 13 / 22 | |
| UMSARS (mean ± SD, median, IQR) | 52.3 ± 18.3, 53.5, 31.0 | |
| UMSARS ADL (mean ± SD, median, IQR) | 25.8 ± 9.6, 25.0, 16.0 | |
| UMSARS ME (mean ± SD, median, IQR) | 26.5 ± 9.6, 25.5, 13.5 | |
MSA-P/C … multiple system atrophy-parkinsonian subtype/cerebellar subtype; H & Y stage … Hoehn & Yahr staging (0 – 5); 3-point severity scale … mild / moderate / severe; UMSARS … Unified MSA rating scale (0 – 104); UMSARS ADL … UMSARS subscale “Activities of daily living” (0 – 48); UMSARS ME … UMSARS subscale “Motor examination” (0 – 56)
Intrarater agreement
The results of the intrarater reliability analysis are summarized in Table 2. Overall weighted κ values were at least substantial (κ (w) = 0.6–0.8) or excellent (κ (w) ≥ 0.8) for all UMSARS motor examination items except for ocular motor dysfunction which showed moderate intrarater agreement. Intrarater reliability was comparable between senior and junior raters with all κ discrepancies being ≤ 0.22. Finally, intraclass correlation coefficients proved the UMSARS motor examination subscore to be reliable.
Table 2.
UMSARS motor examination: Intrarater reliability. κ (w) over all centres; 0– 0.20, slight agreement; 0.21– 0.40, fair agreement; 0.41– 0.60, moderate agreement; 0.61– 0.80, substantial agreement; 0.81–1.00, excellent agreement.
| Item | κ (w) | ||
|---|---|---|---|
| SR | JR | Overall | |
| 1 Facial expression | 0.75 | 0.71 | 0.74 |
| 2 Speech | 0.84 | 0.75 | 0.79 |
| 3 Ocular motor dysfunction | 0.57 | 0.44 | 0.50 |
| 4 Tremor at rest | 0.73 | 0.73 | 0.73 |
| 5 Action tremor | 0.82 | 0.69 | 0.76 |
| 6 Increased tone | Not applicable | ||
| 7 Rapid alternating movement of hands | 0.68 | 0.60 | 0.64 |
| 8 Finger tapping | 0.78 | 0.74 | 0.76 |
| 9 Leg agility | 0.80 | 0.60 | 0.71 |
| 10 Heel-Shin test | 0.84 | 0.82 | 0.83 |
| 11 Arising from chair | 0.96 | 0.97 | 0.96 |
| 12 Posture | 0.86 | 0.84 | 0.85 |
| 13 Body sway | 0.90 | 0.91 | 0.91 |
| 14 Gait | 0.89 | 0.90 | 0.89 |
| ICC of UMSARS II score - 1st examination | 0.98 | 0.97 | 0.98 |
Abbreviations: ICC … intraclass coefficient; κ (w) … weighted kappa; SR … senior rater; JR … junior rater; UMSARS … unified MSA rating scale.
Discussion
It was previously demonstrated that the UMSARS is a MSA-specific semi-quantitative assessment instrument showing high internal consistency as well as substantial to excellent interrater reliability6. Moreover, two independent groups have shown that UMSARS scores are sensitive to change.7, 8 To complete the validation process of the scale, we here report the results of an intrarater validation study.
The motor examination part of the UMSARS underwent intrarater reliability testing by comparison of live rating to subsequent video analysis of the initial examination 3 months later. Our results revealed that all but one item (ocular motor dysfunction) had substantial to excellent intrarater agreement. Ocular motor dysfunction showed moderate agreement which might be explained by the challenge of scoring eye movement using video analysis. As expected, κ values were higher in the intrarater reliability study compared to the interrater reliability study published previously.6 Moreover, intrarater agreement was comparable among senior and junior examiners with differences in κ values being below or equal to 0.20 except for item 3 (ocular motor dysfunction) which had a Δκ of 0.22. The latter observation confirms the previous finding that junior investigators can use the UMSARS reliably once they are trained in applying the UMSARS and provided they receive detailed instructions.6
The present study further underscores that the UMSARS has satisfactory intrarater reliability as a disease-specific and multidimensional rating instrument for semi-quantitative assessment of MSA patients. However, some limitations have to be acknowledged. Video analysis may not perfectly replicate a face-to-face examination. An intrarater study with two live examinations separated by a certain time interval is difficult to perform in MSA. Choosing an appropriate time interval between two live examinations is limited by the rapidly progressive disease course of MSA. Thus, as demonstrated by two studies with the serial application of the UMSARS at patients with MSA,7, 8 the progressive nature of the disease would have led to a deterioration of the UMSARS score and several of the UMSARS items at follow-up. On the other hand, a short follow-up between two potential live examinations might have introduced a bias towards overestimation of the intrarater reliability values. Therefore, we chose this study design of a videotaped face-to-face examination of the patients followed by a re-evaluation using the same video 3 months later. Furthermore, since the scale was validated in European Caucasians, its validity and applicability in a different racial or ethnic context remains to be established. Finally, UMSARS does not cover every aspect of the complex phenomenology of MSA, so that other validated scales may need to be developed to document a more comprehensive picture of the disease.
Acknowledgment
We would like to thank all patients and families as well as referring physicians for their support. We also thank D. Burtscher, H. Granbichler, S. Michlmair, U. Zijerveld, and M. Stampfer-Kountchev (all of Innsbruck, Austria) for their help with the coordination and conduct of the validation study. This study was supported by funds of the 5th framework program of the European Community (QLK6-CT-2000-00661) and the Austrian Science Fund (FWF): F04404-B19.
Funding sources for the present study: This study was supported by funds of the 5th framework program of the European Community (QLK6-CT-2000-00661) and the Austrian Science Fund (FWF): F04404-B19.
Financial disclosure/conflict of interest related to the manuscript:
| Author | Financial disclosure/conflict of interest related to the manuscript |
|---|---|
| Florian Krismer | Nothing to disclose |
| Klaus Seppi | Nothing to disclose |
| François Tison | Nothing to disclose |
| Cristina Sampaio | Nothing to disclose |
| Anja Zangerl | Nothing to disclose |
| Cecilia Peralta | Nothing to disclose |
| Farid Yekhlef | Nothing to disclose |
| Imad Ghorayeb | Nothing to disclose |
| Fabienne Ory-Magne | Nothing to disclose |
| Monique Galitzky | Nothing to disclose |
| Tommaso Scaravilli | Nothing to disclose |
| Maria Bozi | Nothing to disclose |
| Carlo Colosimo | Nothing to disclose |
| Felix Geser | Nothing to disclose |
| Olivier Rascol | Nothing to disclose |
| Niall P. Quinn | Nothing to disclose |
| Werner Poewe | Nothing to disclose |
| Gregor K. Wenning | Nothing to disclose |
Authors' Roles
- Research project:
- Conception
- Organization
- Execution
- Statistical Analysis:
- Design
- Execution
- Review and Critique
- Manuscript:
- Writing of the first draft
- Review and Critique
| Author | Financial disclosure/conflict of interest related to the manuscript |
|---|---|
| Florian Krismer | 2a, 2b, 3a |
| Klaus Seppi | 1a, 1b, 1c, 2a, 2c, 3b |
| François Tison | 1a, 1b, 1c, 2c, 3b |
| Cristina Sampaio | 1a, 2c, 3b |
| Anja Diem-Zangerl | 1a, 1b, 1c, 2a, 2c, 3b |
| Cecilia Peralta | 2a, 2b, 3b |
| Farid Yekhlef | 1a, 1b, 1c, 2c, 3b |
| Imad Ghorayeb | 1a, 1b, 1c, 2c, 3b |
| Fabienne Ory-Magne | 1a, 1b, 1c, 2c, 3b |
| Monique Galitzky | 1a, 1b, 1c, 2c, 3b |
| Tommaso Scaravilli | 1a, 1b, 1c, 2c, 3b |
| Maria Bozi | 1a, 1b, 1c, 2c, 3b |
| Carlo Colosimo | 1a, 2c, 3b |
| Felix Geser | 1b, 3b |
| Olivier Rascol | 1a, 1b, 1c, 2c, 3b |
| Niall P. Quinn | 1a, 1b, 1c, 2c, 3b |
| Werner Poewe | 1a, 1b, 1c, 2a, 2c, 3b |
| Gregor K. Wenning | 1a, 1b, 1c, 2a, 2c, 3b |
Financial Disclosures
Florian Krismer
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Division of Neurobiology, Department of Neurology, Innsbruck Medical University |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Klaus Seppi
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | Novartis, Boehringer Ingelheim, Lundbeck, Schwarz Pharma, UCB, GlaxoSmithKline |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Department of Neurology, Innsbruck Medical University |
| Partnerships | None |
| Contracts | None |
| Honoraria | Novartis, Boehringer Ingelheim, Lundbeck, Schwarz Pharma, UCB, GlaxoSmithKline |
| Royalties | None |
| Grants | None |
| Other | None |
François Tison
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | ADDEX Pharma |
| Expert Testimony | None |
| Advisory Boards | Novartis, GlaxoSmithKline and Boehringer Ingelheim |
| Employment | Le Centre hospitalier universitaire de Bordeaux |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | Travel grants from Novartis, Lundbeck and UCB and a research grant from the Michael J. Fox Foundation |
| Other | Lecture fees from Novartis, GlaxoSmithKline and UCB |
Cristina Sampaio
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | None |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Anja Zangerl
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Department of Neurology, Innsbruck Medical University |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Cecilia Peralta
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | Teva, Boehringer-Ingelheim |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Department of Neurology, CEMIC Medical University |
| Partnerships | None |
| Contracts | None |
| Honoraria | Boehringer-Ingelheim, Teva, Buxton, Novartis |
| Royalties | None |
| Grants | None |
| Other | None |
Farid Yekhlef
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Centre hospitalier René Dubos, 95 300 Pontoise, France |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | Biogen Idec |
Imad Ghorayeb
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Institut des Maladies Neurodégénératives, Université Victor Segalen Bordeaux 2, CNRS UMR 5293 |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Fabienne Ory-Magne
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | Served on Scientific Advisory Board for Novartis |
| Employment | Department of Neurology, CHU Toulouse, France |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Monique Galitzky
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Clinical Investigation Center- CHU Toulouse - France |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Tommaso Scaravilli
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | U.O.C. Neurologia, Ospedale dell'Angelo, Venice |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Maria Bozi
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | “Hygeia” Diagnostic and Therapeutic Medical Center, Athens, Greece. EOPYY Branch of Nea Ionia, Athens, Greece |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Carlo Colosimo
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | Allergan, Ipsen, Lundbeck, UCB/Schwarz |
| Employment | Sapienza University |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | CIC Edizioni Internazionali Publishers, Cambridge University Press |
| Grants | Teva |
| Other | None |
Felix Geser
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Universitätsklinikum Ulm, Medizinische Fakultät der Universität Ulm |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
Olivier Rascol
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | Abbott, Addex, BIAL, Boehringer Ingelheim, Impax Pharmaceuticals, Lundbeck, Merck Serono, Movement Disorders Society, Novartis, Oxford Biomedica, Teva, Schering-Plough, UCB and XenoPort |
| Expert Testimony | None |
| Advisory Boards | Abbott, Addex, Impax Pharmaceuticals, Lundbeck, Merck Serono, Merz, Novartis, Oxford Biomedica, Schering-Plough, Teva, UCB and XenoPort |
| Employment | None |
| Partnerships | None |
| Contracts | None |
| Honoraria | Boehringer Ingelheim, GSK, Lundbeck, MDS, Novartis, Teva, UCB |
| Royalties | None |
| Grants | CHU de Toulouse, France-Parkinson, INSERM-DHOS Recherche Clinique Translationnelle, MJFox Foundation, Programme Hospitalier de Recherche Clinique, Boehringer Ingelheim, Lundbeck, Teva and UCB |
| Other | None |
Werner Poewe
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | Astra Zeneca, Teva, Novartis, GSK, Boehringer-Ingelheim, UCB, Orion Pharma and Merck Serono |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Department of Neurology, Innsbruck Medical University |
| Partnerships | None |
| Contracts | None |
| Honoraria | Astra Zeneca, Teva, Novartis, GSK, Boehringer-Ingelheim, UCB, Orion Pharma, and Merck Serono (Lecture fees) |
| Royalties | None |
| Grants | Movement Disorders Society and the Michael J. Fox Foundation for Parkinson's Research |
| Other | None |
Niall P. Quinn
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | Expert testimony on 2 cases of Parkinson’s disease |
| Advisory Boards | None |
| Employment | None |
| Partnerships | None |
| Contracts | None |
| Honoraria | Lecture honoraria from Orion Pharma |
| Royalties | None |
| Grants | None |
| Other | None |
Gregor K. Wenning
| Stock Ownership in medically-related fields | None |
| Intellectual Property Rights | None |
| Consultancies | None |
| Expert Testimony | None |
| Advisory Boards | None |
| Employment | Division of Neurobiology, Department of Neurology, Innsbruck Medical University |
| Partnerships | None |
| Contracts | None |
| Honoraria | None |
| Royalties | None |
| Grants | None |
| Other | None |
References
- 1.Wenning GK, Colosimo C, Geser F, Poewe W. Multiple system atrophy. Lancet Neurology. 2004;3(2):93–103. doi: 10.1016/s1474-4422(03)00662-8. [DOI] [PubMed] [Google Scholar]
- 2.Kollensperger M, Geser F, Ndayisaba JP, et al. Presentation, diagnosis, and management of multiple system atrophy in Europe: final analysis of the European multiple system atrophy registry. Mov Disord. 2010;25(15):2604–2612. doi: 10.1002/mds.23192. [DOI] [PubMed] [Google Scholar]
- 3.Gilman S, Wenning GK, Low PA, et al. Second consensus statement on the diagnosis of multiple system atrophy. Neurology. 2008;71(9):670–676. doi: 10.1212/01.wnl.0000324625.00404.15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Trojanowski JQ, Revesz T. Proposed neuropathological criteria for the post mortem diagnosis of multiple system atrophy. Neuropathology and applied neurobiology. 2007;33(6):615–620. doi: 10.1111/j.1365-2990.2007.00907.x. [DOI] [PubMed] [Google Scholar]
- 5.Stefanova N, Bucke P, Duerr S, Wenning GK. Multiple system atrophy: an update. Lancet Neurol. 2009;8(12):1172–1178. doi: 10.1016/S1474-4422(09)70288-1. [DOI] [PubMed] [Google Scholar]
- 6.Wenning GK, Tison F, Seppi K, et al. Development and validation of the Unified Multiple System Atrophy Rating Scale (UMSARS) Mov Disord. 2004;19(12):1391–1402. doi: 10.1002/mds.20255. [DOI] [PubMed] [Google Scholar]
- 7.Geser F, Wenning GK, Seppi K, et al. Progression of multiple system atrophy (MSA): a prospective natural history study by the European MSA Study Group (EMSA SG) Mov Disord. 2006;21(2):179–186. doi: 10.1002/mds.20678. [DOI] [PubMed] [Google Scholar]
- 8.May S, Gilman S, Sowell BB, et al. Potential outcome measures and trial design issues for multiple system atrophy. Mov Disord. 2007;22(16):2371–2377. doi: 10.1002/mds.21734. [DOI] [PubMed] [Google Scholar]
- 9.Holmberg B, Johansson JO, Poewe W, et al. Safety and tolerability of growth hormone therapy in multiple system atrophy: a double-blind, placebo-controlled study. Mov Disord. 2007;22(8):1138–1144. doi: 10.1002/mds.21501. [DOI] [PubMed] [Google Scholar]
- 10.Dodel R, Spottke A, Gerhard A, et al. Minocycline 1-year therapy in multiple-system-atrophy: effect on clinical symptoms and [(11)C] (R)-PK11195 PET (MEMSA-trial) Mov Disord. 2010;25(1):97–107. doi: 10.1002/mds.22732. [DOI] [PubMed] [Google Scholar]
- 11.Gilman S, Low P, Quinn N, et al. Consensus statement on the diagnosis of multiple system atrophy. American Autonomic Society and American Academy of Neurology. Clinical autonomic research : official journal of the Clinical Autonomic Research Society. 1998;8(6):359–362. doi: 10.1007/BF02309628. [DOI] [PubMed] [Google Scholar]
- 12.Cohen J. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement. 1960;20(1):37–46. [Google Scholar]
- 13.Fleiss JL. Statistical methods for rates and proportions. Wiley; 1981. [Google Scholar]
- 14.Agresti A. Categorical data analysis. Wiley; 1990. [Google Scholar]
- 15.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 16.Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin. 1979;86(2):420–428. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]