Abstract
Background
Movement Disorders Society Unified Parkinson's Disease Rating Scale (MDS‐UPDRS) and Unified Dyskinesia Rating Scale (UDysRS) were developed as standard tools to rate Parkinson's disease (PD) and drug‐induced dyskinesias of PD. As these scales have become widely used, there is a need for translation to non‐English languages. Here we present the standardization for the Turkish translations.
Methods
The scales were translated into Turkish and then back‐translated to English. These back‐translations were reviewed by the MDS team. After cognitive pretesting, movement disorder specialists from nine centers tested 352 patients for MDS‐UPDRS, and 250 patients for UDysRS. Confirmatory factor analyses (CFAs) were used to determine if the factor structures for the reference standards could be confirmed in the Turkish data. The comparative fit indexes (CFIs) for the scales were required to be 0.90 or higher. Exploratory factor analyses (EFAs) were conducted to explore the underlying factor structure without the constraint of a pre‐specified factor structure.
Results
For both scales, the CFIs were 0.94 or greater as compared to the reference standard factor structures. The factor structures were consistent with that of reference standards, although there were some differences in some areas as compared to the EFA of the reference standard dataset. This may be due to the inclusion of patients with different stages of PD and different cultural properties of raters and patients.
Conclusions
These results demonstrate that the Turkish translations of MDS‐UPDRS and UDysRS have adequate clinimetric properties. They are established as the official translations and can be reliably used in Turkish speaking populations.
Keywords: Dyskinesia, Parkinson's disease, rating scale
Measurement is a sine qua non of science.1 The most frequently used way to measure clinical parameters, such as symptoms or signs of a disease, is rating scales. A rating scale should bereliable (a given clinical parameter gets the same score by every rater at every time point in time) and valid (measures what it intends to measure). Demonstration that a rating scale is reliable and valid, requires clinical testing supported by appropriate statistics. This has been done by the International Parkinson and Movement Disorders Society (MDS), for the Movement Disorders Society‐sponsored Unified Parkinson's Disease Rating Scale (MDS‐UPDRS) and Unified Dyskinesia Rating Scale (UDysRS).
The UPDRS has often been the main outcome measure in clinical trials in Parkinson's disease (PD).2 The scale consisted of four different parts based on mentation, behavior, and mood; daily life activities, motor examination, and treatment complications. It was originally designed to become a single scale used by the clinicians all around the world so that comparisons between different studies could be made reliably.3 Although it fulfilled this original aim, the scale had limitations due to ambiguous wording, weakness in assessing various non‐motor symptoms, lack of uniform instructions, and several metric flaws.4, 5 Thus, the MDS‐UPDRS was designed to be more comprehensive than the original UPDRS, with new items devoted to several non‐motor symptoms of PD. It also provides more detailed instructions and includes both patients and caregivers for the assessment of various motor and non‐motor aspects of daily living.6 The five‐point scoring system for each item was retained, and clinical anchors of normal (0), slight (1), mild (2), moderate (3), and severe (4) were added to provide a consistency across ratings.
The UDysRS was developed as a comprehensive tool to rate all aspects of drug‐induced dyskinesias in PD.7 It has four parts: (1) historical disability (patient perceptions) of on‐dyskinesia impact; (2) historical disability (patient perceptions) of off‐dystonia impact; (3) objective ımpairment (dyskinesia severity, anatomical distribution, and type [choreic or dystonic] based on four activities observed or video‐recorded); (4) objective disability based on part three activities.
Simple translation of a validated rating scale into another language does not necessarily mean that the scale would also be valid and reliable in the translated language. For instance, the word “trick” has associations with “wittiness” in English, whereas it could mean “dishonesty” in Turkish; hence, a direct translation of “sensory trick” would make a cervical dystonia scale's sensory trick item useless. Consequently, translation of a clinical scale, similar to its development, requires an equally rigorous clinical validation and advanced statistics.
The method for the validation of the scales have been previously determined by the original team involved in the development and translation processes of these scales.8 The purpose of this approach is to assure that the interpretation of the scales are consistent acrossall languages.Here we present the Turkish validation studies of MDS‐UPDRS and UDysRSbased on this predetermined method.
Methods
Translation
The scales were translated into Turkish by a team of Turkish investigators who were fluent in English and led by one of the authors (MCA). These translations were then back‐translated into English by a team not involved in the original translation, but acquainted with the use of both scales. These back‐translations were reviewed by an US team (Stebbins, Goetz, LaPelle, Tilley).
Cognitive Pretesting
Cognitive pretesting is a qualitative approach assessing tool usability (or ease of completion) in terms of task difficulty for examiner and respondent as well as respondent interest, attention span, discomfort and comprehension.9 All items in the MDS‐UPDRS were subjected to pretesting. Questions included in cognitive pretesting were: Instructions to Raters and Instructions to Patients, Time Spent with Dyskinesia, Time with OFF Dystonia, Chewing and Swallowing, Hobbies, Walking and Balance, Exciting or Emotional Settings, Objective Impairment Ratings, and Objective Disability Ratings. Based on the initial cognitive pretesting results, translation, back‐translation, and cognitive pretesting phases were repeated for items in need of improvement. Over all, five movement disorder specialists, 25 patients for MDS‐UPDRS, and 13 for UDysRS were involved in this process. Selected items were reassessed based on the initial results. As no problems were noted after the cognitive pretesting, the final translation was obtained.
Testing of the Turkish Version
A team of experienced Turkish movement disorder specialists from nine centers, including Ankara University, Cumhuriyet University, Ege University, Eskisehir Osmangazi University, Gazi University, Hacettepe University, Istanbul University, Mersin University, and Uludag University took part in the testing phase. All raters were trained on the MDS‐UPDRS and UDysRS through the MDS Certificate Program.10, 11 Ethics approval was obtained from the local ethics committee. Only participants providing informed consents were included. The sample size requirements for each scale were based on the number of items and scaling characteristics, thus a minimum of 350 patients were required for the MDS‐UPDRS validation and a minimum of 250 paients were required for the UDysRS validation.12 Patients with missing values within a part in the MDS‐UPDRS or the UDysRS were excluded from the analysis of that specific part. Patient data for both scales were anonymously transferred to a secure website.
Data Analysis
MDS‐UPDRS M‐plus, Version 7 and UDysRS M‐plus, Version 6.11 were used to perform confirmatory and secondary exploratory factor analyses. An unweighted least squares (ULS) approach to factor estimation was used, minimizing the sum of squared differences between observed and estimated correlation matrices not counting diagonal elements. An orthogonal CF‐VARIMAX rotation setting the factors to be uncorrelated was used to assist in interpretation of the factors.
In the UDysRS, Question 1 (time of ON dyskinesia) and Question 12 (time of OFF dystonia) were considered as descriptive indices, rather than impairment or disability measures, and were omitted from the factor analysis. In order to maximize the accuracy of these time indices, three clarifying statements were added to reconcile time‐based questions with the patient/caregiver questionnaire and interview items. In the initial instructions, the rater was informed to review the patient questionnaire after it was filled‐out. This was done to make sure that when individual‐item scores pointed to the presence of dyskinesia or dystonia (over the previous week), the time‐based items supported this ascertion (rating 1, 2, 3, or 4, but not zero). The same alerting question was presented at the end of each questionnaire section (ON dyskinesia and OFF dystonia).
Primary Analysis
A confirmatory factor analysis (CFA) was conducted to determine if the factor structure for the Turkish scales could be confirmed against the reference standard versions (English language MDS‐UPDRS and Spanish version of the UDysRS).6, 13 The CFA was conducted for the total score of the UDysRS and for the total score from each of the four parts of the MDS‐UPDRS (Parts I to IV) with the Turkish data constrained to fall into the factors defined by the reference standard. The CFA results were evaluated based on the comparative fit ındex (CFI). To establish a successful translation of the scales, and to designate these as OFFICIAL MDS translations, the CFI for each part (I‐IV) of the translated MDS‐UPDRS and UDysRS was required to be 0.90 or higher relative to the reference standard versions. Mean and variance adjusted weighted least square (WLSMV) estimator was used to confirm model fit. For both the MDS‐UPDRS and UDysRS, goodness of fit was determined by root mean square error of approximation (RMSEA). This is a population‐based index relying on the noncentral χ2 distribution, which is the distribution of the fitting function when the fit of the model is not perfect.
Secondary Analysis
As secondary analysis an exploratory factor analysis (EFA) of the Turkish version of the scales was conducted to explore the underlying factor structure without the constraint of a pre‐specified factor structure. An unweighted least squares (ULS) approach was used. To determine the factor number to be retain in each MDS‐UPDRS Part and UDysRS, a subjective SCREE test, alongside the information from the SCREE plot of the reference standard versions, was used. The subjective SCREE test uses a scatter plot of eigenvalues, plotted against their ranks with respect to magnitude, to extract as many factors as there are eigenvalues that fall before the last large drop (i.e., an “elbow” shape) in the plot.14 Once the factors were chosen, an item was retained in a given factor if the factor loading for that item was 0.40 or greater. To assist interpretation of the factors, an orthogonal CF‐VARIMAX rotation was used which sets the factors to be uncorrelated.
Results
The demographic characteristics of the patient population are shown in Table 1 and Fig. 1. The study population included native Turkish‐speaking PD patients with or without dyskinesia.
Table 1.
The demographic characteristics of the patients
| Total | Male, n (%) | Age | Duration of dyskinesia (years) | Dyskinesia severity range | Duration of Parkinson's disease (years) | |
|---|---|---|---|---|---|---|
| Turkish | 250 | 136 (54.4) | 63.5 (10.4) | 3.8 (3.6) | 0‐82 | 11.0 (5.9) |
| Reference Standard (Spanish) | 253 | 122 (48.2) | 69.2 (10.5) | 4.9 (4.6) | 12.5 (6.8) | |
| Turkish | 352 | 201 (57.1) | 63.4 (10.6) | na | na | 9.5 (6.2) |
| Reference Standard (English) | 876 | 554 (63.2) | 67.5 (10.9) | na | na | 8.3 (6.7) |
All variables are reported as mean (standard deviation) unless stated otherwise. na=not available.
Figure 1.
The percentage of patients recruited for MDS‐UPDRS/UDysRS (the overall distribution of population) from each region in Turkey.
Cognitive Pretesting
MDS‐UPDRS
Five raters used structured interview to evaluate a total of 15 PD patients for cognitive pretesting. On the first round of cognitive pretesting, minor changes were suggested for the instructions and selected wording for individual items. As most hobbies among Turkish people are gender‐specific, it was difficult to find a hobby which could be shown as an example that would be valid for both sexes; eventually “following a series on television” was chosen. Following these modifications, a second round of cognitive pretesting was performed with an independent sample of 10 PD patients and 3 raters. No items were identified as problematic during this second round. The modified version of the scale was approved as the official working document of the Turkish MDS‐UPDRS for testing in a larger group of PD patients.
UDysRS
The UDysRS was performed the same way as the MDS‐UPDRS; the first round with 10 patients and three raters. After some minor changes in wording, the second round was completed with three patients and two raters with no additional issues. The modified version of the scale was approved as the official working document of the Turkish UDysRS.
Primary Analysis: Confirmatory Factor Analyses (CFA)
For all four parts of the Turkish MDS‐UPDRS, the CFI, in comparison with the reference standard factor structure, was 0.94 or greater (Table 2). For UDysRS, the CFI, in comparison with the reference standard factor structure, was 0.98. Our pre‐specified criterion was a CFI of 0.90 or greater. Hence, the pre‐specified reference standard factor structure was confirmed in the Turkish dataset.
Table 2.
CFA model fit for Turkish dataset for Movement Disorders Society‐Unified Parkinson's Disease Rating Scale (MDS‐UPDRS) and UDysRS (Unified Dyskinesia Rating Scale)
| MDS‐UPDRS | |
|---|---|
| Part I: Non‐Motor Aspects of Experiences of Daily Living (a 2‐factor model)a | |
| Turkish | CFI = 0.94, RMSEA = 0.08 (352 patients) |
| English language | CFI = 0.96, RMSEA = 0.06 (849 patients) |
| Part II: Motor Aspects of Experience of Daily Living (a 3‐factor model) | |
| Turkish | CFI = 0.98, RMSEA = 0.10 (352 patients) |
| English language | CFI = 0.97, RMSEA = 0.09 (851 patients) |
| Part III: Motor Examination (a 7‐factor model) | |
| Turkish | CFI = 0.94, RMSEA = 0.09 (352 patients) |
| English language | CFI = 0.95, RMSEA = 0.07 (801 patients) |
| Part IV: Motor Complications (a 2‐factor model) | |
| Turkish | CFI = 0.98, RMSEA = 0.16 (352 patients) |
| English language | CFI = 1.00, RMSEA = 0.04 (848 patients) |
| UDysRS | |
| Turkish language | CFI = 0.98, RMSEA = 0.10 (250 patients) |
| Reference Standard | CFI = 0.98, RMSEA = 0.08 (247 patients) |
CFI, comparative fit index; RMSEA, root mean square error of approximation
DDS was not included in this analysis as it did not load on any factor. The MDS has been notified of the need for correction to factor 1.6
Secondary Analysis: Exploratory Factor Analyses (EFA)
Mds‐updrs
Similar numbers of factors were extracted for all four parts of MDS‐UPDRS from the Turkish and reference standard versions.
Our EFA analysis for the Turkish dataset differed from the EFA of the reference standard dataset in some areas. For Part I, in contrast to the English version of the MDS‐UPDRS, “fatigue” loaded on factor 2. “Sleep problems, daytime sleepiness, and pain and other sensations” did not load on any of the two factors. For Part II, “speech” and “eating tasks” loaded on more than one factor with factor loading ≥ 0.40. In Part III, “rigidity” items loaded on factor 3, 4, and 5. “Toe tapping” and “leg agility” items loaded on factor 1, 4, and 5. In Part IV, “painful off state dystonia” loaded on both factors.
UDysRS
From the SCREE Plot we extracted three factors. The factor structure of Turkish UDysRS was consistent with that of reference standard UDysRS, differing from the EFA of the reference standard dataset only in few areas. Most of the items that loaded on different factors in the two versions also had cross loadings on multiple factors.
Discussion
The objective of this study was to develop a validated Turkish translation of the MDS‐UPDRS and UDysRS, equivalent to the original version.
The original English version of these scales included clear instructions to ensure all patients could understand the questions easily. We emphasized the need for this clarity during the translation process. Since both rating scales had CFI's that were higher than the pre‐specified 0.90 in comparison with the reference standard language factor structure, our translated scales appear to be measuring similar constructs as the original versions. Thus, both of the scales were accepted as official MDS Turkish translations (Table 2).
As several different centers contributed to this study, we were able to collect data from all of the regions in Turkey. Although the distribution of our patients was not the same as overall population distribution, we believe our results still provide an adequate geographical representation of the country as a number of patients from all of the geographical regions were included (Fig. 1).15 Therefore, these scales can be reliably used in Turkish‐speaking populations living in Turkey, Cyprus, and European countries. However, they cannot be used in Turkish‐speaking populations living in Azerbaijan, Turkmenistan, Uzbekistan, Kazakhstan, Syria, and Iraq as there are substantial differences in the dialects. Original Turkish texts can be found at the International Parkinson and Movement Disorders Society's web page (http://www.movementdisorders.org/MDS/Education/Rating-Scales.htm), and can be used for research or clinical care programs with the society's permission.
The purpose of EFA is to examine subtle differences in the factor loadings that may be observed across various languages and cultures.Similar to other translations of the rating scales, the Turkish dataset differed from the reference standard language datasets in some areas9 The variation may stem from different cultural backgrounds of raters and patients. Perceptions of some symptoms may show transcultural variability. For example, in the Japanese validation of MDS‐UPDRS, a much greater percentage (62.2%) of patients had zero scores for cognitive impairment when compared to the English‐speaking sample (48.9%).16 Another source of variability may be inclusion of patients with different stages of PD. Since we conducted validation studies of both MDS‐UPDRS and UDysRS simultaneously, our first 250 patients were selected to have dyskinesias by design. This may be a cause of variability specific to Turkish dataset.
Unlike the MDS‐UPDRS, the UDysRS was not validated in a large population in the original English version. The first large‐scale field testing was done with the Spanish version. Thus, reliability and construct validity testing were done by comparing Spanish and Turkish versions.8, 13 To become an official translation, the scale has to undergo a confirmatory factor analysis against the Spanish‐version factor structure. Similar to MDS‐UPDRS, a CFI over 0.90 (0.98) was obtained for Turkish UDysRS, establishing the scale as an official translation.
All published translations reported CFI's greater than 0.90, which indicates a strong consistency with the reference standard languages. On the other hand, all showed some variability in EFAs in the secondary analyses, these were accepted to be within a reasonable range by the analysis team.
As Lord Nelson said, “One's knowledge of science begins when he can measure what he is speaking about and express it in numbers.” 17 The most trustworthy way of quantizing abundant aspects of neurological diseases are rating scales that are proven to be reliable and valid. With an invaluable effort of an MDS‐assigned team, it is now possible to rate PD in several languages, including Turkish, more reliably than ever.
Author Roles
1. Research project: A. Conception, B. Organization, C. Execution; 2. Statistical Analysis: A. Design, B. Execution, C. Review and Critique; 3. Manuscript Preparation: A.Writing of the first draft, B. Review and Critique.
M.C.A.: 1B, 1C, 2C, 3A, 3B
E.B.: 1C, 3A, 3B
V.Y.: 1C, 3B
S.R.: 1C, 3B
S.Ö.: 1C, 3B
A.B.T.: 1C, 3B
E.S.: 1C, 3B
F.N.D.C.: 1C, 3B
B.Ö.B.: 1C, 3B
Z.T.: 1C, 3B
A.A.: 1C, 3B
H.B.: 1C, 3B
C.S.E.Ö.: 1C, 3B
O.D.: 1C, 3B
S.K.: 1C, 3B
N.A.: 1C, 3B
H.H.: 1C, 3B
B.E.: 1C, 3B
M.E.: 1A, 3B
G.T.S.: 1A, 1B, 2A, 2B, 2C, 3A, 3B
C.G.G.: 1A, 1B, 2A, 2B, 2C, 3A, 3B
Disclosures
Ethical Compliance Statement: We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this work is consistent with those guidelines.
Financial sources/Conflicts of interest: All authors received financial support provided by the International Parkinson and Movement Disorder Society for this study.
Full financial disclosure for the previous 12 months: M.C.A.: Honoraria from Abbott, Abdi Ibrahim, Allergan, Bohringer Ingelheim, Gen Ilaç, Generica, Glaxo Smith Kline, Medtronic, Lundbeck, Novartis, Ilko and SantaFarma; Project promotion bonus from The Scientific and Technological Research Council of Turkey (TÜBITAK). V.Y., S.R., B.O.B., Z.T., H.B., C.S.E.O., S.K., and N.A. have no disclosures to report.
V.Y., S.R., B.O.B., Z.T., H.B., C.S.E.O., S.K., and N.A. have no disclosures to report. E.B.: Bursary from TÜBITAK for project #115S812.
S.Ö.: Honoraria from Abbott, Abdi Ibrahim, Allergan, TEVA, Ali Raif.
A.B.T.: Honoraria from Abbott, Abdi Ibrahim, Allergan, Medtronic, Ilko.
E.S.: Honoraria from Pfizer ve Nutricia.
F.N.D.C.: Honoraria from TEVA.
A.A.:Honoraria from TEVA, Abbvie.
O.D.: Honoraria from Abbvie, Abdi İbrahim, Allergan, Aris, Gen İlaç, İlko ilaç, Medtronic, Nobel, Novartis, TEVA.
H.H.: Honoraria from Abbvie, Abdi Ibrahim, Lundbeck, Novartis, Pfizer. Project promotion bonus from TÜBITAK.
B.E.: Honoraria from Abbvie, Abdi İbrahim, Allergan, Gen İlaç, Medtronic.
M.E.: Honoraria from Abdi İbrahim, ARIS, Ilko, Novartis and Teva.
G.T.S.: Consulting and Advisory Board Membership with honoraria: Acadia, Pharmaceuticals, Adamas Pharmaceuticals, Inc., Ceregene, Inc., CHDI Management, Inc., Ingenix Pharmaceutical Services (i3 Research), Neurocrine Biosciences, Inc., Pfizer, Inc., Ultragenyx, Inc. Grants and Research: National Institutes of Health, Michael J. Fox Foundation for Parkinson's Research, Dystonia Coalition, CHDI, International Parkinson and Movement Disorder Society, CBD Solutions. Honoraria: International Parkinson and Movement Disorder Society, American Academy of Neurology, Michael J. Fox Foundation for Parkinson's Research, Food and Drug Administration, National Institutes of Health. Salary: Rush University Medical Center.
C.G.G.: Consulting or Advisory Board Membership with honoraria: Acadia, Addex, Avanir, Boston Scientific, Neurocrine, Oxford Biomedica, WebMD. Grants/Research: Funding to Rush University Medical Center from NIH, Michael J. Fox Foundation for research C.G.G conducted.. C.G.G. directs the Rush Parkinson's Disease Research Center that receives support from the Parkinson's Disease Foundation and some of these funds support C.G.G.'s salary as well as his research efforts. He directed the translation program for the MDS‐UPDRS and UDysRS and received funds directed to Rush University Medical Center from the International Parkinson and Movement Disorder Society (IPMDS) for this effort. Honoraria: Oregon Health and Science University. Royalties: Elsevier Publishers, Oxford University Press, Wolters Kluwer, Salary: Rush University Medical Center.
Acknowledgements
We acknowledge Yalın Ünsal for technical support.
Relevant disclosures and conflicts of interest are listed at the end of this article.
References
- 1. Alvin DF. Preface. Margins of error: A Study of Reliability in Survey Measurement. Hoboken, N.J.: Wiley‐Interscience, 2007, p xi. [Google Scholar]
- 2. Gallagher DA, Goetz CG, Stebbins G, Lees AJ, Schrag A. Validation of the MDS‐UPDRS Part I for nonmotor symptoms in Parkinson's disease. Mov Disord 2012;27(1):79–83. [DOI] [PubMed] [Google Scholar]
- 3. Fahn S, Elton R, UPDRS Development Committee . Unified Parkinson's Disease Rating Scale. In: Fahn S, Marsden CD, Calne DB, Goldstein M, editors.Recent developments in Parkinson's disease. Vol. 2 Florham Park, NJ: Macmillan Health Care Information; 1987. pp. 153–163. [Google Scholar]
- 4. Movement Disorder Society Task Force on Rating Scales for Parkinson's Disease . The Unified Parkinson's Disease Rating Scale (UPDRS): status and recommendations. Mov Disord 2003;18(7):738–750. [DOI] [PubMed] [Google Scholar]
- 5. Martinez‐Martin P, Rodriguez‐Blazquez C, Alvarez‐Sanchez M, et al. Expanded and independent validation of the Movement Disorder Society‐Unified Parkinson's Disease Rating Scale (MDS‐UPDRS). J Neurol 2013;260(1):228–236. [DOI] [PubMed] [Google Scholar]
- 6. Goetz CG, Tilley BC, Shaftman SR, et al. Movement Disorder Society UPDRS Revision Task Force. Movement Disorder Society‐sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS‐UPDRS): scale presentation and clinimetric testing results. Mov Disord 2008;23(15):2129–2170. [DOI] [PubMed] [Google Scholar]
- 7. Goetz CG, Nutt JG, Stebbins GT. The Unified Dyskinesia Rating Scale: presentation and clinimetric profile. Mov Disord 2008;23(16):2398–2403. [DOI] [PubMed] [Google Scholar]
- 8. Goetz CG, Stebbins GT, Wang L, LaPelle NR, Luo S, Tilley BC. IPMDS‐Sponsored Scale Translation Program: Process, Format, and Clinimetric Testing Plan for the MDS‐UPDRS and UDysRS. Mov Disord Clin Pract 2014;1(2):97–101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Fowler FJ. Improving Survey Questions. Thousand Oaks, CA: Sage; 1995. [Google Scholar]
- 10. MDS‐UPDRS . (n.d.). Retrieved October 25, 2015, from https://mds.movementdisorders.org/updrs/
- 11. The Movement Disorder Society ‐ MDS. (n.d.). Retrieved October 25, 2015, from http://udysrs.movementdisorders.org/
- 12. Osborne JW, Costello AB. Sample size and subject to item ratio in principal components analysis. Prac Assess Res Eval 2004;9:1–12. [Google Scholar]
- 13. Cubo E, Goetz CG, Stebbins GT, et al. Independent Spanish validation of the unified Dyskinesia rating scale. Mov Disord Clin Pract 2014;3:213–218. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Gorsuch R. Factor Analysis, 2nd ed Hillsdale, NJ: Lawrence Erlbaum Associations Inc; 1983. [Google Scholar]
- 15. (n.d.). Retrieved June 25, 2017, from http://www.tuik.gov.tr/UstMenu.do?metod=temelist
- 16. Kashihara K, Kondo T, Mizuno Y, et al. MDS‐UPDRS Japanese Validation Study Group. Official Japanese Version of the Movement Disorder Society‐Unified Parkinson's Disease Rating Scale: validation against the original English version. Mov Disord Clin Pract 2014;1(3):200–212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Herndon RM. Introduction to Clinical Neurological Scales. In: Handbook of Neurologic Rating Scales, Editor Herndon RM, New York:Demos Vermande: 1997, p 1–6. [Google Scholar]