Clinimetric Analysis of the Motor Section of the Progressive Supranuclear Palsy Rating Scale: Reliability and Factor Analysis

Abstract

Introduction

PSP is a rare degenerative disorder associated with significant morbidity. Recently, investigations of the etiology and treatment of PSP have been initiated. The aim of the present study was to validate the motor domain of the Progressive Supranuclear Palsy Rating Scale (PSPRS) as part of a larger epidemiological study.

Methods

Fifty videos of patients with PSP were rated by four trained movement disorder neurologists using the PSPRS. Reliability and construct validity of the scale were evaluated using standard methods.

Results

Inter‐rater reliability for the total scale score was good. Internal consistency of the scale improved with the removal of three items. Four factors accounted for the majority of the variance of the scale as determined by principal components analysis.

Discussion

This study shows that the motor domain of the PSPRS is a reliable scale, with a factor structure that suggests construct validity, for the assessment of motor signs of the disease. Removal or modification of items may improve the clinimetric features of the motor domain of the scale.

PSP is a rare neurodegenerative disorder characterized by postural instability with falls, axial rigidity, oculomotor deficits, spastic/ataxic dysarthria, and frontal‐type dysfunction. The PSP Rating Scale (PSPRS), developed by Golbe and Ohman‐Strickland, has 28 items with total scores ranging from 0 to 100.1 The scale is divided into interview and examination sections and assesses five domains: history, mentation, bulbar, ocular motor, and limb motor functions. The history items (n = 7) include questions regarding activities of daily living (ADLs) and subjective measures of symptoms. The mentation items (n = 4) are scored by the examiner based on the interference of the sign with ADLs. Bulbar items (n = 2) address dysarthria and dysphagia, and ocular motor items (n = 4) allow the examiner to assess saccades and eyelid function. The largest domain, limb motor (n = 11), includes items for evaluation of bradykinesia, apraxia, tremor, rigidity, and gait. The aim of this study was to validate the motor domain of the PSPRS, as part of a larger study investigating environmental and genetic risk factors for PSP (R01 AG024040).

Materials and Methods

A structured videotape protocol was developed to capture the PSPRS motor domain, including bulbar, ocular motor, limb motor, and gait/midline items (16 items total). Only the motor items of the PSPRS were included owing to the fact that the other sections of the scale are more difficult to capture with video. The limb rigidity item was not videotaped. The following items did not have instructions on the PSPRS, so the score of the worse of the two sides was used, with the worse score counted as one item: voluntary left and right command movement (ocular motor); apraxia of hand movement; and tremor in any part. Neck rigidity/dystonia was scored, with the caveat that rigidity may not be assessable by video. Video recordings were divided into individual PSPRS items for scoring review. Four movement disorder neurologists, who were trained in person by the principal investigator (PI; I.L.) of this study to use the scale, rated videos of 50 subjects consisting of 44 PSP patients, 5 Parkinson's disease (PD) patients, and 1 patient with a corticobasal syndrome. PSP patients had a Richardson's syndrome phenotype.2 Patients without PSP were included to provide a spread of scores for kappa determination for items such as tremor, which may be less prominent in PSP, and oculomotor disturbances, which do not occur in PD. An additional two movement disorder neurologists received the instructions for use of the scale and rated the same patients, but were not trained by the PI in person before the beginning of this study, and their scores were not included in the analysis. Reliability of the scale was assessed using Cronbach's alpha test of internal consistency; inter‐rater reliability for total scores was assessed with interclass correlation coefficient (ICC),3 and generalized weighted kappa (κ), with quadratic weights,4 were used for inter‐rater reliability for each individual item. The ICC method was a two‐way analysis of variance model, where both patients and raters are considered random. This allowed generalization of the findings across all potential raters and all potential patients. Construct validity of the scale, as indicated by the factor structure, was examined using principal components analysis (PCA) with varimax rotation. The ratings of the PI (I.L.) of the study, a PSP expert, were used as the gold standard, and her intrarater reliability was tested. Her ratings were chosen as the gold standard because there is not a standardized training program to teach administration of the scale, and she was very experienced in using it in the past. Her standard was used in training the other raters, and she did not alter the published instructions of the scale. Her ratings were used to assess internal consistency and factor structure. The study was approved by the institutional review board at each institution, and each subject signed an informed consent form before participating.

Results

The subjects who were rated represented the gamut of PSP severity, with a mean PSPRS score of 18.8 (± 8.9) and individual score ranging from 1.0 to 36.0. Sixteen items were used for the inter‐rater reliability analysis. The dysphagia item for liquids that requires evaluation of the subject drinking a glass of water was captured on 32 patients. Intrarater reliability for the gold‐standard rater on 20% of the samples showed excellent agreement for the total score (ICC = 0.95) and excellent individual item agreement (κ > 0.75) for all items except dysarthria, finger tapping, and sitting, which were fair to good agreement (κ = 0.4–0.75) and eyelid dysfunction with poor agreement (κ = 0.091). Inter‐rater reliability for individual items ranged from poor (minimum κ = 0.02 for limb dystonia) to very good (maximum κ = 0.77 for postural stability). Inter‐rater reliability for total score was acceptable with intraclass correlation coefficient of 0.64 (95% confidence interval: 0.49, 0.767).3 Internal consistency using 15 items, with the removal of the tremor item, which had no variance, was good (Cronbach's alpha = 0.82; item to total correlations: 0.02–0.83). Elimination of an additional two items with low item to total correlations (limb dystonia and dysphagia) increased Cronbach's alpha to 0.87. PCA with varimax factor rotation resulted in a parsimonious solution of four factors, accounting for approximately 76% of the scale variance, after removing the poorly performing three items: Factor 1: dysarthia, toe tapping, arising from chair, gait, postural stability, and sitting; Factor 2: upward, downward, and left/right saccades, and eyelid dysfunction; Factor 3: hand apraxia and neck rigidity/dystonia; and Factor 4: finger tapping (Table 1). The factor solutions were domain independent, with no items from different domains with factor loading greater than 0.5 on multiple factors.

Table 1.

Factor analysis results using principle component extraction and varimax rotation

Factor Loadings
	Factor
	1	2	3	4
Dysarthia	0.561
Voluntary up gaze		0.805
Voluntary down gaze		0.843
Voluntary Left/right gaze		0.759
Eyelid dyspraxia		0.636
Finger tap				0.860
Toe tap	0.585
Apraxia hand			0.810
Neck rigid dystonia			0.675	0.441
Arising chair	0.872
Gait	0.830
Postural stability	0.838
Sitting	0.903

All factor loading greater than 0.40 are displayed.

Discussion

These results suggest that the internal structure of the PSPRS has good internal consistency. When measures of limb dystonia, tremor, and dysphagia are removed from the analysis, improvements in internal consistency occur and a parsimonious factor structure emerges. Given the phenotype of PSP and the lower likelihood of the presence of limb dystonia and tremor, this result is not unexpected. This may suggest that limb dystonia and tremor could be removed from a modified version of the scale. However, the data concerning the limb rigidity item and the dysphagia item are less clear. The dysphagia item was only rated on 32 patients, and limb rigidity was not scored owing to the study design of videotape rating. These items may need to be reevaluated in subsequent scale modifications.

Although our sample size was somewhat limited for a factor analysis, the preliminary results demonstrate a structure with four factors. The first factor was a midline factor, second was eye movements, the third was apraxia and dystonia, and the fourth appendicular speed. This breakdown into factors of midline function and appendicular speed is reminiscent of the factor structure of the International Parkinson and Movement Disorder Society UPDRS.5 Inter‐rater reliability of the total score for trained raters is good, whereas inter‐rater reliability for individual items ranged from poor to excellent. Standardized training on the collection and scoring of items might improve inter‐rater reliability for individual items. In fact, scores from the two untrained raters were missing many variables because of a lack of understanding on how to score particular items and could not be included in the analysis. This may be a concern if the scale is to be used by many investigators at different sites in the context of a clinical trial.

Preliminary validation of the PSPRS in the initial description of the scale was completed by using data from a total of 5 patients scored by three raters.1 In addition, the initial validation used total PSPRS score rather than item scores. Our study, with five raters and 50 subjects, adds to our knowledge of the structure and construct validity of the scale for more‐widespread use. Although the scale was not developed to be used with videotaped patients, our data may be useful in further refining the scale.

Recently, a new scale, the Neuroprotection and Natural History in Parkinson Plus Syndromes (NNIPPS) Parkinson Plus Scale, was published.6 This scale includes many more items (n = 85) and was developed for use in studies where the diagnosis of PSP or MSA may not yet be certain. PCA of this scale extracted 15 factors, with the top five including: ADLs and mobility; axial bradykinesia; limb bradykinesia; rigidity; and oculomotor. There is some overlap with the factor loadings in our study, with differences accounted for by the inclusion of MSA symptoms and signs in the development and validation of the NNIPPS Parkinson Plus Scale. It is unclear whether the NNIPPS is superior to the PSPRS because there has not been a comparison between the two scales.

Overall, this study shows that the PSPRS is a useful scale for the assessment of motor signs of the disease, but some modifications are needed to improve the reliability and validity of the scale. The next steps to accomplish this are to expand the scale by adding other motor items and testing the history and mentation domains. Additionally, noncontributory items should be deleted and clinically important items that were removed should be modified. Then, pilot testing of a modified scale in a larger sample should be conducted for item analysis, construct validity, as indicated by factor structure. Finally, a confirmatory factor analyses should be completed. Measurement of content or criterion validity would be ideal, but lack of a good quantitative biomarker is an issue.

Author Roles

(1) Research Project: A. Conception, B. Organization, C. Execution; (2) Statistical Analysis: A. Design, B. Execution, C. Review and Critique; (3) Manuscript Preparation: A. Writing of the First Draft, B. Review and Critique.

D.A.H.: 1A, 1B, 1C, 2A, 2B, 2C, 3A, 3B

G.T.S.: 2A, 2B, 2C, 3B

I.L.: 1A, 1B, 1C, 2A, 2C, 3B

Y.B.: 1C, 2C, 3B

D.E.R.: 1C, 2C, 3B

J.L.: 1C, 2C, 3B

D.G.S.: 1C, 2C, 3B

Disclosures

Funding Sources and Conflicts of Interest: This study was funded by the National Institute for Aging (R01AG024040; to I.L.). The authors report no conflicts of interest.

Financial disclosures for previous 12 months: Dr. Hall receives research support from the National Institutes of Health (NIH), Shapiro Foundation, Parkinson Disease Foundation, Pfizer, and Neurocrine. Dr. Stebbins serves on the editorial board for the Journal of Clinical and Experimental Neuropsychology and as a consultant to IMPAX Laboratories, Ceregene, Biovail Technologies, Santhera, and Ingenix. He has received research support from the NIH, Michael J. Fox Foundation (MJFF) for Parkinson's Research, and the American Cancer Society. Dr. Litvan has research support from NIH, MJFF, Parkinson Study Group, CurePSP, CBD Solutions, and TEVA. She served as a consultant to ACADIA, Pfizer, MJFF, TEVA, and Biotie. Dr. Bordelon serves on the speakers bureaus for Lundbeck and Teva and has received honorarium from Medical Education Speakers Network. She has received research support from the MJFF. Dr. Riley serves on the medical advisory board for CurePSP and on the speakers bureaus of Allergan, Ipsen, Lundbeck, Merz, and Teva. Dr. Leverenz has served as a consultant for Bayer, Novartis, and Teva Pharmaceuticals. Dr. Standaert has served as a consultant or received honorarium from Teva, Solvay, Nupathe, MJFF, Partners Healthcare, University of Michigan, Balch and Bingham, University of Arizona, Northshore Hospital, the Thomas Hartman Foundation, and the Bachmann‐Strauss Foundation and has received royalties from McGraw Hill, Inc. He receives research support from Abbott, Addex, Allon, MJFF, American Parkinson Disease Association, National Multiple Sclerosis Society, American Sleep Foundation, the RJG Foundation, and the NIH.