Clinimetric Testing of the Comprehensive Cervical Dystonia Rating Scale

Abstract

Objective

To test the clinimetric properties of the Comprehensive Cervical Dystonia Rating Scale.

Background

This is a modular scale with modifications of the Toronto Western Spasmodic Torticollis Rating Scale (composed of three subscales assessing motor severity, disability and pain) now referred to as the revised Toronto Western Spasmodic Torticollis Scale-2.; a newly developed psychiatric screening instrument; and the Cervical Dystonia Impact Profile-58 as a quality of life measure.

Methods

Ten dystonia experts rated subjects with cervical dystonia using the comprehensive scale. Clinimetric techniques assessed each module of the scale for reliability, item correlation and factor structure.

Results

There were 208 cervical dystonia patients (73% women, age 59±10 years, duration 15±12 years). The internal consistency of the motor severity subscale was acceptable (Cronbach’s alpha = 0.57). Item to total correlations showed that elimination of items with low correlations (<0.20) increased alpha to 0.71. Internal consistency estimates for the subscales for disability and pain were 0.88 and 0.95 respectively. The psychiatric screening scale had a Cronbach’s alpha of 0.84 and satisfactory item to total correlations. When the subscales of the Toronto Western Spasmodic Torticollis scale -2 were combined with the psychiatric screening scale, Cronbach's alpha was 0.88, and construct validity assessment demonstrated four rational factors: motor, disability, pain and psychiatric disorders. The Cervical Dystonia Impact Profile-58 had an alpha of 0.98 and its construction was validated through a confirmatory factor analysis.

Conclusions

The modules of the Comprehensive Cervical Dystonia Rating Scale are internally consistent with a logical factor structure.

Introduction

Cervical dystonia (CD) is a complex disorder marked by involuntary movements of neck and shoulders, pain, impaired activities of daily living and reduced quality of life. The abnormal movements often combine head turn, tilt, forward or backward flexion, anterior or sagittal shift and shoulder elevation.^{1, 2} The involuntary movements are associated with significant disability. In addition, pain occurs in 75% of patients and contributes to a greater degree of disability.³ CD has also been associated with psychiatric disorders, including depression, anxiety, panic disorders and social phobia.^4–7 Furthermore, several studies have also demonstrated impaired health-related QOL in CD^8–14.

Although there have been many rating scales developed for motor symptoms of CD¹⁵, only 3 of these, the Tsui scale¹⁶, the Cervical Dystonia Severity Scale¹⁷ and the Toronto Western Spasmodic Torticollis Rating Scale (TWSTRS) have had clinimetric evaluation. None of these scales address the psychiatric symptoms or quality of life. The Tsui rating scale is a 6-item scale that assesses amplitude and duration of involuntary neck movements, shoulder elevation, and head tremor¹⁶. This scale is designed to assess head and shoulder postures and head tremor but does not take into account the other manifestations of CD. The Cervical Dystonia Severity Scale uses a protractor and wall chart to rate angles of head deviation from neutral in each of three planes¹⁷. This scale does not evaluate shoulder elevation, tremor or sagittal shift. The Tsui rating scale and CDSS do not address pain, activities of daily living, psychiatric symptoms, or quality of life.

The standard TWSTRS consists of three domains that assess motor severity, pain, and disability¹⁸. The motor severity subscale consists of 10 items, with variable scaling and weighting. It also includes a disability scale with 6 items, and a pain scale with 3 items. The total score is the sum of each of the subscales. Only the motor domain has undergone evaluation for inter-rater reliability and construct validity, with good to excellent inter-rater reliability¹⁹. Despite the limited clinimetric studies of the TWSTRS, it has been used extensively in clinical studies of CD and is the scale currently recommended by the Movement Disorder Society task force on dystonia rating scales.¹⁵

There are no psychiatric rating scales validated for use in CD. While the DSM-V criteria are the gold standard for diagnosis of psychiatric disease, their application requires specific training and is impractical for routine use by most CD providers. There are several self-administered scales that are easy to administer, require no examiner training, and have been assessed for clinimetric properties in primary depression and anxiety. The Beck Depression Inventory²⁰ is a self-administered scale with 21 components that takes 10 – 15 min to complete. This scale does not emphasize somatic components and therefore avoids the confounding factors of the movement associated with CD.²¹ The Hospital Anxiety and Depression Rating Scale is a self-administered scale that consists of 14-item subscales for both depression and anxiety.^{22, 23} This scale was specifically developed for use in patients with somatic co-morbidity and has no questions related to the physical signs of depression or anxiety. The Beck Anxiety Index is a self-reported scale²¹ designed as a screening tool for anxiety with good positive predictive value for panic disorders.²⁴ Although these psychiatric rating scales are all well validated in psychiatric practice, they have not been systematically applied to CD.

The effect of CD on quality of life is comparable to that seen in multiple sclerosis, Parkinson's disease, stroke¹⁴ and other chronic diseases.²⁵ Standard measures of QOL, including generic health-related QOL,²⁵ EuroQoL, SF-36 and Rosenbergs’ self-esteem scale²⁶ are not consistent in identifying factors predicting reduced quality of life in CD and do not correlate with effective treatment for CD, such as botulinum toxin injections.^{12, 27} The Craniocervical Dystonia Questionnaire (CDQ-24), although designed specifically for blepharospasm and CD, has not been extensively used or tested against other scales.²⁸ The Cervical Dystonia Impact Profile – 58 item (CDIP-58) was developed using a modified Delphi method with Rasch methodology.²⁹ It is a self-administered scale with 8 subscales measuring the impact of head and neck symptoms on a variety of quality of life items.²⁹ The CDIP-58 has been evaluated for reliability and validity in CD³⁰ and shown to be superior to the SF-36, a widely used but generic QOL measure. The CDIP-58 also demonstrates sensitivity to change following botulinum toxin injections^{29, 31} and is the recommended scale for quality of life in CD.¹⁵

In this study, the original TWSTRS was revised to the TWSTRS-2 to address identified deficiencies, including the variable scaling of items, the lack of an item for head tremor and the weighting of the duration factor by two.³² The TWSTRS-PSYCH was developed to screen for psychiatric disorders associated with CD. The CDIP-58 with previously established reliability and validity was included in its original form. We combined the TWSTRS-2, TWSTRS-PSYCH and the CDIP-58 to produce the modular Comprehensive Cervical Dystonia Rating Scale or CCDRS. The specific aim of the study was to assess the reliability and construct validity of the CCDRS.

Methods

The methods for development of the CCDRS have been described in a prior publication.³² Briefly, the existing TWSTRS motor severity was revised to the TWSTRS-2 motor severity using a modified Delphi method with input from dystonia experts. The TWSTRS–PSYCH was developed using a similar methodology with input from psychiatrists, dystonia experts and patients. The draft TWSTRS-2 included assessments for motor severity (12 items), pain (5 items) and disability (6 items). The TWSTRS-PSYCH included 6 items rated on a 5-point scale from 0 (absent) to 4 (severe) for occurrence over the past month (Figure 2). The maximal score of the TWSTRS-PSYCH was 24. The CDIP-58 includes 58 self-administered questions that define 8 subscales and are transformed into a total score, with a maximal score of 100. The TWSTRS-2, TWSTRS-PYSCH, and CDIP-58 then were combined into the CCDRS and used in the data collection phase of the study along with other demographic and disease-related measures.

Subjects with isolated CD, previously known as primary dystonia, were recruited from 10 sites. Demographic information, including age, gender, and duration of CD were collected. For this study, subjects were videotaped using a standardized protocol during the time that the site investigator rated the subject severity using the TWSTRS-2 motor severity subscale.³² Subjects were interviewed to complete the TWSTRS-2 disability and pain subscales, as well as the TWSTRS-PSYCH. The subjects completed the self-reported CDIP-58.

There are no accepted formulae for calculating required sample sizes for scale validation studies, particularly factor analytic methods, at given levels of power³³. Instead, recommended subject-to-item ratios are employed. For the present study, we have a 9.1:1 subject-to-item ratio, which exceeds the recommended 8:1 ratio shown to be adequate for this analysis^{34, 35}.

Rating scores and video were electronically sent to a central database at Washington University, St. Louis, MO³⁶. The video and data were assessed for completeness. Queries regarding missing data were resolved. Accuracy of the data entry was verified through cross referencing electronic data to paper data collection forms in 10% of cases.

Statistical approach

Subject demographics and disease-related variables were examined using frequency counts and measures of central tendency and variability, as appropriate. To assess the reliability and validity of the TWSTRS-2 and TWSTRS - PSYCH components of the CCDRS, we employed both Classical Test Theory (CRT) and Item Response Theory (IRT). Classical Test Theory focuses on the relationships of individual items to the entire scale³⁷ while Item Response Theory focuses on the measurement characteristics of the items in relation to the individual completing the scale³⁷. Using Classical Test Theory we examined Cronbach's alpha, a measure of scale reliability, item-to-total correlations, changes in alpha if selected items were removed and distributional skewness, a measure of potential floor or ceiling effects, for the separate subscales of the TWSTRS-2 (motor severity, disability, pain) and TWSTRS-PSYCH modules of the CCDRS. These analyses were conducted using SPSS (Version 21). Additionally, we examined the construct validity through exploratory factor analyses. Because of the ordered categorical level of measurement of the CCDRS we employed an unweighted least squares approach for the factor estimate and a CF-Varimax orthogonal rotation to improve the interpretability of the factors. MPlus (Version 7) was used for these analyses. For the Item Response Theory approach, we used a graded response model analysis with maximum likelihood parameter estimation³⁸ to examine item discrimination, or the strength of the relationship between the item and the measured domain, and item threshold, or the level of item response to the overall severity of the measured domain. MPlus (version 7) was used for these analyses.

To assess each item's utility in the CCDRS, we identified items with low item-to-total correlations (defined as ≤ 0.3), improvement in Cronbach's alpha if omitted, low factor loading (defined as ≤ 0.4), a skewness outside of the range −1.50 to +1.50 representing possible floor or ceiling effects, non-significant Item Response Theory discrimination scores and thresholds that did not encompass a value of zero. Based on this assessment, each item was considered either as one to keep in the scale or as one to drop or modify. If an item met the criteria for acceptable item-to-total correlation, change in alpha if the item were omitted, appropriate factor loading, skewness and Item Response Theory discrimination and threshold, it was retained. Items not meeting these criteria were deleted.

Because the CDIP-58 module of the CCDRS had already undergone clinimetric examination for reliability and validity, we limited our analysis to assessments of internal consistency (Cronbach's alpha) and confirmatory factor structure (CFA). The CFA was conducted to determine if the 8 factors found in the original publication²⁹ could be confirmed with the data collected for this study. We evaluated the CFA results based on the Comparative Fit Index (CFI)³⁹. To confirm a good fit between the original factor structure and our data, the CFI was required to be 0.90 or greater. Mean and variance adjusted weighted least square (WLSMV) estimator was used to confirm model fit. We also used the root mean square error of approximation (RMSEA) to check the goodness of fit, with values less or equal than 0.10 indicating an acceptable index. MPlus (version 7) was used for these analyses.

Results

A total of 208 CD subjects (73% women, mean age 59 years SD± 9.95), onset of CD 44 years (SD ± 12.11) from 10 sites in the United States were included. The mean severity of CD as measured using the TWSTRS 2 total score was (33.24 (SD ± 13.22)), with subscale scores for motor severity of (16.29 (SD ± 5.54)), disability (9.21 (SD ± 5.72)) and pain (7.88 (SD ± 5.56))

TWSTRS-2 Motor Severity Subscale (Table 1)

Table 1.

Classical test theory and item response analyses results for items on the TWSTRS-2 Motor, TWSTRS-2 Disability, TWSTRS-2 Pain and TWSTRS-2 Psych components of the Comprehensive Cervical Dystonia Rating Scale.

Item	Item-Total Correlation	Alpha-if- Item- Removed	Factor Loading	Skewness	IRT Discrimination (p value)	IRT Threshold (Min; Max)	Action
TWSTRS-2 Motor
Rotation	0.314	No Increase	0.420	0.314	1.00 (<0.0005)	−2.75; 2.50	Keep
Laterocollis	0.466	No Increase	0.519	0.608	1.40 (<0.0005)	−2.21; 6.22	Keep
Anterocollis	0.180	Increased	<0.3	1.780	0.69 (0.06)	0.68; 3.53	Drop/Modify
Retrocollis	0.091	Increased	<0.3	1.536	0.17 (0.44)	0.62; 5.31	Drop/Modify
Lateral Shift	−0.068	Increased	<0.3	1.723	−0.24 (0.19)	0.71; 5.35	Drop/Modify
Sagittal Shift	0.136	Increased	<0.3	1.575	0.34 (0.05)	0.45; 5.37	Drop/Modify
Head Tremor	0.000	Increased	<0.3	0.472	0.04 (0.84)	−0.59; 3.91	Drop
Shoulder Elevation	0.349	No Increase	0.481	0.542	1.11 (<0.0005)	−1.79; 2.88	Keep
Duration	0.527	No Increase	0.626	−0.648	1.58 (<0.0005)	−6.53; 1.06	Keep
Sensory Trick	0.011	Increased	<0.3	1.107	0.09 (0.63)	−1.42; 2.23	Drop/Modify
ROM	0.373	No Increase	0.428	0.877	1.16 (<0.0005)	−1.81; 3.88	Keep
Time in Midline	0.472	No Increase	0.631	−0.563	1.60 (<0.0005)	−1.59; 0.49	Keep
TWSTRS- 2 Disability
Work	0.722	No Increase	0.704	0.483	1.00 (<0.0005)	−1.46; 5.50	Keep
ADL	0.654	No Increase	0.604	0.774	1.66 (<0.0005)	−0.77; 5.45	Keep
Driving	0.587	No Increase	0.542	0.294	1.40 (<0.0005)	−1.90; 5.13	Keep
Reading	0.748	No Increase	0.805	0.183	3.32 (<0.0005)	−3.00; 8.12	Keep
Television	0.734	No Increase	0.759	−0.052	3.35 (<0.0005)	−2.92; 6.80	Keep
Outside of Home	0.728	No Increase	0.690	0.685	2.22 (<0.0005)	−2.00; 5.91	Keep
TWSTRS- 2 Pain
Pain Best	0.778	No Increase	0.752	1.13	1.000 (<0.0005)	−0.11; 2.92	Keep
Pain Worst	0.884	No Increase	0.848	−0.210	4.853 (<0.0005)	−1.20; 0.96	Keep
Pain Usual	0.944	No Increase	0.900	0.312	6.169 (<0.0005)	−0.77; 1.87	Keep
Pain Duration	0.835	No Increase	0.778	−0.169	3.564 (<0.0005)	−1.130; 0.55	Keep
Pain Disability	0.735	No Increase	0.669	0.472	2.320 (<0.0005)	−0.50; 1.74	Keep
TWSTRS- 2 Psych
Depression	0.657	No Increase	0.724	0.557	1.00 (< 0.0005)	−0.44; 2.38	Keep
Loss of Interests	0.673	No Increase	0.740	1.263	2.20 (< 0.0005)	0.22; 2.08	Keep
Discomfort in Public	0.570	No Increase	0.627	0.728	1.57 (< 0.0005)	−0.17; 2.34	Keep
Anxiety	0.672	No Increase	0.741	0.907	2.04 (< 0.0005)	−0.22; 2.24	Keep
Panic	0.559	No Increase	0.626	3.646	2.37 (< 0.0005)	1.18; 2.57	Keep
Afraid Going Out	0.576	No Increase	0.645	2.673	2.16 (< 0.0005)	0.93; 2.85	Keep

Note: Item-To-Total = correct item score to total score correlation; Alpha-If-Item-Removed = increase of decrease in Cronbach's alpha if the item is removed from the analysis; Factor Loading = maximum factor loading value; Skewness = a measure of distributional asymmetry; IRT Discrimination = strength of relationship between item and the measured construct; IRT Threshold = level of item response in relation to severity of measured construct, minimum and maximum thresholds displayed.

Overall Cronbach's alpha for the TWSTRS-2 motor severity subscale was 0.57. Items assessing Rotation, Laterocollis, Shoulder Elevation, Duration, Range of Motion and Time in Midline met criteria for acceptable item-to-total correlation, change in alpha if item omitted, factor loading, skewness, IRT discrimination and IRT threshold. Items assessing anterocollis, retrocollis, lateral shift, sagittal shift, head tremor and effect of a sensory trick failed to meet the criteria for utility in the CCDRS and were deleted from the CCDRS.

TWSTRS-2 Disability Subscale (Table 1)

Overall Cronbach's alpha for the TWSTRS-2 disability subscale was 0.88. Items assessing Work, Activities of Daily Living, Driving, Reading, Television and Outside of Home Disability met criteria for acceptable item-to-total correlation, change in alpha if item omitted, factor loading, skewness, IRT discrimination and IRT threshold. All items met the criteria for utility in the CCDRS and were retained in the CCDRS.

TWSTRS-2 Pain Subscale (Table 1)

Overall Cronbach's alpha for the TWSTRS-2 pain subscale was 0.95. Items assessing Pain at its Best, Pain at its Worst, Usual Pain, Pain Duration and Pain Disability met criteria for acceptable item-to-total correlation, change in alpha if item omitted, factor loading, skewness, IRT discrimination and IRT threshold. All items met the criteria for utility in the CCDRS and were retained in the CCDRS. The revised TWSTRS-2 scale is included in figure 1.

Figure 1.

Toronto Western Spasmodic Torticollis Rating Scale-2 (TWSTRS-2)

TWSTRS- PSYCH (Table 1)

Overall Cronbach's alpha for the TWSTRS-PSYCH was 0.84. Items assessing Depression, Loss of Interest, Discomfort and Anxiety met criteria for acceptable item-to-total correlation, change in alpha if item omitted, factor loading, skewness, IRT discrimination and IRT threshold and were retained in the CCDRS. Items assessing Panic and Afraid of Going Outside met all criteria except for skewness. The skewed distribution appears to be due to the high percentages of zero scores for Panic (88%) and Afraid of Going Outside (82%). The TWSTRS-PSYCH scale is included in figure 2.

Figure 2. TWSTRS-PSYCH.

Circle one number for each question.

Combined TWSTRS-2 and TWSTRS-PSYCH

Overall Cronbach's alpha for the combined TWSTRS2 (after removing items assessing anterocollis, retrocollis, lateral shift, sagittal shift, head tremor and effect of a sensory trick) and TWSTRS-PSYCH was 0.88. All items met criteria for acceptable item-to-total correlation, change in alpha if item omitted, skewness, IRT discrimination and IRT threshold. Factor analysis revealed a satisfactory four-factor solution with items assessing motor severity, disability, pain and psychiatric manifestation loading on separate factors (all factor loadings > 0.40) (Table 2).

Table 2.

Factor solution for combined TWSTRS-2 and TWSTRS-PSYCH after deleting TWSTRS-2 Severity items not meeting criteria for inclusion in the Comprehensive Cervical Dystonia Rating Scale.

Item	Factor
	1	2	3	4
Rotation				.41
Laterocollis				.51
Shoulder elevation				.53
Duration				.62
Range of movement				.42
Time midline				.63
Work disability		.59
ADL disability		.52
Driving disability		.52
Reading disability		.80
Television disability		.75
Outside disability		.63
Pain at best	.76
Pain at worst	.84
Pain at usual	.92
Pain duration	.82
Pain disability	.67
Depression			.68
Loss of interest			.70
Discomfort in public			.58
Anxiety			.71
Panic attack			.61
Fear of outside			.62

Note: All factor loadings < 0.40 are not shown in the Table.

CDIP-58

Overall Cronbach's alpha for the CDIP-58 was 0.98. The CFA of the 8-factor solution of the original CDIP-58 resulted in a CFI of 0.97 with a RMSEA of 0.07 and a model fit chi-square of 48.96 (p < 0.0005) using the data from the current study. Thus the pre-specified 8-factor structure was confirmed.

Discussion

This study demonstrates that the CCDRS assesses distinct components of CD and can be applied as a complete scale or used in a modular format. The current study provides a realistic picture of the clinimetric properties of this scale and each of its modules, and allows the deletion of items that do not demonstrate clinical utility.

The revised motor severity subscale of the TWSTRS-2 demonstrated that certain items (anterocollis, retrocollis, lateral shift, sagittal shift, head tremor and effect of sensory trick) had multiple indicators of poor utility on both CCT and IRT analyses. The reasons for the lack of utility of these items are varied. Anterocollis, retrocollis, lateral shift and sagittal shift ratings had highly skewed distributions, suggesting possible floor-effects. Head tremor and effect of sensory trick had more normal-shaped distributions. However, these items had low item-to-total correlations, and increased the alpha if omitted. Further, the low factor loading of these items indicates that they may not directly contribute to overall CD severity in contrast to the other items, although these may be features of the disorder. Hence, these items were deleted from the rating of motor severity, resulting in a simplified scale that can be used efficiently in a clinical study (Figure 1)

The TWSTRS-2 disability subscale, which was unchanged from the standard TWSTRS, was not revised and had good reliability and content validity. The TWSTRS-2 pain subscale was revised, removing the mathematical manipulations (the multiplication of the usual level of pain by 2 and eliminating the division of the pain scores by 4), and was found to be reliable and valid. The first psychiatric screening tool for CD, TWSTRS-PSYCH (Figure 2), demonstrated good clinimetric properties. The CDIP-58, which has previously been assessed for reliability and validity using a different scale development technique, was found to have acceptable internal consistency and a confirmed factor structure of 8 factors. Inclusion of the CDIP-58 provides a patient reported measure of the impact of CD on quality of life that is distinct from information provided by the other scales in the CCDRS.

Although many rating scales have been developed to evaluate CD, none has been comprehensive^{10, 15, 40, 41}. The CCDRS includes measures for motor severity, disability, pain, psychiatric disorders and quality of life measures. Each of these domains may be affected in CD and contribute to overall severity of the condition. The reduction in total items in the TWSTRS-2 motor severity subscale based on these results will allow for easier use. While the deleted items may be useful as descriptors for CD, these items do not contribute to the overall assessment of CD severity

The results of the factor analysis for the modified TWSTRS-2 and TWSTRS-PSYCH suggest that the scores of the four subcomponents (motor severity, disability, pain and psychiatric concerns) can be used either as independent measures or summarized into a single measure of CD impairment. The previously defined factor structure of the CDIP-58 was confirmed in the present analysis.

The CCDRS provides a tool that allows an assessment of all aspects of CD and can be used in modular format. This study provides the framework for development of rating scales that can be used to assess the varied clinical aspects of focal dystonias involving other body regions. As new therapeutic modalities become available for the treatment of the focal dystonias⁴², it is critical that validated outcome measures capture not only the motor features, but also those related to psychological disorders and impact on quality of life.