Reliability of the CMT neuropathy score (second version) in Charcot-Marie-Tooth disease

Abstract

The Charcot-Marie-Tooth neuropathy score (CMTNS) is a reliable and valid composite score comprising symptoms, signs, and neurophysiological tests, which has been used in natural history studies of CMT1A and CMT1X and as an outcome measure in treatment trials of CMT1A. Following an international workshop on outcome measures in Charcot-Marie-Tooth disease (CMT), the CMTNS was modified to attempt to reduce floor and ceiling effects and to standardize patient assessment, aiming to improve its sensitivity for detecting change over time and the effect of an intervention. After agreeing on the modifications made to the CMTNS (CMTNS2), three examiners evaluated 16 patients to determine inter-rater reliability; one examiner evaluated 18 patients twice within 8 weeks to determine intra-rater reliability. Three examiners evaluated 63 patients using the CMTNS and the CMTNS2 to determine how the modifications altered scoring. For inter- and intra-rater reliability, intra-class correlation coefficients (ICCs) were ≥0.96 for the CMT symptom score and the CMT examination score. There were small but significant differences in some of the individual components of the CMTNS compared with the CMTNS2, mainly in the components that had been modified the most. A longitudinal study is in progress to determine whether the CMTNS2 is more sensitive than the CMTNS for detecting change over time.

Introduction

Charcot-Marie-Tooth disease (CMT) is the most common inherited neuromuscular disorder, affecting 1 in 2,500 individuals (Skre, 1974). More than 40 causative genes have been described to date, which can be inherited in an autosomal dominant (AD), autosomal recessive (AR), or X-linked fashion (Reilly et al., 2011). Patients typically present with distal pre- dominant wasting, weakness, and sensory loss, often with foot deformities. CMT is classified based on a combination of genetic cause and upper limb motor nerve conduction velocity (MNCV) (Reilly et al., 2011). MNCV < 38 m/s is considered demyelinating (CMT1), whereas MNCV > 38 m/s is considered axonal (CMT2) (Harding and Thomas, 1980), although some forms are characterized by “intermediate” conduction velocities (Nicholson and Myers, 2006). Regardless of the form of CMT, however, clinical disability correlates with axonal loss (Sahenk and Chen, 1998; Krajewski et al., 2000; Verhamme et al., 2004). Data on the natural history of CMT are limited to the most frequent subtypes, CMT1A (Padua et al., 2008; Shy et al., 2008; Verhamme et al., 2009) and CMT1X (Shy et al., 2007). Because CMT progresses gradually, to measure worsening, one requires scales that can document small changes over time.

The Charcot-Marie-Tooth neuropathy score (CMTNS, Table S1) is a reliable and valid composite of nine assessments: symptoms (three items), signs (four items), and neurophysiology (two items). It is designed to measure length-dependent motor and sensory impairment in genetic neuropathies (Shy et al., 2005). Each assessment is scored on a 0 – 4 point scale, reflecting severity of impairment. Patients are classified as mild (CMTNS ≤ 10), moderate (CMTNS 11 – 20), or severe (CMTNS > 20). The CMTNS correlates well with other measures of disability including ambulation index, self-assessment, hand function, 9-hole peg test, and neuropathy impairment score (Shy et al., 2005). The CMT symptom score (CMTSS) and CMT examination score (CMTES) are subscores of the CMTNS, calculated by the sum of the symptoms (CMTSS) or the sum of the symptoms plus the signs (CMTES). Although the CMTNS is a good measure of CMT severity, after using this scale for several years in natural history studies and therapeutic trials (Shy et al., 2007; 2008; Pareyson et al., 2011), we have found that some of the items have a ceiling or floor effect. For example, when the ulnar sensory action potential (SAP) is absent, a patient will obtain the highest score possible (4). However, the ulnar SAP is often absent in patients with CMT, even mildly affected individuals. Thus, even if the patient progresses, this item cannot get any worse and will not, therefore, detect change over time. In addition, the way in which examiners asked questions of patients has not always been standardized, leading to possible differences in interpretation of symptoms.

Following the 168th ENMC International Work- shop on outcome measures and clinical trials in CMT (Reilly et al., 2006), it was decided to make some modifications to the CMTNS to reduce floor and ceiling effects and to standardize patient assessment, aiming to improve its sensitivity for detecting change over time. In this article, we describe the updated version of the CMTNS (CMTNS2) and report the reliability of this scale.

Materials and Methods

Modifications to the CMTNS

Three of the authors (D.P., M.E.S., and M.M.R.) agreed on the changes to be made, after seeking input from additional peripheral neuropathy specialists who have performed the CMTNS. The CMTNS2 is shown in Table 1; it differs from the original CMTNS in several ways. Spoken instructions are now provided so that patients are asked exactly the same questions in the same way each time (Table 2). Sensory symptoms were changed to discern those with symptoms extending to the distal calf vs. the proximal calf; previously a change in sensory symptoms from just above the ankle to the knee would not have been detected by this item. In addition, a picture is shown to the patients to ensure standardized scoring. Motor symptoms (legs) were changed to add the use of shoe inserts (1 point) and the weight of prior ankle surgery was decreased to reduce the floor effect of this item, as many patients with CMT (especially older patients) have had ankle surgery because of variable orthopedic practice rather than because of severe ankle weakness. Motor symptoms (arms) were changed from any difficulty with buttons/zippers (1 point) to mild difficulty with buttons (1 point) and severe difficulty or unable to do buttons (2 points). We also added “unable to cut most food”(3 points) instead of unable to write or use a keyboard; many people do not write other than signing their name and many CMT patients can use a keyboard with aids, which the original CMTNS would not have recognized. Pinprick sensibility was revised to mirror the change in sensory symptoms and we stipulated that a Neurotip must be used to test this item. For vibration testing, we stipulated the use of a Rydel-Seiffer tuning fork, a reliable and sensitive instrument for determining perception of vibration (Bergin et al., 1995; Martina et al., 1998). Previously, methods for examining vibration varied among centers and examiners, which caused variation among raters. The Rydel-Seiffer tuning fork allows grading of a patient’s vibratory perception from 8 (full) to 0 (absent). We used a score of 5 or greater to indicate normality based on previous studies (Martina et al., 1998; Pestronk et al., 2004; Whitton et al., 2005). Vibration was tested at the dorsum of the metatarsophalangeal joint of the great toe, medial malleolus, and tibial tuberosity. Lower limb strength was changed to include weakness of ankle plantarflexion in scores 1 and 2, as some subtypes of CMT cause weakness of ankle plantarflexion more than dorsiflexion. In addition, we stipulated that to obtain a higher score a patient must fulfill the requirements of all scores below, for example, to score 4 (proximal weakness) a patient must also have weakness of ankle dorsiflexion and plantarflexion of Medical Research Council (MRC) grade 3 or less. Arm strength was changed so that the strongest of first dorsal interosseus (FDI) and abductor pollicis brevis (APB) are used to score this item, meaning that deteriorating strength in the hands should be evaluated better with the CMTNS2. Ulnar or median compound muscle action potential (CMAP) was not changed; however, we decided to use radial SAP in CMTNS2 rather than ulnar SAP as, in our experience, the ulnar SAP is often absent early in the disease course compared with the radial SAP which may be more often preserved.

Table 1.

CMT neuropathy score – Version 2.

Parameter	0	1	2	3	4
Sensory symptoms*	None	Symptoms below or at ankle bones	Symptoms up to the distal half of the calf	Symptoms up to the proximal half of the calf, including knee	Symptoms above knee (above the top of the patella)
Motor symptoms (legs)†	None	Trips, catches toes, slaps feet Shoe inserts	Ankle support or stabilization (AFOs) Foot surgery‡	Walking aids (cane, walker)	Wheelchair
Motor symptoms (arms)	None	Mild difficulty with buttons	Severe difficulty or unable to do buttons	Unable to cut most foods	Proximal weakness (affect movements involving the elbow and above)
Pinprick sensibility*§	Normal	Decreased below or at ankle bones	Decreased up to the distal half of the calf	Decreased up to the proximal half of the calf, Including knee	Decreased above knee (above the top of the patella)
Vibration|	Normal	Reduced at great toe	Reduced at ankle	Reduced at knee (tibial tuberosity)	Absent at knee and ankle
Strength (legs)¶	Normal	4+, 4, or 4− on foot dorsiflexion or plantar flexion	≤3 on foot dorsiflexion or ≤3 on foot plantar flexion	≤3 on foot dorsiflexion and ≤3 on plantar flexion	Proximal weakness
Strength (arms)¶	Normal	4+, 4, or 4− on intrinsic hand muscles**	≤3 on intrinsic Hand muscles**	≤5 on wrist extensors	Weak above elbow
Ulnar CMAP	≥6 mV	4–5.9 mV	2–3.9 mV	0.1–1.9 mV	Absent
(median)	(≥4 mV)	(2.8–3.9)	(1.2–2.7)	(0.1–1.1)	(absent)
Radial SAP amplitude, antidromic testing	≥15 µV	10–14.9 µV	5–9.9 µV	1–4.9 µV	<1 µV

AFO, ankle-foot orthoses; CMAP, compound muscle action potential; SAP, sensory action potential.

^*Use the picture below to discriminate the level of the symptoms.

^†Uses aid most of the time. The patient was prescribed to wear/use or should be wearing/using the aid in the examiner’s opinion (see written instructions, Table S2).

^‡See written instructions for details of eligible foot surgery.

^§Abnormal if patient says it is definitely decreased compared to a normal reference point.

^|Use Rydel-Seiffer tuning fork. Definition of normal: ≥5.

^¶Limb strength scores refer to MRC grade.

^**Intrinsic hand muscles strength assessment: test only abductor pollicis brevis (APB) and first dorsal interosseus (FDI), then choose the stronger to give the score.

Table 2.

CMTNS spoken instructions

Sensory loss

Do you have loss of feeling anywhere in your feet or legs? If so, does the loss of feeling extend above your toes?

Do they extend above the ankle?

Please identify the point on this drawing of the leg where the sensation becomes normal or nearly normal.

Are these symptoms constant (present all the time), present most of your daytime, less than one- half of the daytime, or just occasional? Daytime is defined as time between getting up and going to bed.

Motor symptoms (legs)

Do you have weakness in your legs or feet?

Do you ever trip over your toes/feet or turn or sprain your ankles? Do your feet slap when you walk?

Do you wear shoe inserts/insoles (below the ankle)?

Do you wear braces, splints, or equivalent type of orthotics that extend above your ankle?

Have the above ankle orthotics described above ever been prescribed or suggested by healthcare professionals? Have you had surgery on your feet or ankles?

If so, do you know if the surgery involved fusion of bones, a transfer of tendons, heel cord lengthening, or lowering of the arch?

Do you use a cane, stick, or walker to help you walk most of the time outside the home? Do you use a wheelchair most of the time because of weakness?

Motor symptoms (arms)

Do you have difficulty with buttoning clothes (standard shirt buttons)? If yes, are the difficulties mild or severe (severe includes unable)?

Can you cut most food including meat and pizza with normal utensils?

Do you have difficulty with activities that require extending or flexing your arms or activities using the upper arms?

Design of the reliability study for CMTNS2

After agreeing on the modifications, three examiners (M. M. R., D. P., and M. E. S.) met and examined two patients as a group to standardize examination techniques. A video of the agreed method of examination is available at http://rarediseasesnetwork.epi.usf.edu/training/confevents.asp?ConfID=6601. Sixteen patients with CMT were then prospectively evaluated by each of the examiners and scored based on the CMTNS2. The patients had a range of disability, allowing evaluation of the entire range of the CMTNS2. All patients gave written informed consent to participate in a National Institutes of Health (NIH)-funded natural history study, which had been approved by the local ethics committee. Patients were examined independently by the three examiners, who did not communicate with each other during the evaluations. All examiners utilized the same tuning fork to eliminate variability between instruments. Inter-rater reliability was based on data collected by the examiners at the same visit. Intra-rater reliability was based on repeat scoring of all 18 patients by one of the examiners (D.P.) within 8 weeks of the initial examination. Nerve conduction studies were performed on a single occasion using standard techniques with temperature controlled at >32 C. Ulnar SAP was performed using orthodromic testing and used to calculate the CMTNS, whereas radial SAP was performed using antidromic testing and used to calculate the CMTNS2. As these studies were performed only on a single occasion, they were not part of the reliability assessments. Subsequently, 63 patients in three centers were evaluated prospectively using both the original CMTNS as well as the CMTNS2 to compare the two tests.

Statistical analyses

The intra-class correlation coefficient (ICC), computed using a two-way random effects model with patient and rater as random effects, was used to assess inter-rater reliability. Intra-rater reliability was assessed by an ICC computed using a two-way mixed effects model with a fixed effect for visit and a random effect for patient (Shrout and Fleiss, 1979). Weighted kappa statistics were used to assess intra- and inter-rater agreement for individual components of the CMTNS2. Inter-rater agreement was assessed between each pair of raters for the individual CMTNS2 components. Paired t-tests were used to compare responses in individual categories between the CMTNS and the CMTNS2. A p-value <0.05 (two-tailed) was considered significant. All data analyses were performed using PASW Statistics version 18.0 (2009).

Results

Cohort

The 18 patients evaluated for the intra- and inter- rater reliability study included 13 patients with CMT1A, 2 with a genetically undetermined form of CMT1, and 1 with each of CMT1X, CMT2A, and CMT2. Their ages ranged from 24 to 59 years, with a mean of 40.2 years. Mean scores (standard deviation) for the CMTSS2, CMTES2, and CMTNS2 were 4.3 (2.5), 11.5 (5.8), and 16.8 (7.8), respectively. The 63 patients evaluated for the purposes of comparing individual components between the CMTNS and the CMTNS2 included 34 patients with CMT1A, 9 with CMT1X, 3 with CMT1C, 2 with CMT1B, 2 with CMT1A/1E, 1 with each of CMT2A, CMT2L, and CMT2F, 3 with a genetically undetermined form of CMT1, and 7 with a genetically undetermined form of CMT2. Mean scores (standard deviation) for the CMTSS2, CMTES2, and CMTNS2 were 4.6 (2.65), 11.55 (5.9), and 15.92 (7.41), respectively. Thus, the two groups were similar in terms of their diagnosis and CMT severity.

Inter-rater reliability

To measure inter-rater reliability, all three examiners evaluated the same 16 patients on the same day. The ICCs for the CMTSS2 and CMTES2 were 0.97 and 0.96, respectively (Table 3). For logistical reasons and patient preference, only eight of these patients had neurophysiological testing performed. The weighted kappa statistics for all pairs of the three examiners for each of the component scores are shown in Table 3. The three examiners scored exactly the same in 63% of the 112 (16 × 7) individual patient components. On 37% of these components, the examiners differed from each other; in one case the difference was 3 points, in three cases the difference was 2 points, the remainder were differences of only 1 point. Pin sensibility was the most common source of rater disagreement, with examiners’ scores differing by 1 point in 12/16 patients. Examiners’ scores differed in 7/16 patients for vibration; however, the lower weighted kappa statistics for vibration reflect the fact that when the examiners scored differently for pin they only differed by 1 point, whereas when they scored differently for vibration they differed by up to 3 points.

Table 3.

Inter-rater reliability

Component	ICC (95% CI)	Weighted kappa
Sensory symptoms	–	0.93, 0.95, 0.89
Motor symptoms (legs)	–	0.97, 0.93, 0.97
Motor symptoms (arms)	–	0.87, 0.95, 0.92
Pinprick sensibility	–	0.88, 0.78, 0.74
Vibration	–	0.71, 0.59, 0.77
Strength (legs)	–	0.88, 0.90, 0.97
Strength (arms)	–	0.88, 0.81, 0.88
CMTSS2	0.97 (0.94 – 0.99)	–
CMTES2	0.96 (0.90 – 0.98)	–

CI, confidence interval; CMTES2, CMT examination score (second version); CMTSS2, CMT symptom score (second version); ICC, intra- class correlation coefficient.

^*Weighted kappa statistics for each pair of raters (rater 1 vs. rater 2, rater 1 vs. rater 3, rater 2 vs. rater 3).

Intra-rater reliability

To assess intra-rater reliability, one examiner evaluated 18 patients twice within 8 weeks. The intra-rater ICCs for the CMTSS2 and CMTES2 were 0.96 and 0.97, respectively. The weighted kappa statistics for the component scores are shown in Table 4.

Table 4.

Intra-rater reliability

Component	ICC (95% CI)	Weighted kappa
Sensory symptoms	–	0.82
Motor symptoms (legs)	–	0.97
Motor symptoms (arms)	–	0.97
Pin sensibility	–	0.65
Vibration	–	0.78
Strength (legs)	–	0.88
Strength (arms)	–	0.98
CMTSS2	0.96 (0.89 – 0.98)	–
CMTES2	0.97 (0.92 – 0.99)	–

CI, confidence interval; CMTES2, CMT examination score (second version); CMTSS2, CMT symptom score (second version); ICC, intraclass correlation coefficient. CI, confidence interval; CMTES2, CMT examination score (second version); CMTSS2, CMT symptom score (second version); ICC, intraclass correlation coefficient.

Comparing CMTNS2 with CMTNS

Sixty-three patients were evaluated using the CMTNS and CMTNS2 during the same assessment. Fifty patients (79.4%) had ulnar CMAP, SAP, and radial SAP performed at the same assessment. Complete CMTNS and CMTNS2 data were available on 48 patients. Mean scores for the symptom score, examination score, and total neuropathy score did not differ significantly between the CMTNS and CMTNS2. However, mean scores of individual components differed significantly for sensory symptoms, motor symptoms (arms), pinprick sensibility, strength (arms), and ulnar vs. radial SAP (Table 5). Scores for each of these categories changed in a direction that would have been predicted based on the modifications contained in the CMTNS2. Ulnar SAP was absent in 36/56 (64%) patients, whereas radial SAP was absent in 25/52 (48%) patients. The percentages of patients falling into the different CMT severity categories (mild 0 – 10, moderate 11 – 20, severe >20) were similar for the CMTNS2 and the CMTNS. Forty-three patients of the 48 with complete data (90%) were classified as being of the same severity using the CMTNS and CMTNS2; only 5 patients moved into a different category of severity. Three of these patients were classified as moderate severity using the CMTNS and mild by the CMTNS2, one had been classified as severe by the CMTNS and moderate by the CMTNS2, and another had been classified as moderate by the CMTNS and severe by the CMTNS2. All of these patients had borderline scores such that a small change meant crossing the threshold into a different level of severity.

Table 5.

Comparison of mean scores between CMTNS and CMTNS2.

	Mean score (SD)
	CMTNS	CMTNS2	p-value
Sensory symptoms	1.70 (1.36)	1.59 (1.39)	0.03
Motor symptoms (legs)	1.89 (1.15)	1.97 (1.04)	0.17
Motor symptoms (arms)	0.87 (0.81)	1.05 (0.97)	0.0006
Pinprick sensibility	2.10 (1.13)	1.92 (1.20)	0.002
Vibration	2.11 (1.31)	2.29 (1.24)	0.37
Strength (legs)	1.70 (1.30)	1.66 (1.32)	0.46
Strength (arms)	1.48 (0.98)	1.13 (0.92)	0.0001
SAP (ulnar/radial)	3.25 (1.16)	3.06 (1.11)	0.03
Total CMTSS	4.46 (2.61)	4.60 (2.65)	0.09
Total CMTES	11.77 (5.97)	11.55 (5.90)	0.27
Total CMTNS	16.53 (7.72)	15.92 (7.41)	0.22

CMTES, CMT examination score; CMTNS, CMT neuropathy score; CMTNS2, CMT neuropathy score (second version); CMTSS, CMT symptom score; SD, standard deviation.

Discussion

We have described the modifications made to the CMTNS to create the CMTNS2, a composite score based on patients’ symptoms, signs, and neurophysiological testing. We showed that this test is reliable, with inter- and intra-rater ICCs of 0.96 – 0.97.

We found that intra- and inter-rater ICCs were similar for the CMTNS2, in contrast to the CMTNS, in which the intra-rater agreement was higher than the inter-rater agreement (Shy et al., 2005). This may reflect the fact that the CMTNS2 is more standardized; thus, there is less potential for individual variation between different examiners’ scoring of symptoms and signs. This was a limitation during reliability testing of the CMTNS when one of the examiners tended to score higher than the other examiner and mean scores were significantly different between examiners (Shy et al., 2005); this was not the case in this study.

One reason for modifying the CMTNS was that when the CMTNS was used in longitudinal studies of patients with CMT there was minimal change over time. In patients with CMT1A, Verhamme et al., using a modified version of the CMTNS, showed a deterioration of only 1.5 points over 5 years (Verhamme et al., 2009). Shy et al. showed a deterioration of 0.68 points/year (Shy et al., 2008). The placebo arm of a French trial of ascorbic acid progressed by 0.5 points/year (Micallef et al., 2009), while our study of ascorbic acid in CMT1A showed progression of only 0.2 points/year in the placebo group (Pareyson et al., 2011). This is a particular problem in CMT1A, a disease in which trials of treatment have already commenced. For treatment trials lasting 1 or 2 years, a treatment would likely have to yield an improvement (rather than a slowing) of the disease course for the mean response in the treatment group to be statistically apparent when compared with that of the placebo group.

When comparing components of the CMTNS with those of the CMTNS2, the components that differed significantly included sensory symptoms, motor symptoms (arms), pinprick sensibility, strength (arms), and SAP. These were the items to which most modifications had been made. As would be predicted, mean scores for sensory symptoms, pinprick sensibility, strength (arms), and SAP were lower for the CMTNS2 than for the CMTNS, whereas the mean score for motor symptoms (arms) was higher for the CMTNS2. It is hoped that these differences will make the scale more sensitive to change, for example, a change in sensory symptoms from just above the ankle to knee level will now result in an extra point in the CMTNS2, whereas previously this change would not have altered the CMTNS score.

The use of the radial rather than ulnar SAP reduces the ceiling effect for this item. Ulnar SAP was absent in 64% of patients, compared with 48% for radial SAP; thus, a deterioration in SAP should be detected better with the use of the CMTNS2. As in the reliability study of the original CMTNS, we did not perform neurophysiology on two separate occasions on each patient; thus, the scores for the neurophysiology items are based on a single evaluation. We elected not to perform these studies twice due in part to logistical reasons, and partly because nerve conduction studies are uncomfortable for patients. CMAP and SAP amplitudes have been shown to have excellent test-retest reliability with ICC > 0.92 (Ven et al., 2008; Kong et al., 2009; Lewlet et al., 2010).

We acknowledge that there remain some limitations of the CMTNS2 despite modification. In particular, pinprick sensibility and vibration appear to show less inter- and intra-rater agreement than other components of the test. These two items are perhaps more subjective than motor scoring and neurophysiological testing, although we attempted to make them more objective by the use of standardized instruments (Rydel-Seiffer tuning fork and Neurotip). This highlights the importance of giving standardized instructions to patients and examiners using the same method of examination (as described in written instructions, Table S2). Despite the modest agreement on these two items, overall inter- and intra-rater agreement for the CMTSS2 and CMTES2 remained high, suggesting that this modified version can be used among different examiners and across several sites for the purposes of future studies, assuming that the raters have been appropriately trained in its use. In addition, the CMTNS2 is a composite score comprising symptoms, signs, and neurophysiology and is not a continuous measure. For this reason, similar to other ordinal scales, a difference in total score may not mean the same throughout the whole range of the score.

In conclusion, the CMTNS2 is a reliable scale in patients with CMT, with high inter- and intra-rater reliability for its clinical components (CMTSS2 and CMTES2). To determine whether the CMTNS2 is a more sensitive measure for showing change over time than the CMTNS, we are using this scale to assess patients with CMT longitudinally in a large natural history study.