Development and Validation of a Clinical Scale for Rating the Severity of Blepharospasm

Abstract

Existing scales for rating the severity of blepharospasm (BSP) are limited by a number of potential drawbacks. We therefore developed and validated a novel scale for rating the severity of BSP. The development of the scale started with careful examination of the clinical spectrum of the condition by a panel of experts who selected phenomenological aspects thought to be relevant to disease severity. Thereafter, selected items were first checked for reliability, then reliable items were combined to generate the scale, and clinimetric properties of the scale were evaluated. Finally, the confidence with which the scale could be used by people without high levels of movement disorders skill was assessed. The new scale, based on objective criteria, yielded moderate to almost perfect reliability, acceptable internal consistency, satisfactory scaling assumptions, lack of floor and ceiling effects, partial correlations with a prior severity scale and with a quality of life scale, and good sensitivity to change. Despite a few limitations, the foregoing features make the novel scale more suitable than existing scales to assess the severity of BSP in natural history and pathophysiologic studies as well as in clinical trials.

Blepharospasm (BSP)¹ is characterized by bilateral, synchronous, and symmetric involuntary orbicularis oculi (OO) muscle spasms leading to partial/total eyelid rim closure.^2–4 Spasms may be brief or prolonged, are usually aggravated by bright light, stress, or voluntary muscle contraction, and are reduced by attention-demanding tasks such as writing, particularly in severe cases.^2–4 Additional features are eye symptoms, increased blinking, sensory tricks, apraxia of eyelid opening (AEO), and dystonia in other body parts.^2–6

Clinical evaluation of BSP poses a number of challenges, particularly in the diagnosis and severity rating. Although a specific diagnostic guideline has been recently validated,⁷ the widespread use of existing severity scales is limited by several drawbacks.⁸ The Burke-Fahn-Marsden rating scale,⁹ the Unified Dystonia Rating Scale,¹⁰ and the Global Dystonia Severity Rating Scale¹⁰ comprehensively measure dystonia severity in all body parts. In these scales, severity grading is based on the intensity of dystonic contractions merged with (as in the Burke-Fahn-Marsden rating scale), or weighted by (as in the Unified Dystonia Rating Scale) duration and daily frequency of the spasms. Whereas spasm intensity can be evaluated by clinical examination, an assessment of spasm frequency relies on subjective patient’s report. Moreover, these scales adopt the same grading modality for dystonia at different body sites, each of which ideally warrants a specific severity assessment system. Finally, these rating scales may also prove unreliable because of the relatively high number of severity gradations used. Although these scales have excellent internal consistency,^9,10 inter-rater agreement is not optimal for dystonia affecting the upper/lower face, jaw, neck, and larynx.^8–10 To date, the only severity scale specifically developed for BSP is the Jankovic Rating Scale (JRS).¹¹ This scale includes two subscales that measure intensity and frequency of eyelid spasms, both based on a 5-point grading system. This scale may be subject to several criticisms, including the lack of a clear definition of spasms regarding the degree of rim closure, the combination of examiner-based and patient-based information, and the lack of attention to clinical features. Although the JRS displays excellent internal consistency, reliability measures and other clinimetric properties are not available.¹¹

All of the aforementioned considerations highlight the need for a novel BSP severity scale overcoming limitations of existing scales.

Methods

A Novel BSP Severity Scale Was Developed and Validated in a Four-Step Procedure

In the first step, senior movement disorder specialists (G.D., M.H., H.J., and A.B.) identified a list of phenomenological aspects possibly related to BSP severity (Table 1) that was submitted to eight neurologists and two ophthalmologists with long-standing experience in BSP. These experts evaluated whether each aspect was, or was not, relevant to severity rating by calculating the content validity ratio (CVR) as described in Supplemental Data Appendix 1. We arbitrarily considered as potentially useful those items that reached a CVR greater than 0.5.⁷

TABLE 1.

Content validity analysis of clinical items thought to be relevant for the severity assessment of blepharospasm

Items	Content Validity Ratio
A. Degree of eyelid rim narrowing	0.8
B. Eyebrow lowering below the superior margin of the orbit	0.2
C. Duration of eyelid spasms	0.8
D. Frequency of eyelid spasms	1
E. Complexity of eyelid spasms (isolated spasms or “‘trains’” of spasms)	0.2
F. Signs/symptoms associated with eyelid spasms	0.2
- Eye symptoms	0.25
- Effective sensory tricks	0.0
- Duration of effect of the sensory trick	0.0
- Forcible quality of the trick	0.8
- Blink rate	0.55
- Apraxia of eyelid opening	0.55
- Dystonia in the lower face	0.0
- Dystonia in body parts other than the face

In the second step, interobserver and intraobserver reliability of selected items was assessed using 68 video-recordings of Italian outpatients providing written informed consent (institutional review board approval of the University of Bari n. 483, April 18, 2011). Thirty-eight women and 30 men aged 71 years [SD, 9] who had had BSP for 11 years [SD, 8] on average, were studied. Video-recordings were performed according to a standardized video protocol lasting approximately 5 minutes (Supplemental Data Appendix 1). Two neurologists who did not know the patients reviewed video-recordings. The raters, who were experienced in movement disorders but had no specific expertise in BSP, underwent brief training. A sudden OO muscle contraction causing eyelid rim narrowing/closure was classified as a spasm that could induce brief (<3 sec) or prolonged eyelid closure (≥3 sec). A bilateral, synchronous short duration (<1 sec) OO muscle contraction causing a transient eyelid drop was considered as a blink. Delay in reopening the eyelids after involuntary closure associated with no overt OO contraction and raising of the eyebrow above the superior orbital margin was considered as AEO. A repeat rating was performed by one of the neurologists 5 months later. Reliability was calculated by kappa statistics or the intraclass correlation coefficient (ICC), as appropriate. Kappa values greater than 0.4 and ICC values greater than 0.7 indicated satisfactory agreement.

In the third step, a scale was designed on the basis of the items that survived the CVR criterion and demonstrated acceptable reliability. We assumed that both intensity and frequency would contribute to the severity of spasms. To rate intensity, we first assigned a basic score to each type of spasm (prolonged spasms were given higher basic scores than brief spasms); then the basic score was graded whenever possible by spasm-associated features, including AEO, lower face (LF) spasms, and effect of writing on OO spasms. We also considered the average duration of prolonged spasms with complete eyelid rim closure (recorded during the last 120 seconds of video-recording) as a further determinant of intensity. Duration of prolonged spasms was then transformed into tertiles calculated from the frequency distribution of the variable in the overall BSP sample. Finally, to rate frequency, we counted both brief spasms and blinks together as well as the number of prolonged spasms occurring during the last 120 seconds of video-recording. Number of spasms/min was then transformed into tertiles obtained from the frequency distribution of each variable in the overall BSP population being studied. Total severity score was calculated by adding subscores from all items.

In the fourth step, we evaluated clinimetric properties of the scale, including item-to-total correlation,^12,13 internal consistency,¹⁴ floor and ceiling effects,¹⁵ precision,¹² convergent and discriminant validity,^16,17 and sensitivity to change.

In the fifth step, three neurological residents from the University of Bari were trained on the phenomenology of BSP as described (see second step) and assessed 15 video-recordings of patients who did not participate in the scale development. The rating was repeated by one resident 10 days later. Reliability of total and partial scores were calculated by kappa statistics or the ICC, as appropriate.

Results

Generation of the Items

Among the 13 clinical aspects initially identified by the movement disorder specialists, six were given a CVR greater than 0.5 by the panel of experts (Table 1). These were degree and duration of OO spasms, frequency of spasms, blink rate, AEO, and LF spasms. As a seventh aspect, we suggested that attention be paid as to whether spasms occurred during writing (attention-demanding task). According to degree and duration, we identified four types of OO spasms in the study population: brief spasms (lasting < 3 sec) and prolonged spasms (lasting ≥ 3 sec), both leading to complete or incomplete rim closure. Prolonged spasms with incomplete rim closure were typically long lasting, being present for most of the examination period. Prolonged spasms with complete rim closure lasted on average 3 to 9 sec. Overall, brief spasms (particularly those associated with complete rim closure) were present in all patients, prolonged spasms with incomplete rim closure were present in 47 patients, and prolonged spasms with complete rim closure were present in 18 patients. Counting the number of blinks + brief OO spasms with complete rim closure in the last video segment and stratifying the values in tertiles (Fig. 1) yielded 23 patients in the first tertile, 25 patients in the second tertile, and 20 patients in the third tertile. The sustained quality of prolonged spasms with incomplete eyelid rim closure made their frequency difficult to measure. Stratification of the number of prolonged spasms with complete rim closure counted during the last video-segment in tertiles (Fig. 1) yielded six patients in the first tertile, eight patients in the second tertile, and four patients in the third tertile. Stratification of the average duration of prolonged spasms with complete rim closure in tertiles (Fig. 1) yielded eight patients in the first tertile, five patients in the second tertile, and five patients in the third tertile. The AEO and LF spasms were present in 10 and 36 patients, respectively. Orbicularis oculi spasms during writing were observed in 23 of 68 patients.

FIG. 1.

Blepharospasm severity scale

Inter- and Intra-rater Reliability of the Items

Eight items derived from the seven selected phenomenological aspects and categorized as shown in Table 2 underwent reliability assessment. Satisfactory inter-observer reliability was reached for prolonged spasms, brief spasms inducing complete eyelid rim closure, number of blinks + brief spasms/min, number and average duration of prolonged spasms inducing complete rim closure/min, presence of AEO and of LF spasms, and occurrence of spasms during writing (Table 2). Repeat rating of videotapes yielded acceptable intra-rater reliability for all of these items (kappa > 0.76; ICC > 0.86).

TABLE 2.

Inter-observer agreement on the proposed blepharospasm severity items.

Items	K / ICC	P
Items 1 and 2	0.03	0.3
Degree and duration of eyelid narrowing induced by eyelid spasms	0.82	0.0003
Brief spasms with partial eyelid rim closure (< 3 sec)	0.65	0.0004
Prolonged spasms with partial eyelid rim closure (> 3 sec)	0.76	0.0006
Brief spasms with complete eyelid rim closure (< 3 sec)
Prolonged spasms with complete eyelid rim closure (> 3 sec)
Item 3	0.92	0.001
Rate of Blink + brief eyelid spasms with complete eyelid rim closure/min
Item 4	0.87	<0.001
Rate of prolonged eyelid spasms with complete eyelid rim closure/min
Item 5	0.98	<0.001
Duration of prolonged eyelid spasms with complete eyelid rim closure
Item 6	0.28	0.002
Apraxia of eyelid opening	0.29	0.001
Eyelid spasms are followed by transient brief inability (< 3 sec duration) to raise eyelids associated with frontalis muscle hyperactivity (inducing raised eyebrow)	0.60	0.007
Eyelid spasms are followed by transient prolonged (≥ > 3 sec duration) inability to raise eyelids, associated with frontalis muscle hyperactivity (inducing raised eyebrow)
Apraxia of eyelid opening (y/n)
Item 7	0.17	0.04
Lower face spasms	0.52	0.001
- Sporadically associated with the eyelid spasms - Usually associated with eyelid spasms - Lower face spasms (y/n)	0.55	0.01
Item 8	0.88	0.001
Attention-demanding task
- Eyelid spasms that occur during writing (y/n)

Scale Generation and Clinimetric Properties

The eight items that had acceptable CVR and reliability were then grouped in a preliminary version of the scale (Table 3). Most items gained item-to-total correlation greater than the criterion 0.30; LF spasms yielded the weakest scaling assumption (rho = 0.27) and was therefore excluded.

TABLE 3.

Items included in the preliminary version of the blepharospasm severity scale (scoring system is in parenthesis) and item-to-total correlations (Spearman rho)

Items	Item-to-total correlation (Spearman rho)
Items 1 and 2. Degree and duration of eyelid narrowing induced by orbicularis oculi spasms (score: 1/2/3)	0.82
Item 3. Rate of blinks + brief eyelid spasms (tertiles, score: 1–3)	0.39
Item 4. Rate of prolonged eyelid spasms (tertiles, score: 1–3)	0.76
Item 5. Duration of prolonged spasms (tertiles, score 1–3)	0.77
Item 5. Apraxia of eyelid opening (score: 0/2)	0.41
Item 6. Lower face spasms (score: 0/1)	0.27
Item 7. Effect of writing on eyelid spasms (score: 0/1)	0.35

We tested the clinimetric properties of two final versions of the scale, a complete version including all of the selected items (Fig. 1) and a shorter version that did not consider the average duration of prolonged spasms with complete eye closure (item A4 in Fig. 1). Theoretically, the complete and the short scale yielded a maximum score of 18 and 15, respectively. Among the 68 participants, the mean total severity score was 6.7 for the complete scale (range, 2–15; SD, 2.4; standard error of the mean [SEM], 0.29), 6.2 for the short scale (range, 2–12; SD, 2.6; SEM, 0.31). No patient achieved theoretical minimum and maximum score values. Minimum and maximum scores (2 and 15 in the complete scale, 2 and 12 in the short scale) were reached by four patients (5.9%) and six patients (8.8%), respectively. No significant correlation was found between total severity score from either version of the scale and age or disease duration (data not shown). Internal consistency yielded similar values for both the complete and the short scale (Cronbach’s alpha, 0.75 vs. 0.71). With regard to the convergent validity, a significant partial correlation emerged between the JRS and both the complete scale (rho = 0.67, P < 0.0001) and the short scale (rho = 0.66, P < 0.0001). An assessment of discriminant validity revealed significant correlation between both versions of the severity scale and two of the five domains of CDQ-24 scale (complete scale: emotional well-being, rho = 0.49, P = 0.01; activities of daily living: rho = 0.52, P = 0.01. Short scale: emotional well-being, rho = 0.45, P = 0.008; activities of daily living: rho = 0.54, P = 0.01). Finally, comparisons of the total severity score before and after botulinum toxin (BoNT) treatment in 12 patients revealed a significant decrease in the score after BoNT (Complete scale: 8.2 ± 2.1 vs. 5.2 ± 2.3, P < 0.0001; short scale: 7.7 ± 1.6 vs. 4.9 ± 2.0, P < 0.0001).

Scale Done by Residents

When the scale was administered by three neurological residents to 15 BSP patients, satisfactory inter-rater reliability was seen for both total score (ICC, 0.72–0.94) and subscores from individual items (type of eyelid spasm score: ICC, 0.77–0.88; AEO score: k = 0.78; writing score: k = 0.78; duration of prolonged spasm score: K = 1; number of blinks + brief eyelid spasm score: k = 1; number of prolonged eyelid spasm score: k = 0.93). Intra-rater reliability yielded satisfactory results for both total score (ICC = 0.83) and subscores from individual items (data not shown).

Discussion

A novel scale for rating BSP severity was developed and validated by a multistep procedure that started with selection of phenomenological aspects possibly relevant to BSP severity by a panel of experts. Thereafter, selected items were first checked for reliability, then reliable items were combined to generate the scale, and clinimetrics properties were evaluated. Reliability of scale administration by three residents without high levels of movement disorder skill was also assessed.

Seven clinical items contributed to the final version of the scale. Among them, degree and duration of eyelid closure caused by spasms and frequency of spasms are the core clinical hallmarks of BSP severity; increased blinking, AEO, and occurrence of spasms during writing are useful to grade severity. Our procedure showed that the selected items are reliable and have satisfactory scaling assumptions^12,13 except for the item assessing LF spasms that was omitted from the final scale formulation. Although LF spasms add to the severity of the overall disorder when present, they are clearly separable from the eyelid closures.

Most selected items can be easily administered and measured during a brief clinical examination. Only accurate measurement of duration of prolonged spasms with complete eyelid rim closure would require the examination to be video-recorded. However, both the complete scale and a shorter version of the scale that did not include duration of prolonged spasms have similar and satisfactory clinimetrics. Internal consistency was acceptable for a scale with a relatively small number of items, particularly if one considers that Cronbach alpha is also dependent on the number of items.¹⁴ Because we observed that subjects with total score near the bottom or the top of the scale did not exceed 15% in either scale formulation, we could rule out the possibility of floor or ceiling effects. Our analysis found a partial correlation between both versions of the scale and the JRS,¹¹ or the quality of life (QoL) scale CDQ-24.^16,17 The scale presented here explores domains that are not considered in the prior scales: estimation of spasm severity is present in the JRS¹¹ but not in the CDQ-24,^16,17 whereas spasm-associated features are not considered in the JRS but may contribute to QoL assessment by the CDQ-24. Comparison of total severity scores at baseline and 4 weeks after BoNT treatment highlighted the sensitivity to change of both versions of the scale. Finally, our analysis did not detect any significant relationship between the novel scale and age, or disease duration, which is consistent with the notion that BSP severity is independent of these variables.

Other features of the current scale are advantageous in comparison with existing scales. The new scale is based on objective clinical parameters and includes clear definitions of different types of OO spasms based on the degree and duration of eyelid rim closure. Because of the variable duration of spasms in different patients or within the same patient over time, we were aware that any way of stratifying by spasm duration is arbitrary. We chose to distinguish brief spasms (<3 sec) and prolonged spasms (>3 sec). This approach was reliable and likely reflected the spectrum of BSP severity. Brief spasms (probably carrying a lower severity burden) were present in all patients, whereas prolonged spasms were present in only some of them. Some spasm-associated features were included in the scale as further clinical determinants of severity. All of the items that contributed to the scale proved to be reliable. Finally, to maximize reliability, we did not assign gradations to individual items. To grade severity, we instead assigned a basic score for each type of spasm, which was then adjusted according to the presence of further clinical determinants of severity and added to a specific frequency factor.

The present study has limitations. Eye symptoms, for example, are not considered in the scale. They are, however, considered to be the consequence of eye diseases possibly triggering BSP. Other BSP features, such as the need to wear sunglasses or the inability to read or watch television, do not determine the severity of BSP, but more likely are BSP consequences, which can be influenced by individual factors impacting on QoL, including age, sex, socioeconomic status, and social support. In assessing the severity of BSP, pure measures of motor impairment appear to be more suitable than disability or QoL measures. We counted brief OO spasms with complete rim closure and blinks together. This may be reasonable because such spasms are sometimes difficult to distinguish from blinks; moreover, blinking is thought to be related to BSP pathophysiology¹⁸ and thus may contribute to BSP severity. We did not consider whether the severity of AEO should be considered along with its presence or absence. However, our attempt to stratify by AEO duration/intensity yielded unreliable results. The score distribution in our study sample did not cover the complete range of total scoring, because of the lack of patients scoring the highest three total points. This may suggest that the study sample was not completely representative of all severity stages, even though patients in all stages of BSP severity as categorized by the JRS were present. Scoring the upper three points in both versions of the new scale applies to patients manifesting all types of spasms and yielding the maximum score on almost all determinants of severity and frequency. These patients are not frequent and are not considered by existing scales as a separate category. By contrast, the new scale we propose would theoretically detect even these severe forms of BSP. Reserving three points (one fifth of the scale) to this aim would be adequate to grade the most severe BSP forms.

In conclusion, we developed a severity scale that takes into account the most relevant BSP motor abnormalities, is based on objective criteria, and yields moderate to almost perfect reliability, satisfactory scaling assumptions, acceptable internal consistency, lack of floor and ceiling effects, partial correlations with prior severity and QoL scales, and sensitivity to change. A further advantage is that the scale can be reliably administered by people without high levels of movement disorders experience. Administration of the shorter scale is relatively easy, only requiring a brief standardized clinical examination lasting approximately 5 minutes. To measure the average duration of prolonged spasms with complete eyelid rim closure accurately, as is needed for the complete scale, would require the clinical examination to be video-recorded. Because both scale versions share similar clinimetrics, we would suggest using the shorter scale in routine clinical practice, whereas the complete scale would be suitable to assess BSP severity in research settings.