Development and validation of a clinical guideline for diagnosing blepharospasm

Abstract

Objective:

To design and validate a clinical diagnostic guideline for aiding physicians in confirming or refuting suspected blepharospasm.

Methods:

The guideline was developed and validated in a 3-step procedure: 1) identification of clinical items related to the phenomenology of blepharospasm, 2) assessment of the relevance of each item to the diagnosis of blepharospasm, and 3) evaluation of the reliability and diagnostic sensitivity/specificity of the selected clinical items.

Results:

Of 19 clinical items initially identified, 7 were admitted by content validity analysis to further assessment. Both neurologists and ophthalmologists achieved satisfactory interobserver agreement for all 7 items, including “involuntary eyelid narrowing/closure due to orbicularis oculi spasms,” “bilateral spasms,” “synchronous spasms,” “stereotyped spasm pattern,” “sensory trick,” “inability to voluntarily suppress the spasms,” and “blink count at rest.” Each selected item yielded unsatisfactory accuracy in discriminating patients with blepharospasm from healthy subjects and patients with other eyelid disturbances. Combining the selected items, however, improved diagnostic sensitivity/specificity. The best combination, yielding 93% sensitivity and 90% specificity, was an algorithm starting with the item “stereotyped, bilateral, and synchronous orbicularis oculi spasms inducing eyelid narrowing/closure” and followed by recognition of “sensory trick” or, alternatively, “increased blinking.”

Conclusion:

This study provides an accurate and valid clinical guideline for diagnosing blepharospasm. Use of this guideline would make it easier for providers to recognize dystonia in clinical and research settings.

Blepharospasm (BSP) is characterized by involuntary orbicularis oculi (OO) muscle spasms that are usually bilateral, synchronous, and symmetrical.¹ Other signs and symptoms possibly accompanying BSP include sensory symptoms in the eyes often indicating ocular diseases (e.g., dry eye syndrome),² an increased spontaneous blink rate (BR),³ and sensory tricks that can transiently improve eyelid spasms.⁴

Diagnosing BSP and assessing the impact of treatments on the condition are major needs in clinical practice and research. Although a new scale specifically developed for rating the severity of BSP has been validated,⁵ the diagnosis of BSP is presently based on clinical grounds and is open to bias. The lack of validated diagnostic criteria sometimes makes it difficult to distinguish BSP from other conditions of involuntary eyelid closure such as eyelid tics, hemifacial spasms, facial chorea, apraxia of eyelid opening, frequent blinking, and lid ptosis due to myasthenia or other causes.⁶

BSP may be diagnosed several years after the first symptoms manifest, and in the interim, many patients visit numerous physicians, delaying access to treatment.⁷ Family studies indicate that up to half of people with dystonia may be undiagnosed or misdiagnosed.^8,9

The lack of a diagnostic biomarker makes it essential to demarcate accurate diagnostic criteria to identify patients to be included in epidemiologic studies as well as in possible future clinical trials, reducing diagnostic variability among physicians. In this study, we designed and validated a clinical guideline to aid the physician in confirming or refuting suspected BSP.

METHODS

The diagnostic guideline was developed and validated in a 3-step procedure. First, 4 senior movement-disorder experts (G.D., M.H., H.J., and A.B.) identified a list of clinical items related to the phenomenology of BSP. In the second step, the list was submitted to a panel of 10 experts with long-standing experience in diagnosing and managing BSP (8 neurologists and 2 ophthalmologists) who evaluated whether each item was relevant or not to the diagnosis of BSP (see appendix e-1 on the Neurology^® Web site at www.neurology.org). To assess the importance the expert panel assigned to each item, the content validity ratio (CVR) was then calculated according to the following formula:

where ne = number of raters indicating the item as “relevant” and N = total number of raters. The CVR ranged between −1 (this means that all raters judged the item as nonrelevant) and +1 (all raters judged the item as relevant). For the diagnosis of BSP, we arbitrarily considered potentially useful those items reaching a CVR of ≥0.5. In the third step, 3 neurologists and 2 ophthalmologists who did not participate in the expert panel reviewed the video recordings of 30 patients with BSP (11 men and 19 women aged 49–65 years), 10 healthy subjects, and 30 control patients with conditions causing eyelid closure other than BSP (6 patients in each of the following diagnostic groups: eyelid tics, lid ptosis due to myasthenia or oculomotor palsy, apraxia of eyelid opening, hemifacial spasm, and chorea) recruited at the Movement Disorders Centers at the Universities of Bari and Rome “Sapienza.” The 5 raters were neurologists and ophthalmologists from Italian institutions other than the recruiting centers who were experienced in movement disorders without any specific expertise in BSP. The standardized video protocol (see appendix e-2) lasted long enough (approximately 5 minutes) to reproduce all the major/distinctive diagnostic features identified by the clinical examination and was integrated with standard maneuvers triggering facial spasms, attempting to observe a sensory trick, and excluding causes of eyelid closure other than dystonia.

Neurologists and ophthalmologists reviewed the video recordings to identify those clinical items that reached a CVR of ≥0.5 in the previous phase. Raters received brief training on the phenomenology of BSP possibly relevant to diagnosis. A sudden, involuntary, long-lasting OO muscle contraction causing eyelid narrowing/closure was classified as a muscle spasm, whereas a bilateral, synchronous, short-duration OO muscle contraction causing a transient eyelid drop was considered as a blink. BR (expressed as blinks per minute) was calculated with subjects at rest and eyes open during the last video segment as described. A sensory trick was defined as any kind of manual maneuver performed by the patient that led to a transient reduction of the severity of dystonic posturing or movements in the period of time immediately after its execution. Finally, the item “voluntary spasm suppression” was defined as an inner volitional effort rather than voluntary compensatory frontalis muscle overactivity.

Based on the results of video-recording assessment, inter- and intrarater reliability for each selected clinical item was calculated using κ statistics or the intraclass correlation coefficient (ICC) as appropriate. The level of agreement indicated by κ indexes was assessed according to the Landis classification.¹⁰ ICC >0.75 indicated high reproducibility.¹¹ BSP as diagnosed by the 2 senior neurologists (G.D. and A.B.) from the recruiting centers (reference standard) was compared with BSP as diagnosed by each rater. Sensitivity was the proportion of subjects (patients with BSP, healthy controls, and patients with eyelid disorders other than BSP) that the rater considered as having BSP among those diagnosed by the senior neurologists as having BSP. Specificity was the proportion of subjects (patients with BSP, healthy controls, and patients with eyelid disorders other than BSP) who screened negative among those diagnosed by the senior neurologists as unaffected by BSP. To estimate sensitivity and specificity of BR values in discriminating between patients with BSP and controls, a receiver operating characteristic curve was plotted. The point closest to 80% sensitivity and 80% specificity was defined as the best trade-off threshold discriminating the 2 groups.

Standard protocol approvals, registrations, and patient consents.

All participating patients and healthy subjects gave informed consent to the study, which was approved by the ethics committee of the University of Bari (IRB approval no. 483, April 18, 2011).

RESULTS

Among the 19 clinical diagnostic items initially identified and submitted to the expert panel, 7 yielded a CVR >0.5 (table 1) and underwent further assessment.

Table 1.

Content validity analysis testing the clinical phenomenology of blepharospasm

The 5 observers (3 neurologists and 2 ophthalmologists) achieved significant interobserver agreement for all 7 items, including “involuntary eyelid narrowing/closure due to orbicularis oculi spasms” (κ = 0.91, p < 0.00001), “bilateral spasms” (κ = 0.96, p < 0.00001), “synchronous spasms” (κ = 0.86, p < 0.0001), “stereotyped spasm pattern” (κ = 0.72, p < 0.0001), “sensory trick” (κ = 0.7, p < 0.0001), and “inability to voluntarily suppress the spasms” (κ = 0.74, p < 0.0001). Likewise, ICCs >0.81 (p < 0.001) were computed between each pair of raters for “blink count at rest,” indicating that the item achieved high reproducibility. The analysis assessing interobserver agreement separately in neurologists and ophthalmologists yielded similar findings (data not shown). Two neurologists repeated their rating 8 months after the first assessment, showing acceptable internal consistency for each item (κ index always >0.76).

BR at rest (mean values from neurologists and ophthalmologists) was significantly higher in patients with BSP than in control subjects (46.1 ± 27.1 vs 15 ± 15.4, p < 0.0001). In our sample, the cutoff value for discriminating patients with BSP from controls closest to 80% sensitivity and 80% specificity was 15.6 blinks/min (figure 1).

Figure 1. Receiver operating characteristic curve displaying sensitivity and specificity for the blink rate.

The arrow indicates the cutoff value (16 blinks/min) discriminating patients with blepharospasm from controls.

Because the items “involuntary eyelid narrowing/closure due to orbicularis oculi spasms,” “bilateral spasms,” “synchronous spasms,” and “stereotyped spasm pattern” were all considered to belong to the same phenomenologic domain, they were combined for sensitivity and specificity assessment. Referring to the selected items, neurologists and ophthalmologists achieved similar accuracy in discriminating patients with BSP from healthy subjects and patients with eyelid disturbances other than BSP (table 2).

Table 2.

Sensitivity and specificity of clinical items scoring >0.5 on content validity ratio analyses in diagnosing blepharospasm^a

When we tested whether combining the selected items would improve diagnostic sensitivity/specificity, we found that the best combination, yielding 93% sensitivity and 90% specificity in the 5 observers, was the diagnostic algorithm described in figure 2. This algorithm started with the item that reached the greatest sensitivity, “stereotyped, bilateral, and synchronous orbicularis oculi spasms inducing eyelid narrowing/closure” (table 2). The second step was recognition of sensory trick, the item reaching the greatest specificity (table 2). In the absence of a sensory trick, including in the guideline the term “increased blinking” yielded the greatest diagnostic accuracy (93% sensitivity and 90% specificity) as compared with the item “inability to voluntarily suppress the spasms” (75% sensitivity and 95% specificity) or both combined (72% sensitivity and 95% specificity). Subjects who yielded false-positive results according to the finally proposed algorithm included 2 healthy subjects, one patient with chorea, and one with myasthenia.

Figure 2. Guideline for diagnosing blepharospasm.

DISCUSSION

Of the 19 clinical diagnostic items initially proposed, 7 items were considered important for diagnosing BSP by the expert panel that performed content validity analysis. Among the selected items, “eyelid narrowing/closure due to spasms” proved useful to differentiate BSP from bilateral ptosis (e.g., myasthenia) as well as from isolated apraxia of eyelid opening (i.e., isolated inability to open the eyelids after a voluntary eye closure)⁶; “stereotyped or patterned spasms” usually helped to differentiate BSP from chorea⁶; and finally, “bilateral spasms” and “synchronous spasms” differentiated BSP from unilateral or bilateral hemifacial spasm and unilateral ptosis.⁶ Because all of these items directly reflected OO spasms, we assessed them in combination in the further validation steps. “Sensory trick” (a highly specific dystonic feature),⁴ increased blinking (thought to represent a forme fruste of BSP),³ and “inability to voluntarily suppress the spasms” (a feature that helps to differentiate BSP from tics) were also added to the content validity analysis.¹²

The following clinical items were excluded from the validation procedure because of a CVR <0.5: features of dystonic spasms sometimes absent during OO contractions (Charcot sign and symmetrical OO spasms); conditions possibly reflecting involvement of muscles other than the OO muscles (apraxia of eyelid opening, frontal hyperactivity, spasms in the lower face, and dystonia in other body sites); signs that may merely reflect eye diseases (eye symptoms and photophobia/photo-oculodynia)⁶; and features that are usually absent in dystonic BSP (premonitory sensations, OO muscle paresis, eyelid ptosis, and double vision) and present in other eyelid disorders.^6,12

The items selected by content validity analysis reached satisfactory inter- and intraobserver agreement but achieved variable diagnostic accuracy. The presence of bilateral, synchronous, and stereotyped OO spasms inducing eyelid narrowing/closure yielded high diagnostic sensitivity, thus confirming that this clinical feature is crucial for diagnosing BSP. When applying these criteria alone, however, physicians incurred the risk of misclassifying many cases, as indicated by 85% specificity. This value was unsatisfactory, in part because a 5-minute standardized video recording cannot take into account variability caused by the patient's psychological status and the test circumstances. None of the other selected items allowed an accurate diagnosis of BSP (table 2). The low accuracy reached by the item “inability to voluntarily suppress the spasm” probably reflected variability caused by the patient's education, test circumstances, and attitude of the observer. Because of the significant amount of variability in BR among normal subjects due to the psychological status and behavioral condition, the 16 blinks per minute derived from the receiver operating characteristic curve analysis should be considered as a cutoff value specific for our test circumstances.

Despite the foregoing limitations, combining “presence of bilateral, synchronous, and stereotyped OO spasms inducing eyelid narrowing/closure” with “sensory trick” or “increased blinking” yielded a diagnostic algorithm that was sensitive and specific enough to be proposed as a guideline for presumptive diagnosis of BSP (figure 2).

The present study has some limitations. We did not check whether incorporating the proposed guideline is better than providing only “brief training” to the raters but without the specific criteria. Nevertheless, there are several lines of evidence indicating that, in the absence of specific criteria, there is considerable variability in the diagnostic attitude among physicians. Our aim was to provide a valid and accurate guideline capable of reducing variability among physicians. Finally, because all of the patients and the evaluating physicians involved in the study were from the same country, the results of the study need to be confirmed in different patient and physician populations.

Our study had several strengths. First, the validation procedure included patients with BSP (whose demographic and clinical characteristics resembled those of patients reported in other published series), healthy controls, and subjects with eyelid disorders other than BSP. Second, the standardized videotape protocol reproduced all the major features seen during the clinical examination. Finally, our raters included both neurologists and ophthalmologists, physicians who usually care for patients with BSP.

This study provides an accurate and valid clinical guideline, based on objective criteria, to diagnose BSP in both clinical and research settings.

GLOSSARY

BR

blink rate

BSP

blepharospasm

CVR

content validity ratio

ICC

intraclass correlation coefficient

OO

orbicularis oculi

AUTHOR CONTRIBUTIONS

Dr. Defazio: study concept and design, acquisition of data, analysis and interpretation, critical revision of the manuscript for important intellectual content, study supervision. Dr. Hallett and Dr. Jinnah: study concept and design, critical revision of the manuscript for important intellectual content. Dr. Berardelli: study concept and design, acquisition of data, critical revision of the manuscript for important intellectual content.

STUDY FUNDING

Funded by the Benign Essential Blepharospasm Research Foundation and a Pilot Project Grant from the Dystonia Coalition (NS065701 from the Office of Rare Diseases Research and the National Institute of Neurological Disorders and Stroke at the National Center for Clinical and Translational Sciences).

DISCLOSURE

G. Defazio received funds from the Italian Ministry of University and from the Benign Essential Blepharospasm Research Foundation for research projects on dystonia. He has received honoraria for lecturing from GlaxoSmithKline, UCB Pharma, Lundbeck, and Allergan. M. Hallett may accrue revenue on US Patent 6,780,413 B2: immunotoxin (MAB-Ricin) for the treatment of focal movement disorders; and US Patent 7,407,478: coil for magnetic stimulation and methods for using the same (Hcoil); in relation to the latter, he has received license fee payments from the NIH (from Brainsway) for licensing of this patent. Dr. Hallett's research at the NIH is largely supported by the NIH Intramural Program. Supplemental research funds were provided by Ariston Pharmaceutical Company via a Cooperative Research and Development Agreement with NIH for treatment studies of essential tremor, and the Kinetics Foundation, for studies of instrumental methods to monitor Parkinson disease, BCN Peptides, S.A., for treatment studies of blepharospasm, and Medtronics, Inc., for studies of deep brain stimulation, via Clinical Trials Agreements with NIH. H. Jinnah serves on the scientific advisory boards for the Dystonia Medical Research Foundation, Tyler's Hope for a Cure, the Lesch-Nyhan Syndrome Children's Research Foundation, and Lesch-Nyhan Action France. Dr. Jinnah has received grant support from the US NIH, The Bachmann-Strauss Dystonia and Parkinson's Foundation, Dystonia Medical Research Foundation, the Lesch-Nyhan Syndrome Children's Research Foundation, and Psyadon Inc. Dr. Jinnah is Director of the Dystonia Coalition, which receives the majority of its support through NIH grant NS065701 from the Office of Rare Diseases Research and National Institutes of Neurological Disorders and Stroke. The Dystonia Coalition receives additional material or administrative support from industry sponsors (Allergan Inc., Ipsen, Medtronic Inc., Merz Pharmaceuticals) as well as private foundations (the American Dystonia Society, The Bachmann-Strauss Dystonia and Parkinson's Foundation, the Benign Essential Blepharospasm Foundation, Dystonia Inc., Dystonia Ireland, the Dystonia Medical Research Foundation, the European Dystonia Federation, the Foundation for Dystonia Research, the National Spasmodic Dysphonia Association, the National Spasmodic Torticollis Association, and WeMove). A. Berardelli received grant support from the Italian Ministry of University, the Benign Essential Blepharospasm Research Foundation for research projects on dystonia, and from Boehringer Ingelheim, Lundbeck, UCB, Allergan, and Merz Pharmaceuticals. Go to Neurology.org for full disclosures.