Preliminary Findings of a Dedicated Ocular Myasthenia Gravis Rating Scale: The OMGRate

ABSTRACT

There is a pressing need for a robust rating scale for ocular myasthenia gravis (OMG). Rating scales for myasthenia gravis (MG) research have a predominant focus on generalised disease. We present results of the first dedicated rating scale for OMG: the ocular myasthenia gravis rating scale (OMGRate). The OMGRate was developed through an international collaboration between neuromuscular and neuro-ophthalmology experts in OMG. It comprises two components: a physician- examination (OMGRate-e) and a patient questionnaire (OMGRate-q).. The OMGRate was prospectively validated in patients attending a neuro-ophthalmology clinic from April 2017 to October 2018. External validity and reliability of OMGRate were evaluated using validated MG rating scales: the Myasthenia Gravis Composite (MGC), the Myasthenia Gravis Quality of Life (MG-QOL), and the ocular component from the Myasthenia Gravis Impairment Index questionnaire (MGII). Two hundred and eleven assessments were completed in 104 patients (67 males, mean age 55 y, range 18–86 y). There was very good external validity of the OMGRate: good correlation between OMGRate-e and MGC (r = 0.64, 95% confidence intervals [CI] 0.54–0.74, p < .0001); excellent correlation between OMGRate-q and MGII (r = 0.85, 95% CI 0.78–0.91, p < .0001) and good correlation between OMGRate and MG-QOL (r = 0.68, 95% CI 0.60–0.77, p < .0001). A higher correlation of OMGRate and MG-QOL compared with MGC and MG-QOL (r = 0.47, 95% CI 0.34–0.59, p < .0001) suggests that OMGRate is better able to capture significant QOL information in patients with OMG. It had excellent reliability with an intraclass correlation coefficient of 0.83 (95% CI 0.67–0.92). Feedback from examiners and patients indicated that the OMGRate was easy to use. In conclusion, OMGRate is an easy-to-use, valid and reliable rating scale for monitoring the severity of OMG.

Background

Robust methodology to assess the severity of ocular myasthenia gravis (OMG) symptoms is important for monitoring disease and treatment. Current rating scales recommended for myasthenia gravis (MG) research have a predominant focus on generalised disease,^1,2 and are insufficiently sensitive for OMG.^3-5 Our previous work found that the examiner-based rating scales of the Myasthenia Gravis Composite scale¹ and the Ocular-QMG scale⁶ were insufficiently sensitive for OMG.³ We also found the patient questionnaires of the Myasthenia Gravis Quality of Life (MG-QOL), the National Eye Institute Visual Function (VFQ-25) and the Neuro-ophthalmic supplement of the VFQ-25 (NO-VFQ25) insufficient for assessing severity of OMG.³ More recently, the Myasthenia Gravis Impairment Index (MGII) has been developed as an improved rating scale for MG research.^7,8 The MGII does include an ocular sub-score that demonstrated validity, reliability and responsiveness. However, as the MGII was designed primarily for generalised MG it has limited ocular examination items.

We present a dedicated rating scale for ocular myasthenia gravis (OMG), developed through an international collaboration between neuromuscular and neuro-ophthalmology experts in OMG. We aimed to create a concise and user-friendly rating scale for OMG, to better assess the fluctuations and severity of signs and symptoms caused by OMG. We assessed this new rating scale’s usability and inter-rater reliability.

Method

For this study, patients with OMG were defined as patients who experience ocular only myasthenia symptoms without current or prior myasthenic symptoms in the limbs, bulbar or respiratory muscles. The diagnosis of OMG was based on: the presence of variability of symptoms or signs; the presence of one or more ocular muscle involvement; supported by either a positive antibodies against the acetylcholine receptor (AChR) or muscle-specific kinase (MuSK) or low-density lipoprotein receptor 4 (LRP4) antibodies; supportive changes on neurophysiology; positive ice test or positive response to anti-cholinesterase (pyridostigmine) test. Patients provided written informed consent to be part of this observational study, approved by the London Bridge Research Ethics Committee (ref 15/LO/0943).

Development of rating scale

We developed the ocular myasthenia gravis rating scale (OMGRate) in collaboration, combining our clinical expertise in managing numerous patients with varying severity of OMG (SHW, EE, WC), in MG and in developing rating scales for MG (CB). This was developed through an iterative process: items on the rating scale were selected based on expert opinion (SHW, EE, WC); tested by SHW before implementation for this study; improvements made upon further discussions (SHW, EE, WC), e.g. on wording of the instructions and ratings were performed with the patient wearing spectacles or prisms, if applicable. This was to better reflect the level of function and disability experienced by the patients who would use their spectacles or prisms at home.

The OMGRate has two components (see Appendix). The physician-rated component (OMGRate-e) assesses diplopia and ptosis. Binocular diplopia is assessed using a target equivalent to 20/400 of the Jaeger Near Card (or Arial Bold font size 45), held at 1 m in front of the patient. The presence of binocular diplopia is recorded in nine positions of gaze held for up to 3 s. All positions apart from the primary position (looking straight ahead) are held at approximately 45 degrees from the primary position. If diplopia is present, a cover test is performed to confirm if the diplopia is binocular. Ptosis is rated after 30 s of sustained upgaze with the brow stabilised on the brow ridge to prevent frontalis overaction (see Figure 1). The upper lid margin is then rated in relation to the pupil, i.e. above, crosses, or below the pupil. The presence of amblyopia or factors that affect binocularity, and any potential non-MG ptosis are indicated, e.g. mechanical ptosis, is indicated on the form. Scores are weighted for diplopia in the primary position and down-gaze to reflect impact on daily function.

Figure 1.

Ptosis is rated after 30 s of sustained up gaze with the brow stabilised on the brow ridge to prevent frontalis overaction.

The patient-rated component (OMGRate-q) has items chosen based upon expert knowledge of OMG, with clinical experience in managing OMG symptoms of varying severity, taking into account feedback from patients during consultations about the most troublesome aspects of OMG. OMGRate-q incorporates relevant questions from validated questionnaires: question number 10 about ptosis from the Neuro-ophthalmic supplement of the NEI-VFQ-25 (NO-NEI-VFQ25)⁹; question number 11 about monocular occlusion and question no.14 about depth perception from the Adult Strabismus Quality of Life Questionnaire (AS-20)¹⁰ and all eight questions from the Diplopia Questionnaire used in strabismus research.¹¹ Scores are weighted for diplopia when reading, looking straight ahead and looking down, to reflect the impact on daily function.

The Likert-style response order for all questions was streamlined, from low to high rating (e.g. never; rarely; sometimes; often; always). This resulted in a change of order from the originally published questions from the Diplopia Questionnaire and the NO-NEI-VFQ25 question on ptosis. For consistency, the wording of options was also standardised to “never; rarely; sometimes; often; always”. The duration of symptoms was assessed over two weeks, in line with previous work done for the MGII⁸ showing that this is a useful timescale to capture the fluctuations of symptoms in MG. The OMGRate-q had 11 questions, with total possible score ranging from 0 to 53, and higher scores indicate worse OMG symptoms.

Data collection

We collected the OMGRate scores prospectively in patients with confirmed OMG seen in a single neuro-ophthalmology clinic (SW) from April 2017 to Oct 2018.

We also assessed patients with other validated disease-specific severity measures: the MGC and the ocular questions of the MGII patient questionnaire.⁸ The MGC has 10 questions with total possible scores ranging from 0 (no symptoms) to 50 (maximum severity). All aspects of the MGC were tested in this study, although the study design only included OMG patients. Therefore, by definition the maximum possible MGC score was nine for OMG patients, i.e. scoring only for symptoms ocular questions and zero for the generalised MG questions. The MGII ocular questionnaire has 6 questions, and total scores range between 0 and 18 points, with higher scores indicating worse ocular MG symptoms.

Patients also completed the Myasthenia Gravis Quality of Life (MG-QOL),^12-14 a 15-item disease specific QoL measure, with scores ranging from 0 to 60, and higher scores indicate worse QoL. The MGII data collection started in December 2017. Patients who had difficulty with language or literacy were provided assistance to complete patient questionnaires. We aimed to collect data in more than 100 patients in accordance with the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) group recommendations for validation of health measures (https://www.cosmin.nl/tools/guideline-selecting-proms-cos/).

Reliability

Inter-rater reliability of the examination component was assessed via sequential application of the OMGRate-e by two blinded raters separated by at least 15 min, to allow patient rest and prevent fatigue from affecting the results. For individual item reliability, we estimated weighted kappa values. For the sum-scores we used intra-class correlation coefficients (ICC), considering the raters as random effects (ICC 2,1). Reliability was tested for OMGRate-e and not for OMGRate-q.

Construct validity

For construct validity, the OMGRate-q scores were compared with the responses to the ocular questions of the MGII, using correlation coefficients to test the relationship between the new scale and other validated measures. OMGRate-e scores were compared with the MGC. We expected high, positive correlations (r > 0.6) between the OMGRate scores and MGC and MGII(ocular) scores. We expected moderate correlations (r = 0.4–0.6) between the OMGRate scores and MG-QoL scores. Considering a minimal expected correlation of 0.4, with alpha 0.05 and 80% power, a minimum of 40 patients were needed.

Refinement of the scale was planned based on initial results.

Results

A total of 211 assessments were completed in 104 patients (64% males, mean age 55 y, range 18–86 y, anti-AChR+ 27%, anti-MUSK+ 2%, anti-LRP4 + 18%, seronegative 53%). Of the 55 seronegative patients: 17 had EMG changes in keeping with MG; 27 had positive response to pyridostigmine challenge; four had positive ice test for ptosis and seven were diagnosed based on clinical signs and symptoms of variability characteristic for MG.

All patients had OMGRate-e and MGC ratings completed. One hundred and ninety-seven assessments included MG-QOL, 188 had additional OMGRate-q and 111 assessments included MGII-ocular questions. See Figure 2 for CONSORT flow diagram.

Figure 2.

CONSORT flow diagram of the number of assessments completed.

The median score for OMGRate-e was 1 (range 0–16, maximum possible score 17), while the median OMGRate-q was 7 (range 0–50; maximum possible score 53). The median for OMGRate, i.e. a sum of OMGRate-e and OMGRate-q, was 8 (range 1–62, maximum possible score of 70); for MGC was 1 (range 0–5, maximum possible score 9 for ocular aspects or 50 for full MGC score); for MGII (ocular questions) was 4 (range 0–18, maximum possible score 18) and for MG-QOL was 5 (range 0–58; maximum possible score 60). The distribution of scores is shown in Figure 3.

Figure 3.

Distribution of scores for (a) MGC, (b) MG-QOL, (c) OMGRate-e, (d) OMGRate-q and (e) MGII (ocular questions).

The examiner component of OMGRate (OMGRate-e) and the patient-rated component (OMGRate-q) demonstrated a good correlation coefficient of 0.66 (95% confidence intervals [CI] 0.58–0.75, p < .0001).

The correlation between OMGRate-e & MGC was also strong (r = 0.64, 95% CI 0.54–0.74, p < .0001). TheOMGRate-q and the ocular questions of the MGII had a high positive correlation (r = 0.85, 95% CI 0.78–0.91, p < .0001). Finally, the correlation coefficient between the total OMGRate and MG-QOL scores was 0.68 (95% CI 0.60–0.77, p < .0001). Overall all correlations indicated good construct validity.

The OMGRate outperformed the MGC correlation with MG-QOL: the correlation coefficient between MGC & MG-QOL was 0.47 (CI: 0.34, 0.59, p < .0001), compared with 0.68 (95% CI 0.60–0.77, p < .0001) between the OMGRate and MG-QOL. The MGII(ocular) performed similarly to the OMGRate, with a high correlation coefficient (r = 0.74, 95%CI 0.64–0.85, p < .0001).

See Figures 4 and 5 for the above correlation graphs.

Figure 4.

Correlation graphs for OMGRate, showing good validity. (a) Correlation between OMGrate-e and OMGrate-q. (b) Correlation between OMGRate-e and MGC scores. (c) Correlation between OMGRate-q & MG-QOL scores. (d) Correlation between OMGRate-q & MGII(eye) scores.

Figure 5.

Comparison of correlations with MG-QOL for OMGRate (a) (b) and MGC (c), showing better correlations of OMGRate with MG-QOL scores.

Inter-observer reliability was completed in 28 patients, each assessed by two raters (SHW & either AX, EM, HL or AB). The total OMGRate-e score ICC was excellent at 0.82 (95% CI 0.67–0.92). The weighted kappa values and percentage inter-rater agreement for individual item reliability of diplopia positions are as listed in Appendix B (online supplement). Percentage inter-rater agreement was excellent at 82–93% for all positions, apart from right-up and left-up gaze, with percentage inter-rater agreements of ≤75%.

Refinement of the OMGRate-e

A review of the results and reflections of the practical applications of the OMGRate-e during further investigator discussions (SHW, CB, EE, WC) led to recommendations on the next iteration of the OMGRate-e. For simplicity and improvement of reliability, we removed the right-up left-up right-down and left-down gaze positions from the examination score. We also changed the scoring of diplopia at primary position to be counted if present either at the start or at the end instead of both. This reduced the total score of OMGRate-e to a maximum of 10.

To assess if this simplification was effective, correlations of this simplified OMGRate-e were calculated. This showed correlation coefficients with MGC of 0.62 (95% CI 0.52–0.72, p < .0001) and with OMGRate-q of 0.65 (95% CI 0.56–0.74, p < .0001). The inter-rater reliability remained excellent, with an ICC of 0.81 (95% CI 0.57–0.91).

Discussion

OMG causes ptosis and diplopia, which can significantly impact a patient’s daily function and quality of life. An ideal rating scale would include both physician- and patient-rated components. Due to the inherent variability of symptoms and signs of OMG, sole reliance on physician examination is insufficient to fully capture the severity of the condition. We present a dedicated rating scale for OMG and show that this first iteration of the OMGRate is a promising tool for assessing the severity of OMG.

With regards to content validity, the OMGRate-q builds on extensive work on patient-reported outcome measures in strabismus research.^10,11 The scoring for the Diplopia Questionnaire¹¹ was modified with weighting for diplopia when reading, primary position and down gaze. This was to reflect the impact on daily function as reported by our patients and from our clinical observations. The OMGRate-e was designed to reflect the relevant MG signs we considered problematic for daily function based on clinical experience. Weighting was given to diplopia in the primary position and on down gaze, with the latter being relevant for reading, eating, walking and using stairs. The other questions about ptosis, monocular occlusion to see better and depth perception were derived from validated questionnaires (NEI-VFQ-25 and AS-20). The OMGRate-e and OMGRate-q had a good correlation indicating that the examination reflects the severity of patient symptoms over a two-week period.

In terms of construct validity, our pre-defined hypotheses were met, with a moderate to high correlation between the OMGRate-q and the MGC and MGII-ocular questionnaire. In this study, the OMGRate had a better correlation with MG-QOL than the MGC, which could indicate that the OMGRate is a better indicator for the impact of OMG on the patient’s quality of life. Additionally, the median and range of scores of MGC compared with MGII(ocular) and OMGRate-q suggests that the latter two had less floor effect compared with the MGC, and therefore may be a more sensitive tools for OMG.

In this study, the OMGRate-e was easy to use, taking approximately 1 min of examination time, which makes it practical in a busy clinical setting. In this study, the OMGRate-e was tested by ophthalmologists or neuro-ophthalmology trainees; therefore, wider testing of its usability among neurologists will be part of the next phase of a larger multi-centre study. The OMGRate-q was also easy to use. The time taken to complete this was not collated systematically as part of the study design but was observed to take approximately 2 min for patient completion, as it was completed by the patient in the consultation room.

During the use of the OMGRate-q, patients voiced a preference for the Likert-style questions on the OMGRate-q compared with the MGII questions, which rated severity according to time of day or duration of activity. For example, some patients had constant diplopia on right gaze, which lead to a high score for severity according to timing, but the diplopia was not considered problematic as it spared the primary position. For the next iteration of the OMGRate-q, we will explore the addition of a question regarding the patient’s perceived extent of disability affecting their daily life related to their ocular myasthenia symptoms, to further improve on this as a patient-reported outcome measure. Further qualitative work on the OMGRate-q with semi-structured patient interviews is also planned as part of the next study phase.

The overall inter-rater reliability was good. The weakest inter-rater correlations were found in the right-up and left-up positions. This supports the simplification of the OMGRate-e by removing examination of diplopia at right-up gaze, left-up gaze, right-down gaze and left-down gaze positions. Reducing the number of items scored would improve inter-rater reliability; hence, the suggestion to score diplopia at primary position only once: when present either at start or the end of the examination, not both. The correlation scores with MGC and OMGRate-q with this reduced scoring for diplopia remained excellent, as was inter-rater reliability. A limitation of our reliability study is the small number of patients.

Patients with amblyopia and large deviation of ocular alignment may not experience troublesome diplopia and therefore their severity of signs may not be captured on the OMGRate-e. However, for these patients diplopia may not be as problematic, and therefore their score on the OMGRate-e will be a reflection of their symptomatic observations.

The OMGRate-e was scored with glasses and prisms (if habitually worn), to reflect the effect of any treatment given. Patients on pyridostigmine treatment were also examined on treatment, although the timing of pyridostigmine from time of OMGRate-e was not standardised. For the next multi-centre study, we will explore standardised timing intervals regarding pyridostigmine dosing.

The OMGRate will need to be validated in other populations. One of the limitations of this study is the predominance of patients with low scores on OMGRate-e and OMGRate-q, suggesting a milder disease cohort. This floor effect will be addressed with a larger multi-centre study capturing a wider range of disease severity. Another limitation of this study cohort is that OMG diagnosis in 27/107 was based on positive response to pyridostigmine challenge and 7/107 were diagnosed based on clinical signs and symptoms of variability characteristic for MG. This has benefits of translating OMGRate use in seronegative OMG patients but could raise the concern of false-positive cases being included in the study.

The characteristics of an ideal OMG rating scale include ease of application and reliability when applied by a wide range of health-care providers. This will be assessed in the next phase of the study, with inter-rater reliability studies to include technicians, orthoptists, and trainees in ophthalmology and neurology.

It is an important reminder that this rating scale is designed to monitor severity of ocular symptoms only. It is not designed to pick up insidious generalisation of disease, which should be routinely enquired through careful history and examination as appropriate.

Further work is needed to assess the test–retest reliability of the OMGRate, which will involve retesting stable patients over time. Studies will also be needed to assess the effectiveness of OMGRate to detect change following treatment, through longitudinal studies. Scoring the OMGRate-e before and after pyridostigmine may also be useful to assess whether the OMGRate-e can detect change. Another aspect we are interested to asses in the next phase of the study, is whether the OMGRate-q is sufficient as a standalone outcome measure, i.e. if it better-reflects the impact of OMG symptoms, particularly as the examination may be done during the day and symptoms that impact daily function may occurs later in the day, outside of the consultation setting.

In summary, we have demonstrated excellent construct validity inter-rater reliability of the OMGRate. Patient and examiner feedback has also been positive with regards to its usability. Further work is planned with a larger, multi-centre study to further validate and refine the second iteration of the OMGRate.

Acknowledgments

SHW thanks Drs Jonathan Holmes, Ted Burns, Lanning Kline, and Gordon Plant for correspondence and support; Myaware for supporting the prospective observational cohort of OMG patients in Moorfields Eye Hospital; Dr L.Dell’Arti for helpful feedback on the OMGRate-e; and patients for their participation and input. Preliminary results of this study were presented at the Association of British Neurologists annual meeting 2018; Muscle Meeting Group annual scientific meeting 2018; North American Neuro-ophthalmology Society annual meeting 2019.

Authorship contribution

Study design, data interpretation and write up of study: SHW, EE, WC, CB

Data collection, analysis, and first draft of paper: SHW

Data analysis: SHW, CB

Data entry: SHW, AXE, EM

Data collection for inter-rater reliability: AXE, EM, HL, AB

Declaration of interest

Dr Carolina Barnett has served as a consultant for UCB, CSL, Alexion, and Akcea. She has received research support from Octapharma and Grifols. She is also the primary developer of the MGII and might receive royalties. All other authors have no interests to declare.

Data availability statement

Anonymous data will be shared by request from a qualified investigator.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website.