Test-retest variability of multifocal electroretinography in normal volunteers and short-term variability in hydroxychloroquine users

Abstract

Purpose

To determine measurement variability of N₁P₁ amplitudes and the R₁/R₂ ratio in normal subjects and hydroxychloroquine users without retinopathy.

Design

Retrospective, observational study.

Subjects

Normal subjects (n=21) and 44 patients taking hydroxychloroquine (n=44) without retinopathy.

Methods

Multifocal electroretinography (mfERG) was performed twice in one session in the 21 normal subjects and twice within 1 year in the hydroxychloroquine users, during which time no clinical change in macular status occurred.

Main outcome measures

N₁P₁ amplitudes of rings R₁–R₅, the R₁/R₂ ratio, and coefficients of repeatability (COR) for these measurements.

Results

Values for N₁P₁ amplitudes in hydroxychloroquine users were reduced compared with normal subjects by the known effect of age, but R₁/R₂ was not affected by age. The COR for R₁–R₅ ranged from 43% to 52% for normal subjects and from 43% to 59% for hydroxychloroquine users; for R₁/R₂ the COR was 29% in normal subjects and 45% in hydroxychloroquine users.

Conclusion

mfERG measurements show high test-retest variability, limiting the ability of a single mfERG test to influence a decision to stop hydroxychloroquine; corroborative evidence with a different ancillary test is recommended in a suspicious case.

Introduction

In 2011, the American Academy of Ophthalmology (AAO) published revised guidelines for screening hydroxychloroquine retinopathy.¹ These recommended the use of what were termed objective ancillary tests, including multifocal electroretinography (mfERG), when available.¹ Screening for retinopathy occurs over a period of years, typically with tests taken multiple times and interpretations compared longitudinally. In this situation, it is necessary to quantitate both measurement noise and short-term variation of the measurements in order to be able to recognize true changes that represent disease onset and progression. In this article, test-retest variability (measurement noise) in normal volunteers and short-term variation in hydroxychloroquine users without retinopathy were assessed.

A clinical case will make the issues concrete and motivate the need for the study.

Case report

A 64-year-old woman with rheumatoid arthritis was placed on hydroxychloroquine in 2007. She was 170 cm and weighed 84 kg. Her ideal body weight according to the National Heart Lung and Blood Institute table was 69 kg.^2,3 She took hydroxychloroquine 400 mg/day for 6 years. Her best corrected visual acuity was 20/25 in each eye, secondary to early nuclear sclerotic cataracts. Her maculas at baseline were normal. She had no renal or liver disease.

Yearly 10-2 visual field (VF) testing with a III, red test object was normal for 6 consecutive years (Figure 1). When the AAO revised guidelines were published, mfERG testing was began. The results are shown for 3 consecutive years (Figure 2). Because of reductions in N₁P₁ amplitudes noted in 2012, the question arose as to whether she had toxicity and needed to be taken off hydroxychloroquine.

Figure 1.

10-2 visual fields over six years.

Abbreviation: FP, false positives.

Figure 2.

mfERGS for three consecutive years.

Notes: (A) Study of 15 August, 2011; (B) study of 20 August, 2012; (C), study of 19th August, 2013.

Methods

This was a retrospective study of 21 normal subjects and 44 patients taking hydroxychloroquine, who were free of retinopathy, in a private multispecialty practice with 26 ophthalmologists and three optometrists. The 21 normal subjects were tested twice in a single session. mfERGs were recorded twice within 1 year from the 44 patients taking hydroxychloroquine. These patients had no change in their clinical condition between the first and second measurements and were not receiving toxic daily dosing. The two measurements provided data on short-term variation in the mfERG.

The radius of the stimulus zone was 25 degrees. The variables examined were the R₁–R₅ ring N₁P₁ amplitudes, the R₁/R₂ ring ratio, the R₅/R₄ ring ratio, and the R₅/R₃ ring ratio. One eye for each subject was chosen for analysis. For each index, the sample mean for the combination of both trials was used to normalize the measurements. The normalized measurements were analyzed via the method of Bland and Altman to calculate the coefficient of repeatability (COR), defined as 1.96 times the standard deviation of the differences between the paired measurements.⁴ The probability is 95% that a change in an index that exceeds the COR can be judged to be real, and not random variation in the measurement (noise).⁴

To avoid problems of correlated results between eyes, only one eye was included per patient.⁵ When only one of two eyes had good-quality testing (no 60-cycle noise and no evidence of eccentric fixation), that eye was chosen. When two eyes had good-quality testing, a random number generator was used to pick which of the two was included.

mfERGs were performed following International Society for Clinical Electrophysiology of Vision (ISCEV) guidelines as previously described.^6,7 The mfERGs were performed with the Espion system (Diagnosys LLC, Lowell, MA, USA) running under version 6+ software. DTL (Dawson, Trick, and Litzkow) fiber electrodes were used. The patients’ eyes were dilated and topical anesthetic was used. The stimulation pattern was dictated by an m-sequence controlling the illumination of 61 contiguous hexagons subtending 30 degrees of VF to either side of fixation. The luminances of the white and black hexagons were 1,000 cd/m² and 0 cd/m², respectively. Signals were processed through a 10–100 hertz bandpass filter. The first order kernel response was analyzed. The waveform amplitudes refer to the response density measured from the trough of the N₁ wave to the peak of the P₁ wave. The displays are shown in the retinal view (as though looking at a fundus photograph, not as though looking at a VF).

Waiver of informed consent and waiver of Health Insurance Portability and Accountability Act (HIPAA) authorization were approved by the Presbyterian Hospital Institutional Review Board.

Results

The demographic characteristics of the normal subjects and patients taking hydroxychloroquine without retinopathy are shown in Table 1. The hydroxychloroquine users were older than the normal subjects (median age 62 and 38 years, respectively). The mfERG amplitudes from the normal subjects and hydroxychloroquine users are shown in Table 2. The amplitudes were lower in the hydroxychloroquine users, consistent with their increased age relative to the normal subjects.⁸ The test-retest variability in the normal subjects is shown in Table 3. Neither fixed nor proportional bias was manifest in the test-retest measurements (Figure 3). The COR was 16.1 nV/deg² and 7.2 nV/deg² for the averaged measurements of rings R₁ and R₂, respectively, the rings most closely assessed in evaluating hydroxychloroquine retinopathy. The COR for the R₁/R₂ ratio was 0.54. Expressing the measurements in relative terms, the CORs for R₁, R₂ and the R₁/R₂ ratio were 52%, 43%, and 29%, respectively. The smallest CORs were found for R₅/R₄ and R₅/R₃ (Table 3).

Table 1.

Demographic characteristics of subjects

Characteristic	Controls	HC users
Age (years)	38 (28–55)	62 (54–72)
Women:men	19:2	41:3
Height (cm)	NA	165 (160–168), n=41; not checked in 3
Weight (kg)	NA	77 (64–88), n=39; not checked in 5
Daily dose (mg/day)	NA	400 (n=22); 200 (n=14); 300 (n=6); not checked in 2
Cumulative dose (grams)	NA	706 (406–1,066); n=32; not checked in 12

Note: Data are presented as median (interquartile range) unless otherwise indicated.

Abbreviations: HC, hydroxychloroquine; NA, not applicable.

Table 2.

Values of multifocal electroretinogram measurements in normal subjects and hydroxychloroquine users without retinopathy

Variable	Units	Normal subjects (n=21)	HC users (n=44)
R1	nV/deg2	31.4±7.9	26.9±5.7
R2	nV/deg2	17.0±4.5	14.8±3.3
R3	nV/deg2	10.2±2.3	9.0±2.5
R4	nV/deg2	7.5±1.8	6.7±2.0
R5	nV/deg2	6.4±1.5	5.9±1.8
R1/R2	No unit	1.84±0.15	1.83±0.23
R5/R4	No unit	0.87±0.07	0.89±0.08
R5/R3	No unit	0.64±0.08	0.66±0.09

Notes: R₁ to R₅ refer to ring averages of N₁P₁ amplitudes from the multifocal electroretinogram. R₁/R₂, R₅/R₄, and R₅/R₃ refer to ratios of these ring averages. Data are presented as means ± standard deviations.

Abbreviations: deg, degree; HC, hydroxychloroquine; nV, nanovolts.

Table 3.

Test-retest variability of multifocal electroretinogram measurements in normal subjects (n=21)

Measurement	Units	Trial #1 (mean ± SD)	Trial #2 (mean ± SD)	COR, actual values	COR, normalized values, %
R1	nV/deg2	32.4±8.5	30.4±9.3	16.1	52
R2	nV/deg2	17.6±4.3	16.4±4.5	7.2	43
R3	nV/deg2	10.5±2.6	9.8±2.5	4.3	43
R4	nV/deg2	7.8±1.9	7.2±2.0	3.2	43
R5	nV/deg2	6.7±1.6	6.2±1.8	2.9	46
R1/R2	No unit	1.83±0.19	1.85±0.21	0.54	29
R5/R4	No unit	0.87±0.07	0.86±0.06	0.11	13
R5/R3	No unit	0.65±0.08	0.63±0.09	0.14	22

Notes: R₁ to R₅ refer to ring averages of N₁P₁ amplitudes from the multifocal electroretinogram. R₁/R₂, R₅/R₄, and R₅/R₃ refer to ratios of these ring averages.

Abbreviations: COR, coefficient of repeatability; deg, degree; nV, nanovolts; SD, standard deviation.

Figure 3.

Bland–Altman analysis of repeatability for normal subjects and hydroxychloroquine users.

Abbreviations: SD, standard deviation; HC, hydroxychloroquine.

The short-term variability in hydroxychloroquine users is shown in Table 4. The magnitudes of the measurement differences from tests separated by months were no greater than the measurement differences in tests performed minutes apart in normal subjects. The COR extracted from these measurements was 16.0 nV/deg² and 7.6 nV/deg² for the averaged measurements of rings R₁ and R₂, respectively. The COR for the R₁/R₂ ratio in hydroxychloroquine users was 0.89. Expressing the measurements in relative terms, the CORs for R₁, R₂, and the R₁/R₂ ratio were 59%, 53%, and 45%, respectively. The smallest CORs were found for R₅/R₄ and R₅/R₃ (Table 4).

Table 4.

Short-term variation of multifocal electroretinogram measurements in patients taking hydroxychloroquine without retinopathy (n=44)

Measurement	Units	Trial #1 (mean ± SD)	Trial #2 (mean ± SD)	COR, actual values	COR, normalized values, %
R1	nV/deg2	26.5±7.0	27.3±6.9	16.0	59
R2	nV/deg2	14.4±3.9	15.3±3.9	7.6	53
R3	nV/deg2	8.6±2.6	9.3±2.8	4.2	47
R4	nV/deg2	6.5±2.1	6.9±2.3	3.2	49
R5	nV/deg2	5.9±1.9	6.0±2.0	3.0	55
R1/R2	No unit	1.88±0.36	1.80±0.25	0.89	45
R5/R4	No unit	0.91±0.09	0.86±0.07	0.19	21
R5/R3	No unit	0.69±0.12	0.64±0.09	0.24	36

Notes: R₁ to R₅ refer to ring averages from the multifocal electroretinogram. R₁/R₂, R₅/R₄, and R₅/R₃ refer to ratios of these ring averages.

Abbreviations: COR, coefficient of repeatability; deg, degree; nV, nanovolts; SD, standard deviation.

Discussion

mfERG measurements suffer from high inter-test variability, in part due to the technical skills of the person administering the test.^9–14 This limitation is not appreciated by many clinicians who have been advised to use the test in hydroxychloroquine screening.¹ In the case report, for the right eye, R₁ in the third study is decreased by 41% compared with baseline, which is less than the 59% COR for R₁ in hydroxychloroquine users. Therefore, this change probably reflects measurement variability. Similar comments apply to the changes observed in R₂ for the right eye and both R₁ and R₂ in the left eye. R₁/R₂ is not abnormal for either eye in any of the three studies.

Some authors place diagnostic weight on individual hexagonal waveforms of the mfERG. There is no evidence that the variability of individual hexagonal waveforms is less than for ring averages and ratios.^10,13

The overall clinical picture carries more weight than isolated mfERG testing. For example, the patient in the case report had been on hydroxcyhloroquine <6.0 mg/kg/day based on ideal body weight and had consistently normal 10-2 VFs. There was no need to stop hydroxychloroquine nor to reduce the daily dose despite the decrease in the mfERG amplitudes in the 8-19-2013 study.

The short-term variability of mfERG measurements in patients taking hydroxychloroquine was similar to the test-retest variability in normal subjects, as has been reported by other investigators.¹⁴ Some have reported that variability of N₁P₁ amplitudes increases in the outer rings, but our measurements did not show this. The variability of the R₁/R₂ ratio was not better than that of N₁P₁ amplitudes to a clinically important extent. The usefulness of mfERG measurements in screening for hydroxychloroquine retinopathy is limited by their variability compared with the low variability of spectral domain optical coherence tomography (SD-OCT) images and thickness measurements, which can also be used for this purpose.¹⁵

Certain mfERG variables are age dependent.¹⁶ The response densities of individual hexagons can decrease from 5% to 12% per decade of age.¹⁷ Ring amplitudes decrease with age at approximately 11% per decade.^16,18 Both N₁ and P₁ implicit times decrease at approximately 1% per decade.¹⁶ This adds an additional level of complexity to mfERG analysis of patients taking hydroxychloroquine, because patients taking hydroxychloroquine are often older than the normal population from which tables of normal mfERG values are derived. One method to circumvent this difficulty is to emphasize ring ratios, some of which are invariant with respect to age. For example, R₁/R₂, R₁/R₃, and R₁/R₄ have been reported to be invariant with respect to age.¹⁸ In particular, R₁/R₂ has been reported as particularly useful in assessing hydroxychloroquine toxicity, with 2.6 used as a cut point for classification of responses as abnormal or not.⁸ Another advantage cited for ring ratio analysis has been better reproducibility than amplitude measurements.¹⁸ The data herein do not support this notion. CORs for R₁/R₂ ratios in this study were no better than for N₁P₁ amplitudes. CORs for R₅/R₄ and R₅/R₃ were smaller, but these indices have only been used in one report to date.²⁴

Because of the variability of the mfERGs, several tests should usually be conducted before deciding that hydroxychloroquine should be stopped, if other tests have not clarified the patient’s status. Otherwise, one might stop a useful drug based on a decrease in mfERG amplitude that in reality reflected only intersession variability of the measurements.^19–22 In fact, some investigators think that the sole purpose of the mfERG is to raise suspicion, with other tests necessary to confirm or disconfirm the presence of retinopathy.^20,23 At the other end of the spectrum of hydroxychloroquine retinopathy, the value of mfERG in more advanced cases has been questioned; if a more reliable test such as SD-OCT shows retinopathy, mfERG adds little.^21,22

The limitations of this study include relatively small numbers of normal subjects and patients taking hydroxychloroquine. In addition, mfERG results vary across instruments, laboratories, and testing conditions, making it difficult to generalize to other settings.

Conclusion

These results provide estimates for variability in mfERG testing among patients being screened for hydroxychloroquine retinopathy. Caution against over-reliance on the results of a single mfERG test is warranted. Corroborative testing with other modalities such as the 10-2 VF and SD-OCT are indicated before stopping therapy based on an abnormal mfERG test.