Effect of Operator and Optical Defocus on the Variability of Pattern Electroretinogram Optimized for Glaucoma Detection (PERGLA)

Abstract

Purpose

To evaluate the effect of operator and optical defocus on the variability of Pattern Electroretinogram Optimized for Glaucoma Detection (PERGLA).

Methods

Two different operators obtained two PERGLA recordings each from 10 healthy participants (5 women, mean age 32.1±10.3 years). In addition, one of the operators obtained recordings in which corrective lenses of various diopters (±0.5, ±1, ±2, ±3) were used to generate optical defocus in both eyes. The effect of operator on PERGLA amplitude and phase variability was determined using a single nested variance components' analysis model and by using Bland-Altman plots. One-way analysis of variance (ANOVA) was used to determine the effect of optical defocus on amplitude and phase.

Results

Differences in measurements between operators accounted for approximately 26.6% and 18.2% of the total variance for amplitude and phase, respectively. Results were confirmed by the use of Bland-Altman plots. ANOVA identified a significant effect of defocus on mean amplitude (F= 2.65, p=0.01), but not phase (F=1.02, p=0.42).

Conclusions

Measurements obtained by different operators can result in significant differences in PERGLA amplitude. In addition, although optical defocus leads to a decrease in PERGLA amplitude by reducing visual acuity, this can be avoided by obtaining J1 or better vision before testing.

Introduction

When implementing a new diagnostic instrument for use in clinical practice, its reliability and accuracy must be determined. Ultimately, the device must successfully discriminate between diseased and non-diseased individuals, thus decreasing the level of diagnostic uncertainty for the examining clinician. Recently, a pattern electroretinogram (PERG) paradigm designed specifically for glaucoma detection (PERGLA Glaid, Lace Elettronica, Pisa, Italy) has been introduced that compares PERG testing results to an internal normative database to help clinicians identify glaucomatous functional damage.^1,2 Although several studies have shown that PERGLA measurements are repeatable in healthy and glaucoma eyes,^1,3,4 some factors affecting the long-term variability of measurements have not been systematically evaluated. It is important to evaluate long-term variability components because these components could bias the detection of glaucomatous change over time using the PERGLA instrument.

Long-term variability of PERGLA measurements likely is affected by recording methods. First, the instrument relies on the operator to correctly position skin electrodes prior to each recording to acquire responses from both eyes simultaneously.⁵ Second, changes in appropriate correction for viewing distance over time must be compensated (near Jaeger acuity of approximately J1 or better is required to produce acceptable PERGLA signal strength).^1,2 These potential variance components deserve systematic evaluation. This study was designed to evaluate the effect of electrode placement by two different operators and to investigate the effect of uncorrected viewing distance (accomplished by optical defocus) on the variability of PERGLA measurements.

Methods

Subjects

PERG recordings from 10 healthy participants (20 eyes) (5 women, mean age 32.1±10.3 years) were evaluated at the Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego (UCSD). The UCSD Institutional Review Board approved all methodology and all methods adhered to the Declaration of Helsinki for research involving human participants and the Health Insurance Portability and Accountability Act (HIPAA).

Each study participant underwent a comprehensive ophthalmologic evaluation including medical history review, automatic refraction using the Zeiss Humphrey autorefractor (Carl Zeiss Meditec, Dublin, CA, USA), best-corrected visual acuity testing, slit-lamp biomicroscopy, intraocular pressure (IOP) measurement with Goldmann applanation tonometry, gonioscopy, stereoscopic slit lamp fundus examination with a 78 diopter lens, and reliable (false positives, fixation losses and false negatives ≤ 33% with no observable testing artifacts) standard automated perimetry using the 24-2 Swedish Interactive Thresholding Algorithm (Humphrey visual field analyzer, Carl Zeiss Meditec, Dublin, CA, USA). To be included in the study, participants had to have a best-corrected visual acuity better than or equal to 20/40, spherical refraction within ± 5.0D, cylinder correction within ± 3.0D, open angles on gonioscopy, IOPs of 22 mmHg or less with no history of increased IOP, and a normal visual field result in both eyes. A normal visual field result was defined as a pattern standard deviation within the 95% normal confidence limits and a Glaucoma Hemifield Test results within normal limits. All eyes had a healthy fundus examination including healthy appearance of the optic disc and retinal nerve fiber layer.

PERG Testing

A commercially available modification of the Glaid (Lace Elettronica, Pisa, Italy, software version 2.1.14) electrophysiology instrument, called PERGLA was used to measure the PERG response.^1,2,5 The PERGLA stimulus is a black and white (contrast 98%, mean luminance 40 cd/m²), horizontal square wave grating (1.6 c/deg), counter-phasing at 8.14 Hz, presented on a computer monitor (14.1 cm diameter circular field). At a viewing distance of 30 cm, the display subtends 25 deg centered on the fovea (assuming fixation towards a prominent central fixation circle). Responses from both eyes are measured simultaneously. Electrical signals from silver-chloride skin electrodes (9 mm adhered with conductive cream and tape) (both lower eyelids active, both temples reference, forehead ground) are fed into a two-channel differential amplifier, amplified (100,000 fold), filtered (1-30 Hz), then digitized with 12-bit resolution at 4169 Hz. Before testing, the electrode impedance is monitored automatically and an on-screen indicator signals acceptable impedance (≤ 5 kΩ). Additionally, an on-screen oscilloscope displays background noise.

The PERGLA software obtains each waveform by averaging 600 artifact-free time-periods of 122.8 msec each, synchronized with the contrast alternation of the stimulus grating. Two independent response blocks of 330 sweeps each are recorded and separated by a user-defined pause (i.e., inter-stimulus interval). For each block, the first 30 sweeps are rejected from the average to eliminate onset effects from the steady-state recording. Sweeps containing spurious signals attributable to blinks and eye movements are rejected over a threshold voltage of ± 25 μV. Resulting steady-state PERGs take the form of near-sine waves that are Fourier transformed to isolate the harmonic component at the contrast reversal rate (16.28 Hz, 2 contrast reversals per cycle). In addition, a noise response is obtained by multiplying alternate sweeps by 1 and −1 before averaging. The noise response also is Fourier transformed at 16.28 Hz to allow calculation of SNR.

Independent study variables were software-provided response amplitudes (μV), latencies (i.e., phase shifts) (π rad) and amplitude noise measurements.

Effect of Operator

Each participant was tested twice each (two trials) by two operators (GV and AT). Both operators were familiar with manufacturer-suggested PERGLA testing protocol, including electrode placement. However, operators had little or no previous experience with electrophysiology testing. Participants were requested to fixate towards a prominent fixation circle placed in the middle of the display. Fixation was monitored visually by the operator, because no automatic fixation monitor is provided. Room lights were turned off and blinds were kept closed. Ambient light from various computer monitors was present. Subjects adapted to this condition for approximately 5 minutes before testing.

The first operator obtained two consecutive PERG recordings (trials) with recording electrodes in place. Electrodes were removed and re-applied by the second operator for a subsequent two trials. The order in which operators completed the testing was randomized. Test time was approximately 4 minutes per test, although preparation and electrode placement added from approximately 5 to 10 minutes to each examination. All tests were conducted within a two-hour period. All eyes were refracted, appropriate corrections for viewing distance were made and near Jaeger acuity was J1 or better for all participants.

Effect of Optical Defocus

Next, operator AT conducted a session during which corrective lenses of different diopters (±0.5, ±1, ±2, ±3) were used to generate optical defocus of both eyes. Lens powers up to ±3 diopters were used to approximate the expected diopter change over the course of several decades (i.e., course of glaucoma follow-up). The order with which lenses of different power were used was randomized. Participants were allowed a few minutes to sit back and rest between each trial. However, skin electrodes were kept in place. Immediately before defocused PERG trials, subjects were tested for near Jaeger acuity to estimate the change in visual acuity induced by the optical defocus. Amplitude and phase measurements at baseline were compared to those at different levels of defocus.

Statistical Analyses

The effect of operator on the variability of PERGLA measurements was assessed using a single nested variance components' analysis model for amplitude and for phase to produce variance components attributable to differences between operators, differences between subjects, differences between eyes of the same subject, differences between trials taken by the two operators, and unexplained variability. The coefficients of variation (CV), calculated both within-operators (repeatability between two consecutive trials) and between-operators (repeatability between trials performed by two different operators after electrode replacement) were also reported separately for amplitude and phase. Agreement between measurements obtained within- and between-operators also was illustrated using Bland-Altman plots.⁶

One-way analysis of variance (ANOVA) with Dunnett's test for post-hoc analysis of pairwise comparisons was used to compare the effect of optical defocus on the variability of PERG amplitude and phase. For the Dunnett's test, the means of amplitude and phase at baseline were used as the control means. Regression analyses and the method of generalized estimating equations (GEE) that adjusts for the correlation between two eyes of the same subject also were applied.⁷ Separate models were constructed to describe the effect of optical defocus for positive and negative diopters.

Statistical analyses were performed using SPSS (version 15.0; SPSS Inc., Chicago, IL, USA), MedCalc (version 8.0.1.0; MedCalc Software, Belgium) and JMP (version 6.1 software; SAS Institute, Cary, NC, USA). A p-value less than 0.05 was considered statistically significant.

Results

For all recordings combined, mean (SD) amplitude, amplitudes noise and phase were 1.32 (0.32) μV, 0.15 (0.06) μV and 1.87 (0.06) π rad, respectively. Amplitude, phase and noise recordings were similar to those previously reported in young healthy subjects.^1,4 There were no statistically significant differences in the mean amplitude and phase within- and between-operators and no significant differences in amplitude noise within-operators (ANOVA, all p> 0.3). However, a small but significant difference was found in the amplitude noise levels between-operators. Specifically, the mean amplitude noise was 0.14 (0.04) μV for operator 1 and 0.16 (0.07) μV for operator 2 (ANOVA, F= 4.8; p= 0.03). As expected, amplitude showed greater variability than phase, which remained relatively stable (i.e., between-operator CVs for amplitude and phase were 11.0% and 0.8%, respectively, while corresponding values for within-operator CVs were 5.5% and 0.7%). Table 1 shows results from the variance component analyses performed for both amplitude and phase. According to the model, most of the amplitude and phase measurement variability could be explained by differences between subjects. However, differences between operators accounted for 26.6% and 18.2% of the total variance for amplitude and phase, respectively. The effect of trials by operators on the overall variability of PERGLA measurements was modest.

Table 1.

Results from a single nested variance components' analysis model for amplitude and for phase with percentages of total variance from each variance component factor analyzed.

	Variance component	Parameter estimate	Standard error	Percentage of total variance (%)	P value
Amplitude (μV)	Subject	0.06	0.04	53.1	0.14
	Eye	0.01	0.007	8.8	0.05
	Operator	0.03	0.01	26.6	0.05
	Trial*Operator	0.005	0.003	4.4	0.09
	Unexplained variability	0.008	0.002	7.1	<0.001
Phase (π rad)	Subject	0.004	0.002	36.4	0.04
	Eye	0.0001	3.3E-5	9.1	0.12
	Operator	0.0002	0.0001	18.2	0.25
	Trial*Operator	0.0001	9.4E-5	9.1	0.19
	Unexplained variability	0.0003	6.9E-5	27.2	<0.001

Bland-Altman plots (differences in amplitude measurements as a function of average amplitude measurements) with 95% limits of agreement describing within-operator and between-operator agreement are shown in Figure 1A and 1B, respectively. These plots suggest that agreement was somewhat worse when measurements were compared between-operators than within-operators, evidenced by wider 95% limits of agreement in Figure 1B than in Figure 1A. Absolute amplitude differences also were significantly different within- and between-operators (0.10 μV and 0.23 μV, respectively; F=15.95, p<0.001).

Figure 1.

Bland-Altman plots showing within-operator (A) and between-operator (B) agreement between PERGLA amplitude measurements.

The within-operator Bland-Altman plot (Figure 1A) suggests that amplitude for the first trial was higher than amplitude for the second trial in eyes with higher average amplitudes (R²=0.10, p=0.04). A similar proportional bias was not present in the between-operators Bland-Altman plot (Figure 1B; R²=0.05, p=0.15). This finding may reflect the percentage of amplitude decrease attributable to adaptation to the PERG stimulus when the inter-stimulus interval is short.⁸ However, because the 95% confidence intervals of the regression line include the zero line, the proportional bias found may simply be the result of chance. Bland-Altman plots also indicated that, in general, measurement agreement tended to depend on amplitude magnitude (i.e., agreement decreased as the amplitude increased), as previously described.⁴

Table 2 shows mean amplitude and phase (SD) at baseline with appropriate correction for viewing distance and at different powers of optical defocus. ANOVA identified a significant effect of defocus on mean amplitude (F= 2.65, p=0.01) but not phase (F=1.02, p=0.42). Post-hoc analyses showed that mean amplitudes at -2 and ±3 lens powers were significantly lower than at baseline.

Table 2.

Jaeger (J) visual acuity, PERGLA amplitude and phase at baseline and at different diopters (dpt) of optical defocus. P-value indicates significance in comparison to baseline measurements.

	Jaeger acuity	Amplitude (μV) Mean (SD)	P-value*	Phase (π rad) Mean (SD)	P-value**
Baseline	J1 (n=20)	1.35 (0.40)		1.86 (0.06)
+0.50 dpt	J1 (n=20)	1.26 (0.33)	0.93	1.88 (0.05)	0.91
-0.50 dpt	J1 (n=18) J2 (n=1) J3 (n=1)	1.25 (0.30)	0.88	1.87 (0.05)	0.98
+1.00 dpt	J1 (n=20)	1.26 (0.34)	0.91	1.88 (0.05)	0.84
-1.00 dpt	J1 (n=18) J3 (n=1) J4 (n=1)	1.15 (0.25)	0.23	1.87 (0.06)	0.99
+2.00 dpt	J1 (n=20)	1.21 (0.31)	0.61	1.89 (0.05)	0.58
-2.00 dpt	J1 (n=16) J3 (n=2) J4 (n=2)	1.06 (0.26)	0.03	1.88 (0.06)	0.75
+3.00 dpt	J1 (n=18) J2 (n=2)	1.06 (0.32)	0.03	1.90 (0.05)	0.08
-3.00 dpt	J1 (n=14) J3 (n=2) J5 (n=2) J7 (n=2)	0.99 (0.37)	0.004	1.88 (0.06)	0.94

^*After one-way analysis of variance (ANOVA) with Dunnett's test for post-hoc analysis of pairwise comparisons: F=2.65, p=0.0092. For the Dunnett's test, the mean amplitude at baseline was used as control mean.

^**After one-way analysis of variance (ANOVA) with Dunnett's test for post-hoc analysis of pairwise comparisons: F=1.02, p=0.42. For the Dunnett's test, the mean phase at baseline was used as control mean.

In linear regression analysis, amplitude tended to decrease somewhat with increasing lens power, regardless of direction (+ or -) (R²=0.07, p=0.01 and R²=0.13, p<0.001; for positive and negative lens powers, respectively). Phase was apparently less affected by optical defocus, increasing slightly with positive lens powers (R²=0.06, p=0.01) but not with negative lens powers (R²=0.01, p=0.30). It is unlikely that defocus results were driven by participant fatigue because the order of lens presentation was randomized.

Table 3 shows results from the GEE models used to describe the effect of optical defocus on amplitude and phase for positive and negative diopter changes. These results confirm that amplitude was more affected by optical defocus than phase.

Table 3.

Intercept, unstandardized beta coefficient (B) and its standard error (SE) with associated p-value to describe the effect of optical defocus on amplitude and phase for positive and negative diopters (dpt).

		Intercept	B	SE	P value
Amplitude (μV)	Positive dpt	1.34	-0.09	0.04	0.01
	Negative dpt	1.27	0.09	0.04	0.02
Phase (π rad)	Positive dpt	1.87	0.01	0.003	0.02
	Negative dpt	1.87	-0.005	0.003	0.06

Discussion

Many studies have evaluated conventional PERG techniques in terms of their reliability and potential to assist clinicians in the diagnosis of glaucoma (see for instance^9-14). Results from most of these studies likely are applicable to the PERGLA paradigm because PERGLA retains most of the characteristics of conventional PERG.¹⁵ Nevertheless, because PERGLA is a non-conventional, glaucoma-specific and commercially available clinical tool, it is important to identify non-disease related measurement variability that could compromise the clinical usefulness of this paradigm. To accurately detect and monitor disease, a good understanding of these factors is necessary.

The current results indicate that, although CVs for amplitude and phase were similar to those previously reported,^1,3,4 a significant percentage of total PERGLA measurement variability can be explained by differences between operators, particularly for amplitude (see Table 1). In addition, our results indicate that optical defocus produces a significant decrease in amplitude while leaving phase relatively unaffected.

The PERGLA instrument was originally designed to be minimally invasive and user-friendly, thus allowing novice operators to easily perform the test and interpret the results.^1,16 These potential advantages over other electrophysiology techniques make the PERGLA appealing for general clinical practice. Theoretically, an instrument that objectively measures retinal ganglion cell function with measurement variability that is operator-independent is ideal for use in a glaucoma clinic. However, our results suggest that PERGLA (and likely all PERG) measurement variability is not wholly operator-independent. This is likely due in part to variable positioning of recording electrodes. Although care was taken to ensure that the electrodes were correctly positioned and the impedance was below acceptable levels based on manufacturer's suggestion (≤5 kΩ), it is possible that differences in the position of the electrodes and in the impedance levels obtained by different operators caused variations in amplitude.

Electrode impedance, even for values higher than 5kΩ, is not expected to alter the signal, because signal attenuation resulting from such impedance levels cannot be measured by modern preamplifiers. However, increased impedance potentially could increase the noise level, due to increased detection of electromagnetic disturbances, ultimately resulting in increased variability of amplitude and phase. In this study, we found that the noise levels were significantly different between operators and that measurements reproducibility was slightly better for the operator obtaining lower amplitude noise on average (the CV for operator 1 with lower mean amplitude noise was 5% while the CV for operator 2 was 6%). In addition, changes in noise level were significantly associated with changes in amplitude levels between-operators, but not within-operators (R²<0.01, p=0.91 and R²=0.12, p=0.03 for within- and between-operators, respectively). It is possible that operator 1 achieved lower impedance levels prior to or during the test. However, like most other electrophysiology techniques, the PERGLA does not allow monitoring of impedance levels during recording and summary impedance is not provided at the end of testing. Recent developments in bioelectric recording indicate that continuous impedance monitoring currently is possible¹⁷ and this feature could be added to PERGLA signal acquisition software in the future if this variable was deemed important. Our study suggests that, with the current version of PERGLA, it may be important to record the amplitude noise levels and maintain these levels across tests. The amplitude noise levels used to calculate the signal to noise ratios (SNR) reported in this study and others¹⁶ do not represent the true noise but rather contain some residual signal that contributes to the noise index measurement. This is due to the way that the noise index is obtained with PERGLA¹. The PERGLA software assumes identical signal in the even and odd sweeps used to calculate the noise index. The signal from these sweeps is then subtracted and residual signal is used to describe the noise index. However, it is likely that signal is not identical in both even and odd sweeps, resulting in incomplete cancellation and therefore some addition to the reported noise level. Therefore, the true noise is expected to have smaller amplitude than the noise index provided by the recording system and the true SNR may not reflect the one that is currently being measured.

Other variables such as subject fatigue may have contributed to the greater between-operator than within-operator variability observed. However, Bland-Altman plots show that a decrease in amplitude, possibly attributable to adaptation to the PERG stimulus, was more evident within-operators than between-operators. It is possible that fatigue may result in loss of target fixation but this cannot be systematically evaluated because the PERGLA instrument lacks a feature that allows gaze tracking during testing.

In addition to electrode placement, one of the essential components of PERG testing is obtaining an accurate refraction to provide adequate correction for viewing distance. Because significant effects of optical defocus on various PERG recording techniques has been previously described^18-22 and because PERGLA represents a unique PERG paradigm, the current study explored the effect of optical defocus on amplitude and phase measurements to investigate the case in which PERGLA testing is conducted without optimal correction. Previous studies have shown that amplitude, more than phase, is significantly affected by optical defocus. ^18,20-22 In addition, it has been shown that scatter has significant detrimental effects on outcomes beyond those of blur, an issue relevant to patients developing age-related cataract where both scatter and blur can lead to a significant reduction in visual acuity¹⁸. Our results showed that the effect of optical defocus on PERG amplitude is significant at defocus greater than -2 diopters and +3 diopters and that the effect of defocus on phase is minimal. Table 2 shows that 14 eyes still retained J1 vision with a -3 diopter lens. To evaluate whether the effect of optical defocus remained in subject eyes with J1 acuity, a subgroup analysis was conducted by excluding eyes with J2 acuity or worse at any diopter change. The results showed that mean amplitude in eyes that retained J1 acuity did not significantly decrease from baseline (F=1.24; p=0.28). However, when all recordings obtained at > J1 were considered, an average decrease in amplitude of 0.4 μV (30.4%) compared to baseline was observed. Interestingly, this is the same amplitude decrease observed in early glaucoma eyes (compared to healthy eyes) in a previous study.²³ Plotting amplitude as a function of the “J” near scale, we found a significant negative correlation (R²=0.15; p<0.0001) indicating that amplitude decreased by 10.9% on average from baseline for each unit decrease in the “J” near scale. Similar results have been reported elsewhere.¹⁵ Overall, current results confirm that with PERGLA, if visual acuity is kept appropriate for viewing distance (e.g., J1 vision or better), little or no change in response amplitude should be expected.

In general, PERG phase was relatively unaffected by optical defocus up ± 3 diopters. The relative stability of this parameters might imply that it is promising for detecting glaucoma and monitoring its progression. However, a recent study by our group found that phase discriminates poorly (at chance level) between healthy eyes and those with glaucomatous optic neuropathy, apparently limiting the PERGLA potential diagnostic ability to changes in amplitude only.²³

In the current study, repeat testing used to describe the effect of operator on PERGLA measurement variability was conducted by only two operators on the same day. The former potential limitation was the result of keeping testing duration manageable for study participants (e.g., to minimize fatigue; our study participants were tested for 12 PERGLA trials in a single session). A recent study by Fredette and colleagues reporting the variability of PERGLA measurements obtained on each of five days within an eight-week period addresses the latter potential limitation.³

Previous studies indicate that measurements of PERGLA amplitude can discriminate between healthy and glaucoma eyes²³ and that a decrease in PERG amplitude is suggestive of glaucomatous functional damage.²⁴ Therefore, it is important to better evaluate factors that also may affect PERGLA amplitude. The current study showed that measurements obtained by different operators result in significant differences in PERGLA amplitude. This is likely due to variations in electrode positioning and impedance across operators indicating that these variables should be considered in a clinical setting when different technicians are involved in PERGLA testing. In addition, although optical defocus leads to a decrease in PERGLA amplitude by reducing visual acuity, this situation can be avoided by performing an accurate refraction to obtain J1 or better vision before testing.