Factors affecting the test-retest variability of Heidelberg retina tomograph and Heidelberg retina tomograph II measurements

Abstract

Aims: To evaluate the test-retest variability of stereometric parameter measurements made with the Heidelberg retina tomograph (HRT) and Heidelberg retina tomograph-II (HRT-II), and to establish which parameter(s) provided the most repeatable and reliable measurements with both devices. An investigation into the factors affecting the repeatability of the measurements of this parameter(s) was conducted.

Methods: 43 ocular hypertensive and 31 glaucoma subjects were recruited to a test-retest study. One eye from each subject underwent HRT and HRT-II imaging by two observers on each of two occasions within 6 weeks of each other. Lens grading was carried out by LOCS III grading and Scheimpflug camera generated densitogram analysis.

Results: Rim area (RA) and mean cup depth measurements were found to be least variable. Both inter-test reference height difference and image quality had a strong relation (R²>0.5, p<0.0001) with inter-test RA difference and, together, are responsible for 70% of RA measurement variability. Image quality was influenced by lens opacity, cylindrical error, and age. Inter-test RA measurement differences were unrelated to the observer or visit interval.

Conclusions: RA represents an appropriate measure for monitoring glaucomatous progression. Reference height difference and image quality were the factors that most influenced RA measurement variability. Image analysis strategies that address these factors may reduce test-retest variability.

The ability to evaluate disease progression is of great importance in the management of patients with primary open angle glaucoma (POAG). Likewise, the detection of progression in subjects with ocular hypertension (OHT) could enable early intervention and delay the onset of visual field (VF) loss.^1,2 POAG is a chronic, progressive disorder in which damage to retinal ganglion cell axons results in characteristic VF defects. However, there is evidence to suggest that structural changes to the optic nerve head (ONH) may occur before identifiable VF changes are detected by standard perimetry.^3–7 Clinical examination of the ONH is of limited value in detecting progression because of interobserver variation.⁸ An objective approach, longitudinal quantitative imaging of the ONH, may be more useful for the detection of glaucomatous progression.

One imaging method is scanning laser tomography performed with the Heidelberg retina tomograph (HRT) and, more recently, the Heidelberg retina tomograph II (HRT-II). The HRT-II is intended for use in the clinical milieu by various operators, some of whom may not possess the level of experience required for the successful use of the previous model.⁹ Both devices generate three dimensional mean topography images from which a range of stereometric parameter values can be calculated. These parameters can be measured over time to detect progression. It is useful to estimate the test-retest repeatability of stereometric parameter measurements so that changes caused by disease progression and not by measurement error can be identified correctly.

The purpose of this study was to evaluate the test-retest variability of stereometric parameter measurements made with the HRT and HRT-II, and to establish which parameter(s) provided the most repeatable and reliable measurements. An investigation into the factors that may affect the repeatability of the measurement of this parameter was conducted.

MATERIALS AND METHODS

Subject selection

A total of 74 (43 OHT; 31 POAG) subjects were recruited from the ocular hypertension clinic at Moorfields Eye Hospital. OHT was defined as intraocular pressure (IOP) more than 21 mm Hg on two or more occasions and a baseline Humphrey 24–2 full threshold Advanced Glaucoma Intervention Study (AGIS) score of 0.¹⁰ POAG was defined as a consistent AGIS VF score greater than 0, and a pretreatment IOP greater than 21 mm Hg on two or more occasions. All subjects had previous experience of scanning laser tomography. For each subject, one eye was selected on the basis that it had a refractive error less than 12 dioptres of spherical power and no history of previous intraocular surgery. In subjects with lens opacity, the eye with the greater degree of opacity was preferentially selected although the presence of lens opacity itself was not a criterion for subject selection. This study adhered to the tenets of the Declaration of Helsinki and had local ethics committee approval and the subjects’ informed consent.

Testing protocol

Image acquisition was carried out by two experienced observers (ETW and NGS) at each of two visits within 6 weeks of each other. The testing sequences were performed using both the HRT (Version 2.01b; Heidelberg Engineering, Heidelberg, Germany) and HRT-II (Eye Explorer Version 1.7.0). In each subject, the scan focus (HRT and HRT-II) and depth of focus (HRT) that were used in the first visit were also used in the second visit. A series of three scans was acquired by each observer at a 10° field of view for the HRT, and 15° for the HRT-II (the two different scanning angles have the same degree of resolution). The following imaging protocol was adhered to for all subjects:

• Visit 1: ETW then NGS then ETW

• Visit 2: ETW then NGS.

Following IOP measurement at visit 1, the eyes were dilated using tropicamide 1% to enable one observer (NGS) to carry out lens grading. Subjective grading was carried out using the Lens Opacity Classification System III (LOCS III).¹¹ Nuclear opalescence (NO, range 0.1–6.9), nuclear colour (NC, range 0.1–6.9), posterior subcapsular (P, range 0.1–5.9) and cortical (C, range 0.1–5.9) scores were graded against a standardised transparency. Scheimpflug photography was performed using the Case 2000 series (Marcher Diagnostics, Hereford, UK). The central nuclear dip (CND) value, derived from digitised densitograms, was used as an objective lens score.¹²

Image analysis

Heidelberg Eye Explorer (Version 1.7.0), the operating system of the HRT-II, was used to generate mean topography images and to perform image analysis. The term “HRT-II Explore” is used to indicate when Explorer was used to analyse the HRT-II mean topographies. The HRT topographies were imported into the HRT-II operating platform as HRT-Port files. HRT mean topographies were generated and analysed using the same Explorer software as the HRT-II images (termed “HRT Explorer”). HRT mean topographies were also generated and analysed using an option on the Explorer software called “HRT Classic ” which is derived from the older MS-DOS HRT software. HRT and HRT-II images may be examined interchangeably, and therefore longitudinally, using Explorer; HRT-II images are not compatible with HRT Classic. Contour lines were drawn by one observer (NGS) on the baseline mean topographies and exported to the subsequent images. Four different image sequences were analysed for both imaging devices:

• intraobserver/intravisit (ETW then ETW, same visit)

• interobserver/intravisit (ETW then NGS, same visit)

• intraobserver/intervisit (ETW then ETW, different visits)

• interobserver/intervisit (ETW then NGS, different visits).

The Standard reference plane was used for all analyses in this study. The mean pixel height standard deviation (MPHSD) was recorded for each mean topography as a proxy measure of image quality.

Statistical methods

Within subject coefficient of variation was used to examine the repeatability of each stereometric parameter. Within subject coefficient of variation was calculated with the following equations:

CV_w = 100×s_w/mean of all repeated measurements

where s_w is the common standard deviation of repeated measurements (within subject standard deviation) and CV_w is the within subject coefficient of variation.

Test-retest repeatability was also assessed by constructing Bland-Altman plots and by estimating the repeatability coefficient (RC) as:

RC = sqrt(2)×1.96s_w

This statistical method was applied when no relation was observed between observation magnitude and observation difference, and when the observation differences were normally distributed¹³ (JM Bland, personal communication, 2004).

Intraclass correlation coefficient (ICC) was used to estimate the reliability of the parameters generated.

Scatter plots and regression lines were constructed to identify which factors influenced test-retest measurement variability with significant associations assumed at p<0.05. The factors evaluated were age, refractive error (spherical and cylindrical power), IOP, lens score (NO, NC, PS, C, and CND), MPHSD, inter-test reference height difference, disc area, and baseline rim area.

All statistical analyses were performed using Medcalc Version 7.4.2.0 (Medcalc Software, Mariakerke, Belgium) and SPSS Version 11.5 (SPSS Inc, Chicago, IL, USA).

RESULTS

The male:female ratio of the subjects was 41:33 and the right:left eye ratio was 43:31. The baseline subject characteristics are summarised in table 1.

Table 1.

 Summary of baseline characteristics of test-retest subjects

	Mean	SD	Range
Age (years)	68.2	10.2	20.4–84.8
Mean deviation (dB)	−2.2	2.4	−11.3–1.3
Disc area (mm2)	1.9	0.4	1.0–3.4
Baseline rim area (mm2)	1.2	0.4	0.4–2.2
Spherical power (dioptres)	−0.2	2.8	−7.7–5.0
Cylindrical power (dioptres)	1.0	0.8	0–4.2
Spherical equivalent (dioptres)	0.4	2.7	−7.4–5.7
LOCS III nuclear opalescence	2.2	0.9	0.4–5.3
LOCS III nuclear colour	2.0	0.8	0.3–5.1
LOCS III posterior subcapsular	0.4	0.5	0.1–3.8
LOCS III cortical	0.8	1.0	0.1–3.8
Central nuclear dip	17	3	12–29
MPHSD (HRT)	31	22	11–121
MPHSD (HRT-II)	27	21	10–106

HRT = Heidelberg retina tomograph; LOCS III = Lens Opacity Classification System III; MPHSD = mean pixel height standard deviation.

Judged by the coefficient of variation, RA and mean cup depth had the highest measurement repeatability (table 2).

Table 2.

 Coefficient of variation values for the stereometric parameters generated in the test-retest study

Stereometric parameter	Coefficient of variation (%)
	HRT Classic				HRT Explorer				HRT-II Explorer
	I	II	III	IV	I	II	III	IV	I	II	III	IV
Cup area	26	21	16	27	25	25	25	24	25	25	23	19
Rim area	7	11	7	10	9	10	9	10	9	12	7	8
Cup volume	26	27	23	30	27	30	31	30	32	37	29	26
Rim volume	16	19	15	17	18	19	19	19	18	22	17	18
Reference height	11	14	11	13	11	15	13	16	17	17	13	13
Height variation contour	14	12	15	15	13	16	14	16	20	15	20	16
Maximum cup depth	8	7	10	8	8	7	10	11	12	13	10	12
Mean cup depth	8	8	9	7	7	8	9	11	11	11	9	10
Mean retinal nerve fibre layer thickness	21	21	32	19	20	23	18	22	31	27	25	22
Retinal nerve fibre layer cross sectional area	22	21	22	19	20	23	18	22	31	27	25	22
Cup shape measure	19	28	28	23	24	32	32	27	24	26	28	29

I = intraobserver/intravisit; II = interobserver/intravisit; III = intraobserver/intervisit; IV = interobserver/intervisit.

Low coefficient values equate to high repeatability.

Judged by the ICC, mean cup depth, cup volume, cup area and RA were the most reliable parameters (table 3). There is no significant difference between the coefficients generated for these parameters in the situation most likely to be encountered in the longitudinal setting (interobserver/intervisit, IV).

Table 3.

 Intraclass correlation coefficients for the stereometric parameters generated in the test-retest study

Stereometric parameter	Intraclass correlation coefficient
	HRT Classic				HRT Explorer				HRT-II Explorer
	I	II	III	IV	I	II	III	IV	I	II	III	IV
Cup area	0.97	0.94	0.97	0.96	0.94	0.93	0.93	0.95	0.95	0.92	0.96	0.96
Rim area	0.95	0.90	0.96	0.94	0.92	0.90	0.91	0.93	0.93	0.89	0.95	0.95
Cup volume	0.97	0.97	0.96	0.95	0.96	0.96	0.95	0.96	0.95	0.92	0.97	0.97
Rim volume	0.87	0.81	0.90	0.84	0.84	0.80	0.86	0.87	0.84	0.79	0.88	0.87
Reference height	0.86	0.85	0.88	0.84	0.81	0.80	0.83	0.81	0.84	0.74	0.88	0.88
Height variation contour	0.77	0.87	0.81	0.82	0.86	0.73	0.82	0.81	0.65	0.75	0.66	0.78
Maximum cup depth	0.92	0.96	0.89	0.95	0.91	0.97	0.91	0.93	0.76	0.85	0.90	0.88
Mean cup depth	0.97	0.97	0.96	0.98	0.98	0.97	0.97	0.97	0.93	0.95	0.97	0.95
Mean retinal nerve fibre layer thickness	0.82	0.75	0.83	0.79	0.79	0.76	0.81	0.76	0.77	0.78	0.80	0.85
Retinal nerve fibre layer cross sectional area	0.87	0.82	0.89	0.83	0.79	0.77	0.82	0.79	0.85	0.66	0.85	0.85
Cup shape measure	0.93	0.92	0.89	0.92	0.93	0.90	0.88	0.88	0.87	0.87	0.86	0.88

I = intraobserver/intravisit; II = interobserver/intravisit; III = intraobserver/intervisit; IV = interobserver/intervisit.

High coefficient values equate to high reliability.

RA and mean cup depth measurements were the most consistently repeatable and reliable. As RA represents the more clinically meaningful measure, subsequent analyses were performed on this parameter.

RA interobserver/intervisit CV_w (%) values were 10.3% (HRT Classic), 10.2% (HRT Explorer), and 7.8% (HRT-II Explorer). RA repeatability coefficients were similar for HRT Classic, HRT Explorer, and HRT-II Explorer, and were not affected by the observer or test interval (table 4). There was a tendency towards more repeatable measurements with HRT Classic, compared with HRT Explorer.

Table 4.

 Rim area repeatability coefficients obtained with different types of HRT at different image sequences

HRT type	Rim area repeatability coefficients (mm2)
	I	II	III	IV
HRT Classic	0.21	0.29	0.19	0.24
HRT Explorer	0.28	0.31	0.30	0.26
HRT-II Explorer	0.28	0.34	0.24	0.23

I = intraobserver/intravisit; II = interobserver/intravisit; III = intraobserver/intervisit; IV = interobserver/intervisit.

Low coefficient values equate to high repeatability; the difference of two measurements were within this value in 95% of cases.

Bland-Altman plots (fig 1) show similar interobserver/intervisit repeatability for the three HRT software platforms. The mean inter-test difference in all cases approximates zero.

Figure 1.

 Bland-Altman plots of interobserver/intervisit rim area obtained with HRT Classic (A), HRT Explorer (B), and HRT-II Explorer (C).

Factors affecting repeatability

Inter-test reference height difference and mean image MPHSD were the two factors which consistently had a strong relation (R²>0.5) with inter-test RA difference for all testing permutations. Figures 2 and 3 show scatter plots, with a regression line, of intraobserver/intravisit RA difference against mean MPHSD and against inter-test reference height difference, respectively. Weaker relations (R²<0.5) of inter-test RA difference were observed with CND, LOCS III NC, and NO scores, and cylindrical power. Table 5 summarises these relations for intraobserver/intravisit RA.

Figure 2.

 Scatter plot of intraobserver/intravisit rim area difference (mm²) against reference height difference (mm) using HRT Explorer. The regression line is also shown (R² = 0.7, p<0.0001).

Figure 3.

 Scatter plot of intraobserver/intravisit rim area difference (mm²) against mean image quality (MPHSD) using HRT Explorer. The regression line is also shown (R² = 0.4, p<0.0001).

Table 5.

 Association (R²) between sources of variability and intraobserver/intravisit rim area differences (mm²)

Source of variability	HRT Classic		HRT Explorer		HRT-II Explorer
	R2	p Value	R2	p Value	R2	p Value
Reference height difference	0.7	<0.0001	0.7	<0.0001	0.5	<0.0001
Mean MPHSD	0.5	<0.0001	0.4	<0.0001	0.5	<0.0001
Central nuclear dip	0.1	0.005	0.3	<0.0001	0.2	<0.0001
LOCS III nuclear opalescence	0.1	0.001	0.2	0.0006	0.2	<0.0001
LOCS III nuclear colour	0.03	0.1	0.1	0.001	0.2	<0.0001
Cylindrical power	0.2	<0.0001	0.03	0.4	0.1	0.005
Spherical power	0.07	0.02	0.02	0.2	0.04	0.1
Spherical equivalent	0.04	0.08	0.01	0.3	0.02	0.2
Age	0.05	0.05	0.04	0.1	0.07	0.02
Baseline rim area	0.05	0.05	0.02	0.2	0.0003	0.9
Disc area	0.001	0.8	0.01	0.5	0.001	0.8
Intraocular pressure	0.05	0.05	0.05	0.07	0.01	0.4
LOCS III posterior subcapsular	0.01	0.4	0.001	0.8	0.005	0.6
LOCS III cortical	0.0001	0.9	0.005	0.6	0.01	0.5

MPHSD = mean pixel height standard deviation, LOCS III = Lens Opacity Classification System III.

A weak relation (R² = 0.1) with age was also found with HRT Classic and HRT-II in the interobserver/intravisit and interobserver/intervisit settings. No relation was observed with disc area, baseline RA, spherical power, spherical equivalent (spherical power + cylindrical power/2), IOP, time between visits, LOCS III PS, or LOCS III C scores.

A multiple regression was performed, with RA difference as the dependent variable, and the factors identified as influencing RA difference (that is, reference height difference, MPHSD, CND, NO, NC, age, and cylindrical power) as independent variables. Reference height difference (p<0.0001) and MPHSD (p = 0.05) were the only two significant variables (R² = 0.7).

To elucidate which factors influenced image quality (as determined by MPHSD), a multiple regression was carried out for intraobserver/intravisit HRT Explorer, with MPHSD as the dependent variable and CND (as a single, objective measure of lens opacity), age, and cylindrical power as independent variables (R² = 0.5). CND and cylindrical power displayed a highly significant (p<0.0001) relation with image quality, and age showed a weaker but significant (p = 0.03) relation.

DISCUSSION

Scanning laser tomography is a well established technique that provides reproducible ONH measurements.^14,15 The topographic measures produced by the HRT and its predecessor, the laser tomographic scanner (LTS, Heidelberg Engineering, Heidelberg, Germany), have been demonstrated to be repeatable,^16,17 and to have less variation compared with other techniques such as computer assisted planimetry.¹⁸ Little has been published about the reproducibility of the HRT-II.^19,20 As the HRT-II is intended as a “clinical” instrument, its reproducibility under clinical conditions needs to be established. The profile of the subjects in this study was heterogeneous in terms of demographics, disease stage, refractive error, media opacity, and image quality, and therefore simulates the patient profile encountered in clinic. Image quality has previously been shown to be associated with pupil size and the degree of lens opacity (both objective scoring and LOCS III grading). Image quality was seen to improve with pupillary dilation but the improvements were often small.²¹ Pupil size was therefore not taken into consideration in this study. None of the subjects was taking miotic medications at the time of the study, although this was not a recruitment criterion.

Since the publication of the original reproducibility studies of the HRT,^17,22 the Windows based Explorer platform has been introduced. From this perspective, this study is the first to examine the repeatability of HRT defined morphometric parameters using the newer software.

RA and mean cup depth were the most repeatable parameters for both devices. Some caution is required when interpreting coefficients of variability as some parameters, such as cup area and cup shape measure, have mean values of low magnitude (approaching zero). Another difficulty is the interpretation of differences between ICC values of a similar magnitude. It is therefore unlikely that there is any real difference in measurement reliability between mean cup depth, cup area, cup volume, and RA. It should also be noted that the ICC values depend on the variability of the sample population. As our sample was enriched with eyes with lenticular opacity, the ICC values may not be applicable to other populations with less cataract.

Overall, RA and mean cup depth were consistently the most repeatable and reliable of the parameters measured. This concurs with previous findings.²³ The findings from another study showed that mean cup depth and cup area were the least variable parameters measured with the HRT-II.¹⁹ There is, however, no advantage in measuring cup area as it is merely the difference between disc area (kept constant in Explorer) and RA. As it contains the retinal ganglion cell axons, RA is a meaningful parameter for physicians. It has also been shown to discriminate between normal, glaucoma, and OHT subjects,^24–26 and is therefore an appropriate candidate for the assessment of progression.

In this study, the repeatability of RA measurements was similar with both devices (RC = 0.2 – 0.3 mm²), irrespective of observer or test interval. Similar repeatability between imaging performed at the same visit or at different visits is consistent with a previous study, where no difference was identified in the short term and long term variability of topographic measurements.²⁷ The HRT-II performs at least as well as the HRT. The similar level of RA repeatability between HRT-II and HRT Explorer analyses indicate that the two methods could theoretically be used interchangeably in a longitudinal setting.

The sources of variability for the HRT have been documented and include patient/scanner misalignment,²⁸ and interobserver differences in optic disc contour line drawing.^18,29,30 The present study identifies inter-test reference height difference to be the most consistent factor related to test-retest variability. It has previously been reported that the use of a 320 μm reference plane reduced RA variability, compared with the Standard reference plane.³¹ Image quality, as recorded by MPHSD, is another factor consistently found to influence variability. MPHSD is a gauge of the variability of pixel height measurements across the three topographic images used to construct the mean image.¹⁴ This study shows that image quality was, in turn, influenced by lens opacity, age, and degree of astigmatism. Sihota et al also found a significant correlation between the test-retest variability of the HRT-II and age and degree of astigmatism.¹⁹ Our results suggest that MPHSD may be an appropriate summary measure for the effect of these factors. It is therefore possible to predict repeatability coefficients for various levels of image quality without having to measure the patient’s age, degree of media opacity, or astigmatism.

In conclusion, this study indicates that RA may be an appropriate measure when monitoring glaucoma progression, as its measurements consistently showed excellent repeatability and reliability. Repeatability was similar with both the HRT and HRT-II, irrespective of observer or test interval. Reference height difference and image quality were found to be the factors that influenced RA variability most. The findings of this study will be used as the basis for suggesting strategies for improving test-retest repeatability. Once this is achieved, strategies for monitoring stereometric parameter progression can be devised and tested.

Abbreviations

• CND, central nuclear dip

• HRT, Heidelberg retina tomograph

• ICC, intraclass correlation coefficient

• LOCS III, Lens Opacity Classification System III

• NC, nuclear colour

• NO, nuclear opalescence

• OHT, ocular hypertension

• ONH, optic nerve head

• POAG, primary open angle glaucoma

• RA, rim area

• VF, visual field