Abstract
Purpose
To assess the degree of concordance among clinicians reviewing 3 Heidelberg retina tomograph (HRT) printouts used to detect progression, the Moorfields regression analysis (MRA), the topographic change analysis (TCA), and trend analysis (TA), and to compare with progression identified by stereophotographs.
Design
Observational cohort study.
Participants
We longitudinally followed 237 eyes of 168 patients (50 glaucomatous eyes, 187 glaucoma suspects) from the Diagnostic Innovation in Glaucoma Study (mean follow-up, 46.8±14.2 months), with a minimum of 4 HRT images (range, 4–8).
Methods
Three experienced observers judged the presence of progression using the HRT follow-up printouts available for each HRT method of analysis (MRA, TCA, TA). The overall assessment was based on majority rule, with ≥2 graders agreeing on the classification.
Main Outcome Measures
Observers agreement in assessing HRT progression and agreement for progression or no progression between the HRT methods of analysis and the reference standard represented by masked stereophotograph assessment. The κ test was used to assess the interobserver agreement.
Results
In general, agreement among clinicians for subjective assessment of progression based on HRT printouts was moderate to good; agreement (κ) ranged from 0.52 to 0.71 for MRA, 0.61 to 0.63 for TCA, and 0.45 to 0.74 for TA. Of the 237 eyes, 16 (6.8%) were found to progress during follow-up based on masked stereophotograph assessment. Agreement for progression/no progression between the HRT methods and stereophotography was similar among MRA (84.8%, agreement on 5 progressing eyes and 196 nonprogressing eyes; κ = 0.14), TCA, (82.3%, agreement on 8 progressing eyes and 187 nonprogressing eyes; κ = 0.2), and TA (84%, agreement on 2 progressing eyes and 197 nonprogressing eyes; κ = 0.01).
Conclusions
Clinicians' agreement in identifying suspected glaucomatous progression using different HRT methods of analysis was moderate to good and was similar among all methods, including MRA, which is not designed to detect progression. Agreement between progression identified by HRT and masked stereophotograph assessment was poor. These results suggest that assessment of the HRT and stereophotography may be identifying different aspects of structural change.
One of the advantages of integrating the use of new technologies in clinical practice is to help clinicians objectively assess the presence of the disease and monitor its progression. Confocal scanning laser ophthalmoscopy (CSLO) is currently used to quantitatively assess optic disc topography.1–3 Cross-sectional studies have demonstrated that CSLO can help to distinguish between normal individuals and glaucoma patients distributed across the continuum of the disease.4–13 However, few longitudinal studies have evaluated its ability to detect glaucomatous progression.14–18
There are 2 different options originally developed for assessing glaucomatous progression with the Heidelberg Retinal Tomograph (HRT; Heidelberg Engineering, Heidelberg, Germany). Topographic change analysis (TCA) is based on the probability of change in a cluster of pixels within the optic disc margin19 and the stereometric trend analysis (TA) reports changes in normalized topographic parameters over time.
Printouts of TCA, TA, and chronological series of CSLO images classified by the Moorfields regression analysis (MRA) are available for review by clinicians during the follow-up of glaucoma.
Because one may easily assess progression not only by means of these progression tools, but also by evaluating the available CSLO printouts for follow-up, the main objective of this study was to assess the degree of concordance among clinicians for detecting glaucomatous progression using the MRA, TCA, and TA methods. In addition, results were compared with the accepted reference standard for assessing progression, changes in the optic disc as assessed by stereophotographs.20–22
Methods
Patients were prospectively evaluated at the Hamilton Glaucoma Center, University of California, San Diego, as part of the Diagnostic Innovations in Glaucoma Study, a prospective, longitudinal study designed to evaluate optic nerve structure and visual function in glaucoma. Informed written consent was obtained from all participants and all methods were approved by the University of California, San Diego, Institutional Review Board. The study adhered to the Declaration of Helsinki for research involving human subjects.
As part of Diagnostic Innovations in Glaucoma Study, each study participant underwent a complete ophthalmologic examination, including review of medical history, best-corrected visual acuity testing, slit-lamp biomicroscopy, intraocular pressure measurement, dilated stereoscopic fundus examination using a 78-diopter lens, gonioscopy, simultaneous stereoscopic optic disc photography (TRC-SS, Topcon Instruments Corporation of America, Paramus, NJ), and standard automated perimetry (SAP) using a 24-2 Swedish Interactive Threshold Algorithm (Humphrey Visual Field Analyzer, Carl Zeiss Meditec, Dublin, CA) annually during the course of follow-up.
At study entry, all eyes had best-corrected visual acuity of ≥20/40, sphere within ±5.0 diopters, cylinder within ±3.0 diopters, and open angle at gonioscopy.
Participants were excluded if they had a history of intraocular surgery except for uncomplicated cataract or glaucoma surgery. Participants with secondary causes of elevated intraocular pressure (e.g., iridocyclitis, trauma), other intraocular eye disease, other diseases affecting the visual field (e.g., pituitary lesions, demyelinating diseases, HIV or AIDS, or diabetic retinopathy), or under medications known to affect visual field sensitivity were also excluded.
To be included in the study, patients were required to have a follow-up of >2 years and ≥4 good-quality CSLO examinations, in addition to reliable SAP testing (≤33% false positives, false negatives, fixation losses) and stereophotographs of the optic disc within 6 months of their first and most recent HRT examination.
Instrumentation
Confocal Scanning Laser Ophthalmoscopy
The HRT provides topographical measurements of the optic disc and peripapillary retina. Details on the instrument and its principle of use have been described elsewhere.1 For each study eye, at least once a year, 3 scans were centered on the optic disc and a mean topography was automatically generated. Magnification errors were corrected using patient's corneal curvature measurements. An experienced examiner outlined the optic disc margin on the mean topographic image while viewing simultaneous stereoscopic photographs of the optic disc. All images included in the analysis were carefully examined and reviewed for proper centering, focus, and illumination; all mean topography images had a standard deviation of <50 microns. Mean topography images with a standard deviation of ≥50 microns were discarded. We used HRT software version 1.5.9.0 or earlier for acquisition and analysis was performed with the recently released software version 3.0.
For this study, 3 experienced observers (GV, JMC, and LA) judged the presence of progression using the HRT follow-up printouts for each of the proposed methods of analysis (MRA, TCA, and TA). Figure 1 shows an example of the HRT printouts available for MRA (Fig 1a), TCA (Fig 1b), and TA (Fig 1c). The observers were glaucoma specialists experienced with the use and interpretation of CSLO images. Completion of online tutorials on the use of HRT and different methods of analysis available at www.heidelbergengineering.com (accessed July 25, 2007) was required. Each method of analysis was evaluated independently. The graders were masked from the patient's clinical history, visual field results, optic disc stereophotograph assessment results, other HRT printouts and other graders' results.
Figure 1.
An example of the Heidelberg Retina Tomograph (HRT) printouts used by clinicians for identifying progression. a, Moorfields regression analysis (MRA) printouts with the topography maps along with the MRA analysis for each year of follow-up. b, Topographic change (TCA) printouts. The TCA total size and total volume change over time in mm2 and 1/1000 mm3, respectively. c, Trend analysis (TA) printouts. Changes over time in the average of several HRT parameters are combined for Global, Superior, and Inferior sectors (see text for details). The TA is based on the normalized “average” parameter defined as the average of several HRT parameters combined (rim area, rim volume, cup volume, cup shape, mean retinal nerve fiber layer thickness, mean height of contour, mean contour elevation, contour line modulation temporal, mean cup depth, and mean height inside contour line).
Methods of Analysis
Moorfield Regression Analysis
Moorfields Standard Reports at baseline and per each year of follow-up were selected. All the printouts were displayed in series and were available for simultaneous examination. Each observer examined the changes in the topography, reflection map, MRA classification, and rim area. As a general rule, progression was assessed based on the subjective evaluation of each image and in consideration of a change in the MRA classification (from within normal limits to borderline or from borderline to outside normal limits) in any of the 6 predefined sectors in 2 consecutive images. The series of bars displayed for the whole disc and each predefined segment, describing the ratio between cup area (the portion of the bar in red) and rim area (the portion of the bar in green) in relation to the MRA predicted values, were also subjectively evaluated by each grader for detecting change (Fig 1a).
Baseline global MRA classification was used to infer on the severity of glaucomatous damage in progressing and nonprogressing eyes. Table 1 shows the percentage of eyes with a global MRA classification outside normal limits for the 2 groups (progressors and nonprogressors).
Table 1.
Characteristics of Progressed and Nonprogressed Eyes Based on Stereophotograph Assessment
| Progressors (n = 16) | Nonprogressors (n = 221) | P | |
|---|---|---|---|
| Age (yrs), mean (95% CI)† | 68.1 (53.1–83.2) | 66.1 (41.5–90.7) | 0.11 |
| Right eye (%)* | 8 (50%) | 115 (52%) | 0.84 |
| Baseline abnormal disc by stereophotography (%)* | 13 (81.3%) | 101 (45.7%) | <0.05 |
| Baseline HRT cup disc area ratio† | 0.47 (0.37–0.56) | 0.34 (0.30–0.37) | <0.05 |
| Baseline disc area (mm2)† | 2.07 (1.92–2.22) | 1.95 (1.88–2.03) | 0.29 |
| Baseline MRA any sector ONL (%)* | 13 (81.3%) | 80 (36.2%) | <0.05 |
| Baseline global MRA ONL (%)* | 8 (50%) | 34 (15.4%) | <0.05 |
| Baseline visual field repeatable abnormal result (%)* | 7 (43.8%) | 74 (33.5%) | 0.41 |
| Baseline visual field mean deviation (dB)‡ | −1.17 (−5.45 to −0.29) | −0.87 (−2.33 to −0.07) | 0.44 |
| Baseline visual field PSD (dB)‡ | 2.17 (1.7–9.16) | 1.73 (1.43–2.48) | <0.05 |
CI = confidence intervals
HRT = Heidelberg retina tomograph
MRA = Moorfields regression analysis
ONL = outside normal limits
PSD = pattern standard deviation.
Number of eyes (%) – P value based on chi-square.
Mean (95% CI) – P value based on t test.
Median (Q1–Q3) – P value based on Wilcoxon test.
Topographic Change Analysis
All TCA reports available at follow-ups were selected and displayed for simultaneous examination. The observer examined each of the available reflectivity maps, with red and green colors showing significantly changed superpixels (with red demonstrating depression and green demonstrating elevation) compared with baseline. As a general rule, progression was defined based on the subjective evaluation of each image and in consideration of the presence, repeatability and distribution of green and red superpixels (Fig 1b).
Trend Analysis
One printout with the stereometric progression chart showing the changes over time in the average of several HRT parameters combined (rim area, rim volume, cup volume, cup shape, mean retinal nerve fiber layer (RNFL) thickness, mean height of contour, mean contour elevation, contour line modulation temporal, mean cup depth, and mean height inside contour line) was used for each patient and was evaluated for the combination G, S, I: HRT defined Global, Superior sector, and Inferior sector. The TA grading was conducted based on the subjective evaluation of the trend displayed (Fig 1c).
The same baseline and follow-up HRT scans were included for each analysis. The observers graded for progression based on a progression scale ranging from 1 (no progression) to 4 (progression), with 2 and 3 being possible and probable progression, respectively. A similar scale was applied separately for the superior and inferior hemifields. Observers were then asked to record the presence of progression as a forced choice of progression or no progression. The overall assessment was based on majority rule, with ≥2 graders agreeing on the classification.
Stereophotography
Simultaneous stereophotographs were obtained annually after maximal pupil dilation using a Topcon camera. Baseline photos were taken on the date of HRT imaging for >50% of patients. Each stereophotograph was graded as glaucomatous or normal based on the presence or absence of neuro-retinal rim thinning, RNFL thinning (focal or diffuse), or excavation and/or undermining of the cup characteristic of glaucoma. Also, cup-to-disc ratio asymmetry of >0.2 was noted, in which case the eye with the greater cup-to-disc was considered glaucomatous if the discs were similarly sized. To detect progressive glaucomatous optic neuropathy, the photograph closest to the baseline HRT date was selected and paired with the most recent follow-up photograph. Classifications were assigned as progression or no progression based on the presence or absence of increased rim thinning, excavation, or RNFL defects (new or enlarged). Each photograph was graded by 2 experienced graders using a stereoscopic viewer (Asahi Pentax StereoViewer II; Asahi Optical Co, Tokyo, Japan) and a standard fluorescent light box.
The temporal order of each progression pair was unmasked after final assessment. Graders were masked to patient identity, date, diagnosis, and other graders' results. Disagreements in grading were resolved by adjudication by a third experienced grader.
Visual Field Testing
A visual field was considered abnormal if repeatable abnormal SAP results on 2 consecutive examinations were present with either pattern standard deviation >5% of normal or Glaucoma Hemifield Test outside normal limits (both defined based on the instrument's normative database).
Statistical Analysis
Statistical analysis was performed using SPSS (version 12.0; SPSS Inc., Chicago, IL) and MedCalc (version 8.0.1.0 MedCalc Software, Belgium) statistical software. The κ test was used to evaluate the interobserver agreement; κ<0.20 was considered poor agreement, κ = 0.21– 0.40 was considered fair agreement, κ = 0.41– 0.60 was considered moderate agreement, κ = 0.61– 0.80 was considered good agreement, and κ = 0.81–1.00 was considered very good agreement.23
Agreement between the majority rule classification (progression or no progression) and the current available reference standard (progression based on stereophotographs) was also calculated. In addition, eyes with 100% agreement among observers for progression were compared among methods.
Results
A total of 237 eyes of 168 patients met the inclusion/exclusion criteria and were included in the analysis. Of these, 50 eyes were glaucomatous, defined at baseline as those with glaucomatous appearing optic disc by stereophotograph assessment and consecutive and repeatable abnormal SAP results; 187 were considered glaucoma suspects based on an abnormal-appearing optic disc by masked stereophotograph assessment or an abnormal visual field or based on intraocular pressure >22 mmHg on ≥2 visits, with healthy appearing optic discs without repeatable abnormal SAP results. The average (SD) follow-up time was 46.8 (14.2) months, with a minimum of 4 HRT images (range, 4–8). Of 237 eyes, 16 (6.8%) were found to progress during follow-up based on optic disc stereophotographs grading. Five of these 16 eyes required adjudication for progression and 13 of 221 eyes required adjudication for nonprogression by a third experienced grader, for an agreement of 92.4% (κ = 0.51) between the 2 primary graders. After unmasking the temporal order of each progression, 1 disc was found to have a rim appearance improved over time, according to both graders. This case was considered as nonprogression.
Table 1 summarizes the characteristics of progressed and non-progressed eyes. The κ values for agreement among different observers for all methods of analysis evaluated are summarized in Table 2. In general, the agreement was found to be moderate to good (κ values ranging from 0.52 to 0.71 for MRA, 0.61 to 0.63 for TCA, and 0.45 to 0.74 for TA), with good agreement shown among different observers for TCA, and moderate to good agreement found for the TA and MRA analysis. In general, for each method, 100% agreement was reached when change was noticeable in consecutive follow-up printouts and the damage was consistent and within the same location of the disc (e.g., for TCA, a significant cluster of red pixels shown in the same location at follow-up examinations; for TA, a consistent downward trend in ≥1 sectors as displayed in the printout graph). For the MRA, agreement for progression among all observers was reached only in eyes in which the change prompted the overall or sectors MRA classification to switch consistently from within normal limits to borderline or outside normal limits in follow-up scans. Table 3 summarizes the results obtained by computing the number of eyes (out of the total 237 examined) with all 3 graders in agreement for progression and for no progression for the different HRT methods of analysis. Based on majority rule assessment, TCA had 42 progressing eyes, MRA 30 progressing eyes, and TA 26 progressing eyes. The majority of eyes were defined as progressing by TCA by consensus, with an agreement among observers of 100% in 59.5% of the cases (25 out of the 42 progressed eyes). Seventeen eyes (56.7% of cases) graded using the MRA and 15 (57.7% of cases) using TA also met full agreement among observers for the presence of glaucomatous progression.
Table 2.
Kappa Values and Standard Error (SE) for Agreement among Different Observers for the Heidelberg Retina Tomograph Methods of Analysis for Detecting Progression
| Agreement between Observers | Kappa | SE |
|---|---|---|
| Moorfields regression analysis | ||
| A vs B | 0.52 | 0.08 |
| A vs C | 0.61 | 0.08 |
| B vs C | 0.71 | 0.06 |
| Topographic change analysis | ||
| A vs B | 0.61 | 0.07 |
| A vs C | 0.63 | 0.07 |
| B vs C | 0.62 | 0.06 |
| Trend analysis | ||
| A vs B | 0.45 | 0.09 |
| A vs C | 0.74 | 0.07 |
| B vs C | 0.58 | 0.08 |
Table 3.
Number of Eyes (N = 237) in Agreement for Progression and No Progression for Each Heidelberg Retina Tomograph Method of Analysis
| Moorfields Regression Analysis | Topographic Change Analysis | Trend Analysis | |
|---|---|---|---|
| Majority rule (2/3) | |||
| Progression | 30 | 42 | 26 |
| No progression | 207 | 195 | 211 |
| Complete agreement (3/3) | |||
| Progression | 17 | 25 | 15 |
| No progression | 189 | 173 | 191 |
Table 4 shows the agreement among the different methods of analysis for progression and no progression defined by majority rule and for progression and no progression based on 100% agreement among observers for each method. Using majority rule, the 3 methods were consistent with the presence of progression in 8 eyes and no progression in 170 eyes and, using 100% agreement, 5 eyes were identified as progressing and 138 as nonprogressing.
Table 4.
Number of Eyes (N = 237) in Agreement for Progression and no Progression between Heidelberg Retina Tomograph Methods
| Majority Rule (2/3) |
Complete Agreement (3/3) |
|||
|---|---|---|---|---|
| Progression | No Progression | Progression | No Progression | |
| MRA + TCA + TA | 8 | 170 | 5 | 138 |
| MRA + TA | 15 | 180 | 7 | 151 |
| MRA + TA | 14 | 195 | 6 | 172 |
| TCA + TA | 10 | 179 | 6 | 151 |
HRT = Heidelberg retina tomograph
MRA = Moorfields regression analysis
TCA = topographic change analysis
TA = trend analysis.
Figure 2 shows the agreement for progression for each of the different methods of analysis examined with progression defined by majority rule and the reference standard, represented by progression by stereophotographs. Specifically, in 82.3% of eyes, there was agreement between TCA and stereophotography (agreement on 8 of 16 progressing eyes and 187 of 221 nonprogressing eyes) resulting in a κ value of 0.2. On TCA, progression was detected in 34 (14.3%) eyes that were not progressed by stereophotography and 8 (3.4%) eyes were classified as progressed by stereophotography, but not by TCA. Assessment of MRA agreed on 196 of the nonprogressing eyes and 5 of the progressing eyes based on stereophotographs for an overall agreement of 84.8% and a κ value of 0.14. Assessment of TA agreed on 197 of the nonprogressing eyes and 2 of the progressing eyes based on stereophotographs for an overall agreement of 84% and a κ value of 0.01. Only in 2 eyes there was full agreement between all the methods of analysis and the gold standard for the presence of progression.
Figure 2.
Venn diagram showing the agreement on progression between Moorfields regression analysis (MRA), topographic change analysis (TCA), and trend analysis (TA) and the reference standard. Progression for the MRA, TCA, and TA was defined based on majority rule. The TA is based on the normalized “average” parameter defined as the average of several HRT parameters combined (rim area, rim volume, cup volume, cup shape, mean retinal nerve fiber layer thickness, mean height of contour, mean contour elevation, contour line modulation temporal, mean cup depth, and mean height inside contour line). HRT = Heidelberg retina tomograph.
Figures 3, 4, and 5 show 3 examples of the MRA series, TCA, and TA graphs graded by clinicians to detect glaucomatous progression in this study. In Figure 3, all 3 HRT methods agreed on progression with 100% agreement among observers in accordance with stereophotography, whereas in Figure 4 progression was detected by stereophotography, but not by HRT. Figure 5 shows progression by TCA by majority rule that was not confirmed by stereophotography.
Figure 3.
This eye was classified as progressed by Moorfields regression analysis (MRA), topographic change analysis (TCA), and trend analysis (TA) with 100% agreement among observers. Glaucomatous progression was also confirmed by stereophotographs assessment, as evidenced by the presence of an enlarged retinal nerve fiber layer (RNFL) defect (indicated by the 2 arrows pointing toward each other at the boundary of the RNFL defect) with kinking of the vessel in the proximity of a disc hemorrhage (indicated by the arrow in the follow-up image). The TA is based on the normalized “average” parameter defined as the average of several HRT parameters combined (rim area, rim volume, cup volume, cup shape, mean RNFL thickness, mean height of contour, mean contour elevation, contour line modulation temporal, mean cup depth and mean height inside contour line). HRT = Heidelberg Retina Tomograph.
Figure 4.
This eye was classified as stable by Moorfields regression analysis (MRA), topographic change analysis (TCA), and trend analysis (TA), with all graders agreeing on no progression. However, glaucomatous progression was detected by the appearance of a new retinal nerve fiber layer (RNFL) defect in the inferotemporal region in the 2007 stereophotograph, in addition to the preexisting defect in the superotemporal region (both RNFL defects are indicated by the 2 arrows pointing towards each other at the boundary of the RNFL defect). This new RNFL defect can be detected on the HRT MRA and TCA (though not as a statistically significant change), but did not result in classification of progressed by any grader. The TA is based on the normalized “average” parameter defined as the average of several HRT parameters combined (rim area, rim volume, cup volume, cup shape, mean RNFL thickness, mean height of contour, mean contour elevation, contour line modulation temporal, mean cup depth and mean height inside contour line). HRT = Heidelberg Retina Tomograph.
Figure 5.
This eye was classified as progressed by topographic change analysis (TCA) based on majority rule based on the neuroretinal rim loss in the inferior region. No progression was detected using Moorfields regression analysis (MRA), trend analysis (TA), or by stereophotography. The TA is based on the normalized “average” parameter defined as the average of several HRT parameters combined (rim area, rim volume, cup volume, cup shape, mean retinal nerve fiber layer thickness, mean height of contour, mean contour elevation, contour line modulation temporal, mean cup depth and mean height inside contour line). HRT = Heidelberg Retina Tomograph.
Discussion
These results show that, in general, agreement among clinicians when assessing progression using the HRT was moderate to good. For the TCA, it was consistently good across all graders. The 2 methods of analysis specifically designed to detect progression, the TCA and the TA, performed similarly to MRA with regard to clinicians agreement (i.e., MRA was as good in terms of agreement among observers, as the 2 methods designed to detect progression).
In an attempt to replicate what normally occurs in practice when clinicians evaluate an HRT examination to detect glaucomatous structural change, specific criteria were not established for each method of analysis. Rather, interpretation of the images, and deciding whether or not there was progression, was based on clinical judgment. The observers that participated in this study were all clinicians experienced with the use of the HRT and the interpretation of topographic images. This study suggests that the clinical interpretation of the HRT results for detecting progression is similar among experienced clinicians, even in the absence of predefined criteria and masked to patients' clinical history and the results of other tests.
Previous studies have shown that the HRT, because of its ability to perform topographic analysis of the disc, can be helpful in determining glaucomatous progression.14–18 However, how progression should be identified using disc topographies and which HRT features should be used for this purpose still remains to be established. The interobserver agreement for evaluation of TCA and TA was good for TCA and ranged from moderate to good for TA. This is not surprising, because change is relatively easy to detect in the TCA and TA printouts, which are specifically designed for longitudinal follow-up. Although there was good agreement among the observers, it should be noted that the absence of predefined criteria may have limited the degree of their concordance. Although well informed on the current suggested criteria for detecting change using the TCA,24 observers qualitatively identified change by considering the location and distribution and size of the red pixel cluster. Similarly, for the TA, the reviewers subjectively assessed the graph for identification of change because no formal regression analysis for determination of statistically significant change is currently available.
In addition, in this study an average of 4 HRT images was available for the detection of change; it is possible that a greater number of follow-ups may yield a better agreement among observers, because a consistent downward trend in the TA, for instance, in several consecutive examinations can be more suggestive of change.
Interobserver agreement for the MRA was similar to that of TCA or TA. However, MRA was not designed to detect progression and the subjective evaluation using MRA mainly relies on identifying coherent changes in the sectoral and/or overall classification as normal, borderline, or outside normal limits across multiple visits. The observer may assess the existence of progression by looking at whether the disc and/or the sectors, previously classified as normal, have shifted to borderline or outside normal limits in follow-up examinations. However, baseline characteristics of the patients examined may influence the performance of MRA. In this study, 8 out of 16 eyes (50%) that progressed based on stereophotographs had a baseline global MRA classification of outside normal limits and 13 (81.3%) ≥1 sector outside normal limits (Table 1), making it difficult for the observer to detect significant topographic changes in the absence of additional changes in the MRA classification. As expected, agreement with stereophotographs in eyes with more advanced glaucomatous damage as evidenced by the overall MRA classification outside normal limits at baseline was worse for MRA, whereas it was better for TCA (κ values in eyes with overall MRA classification outside normal limits, borderline, or within normal limits were 0.01, 0.2, and 0.18 respectively for MRA and 0.3, 0.16, and 0.13 respectively for TCA, with agreement for progression based on majority rule). In addition, MRA assessment across visits is time consuming and requires several printouts to be available for simultaneous inspection as opposed to 1 single printout necessary for TCA and TA.
When comparing MRA, TCA, and TA for detecting progression, results show a poor concordance between eyes that progressed, particularly when 100% agreement for progression among observers is used for comparison (Table 4). One might argue that these methods should be used in conjunction to improve the sensitivity of HRT. However, Figure 2 shows that only MRA made little contribution (1 eye) in identifying eyes progressed by stereophotography. This suggests that, by adding MRA and TA to the TCA analysis, the sensitivity of HRT is unlikely to increase when using stereophotography as the gold standard.
The results of this study show that the concordance between the different HRT methods of analysis to detect progression and the currently accepted gold standard, stereophotography, was similar for MRA (85%), TCA (82%), and TA (84%). Previous studies have reported similar agreement for progression by TCA and stereophotographs. Chauhan et al14 reported a concordance of 81% with only 1 patient showing disc progression with photography alone, and Kourkoutas et al18 reported an agreement for progression of 65% with 16 patients (30%) showing progression on HRT only and 3 patients (6%) showing progression on stereophotographs only. These results are comparable with the 82.3% overall agreement by majority rule found for TCA in this study, with 8 (3.4%) eyes progressing by stereophotographs only and 34 (14.3%) eyes progressing by TCA only. However, the κ found is small, indicating poor agreement. This is not surprising, as κ is affected by the overall prevalence, that is, the small number of progressing eyes found in the study population leads to symmetrical imbalances of the marginal totals (referred to as the first paradox of κ25). Although previous studies did not use κ statistics, in this study the limited agreement in the small number of progressing eyes resulted in a very low κ values.
To better evaluate our findings, the 8 eyes that progressed by stereophotographs only and not by TCA underwent a retrospective evaluation to identify possible reasons for disagreement between TCA and stereophotographs. In 3 eyes, progression on stereophotographs only was established by the appearance of 2 new RNFL defects and by the enlargement of a preexisting one. The change in the TCA map in these cases was insufficient for clinicians to judge the eyes as progressed. Figure 4 shows an eye in which the appearance of a new RNFL defect was evident in the follow-up stereophotograph, but not on the TCA printout. In the remaining 5 eyes, progression by stereophotographs was defined based on the presence of increased rim thinning compared with baseline. When examining the TCA maps, TCA did show some change in areas that corresponded to rim thinning on stereophotographs, but this change was modest, and the majority of the red pixels were located either over the vessels or outside the disc margin.
Of the 34 eyes that progressed by HRT only, 22 (65%) eyes were progressed based on 100% agreement among observers, indicating that the TCA map was suggestive of change, whereas 12 (35%) eyes were judged progressed based on majority rule. In the latter eyes, changes were consistent in follow-up scans, but some of the red clusters were positioned either outside the disc margin or immediately adjacent to the vessels, making it difficult for the observer to rule out the existence of progression. An example of such a case is provided in Figure 5. However, the concordance among observers for the hemifields judged as progressed (superior, inferior, or both) was high and was similar for the eyes progressed by both TCA and stereophotography and those progressed by TCA only, with the observers concordant in 87.5% and 85.3% of the cases, respectively. Interestingly, of all the eyes progressed by HRT only, only 1 required stereophotographs adjudication for nonprogression by a third experienced grader, suggesting that stereophotographs were indicative of no change. It is possible that in some of these cases HRT detected progression that could not be seen on the photos, as suggested by a recent study by Bowd et al.26
It has been suggested that the presence of documented progressive change in the optic disc appearance as assessed by stereophotography currently is the best available reference standard for glaucoma diagnosis.27 Because there is no perfect reference standard for glaucomatous structural changes, these results suggest that, in some eyes, CSLO detects change before stereophotographs, whereas in other eyes stereophotographs may identify subtle changes otherwise undetected by clinicians who rely exclusively on current CSLO techniques. It is possible that, with longer follow-up, the changes detected by TCA only or stereophotography only will at some point be detected by both techniques.
One should also be reminded that these changes might not represent the loss of retinal ganglion cells and their axons, which is ultimately the true indicator for glaucomatous progression. By accepting optic disc stereophotography as the gold standard for comparison, it follows that the diagnostic accuracy of any other diagnostic test in glaucoma, including CSLO, will always be lower than that of optic disc stereophotography.
In conclusion, clinicians' agreement in establishing glaucomatous progression using different HRT methods of analysis was moderate to good. Based on agreement by majority rule, TCA alone showed change in half of the eyes progressed by stereophotographs. Further studies are needed to determine predefined objective criteria for detecting change over time when assessing HRT progression in clinical practice. These results suggest that stereophotography and HRT may be identifying different aspects of structural change. Further research and longer follow-up are needed to identify what structural features are best indicators of change that reflects true disease progression.
Acknowledgments
Supported by NIH EY011008, NIH EY008208 and participant incentive grants in the form of glaucoma medication at no cost from Alcon Laboratories Inc., Allergan, Pfizer Inc., and SANTEN Inc.
Footnotes
Financial Disclosure(s): Carl Zeiss Meditec, FAM (F, R) PAS (F), RNW (F, C), LMZ (F); Haag-Streit, PAS (F); Heidelberg Engineering, MB (F), FAM (F), RNW (F), LMZ (F); Lace Elettronica, CB (F); Optovue, LMZ (F); Reichart Instruments, FAM (R); and Welch-Allyn, PAS (F).
References
- 1.Weinreb RN. Laser scanning tomography to diagnose and monitor glaucoma. Curr Opin Ophthalmol. 1993;4:3–6. [PubMed] [Google Scholar]
- 2.Cioffi GA, Robin AL, Eastman RD, et al. Confocal laser scanning ophthalmoscope: reproducibility of optic nerve head topographic measurements with the confocal laser scanning ophthalmoscope. Ophthalmology. 1993;100:57–62. [PubMed] [Google Scholar]
- 3.Rohrschneider K, Burk RO, Kruse FE, Volcker HE. Reproducibility of the optic nerve head topography with a new laser tomographic scanning device. Ophthalmology. 1994;101:1044–9. doi: 10.1016/s0161-6420(94)31220-6. [DOI] [PubMed] [Google Scholar]
- 4.Mikelberg FS, Parfitt CM, Swindale NV, et al. Ability of the Heidelberg retina tomograph to detect early glaucomatous visual field loss. J Glaucoma. 1995;4:242–7. [PubMed] [Google Scholar]
- 5.Zangwill LM, van Horn S, de Souza Lima M, et al. Optic nerve head topography in ocular hypertensive eyes using confocal scanning laser ophthalmoscopy. Am J Ophthalmol. 1996;122:520–5. doi: 10.1016/s0002-9394(14)72112-9. [DOI] [PubMed] [Google Scholar]
- 6.Uchida H, Brigatti L, Caprioli J. Detection of structural damage from glaucoma with confocal laser image analysis. Invest Ophthalmol Vis Sci. 1996;37:2393–401. [PubMed] [Google Scholar]
- 7.Hatch WV, Flanagan JG, Etchells EE, et al. Laser scanning tomography of the optic nerve head in ocular hypertension and glaucoma. Br J Ophthalmol. 1997;81:871–6. doi: 10.1136/bjo.81.10.871. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wollstein G, Garway-Heath DF, Hitchings RA. Identification of early glaucoma cases with the scanning laser ophthalmoscope. Ophthalmology. 1998;105:1557–63. doi: 10.1016/S0161-6420(98)98047-2. [DOI] [PubMed] [Google Scholar]
- 9.Zangwill LM, Bowd C, Berry CC, et al. Discriminating between normal and glaucomatous eyes using the Heidelberg Retina Tomograph, GDx Nerve Fiber Analyzer, and Optical Coherence Tomograph. Arch Ophthalmol. 2001;119:985–93. doi: 10.1001/archopht.119.7.985. [DOI] [PubMed] [Google Scholar]
- 10.Sanchez-Galeana C, Bowd C, Blumenthal EZ, et al. Using optical imaging summary data to detect glaucoma. Ophthalmology. 2001;108:1812–8. doi: 10.1016/s0161-6420(01)00768-0. [DOI] [PubMed] [Google Scholar]
- 11.Medeiros FA, Zangwill LM, Bowd C, et al. Influence of disease severity and optic disc size on the diagnostic performance of imaging instruments in glaucoma. Invest Ophthalmol Vis Sci. 2006;47:1008–15. doi: 10.1167/iovs.05-1133. [DOI] [PubMed] [Google Scholar]
- 12.Bowd C, Zangwill LM, Medeiros FA, et al. Structure-function relationships using confocal scanning laser ophthalmoscopy, optical coherence tomography, and scanning laser polarimetry. Invest Ophthalmol Vis Sci. 2006;47:2889–95. doi: 10.1167/iovs.05-1489. [DOI] [PubMed] [Google Scholar]
- 13.Pueyo V, Polo V, Larrosa JM, et al. Diagnostic ability of the Heidelberg retina tomograph, optical coherence tomograph, and scanning laser polarimeter in open-angle glaucoma. J Glaucoma. 2007;16:173–7. doi: 10.1097/IJG.0b013e31802dfc1d. [DOI] [PubMed] [Google Scholar]
- 14.Chauhan BC, McCormick TA, Nicolela MT, LeBlanc RP. Optic disc and visual field changes in a prospective longitudinal study of patients with glaucoma: comparison of scanning laser tomography with conventional perimetry and optic disc photography. Arch Ophthalmol. 2001;119:1492–9. doi: 10.1001/archopht.119.10.1492. [DOI] [PubMed] [Google Scholar]
- 15.Artes PH, Chauhan BC. Longitudinal changes in the visual field and optic disc in glaucoma. Prog Retin Eye Res. 2005;24:333–54. doi: 10.1016/j.preteyeres.2004.10.002. [DOI] [PubMed] [Google Scholar]
- 16.Patterson AJ, Garway-Heath DF, Strouthidis NG, Crabb DP. A new statistical approach for quantifying change in series of retinal and optic nerve head topography images. Invest Ophthalmol Vis Sci. 2005;46:1659–67. doi: 10.1167/iovs.04-0953. [DOI] [PubMed] [Google Scholar]
- 17.Kalaboukhova L, Fridhammar V, Lindblom B. Glaucoma follow-up by the Heidelberg retina tomograph—new graphical analysis of optic disc topography changes. Graefes Arch Clin Exp Ophthalmol. 2006;244:654–62. doi: 10.1007/s00417-005-0107-3. [DOI] [PubMed] [Google Scholar]
- 18.Kourkoutas D, Buys YM, Flanagan JG, et al. Comparison of glaucoma progression evaluated with Heidelberg retina tomograph II versus optic nerve head stereophotographs. Can J Ophthalmol. 2007;42:82–8. [PubMed] [Google Scholar]
- 19.Chauhan BC, Blanchard JW, Hamilton DC, LeBlanc RP. Technique for detecting serial topographic changes in the optic disc and peripapillary retina using scanning laser tomography. Invest Ophthalmol Vis Sci. 2000;41:775–82. [PubMed] [Google Scholar]
- 20.Werner EB, Bishop KI, Koelle J, et al. A comparison of experienced clinical observers and statistical tests in detection of progressive visual field loss in glaucoma using automated perimetry. Arch Ophthalmol. 1988;106:619–23. doi: 10.1001/archopht.1988.01060130673024. [DOI] [PubMed] [Google Scholar]
- 21.Heijl A, Leske MC, Bengtsson B, et al. Early Manifest Glaucoma Trial Group. Reduction of intraocular pressure and glaucoma progression: results from the Early Manifest Glaucoma Trial. Arch Ophthalmol. 2002;120:1268–79. doi: 10.1001/archopht.120.10.1268. [DOI] [PubMed] [Google Scholar]
- 22.Collaborative Normal-Tension Glaucoma Study Group Comparison of glaucomatous progression between untreated patients with normal-tension glaucoma and patients with therapeutically reduced intraocular pressures. Am J Ophthalmol. 1998;126:487–97. doi: 10.1016/s0002-9394(98)00223-2. [DOI] [PubMed] [Google Scholar]
- 23.Cohen J. A coefficient for agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46. [Google Scholar]
- 24.HRT. Heidelberg Engineering; Dossenheim, Germany: How to interpret progression. year:xx–xx. Available at: http://www.heidelbergengineering.com/wp-content/uploads/hrttutqg_doc_howtointerpretprogression_031705.pdf. Accessed August 11, 2005. [Google Scholar]
- 25.Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43:543–9. doi: 10.1016/0895-4356(90)90158-l. [DOI] [PubMed] [Google Scholar]
- 26.Bowd C, Balasubramanian M, Weinreb RN, et al. Performance of confocal scanning laser tomograph topographic change analysis (TCA) for assessing glaucomatous progression. Invest Ophthalmol Vis Sci. 2008 Oct 3; doi: 10.1167/iovs.08-2136. epub ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Coleman A, Friedman D, Gandolfi S, et al. Levels of evidence of diagnostic studies. In: Weinreb RN, Greve EL, editors. Glaucoma Diagnosis Structure and Function: Reports and consensus statements of the 1st Global AIGS Meeting. vol. 1. Kugler; The Hague, The Netherlands: 2004. pp. 9–12. Consensus Series. Association of International Glaucoma Societies. [Google Scholar]