The Relative Odds of Progressing by Structural and Functional Tests in Glaucoma

Abstract

Purpose

The purpose of this study was to evaluate the effect of disease severity and number of tests acquired during follow-up on the relative odds of identifying progression by structural or functional tests in glaucoma.

Methods

This was an observational cohort study involving 462 eyes of 305 patients with glaucoma and 62 eyes of 49 healthy subjects. Glaucoma patients and healthy subjects were followed for an average of 3.6 ± 0.9 and 3.8 ± 0.9 years, with a median (interquantile range) of 8 (6–9) and 7 (6–8) visits, respectively. At each visit, subjects underwent visual field assessment with standard automated perimetry (SAP) and retinal nerve fiber layer (RNFL) evaluation by spectral-domain optical coherence tomography (SD-OCT). Slopes of change in SAP mean sensitivity and OCT RNFL thickness over time were estimated by linear regression using progressively cumulative visits over time. Cutoff values for age-related expected rates of change for each test were obtained from the healthy group. Progression by SD-OCT and/or SAP was determined if the slope of change was statistically significant and also lower (faster) than the fifth percentile cutoff calculated from the healthy group. A generalized estimating equation logistic regression model was used to evaluate the relative odds of progressing by OCT versus SAP in glaucoma eyes.

Results

Eyes with less severe disease at baseline had a higher chance of being detected as progressing by SD-OCT but not by SAP, whereas an increase in disease severity at baseline increased the chance that the eye would be detected as progressing by SAP but not SD-OCT. Each 1 dB higher MD was associated with a 5% increase in the odds of detecting progression by SD-OCT versus SAP (odds ratio = 1.05 per 1 dB; 95% confidence interval: 1.01–1.09; P = 0.005).

Conclusions

The ability to detect glaucoma progression by SAP versus SD-OCT is significantly influenced by the stage of disease. Our results may provide useful information for guiding clinicians on the relative utility of these tests for detecting change throughout the disease continuum.

Glaucoma is a progressive optic neuropathy associated with loss of retinal ganglion cells, leading to characteristic optic nerve and retinal nerve fiber layer (RNFL) changes and loss of visual function.¹ Reduction of intraocular pressure has been shown to halt or slow down the rate of deterioration; therefore, detection of progressive damage is essential in management decisions about when to initiate or advance therapy, as well as in determining response to treatment.^2,3

Despite its fundamental importance in the management of the disease, there is no consensus with regard to the best method for detecting progression and measuring rates of change in glaucoma. Although standard automated perimetry (SAP) is the most widely used method for assessing functional loss, many patients can show progressive structural changes to the optic disc and RNFL despite the absence of detectable changes on standard perimetry. These structural losses are usually quantified by imaging technologies, such as confocal scanning laser ophthalmoscopy, scanning laser polarimetry, or optical coherence tomography (OCT).^3–5 Structural changes have been shown to be predictive of future functional losses and to be associated with a decrease in quality of life, carrying important prognostic significance for the patient.^6,7 However, a large proportion of patients with glaucoma may also show progressive functional loss on SAP in the absence of detectable structural change.⁸ Although several studies have attempted to clarify the reasons for this apparent disagreement, it is still unclear in what circumstances detection of progression is facilitated by one or the other method.

The ability of a specific test to detect glaucomatous progression may depend on several factors, such as stage of the disease, test–retest variability, and number or frequency of tests acquired during follow-up.^9,10 Previous studies have suggested that structural tests may perform better for detecting progression at earlier stages of damage, whereas functional ones may perform better at later stages.¹¹ However, the relationship between stage of disease and ability to detect progression has not been well characterized and quantified in the literature. In addition, the influence of number of tests acquired during follow-up on the relative ability to detect progression by different structural and functional tests has not been reported. Although increasing the number of tests will generally lead to higher chances of detecting progressive damage, the increase in the probability of detecting progression may be different according to the type of test being using and the stage of the disease.¹² These are important considerations as an understanding and quantification of such relationships may provide a better allocation of resources by allowing clinicians to tailor the frequency of testing by one or the other method according to the stage of disease and the expected probability that the test will be able to detect progressive damage. Such a strategy could help maximize the chances of detecting progression while minimizing the costs associated with repeated testing.

In the current study, we evaluated the influence of factors such as stage of disease and number of tests on the relative ability of SAP and spectral-domain (SD)-OCT in detecting progression in a cohort of glaucoma patients followed over time.

Methods

This was an observational study. Participants from this study were included in a prospective longitudinal study designed to evaluate optic nerve structure and visual function in glaucoma (the Diagnostic Innovations in Glaucoma Study) conducted at the Hamilton Glaucoma Center at the Department of Ophthalmology, University of California, San Diego (UCSD). The institutional review board approved the study methodology, which adhered to the tenets of the Declaration of Helsinki and to the Health Insurance Portability and Accountability Act.

At each visit during follow-up, patients underwent a comprehensive ophthalmologic examination including review of medical history, best-corrected visual acuity, slit-lamp biomicroscopy, intraocular pressure, gonioscopy, dilated ophthalmoscopic examination, stereoscopic optic disc photograph (Kowa Nonmyd WX3D; Kowa Optimed, Inc., Torrance, CA, USA), SD-OCT testing (Spectralis; Heidelberg Engineering, Dossenheim, Germany), and SAP using the Swedish interactive threshold algorithm (SITA standard 24-2; Carl Zeiss Meditec, Inc., Dublin, CA, USA). Subjects were excluded if they had any ocular or systemic disease that could affect the optic nerve or the visual field.

All patients had a diagnosis of glaucoma at baseline, based on the presence of repeatable visual field defects on SAP and/or glaucomatous optic neuropathy on optic disc stereophotographs. Repeatable visual field defects on SAP were defined as at least three consecutive abnormal SAP results with pattern standard deviation (PSD) with P < 0.05, and/or glaucoma hemifield test results outside normal limits. Glaucomatous optic neuropathy was evaluated by masked assessment of stereophotographs and defined based on the presence of neuroretinal rim thinning, excavation, notching, or characteristic RNFL defects.¹³ Healthy subjects were recruited from the general population through advertisements or from the staff and employees at the UCSD. Healthy subjects had intraocular pressures less than 22 mm Hg with no history of increased intraocular pressure and at least two reliable normal visual fields in both eyes, which were defined as a pattern SD within 95% confidence limits and a glaucoma hemifield test result within normal limits. For inclusion in the analysis, each glaucoma patient or healthy subject was required to have at least five visits (with five SAP tests and five OCT tests) over a follow-up duration of at least 2 years.

Optical Coherence Tomography

The Spectralis SD-OCT (software version 5.4.7.0) was used to measure peripapillary RNFL thickness in the present study. The device has been described in detail previously.^14,15 All images were reviewed by the UCSD Imaging Data Evaluation and Analysis Center to ensure that the scan was centered, the signal strength was more than 15 dB, that there were no artifacts, and that there were no RNFL segmentation algorithm errors. The OCT parameter used in the study was the global RNFL thickness corresponding to the average of all RNFL thickness measurements acquired from a 3.45-mm circle centered on the optic disc consisting of 1536 A-scan points.

Standard Automated Perimetry

Visual fields were performed using SAP SITA 24-2 and evaluated by the UCSD Visual Field Assessment Center (VisFACT).¹³ Visual fields were excluded if they had more than 33% fixation losses or more than 15% false-positive errors. Visual fields were excluded in the presence of the following artifacts: eyelid, rim artifacts, fatigue effects, inappropriate fixation, or evidence that the visual field results were caused by a disease other than glaucoma or inattention. Visual fields exhibiting a learning effect (i.e., initial tests showing consistent improvement on visual field indices) were also excluded. The mean sensitivity (MS) in decibels of each eye was calculated by converting each threshold point into a linear scale (apostilbs) and then averaging all 52 threshold points.¹⁶ The final value was then converted back to logarithmic scale (decibels).

Data Analysis

Global parameters of structural and functional damage were selected for evaluation in this study to provide a fair comparison between the different tests. As comparable localized parameters would be difficult to establish for SAP and OCT, we restrained our analyses to global structural loss as measured by the OCT RNFL average thickness and global functional loss as measured by SAP MS. We used MS instead of mean deviation (MD), as MD is an age-corrected parameter, whereas global RNFL thickness and MS are not.

A challenge in evaluating the relative ability of different tests in detecting glaucoma progression resides in equating their specificities, so that sensitivities can be meaningfully compared. When evaluating progression based on slopes of change, the specificity can be set by the type I error. A statistically significant slope at P < 0.05 implies that in only 5% of the cases the slope would be declared statistically significantly different from zero in cases where no change truly existed. However, such approach may be insufficient to establish specificity due to possible changes over time that can occur from aging, instead of glaucoma progression. Therefore, to take into account possible age effects, we established confidence limits for the rates of change of OCT average RNFL thickness and SAP MS using longitudinal data available from healthy eyes. Slopes of change were obtained using linear mixed models and cutoffs corresponding to the fifth percentile of rates of change in the healthy group were obtained for each parameter.¹⁷ It is expected that 95% of healthy eyes will have rates of change slower (i.e., more positive) than the cutoff value. After cutoffs were obtained from the healthy group, individual slopes were calculated for each glaucoma eye and for each test using ordinary least squares regression. Progression was then determined to have occurred if the slope was statistically significant at P < 0.05 and also faster (i.e., more negative) than the fifth percentile cutoff calculated from the healthy group. This approach equated the specificities of the tests by requiring statistically significant slopes at the same type I error and also by taking into account age-related changes by establishing confidence limits from the healthy group.

For each glaucoma eye, slopes of change for each test were calculated using progressively cumulative visits over time, with a minimum of five visits. For example, for an eye that had a total of 10 visits during follow-up, slopes of change were calculated from data including only the first 5 visits, as well as from the data including the first 6 visits, first 7 visits, and so on, up to 10 visits. Therefore, for an eye with 10 visits during follow-up, six slopes were calculated for each test. Progression (yes/no) was then determined based on the criteria set on the previous paragraph for each one of these slopes.

After progression (yes/no) was determined at each visit, a logistic regression model was used to evaluate the relative odds of progressing by OCT and SAP in the population.¹⁸ For the logistic regression, eyes that progressed exclusively by OCT were assigned the value 1, whereas eyes that progressed exclusively by SAP were assigned the value 0. As we were interested in evaluating the factors associated with progressing by one method versus the other, eyes that progressed by both methods or by none were excluded. As each eye could have had multiple slopes for each test depending on the number of visits during follow-up, each eye had a combination of 1s and 0s during follow-up, depending on whether progression was detected solely by OCT or by SAP at each point during follow-up. A generalized estimating equation (GEE) model was used to deal with correlated observations from each eye.¹⁹ The GEE logistic model evaluated the effect of the number of tests during follow-up and disease severity at baseline on the relative odds of progressing by OCT versus SAP. Baseline disease severity was classified according to SAP MD in early (>−6 dB), moderate (<−6 and >−12 dB), and advanced (<−12 dB).

Statistical analyses were performed using commercially available software Stata, version 13 (StataCorp LP, College Station, TX, USA). The α level (type I error) was set at 0.05.

Results

The study included 462 eyes of 305 glaucoma patients and 62 eyes of 49 healthy controls. Table 1 shows the baseline clinical and demographic characteristics of the subjects included in the study. Glaucoma patients had a median of 8 visits (range, 5–18) during an average follow-up of 3.6 ± 0.9 years. For healthy subjects, the median number of visits was 7 (range, 5–10), over an average follow-up period of 3.8 ± 0.9 years. Mean age at baseline was 70.0 ± 8.2 years in the glaucoma group and 69.1 ± 8.5 years in the healthy group (P = 0.470). Average SAP MD was −4.8 ± 5.9 and 0.0 ± 1.2 dB, and global RNFL thickness at baseline was 78.4 ± 17.6 and 94.9 ± 11.0 μm, in glaucoma and healthy subjects, respectively.

Table 1.

Baseline Clinical and Demographic Characteristics of Healthy Subjects and Glaucoma Patients Included in the Study

As the number of visits for glaucoma eyes ranged from 5 to 18 visits, the number of slopes ranged from 1 to 14 for each eye. There were in total 1910 slopes for all the glaucomatous eyes included in the study. The fifth percentile cutoff values for rates of change estimated from the healthy group were −0.35 dB/y for SAP MS and −1.48 μm/y for global RNFL thickness. Progression was determined for each eye and each test if the slope was faster than these cutoff values and also if the slope was statistically significant at P < 0.05. From the 1910 slopes, progression was detected solely by OCT in 363 slopes (19.0%), solely by SAP in 183 slopes (9.5%), and by both OCT and SAP in 80 slopes (4.1%). Progression was not detected in 1284 slopes (67.2%). As we were interested in factors explaining the relative odds of progressing by one test versus the other, subsequent analyses were conducted with the 546 slopes that showed progression exclusively by OCT or exclusively by SAP.

Table 2 shows results of progression by OCT versus SAP according to the number of visits and baseline disease severity. The raw data suggested that OCT was able to detect more eyes as progressing in comparison to SAP, except for eyes with moderate and severe disease and when a large number of tests were available (Fig. 1).

Table 2.

Total Number of Visits With Progression Detected by SAP and SD-OCT, According to Baseline Disease Severity and the Number of Visits During Follow-Up

Figure 1.

Estimated probabilities of detecting progression by SD-OCT versus SAP, according to different levels of disease severity at baseline and number of tests acquired during follow-up.

This was confirmed by the results of the GEE logistic model, as shown in Table 3. Each 1 dB higher MD was associated with a 5% increase in the odds of detecting progression by OCT versus SAP (odds ratio [OR] = 1.05 per 1 dB; 95% confidence interval [CI]: 1.01–1.09; P = 0.005). That is, eyes with less severe disease at baseline had higher chance of being detected as progressing by OCT but not by SAP, whereas an increase in disease severity at baseline increased the chance that the eye would be detected as progressing by SAP but not OCT. A higher number of visits during follow-up was associated with a decrease in the relative odds of progressing by OCT versus SAP (OR = 0.86 per visit; 95% CI: 0.81–0.93; P < 0.001). Although an increase in the number of visits would be expected to increase the chance of detecting progression by both OCT and SAP, it is important to keep in mind that the model evaluated the odds of progressing by one test versus the other. The results show that for eyes with few visits during follow-up, OCT had a higher chance of detecting progression compared with SAP. Importantly, there was also a significant interaction between baseline disease severity and number of visits in explaining the relative odds of progressing by OCT versus SAP. For eyes with early disease and relatively small number of visits, the odds of detecting progression by OCT were in general much larger for OCT compared with SAP (Fig. 1). Conversely, for eyes with advanced disease and relatively large number of visits available during follow-up, the odds of detecting progression by SAP were greater than by OCT. Figures 2 and 3 show cases of eyes included in the study illustrating differences in the ability to detect progression by OCT and SAP.

Table 3.

Logistic GEE Model Evaluating Factors Explaining the Relative Odds of Progressing by OCT Versus SAP

Figure 2.

Example of an eye of a glaucoma patient with glaucomatous optic neuropathy but normal visual field at baseline (mild disease). Significant progression was seen on SD-OCT RNFL thickness measurements over time, with a rate of change of −2.28 μm/y (P = 0.001). Progression was not seen on SAP mean sensitivity measurements over time, with a slope of −0.25 dB/y (P = 0.188).

Figure 3.

Example of an eye of a glaucoma patient with moderate disease at baseline (baseline MD deviation of −10.96 dB). Significant progression was seen on SAP with a slope of change in mean sensitivity of −0.51 dB/y (P = 0.009). However, no significant change was seen on SD-OCT measurements, with a slope of change of −0.02 μm/y (P = 0.961).

Discussion

The present study demonstrated that the relative odds of identifying progression by structural or functional tests varies according to the number of tests performed during the follow-up and also according to the disease severity at baseline. These findings may have significant implications for the evaluation of glaucoma progression in clinical practice by allowing clinicians to tailor the frequency of SAP and OCT tests during follow-up, taking into account the relative odds of detecting progression at different levels of disease severity.

Several studies have previously shown substantial disagreement when evaluating detection of glaucoma progression by structural and functional tests.^8,11,20–24 Our results agree with this observation. From the 626 slopes showing progression, progression was detected simultaneously by SAP and SD-OCT in only 80 (13%) of them. In the vast majority of cases, progression was either shown by SD-OCT or by SAP, but not by both. Therefore, we built a model evaluating the relative odds of progression by SD-OCT versus SAP in an attempt to understand factors that could explain why some eyes are detected as changing by one test but not by the other. The model showed that in eyes with less severe disease at baseline the odds of detecting progression by SD-OCT were generally higher than the odds of detecting change by SAP. Each 1 dB higher baseline MD was associated with a 5% increase in the odds of progression being detected by SD-OCT versus SAP. This can be clearly seen on Figure 1. When slopes were calculated from eight visits, the probability of detecting progression by SD-OCT when the baseline MD was −1 dB, for example, was 70% compared to only 30% for SAP. Equal probabilities (i.e., 50% each) for detecting progression for SAP and SD-OCT would occur for a baseline MD of −10 dB. For eyes with baseline MD worse than −10 dB, the probability of detecting progression by SAP would be larger than by SD-OCT considering slopes calculated from eight visits. For example, for a baseline MD of −20 dB, the estimated probability of detecting progression by SAP was 70% versus only 30% for SD-OCT. These results are in agreement with previous suggestions in the literature that structural assessment by SD-OCT would perform better for detection of disease progression in early stages of damage, whereas SAP would perform better for more advanced cases.^11,21,25 However, to the best of our knowledge, no previous study has provided a quantification of the impact of disease severity on the relative odds of detecting change by these two methods.

The relationship between ability to detect change and disease severity may be explained by a number of factors. Previous studies investigating the structure and function relationship in glaucoma have shown that when measurements are expressed in their original scales (i.e., dB for SAP and μm for RNFL), a nonlinear relationship is seen.^11,26–31 From this nonlinear relationship, it can be extracted that for eyes at relatively early stages of damage, substantial change in RNFL measurements might be seen when only relatively small changes in visual function are seen in decibels. In contrast, for eyes at advanced stages of disease, even relatively small neural losses will produce large changes in the decibel scale. It is important to note that the nonlinear relationship between structure and function seems to be mostly the result of the logarithmic scaling of visual field data, as transformation of functional data to a linear scale (or structural data to a decibel scale) seems to produce a closely linear relationship between functional and structural measurements.^11,28,32 However, such post hoc transformations will not influence the ability of a test to detect change as long as the original measurements are still acquired in a decibel scale (i.e., the staircase procedures for threshold acquisition still acquired on decibel steps). Another factor influencing the relative ability of structural and functional tests in detecting progression is the presence of floor effects. For SD-OCT average RNFL thickness measurements, a floor effect seems to occur when measurements are close to 50 μm, although there are variations according to specific instruments.^33–36 When the floor is reached, even further neural losses will go relatively undetected by RNFL thickness measurements. The presence of such floor has been shown by several previous investigations and seems to be related to the presence of nonneural or glial tissue, as well as to the dynamic range of the instrument.³⁶

Several previous studies have shown that SAP variability increases with disease severity.^37–40 This would seem to contradict our findings of a relatively superior ability of SAP in detecting progression as disease severity gets worse. However, it is important to note that such relationship between disease severity and SAP variability is valid for measurements expressed in the decibel scale. Even though variability in decibels increases with worsening visual field damage, due to the logarithmic scaling, a 1-dB change in later stages of the disease corresponds to a much smaller change in terms of neural loss or structural damage than a 1-dB change in early disease.^11,41,42 Therefore, even relatively small losses of neural tissue in later stages of the disease may produce relatively large changes in visual sensitivity as measured in decibels, overcoming at least in part the limitation of increased variability and making detection of change in decibels relatively easier.¹⁰

As expected, the number of tests (visits) acquired during follow-up also significantly influenced the ability to detect change. It is expected that a greater number of visits will make detection of change easier for both structural and functional tests. However, we also found a significant interaction between baseline disease severity and number of tests in explaining the relative odds of detecting change by SD-OCT versus SAP. This is illustrated by Figure 1 and also by the analysis of the raw data presented in Table 2. When only a small number of tests were available during follow-up (less than eight), detection of progression was more common for SD-OCT than SAP for all stages of the disease, even in eyes with moderate and advanced damage. Conversely, when a large number of tests were available (>12 tests), the odds of detecting change were higher for SAP compared with SD-OCT for eyes with moderate and advanced disease at baseline.

Our findings may be important in providing clinicians with information that can be used to tailor the frequency and type of test according to the stage of disease. However, such test customization strategy might be difficult to implement in clinical practice, as there are no clear cutoffs of when to perform one test versus the other. Additionally, even though under certain circumstances the relative odds of detecting change may be higher for test A compared with test B, this does not necessarily imply that all cases of progression will be detected by test A. Therefore, algorithms that provide a combination of structural and functional data allowing effective monitoring throughout the disease continuum might offer a better solution to this issue.^43–48

The present study has limitations. The ability to detect change was investigated using only trend analysis of global parameters. This was done to allow equating the specificities of the tests, an essential requirement for comparing their abilities to detect change. As there is no current gold standard for detecting change in glaucoma, our comparison was based on the relative proportion of glaucoma eyes that were detected as having statistically significant slopes of progression considering rates of change that would have only a 5% chance of occurring in a healthy population. This was done to provide not only a statistically but also a clinically relevant criteria. Future studies should, however, investigate the relative ability to detect change by using other methods such as event-based detection of localized change or other techniques. As another limitation of our study, we assumed linear rates of change over time. Although this assumption is likely to correspond to how these tests are currently used in clinical practice for relatively short-term management decisions, it is likely that the true course of the disease is not linear in the long run. Recent investigations have also suggested nonlinear models for investigation of rates of visual field change that might provide an improvement over current simple linear models.^49–52 Future studies should also investigate this issue.

In conclusion, our results show that the ability to detect glaucoma progression by SAP and SD-OCT is significantly influenced by the stage of disease. Our results may provide useful information for guiding clinicians on the relative utility of these tests for detecting change throughout the disease continuum. Furthermore, they may also serve to guide the construction of indices or strategies combining structural and functional information for detection of change over time.