Risk Calculation in the Medication Arm of the Ocular Hypertension Treatment Study

Abstract

Purpose

Risk assessment is integral to the management of individuals with ocular hypertension (OHTN). This study aims to determine the predictive accuracy of the Ocular Hypertension Treatment Study 5-year risk calculator (OHTS Calculator) among treated OHTN patients by applying it to patients randomized to the OHTS medication arm.

Design

Post hoc secondary analysis of a randomized clinical trial.

Subjects

Individuals participating in the OHTS who were randomized to the medication arm. Only participants with complete baseline data in both eyes were included (n=726).

Methods

The hazard ratios (HR) of the medication group in OHTS were compared to the HR used for the OHTS Calculator using the Z-test statistic to establish the OHTS calculator’s generalizability to the OHTS medication arm. The performance of the OHTS Calculator among the OHTS medication group was evaluated twice, using both untreated baseline IOP and average treated IOP during the first 24 months for the IOP variable.

Main Outcome Measures

The performance was determined based on the model’s accuracy in estimating the risk of reaching an OHTS primary open-angle glaucoma (POAG) endpoint using calibration chi-square and discriminating between participants who did or did not develop POAG.

Results

The HRs for the OHTS medication arm were not significantly different from those used in the OHTS Calculator for untreated OHTN derived from observation arm data (P>0.1). Based on the calibration chi-square test for the medication group, the OHTS Calculator prediction model had good predictive accuracy when using the mean treated IOP and poorer predictive accuracy with the untreated baseline IOP (chi-square 10 and 29, respectively). The model had good discrimination with treated IOP (c-statistic = 0.77), comparable to what has been reported for the OHTS Calculator in the OHTS observation group.

Conclusion

The OHTS Calculator can be applied to treated OHTN patients, and repeat risk calculation after initiating IOP reduction may provide useful information that can aid in disease management.

Precis

This study describes the predictive accuracy of the ocular hypertension treatment study risk calculator among treated ocular hypertension patients by applying it to individuals randomized to the medication arm of the ocular hypertension treatment study.

Introduction

The Ocular Hypertension Treatment Study (OHTS) was a pivotal clinical trial that concluded that treating ocular hypertension (OHTN) reduced or delayed the risk of developing glaucoma.¹ The OHTS 5-year risk prediction model was first introduced in 2002 based on data collected from both the medication and observation arms of the OHTS² and is often used in clinical practice to estimate the risk for progression from OHTN to primary open-angle glaucoma (POAG). The 2002 calculator formula was then validated using data from the observation arm of the OHTS and the European Glaucoma Prevention Study (EGPS) placebo arm.^3,4 The model was also validated using a separate cohort.^5,6 Since its publication, the OHTS risk calculator has become an invaluable tool in the decision-making process of who should be treated for OHT, both in clinical practice and for cost-benefit health policy analyses.⁷

The prediction model requires the entry of five identified risk factors as numerical variables, including age (years), vertical cup/disc ratio by contour, intraocular pressure (IOP, mmHg), central corneal thickness (CCT, microns), and pattern standard deviation (PSD, dB) from 24–2 or 30–2 Humphrey visual fields or the equivalent corrected loss variance from Octopus perimetry. The OHTS calculator uses these five factors to provide the 5-year probability of transition to POAG. This estimation aids in the decision on whether to initiate treatment to reduce the risk of progression. Although the benefits of initiating therapy for OHTN are well described, both in terms of overall risk reduction to glaucoma progression,¹ as well as the rate of visual field change,^8,9 the applicability of the OHTS 5-year risk prediction model in OHTN patients under treatment has not yet been reported.

The current study aims to determine the utility of using treated IOP in the OHTS risk calculator by comparing the 5-year predictions using IOP at baseline (IOP_baseline) versus using the IOP after initiation of treatment (IOP_treated) in the treated arm of the OHTS trial. We hypothesized that using the treated IOP to calculate the 5-year risk would result in a more accurate prediction of the treated OHTN patient’s outcome than the baseline pressure.

Methods

Subjects

The OHTS and EGPS were prospective, randomized clinical trials that tested the safety and efficacy of topical ocular hypotensive medication in delaying or preventing the development of POAG in individuals with ocular hypertension. The study protocols are described in their baseline design papers.^4,10 The OHTS protocol is available on the web at https://vrcc.wustl.edu (date accessed: 09/21/2022). Each study’s protocol was compliant with the Health Insurance Portability and Accountability Act (HIPAA) of 1996 and was approved by the institutional review boards of all participating clinics and resource centers. The OHTS registration number is NCT00000125 and can be found at www.clinicaltrials.gov.

In both studies, participants were randomized in equal proportions to an observation group or a medication group. In the OHTS, the observation group did not receive ocular hypotensive medication or placebo, whereas the medication group was treated with IOP-lowering medication aimed at lowering IOP by 20% from baseline. The EGPS observation group was a placebo group, which received only the vehicle for the medication. The data collected from the observation groups in both trials provided insights into the relationship between baseline factors, including untreated IOP, and the risk of POAG development, serving as the foundation for the OHTS calculator. In the current study, we used data only from participants in the OHTS who were randomized to the medication group. This group’s demographic and baseline characteristics have already been reported and found similar to the observation group.³

The dataset for this from the first phase of the OHTS trial, encompassing the period from February 1994 to June 2002. The data pertained to OHTS participants in the medication arm. Only participants with complete baseline data in both eyes were included in this report. The protocol for measuring eye-specific factors and POAG endpoints has been described previously.³ The date of onset for POAG is the date of the first of three consecutive abnormal visual fields or the first of two sets of optic disc photographs that masked readers classify as meeting the definition for change and that was subsequently attributed to POAG by a masked endpoint committee. Baseline demographic and clinical information was collected on each participant prior to randomization, except for central corneal thickness (CCT) measurements, which were performed 1–3 years after randomization.

Calculating 5-year risk after treatment

Although the OHTS calculator is available online (https://ohts.wustl.edu/risk/), it is limited for use for eyes with IOP that is 20mmHg or higher. To obtain the estimated risk for patients with lower treated IOP, we have created a custom risk estimation formula using hazard ratios (HR) as previously reported (online supplement 1).³ A simulation of 30 cases with random risk factors was used to validate the accuracy of the formula. It was found to have excellent agreement with the online OHTS calculator version (mean absolute difference 0.16%, median 0.14% range 0–0.45%).

The risk estimation was calculated twice for each eye with either IOP_baseline or IOP_treated for the IOP variable. For IOP_baseline, we used the average measure from both eyes during the baseline visit (i.e., prior to randomization and treatment initiation). To establish a reliable measure of the treated IOP for all participants, IOP_treated was calculated based on the average measurements obtained during the first four follow-up visits after randomization (6,12,18,24 months).

Statistical Analysis

The validity and generalizability of the previously published prediction model from the OHTS/EGPS observation groups were evaluated in the medication OHTS group using three methods:

1. Comparisons of multivariate HR from the OHTS medication group to those previously reported for the OHTS/EGPS observation groups³ using the z-test statistic.¹¹ A test statistic z was calculated, where z = (b[C] − b[T])/SE, and where b(C) and b(T) are, respectively, the regression coefficients of the OHTS control and treatment groups, while SE is the standard error of the difference in the coefficients. This was computed as the square root of the sum of the squares of the SEs for the two coefficients. Because the HR of a variable is calculated by exponentiation of its regression coefficient, the z statistic tests the equality of HRs between control and treatment cohorts

2. Determining the over- and underestimation of the actual number of POAG events in the OHTS medication group using the calibration chi-square.¹¹ The calibration chi-square was calculated by dividing the OHTS medication group into ten levels of risk according to the estimated probability of transition to POAG by the OHTS prediction model. For each decile, the predicted risk of developing POAG was compared to the observed proportion of participants developing POAG. A calibration chi-square of 20.00 and below indicates good agreement between the predicted and the observed event rate.¹¹

3. Assessment of the accuracy of the OHTS prediction model using IOP_treated in discriminating between participants in the medication OHTS group who did/did not develop POAG using the c-statistic.¹² The c-statistic represents the probability that a randomly selected subject who experienced progression to POAG will have a higher predicted probability of developing POAG than a randomly selected subject who did not progress to POAG.

Results

Generalizability of the OHTS calculator to the OHTS medication arm

Out of a total of 817 participants in the OHTS medication arm, 726 had complete baseline data and were included in the multivariate analyses (table 1). POAG developed in 5.8% (42 of the 726 participants included in the analysis). The multivariate HR for baseline age, CCT, VCDR, and PSD were similar (P>0.1) to those previously described for the OHTS calculator (Figure 1). The HR for IOP_treated in the OHTS medication arm also had significant overlap (P = 0.265) with the HR found for baseline IOP in the OHTS observation arm, used in the OHTS calculator.

Table 1.

Demographics and baseline characteristics

	Mean ± SD
Age, years	55.9±9.5
Sex	Female – 413 (56.9%) Male – 313 (43.1%)
Average baseline IOP, mmHg	24.9±2.7
Average CCT	571.0±38.9
Average vertical cup-to-disc ratio	0.37±0.18
Average pattern-standard deviation	2.0±0.27

Figure 1.

Comparison of the hazard ratios for risk factors used in the OHTS calculator. Hazard ratios for baseline age, central corneal thickness (CCT), vertical cup/disc ratio (VCDR), and pattern standard deviation (PSD) and intraocular pressure (IOP) reported for the OHTS-calculator to those calculated for the treatment group of the OHTS using mean treated IOP instead of baseline IOP.

Comparison of POAG Prediction with baseline vs. treated IOP

The results of the goodness-of-fit model comparing the observed rates of progression to POAG with the OHTS calculator estimated risk with either IOP_baseline or IOP_treated as the IOP variable are depicted in figure 2.

Figure 2.

The goodness of fit for using IOP_treated (a) or IOP_baseline (b) to estimate the 5-year risk of POAG. Comparison of observed and predicted five-year incidence of POAG for the OHTS medication group using IOP_treated (a) or IOP_baseline (b). The x-axis refers to the predicted probability of risk calculated using the OHTS calculator with IOP_treated (a) or IOP_baseline (b) for the IOP variable, divided into 10 groups of approximately 70 participants each. POAG=Primary open-angle glaucoma; OHTS=Ocular Hypertension Treatment Study; IOP = Intraocular pressure;

The predictions provided by the model with IOP_treated as the IOP variable in the OHTS calculator were in good agreement with the observed rates of progression to POAG with an acceptable rate of over- or underestimation (calibration chi-square test = 10.4). When IOP_baseline was used as the IOP variable, there was a significant discrepancy between the predicted and observed rates of progression and a high rate of over- or underestimations (calibration chi-square test = 28.95).

The c-statistics for concordance between observed and predicted development when treated IOP was used in the model was 0.77 (95% CI 0.70–0.84, P<0.001), comparable to the c-statistic reported previously for using the OHTS calculator on non-treated individuals with baseline IOP in the OHTS and EGPS cohorts of 0.74 (95% CR 0.70–0.78).³

Discussion

The OHTS calculator is a well-established and valuable tool in clinical practice to estimate the risk for progression to POAG among untreated OHTN patients. Here we tested the performance of this calculator on treated OHTN patients. Our results demonstrate that the OHTS calculator can provide good estimations for the risk of progression to POAG among treated OHTN patients when the treated IOP is used instead of the untreated baseline IOP. The model’s performance was comparable to those previously reported for non-treated OHTN patients³, indicating that the OHTS calculator can also be applied to treated OHTN patients.

Deciding on who should be treated for OHTN and when to initiate therapy is a common ophthalmic dilemma. Clinicians need to check for multiple risk factors and integrate various sources of information, including personal medical history, family history, and clinical examination, to make an informed recommendation. The OHTS study, followed by the introduction of the OHTS calculator, provided a practical and efficient tool to quantitatively assess the overall risk and assist the clinician in identifying untreated OHTN patients most likely to develop glaucoma and who are most likely to benefit from the treatment. The predictions provided by the OHTS calculator are based on the natural history of the disease. Out of the five risk factors included in the model, IOP is the only one that can be altered through treatment. Although intended for use among untreated OHTN patients, it is possible to use the OHTS calculator to estimate the risk with treated IOP as well. While this technique is sometimes applied in clinical practice, the accuracy of the OHTS calculator using treated IOP has not been validated previously to the best of our knowledge.

To substantiate our hypothesis, we required a comprehensive 5-year prospective data set of OHTN patients receiving IOP-lowering medication. The data had to include all five OHTS risk factors to estimate the 5-year risk with the OHTS calculator. To that end, the most suitable cohort was found among patients enrolled in the medication arm of the OHTS. While there have been other studies on the effect of treatment on the risk of glaucoma progression, the OHTS dataset has several unique advantages. First, it is the largest and most comprehensive study to date, with a sample size significantly larger than other similar studies.^13–19 Second, the design of the OHTS included a unique predefined POAG endpoint using a two-step systematic approach that included (1) confirmation of optic disc deterioration and/or visual field abnormality that was identified by masked readers at reading centers through repeat testing,²⁰ and (2) determination that this deterioration and/or abnormality was associated with POAG based on the assessment of a masked endpoint committee.²¹ Third, the OHTS results have been validated by the EGPS, the second-largest study on OHTN.³ It should be noted that although the EGPS found similar risk factors as the OHTS for glaucoma progression, the EGPS failed to show a significant reduction in risk of progression with treatment, primarily due to insufficient reduction in IOP. Therefore, we considered the EGPS less suitable for our current study. However, combination of datasets might have also enabled sub-analysis based on achieving adequate IOP reduction, which was not feasible for the OHTS dataset alone given the low conversion rate.

Extending the risk calculator to be also used for treated OHTN patients has the potential to further expand the utility of the calculator beyond the decision to initiate IOP-lowering treatment in OHTN eyes. It is imperative to remember that even after the decision to initiate treatment has been made, surveillance is required to monitor for treatment efficacy. As both the OHTS and EGPS have shown, although treatment helps to significantly reduce the risk of developing glaucoma, slow progression often continues, and periodic assessment remains essential. Data from 10 years of follow-up of OHTS (OHTS phase 2) found that the 50% reduction in the incidence of POAG associated with treatment extended beyond the first five years of the study (OHTS phase 1).²² Nevertheless, the prevalence of POAG continued to increase even though both groups received treatment.²² The last phase of OHTS ( OHTS phase 3) was merely observational, and treatment was continued or stopped at the discretion of the clinicians and participants. The cumulative 20-year incidence of POAG was 45.6%, with 25.2% of participants developing VF loss in at least one eye. This was despite the fact that these patients were monitored in a clinical study, and over 70% received ocular hypotensive medical treatment or even underwent glaucoma surgery.²³ The likelihood for progression is not static, and risk calculation may vary based on changing parameters such as age and IOP. Adjusted risk calculations may alter treatment goals and surveillance frequency. It is also important to consider that risk estimation not only predicts who will progress but perhaps also who will progress at the fastest rate. This information can guide the development of an appropriate surveillance plan (i.e., frequency of visits and testing); high-risk patients may require biannual testing, whereas monitoring every two years might also be acceptable for patients who are at low risk.^7,24

Interestingly, although the HR we calculated for the treatment group were not statistically different from those previously reported for the OHTS-EGPS observation group, it seems that for the OHTS medication group IOP and CCT were weaker risk factors. In contrast, the other risk factors became stronger. This observation suggests that treatment made IOP a less significant factor in predicting progression for treated patients. Reduction of IOP decreases the 5-year risk of conversion from OHTN to POAG by approximately 50%, and an increase of mean IOP by one mmHg translates to a 10–20% increased risk.^2,4,25 Similarly, multiple studies have shown that a reduction in IOP reduces the rates of VF loss in POAG and other types of glaucoma.^26–31 Nevertheless, even with very effective treatment and a significant reduction in IOP, some patients will continue to progress, albeit at a slower rate than without treatment. Although IOP is currently the only modifiable risk factor for glaucoma progression, many other IOP-independent risk factors are not affected by available treatment options, including age, family history, exfoliation syndrome, or presence of beta-zone parapapillary atrophy.^32–37 In fact, eliminating the IOP-related risk inevitably increases the importance of the IOP-independent risk factors in predicting progression. Note that despite the low correlation between IOP and CCT, treatment has also made CCT a weaker risk factor. The other risk factors, however, became more significant as they were not affected by treatment.

Limitations

This study has several limitations. First, although the OHTS calculator was based on combined data from the observation groups of both the OHTS and EGPS trials, our analysis was limited to data from the OHTS and therefore relied on a relatively low conversion rate. While including the EGPS treatment would have increased the statistical power of our analysis and might have affected the calculated HRs and the measures of the model’s performance (i.e., c-statistics and goodness-of-fit tests), our results are sufficiently robust and comparable to previous publications for untreated OHTN patients. Therefore, it seems unlikely that this would have had a significant implication on the conclusion that the OHTS calculator can be applied to treated OHTN. The mean IOP_treated variable used in our analysis only reflects the average treated IOP in the first 24 months of treatment during which time additional data would be collected, possibly affecting management decisions, such as possible variability in IOP over time and measures of the retinal nerve fiber layer (RNFL) or ganglion cell layer (GCL). While recent reports suggest that long-term variability does not add substantial explanatory power to the prediction model in individuals with untreated OHTN,³⁸ further analysis is needed to confirm the same findings for treated individuals. It should also be noted that the relatively long follow-up duration used to collect sufficient IOP data for averaging and establishing a reliable measure of the treated IOP was determined by the retrospective nature of our analysis, bound by the study protocol and data collection of the OHTS. However, in clinical practice, these repeated measurements can be clustered into a shorter timeframe, providing a reliable measure of the treated IOP for the prediction model. In addition, it is important to note that the use of additional factors, such as OCT measures, in predicting the transition from OHTN to POAG has yet to be validated to the same extent as the factors used in the OHTS calculator. Lastly, our analysis did not limit eyes based on a specific IOP_treated range; it is important to remember that the OHTS calculator was based on patients with an IOP range of 20–34 mmHg. While our results indicate that the OHTS calculator can be applied to IOP_treated lower than 20 mmHg, this does not mean that the same results will be reached for untreated IOP within the same range. Data from large prospective populations observational studies might be helpful in determining the value of the OHTS calculator in this patient population. We should note that although our results support the use of the OHTS 5-year risk calculator in its current form for treated OHTN patients, it is possible that a modified version, looking at other risk factors not used in the current version, might provide better risk estimations. However, such an evaluation was beyond the scope of this study.

Summary

The OHTS Calculator can be applied to treated OHTN patients. Using the treated IOP for risk calculations after initiation of treatment can provide useful information to aid in disease management. This risk estimation is not limited to a certain IOP threshold and can be applied to eyes with IOP values below 20 mmHg. Adjusted risk calculations can help determine treatment efficacy and indicate whether more aggressive IOP lowering is appropriate. In addition, it can be used as a guideline for surveillance intensity. Simulating the change in 5-year risk, based on predicted IOP reduction with treatment, may also be useful in treatment selection and managing patient expectations in clinical practice.

Abbreviations:

IOP

intraocular pressure

OHTS

ocular hypertension treatment study

POAG

primary open-angle glaucoma

OHT

ocular hypertension

CCT

central corneal thickness

VCDR

vertical cup-to-disc ratio

PSD

pattern standard deviation

VF

visual field

EGPS

European Glaucoma Prevention Study