The Development of a Decision Analytic Model of Changes in Mean Deviation in People with Glaucoma: The COA Model

Abstract

Purpose

To create and validate a statistical model predicting progression of primary open angle glaucoma (POAG) assessed by loss of visual field as measured in mean deviation (MD) using three landmark studies of glaucoma progression and treatment.

Design

A Markov decision analytic model using patient level data described longitudinal MD changes over seven years.

Participants

Patient level data from the Collaborative Initial Glaucoma Treatment Study (CIGTS, n=607), the Ocular Hypertension Treatment Study (OHTS, n=148, only those who developed POAG in the first five years of OHTS) and Advanced Glaucoma Intervention Study (AGIS, n=591), the COA model.

Methods

We developed a Markov model with transition matrices stratified by current MD, age, race and intraocular pressure categories and used a microsimulation approach to estimate change in MD over seven years. Internal validation compared model prediction for seven years to actual MD for COA participants. External validation used a cohort of glaucoma patients drawn from university clinical practices.

Main Outcome Measures

Change in visual field as measured in MD in decibels (dB).

Results

Regressing the actual MD against the predicted produced an R² of 0.68 for the right eye and 0.63 for the left. The model predicted ending MD for right eyes of 65% of participants and for 63% of left eyes within 3 dB of actual results at seven years. In external validation the model had an R² of 0.79 in the right eye and 0.77 in the left at five years.

Conclusion

The COA model is a validated tool for clinicians, patients and health policy makers seeking to understand longitudinal changes in mean deviation in people with glaucoma..

Introduction

In 2006, the Ocular Hypertension Treatment Study (OHTS) group reported that the lack of understanding of the change in glaucoma severity over time represents an important limitation in evaluation of the cost-effectiveness of glaucoma treatment.¹ While there have been a number of cohort studies of ocular disease, none of these have been of sufficient size or duration to reliably describe changes in of glaucoma severity.^2–5 However, the National Eye Institute has funded four large landmark studies of the treatment of glaucoma which might provide insights into changes in disease severity over time: the Collaborative Initial Glaucoma Treatment Study (CIGTS) compared surgical treatment to medical treatment of glaucoma in people newly diagnosed with open-angle glaucoma;⁶ the Ocular Hypertension Treatment Study (OHTS) evaluated the efficacy of treatment of ocular hypertension in the prevention of progression to glaucoma in people with normal ocular discs and visual fields;⁷ the Advanced Glaucoma Treatment Study (AGIS) evaluated surgical methods for the prevention of glaucoma progression in people with advanced glaucoma;⁸ and the Early Manifest Glaucoma Study evaluated the treatment of people with early disease.⁹

Using data from clinical trials, it is possible to develop reliable models to describe the progression of chronic diseases if the investigator uses robust methods of model validation.¹⁰ In this report we describe development and validation of a statistical model that brings together patient level data from three of these landmark clinical trials (CIGTS, OHTS and AGIS) to develop a mathematical model describing changes in mean deviation in people with glaucoma over a seven year period.

Methods

We constructed a Markov decision model to estimate changes in MD of a person with open angle glaucoma over seven years based upon a patient’s current MD, intraocular pressure (IOP), vertical cup-to-disc ratio (VCD), age, and race. Patient level data from the CIGTS, OHTS, and AGIS clinical trials was pooled to determine the probability and magnitude of annual change in MD for each year. In the remainder of this paper, we will refer to the pooled model as the “COA model” for the three trials that contributed data. We assessed the internal validity of the model by comparing the MD predicted by the model at year seven to that reported for each study participant with similar characteristics. External validity was assessed by comparing the predicted to actual MD at five years for patients drawn from an academically based glaucoma practice.

Acquiring and preparing the COA data

Written requests for patient level data were made to the principal investigators of CIGTS and OHTS and de-identified data were provided after review. De-identified AGIS study data are in the public domain and were obtained from the AGIS coordinating center. As the purpose of the COA model was to estimate the change in MD over time, only summary quantitative measures of visual field status from reliable visual fields were requested along with supporting demographic and clinical information. In addition, as the model is concerned only with the change in MD for patients with glaucoma, we used data only from those OHTS participants who developed primary open angle glaucoma (POAG) during the study period. This project was approved and monitored by the Washington University Human Research Protection Office.

Building the COA Model

Data from the three studies were merged into a single data set and variable names harmonized. To ease the decision analytic modeling process, MD was converted to an integer measure by rounding off at 0.5 dB (e.g., −0.5 to −1.4 dB would be changed to −1.0 dB; −1.5 to −2.4 would be changed to −2.0 dB). This simplification was well within the test/retest reliability of the test;¹¹ therefore, we have no reason to expect that this led to any clinically relevant bias in prediction of results. Initial descriptive and exploratory analyses were conducted with SAS 9.0 (SAS Institute, Cary N.C.). While each study took place at a different period of time, we treated each year as if it occurred at the same time: i.e., data for the first year of study following enrollment for each participants was considered “year 1” regardless whether that year occurred in 1989 (AGIS), 1994 (CIGTS), or 1995 (OHTS).

Transition matrices were calculated for each year by combining data from the three studies and stratifying by MD, age, race, VCD and IOP category (see Results for discussion of the categories). The transition probabilities were based upon the changes observed in the combined studies for each year. In other words, during each year modeled, the simulated participant has the opportunity to transition to a new MD in the following year that is worse (i.e., more negative) by as much as 4 dB, or to improve (become more positive) by as much as 3 dB, or to remain at the current MD.

Where the stratification resulted in data too sparse to estimate the transition matrix, we used a “bagging” process to expand and smooth the available data. Bagging is similar to bootstrap sampling in that an iterative process is used to generate modeling estimates by resampling the available data with replacement, then averaging the fitted values. Thus, the available data for model estimation are increased by relying on a data set bounded by the available values and thereby avoiding extrapolation beyond the available data.¹²

The Markov model was constructed using TreeAge Pro 2009 (TreeAge Software; Williamsport, MA 2009). A Markov model is a mathematical representation of an iterative process.¹³ In this case, the iterative process describes the annual change in MD seen in people with glaucoma. A schematic of the Markov model is provided in Figure 1. The model estimates the change in MD for a person diagnosed with glaucoma over seven years. Each simulated “participant” entering the model is assigned an initial MD, age, race characteristic (African- American or not African-American), IOP and VCD based upon their baseline characteristics (these could be either the characteristics of an actual patient, or characteristics of a hypothetical patients depending on whether an actual person or hypothetical person is being modeled). Each “year” (Markov cycle), the simulated participant proceeds through the model by randomly selecting a value from the transition table that determines the change in IOP and MD for that year, first for the right and then the left eye. The distribution of the change in IOP and MD was determined by the frequency of that change seen in the three studies for the year (within the categories of age, race, and VCD). Following the determination of change in the left eye, the probability of surviving to the next year is determined by assessing the age specific mortality rate for the participant estimated by the U.S. Census life tables for 2004.¹⁴

Figure 1. Schematic of the COA model.

Simulated participants with glaucoma experience (or not experience) progress during a year, and at the end of the year, they will die (or live). If they live they return to the “person with glaucoma” stage to begin the next year.

COA- Collaborative Initial Glaucoma Treatment Study, the Ocular Hypertension Treatment Study, and Advanced Glaucoma Intervention Study

The Markov model was estimated using a microsimulation approach. In a microsimulation, the underlying simulated cohort is estimated with each simulated participant proceeding through the model individually.¹⁵ For example, let us assume that there are 10,000 cohort members who will be “walking” through the model described in Figure 1. At the beginning (root node) on the far left, each participant is assigned an age, race, VCD, IOP, and starting MD. Each participant proceeds individually to the right, and encounters a probability node defining what change in MD she will experience during her first year. She is assigned her progression based upon a random draw from a distribution specific to her age, race, current MD, VCD, current IOP and the visit number (i.e., years 1–7). She then proceeds through the model to the next probability node “remembering” this information. This process is repeated for seven years, and once completed the next “participant” proceeds through the model. The new person would have different baseline characteristics than the first participant, and thus would be likely to face different transition matrices depending on her prognostic factors. As we repeat the process 10,000 times (or other arbitrary sample size) we develop a distribution that reflects the range and probabilities of change in MD for a cohort of people with the baseline characteristics in question. A similar process might be performed to assess the possible outcomes for a single individual with glaucoma.

Validation of the COA Model

Validation of a model is an essential element of its development, as a model that does not properly represent the underlying epidemiological process will not provide useful information for clinicians or health policy makers.¹⁶ In this project, we conducted internal validation of the model following the example of Kennedy.¹⁷ For those CIGTS, OHTS and AGIS participants for whom we had seven years of follow-up data, we compared their actual outcome to that predicted by the COA model for a simulated participant with the same characteristics (i.e., age, race, starting IOP, starting VCD, IOP at each visit and starting MD). The microsimulation was run with a cohort of 10,000 simulated participants for each combination of characteristics seen among the participants in the CIGTS, OHTS and AGIS studies. The R² as calculated by ordinary least squares regression was used to evaluate the accuracy with which the predicted seven year MD correlated with the observed. In addition, we assessed the proportion of predicted values that fell outside of a 3 dB range of the observed value at seven years by following the example of the CIGTS group that found that a 3 dB difference in MD was determined to represent a clinically significant change in MD.¹⁸ We also report the number of these observations that were over-predicted (i.e., the predicted value was worse than the observed) or under-predicted (i.e., the predicted value was better than the observed).

Internal validation provides insights into the model’s internal consistency of structure; external validation provides insights into the model’s generalizability beyond the data used in its construction. We conducted external validation of the COA model by comparing the predicted outcome at five years to that seen in a cohort of participants drawn from the glaucoma practice at Washington University School of Medicine (WUSM). A report was generated from the medical records system for patients with a diagnosis of POAG for whom there was five years of data records within a seven year period. Each record was reviewed independently by two research coordinators trained in review of ophthalmic records. Each coordinator confirmed that the patients met the inclusion criteria and that the patient had at least five visual field tests. Demographic data along with MD, IOP, and VCD were recorded for each visit. All data were entered into Excel files and converted to SAS for cleaning and reconciliation. The R² for the external sample was compared to the R² for the internal sample using the Fisher Z-transformation to determine similarity of fit.¹⁹ A non-significant Z-statistic was considered to be evidence of similar performance of the model between the external and internal sample, and thus, good external validity.

Results

We have summarized in Table 1 the characteristics of the data provided for participants from the three trials. These characteristics did not differ in any significant manner from that reported in the design or primary papers from these studies. The underlying theory in our model was that the only modifiable risk factor affecting change in MD was IOP and that the only effect of treatment on MD was through this mechanism. We did not consider treatment status of the eye (in CIGTS and AGIS) or participant (in OHTS). Therefore, all eyes of each participant were included in our modeling regardless of treatment status; thus slightly changing the number of participants included in our analyses from that seen in the trials. African-Americans represented 45% of the sample. CIGTS and OHTS participants were of similar age, and AGIS participants were almost a decade older. We expect that this heterogeneity in the training sample used for the model will increase the generalizability of the model.

Table 1.

Description of the participants from the CIGTS, OHTS and AGIS studies whose records provided the data for the COA model.

	CIGTS		OHTS		AGIS
	n=574*		n=148*		n=580*
	Left	Right	Left	Right	Left	Right
Age (years, sd1)	58.2 (11.0)		59.7 (9.1)		66.0 (9.2)
IOP2 (mm Hg3, sd)	25.7 (5.6)	26.0 (5.4)	26.3 (2.6)	26.2 (2.7)	23.0 (6.1)	22.7 (5.8)
Mean Deviation (MD) (dB4, sd)	−4.15 (4.0)	−3.85 (3.7)	0.08 (1.1)	0.11 (1.0)	−10.8 (7.1)	−9.8 (7.5)
Change in MD (dB, sd)**	−0.64 (4.2)	−0.46 (3.7)	−2.04 (2.9)	−1.68 (2.9)	−2.15 (5.56)	−2.02 (4.88)
Race (%)
White	328 (57.1)		90 (60.8)		247 (42.2)
Black	210 (36.6)		47 (31.7)		329 (56.1)
Latino	27 (4.7)		8 (5.4)		9 (1.5)
Asian	9 (1.6)		1 (0.7)		0 (0.0)
Other	0 (0.0)		2 (1.4)		1 (0.2)

COA - CIGTS – Collaborative Initial Glaucoma Treatment Study; OHTS – Ocular Hypertension Treatment Study; AGIS – Advanced Glaucoma Intervention Study

^*The Ns listed for each study differ from previous published totals due to use of both eyes in the model and requirement of follow-up (see text). For OHTS, only those who achieved a glaucoma endpoint as reported in Kass 2002 are included.¹¹

^**Average total change in mean deviation (MD) over seven years

¹sd - standard deviation

²IOP – intra ocular pressure

³mm Hg – millimeters mercury

⁴dB - MD in decibels

In Table 2, we report the follow-up seen from the participants of the combined trials over time. At eighty-four months (seven years), 66% of the participants remained in the sample. At ninety-six months, less than half remained. This was due to a lack of data available for the OHTS cohort after year seven, as well as a drop-off in the follow-up for AGIS and CIGTS after year 7. This was the basis of our rationale to limit our model to change at seven years.

Table 2.

The number of combined COA participants examined at each annual follow-up visit

Visit Number (months since randomization)	Number of Participants
Baseline	1,418
12	1,300
24	1.264
36	1,218
48	1,173
60	1,144
72	1,094
84	929
96	682

COA - CIGTS – Collaborative Initial Glaucoma Treatment Study; OHTS – Ocular Hypertension Treatment Study; AGIS – Advanced Glaucoma Intervention Study

In Table 3 we detail the changes in MD over seven years related to baseline factors: age, race, VCD and IOP. It is clear that older age, African-American race, and a worse VCD at baseline are associated with a larger loss of visual field (as measured by MD) over time. The relationship between baseline IOP and changes over time are not as clear, but as the model considers not only the IOP at baseline, but also IOP at subsequent visits, the influence of baseline IOP did not extend beyond the first year. We estimated transition probabilities for each combination of MD (at integer) and time (i.e., year 1 to 7) stratified by combinations of age (categorized as shown in Table 3), race and IOP (categorized as shown in Table 3).

Table 3.

The average rate of change in MD seen in the right and left eye over seven years by risk factor

Risk Factor /Category		Average Change in MD1 over 7 Years (dB2)
		Right Eye	Left Eye
Age	< 60	−0.32	−0.60
	≥ 60 and < 70	−1.86	−1.63
	≥ 70	−2.24	−2.94
Race	African-American	−.1.49	−1.89
	Non-African American	−1.24	−1.23
Baseline IOP3 (mm Hg4)	≤17.5	−1.40	−2.13
	>17.5 and ≤ 20.5	−0.86	−1.45
	> 20.5 and ≤ 23.5	−1.29	−0.54
	>23.5 and ≤ 26.5	−1.40	−1.82
	>26.5	−1.54	−1.87
Vertical Cup to Disc Ratio	< 0.6	−0.96	−1.52
	≥ 0.6	−1.49	−1.50

¹MD – mean deviation

²dB – decibels

³IOP – intraocular pressure

⁴mm Hg - millimeters mercury

The result of internal validation of the COA model is shown in Figure 2. Our internal validation sample consisted of 470 participants from the COA sample (36%) for whom we had complete data over the seven years. The triangles in the scatter plots represent the combination of the observed and predicted value for a participant in one of the COA trials (i.e., CIGTS, OHTS, or AGIS). The dark parallel lines represent a 3 dB bracket around “perfect” prediction. The R² of 0.68 for the right eye and 0.63 for the left indicates a good fit for the COA model, although the model tends to over-predict the severity of disease. In Table 4 we detail the participant level comparison of the model’s predicted versus the actual for COA participants. We found that the model predicts the outcome of 65% of participants (309/470) within 3 dB of their actual result at 7 years in the right eye, and 63% in the left (298/470). Of those whose prediction fell outside of the 3 dB band (161 in the right eye; 172 in the left), 54.6% (88 of 161) were “over-predicted” (i.e., the predicted result was worse than the actual) in the right eye and 56.3% (97 of 172) in the left. In Table 4, “better” indicates the model predicted that MD at seven years would be more negative (worse) than participant’s actual reported outcome at seven years. “Worse” means that the model predicted the patient would have an MD less negative than the actual.

Figure 2. Regression of Actual vs. Predicted Outcome of COA Participants.

Each triangle represents a COA study participant’s final MD at 7 years and the predicted MD. The diagonal lines bound those observations for which the predicted MD was ≥2dB of the actual reported MD.

COA- Collaborative Initial Glaucoma Treatment Study, the Ocular Hypertension Treatment Study, and Advanced Glaucoma Intervention Study. MD-mean deviation. dB-decibels.

Table 4.

A comparison of the predicted MD at seven years by COA model to the actual MD for a person with the same age, race and starting MD and IOP at seven years

Difference between predicted and actual at seven years	Source of Study Eye
	Right Eye (n and %)				Left Eye (n and %)
	CIGTS	OHTS	AGIS	Total	CIGTS	OHTS	AGIS	Total
Better than 4 dB	29 (12%)	6 (7%)	12 (9%)	47 (10%)	33 (13%)	6 (7%)	15 (11%)	54 (11%)
Better than 3 but worse 4 dB	26 (10%)	8 (9%)	7 (5%)	41 (9%)	26 (10%)	6 (7%)	11 (8%)	43 (9%)
Better than 2 but worse than 3 dB	43 (17%)	15 (17%)	13 (10%)	71 (15%)	42 (17%)	14 (16%)	10 (7%)	66 (14%)
Worse than 2 dB but better than 2 dB	107 (43%)	44 (51%)	62 (46%)	213 (45%)	109 (44%)	48 (55%)	55 (41%)	212 (45%)
Worse than 2 but better than 3 dB	8 (3%)	5 (6%)	12 (9%)	25 (5%)	8 (3%)	4 (5%)	8 (6%)	20 (4%)
Worse than 3 dB but better than 4 dB	9 (4%)	2 (2%)	5 (4%)	16 (3%)	5 (2%)	0 (0%)	9 (7%)	14 (3%)
Worse than 4 dB	26 (10%)	7 (8%)	24 (18%)	57 (12%)	25 (10%)	9 (10%)	27 (20%)	61 (13%)
Total	248	87	135	470	248	87	135	470

COA - CIGTS – Collaborative Initial Glaucoma Treatment Study; OHTS – Ocular Hypertension Treatment Study; AGIS – Advanced Glaucoma Intervention Study

MD – mean deviation measured in dB – decibels

IOP – intra ocular pressure

Our external validation sample is detailed in Table 5. Data on 300 participants were extracted from clinic files. Of these, 150 had sufficient longitudinal data to use for validation at five years (data for seven years were too sparse for validation purposes). The characteristics of this subset are not significantly different than the rest of the data extracted from patient records. The predicted outcome for these patients was compared to the actual at five years and found to have an R² of 0.79 for the right eye and 0.77 for the left. Comparison of these correlation statistics to those of the internal validation found that the differences were not significant, indicating good external validity of the model.

Table 5.

Comparison of Internal and External Validation Samples for the COA Model

		Internal Validation* n = 470		External Validation** n = 151		p-value
		Left	Right	Left	Right
Age (years, sd1)		58.4 (10.4)		67.5 (10.9)		0.0001
IOP2 (mm Hg3, sd)		24.7 (5.2)	25.1 (5.4)	15.9 (4.4)	16.5 (4.7)	0.0001	0.0001
MD4 (dB5, sd)		−4.89 (5.91)	−4.39 (5.73)	−6.53 (6.41)	−6.96 (6.50)	0.004	0.0001
Change in MD (dB, sd)		−1.02 (4.30)	−0.74 (3.71)	−1.06 (3.14)	−1.07 (3.03)	n/a	n/a
Vertical Cup to Disk Ratio		0.63 (0.20)	0.64 (0.20)	0.76 (0.16)	0.77 (0.15)	0.0001	0.0001
Race
White		244 (53.0)		116 (77.8)
Black		196 (42.6)		29 (19.5)		0.0001
Latino		6 (1.3)		0 (0.0)
Asian		0 (0.0)		3 (1.0)
Other		14 (3.0)		1 (0.7)

COA - CIGTS – Collaborative Initial Glaucoma Treatment Study; OHTS – Ocular Hypertension Treatment Study; AGIS – Advanced Glaucoma Intervention Study

^*Internal validation sample for CIGTS, OHTS and AGIS participants providing 7 years of data.

^**External validation sample based on clinic patients from a university practice providing 5 years of data.

¹sd - standard deviation

²IOP - intra ocular pressure

³mm Hg – millimeters mercury

⁴MD - mean deviation

⁵dB - decibels

Discussion

For this report we pooled patient level data from the CIGTS, OHTS, and AGIS studies in order to create a model of change in MD over time and over most of the range of glaucomatous damage classifications. We extensively validated the model, including comparison to a sample of patients not enrolled in the clinical trials. Our validation process demonstrates that the model provides excellent prediction of the changes in MD of patients with POAG over seven years. This could be an important tool for people with glaucoma and their physicians who are seeking to understand their disease prognosis, as well as clinical and health policy investigators seeking to model the impact of population based interventions for glaucoma.

In our 2006 report from the OHTS Group, we demonstrated that treatment of people with ocular hypertension with moderate to high risk of progression (i.e., ≥ 2% annual risk of developing POAG) was cost effective.¹ However, to make it clear that our results said nothing concerning the treatment of glaucoma to prevent disease progression, we added a key caveat:

Our....simulation also makes clear the significant limits to our knowledge of important aspects of the natural history of glaucoma and its impact on quality of life…Among the most influential parameters in the model are the....estimate of the progression of POAG. Our results demonstrate that investigation concerning these aspects of the disease would provide important information to support future evaluations of treatment to prevent glaucoma or slow its progression.

As the OHTS Economic Model lacked a validated estimate for glaucoma progression it would not provide a basis for conducting cost-effectiveness or cost-benefit of glaucoma treatments such as improved IOP control or neuroprotection. We are not aware of any reports that have addressed this limitation since our paper in 2006.

The ability of clinical investigators to address this weakness of the literature has been limited by the expense and logistic difficulty of assembling the necessary cohort of glaucoma patients to evaluate changes in MD over time. Broman and colleagues brought together cross-sectional data from nine studies totaling 1,066 participants, and used a parametric approach to estimate changes in MD relying on age and age specific incidence to simulate the duration of disease.²⁰ Using these methods, the investigators found an average annual rate of progression of glaucoma ranging from 1.12 dB/year for people of European ancestry to 1.56 dB/year for people of Chinese ancestry. People of African-American ancestry progressed at a rate of 1.33 dB/year. The investigators from the Early Manifest Glaucoma Study (EMGT) found that among their participants who progressed, there was an average worsening of 1.93 dB over five years.²¹ Broman and colleagues acknowledged the discrepancy between their findings and those of the EMGT and pointed out that this might be due to the difference in treatment and IOP control between the participants in the clinical trials and those enrolled in the cross-sectional population based cohort studies.

It is difficult to compare the results of the COA model to those of the natural history arm of EMGT and the work of Broman et al. EMGT and Broman are designed as natural history studies and report the result of a population mean. COA is designed to predict the progression of individual patients given a specific vector of risk factors. It is interesting to note however that the average change in MD over seven years in both OHTS and AGIS was approximately the same seen in EMGT at five years (see Table 1). We also found that in modeling the change over time for a hypothetical patient with a starting MD of −4 dB the average change in MD was similar to that seen in EMGT (results available by contacting the corresponding author).

Broman et al has the additional limitation for assessing the influence of treatment on progression in that it does not assess the influence of IOP control, and assumes a linear relationship across the disease spectrum. The COA model addresses both of these limitations. We predicted ending MD within 3 dB in nearly 2/3 of participants at seven years. An important strength to our model over the other models mentioned above is that we have explicitly incorporated longitudinal changes in IOP as a risk factor. This will allow investigators to assess the importance of IOP reduction and stability in slowing patient progression.

Our test of internal validity found that this model has a good fit, as measured by our R²s of 0.68 for the right eye and 0.63 for the left (see Figure 2). We searched the literature to determine how the fit of this model compared to other predictive models relying on continuous outcomes such as MD and identified a range of R² ranging from 0.45 to 0.76.^22,23 Other examples we found had a fit considerably lower than these. While we do not claim that this is an exhaustive review of the predictive literature in medicine, we believe that this indicates that our model well meets most accepted standards of prediction. Further, the fact that the model correctly classifies nearly 2/3 of COA participants indicates that the model has clinical value in providing prognostic information to people with glaucoma, particularly those with early disease.

Our external validation of the COA model is an important strength. It demonstrates that our model is capable of prediction beyond the component studies and thus has good generalizability, an essential quality if the model is to be used by policy makers or physicians. Some might find it surprising that the COA model is more accurate in predicting the outcomes of the external validation sample than it does for the sample on which it was based. We posit two reasons for this. First, due to problems of obtaining a sufficient sample, we were only able to model five years of data with our clinic based sample. It is understandable that the COA model would be more accurate at five years than seven. Secondly, what we have actually found is that the R² for the external sample is not significantly different than that of the internal sample. The proper interpretation of this is not that the external sample fits better than the internal one, but that the difference between the samples is due to random variation, not any systemic source. In other words, we found that the fit statistics for each sample lies with the confidence bound of the other.

Some might also question our external validation using the results of a single institution. It is important to note here that the COA sample itself is a national sample (albeit one that incorporates participants in clinical trials), therefore our external validation consists of comparison of a local sample to a national one and we recognize that our claim of generalizability is limited by the use a sample drawn from a single institution. This limitation is of course necessitated by the difficulty of obtaining similar data from other centers or nationally. Therefore, we invite centers that are interested in collaborating with us in conducting further validation of the model using their own data to contact the corresponding author.

It is an important limitation to our knowledge of the epidemiology of glaucoma that there are few validated models describing changes in glaucoma severity. This deficit has important consequences for patients and policy makers. For the newly diagnosed patient and her physician, this makes it difficult to describe prognosis and the benefit of treatment. For the health policy maker, this deficit makes it nearly impossible to properly characterize the value of new medical and surgical treatments. The COA model represents an innovative evidence based approach to prediction of changes in MD using rigorous statistical methods. In the future, we hope to make this important tool readily available to patients, physicians and investigators interested in better predicting change in visual function over time.