This cohort study uses data from patients with ocular hypertension to assess how well Kalman filtering, a machine learning approach used in aviation, can forecast disease trajectory up to 5 years into the future.
Key Points
Question
Can Kalman filtering, a machine learning approach, accurately forecast future values of mean deviation, pattern standard deviation, and intraocular pressure for patients with ocular hypertension?
Findings
In this cohort study, the algorithm forecasts values of mean deviation on perimetry 5 years into the future within 1.0 decibel of the actual value for 61% of all eyes with ocular hypertension and 68% for the subset of eyes in which ocular hypertension eventually progressed to open-angle glaucoma.
Meaning
This study’s findings indicate that machine learning approaches such as Kalman filtering have the potential to assist clinicians in caring for patients with ocular hypertension and in identifying patients whose ocular hypertension is more likely to progress to glaucoma.
Abstract
Importance
Techniques that properly identify patients in whom ocular hypertension (OHTN) is likely to progress to open-angle glaucoma can assist clinicians with deciding on the frequency of monitoring and the potential benefit of early treatment.
Objective
To test whether Kalman filtering (KF), a machine learning technique, can accurately forecast mean deviation (MD), pattern standard deviation, and intraocular pressure values 5 years into the future for patients with OHTN.
Design, Setting, and Participants
This cohort study was a secondary analysis of data from patients with OHTN from the Ocular Hypertension Treatment Study, performed between February 1994 and March 2009. Patients underwent tonometry and perimetry every 6 months for up to 15 years. A KF (KF-OHTN) model was trained, validated, and tested to assess how well it could forecast MD, pattern standard deviation, and intraocular pressure at up to 5 years, and the forecasts were compared with results from the actual trial. Kalman filtering for OHTN was compared with a previously developed KF for patients with high-tension glaucoma (KF-HTG) and 3 traditional forecasting algorithms. Statistical analysis for the present study was performed between May 2018 and May 2019.
Main Outcomes and Measures
Prediction error and root-mean-square error at 12, 24, 36, 48, and 60 months for MD, pattern standard deviation, and intraocular pressure.
Results
Among 1407 eligible patients (2806 eyes), 809 (57.5%) were female and the mean (SD) age at baseline was 57.5 (9.6) years. For 2124 eyes with sufficient measurements, KF-OHTN forecast MD values 60 months into the future within 0.5 dB of the actual value for 696 eyes (32.8%), 1.0 dB for 1295 eyes (61.0%), and 2.5 dB for 1980 eyes (93.2%). Among the 5 forecasting algorithms tested, KF-OHTN achieved the lowest root-mean-square error (1.72 vs 1.85-4.28) for MD values 60 months into the future. For the subset of eyes that progressed to open-angle glaucoma, KF-OHTN and KF-HTG forecast MD values 60 months into the future within 1 dB of the actual value for 30 eyes (68.2%; 95% CI, 54.4%-82.0%) and achieved the lowest root-mean-square error among all models.
Conclusions and Relevance
These findings suggest that machine learning algorithms such as KF can accurately forecast MD, pattern standard deviation, and intraocular pressure 5 years into the future for many patients with OHTN. These algorithms may aid clinicians in managing OHTN in their patients.
Introduction
Ocular hypertension (OHTN) affects an estimated 3 to 6 million individuals in the United States1 and is a well-established risk factor for open-angle glaucoma (OAG).2 However, according to landmark clinical trials, including the Ocular Hypertension Treatment Study (OHTS),3 few patients with OHTN experience progression to OAG. Although lowering intraocular pressure (IOP) reduces the risk of conversion to OAG,3 common IOP-lowering interventions are associated with adverse effects or are costly. Therefore, if clinicians can more effectively assess which patients with OHTN are at greatest risk for OAG, they can more closely monitor and treat that subset of patients while safely monitoring without treatment many others who have a lower risk.
Kalman filtering (KF), a machine learning approach, is frequently used in commercial aviation to forecast flight paths from one destination to another.4 Recently, researchers have applied KF to forecast disease trajectory dynamics for patients with chronic diseases.5,6,7,8 Kalman filtering initially relies on the disease dynamics of an underlying population to generate its first few forecasts. However, the more KF learns about the unique dynamics of an individual patient, the less it prioritizes information from the population and the more it prioritizes the disease dynamics of the actual patient. This permits KF to generate a personalized forecast of each patient’s disease trajectory and how it changes over time. Kalman filtering has been successfully applied in forecasting disease progression for patients with high-tension glaucoma (HTG)8 and normal-tension glaucoma.9
Here we study how well KF can forecast values of mean deviation (MD), pattern standard deviation (PSD), and IOP for patients with OHTN, and whether the same model previously created for HTG can effectively forecast progression for patients with OHTN or whether a different KF model is better suited for this new population. Thus, we developed 2 KF models: 1 trained, calibrated, and parameterized using data from patients with HTG (KF-HTG) and 1 using data from patients with OHTN (KF-OHTN). We compared these 2 models with 3 standard forecasting methods to assess which of the 5 prediction algorithms most accurately forecasts visual field global indexes and IOP dynamics up to 60 months into the future for patients with OHTN.
Methods
Data Source
We used data from the OHTS trial.3 The OHTS, performed between February 1994 and March 2009, involved 1636 patients with OHTN from 22 US centers. Patients were randomized to treatment with IOP-lowering medications or observation and were followed up for as long as 15 years by using standardized measurements of tonometry and perimetry at baseline and every 6 months. The trial sought to identify differences in conversion rates from OHTN to OAG between the randomization groups based on evidence of glaucomatous optic neuropathy or visual field loss, which was adjudicated as to cause by an end point committee review. This cohort study, a secondary analysis of data from the OHTS, was approved by the University of Michigan and Washington University institutional review boards, which waived the need for patient informed consent because all data were deidentified before we obtained them.
Sample Selection
To be eligible, OHTS participants had baseline IOPs of 21 to 32 mm Hg in both eyes, reliable and normal 30-2 Humphrey visual fields (Carl Zeiss Meditec), and no detectable evidence of glaucoma based on optic nerve evaluation.3 We included the eyes of all persons with 6 or more Swedish interactive threshold algorithm standard or full-threshold Humphrey visual fields and 6 IOP measurements on separate dates over a span of 2 or more years. These 6 measurements ensured that we could forecast using at least 3 warm-up measurements and validate at least 12 months into the future. Patients were censored when they experienced nonglaucomatous visual field loss or underwent any incisional intraocular surgery other than uncomplicated cataract surgery.
Statistical Analysis
All statistical analyses were conducted between May 2018 and May 2019 using MatLab, version R2016b (MathWorks). Participant chracteristics were summarized for the entire sample by using means and SDs for continuous variables and frequencies and percentages for categorical variables. Participants with OHTN and HTG were compared using the independent-samples t test for continuous variables and the Pearson χ2 test for categorical variables, with 2-sided P < .05 indicating statistical significance. We also computed 95% Wald CIs for the proportions of patients with predicted values of MD, PSD, and IOP within 0.5, 1.0, and 2.5 dB of the actual value.
Forecasting MD, PSD, and IOP Values
Kalman Filter
When modeling disease trajectories, KF combines population-level and patient-level disease dynamics to produce personalized forecasts of each model variable. Unlike traditional forecasting techniques, KF uses past prediction errors to correct its future estimates, dynamically updating forecasts as new measurements become available. We used KF to forecast future MD, PSD, and IOP values based on their current values and rates of change. Kalman filtering learns each patient’s dynamics before it begins to forecast. Our analyses required 6 previous sets of readings before forecasting. We also performed an ancillary analysis using 3 sets of initial readings to assess whether KFs could effectively forecast future values with fewer initial inputs.
Data Elements
We obtained demographic information (ie, sex, age, and race) and MD, PSD, and IOP measurements for each eye beginning at study randomization and ending at last follow-up visit or censoring. We parameterized and trained our KF model (KF-OHTN) using MD and PSD on perimetry and IOP measurements, along with their velocities and accelerations. Because KF requires relatively evenly spaced time intervals between consecutive measurements, we performed linear interpolation when necessary. Little interpolation was required because most patient visits adhered closely to OHTS protocol visit windows.
Comparison Models
High-Tension Glaucoma Kalman Filter
Previously, we parameterized a KF (KF-HTG) using data from the Advanced Glaucoma Intervention Study10 and the Collaborative Initial Glaucoma Treatment Study.8 Briefly, the Advanced Glaucoma Intervention Study studied the efficacy of different treatments for advanced glaucoma, and the Collaborative Initial Glaucoma Treatment Study11 studied patients with early to moderate OAG who were randomized to initial treatment with medication or surgery. Patients from the Advanced Glaucoma Intervention Study and the Collaborative Initial Glaucoma Treatment Study are hereinafter referred to as the HTG group. We assessed how well KF-HTG could forecast MD, PSD, and IOP among patients with OHTN.
Personalized Mean Model
The personalized mean (PM) model assumes that MD, PSD, and IOP remain constant at the mean value of the initial 6 readings. For example, consider a patient with initial MD readings of −0.6, −0.9, −1.1, −2.2, −1.0, and −2.0 dB. The PM estimates that MD at 60 months into the future will be −1.3 dB.
Linear Regression Models
Linear regression models assume a linear relationship between the variable of interest (eg, MD) for each time period. We created 2 models: a simple linear regression model (LR1) and an econometric forecasting model12 (LR2). Compared with LR1, LR2 has the same slope but instead passes through the most recent observation (ie, LR2 has a different intercept). These models are described in detail elsewhere.9
For all 5 models, we forecast MD, PSD, and IOP values at 12, 24, 36, 48, and 60 months into the future for all eligible patients with OHTN. Because most patients with OHTN have stable disease over time, we hypothesized that one could adequately forecast future MD, PSD, and IOP values for most patients by assuming no change over time. However, some patients with OHTN ultimately experience progression to OAG, and identifying these patients early is helpful for clinical decision-making. Therefore, a specific aim of our study was to perform a separate analysis investigating the models’ performance on the subset of eyes progressing from OHTN to OAG owing to changes in visual field as determined by the OHTS Endpoint Committee.
Performance Measures
We computed MD, PSD, and IOP prediction errors at 12, 24, 36, 48, and 60 months (ie, the difference between the forecast and the value reported in OHTS) as our primary measure of model performance. Because KF-OHTN required entire patient trajectories for parameterization, we used leave-1-out cross-validation to estimate its performance. That is, for each patient with OHTN, we parameterized KF-OHTN by using all eyes in our sample except that patient’s eyes. Then, we forecast glaucoma dynamics for each eye of that patient with KF-OHTN. Because KF-HTG was trained using patients with HTG, and PM, LR1, and LR2 were trained using only past patient readings, leave-1-out cross-validation was unnecessary for these models.
We analyzed the distribution and magnitude of prediction errors when forecasting 12, 24, 36, 48, and 60 months into the future. To study the distribution of errors, we determined the proportion of MD, PSD, and IOP prediction errors falling within clinically relevant boundaries. For MD and PSD, these thresholds were 0.5, 1.0, and 2.5 dB from the actual value; for IOP, they were 1.0, 2.5, and 5.0 mm Hg from the actual value. We compared prediction error distributions across models by using the Bhapkar test for equality of marginal distributions13 and illustrated these distributions using violin plots. To evaluate prediction error magnitude for each model, we computed root-mean-square error, which estimates the typical deviation between observations and forecasts (eg, for MD, a root-mean-square error equal to 0.8 dB indicates that the actual values will be approximately 0.8 dB greater or less than the forecasts, on average). Lower root-mean-square error values indicate better estimations.
Outlier Analysis
We identified eyes of patients for which KF-OHTN forecast MD 2.5 dB or less (ie, accurate) and greater than 2.5 dB (ie, outliers) from the actual value when forecasting 60 months into the future. We separated outliers into those with forecast values 2.5 dB greater than the actual value (ie, high outliers) and lower than the actual value (ie, low outliers). We compared the accurate group, high outliers, and low outliers on baseline MD, PSD, and IOP by using analysis of variance.
Results
Study Sample
From OHTS, there were 2806 eyes from 1407 eligible patients with OHTN. The mean (SD) age at baseline was 57.5 (9.6) years. Among these patients, 809 (57.5%) were female; the racial composition (by self-identification) included 971 white participants (69.0%), 357 black participants (25.4%), and 79 participants (5.6%) classified as another race. Mean (SD) follow-up was 9.8 (3.7) years. Of the 2806 eyes, 2124 (75.7%) had sufficient measurements to forecast 60 months into the future. Overall, 108 eyes (7.7%) were identified by the OHTN Endpoint Committee as progressing from OHTN to OAG based on perimetry results. The mean (SD) time to conversion was 6.8 (3.7) years (Table 1 and eTable 1 in the Supplement).
Table 1. Description of Study Sample.
| Characteristic | Mean (SD) | P Valuea | |
|---|---|---|---|
| Patients With HTG | Patients From OHTS | ||
| No. of eyes | 601 | 2806 | NA |
| No. of patients | 601 | 1407 | NA |
| Sex, No. (%) | |||
| Male | 291 (48.4) | 598 (42.5) | .05 |
| Female | 310 (51.6) | 809 (57.5) | |
| Race/ethnicity, No. (%)b | |||
| White | 275 (45.8) | 971 (69.0) | <.001 |
| Black | 304 (50.6) | 357 (25.4) | |
| Other | 22 (3.7) | 79 (5.6) | |
| Total No. of visits | 8228 | 57 809 | NA |
| No. of visual field tests per eye | 13.7 (5.5) | 20.6 (7.3) | <.001 |
| No. of IOP measurements per eye | 13.7 (5.5) | 20.6 (7.3) | <.001 |
| Follow-up, y | 6.3 (2.8) | 9.8 (3.7) | <.001 |
| Age at baseline, y | 63.1 (10.9) | 57.5 (9.6) | <.001 |
| Initial | |||
| Mean deviation, dB | −7.5 (5.6) | 0.0 (1.4) | <.001 |
| Pattern standard deviation, dB | 6.4 (3.7) | 2.0 (0.5) | <.001 |
| IOP, mm Hg | 17.7 (4.2) | 25.2 (3.0) | <.001 |
| Changec | |||
| Mean deviation, dB | −1.2 (4.3) | −0.2 (2.0) | <.001 |
| Pattern standard deviation, dB | 0.3 (2.3) | 0.2 (0.9) | .13 |
| IOP, mm Hg | −0.2 (5.0) | −1.1 (3.4) | <.001 |
Abbreviations: HTG, high-tension glaucoma; IOP, intraocular pressure; NA, not applicable; OHTS, Ocular Hypertension Treatment Study.
P values for sex, race/ethnicity, and number of eyes that progressed were computed using Pearson χ2 test for independent samples. All other P values were computed using a 2-sample t test.
Because of rounding, percentages may not sum to 100.
Change was computed as the reading at 60 months minus the initial reading; this statistic was computed only for those with enough measurements.
We used 601 eyes from 601 patients with HTG to develop KF-HTG. Of these patients, 324 (53.9%) came from the Advanced Glaucoma Intervention Study and 277 (46.1%) from the Collaborative Initial Glaucoma Treatment Study. These patients had a mean (SD) age of 63.1 (10.9) years. Three hundred ten patients (51.6%) were female, 275 (45.8%) were white, 304 (50.6%) were black, and 22 persons (3.7%) reported being of another race. Mean (SD) follow-up was 6.3 (2.8) years. At baseline, the eyes with OHTN and HTG differed significantly on mean (SD) MD (0.0 [1.4] vs −7.5 [5.6] dB), PSD (2.0 [0.5] vs 6.4 [3.7] dB), and IOP (25.2 [3.0] vs 17.7 [4.2] mm Hg) (P < .001 for all) (Table 1).
Forecasting MD, PSD, and IOP With Kalman Filter Models
We forecast future MDs, PSDs, and IOPs at 12, 24, 36, 48, and 60 months for each eligible eye with OHTN. Figure 1 illustrates the forecasting of a sample patient from OHTS by using KF-OHTN. Comparisons among KF-OHTN, PM, and LR1 are shown in eFigure 1 in the Supplement. When MD was forecast 60 months into the future, KF-OHTN forecast MD within 0.5 dB for 696 eyes (32.8%; 95% CI, 30.8%-34.8%), within 1.0 dB for 1295 eyes (61.0%; 95% CI, 58.9%-63.0%), and within 2.5 dB for 1980 eyes (93.2%; 95% CI, 92.2%-94.3%). For 144 eyes, (6.8%; 95% CI, 5.7%-7.8%) KF-OHTN estimated an MD greater than 2.5 dB of the actual value. Similarly, KF-HTG estimated an MD within 0.5 dB for 578 eyes (27.2%; 95% CI, 25.3%-29.1%), within 1.0 dB for 1085 eyes (51.1%; 95% CI, 49.0%-53.2%), and within 2.5 dB for 1911 eyes (90.0%; 95% CI, 88.7%-91.2%). The personalized mean model also forecast MD well for many patients, whereas LR1 and LR2 did not (Table 2). The personalized mean model estimated an MD within 0.5 dB of the actual value for 728 eyes (34.3%; 95% CI, 32.3%-36.3%), within 1.0 dB for 1309 eyes (61.6%; 95% CI, 59.6%-63.7%), and within 2.5 dB for 1980 eyes (93.2%; 95% CI, 92.2%-94.3%). The LR1 model estimated an MD within 0.5 dB for 242 eyes (11.4%; 95% CI, 10.0%-12.7%), within 1.0 dB for 484 eyes (22.8%; 95% CI, 21.0%-24.6%), and within 2.5 dB for 1173 eyes (55.2%; 95% CI, 33.1%-57.3%). The LR2 model estimated an MD within 0.5 dB of the actual value for 246 eyes (11.4%; 95% CI, 10.2%-12.9%), within 1.0 dB for 496 eyes (23.4%; 95% CI, 21.6%-25.2%), and within 2.5 dB for 1216 eyes (57.3%; 95% CI, 55.1%-59.4%). Figure 2 compares the distribution of MD prediction errors by using each model when forecasting 12 to 60 months into the future for the entire OHTS sample. Shorter and fatter violin plots indicate greater concentration and less variation in prediction error compared with longer, thinner plots. For PSD, KF-OHTN forecast 1402 eyes (66.0%; 95% CI, 64.0%-68.0%) within 0.5 dB, 1914 eyes (90.1%; 95% CI, 88.8%-91.4%) within 1.0 dB, and 2082 eyes (98.0%; 95% CI, 97.4%-98.6%) within 2.5 dB of the actual value (eTable 2 in the Supplement). For IOP, KF-OHTN forecast within 1.0 mm Hg of the actual value in 560 eyes (26.4%; 95% CI, 24.5%-28.3%), within 2.5 mm Hg of the actual value in 1255 eyes (59.1%; 95% CI, 57.0%-61.2%), and within 5 mm Hg of the actual value in 1854 eyes (87.3%; 95% CI, 85.9%-88.7%), and the KF-HTG forecast 517 eyes (24.3%; 95% CI, 22.5%-26.1%) within 0.5 mm Hg, 1118 eyes (52.6%; 95% CI, 50.5%-54.7%) within 1.0 mm Hg, and 1703 eyes (80.2%; 95% CI, 78.5%-81.9%) within 5 mm Hg of the actual value (eTable 3 in the Supplement).
Figure 1. Forecast of Sample Patient With Ocular Hypertension From the Ocular Hypertension Treatment Study (OHTS) Using the Kalman Filter Model Parameterized Using Patients With Ocular Hypertension (KF-OHTN) Algorithm.
Actual values represent measurements from the OHTS, filtered values are self-corrected past forecasts, and predicted values represent forecast measurements. PI indicates 95% prediction interval (error bars).
Table 2. Proportion of Eyes From Patients in the Ocular Hypertension Treatment Study With Forecasts of MD Within 0.5, 1.0, and 2.5 dB and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months for Each of the 5 Forecasting Models.
| Months Forecast Ahead and Amount of Error in MD Forecast, dB | No. of Eyes (%) | ||||
|---|---|---|---|---|---|
| KF-OHTN | KF-HTGa,b | PM | LR1a,b | LR2a,b | |
| 12c | |||||
| 0.5 | 1067 (42.1) | 942 (37.1) | 1066 (42.0) | 805 (31.7) | 869 (34.3) |
| 1.0 | 1794 (70.7) | 1706 (67.2) | 1789 (70.5) | 1419 (55.9) | 1527 (60.2) |
| 2.5 | 2429 (95.7) | 2413 (95.1) | 2439 (96.1) | 2310 (91.1) | 2340 (92.2) |
| >2.5 | 108 (4.3) | 124 (4.9) | 98 (3.9) | 227 (8.9) | 197 (7.8) |
| 24 | |||||
| 0.5 | 939 (38.8) | 890 (36.8) | 940 (38.9) | 530 (21.9) | 555 (23.0) |
| 1.0 | 1602 (66.3) | 1581 (65.4) | 1624 (67.2) | 1057 (43.7) | 1083 (44.8) |
| 2.5 | 2274 (94.0) | 2280 (94.3) | 2301 (95.2) | 1980 (81.9) | 2011 (83.2) |
| >2.5 | 144 (6.0) | 138 (5.7) | 117 (4.8) | 438 (18.1) | 407 (16.8) |
| 36 | |||||
| 0.5 | 818 (35.3) | 784 (33.8) | 886 (38.2) | 386 (16.7) | 414 (17.9) |
| 1.0 | 1489 (64.3) | 1424 (61.5) | 1516 (65.4) | 791 (34.1) | 809 (34.9) |
| 2.5 | 2175 (93.9) | 2171 (93.7) | 2181 (94.1) | 1623 (70.0) | 1654 (71.4) |
| >2.5 | 142 (6.1) | 146 (6.3) | 136 (5.9) | 694 (30.0) | 663 (28.6) |
| 48 | |||||
| 0.5 | 803 (35.7) | 704 (31.3) | 830 (36.9) | 318 (14.1) | 336 (14.9) |
| 1.0 | 1424 (63.3) | 1313 (58.4) | 1433 (63.7) | 599 (26.6) | 606 (26.9) |
| 2.5 | 2114 (94.0) | 2100 (93.4) | 2114 (94.0) | 1389 (61.8) | 1411 (62.7) |
| >2.5 | 135 (6.0) | 149 (6.6) | 135 (6.0) | 860 (38.2) | 838 (37.3) |
| 60d | |||||
| 0.5 | 696 (32.8) | 578 (27.2) | 728 (34.3) | 242 (11.4) | 246 (11.6) |
| 1.0 | 1295 (61.0) | 1085 (51.1) | 1309 (61.6) | 484 (22.8) | 496 (23.4) |
| 2.5 | 1980 (93.2) | 1911 (90.0) | 1980 (93.2) | 1173 (55.2) | 1216 (57.3) |
| >2.5 | 144 (6.8) | 213 (10.0) | 144 (6.8) | 951 (44.8) | 908 (42.7) |
Abbreviations: KF-HTG, Kalman filter built using a sample of patients with high-tension glaucoma; KF-OHTN, KF built using a sample of patients with ocular hypertension from the Ocular Hypertension Treatment Study; LR, linear regression model; MD, mean deviation; PM, personalized mean model.
This model had a significantly different distribution of prediction errors compared with the KF-OHTN model when estimating MD 12 months into the future at a significance level of P = .01 based on the Bhapkar test for equality of marginal distributions.13
This model had a significantly different distribution of prediction errors compared with the KF-OHTN model when estimating MD 60 months into the future at a significance level of P = .01 based on the Bhapkar test for equality of marginal distributions.
Of 2806 eyes from patients in the Ocular Hypertension Treatment Study, 2537 (90.4%) underwent enough measurements for this analysis.
Of 2806 eyes from patients in the Ocular Hypertension Treatment Study, 2124 (75.7%) underwent enough measurements for this analysis.
Figure 2. The Proportion of Mean Deviation (MD) Forecasting Errors Between 0.5, 1.0, and 2.5 dB of the Actual Value for Patients With Ocular Hypertension.
Shorter, fatter violin plots indicate greater concentration and less variation in prediction error compared with longer, thinner plots. The horizontal lines capture values within 0.5 dB (orange solid lines), 1.0 dB (black dotted lines), and 2.5 dB (black dashed and dotted lines) of the actual values. The white plus sign in the middle of each plot represents the mean, and the cyan rectangle indicates the median. KF-HTG indicates Kalman filter parameterized using patients with high-tension glaucoma; KF-OHTN, KF parameterized using patients from the Ocular Hypertension Treatment Study with ocular hypertension; LR1 and LR2, linear regression 1 and 2 models; and PM, personalized mean.
We replicated these analyses on the subset of eligible eyes that progressed from OHTN to OAG owing to visual field changes. When MD values were forecast 60 months into the future (44 eyes), KF-OHTN and KF-HTG performed identically: within 0.5 dB of the actual value for 13 eyes (29.5%; 95% CI, 16.0%-43.3%), within 1 dB of the actual value for 30 eyes (68.2%; 95% CI, 54.4%-82.0%) and within 2.5 dB of the actual value for 41 eyes (93.2%; 95% CI, 85.8%-100%). By comparison, PM forecast within 0.5 dB of the actual value in 7 eyes (15.9%; 95% CI, 5.1%-26.7%), within 1 dB in 22 eyes (50.0%; 95% CI, 35.2%-64.8%), and within 2.5 dB in 41 eyes (93.2%; 95% CI, 85.8%-100%) (eTable 4 and eFigure 2 in the Supplement). For this subset of eyes with progression, KF-OHTN and KF-HTG also forecast similarly for PSDs and IOPs 60 months into the future (eTables 5 and 6 in the Supplement). For PSD, KF-OHTN and KF-HTG both forecast within 2.5 dB for all 44 eyes (100%; 95% CI, 100%-100%). For IOP, KF-OHTN forecast within 5 mm Hg for 34 eyes (77.3%; 95% CI, 64.9%-88.7%) and KF-HTG for 29 eyes (65.9%; 95% CI, 51.9%-79.9%).
As an ancillary analysis, we assessed how well the KF models forecast MD, PSD, and IOP up to 60 months using 3 initial observations as warm-up instead of 6. The use of 3 initial observations resulted in only modest increases in forecasting error (data unreported).
Comparison of Magnitude of Prediction Errors
When MD values were forecast 60 months into the future, KF-OHTN had a lower root-mean-square error (1.72) than the other models (1.85-4.28). Similar findings were observed for IOP (root-mean-square error for KF-OHTN, 3.42; for other models, 4.08-8.25). For PSD, the KF-OHTN and PM models achieved the lowest root-mean-square error (1.04 and 1.00, respectively), followed by KF-HTG (1.48) and the LR models (2.01 for LR1 and 2.00 for LR2) (Table 3). For eyes with OHTN that progressed to glaucoma, KF-OHTN (root-mean-square error, 1.20) outperformed PM (1.40), LR1 (3.40), and LR2 (3.28) at forecasting 60 months into the future but did not outperform KF-HTG (1.19) (eTable 7 in the Supplement).
Table 3. Comparison of the RMSE of the 5 Models at Forecasting MD, PSD, and IOP for All Patients in the Ocular Hypertension Treatment Study at 12, 24, 36, 48, and 60 Months.
| Months Ahead by Metric | RMSE (% Improvement)a,b | ||||
|---|---|---|---|---|---|
| KF-OHTNc | KF-HTG | PM | LR1 | LR2 | |
| MD | |||||
| 12 | 1.44 (18.8) | 1.48 (16.4) | 1.52 (14.3) | 1.77 | 1.76 (0.7) |
| 24 | 1.64 (32.2) | 1.64 (32.3) | 1.70 (29.7) | 2.42 | 2.43 (–0.3) |
| 36 | 1.59 (47.1) | 1.60 (46.7) | 1.67 (44.5) | 3.00 | 3.01 (–0.4) |
| 48 | 1.61 (55.5) | 1.64 (54.6) | 1.76 (51.4) | 3.62 | 3.63 (–0.4) |
| 60 | 1.72 (59.7) | 1.85 (56.7) | 1.89 (55.8) | 4.27 | 4.28 (–0.4) |
| PSD | |||||
| 12 | 0.73 (14.7) | 0.71 (17.0) | 0.68 (19.6) | 0.85 | 0.86 (–1.5) |
| 24 | 0.80 (28.3) | 0.81 (28.0) | 0.76 (32.2) | 1.12 | 1.13 (–0.7) |
| 36 | 0.84 (39.3) | 0.97 (29.8) | 0.81 (41.2) | 1.38 | 1.38 (0.2) |
| 48 | 0.93 (45.0) | 1.19 (29.7) | 0.90 (46.5) | 1.69 | 1.69 (0.1) |
| 60 | 1.04 (48.5) | 1.48 (26.4) | 1.00 (50.2) | 2.01 | 2.00 (0.6) |
| IOP | |||||
| 12 | 2.33 (26.6) | 2.67 (15.8) | 2.48 (21.7) | 3.17 | 3.15 (0.5) |
| 24 | 2.67 (40.4) | 2.83 (36.7) | 2.72 (39.2) | 4.47 | 4.45 (0.6) |
| 36 | 3.02 (46.7) | 2.99 (47.1) | 3.02 (46.7) | 5.66 | 5.60 (1.0) |
| 48 | 3.23 (53.0) | 3.36 (51.1) | 3.54 (48.5) | 6.88 | 6.83 (0.8) |
| 60 | 3.42 (58.5) | 4.08 (50.6) | 4.56 (44.8) | 8.25 | 8.14 (1.4) |
Abbreviations: IOP, intraocular pressure; KF-OHTN, Kalman filter built using a sample of patients with ocular hypertension from the Ocular Hypertension Treatment Study; KF-HTG, KF built using a sample of patients with high-tension glaucoma; LR, linear regression model; MD, mean deviation; PM, personalized mean model; PSD, pattern standard deviation; RMSE, root-mean-square error.
The RMSE values closer to 0 indicate estimations closer to the actual values obtained in the trial.
Percentage improvement is measured with respect to the LR1 model and computed as (RMSELR1 − RMSEM)/(RMSELR1) × 100, where RMSEM is the RMSE belonging to the KF-OHTN, KF-HTG, PM, or LR2 model. Positive percentage improvement values indicate improved performance compared with the LR1 model.
The RMSE was estimated using leave-1-out cross-validation.
Outlier Analysis
Using KF-OHTN, there were 88 high outliers (forecast value was ≥2.5 dB greater than the actual value at 60 months) (61.1%) and 56 low outliers (forecast value was ≥2.5 dB lower than the actual value at 60 months) (38.9%). There were no differences between the accurately forecast group and the 2 outlier groups with respect to mean initial MD, PSD, or IOP (eTable 8 in the Supplement).
Discussion
We applied KF, a machine learning technique, to generate personalized forecasts of future MD, PSD, and IOP values for patients with OHTN. A KF model built using a sample of patients with ocular hypertension from OHTS successfully forecast MD within 0.5 dB of the actual value at 60 months for nearly one-third of the patients with OHTN and within 1.0 dB of the actual value for more than 60% of the patients. Although KF-OHTN outperformed the other models on most metrics studied, we found that another KF that was previously parameterized and validated on patients with HTG (KF-HTG)8 also did reasonably well at forecasting disease trajectory for most patients with OHTN. Although the PM model also performed relatively well on the overall sample, it was notably less accurate than the KF models on the small, albeit important subset of patients who experienced progression from OHTN to OAG.
The simple PM model, which takes the mean of the initial few measurements, performed well when forecasting MD, PSD, and IOP for the entire sample of patients with OHTN. This result is unsurprising because most patients with OHTN never experience progression to OAG. Although the PM model’s simplicity is appealing, it underestimates future MD and PSD values for patients with OHTN who experience progression to OAG because it assumes that future perimetric parameters are similar to their initial values. Because it is unknown a priori which patients with OHTN will eventually develop OAG, our analysis shows the benefits of using KF over models like PM. Furthermore, past studies using other patient populations with more advanced disease have shown that simple models like PM do not forecast progression well.9
Compared with other forecasting algorithms, KF has the advantage of integrating new data into its forecasts over time. Therefore, although initial forecasts are heavily influenced by the underlying population, KF becomes more influenced by the dynamics of the actual patient as additional measurements are obtained. This likely explains why KF-HTG—which was developed using patients with moderate to severe OAG—accurately estimated the trajectory of many patients with OHTN. We suspect that KF-HTG quickly learned that many patients with OHTN did not experience changes in MD or PSD at IOP levels that were associated with disease progression for many patients with HTG. Likewise, a recently published complementary study9 found that KF-HTG successfully forecast the trajectory of many patients with normal-tension glaucoma, again showing the adaptability of these algorithms. Although additional studies are needed to test and validate our KFs on other glaucoma types and severities, it is possible that a single KF algorithm can accurately forecast disease trajectory across the disease spectrum, from OHTN to advanced OAG. Practically speaking, it would be easier to implement into clinical practice a single KF model that could forecast for all patients with OHTN and OAG instead of selecting from multiple algorithms depending on a patient’s glaucoma type and severity.
We evaluated the performance of our KF models by using 3 instead of 6 initial observations. Although the use of 6 observations resulted in more accurate forecasts, the improvement was modest. Algorithms that can begin forecasting with fewer measurements are preferable because fewer data points are required before the model can be used, allowing it to forecast for more patients.
We hope soon to incorporate our KFs into perimeters or electronic health record software to generate automated forecasts delivered in real time to clinicians to aid with clinical decision-making. We have already developed tools involving KFs that clinicians could use to assess the optimal frequency for performing diagnostic testing to monitor OAG8 and identify rates of deterioration of MD at different target IOP levels.14 Similar tools could be created for patients with OHTN based on KF-OHTN.
Limitations
Our study has limitations. First, OHTS collected data on the optic nerve appearance to assess for conversion to OAG3; however, because these evaluations were qualitative and not quantitative, they could not be integrated into our KF models. Second, in OHTS, the intervals between study visits were nearly evenly spaced. In clinical practice, timing between visits often varies. Because KF models require evenly spaced measurements, future studies must extend this method to handle irregularly spaced intervals. The precision of KF models may be greater in clinical trials that standardize and monitor the quality of measurements.
Conclusions
We trained and validated a KF model to accurately forecast MD, PSD, and IOP values to 60 months. The estimated values from our model aligned with the actual values for most patients we studied. Integrating findings from machine learning algorithms, such as those presented herein, into clinical decision-making may enhance clinicians’ ability to make more informed decisions regarding which patients with OHTN would most benefit from treatment.
eTable 1. Description of Eyes From OHTS Which Progressed Due to Changes in VF and Did Not Progress Due to Changes in VF
eTable 2. Proportion of Eyes From OHTS Patients With Forecasts of Pattern Standard Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 3. Proportion of Eyes from OHTS Patients With Forecasts of Intraocular Pressure Within 1.0, 2.5, and 5.0 mm Hg, and Beyond 5.0 mm Hg of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 4. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Mean Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months into the Future for Each of the Five Forecasting Models
eTable 5. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Pattern Standard Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 6. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Intraocular Pressure Within 1.0, 2.5, and 5.0 dB, and Beyond 5.0 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 7. Comparison of the Root Mean Square Error of the 5 Models at Forecasting Key Glaucoma Metrics for Progressed Eyes From OHTS Patients at 12, 24, 36, 48, and 60 Months Into the Future
eTable 8. Characteristics of Patients With Ocular Hypertension Stratified by Whether the KF-OHTN Model Accurately Forecasted Mean Deviation 60 Months Into the Future
eFigure 1. Forecast of Sample Patient With Ocular Hypertension From the OHTS Trial Comparing [A] KF-OHTN With PM and [B] KF-OHTN With LR1
eFigure 2. Violin Plots Showing the Proportion of MD Forecasting Errors Between 0.5, 1.0, and 2.5 dB of the Actual Value for Patients With Ocular Hypertension Who Progressed to Open-Angle Glaucoma
References
- 1.Leibowitz HM, Krueger DE, Maunder LR, et al. . The Framingham Eye Study monograph: an ophthalmological and epidemiological study of cataract, glaucoma, diabetic retinopathy, macular degeneration, and visual acuity in a general population of 2631 adults, 1973-1975. Surv Ophthalmol. 1980;24(suppl):1416-1423. [PubMed] [Google Scholar]
- 2.Weinreb RN, Aung T, Medeiros FA. The pathophysiology and treatment of glaucoma: a review. JAMA. 2014;311(18):1901-1911. doi: 10.1001/jama.2014.3192 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Gordon MO, Kass MA. The Ocular Hypertension Treatment Study: design and baseline description of the participants. Arch Ophthalmol. 1999;117(5):573-583. doi: 10.1001/archopht.117.5.573 [DOI] [PubMed] [Google Scholar]
- 4.Lefferts EJ, Markley FL, Shuster MD. Kalman filtering for spacecraft attitude estimation. J Guid Control Dyn. 1982;5:417-429. doi: 10.2514/3.56190 [DOI] [Google Scholar]
- 5.Eberle C, Ament C. The unscented Kalman filter estimates the plasma insulin from glucose measurement. Biosystems. 2011;103(1):67-72. doi: 10.1016/j.biosystems.2010.09.012 [DOI] [PubMed] [Google Scholar]
- 6.Lavieri MS, Puterman ML, Tyldesley S, Morris WJ. When to treat prostate cancer patients based on their PSA dynamics. IIE Trans Healthc Syst Eng. 2012;2:62-77. doi: 10.1080/19488300.2012.666631 [DOI] [Google Scholar]
- 7.Wang Q, Molenaar P, Harsh S, et al. . Personalized state-space modeling of glucose dynamics for type 1 diabetes using continuously monitored glucose, insulin dose, and meal intake: an extended Kalman filter approach. J Diabetes Sci Technol. 2014;8(2):331-345. doi: 10.1177/1932296814524080 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Schell GJ, Lavieri MS, Helm JE, et al. . Using filtered forecasting techniques to determine personalized monitoring schedules for patients with open-angle glaucoma. Ophthalmology. 2014;121(8):1539-1546. doi: 10.1016/j.ophtha.2014.02.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Garcia GP, Nitta K, Lavieri MS, et al. . Using Kalman filtering to forecast disease trajectory for patients with normal tension glaucoma. Am J Ophthalmol. 2019;199:111-119. doi: 10.1016/j.ajo.2018.10.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ederer F, Gaasterland DE, Sullivan EK; AGIS Investigators . The Advanced Glaucoma Intervention Study (AGIS), 1: study design and methods and baseline characteristics of study patients. Control Clin Trials. 1994;15(4):299-325. doi: 10.1016/0197-2456(94)90046-9 [DOI] [PubMed] [Google Scholar]
- 11.Musch DC, Gillespie BW, Lichter PR, Niziol LM, Janz NK; CIGTS Study Investigators . Visual field progression in the Collaborative Initial Glaucoma Treatment Study: the impact of treatment and other baseline factors. Ophthalmology. 2009;116(2):200-207. doi: 10.1016/j.ophtha.2008.08.051 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Krajewski L, Ritzman L, Malhotra M. Forecasting demand In: Operations Management: Processes and Supply Chains. 10th ed Boston, MA: Pearson; 2013:490-500. [Google Scholar]
- 13.Bhapkar VP. A note on the equivalence of two test criteria for hypotheses in categorical data. J Am Stat Assoc. 1966;61:228-235. doi: 10.1080/01621459.1966.10502021 [DOI] [Google Scholar]
- 14.Kazemian P, Lavieri MS, Van Oyen MP, Andrews C, Stein JD. Personalized prediction of glaucoma progression under different target intraocular pressure levels using filtered forecasting methods. Ophthalmology. 2018;125(4):569-577. doi: 10.1016/j.ophtha.2017.10.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable 1. Description of Eyes From OHTS Which Progressed Due to Changes in VF and Did Not Progress Due to Changes in VF
eTable 2. Proportion of Eyes From OHTS Patients With Forecasts of Pattern Standard Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 3. Proportion of Eyes from OHTS Patients With Forecasts of Intraocular Pressure Within 1.0, 2.5, and 5.0 mm Hg, and Beyond 5.0 mm Hg of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 4. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Mean Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months into the Future for Each of the Five Forecasting Models
eTable 5. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Pattern Standard Deviation Within 0.5, 1.0, and 2.5 dB, and Beyond 2.5 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 6. Proportion of Progressed Eyes from OHTS Patients With Forecasts of Intraocular Pressure Within 1.0, 2.5, and 5.0 dB, and Beyond 5.0 dB of the Actual Value at 12, 24, 36, 48, and 60 Months Into the Future for Each of the Five Forecasting Models
eTable 7. Comparison of the Root Mean Square Error of the 5 Models at Forecasting Key Glaucoma Metrics for Progressed Eyes From OHTS Patients at 12, 24, 36, 48, and 60 Months Into the Future
eTable 8. Characteristics of Patients With Ocular Hypertension Stratified by Whether the KF-OHTN Model Accurately Forecasted Mean Deviation 60 Months Into the Future
eFigure 1. Forecast of Sample Patient With Ocular Hypertension From the OHTS Trial Comparing [A] KF-OHTN With PM and [B] KF-OHTN With LR1
eFigure 2. Violin Plots Showing the Proportion of MD Forecasting Errors Between 0.5, 1.0, and 2.5 dB of the Actual Value for Patients With Ocular Hypertension Who Progressed to Open-Angle Glaucoma