Prediction of Maximum Keratometry Progression With Stratified Progression Criteria in Keratoconus Patients Using Machine Learning

Abstract

Purpose

The aim of this study was to use machine learning to predict 12-months progression of maximum keratometry (K_max) within stratified repeatability limits based on a baseline topography/optical coherence tomography measurement in patients with keratoconus (KC).

Methods

This study included patients with diagnosed KC in which baseline and follow-up measurements were performed with a Placido disc topographer combined with optical coherence tomography (OCT) at a time interval of at least 7 months. Eyes were classified as “progressive” or “stable” based on the repeatability-adjusted difference in K_max between the baseline measurement and a follow-up measurement. Nested cross-validation was used to train different algorithms, and the algorithm with the best area under the curve (AUC) value was used as final model.

Results

A total of 102 eyes of 65 patients were included. Mean age was 33.2 ± 13.0 years. Predictors with high outer fold stability included specific KC markers such as the CSI^B index and posterior K values. Within the internal folds, the model achieved an AUC of 0.82, a sensitivity of 62.2%, and a specificity of 83.2% and, in the outer left-out folds, a sensitivity of 65.6%, a specificity of 84.3%, and thus an accuracy of 78.4%.

Conclusions

Progression of K_max can be predicted with decent accuracy using a single OCT-based topographic measurement. Nested cross-validation represents a valuable tool in small datasets.

Translational Relevance

Machine learning–based prediction of K_max progression may aid in scheduling risk-adapted follow-up appointments, potentially leading to more rapid identification of progression-suspect KC cases.

Introduction

Keratoconus (KC) is a bilateral, frequently asymmetric progressive corneal disease leading to corneal steepening and thinning especially of the central corneal area, which results in progressive worsening of vision.¹ The gold standard to slow down or even halt the progression of the disease is corneal cross-linking (CXL).² However, the indication for CXL is commonly based on the presence of disease progression, which is evaluated according to corneal tomographical/topographical or combined measurements.³ Although different strategies to determine disease progression have been introduced, one of the most commonly used topographic progression criteria is the increase of maximum keratometry (K_max).^4,5 K_max values obtained by a Placido disc topographer combined with high-resolution anterior segment optical coherence tomography (AS-OCT; MS-39; Costruzione Strumenti Oftalmici, Florence, Italy) have been shown to provide good repeatability in patients with KC.⁶ Nonetheless, Seiler et al.⁶ reported that, in patients with KC, the test–retest repeatability of K_max in this device is proportional to the total amount of K_max; however, when using K_max as the central progression criterion, it is necessary to adjust the classification of progression according to the total amount of K_max. Consequently, to make a clear statement about disease progression, one single measurement is not sufficient: At least two measurements with a minimum interval of several months are needed. During this time, further disease progression is possible, and multiple clinical visits are needed.⁷ Thus, approaches to detect progressive cases at the first presentation without timely delay are desired. Although an increasing pool of studies have reported on the machine learning (ML)-based automated detection of KC, subclinical KC, and forme fruste KC from topographical and/or tomographical measurements with high accuracy, data on the detection of KC progression, especially on repeatability-based progression, are rare. The aim of this study was to provide a ML algorithm to automatically detect eyes with KC affected by K_max progression after 12 months defined by the repeatability-adjusted K_max values measured by a Placido disc topographer combined with AS-OCT.

Materials and Methods

Before the collection of data, the study protocols were approved by the Ethics Committee of the City of Vienna (EC number: EK-24-158-VK) and the Ethics Committee of the Johannes Kepler University (EC number: 1049/2024). The study was registered at www.clinicaltrials.gov (Identifier: NCT06873399) and was conducted in compliance with the tenets of the Declarations of Helsinki and its later amendments,⁸ as well as the Good Clinical Practice guidelines of the European Union. Due to the retrospective nature of the study, informed consent was not obtainable. The collection of data was performed at two centers: Vienna Institute for Research in Ocular Surgery (VIROS) at the Department of Ophthalmology of the Hanusch Hospital (Vienna, Austria) and the Department of Ophthalmology, Kepler University Clinic (Linz, Austria).

Patients

Patients of two cornea-specialized outpatient clinics with a diagnosis of KC, based on corneal experts’ assessments using biomicroscopic features and topographical conformation, were included. Due to the asymmetrical nature of the disease, both eyes of each patient were included if both eyes fulfilled the respective inclusion and exclusion criteria. Patients were excluded from the study if there were any other relevant ophthalmic diseases that were likely to influence the measurement outcome, such as corneal scars, corneal dystrophies, a history of CXL, or penetrating keratoplasty that took place before the baseline measurement or between the baseline and follow-up measurement, or if there were too few measurements/too short follow-ups to define the progression of KC. A minimum interval of 7 months was required between the two measurements, and each progression value was extrapolated to 12 months to obtain comparable progression rates. All measurements were performed between February 1, 2020, and June 31, 2024.

The included eyes were divided into two groups. The first group was classified as progressive regarding the change of K_max, and the second group consisted of stable eyes without significant change of K_max. Eyes with a difference of K_max between two measurements of >0.5 diopters (D) per year (if the total amount of K_max was <48 D), >0.6 D per year (if the total amount of K_max was between 48.01 and 53 D), >0.8 D per year (if the total amount of K_max was between 53.01 and 58 D), and >1.5 D per year (if the total amount of K_max was >58 D) were classified as progressive, according to the reported test–retest repeatability values of the combined Placido disc topographer/OCT device reported by Seiler et al.⁶ If the difference between the measurements of K_max per year was lower than the mentioned cut-offs, the eyes were classified as stable. In both clinics, the patients are routinely advised to pause any form of contact lens wear at least 5 days before every consultation to avoid any influence on the measurements.

Methods

The MS-39 device is a combination of a Placido disc topographer and a high-resolution spectral-domain AS-OCT device. Corneal topography of the central 10 mm of the anterior corneal surface is obtained from 22 Placido disc rings that are emitted by a laser-emitting diode light source at 635 nm. The keratometry index of the device is 1.3375. The spectral-domain OCT of the device has a light source operating at a wavelength of 845 nm, providing cross-sectional OCT imaging at an axial resolution of 3.6 µm.⁹ To also include geometrical centers of different parameters in the models while taking laterality into account, the x-coordinates in millimeters (mm) of the right eyes were mirrored and therefore, classified as “temporal” if positive or “nasal” if negative.

Data Collection

The following data were collected from all included patients: age, gender, laterality, history of CXL, date of CXL, and the following indices of the Placido disc topographer/AS-OCT device: quality indices, summary indices, KC screening indices, epithelial thickness values, and front and back K-readings. All index duplicates were removed. If there was more than one measurement at one visit, the measurement with the highest quality index regarding Placido coverage was used for the analysis. In total, two measurements of each eye were obtained: As the baseline measurement, the first measurement obtained in the predefined time span was used; if there was more than one follow-up measurement, then the measurement closest to 1 year of follow-up was used for defining the ground truth (progressive/stable).

ML Algorithm

To develop a ML model for the prediction of progression, RStudio software (R Foundation for Statistical Computing, Vienna, Austria) was used. Due to the small dataset, instead of the classic train/test split method, nested cross-validation was used for model generation and validation.¹⁰ The advantages of nested cross-validation rest on the feasibility to train and test a model without data leakage even in small datasets. In short, by default, 10 inner and 10 outer folds are created. Although filters are applied to the training data of the outer loop, hyperparameter tuning is performed within the inner folds, and the model is then fitted on the whole outer training fold.

For validation, the data from the held-out outer folds are used. By performing a final round of cross-validation, the final hyperparameter settings are determined. An illustration of the principle of nested cross-validation is shown in Supplementary Figure S1. The following filters were applied to reduce the total amount of predictors, and the receiver operating characteristic (ROC) values were then compared to the full model: Boruta filter, t-test filter, partial least squares regression filter, and random forest variable importance filter. Using the caret package, different models were tested—namely, random forest, k-nearest neighbor, neural network, support vector machine, and a generalized linear model.¹¹ The model with the highest internal performance according to the area under the curve (AUC) was selected as the final model. For training, summary indices, KC screening indices, epithelial thickness values, and front and back K-readings of the Placido disc topographer/AS-OCT device were used.

Outcomes

The performance measures (sensitivity and specificity, AUC) of the model regarding the prediction of progression in the left-out data of the outer folds were chosen as the main outcome. In addition, predictors with variable stability >0.7 are reported. Variable stability evaluates the consistency of variables across the outer folds.

Statistical Analysis

The baseline characteristics are given as mean ± standard deviation (SD). To evaluate the performance on the inner and outer folds, the following metrics were used: accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. In addition, confusion matrices for the predictions and the actual values are shown. The difference between the performance of the final model and the no-information rate (NIR) was tested using a χ² test. The NIR is the accuracy a model would achieve by always predicting the most frequent class. The differences of the most stable predictors between the groups were tested according to the distribution of the data, which was calculated using a Shapiro–Wilk test. For statistical testing, based on distribution of the data, an unpaired t-test or a Mann–Whitney U test was used. For categorical values, a χ² test was used. In addition, the direction of the most stable predictors was analyzed using the var_stability function. P < 0.05 was considered significant. To adjust for multiple comparisons, Bonferroni–Holm adjustment was applied. All statistical analyses were computed using RStudio 4.2.2, and the following R packages were used: caret, nestedcv, ggplot2, and dplyr.

Results

In total, 102 eyes of 65 patients with diagnosed KC with a mean age of 33.2 ± 13.0 years were included in this study. Thirty eyes had a K_max of lower than 48 D, 39 eyes had a K_max of >48 to 53 D, 16 eyes had a K_max of >53 to 58 D, and 17 eyes had a K_max of >58 D. The mean follow-up time between the two measurements was 12.6 ± 3.0 months. The baseline characteristics of both groups are included in Table 1.

Table 1.

Demographic Data and Differences Between Progressive and Stable Eyes

Demographic	Progression (n = 32)	Stable (n = 70)	P
Age (y), mean ± SD	32.7 ± 15.8	33.7 ± 11.6	0.24a
Females/males, n	12/20	20/50	0.43b
Laterality (R/L), n	19/13	31/39	0.16b
Follow-up (mo), mean ± SD	11.50 ± 2.18	13.06 ± 3.20	0.03a,*
Kmax (D), mean ± SD	51.60 ± 8.97	52.91 ± 7.48	0.10a
simKf steep (mm), mean ± SD	7.22 ± 0.76	7.12 ± 0.66	0.17a
simKf flat (mm), mean ± SD	7.70 ± 0.63	7.55 ± 0.60	0.10a

simKf flat, simulated front keratometry, flat meridian; simKf steep, simulated front keratometry, steep meridian.

^aMann–Whitney U test.

^bχ² test.

^*Significant at P < 0.05.

Of all 102 eyes, 70 were classified as stable/non-progressive and 32 were classified as progressive. Regarding the baseline characteristics of both groups, there was no difference between the mean age of the patients in the progressive group and the stable group (mean difference, −1.0 year; P = 0.24, Mann–Whitney U test). The progressive eyes had a K_max of 51.60 ± 8.97 D, and the stable eyes had a K_max of 52.91 ± 7.48 D, with a mean difference of 1.30 D (P = 0.1, Mann–Whitney U test). The mean change of K_max was 5.78 ± 12.66 D/y in the progressive group and was –0.59 ± 1.16 D/y in the non-progressive group, and there was a mean difference of 6.37 D/y (P < 0.001, Mann–Whitney U test) between the groups.

Inner Loop Performance

The algorithm with the best AUC was the random forest algorithm. Despite various filtering methods being applied, the AUC of the full model was higher than that of the filtered models. Therefore, the full model was used as the final model. Due to the imbalance of the data, decision thresholds of 0.4/0.6 in favor of the minor class (progressive) were implemented. Within the inner folds, the final model achieved an AUC of 0.82, a sensitivity of 62.2%, and a specificity of 83.2%. The ROC curve and the confusion matrix of the inner folds are shown in Figures 1 and 3, respectively.

Figure 1.

Inner fold ROC curve of the random forest model.

Figure 3.

Inner fold confusion matrix.

Outer Loop Performance

In the outer loop data, the model achieved an accuracy of 78.4% (95% confidence interval [CI], 69.2–86.0), a sensitivity of 65.6%, a specificity of 84.3%, a positive predictive value of 65.6%, and a negative predictive value of 84.3%. The detection rate was 20.6%, and the model performed significantly better than the NIR (P = 0.02). The ROC curve and the confusion matrix of the outer folds are shown in Figures 2³ and 4, respectively.

Figure 2.

Outer fold ROC curve of the random forest model.

Figure 4.

Outer fold confusion matrix.

Stability of Predictors

All predictors that achieved a stability of >0.7 are shown in Table 2. In total, five predictors achieved a stability exceeding the mentioned cut-off: central-surrounding index back (CSI^B), K^B Meridians 3 mm K_flat (mm), K^B Meridians 3 mm K_avg (mm), K^F Meridians 7 mm KJ0 (D), and K^B Meridians 3 mm K_steep (mm). The stability, mean ± SD, and P values for comparisons between the two groups and their direction are also given in Table 2.

Table 2.

Predictors With Fold Stability of >0.7

Predictor	Stability	Progression (n = 32), Mean ± SD	Stable (n = 70), Mean ± SD	P	Direction
CSIB	94.4	0.35 ± 1.15	0.97 ± 1.08	<0.001a,*	Stable
KB Meridians 3 mm Kflat (mm)	73.7	6.67 ± 1.06	5.93 ± 0.81	<0.001a,*	Progression
KB Meridians 3 mm Kavg (mm)	72.8	6.30 ± 0.94	5.64 ± 0.79	<0.001a,*	Progression
KF Meridians 7 mm KJ0 (D)	70.3	0.32 ± 1.40	0.58 ± 0.84	0.48b	Stable
KB Meridians 3 mm Ksteep (mm)	70.2	5.92 ± 0.96	5.35 ± 0.80	<0.001a,*	Progression

^aMann–Whitney U test.

^bUnpaired t-test.

^*Significant at P < 0.05.

Discussion

Several previous studies have investigated the capability of different topographical and/or tomographical devices to discriminate healthy eyes from eyes affected by KC, subclinical KC, and forme fruste KC.^12–17 Also, an algorithm predicting eyes at risk for penetrating keratoplasty has been introduced.¹⁸ However, our study did not aim to predict the presence of KC. Indeed, we aimed to investigate if the detection of progression in KC eyes after 12 months according to the mean change in stratified K_max values can be predicted using a ML model. Notably, the stratification of K_max is an important step: Gustafsson et al.^19,20 showed that K_max values should be stratified according to disease severity to assess progression based on measurement precision. Therefore, we stratified the amount of K_max according to the test–retest repeatability of the used device as reported by Seiler et al.⁶

Achieving inner- and outer-loop sensitivity/specificity values of 62.2%/83.2% and 65.6%/84.3%, respectively, our data indicate that disease stability may be identified more reliably than progression using a single Placido disc topography/OCT measurement. In addition, by analyzing the stability of variable importance, we identified parameters that showed significant differences between progressive and non-progressive eyes. Our analysis indicates that the K-readings of the back surface preferably support the detection of progression, whereas other indices such as CSI_b support the detection of disease stability. Therefore, our results further support the premise that the central corneal back surface plays an important role for progression detection in KC patients. This is in good agreement with the previous published literature: Similar to our results, Ribeiro et al.²¹ showed that indices of the corneal back surface facilitate the detection of disease progression. Although a few studies have investigated the capability of image classification for progression prediction using other devices than the one used in this study, this study focused on detecting K_max progression according to structured numerical and categorical data.^22,23 Comparing the performance of our model to that of similar studies, Cohen-Tayar et al.²⁴ classified eyes as “stable” or “unstable” based on clinical decision making (referral to corneal procedures/CXL or observance) and trained a XGBoost algorithm to detect stable or unstable disease from raw data obtained by the Pentacam device (Oculus Optikgeräte GmbH, Wetzlar, Germany). They reported a ROC value of 0.75 when using multiple scans and 0.59 when using a single scan, which is lower than the ROC value of 0.82 found in this study. A possible explanation for the lower ROC value is the wider defined term of progression in the study by Cohen-Tayar et al.²⁴ Another similar study by Jiménez-García et al.²⁵ reported a time delay neural network to detect progression of KC with a sensitivity of 70.8% and a specificity of 80.6%. In their study, patients were measured with the Pentacam device, and progression was defined as a change in KC specific markers above the noise threshold of the device. In contrast to the Placido disc topographer/OCT device used in our study, the device used in the studies by Jiménez-García et al.²⁵ and Cohen-Tayar et al.²⁴ measures the curvature of the back surface by Scheimpflug imaging. Within this regard, it is important to note that the measurements of the Placido disc topography/OCT device cannot be used interchangeably with other devices; therefore, the proposed model cannot be applied to Scheimpflug rotating cameras.²⁶ The advantages of the Placido disc topography/OCT device are the higher resolution and the possibility to measure epithelial thickness maps (ETMs).²⁷ Thus, compared with the most stable predictors of the random forest model, ETM values seem to play a minor role in progression prediction. In addition, specific KC markers are commonly device related, and, to our best knowledge, there are currently no studies investigating ML-based KC progression prediction with the Placido disc topographer/OCT device. In our model, the specific KC marker CSI^B was the predictor with the highest stability. CSI^B is defined as the difference of an average Gaussian curvature (indicated in D) of a central circular zone with a radius of 1.5 mm and the concentric annulus with a major radius of 3 mm and a minor radius of 1.5 mm.⁹ In a previous study,⁹ this index was shown to be able to excellently discriminate between normal corneas and corneas affected by KC. In our study, there was a significant difference in CSI_b values between progressive and stable KC eyes (mean difference = 0.73; P < 0.001).

Although the strengths of the study by Jiménez-García et al.²⁵ lie in the high case number, they built multiple triplets of measurements for each patient to maximize data availability. In contrast, in our study, measurements of each patient were selected only once. In addition, one advantage of our study is that predictions were made on a single baseline examination, whereas in the study by Jiménez-García et al.²⁵ two measurements with a time interval between were required to predict the trend. Generally, using the average of multiple measurements on a single study day would further increase the independence from measurement variability.²⁸ However, this approach was not applicable in this study due to its retrospective nature and the differences in the amount of measurement repeats.

The studies of both Cohen-Tayar et al.²⁴ and Jiménez-García et al.²⁵ used more than one baseline measurement for predicting progression, but only a few other studies have reported on predicting progression with only one baseline measurement, as in our study. For example, Hartmann et al.²² predicted progression at the first visit according to a change in K_max of 1 D by training a combined image and clinical data predictor model and achieved an accuracy of 0.83 with an underlying sensitivity of 53% and a specificity of 95%. The distinct mismatch between sensitivity and specificity may be explained by the existent imbalance of the data. Kato et al.²⁹ used deep learning to predict a change in K_max/cylinder/spherical equivalent of 1 D and reported AUC values of up to 81.4%. However, in their study, the mean follow-up time ranged from 6 weeks to 8.6 years, which can be interpreted as widely inconsistent. In another study, Kamiya et al.²³ showed that age stratification can enhance the predictive power of a deep learning approach to identify progression of K_max using AS-OCT maps.

Shetty et al.³⁰ used three different K_max cut-offs (0.75 D, 1.0 D, and 1.25 D) to predict progression and achieved AUC values of >0.9 in each model, but they found a discordance between the increase in K_max and concomitant associated changes in the other parameters in approximately 40% of eyes. In another study, Reddy et al.³¹ found that image-based analysis may detect progression in keratoconus patients earlier than keratometric-based approaches. An unsupervised ML model introduced by Cao et al.³² identified three different progression clusters, helping to identify eyes with rapidly changing tomographical parameters. In a different approach, Kundu et al.³³ used clinical risk factors such as serum IgE levels or eye-rubbing to predict KC progression and achieved an AUC of 0.812. Particularly, the combination of tomographical and clinical data represents an interesting approach for future prediction studies.

A major challenge in detecting KC progression, both in scientific research and in clinical practice, lies in defining progression itself. The 2015 Global Consensus on Keratoconus and Ectatic Disease recommended that progression should be identified through assessment of steepening of the anterior and posterior corneal surfaces, as well as “thinning and/or an increase in the rate of corneal thickness change from the periphery to the thinnest point.”³⁴ A systematic review by Henriquez et al.⁵ found that 85.5% of included studies used K_max as the criterion for progression, making it the most commonly applied parameter. Accordingly, the primary aim of our study was to detect progression based on changes in K_max.

Generally, in the treatment of KC it is particularly important to detect disease progression in a timely manner; for example, Romano et al.⁷ reported that 25% of patients with progressive KC waiting for scheduled CXL showed further progression in the time span of a few months. Therefore, prediction of progression based on a single measurement could enable more rapid scheduling of CXL. Nonetheless, although the random forest model showed good performance in detecting progressive/stable KC, there were still cases that were classified falsely; therefore, we discourage practitioners from making ML-based decisions regarding referral to CXL. Indeed, our study has other implications that aid future classification of KC progression. Concordant with Jiménez-García et al.,²⁵ we propose that the use of ML regarding KC progression detection may be used for appointment scheduling strategies; if a patient is classified as high risk for progression, then shorter follow-up intervals should be scheduled rather than performing immediate CXL. However, the decision regarding whether or not CXL should be applied should not be made by ML alone until perfect classifiers have been developed. A strength of our study is that we strictly included patients without previous CXL. Although this measure substantially limited the sample size of this study due to the referral-based medical supply strategy, a possible bias of previous CXL on topography could be circumvented. Consequently, in comparison with studies included in a systematic review by Hashemi et al.,³⁵ our study has a lower sample size. It should be mentioned that we used a simple and classic ML algorithm rather than more recent approaches. Although more modern approaches represent interesting and promising alternatives, the limited sample size in the present study favored the use of established algorithms that can be reliably trained and validated in small datasets. Future studies with larger cohorts should investigate whether newer algorithms provide additional predictive benefit.

Due to the limited sample size, the application of nested cross-validation was implemented. The strengths of this method include its ability to provide stable model measures even in small sample sizes, the prevention of data leakage from the test and training dataset, the possibility of filtering important predictors, and its robustness against overly optimistic validation results due to small test sets. Therefore, nested cross-validation can be seen as a valuable alternative to the classic test/train split scheme for studies investigating ML-based prediction algorithms with small sample sizes.

This study has some limitations that must be addressed. First, its retrospective nature comes along with the risk of selection bias. Second, extrapolating progression rates may be seen as controversial, as the progression of KC does not always follow a strict linear tendency. To counteract this issue, we used the follow-up measurement closest to 1 year as ground truth; with mean follow-ups of 11.50 months and 13.06 months and small standard deviations (<3.2 months) in both groups, we achieved homogeneous conditions. Third, an issue that can be found in similar studies is the presence of imbalanced data. This issue is caused by the fact that a majority of KC patients visiting outpatient departments without previous CXL do not have progressive disease. In our study, we tried to counteract this issue by applying decision thresholds in support of the minority class, thus the progressive group. Fourth, despite stratifying progression cut-offs according to the repeatability limits of K_max following a study by Seiler et al.,⁶ a more recent study by Balal et al.³⁶ further refined the precision limits of the device used based on repeatability on a granular level using a large sample size, therefore leading to more tailored decision cut-offs. Using these cut-offs, the classification in progressive/stable disease would have been even more adapted to the repeatability limits of the device. Future studies should implement these cut-offs for progression stratification.

Finally, when possible, we decided to include both eyes of a single patient to increase sample size, which is lower than in previous studies. Despite using data from two independent clinical centers, the difficulty in generating larger data sets relies on the prevalence of the disease and the referral-based healthcare system in Austria. Patients that are referred to a specialized outpatient department are often affected by progressive disease and receive early CXL intervention, which is in contradiction with the respective exclusion criteria of this study.

To further counteract this issue, we decided to use nested cross-validation, a rigorous approach to handle situations with limited sample size helping to avoid optimism bias and overfitting, which is especially important in small samples. However, the implementation of external validation of the model would further strengthen its clinical applicability. Additionally, because patient-level splitting was not performed, data leakage or contamination cannot be completely excluded. However, keratoconus is typically an asymmetric disease, and inter-eye correlations may therefore be limited. Furthermore, fellow eyes may even have different labels (e.g., progressive vs. stable), which further reduces the risk of data leakage.

Conclusions

This study presented a simple yet effective random forest model for predicting the progression of K_max based on a single combined topographic/tomographic measurement. If validated in larger cohorts, such an approach could support the identification of eyes at increased risk of progression and help make decisions regarding follow-up intervals and monitoring frequency. Additionally, our findings suggest that specific keratometric measurements and posterior corneal surface indices may serve as potential markers associated with KC progression. Finally, we can endorse the applicability of nested cross-validation in ML based studies with small sample sizes.