Automatic Determination of Endothelial Cell Density From Donor Cornea Endothelial Cell Images

Abstract

Purpose

To determine endothelial cell density (ECD) from real-world donor cornea endothelial cell (EC) images using a self-supervised deep learning segmentation model.

Methods

Two eye banks (Eversight, VisionGift) provided 15,138 single, unique EC images from 8169 donors along with their demographics, tissue characteristics, and ECD. This dataset was utilized for self-supervised training and deep learning inference. The Cornea Image Analysis Reading Center (CIARC) provided a second dataset of 174 donor EC images based on image and tissue quality. These images were used to train a supervised deep learning cell border segmentation model. Evaluation between manual and automated determination of ECD was restricted to the 1939 test EC images with at least 100 cells counted by both methods.

Results

The ECD measurements from both methods were in excellent agreement with r_c of 0.77 (95% confidence interval [CI], 0.75–0.79; P < 0.001) and bias of 123 cells/mm² (95% CI, 114–131; P < 0.001); 81% of the automated ECD values were within 10% of the manual ECD values. When the analysis was further restricted to the cropped image, the r_c was 0.88 (95% CI, 0.87–0.89; P < 0.001), bias was 46 cells/mm² (95% CI, 39–53; P < 0.001), and 93% of the automated ECD values were within 10% of the manual ECD values.

Conclusions

Deep learning analysis provides accurate ECDs of donor images, potentially reducing analysis time and training requirements.

Translational Relevance

The approach of this study, a robust methodology for automatically evaluating donor cornea EC images, could expand the quantitative determination of endothelial health beyond ECD.

Introduction

The determination of the central endothelial cell density (ECD) of the donor corneal endothelium has been a certification requirement for donor tissue suitability of the Eye Bank Association of America (EBAA) since 2001.¹ However, the training and operating procedures for specular microscopic image capture, quality, and analysis for ECD determination have been left up to the individual eye bank medical directors. The impact of this variability in specular microscopic procedures on ECD measurement accuracy has been well demonstrated with comparisons to an image analysis reading center for standardized image analysis procedures.^2,3

Nevertheless, eye-bank–determined ECD accuracy when compared to an image analysis reading center has significantly improved with training and a similar image analysis method.⁴ Tissue warming implementation prior to imaging has improved image quality,^5,6 and wider field eye bank specular microscopy (Konan CellChek D; Konan Medical, Irvine, CA)⁷ has enabled the the analysis of more endothelial cells (ECs) from multiple areas within the image field. These improvements, however, may come at the expense of longer image analysis time and greater technician training. The EBAA has also recently established a standardized specular microscopic image capture technique to maximize image quality.⁸ Whether the central ECD is determined by the eye bank or an image analysis reading center, the number of ECs analyzed remains limited to usually no more than 100 to 150 cells using a manual method of counting, principally the center method.⁷ The accuracy of this approach can be influenced by cell selection, poor focusing, deficient image quality, improper application of the center method, Desçemet folds, and confounding image opacities (e.g., blood, debris, guttae).⁷ Furthermore, Huang et al.⁹ reported that the Konan CellChek fully automated analysis overestimated ECDs in central EC images with polymegethism and glaucomatous eyes.

Currently, a fully automated, commercially available donor cornea endothelial image analysis system is not on the market. Given the challenges of technician training, turnover, and work demands; image quality issues; and image analysis errors, an efficient, accurate, and automated analysis of donor EC images would be ideal. With the plentiful quantity of donor specular microscopic EC images routinely obtained as part of the standard operating procedure for eye banks, this is a promising opportunity for machine learning modeling.

Our group recently reported the application of deep learning segmentation to post-keratoplasty images and identified machine learning classifiers predicting rejections 1 to 24 months prior to a clinically apparent rejection.¹⁰ With lessons learned from our in vivo clinical machine learning image analysis work, we have directed efforts toward the more challenging machine learning image analysis of donor ex vivo endothelial images with higher ECDs with smaller cells, lower magnification, and optical interference from the plastic viewing storage chamber, confounded by other imaging issues outlined above. Our objective with this study was to develop a deep learning model that can accurately, rapidly, and objectively determine the ECD of the donor cornea endothelium to facilitate a more efficient determination of this key element in tissue suitability.

Methods

Unlabeled Data Cohort

Eversight (Ann Arbor, MI) and VisionGift (Portland, OR) eye banks provided central donor corneal EC specular microscopic images obtained at screenings between 2010 and 2023. A subset of only those donors that went on to keratoplasty with a valid manual ECD measurement or who were deemed keratoplasty ineligible based on slit-lamp findings were included. Utilizing the Konan CellChek D specular microscope, Eversight provided one analyzed image each with a single selected cell area from 13,942 donor corneas, and VisionGift provided one analyzed image with up to three selected cell areas from 1196 donor corneas.

Both eye banks participated during this period in the National Eye Institute–supported Specular Microscopy Ancillary Study (SMAS)^3,11 and Cornea Preservation Time Study (CPTS),¹² as well as the Diabetes Endothelial Keratoplasty Study (DEKS)¹³ involving defined donor specular microscopy procedures. Technician training at both eye banks has been previously described.⁴ Donor data obtained included donor age and sex, cause of death, diabetes status, lens type, death to preservation time, death to imaging time, tissue suitability evaluations, and the keratoplasty type the donor was used for. Both eye banks analyzed a minimum of 40 ECs per image and ideally more than 100 ECs, and they provided their final ECDs to our biostatistician (RCO) for inclusion in the statistical analyses.

Labeled Data Cohort

A second subset of 174 images (81 Eversight, 93 VisionGift) selected from the same dataset reported by Huang et al.⁴ was utilized for supervised training of the deep learning algorithm. The selection criterion for this subset was analyzability as determined by an expert reader analyst (BMB). These images underwent automatic cell border segmentation as previously described.¹⁴ Following postprocessing, the predictions were manually edited using an in-house centroid-selection algorithm.¹⁵ The centroid-selection algorithm started with identifying cellular connected components and their centroid locations. These were visualized by plotting the centroids as red circles and overlaying the binarized segmentation prediction in green on the original EC image displayed in the software viewing window. From here, the image analyst clicked on visible centroids to exclude them from the final segmentation if the corresponding cell borders were incorrectly annotated. The final ground-truth labels were binary segmentations with single-pixel-width white borders (pixel value 1) and black cells and background (pixel value 0). To enhance the deep learning training performance, the borders were dilated with a 3 × 3 square kernel for one iteration.

Deep Learning Networks

In this study, training was conducted in two stages and employed two network architectures (Fig. 1). A self-supervised learning technique was implemented in the first stage using the Vision Transformer (ViT) architecture.¹⁶ The second stage involved using supervised learning to train a U-Net Transformer (UNETR) model.¹⁷ Self-supervised learning enables the model to learn important imaging features by masking areas of the training images and evaluating the attempt of the model to recreate these masked areas via comparison with the ground-truth input images. Supervised learning requires labeled datasets to train deep learning algorithms to perform a desired task—in this study, cell-border segmentation. The benefit of combining these deep learning techniques is that the model can achieve high segmentation accuracy performance despite a limited labeled dataset by utilizing thousands of unlabeled images to learn pre-existing features.

Figure 1.

Training workflow for deep learning–based endothelial cell segmentation. Self-supervised training was performed using 12,114 images through a ViT to learn feature representations without labeled data. Learned weights were then used to initialize a UNETR for supervised cell border segmentation training with 174 images and labels. Final output after postprocessing was a single-pixel-width cell border segmentation from which ECD was automatically calculated. The illustrations of the ViT and UNETR are based on work by Dosovitskiy et al.¹⁶ and Hatamizadeh et al.,¹⁷ respectively.

The ViT and UNETR architectures first deconstructed an image into a sequence of patches imitating natural language processing (NLP) transformer models that interpret sentences as a sequence of words. Each patch of the input image was 16 × 16 pixels. The ViT is composed of a transformer encoder with two convolutional transpose layers to output a reconstructed image of the same size as the input image.¹⁶ Because the ViT architecture is the backbone of the UNETR, the weights learned during the ViT training initialized the UNETR encoder weights. The UNETR decoder weights remained randomly initialized, refined thereafter during supervised learning.

Data augmentations, network parameters, and training parameters (i.e., loss functions, validation metrics, and learning rates) described by Tang et al.¹⁸ were utilized and are described briefly. Prior to ViT self-supervised training, input images were scaled to [0, 1] and randomly cropped into 10 samples of 128 × 128 pixels before an additional copy of the cropped image was generated. From here, random patches no larger than 64 × 64 pixels were either dropped out or mutated from both image copies independently. The ViT was trained with two loss functions: contrastive loss¹⁹ and mean absolute error. The contrastive loss maximized agreement between the reconstructed image patches of the original input and the reconstructed image patches of the copied input. The mean absolute error loss was used to minimize the absolute pixel value difference between the reconstructed original input image patch and the ground-truth input image patch. The data augmentations of the supervised learning task with UNETR shared intensity scaling and random cropping of input images into 10 samples of 128 × 128 pixels. Thereafter, images were randomly flipped, rotated, or underwent a 10% shift in intensity values. The UNETR model was trained using a combined Dice and cross-entropy loss.

Deep Learning Experiments

The 15,138 unlabeled donor EC images from 8169 donors were rescaled from 1296 × 972 to 1004 × 1338 pixels to maintain a 0.8-µm resolution shared with the 174 EC images of the labeled dataset. For self-supervised learning, a type of unsupervised learning where the model predicts parts of its input data from other features of the same data, 9085 randomly selected EC images (8327 from Eversight, 758 from VisionGift) were used to train our ViT model, and the remaining 3029 EC images (2807 from Eversight, 222 from VisionGift) were used for validation. The ViT model was trained for 300 epochs with batch size of 48 and 1e-4 learning rate.

For supervised learning, the best ViT model weights were loaded onto a UNETR model before fine-tuning on 174 labeled donor EC images (139 for training and 35 for validation) for 215 epochs, equivalently 30,000 iterations, with a batch size of 1. Finally, 3024 unlabeled donor EC images (2808 from Eversight, 216 from VisionGift) were used as the held-out testing dataset for the final UNETR model to conduct inference. The final model binarized each segmentation prediction by determining which class (border or cell/background) received the higher probability. The deep learning training workflow is illustrated in Figure 1.

Post-Processing

Predicted segmentations contained thick cell borders and stray “branches,” a consequence of fine-tuning with labels that contained 3-pixel-wide cell borders and varying image quality and physiological artifacts in donor EC images. Thus, a postprocessing step was implemented (see Supplementary Fig. S1). The prediction segmentations were first skeletonized to single-pixel width, after which stray borders were opened and subsequently closed with a 3 × 3 square kernel to remove smaller disconnected segments and fill small cell border gaps, respectively. Finally, all external, stray “branches” or borders were removed by exploiting an eight-way connected component function from Bolelli et al.²⁰ to find endothelial cells fully enclosed by a segmented border.

Clinical Morphometrics

ECD was determined for the whole image and a cropped image. To calculate the ECD for the whole image, the predictions underwent a filling operation to convert each fully enclosed cell from black pixels to white. This area of white borders and enclosed cells was deemed as the cellular area. The total number of pixels in the cellular area was multiplied by the pixel area (e.g., 0.8 µm × 0.8 µm) to obtain the physical cell area. The number of enclosed cells, determined by a connected components function, was divided by the physical cell area to obtain the ECD for the segmentation. ECD was calculated for the cropped image by using a circular region to encompass the manually annotated cells after postprocessing for the whole image analysis was complete.

Statistical Analyses

Statistical analyses were performed using R 4.3.2 (R Foundation for Statistical Computing, Vienna, Austria) with the SimplyAgree, boot, boot.pval, and tidyverse packages.^21–25 Agreement between manual and automated determinations of ECD was evaluated with Bland–Altman analysis and Lin's concordance correlation coefficient (r_c).^26,27 Associations among donor age, diabetes, death to preservation time, and ECD (average of manual and automated measurements) and differences in ECD between automated and manual methods were explored via linear regression models. Analysis was restricted to the test EC images with at least 100 cells counted by both methods to ensure reliable results.² Cluster bootstrap resampling to account for clustering of cornea EC images from the same donor was used to assess the statistical significance of differences between the automated and manual methods. P values were obtained via confidence interval inversion, and a two-sided P < 0.05 was considered statistically significant.

Computational Resources

All image processing and deep learning algorithm development were conducted using Python 3.9 libraries such as MONAI, PyTorch (version 1.13.1), numpy, opencv, scikit-learn, and PyQt. ViT and UNETR were trained using four 48-GB NVIDIA A600 GPUs (Nvidia Corporation, Santa Clara, CA). The overall architecture required approximately 87.1 million learning parameters and approximately 520 GFLOPs per image. Self-training took approximately 3 hours, and fine-tuning using one NVIDIA GPU took approximately 2 hours for training. The inference pipeline of segmentation, postprocessing, and ECD calculation required less than 2 seconds.

Results

Of the 3024 EC test images, there were 1939 (66%) with at least 100 ECs counted by both methods (see Supplementary Fig. S2). Within this cohort, the proposed automated approach identified between 100 and 1263 ECs per image, whereas the manual approach identified between 100 and 371 ECs per image. This difference in cell identification is illustrated in Figure 2, where example images from Eversight and VisionGift have undergone both automated and manual analyses. Furthermore, it is evident that the automated analysis is not limited to the brightest region of the image, nor a contiguous region of cells. In fact, the automated approach identified many cells in the darker regions of the Eversight image.

Figure 2.

Example of donor cornea endothelial cell images and comparative display of corresponding manual and automatic annotations overlay. Top row: (1a) and (2a) are specular microscopic images from donor cornea endothelium provided by Eversight and VisionGift, respectively. Bottom row: (1b) and (2b) depict automated cell border annotations (green) and manual cell centroid annotations (red, blue, and purple).

The ECD measurements from the two methods were in excellent agreement with r_c of 0.77 (95% confidence interval [CI], 0.75–0.79; P < 0.001) and bias (mean difference) of 123 cells/mm² (95% CI, 114–131; P < 0.001). Eighty-one percent of the automated ECDs were within 10% of the manual ECDs. When the analysis was further restricted to the cropped image, the r_c was 0.88 (95% CI, 0.87–0.89; P < 0.001), bias was 46 cells/mm² (95% CI, 39–53; P < 0.001), and 93% of the automated ECDs were within 10% of the manual ECDs (Table). The Bland–Altman plots with limits of agreement for the whole and cropped images are displayed in Figures 3A and 3B, respectively.

Table.

Comparison of Whole and Cropped Image ECD for Test Images by Manual and Automated Methods (N = 1939 EC Images)

ECD (cells/mm2)	Manual Test, Mean ± SD [Min, Max]	Automated Test, Mean ± SD [Min, Max]	Bias (95% CI)a	rc (95% CI)
Whole image	2730 ± 349)	2607 ± 272)	123 (114–131)*	0.77 (0.75–0.79)*
	[1393, 4132]	[1612, 3607]
Cropped image	2730 ± 349)	2683 ± 277)	46 (39–53)*	0.88 (0.87–0.89)*
	[1393, 4132]	[1655, 3765]

^aCluster bootstrap percentile confidence intervals are based on 100,000 bootstrap samples.

^*P < 0.001.

Figure 3.

(A) Bland–Altman plot of whole image manual versus automated ECD (N = 1939). Bland–Altman plot of whole image ECDs (cells/mm²) determined by study eye banks versus the ECDs determined by the deep learning model. The difference in ECDs is plotted against their average value. The Bland–Altman plot shows a small, estimated bias (mean difference) of 123 (95% CI, 114–131), with a lower limit of agreement (LoA) of −234 (95% CI, −248 to −220) and an upper limit of agreement of 479 (95% CI, 462–497). Auto, automated. (B) Bland–Altman plot of cropped images of manual versus automated ECDs (N = 1939). Bland–Altman plot of cropped image ECDs (cells/mm²) was determined by study eye banks versus the ECDs determined by the deep learning model. The difference in ECDs is plotted against their average value. The Bland–Altman plot shows a small, estimated bias (mean difference) of 46 (95% CI, 39–53), with a lower limit of agreement of −243 (95% CI, −258 to −228) and an upper limit of agreement of 335 (95% CI, 319–352).

In the univariable and multivariable linear models predicting differences in ECD between manual and automated methods, only donor age was statistically significant (P < 0.001) but not clinically significant. The estimated coefficient for donor age in the multivariable model was −25 cells/mm² (95% CI, −33 to −17) per decade. For the cropped image, the estimated coefficient for donor age in the multivariable model was −20 cells/mm² (95% CI, −27 to −14) per decade (see Supplementary Table S1). Differences in ECDs between the manual and automated methods monotonically increased with increasing ECD (P < 0.001); the estimated increase from the linear model was 134 cells/mm² (95% CI, 122–147) per 500 cells/mm² for whole images and 120 cells/mm² (95% CI, 110–131) per 500 cells/mm² for cropped images (see Supplementary Table S2). This “proportional bias” results in limits of agreement that tend to be too wide for donor corneas with lower ECDs and too narrow for corneas with higher ECD values.²⁸

Discussion

We have shown that the donor cornea ECD determined by our automated, deep learning algorithm compares favorably to the real-world eye-bank–determined ECDs from two major U.S. eye banks. The two methods only differed by a mean of 123 cells/mm², and 81% of the automated ECDs were within 10% of the eye-bank–determined ECD; the agreement was even tighter when the analysis was conducted within the comparable cropped area of cells that the eye bank had analyzed. Only a mean difference of 46 cells/mm² and 93% of the automated ECDs were within 10% of the eye-bank–determined ECD. Although these differences were statistically significant, from a medical director standpoint these differences would not impact tissue suitability. Not surprisingly, the difference between the automated-determined and eye-bank–determined ECDs increased with increasing ECD, whether with the whole or cropped image. Similar findings have been reported comparing an automated approach for cell segmentation of in vivo EC images to a flex-center manual analysis.²⁹ Other authors, however, have reported a decrease in the difference between automated and manual ECD methods as more cells in post-keratoplasty EC images were identified.³⁰

Notably, for the analysis of donor factors that could affect the accuracy of automated analysis (specifically, a more polymegethetic endothelial cell population with increasing donor age^7,31 and diabetes³²), only donor age was found to have a statistically but not clinically significant difference (difference of −25 cells/mm²; 95% CI, −33 to −17) per decade in the multivariable model). This difference may be due to technician cell selection with greater polymegethism and pleomorphism with increasing age.^33,34 Conversely, the automated approach is applicable across the entire spectrum of donor conditions. With comparable ECDs between the two ECD analysis methods, the advantages of the automated analysis are its rapidity for ECD determination within seconds and technician training being primarily limited to image acquisition.

Machine learning techniques to automate methodologies for determining endothelial characteristics and morphometrics with in vivo corneal endothelial image analysis have been more commonly described.^9,30,35–43 These efforts have included U-Net–based convolutional neural networks to automatically identify cell edges within in vivo EC images⁴² and a sliding window technique in similar network architectures for cell border segmentation.^37,39 ECD calculation improvement was achieved with DenseUNets for cell border segmentation of in vivo EC images and watershed-based postprocessing to clean corresponding deep learning predictions.⁴¹ Most recently, efforts to reduce computational resources of deep learning algorithms while maintaining high performance have included parallel networks for cell border segmentation and region of interest extraction followed by a postprocessing pipeline.⁴³ Our efforts build on these works by using self-supervised learning to train transformer network cell border segmentation of donor endothelial images. These images have distinct tissue morphologies, imaging artifacts, and pathologies in comparison to clinical in vivo EC images.

The number of ECs that must be analyzed to be reflective of the central ECD has been debated.^33,44–50 Estimates have ranged from at least 30 cells per image³³ to as many cells as possible,⁴⁵ all influenced by the practical limitations of technician time and analysis method accuracy.⁷ Accurate automated ECD analysis with or without manual intervention is commonly performed clinically with images of good to excellent image quality for a homogeneous cell population with normal cell structure and in the absence of endothelial pathology.^9,51–55 However, unlike clinical endothelial image analysis, a fully automated analysis of donor endothelium for ECD has not proven successful,⁵⁶ whereas semi-automated analytical approaches currently employed in eye banking have raised questions of automation error, human bias, and technique.^2,3,57 The EBAA Procedures Manual does not specify the ideal number of endothelial cells to be analyzed; it simply states that, to obtain the most accurate analysis, a large field and/or multiple fields should be captured, counted, and/or averaged.⁸ In our study, we chose a minimum of 100 ECs minimum to be analyzed by the eye bank, as is commonly practiced in eye banking. Our proposed automated approach for image analysis and donor central ECD determination would more than satisfy EBAA procedural standards, with efficiency and without bias.⁸

Donor specular microscopy and central ECD analysis have been crucial in determining a minimum ECD of at least 2000 cells/mm² as part of donor tissue suitability over the past 30 years. The contribution of the determination of donor central ECD toward donor tissue suitability has paid off, as primary graft failure is uncommon in the United States, with a rate of 0.2% annually between 2019 and 2022 according to the EBAA Online Adverse Reaction Reporting System (Jennifer DeMatteo, personal communication, 2024). Notably, long-term graft success (e.g., at 5 years for Fuchs dystrophy at a single site) following primary penetrating keratoplasty, Desçemet stripping automated endothelial keratoplasty (DSAEK), and Desçemet membrane endothelial keratoplasty (DMEK) has been excellent: 95%,⁵⁸ 93%,⁵⁹ and 93%,⁵⁹ respectively. However, for the use of deep learning–determined donor central ECD as part of donor tissue suitability to be accepted by the eye banking and surgeon community, there will have to be data relating the deep learning–determined ECD to short-term primary graft failures and long-term graft outcomes (graft success, endothelial cell loss).

Other fields of ophthalmology with large quantities of diagnostic images have already taken advantage of machine learning modeling and their AI applications toward management decisions.^60–68 Initial diagnostic AI models developed using in vivo clinical images have been trained to predict corneal disease such as microbial keratitis, keratoconus, dry eye syndrome, and Fuchs dystrophy.^36,69 One of the most expeditious ways for graft outcomes to be related to deep learning–determined donor ECDs is for single-site and multicenter study groups with access to the donor images of their recipients to compare automatically determined ECDs to eye-bank–determined ECDs and the graft outcomes of corresponding recipients.

Our study had several limitations. First, our two eye banks differed in the number of areas analyzed for each donor's central endothelium, so their representation of the central endothelium may have differed; however, the specular microscope utilized, technician training, and image analysis techniques did not differ. In this study, the cell border segmentations of the test set donor cornea endothelial images were not manually edited, as were the 174 donor cornea endothelial images used for supervised deep learning training. Although our study proposes an automatic approach for ECD determination, a brief manual check of the image segmentation would not be unprecedented. Furthermore, we are continuing our efforts to optimize and prove the consistency, robustness, and generalizability of our proposed deep learning segmentation algorithm by investigating other transformer network architectures and self-supervised learning approaches such as generative adversarial learning (e.g., boundary attention learning). Also, the influence of polymegethism and pleomorphism on the comparison between the eye-bank–determined ECD and the deep learning–determined ECD was not directly measured. As McCarey et al.⁴⁹ pointed out, due to the greater variation in cell area and cell shape, cell selection has a greater effect on ECD determination; thus, cell selection and bias by the technician to analyze an image area with possibly smaller cells resulting in a higher ECD could have an influence on the central ECD determined. This influence may have been present, as we noted higher agreement between the eye-bank–determined ECD and deep learning–determined ECD when the deep learning analysis was cropped to approximately the same area of cells that the eye bank analyzed. Examining the coefficient of variation and percentage of hexagonal cells could perhaps shed further light on this finding but was beyond the scope of our study.

In summary, an automated image analysis approach involving deep learning techniques provided accurate ECDs of donor images comparable to real-world eye-bank–determined ECDs for donors suitable for keratoplasty and achieved a significantly reduced analysis time. This analytical approach would also eliminate cell selection bias and limit training requirements to the best conditions and techniques for achieving excellent donor endothelial image quality. It also offers the potential for future studies that would go beyond ECD as a tissue suitability marker with the discovery of machine learning morphologic features, which, as we have shown with postoperative image analysis and graft rejection,¹⁰ may be even more predictive of graft success.