Three-Dimensional Map of Descemet Membrane Endothelial Keratoplasty Detachment

Abstract

Purpose

To develop and apply a neural network for quantification of endothelial corneal graft detachment using anterior segment (AS) OCT.

Design

Training and validation of a neural network and application within a prospective cohort.

Participants

Patients two weeks after Descemet membrane endothelial keratoplasty.

Methods

Investigators manually labeled the posterior cornea and the graft in cross-sectional images of rotational AS OCT scans. Neural networks for image segmentation were trained to identify the area of graft detachment on cross-sectional images. The best-performing neural network with the lowest misclassification (Youden index) and highest spatial overlap with the ground truth (Dice coefficient) was selected and evaluated in a separate dataset. Three-dimensional maps of the area and volume of graft detachment were calculated. For application, the neural network’s rating on the detachment was compared with slit-lamp–based ratings of cornea specialists on the same day as the AS OCT imaging took place.

Main Outcome Measures

Youden index and Dice coefficient.

Results

Neural networks were trained on 27 AS OCT scans with 6912 labeled images. Among 48 combinations of probability thresholds and epoch states, the best-performing neural network showed a Youden index of 0.99 and a Dice coefficient of 0.77, indicating low misclassification and good spatial overlap on individual image segmentation. In the validation set unknown to the neural network with 20 scans (5120 images), the Youden index was 0.85 and the Dice coefficient was 0.73, and a high overall performance compared with the manually labeled ground truth (R² = 0.90). In the application set with 107 eyes, the neural network estimated the mean percent detachment larger than the cornea specialist (mean difference, 8.2 percentage points; 95% confidence interval, 6.2–10.2). Masked review of 42 AS OCTs with more than ±10 percentage points difference in ratings showed that clinicians underestimated the true detachment in cases with significant detachment requiring intervention.

Conclusions

Deep learning-based segmentation of AS OCT images quantified the percent and the volume of DMEK graft detachment with high precision. Fully automated 3-dimensional quantification of graft detachment is highly sensitive, particularly in corneas with a significant amount of detachment, and may support decision making.

Incomplete graft attachment is the number one complication after Descemet membrane endothelial keratoplasty (DMEK).¹ The decision to perform a rebubbling procedure to attach the graft relies on the “one third rule of thumb,” which suggests rebubbling with air or gas tamponade if more than 33% of the graft surface is not attached.^2,3 Alternative strategies rely on watchful waiting over months.⁴ The impact of such strategies to address incomplete attachment on long-term graft survival is unknown.^1,5,6 To identify and modify risk factors of incomplete attachment, define indications for rebubbling precisely, and assess the impact of incomplete graft attachment on long-term outcomes, exact quantification of detachment will be key.

To assess the amount of graft detachment, clinicians and researchers at present rely on their own mental map of the detached area based on individual cross-sectional anterior segment (AS) OCT images or on slit-lamp examinations.^2,7, 8, 9, 10 Custom-made segmentation algorithms of high-resolution AS OCT images allow detection of the presence of graft detachment,¹¹ length of detachment,¹² and thickness of the endothelium–Descemet complex in Fuchs’ dystrophy.¹³ In neuro-ophthalmology and retina disciplines, 3-dimensional maps are used successfully in clinical practice for evaluation of the ganglion cell layer or retinal fluid on macular OCT imaging. We believe a similar tool could be useful for corneal grafts for a more profound understanding of graft detachment and an objective evaluation of therapeutic approaches.

In this study, we trained and validated a neuronal network to generate maps for the percent detachment and for 3-dimensional volume representation of the detachment after DMEK. To this end, the posterior corneal curvature and, if present, the detached graft were labeled manually on cross-sectional AS OCT images. The best-performing neural network in the training set was selected, locked, and validated in a separate dataset of AS OCT scans not used for training. Next, we applied the neural network to eyes from a prospective study unknown to the neural network, with rating of the percent graft detachment by corneal experts based on slit-lamp examination.

Methods

Study Population

All participants were enrolled in a prospective corneal endothelial transplantation study (German Clinical Trials Registry identifier, DRKS00020947).^14,15 Written consent was obtained from all patients. The study was approved by the institutional ethics committee at the Faculty of Medicine Freiburg and conformed to the tenets of the Declaration of Helsinki. In brief, participants underwent routine DMEK surgery for Fuchs’ dystrophy or pseudophakic bullous keratopathy, combined with cataract surgery if participants were phakic before DMEK. Participants with previous surgery other than cataract surgery or infections in the anterior segment were not included in the study. To assess graft attachment 2 weeks after DMEK, all participants underwent a clinical slit-lamp examination and corneal imaging using an AS OCT (Casia-1; Tomey GmbH).

Image Acquisition and Processing

From each individual 360° AS OCT scan, 256 cross-sectional images of the anterior segment of the eye in 15° steps were acquired in a routine clinical practice setup. The resolution of each image was 512 × 900 pixels. The bit-depth for a single pixel was 16 bits, corresponding to 2¹⁶ gray values from pure black to white.

Each scan was exported as a single file in the instrument’s proprietary file format and was separated again in 256 images using a custom-made extraction tool. To enhance contrast and to resize images, all images were postprocessed and stored as 8-bit images (ImageJ; National Institutes of Health).¹⁶ In detail, a median filter was applied with a kernel size of 2 × 2 replacing each pixel by the median grey value of 3 neighboring pixels to reduce random noise. The image was cropped automatically to remove structures not of interest, for example, the iris, and resized. For labeling, an image size of 1350 × 600 pixels was selected to mimic the format of the AS OCT image viewer. To train the neural network, images again were resized to 512 × 512 pixels, satisfying prerequisites of a square image format as a multiple of 2ⁿ pixels.

Ground Truth Annotation of Anterior Segment OCT Images

For each image, 1 of 5 investigators (A.G., V.G., M.F., C.M., and K.W.) labeled the posterior corneal curvature and, if present, the detached graft using custom-made software (Fig 1) after completing a set of training images. To trace the contour and label the structures, the software drew a line by connecting adjacent points that were placed on the hyperreflective layer of the posterior cornea or the graft (click tracing). Investigators first reached a consensus regarding how to label the posterior cornea and the graft. Investigators set labels only if the graft was visible on the slide. If the graft was visible on adjacent slides only, but not on the image itself, the investigators set no labels. One reviewer then manually re-reviewed all annotated images and, if necessary, in very rare cases, made corrections to the annotations.

Figure 1.

Diagrams showing study design: (1) anterior segment (AS) OCT captures a 360° radial scan with 256 individual images; (2) the posterior cornea and the endothelial graft, if detached, were labeled manually by click tracing or were identified by the neural network; (3) the number of pixels between the graft and the posterior cornea were calculated, resulting in a histogram; and (4) using polar transformation and postprocessing, a 2-dimensional map of the area and a color-coded 3-dimensional map of the volume of detachment were generated.

To determine the area of graft detachment on each individual image, an inverted second-degree polynomial function centered at the anterior corneal apex of each image was used. All pixels between the graft and the posterior cornea located on the normal, the perpendicular line between the polynomial and the graft, were calculated using a modified Bresenham’s line algorithm.¹⁷ To fill gaps in the resulting diagonal pixel raster, a binary morphologic close operation was applied for vertical columns. The area of detachment identified based on the annotations of posterior cornea and graft by 1 investigator per image was highly reproducible between investigators; on 19 representative training images, the between-rater intraclass correlation coefficient from a 2-way random-effects linear model was 0.99 (95% confidence interval, 0.99–1.00).

Training of the Neural Network

A neural network for image segmentation, the UNet++,¹⁸ was trained to detect the area of graft detachment in the manually labeled training images. The implementation of the network for this study was based on open-source libraries (TensorFlow version 1.4.1 [www.tensorflow.org]; Keras version 2.2.2 [www.keras.io]).¹⁹ We trained a convolutional neural network with a range of epochs (50, 150, and 250) using 85% of randomly sequenced training images. The number of training images was augmented artificially using random vertical and horizontal shifting (−20% to +20%), horizontal flipping, decreasing or increasing image size (–0.1% to +0.1%), image rotation (−10° to +10°), and image shearing (–0.1% to +0.1%).²⁰ In the training set, the neural network used 15% of the training images for validation purposes. All networks used the weight optimizer Adam to update network weights of training data and a binary cross entropy loss function.²¹

Selection of the Neural Network

To select the best-performing neural network, we compared the area of graft detachment identified by the neural network with the manually labeled area of graft detachment on each image. On each image, we determined the proportion of matching pixels of graft detachment (true positive [TP]), the number of matching pixels of no detachment (true negative [TN]), the number of falsely labeled pixels of detachment identified by the neural network only (false positive [FP]), and the number of pixels of detachment not identified by the neural network (false negative [FN]). Mean sensitivity (true positive pixels over total manually labeled positive pixels) and mean specificity (true negative pixels over total manually not-labeled pixels) were calculated for each image.

To select the best-performing network with equal weights for false-positive and false-negative values of image misclassification, we calculated the Youden index (J) and the Sørensen–Dice coefficient (DSC), indicating similarity and special overlap between 2 samples or pixels the closer the coefficients approach a value of 122, 23, 24:

$$J=\frac{TP}{TP+FN}+\frac{TN}{TN+FP}-1\text{and}$$

$$DSC=\frac{2\ast TP}{2\ast TP+FP+FN}.$$

Postprocessing of Images and Development of a 3-Dimensional Map

To display the area and volume of graft detachment on a 2-dimensional map, all 256 cross-sectional individual images of an AS OCT scan were combined to a detachment map using polar transformation and were postprocessed. The volume of detachment in the map was recorded by the height of detached pixel columns for each pixel. To improve visualization of the maps, an opening gray morphologic operator and a Gaussian filter were applied. To remove artifacts, each map was subtracted from its respective binarized detachment map to remove detachments of 1 pixel only and detachment areas of fewer than 300 pixels.

To create true heatmaps indicating both surface and area of detachment, the height of detachment was color coded using a color scheme visible for users with red–green deficiency. The areas of detachment were calculated based on positive pixels of the final, binarized map and were multiplied by the size of a pixel. After converting the area of detachment from pixels to square millimeters, the percent of the area of detachment was calculated with respect to the trephined graft size.

Validation of the Selected Neural Network in a Separate Set of Eyes

The best-performing neural network was validated with AS OCT scans from eyes unknown to the network. Images were selected randomly from the study cohort according to the slit-lamp–graded percent detachment in a 3:7:10 ratio (no detachment, <1/3 of detachment, and ≥1/3 of detachment). Preprocessing, manual labeling, and evaluation of the trained network were identical to those of the training set. Overall performance of the network on detecting the total amount of detached pixels per AS OCT scan compared with the gold standard was assessed using R² values.

Application of the Neural Network in a Prospective Study

To evaluate and apply the best-performing neural network further, we leveraged AS OCT images of participants from the same prospective cohort study who were not assigned to the training and validation cohort.^14,15 In the study, the percent graft detachment was estimated by cornea specialists based on slit-lamp biomicroscopy 2 weeks after DMEK. Cornea specialists were instructed to hatch the area and to estimate the percent detachment as illustrated in Supplemental Figure 1. Interventions were defined as repeat air injection (rebubbling) or repeat clinical visit for incomplete graft attachment. All study eyes were included that had undergone AS OCT imaging on the same day of the cornea specialists’ evaluation. The neural network’s estimate of the area of detachment was resized with respect to the graft’s size as determined by the trephine used for DMEK preparation (percent detachment). We compared the difference in percent graft detachment and the overall performance of the network on detecting the total amount of detached pixels (R²). In case of more than 10 percentage points of absolute difference in rating (an arbitrary cutoff), the complete AS OCT scan was reviewed manually by 2 investigators in parallel (A.G. and K.W.), blinded to each other, the network’s rating, and the slit-lamp–based rating, to estimate the percent detachment based on the individual images of the radial scans.

Results

Training Set Characteristics

Twenty-seven participants with Fuchs’ dystrophy contributed 27 AS OCT scans to the training set. The AS OCT scans were acquired at a median of 16 days (interquartile range [IQR], 13–19 days) after DMEK (Table 1). The training set consisted of 6912 manually labeled AS OCT images, including images with detached and attached grafts. Fourteen scans showed no graft detachment, 7 scans showed graft detachment of less than one-third of the area, and 6 scans showed significant graft detachment (≥30% of the graft’s area). The investigators used a median number of 72 connected line segments (IQR, 23–94 segments) per image for labeling the posterior corneal shape. Any amount of graft detachment was present in 4027 images (58%). In these images, the graft was labeled on median with 11 connected lines (IQR, 6–21 lines).

Table 1.

Participant Characteristics

Characteristic	Training Set	Validation Set	Application
Baseline
No. of participants (eyes)	26 (27)	17 (20)	103 (107)
Age (yrs)	69 (62–79)	73 (58–78)	69 (62–78)
Women (%)	14 (54)	12 (71)	70 (68)
DMEK plus cataract surgery (%)	17 (63)	11 (55)	67 (63)
Trephine size (mm) (%)
7.0	0 (0)	0 (0)	0 (0)
7.5	3 (11)	3 (15)	9 (8)
8.0	24 (89)	17 (85)	98 (92)
Postoperative evaluation 2 wks after DMEK
Area of graft detachment∗	0 (0–20)	20 (5–30)	0 (0–10)
Intervention, no. (%)
No action required	20 (74)	12 (60)	90 (84)
Clinical reevaluation	2 (7)	1 (5)	8 (7)
Rebubbling	5 (19)	7 (35)	9 (8)
Manually labeled images	6912	5120

DMEK = Descemet membrane endothelial keratoplasty; IQR = interquartile range.Continous data are presented as median (interquartile range), and categorical data as count (%).

^∗Cornea specialists estimated the area of graft detachment in percent using slit-lamp biomicroscopy.¹⁵

Neural Network Performance and Selection in the Training Set

We compared 16 probability thresholds (range, 0.001–0.5) and 50, 150, or 250 epoch states in neural networks trained to detect the area of graft detachment. Among 48 combinations (Supplement Table 1), the best-performing neural network was the network with 250 epochs and a probability threshold of 0.015 with a Youden index of 0.994 and a Dice coefficient of 0.767, indicating low misclassification and good spatial overlap of the pixels labeled by the neural network compared with the manually labeled gold standard. In the training set, the neural network’s sensitivity was 0.998 and specificity was 0.997.

Combining all 256 images of an AS OCT scan resulted in a map of the area and the volume of detachment (Fig 1). Comparing the total detachment area of the neural network with the heatmaps calculated from manually labeled images, the neural network achieved a Youden index of 0.994, a Dice coefficient of 0.836, a sensitivity of 0.970, and a specificity of 0.965. The overall performance of the neural network compared with the manually labeled images was high (R² = 0.96; Fig 2).

Figure 2.

Examples from the training cohort. Maps of the volume of graft detachment derived from 6912 manually labeled anterior segment OCT images (gold standard) and images labeled by the neural network (UNet++) are shown for randomly drawn examples from the training cohort. The color gradient displays the height of detachment from no detachment (dark blue; pixel value, 0) to high detachment (yellow; pixel value, ≥70).

Validation of the Final Neural Network

The best-performing neural network in the training set was evaluated in a separate validation set of 20 AS OCT scans with 5120 manually labeled images unknown to the neural network. Three scans were without graft detachment, 7 scans showed graft detachment of less than one-third of the area, and 10 scans showed significant graft detachment (≥ 30% of the graft’s area). All AS OCT scans were acquired at a median of 16 days (IQR, 13–19 days) after DMEK (Table 1). The median number of manually set line segments for labeling the posterior cornea was 97 (IQR, 91–103 segments). In 3164 images (62%), any amount of graft detachment was present and manually labeled a median of 28 lines per image (IQR, 13–56 lines).

In the validation set (Fig 3), the neural network achieved a Youden index of 0.850 and a Dice coefficient of 0.729, with a sensitivity of 0.854 and a specificity of 0.996. At determining the area of graft detachment, the neural network achieved a Youden index of 0.908, a Dice coefficient of 0.833, a sensitivity of 0.955, and a specificity of 0.953. The overall performance of the network on detecting the total volume of graft detachment was high compared with the manually labeled gold standard (R² = 0.90).

Figure 3.

Validation cohort. Maps of the volume of graft detachment derived from 5120 manually labeled anterior segment OCT images (gold standard) and images labeled by the neural network (UNet++) are shown for all eyes in the validation cohort. The color gradient displays the height of detachment from no detachment (dark blue; pixel value, 0) to high detachment (yellow; pixel value, ≥70).

Application of the Neural Network in the Corneal Endothelial Transplantation Study

To apply the neural network, we compared the network’s results on the percent detachment with the clinical evaluation of cornea specialists based slit-lamp examination findings. A total of 107 eyes of 103 participants not included in the training and validation of the neural network underwent AS OCT imaging on the same day as the clinical evaluation (Table 1). Postoperative evaluation took place at a median of 16 days (IQR, 15–21 days) after DMEK.

The median percent graft detachment was 8.5% (IQR, 3.8%–23.6%) estimated by the neural network as compared with 0.0% (IQR, 0.0%–10.0%) estimated by cornea specialists, indicating that the neural network estimated the percent graft larger than the cornea specialist (mean difference, 8.2 percentage points; 95% confidence interval, 6.2–10.2; Fig 4A).

Figure 4.

Application of the neural network and comparison with slit-lamp–based evaluation of graft detachment. A, Scatterplot showing the percent detachment estimated by cornea specialists based on slit-lamp examination findings compared with the estimate of the neural network on anterior segment (AS) OCT images obtained the same day. Eyes with the need for repeat visits and for rebubbling because of incomplete attachment are highlighted (small open circles). The solid line represents x = y, and the dashed line represents the slope. B, Scatterplot showing results in eyes with more than a 10-percentage point difference in estimated percent detachment (large open circles) for which manual masked reannotation of AS OCT imaging was performed.

Forty-two AS OCTs with more than ±10 percentage points absolute difference between the neural network’s rating and the cornea specialists’ slit-lamp–based rating (39% of eyes) were annotated by 2 masked investigators. Manual review of the AS OCT images revealed that the cornea specialists’ rating based on slit-lamp examination findings alone underestimated the true percent detachment, particularly in those 17 eyes with indication for intervention because of incomplete attachment, in complicated configurations with multiple small detachments, and in cases of flat or planar detachment over multiple clock hours. In contrast, the network underestimated the true volume of detachment in one case with an artificially reduced graft surface because of the scrolled edges of the graft (Supplemental Fig 1). After reannotation, the mean difference was 8.1% (95% confidence interval, 5.0%–11.2%) among these scans. The neural network’s overall performance was high in detecting the percent of detachment when compared with partially reannotated clinical ratings (R² = 0.85).

Discussion

This study developed and validated a neural network to quantify graft detachment after endothelial keratoplasty using a separate training and validation cohort of manually labeled AS OCT images. The selected neural network precisely and selectively detected the area and volume of graft detachment in the training set. The network’s performance was good in a separate validation set. The resulting color-coded 3-dimensional volume maps allow identification and quantification of graft detachment easily, as demonstrated in the network’s application in the corneal endothelial transplantation study. The neural network detected graft detachment with much greater sensitivity in particular in cases with a significant amount of graft detachment that were missed or underestimated by the cornea specialists.

Quantifying and classifying graft detachment is required for management of patients after endothelial keratoplasty. The physician’s cognitive map of individual parallel slit-lamp or AS OCT images25, 26, 27, 28, 29 is prone to misinterpretation, particularly in eyes with significant extent of graft detachment, as demonstrated in the application. The neural network of this study performed well in detecting the area of detachment in individual images, resulting in highly reliable maps of the area of graft detachment in the training and validation set (Figs 2 and 3).

Three-dimensional representation of cross-sectional images adds an additional layer of complexity for humans that is natural for computers. The performance of the neural network to detect the volume of graft detachment was good and comparable with clinical grading by cornea specialists when applied to a prospective cohort study. Importantly, the neural network identified detachments missed or underestimated by a cornea specialist when the detachment was planar and flat and quantified the percent detachment when the extent detachment was big and the translucent Descemet’s membrane was obscured by the opaque and edematous corneal edema. Similar to machine learning approaches in the retina discipine,³⁰ the highly sensitive neural network may assist the clinician as an additional tool to conventional slit-lamp examination by drawing attention to areas of detachment in the en face heatmaps.

Implementation of the neural network in the manufacturer’s software or institutional analytical pipeline and the addition of eye tracking or alternative approaches for image registration¹² may allow for standardized comparison of follow-up examinations of the same patient and for assessment of algorithms and decision making.²⁷ For researchers interested in generating color-coded 3-dimensional maps of AS OCT images after DMEK, we provide the fully trained network on request to the corresponding author. Using such software, a technician acquiring the AS OCT image after DMEK can save the detachment map and 10 cross-sectional images (Supplemental Fig 1). Application of the neural network may help to understand detachment patterns, within-eye variability in detachment over time, and their risk factors ultimately to guide clinical decision making regarding how to improve long-term graft survival.³¹ Relevant questions for future studies include revisiting surgical and nonsurgical strategies for managing incomplete attachment. For example, indications for and timing of surgical interventions with descemetopexy using filtered air versus isoexpansile gas, or sutures,³² or nonsurgical approaches such as hyperosmolar agents,²⁷ which have been shown to be of limited use in Fuchs’ dystrophy eyes before DMEK.³³ Assessing intraoperative Descemet membrane detachment³⁴ or wound healing after Descemet stripping only also might be fields of application of 3-dimensional maps.

Limitations of the study include the development and validation at a single institution and the image resolution of the AS OCT device used in this study. Applying the network development process to scans from devices with higher resolution likely will improve the network’s performance further. Such networks also may be trained for image segmentation despite bandage lenses when DMEK is combined with corneal epithelial abrasion,³⁵ for various sizes of air or gas bubbles in the anterior chamber, or extreme detachment with freely floating grafts that were not present in this study.

In summary, we developed an instrument to quantify detachment of DMEK grafts automatically on AS OCT imaging and to visualize the area and volume of detachment using 3-dimensional maps. Given the high number of DMEK surgeries and an estimated need for intervention in every tenth case with rebubbling,^1,6,15 the maps provided by the neural network proved relevant in quantifying and localizing the area and volume of detachment in particular in those patients with significant detachment when the edema blocks the view on the thin graft. Beyond potential patient safety when assessing the site of injection for rebubbling without touching the graft, the neural network may allow identification of modifiable risk factors for incomplete graft attachment and precisely monitoring natural history of incomplete graft attachment over time.

Manuscript no. D-21-00067