Quantifying Meibomian Gland Morphology Using Artificial Intelligence

Abstract

Significance:

Quantifying Meibomian gland morphology from meibography images is used for diagnosis, treatment, and management of Meibomian gland dysfunction in clinics. A novel and automated method is described for quantifying Meibomian gland morphology from meibography images.

Purpose:

Meibomian gland morphological abnormality is a common clinical sign of Meibomian gland dysfunction, yet there exist no automated methods that provide standard quantifications of morphological features for individual glands. This study introduces an automated artificial intelligence approach to segmenting individual Meibomian gland regions in infrared meibography images and analyzing their morphological features.

Methods:

A total of 1443 meibography images were collected and annotated. The dataset was then divided into development and evaluation sets. The development set was used to train and tune deep learning models for segmenting glands and identifying ghost glands from images, while the evaluation set was used to evaluate the performance of the model. The gland segmentations were further used to analyze individual gland features, including gland local contrast, length, width and tortuosity.

Results:

1039 meibography images (including 486 upper and 553 lower eyelids) were used for training and tuning the deep learning model while the remaining 404 images (including 203 upper and 201 lower eyelids) were used for evaluations. The algorithm on average achieved 63% mean intersection over union in segmenting glands, and 84.4% sensitivity and 71.7% specificity in identifying ghost glands. Morphological features of each gland were also fed to a support vector machine for analyzing their associations to ghost glands. Analysis of model coefficients indicated that low gland local contrast was the primary indicator for ghost glands.

Conclusions:

The proposed approach can automatically segment individual Meibomian glands in infrared meibography images, identify ghost glands, and quantitatively analyze gland morphological features.

Meibomian glands, located in the upper and lower eyelids with orifices at the eyelid margins, are believed to play a critical role in ocular surface health by secreting lipids into the tears, which form a polar lipid film that serves to retard aqueous evaporation. Meibomian gland dysfunction can result in an unstable tear film and excessive evaporation of tears from the ocular surface and is the most frequent cause of dry eye symptoms.¹ The ability to visualize Meibomian glands and to monitor their temporal changes is important for evaluating the risk of Meibomian gland dysfunction, identifying active pathology, and following treatment outcomes. Meibography, which is the photo documentation of Meibomian glands in vivo using either trans-illumination or infrared imaging, is now commonly used in clinics for the diagnosis, treatment, and management of Meibomian gland dysfunction.^2,3

The relative size of areas of Meibomian gland atrophy, or gland loss area, is an important clinical measure for assessing Meibomian gland dysfunction severity. Currently, clinicians estimate the degree of Meibomian gland atrophy subjectively by comparing the area of glandular loss to the total eyelid area.^3,4 Though commonly used, the method only evaluates the overall severity of gland atrophy, not detailed individual Meibomian gland morphological features. Recent studies have shown that Meibomian gland morphological features, such as length, curvature or tortuosity, and local contrast (average gland region intensity normalized by its surrounding intensity) may also be indicative of Meibomian gland dysfunction severity and related to ocular surface disease.^5–8 Additionally, the presence of glands that were once patent but now contain little or no meibum, referred to as ghost glands, is also thought to be associated with Meibomian gland dysfunction. These glands have a faint or “ghostly” appearance in meibography images due to decreased local contrast of the central duct.^8,9

To accurately measure individual Meibomian gland morphological features from meibography images would be prohibitively time consuming in a patient care setting and would require extensive training and standardization across the field to be accurate and repeatable for research purposes. To date, there has been little progress made in understanding the roles played by various Meibomian gland morphological features in Meibomian gland dysfunction and various related pathologies (e.g., dry eye, tear film instability). The lack of quantification tools for individual gland morphology hinders such progress. Fortunately, recent advances in deep learning,^10–12 a form of artificial intelligence, demonstrate the ability of deep neural networks to learn to perform quantification tasks directly from a large dataset of images without explicitly specifying rules. Recent papers^13,14 have proposed deep-learning-based approaches to analyzing Meibomian gland morphology from meibography images and some have outperformed trained human observers. However, such methods are only capable of evaluating global Meibomian gland morphology rather than the fine-grained morphology of individual glands. Therefore, employing deep learning approaches to partitioning individual Meibomian gland regions from meibography images, a process known as image segmentation, quantitatively measuring gland morphological features, and identifying the presence of ghost glands is of timely interest to many clinicians.

This study aims to develop an automated approach based on deep learning that segments Meibomian glands in meibography images, identifies ghost glands, and quantifies individual Meibomian gland morphological features. The health and morphology of Meibomian glands are likely to be related to the quality of the lipid layer and the stability of the tear film. If such detailed morphological features could be obtained and quantified quickly and accurately, this would provide researchers with a powerful tool to quantitatively explore their etiological significance in ocular surface disease. In the longer term, the proposed tool could make a significant contribution to improving the diagnosis, treatment, and long-term management of tear film instability and evaporative dry eye.

METHODS

Subject Recruitment and Imaging

Adult human subjects, mean (standard deviation) age 27.8 (12.9) years, were recruited from the University of California, Berkeley campus and surrounding community for single-visit ocular surface evaluations during the period from 2012 to 2017. Eligible subjects were 18 years of age or older with no history of ocular injury or surgery, and not taking prescription ocular medications or systemic medications with effects on the ocular surface or tear film. Meibography images of the upper and lower eyelids of both eyes illuminated with 800 nm infrared light were captured with the OCULUS Keratograph 5M® (OCULUS, Arlington, WA).² During image capture for all subjects, the ambient light was off with the subject’s head positioned in a chin rest and forehead strap apparatus.

Informed consent was obtained from all subjects after a thorough description of the goals, procedures, risks and potential benefits of the study. This study conformed to the tenets of the Declaration of Helsinki, and was approved by the U.C. Berkeley Committee for Protection of Human Subjects.

Data Annotations

A total of 1550 images were collected and pre-screened to rule out images that did not capture the entire eyelid (107 images or 6.9%), thus 1443 images were used for the study. This study utilized a meibography image dataset from a previous paper,¹³ in which the percent area of Meibomian gland atrophy was estimated in upper eyelid meibography images. In the current paper, 754 lower eyelid images were added to 689 of these upper eyelid images from the same subjects. Both upper and lower eyelid images then had the total eyelid area, any regions of Meibomian gland atrophy, and each individual Meibomian gland contour traced, or “annotated”, by one of 10 trained observers, using the polygon tool in Fiji (ImageJ version 2.0.0-rc-59)¹⁵. For lower eyelid images, only the individual Meibomian gland was annotated, not the overall lid and atrophy regions, because the border of the lower tarsal plate could not be easily defined due to inter-subject variation in lower eyelid eversion. Far temporal and nasal gland contours were not annotated because these regions are generally out of focus in meibography images due to the anterior curvature of the eye. All visible glands within approximately the central 50% of the eyelid area of each meibography image were annotated. Every gland was also identified as a ghost or non-ghost gland. Note that ghost gland regions and Meibomian gland atrophy regions are mutually exclusive because ghost glands are faint but still visible, while atrophy regions have no discernable glands for annotation. Final annotations were verified by a single lead clinical investigator (TNY) before they were made available to the machine learning algorithm, in order to minimize variability in the ground truth data (i.e., the manual annotations from trained observers) that would arise from using multiple unsupervised observers. Figure 1 depicts several examples of Meibomian gland region annotations. Detailed statistics can be found in Table 1. Algorithms were considered to achieve 100% accuracy if they predicted results identical to the ground truth annotations.

Figure 1.

Meibography images with ground truth MG regions (outlined in color). The first two rows refer to upper eyelids while the last two refer to lower eyelids. Different colors correspond to different glands. Only MGs overlapping with the central region with width equaling to 50% eyelid width were considered since the far temporal and far nasal regions are out of focus.

Table 1.

Subject Demographics of the Meibography Image Dataset.

Upper eyelid	Development		Evaluation
	Train	Tune
Images (No.)	389	97	203
Patient Demographics
Unique individuals (No.)	260	94	109
Age (average ± SD)	27.8 ± 13.1	27.0 ± 11.5	27.9 ± 12.7
Female/ total patients	69.6%	66.0%	69.4%
Lower eyelid	Development		Evaluation
	Train	Tune
Images (No.)	445	108	201
Patient Demographics
Unique individuals (No.)	272	97	106
Age (average ± SD)	27.9 ± 13.1	28.2 ± 13.4	27.7 ± 12.6
Female/ total patients	68.4%	63.9%	67.6%

Data Allocations

Meibography images were allocated into two subsets according to collection time. The development set contained meibography images collected from the years 2015 to 2017, while the evaluation set contained those collected from the years 2012 to 2013. The development set was used for developing the deep learning algorithm, while the evaluation set was used for evaluating the performance of the algorithm. For algorithm development, the development set was further divided randomly into 2 subsets for training and validation. The images in the training set were used to train the deep learning model, while the validation set was used for tuning the model hyperparameters (e.g., network architectures, learning rate). The evaluation set, which did not have any overlapping images/subjects with the development dataset, was evaluated using the model that achieved the best performance from the validation set.

Algorithm Design and Training

Data Processing

Raw Meibomian gland annotations were preprocessed to a standardized format before feeding to the deep learning model. Detailed procedures are available in the supplementary materials.

Gland Segmentation

In computer vision, image segmentation is the process of partitioning an image into multiple regions.^16,17 Instance segmentation, one important task of image segmentation, aims to identify the contour of each object at the detailed pixel level.^18,19 Instance segmentation techniques are suitable for segmenting Meibomian glands since contours of individual Meibomian glands are desired outputs. Recently, deep learning approaches have outperformed all other methods in instance segmentation.^19–21 A deep learning algorithm was built upon an instance segmentation network with discriminative loss²¹ to predict individual Meibomian gland regions in meibography images not previously seen in the training phase. Figure 2 depicts the architecture of the proposed Meibomian gland segmentation network, which takes a meibography image as input and produces the contour for each gland as output. Details on the network design and training can be found in the supplementary materials.

Figure 2.

The MG segmentation network. The gland segmentation network takes a meibography image as an input and produces two intermediate outputs: the entire MG region segmentation (along with pixel-wise feature embedding, see the supplementary materials for details) and the number of glands. The predicted MG contour/embedding and gland number are further exploited by the network to predict instance MG region segmentation.

Ghost Gland Identification

The deep learning model was also designed to identify ghost glands (Figure 3) at the individual Meibomian gland level from a meibography image. Details on the network design and training can be found in the supplementary materials.

Figure 3.

Ghost gland classification network. The neural network has three inputs: an entire meibography image, a specific gland mask (the region within the predicted contour) and a corresponding cropped gland image. Inputs consist of a global view of the meibography image and a local view of individual gland, and thus two subnetworks are used to learn to identify ghost glands. The global network learns a representation from the entire meibography image and the individual gland mask, while the local network learns a representation from the individual gland image. Two representations are concatenated to predict ghost glands. In this example, the network outputs a value of 0, indicating it is a non-ghost gland.

Evaluation Metrics

After finishing the training of deep learning models and identifying the model with the best performance on the tuning set of images, its performance with respect to Meibomian gland instance segmentation and ghost gland identification was evaluated on the evaluation dataset.

Gland Segmentation

To quantify the similarity between the predicted Meibomian gland region (i.e., the entire Meibomian gland region within the predicted contour) and the ground truth, the mean intersection over union (mean IU) was used. This metric was also used by a previous paper evaluating predicted Meibomian gland atrophy segmentation.¹³ Mean IU, or the Jaccard index, quantifies the percent overlap between the target region and the prediction output. It measures the number of pixels in common between the target and prediction regions divided by the total number of pixels present across both regions. Denoting ground-truth gland region segmentation as GT, network predicted segmentation as P, the mean IU²² can be written as follows:

$$\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}\mathrm{I}\mathrm{U}=\frac{\mathrm{G}\mathrm{T}\bigcap \mathrm{P}}{\mathrm{G}\mathrm{T}\cup \mathrm{P}}$$

Intuitively, mean IU is analogous to the harmonic average of the precision and recall (F1-score) and provides a fair evaluation of the segmentation algorithm by considering both precision and recall. For segmentation tasks, a higher mean IU value indicates a better alignment of the algorithm prediction with the ground truth.

Gland Detection

In addition to evaluating the similarity between the predicted region and the ground truth, it is also necessary to evaluate how well the model detects objects for instance segmentation. Since there are multiple glands presented in an image, a high detection performance means the model does not miss or wrongly identify individual glands. The evaluation metric is available in the supplementary materials.

Ghost Gland Identification

Confusion matrices and receiver operating characteristic curves were used to evaluate ghost gland classification performance. A confusion matrix is a table that is used to describe the performance of a classification model on a set of test data for which the true class labels are known. Each row of the confusion matrix represents a true class label while each column represents a model-predicted label. Sensitivity and specificity (with 50% probability threshold) were calculated from the confusion matrix. A receiver operating characteristic curve²³ graphically depicts the diagnostic ability of a binary classifier system as its threshold is varied.

Morphology Analysis

A meibography image reveals the morphology of multiple glands. Individual gland features as well as global morphology are both important. We thus aimed to quantify both local and global morphological features.

Local Morphological Features

Based on individual Meibomian gland segmentations, morphological features such as average gland local contrast, gland length, width and tortuosity can be quantitatively measured. Details on measuring each feature from a meibography image are available in the supplementary materials. Visualizations of gland local contrast, length and tortuosity extraction are shown in Figure 4, Figure 5 and Figure 6, respectively. Specifically, local contrast of a gland region represents its pixel intensity normalized by the surrounding pixel intensity (gray scale: 0–255). Glands containing little or no meibum have a faint, ghost-like appearance⁹ and low contrast in the meibography images. Additionally, Yeh and Lin⁷ showed that gland local contrast in meibography captured using the same instrument was repeatable and invariant to ambient light conditions. Therefore, it is of interest to include local contrast of a gland as a morphological feature in the present study.

Figure 4.

Local contrast calculations of MGs. To “normalize” the individual gland intensity, the average gland local contrast is defined as the difference of average intensity of the gland region and surrounding area with negative values set to 0. Specifically, given an input image (left), instance glands are segmented. For a segmented gland region (middle), its surrounding area (outlined in orange) can be obtained by binary dilation operation (see the supplementary materials for details). The difference in intensity is thus calculated. The surrounding areas of all glands are also shown (right).

Figure 5.

Principal curves (marked in yellow) fit for gland length calculation. Gland length is defined as the number of pixels in each principal curve.

Figure 6.

Two sample MGs with tortuosity marked below. For each MG (outlined in red), the convex hull (outlined in blue) of the shape is extracted. Gland tortuosity is defined as the convexity defect of a gland, the ratio of the cavity area (green region) to the convex hull area. More tortuous glands have greater gland tortuosity values. The left gland is more tortuous.

Association of Local Morphological Features with Ghost Glands

One goal of analyzing Meibomian gland morphology is to understand which morphological features are most associated with ghost glands. A linear support vector machine for classifying Meibomian glands as ghost glands or non-ghost glands was constructed with the four aforementioned morphological features as explanatory variables (inputs). The model could be considered to be using the weighted sum of input morphological features as the probability of classifying a gland as a ghost gland. Weights are learned by training the model. Feature coefficients, or model weights, thus reflect the relative importance of different features for the ghost gland decisions of the model.

Global Morphological Features

Several global morphological features can also be measured quantitively with this deep learning approach. Global features that can be extracted include number of glands per image, gland density (the ratio of the gland area to the eyelid area), and percent atrophy (the ratio of atrophy area to the eyelid area).The number of glands was calculated for both upper and lower eyelids while gland density and percent atrophy were calculated over upper eyelids only due to the inability to annotate the full eyelid and atrophy regions in lower eyelid images as explained above.

RESULTS

The algorithm development and evaluation were performed for the whole dataset with both upper and lower eyelid images for local morphological features, and with upper eyelid images for global morphological features. Normalized histograms of global morphological features (total number of glands, gland density, and percent atrophy) are shown in Figures 7 (a)–(c) for upper eyelid images, and of local morphological features (local contrast, length, width, tortuosity, and identification of ghost glands) in Figures 7 (d)–(h).

Figure 7.

Histograms (in percentage) of various morphological features of our meibography image dataset.

Dataset and Training Details

Subject demographics stratified on the development and evaluation datasets can be found in Table 1. Details of data preprocessing and network training are available in the supplementary materials.

Meibomian Gland Segmentation Performance

Table 2 reports the performance of the Meibomian gland instance segmentation algorithm. For gland segmentation, mean IUs were 58.4% and 68.0%. Gland detection performance is available in the supplementary materials. Figure 8 and Figure 9 provide visualizations of the ground truth and predicted Meibomian gland segmentations for upper and lower eyelids, respectively, with the last row in each figure presenting illustrative examples of failure cases.

Table 2.

Gland Segmentation Performance.

	Upper eyelid	Lower eyelid
Precision	62.7%	76.9%
Recall	53.9%	74.1%
Mean IU	58.4%	68.0%

Figure 8.

Upper eyelid MG segmentation ground truths versus predictions. Different colors refer to different MGs. The predictions were visually similar to the ground truths. The last row depicts some failure cases.

Figure 9.

Lower eyelid MG segmentation ground truths versus predictions. Different colors refer to different MGs. The predictions were visually similar to the ground truths. The last row depicts some failure cases.

Ghost Gland Identification Performance

Table 3 reports confusion matrices of ghost gland classification for the upper and lower eyelids. The deep learning algorithm achieved 84.7% sensitivity and 72.5% specificity for ghost gland classification in upper eyelid images, and 84.1% sensitivity and 70.8% specificity for ghost gland classification in lower eyelid images. Receiver operating characteristic curves for ghost gland classifications for the upper and lower eyelids are shown in Figure 10.

Table 3.

Performance of Ghost Gland Classification.

Upper eyelid	Predicted: Ghost	Predicted: Non-ghost
Actual: Ghost	188 (84.7%)	34 (15.3%)
Actual: Non-ghost	789 (27.5%)	2085 (72.5%)
Lower eyelid	Predicted: Ghost	Predicted: Non-ghost
Actual: Ghost	74 (84.1%)	14 (15.9%)
Actual: Non-ghost	699 (29.2%)	1693 (70.8%)

Figure 10.

Receiver operating characteristic curves for ghost gland classification.

Morphological Features and Associations with Ghost Glands

To give readers a sense of extracted features, individual Meibomian gland morphological features from an example meibography image are shown in Figure 11. Detailed morphological features including local contrast, gland length, width, tortuosity and ghost gland identification of the 13 visible glands in that image are shown. Figure 7 depicts the normalized histograms of local and global morphological features for the whole meibography image dataset.

Figure 11.

Individual MG morphological features. Based on MG regions (upper right), individual MG morphological features (lower) can be measured. In the last row, 1 indicates a ghost gland while 0 indicates a non-ghost gland.

To determine the important morphological gland features associated with ghost glands, the feature coefficient of a support vector machine model constructed on Meibomian gland features was analyzed. The feature coefficient of the support vector machine model reflects the importance of different input features. A support vector machine model achieving 57% sensitivity and 65% specificity was used. The corresponding feature coefficient of gland local contrast was −73%, gland length was 69%, gland width was 14% and gland tortuosity was 30%. The negative sign indicates that glands with lower local contrast are more likely to be ghost glands. Gland local contrast and length had comparably large coefficient values.

DISCUSSION

This work presents a deep learning-based approach to automatically analyzing Meibomian gland morphology by segmenting individual glands from meibography images. This approach introduces the following three innovations: 1) Instance gland regions are automatically segmented from the meibography image; 2) Ghost glands are automatically identified; 3) Morphological features at the individual Meibomian gland level are analyzed. The proposed approach to quantifying gland morphological features makes it possible to further study the quantitative relationships between gland morphology and clinical signs, to improve the understanding of Meibomian gland morphology and pathology, and could ultimately provide quick and accurate diagnostic information in the clinical setting.

Previous works have explored the plausibility of automatically segmenting glands from meibography images with the goal of Meibomian gland morphological analysis, and have achieved satisfactory results for certain images.^14,24 However, such methods only segment entire Meibomian gland regions instead of individual Meibomian gland regions, which is a serious limitation for further morphological analysis. While additional post-processing could break the entire region into several smaller parts, such methods are unlikely to achieve adequate results as Meibomian glands are usually dense, close to or even overlapping with each other. The proposed method, in contrast, is able to segment instance glands even when they are close or slightly overlapping, which makes it more appropriate for further morphological analysis. Empirically, a mean IU with value of 50% or higher suggests a good match between the prediction and the ground truth in computer vision. Thus, both the numerical performance evaluation metric (58% and 68% for upper and lower eyelid, respectively) and the visual examples reported in the previous section validated the acceptable performance of the proposed algorithm.

The proposed approach is able to identify ghost glands, which have been shown previously to be indicative of Meibomian gland dysfunction.^8,9 Based on gland segmentation and quantified morphological features, further analysis of the association between individual gland morphological features and ghost glands suggested that low Meibomian gland local contrast was most associated with the presence of ghost glands. This is consistent with the definition of ghost glands, which have low contrast to their surroundings in images since they contain little or no meibum. The coefficient of gland length is the second largest, meaning that longer glands are more likely to be ghost glands. Future studies of the links between gland health, pathology and morphological features such as gland local contrast and length are warranted to verify and further understand this finding.

The average processing time (including segmentation, classification and morphological analysis) per image was approximately 0.32 seconds. Experiments were performed on a single graphics processing unit (GeForce RTX 2080 Ti®. NVIDIA, Santa Clara, CA.). This means that more than 900 unprocessed or raw meibography images can be evaluated for atrophy severity in 5 minutes without additional human intervention. The processing speed renders it possible to automatically analyze a large number of meibography images within a short time and greatly improve efficiency in the clinical and research settings. Such evaluations currently would be prohibitively time consuming and without standardization for large-scale clinical trials or on-site ocular health screenings with large numbers of patients.

As with any emerging methodology, the proposed approach has certain limitations. Only glands in the central eyelid region were analyzed. Because the entire eyelid is a curved surface, imaging the central region of the tarsal plate with an optimal focus causes defocus of the peripheral glands. Additionally, incomplete eyelid eversion may also occlude far peripheral glands, making accurate identification of Meibomian gland morphological features challenging. In terms of global morphological features, gland density and percent atrophy were not analyzed for the lower eyelids in this study due to the inability to accurately annotate lower eyelid boundaries for many subjects. The tarsal plate of the lower eyelid is about 5 mm in length,²⁵ which is approximately half the length of the upper eyelid. Variability among patients in the eyelid area exposed by eversion of the lower lid makes it difficult to consistently determine its boundaries. It is also important to note that deep learning model performance is measured against, and ultimately depends upon, human expert annotations of the original meibography images. It is thus not possible to eliminate completely the subjective nature of the expertise needed to train the algorithm.²⁶ Finally, it should be noted that infrared imaging of the eyelids may or may not provide the best images of all Meibomian glands, as some researchers have found significant differences in detecting gland dropout between infrared imaging and other methods such as optical coherence tomography. ²⁷

The scope of this study is restricted to reporting a new approach to evaluating morphological features in meibography without providing clinical understanding of what the variability of these features may indicate. Few other studies to date have attempted to investigate the associations among Meibomian gland morphological features, especially local features, and basic physiological or ocular health outcomes. Some researchers have used the 0–3 (per eyelid) discrete ordinal meiboscore proposed by Arita et al., ²⁸ or variations thereof. ²⁹ These scores are generally based on global features such as percent area of the eyelid affected, and do not have fine resolution or consider local morphological features of individual Meibomian glands. For gland tortuosity, some researchers have proposed a categorical classification with a threshold of 45 degrees,³⁰ and some researchers have proposed different tortuosity quantification methods.³¹ With the proposed method, a great deal more detailed and accurate information about both the overall status and the fine-grained morphology of the Meibomian gland array in the eyelid can be made available to researchers. Future work is warranted using the proposed methodology to study the relationships among Meibomian gland morphological features, gland health and pathology, the quality and thickness of the tear lipid layer, tear film instability and dry eye.

In conclusion, a deep learning model to automatically segment individual Meibomian glands, analyze both global and local Meibomian gland morphology and predict the presence of ghost glands in meibography images has been developed. Given its accuracy and efficiency, the quantitative outputs (morphological features at the individual gland level) can potentially be helpful in furthering our understanding of the interplay between Meibomian gland features and clinical signs and symptoms by analyzing large-scale image datasets⁸. Future work is warranted to improve Meibomian gland segmentation performance for highly overlapping glands and peripheral glands in the nasal and temporal regions of the tarsal plate, and to expand our understanding of the relationships between Meibomian gland morphology and various ocular pathologies.

APPENDIX

We report additional details of the proposed method. Specifically, we present detailed data processing procedures, network design, loss functions and training details of the proposed model, Meibomian gland morphological feature measuring methods, and Meibomian gland detection evaluation details.