Machine learning in optical coherence tomography angiography

Abstract

Optical coherence tomography angiography (OCTA) offers a noninvasive label-free solution for imaging retinal vasculatures at the capillary level resolution. In principle, improved resolution implies a better chance to reveal subtle microvascular distortions associated with eye diseases that are asymptomatic in early stages. However, massive screening requires experienced clinicians to manually examine retinal images, which may result in human error and hinder objective screening. Recently, quantitative OCTA features have been developed to standardize and document retinal vascular changes. The feasibility of using quantitative OCTA features for machine learning classification of different retinopathies has been demonstrated. Deep learning-based applications have also been explored for automatic OCTA image analysis and disease classification. In this article, we summarize recent developments of quantitative OCTA features, machine learning image analysis, and classification.

Impact statement

High-resolution optical coherence tomography angiography (OCTA) can capture subtle microvascular changes associated with early eye disease. This article summarizes the technical rationale of quantitative OCTA features, machine learning image analysis, and classification. Further developments of the machine learning OCTA image analysis and classification will foster artificial intelligence OCTA screening of eye diseases.

Introduction

Research into artificial intelligence (AI) technology has accumulated increased interest in biomedical research in recent years. ¹ One branch of AI known as machine learning has been at the forefront of research. Deep learning, a subset of machine learning, has also grown in popularity. The rise of deep learning is due to two main reasons. First, the recent advancement in computational hardware, i.e., the graphics processing unit (GPU), has enabled for the use of deep learning algorithms. Second, the overabundance of medical data, termed “big data”, can enable machine learning algorithms to learn and perform with high accuracy. There are several advantages of using AI in the medical field. Namely, the automation of disease classification or detection of regions of interests, the reduction in human errors, and the detection of early disease stages. Over recent years, machine learning applications have been explored for various diseases in a wide selection of medical imaging modalities, such as for classification of brain tumors using magnetic resonance imaging (MRI),^2–4 classification of breast cancer in ultrasound,^5–7 and detection of pulmonary diseases in chest X-rays ^8–10 and in computed tomography (CT) .^11–13

In ophthalmic clinics, massive screening for common ocular conditions is heavily dependent upon experienced physicians to examine and evaluate retinal images. Patients with early onset of retinopathies such as diabetic retinopathy (DR) or age-related macular degeneration (AMD) are initially asymptomatic yet require monitoring to ensure prompt medical interventions to prevent vision losses. However, this process is both time consuming and expensive, therefore it is not feasible to screen 65 million people in the USA over the age of 50 years ¹ to identify individuals with signs of early retinopathy (AMD, DR, or other disease). An AI-based diagnostic tool for retinal imaging with the capability for multiple-disease differentiation would have tremendous potential to advance massive screening of eye diseases. ¹⁴ For ophthalmology research, application of machine learning has led to excellent diagnostic accuracy for various eye diseases such as DR,^15,16 AMD,^17,18 sickle cell retinopathy (SCR),^19,20 and central retinal vein occlusion (CRVO) ²¹ in fundus photography, optical coherence tomography (OCT), and recently, OCT-angiography (OCTA).

To date, most of the reported studies of AI diagnostic systems in literature are based on color fundus photography.^22–25 Fundus photography is one of the most common clinical imaging modalities and is established for evaluating retinal abnormalities. However, fundus images provide limited spatial resolution to provide detailed retinal vascular information, especially the microvascular distortions near the fovea and it does not provide layer information of the retina (Figure 1(a)). OCT can provide sectioning capability to examine abnormalities in different retinal layers. As a new OCT modality, OCTA can further enhance the capability to visualize trilaminar vascular network with capillary level resolution. ²² OCTA provides improved capability for detecting subtle vascular distortions associated with progression of retinal pathology, such as vessel dropout, foveal abnormalities, increased vessel tortuosity, etc. (Figure 1(c) to (e)). By providing unparalleled morphological detail of retinal vasculatures, OCTA has been rapidly adopted for clinical management of DR, AMD, glaucoma, etc. Quantitative OCTA analysis has been explored to standardize objective interpretation of clinical outcomes. Multiple OCTA features have been developed for quantitative analysis of retinal vascular distortions due to eye conditions. Machine learning approaches can utilize OCTA for the classification and staging of retinal diseases.

Figure 1.

Representative retinal images in ophthalmology. (a) Fundus photograph, (b) corresponding enface OCT, and (c1, c2) OCTA of the same eye, control eye. (d1, d2) OCTA images of mild non proliferative diabetic retinopathy (NPDR) and (e1, e2) moderate NPDR eyes. (c1, d1 and e1) Enface OCTA of the superficial vascular plexus. (c2, d2 and e2) Enface OCTA of the deep vascular plexus. The scale bar in yellow corresponds to 1 mm and applies to all images. Images are from Xincheng Yao’s lab image gallery. (A color version of this figure is available in the online journal.)

In this article, we provide a brief review of machine learning in OCTA for classification of retinal diseases. The following section describes the basic principles of machine learning, where we discuss the pipeline of machine learning implementation and differences between traditional machine learning approaches and deep learning. A further section summarizes the quantitative OCTA features that have been developed and their technical rationale. The subsequent section provides recent development in traditional machine learning for retinopathy classification and deep learning for different tasks, such as automated classification and segmentation of the retinal vasculature. The penultimate section discusses the current limitations and prospective developments for machine learning in OCTA.

Basic principle of machine learning

The field of AI encompasses a broad range of applications. The machine learning is one of the branches of AI. The basic principle of machine learning is to develop and implement algorithms that can learn from the data without being explicitly programed. ²⁶ For machine learning practices, generally the data would be divided into two main datasets, the training set and the testing set. The training set is used to optimize the algorithm, also referred to as model, and the testing set is used to evaluate the performance of the model. There are different tasks or applications in which the model can perform, such as classification, the goal of categorizing the data, and regression, the goal of predicting continuous values.

To optimize the model, there are two main approaches, supervised and unsupervised learning. Using the example of a classification model, in supervised learning, each of the samples in the dataset has a corresponding label or ground truth, whereas in the unsupervised learning approach, the data does not have any associated labels. The type of learning depends on the dataset, if the dataset has labels then the user can implement either type. And if there are no labels, then only unsupervised learning can be implemented. Since unsupervised learning does not involve any manual labor to generate the labels, it may be useful if the user has a substantially large dataset. However, the caveat is since there are no labels, the performance may be unreliable for the desired application.

There are different machine learning algorithms that have been developed over the years. Common algorithms include, support vector machines (SVMs), k-nearest neighbors (kNN), and linear regression.^27,28 For retinal image classification, to apply these algorithms, there are generally two segments. Digital image processing and machine learning. A pipeline of machine learning is illustrated in Figure 2. In the digital image processing segment, the user must first perform digital image processing and extract features or measurements from the image. Once the features have been extracted and compiled, it can then be used in the second segment to train the machine learning model. Once the model has been trained, its performance must be evaluated.

Figure 2.

Overview of general machine learning steps. First, digital image processing and feature extraction are performed. Second, the machine learning model is trained with the dataset to perform the classification task for either different diseases or staging. (A color version of this figure is available in the online journal.)

To evaluate machine learning models, common metrics such as accuracy, sensitivity, and specificity are used. Sensitivity evaluates the model’s ability to identify cases with the disease, and specificity evaluates the model’s ability to identify cases without the disease. To evaluate the model’s overall performance, accuracy would be determined. The following equations are used to calculate sensitivity, specificity, and accuracy

$$\begin{array}{c}S\text{ensitivity}=\frac{TP}{TP+FN}\end{array}$$

$$\begin{array}{c}S\text{pecificity}=\frac{TN}{TN+FP}\end{array}$$

$$\begin{array}{c}A\text{ccuracy}=\frac{TP+TN}{TP+FN+TN+FP}\end{array}$$

where TP is true positive, TN is true negative, FP is false positive, and FN is false negative.

Recently, deep learning has sparked tremendous interest in all research fields. Deep learning is a subset of machine learning, and generally refers to the use of a specific algorithm, convolutional neural networks (CNNs), which are inspired by the human visual pathway. The CNN gets its name from the use of the convolutional layers, which consist of sets of trainable filters that are adept to process spatial patterns. ²⁹ In contrast to traditional machine learning algorithms, the input into the CNN is the image itself. The CNN is trained on the image dataset and learns how to perform both the feature extraction and classification. Therefore, the user does not have to perform manual feature development and extraction. Deep learning has risen in popularity due it is ability to learn directly from image data, thereby reducing the burden of manual feature development and complexity for clinical use. However, one disadvantage is its “black box” nature. Since the CNN performs the feature extraction automatically, the user does not know what types of features are being used for the classification. In contrast, for traditional machine learning approaches, the input into the algorithm are the quantitative features, thus enables high interpretability.

Quantitative feature in OCTA

Different retinal disease affects the blood flow inside the vessels resulting in structural changes in the vasculature. OCTA is uniquely suited to capture these vascular changes. Utilizing digital image processing, different types of vascular maps can be derived and employed in the development of quantitative features (Figure 3). In this section, we briefly describe the technical rationale of quantitative OCTA features that have been developed in recent years. Commonly used quantitative OCTA features include, blood vessel density (BVD), blood vessel caliber (BVC), blood vessel tortuosity (BVT), vessel perimeter index (VPI), vessel complexity index (VCI), foveal avascular zone (FAZ) area (FAZA), and FAZ contour irregularity (FAZCI).

Figure 3.

Representative OCTA image processing, (a) enface OCTA image, (b) vessel map, (c) vessel perimeter map, (d) skeleton map with the FAZ area segmentation in blue, FAZ contour demarcated by the green boundary, and the yellow line is an example measurement of vessel tortuosity from point X to Y. (e, f) Contour maps created with normalized values of local fractal dimension in superficial and deep layers, respectively. (e) Retinal quadrants (superior, inferior, temporal, and nasal regions). (f) Circular zones of diameter, 2, 4, and 5.5 mm. The scale bar in yellow corresponds to 1 mm and applies to all images. Images are from Xincheng Yao’s lab image gallery. (A color version of this figure is available in the online journal.)

The BVC index was developed to estimate the mean vessel width and can therefore be used as a biomarker for vessel dilation or shrinkage, whereas the VPI encapsulates changes to overall vessel area and length. Thus, VPI may reflect vessel dropout or early ischemia. ³⁰ Eye diseases can also occlude blood vessels, such as in DR or SCR, resulting in more tortuous blood vessels compared to the vasculature of healthy subjects. This observation can be measured by the BVT index. In addition to features that measures changes in the vasculature, the fovea is an important area of investigation as well. ³¹ The FAZ is the region devoid of retinal capillaries that surround the fovea. Quantitative parameters of the FAZ include FAZA and FAZCI. Alterations of the FAZ have been demonstrated to be good indicators of retinal pathology such as in DR and retinal vascular occlusion.^32,33

For different retinal complications, the changes may not be uniform. Therefore, complexity indices may be robust for detecting local vessel dropout (low complexity) or areas of neovascularization (high complexity). The VCI is derived from the digital image processing research, ³⁴ and can be used to capture localized changes. ¹⁵ Another commonly used measurement is fractal dimension (FD), which measures the texture complexity of non-Euclidean structures that show similarity at different scales. ³⁵ FD can be used to derive highly detailed vessel maps, and therefore is commonly used to determine BVD. Recent studies have also measured BVD for different retinal regions (Figure 3(e)), i.e. superior, inferior, temporal, and nasal quadrants, and eccentricities (Figure 3(f)), i.e. central fovea, parafovea, and perifoveal retina.

Machine learning in OCTA

Quantitative OCTA features can reduce human error in subjective screening of retinal images. Furthermore, certain features may be sensitive to reveal subtle microvascular changes associated with early eye diseases. Machine learning algorithms can utilize these features to enable robust early disease detection. In this section, we review traditional machine learning approaches for classification and deep learning applications in OCTA.

Traditional machine learning for classification of retinopathy

Traditional machine learning methods involves the use of manual feature extraction, e.g. quantitative OCTA features. Several different studies have explored the use of machine learning using quantitative OCTA features for computer-aided disease detection and AI classification of different retinopathies. One study employed six OCTA features, i.e. BVD, BVC, BVT, VPI, FAZA, and FAZCI to train different machine learning classifiers, i.e. SVM, kNN, and discriminant analysis, for the staging of SCR patients. ¹⁹ The study reports that the trained SVM classifier was able to identify control versus SCR eyes and mild versus severe SCR with 100% and 97% accuracy, respectively. A similar approach was also demonstrated for nonproliferative diabetic retinopathy (NPDR) classification, employing the same six quantitative features. ¹⁵ The study evaluated the individual performance of each quantitative feature. The best performance was achieved on the SVM classifier trained on the combined feature set, with a reported 94.41% accuracy for control versus NPDR eyes, and 92.96% for control versus mild NPDR eyes. The individual features that had the highest sensitivity were BVD, FAZA, and FAZCI. The performance of the individual features can be explained by observations of the quantitative OCTA maps (Figure 4). For example, BVD had the best performance out of all the other features due to large areas of vessel dropout observed in severe NPDR eyes (Figure 4(d1) to (d4)). Likewise, the FAZ parameters also had excellent performance to differentiate control from NPDR eyes. This can be attributed to the observation that the FAZ in control subjects are much smaller and have high circularity, in contrast NPDR cohorts (mild to severe) have much larger FAZ and high acircularity (Figure 4(a2) to (d2)).

Figure 4.

Representative OCTA images for illustrating the feature extraction. (a1–a4) Control subject, (b1–b4) Mild NPDR subject, (c1-c4) Moderate NPDR subject, (d1–d4) Severe NPDR subject. (a1, b1, c1, d1) Segmented blood vessel map including large blood vessels and small capillaries. (a2, b2, c2, d2) Skeletonized blood vessel map (red) with segmented FAZ (marked blue region) and FAZ contour (green boundary marked around FAZ). (a3, b3, c3, d3) Vessel perimeter map. (a4, b4, c4, d4) Contour maps created with normalized values of local fractal dimension. Scale bar shown in A1 corresponds to 1 mm and applies to all the images. Modified from Alam et al. ³⁰ (A color version of this figure is available in the online journal.)

Localized quantitative OCTA analysis was recently demonstrated for the staging of controls, NPDR, and proliferative DR patients (PDR) . ¹⁵ This study performed a comparative analysis between whole-image and shifting-window measurements. For this study, eight features, namely, BVT, and VCI for the superficial vascular plexus (SVP) only, FD, BVD, and FAZA for both the superficial and deep capillary plexus (DCP) were utilized. Localized complexity maps were generated and revealed areas of increased vascular complexity in the PDR eye. In this study, eight features were employed for whole-image and shifting-window, for a total of 16 features. The study employed a multivariate logistic regression model and implemented a backward elimination to iteratively select the best features for classification. Out of the 16 features, this study reports that three shifting-window features, VCI, FD and BVT performed the best with reported accuracies of 91.26%, 85.10%, and 87.62%, respectively. The combination of these three features achieved a classification performance of 94.75% accuracy.

In addition to morphological changes in OCTA, several studies have employed intensity features in addition to OCTA features. A recent study incorporated different features such as the FAZ circularity and BVD, and also the average intensity of capillaries and large vessels for machine learning-based classification of healthy controls, diabetic with and without retinopathy. ³⁶ This study performed a comparison of different machine learning algorithms, i.e. decision trees, random forest, and logistic regression, etc. For the binary classification of diabetic eyes from healthy eyes, the random forest classifier achieved an area under the curve (AUC) of 0.80, whereas for the classification of diabetic retinopathy versus non-diabetic retinopathy eyes, the logistic regression model achieved an AUC of 0.91. Overall, the study suggests that machine learning can be used for detection of diabetics and early staging of NPDR. The observations in this study indicate that vessel density and avascular areas are the most sensitive parameters in DR classification. However, quantifiable changes in the intensity of the capillary vessels, i.e., reduction in blood flow, were also observed. Suggesting that OCTA intensity-based features may be leveraged in future studies for disease detection.

Another study explored the feasibility of a computer aided diagnostic (CAD) system using a multimodal approach involving both OCT and OCTA feature for the classification of NPDR. ³⁷ The study extracted three OCT features, namely, curvature, reflectivity, and thickness, and four OCTA features, FAZA, BVD, BVC, and bifurcation points. Additionally, six clinical features were also utilized, i.e. age, sex, glycated hemoglobin (HbA1c), hypertension, dyslipidemia prevalence, and edema prevalence. A multi-step random forest classifier was employed for machine learning classification (Figure 5). The first random forest performs binary classification of DR eyes and healthy control eyes. If the eye is classified as DR, a second random forest further differentiates the eye as either mild or moderate DR. The study reports a 94.7% accuracy using OCTA features only, and when supplemented with additional information such as OCT and the clinical data, the classifier achieves a 98.7% accuracy.

Figure 5.

A pipeline of a computer aided diagnostic system that incorporates OCT and OCTA feature extraction, and the inclusion of clinical biomarkers for the classification of healthy normal eyes, mild, and moderate DR eyes. Reprinted from Lee et al. ³⁸ (A color version of this figure is available in the online journal.)

Multi-classification of retinopathies was also demonstrated in a recent study. The study employed six quantitative OCTA features, i.e. BVT, BVC, VPI, BVD, FAZA, and FAZCI for the classification and individual staging retinopathy in a cohort of healthy subjects, NPDR and SCR patients. ²⁰ All features were extracted from both the SVP and DCP. Measurements of BVD were further differentiated for retinal quadrants (superior, inferior, temporal, and nasal) and different eccentricities (central fovea, parafovea, and perifovea). Therefore, the study implemented backward elimination to select the best optical feature sets for the multi-task classification. This study reports that four sensitive features, namely, BVT, FAZA (S), FAZCI (D), and BVD (S-6mm) for the differentiation of healthy, NPDR, and SCR (Figure 6). The analysis of the scatter plot reveals that the three cohorts, healthy controls, DR, and SCR subjects have excellent separability. Quantitative OCTA of SCR revealed that the FAZ parameters, i.e., area and contour irregularity, are most sensitive for SCR classification. ³⁹ In addition to FAZ changes, previous observations have indicated that the BVD is the most sensitive feature for DR classification. ³⁰ Therefore, quantitative OCTA features can enable multiclass classification because of how different retinopathies affect the retinal vasculature differently. The overall performance reported in the study was 95.01% sensitivity for DR versus SCR, 92.18% sensitivity for NPDR staging, and 93.19% sensitivity for SCR staging.

Figure 6.

Correlation analysis of the four most sensitive features for the classification of healthy controls, NPDR, and SCR eyes. The scatter plot illustrates the distribution of control, NPDR, and SCR OCTA features for different feature combinations. Reprinted from Alam et al. ²⁰ (A color version of this figure is available in the online journal.)

Deep learning for classification of retinopathy

Deep learning-based technologies have captivated attention from ophthalmology due to the multitude of applications in OCTA, including OCTA reconstruction,^38,40,41 OCTA denoising,^42–44 and segmentation of different regions of interest, e.g. vessels, FAZ, etc. .^45–48 Recent development and the implementation of quantitative OCTA features for machine learning classification indicate that OCTA images contain the necessary information to identify different retinopathies and perform disease staging. In principle, the CNN can automatically perform the feature extraction and classification, thereby reducing the burden for manual feature engineering. Additionally, there may be features that have yet to be examined, by using the image as a direct input into the CNN, the CNN may be able to utilize all types of features for early disease detection. To train a CNN for a specific classification task would require millions of images to optimize the millions of network parameters. This poses a challenge for exploration of deep learning for OCTA classification, as OCTA is a relatively new imaging modality and therefore has limited datasets.

A recent study employed the use of transfer learning to overcome this limitation. Transfer learning is a training method that adopts some weights of a pretrained CNN and appropriately retrains certain layers of the CNN to optimize the weights for a new dataset. This pre-training method is feasible due to how CNNs extract features in a bottom-up hierarchical structure, analogous to the human visual pathway system. ⁴⁹ In this study, the visual geometric group (VGG)-16 network ⁵⁰ was pretrained on the ImageNet dataset, ⁵¹ a dataset comprised of millions of everyday images, e.g., cars, animals, people, etc., and employed transfer learning to retrain the network for classification of control, diabetic eyes with no retinopathy and NPDR eyes and reports an overall accuracy of 87.27%. A similar study implemented ensemble learning using CNNs, i.e., VGG16, in addition to transfer learning, for the task of classification in DR. ⁵² The study trained individual CNNs using a multi-modal approach using enface images of the SVP and DCP from both OCT and OCTA. A majority voting ensemble method combined the classification of each individual network for the classification of eyes as referable and non-referable DR. (Figure 7(a)). The study also employed the use of gradient-weighted class activation maps (grad-CAM), an algorithm that generates a heatmap of areas in the input image that contribute the most to the final prediction score (Figure 7(b)). The study reports an overall accuracy of 92% for the ensemble-based classification method.

Figure 7.

(a) Example of the majority voting ensemble method for combining classification results from multiple component networks. The component networks were previously trained on superficial and deep plexus enface images of OCT and OCTA volumes separately. (b) Grad-CAMs for the (1) superficial and (2) deep enface OCT image and (3) superficial and (4) deep enface OCTA image of a severe DR patient. Hard exudates and regions of fluid are highlighted in the OCT image. Microaneurysms and regions of capillary dropout are highlighted in the OCTA. Modified from Fonseca and Dantas. ⁵³ (A color version of this figure is available in the online journal.)

Classification is one of the many tasks that can be performed using deep learning algorithms, CNNs can also be applied for segmentation tasks, also referred to as pixel-wise classification, in OCTA. Recently, deep learning has been explored in OCTA for artery-vein (AV) classification. Clinical observations have established that different diseases can affect the arteries and veins differently, i.e. arterial narrowing^54,55 and venous beading.^53,56 Therefore, AV analysis can provide valuable information for disease detection and classification. Several studies have proposed methods for differential AV classification in OCTA.^57–59 However, these studies involve multiple different algorithms which may be too complex for clinical deployment.

One recent study proposed “AV-Net” a fully CNN for the automated AV classification (Figure 8(a)). ⁶¹ This study employs a multi-modal training process that involves both OCT and OCTA. The study suggests that the enface OCT can provide the necessary intensity information for AV classification, and the enface OCTA contains the blood flow information. AV-Net is comprised of two parts, an encoder and a decoder. The encoder is equivalent to the classification CNNs, such as VGG16, in that it takes in an image input and performs feature extraction. The decoder identifies and maps the image features to produce an output image.

Figure 8.

Network architecture for AV-Net, (a) overview of the blocks in AV-Net architecture. AV-Net takes in an input of size 320 × 320 with two channels, corresponding to OCTA and OCT and outputs an RGB map of the same size. Representative data used for training and example prediction. (b1) Input enface OCTA and (b2) OCT images, (b3) the ground truth, and (b4) the AV-Net predicted AV-maps. The scale bar in yellow corresponds to 1 mm and applies to all images. Modified from Le et al. ⁶⁰ (A color version of this figure is available in the online journal.)

Since OCTA data are limited, this study leveraged regularization techniques, such as data augmentation, i.e., random flips, rotation, and zoom, to increase the dataset size. Furthermore, transfer learning was implemented. The performance of AV-Net achieves an overall accuracy of 86.75%, with 86.71% and 86.80% for artery and veins, respectively. Qualitative performance of AV-Net is illustrated in Figure 8(b). Overall, AV-Net performs well and achieves robust vessel segmentation. However, since AV-Net performs pixel-wise classification, there are pixels that are misclassified in the vessels (Figure 8(b4)). Furthermore, the vessels do appear dilated in the predicted AV-maps.

Different eye disease, such as DR, may involve ischemia and drop out zones in the retinal and choroidal vasculature. ⁶² Therefore, quantification of avascular area may potentially be a useful biomarker for disease detection and staging. A recent study developed a CNN architecture titled “MEDnet” for the detection of non-perfusion areas in the SVP. ⁶³ The MEDnet architecture follows a similar encoder-decoder network and employs a multimodal input. The input is comprised of an enface angiogram that provides the flow information, and the corresponding reflectance image to compensate for the loss of flow signal due to occlusion of the back-scattered signal (Figure 9(a)). The encoder network was modified to utilize atrous (dilated) convolution operations to extract features at different scales (Figure 9(b)). The study trained and evaluated MEDnet using a dataset comprised of healthy controls, patients without and with NPDR (mild, moderate, and severe), with varying image qualities, i.e. different signal strength index (SSI) values. The study reports an overall F1-score of 80%, indicating that MEDnet has overall good performance. Some sources of misclassification may be attributed to the loss of OCTA signal due to vignetting and vitreous floaters, and lower quality scans will have an increased prevalence for artifacts, such as motion artifacts.

Figure 9.

(a) Network architecture of multi-scaled encoder-decoder neural network (MEDnet). (b) Kernel sizes of atrous convolution blocks with different dilation rates. Representative data used for training. (c) The en face angiogram of the superficial vascular complex from a patient with diabetic retinopathy. (d) The reflectance image acquired by projecting the reflectance OCT data within the same slab used in (c). (e) The ground truth map of the avascular area. (f) Manually segmented avascular area, overlaid on the superficial vascular complex angiogram. Modified from Nagasato et al. ⁶⁴ (A color version of this figure is available in the online journal.)

Discussion

OCTA provides a label-free solution for high resolution examination of ocular vasculatures, and thus has been quickly adopted for clinical management of eye diseases. Quantitative OCTA features have been developed for the standardization of objective interpretation of OCTA. Different OCTA parameters that have been developed and their technical rationale are summarized in Table 1. Different pathological mechanisms may induce OCTA distortions in different ways. For example, DR patients have been commonly observed to have arterial narrowing ⁵⁴ and capillary level ischemia. ³⁰ However, in the current clinical setting, mass screening programs for common ocular conditions such as DR or SCR require experienced physicians to examine and evaluate retinal images. This process is both time consuming and expensive, making it difficult to scale up to incorporate the millions of individuals who harbor systematic diseases which are prone to affect the retina. Patients with early onset of retinopathies such as DR or SCR are initially asymptomatic yet require monitoring to ensure prompt medical interventions to prevent vision losses. AI-based CAD system may enable reliable performance by reducing subjective human error in screening procedures. Additionally, deep learning-based methods that learn from the images may be able to capture subtle changes that would be missed by human graders in the early stages of disease progression.

Table 1.

Summary of quantitative OCTA features.

Features	Equations
BVC	The ratio of the vascular area (Figure 3(b)) and the vascular length (Figure 3(d)) was defined as the average diameter of the vessels 36 $\begin{array}{c}BVC=\frac{{\sum }_{i=1,j=1}^{n}B\left(i,j\right)}{{\sum }_{i=1,j=1}^{n}S\left(i,j\right)}\end{array}$ where $B(i,j)$ * represents the pixels in the vessel map, and $S(i,j)$ represents the pixels in the skeleton map.
VPI	Ratio of the vessel perimeter area (Figure 3(c)) to the total image area 36 $\begin{array}{c}VPI=\frac{{\sum }_{i=1,j=1}^{n}P\left(i,j\right)}{{\sum }_{i=1,j=1}^{n}I\left(i,j\right)}\end{array}$ where $P(i,j)$ represents the pixels occupied by the perimeter map and $I(i,j)$ represents all the pixels in the perimeter map.
BVT	Each branch and corresponding endpoints are identified in the vessel skeleton map (Figure 3(d)). The metric to measure tortuosity was defined as the ratio between the Euclidean distance (ED) and the Geodesic distance (GD) 37 $\begin{array}{c}ED= \sqrt{{\left(x_1-x_2\right)}^2+{\left(y_1-y_2\right)}^2}\end{array}$ $\begin{array}{c}GD= {\int }_{t_0}^{t_1}\sqrt{{\left(\frac{dx\left(t\right)}{dt}\right)}^2+{\left(\frac{dy\left(t\right)}{dt}\right)}^2}dt\end{array}$ $\hspace{1em}\begin{array}{c}BVT=\frac{1}{n}{\sum }_{i=1}^n\left(\frac{GD \text{of\hspace{0.17em}\hspace{0.17em}vessel\hspace{0.17em}\hspace{0.17em}branch\hspace{0.17em}^\hspace{0.17em}}i}{ED\text{\hspace{0.17em}\hspace{0.17em}of\hspace{0.17em}\hspace{0.17em}vessel\hspace{0.17em}\hspace{0.17em}branch\hspace{0.17em}^\hspace{0.17em}}i}\right)\end{array}$ where i is the ith branch and n is the number of vessel branches.
FAZA	Area of the demarcated FAZ region (Figure 3(d)) is determined as follows. $\begin{array}{c}F\mathit{AZA}=\mathit{Area}\times {\sum }_{i=1,j=1}^{n}A\left(i,j\right)\end{array}$ where Area is the digital pixel resolution and A(i,j) represent the pixels occupied by the segmented avascular region.
FAZCI	The irregularity of the FAZ is determined using the FAZ contour map (Figure 3(d)) using the following equation36. $\begin{array}{c}F\mathit{AZCI}=\frac{{\sum }_{i=1,j=1}^{n}O\left(i,j\right)}{{\sum }_{i=1,j=1}^{n}R\left(i,j\right)}\end{array}$ Where O(i,j) represents the pixels in the FAZ contour map and R(i,j) represents the perimeter of a reference circle of the same area.
VCI	The relationship between the vessel map (Figure 3(b)) and perimeter map (Figure 3(c)) defines the complexity index 36 $\begin{array}{c}VCI=\frac{{\left({\sum }_{i=1,j=1}^{n}P\left(i,j\right)\right)}^2}{4\pi {\sum }_{i=1,j=1}^{n}B\left(i,j\right)}\end{array}$ where P(i,j) represents the pixels enclosed by the perimeter map and B(i,j) represents the pixels enclosed by the vessel map.
FD	Fractal dimension utilizes moving windows of varying sizes and is formulated as the following 39 $\begin{array}{c}FD=\frac{\log \left(N_s\right)}{\log \left(s\right)}\end{array}$ where Ns is the number of boxes at magnification s needed to enclose the image. Normalized of localized FD values close to 1 indicate large vessels and close to 0 indicates avascular regions 39
BVD	The ratio between the area occupied by the vessel and the total image area. $\begin{array}{c}BVD=\frac{{\sum }_{i=1,j=1}^{n}B\left(i,j\right)}{{\sum }_{i=1,j=1}^{n}I\left(i,j\right)} \end{array}$ where $B(i,j)$ represents the pixels in the vessel map, and $I(i,j)$ represents all the pixels in the vessel map.

*Unless stated otherwise, i, j represents the row and column positions of each pixel in the image, respectively.

Quantitative OCTA opens a unique opportunity to enable computer-aided disease detection and AI classification of different eye disease. Machine learning techniques have been explored for the different applications in OCTA. One of the main applications of machine learning in OCTA is for the classification of retinopathies, i.e. DR,^{15,30,36,52,60} SCR,^19,20 and CRVO. ⁶⁴ However, one limitation to these studies is the small dataset size. As a relatively new imaging modality, there is a limitation in the available datasets for OCTA. The dataset size is critical in machine learning applications, due to the problem of overfitting, which is when the algorithm has memorized the training data. While this challenge is quite posing for traditional machine learning approaches, strategies to overcome this limitation have been explored for deep learning methods. A commonly used method to increase the dataset size is the use of data augmentation, i.e. randomized rotations, vertical/horizontal flipping. Transfer learning can also be employed to alleviate the concerns of dataset size. By pretraining the CNN, the CNN will already be optimized for feature extraction and thus reducing the requirement for large data.

For model performance evaluations, cross-validation methods have been commonly employed. Cross-validation involves the division of the dataset into several pairs of training and testing datasets. The performance of the model on all the testing sets is averaged and reported. This therefore poses another challenge in machine learning applications which is the use of external validation. A recent study employing transfer learning for the classification of DR reported an overall accuracy of 87.27% on the cross-validation performance; however, additional external validation results revealed a 70.83% accuracy. This indicates that there is some overfitting due to a limited dataset. While the results of machine learning in OCTA show promising results, it is vital for future studies to employ external validation to evaluate the model’s performance on novel data. Additionally, many of these studies employ dataset from a homogenous demographic and one device and therefore may contain biases. To alleviate these issues, a multiple-institution collaboration is required.

While deep learning offers benefit to decrease burden in manual feature development and ease of usability, deep learning models have low interpretability. Since the user does not know what types of features are being learned and used for to make the prediction. Different methods have been explored to alleviate this “black box” view of deep learning, such as occlusion maps ⁶⁵ and grad-cams,^52,64 which highlight areas of the input image contributes most to the prediction. For example, a CNN may predict an image as referrable DR. Employing grad-CAM could highlight areas of hard exudates in the input image that have a strong influence on the final classification (Figure 7(b)). Therefore, future studies should incorporate these techniques to increase the physician’s trust in the network and acceptability of AI for clinical use.

Conclusion

OCTA provides a noninvasive label-free solution for high resolution imaging of the retinal vasculature. Quantitative OCTA features have been developed to standardize vascular distortions associated with eye conditions. This creates a unique opportunity to employ AI-technologies in OCTA. Machine learning algorithms have been employed to utilize OCTA features for the classification and staging of different retinopathies. Deep learning has also been demonstrated as a viable tool for automatic AV-classification, segmentation of avascular areas and other regions of interests in OCTA. AI-based technologies can alleviate the burden for experienced physicians and foster mass screening programs.