A Deep Learning Network for Classifying Arteries and Veins in Montaged Widefield OCT Angiograms

Abstract

Purpose

To propose a deep-learning−based method to differentiate arteries from veins in montaged widefield OCT angiography (OCTA).

Design

Cross-sectional study.

Participants

A total of 232 participants, including 109 participants with diabetic retinopathy (DR), 64 participants with branch retinal vein occlusion (BRVO), 27 participants with diabetes but without DR, and 32 healthy participants.

Methods

We propose a convolutional neural network (CAVnet) to classify retinal blood vessels on montaged widefield OCTA en face images as arteries and veins. A total of 240 retinal angiograms from 88 eyes were used to train CAVnet, and 302 retinal angiograms from 144 eyes were used for testing. This method takes the OCTA images as input and outputs the segmentation results with arteries and veins down to the level of precapillary arterioles and postcapillary venules. The network also identifies their intersections. We evaluated the agreement (in pixels) between segmentation results and the manually graded ground truth using sensitivity, specificity, F1-score, and Intersection over Union (IoU). Measurements of arterial and venous caliber or tortuosity are made on our algorithm’s output of healthy and diseased eyes.

Main Outcome Measures

Classification of arteries and veins, arterial and venous caliber, and arterial and venous tortuosity.

Results

For classification and identification of arteries, the algorithm achieved average sensitivity of 95.3%, specificity of 99.6%, F1 score of 94.2%, and IoU of 89.3%. For veins, the algorithm achieved average sensitivity of 94.4%, specificity of 99.7%, F1 score of 94.1%, and IoU of 89.2%. We also achieved an average sensitivity of 76.3% in identifying intersection points. The results show CAVnet has high accuracy on differentiating arteries and veins in DR and BRVO cases. These classification results are robust across 2 instruments and multiple scan volume sizes. Outputs of CAVnet were used to measure arterial and venous caliber or tortuosity, and pixel-wise caliber and tortuosity maps were generated. Differences between healthy and diseased eyes were demonstrated, indicating potential clinical utility.

Conclusions

The CAVnet can classify arteries and veins and their branches with high accuracy and is potentially useful in the analysis of vessel type-specific features on diseases such as branch retinal artery occlusion and BRVO.

OCT angiography (OCTA) is a powerful imaging modality for noninvasive, 3-dimensional, and detailed assessment of retinal vasculature.^1,2 Numerous studies have demonstrated the utility of OCTA in retinal diseases. For example, nonperfusion area is a key biomarker in the evaluation of diabetic retinopathy (DR),3, 4, 5, 6 and accurate identification and segmentation of choroidal neovascularization in the outer retinal angiogram is helpful for the diagnosis and management of neovascular age-related macular degeneration.⁷

One challenge with OCTA technology is distinguishing arteries from veins. Correctly identifying the vessels can be critical in understanding certain diseases. For example, retinal arteriolar narrowing is associated with hypertension and DR,^8,9 venous beading is a feature of moderate to severe DR,¹⁰ and different types of retinal vascular occlusion (artery or vein) would have vessel-specific dilation or constriction.^11,12 Classifying arteries and veins may enable improved disease characterization by specifically evaluating the vessel for pathologic changes; however, manual segmentation of arteries and veins not only requires specific expertise but is also time-consuming. Automation of this task is indispensable if specific characterization for arteries and veins is to be practical.

A number of algorithms have been proposed to classify arteries and veins using OCTA. However, limitations in each approach still call for improvement. Alam et al^13,14 proposed using fundus photography or structural OCT features to guide the classification of arteries and veins in OCTA images. These algorithms based on OCTA need other imaging modalities or structural OCT to help classify arteries and veins, which in turn may introduce more errors by fusing multiple types of images. Xu et al¹⁵ differentiated veins from arteries in OCTA by identifying deep capillary plexus vortices. Although they described a method for distinguishing retinal arteries from veins on OCTA, it is not an automated approach. Ishibazawa et al¹⁶ evaluated the accuracy and reliability in differentiating retinal arteries from veins using widefield OCTA. Although their study showed that readers can correctly classify large arteries and veins using only widefield OCTA in healthy eyes and eyes with severe DR, trained graders were required for the task. Kim et al¹⁷ proposed differentiating arteries from veins in OCT and OCTA of the mouse retina, and their method manually performed artery and vein classification using vascular morphology and blood flow signatures. This means the approach will not scale well for clinical application.

More recently, deep learning has been used to aid OCTA image enhancement and segmentation of vascular features.¹⁸ Alam et al¹⁹ also proposed a deep-learning approach to artery/vein classification on OCTA. However, their approach focused on just the macular region and required structural OCT input, which increased computation requirements. Furthermore, the requirement for structural OCT input means that graders should review structural information as well as angiographic if a scan requires verification, increasing the amount of time needed to determine if correction is required. Their results also include a significant number of anatomically inaccurate spurs where vessel walls should be smooth.

We propose an end-to-end convolutional neural network, the Classification of Artery and Vein Network (CAVnet), that can classify arteries and veins in montaged widefield OCTA covering a 6 × 17-mm field of view and subimages in the same area. In this algorithm, the classification is purely based on OCTA input, and the vessel types are stratified starting from peripapillary major retinal arteries/veins to precapillary branches. In this study, we also characterize the caliber and tortuosity of arteries and veins of healthy and eyes with retinopathy.

Methods

Data Acquisition

The study was approved by an Institutional Review Board/Ethics Committee of Oregon Health & Science University, and informed consent was collected from all participants, in compliance with the Declaration of Helsinki. In this study, 27 healthy controls, 27 eyes from diabetic eyes without DR, 109 participants diagnosed with DR (52 with mild to moderate nonproliferative diabetic retinopathy [NPDR], 57 with severe NPDR and proliferative diabetic retinopathy [PDR]), and 64 participants diagnosed with branch retinal vein occlusion (BRVO) were enrolled. One eye from each participant underwent 6 × 6-mm volumetric scans centered at the macula and the immediate areas nasal and temporal to the macular scan using a 70-kHz AngioVue OCTA system (RTVue-XR Avanti; Optovue, Inc.) with a central wavelength of 840 nm. In this scan pattern, 2 repeated B-scans were obtained at each of 400 raster positions and each B-scan containing 400 A-lines. Additionally, 5 healthy eyes were scanned using a 120-kHz AngioVue OCTA system (Solix, Optovue, Inc.). The eyes were scanned using 9 × 9-mm scan patterns. The details of the acquired data are presented in Table 1. Two repeated B-scans were taken at each of the 600 raster positions, and each B-scan consisted of 600 A-lines. The split-spectrum amplitude-decorrelation angiography algorithm was used to generate the OCTA data from both instruments.¹ Retinal layer boundaries were segmented by a guided bidirectional graph search algorithm.²⁰ Inner retinal angiograms were generated by maximum projection of the OCTA signal in a slab from the internal limiting membrane to the outer plexiform layer.

Table 1.

Datasets

Dataset for Convolutional Network
	Disease		Nasal	Macula	Temporal	Eyes
Training Dataset	No DR	Healthy Control	10	10	10	10
		Diabetes without DR	19	19	19	19
	DR	Mild to Moderate NPDR	27	22	22	27
		Severe NPDR or PDR	32	28	22	32
Test Dataset	No DR	Healthy Control	17	17	17	17
		Diabetes without DR	8	8	8	8
	DR	Mild to Moderate NPDR	25	25	25	25
		Severe NPDR or PDR	25	25	25	25
BRVO Dataset
	Disease		Scans			Eyes
	BRVO		64			64
9 × 9-mm Scans from Solix
	Disease		Scans			Eyes
	Healthy Control		13			5

BRVO = branch retinal vein occlusion; DR = diabetic retinopathy; NPDR = nonproliferative diabetic retinopathy; PDR = proliferative diabetic retinopathy.

Convolutional Neural Network Architecture

The structure of CAVnet is illustrated in Figure 1 (more details are avalible in Supplemental Fig S1, Tables S1, S2, and Appendix). En face OCT angiograms projected from the retina were input to the network to segment arteries and veins. The CAVnet adopted a U-net−like architecture that is composed of an encoder and a decoder. The encoder includes 4 down-sampling layers to reduce the image resolution and 5 sub-modules to extract features at different scales. Each sub-module fused features extracted by a convolution layer and atrous convolutional layer with different receptive fields. The features with different receptive fields and at different scales provide the network with contextual and global information. The decoder consists of 4 sub-modules and deconvolutional upsampling layers. The resolution is sequentially increased through the up-sampling operation until it is consistent with the resolution of the input image. The network also uses a skip connection to connect the up-sampling result with the output of the sub-module with the same resolution in the encoder as the input of the next sub-module in the decoder. Except for the last convolutional layer, batch normalization and a LeakyRelu activation function are used after each convolutional layer. A softmax activation function is used for multiple classifications in the last convolutional layer.

Figure 1.

Classification of artery and vein network (CAVnet) architecture. A, The input, consisting of an OCT angiogram of the inner retina. CAVnet has a U-net−like structure. The different colored rectangles represent operations with different parameters. The stride is represented by s. The dilation of the atrous convolutional layer is represented by r. The segmentation results output by CAVnet distinguishes between arteries and veins (B).

Training

Subjects and Ground Truth Generation

The datasets included the retinal angiograms of eyes without DR (either healthy or diabetes mellitus without DR), eyes from patients with DR, and eyes with BRVO. For each eye without DR and eye with DR, volumetric OCT and OCTA scans from the nasal, macula, and temporal were acquired. For each eye with BRVO, volumetric OCT and OCTA scans from the macula were collected. A total of 29 scans with poor quality were excluded, yielding a total of 537 retinal angiograms. A total of 240 angiograms from 88 eyes were used for training, and 302 angiograms from 144 eyes were used for testing. No eyes were shared in the training and testing of the model; the number of eyes is shown in Table 1. To make a reliable and accurate quantitative comparison of the test results, we set the number of eyes from DR with different severity to be the same in the test dataset. In the training phase, the angiograms from 3 regions were fed into CAVnet, separately. For predictions, the input can be independent angiograms from these 3 regions or widefield angiograms, which include nasal, macular, or temporal scans. Although a previous study has confirmed that en face OCTA allows for accurate and reliable artery and vein identification,¹⁶ we still used fundus photographs as a guide to delineate accurate ground truth for training. First, we enhanced the OCT angiogram quality using a deep-learning−based capillary reconstruction algorithm²¹ (Fig 2B), which is only used for ground truth delineating. Then, 2 certified graders (G.P. and D.P.) manually delineated arteries and veins on enhanced OCTA images according to the corresponding fundus photographs (Fig 2D, E). The third certified grader (M.G.) reviewed and corrected the delineated ground truth maps. In color fundus photographs and OCTA (Fig 2A, D, E), various characteristics can help to differentiate arteries from veins: (1) Arteries have higher reflectance than veins; (2) the central light reflex is wider in arteries and smaller in veins; (3) arteries are smaller in caliber than adjacent veins; (4) normally, arteries and veins alternate as each vein drains the capillary bed perfused by adjacent arteries;^15,22 (5) arteries do not cross other arteries and veins do not cross other veins;¹⁶ (6) there are obvious capillary-free zones near the retinal arteries in OCTA images.^23,24 These properties provide graders with sufficient information to accurately delineate the ground truth.

Figure 2.

Ground truth generation. A, Three angiograms of separate regions (nasal, macular, and temporal) are montaged to produce a widefield image. B, The enhanced angiograms with low noise intensity, good connectivity, and strong contrast. C, The manually delineated ground truth. Arteries are colored in red, green represents veins, and blue is intersection points. Optic disc (D) and macular fundus (E) photographs corresponding to OCT angiography (OCTA), with arteries and veins indicated by red and green arrows. The original en face images of OCTA and fundus photographs serve as guides to delineate the accurate ground truth.

Training Parameters

During training, we used a combination of dice and categorical cross-entropy (CE) loss as the loss function. The dice coefficient D is used to measure the overlap of the 2 segmentations

$$D=\frac{2\left|Y\cap Y^{\prime }\right|}{\left|Y\right|+\left|Y^{\prime }\right|}$$ (1)

where $Y$ is the ground truth and $Y^{\prime }$ is the predicted image, $\left|Y\cap Y^{\prime }\right|$ is the cardinality of intersection of the $Y$ and $Y^{\prime }$, and $\left|Y\right|$ and $\left|Y^{\prime }\right|$ represent the number of elements in $Y$ and $Y^{\prime }$, respectively. D is a value between 0 and 1, and with 0 indicating no overlap and 1 a perfect overlap.

The CE loss

$$CE=-\sum _i{y}_{i}log{y}_i^{\prime }$$ (2)

is usually used for multiple classifications,²⁵ where $y_i$ is the ground truth of the $i^{th}$ category. $y_i^{\prime }$ is the output results of the $i^{th}$ category. CE is a value between 0 and positive infinity, and close to 0 when the predicted image is approaching the ground truth. The loss function is then defined as

$$L=\left(1-D\right)+CE.$$ (3)

The combination of dice loss and CE helped with category imbalance in the training data.²⁶

We used 400 × 400-pixel 6 × 6-mm images to train CAVnet. Several data augmentation methods were used to expand the training dataset: Each batch of images is randomly flipped horizontally, vertically, and transposed before being fed into the network for iterative training. Considering both convergence time and hardware limitations, the training batch size was set to 2. We used an Adam optimizer²⁷ with an initial learning rate of 0.001 to optimize the loss. The learning rate was reduced by a factor of 0.1 when the loss did not decline after 10 epochs. The minimum learning rate is 1 × 10⁻⁸. When the loss does not change more than 1 × 10⁻⁵ in 20 epochs, training will stop. The CAVnet was implemented using Python 3.7 with Keras (Tensorflow-backend) on a PC with a 64G RAM and Intel i7 CPU, and 2 NVIDIA GeForce GTX1080Ti graphics cards. For more details of hyper-parameter settings, please refer to https://github.com/octangio/CAVnet.

Results

Performance Evaluation

To assess the performance of our algorithm, we separately evaluated arteries, veins, and intersection point accuracy on the test dataset. The pixel-wise sensitivity (Eq. (4)), specificity (Eq. (5)), F1 score (Eq. (6)), and Intersection over Union (IoU) (Eq. (7)) between the segmentation results and the ground truth were evaluated as follows:

$$\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}}$$ (4)

$$\text{Specifivity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$$ (5)

$$\text{F}1-\text{score}=\frac{2\times \text{TP}}{2\times \text{TP}+\text{FP}+\text{FN}}$$ (6)

$$\text{IoU}=\frac{\text{TP}}{\text{TP}+\text{FP}+\text{FN}}$$ (7)

Where TP is true-positive, TN is true-negative, FN is false-negative, FP is false-positive. Our algorithm achieved high performance with each of these metrics for artery and vein classification (Table 2), but artery/vein intersection points were only partially accurate. For determining whether intersection points output by the network are the ones manually graded, we dilated the network output using a 3 × 3 disk-shaped structuring element. This helps to define the match between output and ground truth by confirming the location correlation. We speculate on this performance gap in the discussion.

Table 2.

Artery and Vein Segmentation Performance on Test Dataset (N of scan = 225)

	Sensitivity (%)	Specificity (%)	F1-score (%)	IoU (%)
Artery	95.3 ± 4.1	99.6 ± 0.3	94.2 ± 4.1	89.3 ± 7.0
Vein	94.4 ± 5.0	99.7 ± 0.3	94.1 ± 4.3	89.2 ± 7.2
IntersectionPoints	76.3 ± 18.6	1.0 ± 0.0	85.2 ± 13.2	76.3 ± 18.6

IoU = Intersection over Union.

For qualitative verification of these results, we montaged nasal, macular, and temporal angiograms into a widefield angiogram covering a 6 × 17-mm field of view (Fig 3). The predicted results are highly consistent with the ground truth whether on healthy eyes (Fig 3A1−A3) or on eyes with DR (Fig 3B1−B3). As the image shows, CAVnet classified arteries and veins down to the level of precapillary arterioles and postcapillary venules.

Figure 3.

The performance of the classification of artery and vein network (CAVnet) classification demonstrated by a healthy eye (A1−A3) and an eye with severe proliferative diabetic retinopathy (PDR) (B1−B3). Row 1: retina angiograms montaged by nasal, macular, and temporal scans. Row 2: ground truth of classification of arteries and veins. Row 3: prediction from CAVnet.

The CAVnet may misclassify part of an artery to a vein or vice versa (Fig 4A, B) when some features are ambiguous. Other potentially troublesome regions include artery-vein intersections (Fig 4C, D).

Figure 4.

Examples of misclassification. A, C, The input of the classification of artery and vein network (CAVnet). B, D, The segmentation results produced by CAVnet. The yellow arrows show segmentation errors in the output images. Mis-segmentation may be caused when CAVnet overfits the “artery-vein-alternation rule” or when the capillary-free zone near arteries is unobvious. Artery-vein crossings may also make the algorithm generate misclassifications; however, note that in most places the algorithm correctly interpreted crossing vessels.

Caliber of Arteries and Veins

We calculated the caliber of arteries and veins on the test dataset using CAVnet’s output. The caliber of the blood vessels was defined as the ratio of vascular area and vascular length that were acquired from a binarized and skeletonized artery-vein OCTA map, respectively. To calculate vessel caliber, we consider each foreground pixel as well as the 7 adjoining pixels along the skeletonized artery-vein OCTA map in each direction (15 pixels total) and calculate the ratio of the vessel area to the vessel length. Then the caliber value was mapped to the binarized artery-vein OCTA map (Fig 5A−C). The caliber was calculated in the 6 × 6-mm angiograms excluding a 2-mm diameter circle centered on the optic disc. Caliber changes can be observed on the caliber map of angiograms in Figure 5A−C. The minimal caliber that CAVnet can detect is 1 pixel (15 μm). For statistical analysis of these results, we treated each eye as a single observation. We performed the Kolmogorov−Smirnov test, which indicated that neither the arterial nor venous caliber in nasal, macular, and temporal areas was normally distributed. For each area, a nonparametric Kruskal−Wallis test was applied to suggest if there is a difference between at least 1 pair of groups. Then, to find out which pairs of groups have a difference, post hoc tests with Bonferroni correction were performed on each pair of groups. We found that there is no significant difference in the arterial caliber between eyes without DR and eyes with mild to moderate NPDR in the 3 areas (Fig 5D) (nasal, P = 1.00; macula, P = 0.99; temporal, P = 1.00); however, compared with eyes without DR or eyes with mild to moderate NPDR, the arterial caliber of eyes with either severe NPDR or PDR was significantly reduced (Fig 5D). There is no significant difference in venous caliber between eyes without DR and eyes with DR in the optic disc area (P = 0.34, Kruskal−Wallis test). Compared with eyes without DR, venous caliber in eyes with mild to moderate NPDR had no significant difference in the macular (P = 0.24) and temporal (P = 1.00) areas; however, venous caliber in eyes with either severe NPDR or PDR was smaller in the macular region (P = 0.003) (Fig 5E). Compared with eyes with mild to moderate NPDR, venous caliber in eyes with either severe NPDR or PDR was also greatly reduced in the temporal area (P = 0.01) (Fig 5E). We also used a Mann−Whitney U test to compare the caliber of arteries and veins in 17 healthy eyes from test dataset. The caliber of arteries (mean ± standard deviation, 69.6 ± 25.3 μm) is much smaller than that of veins (82.6 ± 31.0 μm) (P = 0.003) at the nasal location. Arterial (48.2 ± 2.2 μm) and venous (51.3 ± 3.5 μm) caliber (P = 0.006) are also different at temporal locations (Fig 6B), but this difference disappears in the macula (P = 0.92). In addition, we calculated the caliber of major peripapillary arteries and veins (radiated from optic nerve head) in the nasal scans (Fig 7). The caliber of the major veins (113.7 ± 20.2 μm) is much larger than that of the major arteries (100.2 ± 16.1 μm, P = 0.001, Mann−Whitney U test). This result corresponds to the previous findings that major vein caliber (120.9 ± 27.2 μm) is larger than that of the major arteries (103.3 ± 22.2 μm, P < 0.001) by Falavarjani et al²⁸ using a manual measurement method.

Figure 5.

Caliber map of the montaged widefield angiograms from a diabetic eye without diabetic retinopathy (DR) (A), an eye with mild to moderate nonproliferative diabetic retinopathy (NPDR) (B), and an eye with severe PDR (C); boxplots of arterial (D) and venous (E) caliber compared among 3 groups at 3 scan locations. Kruskal−Wallis test (P¹) was first applied to suggest if there is a difference between at least 1 pair of groups, and then a post hoc test with Bonferroni correction (P²) was applied to multiple pairs comparison.

Figure 6.

Comparison of caliber between arteries and veins in healthy eyes. A, Caliber map from a healthy eye. B, Boxplot showing artery and vein caliber at 3 locations in healthy eyes. Mann−Whitney U test (P³) was applied to compare if there is a difference between artery and vein caliber at 3 locations.

Figure 7.

Measurements of arterial and venous caliber at the nasal location. A, B, The caliber (μm) of arteries and veins was measured in the ring area with a diameter of 1.7 mm to 3.4 mm centered on the optic disc (yellow circles). The cyan values indicate the caliber of the veins. The yellow values represent the caliber of arteries.

Tortuosity

We also calculated the tortuosity of arteries and veins using CAVnet’s output. For each foreground pixel in the skeletonized artery-vein map, we use the 100 adjoining pixels to either side (201 pixels total) to calculate the tortuosity. The tortuosity value was mapped to the binarized artery-vein OCTA map (Fig 8A−C). The tortuosity was calculated in the 6 × 6-mm angiograms excluding a 2-mm diameter circle centered on the optic disc. We adopted the same statistic test method from the last section for tortuosity comparisons. Although vessels with large tortuosity values were highlighted in the maps, the tortuosity had no significant changes in eyes with mild to moderate NPDR compared with eyes without DR for either arteries or veins (Fig 8D, E). Compared with eyes without DR or eyes with mild to moderate NPDR, the mean tortuosity also had no significant changes in eyes with either severe NPDR or PDR (Fig 8D, E). No significant difference was found in the mean tortuosity of arteries compared with veins at nasal, macular, or temporal locations in healthy eyes (Fig 9), which is consistent with previous measurements.²⁹

Figure 8.

Tortuosity map of the montaged widefield angiograms from a diabetic eye without diabetic retinopathy (DR) (A), an eye with mild to moderate nonproliferative diabetic retinopathy (NPDR) (B), and an eye with severe proliferative diabetic retinopathy (PDR) (C). Brighter colors (green arrows) indicate the higher tortuosity of the vessel. D, E, Comparison of tortuosity between eyes without DR and eyes with DR. No significant difference in mean tortuosity of eyes with DR compared with eyes without DR was found, as evaluated by the Kruskal−Wallis test (P¹).

Figure 9.

A, Tortuosity map from a healthy eye. B, Comparison of mean tortuosity between arteries and veins in healthy eyes. There is no significant difference between arterial and venous tortuosity in healthy eyes, as evaluated by the Mann−Whitney U test (P³).

Performance on Scans from Eyes with Other Retinal Vascular Diseases

Retinal venous occlusion is the second most common retinal vascular disease after DR³⁰ and is a disease that shows different pathology on arteries and veins, respectively. We applied our algorithm to macular scans from 64 eyes with BRVO (Fig 10A1). We also calculated performance metrics such as sensitivity or F1 score for CAVnet’s output on these data. The results showed high classification accuracy on these eyes (Fig 10B1) (Table 3). We also calculated vessel caliber (Fig 10C1) and the tortuosity (Fig 10D1) for this data set. Results from 17 healthy eyes from the test dataset and 64 eyes with BRVO in the macular center area showed that the caliber of arteries and veins in eyes with BRVO both greatly reduced compared with healthy eyes (P < 0.001, Mann−Whitney U test) (Fig 11A). A previous study using revised Parr–Hubbard formulas also showed that there were significant differences in mean central retinal arterial diameter and venous diameter between the control group and the BRVO group (P < 0.05).³¹ The tortuosity of veins significantly increased (P < 0.001, Mann−Whitney U test), and the tortuosity of arteries had no significant change (P = 0.18, Mann−Whitney U test) (Fig 11B), which is consistent with previously reported work showing increased venous tortuosity in eyes with BRVO by observing the number of tortuous vessels.³²

Figure 10.

The classification of artery and vein network (CAVnet) output in an eye with branch retinal vein occlusion (BRVO) (top row) and a healthy control (bottom row). A, Macular angiograms. B, CAVnet output for (A). C, Vessel caliber map. D, Tortuosity map. The prediction showed high accuracy on scans from eyes with BRVO. The changes in arterial and venous caliber can be observed in the caliber map. The high tortuosity of veins in BRVO is obvious in the tortuosity map.

Table 3.

Artery and Vein Segmentation Performance on Eyes Diagnosed with Branch Retinal Vein Occlusion (N of Scan = 64)

	Sensitivity (%)	Specificity (%)	F1-score (%)	IoU (%)
Artery	97.4 ± 2.7	99.6 ± 6.0	94.5 ± 3.6	89.9 ± 6.4
Vein	92.2 ± 0.1	99.9 ± 0.1	94.6 ± 3.8	90.0 ± 6.6
IntersectionPoints	93.7 ± 8.9	1.0 ± 0.0	96.5 ± 5.2	93.7 ± 8.9

IoU = Intersection over Union.

Figure 11.

Comparison of the mean arterial and venous caliber and tortuosity of 17 healthy eyes and 64 eyes with branch retinal vein occlusion (BRVO) in the macular center area. A, Comparison of caliber. B, Comparison of tortuosity. Mann−Whitney U test (P³) was used to compare whether caliber and tortuosity deviate significantly from a healthy distribution in BRVO eyes. Compared with healthy eyes, the caliber of arteries and veins both greatly reduced in eyes with BRVO. The tortuosity of veins significantly increased.

Performance on Scans from Different Devices and Larger Field-of-View Scans

In addition to testing data from an AngioVue device, we also tested our algorithm on larger field-of-view 9 × 9-mm angiograms from a Solix instrument (Fig 12). The predicted artery and vein maps also appear to be correct, indicating performance did not noticeably degrade on different devices (Table 4).

Figure 12.

The classification of artery and vein network (CAVnet) outputs for larger-field-of-view scans. A, 9 × 9-mm angiogram of the retina. B, Ground truth. C, Predicted result by CAVnet for A. The results show high accuracy on 9 × 9-mm scans.

Table 4.

Artery and Vein Segmentation Performance on 9 × 9-mm Scans from Solix (N of Scan = 13)

	Sensitivity (%)	Specificity (%)	F1-score (%)	IoU (%)
Artery	91.3 ± 0.2	99.7 ± 0.2	92.6 ± 2.2	86.4 ± 3.7
Vein	87.0 ± 4.0	99.9 ± 0.0	92.1 ± 2.4	85.5 ± 4.0
IntersectionPoints	91.0 ± 10.2	1.0 ± 0.0	94.9 ± 6.1	91.0 ± 10.2

IoU = Intersection over Union.

Discussion

The U-Net is a popular architecture developed for biomedical image segmentation that can learn to segment features using relatively small datasets while maintaining feature resolution through the use of skip connections. There are many applications of U-Nets in biomedical image segmentation, which show reliable and highly accurate results.^33,34 A U-net−like architecture we adopted in this study is an end-to-end fully convolutional network that includes an encoder and a decoder. The CAVnet can automatically and accurately classify arteries and veins in montaged widefield OCT angiograms, and includes several innovations. Among these advantages is the input data set. Although arteries and veins can be distinguished by morphologic (i.e., caliber) and anatomic (i.e., the alternating artery-vein rule) features available in OCTA images, such an identification is often difficult and therefore time-consuming for human graders. For this reason, artery/vein differentiation in OCTA images has usually been supplemented with other imaging modalities, for example, oximetry.^13,35 However, combinations of multiple imaging modalities introduce a large burden into clinical practice and so should be avoided if a result can be achieved using just a single imaging technique. Our network achieves artery/vein differentiation using only a single OCT angiogram as input.

For prediction, the input to CAVnet could be independent angiograms from nasal, macular, and temporal regions, which can save computing budget. Widefield images can also function as input. Postprocessing on the CAVnet output can also stratify vessels with caliber ranging from major peripapillary vessels (∼225 μm) to pericapillary levels (∼15 μm). By identifying vessels at multiple orders and scales, CAVnet can provide more comprehensive information concerning retinal circulation, which could be helpful for assessing disease. Lastly, we believe that artery/vein classification is broadly useful for OCTA image analysis. Because arteries and veins are not routinely differentiated, disparate effects on each by various diseases may go undetected, even though such information could be used to improve diagnosis. Furthermore, from a basic research perspective, artery/vein classification could aid investigations of pathophysiology.

We used several metrics to assess the algorithm’s performance, including sensitivity, specificity, F1-score, and IoU. We found that our network performed strongly in each of these categories, with each of sensitivity, specificity, and F1-score approaching or exceeding 95% accuracy. Performance gauged by IoU was slightly lower (∼90% accuracy), but some of the inaccuracy in this metric may represent anatomically meaningless disparities resulting from single-pixel width translocations between the ground truth and the network output. In each case, performance was essentially equivalent for performing artery or vein segmentation, indicating no bias toward one vessel type or the other in output. We also investigated network applicability by characterizing CAVnet’s output on different image sizes, noting no obvious performance dropoff.³⁶ Moreover, our results did not generate inaccurate spurs, which are not encountered anatomically. Finally, we also investigated CAVnet’s performance across multiple diseases (including a spectrum of DR severity) and healthy eyes, and across multiple devices. Our results indicate that CAVnet’s performance was resilient to these different contexts.

Caliber and tortuosity are both important quantitative indicators of diabetes mellitus, cardiovascular disease, hypertension, and atherosclerosis;37, 38, 39, 40, 41, 42 however, because artery/vein differentiation is not usually performed in OCTA images, these quantities are usually not calculated separately for the different vessel types in OCTA imaging. However, our results indicate that pathophysiologic differences between these vessel types in disease are detectable by OCTA. Compared with eyes without DR (either healthy or diabetes mellitus without DR), the caliber of arteries is significantly reduced in eyes with either severe NPDR or PDR. We found that the caliber of veins is also reduced in the macular area, relative to eyes without DR. The reason may be that the basal membrane of arteries and veins in DR cases is thicker than in healthy controls; consequently, the functional vascular lumen imaged by OCTA is smaller, with greater changes in arteries than in veins.^37,43 Likewise, compared with healthy eyes, the arterial and venous caliber of eyes with BRVO significantly decreased in the central macula. As a previous study has demonstrated, a decrease in retinal arterial and venous diameters was seen in eyes with BRVO.³¹ We could also detect variation in vessel caliber based on region. In healthy eyes, the caliber of veins (82.6 ± 31.0 μm) is larger than that of arteries (69.6 ± 25.3 μm) at the nasal location (P = 0.003). There is also a detectable difference in arterial (48.2 ± 2.2 μm) and venous (51.3 ± 3.5 μm) caliber in the temporal area (P = 0.006). At the macula, however, there was no significant difference in artery and vein caliber. In the macula, we would expect differences between artery and vein caliber to diminish, because the size of the higher-order vessels prevalent in this region approaches the capillary limit. In addition, we also calculated the caliber of major vessels at the nasal location in healthy eyes. The mean caliber of the major veins (113.7 ± 20.2 μm) is larger than that of the largest arteries (100.2 ± 16.1 μm) at the nasal location (P = 0.001); these findings are similar to histological results.²⁸ These results on healthy eyes can be taken as a further indication of the validity of CAVnet’s output, because they concur with known retinal vascular physiology.

Unlike vessel caliber, our network’s output produces arteries with tortuosity that is not significantly different than veins in healthy eyes. This result and the tortuosity values we measure are similar to those of a prior study.^29,44 The arterial or venous tortuosity of eyes with DR has no great difference compared with that of eyes without DR. The lack of difference in tortuosity could be a result of large inherent variation in the population. The tortuosity of veins significantly increased in eyes with BRVO compared with healthy eyes, which is in line with clinical observations.⁴⁵

Study Limitations

There are some limitations in CAVnet. First, CAVnet may generate false segmentation in some specific conditions. For example, CAVnet may misclassify part of an artery to a vein or vice versa, which may be because CAVnet over-learned the “artery-vein-alternation rule” or over-relies on the conspicuous capillary-free zone near arteries (which could be mimicked by shadow artifacts). Likewise, the sensitivity for arteries is slightly higher than that of veins due to the same obvious capillary-free zone near the retinal arteries in OCTA images, which would enable vessels to be more readily distinguished from the background. We were also unable to obtain helpful interpretable results within the 2-mm diameter circle centered on the optic disc because this area includes intricate vessel morphologies different than the rest of the retina. In this region, CAVnet had lower classification performance (sensitivity, 73.9% ± 12.2%; F1-score, 80.5% ± 10.1%; IoU, 68.5% ± 12.9%) than other areas. The CAVnet generates incorrect artery-vein intersection segmentations at a higher rate than vein or artery mis-segmentations. This may be caused by the small number of samples of intersections in the training set. Furthermore, compared with arteries and veins, the number of pixels in an intersection is small even after the dilation we applied to the output. This means that a single misplaced pixel can lead to a large deterioration in performance metrics because it will represent a greater part of the sample. This is especially apparent in the IoU metric, which tends to penalize individual classification errors more than the F1-score (similar to L1 vs. L2 loss). We think these facts probably largely explain the performance decline for intersection points. However, it should be noted that although these performance metrics were less satisfactory, they largely represent clinically irrelevant, small differences in the location of a small number of pixels.

Conclusions

We proposed an end-to-end convolutional neural network that classifies arteries and veins: CAVnet. The CAVnet not only classified arteries and veins down to the level of precapillary arterioles and postcapillary venules but also detected the intersection of arteries (or arterioles) and veins (or venules). The CAVnet has a high performance for differentiating arteries and veins, even in severe DR and BRVO cases. Measurements of arterial and venous caliber or tortuosity can potentially help with the diagnosis of DR and BRVO; our method is capable of extracting these measurements with high accuracy. Finally, CAVnet also performs well on widefield images, an important capability in OCTA as bleeding edge research continues to push toward larger fields of view. We believe that this method can have applications in both the clinic and basic research.

Manuscript no. XOPS-D-21-00188.