Cross-modality Labeling Enables Noninvasive Capillary Quantification as a Sensitive Biomarker for Assessing Cardiovascular Risk

Abstract

Purpose

We aim to use fundus fluorescein angiography (FFA) to label the capillaries on color fundus (CF) photographs and train a deep learning model to quantify retinal capillaries noninvasively from CF and apply it to cardiovascular disease (CVD) risk assessment.

Design

Cross-sectional and longitudinal study.

Participants

A total of 90732 pairs of CF-FFA images from 3893 participants for segmentation model development, and 49229 participants in the UK Biobank for association analysis.

Methods

We matched the vessels extracted from FFA and CF, and used vessels from FFA as labels to train a deep learning model (RMHAS-FA) to segment retinal capillaries using CF. We tested the model's accuracy on a manually labeled internal test set (FundusCapi). For external validation, we tested the segmentation model on 7 vessel segmentation datasets, and investigated the clinical value of the segmented vessels in predicting CVD events in the UK Biobank.

Main Outcome Measures

Area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity for segmentation. Hazard ratio (HR; 95% confidence interval [CI]) for Cox regression analysis.

Results

On the FundusCapi dataset, the segmentation performance was AUC = 0.95, accuracy = 0.94, sensitivity = 0.90, and specificity = 0.93. Smaller vessel skeleton density had a stronger correlation with CVD risk factors and incidence (P < 0.01). Reduced density of small vessel skeletons was strongly associated with an increased risk of CVD incidence and mortality for women (HR [95% CI] = 0.91 [0.84-0.98] and 0.68 [0.54-0.86], respectively).

Conclusions

Using paired CF-FFA images, we automated the laborious manual labeling process and enabled noninvasive capillary quantification from CF, supporting its potential as a sensitive screening method for identifying individuals at high risk of future CVD events.

Financial Disclosure(s)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

The use of retinal photographs provides a noninvasive method to evaluate the health and functionality of the vasculature within the human body. This has led to significant interest in identifying retinal vessel alterations as potential biomarkers that may aid in the detection of diseases that affect the eye, brain, and heart, such as cardiovascular diseases (CVDs), which include heart attack and stroke.1, 2, 3, 4 Because of their high rates of morbidity and mortality worldwide, prevention and early detection of CVD are crucial.⁵

Quantitative changes in retinal vessels have been identified as independent predictors of systemic health and disease.^1,2,6 However, manual labeling of these vessels is a laborious process, leading to a limited number of labeled images available for model development, often < 30 images per dataset. Although semiautomatic or automatic retinal vessel measurement programs like QUARTZ,⁷ VAMPIRE,⁸ AutoMorph,⁹ and the Retina-based Microvascular Health Assessment System (RMHAS)¹⁰ have enabled some degree of automated analysis on the retinal vessels, they often fail to segment and quantify the retinal capillaries. These capillaries are a critical component of the retinal vasculature, but their low contrast in color fundus (CF) images makes it challenging to visualize, outline, and segment them. Consequently, automated detection of retinal capillaries remains unavailable, which limits further investigation into understanding their contribution to various systemic diseases, including diabetes mellitus, coronary artery disease,¹¹ chronic kidney disease,¹² and critical illness.¹³

The gold standard for evaluating retinal vascular disease is fundus fluorescein angiography (FFA) because of its ability to provide clear and realistic information on the retinal microvasculature with enhanced contrast. Nevertheless, FFA has several drawbacks, including being time consuming, costly, and invasive because it requires intravenous fluorescein injection that may lead to severe adverse effects such as anaphylaxis. Therefore, utilizing the vasculature information from matched FFA images to label and segment retinal vessels from the CF represents a potential novel approach for quantifying retinal capillaries.

We have developed a novel framework, RMHAS-FA, to overcome the challenges of manual labeling and the failure to segment retinal capillaries. This approach utilizes matched FFA images to label the CF and generate numerous high-resolution labels on the retinal capillary to meet the data requirements of deep learning models. We further demonstrated the clinical value of the segmented capillaries by examining their correlation with CVD risk factors, incidence, and mortality.

Methods

We developed a vessel segmentation model to automatically detect retinal vasculature, including capillaries, by using pseudo-labels generated from CF-FFA pairs. The segmentation process involved registering coarse to fine vessels in CF and FFA images and using the fine vessels as pseudo-labels to train the model to segment delicate retinal vessels from the CF images. Model validation was performed on a dataset of 9068 CF-FFA pairs and 15 manually labeled CF images. External validation was conducted using 7 vessel segmentation datasets to assess the generalizability and robustness of the approach. We also evaluated the clinical significance and associations of the measurements with CVD risk prediction using 10-year follow-up data from the UK Biobank. Figure 1 provides a graphical summary of the pipeline, and Fig S1 shows the study flowchart.

Figure 1.

Workflow of the study. AVR = arteriolar-to-venular ratio; CF = color fundus; CVD = cardiovascular disease; FFA = fundus fluorescein angiography; RMHAS = Retina-based Microvascular Health Assessment System; VSD = vessel skeleton density.

Data

Model Development

CF-FFA Pairs

Color fundus with corresponding FFA pairs was retrospectively collected from clinical service; the CF and FFA images were taken at the same visit. Fundus fluorescein angiography images were captured using Zeiss FF450 Plus (Carl Zeiss, Inc) and Heidelberg Spectralis with resolutions of 512 × 512 and 768 × 768, respectively. Color fundus images were captured using Topcon TRC-50XF, Zeiss Visucam 500, Zeiss FF450 Plus (Carl Zeiss, Inc), and Nikon D7100 (Nikon), with pixel resolutions ranging from 1110 × 1467 to 2600 × 3200. One CF accompanies multiple FFA images, and each CF was registered with multiple FFA images into multiple CF-FFA pairs. The CF-FFA pairs were randomly split into training and testing sets at a ratio of 9 to 1 at the patient level to ensure no patient crossover between the training and testing sets.

FundusCapi

To provide ground truth labels to assess the accuracy of retinal vessel segmentation, a subset of 15 images from the test set of CF-FFA were manually labeled by a trained retinal specialist with the assistance of FFA and CF images. The dataset included 5 images of diabetic retinopathy, 5 images with unremarkable retinal diseases, and 5 images with other retinal abnormalities (retinal vein occlusion, age-related macular degeneration, macular telangiectasia, optic disc edema, and high myopia). Fig S2 shows a demonstration of the labeling procedure.

External Testing

Segmentation Datasets

To test the generalizability and robustness of the segmentation model, 7 publicly available datasets were used for external validation: STARE,¹⁴ CHASEDB,¹⁵ DR HAGIS,¹⁶ HRF,¹⁷ IOSTAR,¹⁸ LES-AV,¹⁹ and DRIVE.²⁰ Detailed descriptions of each dataset can be found in Table 1.

Table 1.

Model Segmentation Performance on Internal Validation∗, Manual Capillary Labeling∗∗ and External Test Sets

Dataset	N	AUC	Accuracy	Sensitivity	Specificity	Camera	Disease	Country
CF-FFA∗	9068	0.94	0.89	0.90	0.89	Various	Various	China
FundusCapi∗∗	15	0.95	0.94	0.90	0.93	TRC-50XF (Topcon)	Various	China
CHASEDB	28	0.96	0.89	0.92	0.89	NM-200D (Nidek, Japan)	Various	UK
DRHAGIS	40	0.96	0.90	0.93	0.90	TRC-NW6s (Topcon), TRC-NW8 (Topcon),or CR-DGi (Canon)	DR, HBP, AMD, glaucoma	UK
HRF	45	0.95	0.88	0.92	0.88	CR-1 (Canon)	DR, glaucoma	-
IOSTAR	30	0.95	0.91	0.87	0.91	SLO (i-Optics Inc., the Netherlands)		Netherlands and China
LES-AV	22	0.97	0.92	0.94	0.92		glaucoma
DRIVE	20	0.97	0.94	0.89	0.94	CR5 non-mydriatic 3CCD camera (Canon)	DR	Netherlands
STARE	20	0.95	0.91	0.91	0.91	TRV-50 fundus camera (Topcon)	Various	USA

AMD = age-related macular degeneration; AUC = area under the receiver operating characteristic curve; CF = color fundus; DR = diabetic retinopathy; FFA = fundus fluorescein angiography; HBP = high blood pressure; SLO = scanning laser ophthalmoscopy; UK = United Kingdom, USA = United States of America.

UK Biobank

To explore the utility of the segmented capillaries, we used UK Biobank, which is a large-scale population-based cohort study that was established in the United Kingdom to facilitate research into various risk factors for diseases that occur in middle and old age. Between 2006 and 2010, over 500 000 individuals aged between 40 and 69 years were recruited from across the United Kingdom, of whom > 60 000 had undergone retinal photography (3D OCT-1000 Mark II). Details about the cohort are available elsewhere.²¹ For the analysis, we used fundus images from the right eye as both eyes provide a consistent window for assessing vascular changes related to CVD.

The primary outcome was CVD events, which were defined as a composite of cardiovascular mortality, nonfatal myocardial infarction, and nonfatal stroke according to the guidelines of the American College of Cardiology/American Heart Association (fatal events [International Classification of Diseases 10th Revision {ICD-10} codes I20-25, I60-64] and nonfatal events [ICD-10 codes I21, I22, I60-64]).²² These events were identified through linkages to death certificate records and hospital episode statistics. The secondary outcome measure was CVD mortality, which was defined as fatal CVD events based on the clinical guidelines of the European Systematic Coronary Risk Evaluation (ICD-10 codes, I10-15, I44-1, I20-25, and I61-73).²³

Ethical Approval

All patients were anonymized and deidentified, and the study followed the principles of the Declaration of Helsinki. The Institutional Review Board of Zhongshan Ophthalmic Center approved the study (No. 2021KYPJ164-3).

Cross-modality Image Registration

Vessel Detection

Retinal vessels in CF images were extracted using a U-net-based retinal artery/vein/optic disc segmentation and measurement system called RMHAS.²⁴ The extracted vessels were used for registration purposes.

Vessels in FFA images were extracted using a patched deep learning model developed by Ding et al²⁵ which was trained using human-in-the-loop labeling. The FFA vessels extracted were used for geometric registration with CF vessel and as pseudo labels for model training.

Coarse Registration

The registration pipeline follows a previous study.²⁶ In summary, to account for the large-scale differences between vessel frames, we used a rigid registration based on keypoint matches, as shown in Figure 1. We applied the AKAZE keypoint detector²⁷ to detect key points by comparing the gradients of small image patches rather than the points themselves. The nearest-neighbor-distance ratio was used for feature matching, and random sample consensuswas used to generate homography matrices and reject outliers. We added a validity restriction to exclude erroneously registered pairs. The rotation scale was restricted to 0.8 to 1.3, and the absolute value of the rotation radian was < 4 before the warping transformation. Additionally, pairs with poor registration performance (e.g., Dice coefficients < 0.5) were filtered out. The threshold value was empirically set based on the dataset used in our experiments.

Fine-registration

To optimize the small misalignment between registered vessels, we used nonrigid registration to conduct pixel-wise alignment on the vessel segmentation maps, as shown in Figure 1. We measured similarity using normalized cross-correlation and used B-spline curves to define a continuous deformation field that maps each pixel in the FFA vessel to a corresponding pixel within the CF vessel. The optimization of the deformation was determined using the gradient-based L-BFGS-B algorithm.²⁸

Deep Learning Microvascular Segmentation

We employed the pix2pixHD²⁹ architecture, which is a generative adversarial network, for retinal vessel segmentation by the following minimax game: the generator G aims to produce a realistic retinal vessel map to deceive the discriminator D, whereas the discriminator D aims to differentiate the generated vessels from the real labels. During training, the CF was cropped to the field of view and fed into the model with a batch size of 1 and a learning rate of 0.0002. To prevent overfitting and increase the ability to detect small vessels, we augmented the images by randomly flipping them horizontally and vertically and resizing them to multiple resolutions (32 × i, 32 × j), where i and j ranged randomly from 14 to 29. We trained the model for a total of 100 epochs. We selected the model with the highest Dice index in the validation set to avoid overfitting.

Segmentation Performance Evaluation

The segmentation results were evaluated both qualitatively and quantitatively by comparing them with ground truth. The accuracy of the segmentation was assessed at the pixel level using metrics of the area under the receiver operating characteristic curve, accuracy, sensitivity, and specificity.

Vessel Measurement

The resulting segmentation maps were binarized, and watershed and filled under the guidance of the RMHAS segmentation module to obtain the arteries and veins. The vessel caliber measure is the same as that described in RMHAS.²⁴

Binary images of the arteries and veins were reduced to skeleton images with a width of 1 pixel. Skeletonized images were then used to calculate the vessel skeleton density (VSD) as the absolute vessel length in the image scan (in pixels).

We quantified the vessels in the entire retina and performed subgroup analysis by dividing the fundus image into 2 regions: 1 disc diameter from the fovea and other regions.

Statistical Analysis

We calculated the follow-up time as person-years for each participant in the UK Biobank from the date of the baseline CF to the first occurrence of CVD, death, loss to follow-up, or February 28th, 2021, whichever came first. Only Caucasian participants were included in the study. At the time of analysis, each participant had been followed up for over 10 years from the date of the baseline visit. Descriptive statistics were reported as mean (standard deviation [SD]) or median (interquartile range) for normally or non-normally distributed continuous variables, respectively, and as percentages for categorical variables. To compare groups, we used either a parametric 1-way analysis of variance test or a nonparametric Kruskal‒Wallis test.

We removed sex-specific extreme outliers in the analysis of retinal measurements by adjusting the upper and lower bounds of a traditional box and whisker plot and accounting for skewness using the Robustbase package in R (range = 3).³⁰ To investigate the clinical relevance of retinal measurements of different sizes, multiple retinal measurements from retinal segments were grouped into “large,” “medium,” or “small” based on the tertiles of retinal caliber (stratified by artery/vein and gender) and summarized as a mean value for each image. Descriptive statistics were stratified by sex and CVD risk level to explore sex differences and trends in retinal measurements. Cardiovascular diseases risk stratification followed the American Heart Association's Pooled Cohort Equation,²² with risk levels defined as < 5% (low risk), 5% to < 7.5% (borderline risk), ≥ 7.5% to < 20% (intermediate risk), and ≥ 20% (high risk). We examined P values for trends by fitting a linear model for the risk categories and retinal measurements. Measurements were then inverse-rank normalized to a mean of 0 and an SD of 1 for regression analysis.

We conducted linear regression models for continuous variables and logistic regression for binary variables to examine the associations between retinal measurements (independent variables) and CVD risk factors (dependent variables), including age, sex, systolic blood pressure, diastolic blood pressure, body mass index, total cholesterol, glycosylated hemoglobin, diabetes mellitus, and smoking. The models were unadjusted, as well as age- and sex-adjusted.

To investigate the associations between retinal vessel measurements and incident CVD and CVD mortality, we used sex-specific Cox proportional hazards models because the strength of the association differs by sex.^31,32 The analyses included unadjusted, age-adjusted, and risk factor-adjusted (age, systolic blood pressure, diastolic blood pressure, body mass index, smoking, total cholesterol, and diabetes mellitus) models. We assessed the proportional hazard assumption using Schoenfeld residuals, and it was satisfied for each model.

The retinal models were developed using PyTorch and trained on 1 NVIDIA GeForce RTX 3090 card. Data were analyzed using R version 4.0.1, and statistical significance was defined as a false discovery rate of < 0.05.

Results

The development of the RMHAS-FA cross-modality model involved 3893 participants with 90 732 CF-FFA pairs (9403 CF images; 56 580 FFA venous phase images), of which 81 664 pairs were used for model development and 9068 pairs for testing. Using CF vessels as references, the mean of the registered vessels' specificity was 0.997 (SD: 0.004), indicating the vessel trunks were matched well, visualization of the registered vessels is presented in Figure 2. The model achieved area under the curve, accuracy, sensitivity, and specificity scores of 0.94, 0.89, 0.90, and 0.89, respectively, on pixel accuracy in the internal test set. When evaluated with manual capillary labeling, the performance was 0.95, 0.94, 0.90, and 0.93, respectively, as shown in Table 1.

Figure 2.

Overlay of registered vessels extracted from color fundus (CF) photography and fundus fluorescein angiography (FFA). First column: CF image, second column: FFA image, third column: CF vessels in green, FFA vessels in red, intersection of CF and FFA vessels in yellow. The vessel trunks overlay well, the FFA image clearly identifies more capillaries than CF, and the registered FFA vessels will be used as labels to train a deep learning model to segment retinal vessels, including capillaries from CF.

When tested on 7 public datasets including different cameras and diseases, the segmentation model achieved notably high sensitivity in detecting retinal vessels compared to previous studies reported in the literature (see Table 1 and Fig S3 for quantitative and qualitative results).³³ Figure 3 presents a side-by-side comparison of CF, FFA, pseudolabels extracted from FFA, manually labeled FundusCapi, and the predictions made by the RMHAS-FA model.

Figure 3.

Demonstration of color fundus photograph, fundus fluorescein angiography (FFA), pseudolabels extracted from FFA, manual segmentation from the FundusCapi dataset, and Retina-based Microvascular Health Assessment System with fluorescein angiography model prediction (based on color fundus).

The external validation of the UK Biobank dataset included 49 229 participants; the median age was 58 years (interquartile range = 50–63), among whom 21863 (44%) were men. The baseline characteristics of the study participants are presented in Table 2. During a median follow-up of 10.91 years (interquartile range = 10.81–11.05), there were 2111 (4.3%) CVD events and 268 (0.5%) CVD mortality cases. Men had smaller arteriolar-to-venular ratio (AVR) (which was more pronounced in large vessels) and smaller VSD in arteries but larger VSD in veins compared to women (Table S1). Retinal measurements stratified by Pooled cohort equation risk levels showed clear trends (Table 3); the AVR and VSD decreased with increasing CVD risk levels in both arteries and veins (P value for trend < 0.001).

Table 2.

Baseline Demographics of the UK Biobank Participants Included in our Analysis

Characteristics	Total (n = 49 229)	Female (n = 27366)	Male (n = 21863)	P∗
Age, median (IQR)	58 (50–63)	57 (50–63)	58 (50–63)	< 0.001
Total cholesterol, mean ± SD, mmol/L	5.7 ± 1.1	5.9 ± 1.1	5.6 ± 1.1	< 0.001
HDL cholesterol, mean ± SD, mmol/L	1.5 ± 0.4	1.6 ± 0.4	1.3 ± 0.3	< 0.001
Body mass index, mean ± SD, kg/m2	27.2 ± 4.7	26.8 ± 5.1	27.7 ± 4.1	< 0.001
Diastolic blood pressure, mean ± SD, mm Hg	81.7 ± 10	80.1 ± 9.9	83.6 ± 9.7	< 0.001
Systolic blood pressure, mean ± SD, mm Hg	136.8 ± 18.3	134.4 ± 19	139.8 ± 16.9	< 0.001
HbA1c, mean ± SD, mmol/mol	35.5 ± 6	35.4 ± 5.5	35.7 ± 6.5	< 0.001
Diabetes mellitus, n (%)				< 0.001
No	47033 (95.5)	26479 (96.8)	20 554 (94)
Yes	2196 (4.5)	887 (3.2)	1309 (6)
Current smoker, n (%)				< 0.001
No	44 527 (90.7)	25 205 (92.4)	19 333 (88.7)
Yes	4545 (9.3)	2079 (7.6)	2466 (11.3)

HbA1c = glycosylated hemoglobin; HDL = high density lipoprotein; IQR = interquartile range; SD = standard deviation.

^∗P value for parametric 1-way analysis of variance test or the nonparametric Kruskal‒Wallis test.

Table 3.

Measurement Characteristics Stratified by 4 Cardiovascular Risk Groups According to the Pooled Cohort Equation (< 5% [low Risk]; 5% to < 7.5% [Borderline Risk]; ≥ 7.5% to <20% [Intermediate Risk]; and ≥ 20% [High Risk])

Measurement	Total (n = 49 229)	Low (n = 20 365)	Borderline (n = 6091)	Intermediate (n = 13 612)	High (n = 2852)	P for Trend
Artery
AVR (large), mean ± VSD	0.88 ± 0.04	0.89 ± 0.04	0.88 ± 0.04	0.88 ± 0.04	0.87 ± 0.05	< 0.001
AVR (medium), mean ± VSD	0.95 ± 0.02	0.96 ± 0.02	0.95 ± 0.02	0.95 ± 0.02	0.95 ± 0.02	< 0.001
AVR (small), mean ± SD	0.97 ± 0.02	0.97 ± 0.02	0.97 ± 0.02	0.97 ± 0.02	0.97 ± 0.02	< 0.001
VSD (large), median (IQR)	1309.14 (1091–1507.26)	1328.3 (1125.57–1521.93)	1302.37 (1081.57–1503.33)	1288.48 (1061.47–1493.98)	1262.66 (1022.87–1475.94)	< 0.001
VSD (medium), median (IQR)	1225.07 (989.36–1453.19)	1269.52 (1048.29–1488.2)	1207.38 (965.79–1438.03)	1178.73 (941.8–1414.27)	1126.79 (867.41–1367.37)	< 0.001
VSD (small), median (IQR)	1033.79 (579.52–1428.98)	1195.97 (730.71–1538.06)	1005.5 (556.74–1394.04)	881.7 (482.68–1294.94)	723.4 (388.87–1147.47)	< 0.001
Vein
AVR (large), mean ± SD	0.88 ± 0.04	0.89 ± 0.04	0.88 ± 0.04	0.88 ± 0.04	0.87 ± 0.05	< 0.001
AVR (medium), mean ± VSD	0.95 ± 0.02	0.96 ± 0.02	0.95 ± 0.02	0.95 ± 0.02	0.95 ± 0.02	< 0.001
AVR (small), mean ± SD	0.97 ± 0.02	0.97 ± 0.02	0.97 ± 0.02	0.97 ± 0.02	0.97 ± 0.02	< 0.001
VSD (large), median (IQR)	1398.54 (1218.09–1572.3)	1382.42 (1216.47–1544.02)	1396.83 (1208.81–1575.79)	1421.64 (1224.57–1599.98)	1429.42 (1215.38–1627.21)	< 0.001
VSD (medium), median (IQR)	1393.39 (1103.81–1634.06)	1427.75 (1165.02–1652.04)	1375.23 (1063.09–1623.85)	1361.48 (1059.88–1617.05)	1323.22 (1018.74–1594.37)	< 0.001
VSD (small), median (IQR)	1112.11 (662.04–1504.93)	1243.33 (789.98–1587.74)	1080.47 (637.27–1463.44)	979.68 (572.7–1407.3)	866.28 (497.96–1288.75)	< 0.001

AVR = arteriolar-to-venular diameter ratio; IQR = interquartile range; SD = standard deviation; VSD = vessel skeleton density.

P value for trend was examined by fitting a linear model for the risk categories and retinal measurements.

Arterial VSD was significantly associated with body mass index, systolic blood pressure, diastolic blood pressure, and glycosylated hemoglobin, whereas small and large vessels had different association effect sizes. The AVR from large vessels had a stronger negative association with blood pressure, blood lipids, and smoking status than that from small vessels. The VSD from small vessels had a stronger negative association with glycosylated hemoglobin and diabetes status than that from large vessels for both arteries and veins. The VSD in the macular region (1 disc diameter from the fovea) was more strongly associated with systemic biomarkers than that in the other regions (see Figs S4–S9).

In the Cox regression analyses stratified by sex (Figure 4), each SD decrease in the arterial VSD (small) was associated with an increased risk of CVD and CVD mortality for women. The hazard ratios (HRs) and 95% confidence intervals (CIs) for crude, age-adjusted, and risk-adjusted models were 0.74 (95% CI 0.69–0.79), 0.89 (95% CI 0.83–0.96), 0.91 (95% CI 0.84–0.98) for CVD risk, and 0.53 (95% CI 0.43–0.65), 0.66 (95% CI 0.53–0.82), 0.68 (95% CI 0.54–0.86), for CVD mortality. For men, a decrease in arterial VSD (small) per SD was associated with an increased risk of CVD in the single-variable model (HR of 0.8 [95% CI 0.76–0.85]). However, after adjusting for age and risk factors, the effect was insignificant, whereas the association with CVD mortality remained significant after multiple adjustments (HR for crude, age-adjusted, and risk-adjusted models of 0.67 [95% CI 0.58–0.78], 0.83 [95% CI 0.71–0.98], and 0.83 [95% CI 0.70–0.98], respectively). The analyses of vessel measurements in different retina regions (Figure 5) showed that VSD from the macular region was more significantly associated with incident CVD and CVD mortality for both sexes than VSD from other regions.

Figure 4.

Association of retinal vessel measurements with incident cardiovascular disease (CVD) in the UK Biobank study. A, Female participants; B, Male participants. Pink denotes artery, blue denotes vein. Model 1 unadjusted, Model 2 adjusted for age, Model 3 adjusted for age, systolic blood pressure, diastolic blood pressure, body mass index, smoking status, blood cholesterol, and diabetes. AVR = artery to vein ratio; CI = confidence interval; HR = hazard ratio; VSD = vessel skeleton density (in pixel unit).

Figure 5.

Association of retinal vessel measurements in different retinal region (macular region and other region) with incident cardiovascular disease (CVD) in the UK Biobank study. A, Female participants; B, Male participants. Pink denotes artery, blue denotes vein. Model 1 unadjusted, Model 2 adjusted for age, Model 3 adjusted for age, systolic blood pressure, diastolic blood pressure, body mass index, smoking status, blood cholesterol and diabetes. CI = confidence interval; DD = 1 disc diameter from the fovea; HR = hazard ratio; VSD = vessel skeleton density.

Discussion

In this study, we utilized vasculature data from FFA to automatically label the CF for small retinal vessel segmentation. We assessed the performance of the algorithm on multiple external datasets and further investigated the contribution of retinal vessels of various sizes on incident CVD and CVD mortality using the UK Biobank data.

Previous studies on the automated segmentation of retinal vessels based on CF were developed solely based on manual labeling of the vessels, which can be labor-intensive, time-consuming, and subject to significant interobserver variations. Automated segmentation of retinal capillaries is even more difficult because they are often small and elusive in CF. Inspired by previous investigation of generating realistic FFA images from CF,³⁴ we used more than 90 000 CF-FFA pairwise images to develop a method that allows us to use FFA images to automatically generate retinal vessel labels for CF images. Then, we developed the RMHAS-FA algorithm that can detect retinal vessels, including capillaries on the CF without requiring FFA. We proved that this automated segmentation algorithm was accurate, and robust to various fundus cameras, even among the images with multiple established retinal diseases. The system can automatically complete the retinal vascular analysis on a lot of images, and it is faster (taking < 1 second in general) than the existing tools. We assessed the accuracy of automated vessel segmentation based on our manually labeled capillary dataset and 7 public datasets. Instead of being trained and tested on the same public dataset in the development of other algorithms, our algorithm was assessed on multiple external validation sets to prove the generalizability among various fundus cameras and images with various retinal diseases.

Our study further demonstrated that, in the UK Biobank analysis, retinal capillary density was closely associated with systemic diseases compared to large and medium retinal vessels, whereas AVR was the opposite. For example, AVR from larger vessels was more significantly associated with blood pressure and CVD events, whereas skeleton density from retinal capillaries was more associated with diabetes and CVD events. This is in agreement with previous studies where using optical coherence tomography angiography, Rakusiewicz et al³⁵ reported significantly decreased vessel density in the superficial capillary plexus in children with chronic heart failure. Wang et al found that decreased vessel density and blood flow were associated with coronary artery and branch stenosis. Vessel density was also associated with CVD, CVD risk factors,^11,36,37 and acute coronary syndrome.³⁸

Finally, assessing the microvasculature changes through retinal photography might be the most cost-effective, noninvasive, and convenient way to assess CVD risk. Because the coronary microvasculature changes have to be visualized by invasive angiogram technology, numerous studies have attempted to develop less invasive imaging biomarkers or techniques.³⁹ Compared to optical coherence tomography angiography and FFA, our technology may provide a new way to visualize systemic microvascular changes as an easy-to-perform, cheaper, and quick screening tool for CVD risk stratification in general practice settings.

Several limitations should be acknowledged. First, while we reckon that there is currently no gold standard for retinal capillary segmentation, we, therefore, have manually labeled a capillary dataset (FundusCapi) for validation purposes. Second, despite having reported the clinical utility of the algorithm for retinal vessel segmentation in the context of its association with CVD risk factors and events, we believe other potentials for predicting other eye diseases or systemic conditions should be further explored. Third, our study was conducted on a cohort of primarily Caucasian individuals; further studies are needed to explore the associations between retinal capillaries and CVD or systemic diseases in other ethnic groups.

Conclusion

The utilization of cross-modality labeling enables a more accurate segmentation of retinal vessels including the retinal capillary from CF that is not possible before. The study demonstrated that retinal capillaries can be accurately segmented from CF. This quantification of capillary will become specific biomarkers that potentially can be used in the prevention and management of CVD.

Manuscript no. XOPS-D-23-00159.