Abstract
Purpose
Investigate the association between the optical coherence tomography angiography (OCTA) metrics derived from different analysis programs to understand the comparability of studies using these different approaches.
Methods
Secondary analysis of a prospective observational study (March 2018–September 2021). Forty-four right eyes and 42 left eyes from 44 patients were included. Patients were either undergoing upper gastrointestinal surgery with a critical care stay planned or were already in the critical care unit with sepsis. OCTA scans were obtained in an ophthalmology department or critical care setting. Fourteen OCTA metrics were compared within and between the programs, and agreement was measured by Pearson's R coefficient and intraclass correlation coefficient.
Results
Correlation was highest between all Heidelberg metrics and Fractalyse (all >0.84), and lowest between Matlab skeletonized or foveal avascular zone metrics and all other measures (e.g., skeletal fractal dimension and vessel density at −0.02). Agreement between eyes was moderate to excellent in all metrics (0.60–0.90).
Conclusions
The significant variability between metrics and programs used for OCTA analysis demonstrates that they are not interchangeable and supports a recommendation for perfusion density metrics to be reported as standard.
Translational Relevance
Agreement between different OCTA analyses is variable and not interchangeable. The high agreement between non-skeletonized vessel density metrics suggests that these should be routinely reported.
Keywords: OCTA, retinal blood flow, image analysis
Introduction
Optical coherence tomography (OCT) allows noninvasive visualization and measurement of retinal structure, generating high-resolution cross-sectional retinal images.1 OCT angiography (OCTA) creates three-dimensional images of the choroidal and retinal blood flow by using moving red blood cells as an intrinsic contrast medium.2 OCTA is used extensively in ophthalmic clinical practice and research,3 requiring no injection of contrast or dye—such as fluorescein—and so is faster and cheaper and poses no risk of adverse reactions compared to traditional angiography.4 OCTA algorithms use the component differences of the varying B-scans,5 with different devices using different algorithms to detect blood flow and segment retinal layers and capillary boundaries.6 Furthermore, many commercial devices do not include automated calculation of blood flow characteristics,7 necessitating the use of third-party software to quantify blood flow in many studies, causing additional variation between studies.8 Methods of quantification include measuring foveal avascular zone (FAZ) area and perimeter7 and using binarized and skeletonized images to calculate perfusion density, vessel length density, and fractal dimension.8 Differing methods to threshold or binarize scans also add to consistency and reproducibility problems.9,10
Although studies have looked at agreement in measures of macular perfusion using OCTA between devices11 and at methods to calculate macular vessel density,12 the extent of agreement between different OCTA analysis methods and metrics is not previously reported, impairing comparison of OCTA studies using different measures. We aimed to investigate the correlation and agreement of analysis methods derived from four different approaches using the following third-party and manufacturer's software applied to Heidelberg Spectralis OCTA scans: (1) MatLab (M; MATLAB R2021a; MathWorks, Natick, MA, USA)13–16; (2) Heidelberg SP-X1902 (H; Heidelberg Engineering, Heidelberg, Germany); (3) ImageJ Fiji (IJ; National Institutes of Health, Bethesda MD, USA)17–20; and (4) Fractalyse (F; ThéMA, Besançon, France)21–25, to determine their comparability and preferred methods to use going forward.
Methods
Study Design and Setting
We performed a secondary analysis of a prospective observational study of OCTA in a critical care setting. OCTA scans analyzed were from patients recruited to the Ophthalmic and Neurocognitive Assessment in the Management of Critically Ill Patients study: 19/YH/0113; and Defining Outcome Measures in Ocular Inflammatory Disease: 14/EM/1163, both approved by the NHS Research Ethics Service and conducted between March 2018 and September 2021 in the Ophthalmology Department and Critical Care Unit (CCU) at the Queen Elizabeth Hospital Birmingham of University Hospitals Birmingham NHS Foundation Trust (UK). Inclusion criteria were patients over the age of 18 years with either planned esophagectomy or septic patients already admitted to the CCU. Exclusion criteria were individuals with pre-existing retinal pathology, optic nerve pathology, or known neurological conditions; this was assessed using patient history and the review of case notes. In the planned esophagectomy cohort, written informed consent was obtained from each individual before preoperative imaging. In the septic cohort, eligibility was confirmed, and nominated consent was sought by the participant's CCU consultant, with written informed consent obtained from the patient after recovery. The planned esophagectomy patient cohort was included because postoperative care is routinely performed on the CCU, and the surgery is elective, allowing researchers to identify and recruit eligible participants and perform baseline imaging before their CCU stay.26 The study was conducted in accordance with the Declaration of Helsinki.
Acquisition Devices
Scans were performed using the SPECTRALIS Heidelberg HRA+OCT flex or tabletop modules (Heidelberg Engineering).
Scanning Protocol
A total of 44 right eyes (oculus dexter [OD]) and 42 left eyes (oculus sinister [OS]) were included from 34 males and 11 females. OD was always scanned first followed by OS; therefore the difference in the number of eyes included is due to inability to scan the OS in two patients from either patient fatigue or not attempting to image because of time constraints imposed by other medical interventions. The average quality of the scans included was 35db, with no scans included having a quality below 26db.
Macula OCTA (512 B-scans over an area of 2.8 mm2, with an automatic real time setting of 5 A-scans averaged) was performed in at least one eye of 44 participants, with a total of 86 OCTA scans included. Pupil dilatation using tropicamide 0.5% to 1.0% eye drops solution (Minims; Bausch & Lomb, Surrey, UK) was instilled in the septic patients (seven participants).27
OCTA Analysis
The superficial vascular plexus (SVP) and intermediate capillary plexus (ICP) layers of the OCTA scans were automatically segmented by the manufacturer's software, with accuracy manually verified before exporting en face tif images of the OCTA scans in each plexus for ImageJ, MatLab, and Fractalyse analyses and exporting as E2E files for SP-X1902 analysis. Metric definitions are detailed in Table 1.
Table 1.
Metric Descriptions
| Heidelberg SP-X1902 | Mean | Expected probability of OCTA signal for a pixel in the vasculature slab |
| Skeleton | Estimation of the center line of a blood vessel – equivalent to vessel length density | |
| Sum | Amount of OCTA signal within a vasculature slab—equivalent to vessel density | |
| Prob | Probability that OCTA signal exists in the vasculature slab | |
| Binary | Binary classification determined by whether the A-scan in the vasculature slab includes a vessel | |
| Ves en | Binarized classification, preserving small vessels | |
| MatLab | FD | Mathematical measure describing the complexity of a biological structure |
| SFD | Measure of FD when the vessels are represented as a 1-pixel wide central line of the vessel | |
| VD | Total perfused vasculature in area of binarized measurement | |
| SVD | Measure of VD when vessels are represented as a 1-pixel wide central line (vessel length density) | |
| ImageJ | FAZ area | Area of foveal avascular zone (mm2) |
| FAZ perimeter | Perimeter of foveal avascular zone (mm) | |
| VD | Total perfused vasculature in area of measurement | |
| Fractalyse | FD | Complexity of vascular branching on binarized images. |
Units in brackets. Arbitrary units where no units are given. OCTA, optical coherence tomography angiography; Ves en, vessel enhanced; FD, fractal dimension; SFD; skeletal fractal dimension; VD, vessel density; SVD; skeletal vessel density; FAZ, foveal avascular zone.
MatLab
To analyze the OCTA scans in MatLab, we used the approach and custom code reported by Zahid et al.,14 assessing vascular morphology by measuring fractal dimension (FD) and skeletal fractal dimension (SFD), and vessel density assessed by vessel density (VD) and skeletal VD (SVD), all derived from binarized images and expressed as arbitrary units, as previously reported.13,14
Heidelberg SP-X1902
Raw OCTA scans were imported into the Heidelberg SP-X1902 software, which analyses all vascular layers. Average perfusion within a macula 3 × 3 grid was exported and summed to generate a final value comparable to the MatLab, Fiji ImageJ, and Fractalyse analyses for use in the reliability analysis. Metrics exported were mean, skeleton, sum, prob, binary, and vessel enhanced (Table 1).
ImageJ Fiji
OCTA tif image files of the SVP and ICP were processed to 8-bit and then binarized, with the threshold adjusted using the Otsu method21,23,28,29 set to intensity 135 for SVP images and 110 for ICP images and then inverted (Figure). Binarized image intensity was measured, with VD calculated as vessel pixels divided by sum of pixels in the region of interest.
Figure.
OCTA scan of the OD SVP. (A) Raw OCTA. (B) Inverted image of A to calculate VD in IJ. (C) Example of the skeletonized image of B to calculate SFD/SVD.
FAZ metrics were measured by opening the tif OCTA scans and setting the scale to 420 pixels/mm. The “polygon” setting was used to draw round the boundary of the FAZ to calculate area and perimeter.
Fractalyse
The binarized OCTA images from ImageJ were uploaded to Fractalyse. FD was calculated for each image by the box counting method.
Statistics
All blood flow measures from the OD and OS eye were analyzed and compared for each analysis program, and then compared between analysis programs. The correlation of the different analyses for blood flow measures within and between programs was estimated using Pearson's r coefficient (r), as the different scales between the different metrics analyzed prevent intraclass correlation.30 The correlation of each metric between eyes was measured by intraclass correlation coefficient (ICC). Pearson's r values were calculated as a two-tailed test of significance, with ICC calculated using ICC1 values based on the Shrout and Fleiss classification,31 assuming two-way mixed effects, consistency, and single rater/measurement models in IBM SPSS Statistics (IBM Corp, Released 2020. IBM SPSS Statistics for Macintosh, Version 27; IBM Corp, Armonk, NY, USA). ICC for consistency was chosen as fellow eyes are not expected to be identical. Results were summarized as r and ICC values with 95% confidence intervals, with values less than 0.5, between 0.5 and 0.74, between 0.75 and 0.9, and greater than 0.90 are taken to be indicative of poor, moderate, good, and excellent reliability, respectively.32
Results
Correlation Between MatLab, Heidelberg, ImageJ, and Fractalyse Measures
To examine the extent to which perfusion measures from different software packages are interchangeable, we compared outputs using Pearson's r. There was good to excellent correlation between VD measures (meanH, VDM and VDIJ; 0.78–0.96), good to excellent correlation between skeletonH and the non-skeletonized VD measures (0.76–0.96), moderate to excellent correlation between VDIJ and FDM metrics (0.62–0.92), variable correlation between FDF and H/M/IJ blood flow metrics (0.43–0.98) and a poor to moderate inverse relationship between FAZ area and perimeter with other metrics, which were strongest in the SVP compared to the ICP (up to −0.53) (Table 2).
Table 2.
Agreement (r) Between MatLab (M), Heidelberg (H), ImageJ (IJ), and Fractalyse (F) Measures in the OD SVP, OD ICP, OS SVP, and OS ICP
| MeanH | SkeletonH | SumH | ProbH | BinaryH | Ves EnH | VDIJ | FDF | FAZ Area (mm2)IJ | FAZ Perimeter (mm)IJ | |
|---|---|---|---|---|---|---|---|---|---|---|
| OD SVP | ||||||||||
| FDM | 0.76 | 0.76 | 0.76 | 0.72 | 0.76 | 0.79 | 0.81 | 0.87 | −0.46 | −0.24 |
| SFDM | 0.29 | 0.33 | 0.37 | 0.28 | 0.31 | 0.35 | 0.34 | 0.40 | −0.25 | −0.26 |
| VDM | 0.78 | 0.76 | 0.76 | 0.74 | 0.77 | 0.78 | 0.83 | 0.82 | −0.37 | −0.46 |
| SVDM | 0.38 | 0.44 | 0.48 | 0.39 | 0.41 | 0.44 | 0.44 | 0.50 | −0.25 | 0.24 |
| FAZ perimIJ | −0.41 | −0.38 | −0.29 | −0.31 | −0.41 | −0.39 | −0.38 | −0.34 | 0.80 | |
| FAZ areaIJ | −0.48 | −0.49 | −0.38 | −0.34 | −0.49 | −0.50 | −0.45 | −0.45 | ||
| FDF | 0.92 | 0.91 | 0.92 | 0.87 | 0.92 | 0.94 | 0.95 | |||
| VDIJ | 0.91 | 0.90 | 0.96 | 0.88 | 0.92 | 0.92 | ||||
| OD ICP | ||||||||||
| FDM | 0.78 | 0.85 | 0.78 | 0.83 | 0.82 | 0.83 | 0.62 | 0.88 | −0.02 | 0.08 |
| SFDM | 0.35 | 0.40 | 0.35 | 0.39 | 0.41 | 0.42 | 0.22 | 0.44 | 0.02 | 0.45 |
| VDM | 0.80 | 0.80 | 0.80 | 0.81 | 0.76 | 0.76 | 0.71 | 0.77 | −0.10 | −0.30 |
| SVDM | 0.44 | 0.48 | 0.44 | 0.48 | 0.48 | 0.50 | 0.33 | 0.50 | 0.01 | 0.42 |
| FAZ perimeterIJ | −0.10 | −0.02 | −0.10 | −0.07 | −0.02 | −0.01 | −0.12 | −0.03 | 0.81 | |
| FAZ areaIJ | −0.07 | −0.05 | −0.07 | −0.08 | −0.06 | −0.06 | 0.00 | −0.04 | ||
| FDF | 0.93 | 0.90 | 0.93 | 0.92 | 0.89 | 0.90 | 0.84 | |||
| VDIJ | 0.93 | 0.78 | 0.93 | 0.86 | 0.76 | 0.75 | ||||
| OS SVP | ||||||||||
| FDM | 0.87 | 0.90 | 0.91 | 0.93 | 0.89 | 0.91 | 0.92 | 0.94 | −0.44 | −0.35 |
| SFDM | 0.50 | 0.58 | 0.59 | 0.60 | 0.52 | 0.55 | 0.58 | 0.61 | −0.14 | 0.17 |
| VDM | 0.85 | 0.85 | 0.87 | 0.89 | 0.87 | 0.87 | 0.89 | 0.88 | −0.45 | −0.53 |
| SVDM | 0.56 | 0.64 | 0.65 | 0.65 | 0.58 | 0.60 | 0.64 | 0.66 | −0.18 | 0.14 |
| FAZ perimeterIJ | −0.47 | −0.42 | −0.40 | −0.41 | −0.46 | −0.45 | −0.43 | −0.43 | 0.89 | |
| FAZ areaIJ | −0.49 | −0.47 | −0.45 | −0.47 | −0.48 | −0.48 | −0.48 | −0.48 | ||
| FDF | 0.95 | 0.96 | 0.97 | 0.98 | 0.96 | 0.97 | 0.98 | |||
| VDIJ | 0.96 | 0.96 | 0.99 | 0.98 | 0.97 | 0.97 | ||||
| OS ICP | ||||||||||
| FDM | 0.85 | 0.85 | 0.85 | 0.89 | 0.89 | 0.90 | 0.75 | 0.84 | −0.38 | −0.19 |
| SFDM | 0.42 | 0.47 | 0.42 | 0.43 | 0.48 | 0.48 | 0.40 | 0.43 | −0.11 | 0.40 |
| VDM | 0.83 | 0.79 | 0.83 | 0.85 | 0.81 | 0.82 | 0.74 | 0.74 | −0.35 | −0.45 |
| SVDM | 0.54 | 0.56 | 0.54 | 0.53 | 0.58 | 0.57 | 0.53 | 0.52 | −0.14 | 0.35 |
| FAZ perimeterIJ | −0.28 | −0.27 | −0.28 | −0.30 | −0.26 | −0.27 | −0.17 | −0.31 | 0.76 | |
| FAZ areaIJ | −0.37 | −0.40 | −0.37 | −0.41 | −0.41 | −0.43 | −0.24 | −0.46 | ||
| FDF | 0.90 | 0.84 | 0.90 | 0.91 | 0.87 | 0.87 | 0.83 | |||
| VDIJ | 0.96 | 0.86 | 0.96 | 0.92 | 0.87 | 0.86 |
Correlation Between MatLab Blood Flow Measures
There was good correlation between the FD and VD measures, with an r of 0.78 to 0.88 in both eyes and vascular layers. There was excellent agreement between SFD and SVD of 0.97 in OD SVP, ICP and OS ICP, and 0.98 in OS SVP.
Interestingly, correlation was lower and more variable when comparing the SFD and SVD measures with their non-skeletonized counterparts, ranging from −0.02 to 0.74 depending on metrics, eye, and vascular layer. The correlation was lowest between SFD and VD at 0.01 and −0.02 in the OD SVP and ICP, respectively, and 0.31 and 0.05 in the OS SVP and ICP, respectively (Table 3; Supplementary Table S1a-d).
Table 3.
Agreement (r) Between MatLab Measures in the SVP and ICP of Each Eye
| MatLab Measures | OD SVP | OD ICP | OS SVP | OS ICP |
|---|---|---|---|---|
| FD vs VD | 0.80 | 0.78 | 0.88 | 0.85 |
| FD vs SVD | 0.63 | 0.57 | 0.74 | 0.56 |
| FD vs SFD | 0.57 | 0.57 | 0.71 | 0.52 |
| SFD vs SVD | 0.97 | 0.97 | 0.98 | 0.97 |
| SFD vs VD | 0.01 | −0.02 | 0.31 | 0.05 |
| SVD vs VD | 0.13 | 0.04 | 0.38 | 0.14 |
Correlation Between Heidelberg Measures
There was excellent correlation among all metrics with little variability. There was good to excellent correlation between skeleton and mean measures; for example, in the OD/OS SVP and OD/OS ICP, r was 0.96 and 0.93, respectively. The greatest correlation was between mean and sum, and binary and vessel enhanced measures, with r reaching 1.00 in mean versus sum (OD and OS ICP), and 1.00 in both eyes and vascular layers for binary versus vessel enhanced (Table 4; Supplementary Table S1a–d).
Table 4.
Agreement (r) Between Heidelberg Measures in the SVP and ICP of Each Eye
| Measures | OD SVP | OD ICP | OS SVP | OS ICP |
|---|---|---|---|---|
| Mean vs Skeleton | 0.96 | 0.93 | 0.96 | 0.93 |
| Mean vs Sum | 0.91 | 1.00 | 0.95 | 1.00 |
| Mean vs Probability | 0.83 | 0.98 | 0.94 | 0.99 |
| Mean vs Binary | 0.97 | 0.90 | 0.97 | 0.94 |
| Mean vs Ves En | 0.97 | 0.90 | 0.97 | 0.94 |
| Skeleton vs Sum | 0.92 | 0.93 | 0.95 | 0.93 |
| Skeleton vs Prob | 0.81 | 0.97 | 0.94 | 0.93 |
| Skeleton vs Binary | 0.99 | 0.97 | 0.99 | 0.94 |
| Skeleton vs Ves En | 0.98 | 0.98 | 0.99 | 0.95 |
| Sum vs Prob | 0.88 | 0.98 | 0.97 | 0.99 |
| Sum vs Binary | 0.93 | 0.90 | 0.96 | 0.94 |
| Sum vs Ves En | 0.93 | 0.90 | 0.96 | 0.95 |
| Prob vs Binary | 0.82 | 0.93 | 0.95 | 0.96 |
| Prob vs Ves En | 0.84 | 0.94 | 0.96 | 0.96 |
| Binary vs Ves En | 1.00 | 1.00 | 1.00 | 1.00 |
Correlation Between ImageJ Measures
There was poor inverse correlation between FAZ area or perimeter and VD in either eye or vascular layer (r = −0.48 to 0.00). However, there was good correlation between FAZ area and perimeter in both eyes and vascular layers (ICC = 0.76–0.89) (Table 2; Supplementary Table S1a-d).
Correlation Between Eyes of the Same Patient
Because retinal blood flow is strongly correlated between right and left eyes in healthy individuals,33–35 we looked at inter-eye agreement. There was moderate to excellent agreement between OD and OS for each metric in both layers with the lowest in the FAZ areaIJ and SFDM (SVP) metrics (Table 5).
Table 5.
ICC Between Eyes for the Same Patient for MatLab (M), Heidelberg (H), ImageJ (IJ), and Fractalyse (F) in the SVP and ICP
| Measure | SVP | ICP |
|---|---|---|
| MeanH | 0.74 | 0.80 |
| SkeletonH | 0.78 | 0.69 |
| SumH | 0.81 | 0.80 |
| ProbabilityH | 0.81 | 0.80 |
| BinaryH | 0.80 | 0.76 |
| Ves EnH | 0.79 | 0.76 |
| FDM | 0.69 | 0.64 |
| SFDM | 0.63 | 0.71 |
| VDM | 0.90 | 0.85 |
| SVDM | 0.75 | 0.75 |
| FAZ area (mm2)IJ | 0.60 | 0.81 |
| FAZ perimeter (mm)IJ | 0.81 | 0.87 |
| FDF | 0.77 | 0.74 |
| VDIJ | 0.84 | 0.76 |
Discussion
We compared four different analysis programs commonly used for the analysis of retinal OCTA signals, exploring the correlation between metrics derived from these different approaches after application to images obtained on a single manufacturer's platform, defining for the first time the very variable correlations between different analysis methods. These findings suggest that the choice of the analysis metric may determine the agreement of results between OCTA studies and the ability to detect pathology.
We also report, for the first time, the relationship between the traditional binarized metrics, which collapse all blood flow data to either 0 or 1, and the novel metrics mean, sum and prob, that are sensitive to different levels of blood flow within the vessels at the low end of the range of detection, albeit that the OCTA signal saturates quickly meaning that at normal or higher levels of blood flow the signal is essentially binary.
We assessed correlation as a measure of agreement, because different metrics have different scales and means and so would not be expected to generate absolute agreement in mean values or magnitude of variation. However, metrics that assess the same underlying physiological property of the retina being imaged (assumed to be perfusion) should vary together across the population, and the low correlation among some metrics suggests that they are not effectively measuring the same aspects of retinal perfusion.
The varying analytic tools available to analyze OCTA limits the comparison of OCTA studies. A meta-analysis investigating FD and retinal pathology concluded that variation between studies in methods for FD calculation made it difficult to determine effects and that a standardized protocol for image acquisition and processing must be sought to allow reliable study comparison.36 Other studies have investigated methods of quantifying OCTA VD12 and compared different metrics when considering vessel drop out in diabetic retinopathy.37 Another study has developed a standardized method to analyze VD and FAZ measurements of OCTA scans because of lack of quantitative analysis reproducibility.38 An “open-source toolbox” has been developed with the goal to standardize OCTA data analysis in an attempt to mitigate the wide variety of OCTA imaging modalities and analysis methods that make OCTA interstudy comparison difficult.39
Adding to the complexity of using different imaging modalities, a study by Dave et al.40 investigated the agreement of FAZ measurements between two OCTA devices and found, although repeated measurements using the same device were reproducible (ICC of between 0.92 and 0.99), agreement between devices was poor with an ICC of 0.21. However, conversion formulas were devised to convert FAZ measurements between devices, which could be a method applied in the future when comparing results from different studies.
Our results show good internal correlation between the VD and FD MatLab metrics and between SVD and SFD, consistent with the different MatLab codes all assessing a consistent underlying feature of the OCTA image (assumed to be retinal blood flow), although skeletonization reduced correlation when compared with the non-skeletonized metrics. There was good to excellent internal correlation among many of the different Heidelberg metrics consistent with the different Heidelberg metrics all assessing consistent aspects of retinal blood flow. Because all of the metrics examined are different methods of measuring perfused VD, high levels of correlation are expected. It is notable that the Heidelberg skeleton metric had excellent agreement with other non-skeletonized Heidelberg metrics, whereas skeletonized MatLab metrics had much lower correlation, suggesting that skeletonization outputs are heavily dependent on the skeletonization algorithm and further affect comparison. The poor correlation of the MatLab skeletonized values with other metrics could be caused by loss of smaller blood vessels in the skeletonization process, preventing the detection of physiological variability in perfusion.
There was very variable agreement among the Heidelberg, MatLab, ImageJ, and Fractalyse programs. All Heidelberg metrics, VDIJ and FDF showed moderate to excellent agreement with each other and the non-skeletonized MatLab metrics but showed low agreement with the MatLab skeletonized metrics and variable agreement with the FAZ metrics. This suggests that the non-skeletonized VD metrics assessed more consistent features of the images that varied in the same way between patients and therefore more reliably assessed an aspect of retinal blood flow than MatLab skeletonized and FAZ metrics. The agreement was similar between eyes and vascular layers, indicating the reproducibility of these findings.
The good overall agreement between right and left eyes for each analysis method suggests that a high proportion of the variability in the blood flow measured from OCTA images among patients derives from real differences in perfusion and underlying variation in patient physiology, such as cardiovascular status, because retinal blood flow is strongly correlated between right and left eyes in healthy individuals.33–35 That being the case, the lower between-eye correlation for MatLab FD and SFD may suggest that these analyses do not so effectively assess underlying patient physiology or retinal perfusion.
Consistent with increasing FAZ size reflecting lower retinal blood flow, FAZ area and perimeter were both negatively correlated with other blood flow metrics, although agreement was lower. Interestingly, SVP FAZ associated more with other blood flow metrics than ICP FAZ. Previous studies vary regarding which vascular layer is best for measuring FAZ, showing more reliable quantification in the SVP than deep plexus,18 whereas another study suggests that human FAZ measurements are most consistent in the ICP.41 We have previously shown higher FAZ repeatability (ICC) in the ICP,13 and in our data here, there was greater inter-eye agreement in ICP FAZ measures. Our data therefore suggest that changes in SVP FAZ may better reflect variation in retinal blood flow and have better agreement with other blood flow metrics than ICP, whereas ICP FAZ may be more repeatable and more symmetric between eyes, although possibly reflecting retinal anatomy rather than blood flow variation.
The main limitation to this study is that all metrics investigated were from OCTA scans obtained by the Heidelberg Spectralis device, limiting extrapolation of our results to comparison of OCTA metrics derived from difference imaging platforms. Because manufacturers use different algorithms to calculate segmentation, flow detection has varying automated vessel perfusion calculations, so it is difficult to compare values between different OCTA equipment used. Other studies have looked at the performance of different OCTA acquisition devices, finding poor agreement of measures between systems.6,42,43 An additional limitation is that our cohort of patients varied in systemic perfusion but did not have focal abnormalities as might be seen in diabetic retinopathy or age-related macular degeneration, limiting application where localized variation is sought. We did not attempt to define sensitivity or specificity for the detection of specific diseases, concentrating instead on agreement, but it is also possible that the presence of different retinal pathologies may affect agreement between metrics. FAZ metrics may vary with ocular anatomy, such as axial length, which we did not measure. They therefore do not necessarily follow other perfusion metrics but are commonly used to examine retinal perfusion, and we therefore included them for comparison.
To our knowledge, this is the first study to investigate the correlation between different retinal blood flow analysis methods by different programs. The good agreement of VD metrics between software platforms and good correlation between eyes in the same patient suggests that these metrics are likely to be the most reproducible across different software packages and the consistency of VD measures between algorithms has been previously reported.12 We therefore suggest that VD metrics should be reported in OCTA studies and potentially be preferred to skeletonized metrics in the absence of software standardization.
Conclusion
Agreement among different OCTA analysis programs is highly variable and is important to consider when reporting and comparing results between OCTA studies. The high agreement of non-skeletonized VD metrics suggests that these should routinely be reported.
Supplementary Material
Acknowledgments
Supported by the National Institute for Health Research (NIHR) Surgical Reconstruction and Microbiology Research Centre (SRMRC). TV's research programme was supported by Medical Research Council. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
Heidelberg Engineering provided the Heidelberg SPECTRALIS flex module for this study, but were not involved in data collection, analysis, or manuscript preparation.
Disclosure: E.F. Courtie, None; A. Gilani, None; N. Capewell, None; A.U. Kale, None; B.T.K. Hui, None; X. Liu, None; G. Montesano, None; M. Teussink, None; A.K. Denniston, None; T. Veenith, None; R.J. Blanch, None
References
- 1. Adhi M, Duker JS.. Optical coherence tomography—current and future applications. Curr Opin Ophthalmol. 2013; 24: 213–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Chen C-L, Wang RK.. Optical coherence tomography based angiography [Invited]. Biomed Opt Express. 2017; 8: 1056–1082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Courtie E, Veenith T, Logan A, Denniston AK, Blanch RJ.. Retinal blood flow in critical illness and systemic disease: a review. Ann Intensive Care. 2020; 10: 1–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. de Carlo TE, Romano A.. A review of optical coherence tomography angiography (OCTA). Int J Retin Vitr. 2015; 1: 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Rocholz R, Corvi F, Weichsel J, Schmidt S, Staurenghi G. OCT angiography (OCTA) in retinal diagnostics. In: High Resolution Imaging in Microscopy and Ophthalmology. Berlin: Springer International Publishing; 2019: 135–160. [PubMed] [Google Scholar]
- 6. Li XX, Wu W, Zhou H, et al.. A quantitative comparison of five optical coherence tomography angiography systems in clinical performance. Int J Ophthalmol. 2018; 11: 1784–1795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Arya M, Rashad R, Sorour O, Moult EM, Fujimoto JG, Waheed NK.. Optical coherence tomography angiography (OCTA) flow speed mapping technology for retinal diseases. Expert Rev Med Devices. 2018; 15: 875–882. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Borrelli E, Sadda SVR, Uji A, Querques G.. Pearls and pitfalls of optical coherence tomography angiography imaging: a review. Ophthalmol Ther. 2019; 8: 215–226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Binotti WW, Saukkonen D, Seyed-Razavi Y, Jamali A, Hamrah P.. Automated image threshold method comparison for conjunctival vessel quantification on optical coherence tomography angiography. Transl Vis Sci Technol. 2022; 11(7): 15–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Mehta N, Liu K, Alibhai AY, et al.. Impact of binarization thresholding and brightness/contrast adjustment methodology on optical coherence tomography angiography image quantification. Am J Ophthalmol. 2019; 205: 54–65. [DOI] [PubMed] [Google Scholar]
- 11. Dai W, Chee M-L, Majithia S, et al.. Agreement in measures of macular perfusion between optical coherence tomography angiography machines. Sci Rep. 2020; 10(1): 8345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Rabiolo A, Gelormini F, Sacconi R, et al.. Comparison of methods to quantify macular and peripapillary vessel density in optical coherence tomography angiography. PLoS One. 2018; 13(10): e0205773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Courtie EF, Kale AU, Hui BTK, et al.. Stability of OCT and OCTA in the intensive therapy unit setting. Diagnostics. 2021; 11: 1516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Zahid S, Dolz-Marco R, Freund KB, et al.. Fractal dimensional analysis of optical coherence tomography angiography in eyes with diabetic retinopathy. Invest Ophthalmol Vis Sci. 2016; 57: 4940–4947. [DOI] [PubMed] [Google Scholar]
- 15. Cabral D, Pereira T, Ledesma-Gil G, et al.. Volume rendering of dense B-scan optical coherence tomography angiography to evaluate the connectivity of macular blood flow. Invest Ophthalmol Vis Sci. 2020; 61(6): 44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Do JT-T, Leeman S, Renneburg C, et al.. Identifying microvascular differences in multiple sclerosis via optical coherence tomography angiography (OCT-A). Invest Ophthalmol Vis Sci. 2020; 61: 4829–4829. [Google Scholar]
- 17. Choi E, Kim M.. Analysis of foveal avascular zone and macular vasculature using optical coherence tomography angiography in type 2 diabetic eyes without diabetic retinopathy. Invest Ophthalmol Vis Sci. 2018; 59(9): 1931. [Google Scholar]
- 18. Shahlaee A, Pefkianaki M, Hsu J, Ho AC.. Measurement of foveal avascular zone dimensions and its reliability in healthy eyes using optical coherence tomography angiography. Am J Ophthalmol. 2016; 161: 50–55.e1. [DOI] [PubMed] [Google Scholar]
- 19. Kaizu Y, Nakao S, Yoshida S, et al.. Optical coherence tomography angiography reveals spatial bias of macular capillary dropout in diabetic retinopathy. Invest Ophthalmol Vis Sci. 2017; 58: 4889–4897. [DOI] [PubMed] [Google Scholar]
- 20. Collins LF, Shantha JG, Yeh S, et al.. Use of optical coherence tomography angiography to assess microvascular health among persons with HIV: employing the retina as a convenient window. Open Forum Infect Dis. 2020; 7(Suppl 1): S507. [Google Scholar]
- 21. Hayden C, Fowler N, Kitchens J, Maldonado R.. A comparison of automated thresholding methods to quantify vessel density from wide field optical coherence tomography angiography in a cohort of normal subjects. Invest Ophthalmol Vis Sci. 2021; 62(11): 85. [Google Scholar]
- 22. Arrigo A, Aragona E, Di Nunzio C, et al.. Octa-based identification of different vascular patterns in stargardt disease. Transl Vis Sci Technol. 2019; 8(6): 26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Al-Sheikh M, Falavarjani KG, Akil H, Sadda SR.. Impact of image quality on OCT angiography based quantitative measurements. Int J Retina Vitreous. 2017; 3: 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Vuidel G, Frankhauser P, Tannier C. Fractalyse. Available at: https://sourcesup.renater.fr/www/fractalyse/. Accessed March 03, 2022.
- 25. Fawzi AA, Fayed AE, Abdelbaki AM, El Zawahry OM. Optical coherence tomography angiography reveals progressive worsening of retinal vascular geometry in diabetic retinopathy and improved geometry after panretinal photocoagulation. PLoS One. 2019; 14(12): e0226629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Weledji EP, Verla V.. Failure to rescue patients from early critical complications of oesophagogastric cancer surgery. Ann Med Surg. 2016; 7: 34–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Liu X, Kale AU, Capewell N, et al.. Optical coherence tomography (OCT) in unconscious and systemically unwell patients using a mobile OCT device: a pilot study. BMJ Open. 2019; 9(11): e030882. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Liu Z, Wang H, Jiang H, Gameiro GR, Wang J.. Quantitative analysis of conjunctival microvasculature imaged using optical coherence tomography angiography. Eye Vis. 2019; 6(1): 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Terheyden JH, Wintergerst MWM, Falahat P, Berger M, Holz FG, Finger RP.. Automated thresholding algorithms outperform manual thresholding in macular optical coherence tomography angiography image analysis. PLoS One. 2020; 15(3): e0230260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Cicchetti D V. Choice of Agreement Statistics: A Discussion of the Underlying Biostatistics, With Heuristic Examples From the Vineland-3. Bloomington, MN: NCS Pearson, Inc. 2017: 1–9. [Google Scholar]
- 31. Shrout PE, Fleiss JL.. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979; 86: 420–428. [DOI] [PubMed] [Google Scholar]
- 32. Koo TK, Li MY.. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016; 15: 155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Wang L, Kwakyi O, Nguyen J, et al.. Microvascular blood flow velocities measured with a retinal function imager: inter-eye correlations in healthy controls and an exploration in multiple sclerosis. Eye Vis. 2018; 5: 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Koch R, Seto B, Yamada K, et al.. Relative retinal blood flow: a novel and informative measure of unilateral retinal vein occlusion severity. Transl Vis Sci Technol. 2021; 10(3): 15–15. [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
- 35. Cosatto V, Wainwright A, Rochtchina E, et al.. Correlation Between Right and Left Eyes in the Measurement of Retinal Vascular Fractal Dimension in an Older Population. IOVS. 2009; 50(13): 99–99. [Google Scholar]
- 36. Yu S, Lakshminarayanan V.. Fractal dimension and retinal pathology: a meta-analysis. Appl Sci. 2021; 11(5): 1–23. [Google Scholar]
- 37. Mendes L, Marques IP, Cunha-Vaz J.. Comparison of different metrics for the identification of vascular changes in diabetic retinopathy using OCTA. Front Neurosci. 2021; 15: 1553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Mello LGM, Rodrigues Neto T dos S, da Silva Neto ED, Preti RC, Monteiro MLR, Zacharias LC. A standardized method to quantitatively analyze optical coherence tomography angiography images of the macular and peripapillary vessels. Int J Retin Vitr. 2022; 8(1): 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Untracht GR, Matos R, Dikaios N, et al.. OCTAVA: an open-source toolbox for quantitative analysis of optical coherence tomography angiography images. Plos One. 2021; 16(12): e0261052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Dave PA, Dansingani KK, Jabeen A, et al.. Comparative evaluation of foveal avascular zone on two optical coherence tomography angiography devices. Optom Vis Sci. 2018; 95: 602–607. [DOI] [PubMed] [Google Scholar]
- 41. Gutierrez-Benitez L, Palomino Y, Casas N, Asaad M.. Automated measurement of the foveal avascular zone in healthy eyes on Heidelberg spectralis optical coherence tomography angiography. Arch Soc Esp Oftalmol (English Ed). 2022; 97: 432–442. [DOI] [PubMed] [Google Scholar]
- 42. Chen Y, Laotaweerungsawat S, Zhao T, et al.. Discordant vascular parameter measurements in diabetic and non-diabetic eyes detected by different optical coherence tomography angiography devices. PLoS One. 2020; 15(6): e0234664. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Corvi F, Pellegrini M, Erba S, Cozzi M, Staurenghi G, Giani A.. Reproducibility of vessel density, fractal dimension, and foveal avascular zone using 7 different optical coherence tomography angiography devices. Am J Ophthalmol. 2018; 186: 25–31. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.