Evaluation of the different thresholding strategies for quantifying choriocapillaris using optical coherence tomography angiography

Abstract

Background

In this paper, we evaluate the different thresholding strategies that have been used for the quantification of the choriocapillaris (CC) and explore their repeatability and the interchangeability of the measurements resulting from its application.

Methods

Observational study. Eighteen eyes from nine healthy volunteers aged >18 years were imaged four consecutive times with a SD-OCTA system (Heidelberg Engineering, Germany) using a 10°×10° high-resolution protocol centered on the fovea. Projection artifacts were removed, and the CC was bracketed between 10 and 30 µm below Bruch’s membrane. For the quantification of CC, we used four flow deficits (FD) parameters: FD number, mean FD size, total FD area and FD density. We performed a systematic review of literature to collect the thresholding methods that have been used for the quantification of CC. The CC quantification parameters were then evaluated after applying each of the thresholding strategies. Intraclass correlation coefficient (ICC) and Pearson’s correlation analysis were used to compare the repeatability and interchangeability among the different thresholding strategies for quantifying the CC.

Results

A total of 72 optical coherence tomography angiography (OCTA) examinations were considered. The systematic review allowed us to conclude that three local thresholding strategies (Phansalkar, mean and Niblack) and three global thresholding strategies (mean, default, Otsu) have been used for CC quantification. These strategies were evaluated in our observational study. We found a high agreement within the same method in the quantification of FD number, mean FD size, total FD area and FD density but a poor agreement with different strategies. Local strategies achieved a significantly superior ICC than global ones in CC quantification.

Conclusions

In conclusion, the interchangeability of the CC quantification using different thresholding strategies is low, and direct comparisons should not be performed. Local thresholding strategies are significantly superior to global ones for quantifying CC and should be preferred. There is an unmet need for a uniform strategy to quantify CC in future studies.

Introduction

The choriocapillaris (CC) is a dense vascular layer that is located beneath Bruch’s membrane and provides metabolic support for the outer retina, retinal pigmented epithelium and choroidal stroma (1). The study of CC properties and morphology has been of interest throughout the years as histopathological studies revealed that the CC plays an important role in prevalent retinal and choroidal diseases (2-4). However, quantitative studies of the CC have been limited by its in-vivo inaccessibility (5,6).

Recently, optical coherence tomography angiography (OCTA) emerged as means of providing detailed images of retinal vasculature. Currently available OCTA devices are capable of generating high-quality en-face images of the retinal plexuses that enable retinal vasculature quantification (7). Such ability of resolving the microvascular networks of the retina is possible as the inter-capillary distance of the vessels in these plexuses (71.3±5.2 µm) (8) are generally larger than the lateral resolution of the system (15–20 µm) (9). Particularly for research purposes, the ability to reproducibly and objectively quantify OCTA scans is critical as it allows images to be compared. Not all OCTA instruments have built-in software to calculate the vasculature metrics. As such, researchers and clinicians currently face the challenges of exploring quantification methodologies on en-face OCTA exported images (10-12). The process of OCTA image quantification requires the application of a thresholding strategy (in other words, a process that enables the separation of the region of interest from the background) (13). The output of the thresholding operation is a binary image, where the foreground and the background are represented by white or black pixels, according to a pre-definition. Different thresholding strategies have been described by researchers, most of them based on the use of open-source software, such as Image J (National Institutes of Health, Bethesda, available at https://imagej.nih.gov/ij/) (14).

Currently, there are no clear consensus on the best thresholding strategy for the binarization of CC images and multiple strategies have been reported. With OCTA quantification becoming increasingly common among researchers, there is a pressing need to understand how different methodologies can affect metrics. It is also important to understand how well different methods perform in both producing accurate metrics and minimizing variability. To our knowledge, few studies have reported the reproducibility and reliability of the different thresholding strategies for the quantification of CC vasculature in OCTA.

The aim of our work is to evaluate the repeatability of the different thresholding strategies that have been used for the quantification of CC as well as to evaluate how the CC images are affected by the application of different thresholding strategies.

Methods

Systematic review

We conducted a systematic review and comprehensive search to identify the thresholding strategies that have been applied for the purpose of CC quantification in OCTA scans.

Terminology

For the purpose of this paper, the following terminology was used:

1. CC: the capillary plexus of the choroid located between the Sattler’s layer and Bruch’s membrane. For the purpose of this review, we considered the OCTA slab defined by the authors in each study.

2. Thresholding strategy: this term is used to refer to the strategy that was applied in raw greyscale CC scans to convert them into binary images (separate the region of interest from the background).

Search strategy

We conducted a systematic review from 1st January 2014 to 31st December 2019 using the PubMed electronic database. We’ve used the following query: “(choriocapillaris AND [(optical coherence tomography angiography) OR (OCTA) OR (OCT angiography)]”. We opted to include a broad query so that we could encompass the larger number of studies possible.

Eligibility criteria

We only included studies that reported macular CC vascular quantification strategies in OCTA images in humans (either as flow voids/deficits or as vascular density or similar concepts). Identified publications were screened manually based on the title and abstract. We placed no restrictions or limits during the search process (language, time or country of origin).

Data extraction and synthesis

After excluding publications that did not analyze quantitatively CC OCTA macular scans, we then manually searched on the manuscripts and the thresholding strategy that had been used was collected.

Observational study

A prospective study was performed at Centro Hospitalar de Entre o Douro e Vouga (Portugal). The study was approved by the Institutional Ethics Committee of Centro Hospitalar de Entre o Douro e Vouga (No. CA-0708/18-0t_MP/AC) and adhered to the tenets of the Declaration of Helsinki and its later amendments. Informed consent was obtained from participants before the inclusion in the study.

Sample

Eighteen eyes from nine healthy individuals >18 years (8 females, mean age of 38.2±11.4 years) with no systemic or ocular history, no visual complaints, and no identified optic disc, retinal or choroidal pathologies on examination and a refractive error <6.00 diopters were enrolled in the study.

Image acquisition

The individuals were scanned in both eyes using a 10°×10° scan high-resolution protocol centered on the fovea with the Heidelberg Spectralis OCTA system (version 1.10.2.0, Spectralis; Heidelberg Engineering, Heidelberg, Germany). The device uses an 870 µm central wavelength and images at 85,000 per second with an isotropic lateral resolution of 5.7 µm/pixel. The image cubes were acquired using 5 repeated scans with the TruTrack technology from Heidelberg. The imaging protocol was explained to the participants before any acquisition. For evaluating repeatability, we performed four consecutive OCTA acquisitions in each eye, separated by a few seconds. Images were manually reviewed, and low-quality scans were excluded (scans with motion, projection or other image artifacts). All images were obtained by the same trained ophthalmic professional and in the same environment conditions. Acquisitions were repeated if necessary, to obtain high-quality images.

Image processing

Automated segmentation of the CC was performed using the software provided within the Spectralis^® (Heidelberg Engineering). The boundary of the CC slab is defined 10–30 µm below the Bruch’s membrane. The retinal projection artifacts were removed using the projection artifact removal tool from Heidelberg (Software version 6.14.1) before the images were further processed for quantification. The algorithm removes flow projection from the normally avascular outer retinal slab and preserves in situ flow signal of the deeper vessels. The angiograms were exported for analysis as Tagged Image File Format (tiff) format. Image analysis was performed using Image J V. 1.51 (National Institutes of Health, Bethesda) (14). Raw data was cut using the same frame (960×960 pixels) in order to exclude artifacts that sometimes occur in the margin of the scan. Brightness and contrast adjustments were not performed, the images were manipulated in the native form. Before any conversion, the initial pixel values were coded as 8-bit values, ranging from 0–255.

Histogram analysis

For analysis purpose, we extracted the images histograms of each CC image to verify if different eyes followed similar distributions. Histogram analysis was performed using Image J V. 1.51 (National Institutes of Health, Bethesda) (14).

Image post-processing

Following the mentioned transformations, each CC angiogram was then processed with all of the thresholding strategies found in the initial systematic review using Image J V. 1.51 (National Institutes of Health, Bethesda) (14).

Quantification parameters

Contrasting to the retinal plexuses, the CC is a much denser vascular network (15,16) with a much smaller inter-capillary distance [average 10–25 µm (17)] that cannot be resolved by OCTA devices. Thus, a different approach is necessary for quantification purposes. This limitation has been resolved by the quantification of the flow deficits (FD) in the exported CC images (18-26). FD are defined as regions of non-perfusion or low perfusion, where the flow is below the sensitivity limit of the current OCTA technology (19). In a binary image of the CC, FD correspond to black pixels or the background. In our study, to quantify the FD, images were processed with the ‘Analyze Particles’ command (Image J V. 1.51, National Institutes of Health, Bethesda) (14). This function retrieved the following measurements: (I) the FD number, (II) mean FD size, (III) total FD area and (IV) FD density. These variables were represented using box blots to allow the visualization the quantitative distribution of the values according to the thresholding strategy.

Reproducibility of CC FD quantification

We evaluated the reproducibility of CC FD quantification for each thresholding strategy found in the systematic review. This was performed using the four consecutive scans from the same eye, acquired in the same environmental conditions, as previously described. After applying the same thresholding strategy to the four scans per eye, we calculated the FD number, the mean FD size, the total FD area and the FD density for each scan. The repeatability of the CC FD quantification for each strategy was then estimated using the intraclass correlation coefficient (ICC) and the respective 95% confidence interval (CI), with the two-way mixed, single measures, absolute agreement mode. All statistical analysis was performed using IBM SPSS Statistics v. 25 (SPSS Inc., Chicago, IL, USA). Significance was set at 0.05.

Agreement among the strategies for CC quantification

For this purpose, the first CC scan of each eye was used. We first evaluated the correlation among the same FD quantification parameters after applying the different thresholding strategies to verify if the algorithms retrieved similar trends in values variation. Correlation among the different obtained values using different strategies was evaluated using the Pearson’s correlation coefficient (both r values and P values are reported). In the absence of correlation, either the thresholding strategies had a complete absolute agreement, either they fail in retrieving FD. We then tested the absolute agreement for the different thresholding strategies for measuring the same scan and evaluated the correlation between the different parameters of FD obtained using different thresholding strategies. The agreement among the different thresholding strategies for quantifying FD in the same CC scan was evaluated using the ICC and the respective 95% CI, as previously described. All statistical analysis was performed using IBM SPSS Statistics v. 25 (SPSS Inc., Chicago, IL, USA). Significance was set at 0.05.

Results

Systematic review

Figure S1 presents the search strategy and its results. One thousand six hundred and seventy-four studies were identified by the query. From these, 1,490 were excluded after a primary screening. From the remaining 184 studies, 51 were excluded as they did not meet the inclusion criteria. One hundred and thirty-three studies were included in the final synthesis. The detailed results are summarized in Table S1. We grouped the found thresholding strategies in major categories as:

1. OCTA device-related thresholding strategies (n=40): studies that used the thresholding method that is intrinsic to the OCTA device. These algorithms are not customizable and are device-dependent and thus were not considered for the observational study.

2. Thresholding strategies customized by the author (n=37): studies that used customized thresholding strategies designed by the authors. These strategies were not included in the observational study.

3. Unknown/inaccessible (n=5): studies in which the thresholding strategy was not accessible, and authors did not answer to direct email questioning. These strategies were not included in the observational study.

4. Global thresholding strategies (n=25): the simplest form of binarization uses global thresholds. These methods employ a threshold value, t, pixel values greater that t are set to 1 and pixel values smaller or equal to t are set to 0, or vice versa (27-30). Image histogram (the representation of the number of pixels in an image as a function of their intensity) computing is an important tool in the decision for global thresholding methods and values (30). In an ideal case, the histogram has a deep and sharp valley between two peaks representing objects and background, respectively, so that the threshold can be chosen at the bottom of this valley. As a single threshold is applied to the entire image, these methods produce incompetent binarization in conditions as the presence of noise or uneven background. Three global thresholding strategies have been used to quantify CC OCTA scans according to our review and were evaluated in the observational study: global Otsu, global mean and global default. The definition of each algorithm is presented as a Supplementary file 1.

5. Local thresholding strategies (n=32): local binarization methods were created to surpass the limitations of global methods as they assign different threshold values according to local properties of the image (28). Various factors, such as nonstationary and correlated noise, ambient illumination, busyness of gray levels within the object and its background, inadequate contrast, and object size not commensurate with the scene, complicate the thresholding operation. Thus, the ideal method must be selected case by case, depending on the image properties. Three local thresholding strategies have been used to quantify CC OCTA scans according to our review and were evaluated in the observational study: local Phalsankar, local mean/median and local Niblack. The definition of each algorithm is presented as a Supplementary file 1.

Observational study

In the following paragraphs we present the results of CC scans analysis using the three global (global Otsu, global mean and global default) and the three local (local Phalsankar, local mean/median and local Niblack) thresholding strategies obtained in the systematic review.

Histogram analysis

A typical histogram from the en-face CC images after 8-bit conversion is demonstrated in Figure 1 as an example. Inspection of the histogram showed a Gaussian curve of grey-scale values. In these histograms, it is not possible to trace a deep and sharp valley between two peaks representing objects and background, so that the threshold can be chosen at the bottom of this valley. Thus, from the visual inspection of this histogram no single thresholding value can be inferred to segment the region of interest in the image.

Figure 1.

The figure illustrates the histogram from one of the CC exported images after 8-bit conversion. Analysis was performed using Image J V. 1.51 (National Institutes of Health, Bethesda). CC, choriocapillaris.

Image post-processing

In Figure 2, we demonstrate the visual result of applying the different thresholding strategies (in the left square the result of applying each of the global strategies, and in the right square the result of applying each of the local strategies). The original image of the CC is in the center for comparison. Figure 2 illustrates the pronounced differences of applying local and global thresholding strategies, that are evidenced by the different black and white regions highlighted by each thresholding strategy. All the global thresholding strategies produced binary images that qualitatively appeared identical. The local methods generated a more homogeneous appearance.

Figure 2.

The figure illustrates one of the CC images in its 8-bit original form and the results after applying the 6 different thresholding strategies. Analysis was performed using Image J V. 1.51 (National Institutes of Health, Bethesda). CC, choriocapillaris.

Quantification parameters

Figure 3 illustrates the differences in the quantification of the four considered CC FD metrics (either the FD number—A, total FD area—B, mean FD size—C, and FD density—D) using the six considered thresholding strategies applied to a single typical OCTA scan. As seen in the graphs, we verified a high amount of variation for each of the quantitative parameters according to the used method. The quantitative parameters generated after applying global strategies achieved a less dispersive distribution, that is in accordance with the observation in Figure 2. By contrast, measurements resulting from applying local strategies generated more dispersed values, as seen in the boxplots.

Figure 3.

Differences in the quantification of the four considered CC metrics (either the FD number—A, total FD area—B, mean FD size—C, and FD density—D) using the six considered thresholding strategies (local mean, local Phansalkar, local Niblack, global mean, global Otsu, global default) in the same scan. Individual boxplots show the median with the 25th–75th percentile (box) and range (brackets). CC, choriocapillaris; FD, flow deficits; L., local; G., global.

Reproducibility of CC FD quantification

The repeatability of CC FD quantification after applying each thresholding strategy was evaluated for each individual CC quantitative parameter (FD number, mean FD size, total FD area and FD density) using the four OCTA consecutive scans (Table 1). Globally, the ICC was superior in the quantification of FD number (ranging between 0.852 to 0.964) followed by FD density (ranging from 0.786 to 0.974). No isolated strategy was found to be significantly superior than the others in analyzing repeated scans, as demonstrated by the overlapping of the ICC 95% CI presented in Table 1.

Table 1. Agreement among the four scans for each parameter measured using each of the six thresholding strategies.

Quantitative parameter	Thresholding strategy	ICC (95% CI)
N. FD	G. Otsu	0.909 (0.820-0.960)
	G. Mean	0.852 (0.730-0.983)
	G. Default	0.904 (0.818-0.958)
	L. Mean	0.925 (0.856-0.968)
	L. Phansalkar	0.964 (0.928-0.985)
	L. Niblack	0.955 (0.910-0.981)
FD area	G. Otsu	0.880 (0.777-0.947)
	G. Mean	0.115 (–0.064-0.395)
	G. Default	0.838 (0.707-0.927)
	L. Mean	0.880 (0.777-0.947)
	L. Phansalkar	0.937 (0.877-0-973)
	L. Niblack	0.881 (0.777-0.948)
FD average size	G. Otsu	0.771 (0.605-0.894)
	G. Mean	0.627 (0.412-0.813)
	G. Default	0.684 (0.484-0.847)
	L. Mean	0.825 (0.687-0.921)
	L. Phansalkar	0.850 (0.727-0.933)
	L. Niblack	0.843 (0.716-0.930)
FD density	G. Otsu	0.873 (0.763-0.944)
	G. Mean	0.786 (0.618-0.902)
	G. Default	0.833 (0.699-0.925)
	L. Mean	0.879 (0.774-0.946)
	L. Phansalkar	0.917 (0.840-0.964)
	L. Niblack	0.974 (0.946-0.989)

Values presented correspond to ICC. ICC, intraclass correlation coefficient; G., global; L., local; N., number; CI, confidence interval; FD, flow deficits.

We compared the repeatability of global versus local thresholding strategies for evaluating each of the four considered CC quantitative parameters (FD number, mean FD size, total FD area and FD density). As seen in Table 2, the agreement was significantly superior for local versus global strategies for all the four quantitative parameters. Local strategies achieved an ICC of 0.978 (95% CI: 0.967–0.986) for FD number quantification, 0.950 (95% CI: 0.926–0.969) for FD area, 0.925 (95% CI: 0.889–0.952) for average FD size and 0.958 (95% CI: 0.936–0.974) for FD density.

Table 2. Agreement among the four scans for each parameter measured using either local or global thresholding strategies.

Quantitative parameter	Thresholding strategy	ICC (95% CI)
N. FD	Local	0.978 (0.967–0.986)
	Global	0.901 (0.854–0.936)
Area FD	Local	0.950 (0.926–0.969)
	Global	0.446 (0.309–0.589)
Average size FD	Local	0.925 (0.889–0.952)
	Global	0.701 (0.591–0.796)
FD density	Local	0.958 (0.936–0.974)
	Global	0.843 (0.770–0.899)

Values presented correspond to ICC. ICC, intraclass correlation coefficient; FD, flow deficits; N., number; CI, confidence interval.

Agreement among the strategies for CC quantification

We verified variable degrees of correlation among the different thresholding strategies for measuring the same quantitative parameter (either the FD number, mean FD size, total FD area and FD density). All the correlation coefficients, r, and respective P value are in Table 3.

Table 3. Correlation among the different parameters measured using six thresholding strategies.

Thresholding strategy 1	Thresholding strategy 2	FD number	Area FD	Average size FD	FD density
G. Otsu	G. Mean	r=0.951; P<0.001*	r=0.892; P<0.001*	r=0.578; P=0.015*	r=0.920; P<0.001*
	G. Default	r=0.999; P<0.001*	r=0.992; P<0.001*	r=0.969; P<0.001*	r=0.992; P<0.001*
	L. Mean	r=0.823; P<0.001*	r=0.741; P<0.001*	r=0.687; P=0.002*	r=0.735; P<0.001*
	L. Phansalkar	r=0.956; P<0.001*	r=0.837; P<0.001*	r=0.681; P=0.003*	r=0.859; P<0.001*
	L. Niblack	r=0.467; P=0.051	r=0.686; P=0.002*	r=–0.62; P=0.814	r=0.826; P<0.001*
G. Mean	G. Otsu	r=0.951; P<0.001*	r=0.892; P<0.001*	r=0.578; P=0.015*	r=0.920; P<0.001*
	G. Default	r=0.957; P<0.001*	r=0.875; P<0.001*	r=0.547; P=0.023*	r=0.890; P<0.001*
	L. Mean	r=0.680; P<0.001*	r=0.556; P=0.017*	r=0.365; P=0.150	r=0.545; P=0.016*
	L. Phansalkar	r=0.935; P<0.001*	r=0.876; P<0.001*	r=0.502; P=0.040*	r=0.771; P<0.001*
	L. Niblack	r=0.427; P=0.077	r=0.695; P=0.001*	r=–0.286; P=0.266	r=0.814; P<0.001*
G. Default	G. Otsu	r=0.999; P<0.001*	r=0.992; P<0.001*	r=0.969; P<0.001*	r=0.992; P<0.001*
	G. Mean	r=0.957; P<0.001*	r=0.875; P<0.001*	r=0.547; P=0.023*	r=0.890; P<0.001*
	L. Mean	r=0.813; P<0.001*	r=0.782; P<0.001*	r=0.717; P=0.001*	r=0.779; P<0.001*
	L. Phansalkar	r=0.958; P<0.001*	r=0.802; P<0.001*	r=0.630; P=0.007*	r=0.851; P<0.001*
	L. Niblack	r=0.468; P=0.050	r=0.671; P=0.002*	r=–0.069; P=0.791	r=0.795; P<0.001*
L. Mean	G. Otsu	r=0.823; P<0.001*	r=0.741; P<0.001*	r=0.687; P=0.002*	r=0.735; P<0.001*
	G. Mean	r=0.680; P<0.001*	r=0.556; P=0.017*	r=0.365; P=0.150	r=0.545; P=0.016*
	G. Default	r=0.813; P<0.001*	r=0.782; P<0.001*	r=0.717; P=0.001*	r=0.779; P<0.001*
	L. Phansalkar	r=0.789; P<0.001*	r=0.365; P=0.136	r=0.191; P=0.463	r=0.841; P=0.037*
	L. Niblack	r=0.495; P=0.037*	r=0.409; P=0.092	r=–0.020; P=0.938	r=0.453; P=0.051
L. Phansalkar	G. Otsu	r=0.956; P<0.001*	r=0.837; P<0.001*	r=0.681; P=0.003*	r=0.859; P<0.001*
	G. Mean	r=0.935; P<0.001*	r=0.876; P<0.001*	r=0.502; P=0.040*	r=0.771; P<0.001*
	G. Default	r=0.958; P<0.001*	r=0.802; P<0.001*	r=0.630; P=0.007*	r=0.851; P<0.001*
	L. Mean	r=0.789; P<0.001*	r=0.365; P=0.136	r=0.191; P=0.463	r=0.481; P=0.037*
	L. Niblack	r=0.474; P=0.047*	r=0.628; P=0.005*	r=–0.012; P=0.963	r=0.863; P<0.001*
L. Niblack	G. Otsu	r=0.467; P=0.051	r=0.686; P=0.002*	r=–0.062; P=0.814	r=0.826; P<0.001*
	G. Mean	r=0.427; P=0.077	r=0.695; P=0.001*	r=–0.286; P=0.266	r=0.814; P<0.001*
	G. Default	r=0.468; P=0.050	r=0.671; P=0.002*	r=–0.069; P=0.791	r=0.795; P<0.001*
	L. Mean	r=0.495; P=0.037	r=0.409; P=0.092	r=–0.020; P=0.938	r=0.453; P=0.051
	L. Phansalkar	r=0.474; P=0.047*	r=0.628; P=0.005*	r=–0.012; P=0.963	r=0.863; P<0.001*

Values of Pearson correlation coefficient (r) and P value are presented. *, P<0.05. FD, flow deficits; G., global; L., local; N., number.

Although some degree of correlation may be found among the measurements, the absolute agreement among the six thresholding methods for measuring the same quantitative parameter in the CC images was low, with an ICC ranging from 0.000 to 0.339—Table 4. The lowest absolute agreement was found for average FD size, followed by FD area, FD density and FD number.

Table 4. Absolute agreement among the six thresholding strategies for measuring the same quantitative parameter in the same scan.

Quantitative parameter	ICC (95% CI)
FD number	0.339 (0.124–0.601)
Area FD	0.224 (0.063–0.468)
Average size FD	0.000 (–0.090–0.191)
FD density	0.278 (0.091–0.530)

Mean value of the four acquisitions was considered in each pair. Values presented correspond to ICC. ICC, intraclass correlation coefficient; CI, confidence interval; FD, flow deficits.

Discussion

In this study we systematically reviewed the thresholding strategies that have been recently used to quantify CC and evaluated their interchangeability and reproducibility. From our systematic review, we concluded that there is an unmet need for the homogenization of the methods that are used to threshold CC images. We found a marked variability in the thresholding methods that have been applied to CC angiograms, thus conditioning future aggregation of the results.

A significant number of authors choose to use customized or device-included algorithms. Although they may improve the quantification process, they have a narrow spectrum of application as they are not universally applicable due either to device conditioning or expertise programing needs. In our observational study, we evaluated three global and three local thresholding methods. We found that local thresholding methods have a superior performance in CC angiograms (either by a superior repeatability and an adequacy to CC histogram properties) and should be preferred to global ones. We also concluded that the CC metrics that are obtained through them are not interchangeable. Thus, direct comparisons cannot be made from studies that adopt different thresholding strategies.

OCTA quantification was a change in the paradigm for the evaluation of most macular diseases. However, there is currently a need to improve and uniformize the segmentation methods for OCTA quantification. In addition, before any clinical trial, it is essential to precisely know the strengths and limitations of the image segmentation strategies that are being applied. As previously mentioned, the CC images are challenging as the intracapillary distance of the vessels is general under the resolution of OCTA devices, what does not occur in retinal capillary plexus. Thus, the CC images quantification is more prone to be influenced by the image processing methods and all sources of variability must be explored to prevent biased conclusions.

Thresholding is known to be a critical step in image segmentation as it will influence all the analysis that are subsequently applied to CC angiograms. When thresholding CC OCTA images, a balance must be struck between excluding noise and including valid signal. If the threshold is set low, then there will be more noise above the threshold (and hence in the resulting thresholded OCTA image), but the subsequent OCTA output will be less likely to have eliminated any true vasculature.

In the current study, we explored the influence of applying different thresholding strategies in the quantification of the CC FD. We found that, considering each of the thresholding strategy individually, the four CC quantitative parameters were highly correlated. This is in accordance with the study performed by Shi et al. (26), that also found excellent correlations among the percentage of CC FD and the average area of FD. However, we found that the absolute agreement among the evaluated thresholding strategies for measuring the same quantitative parameter in the CC images was low, with ICC ranging from 0.000 to 0.339. Therefore, although somewhat correlated, the CC FD values obtained using different thresholding strategies are not interchangeable and direct comparisons should not be performed among studies that use different strategies.

In this study we also demonstrated that the histogram of the CC en-face 8-bit image follow a Gaussian distribution of grey-scale values. This is in accordance to what has been described in the literature (6,31) and, from this, we can conclude that global thresholding strategies will not have an adequate performance as the distribution does not suggest a value to separate the background from the foreground. By using a global thresholding strategy, we will erroneously classify noise as FD or the opposite. This was corroborated by our findings as the global thresholding strategies, as a group, achieved a significant lower reliability in the quantification of CC consecutive scans when comparing to the local ones. This was verified irrespective of FD characteristics. Therefore, local strategies should be preferred to global ones for CC FD quantitative analysis purpose. Among local strategies, we found similar repeatability and no standard could be inferred from this analysis.

To our knowledge, few studies have investigated the variability in CC quantification induced by the thresholding process. Yun et al. (32) compared the Phansalkar and a device-specific global thresholding method in images from four OCTA devices. Mehta et al. (33) also investigated CC quantitative measurements variability using four different thresholding strategies (global default, global mean, global Otsu, local mean, and local Phansalkar) in a SS OCTA. Although the previous authors concluded about the differences that exist in FD quantification when each strategy is applied in a single examination, no repeatability analysis is reported in the study and no comparison is reported among the local and global strategies as major groups. Chu et al. (20) also used SS OCTA and compared the correlation and agreement between one local thresholding method (fuzzy C-means algorithm) and one global thresholding method (an algorithm that uses standard deviation from a young normal database) and concluded about the strong correlation between the two methods for measuring FD density and mean FD size. In the another study (23), Chu et al. compared the variation in quantitative CC metrics after applying both fuzzy C-means thresholding method and Phansalkar method (using different pixel radius) and found heterogeneous results. Interestingly the authors also performed a repeatability study and ICC values reported for CC metrics using Phansalkar method are superior than those we found. This may be explained either by the different post-processing of the signal, either by the use SS OCTA device to perform acquisitions. In SS OCTA, the longer wavelength that has better penetration through the RPE and less sensitivity roll-off into the choroid, which results in an improvement in the likelihood of detecting the weaker signals from under the retinal pigment epithelium (34). Previous reports have reported the superiority of SS-OCTA for detecting macular neovascularization under the retinal pigmented epithelium (35,36). Thus, we may infer that SS technology will have a superior reliability for the quantification of CC FD than SD do.

Our study has several limitations. Firstly, as previously mentioned by Chu et al. (23), there is a lack of ground truth for CC vasculature, as all the techniques to validate it are invasive involving sacrificing animals to compare harvested eyes with previous imaging. Secondly, we are aware that the quantitative CC parameters may be significantly influenced by small differences in the slab selection (21). We did not stratify the results by slab classification as that was not the purpose of the review. In our observational study, we opted to maintain the manufacturer CC segmentation for clinical relevance. We are also aware of other sources of variability in OCTA images quantification as the use of different algorithms and averaging (13,37). Thus, more studies in the field are needed. Finally, our study only included healthy eyes from young patients. It is thus unpredictable how much our results would change if the validation was performed in patients with significant retinal diseases. Further studies are needed to evaluate the behavior of these thresholding methods in more complex and noisy scans as in the presence of macular neovascularization and to evaluate their discriminative power to differentiate normal versus abnormal CC.

Besides all the potential limitations, our study has several strengths. We report the heterogeneity of thresholding process in CC quantification thus raising awareness for the need of uniformization to achieve comparable conclusions. We also evaluated the interchangeability among different algorithms for measuring the same parameter and the reproducibility of each algorithm for measuring repeated scans. This gives a broad perspective of the variability that is inherent to choosing different thresholding strategies. In addition, no data exists for SD device regarding this topic. We believe our research will help future researchers in the field to improve their thresholding selection.

Conclusions

As OCTA becomes incorporated into clinical decision making, the ability to understand the thresholding process, and the artifacts that this process introduces in CC FD quantification, is of utmost importance. We found no interchangeability among different thresholding strategies for quantifying CC FD. Thus, direct comparisons should not be considered in future studies. Local thresholding strategies demonstrated a superior repeatability and should be preferred to the global ones for CC quantitative analysis in OCTA angiograms. There is currently an unmet need for a uniform strategy to quantify CC in future studies.

Supplementary

Supplementary file 1 Thresholding strategies—definitions

Global thresholding

The mean global threshold method (27) takes the average grayscale value across the image as a threshold value. The default global method which is a variation of the IsoData algorithm that divides the image into foreground and background pixels and iteratively tries different threshold values until finding one larger than the mean value (38). The Otsu global algorithm determines a threshold value that minimizes the variance in grayscale values within each class and maximizes variance between the classes (39).

Local thresholding

Local mean method selects the threshold as the mean of the local grayscale distribution (14). In local thresholding methods, the threshold for binarization is computed for each pixel according to the image characteristics within a window of radius r. The default r value in Image J is 15 pixels. This valued was not changed as we conclude that the most of the authors from previous CC quantitative studies left this value unchanged, according to what was originally reported by Spaide (40). Niblack’s thresholding method is the oldest local binarization method found in the literature. In this method, the estimation of a threshold value is based on the calculation of local mean and standard deviation of pixels value in a local window confined to an image (41). The Phansalkar algorithm incorporates the mean and standard deviations of the grayscale values in the local area. It was designed for images with a variable appearance and particularly to optimize binarization thresholding in low contrast images (42).

Figure S1.

PRISMA 2009 flow diagram.

Table S1. Summary of the thresholding strategies found through the systematic review.

OCTA device related (n=40)	Customized by the author (n=37)	Unknown/inaccessible (n=5)	Global thresholding strategies				Local thresholding strategies
			G. Otsu (n=5)	G. Mean (n=14)	G. Default (n=6)		L. Phansalkar (n=27)	L. Mean/median (n=3)	L. Niblack (n=2)
Abbouda et al. [2018], (43)	Ahn et al. [2018], (44)	Al-Sheikh et al. [2017], (45)	Abroug et al. [2019], (46)	Borrelli et al. [2018], (47)	Cicinelli et al. [2017], (48)		Alagorie et al. [2019], (22)	Mehta et al. [2019]**, (33)	Wang et al. [2018], (49)
Agemy et al. [2015], (50)	Alten et al. [2016], (51)	Guduru et al. [2018], (52)	Al-Sheikh et al. [2017], (53)	Borrelli et al. [2017], (54)	Mehta et al. [2019]**, (33)		Borrelli et al. [2018], (55)	Tepelus et al. [2019], (56)	Kaur et al. [2019], (57)
Alabduljalil et al. [2019], (58)	Cakir et al. [2019], (59)	Qu et al. [2017], (60)	Mehta et al. [2019]**, (33)	Caplash et al. [2019], (61)	Murro et al. [2019], (62)		Borrelli et al. [2018], (63)	Mastropasqua et al. [2019], (64)
Augstburger et al. [2018], (65)	Camino et al. [2019], (66)	Yang et al. [2019], (67)	Nicolò et al. [2017], (68)	Carnevali et al. [2017], (69)	Battaglia Parodi et al. [2017], (70)		Borrelli et al. [2018], (71)
Ayhan et al. [2017], (72)	Chu et al. [2019]**, (23)	Yu et al. [2017], (73)	Rodrigues et al. [2019], (74)	Chu et al. [2018]**, (20)	Battaglia Parodi et al. [2018], (75)		Borrelli et al. [2019], (76)
Cao et al. [2018], (77)	Chu et al. [2018]**, (20)			Costanzo et al. [2019], (78)	Sakurada et al. [2020], (79)		Braun et al. [2019], (80)
Cennamo et al. [2019], (81)	Chu et al. [2018], (24)			Jauregui et al. [2018], (82)			Byon et al. [2019], (21)
Cennamo et al. [2020], (83)	Chua et al. [2019], (84)			Mastropasqua et al. [2019], (85)			Chanwimol et al. [2019], (86)
Chan et al. [2019], (87)	Fernández-Vigo et al. [2020], (88)			Mehta et al. [2019]**, (33)			Chu et al. [2019]**, (23)
Chao et al. [2019], (89)	Fernández-Vigo et al. [2020], (90)			Nassisi et al. [2017], (91)			Mastropasqua et al. [2019], (92)
Chun et al. [2019], (93)	Forte et al. [2019], (94)			Nesper et al. [2017], (95)			Mehta et al. [2019]**, (33)
Çömez et al. [2019], (96)	Hagag et al. [2020], (97)			Sacconi et al. [2019], (98)			Müller et al. [2018], (99)
Conti et al. [2019], (100)	Keiner et al. [2019], (101)			Treister et al. [2018], (102)			Nassisi et al. [2019], (103)
Conti et al. [2019], (104)	Lauermann et al. [2017], (105)			Wang et al. [2016], (106)			Nassisi et al. [2018], (107)
Demirel et al. [2019], (108)	Lee et al. [2019], (109)						Nassisi et al. [2019], (110)
Hikichi et al. [2019], (111)	Lim et al. [2018], (112)						Nassisi et al. [2019], (18)
Hua et al. [2020], (113)	Liu et al. [2019], (114)						Rochepeau et al. [2018], (115)
Jain et al. [2016], (116)	Matet et al. [2019], (117)						Sacconi et al. [2019], (118)
Karabulut et al. [2019], (119)	Montesano et al. [2017], (120)						Spaide [2018], (31)
Karaküçük et al. [2019], (121)	Moult et al. [2020], (122)						Spaide [2017], (123)
Khodabandeh et al. [2018], (124)	Moult et al. [2020], (125)						Spaide [2016], (40)
Kýlýnç et al. [2020], (126)	Nesper et al. [2017], (127)						Sugano et al. [2018], (128)
Lee et al. [2019], (129)	Oh et al. [2019], (130)						Tzaridis et al. [2019], (131)
Milani et al. [2018], (132)	Reich et al. [2019], (133)						Uchida et al. [2019], (134)
Mo et al. [2017], (135)	Reich et al. [2019], (136)						Uji et al. [2017], (137)
Pettenkofer et al. [2019], (138)	Rinella et al. [2019], (139)						Yun et al. [2019], (140)
Rispoli et al. [2018], (141)	Shi et al. [2020], (26)						Yun et al. [2020], (32)
Mastropasqua et al. [2017], (142)	Shin et al. [2018], (143)
Sarwar et al. [2018], (144)	Thulliez et al. [2019], (145)
Scherm et al. [2019], (146)	Vujosevic et al. [2018], (147)
Takayama et al. [2018], (148)	Vujosevic et al. [2020], (149)
Teng et al. [2017], (150)	Vujosevic et al. [2019], (151)
Toto et al. [2017], (152)	Yip et al. [2019], (153)
Tsai et al. [2018], (154)	Zhang et al. [2018], (19)
Tsen et al. [2019], (155)	Zhang et al. [2018], (6)
Urfalýoglu et al. [2019], (156)	Zhang et al. [2018], (157)
Wang et al. [2019], (158)	Zheng et al. [2018], (25)
Wang et al. [2019], (159)
Yang et al. [2019], (160)
Zhang et al. [2017], (161)

**, Studies that reported various strategies. G., global; L., local.

Ethical Statement: The study was approved by the Institutional Ethics Committee of Centro Hospitalar de Entre o Douro e Vouga (No. CA-0708/18-0t_MP/AC) and adhered to the tenets of the Declaration of Helsinki and its later amendments. Informed consent was obtained from participants before the inclusion in the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.