Optimizing 3D retinal vasculature imaging in diabetic retinopathy using registration and averaging of OCT-A

Abstract

High resolution visualization of optical coherence tomography (OCT) and OCT angiography (OCT-A) data is required to fully take advantage of the imaging modality’s three-dimensional nature. However, artifacts induced by patient motion often degrade OCT-A data quality. This is especially true for patients with deteriorated focal vision, such as those with diabetic retinopathy (DR). We propose a novel methodology for software-based OCT-A motion correction achieved through serial acquisition, volumetric registration, and averaging. Motion artifacts are removed via a multi-step 3D registration process, and visibility is significantly enhanced through volumetric averaging. We demonstrate that this method permits clear 3D visualization of retinal pathologies and their surrounding features, 3D visualization of inner retinal capillary connections, as well as reliable visualization of the choriocapillaris layer.

1. Introduction

Diabetic retinopathy (DR) is one of the most prevalent retinal vascular diseases worldwide. Affecting approximately one in three people with diabetes, an estimated 126 million people had the disease in 2010 [1]. As the prevalence of diabetes increases with population growth and societal factors, this figure is expected to reach 191 million by 2030, cementing DR as a global epidemic [2]. Ophthalmic angiography, the examination of retinal blood flow, has played a critical role in the understanding and care of the disease. Specifically, Fluorescein Angiography (FA) is routinely used as a method for diagnosing DR, as well as monitoring its progression. However, FA requires venipuncture and intravenous injection of a dye, and reports of adverse reactions ranging from minor (1:63) to severe (1:1900) and even death (1:222,000) have been documented [3,4]. Nevertheless, FA still remains as the gold standard in the analysis of DR features.

Optical coherence tomography angiography (OCT-A) is a recently established in vivo imaging technique that measures variation between OCT scans from the same transverse area to construct a volumetric angiogram [5 –7]. This non-invasive imaging method offers an alternative technique without the requirement of an exogenous contrast agent. An additional benefit is that OCT-A facilitates the study of the full retinal thickness, notably the intermediate and deep capillary plexuses (ICP and DCP, respectively), which together are often referred to as the deep retinal layers (DRL) or deep vascular complex (DVC) [8,9]. In contrast, conventional FA may only visualize the surface-level superficial capillary plexus (SCP). This is a crucial shortcoming of the current clinical standard, as there is evidence that pathological changes may occur in the DVC earlier than in the SCP, which may indicate the DVC as the layers of initial insult in DR [10 –12]. It has also been shown that metrics obtained from DVC analysis (such as vessel density decline) are more effective at disease stratification than their SCP counterparts [13]. Thus, the increased use of three-dimensional imaging modalities like OCT-A is required to further our understanding of retinal diseases like DR.

However, the clinical utility of OCT-A is constrained by several factors. For instance, studies have shown that for the detection of a key DR biomarker - microaneurysms (MAs) – the number of MAs detectable from single OCT-A en face images is significantly lower (less than 50%) than what is visible on corresponding FA images [14,15]. Additionally, the reduced light penetration into the deeper retinal layers results in poor vessel contrast of the DVC in single acquisition en face images. Another major limitation of clinical OCT-A is the issue of motion artifact. The acquisition of a single OCT-A volume requires that the patient has steady fixation during image acquisition (several seconds), whereas a single FA frame can be acquired in a fraction of a second. Any patient motion, especially microsaccadic motion, during the multi-second acquisition interval required for OCT-A can severely deteriorate the quality of the resulting data. Further compounding the problem is that patients with deteriorated focal vision, including those with DR, may struggle to fixate and thus may exhibit motion artifacts more frequently and with larger amplitude than their healthy counterparts [16,17]. Approaches exist for addressing this issue, including hardware-based motion tracking that is featured in many state-of-the-art commercial OCT-A systems [18 –22]. However, additional hardware may increase the cost and complexity of the system and may induce other subtle artifacts when the system cannot fully compensate for patient motion, which can manifest as small capillary distortions where some vessel segments appear non-continuous or broken [23,24].

Alternatively, other approaches have combined unique scanning patterns and post-acquisition software-based motion correction [25 –29]. A recent review article created a tabular summary comparing the existing strategies for approaching motion correction in OCT [30]. Of these, one strategy that shows promise is that of serial acquisition, registration and averaging of OCT-A images. This software-only approach is especially appealing as it does not require the addition of any motion-correction hardware. Specifically, it has been shown that multi-acquisition averaging of OCT-A images can not only remove motion artifacts, but also significantly improve image quality [31 –33]. However, these algorithms often rely upon the successful acquisition of at least one high quality, microsaccade-free template image that guides the registration and averaging of lower quality images. The requirement of obtaining a high quality, motion-free image is a significant barrier to realistic clinical application, as it is often difficult to obtain even one such image from patients with poor fixation. Indeed, while we were able to acquire a microsaccade-free template image from a normal subject imaged for this study, no entirely microsaccade-free volumes were found in our diabetic data set of nearly 100 volumes.

In this work, we propose a 3D registration and averaging methodology for OCT-A volumes, which simultaneously addresses both issues of weak vessel contrast in OCT-A images of the deeper retinal layers as well as motion correction. We have improved upon our original multi-acquisition registration and averaging algorithm to free it from the requirement of a microsaccade-free template image by automatically stitching together small, microsaccade-free sub-images [33]. An important feature of this improvement is that the field of view of the averaged output image is not limited to the largest microsaccade-free field of view of the input image set. Rather, by precisely identifying overlapping regions between volumes and stitching them together, microsaccade-free images of the full field of view can be produced without having to repeatedly image a patient in the hopes of eventually acquiring a microsaccade-free template.

We have also expanded the algorithm to operate volumetrically rather than on a discrete number of layer-based en face images, thus facilitating the production of exquisite 3D renderings of the retinal microvasculature. Through the volumetric approach, we also demonstrate an enhanced ability to visualize the structures interconnecting the superficial, intermediate, and deep vascular plexuses, the series and parallel organization of which have been established as playing a role in the stratification of DR [11,12]. Finally, we demonstrate that our approach is capable of reliably visualizing the choriocapillaris of the outer retina in addition to the microvasculature of the inner retina, the importance of which has been established in numerous studies [34 –42]. Visualization of the choriocapillaris is of particular interest in the context of DR, as it has been established that choriocapillaris perfusion is negatively impacted in diabetic patients [43 –50]. We present these results qualitatively in DR patients to emphasize that this is a clinically viable methodology that can facilitate enhanced visualization of retinal disease pathophysiology.

2. Methods

All subject recruitment and imaging took place at the Lions Eye Institute in Perth, Western Australia. The project protocol was approved by the Human Ethics Committee at the University of Western Australia and performed in accordance with the tenets of the Declaration of Helsinki. Written informed consent was obtained from all subjects.

2.1. Optical coherence tomography angiography instrumentation

The system used in this study was a custom-built swept source OCT (SS-OCT) system described previously [51]. Briefly, the SS-OCT system used a 200 kHz swept source (Axsun Technology Inc., MA) with a center wavelength of 1.06 um and a full width half maximum (FWHM) of 80 nm. The parameters of the light source, 500 MHz sampling frequency of the spectral interference signal and spectral windowing define the measured depth-resolution of 8.7 µm in air, corresponding to 6.3 µm in tissue (n=1.38). The beam diameter incident on the cornea and spot size at a retina were around 1.6 mm and 13 µm, respectively. The optical power on the cornea is configured to be below 1.5 mW in order to satisfy the safety standard defined by ANSI Z136.1 (2014) safety limits [52]. The transversal area of ∼2 mm (horizontal) × ∼2 mm (vertical) was scanned with 400 × 800 A-scans in 1.6 seconds. In this scanning protocol, two B-scans were taken at a single location using step bidirectional scanning in which the slow scan axis is stepped at the B-scan frequency to obtain a constant time interval between the two B-scans [51]. The OCT-A clinical prototype used in this study did not possess hardware-based motion tracking capabilities. During the image acquisition period, patients were asked to maintain their gaze on an external fixation target and encouraged to blink outside the brief acquisition interval to prevent drying of the cornea.

2.2. OCT angiography data acquisition and pre-processing

Twenty OCT-A volumes were acquired sequentially per eye. The acquisition time was 1.6 seconds per volume. The OCT-A data was acquired using a stepped bidirectional scanning pattern with two repeated B-scans per BM scan, or 2BM. ‘BM’ denotes that multiple B-scans are acquired at each B-scan location [53]. This stepped OCT-A scanning pattern differs from conventional raster and bidirectional scans, in that it allows for each B-scan within a BM scan to be acquired in the same direction, without fly-back, and with equal time intervals between corresponding A-scans [51]. Briefly, after the initial forward sweep of the beam to collect a B-scan at the first location, the position is changed to the next location to collect another B-scan on the backward sweep. The beam is then repositioned at the original location to collect repeated B-scans in these two locations following this forward-backward collection pattern until the desired number of BM scans is acquired [51]. This scanning protocol facilitates increased blood flow sensitivity by adjusting the time interval and reducing acquisition time compared to raster and bidirectional scan patterns. While higher BM values (i.e., 3BM, 4BM, 5BM) would further increase flow sensitivity and vessel contrast, the longer scan times significantly increase susceptibility to motion artifact. To that end, the use of serial acquisition and averaging post-acquisition mitigates the lower flow sensitivity of the 2BM approach.

Following acquisition, standard SS-OCT processes such as wavenumber resampling, numerical dispersion compensation, and FFT was performed on the acquired raw OCT data. Prior to the 3D registration algorithm, coarse axial motion artifact was corrected using cross-correlation between adjacent B-scans. The OCT-A microvascular signal was extracted from the complex OCT data via an absolute complex difference OCT-A technique [54]. Three-dimensional bounded variance smoothing was applied to the motion-corrected OCT intensity B-scans to enhance the boundaries between retinal layers and aid layer segmentation [55,56]. The posterior boundary of the outer nuclear layer (ONL) was automatically segmented from the smoothed OCT data in 3-D using a graph-cut algorithm [57 –60]. The OCT intensity, OCT-A, and ONL segmentation data were then passed as inputs into the 3D registration and averaging algorithm.

2.3. Three-dimensional registration and averaging pipeline

The algorithm overview is shown in Fig. 1. The structural OCT and OCT-A data were processed through the pipeline concurrently, but all transformation matrices were derived from the OCT-A data only. The pipeline is an extension of our previously published template-based 2D en face OCT-A registration algorithm [33] and was implemented in MATLAB (2019b; MathWorks). This novel 3D algorithm is template-less, such that a microsaccade-free ‘template’ image is not required for registration. Instead, the template-less algorithm proceeds iteratively, wherein one ‘moving’ slab is registered to a ‘target’ slab at each iteration, and the two registered slabs are combined to create a larger target for the subsequent iteration. Hence, where our original algorithm could produce an output image with a field of view no larger than that of the largest microsaccade-free strip, this algorithm can stitch together a final image with a field of view much larger than the microsaccade-free regions of the input data. It should be noted that if a microsaccade-free template exists in a data set, it is automatically selected as the initial registration target. However, no such templates were found in this diabetic data set of nearly 100 volumes.

Fig. 1.

Algorithmic flowchart of the 3D registration and averaging pipeline.

Briefly, the pipeline begins with B-scan flattening of all volumes with respect to the ONL segmentation, which also serves as the axial registration step. The flattened volumes were then divided into microsaccade-free volumetric slabs. The largest slab is identified and used as the initial reference for registration, or ‘target’ slab. At each iteration, the slab with the most overlap with the target slab is identified via a maximum cross-correlation calculation and used as the ‘moving’ slab for that round. As shown in Fig. 1, each slab is registered once axially, then three times C-scan-wise in increasing order of precision: first translationally via cross-correlation; then rigidly using a feature-based affine transformation; and finally, non-rigidly by pixel-wise local neighborhood matching to correct for finer deformations. C-scans in this work refer to single en face depth slices of a volume. A graphic illustration of this pipeline is presented in Fig. 2.

Fig. 2.

Graphic illustration of the 3D registration and averaging pipeline, which reconstructs full field-of-view images from microsaccade-free slabs (see Visualization 1 ^{(249.1KB, mp4)} ).

2.3.1. Axial registration and flattening with respect to outer nuclear layer

To expand our previously published algorithm from two to three dimensions, a more comprehensive axial registration step (i.e., B-scan registration) between volumes was required in addition to intra-volume axial registration and lateral registration (i.e., en face registration) to fully align the volumetric data in three-dimensional space. First, the ONL segmentations for each volume were smoothed to fit a low-order polynomial to the outer-retinal curvature. C-scans were then extracted with respect to the curvature of the smoothed ONL boundary and stacked, effectively generating flattened volumes spanning the full retinal thickness, which were aligned axially in space.

2.3.2. Microsaccade-free slab generation

In OCT-A en face images, microsaccadic motion manifests as bright white stripes parallel to the fast scan axis, as shown in Fig. 2. To detect these, mean 2D projections of the inner retina (ILM to ONL) of each volume were produced and artifacts were automatically identified by measuring intensity peaks along the slow-scan axis. The pixel location of the intensity peak as well as 3 pixels on either side were removed to ensure full deletion of the artifact. The regions between the artifacts were extracted as microsaccade-free slabs. The largest slab was chosen as the initial registration target. An optimal registration order for the remaining slabs was automatically derived by identifying the slab with the maximum cross-correlation value with the target slab at each iteration.

2.3.3. Slab-based affine registration

The third registration step of the algorithm is a feature-based affine registration. Affine registration transforms an image via scaling, translation, shear, and rotation. However, parallel lines remain parallel after an affine transform. This makes the affine transform an ideal precursor to non-rigid registration, in which localized deformations can badly distort vascular structures if they are not already roughly aligned. The mean projections of the inner retina (ILM to ONL) from the previous step were used as feature-rich images from which to derive deformation matrices. The 2D deformation maps were then applied to each C-scan of the volume.

To perform this feature-based affine registration, scale invariant feature transform (SIFT) keypoints were extracted from both the target and moving slabs at each iteration [61]. As the SIFT feature descriptor is invariant to uniform scaling and orientation, it is ideal for identifying matching features in noisy or speckled images such as OCT-A. If a minimum of four matched keypoints were identified in the 2D mean projections of the target and moving OCT-A slabs, the matching keypoints pairs were used to derive an affine transformation matrix. The matrix was applied to each C-scan of the moving slab, thus transforming all en face layers of the slab equally. A more detailed description of how keypoints are selected can be found in our previously published template-based algorithm [33], and a full description of the SIFT algorithm can be found in the original algorithm publication [61].

2.3.4. Slab-based non-rigid registration

The slab-based affine registration described in the previous section corrects for microsaccadic motion artifacts marked by bright white lines such as those shown in Fig. 2. However, localized deformations caused by smoother tremor and drift motion still remain in the aligned slabs. The next step of the algorithm accounts for these deformations by using our previously published non-rigid registration algorithm [33]. Briefly, a 2D averaging filter consisting of 2 × 2 pixels was applied to both the target slab and affine-registered moving slab to smooth any fine speckle that may affect the non-rigid registration. For each pixel in the 2D projection of the moving slab, the normalized cross-correlation was calculated as:

$$xcor{r}_{norm}\left(s,t\right)=\frac{{\sum }_{x,y}\left[f\left(x,y\right)-\overline{f\left(s,t\right)}\right]\left[m\left(x-s,y-t\right)-\overline{m}\right]}{\sqrt{{\sum }_{x,y}{\left[f\left(x,y\right)-\overline{f_{s,t}}\right]}^2{\sum }_{x,y}{\left[m\left(x-s,y-t\right)-\overline{m}\right]}^2}}$$ (1)

where $f(x,y)$ is the 29 × 29 pixel matrix field centered on the $(x,y)$ pixel of the target slab image, $\overline{f(s,t)}$ is the mean of the moving slab image in the region under the mask, m is the 15 × 15 pixels mask matrix centered on the pixel of the moving slab image, and $\overline{m}$ is the mean of the mask. This was repeated with the matrix rotated from -15 to 15 degrees in 5-degree increments. The pixel located at the index of the maximum normalized cross-correlation was then used as the registered pixel for the moving slab.

Upon completion of this final registration step, the moving slab is fully registered to the target slab. The next step of the iterative process is to combine the registered slabs so that they form a larger target for the subsequent iteration of the algorithm. This is done by averaging the target and moving slabs at each iteration. Visualization 1 ^{(249.1KB, mp4)} illustrates how this process reconstructs the full foveal avascular zone (FAZ) field of view from the registered microsaccade-free slabs.

2.3.5. Quantitative validation of image improvement

Improvement in image quality was quantified via evaluation of two image quality metrics, namely: the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) [62 –65]. The SNR is defined as:

$$SNR=10\text{log}\frac{max(X_{lin})}{{{\sigma }_{lin}}^2}$$ (2)

where $\text{max}(X_{lin})$ is the maximum of the matrix of pixel values in the angiogram on a linear intensity scale and ${{\sigma }_{lin}}^2$ is the noise variance on a linear intensity scale.

The CNR is defined as:

$$CNR=\frac{{\mu }_s-{\mu }_b}{\sqrt{{{\sigma }_s}^2+{{\sigma }_b}^2}}$$ (3)

where ${\mu }_s$ is the mean value of the signal image, ${\mu }_b$ is the mean value of the noise image, ${{\sigma }_s}^2$ is the variance of the signal image, and ${{\sigma }_b}^2$ is the variance of the noise image.

To facilitate the quantification of these metrics, the vasculature of the 2D inner retina projections of each eye were segmented by a deep neural network (DNN) before and after averaging [66]. These vessel segmentations were used to isolate vascular signal from background noise, which are necessary to compute SNR and CNR. For each input image, the DNN outputs a grayscale image with the same dimensions as the input image, where each output pixel is a value between 0 and 1, corresponding to the network’s confidence that said pixel belongs to a blood vessel, as illustrated in Fig. 3. Hence, any DNN output pixel with a value greater than 0.5 denotes a pixel which the DNN identified as having above 50% likelihood of belonging to a blood vessel, and any pixel less than 0.5 in value is less than 50% likely to belong to a blood vessel. An example of how this DNN segmentation process is used to delineate signal from noise is shown in Fig. 3. The DNN used in this study was trained on single frame Zeiss Plex Elite images at 3 × 3mm scale with both control and diabetic patients as in [66]. For a detailed exploration of the DNN architecture employed in this study, readers are encouraged to refer to [66].

Fig. 3.

Illustration of how a vessel-segmentation DNN [66] is used to facilitate the quantification of SNR and CNR image quality metrics before and after averaging. *The noise images have intentionally been made brighter in this report to ease visualization for the reader.

The signal and noise were extracted from the angiogram as follows. The DNN output probability maps were thresholded at a value of 0.5, which denotes the boundary at which the DNN has 50% confidence that a given pixel belongs to a blood vessel. Examples of these binarized probability maps are shown in the ‘DNN Output’ column of Fig. 3. The masks were then pixel-wise multiplied with their respective input images to generate distinct signal and noise images, which were then used to calculate SNR and CNR as defined above. SNR and CNR were calculated for each inner retinal vascular plexus of each DR eye before and after averaging.

3. Results

Data from a total of six eyes from four volunteers were acquired according to the imaging protocol described in Section 2.2. Five eyes from three volunteers (2 males, 1 female) were affected with DR two eyes with moderate non-proliferative DR (NPDR), and three eyes with proliferative DR (PDR). One normal eye from one healthy male volunteer was also imaged.

3.1. Quantitative validation results

Quantification of image enhancement as a result of registration and averaging was evaluated via the two metrics described in Section 2.3.5, namely: SNR (Table 1), and CNR (Table 2). All increases in validation metrics were statistically significant (p < 0.01) using a paired t-test.

Table 1. Summary of SNR evaluation from single to averaged OCT-A inner-retina images.

Table 2. Summary of CNR evaluation from single to averaged OCT-A inner-retina images.

3.2. Volumetric averaging of OCT and OCT-A

A comparison between single frame and averaged OCT and OCT-A data from a patient with moderate non-proliferative diabetic retinopathy (NPDR) is shown in Fig. 4. In the single frame OCT-A en face images, there are two prominent microsaccadic motion artifacts, which manifest as bright white lines and correspond to the point when the patient lost fixation during the 1.6 second scan interval.

Fig. 4.

Illustration of the impact of serial acquisition and 3D averaging applied to OCT-A data acquired from a patient with moderate non-proliferative diabetic retinopathy. (A, B, C) Single frame OCT-A 2D projections of the superficial, intermediate, and deep vascular plexuses, respectively. (D, E, F) Averaged images of 20 serially acquired and registered volumes of the superficial, intermediate, and deep vascular plexuses, respectively. (G, H) Single angiographic OCT-A and structural OCT B-scans, respectively, extracted from the FAZ region of the same patient. (I, J) 3D-averaged angiographic OCT-A and structural OCT B-scans, respectively.

The second row of Fig. 4 shows the averaged images of each inner retinal vascular plexus after full registration of the 20 serially acquired volumes. The improvement in vessel contrast is particularly marked in the intermediate and deep vascular plexuses, where there is necessarily a reduced signal-to-noise ratio due to worse light penetration in the deeper layers compared with the superficial layers. In these deeper layers, the single frame OCT-A images show weak contrast and discontinuous vessels, whereas the averaged images show clear and continuous vessel networks, and the distinct vessel morphology of each layer is made readily visible as a result of this averaging process. Finally, the third and fourth rows of Fig. 4 show a single OCT and OCT-A B-scan, and the corresponding averaged B-scans. Features, such as vessels, are much more clearly defined in the averaged B-scans.

3.3. 3D visualization of averaged OCT-A

Figure 5 illustrates the impact of this volumetric averaging process through the use of color-coded depth profiles and three-dimensional renderings. Several works have demonstrated the usefulness of encoding depth in OCT images with color profiles [6,67 –71]. Importantly, this tool facilitates not only the clear visualization of the three individual vascular plexuses, but also the regions and structures connecting them. This is a critical advancement in retinal visualization, as it is theorized that the interconnections between these layers and the branching patterns of the capillaries therein may play a role in the pathogenesis of retinal diseases including DR [11,12].

Fig. 5.

(A-C) Fluorescein angiogram (zoomed and cropped), averaged OCT-A, and single OCT-A, respectively, acquired from a patient with PDR. Fluorescein angiogram was acquired 7 months prior to OCT-A. (D) Three 3D views of an inter-layer capillary branch pattern. The yellow arrow indicates a superficial capillary branch that remains in the superficial vascular plexus, and the white arrow indicates a branch that diverges into the deep vascular plexus. A 3D rendering of the full field of view extracted to (D) is shown in Visualization 2 ^{(30.1MB, mp4)} .

Figure 5 also demonstrates the disparity in data quality between the clinical gold standard, i.e., fluorescein angiography, and that which is attainable through the advanced OCT-A methods described herein. Figure 5(C) illustrates how even a single frame 2D OCT-A image affords superior detail of the retinal microvasculature than the corresponding FA (although it should be noted that the original FA image is much larger and was zoomed-in and cropped to match the same region represented in the 2×2mm OCT-A). Figure 5(B) displays a 2D mean projection of the volumetrically averaged OCT-A and color-coded along the depth axis. This image, even while restricted to the 2D plane, provides valuable information on the interactions between the superficial (blue), intermediate (green), and deep (red) vascular plexuses with superior resolution than the single frame OCT-A, and without the need of a sophisticated 3D rendering tool. However, if such 3D tools are available, importing the same volume represented in Fig. 5(B) into a 3D viewer allows for three-dimensional visualization of the microvasculature such as that shown in Fig. 5(D). A 3D rendering of the full field of view is shown as a video in Visualization 2 ^{(30.1MB, mp4)} .

The three-dimensional morphology of DR biomarkers and their surrounding areas are of particular interest in DR pathology research. Figure 6 demonstrates how results from the pipeline can facilitate 3D analysis of key regions of interest. A region containing several potential microaneurysms and the surrounding microvasculature is shown in 2D and 3D OCT-A and held in comparison against the corresponding FA. In particular, abnormal blind-end capillaries are accompanied by a notable avascular region in the averaged OCT-A, whereas it is difficult to discern these features in the FA.

Fig. 6.

(A-C) Fluorescein angiogram (zoomed and cropped), averaged OCT-A, and single OCT-A, respectively, acquired from a patient with PDR. Fluorescein angiogram was acquired on the same day as the OCT-A. (D) Three 3D views of a region containing several microaneurysms and the surrounding microvascular network. The yellow dashed circle highlights that the sub-region containing these microaneurysms also features an avascular zone lacking normal vasculature.

3.4. Choriocapillaris imaging through 3D averaging

The results presented thus far have been limited to the inner retina. However, despite deriving feature-based transformations from the inner retina only, the algorithm extends these transformations to the full retinal thickness of the co-registered OCT and OCT-A volumes. The result is the ability to produce high quality images of the outer retina as well. Figure 7 demonstrates this capability through the visualization of the choriocapillaris. Significant effort has been made in the attempt to reliably visualize and quantify the features of the very thin choriocapillaris layer, driven by the belief that it may play a role in the pathogenesis of several retinal diseases [44 –50]. Through this work, we demonstrate that we can reliably visualize the choriocapillaris in healthy and diseased patients, and can do so using post-acquisition software methods, as opposed to optimizing the OCT-A hardware for this specific imaging purpose.

Fig. 7.

Illustration of the algorithm’s ability to extract OCT-A images of the choriocapillaris and correlate with inner retina features. Data acquired from a patient with PDR. (A-C) ∼1 × 1 mm SCP, choriocapillaris, and DVC images, respectively. (D-F) ∼2 × 2 mm SCP, Choriocapillaris, and DVC images, respectively, acquired from a separate imaging session as those in the row above. (G,H) OCT and OCT-A averaged B-scans, respectively, from the location indicated in (D). (I) FA, zoomed and cropped, acquired on the same day as the OCT-A. (J,K) OCT and OCT-A averaged B-scans, respectively, from the location indicated in (F). Red and purple arrows denote two groups of DR disease features localized to their three-dimensional location using averaged en face and orthogonal views. Pink dashed boxes denote regions of apparent signal attenuation in the choriocapillaris layer.

Two notes should be made about the results presented in Fig. 7. First, the top and middle image rows were generated from data sets acquired in two separate imaging sessions which were then processed through the 3D registration pipeline independently. Hence, the qualitative consistency of the retinal features seen in both image sets is a testament to the robustness of the registration algorithm. Second, the region of the choriocapillaris image outlined in the dashed pink rectangle would, on its own, appear to signify a region of choriocapillaris vessel dropout. However, the ability to correlate these features with the B-scan dimension allows us to see that this dark choriocapillaris region is instead likely a shadow artifact induced by the hyperreflective DR features within the inner retina directly above, denoted by the red and purple arrows.

4. Discussion

This study has demonstrated the following benefits of using a serial acquisition, 3D template-less registration and averaging OCT-A in vivo imaging paradigm of the human retina: 1) the template-less approach facilitates the implementation of an imaging paradigm without requiring the presence of a microsaccade-free template image, which is exceedingly difficult to obtain from patients with deteriorated focal vision; 2) significantly enhanced vessel contrast of the macular microvascular network compared to single frame OCT-A images and traditional clinical imaging such as fluorescein angiography; 3) facilitates three-dimensional analysis of the macular microvascular network, including clear visualization of the structures interconnecting the superficial, intermediate, and deep vascular plexuses; 4) facilitates three-dimensional analysis of key DR features and their surrounding microvasculature networks; and 5) ability to extract high-resolution images of the choriocapillaris via extension of feature-based transformation matrices derived from the inner retina. The increase in OCT-A image quality has been quantified through SNR and CNR image metrics, facilitated by vessel segmentations produced by a DNN.

High resolution images of the macular microvasculature are an important asset for clinicians and researchers. However, single frame OCT-A images of the macula are often of insufficient quality to permit meaningful analysis because of 1) low vessel contrast, especially in the DVC, and 2) motion artifacts. We have demonstrated the capability of a novel 3D registration and averaging algorithm to correct for both of these issues. The volumetric nature of the algorithm, taken with the fact that it simultaneously processes structural OCT and OCT-A data, produces high-resolution images of the OCT and OCT-A B-scans in addition to en face images. This is a valuable asset for ophthalmic clinicians who may want to correlate disease features seen in the structural OCT volume with the angiogram, as demonstrated in Fig. 7.

The volumetric approach also facilitates exquisite 3D visualizations and manipulation of the averaged volumes. We have demonstrated this qualitatively in Fig. 5, Fig. 6, and Visualization 2 ^{(30.1MB, mp4)} , wherein capillary branching patterns between the SCP and DVC can be seen clearly from different viewing angles. These renderings also illustrate the algorithm’s ability to localize DR features to their depth locations in the retina with high clarity, including neovascularization that is characteristic of PDR, regions of capillary non-perfusion, MAs and hard exudates. The work performed in this study demonstrates that through template-less registration and averaging, the signal and contrast of 3D OCTA images is sufficiently improved to facilitate similar analyses in vivo as done in donor eyes ex vivo [11,12].

This work has demonstrated how key DR features, including microaneurysms and altered capillary branching patterns, are more clearly visualized as a result of the averaging process. The importance of these DR biomarkers and their detection via OCT-A has been reported in numerous studies [72 –74] In particular, DR is characterized by microaneurysms (MAs), capillary non-perfusion and ischemia. Of these, MAs have been found to be one of the key biomarkers used for DR analysis and staging and contribute to DR progression themselves [75]. However, it has been shown that the number of MAs detectable from single OCT-A images is up to 50% less than that detectable from corresponding FAs [14,15]. It is likely that serial acquisition and averaging of OCT-A images can greatly mitigate this issue. Using the methodology described in this work, we are currently investigating the change in sensitivity of MA detection as a result of applying this 3D averaging process.

Additionally, we have demonstrated that the volumetric registration approach also facilitates image production of layers under the outer retina, notably the choriocapillaris, as shown in Fig. 7. While our presentation of choriocapillaris data in this work is limited to qualitative observations, our ability to produce images of this crucial choroidal layer without an imaging paradigm specifically designed to extract it, and by using image processing transformations identical to those used for the inner retina, is an encouraging step towards making choriocapillaris analysis a clinically realistic tool for investigating DR pathogenesis. Indeed, using this methodology, we are actively acquiring more DR data of this kind so that we may make statistically significant quantitative comments on the role of the choriocapillaris in DR progression in the near future.

While the methodology presented herein has yielded promising results in the pursuit of high resolution, three-dimensional visualization of the living human retina, we acknowledge several limitations of this work. First, the small number of diabetic eyes imaged does not facilitate any quantification that would permit statistical insights into the disease. Future work will use this tool to quantify, among other factors, the prevalence and morphology of microaneurysms and neovascularization in 3D. Second, this work focused only on a small field of view centered on the FAZ. However, given that the template-less algorithm iteratively stitches together small fields of view to create larger ones, this methodology may facilitate the creation of montage-like fields of view that are much larger than that of a single volume. Follow up studies should investigate the ability of this algorithm to produce similar results at larger fields of view, and at different retinal locations such as the peripapillary region. Third, since our feature-based registration algorithm uses only the microvasculature signal to guide the registration, the region within the FAZ remains mostly unregistered due to its avascular nature. This is evident in the choriocapillaris image shown in Fig. 7. Improvements to the registration algorithm should be made to address this issue, potentially by incorporating the structural OCT signal to guide the registration in addition to the OCT-A signal.

Fourth, our work has emphasized microsaccadic motion as the primary source of deleterious motion artifact in OCT-A images. However, we acknowledge that more subtle sources of motion artifact may persist after processing through our algorithm, including slow drift motion, which can distort the appearance of the microvasculature and induce erroneous registration. Several works have focused on quantifying and removing this type of motion artifact [76,77]. While we believe that the non-rigid-registration step described herein adequately accounts for major drift motion, future work should include a motion-correction step which explicitly quantifies and compensates for subtle drift motion. Fifth, the graph-cut layer segmentation tools used in this study for flattening and axial registration are sometimes unable to reliably segment diseased retinas, particularly in the presence of diabetic macular edema. The layer segmentation tool should be improved, or a new one created altogether, to account for abnormal retinal morphology. Finally, some projection artifacts of superficial vessels still remain in the final averaged volumes which sometimes obscure the deeper vasculature. Future work should incorporate 3D projection artifact compensation, such as projection-resolved OCT-A [78], to further improve the appearance of 3D renderings of the retinal microvasculature.

5. Conclusion

OCT-Angiography is a recently established, non-invasive tool used for in vivo 3D imaging of the human retina, offering exquisite resolution of the retinal microvasculature superior than traditional 2D Fluorescein Angiography. However, single-frame OCT-A images are often inadequate to visualize the deeper retinal layers like the choriocapillaris, and a relatively longer image acquisition time induces motion artifacts which can severely degrade the resulting data. This work has addressed these issues by iterating on our previous work to create a serial acquisition, three-dimensional registration and averaging methodology for visualizing the human retina. The registered and averaged volumetric data offers significantly enhanced 3D visualization of the human microvascular network. We have validated that this methodology is clinically realistic through application to macular data acquired from patients with DR in a clinical environment. Future work will apply this tool on larger data sets to extract statistically significant conclusions about DR, and the work will be further extended to include analysis of other debilitating retinal pathologies.

Funding

National Health and Medical Research Council10.13039/501100000925; Mitacs10.13039/501100004489; Canadian Institutes of Health Research10.13039/501100000024; Natural Sciences and Engineering Research Council of Canada10.13039/501100000038.

Disclosures

MVS: Seymour Vision (I).