Automated Deformation-Based Analysis of 3D Optical Coherence Tomography in Diabetic Retinopathy

Abstract

Diabetic retinopathy (DR) is a significant microvascular complication of diabetes mellitus and a leading cause of vision impairment in working age adults. Optical coherence tomography (OCT) is a routinely used clinical tool to observe retinal structural and thickness alterations in DR. Pathological changes that alter the normal anatomy of the retina, such as intraretinal edema, pose great challenges for conventional layer-based analysis of OCT images. We present an alternative approach for the automated analysis of OCT volumes in DR research based on nonlinear registration. In our work, we first obtain an anatomically consistent volume of interest (VOI) in different OCT images via carefully designed masking and affine registration. After that, efficient B-spline transformations are computed using stochastic gradient descent optimization. Using the OCT volumes of normal controls, for which layer-based segmentation works well, we demonstrate the accuracy of our registration-based analysis in aligning layer boundaries. By nonlinearly registering the OCT volumes of DR subjects to an atlas constructed from normal controls and measuring the Jacobian determinant of the deformation, we can simultaneously visualize tissue contraction and expansion due to DR pathology. Tensor-based morphometry (TBM) can also be performed for quantitative analysis of local structural changes. In our experimental results, we apply our method to a dataset of 105 subjects and demonstrate that volumetric OCT registration and TBM analysis can successfully detect local retinal structural alterations due to DR.

I. Introduction

DIABETIC retinopathy (DR) is a significant microvascular complication of diabetes mellitus (DM) and a leading cause of visual impairment in the developed world [1], [2]. DR-related vision impairment is expected to remain a major health concern since the prevalence of diabetes is projected to increase from 14% in 2010 to 21% in 2050 [3] and the lifetime prevalence of DR in subjects with DM is well over 50% [1]. Current approach to prevent vision loss in DR requires early diagnosis, frequent monitoring and timely treatment [1]. Nevertheless, the challenge of identifying subjects at risk of DR development and vision impairment remains open.

Optical coherence tomography (OCT) is a non-invasive 3D imaging technique for high resolution visualization of retinal layers [4], [5]. The high resolution allows visualization of normal retinal anatomy [6], [7] and detection of microscopic retinal changes due to systemic and vision threatening diseases such as diabetes [8], glaucoma [9], age-related macular degeneration [10], and even neurodegenerative diseases [11]. In DR, OCT is routinely used to identify the presence or absence of microscopic abnormalities such as retinal thickening and thinning from intraretinal edema [12]. Indeed, numerous methods have been proposed for segmentation of retinal layers [13]–[17], and their performance have been significantly improved recently [18]. However, locating layer boundaries in DR pathologies such as diabetic macular edema which interrupts layer integrity may still be unreliable [8]. Hence, there is a need for techniques that allow analysis of retinal abnormalities in DR, independently of layer segmentation.

OCT registration has been performed for studying retinal diseases [19]–[21], evaluating treatment efficiency [20], assisting with layer segmentation [22], [23], motion correction [24], noise compensation [25], and analysis of same subject longitudinal data [26]. To our knowledge, three methods have focused explicitly on cross-subject OCT registration. Non-rigid registration of surfaces extracted from OCT volumes has been reported [27]. In a similar study, retinal layers were segmented and a method for surface-based registration of the segmented layers was presented [20]. Finally, Chen et al, presented intensity-based registration of macular OCT using initial rigid followed by nonlinear transformation along each A-scan using 1D radial function [28]. The latter is the only study to our knowledge presenting a method for registration of entire OCT volume without segmentation of individual layers. However, their technique cannot register tilted OCT and is not suitable in the presence of DR abnormalities such as edema.

The limited number of techniques available for cross-subject OCT registration in DR research can be attributed to a number of challenges. Foveal pit and layer boundaries are typically the only consistent landmarks across macular OCT of different subjects. Also, foveal shape alteration and disruption in layer integrity due to DR pathology such as edema add difficulty to the registration. The presence of retinal vessels and their shadows, which have little correspondence across subjects, pose additional registration challenge. OCT images also suffer from low signal-to-noise ratio (SNR) and intensity inhomogeneity [28]. Moreover, the distance between B-scans is usually larger than within B-scans resolution. As a result, the OCT data is typically highly anisotropic in terms of spatial resolution. Finally, OCT registration requires extensive computation due to high resolution and large image size [28].

In this work, we develop a systematic approach to address the challenges in OCT registration for DR research. Our method builds upon the publicly available Elastix framework [29], to perform automated and efficient cross-subject OCT registration. As a first step in our method, we apply effective denoising and carefully optimize an affine transformation customized for foveal OCT registration, which results in a properly constructed volume of interest (VOI) to perform meaningful registration between OCT images. After that, we compute a nonlinear deformation to achieve robust OCT registration across subjects. For efficient calculation, we adopt the stochastic gradient descent method in Elastix to minimize a normalized mutual information cost function under the constraint of the VOI. By combining the affine transformation and nonlinear warp, we obtain the overall transformation between OCT volumes. We demonstrate the accuracy of our registration-based method by evaluating its ability in aligning retinal layers. Using the macular OCT of a cohort of normal controls, we iteratively apply the nonlinear registration process and construct an OCT atlas of the macula for population studies. By registering each OCT volume to the atlas volume and computing the Jacobian determinant from the atlas for each subject, we can perform tensor-based morphometry (TBM) for comparison of retinal structure in different DR stages. TBM is a relatively new and automated technique for quantification of anatomical differences in medical images across populations [30]. It has been extensively used in neuroimaging to detect and characterize disorders such as Alzheimer's disease [31], and schizophrenia [32]. However, there is no previous evaluation of TBM for detection of systematic retinal structural changes in stages of DR. Our results suggest that registration-based analysis of OCT volumes has a potential for quantification of structural alterations in DR and contributes to anomaly detection, disease characterization and monitoring.

II. METHODS

A. Subjects and Imaging

The study was approved by the institutional review board of the University of Southern California and according to the Tenets of Declaration of Helsinki. OCT imaging was performed on the right eye of 105 subjects, namely normal control (NC, N=24), non-proliferative diabetic retinopathy (NPDR, N=57) and proliferative diabetic retinopathy (PDR, N=24). There is no significant age difference among the 3 groups (P=0.2, ANOVA). NPDR is characterized by the presence of edema, vessel leakage and restricted blood flow into the retina. In the more advanced stage of PDR, neovascularization also occurs which result in fragile new vessels that are prone to leakage. PDR complications frequently result in vision loss if left untreated. A typical volumetric OCT and B-scan for NC, NPDR and PDR subjects is shown in Fig. 1.

Fig. 1.

Example OCT volumes (left) and B-scans (right) in (a) normal control (NC) (b) non-proliferative diabetic retinopathy (NPDR) and (c) proliferative diabetic retinopathy (PDR) subjects.

Imaging was performed using a commercially available OCT instrument (Cirrus, Carl Zeiss Meditec, Inc. Dublin, CA). The volumes comprised of 245 B-scans (1024 A-scans) at a depth resolution of 2 μm. The macular scan was centered on the fovea and covered a retinal area of 3 mm × 3 mm. The instrument eye tracker was used to compensate for motion artifacts due to eye movements during image acquisition.

B. Pre-processing

One challenge in OCT registration is that the images are usually very noisy, which can affect numerical calculations in registration algorithm. Another challenge is that the OCT volumes contain regions with highly variable appearances such as the choroidal layer. Sometimes imaging artifacts could also occur in the vitreous anterior to the retina. Before we perform OCT registration, we thus perform two preprocessing steps: denoising and masking the retinal layers for registration.

We first apply the non-local means (NLM) filter [33] to reduce the speckle noise in the OCT images. The NLM compares patches across the image for pattern similarity and weighs them based on their similarity to the patch centered at the current voxel. The denoising process is then performed via a weighted average of these patches. While NLM is efficient in 2D, the computational burden is very high in 3D [34]. Block-wise NLM allows computationally tractable filtering without compromising the result [35]. In this approach, the volume is divided into overlapping blocks before performing NLM-like restoration of these blocks and finally restoring voxels based on the restored values of the block they occupy. It was shown previously that this process preserves anatomical detail while suppressing the noise [36], [37]. In the current study, the size of the search window was 3×3×3 in a restricted block (Ω) of size 32×32×32 within each OCT volume. For a voxel x_j with intensity value of U(x_j). w(x_i, x_j) is the weight assigned to U(x_j) in the restoration of voxel x_i. In block-wise NLM, the weights determine the similarity of local neighborhoods N_i and N_j for voxels x_i and x_j, respectively. Calculation of weights is shown in (1).

$$w(x_i,x_j)=\frac{1}{z_i}e\frac{\Vert u(N_i)-u(N_j){\Vert }_{2a}^2}{h^2}$$ (1)

where $\Vert .{\Vert }_{2a}^2$ is Gaussian-weighted Euclidean distance, z_i is a normalization constant to ensure ∑_jw(x_i, x_j) = 1, and h is a smoothing constant to control the decay of the exponential function which is set to 0.05. An example B-scan before and after denoising is shown in Fig. 2(a) and 2(b), respectively. As can be seen, the NLM filter reduced the speckle noise while preserved the detailed structure of the tissue.

Fig. 2.

Example OCT volume preprocessing shown on a selected B-scan. (a) Raw OCT of a NC subject. (b) Denoised image using NLM. (c) Retinal tissue mask by combination of layers segmented by OCTExplorer. The red lines indicate boundary of the mask and the white arrow indicates the choroidal layer which lies outside the mask. (d) Denoised and masked OCT which will be used for registration.

Both the vitreous region and the choroidal layer were eliminated by a mask obtained from combining automatically segmented retinal layers using the software tool OCTExplorer [38]. We found that the inner most and outermost layer boundaries of the whole retina provides a robust mask of the tissue for our purposes and avoided any intraretinal segmentation errors. Fig. 2(c) shows an example of the mask outlined by red lines on a normal B-scan and Fig. 2(d) shows the same B-scan after applying the mask. This denoised and masked volume will be used for image registration.

C. Registration of OCT Volumes

Overall our registration method is divided into two main steps. In the first step, we design an affine registration approach that not only aligns the OCT volumes with tilted position but also provides an anatomically meaningful way of defining corresponding volume of interest (VOI) in the OCT images for cross-subject registration. In the second step, we compute the nonlinear warp within the properly defined VOIs between two OCT images. By combining the affine and nonlinear warp, we obtain the final transformation.

Affine registration is performed to align the tissue and fovea of a moving OCT volume with respect to a fixed OCT volume. One of the volumes from a healthy eye with the retinal tissue centered and not tilted will serve as the fixed volume and the rest of OCTs are considered as moving volumes. In each OCT volume, we denote the direction along the A-scan as the z-axis, and the other two directions as the x- and y-axis. A challenge for accurate affine registration of cross-subject OCT is to avoid unnecessary stretching along x- and y-axis. Along the z-axis, the intensity variation of cell layers provides distinctive features for registration. Along the x- and y-axis, foveal pit which is usually centered serves as a registration feature while distinctive features are often not available to guide registration near the edges of the image volume. This can result in unnecessary stretching if regular affine transformation is used. Particularly, when the foveal pit is not centered on the moving volume, no corresponding tissue is available close to one of the edges on the fixed volume. To address this issue, stretching along the x- and y-axis was not allowed in our method by adopting the affine model from [39]. This model parameterized the transformation by angle, shear, scale and translation into matrices (rather than a single affine matrix) as shown in (2):

$$T(x)=RGS(x-c)+t+c$$ (2)

with R, G and S being the rotation, shear and scaling metrices, respectively, c is the center of rotation, and t is the translation.

Normalized mutual information (NMI) is used as the cost function to determine the affine transformation for global alignment. NMI is suitable for both mono- and multi-modal registration [29], and is robust to intensity inhomogeneity. Mathematically the NMI is defined as:

$$NMI(T;I_F,I_M)=\frac{\sum _{fϵL_F}{p}_F(f)lo{g}_2{p}_F(f)+\sum _{mϵL_M}{p}_M(m;T)lo{g}_2{p}_M(m;T)}{\sum _{mϵL_M}\sum _{fϵL_F}p(f,m;T)lo{g}_{2}p(f,m;T)}$$ (3)

where T is the affine transformation we want to optimize, I_F and I_M are the fixed and moving images, L_F and L_M are sets of regularly spaced and empirically selected intensity bin centers (∣L_F∣ = ∣L_M∣ = 32), p_F and p_M are the marginal discrete probabilities of the fixed and moving volumes provided by the summation of p over m and f, respectively. Finally, p(f, m, T) is the discrete joint probability estimated using B-spline Parzen window as shown in (4):

$$\begin{gathered} p(f,m;T)=\frac{1}{\mid {\Omega }_F\mid }{\sum }_{x_iϵI_F}{w}_F(f∕{\partial }_F-{\mathrm{I}}_F(x_i)∕{\partial }_F)\times \\ {w}_M(m∕{\partial }_M-{\mathrm{I}}_M(T(x_i))∕{\partial }_M) \end{gathered}$$ (4)

where Ω_F is the domain of the fixed image, x_i are spatial coordinates of the randomly selected sample voxels i in the fixed volume, fϵL_F and mϵL_M, and the scaling constants ∂_F and ∂_M are based on the range of gray-values in the fixed and moving volumes and the number of histogram bins (i.e. 32). I_F(x_i) refers to voxels in the fixed volume and I_M(T(x_i)) are their corresponding locations in the moving volume [29]. w_F is a first-order B-spline Parzen window for the fixed image and w_M is a third-order B-spline Parzen window for the moving image [40]. A cubic B-spline moving Parzen window allows computing the image gradient which is needed for the optimization process [29], [41]. To have an exact gradient, B⁽ⁿ⁾(x) needs to be differentiable which is true when n > 1. Cubic spline was selected because it provides better result than quadratic spline with essentially the same computational cost [42]. Selecting Parzen-window parameters including window size and scaling constants (∂_F. ∂_M) is crucial and difficult in high dimensional data such as the current one [43]. Therefore, in this work, a previously validated and automated parameter estimation method was employed in each resolution [29], [43].

We use the adaptive stochastic gradient descent optimization in Elastix to optimize the NMI and estimate the constrained affine transformation [44]. Coordinates are defined in the fixed space and an intensity pyramid with 8 resolutions is used to assure enough initial overlap between the retinas. Voxels are smoothed at each resolution without any down sampling. Down sampling is unnecessary since at each resolution, 10³ samples are selected randomly regardless of the image size. Selected samples could be voxels or coordinates between them. The process of sample selection is repeated for 2×10³ iterations. Increasing number of samples up to 5×10⁵ and iteration up to 10⁴ did not make notable difference in the result. A linear interpolation is used per resolution to provide a good trade-off between quality and speed, and a final third-order B-spline interpolator is utilized to provide the result.

Fig. 3 shows affine registration of OCT volumes with and without x- and y-axis stretching. The B-scan from the moving volume shows that the foveal pit is not centered, and the retina is tilted. Using regular affine, unnecessary stretching is unavoidable near the nasal side of the retina as shown in Fig. 3(c), while disabling x- and y- stretching provides desired result with higher anatomical fidelity as shown in Fig. 3(d).

Fig. 3.

Example showing unnecessary stretching along x- and y-axis by regular affine on a selected B-scan. (a) B-scan of the moving volume. (b) Corresponding B-scan from the fixed volume. (c) Regular affine transformation by stretching in all directions. The red arrow indicates the region in which unnecessary stretching occurred. (d) Affine transformation by precluding x and y stretching. The red arrow points to the region that has been correctly left blank.

After transforming the moving volume to the fixed volume space using affine registration, we mask out the blank space (red arrow in Fig. 3(d)) on the x- or y- directions by defining a rectangular box containing valid and comparable retinal tissue in both volumes. This generates the volume of interest (VOI) within which we will perform the nonlinear registration.

With the VOIs of the fixed and moving image, we then compute the nonlinear B-spline transformation to align anatomical details. It was shown previously that B-spline transformation combined with stochastic gradient descent optimization can improve efficiency of image registration [29]. Mathematically, a B-spline transformation is defined as:

$$T=\left[\begin{array}{c}\hfill T_1(x)\hfill \\ \hfill T_2(x)\hfill \\ \hfill T_3(x)\hfill \end{array}\right]=\left[\begin{array}{c}\hfill x_1\hfill \\ \hfill x_2\hfill \\ \hfill x_3\hfill \end{array}\right]+\left[\begin{array}{c}\hfill \sum _i{\mu }_{i1}{\beta }^3(\frac{x_1-x_1^i}{6_1}){\beta }^3(\frac{x_2-x_2^i}{6_2}){\beta }^3(\frac{x_3-x_3^i}{6_3})\hfill \\ \hfill \sum _i{\mu }_{i2}{\beta }^3(\frac{x_1-x_1^i}{6_1}){\beta }^3(\frac{x_2-x_2^i}{6_2}){\beta }^3(\frac{x_3-x_3^i}{6_3})\hfill \\ \hfill \sum _i{\mu }_{i3}{\beta }^3(\frac{x_1-x_1^i}{6_1}){\beta }^3(\frac{x_2-x_2^i}{6_2}){\beta }^3(\frac{x_3-x_3^i}{6_3})\hfill \end{array}\right]$$ (5)

where β³ is a third-order B-spline basis function, 6 and μ are B-spline grid spacing and parameters, respectively. Similar to the affine registration step, the NMI is used as the cost function for the nonlinear registration as we found it performs better than other commonly used cost functions such as normalized correlation coefficient in aligning the detail of retinal anatomy. Using the stochastic gradient descent optimization in Elastix, coordinates are defined in the fixed volume and 4 pyramid resolutions are used for the B-spline transformation. Voxels are smoothed at each resolution and 5×10³ samples including voxels, or their coordinates are randomly selected. Sampling process is repeated for 10³ iterations per resolution. A multi-grid strategy is used to match larger structures in the first resolution. Afterwards, smaller structures are matched with the reduced grid size in the next resolution up to the final precision. Using this technique, the size of the grid along z-axis is set as 50% less than the grid size along the x- and y-axis. This is crucial since more information is available along the z-axis. The smallest grid size at the highest resolution is 20 times larger than the original image resolution of x- and y-axis, and 10 times larger than the image resolution along the z-axis. The proper choice of grid size is to ensure stability of registration as values smaller than the selected ones would result in irregular transformation. This is particularly important for cross-subject OCT registration to reduce the contribution of inconsistent structures including vessels. Also, the larger grid size introduces necessary regularization to guide the registration in presence of homogenous regions due to DR abnormalities such as edema.

D. Image registration validation

Accuracy of the proposed registration technique will be assessed by first determining the distance between manually selected center of fovea after registration and the foveal center of the fixed volume. Additionally, similarity between the location of retinal layers after registration will be compared with layer segmentation. Validation cohort included 5 NC and 5 DR (3 NPDR and 2 PDR) OCT volumes. Layer segmentation will be first performed semi-automatically using the OCTExplorer to find the location of retinal layers. Fig. 4 shows the location of retinal layer boundaries used for validation. One NC volume, which was not in the validation cohort, with the retinal tissue centered and not tilted will serve as the fixed volume to which the rest of OCT volumes are registered. Deformation field of each registration is then used to transform retinal layer labels of each subject to the common space, i.e., the fixed volume. Dice similarity (D = 2 * ∣A ∩ B∣/(∣A∣ + ∣B∣), where A is the ground truth from fix volume and B is the same area in the moving volume which is brought to the atlas space by registration) will be calculated for corresponding layers between the fixed and moving volumes. Dice value range from minimum of zero when there is no overlap and maximum of one for two identical regions. To obtain a similarity measure in μm, we will calculate average symmetric surface distance (ASD) [45], [46]. For each voxel on the border of the moving image, the closest border voxel on the fixed image is determined. ASD is the mean Euclidean distance between these points. Minimum value of ASD is zero for a perfect overlap. Evaluation will be performed first on full retinal thickness. Second, on a simplified 3-layer segmentation scheme with superficial, deep and avascular layers. Third, on 7 retinal layer segmentation scheme including nerve fiber layer (NFL), ganglion cell layer and inner plexiform layer (GCL/IPL), inner nuclear layer (INL), outer plexiform layer (OPL), outer nuclear layer (ONL), inner/outer photoreceptor segment (PS) and retinal pigment epithelium (RPE). ONL included external limiting membrane and myoid zone.

Fig. 4.

Location of retinal layers used for validation. (a) Boundaries of 3 retinal layers based on a simplified 3-layer scheme with superficial, deep and avascular layers. (b) Boundaries of 7 retinal layers including NFL, GCL/IPL, INL, OPL, ONL, PS and RPE.

E. Normalized atlas space

An important application of OCT registration is the ability to construct a normalized atlas for anomaly detection and population studies [28]. Previous research suggested that the use of an atlas could reduce the bias induced by registering to an individual volume [47]. Using the OCT volumes of all NC subjects, we will apply our nonlinear registration method to construct a population-based atlas for DR research. To prepare the atlas, a good quality NC volume which is not tilted and has clear representation of retinal layers will be selected as the fixed volume and used to register the rest of NC volumes. Registered volumes are then averaged to provide a new fixed volume for the next iteration. This process will be repeated 5 times to minimize bias in the distribution of intensity values and construct the final atlas.

F. Jacobian map and TBM

Jacobian map provides simultaneous visualization of tissue contraction and expansion due to DR. We register all NC and DR volumes to the normalized atlas constructed above using affine and B-spline transformations. For each registration from the atlas to a subject, the Jacobian map is determined based on nonlinear deformation field as shown in (6).

$$J=s\mid \begin{array}{c}\hfill \partial (\mathrm{x}-u_x)∕\partial \mathrm{x}\partial (\mathrm{x}-u_x)∕\partial \mathrm{y}\partial (\mathrm{x}-u_x)∕\partial \mathrm{z}\hfill \\ \hfill \partial (\mathrm{y}-u_y)∕\partial \mathrm{x}\partial (\mathrm{y}-u_y)∕\partial \mathrm{y}\partial (\mathrm{y}-u_y)∕\partial \mathrm{z}\hfill \\ \hfill \partial (\mathrm{z}-u_z)∕\partial \mathrm{x}\partial (\mathrm{z}-u_z)∕\partial \mathrm{y}\partial (\mathrm{z}-u_z)∕\partial \mathrm{z}\hfill \end{array}\mid$$ (6)

where u is the displacement vector from the B-spline transformation and s is the scaling factor in the affine transformation along the z-direction. The Jacobian map shows local tissue differences between each volume and the atlas. It represents magnitude of expansion (J>1) and contraction (J<1) at voxel level [29]. These values are quantitative (e.g. J=0.9 shows 10% contraction and J=1.1 shows 10% expansion) allowing statistical comparison across groups of subjects.

TBM can detect local structural differences in groups of subjects based on gradient of nonlinear transformation to the common space [30]. In our experiment, TBM will be performed by comparing 3D Jacobian maps at voxel level to determine association between local retinal changes and DR progression. For each voxel, corresponding Jacobian map values will be compared between NC and each DR stage using student t-test to obtain a p-value map. Density of significantly deformed voxels in the p-value map at significant level of 0.05 are then determined. Since both tissue loss and tissue expansion are expected in DR, statistical test will be performed separately for significant contraction and expansion.

III. Results

A. Comparison to retinal layer segmentation

The mean and standard deviation (SD) of the distance between the foveal center of the registered NC (N=5) and DR (N=5) OCT with respect to the foveal center of the fixed volume were 24±11 μm and 32±14 μm, respectively. This shows on average the center of fovea after registration was less than 3 voxels apart (maximum voxel spacing is 12μm along the x- and y- direction) from the atlas foveal center. Table I shows validation of proposed 3D OCT registration in NC subjects based on dice similarity measurements to layer segmentation as described in section II-D. In NC subjects, similarity of whole tissue was 0.99 and similarity between layers based on the simplified 3-layer scheme was over 0.88. Dice was over 0.9 for GCL/IPL, ONL and RPE, over 0.8 for INL and PS, and 0.76 and 0.67 for NFL and OPL layers, respectively. Table II shows validation of proposed 3D OCT registration in DR subjects based on dice similarity measurements to layer segmentation as described in section II-D. Similarity of whole tissue was 0.99 and similarity between layers based on the simplified 3-layer scheme was over 0.86. Dice was 0.9 for ONL, over 0.8 for GCL/IPL, PS and RPE, over 0.7 for NFL and INL, and 0.62 for OPL layer. The lower similarity in deeper layers could be due to smaller size of this region and the naturally occurring intensity changes that are found among OCT scans within these vascular regions. Mean and SD of ASD (μm), as described in section II-D, in the validation cohort of NC and DR subjects is reported in Table III and Table IV, respectively. Mean ASD in NC and DR subjects for the whole tissue, 3-layer scheme and 7-layer scheme were less than 1 μm, 6 μm and 8 μm, respectively. This result shows good alignment for location of layer boundaries after registration in NC and DR subjects.

TABLE I.

Dice similarity (Mean±SD) between retinal layers after registration of Normal control subjects (N=5)

NFL	GCL/IPL	INL	OPL	ONL	PS	RPE
76±15	90±06	83±07	67±13	93±05	82±09	91±04
96±02		88±07		97±02
99±00

Dice similarities have been multiplied by a factor of 10².

NFL is nerve fiber layer, GCL is ganglion cell layer, IPL is inner plexiform layer, INL is inner nuclear layer, OPL is outer plexiform layer, ONL is outer nuclear layer, PS is inner/outer photoreceptor segments, RPE is retinal pigment epithelium. ONL layer included external limiting membrane and myoid zone.

TABLE II.

Dice similarity (Mean±SD) between retinal layers after registration of Diabetic Retinopathy subjects (N=5)

NFL	GCL/IPL	INL	OPL	ONL	PS	RPE
74±12	88±05	78±10	62±10	93±03	80±07	89±05
94±03		86±06		96±02
99±00

Dice similarities have been multiplied by a factor of 10².

TABLE III.

average symmetric surface distance (Mean±SD) between retinal layers after registration of Normal control subjects (N=5)

NFL	GCL/IPL	INL	OPL	ONL	PS	RPE
3.7±0.5	3.8±0.9	4.8±0.4	5.6±1.2	3.7±0.6	3.5±0.5	2.7±0.5
2.3±0.2		4.4±1.0		2.7±0.4
0.8±0.1

Measurements are in μm.

TABLE IV.

average symmetric surface distance (Mean±SD) of retinal layers after registration of Diabetic Retinopathy subjects (N=5)

NFL	GCL/IPL	INL	OPL	ONL	PS	RPE
5.0±2.0	7.3±2.6	7.7±2.9	7.4±3.5	5.6±2.5	4.7±1.5	3.9±1.5
4.9±3.1		5.9±1.9		4.2±1.9
1.1±0.7

Measurements are in μm.

Fig. 5 shows a qualitative comparison of registration with layer segmentation by OCTExplorer in a DR subject. Erroneous layer boundary segmentation is shown in Fig. 5(c). The result of nonlinear registration for the same B-scan when registered to the atlas is shown in Fig. 5(d). As can be observed, the proposed registration-based method provides more reasonable correspondence between the volumes.

Fig. 5.

Qualitative comparison of registration and layer segmentation in a PDR subject. (a) Example of a B-scan from PDR volume. (b) Corresponding B-scan from the atlas. (c) Layer segmentation by OCTExplorer without human intervention. The white arrows point to inaccurate layer boundary detection. (d) Result of nonlinear registration for the same B-scan showing good alignment between layers in the moving B-scan with those of the fixed B-scan.

B. Atlas construction

Fig. 6 shows the fixed volume used for atlas construction and the final atlas constructed following the steps in section II-E. The sharp layer boundaries in the atlas indicates good cross-subject registration. Also, as shown in Fig. 6(f), small vessels have been smoothed out during atlas construction. Since small vessel do not correspond between subjects, using the atlas as the fixed volume for group studies improves accuracy of the registration and prevents irregular transformation.

Fig. 6.

OCT atlas construction. (a) OCT of a NC subject that served as fixed volume. (b) Atlas obtained by iterative registration and averaging of NC volumes (N=24) to the fixed one. (c) Example B-scan of the fixed volume showing foveal pit and retinal layers. (d) Corresponding B-scan from the atlas volume. (e) A cut through superficial layer of the fixed NC volume showing small vessels which may negatively influence registration. (f) Same cut through atlas showing that the vessels were smoothed out. Volumes are scaled by 3 along z-axis to provide better visualization of layer boundaries.

C. Group studies

All OCT volumes used in this study were registered to the atlas in Fig. 6(b) for group comparisons. Fig. 7 shows examples of OCT registration for two NPDR and two PDR subjects. For each subject, a B-scan which includes foveal pit is shown together with the corresponding B-scan from the atlas. The results of affine and nonlinear transformations for each subject is presented. Fig. 7(a) shows a B-scan of a NPDR subject with abnormal retinal thickness and foveal contour. As shown in Fig. 7(d), these shape differences were corrected after B-spline transformation. Fig. 7(e) shows a B-scan of another NPDR subject with retinal thickening, fluid-filled regions and hard exudates. Tilted tissue was aligned with initial affine transformation as shown in Fig. 7(g). The fluid-filled regions have significantly influenced the appearance of foveal pit. As shown in Fig. 7(h), layer boundaries and foveal pit were aligned by the nonlinear transformation. Consequently, fluid-filled regions were shrunk to fit between the layers. Hard exudates were also shifted in accordance to the transformation used for aligning the layers. Fig. 7(i), shows a B-scan of a PDR subject with significant thickness alteration due to edema with foveal shape alteration. Hard exudates are also visible. As shown in Fig. 7(l), layer boundaries and foveal pit were aligned by the nonlinear transformation and the fluid-filled regions were shrunk accordingly. Fig. 7(m) shows a B-scan of another PDR subject with tissue loss, hard exudate and partial foveal flattening. As shown in Fig. 7(p), layer boundaries and foveal pit were expanded and aligned well with the fixed atlas volume.

Fig. 7.

Example OCT volume registration for two NPDR and two PDR subjects. (a) B-scan of the first NPDR subject. (b) Corresponding B-scan from the atlas volume. (c) The same B-scan after affine and (d) B-spline registration. (e) B-scan of the second NPDR subject. (h) Corresponding B-scan from the atlas volume. (g) The same B-scan after affine and (h) B-spline registration. (i) B-scan of the first PDR subject with presence of large fluid-filled regions. (j) Corresponding B-scan from the atlas volume. (k) The same B-scan after affine and (l) B-spline registration. (m) B-scan of the second PDR subject with tissue loss in foveal region. (n) Corresponding B-scan from the atlas volume. (o) The same B-scan after affine and (p) B-spline registration.

Example 3D Jacobian maps derived from the deformation field of OCT volumes of DR subjects that were presented in Fig. 7 are demonstrated in Fig. 8 and Fig. 9. Additionally, a cut through the 3D Jacobian maps for the same B-scans in Fig. 7 is demonstrated for each subject. Magnitude of local expansion and contraction are demonstrated using color-coded maps. Local retinal contraction and expansion due to DR progression can be clearly visualized in these examples.

Fig. 8.

Example 3D Jacobian maps based on nonlinear deformation of OCT volumes of the two NPDR subjects shown in Fig. 7 to the atlas. a) Top row: OCT volume of the first NPDR subject and the Jacobian map showing uniform contraction over the retina and local foveal expansion. Bottom row: a cut through OCT volume of the first NPDR subject, the atlas and the Jacobian map. b) Top row: OCT volume of the second NPDR subject and the Jacobian map showing expansion and contraction in different retinal regions. Bottom row: a cut through OCT volume of the second NPDR subject, the atlas and the Jacobian map. Color-bar shows magnitude of contraction and expansion. Volumes are scaled by 3 along z-axis to provide better visualization.

Fig. 9.

Example 3D Jacobian maps based on nonlinear deformation of OCT volumes of the two PDR subjects shown in Fig. 7 to the atlas. a) Top row: OCT volume of the first PDR subject with edema and foveal shape alteration and the Jacobian map showing foveal expansion. Bottom row: a cut through OCT volume of the first PDR subject, the atlas and the Jacobian map. b) Top row: OCT volume of the second PDR subject with severe tissue loss and the Jacobian map showing contractions due to tissue loss. Local expansion is also visible on the corner of the volume. Bottom row: a cut through OCT volume of the second PDR subject, the atlas and the Jacobian map. Color-bar shows magnitude of contraction and expansion. Volumes are scaled by 3 along z-axis to provide better visualization.

Table V demonstrates the density of significantly deformed voxels using TBM analysis in different DR stages. Density of significantly deformed voxels in PDR was consistently higher than NPDR subjects. Fig. 10 shows significantly deformed voxels overlaid on the atlas for the groups of NPDR and PDR subjects. Mean Jacobian value of contracted voxels were 0.90±0.05 and 0.86±0.05 in NPDR and PDR subjects, respectively. There was no overlap between significantly expanded and contracted voxels.

TABLE V.

Density (%) of significantly deformed voxels by TBM analysis

DR stage	Contraction	Expansion
NPDR (N=57)	5.6	5.8
PDR (N=24)	10.8	9.4

DR is diabetic retinopathy, NPDR refers to non-proliferative diabetic retinopathy and PDR refers to proliferative diabetic retinopathy.

Fig. 10.

Location of significantly contracted and expanded voxels in (a) NPDR (N=57) and (b) PDR (N=24) subjects by TBM analysis are overlaid as blue and red on the atlas volume, respectively. Volumes are scaled by 3 along z-axis to provide better visualization.

IV. Discussion and conclusion

With the increase in number of acquired OCT scans and technical complexity associated with layer segmentation in DR, new automated techniques for detection of local tissue alterations are becoming more relevant. In the current study, a systematic approach for rapid 3D registration of OCT volumes in healthy and DR subjects was presented. The usefulness of the Jacobian map as a quantitative visualization tool and TBM for quantification of structural shape alterations in stages of DR have been demonstrated.

Previous reports of cross-subject OCT volume registration were based on 1D nonlinear registration and were not designed to align tilted volumes or those with edema [28], [48]. The current study is the first, to our knowledge, that present a technique for 3D OCT registration in presence of DR abnormalities. The mean dice similarity for the 7-layer segmentation scheme of the current study in normal subjects was 8% higher than the previous report of cross-subject OCT registration [28]. The differences in the similarity measures may be attributed to different field of view. Also, the report of registration accuracy in the current and earlier studies is influenced by quality of layer segmentation. Finally, IS layer similarity was assessed separately in [28], while this layer was combined into OS in the current study as OCTExplorer does not segment this layer. Registering a pair of OCT volumes using the current technique took less than 10 minutes on a PC with 4.5 GHz CPU and 64 GB RAM.

The color-coded Jacobian map provides simultaneous visualization of tissue contraction and expansion due to DR progression. Advanced DR abnormalities such as fluid-filled regions are reflected as expansion, while tissue loss is reflected as contraction in the Jacobian map. DR abnormalities such as neovascularization, microaneurysms and hard exudates may become visible in the Jacobian map depending on their extent. TBM was performed with the inclusion of affine scaling to reduce skewness of the result in case of thickness inhomogeneity. The density of significantly contracted and expanded voxels was higher in PDR compared to NPDR. This result can be expected since there is strong association between increase in number of visible retinal abnormalities and DR progression [49]. In OCT, normal variation of layer thickness exists in NC and DR subjects. However, this variation is smaller than the changes due to DR [50]. As shown in the Jacobian maps (Fig. 8 and Fig. 9), the magnitude of deformation can be used to differentiate between problematic and benign deformation. The individual Jacobian map shows local deformation and the TBM analysis demonstrates patterns of global deformations in a group of subjects. Also, the atlas of the current study was formed from OCT images of NC subjects with age ranging from 21 to 75 years old to reflect intensity variation of each layer in normal subjects. Future studies are needed in a larger cohort to determine clinical implication of TBM analysis of OCT volumes in DR. Such a study can better characterize normative variability which must be considered when evaluating patients for diagnosis, particularly at the lower boundary of normal range. Nevertheless, the current result shows potential for TBM analysis for detecting systematic retinal alterations due to progression of DR.

Recently, methods for simultaneous segmentation of retinal layers and abnormalities in 3D OCT have been developed [51], [52]. These approaches were validated in macular edema [51], and in retinas with central serous retinopathy and age-related macular degeneration [52]. While these methods provide segmentation and visualization of thickness map and abnormalities, they are substantially different from the current registration approach. The 3D registration does not have to focus on specific abnormality, instead it provides quantitative visualization and evaluation of structure differences in OCT with respect to the fixed image. It also allows construction of population atlas that can serve as standard reference for comparison between subjects and performing group-wised comparison using tools such as TBM.

For the current technique to be applied to OCT data acquired with a large field of view, one important aspect to consider is the fan beam effect of OCT scans [53]. In smaller field of view OCT such as the one used in the current study, the distortion due to fan beam effect is minimal [54]. To apply our method to OCT scans of larger field of view, correction for the fan beam effect can first be performed based on measurements of eye geometry if they are available. Nevertheless, the registration-based method developed here can still be valuable for various tasks such as anomaly detection even without the necessary measurements to perform the correction of fan beam effect because of the smoothness of the retinal structure, which will be an important topic of future research. In future work, we plan to apply the current technique to OCT images of DR subjects over time, which could assist with the quantification of longitudinal changes of DR pathology under treatment. Furthermore, future experiment will be performed to evaluate application of the current technique for layer segmentation of OCT scan of pathological eyes by pulling back and fusing high quality segmentation of OCT scans from healthy eyes. We believe this can potentially improve layer segmentation in pathological scans in which layer boundaries are intact. We will also extend the current registration method to align OCT images from different field of view and retinal regions. In addition, we will extend the current technique to perform multi-modal registration of OCT with other retinal imaging modalities such as OCT angiography to help clinicians integrate the information from different modalities.