An automated 3D registration method for optical coherence tomography volumes

Abstract

Optical coherence tomography (OCT) is able to provide high resolution volumetric data for biological tissues. However, the field of view (FOV) of OCT is sometimes smaller than the field of interest, which limits the clinical application of OCT. One way to overcome the drawback is to stitch multiple 3D volumes. In this paper, we propose a novel method to register multiple overlapped volumetric OCT data into a single volume. The relative positions of overlapped volumes were estimated on en face plane and at depth. On en face plane, scale invariant feature transform (SIFT) was implemented to extract the keypoints in each volume. Based on the invariant features, volumes were paired through keypoint matching. Then, we formulated the relationship between paired offsets and absolute positions as a linear model and estimated the centroid of each volume using least square method. Moreover, we calibrated the depth displacement in each paired volume and aligned the z coordinates of volumes globally. The algorithm was validated through stitching multiple volumetric OCT datasets of human cervix tissue and of swine heart. The experimental results demonstrated that our method is capable of visualizing biological samples over a wider FOV, which enhances the investigation of tissue structure such as fiber orientation.

I. Introduction

Optical coherence tomography (OCT) has experienced rapid advancement in recent years. In comparison with existing image modalities, OCT is able to achieve high penetration, high sensitivity, fast data acquisition, and high axial resolution simultaneously. OCT is a promising medical modality for a wide range clinical application such as imaging the myocardium [1], cervix [2], and retina[3]. However, the field of view (FOV) of current OCT systems is limited to the magnitude of millimeters. For some clinical applications, e.g. monitoring the mechanical properties of human cervix tissue [4] or assessing the ventricular septum, such FOV is not sufficient to visualize the whole tissue structure. There is thus an increasing need to extend the FOV. Without complicating the current OCT system design, stitching multiple overlapped OCT volumes into a single three dimensional dataset serves as a possibility to enlarge the FOV.

Though many image registration methods are proposed for image stitching [5, 6], general methods to stitch volumetric OCT data are still scarce. Stitching OCT volumes is challenging because the signal noise ratio (SNR) in OCT image is much weaker than most of other image modalities, such as computed tomography and magnetic resonance imaging. A weak SNR indicates relatively strong noise, which can change the texture of the overlapped area and result in poor correlation and matching between volumes. In addition, in OCT, the volumes should be aligned in three dimensions, which increase the difficulty of global optimization algorithms. An method for mosaicking OCT volumes from bladder tissue is proposed in [7]. Zawadzki et al [8] register multiple retinal OCT volumes through manual stitching, then develop a ray cast method to render the volume and use an annealing algorithm to optimize the alignment among multiple retinal imaging volumes [9]. Multiple OCT volumes are stitched in [10] based on blood vessel matching algorithm. In [3], a B-spline based free form deformation method was used to register multiple volumes for a motion-free composite image of the retinal vessels. Most of current OCT stitching methods are developed for retinal imaging, where blood vessels are a good reference for registration. Furthermore, the retinal tissues have a clear layered structure, which enables the layer information to help register B-scans in 2D.

In this paper, we propose a generic method that can be applied to register multiple volumes of biological tissues. Since the features within en face images sometimes fail to be a reliable reference for registration, we rely upon the scale invariant features from camera images that can be easily obtained through OCT system with integrated microscopes. Scale invariant transform (SIFT) was applied to extract the features that are robust to contrast, brightness, and movements. Though weak SNR undeniably exists in OCT image, the contrast at the surface of tissue is strong due to surface reflection. Therefore, the edge information of OCT is used to quantify the displacement at depth. The three dimensional offsets were considered in a linear model and globally optimized using a least square estimation. The stitched volume is used to enhance the analysis of fiber organization in heart tissue [11, 12] and cervix [13–15] over large FOV.

II. Methodology

A. Data collocation

Three dimensional volumetric data was collected from human cervix and swine heart. The cervix specimen was acquired from a nonpregnant hysterectomy patient using an Institutional Review Board (IRB) approved protocol at the Columbia University Medical Center [16]. The swine heart was obtained from Green Village Packing, Green Village, NJ. The data were acquired using a commercial spectral domain OCT system, Telesto (Thorlabs GmbH, Germany). It is an InGaAs based system centering at 1325 nm, with a bandwidth of 150 nm. The axial and lateral resolutions in water are 4.9 μm and 9.8 μm, respectively. The maximum axial line rate is 92 kHz. In our experiment, each volume consists of 800 × 800 × 512 pixels, corresponding to a tissue volume of 4 mm × 4 mm × 1.8 mm. Samples were placed in a linear transition stage underneath the objective. For each sample, we obtained multiple volumes. There was an overlap proportion of 10% to 20% between two consecutive volumes. Camera images were taken simultaneously with the same FOV. The camera images and OCT images were calibrated by the default factory setting in Thorlab pre-experiment.

B. Algorithm flow

To stitch multiple volumes, our algorithm consists of the registrations on both the en face plane and depth, as shown in Fig. 1. On the en face plane, we first paired individual volumes based on their scale invariant features. Then, the offset between each pair of volumes was measured. We then formulated a linear regression model between the measured offsets and the centroid of each volume in a global coordinate system. The least square estimation of the regression model was the global optimization registration results of en face plane. Based on the estimation of offsets on x-axis and y-axis, we calculated the overlapped area between any pair of volumes and measured displacements at depth through comparing the measured edge in each volume. Another linear regression model for depth estimation was then derived and the global offset of depth was obtained. With the global offsets in x, y, and z axis, all volumes were stitched and visualized in 3D.

Figure 1.

Flowchart of registration algorithm. Scale invariant transform (SIFT) is used for pair matching. Linear regression models are used and least square estimations are calculated in global registration and depth optimization. Estimation in first block and edge detection is used in depth calibration.

C. Registration on en face plane

We consider the camera image of each volume as the reference of registration on en face plane. SIFT algorithm in [5] was applied to each image. Over one hundred keypoints were extracted in each image. A normalized descriptor, with a dimension of 128, was assigned to each keypoint. The local descriptor was based on the gradient in a small area and thereby was robust to illumination and brightness changes caused by movement of transition stage. A Best-Bin-First (BBF) algorithm [6] was used to search a matched keypoint from other image. Denote DES_i and DES_j are the descriptors of keypoint sets in volume i and j respectively. For a keypoint des ∈ DES_i, the set of distance is defined as:

$${\mathit{Dis}}_{\mathit{des}}=\{\mathit{arc}\cos (\mathit{des}•{\mathrm{des}}_k)|k\mathrm{\in }{\mathit{DES}}_j\}$$ (1)

where the ● is the dot product of two vectors. The calculated distance represents the angle between two normalized descriptors. In our experiment, a match between des and the set DES_j is established only if the distance to the nearest neighbor is less than 0.45 times of the distance to the second-nearest neighbor. If at least one of match relationship can be established between keypoints in volume i and volume j, we consider the two volumes are paired. Suppose there are K keypoints matched between two volumes. The offsets, Dx_ij and Dy_ij, can be estimated as

$${\text{Dx}}_{\mathit{ij}}=\text{median}({\{{\mathrm{x}}_{\mathit{ik}}-{\mathrm{x}}_{\mathit{jk}}\}}_{k=1}^K)$$ (2)

$${\text{Dy}}_{\mathit{ij}}=\text{median}({\{{\mathrm{y}}_{\mathit{ik}}-{\mathrm{y}}_{\mathit{jk}}\}}_{k=1}^K)$$ (3)

where (x_ik, y_ik) and (x_jk, y_jk) are the coordinates of the kth matched keypoint. Fig. 2 (a) shows a typical match between volume i and volume j.

Figure 2.

Schematics of registration on en face plane (a) and at depth (b). (a) On en face plane, the offsets between two volumes are measured through mataching its corresponding camera image. The centroids of multiple volumes are estimated globally by considering their relative offsets. (b) To register the OCT volumes at depth, the Bscans at overlaped area are compared. The displacement of detected edge shows the offset of depth among volumes.

Next, we stitched all volumes in a global space. Denoting the centroid of volume i (i = 1, 2, …, N) as (x_i, y_i), we wrote the centroids in all volumes as a vector c = [x₁ x₂ … x_N y₁ y₂ … y_N]^T. Since the samples were moved along x axis and/or y axis, the offsets in each pair were maintained in global space. The geometric relationship between offsets and centroid of OCT volumes were shown in Fig. 2 (a). Suppose there are M pairs of volumes. The measured offset, d = [dx₁ dx₂ … dx_M dy₁ dy₂ … dy_M]^T, had a linear relationship with centroid vector c as

$$\mathit{Wc}=d$$ (4)

W is a matrix with dimension of 2M×2N. Let the mth pair be the offsets between volume i and volume j. The mth row and (m+M)th row of W is given by the following rows:

$$\mathit{mth}\mathit{row}:[\dots 010\dots 0-10\dots 000\dots 000\dots ]$$

$$(m+M)\mathit{th}\mathit{row}:[\dots 000\dots 000\dots 010\dots 0-10\dots ]$$

The ith column in mth row and (i+N)th column in (m+M)th row equal to 1 while The jth column in mth row and (j+N)th column in (m+M)th row equal to −1. Each row in W represents the offset between two volumes in either x or y axis, with a relationship of x_i − x_j = dx_ij. The least square estimates of (4) is:

$$\widehat{c}={(W^{T}W)}^{-1}{W}^{T}d$$ (5)

In our experiment, the matrix W^TW may not be full rank. A generalized inverse can be used to solve (5). Here, we use Moor-Penrose pseudo inverse of matrix during implementation.

D. Registration at depth

Based on the estimation of ĉ, we can arrange the volumes as overlapped tiles in a single space. However, the volumes were not perfectly stitched because of variations at depth. To register multiple volumes at depth, we used the edge of B-scans as reference. We calculated the overlapped region between each volume pairs as the gray area in Fig. 2(b). To calibrate the displacement at depth, we obtained B-scans from two volumes at the same position within the overlapped region. The edge of each B-scans was determined by smooth filter followed by searching maximum gradient point. The displacements of two edges were measured along x axis. A typical scenario of overlayed edges was drawn in Fig. 2 (b). The measured offset Δz_ij between volume i and volume j was computed as the median of all displacements between edges in volume i and j along x axis. Similar to Section II-B, a linear relationship can be formulated between the z coordinates of centroid in all volumes, z = [z₁ z₂ … z_N]^T, and the measured offset, d₂ = [dz₁ dz₂ … dz_M]^T as following:

$$Vz=d_2$$ (6)

where mth row of V can be written as [… 0 1 0 … 0 −1 0 … 0]. The entry of ith column is 1 while the entry of jth column is −1. The least square result of (6) is:

$$\widehat{z}={(V^{T}V)}^{-1}{V}^T{d}_2$$ (7)

Finally, we aligned all volumetric dataset on the basis of estimation ĉ and $\widehat{z}$. Specifically, we used a hard combination of all individual volumes. All non-zero voxel values are maintained when added to the stitched volume.

III. Experiments and Results

We set up an evaluation test to validate the accuracy of our method. We used an OCT volume of 4 mm × 4 mm × 1.8 mm (800 × 800 × 512 pixels) as ground truth. Sticking to the same space, we additionally imaged four overlapped volumes of 2.5 mm × 2.5 mm × 1.8 mm (500 × 500 × 512 pixels). The volumes were stitched by our methods. We set three pairs of landmarks at ground truth and then specified the same landmarks at stitched volume. The 3D distances in each pair were measured. We found that difference of measured distance between stitched volume and ground truth is 10.57±8.51 pixels. Our methods are further validated in human cervix tissue and swine heart. The two dimensional results are shown in Fig. 3. For cervix data, a total of 17 camera images are stitched in Fig. 3 (a) in a semi-circular shape. The combined OCT image was visualized in Fig. 3 (b) based on the offsets measured in Fig. 3 (a). More importantly, the inner canal of the cervix is clearly observable at the center of stitched image. The detailed collagen fiber structure can be directly viewed at depth. Similarly, the structure of ventricular septum is shown on the camera image in Fig. 3 (c) and OCT image in Fig. 3 (d), respectively. In total, 16 small volumes are aligned as a rectangle shape. The inner structure of myocardium is observable. At each B-scan, Fig 3. (e) and (f) show the stitching results. We found that the surfaces of multiple B-scans are stitched smoothly at a range of over 1 cm. Three dimensional registration results are shown in Fig. 4. The cylinder shape of cervix is shown in Fig. 4 (a), and the representative B-scans are overlayed with 3D volume in Fig. 4 (b). Square volumes of ventricular septum are given in Fig. 4 (d) and Fig. 4 (e). As an application of registration, we show the fiber estimation results in a large volume using algorithm in [11, 12]. Fig. 4 (c) and (f) show the fiber trend in cervix tissue and myocardium, respectively.

Figure 3.

Registration results of human cervix (a, b, and e) and swine heart (c,d, and f). Based on the registration algorithm on en face plane, we register the camera images of human cervix (a) and swine heart (c). The OCT images can be registered correspondingly on human cervix (b) and swine heart (d). Using the registering algorithm at depth, multiple B-scans in human cervix (e) and swine heart (f) are aligned.

Figure 4.

Three dimensional registration results of human cervix (a, b, and c) and swine heart (d, e, and f). The stitched OCT volumes of cervix sample is shown in (a) with a size of 21.6 mm × 12.5 mm × 1.5 mm. The stitched myocardium sample is presented in (d) with a size of 12.5 mm × 12.5 mm × 1.5 mm. Typical B-scans are overlayed with OCT volumes in (b) and (e). Fiber orientation estimation on en face plane is processed and ovelayed with original OCT volumes in (c) and (f). The fiber orientation estimation is color coded based on confidence level from high (bright) to low (dark).

IV. Conclusion

We present an automatic registration method to stitch multiple OCT volumes based on SIFT and least square estimation in three dimensions. By showing a large view of cervix and myocardium and investigating the structure property within a large area, we demonstrate our algorithms’ ability in automatic stitching.

Future work will be concentrated on the compensation of overlapped area post registration and the extension of the algorithm on catheter-based OCT system.