Non-rigid registration based on hierarchical deformation of coronary arteries in CCTA images

Abstract

In this paper, we propose an accurate and rapid non-rigid registration method between blood vessels in temporal 3D cardiac computed tomography angiography images of the same patient. This method provides auxiliary information that can be utilized in the diagnosis and treatment of coronary artery diseases. The proposed method consists of the following four steps. First, global registration is conducted through rigid registration between the 3D vessel centerlines obtained from temporal 3D cardiac CT angiography images. Second, point matching between the 3D vessel centerlines in the rigid registration results is performed, and the corresponding points are defined. Third, the outliers in the matched corresponding points are removed by using various information such as thickness and gradient of the vessels. Finally, non-rigid registration is conducted for hierarchical local transformation using an energy function. The experiment results show that the average registration error of the proposed method is 0.987 mm, and the average execution time is 2.137 s, indicating that the registration is accurate and rapid. The proposed method that enables rapid and accurate registration by using the information on blood vessel characteristics in temporal CTA images of the same patient.

Introduction

Coronary artery disease (CAD) is one of the leading causes of death worldwide [1]. It is a disease caused by insufficient blood supply to the cardiac muscle, resulting from the narrowing or clogging of the blood vessels due to stricture, calcification, and lipid deposits in the coronary arteries. Cardiac CT angiography (CTA), known for its non-invasive nature and high accuracy, is utilized for an accurate diagnosis of CAD [2]. CTA can take 3D images of blood vessels using radiopaque contrast media for various tasks. For example, these tasks can be comparing the effects of surgery, searching for coronary artery movements according to the heart cycle, and checking the progress based on the images taken during the initial and follow-up visits. However, the coronary arteries in the CTA images taken from the same patient may entail differences in disease status, imaging equipment, patient posture, protocol, and heart movement. These differences can prolong comparative observations and analysis of multiple CTA images. Active research is conducted on auxiliary technologies that allow direct comparative observations and monitoring of coronary arteries through automatic registration methods of the vessels to compensate for these shortcomings.

Related work

Studies on registration related to medical imaging have been conducted in various modalities such as US/US, CT/CT, CT/MR, and MR/MR.

Studies on MR/MR registration are as follows [3, 4]. Zhang et al. [3] performed an hippocampus segmentation through pre-prepared brain cross-section atlas model registration and then performed sophisticated segmentation using the broad learning model. This method has a potential problem of causing misalignment if the predefined atlas model is different from the brain cross-section of the segmented target. Fontana et al. [4] performed thresholding-based muscle segmentation for skeletal muscle registration in MRI images, performed initial registration through rigid body registration, and then performed non-rigid registration using a B-spline-based free-form transform (FFD). However, as this method matches the divided muscle by a simple thresholding method, it implies a disadvantage because a large matching error may occur depending on the degree of division.

Studies on CT/MR registration include as follows [5–7]. For knee joint registration, Kuniper et al. [5] performed initial registration using the Iterative Closest Point (ICP) method between the point cloud data obtained in the area where water/fat/muscle is divided, and then performed B-spline non-rigid registration based on intensity information. As the matching performance in this method is affected by the brightness similarity of the images, registration becomes difficult if changes occur in the protocol of the imaging equipment or in the patient’s body. Xu et al. [6] segmented the liver area using a U-Net++-based deep learning model applied with the 3D MR images taken before the procedure and the 3D CT images taken during the procedure, and after global alignment via rigid body registration, performed non-rigid registration using the B-spine-based FFD. This method may cause inaccurate segmentation because it does not consider the inter-images of adjacent upper/lower frames in liver segmentation, which could affect the registration performance. Tang et al. [7] divided the liver area through the active contour using the seed points specified by the user and performed non-rigid body registration between the divided liver areas using the B-spline-based FFD. This method has a disadvantage in that a user has to define several seed points through intervention manually.

Studies on US/US registration include as follows [8–11]. Ou et al. [8] conducted supervised-learning based on the texture information extracted from US images using the Gabor filter in order to select meaningful features. Based on this, they performed registration using an FFD technique. For the purpose of registration of brain images for brain shift compensation, Luo et al. [9] performed registration through the Gaussian process between the feature points acquired using Scale-invariant feature transform (SIFT) in the 2D US images, and further improved the registration accuracy by using preoperative US-MRI image information. Machado et al. [10] performed the SIFT-based affine and thin-plate spline registration for 3D US Volume registration. Zhou et al. [11] performed registration through intensity similarity comparison using NCC, for US volume matching. However, registration between US/US images presents difficulties due to the characteristics of ultrasound images such as low contrast detectability and poor quality [12].

Studies of CT/CT registration include as follows [13–18]. Park et al. [13] performed liver segmentation on 3D CT images using a level-set method, initial registration using the boundary regions, then precise registration through the non-rigid registration technique and local affine transformation using the B-spline deformation model. This method has a disadvantage in that the matching technique using the local affine transformation model takes a long time due to the complexity of the calculation. Zhang et al. [14] registered blood vessels divided into each cardiac phase through registration using the B-spline method and the template matching technique based on the blood vessel at the end-diastole point in the 4D CTA image. This method takes a long time to conduct owing to vessel division and registration for each cardiac phase. In addition, it entails a disadvantage of onerous registration because of the rapid heart movement causing blurring and ghost effect phenomena in the image. Hadjiiski et al. [15] proposed a method for performing inter-phase registration between adjacent phases with little changes in the blood vessels by using affine transformation and cubic B-spline transformation methods in the multi-phase CTA images. However, this method makes it challenging to divide blood vessels in the phase image excluding end-diastole and end-systole because of the image noise caused by the rapid transformation of blood vessels due to heartbeats. Furthermore, it can cause registration errors owing to incorrect input of blood vessel information. Luo et al. [16] proposed a registration method using a hidden Markov model (HMM) based on the intensity of the CTA image and the geometry of the 3D vessel centerline. However, registration becomes difficult if the protocol of the imaging equipment changes or internal changes occur in the patient body because such techniques are influenced by the brightness similarity of the images. Zeng et al. [17] matched branching points between two blood vessels using a feature engineering technique by means of SIFT. Next, the authors conducted registration based on thin plate spline on all blood vessels using the matched branching points. Subsequently, the registration correction on blood vessel segments was performed using the Levelset and B-spline transform technique in order to register between 3D vessel centerlines. This method takes a long time to perform, as the operation is performed on the entire blood vessel and some segments. Cao et al. [18] performed rigid registration using coherent point drift to register between 3D vessel centerlines. However, this method can cause registration errors by converging to a local minimum if the transformation between the vessel centerlines is substantial.

In this paper, we propose a non-rigid registration method based on the characteristics of the temporal CTA image of the same patient to overcome the aforementioned shortcomings. Translation, rotation, and rigid transformations between two blood vessels are sequentially optimized to conduct global registration. Next, point matching is performed based on the gradient to consider the corresponding points between the two blood vessels. In this step, outliers are removed to consider CAD and errors in the feature extraction process. Non-rigid registration involves conducting hierarchical local transformation so that transformations occur accurately and naturally. In addition, a more natural transformation is enabled by using an energy function that utilizes blood vessel characteristics, as the phase of the two blood vessels is maintained. In this study, rapid and accurate registration is enabled through outlier removal using blood vessel characteristics and hierarchical local transformations.

The structure of this paper is as follows. Chapter 2 introduces the proposed method, and Chap. 3 details the experimental environment and the results. Finally, Chap. 4 presents the conclusions of this study.

Methodology

In this study, a rapid and robust 3D-3D non-rigid registration process is proposed using a flow chart (Fig. 1) to register blood vessels between 3D CTA images of the same patient photographed with a time difference.

Fig. 1.

3D-3D non-rigid registration process

3D CTA images of the same patient photographed with a time difference are defined as fixed and moving images. Feature-based registration methods [19] allow faster registration compared to light intensity-based registration methods [20], which have relatively high computational complexity. A 3D vessel centerline is extracted [21] from the 3D CTA image to acquire information on blood vessel characteristics for rapid registration. Here, registration is performed based on the assumption that the shapes are similar because the blood vessels of the fixed and moving images are photographed from the same patient.

Rigid Registration

The location or shape of the blood vessel may differ even for the same patient owing to CAD, imaging equipment, patient posture, and protocols [22]. A global registration is performed through rigid registration [23] proposed by Park et al. [23] to minimize these errors before conducting non-rigid registration. First, Eq. (1) is used to optimize the conversion function regarding the two 3D centerlines of the fixed and moving images [24].

$$\begin{array}{cc}& T_T\left(T\right)=M_T\left(T_1^3\right)\hfill \\ \hfill & T_{R+T}\left(R\right)=M_T\left(R_1^3\right)·M_R\left(R_4^6\right)\hfill \\ \hfill & T_{\mathit{Rigid}}\left(A\right)=M_T\left(A_1^3\right)·M_R\left(A_4^6\right)·M_S\left(A_6^9\right)\hfill \\ \hfill & M_T\left(T\right)=\left[\begin{array}{cccc}1& 0& 0& T_x\\ 0& 1& 0& T_y\\ 0& 0& 1& T_z\\ 0& 0& 0& 1\end{array}\right],M_S\left(S\right)=\left[\begin{array}{cccc}{S}_x& 0& 0& 0\\ 0& S_y& 0& 0\\ 0& 0& S_z& 0\\ 0& 0& 0& 1\end{array}\right]\hfill \\ \hfill & M_R\left(\theta \right)=\left[\begin{array}{cccc}{c}_y{c}_z& c_z{s}_x{s}_y-c_x{s}_z& s_x{s}_z+c_x{c}_z{s}_y& 0\\ c_y{s}_z& s_x{s}_y{s}_z+c_x{c}_z& c_x{s}_y{s}_z-c_z{s}_x& 0\\ -s_y& c_y{s}_x& c_x{c}_y& 0\\ 0& 0& 0& 1\end{array}\right]\hfill \end{array}$$ 1

Here, $T_x$, $T_y$, and $T_z$ refer to the motion vector of the $x$, $y$, and $z$ axes, respectively, and $S_x$, $S_y$, and $S_z$ refer to the center size vectors. Further, $T_T$ represents translation transformation, $T_{R+T}$ represents translation and rotation transformations, and $T_{\mathit{Rigid}}$ refers to rigid transformations. Finally, $c$ and $s$ refer to $\text{cos}\left(\theta \right)$ and $\text{sin}\left(\theta \right)$, respectively. Equation (2) is used to evaluate the conversion function [25].

$$AADD=\frac{1}{N_C}{\sum }_{i=0}^{N_C-1}\left|D,i,s,t,a,n,c,e,M,a,p_F,\left(C,e,n,t,e,r,l,i,n,e_M,\left(i\right)\right)\right|$$ 2

Here, $Centerlin{e}_M\left(i\right)$ refers to the i-th point on the moving 3D centerline, and $DistanceMa{p}_F\left(i\right)$ shows the 3D distance map of the fixed 3D centerline [25].

Point matching based on the gradient

In this study, non-rigid registration for the local deformation of each blood vessel was conducted, following global registration through rigid registration. Consequently, all the points, $v_i$, were matched to the 3D centerline acquired from the fixed image based on the 3D centerline obtained from the moving image [24]. The global registration of the two blood vessels occurred through the rigid registration in the previous process, so a gradient-based searching technique was used to search for points instead of a distance-based technique. In the gradient-based searching method, partial derivatives ($\frac{\partial v_i}{\partial x}$, $\frac{\partial v_i}{\partial y}$, $\frac{\partial v_i}{\partial z}$) were calculated for all the points and were matched as shown in Fig. 2 [24].

Fig. 2.

Point matching between points on the 3D-3D centerlines

Outlier removal

Errors can occur in the gradient-based point matching process because of errors that can occur in the extraction process of CAD or the centerline even if the data of the same patient are used in registration. The registration efficiency through the removal of outliers between the matched points was enhanced as follows. First, we assumed the difference in the shape of the two blood vessels to not be significantly different in the CTA images taken from the same patient. Second, $H\left(v_i\right)\to \text{0,1}$ was defined assuming the error in the blood vessels was not too substantial, as the global registration of the two blood vessels occurred through rigid registration. The $H\left(v_i\right)$ uses the distance ($H_D\left(v_i\right)$, gradient ($H_{\theta }\left(v_i\right)$), and thickness $H_D\left(v_i\right)$.

$$H_D\left(x_i\right)=\left\{\begin{array}{c}0\\ 1\end{array}\right)\begin{array}{c}\begin{array}{c}1)NoMatching\\ 2)MatchingDistance>Ma{x}_{\mathit{Distance}}\end{array}\\ Otherwise\end{array}.$$ 3

$H_D\left(x_i\right)$ is used as the outlier if no matched points exist or the distance between the matched points is greater than $Ma{x}_{\mathit{Distance}}$.

$$H_{\partial }\left(x_i\right)=\left\{\begin{array}{cc}0\hfill & G\left(x_i\right)-G\left(x_i^{\prime }\right)>Ma{x}_{\theta }\hfill \\ 1\hfill & Otherwise\hfill \end{array}\right)$$ 4

.

Here, $G\left(x_i\right)$ refers to the adjacent gradient, and the surrounding adjacent points are used to consider the presence of a disease, errors that occur in the division process, and errors that occur during the point matching process.

$$H_{\mathit{Th}}\left(x_i\right)=\left\{\begin{array}{cc}0\hfill & Th\left(x_i\right)-Th\left(x_i^{\prime }\right)>Ma{x}_{\mathit{Th}}\hfill \\ 1\hfill & Otherwise\hfill \end{array}\right)$$ 5

Similar to the gradient, adjacent points are also used for the information regarding the vessel thickness.

Hierarchical local transformation

The non-rigid registration process involves repeated local transformations from the upper level, which includes the root node of the vessel tree, to the lower level to achieve more accurate and natural vessel transformations (Fig. 3).

Fig. 3.

Upper and lower local transformations

Thus, the phase information of the actual blood vessels is first maintained through sampling point matching and transformations, and local errors are reduced. The transformation involves repeated rotations using the angle between matched sampling points (Fig. 4).

Fig. 4.

a LCA registration result. b RCA registration result

After repeated local transformations, the energy minimization method proposed by Park et al. [17] is extended and applied.

$$\begin{array}{cc}& E=\sum _{i=0}^N\left({\alpha }_i,E_{\mathit{cont}},\left(v_i\right),+,{\beta }_i,E_{\mathit{curve}},\left(v_i\right),+,{\gamma }_i,E_{\mathit{img}},\left(v_i\right)\right)\hfill \\ \hfill & E_{\mathit{cont}}\left(v_i\right)=(\overline{d}{|v_i-v_{i-1}|)}^2\hfill \\ \hfill & E_{\mathit{curve}}\left(v_i\right)=\left|\frac{d^2{v}_i}{d{s}^2}\right|\hfill \\ \hfill & E_{\mathit{img}}=H\left(v_i\right){\left(v_i,-,\dot{v_i}\right)}^2\hfill \end{array}$$ 6

Here, α, β, and γ refer to the weight of each function; $v_i$ represents the i-th point on the 3D centerline; and N represents the number of points constituting the 3D centerline. The $E_{\mathit{cont}}$ energy function controls the continuity of the points constituting the 3D centerline and maintains equal spacing between adjacent points. The $\stackrel{-}{d}$ refers to the average distance between neighboring points. The $E_{\mathit{curve}}$ energy function controls the slowness of the 3D centerline and induces a gentle, natural shape to prevent sudden curvature changes and vibrations. The $E_{\mathit{img}}$ energy function controls the location and shape transformations according to the reference 3D centerline. The $\dot{v_i}$ is the corresponding point of $v_i$.

Statistical analysis

The matching performance was compared by verifying the statistical significance between the rigid and non-rigid registration methods of the 3D vessel centerlines. A t-test was conducted to examine the significance of the accuracy, in which the statistical significance was confirmed when the P-value was less than 0.05. Statistical analysis was performed using Medcalc version 20.116 (Frank Schoonjans, Mariakerke, Belgium).

Experiments and results

This experiment was conducted on a system with an Intel Core i7-7800 CPU(3.2 GHz) RAM 32GB. Seven sets of left coronary artery (LCA) CTA images and four sets of right coronary artery (RCA) images photographed with a time difference on 11 patients were used as the experimental data.

Qualitative evaluation

Figure 5 shows the results of registering the blood vessels between 3D CTA images with the method proposed in this study. Figure 5a and b show the LCA and RCA registration results, respectively. The registration results visually confirm that the two 3D vessel centerlines extracted from the CTA images photographed with a time difference are well registered.

Fig. 5.

a LCA registration result. b RCA registration result

Quantitative evaluation

This study uses a distance-based error measurement method to measure the registration accuracy between registered 3D vessel centerlines. Distance-based error measurements are taken by calculating the Euclidean distance-based average of distance difference (ADD) using Eq. (7) between the registered results and the ground truth centerline manually divided in fixed images by an expert [23, 26].

$$ADD=\frac{1}{N}{\sum }_{i=1}^{N}E(p_i-q_i)$$ 7

.

Here, $N$ refers to the number of points constituting the input 3D vessel centerline, $p_i$ and $q_i$ are the matched points in the two 3D vessel centerlines extracted from the fixed and moving images, and $E(p_i-q_i)$ represents the Euclidean distance between these two points.

Figure 6 shows the error in average distance between the two 3D vessel centerlines after applying the rigid and non-rigid registration methods proposed in this study to each patient data. Compared to the case where only the rigid registration method was applied, the non-rigid registration method decreased the error in the left and right coronary arteries of the 11 patients by an average of 0.479 mm and 0.442 mm, respectively. These results indicate that the proposed method successfully enhanced the registration accuracy.

Fig. 6.

Error measurement results for the proposed rigid/non-rigid registration methods

Table 1 shows the registration performance results of the proposed method in this study and previous studies. The average registration error of the proposed method for the data of the 11 patients is 0.987 mm, and the standard deviation of the registration error is 0.397 mm. The average registration error in the left coronary artery is 0.913 mm, and the standard deviation is 0.213 mm. The average registration error in the right coronary artery is 1.061 mm, and the standard deviation is 0.582 mm.

Table 1.

Comparing the proposed method with previous studies

Author	Modality	No. of test subjects	Accuracy
Kaila et al. [27]	Biplane XA / CTA	7	1.41 mm (RMSE1)
Hadjiiski et al. [15]	CTA / CTA	22	LCA − 1.49 pixels RCA − 1.43 pixels
Luo et al. [16]	CTA / CTA	5	1.6 mm (ADD1)
Park et al. [13]	CTA / CTA	10	1.7964 mm
Proposed method	CTA / CTA	11	LCA − 0.913 mm RCA − 1.061 mm (ADD1)

^aRMSE: Root mean square error

^bADD: Average of distance difference

In the result of the comparison between the accuracy of the non-rigid registration method of the 3D vessel centerlines (proposed in this paper) and that of the rigid registration method, the p-value was 0.0129, confirming the statistical significance of the method proposed in this paper.

The imaging modality, accuracy measurement method, and additional data used for registration differ between the studies. However, we acquired superior performance compared to that of the method proposed in the most recent paper. Furthermore, the average time to conduct the proposed method in this study is 2.137 s.

Conclusion

In this study, we propose a non-rigid registration method that enables rapid and accurate registration by using the information on blood vessel characteristics in temporal CTA images of the same patient. First, global registration is conducted through rigid registration between the 3D vessel centerlines. This procedure minimizes the error in the blood vessel location that occurs owing to the patient posture during imaging and imaging protocols and reduces the time required for non-rigid registration. After global registration, the corresponding points between the two 3D vessel centerlines are defined by point matching using a gradient-based searching method. At this point, the point matching error caused by errors in the feature extraction process is reduced using an outlier removal method, which considers the thickness and gradient of the blood vessels to enhance the registration efficacy. Non-rigid registration maintains the topology through hierarchical local transformations and improves the registration accuracy by reducing the error caused by local deformation of the blood vessels according to breathing and CAD. Here, an energy function defined based on vessel characteristics is used to induce a natural blood vessel transformation.

The non-rigid registration method proposed in this paper was able to provide accurate blood vessel information compared to the rigid registration method, by correcting the location and shape of blood vessels that are deformed due to heartbeat and respiration through hierarchical local transformation.

The experimental results show that the average registration error is 0.987 mm, and the standard deviation is 0.397 mm. The average registration error is improved by 0.479 mm compared to that before non-rigid registration. The average conduction time is 2.137 s, indicating that the proposed method enables rapid and accurate registration of the two 3D vessel centerlines.

As a limitation of this study, since blood vessel feature information of the 3D CTA image is extracted using the Han et al. [2] method, the feature information that can be used for registration is limited. In the future, we will collect more data and study a blood vessel feature extraction method using the deep learning, which is being actively studied recently, and link it with registration method. It can be expected to improve the accuracy and processing time by utilizing additional blood vessel features and removing unnecessary computation.

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.