Multi-scale Regularization Approaches of Non-parametric Deformable Registrations

Abstract

Most deformation algorithms use a single-value smoother during optimization. We investigate multi-scale regularizations (smoothers) during the multi-resolution iteration of two non-parametric deformable registrations (demons and diffeomorphic algorithms) and compare them to a conventional single-value smoother. Our results show that as smoothers increase, their convergence rate decreases; however, smaller smoothers also have a large negative value of the Jacobian determinant suggesting that the one-to-one mapping has been lost; i.e., image morphology is not preserved. A better one-to-one mapping of the multi-scale scheme has also been established by the residual vector field measures. In the demons method, the multi-scale smoother calculates faster than the large single-value smoother (Gaussian kernel width larger than 0.5) and is equivalent to the smallest single-value smoother (Gaussian kernel width equals to 0.5 in this study). For the diffeomorphic algorithm, since our multi-scale smoothers were implemented at the deformation field and the update field, calculation times are longer. For the deformed images in this study, the similarity measured by mean square error, normal correlation, and visual comparisons show that the multi-scale implementation has better results than large single-value smoothers, and better or equivalent for smallest single-value smoother. Between the two deformable registrations, diffeormophic method constructs better coherence space of the deformation field while the deformation is large between images.

Introduction

Deformable registration is a method in computer vision that has many applications in medicine, e.g., the simulation of medical surgery, planning of medical intervention, detection of tumor growth, atlas-based segmentation, and recently adaptive radiation therapy. It is a deformable mapping process between two images such that the target image (T) can be warped through this deformation field into the reference image (R). One of the major difficulties in the reconstruction of the deformation field is due to the ill-posed property that the matching between the reference image and target images may have many solutions.

An ill-posed problem (i.e., the solution does not exist, is not unique, not stable, or not continuous) is normally solved with regularization methods. Regularization can be approached through optimization¹, filtering, and iterative methods. In the non-parameter deformable registration, Tikhonove regularization is a standard approach to restrict the solution to a computable subspace and provable uniqueness by constraining the energy of the solution’s derivatives. In other words, the aim of Tikhonove regularization is to find a differentiable function which is close to the original data by means of functional minimization. Bro-Nielsen et al.² and Florack et al.³ have demonstrated that a suitably regularized image can be regarded approximately as a scale space image at a particular scale (i.e., as a filtered image), in which the filter is some approximation of a normalized Gaussian.

Bro-Nielsen⁴ applied a convolution filter to solve the linear PDE of a fluid registration and came out an order of magnitude faster in calculation speed compared to Christensen's⁵ successive overrelaxation implementation of fluid registration. He also compared his fast fluid registration with Thirion⁶ demons registration. He found that the body forces of the two algorithms are almost the same and the Gaussian type low-pass filter used in demons actually corresponds to the linear elastic filter used in his fast fluid algorithm. Despite concluding that the two algorithms are similar, he pointed out that by applying the Gaussian filter instead of the real linear elastic filter, demons registration would have the problems in terms of topology and stability of the model.

To resolve the lack of invertibility of the output transformations by the demons algorithm, Vercauteren⁷ introduced the Lie Group structure in diffeomorphism space and derived the diffeomorphic image registration algorithm. Based on demons registration, this extended algorithm optimizes the global energy over a space of diffeomophism instead of the complete space of non-parametric spatial transformation. Both the demons and diffeomophic registration algorithms apply a Gaussian filter either on the transformation field or the update field. In the original implementation of Thirion⁶ and Vercauteren's⁷ algorithm, a single value of Gaussian smoother is applied iteratively (either with or without multi-resolution scheme). Since a Gaussian filter is a linear elastic regularization, this implementation is equivalent to the application of having single material stiffness during the entire image registration. A single value of Gaussian filter may have transformation which has local minimum and is not able to find the minimum globally.

In the application of tumor growth modeling, Cuadra⁸ regularized the deformation field with an adaptive Gaussian filter to avoid possible discontinuities. Kohlrausch⁹ proposed a method to cope with local as well as global differences in the images by varying the standard deviation of the Gaussian forces. In this study, we investigate multi-scale regularizations (smoothers or different material stiffness at different resolution level) during the multi-resolution iteration of symmetry demons and diffeomorphic algorithms and compare to original implementation of single-value smoother. In this approach, we expect the improvement of continuity and coherency of the deformation field and the registration results as well.

Methods

Experimental Data

Computed tomography (CT) scans of six patients (each patient has an average CT images of 400 × 300 × 70 voxels per CT scan) with large internal organ movement (five patients have 10∼12 sets of CT scans which were scanned with and reconstructed based upon the patient’s respiration cycle to form 4D CT images, one patient has two sets of CT scan which were scanned with full and empty bladder) were evaluated in this study. Each 4D CT images set was acquired at 0.5 s per revolution and were binned into at least 10 phases according to the patient's respiratory cycle. This process reduces the potential motion artifact within a single phase. The voxel size of each CT images set is 2.5 mm in the superior-inferior (SI) direction, ∼1 mm in the anterior-posterior (AP) and right-left (RL) directions. Since all CT images were acquired at the same time with the same position, no pre-registration were required to correct the mis-alignment of the patient orientation. Registration is intensity based and is performed at the entire CT volumes from two CT image sets (as reference image and target image). The average displacement of liver and lung and the bowel movements were 1.0∼1.5 cm. The volume change of the bladder was more than 250 cm³ with a maximum displacement of the bladder wall of more than 4 cm.

Non-parametric Deformation Registration

Demons Algorithm

Demons algorithm was proposed by Thirion¹ who was inspired by the optical flow equation with the idea that a regular grid of forces deform an image by pushing the contours (of iso-intensity) in the normal direction,

1

where, R(x) is the reference image (or fixed image in optical flow), T(x) is the target image (or moving image in the optical flow), and D(x) is the displacement (or optical flow) between the images. To prevent the equation becoming unstable for small values of the image gradient, resulting in large displacement values, Thirion re-normalizes the equation such that:

2

where, K is a normalization factor. Starting with an initial deformation field D⁰(x), the demons algorithm iteratively updates the field using Eq. 2 such that the field at the Nth iteration is given by:

3

To resolve the ill-posed problem of the image matching, the deformation field is smoothed with a Gaussian filter between iterations such that

4

where, K (σ_S) is the Gaussian kernel with width of standard deviation σ_S, u is the update field.

Diffeomorphic Algorithm

With the Gaussian filter, Thirion’s demons algorithm still does not ensure the invertibility of the output transformation. Vercauteren⁷ adapted the demons algorithm and made it diffeomorphic by optimizing the global energy over a space of diffeomophism instead of the complete space of non-parametric spatial transformation. In other words, instead of the spatial transformation with D^(N) = D^{(N − 1)}+u directly, an unconstrained update u is computed with Lie Algebra and is projected back onto the Lie Group through the exponential map:

5

In the implementation of Vercauteren’s diffeomorphic algorithm, a fluid like regularization is taken by smoothing the update field u with Gaussian kernel K (σ_G), and/or a diffusion-like regularization is taken by smoothing the deformation field D with a Gaussian kernel K (σ_S).

6

where, K (σ_S) and K (σ_G) are the Gaussian kernel with width of standard deviations of σ_S and σ_G, respectively. These Gaussian kernels K (σ) are represented by Gaussian probability function P(x) of standard deviation σ,

7

Multi-scale Smoother Scheme

Image registration is a process to minimize the similarity function (the similarity measure between reference image and target image). Often the function surface can be very unsmooth, having many sharp local minima, making it hard to find the overall global minimum. It would be easier to locate the minimum of a smoothed version of the function surface, which can then give a good starting point to locate the minimum of the original function. This study proposes a multi-scale smoother scheme and implements a demons and diffeomorphic registration method which is compared with the mono-smoother approach. Multi-scale smoother scheme takes the advantage of multi-resolution, where a coarse level with a larger (physical) step length is regularized with wider kernels to smooth the deformation field and/or the update field. The result at coarse level is used to seed the optimization working at finer levels, where the registration is more or less locally, and then the deformation field and/or the update field are regularized with a narrower kernel to allow flexible displacement. It is iterative until the solution is found on the original image with smoother of the least kernel size.

Multi-scale scheme was applied at different levels (total five, with reduction factor of 2 at each level: image resolution 516-256-128-64-32, from fine level to coarse level) of multi-resolution image registration process (256-128-64-32-16 iterations each). The coarse levels were applied with large smoothers; the finer levels were applied with smaller smoothers. In our implementation of Eqs. 4 and 6, σ_S (abbreviates as S in the following text and figures) represents the width of the Gaussian filter applied at the deformation field; it starts with 1.5 at the coarse level (resolution of 32), each level reduces linearly by a difference of 0.25 such that S equals 0.5 at the finest level. σ_G (abbreviates as G in the following text and figures.) represents the width of the Gaussian filter applied at the update field in Eq. 6; its values starts with 1.5 at the coarse level, each level reduces linearly by a difference of 0.5 such that no G was applied at the finest and second finest levels (in the software implementation, the smoothers of S or G were on only when their value are larger than 0.1).

Evaluation

The different algorithms with different regularization scheme were evaluated according to convergence rate, CPU calculation time (of Intel Quad 2.33 GHz CPU, 7 GB RAM), similarity measures (mean square error (MSE) and normal correlation (NC)), Jacobian determinant of the displacement vector field (DVF), and the residual vector field (RVF).

Mean Squared Error

The mean squared error is defined as the mean squared pixel-wise difference in intensity between image R and T over the interest region:

8

where, R_i and T_i are the ith pixels of reference image and target image, respectively, and N is the number of pixels considered. The optimal match between the two images produces a zero MSE and poor matches result in large values of the MSE.

Normalized Correlation

Normalized correlation computes pixel-wise cross-correlation and normalizes it by the square root of the autocorrelation of the images:

9

The negative value is introduced when NC is applied as the similarity metric during the iteration of image registration. The optimal value of the metric is then minus one. Misalignment between the images results in small measure values of NC.

Jacobian Determinant of the Displacement Vector Field

The DVF¹⁰ warps the target image (T) onto reference image (R). The Jacobian determinant of the DVF is the determinant of the gradient DVF at voxel volume location P.

10

J measures how a voxel volume changes after registration. The value of J can be understood as follows. For J = 1, there is no deformation. If J > 1, the volume increases; for 0<J<1, the volume decreases and for J < 0, the volume vanishes. In the real world of physical tissue under continuous movement, better image mapping should preserve the morphology of the tissue and prevent tissue from cracking (J→∞) or folding (J < 0).

Residual Vector Field

The RVF defines the difference between the displacement vector fields for the same reference and target images from two different routes. With more than three sets of simultaneous images acquired continuously, this measure compares the difference of the DVFs with image mapping from A to C directly and image mapping from A to B to C.

11

In this study (CT images acquired at the same time with the same position), the above two routes are identical mathematically and physically since they are from the same physical movement. With the exact displacement transformation, proper interpolation of A and C should obtain B. The hypothesis behind this measure is: minimum similarity difference (MSE, NC, etc.) from two different routes of mapping should have minimum similarity difference within the same route of mapping. i.e.,

12

where, T is equivalent to the applied DVF of the correspond target to reference image, S is the similarity measure (MSE, NC, etc.). Under this condition, the RVF should be a minimum, too.

Results and Discussion

Figure 1 compares the convergence rates of the different approaches from a lung case. Since the images in this study were produced from a mono-image modality (only CT images were used in this study), the similarity metric during the optimization of the registration iteration is MSE. From both of the graphs in Figure 1, the MSE curves are the same in the initial level if the starting smoothers are the same (eg, mono-smoother of S = 1.5 and multi-scale smoother which starts at S = 1.5); after the first level, the MSE curves become different since the smoother values are different (the mono-smoother keeps the same S value, the multi-scale approach reduce the S value according to the scheme). While the single small value smoother (S = 0.5 or S = 0.1) converges more quickly initially (i.e., in level 1), the similarity function curves are noisy which are tend to be trapped in local minimum and converge slower at higher levels; in contrast, the multi-scale scheme smoothes the deformation field with large size smoother at coarse level demonstrated smoother similarity function curve and converges faster overall (convergent rate increased at each level).

Fig 1.

Convergent rate of MSE vs. iteration at different level for different regularization scheme. a Demons method: the "S = 0.5–1.5" curve represent an opposite multi-scale scheme (initials with small width soother at coarse level and increases linearly the width of the smoother). b Diffeomorphic algorithm: the "S0.1/G0.1" curve is a registration process without smoother such that the curve is very noisy.z

The subtraction of two image sets of pelvic case with large bowel movement, before and after deformable registration using different smoother values of the demons method is illustrated in Figure 2. For a perfect registration, there should not be a difference between images before and after registration and the subtraction of the images should produce uniform view, i.e., there should not be a residual image. In Figure 2, the large differences (shown as bright and dark spots within the pelvic area) which are due to the bowel movement are dramatically reduced by the deformable registration (Fig. 2b-d). Due to the large deformation in this case, large smoother value (S = 1.5) still have large residual image after the subtraction. With smaller smoother (S = 0.5) applied to the deformation field, the residual image is less in the pelvic area. However, this small regularization scheme also shows larger residual image in the pelvic bone and on the body surface. In contrast, the multi-scale approach has fewer and lighter bright and dark spots within the subtraction image. By evaluating MSE from the subtraction of volume image before and after registration, the multi-scale approach has a value of 2,747, which is smaller than 7,419 and 3,050 produced after two mono-smoother approaches (see caption of Fig. 2).

Fig 2.

a Image difference of pelvic bowel movement before registration. b Image difference after demons registration of large mono-smoother (S = 1.5; MSE 7419; NC −0.964; Min J −0.048; Max J 3.22). c Image difference after small mono-smoother (S = 0.5; MSE 3050; NC −0.985; Min J −6.7; Max J 24.34). d Image difference after multi-scale smoother scheme (MSE 2747; NC −0.988; Min J −15.21; Max J: 21.79).

Figure 3 shows the overlay of the reference image (at the exhalation phase) with the displacement vector field (from the inhalation phase). The images on the right (3a and 3c) are produced using the multi-scale scheme and the left sides (3b and 3d) are the images generated using the single-value scheme. The details of the registration measured with different similarities with MSE and NC are in the caption. This figure demonstrates the difference of the deformation field constructed from multi-scale smoothers and mono-smoothers, respectively. The vectors overlaid on the image represent the magnitude and direction of the displacement between reference and target image. The vectors change direction and magnitude more in Figure 3b and d than Figure 3a and c indicating that the local morphological mapping is unstable with single-value scheme of small smoother (S = 0.5 in these cases). Additionally, in Figure 3b and d, there are significant displacements within the bone and muscle (circled area in figures) which is very unlikely since the patient were scanned without movement except the respiratory motion of the lung and chest wall, and the cardiac motion.

Fig 3.

a Diffeormorphic registration with multi-scale smoother scheme (MSE 2326, NC −0.995; Min J 0.0008; Max J 17). b With mono-smoother: S = 0.5 and without G applied, (MSE 3899, NC −0.991; Min J −4.6; Max J 41). c Demons registration with multi-scale scheme (MSE 2710; NC −0.994; Min J −10.8; Max J:19). d Demon registration with mono-scale smoother: S = 0.5 (MSE:5796; NC −0.987; Min J −16.9; Max J 23).

Table 1 shows a comparison of the different approaches for the efficiency, different similarity measures and the range of J for a variety of different single-value smoother (S) in the first four columns and the multi-scale smoother in the right most column. In terms of the CPU time, the multi-scale approach on the demons method is the most efficient method; while the multi-scale approach using the diffeomorphic algorithm is the most calculation intensive method. The experimental data is a relatively large deformation. The registration results show that smaller S is better in terms of MSE and NC. Too small S (0.5) value leads to large negative J value in Demons method. The diffeomhophic algorithm preserves better image morphology with very few J < 0 (i.e., the minimum J is close to 0 for smoother larger than 0.5), however, without any smoother (S = 0.1, G = 0.1) still leads unstable deformation which is not physically realistic. By applying a paired t test to compare multi-scale smoother approach with other mono-smoother approaches in different similarity measures (MSE and NC), the results show that multi-scale approaches significantly improve NC for both of demons and diffeomorphic methods. In terms of demons method measures with MSE, multi-scale approach is better than mono-smother approaches, as well multi-scale approach does not significantly improve the diffeomorphic method compared with the mono-approaches with small smoothers (S = 0.5 and S = 0.1), however, the minimum J value of the multi-scale scheme proposed in this study indicates that this approach constructs better coherence space of the deformation field than the one without any smoother (S = 0.1). The following examples further demonstrate the points made in this table.

Table 1.

Mean Registration Results of Six Patients in this Study. Upper is the Diffeomorphic Algorithm, Lower is the Demons Method

	S = 1.5;G = 1.5	S = 1;G = 1	S = 5; no G	S = 0.1;G = 0.1	Multi-scale
Time (s)	233 ± 50	226 ± 55	170 ± 36	204 ± 44	249 ± 51
MSE	4811 ± 2457	3853 ± 1843	2275 ± 1456	2242 ± 1127	1888 ± 879
p valuea	0.000	0.000	0.081	0.065
NC	−0.975 ± 0.017	−0.98 ± 0.014	−0.988 ± 0.011	−0.989 ± 0.008	−0.99 ± 0.008
p valueb	0.022	0.014	0.044	0.039
Min J	0.20 ± 0.06	0.08 ± 0.04	−0.01 ± 0.04	−147.48 ± 304.7	0.001 ± 0.00
Max J	3.27 ± 0.91	5.21 ± 1.43	21.93 ± 9.97	375.75 ± 511.85	18.24 ± 4.69
	S = 0.5–1.5	S = 1.5	S = 1.0	S = 0.5	Multi-scale
Time (s)	96 ± 20	96 ± 18	100 ± 16	89 ± 16	89 ± 18
MSE	4212 ± 2102	4185 ± 2096	3170 ± 1458	2638 ± 1406	1953 ± 720
p valuea	0.001	0.001	0.001	0.048
NC	−0.978 ± 0.013	−0.978 ± 0.014	−0.984 ± 0.01	−0.988 ± 0.004	−0.99 ± 0.005
p valueb	0.018	0.024	0.023	0.025
Min J	−0.23 ± 0.23	−0.21 ± 0.24	−0.92 ± 0.30	−24.73 ± 7.59	−13.86 ± 2.41
Max J	3.86 ± 0.82	3.80 ± 0.84	6.08 ± 1.52	37.67 ± 13.52	24.09 ± 5.4

The S = 0.5–1.5 is an opposite direction of multi-scale which shows no benefits

S = 0.1 and G = 0.1 the example to show the effect of no smoother at all during the iterations

^ap value of the paired sample t test at MSE for the mono-smoother with multi-scale smoother

^bp value of the paired sample t test at NC for the mono-smoother with multi-scale smoother

Figure 4 displays the registered images of the empty bladder image morphed into the full bladder image with the overlay of the parsed displacement field. Since the optical flow-based algorithm such as the demons and the diffeomorphic method requires the reference image and the target image to have similar intensity distributions, it is difficult to recover the intensity of the bladder since the intensity has great difference between empty and full bladder. When the smoother is too big (S = 1.5, G = 1.5 in this case), the bladder shape cannot be restored. When the smoother is small, the optimization is less constrained on the deformation field to allow flexible displacement, however, only the diffeormophic one of multi-scale (S starts from 1.5, end at 0.5; no fluid-like smoother G was applied at each level in this case) approach shows sensible topology of the displacement field and shows the best restoration of the bladder shape. This example expresses that the multi-scale scheme of diffeormophic method constructs a better coherent deformation field while the deformation is big.

Fig 4.

a Empty bladder (target) image, b full bladder (reference) image. c∼h show the registered (empty to) full bladder image (DVF is overlaid) with the regularization scheme labeled on the image. The corresponding similarity measures are c, MSE = 1964, NC = −0.988, Min J = 0, Max J = 8.4; d MSE = 784, NC = −0.995, Min J = -1.29, Max J = 63.35; e MSE = 848, NC = −0.995, Min J = −4, Max J = 91.02; f MSE = 1044, NC = −0.994, Min J = −61.63, Max J = 372; (g), MSE = 1381, NC = −0.992, Min J = −18.76, Max J = 27.25; e MSE = 2013, NC = −0.988, Min J = −29.64, Max J = 42.74.

Figure 5 further illustrates the difference of the deformation results of the previous bladder case through the distribution of the Jacobian determinant of the DVF, which only covered the major deformed area within the pelvis. Figure 5a and b compare the J distribution of S = 0.5 with multi-scale approaches. They show similar distributions (p = 0.008 in both cases) except multi-scale have more voxel counts around J = 1 (i.e., no change of the volume or no deformation) and less big tissue volume increasing (J > 10), and tissue volume folding (J < 0). The S = 0.5 has a larger range of deformations, with a greater range of volume changes after deformable registration. Figure 5c compares the difference of J distribution between two different deformable registration algorithm in this study, although the two curves are similar in large scale (p = 0.01), it still shows improvement of diffeomorphic algorithm over the demons method from preventing the tissue folding (J < 0) after the registration. Figure 5d displays two extreme cases of J distribution for S = 1.5 or S = 0.1, which show the different results (p = 0.248) of over regularization or no regularization of the deformation field, respectively. Although the registration of S = 0.1 (& G = 0) has similar results compared to S = 0.5 (& G = 0) and multi-scale approach, if they were measured with NC (caption of Fig. 4), their J distributions (not shown in Fig. 5) are different with p values of 0.06 and 0.07, respectively.

Fig 5.

The J distribution of different regularizations. P values were obtained by performing chi-square test of the two curves after re-binning the curves into J < 0 (volume folding), J = [0,0.2] (big volume shrink), J = [0.2,0.5] (medium tissue shrink), J = [0.5,1.0] (mild volume shrink), J = [1,2] (volume unchanged), J = [2,5] (mild volume increase), J = [5,10] (medium volume increase), and J > 10 (big volume increase); *indicates multi-scale scheme with diffeomorphic method; **multi-scale scheme with demons method.

Figures 6 and 7 illustrate the results of a self-consistency test of the deformable algorithms in this study by comparing the residual displacement field from different routes. The experiment data is the CT images of a liver case at phase 0 (the inhalation phase), phase 1, phase 2, phase 3, phase 4, and phase 5 (the exhalation phase). Figure 6 displays the results of mapping route from phase 5 to phase 0 against the mapping route from phase 5 to phase 3 combined with the route from phase 3 to phase 0. The direct route from phase 5 to phase 0 (Fig. 6d) has the largest displacement (absolute mean of 8.6, 1.6, and 3.4 mm in the SI(Z), RL(X), and AP(Y) directions, respectively). The indirect route from phase 3 to phase 0 (Fig. 6b) has a large displacement (absolute mean of 6.6 mm (Z),1.4 mm(X) and 2.4 mm(Y)) as well. Figure 6h shows the exhalation period from phases 5 to 3, and has the least displacement (absolute mean: 2.2 mm(Z), 1.1 mm (X), 1.7 mm (Y)). The residual vector field (RVF(5:0; 5:3:0)) is overlaid on phase 5 and is shown in Figure 6i. This measure reveal the intrinsic error of the algorithm which include the error propagation form phase 5 to phase 0, phase 5 to phase 3, and phase 3 to phase 0. This RVF (5:0; 5:3:0) is expressed as (P5-P3) in the later analysis.

Fig 6.

Different phases of images are shown at a, e, g, and i. The registered images (b, d, h) and their residual DVF are the results from diffeomorphic algorithm. The registered images of demons method are shown on c and f. Both methods have similar results within soft tissue; however, demons method is less accurate in recovering the position of the marker (bright dots in the posterior of the liver, as c or f was compared with d).

Fig 7.

Box plot of RVF on three separate X, Y, Z components; a is the diffeomorphic algorithm; b is the demons method. From bottom to the top is Q(1%), Q1(25%), Q2(50%), Q3(75%), and Q(99%), “o” marked the absolute average residual error (|D|, see text). Both of the multi-scale schemes have 0.3∼0.5 mm (1/3∼1/2 voxel size) better than the single S (=0.5) in terms of |D| and the QD (quartile difference, i.e., 0.5(Q3-Q1)).

The analysis the RVF at different X, Y, Z components is shown in Figure 7. There are total of 1,440,000 voxels with dimension of 0.98 × 0.98 × 2.5 mm per voxel in each data set. The box plots include the 1% quartile (D(Q1%)), 25% quartile (D(Q1)), 50% quartile (D(Q2)), 75% quartile (D(Q3)) and the 99% quartile (D(Q99%)) of the RVF. The average absolute residual error (|D|) of each RVF for different phase route and different regularization (multi-scale or single smoother, demons or diffeomorphic algorithm) is marked on the plots, too. The overall mean residual errors (|D| ± 1 SD, for the diffeomorphic method) from phase 1 to phase 4 are 1.38 ± 0.33, 1.38 ± 0.3 , and 1.9 ± 0.47 mm at X, Y, and Z direction, respectively. They are 1.4 ± 0.39, 1.41 ± 0.35, and 1.54 ± 0.45 mm using the demons method. These residual errors include the propagation of error from three routes (phases 5 to 0, phases 5 to 3, and phases 3 to 0). The residual error of the mapping within each route should be approximately half of the voxel size at each dimension. These results are compatible to the report of other deformable registration methods^8–10 evaluated either from synthetic image or from those with marker within the image. The difference between multi-scale and single-value smoother is that the multi-scale one has smaller range of RVF in terms of the range between Q1% and Q99% or Q1 and Q3 (or OD in Fig. 7). In terms of the ranges of Q99% and Q1%, for diffeomorphic method with multi-scale approach and mono-smoother approach, Q99% = 11.1 ± 3.6 mm and Q1% = 14.1 ± 3.5 mm; for demons method, Q99% = 8.9 ± 1.3 mm and Q1% = 14.0 ± 2.7 mm. In terms of OD, for diffeomorphic method with multi-scale approach and mono-smoother approach, they are 0.9 ± 0.3 mm and 1.1 ± 0.3 mm, respectively; for demons method, they are 0.8 ± 0.2 and 1.2 ± 0.2 mm, respectively. With significant level of p value <0.05, RVFs are significantly improved by using the multi-scale approach for both of diffeomorphic and demons methods in comparing the range between Q1% and Q99% or Q1 and Q3 (all p < 0.0001). This could be interpreted as the multi-scale method having stable one-to-one mapping compared to the single smoother approach. Both of the demons and diffeomorphic method have similar RVF results in this liver case; however, with visual comparison, the diffeomorphic method is more accurate in recovering the marker position inside liver (Fig. 6c and d)

Conclusions

This study implemented a multi-scale scheme over the demons and diffeormophic deformable registration. In terms of the similarity measured by MSE, NC, and visual comparison, the multi-scale implementation has much better results than large smoothers, better or equivalent than the smallest smoother. Different smoother values change the behavior of the displacement vector field drastically; i.e., a low value of smoother will lose the smoothing power in the deformation space such that a well topology of the deformation field won’t be maintained. Multi-scale regularization combined with multi-resolution has the advantage of preserving the morphology of the images and the registration results. While there is large deformation between images, diffeormophic registration constructs better coherence space of the deformation field. Both of the demons and diffeormophic methods are intensity based registration, which works fine when both of the target and reference image have similar intensity distribution within the entire volume of images. A major intensity difference within organs (e.g., an image set with contrast injected to organ vs. image set without contrast or a full bladder vs. empty bladder) may reduce the accuracy of the registration. A hybrid combined feature- and intensity-based registration methods may help to improve accuracy.

A robust deformable registration can help to monitor the response of a tumor during radiation treatment and then adapt the changing shape for modification of treatment.¹⁰ A motion model acquired after deformable registration can be applied to evaluate the impact of dose coverage due to respiratory^11,12 motion. The experiments in this study cover the most frequent intra- or inter-treatment organ motion in the daily practice of radiation therapy.^13–15 The analysis results also validated implementation of the deformable registration with either demons or diffeomorphic method in clinical radiation therapy.