Few-shot learning for deformable image registration in 4DCT images

Abstract

Objectives:

To develop a rapid and accurate 4D deformable image registration (DIR) approach for online adaptive radiotherapy.

Methods:

We propose a deep learning (DL)-based few-shot registration network (FR-Net) to generate deformation vector fields from each respiratory phase to an implicit reference image, thereby mitigating the bias introduced by the selection of reference images. The proposed FR-Net is pretrained with limited unlabeled 4D data and further optimized by maximizing the intensity similarity of one specific four-dimensional computed tomography (4DCT) scan. Because of the learning ability of DL models, the few-shot learning strategy facilitates the generalization of the model to other 4D data sets and the acceleration of the optimization process.

Results:

The proposed FR-Net is evaluated for 4D groupwise and 3D pairwise registration on thoracic 4DCT data sets DIR-Lab and POPI. FR-Net displays an averaged target registration error of 1.48 mm and 1.16 mm between the maximum inhalation and exhalation phases in the 4DCT of DIR-Lab and POPI, respectively, with approximately 2 min required to optimize one 4DCT. Overall, FR-Net outperforms state-of-the-art methods in terms of registration accuracy and exhibits a low computational time.

Conclusion:

We develop a few-shot groupwise DIR algorithm for 4DCT images. The promising registration performance and computational efficiency demonstrate the prospective applications of this approach in registration tasks for online adaptive radiotherapy.

Advances in knowledge:

This work exploits DL models to solve the optimization problem in registering 4DCT scans while combining groupwise registration and few-shot learning strategy to solve the problem of consuming computational time and inferior registration accuracy.

Introduction

Four-dimensional computed tomography (4DCT) has been employed in diverse clinical applications, including radiation treatment planning and tumor motion tracking for various disease sites.¹ To alleviate the delivery errors caused by respiration, 4DCT provides three-dimensional CT volume sequences at different respiratory phases to realize accurate radiotherapy planning and delivery. The patient-specific respiratory motion information obtained from the 4DCT allows accurate generation of planning target volume for treatment planning. Research has demonstrated that 4DCT facilitates the reduction of the radiation dose for lung normal tissues and enhances the treatment outcomes.² However, additional time-varying data in 4DCT increase the workload and planning time to formulate the 4D treatment plan.³ These requirements necessitate the use of advanced automatic processes for radiotherapy planning, such as contour delineation and treatment planning.

Deformable image registration (DIR) is a potential technique to promote the automation of 4D radiotherapy planning.^4–6 Based on geometric and intensity properties, DIR constructs the voxel to voxel correspondence between the moving and reference images and propagates the contours and dose distribution from reference images to other respiratory phases.^7,8

State-of-the-art DIR could be categorized into conventional and deep learning (DL)-based methods. Conventional 3D registration methods, such as the Demons algorithm,⁹ neglect the temporal coherence across different respiratory phases and incur a potential bias towards a chosen reference image, thereby degenerating the registration performances of 4DCT.¹⁰ To solve these problems, groupwise registration simultaneously registers all phase images to a common space to mitigate the uncertainties associated with the reference image.^10–12 However, conventional groupwise registration through iterative optimization is time-consuming and cannot be practically adopted in online 4D registration applications.

DL has gained considerable interest and has been adopted successfully to perform medical image registration.^13,14 For example, VoxelMorph was proposed to rapidly compute the deformation vector fields (DVFs) between pairwise medical images by using a convolutional neural network.¹³ However, compared to conventional methods, DL-based methods require a large amount of training data and fail to effectively register the unseen images during the inference. Therefore, one-shot registration was proposed and iteratively optimized the parameters of an untrained neural network with the 4D medical images to be registered.^15,16 Nevertheless, these one-shot methods are incapable of leveraging the capability of DL models to learn prior knowledge from the training data and require considerable optimization time as conventional methods.

To overcome the difficulties of time-consuming processes in conventional methods and inferior performances in DL-based methods, we propose a DL-based few-shot groupwise registration network (FR-Net). The proposed FR-Net advances one-shot registration by pretraining the model using limited unlabeled 4D data and further optimizing the registration problem of one specific 4DCT scan until convergence. The pretraining strategy facilitates the acceleration of the optimization processes in the test stage while remaining competitive registration accuracy with state-of-the-art methods. In the following sections, we describe the implementation of the proposed FR-Net and the comprehensive experiments conducted to evaluate the effectiveness and efficiency of the model.

Methods and materials

DL-based few-shot groupwise registration

Consider a 4D (3D + time) medical image sequence $I^N$ , consisting of N 3D images $I_n$ in different respiratory phases. The DIR algorithms for 4D images aim to find an optimal deformation field ${\Phi }_n$ for each phase image $I_n$ to align with the reference image $I_{ref}$ . The groupwise registration scheme obtains the DVFs ${\Phi }^N$ by maximizing the intensity similarity between the 4D image set $I^N$ and reference image $I_{ref}$ in a single optimization procedure:

,$${\widehat{\Phi }}^N=\mathrm{a}\mathrm{r}\mathrm{g}\underset{{\Phi }^N}{\max }S\left(I^N\circ {\Phi }^N,\mathrm{}\mathrm{}{I}_{ref}\right)$$

where $S\left(\bullet \right)$ is the similarity measure function, and $I\circ \Phi$ denotes the transformation operation applied to image $I$ with the corresponding DVF $\Phi$.

To accelerate the optimization process and improve the registration performance, we extend the one-shot registration to few-shot learning with the help of the learning ability of DL models. In contrast to state-of-the-art unsupervised methods for registration, the proposed method only requires a few training data to pretrain an initial model in the training stage. Subsequently, in the test stage, the pretrained model is fine-tuned (optimized in fewer iterations) to ensure the model convergence and generate the optimal DVFs on the test image set.

Figure 1 illustrates the proposed few-shot registration approach. In the training stage, we first downsample the 4D images to accommodate the GPU memory limitation while ensuring that the proposed model extracts the global feature maps with both spatial and temporal correlations. The proposed FR-Net inputs the downsampled images (each phase image as a single channel) and compute the corresponding DVF mapping from each phase image to the reference image. And a spatial transformer¹⁷ is adopted to deform the 4D images to the reference image with the predictive DVFs upsampled to the original image dimension. Moreover, the FR-Net is trained through the loss function between warped 4D images and the reference image through backpropagation. In the test stage, the pretrained FR-Net is further optimized exclusively on the new 4D images and generates the corresponding deformation fields. After the model converges on the test data, the deformed image $I_m^n$ from the m^th phase to the n^th phase can be computed for the evaluation, as follows: $I_m^n=(I_m\circ {\Phi }_m)\circ {\Phi }_n^{-1}$ ,

Figure 1.

Framework of the proposed DL-based few-shot registration method. DL, deep learning; FR-Net, few-shot registration network.

where $I_m\circ {\Phi }_m$ is the warped image of the m^th phase aligned with the reference image, and ${\Phi }_n^{-1}$ is the inverse DVF from the reference image to the n^th phase image, estimated using a fixed-point approach.¹⁸

Few-shot registration network

We design a few-shot registration network based on U-Net¹⁹ to simultaneously predict the DVFs for 4D images of all respiratory phases. The architecture of the proposed FR-Net is shown in Figure 2.

Figure 2.

Architecture of the proposed FR-Net. Blue boxes represent multi channel feature maps with the number of channels denoted on the top. The feature size (height ×width × depth) is presented at the lower left edge of the box. The arrows denote the different operations in the legend. FR-Net, few-shot registration network

The FR-Net inputs the whole deformation region (e.g. lung) to exploit the spatial and temporal coherences of 4D images. The proposed model consists of an encoder and a decoder with the skip connection operation to integrate both the high-and low-level contexts for the final prediction. The encoder includes four repeated convolutional blocks, each consisting of a 3 × 3 × 3 convolutional layer, an instance normalization layer, and a leaky rectified linear unit (LReLU) activation function. The number of initial output channels of the first convolutional layer in the encoder is 32, and we double the number of channels for the following convolutional layers after downsampling the feature maps through the max-pooling layers. In each level of the decoder, we exploit a convolutional block and a squeeze-and-excitation block (SE-block)²⁰ and halve the number of channels of the convolutional layers after each transposed convolutional layer. The SE-blocks adaptively recalibrate the channel-wise feature representations and enhance the registration performance by explicitly modeling the interdependencies among the channels. To avoid losing critical features during the downsampling in the encoder, skip connections are applied to concatenate the feature maps from the encoder of the same level and the decoder of a higher level.

Loss functions

To compute the optimal DVFs for 4D images, the FR-Net is trained by minimizing the following loss function:

,$$L_{total}=-S\left(I^N\circ {\Phi }^N,\mathrm{}\mathrm{}{I}_{ref}\right)+{\lambda }_0{L}_{smooth}(I^N,\mathrm{}\mathrm{}{\Phi }^N)+{\lambda }_1{L}_{cycle}\left({\Phi }^N\right)$$

where $L_{smooth}$ and $L_{cycle}$ are the smoothness and cycle regularization, respectively, and ${\lambda }_0$ and ${\lambda }_1$ are the corresponding regularization parameters to balance the effects of the regularization terms.

The similarity measure function $S\left(\bullet \right)$ averages the normalized cross-correlation (NCC) between the warped images of each respiratory phase and the reference image. In particular, NCC is used to determine the local image average and variance, and has shown to be effective in many template matching applications such as image registration.²¹ $I_{ref}=\frac{1}{N}{\sum }_n{I}_n\circ {\Phi }_n$ represents the implicit reference image to be registered in the common space by averaging the warped 4D images in each iteration. This approach helps achieve a more effective transformation than that obtained using the reference-based registration methods.²²

The smoothness regularization term is defined as follows:

,$${\displaystyle L_{smooth}\left(I^N,{\mathrm{\Phi }}^N\right)=\frac{1}{3NHWD}\sum _n\sum _v\sum _{d\in x,y,z}{\Vert {\mathrm{\nabla }}_d{\mathrm{\Phi }}_n\left(v\right)\Vert }_1\cdot e^{-\left|{\mathrm{\nabla }}_d{I}_n\left(v\right)\right|}}$$

where v denotes the voxels in the 4D images and H, W, D are the height, width, and depth of the image, respectively. This term smooths the predictive DVFs and enforces the warped images to be similar to the neighboring voxels when no significant image gradient exists. The cycle regularization loss is designed to constrain the implicit reference image in the center of the 4D images as follows:

.$$L_{cycle}\left({\Phi }^N\right)=\sqrt{\frac{1}{3HWD}\sum _v\sum _{d\in x,y,z}{\left(\sum _n{\Phi }_{n,d}\left(v\right)\right)}^2}$$

Experimental data

We evaluate the proposed method on the thoracic 4DCT images from public domain data sets DIR-Lab^11,23 and POPI.²⁴

The DIR-Lab data set consists of 10 thoracic 4DCT scans, each with 10 respiratory phase images. The in-plane pixel resolution of the 4DCT scans varies between 0.97 and 1.16 mm, and the slice thickness is 2.50 mm. 300 pairs of landmarks are manually annotated in the end-inhalation (EI) and end-exhalation (EE) phases for the evaluation. Moreover, subsets of 75 landmarks are marked on each of the expiratory phase images.

The POPI data set consists of six thoracic 4DCT scans involving ten 3D volumes in different respiratory phases. The in-plane pixel resolution of the 4DCT scans varies between 0.78 and 1.17 mm, and the slice thickness is 2.00 mm. 100 landmarks are provided in each of 10 frames of the 4DCT of the first three patients; for the remaining three patients, the corresponding landmarks are marked at the EI and EE phases.

Experiment design and evaluation metrics

We conduct experiments on the DIR-Lab and POPI data sets to evaluate the accuracy and efficiency of FR-Net in 4D groupwise and 3D pairwise registration. In 4D groupwise registration process, the entire 4DCT set with all the respiratory phases is used as the input of the proposed FR-Net (each phase image as a single channel). To verify the performance of the few-shot registration in 3D pairwise registration, we train and test the FR-Net on two extreme phase images (i.e. the EI and EE phases). To perform a comprehensive evaluation, we compare the proposed method to the one-shot registration,^15,16 learning-based LungRegNet²⁵ and conventional approaches.^12,24

To evaluate the registration accuracy, the target registration error (TRE) is calculated using the Euclidean distance between the pairs of corresponding landmarks before and after registration. We calculate the TRE of the 4D registration between consecutive phase images and between the EI and EE phase images, denoted as TRE-4D and TRE-3D, respectively. Moreover, the computational time is recorded to assess the efficiency of the proposed FR-Net compared to the one-shot registration methods.

Implementation details

The proposed FR-Net algorithm is implemented with the DL library in PyTorch (v. 1.6.0) and Python (v. 3.7.3). Experiments are conducted on a Windows workstation with an NVIDIA GeForce 2080 Ti GPU with 11 GB memory. The 4DCT images with less than 128 slices are padded with the most superior and inferior slices. Due to the GPU memory limitation and different volume dimensions of images, the thoracic 4DCT series are cropped to a consistent size of 192 × 256×128 centering on the lung region. To avoid gradient explosion during optimization, the Hounsfield units of the CT images are normalized to the range from −1 to 1.

The FR-Net is trained on five patients in the DIR-Lab data set to learn prior knowledge, while the remaining five patients act as the test data. Six 4DCT scans in the POPI data set are evaluated in the test stage to demonstrate the FR-Net performance across data sets (training and testing on data from different institutions). We train the FR-Net using the Adam optimizer at an initial learning rate of 10⁻² until convergence. Specifically, the iterative optimization stops if: (a) the standard deviation $\sigma$ of $N_{stop}$ latest total losses is smaller than the threshold ${\sigma }_{stop}$ ; and (b) the total loss at the current iteration is smaller than the minimum total loss plus ${\sigma }_{stop}/3$ . The hyperparameters $N_{stop}$ and ${\sigma }_{stop}$ are empirically set as 100 and 0.001 for both training and test stages. To ensure the between the computational time and registration accuracy, the values of $N_{stop}$ and ${\sigma }_{stop}$ for FR-Net-early-stop during the test stage are 50 and 0.01, respectively. In addition, the weights of the smoothness and cycle regularization loss ${\lambda }_0$ and ${\lambda }_1$ are empirically set as 10⁻³ and 10⁻² for 4D groupwise registration and 10⁻¹ and 10⁻² for 3D pairwise registration, respectively. The batch size is one during the training and test stages.

Results

4D groupwise registration

As shown in Tables 1 and 2, we compare the proposed few-shot registration method with state-of-the-art methods on the DIR-Lab and POPI data sets. For the proposed FR-Net, the average landmark distances between the maximum inhalation and exhalation phases are reduced from an initial value of 10.43 to 1.31 mm on the DIR-Lab data set and from 8.12to 1.08 mm on the POPI data set. The few-shot FR-Net demonstrates a comparable registration accuracy to that of the one-shot GroupRegNet while decreasing the computational time by half, which will be shown in the following experiments. Compared with the one-shot registration method proposed by Fechter et al., FR-Net decreases the TRE-3D by 0.88 mm and 0.41 mm on the DIR-Lab and POPI data sets, respectively. And the proposed FR-Net outperforms the learning-based LungRegNet and conventional approach proposed by Vandemeulebroucke et al. In the evaluation of the registration to consecutive phase images, FR-Net achieves lower registration errors than the other two methods on both data sets, demonstrating the high accuracy of the proposed few-shot groupwise registration scheme.

Table 1.

Comparisons of the target registration error between extreme phase images (TRE-3D) for FR-Net and state-of-the-art methods on test data setsSee the Supplementary Material 1 for more results

Data set	Before Reg.	FR-Net	GroupRegNet16	Fechter et al.15	LungRegNet25	Spatiotemporal Reg.24
DIR-Lab	10.43 ± 7.45	1.31 ± 0.99	1.31 ± 0.98	2.19 ± -	2.00 ± -	-
POPI	8.12 ± 4.77	1.08 ± 0.73	1.10 ± 0.75	1.49 ± 1.59	-	1.46 ± 1.65

FR-Net, few-shot registration network.

See the Supplementary Material 1 for more results.

Table 2.

Comparisons of the target registration error between consecutive phase images (TRE-4D) for FR-Net and state-of-the-art methods on test data sets

Data set	FR-Net	Fechter et al.15	Metz et al.12
DIR-Lab	1.56 ± 1.42	1.74 ± -	1.83 ± -
POPI	1.10 ± 0.80	1.24 ± 0.90	1.11 ± 0.87

FR-Net, few-shot registration network.

See the Supplementary Material 1 for more results.

Figure 3 illustrates the changes in the TRE-3D as the GroupRegNet and FR-Net optimized the registration problem of the 4DCT data in the DIR-Lab and POPI data sets. With the prior knowledge learned from training data, the pretrained FR-Net starts with lower initial landmark distances than GroupRegNet before the optimization. FR-Net reaches the optimal solution in ~100 s, while GroupRegNet requires more than 300 s to achieve similar results. Although both methods can maximize the similarity between any two phase images in the 4DCT, trivial improvements in the registration accuracy are achieved at the expense of considerable time.

Figure 3.

TRE-3D vs computational time of the proposed FR-Net and GroupRegNet during the optimization of a thoracic 4DCT in the DIR-Lab and POPI data sets. DIR, deformable image registration; FR-Net, few-shot registration network; TRE, target registration error

To further reduce the computational time for 4D registration, we adopt the early termination strategy in the optimization process, denoted as FR-Net-early-stop. Table 3 presents the results of the TRE and optimization time for GroupRegNet, FR-Net and FR-Net-early-stop. FR-Net incurs 44.9 and 50.1% of the computational time required by GroupRegNet to optimize the 4DCT scans in both data sets, demonstrating the efficiency of the proposed few-shot registration method. With the early termination strategy, FR-Net-early-stop significantly decreases the mean computational time from 902.2to 128.3 s on the DIR-Lab data set, with an increase of only 0.09 mm and 0.17 mm in the TRE-4D and TRE-3D, respectively. Similar observations can be made on the POPI data set. The proposed FR-Net-early-stop considerably reduces the computational time with only a slight reduction in the registration accuracy on both data sets. The computational efficiency and promising performance of FR-Net-early-stop illustrate its potential capacity in online registration applications.

Table 3.

Comparisons of target registration error and optimization time for GroupRegNet, FR-Net and FR-Net-early-stop for 4D groupwise registration

Data set	Method	TRE-4D (mm)	TRE-3D (mm)	Time (s)
DIR-Lab	GroupRegNet16	1.55 ± 1.42	1.31 ± 0.98	902.2
	FR-Net	1.56 ± 1.42	1.31 ± 0.99	405.2
	FR-Net-early-stop	1.64 ± 1.43	1.48 ± 0.95	128.3
POPI	GroupRegNet16	1.11 ± 0.82	1.10 ± 0.75	661.4
	FR-Net	1.10 ± 0.80	1.08 ± 0.73	331.4
	FR-Net-early-stop	1.17 ± 0.85	1.16 ± 0.82	120.9

DIR, deformable image registration; FR-Net, few-shot registration network; TRE, target registration error.

Table 4 summarizes the average registration error and computational time of the proposed few-shot registration when applied to the extreme case with only one 4DCT data used for training. Compared with GroupRegNet, the computational time is decreased from 902.2 to 273.1 s on the DIR-Lab data set and from 661.4 to 238.5 s on the POPI data set. Experimental results indicate the potential application of the proposed method to 4D registration even when few data are available for training.

Table 4.

Performance of the proposed method when using one 4DCT scan of the DIR-Lab data set to train the model

Data set	Size of training set	TRE-4D (mm)	TRE-3D (mm)	Time (s)
DIR-Lab	1	1.65 ± 1.43	1.51 ± 1.01	273.1
	5	1.64 ± 1.43	1.48 ± 0.95	128.3
POPI	1	1.18 ± 0.85	1.22 ± 0.79	238.5
	5	1.17 ± 0.85	1.16 ± 0.82	120.9

The mean target registration error and computational time and the corresponding standard deviation on the DIR-Lab and POPI data sets are presented.

DIR, deformable image registration; FR-Net, few-shot registration network; TRE, target registration error.

^aD pairwise registration.

To apply the FR-Net to 3D pairwise registration, only the images of maximum inhalation and exhalation in 4DCT are used as the input of the FR-Net. The results for the application of the proposed method in pairwise registration are listed in Table 5. FR-Net-early-stop significantly decreases the average landmark distances between the maximum inhalation and exhalation phases from 10.43 mm before registration to 1.31 mm on the DIR-Lab data set and from 8.12 to 1.11 mm on the POPI data set. Moreover, this approach achieves the same average registration error as that of GroupRegNet on the DIR-Lab data set while consuming only 29.4% of the computational time.

Table 5.

Mean target registration error and optimization time and the corresponding standard deviation of the proposed methods for 3D pairwise registration

Data set	Method	TRE-3D (mm)	Time (s)
DIR-Lab	Before Reg.	10.43 ± 7.45	-
	GroupRegNet16	1.31 ± 1.29	277.1
	FR-Net-early-stop	1.31 ± 1.11	81.5
POPI	Before Reg.	8.12 ± 4.77	-
	GroupRegNet16	1.04 ± 1.10	394.4
	FR-Net-early-stop	1.11 ± 1.18	79.6

Qualitative results.

DIR, deformable image registration; FR-Net, few-shot registration network; TRE, target registration error.

Figure 4 illustrates the qualitative performance of the proposed few-shot groupwise registration on a sample of the DIR-Lab data set. The deformed image (Figure 4(d)) is generated by registering the maximum inhalation image (Figure 4(a)) to the maximum exhalation image (Figure 4(b)). Moreover, the corresponding deformation vector composited by the DVFs mapping from the two extreme phase images to the implicit reference image is shown in Figure 4(e). By comparing the difference between the original images (Figure 4(c)) with that between the deformed and exhalation images (Figure 4(f)), we can observe that most of the tissues are well aligned using the proposed FR-Net.

Figure 4.

Coronal cross-section images of a patient in the DIR-Lab data set. Images (a) and () and (b) represent the maximum inhalation and maximum exhalation images, respectively; () represent the maximum inhalation and maximum exhalation images, respectively; (d) is the deformed inhalation image; () is the deformed inhalation image; (e) is displacement vector field overlaid on image () is displacement vector field overlaid on image (a); (); (c) and () and (f) show the difference between the exhalation image and inhalation image and that between the exhalation image and deformed image, respectively.) show the difference between the exhalation image and inhalation image and that between the exhalation image and deformed image, respectively. DIR, deformable image registration.

Discussion

In this work, we develop a novel few-shot groupwise registration method to solve the challenging optimization problem of 4DCT registration by using deep neural networks. Comprehensive experiments on two thoracic 4DCT data sets demonstrate the promising registration accuracy of the proposed method over most state-of-the-art approaches, along with a significant reduction in the computational cost. In addition, we adopt an early stopping strategy in the optimization process to achieve more efficient 4D registration at the cost of a slightly increased registration error. The proposed method only requires a few unlabeled training data and accomplishes the registration task for one thoracic 4DCT in approximately 2 min. In conclusion, our DL-based FR-Net can realize efficient and effective image registration on 4D medical images for online medical applications such as motion tracking.

Although conventional iterative optimization-based methods have demonstrated considerable success in 3D pairwise registration, they are infeasible for online 4D registration because of the increased computational cost caused by registering each phase image individually. The average running time for one 4DCT image are usually more than 30 min and 10 min for conventional and DL-based one-shot 4D registration, respectively.^12,15,16,24 However, the proposed FR-Net exhibits considerable potential owing to its reduced computational cost in realizing rapid and accurate 4D registration, even when limited unlabeled data are available for training. When the number of training data is increased from 1 to 5, 53 and 49.3% less computational time is required to achieve comparable performance on the DIR-Lab and POPI data sets, respectively. It can be inferred that the proposed method can exhibit an enhanced computational efficiency if more 4D medical images are available for model training.

The proposed FR-Net exhibits a high registration accuracy while significantly decreasing the computational cost. The proposed groupwise registration can manage both large and small deformations with initial landmark distances ranging from 5.73 to 14.99 mm and ensure the alignment for the images between the consecutive phases. Different from the one-shot method proposed by Fechter et al,¹⁵ our method does not cause an increased registration error between the 4D groupwise registration and 3D pairwise registration. Experimental results show that the differences in the average TRE-3D are smaller than 0.05 mm on both data sets, thereby demonstrating the robustness of the proposed approach in realizing the registration of two or more medical images.

The outstanding enhancements in the registration accuracy and computational cost can be attributed to the initial estimation in the optimization problems and groupwise strategy for 4D registration. In general, the choice of a starting point determines how promptly an algorithm converges to a local optimal solution and the optimal value to which the algorithm converges. Instead of performing the initialization with random parameters, the FR-Net learns the registration task on finite training data and generalizes the optimization for the incoming test data, thereby accelerating the optimization process.

Geng et al²² conduct comprehensive experiments to compare the performances using images in the sequence data an implicit image as the reference image, which demonstrate that the implicit reference-based registration produces smaller registration errors than the reference-based methods. And they theoretically prove that the registration error using implicit reference is no larger than using a chosen reference image. Therefore, the proposed FR-Net simultaneously registers all the phase images to implicit reference images to mitigate the bias introduced by the selection of reference images. We calculate one large deformation between any two phases with the composition of their DVFs to the reference images, which reduces the accumulated error caused by combining multiple small deformations between consecutive phases. In addition, our DL-based model can extract the global representation to predict the DVFs considering both the spatial and temporal coherence for the patient-specific motion patterns, which helps enhance the registration performance.

Conclusion

We propose a rapid and accurate few-shot DIR algorithm for the registration of 4D images by combining the conventional groupwise registration method and DL pretraining strategy. The experimental results demonstrate the promising registration accuracy of the proposed method with significantly decreased computational costs, which can potentially be applied in online registration scenarios, such as tumor motion tracking.