Learning statistical correlation for fast prostate registration in image-guided radiotherapy

Abstract

Purpose: In adaptive radiation therapy of prostate cancer, fast and accurate registration between the planning image and treatment images of the patient is of essential importance. With the authors’ recently developed deformable surface model, prostate boundaries in each treatment image can be rapidly segmented and their correspondences (or relative deformations) to the prostate boundaries in the planning image are also established automatically. However, the dense correspondences on the nonboundary regions, which are important especially for transforming the treatment plan designed in the planning image space to each treatment image space, are remained unresolved. This paper presents a novel approach to learn the statistical correlation between deformations of prostate boundary and nonboundary regions, for rapidly estimating deformations of the nonboundary regions when given the deformations of the prostate boundary at a new treatment image.

Methods: The main contributions of the proposed method lie in the following aspects. First, the statistical deformation correlation will be learned from both current patient and other training patients, and further updated adaptively during the radiotherapy. Specifically, in the initial treatment stage when the number of treatment images collected from the current patient is small, the statistical deformation correlation is mainly learned from other training patients. As more treatment images are collected from the current patient, the patient-specific information will play a more important role in learning patient-specific statistical deformation correlation to effectively reflect prostate deformation of the current patient during the treatment. Eventually, only the patient-specific statistical deformation correlation is used to estimate dense correspondences when a sufficient number of treatment images have been acquired from the current patient. Second, the statistical deformation correlation will be learned by using a multiple linear regression (MLR) model, i.e., ridge regression (RR) model, which has the best prediction accuracy than other MLR models such as canonical correlation analysis (CCA) and principal component regression (PCR).

Results: To demonstrate the performance of the proposed method, we first evaluate its registration accuracy by comparing the deformation field predicted by our method with the deformation field estimated by the thin plate spline (TPS) based correspondence interpolation method on 306 serial prostate CT images of 24 patients. The average predictive error on the voxels around 5 mm of prostate boundary is 0.38 mm for our method of RR-based correlation model. Also, the corresponding maximum error is 2.89 mm. We then compare the speed for deformation interpolation by different methods. When considering the larger region of interest (ROI) with the size of 512 × 512 × 61, our method takes 24.41 seconds to interpolate the dense deformation field while TPS method needs 6.7 minutes; when considering a small ROI (surrounding prostate) with size of 112 × 110 × 93, our method takes 1.80 seconds, while TPS method needs 25 seconds.

Conclusions: Experimental results show that the proposed method can achieve much faster registration speed yet with comparable registration accuracy, compared to the TPS-based correspondence (or deformation) interpolation approach.

INTRODUCTION

Prostate cancer is the second-leading cause of cancer death for America men.¹ Intensity-modulated radiation therapy (IMRT), a state-of-the-art external beam radiation therapy, is the most common treatment paradigm for prostate cancer, by spreading the treatment over a weeks-long series of daily fractions.^{2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12} Since the prostate is surrounded by healthy tissues which can also be harmed by radiation, it is important to maximize the dose delivered to the prostate while minimize the dose delivered to the healthy tissues.⁸ In addition, for adaptive radiation therapy in IMRT, the treatment plan designed in the planning image needs to be adjusted according to the patient position and motion estimated from the treatment images of the patient during the treatment stage.^{9, 10} All these applications require prostate segmentation and also the registration between the planning image and each treatment image. In this paper, we focus on prostate registration.

The goal of prostate registration is to estimate deformable prostate motion during radiotherapy. By registering prostates between planning image and each treatment image of the patient, the treatment plan designed in the planning image space can be warped onto the treatment image space for adaptive radiation therapy. Also, the dose distribution map in the treatment image can be mapped back onto the planning image space, for measuring the total dose that has been actually delivered to the prostate (after a portion of the treatment). This is very important for adaptive dose compensation or modulation during the radiotherapy, in order to reduce side effects caused by excessive dose delivered to the healthy tissues while improve the chance of curing prostate cancer. Furthermore, adaptive dose compensation can be achieved by modifying the initial treatment plan to account for the actual dose already delivered.^{10, 11, 12, 13}

So far, many prostate registration algorithms have been developed in the literature, which can be broadly classified into two categories. The first category of methods is mainly based on correspondence detection or interpolation, i.e., first detecting correspondences through the segmentation of corresponding organs and then interpolating the dense correspondences for the rest regions of the image using thin plate splines (TPSs) based interpolation methods,^{14, 15, 16} finite element methods,^{17, 18, 19, 20} or other techniques.^{20, 21} For instance, Venugopal et al.¹⁵ used TPS to estimate the prostate motion given homologous landmark points in the two prostate images. Bharatha et al.¹⁹ used an elastic finite element model to align the preprocedural images with the intraprocedural images of the prostate and showed a significant increase in overlap between the registered preprocedural and intraprocedural prostate images, comparing to only using rigid transformation. Risholm et al.²⁰ described a probabilistic method for nonrigid registration of prostate images, which could quantify the most probable deformation as well as the uncertainty of the estimated deformation based on a biomechanical finite element model. Note that, although this category of prostate registration methods requires the detection of correspondences (which often involves manual selection or contouring), it has the advantage of ensuring the alignment of important features²² such as the regions around the prostate boundary.

The second category of prostate registration methods is mainly intensity-based, aiming to maximize a similarity function defined based on image intensities or gradients of the two prostate images.^{23, 24, 25, 26, 27, 28, 29} Mutual information is a popular similarity function and has been reported to be effective for prostate registration.²⁵ Besides, Oguro et al.²⁶ have also evaluated the registration performance by integrating the B-spline transformation model with the mutual information. The advantage of intensity-based registration methods is that they can be made fully automatic. However, their registration performance could be easily affected by the possible significant change of prostate image appearance due to the uncertain existence of bowel gas.^{27, 28, 29} Figure 1 shows a typical example from the serial prostate CT images of a patient, where the yellow rectangles in the top row indicate the inconsistent appearance of bowel gas, while the bottom row shows the corresponding zoomed area for better visualization.

Figure 1.

Illumination of inconsistent appearance of bowel gas across different prostate CT images of the same patient.

It is still challenging to use the existing registration methods (as mentioned above) for the CT image guided radiotherapy of prostate cancer. First, due to the low image contrast between prostate and its surrounding tissues in CT images, it is often difficult for the intensity-based registration algorithms to differentiate the corresponding structures in the planning and treatment images.^{23, 24, 25, 6}Second, it is also difficult to perform fast registration using the conventional intensity-based registration algorithms or even using the correspondence-guided registration methods (such as TPS-based interpolation algorithm) since it becomes slow when the number of correspondences in the prostate boundaries or the size of region of interest (ROI) becomes large. Although the TPS-based interpolation algorithm can be implemented block-by-block to improve its speed, the interpolation result could be affected if there is no single block covering the boundary points of the opposite sides of prostate.

In this paper, we propose a fast registration approach to align each treatment image with the planning image of the patient, based on the correspondences established between the prostate boundaries in the treatment and planning images by our recently developed deformable surface model.³⁰ Our main idea is to learn the statistical (deformation) correlation between the prostate boundary and the nonboundary regions (around the prostate) from both the images acquired from the current patient and the images acquired from other training patients. With the learned statistical deformation correlation, the deformations estimated on the prostate boundaries can be used to rapidly predict the deformations on the nonboundary regions, for registering the new treatment image of the current patient to its planning image. More importantly, an online-updated strategy is adopted in our proposed approach to progressively learn the patient-specific statistical deformation correlation. Concretely, in the initial treatment stage when the number of treatment images from the current patient is small, the statistical deformation correlation is mainly learned from the images of other training patients. As more treatment images are collected from the current patient, they play a more important role in learning patient-specific statistical deformation correlation to reflect prostate deformation of the current patient during the treatment. Eventually, only the patient-specific statistical deformation correlation is used to estimate the dense deformations, once a sufficient number of treatment images have been acquired from the current patient.

It is worth noting that the statistical correlation model has been widely used in the field of medical imaging.^{31, 32, 33, 34, 35, 36} For example, Davatzikos et al.³² presented a purely shape-based framework for capturing the principal modes of covariation between anatomy and deformation in principal component analysis (PCA) formulation, in order to statistically represent deformability induced by tumor growth. Thus, when a patient’s anatomy is considered, it can be used in conjunction with the statistical model to predict the way in which the anatomy will deform. Liu et al.³³ proposed a method for predictive modeling of anatomical structures using canonical correlation analysis (CCA).³⁴ With this technique, certain anatomical structures, such as tumor-distorted structures, can be estimated from others by their correlation learned from the training samples. Besides the applications in medical imaging, predictive modeling has also been used widely in the computer vision and chemometrics fields. For example, Reiter et al.³⁵ proposed to estimate face-depth maps from color face images based on CCA, which exploits the correlation between face color texture and surface depth. Castelan et al.³⁶ compared four different subspace multiple linear regression (MLR) methods for 3D face shape prediction from a single 2D intensity image, and then principal component regression (PCR), partial least squares (PLS), CCA, and ridge regression (RR) were used to estimate a regression operator while maximizing specific criteria between 2D and 3D face subspaces. In our study, three types of MLR methods, namely CCA, PCR and RR, are used and compared to learn the statistical deformation correlation between prostate boundary and nonboundary regions.

The rest of the paper is organized as follows. In 2, our fast registration method on learning statistical deformation correlation between prostate boundary and nonboundary regions is presented. Then, our proposed method is intensively evaluated in 3. The key factors which affect the performance of the proposed method are also discussed in 4. Section 5 concludes the whole paper.

METHODOLOGY

In this section, details of the proposed method are described, with its schematic summary provided in Fig. 2. As illustrated in Fig. 2, the first step of the proposed method is to perform statistical shape-based segmentation on the acquired prostate CT images. Our recently developed statistical shape-based segmentation algorithm³⁰ can not only segment the prostate from the planning and treatment images but also obtain correspondence between all segmented prostate boundaries.³⁰ In order to establish the dense correspondences (or deformations), also for the points belonging to the nonboundary regions around the prostate, a correspondence interpolation algorithm, e.g., TPS or others,^{14, 15, 16} is needed. However, these algorithms are generally slow (e.g., taking more than 6.7 minutes for dense correspondence interpolation of prostates in a large region of interest (ROI) with size of 512 × 512 × 61 (Ref. ³⁷)), which is not efficient for clinical application. Therefore, it is important to develop a fast algorithm for dense correspondence interpolation to satisfy the clinical applications.

Figure 2.

A schematic representation of the proposed method used for fast registration of prostate CT images.

As indicated in Fig. 2, the dense deformation (or correspondence) interpolation in the nonboundary regions can be estimated by using the statistical (deformation) correlation established between prostate boundary and nonboundary regions from a population of training patients, which will be further refined by the online-collected patient-specific treatment images during the radiotherapy of a patient. Specifically, the statistical deformation correlation learned from the population can be used to guide the estimation of dense deformations (or correspondences) for the initial treatment images of the current patient at the beginning of the treatment stage. With the acquisition and segmentation of more treatment images from the current patient, the patient-specific information can gradually play a more important role to achieve more accurate correspondence interpolation results, i.e., by increasing the value of coefficient ω_t in Fig. 2. Once a sufficient number of treatment images are acquired from the current patient, the patient-specific information can eventually replace the population information in correspondence interpolation, i.e., setting ω_t to be 1.0 in Fig. 2.

In the following Subsections2A, 2B, we will detail the proposed idea. In particular, Subsection 2A will first introduce how to learn the statistical deformation correlation from a population of training patients by using different multiple linear regression (MLR) methods, such as CCA-, PCR-, and RR-based correlation models. Then, Subsection 2B will introduce how to learn the patient-specific statistical deformation correlation as more treatment images are collected from the current patient, and also introduce how to adaptively combine the patient-specific information with the population-based information for more effective interpolation of dense correspondences in the nonboundary regions.

Learning statistical deformation correlation from a population of training patients

Training samples

Given a number of (manually or automatically) segmented serial prostate images of different training patients, the boundary correspondences across different prostate shapes can be established by using the deformable shape model proposed in Ref. 38. Then we can use the established correspondences to warp the prostate shapes of each training patient onto the planning image space of the current patient to obtain the aligned prostate shapes, with details provided below.

Assume that we have P training patients {Sⁱ|i=1,…,P}, and each patient has d_i segmented prostate shapes, denoted as $\{S_j^i|i=1,...,P;j=1,...,d_i\}$. Also, each shape $S_j^i$ is represented by a vector with M prostate boundary points, denoted as $S_j^i={(x_{j{,}_1}^i,y_{j,1}^i,z_{j,1}^i,\dots x_{j,M}^i,y_{j,M}^i,z_{j,M}^i)}^T$. We can linearly align each prostate shape $S_j^i$ of the i-th training patient onto the prostate shape $S_1^0$ of the planning image of the current patient S⁰, to obtain its aligned prostate shape $S_{j\to 1}^i$ in the planning image space of the current patient. Based on all aligned prostate shapes $S_{j\to 1}^i$, for each patient Sⁱ, we can calculate the mean shape as ${\overline{S}}^i={\sum }_{j=1}^{d_i}{S}_{j\to 1}^i∕d_i$, and then obtain the residual deformation $B_j^i=S_{j\to 1}^i-{\overline{S}}^i$ from each aligned prostate shape $S_{j\to 1}^i$ by subtracting the mean prostate ${\overline{S}}^i$ from $S_{j\to 1}^i$. Thus, based on the prostate shape $S_1^0$ in the planning image of the current patient S⁰ and the learned residuals from different training patients, we can obtain various deformed prostate shapes, $\{C_j^i=B_j^i+S_1^0|i=1,\dots ,P;j=1,\dots ,d_i\}$, to simulate possible prostate deformations for the current patient during the radiotherapy.

Then, by using a correspondence interpolation algorithm, i.e., TPS or other methods,^{14, 15, 16} we can interpolate N dense correspondences $R_j^i$ for the nonboundary regions of prostate, according to the correspondence on each deformed prostate shape $C_j^i$, denoted as $R_j^i={(x_{j,1}^i,y_{j,1}^i,z_{j,1}^i,\dots ,x_{j,N}^i,y_{j,N}^i,z_{j,N}^i)}^T$. Therefore, we can get n paired samples for the prostate shape (or boundary) correspondences and nonboundary regional correspondences, denoted as $\{(C_j^i,R_j^i)|i=1,\dots ,P;j=1,\dots ,d_i\}$, where $n={\sum }_{i=1}^P{d}_i,$$C_j^i$ is a 3M-dimensional vector, and $R_j^i$ is a 3N-dimensional vector. With these training pairs, the goal of our study is to build a statistical deformation correlation model to predict the N dense correspondences in the nonboundary regions based on the M correspondences established on the prostate boundaries using multiple linear regressions (MLR). Next, we will describe three different MLR methods (i.e., CCA, PCR, and RR) one-by-one to build the statistical deformation correlation model.

CCA-based Correlation Model

Learning the correlation model. CCA is used to model the canonical correlation between the prostate boundary correspondences and the nonboundary regional correspondences.^{31, 32, 33, 34, 35, 36} Before performing CCA, PCA should be first performed on boundary correspondences and regional correspondences in order to get a compact representation for $\{C_j^i\}$ and {$R_j^i$}, respectively, as described by Eqs. 1, 2 below.

$$C_j^i=\overline{C}+{\mathbf{\varphi }}_{\mathit{c}}{\alpha }_j^i$$ (1)

$$R_j^i=\overline{R}+{\mathbf{\varphi }}_{\mathit{R}}{\beta }_j^i$$ (2)

where $\overline{C}$ and $\overline{R}$ denote the respective means for $\{C_j^i\}$ and {$R_j^i$}, with _ϕc and ϕ_R as the respective eigenvector matrices (each with q eigenvectors). ${\alpha }_j^i$ and ${\beta }_j^i$ denote the respective coefficient vectors for $C_j^i$ and $R_j^i$, which can also be denoted as matrices $\mathit{A}={({\alpha }_1^1,\dots ,{\alpha }_{d_P}^P)}^T$ and $\mathit{B}={({\beta }_1^1,\dots ,{\beta }_{d_P}^P)}^T$, respectively. Then, CCA can be directly formulated as a linear least squares problem with respect to Eq. 3,

$$\varepsilon (w_{\alpha }^k,w_{\beta }^k)=E\left[{({(w_{\alpha }^k)}^T{\alpha }_j^i-{(w_{\beta }^k)}^T{\beta }_j^i)}^2\right],$$ (3)

by seeking a set of transformation vector pairs, $w_{\alpha }^k$ and $w_{\beta }^k$, which yields the canonical variates with maximum correlation. A number of at most g=min(n,q) leading empirical canonical factor pairs $w^k=\{w_{\alpha }^k,w_{\beta }^k\},k=1,\dots ,g,$ can be solved, as denoted as matrices ${\mathit{W}}_{\alpha }=(w_{\alpha }^1,\dots ,w_{\alpha }^g)$ and ${\mathit{W}}_{\mathbf{\beta }}=(w_{\beta }^1,\dots ,w_{\beta }^g)$, respectively. Finally, the regression parameter H can be computed as $\mathit{H}={({\mathit{W}}_{\alpha }^T\mathit{A})}^{†}{\mathit{B}}^T$. (Here, notation ‘†‘ denotes the pseudoinverse of matrix.)

Predicting the nonboundary regional correspondences. Given a new prostate shape segmented from a new treatment image of the current patient at time-point t, denoted as $S_t^0$, we first obtain its corresponding aligned shape $S_{t\to 1}^0$ by linearly aligning it onto the prostate shape in the planning image, $S_1^0$. And then we obtain $B_t^0=S_{t\to 1}^0-{\overline{S}}^0$ and $C_t^0=S_1^0+B_t^0$. Finally, we can predict the nonboundary regional correspondences by the following steps: first, obtain the PCA-based coefficient vector ${\alpha }_t^0$ according to Eq. 1; second, estimate the coefficient vector ${\beta }_t^0$ using equation ${\beta }_t^0={\mathit{H}}^T{\alpha }_t^0;$ and finally, estimate the regional correspondences $R_t^0$ for the new treatment image using the reversed PCA by Eq. 2.

PCR-based Correlation Model

Learning the correlation model. Another alternative to build the statistical deformation correlation model between prostate boundary correspondences and regional correspondences is the PCR method. PCR is used to overcome the possible colinearity of prostate boundary correspondences. Let X be the matrix constructed by the prostate boundary correspondences, i.e., $\mathit{X}={[C_1^1,\dots ,C_{d_P}^P]}^T$, and Y be the matrix constructed by the nonboundary regional correspondences, i.e., $\mathit{Y}={[R_1^1,\dots ,R_{d_P}^P]}^T$. And then, according to Eq. 1, the first q principal components of the covariance matrix X^TX of the prostate boundary correspondences can be calculated and stored in ϕ_c and the corresponding coefficient vectors are stored in the matrix $\mathit{A}={({\alpha }_1^1,\dots ,{\alpha }_{d_P}^P)}^T$. Then, these coefficient vectors are used as the new predictors for the regional correspondence vectors Y. Finally, the regression matrix H=(A^TA)^-1A^TY is calculated by minimizing the following energy function:

$$\varepsilon (\mathit{H})=E[{(\mathit{Y}-\mathit{AH})}^2]$$ (4)

Predicting the regional correspondences. Given a new prostate shape segmented from a new treatment image of the current patient at time-point t, denoted as $S_t^0$, similar to CCA, the corresponding $C_t^0$ can be obtained, and then the prostate regional correspondences can be predicted by first obtaining the PCA-based coefficient vector ${\alpha }_t^0$ according to Eq. 1, and then estimated by equation $R_t^0={\alpha }_t^0\mathit{H}.$

RR-based Correlation Model

Learning the correlation model. The last method we described in this paper is the RR-based correlation method. The main advantage of the RR-based method is its ability to avoid the over-fitting problem in high-dimensional correspondence prediction. In particular, RR-based correlation model introduces an additional term λ||H|| in the following regression function,

$$\varepsilon (\mathit{H})=E{[\parallel \mathit{Y}-\mathit{XH}\parallel }^2+\lambda {\parallel \mathit{H}\parallel }^2]$$ (5)

where X and Y are defined similarly as PCR’s, λ denotes the regularization parameter, and H denotes the regression matrix. By taking derivative ɛ(H) on H and letting the derivative be zero, the regression matrix is estimated as H=(X^TX+λI)⁻¹X^TY (where I is a unit matrix of 3M-by-3M).

Predicting the regional correspondences. Given a new prostate shape segmented from a new treatment image of the current patient at time-point t, denoted as $S_t^0$, after the similar operations as CCA’s, we obtain $C_t^0$ and finally the nonboundary regional correspondences of prostate by equation $R_t^0=C_t^0\mathit{H}$.

Refining interpolation of dense correspondences by using patient-specific information

For each new treatment image of the current patient, we can use the method proposed in Ref. 30 to segment the prostate shape, and further establish the correspondence on the prostate boundaries between the planning image and the treatment image. With the statistical correlations learned between the prostate boundary correspondences and regional correspondences, we can rapidly predict the dense deformations (or correspondences) for the nonboundary regions based on the prostate boundary correspondences of the new treatment image. Moreover, after each daily radiotherapy, we can also perform offline interpolation using TPS or other interpolation methods to obtain potentially more accurate correspondences for the nonboundary regions, based on the prostate boundary correspondences or even other correspondences provided by the urologist, calcified points detected in the prostate (i.e., bright points in CT), or calypso transponders implanted in the prostate. Note that, after radiotherapy, we are allowed to take more time to update the dense correspondences for the nonboundary regions in the treatment image.

As more treatment images are obtained and segmented from the current patient, we can start to collect new pairs of correspondences in the prostate boundary and the nonboundary regions, which can be used to train a patient-specific deformation correlation model. Intuitively, images obtained from the same patient can reflect the patient-specific anatomical characteristics more accurately; therefore, these images should be used to adaptively update the statistical deformation correlation model, in order to build a patient-specific statistical deformation correlation model with acquisition of more and more treatment images from the current patient. Details about how to control the parameter coefficient ω_t (in Fig. 2) to build or update patient-specific statistical deformation correlation model during the radiotherapy of a patient are given below.

First, we briefly summarize how to generate pairs of patient-specific training data. Assume that we have t-1 segmented prostate shapes $\left\{S_j^0\right|j=1,\dots ,t-1\}$ from the current patient S⁰. Then, we can obtain their linearly aligned prostate shapes $\left\{S_{j\to 1}^0\right|j=1,\dots ,t-1\}$ in the planning image space $S_1^0$. And the mean prostate shape can be calculated as ${\overline{S}}^0={\sum }_{j=1}^{t-1}{S}_{j\to 1}^0∕(t-1)$, and the residual deformation is calculated as $B_j^0=S_{j\to 1}^0-{\overline{S}}^0$. And, eventually, we can obtain all the possible patient-specific prostate deformations for the current patient as $\{C_j^0=B_j^0+S_1^0|j=1,\dots ,t-1\}$. Similarly, the respective dense deformations (or correspondences) in the nonboundary regions $\left\{R_j^0\right|j=1,\dots ,t-1\}$ can also be obtained with TPS interpolation method. In this way, we can obtain a set of patient-specific training samples $\{(C_j^0,R_j^0)|j=1,\dots ,t-1\}.$

Then, based on the available patient-specific training data and also the population-based training data, patient-specific statistical deformation correlation model can be built and progressively updated with more and more treatment images collected from the current patient, i.e., using a parameter ω_t to control the contribution of the current patient in construction of statistical deformation correlation model according to the Eq. 6,

$${\omega }_t=\{\begin{array}{l}\begin{array}{cc}0& t<N_s\\ (t-N_s)∕(N_b-N_s)& N_s\le t\le N_b\end{array}\\ \begin{array}{cc}1& N_b<t\end{array}\end{array}$$ (6)

Here N_s is the minimal number of treatment images required for the current patient to begin building the patient-specific statistical deformation correlation model. N_b is the minimal number of treatment images collected from the current patient required to build the patient-specific correlation model without using population training data. More specifically,

• if t<N_s, we fully rely on the population-based training data to build population-based correlation model for estimating the nonboundary regional correspondence R_t at the t-th time point treatment image for the current patient;

• if N_s≤t≤N_b, we start to build and update patient-specific statistical deformation correlation model with the collected pairs of training sample from the current patient and population training data balanced by parameter ω_t. According to Eq. 6, initially, population training data play a more important role in learning the patient-specific correlation model to predict nonboundary regional correspondences when there are few treatment images from the current patient available. Then with more treatment images collected from the current patient, the contribution from patient-specific information to update the patient-specific correlation model is gradually increased, and simultaneously the contribution from population training data is reduced;

• if t>N_b, we have more than N_b treatment images collected from the current patient. Then, we can stop using the population training data in learning the patient-specific correlation model, since more than N_b treatment images collected from the current patient is enough to build a robust patient-specific statistical deformation correlation model for accurately predicting the nonboundary regional correspondences.

In conclusion, our method for fast correspondence interpolation is summarized as follows (Refer to Fig. 2 for schematic explanations):

• Step (1)

  Use the deformable segmentation method such as proposed in Ref. 30 to segment the prostates from the planning image of the current patient, as well as all images in the available training patients.

• Step (2)

  Align all segmented prostate shapes of each training patient onto the prostate shape in the planning image of the current patient, and then learn its intrapatient motions (i.e., the residual of the aligned patient shapes compared to the mean shape) to simulate pairs of boundary and nonboundary correspondence samples, for constructing the population-based statistical deformation correlation model.

• Step (3)

  For each initial treatment image of the current patient, we first use a deformable segmentation method such as in Ref. 30 to segment the prostate and establish its boundary correspondence with that in the planning image. Then, with the learned population-based correlation model as well as the established correspondences on the prostate boundaries, we can estimate the nonboundary regional correspondences for the current treatment image.

• Step (4)

  As more treatment images are collected from the current patient, we can use them together with the population training data to jointly build a patient-specific statistical deformation correlation model. Once a sufficient number of treatment images are collected from the current patient, we can use only the patient-specific training data to build the patient-specific statistical deformation correlation model (without using the population training data). This built model will be used to guide the estimation of regional correspondences for the new treatment image by following the similar procedure as described in Step (3).

EXPERIMENTAL RESULTS

The performance of our proposed method is evaluated on serial prostate CT images of 24 patients. Most patients have up to 12 scanned images, with image size of 512×512×61 and voxel size of 1×1×3mm³. Prostates in all serial images of each patient have been manually segmented by medical experts, and their correspondences on the prostate boundaries have been established with a statistical shape-based method as proposed in Ref. 38. For comparison, TPS-based correspondence interpolation is used as the gold standard¹⁴ to estimate dense regional correspondences based on the correspondences obtained on the prostate boundaries. For convenience, we denote our methods of using different statistical correlation models as follows: Methods based on statistical correlation models of CCA, PCR, and RR if using only the population-based training data are called as Population CCA, Population PCR, and Population RR, respectively, while the respective methods using both the population-based and patient-specific training data are called as online CCA, online PCR, and online RR, respectively. The performance of different statistical correlation models are demonstrated by both qualitative and quantitative results.

Before providing the detailed experimental results, we first introduce the measures used to evaluate the registration performance and then summarize the parameters used in these experiments. In particular, the quantitative measure, called predictive error, is used to evaluate the registration performance of the proposed method. The predictive error is calculated as the magnitude of difference of displacement vector on each voxel obtained using the TPS-based interpolation method and our proposed method. Table Table I. gives the values of important parameters we used in the experiments. These parameters specify when to collect and use the patient-specific statistical correlation model, the important parameters of PCA, and the number of landmarks used in the whole experiments and so on. All parameters are optimized based on the predictive errors. For example, for the method of CCA or PCR, the predictive error is mainly a function of the number of latent (or canonical) variables used, while for the method of RR, the predictive error is mainly a function of regularization parameter λ. For our data, the number of latent variables for CCA- or PCR-based correlation model is determined by their representation power, while for RR-based model, λ=0.5 was found to have the optimal results.

Table 1.

Parameters used in the paper.

Symbol	Value and Description
Training data
P	(24) Number of patients
di	Number of serial CT images for each patient
n	Number of training samples in the population-based training data, i.e., $n\hspace{1em}=\hspace{1em}{\sum }_{i=1}^P{d}_i$
M	(816) Number of corresponding points on prostate shape (or boundary)
N	Number of points on nonboundary regions
Predictive model
q	Number of principal components used for prostate boundary correspondences, as well as for nonboundary regional correspondence; or the length of coefficient vector ${\alpha }_j^i$ and $B_j^i$
g	Number of empirical transformation factor pairs of CCA-based correlation model
λ	(0.5) Regularization parameter for RR-based correlation model
Algorithm
Ns	(3) Minimal number of treatment images from current patient required to build or update patient-specific statistical deformation correlation model
Nb	(10) Minimal number of treatment images of the current patient required to build a patient-specific statistical deformation correlation model without using population training data

Evaluating registration performance

In the following, we will evaluate the performance of our method on an individual patient and also on all patients, respectively.

Results on an individual patient

Figure 3 shows the predictive error of prostate deformations in the serial images of a patient estimated by our methods, compared to those interpolated by TPS. The top three rows show the spatial distribution of predictive errors at different treatment times using the population-based statistical correlation model by setting ω_t=0 in Eq. 6. And the bottom three rows show the spatial distribution of predictive errors using patient-specific statistical correlation models. In this figure, the middle contour represents the boundary of prostate, while the outer contour and the interior contour represent the iso-distance of 15 mm outward and inward from the prostate boundary contour, respectively. This figure shows that the performance of all online methods (bottom three rows) is better than that by the population methods (top three rows). As for the population methods, the performance of Population PCR and Population RR is better than that by Population CCA, especially in the right side and also the middle part of prostate. We have also evaluated the Online CCA, Online PCR and Online RR methods, indicating (from Fig. 3) that our online methods can achieve the predictive error of less than 1.0 mm when a sufficient number of treatment images are obtained from the current patient. By comparing these three online methods, we can find that the Online RR method can more accurately estimate deformations even when the number of treatment images from the current patient is small.

Figure 3.

The distribution of predictive errors at different treatment times by using the population-based (top three rows) or patient-specific statistical correlation model (bottom three rows).

These results can be further evaluated by comparing the relative deformed positions of four planning image points in different treatment images by TPS method and our population methods or online methods, respectively. Figure 4 shows the four points in the planning image of a patient, as indicated by four crosses. Figure 5 compares the corresponding points estimated, respectively, by TPS method (crosses), our population methods (plus signs in the left three columns), and our online methods (plus signs in the right three columns) at image spaces of time-point 5, 7, and 10. This figure shows that the performance of all online methods (right three columns) is better than that by the population methods (left three columns), as we can see that the four estimated points by the online methods are basically overlaid upon the corresponding points interpolated by TPS method. Notably, in Figs. 45, the middle contour represents the boundary of prostate, while the outer contour and the interior contour represent the iso-distance of 15 mm outward and inward from the prostate boundary contour, respectively. Since these four points are selected far from the prostate boundary, this experimental result shows that our online methods can more accurately estimate tissue deformations.

Figure 4.

Four points are selected in the planning image of the patient, to demonstrate the performance of correspondence interpolation by different methods as shown in Fig. 5.

Figure 5.

Comparison between the point interpolation results (plus signs) by the population methods (left three columns) and the online methods (right three columns). Here, the points (crosses) interpolated by TPS method are shown and used as ground-truth. It can be observed that the online methods can interpolate the points much closer to those interpolated by TPS method.

Figure 6 quantitatively illustrates the error distributions corresponding to those in Fig. 3. From this figure, we can observe that the error distribution in any treatment image is similar to each other for all three population-based methods when using only the population-based statistical correlation model. We can also observe that, when including the patient-specific information, the predictive errors are significantly reduced, compared with those by using only the population-based information. This figure also shows that the performance of PCR and RR is consistently better than that of CCA.

Figure 6.

Overall distributions of predictive errors at six different treatment times (Time 5–Time 10) of a patient, by using the population-based or patient-specific statistical correlation models.

Results on all 24 patients

Table Table II. shows the overall predictive errors for all images of the 24 patients by using only the population-based training data or using both the population-based and patient-specific training data. The predictive errors on the voxels around 5 mm of prostate boundary are considered and reported, since these voxels locate between the prostate and the other normal tissues where high registration accuracy is required for effective treatment to maximize the harm to the tumor while minimize the harm to healthy tissues. It can be observed that the average error is 0.47, 0.38, and 0.38 mm for the Online CCA, Online PCR, and Online RR, respectively, which is much better than 1.15, 0.87, and 0.87 mm achieved by the Population CCA, Population PCR, and Population RR which use the population-based statistical correlation model. Also, the maximum error can be reduced significantly by using patient-specific correlation models, compared with those using only the population-based correlation model. For example, the maximum errors for the Online CCA, Online PCR, and Online RR methods are 3.61, 3.02, and 2.89 mm, respectively, which are much smaller than 4.98, 4.05, and 3.76 mm for the Population CCA, Population PCR, and Population RR methods, respectively.

Table 2.

Comparison of predictive errors for all the images of 24 patients between CCA-, PCR-, and RR-based predictive correlation methods which use the population-based correlation model or patient-specific correlation models. (unit: mm).

Method		Mean ± Std	Minimum	Median	Maximum
CCA	Population	1.15 ± 0.78	0.0030	0.99	4.98
Online	0.47 ± 0.34	0.0003	0.39	3.61
PCR	Population	0.87 ± 0.58	0.0013	0.77	4.05
Online	0.38 ± 0.27	0.0002	0.32	3.02
RR	Population	0.87 ± 0.57	0.0030	0.77	3.76
	Online	0.38 ± 0.27	0.0002	0.32	2.89

Table Table III. shows that the effect on the predictive error with respect to the number of available treatment images of the current patient used for building the patient-specific correlation model. The error in this table is computed from voxels around 5 mm of the prostate shape. This table indicates that the predictive error is reduced as more treatment images are collected from the current patient for all the methods. From Table Table III., we can also observe that RR’s performance is better than PCR’s and CCA’s when the number of available treatment images of the current patient is small, and the performance of these three methods becomes similar when a sufficient number of treatment images are collected from the current patient to serve as the patient-specific training data. For example, the predictive errors obtained by the Online CCA, Online PCR, and Online RR methods are similar to each other when the number of the treatment images collected from the current patient is reaching ten.

Table 3.

The effect on the predictive error with respect to the number of patient’s treatment images used for estimation of deformation correlation. (unit: mm).

Number of patient’s treatment images used	Online CCA		Online PCR		Online RR
	Mean (Std)	Maximum	Mean (Std)	Maximum	Mean (Std)	Maximum
3	0.83 (0.56)	3.61	0.64 (0.43)	3.03	0.64 (0.42)	2.89
4	0.77 (0.52)	3.01	0.59 (0.38)	2.32	0.60 (0.39)	2.27
5	0.65 (0.43)	2.42	0.50 (0.32)	1.95	0.51 (0.32)	1.93
6	0.51 (0.35)	1.94	0.40 (0.27)	1.60	0.41 (0.28)	1.58
7	0.45 (0.33)	2.34	0.37 (0.27)	1.93	0.37 (0.27)	1.94
8	0.31 (0.23)	1.75	0.28 (0.22)	1.82	0.28 (0.21)	1.82
9	0.13 (0.22)	2.54	0.13 (0.23)	2.54	0.13 (0.23)	2.54
10	0.09 (0.07)	1.03	0.09 (0.07)	1.04	0.09 (0.06)	0.96

Evaluating registration speed

The above results (in Figs. 3456 and Tables 2, Table III.) demonstrate that our online-learning method has comparable performance as TPS. More importantly, our method is much faster than TPS when processing the same data in the same machine (OS: 32 bit of Windows 7; CPU: Intel Core 2 Quad Q9400 2.66 Hz; Memory: 4 GB). As shown in Table Table IV., when considering larger ROI with the size of 512 × 512 × 61, our methods, such as CCA, PCR, and RR, are able to predict the dense deformation field within 24.50, 24.45, and 24.41 seconds, respectively, while TPS needs 6.7 minutes When considering a small ROI (surrounding prostate) with the size of 112 × 110 × 93, it takes 1.85, 1.83, and 1.80 seconds for our CCA, PCR, and RR methods to estimate the deformation field, while TPS needs 25 seconds. This result indicates that our method can estimate the dense deformation field in fast time, which shows potential in clinical application.

Table 4.

Comparison of the running time for our methods and TPS-based method.

Prostate data (Number of Landmarks: 816)	TPS	CCA (s)	PCR (s)	RR (s)
Large ROI (Size: 512 × 512 × 61)	6.70 min	24.50	24.45	24.41
Small ROI (Size: 112 × 110 × 93)	25.00 s	1.85	1.83	1.80

DISCUSSION

In the following, we detail the discussions on the choice of multiple linear regressions, the factors affecting the registration speed, and the standard for evaluating the performance.

Choice of multiple linear regressions

In this paper, multiple linear regressions are used to estimate the dense deformation field in the nonboundary regions based on the correspondence established on the prostate boundaries. There exist other regression methods such as principal component regression (PCR) and partial least squares (PLSs). However, the drawback of PCR method is that it can lead to suboptimal solutions because it creates only the principal components to explain the observed variability of prostate boundary vectors, without considering the nonboundary regional correspondence vectors.³⁹ On the other hand, PLS do seek to maximize covariance between prostate boundary correspondences and regional correspondences jointly, which has been proven as the most useful for the low-observation-to-variable ratio problems, i.e., our problem.³¹ However, the longer computation time of PLS leads to its unsuitability in our application, which requires fast registration speed. Furthermore, the experimental results also confirm that RR is more accurate than PCR, and can offer a simple and efficient way to approximate the dense correspondences in the nonboundary regions without requiring the construction of the latent (or canonical) variables between prostate boundaries and nonboundary regions.

Choice of TPS model, and factors affecting the registration speed

When using the correspondence-guided registration methods to estimate dense deformation field during radiotherapy of the prostate cancer, TPS or other interpolation methods^{14, 15, 16} are often used to estimate the deformation within or around the prostate. This is because TPS approach can yield minimal bending energy to produce the smooth deformation field^{14, 15, 16, 40} for modeling the elastic deformation of object, which is exactly assumed for the prostate deformation as investigated in the literature.⁴¹ Also, TPS approach can further improve its performance in modeling prostate deformation by using the extra correspondences established in urethra and calcified points, besides using only the boundary correspondences as described above. On the other hand, this is also the main reason why multiple linear regression techniques such as CCA, PCR and RR are used in this paper to interpolate the dense deformation. If the deformation cannot be formulated by the linear elastic model, other methods such as Gaussian radial basis interpolation^{42, 43} may be needed for deformation interpolation.

The key factors affecting the deformation interpolation speed are mainly the number of corresponding points and the size of region of interest (ROI). Accordingly, two respective ways can be used to improve the deformation interpolation speed. First, interpolation can be confined to a small portion of the image around prostate since the prostate does not occupy the entire image. Second, a limited number of points can be used to interpolate the dense deformation field since the deformations between serial images are assumed smooth.

Moreover, when using conventional intensity-based registration methods to align the planning image to a treatment image of a patient, the running time can be significantly reduced by implementing the algorithm with fast hardware such as graphic processing unit (GPU), or using a deliberately designed parallel implementation. However, a deflation algorithm must be first performed in order to eliminate the possible appearance of bowel gas in one image but not in another image (Refer to Fig. 1),^{27, 28, 29} which is not easy to perform in an automated fashion.

Standard for evaluating the performance

Comparing with the ground-truth landmarks of biomechanical model, the TPS interpolation is not physically based. Although the predicted deformations using the proposed method agree well with those by the direct TPS-based interpolation, neither of these deformation fields might be particularly accurate when compensating for the physical deformation. In particular, it is very important to find the significant registration error for the emerging targeted and dose escalated radiotherapy schemes where the tissue may exhibit complex behavior in terms of deformation. Therefore, in the future, more other ground-truth landmarks provided by the urologist, calcified points (with bright intensity) detected in the prostates, and calypso transponders implanted in the prostates will be considered to obtain better registration accuracy.

CONCLUSION

This paper presented a new algorithm for fast registration between prostates in the planning image and each treatment image of a patient during radiotherapy, by using both the population-based and patient-specific statistical correlation model established between correspondences of prostate boundaries and non-boundary regions. The statistical correlation model is learned by using multiple linear regression (MLR) techniques, including canonical correlation analysis (CCA), principal component regression (PCR), and ridge regression (RR). First, the patient-specific statistical correlation model is progressively learned from the previously collected treatment images of the current patient for guiding the estimation of regional correspondences for the new treatment image. In particular, at the beginning of the treatment stage, the population-based statistical correlation model plays a major role to statistically predict the dense regional correspondences. As more treatment images are collected from the current patient, the patient-specific statistical correlation model gradually plays a more important role than the population-based statistical correlation model. Second, it is found that the RR-based statistical correlation model can achieve more accurate prediction results in estimating the dense regional correspondences, compared with PCR- and CCA-based statistical correlation models. Experimental results also illustrate that our method has comparable registration accuracy with TPS-based interpolation technique, but achieves much faster speed than TPS, which is critical for clinical application.

In the future, our proposed method will be incorporated into the whole image process pipeline for prostate segmentation and registration. With accurate segmentation and registration of the prostate, the day-to-day motion (or deformation) of the prostate during external beam radiation therapy can be better determined, which can be used to adjust the treatment plan, as popularly done in adaptive radiation therapy. Thus, the best treatment for the disease could be achieved by maximizing dose to tumor and minimizing dose to healthy tissue. The performance of the whole segmentation and registration pipeline will be tested in our future work.