A Unified Framework for Joint Segmentation, Nonrigid Registration and Tumor Detection: Application to MR-Guided Radiotherapy

Abstract

Image guided external beam radiation therapy (EBRT) for the treatment of cancer enables accurate placement of radiation dose to the cancerous region. However, the deformation of soft tissue during the course of treatment, such as in cervical cancer, presents significant challenges. Furthermore, the presence of pathologies such as tumors may violate registration constraints and cause registration errors. In this paper, we present a unified MAP framework that performs automatic segmentation, nonrigid registration and tumor detection simultaneously. It can generate a tumor probability map while progressively identifing the boundary of an organ of interest based on the achieved transformation. We demonstrate the approach on a set of 30 T2-weighted MR images, and the results show that the approach performs better than similar methods which separate the registration and segmentation problems. In addition, the detection result generated by the proposed method has a high agreement with the manual delineation by a qualified clinician.

1 Introduction

External beam radiation therapy(EBRT) is the primary modality of treatment for cervical cancer[12]. Imaging systems capable of visualizing patient soft-tissue structure in the treatment position have become the dominant method of positioning patients for both conformal and stereotactic radiation therapy. Traditional CT images suffer from low resolution and low soft tissue contrast, while Magnetic Resonance (MR) imaging can characterize deformable structure with superior visualization and differentiation of normal soft tissue and tumor-infiltrated soft tissue. In addition, advanced MR imaging with modalities such as diffusion, perfusion and spectroscopic imaging has the potential to better localize and understand the disease and its response to treatment. Therefore, magnetic resonance guided radiation therapy (MRgRT) systems with integrated MR imaging in the treatment room are now being developed as an advanced system in radiation therapy[7].

The goal in radiotherapy cancer treatment is to deliver as much dose as possible to the clinical target volume (CTV), while trying to deliver as little dose as possible to surrounding organs at risk[4,10]. When higher doses are to be delivered, precise and accurate targeting is essential because of unpredictable inter- and intra-fractional organ motions over the process of the weekly treatments, which require accurate nonrigid mapping between treatment and planning day. However, the presence of pathologies such as tumors may violate registration constraints and cause registration errors, because the abnormalities often invalidate gray-level dependency assumptions that are usually made in the intensity-based registration. In addition, the registration problem is challenging due to the missing correspondences caused by tumor regression. Incorporating some knowledge about abnormalities can improve the registration. Meanwhile, segmentation of the soft tissue structure over the course of EBRT is challenging due to structure adjacency and deformation over treatment.

Some initial work has been performed in simultaneously registration and segmentation or incorporating abnormality detection. Chelikani et.al. integrated rigid 2D portal to 3D CT registration and pixel classification in an entropy-based formulation [3]. Yezzi et.al. integrated segmentation using deformable model with rigid and affine registration [17]. Pohl et.al. performed voxel-wise classification and registration which could align an atlas to MRI[14]. The automatic mass detection problem mainly focuses on the application on mammography[13]. Hachama et.al. proposed a registration technique which neglects the deformation generated by abnormalities[5], which is not always the case, and raises questions regarding the applicability of the method.

In this paper, we present a probability based technique as an extension to the work in [11]. Our model is based on a MAP framework which can achieve segmentation and nonrigid registration while simultaneously estimating a probability tumor map, which estimates each voxel’s probability of belonging to a tumor. In this manner, we can interleave these processes and take proper advantage of the dependence among them. Using the proposed approach, it is easy to calculate the location changes of the lesion for diagnosis and assessment, thus we can precisely guide the interventional devices toward the tumor during radiation therapy. Escalated dosages can then be administered while maintaining or lowering normal tissue irradiation. There are a number of clinical treatment sites that will benefit from our MR-guided radiotherapy technology, especially when tumor regression and its effect on surrounding tissue can be significant. Here we focus on the treatment of cervical cancer as the key example application.

2 Description of the Model

2.1 A Unified Framework: Bayesian Formulation

Let I₀ and I_d be the planning day (day 0) and treatment day (day d) 3D MR images respectively, and S₀ be the set of segmented planning day organs. A unified framework is developed using Bayesian analysis to calculate the most likely segmentation in treatment day fractions S_d, the deformation field between the planning day and treatment day data T_0d, and the tumor map M at treatment day which associates with each pixel of the image its probability of belonging to a tumor.

$$\widehat{S_d},\widehat{T_{0d}},\widehat{M}=\text{arg}\underset{S_d,T_{0d},M}{\text{max}}[p(S_d,T_{0d},M|I_0,I_d,S_0)]$$ (1)

This multiparameter MAP estimation problem in general is difficult to solve, however, we reformulate this problem such that it can be solved in two basic iterative computational stages using an iterative conditional mode (ICM) strategy[1,11]. With k indexing each iterative step, we have:

$$\widehat{S_d^k}=\text{arg}\underset{S_d^k}{\text{max}}[p(S_d^k|T_{0d}^k(S_0),I_d,M^k)]$$ (2)

$$\widehat{T_{0d}^{k+1}},\widehat{M^{k+1}}=\text{arg}\underset{T_{0d}^{k+1},M}{\text{max}}[p(T_{0d}^{k+1},M^{k+1}|S_d^k,S_0,I_d,I_0)]$$ (3)

These two equations represent the key problems we are addressing. Equation (2) estimates the segmentation of the important day d structure $(S_d^k)$. Equation (3) estimates the next iterative step of the mapping $T_{0d}^{k+1}$ between the day 0 and day d spaces, as well as the probability tumor detection map M^k+1.

2.2 Segmentation

In this paper, we use Eq. (2) to segment the normal or non-cancerous organs, hence we assume the segmentations $S_d^k$ here are independent of the tumor detection map M. Bayes rule is applied to Eq. (2):

$$\widehat{S_d^k}=\text{arg}\underset{S_d^k}{\text{max}}[p(S_d^k|T_{0d}^k(S_0),I_d,M^k)]=\text{arg}\underset{S_d^k}{\text{max}}[p(S_d^k|T_{0d}^k(S_0),I_d)]=\text{arg}\underset{S_d^k}{\text{max}}[p(I_d|S_d^k)·p(T_{0d}^k(S_0)|S_d^k)·p(S_d)]$$ (4)

Shape Prior Information

The segmentation module here is similar to the previous work in [11]. Here we assume that the priors are stationary over the iterations, so we can drop the k index for that term only, i.e. $p(S_d^k)=p(S_d)$. Instead of using a point model to represent the object, we choose level set as the representation of the object to build a model for the shape prior[9], and then define the joint probability density function in Eq.(4).

Consider a training set of n rigidly aligned images, with M objects or structures in each image. The training data were generated from manual segmentation by a qualified clinician. Each object in the training set is embedded as the zero level set of a higher dimensional level set Ψ with negative distances inside and positive distances outside the object. Using the technique developed by Leventon et al. in [9], the mean (Ψ̄) and variance of the boundary of each object can be computed using principal component analysis (PCA). An estimate of the object shape Ψ_i can be represented by k principal components and a k-dimensional vector of coefficients (where k < n) α_i

$${\tilde{\Psi }}_i=U_k{\alpha }_i+\overline{\Psi }$$ (5)

where U_k is a matrix generated by PCA. Under the assumption of a Gaussian distribution of object represented by α_i, we can compute the probability of a certain shape in Eq.(6), where Σ is the diagonal matrix of corresponding singular values.

$$p(S_d)=p({\alpha }_i)=\frac{1}{\sqrt{{(2\pi )}^k|{\Sigma }_k|}}\mathit{\text{exp}}[-\frac{1}{2}{\alpha }_i^T{\Sigma }_k^{-1}{\alpha }_i]$$ (6)

Segmentation Likelihoods

Then we imposed a key assumption: the likelihood term is separable into two independent data-related likelihoods, requiring that the estimate of the structure at day d be close to: i.) the same structure segmented at day 0, but mapped to a new estimated position at day d by the current iterative mapping estimate $T_{od}^k$ and ii.) the intensity-based feature information derived from the day d image [11].

In the Equation (4), $p(T_{0d}^k(S_0)|S_d)$ constrains the organ segmentation in day d to be adherent to the transformed day 0 organs by current mapping estimate $T_{od}^k$. Thus, we made the assumption about the probability density of day 0 segmentation likelihood term:

$$p(T_{0d}^k(S_0)|S_d^k)=\frac{1}{Z}\prod _{(x,y,z)}\mathit{\text{exp}}[-{({\Psi }_{T_{0d}^k(S_0)}-{\Psi }_{S_d^k})}^2]$$ (7)

In equation (4), $p(I_d|S_d^k)$ indicates the probability of producing an image I_d given $S_d^k$. In three-dimensions, assuming gray level homogeneity within each object, we use the imaging model defined by Chan and Vese [2] in Eq.(8), where c₁ and σ₁ are the average and standard deviation of I_d inside $S_d^k$, c₂ and σ₂ are the average and standard deviation of I_d outside $S_d^k$.

$$p(I_d|S_d^k)=\prod _{\mathit{\text{inside}}(S_d^k)}\mathit{\text{exp}}[-{(I_d(\mathbf{x})-c_1)}^2/2{\sigma }_1^2]·\prod _{\mathit{\text{outside}}}\mathit{\text{exp}}[-{(I_d(\mathbf{x})-c_2)}^2/2{\sigma }_2^2]$$ (8)

Energy Function

Combining(4),(6),(7),(8), we introduce the segmentation energy function E_seg defined by:

$$E_{\mathit{\text{seg}}}=-\text{ln}p(S_d^k|T_{0d}^k(S_0),I_d)=-\text{ln}p(I_d|S_d^k)·p(T_{0d}^k(S_0)|S_d^k)·p(S_d^k)\propto {\lambda }_1{\int }_{\mathit{\text{inside}}(S_d^k)}|I_d(\mathbf{x})-c_1{|}^{2}d\mathbf{x}+{\lambda }_2{\int }_{\mathit{\text{outside}}(S_d^k)}|I_d(\mathbf{x})-c_2{|}^{2}d\mathbf{x}+\gamma {\int }_{\mathbf{x}}|{\Psi }_{T_{0d}^k(S_0)}-{\Psi }_{S_d^k}{|}^{2}d\mathbf{x}+{\omega }_i{\alpha }_i^T{\Sigma }_k^{-1}{\alpha }_i$$ (9)

Notice that the MAP estimation of the objects in (4), $\widehat{S_d^k}$, is also the minimizer of the above energy functional E_seg. This minimization problem can be formulated and solved using the level set surface evolving method.

2.3 Registration and Tumor Detection

The second stage of the proposed strategy described above in Eq. (3) can be further developed using Bayes rule as follows in Eq.(10). As indicated in Section 2.2, the segmentations $S_d^k$ are independent of the tumor detection map M, and the priors are stationary over the iterations.

$$\widehat{T_{0d}^{k+1}},\widehat{M^{k+1}}=\text{arg}\underset{T_{0d}^{k+1},M^{k+1}}{\text{max}}[p(T_{0d}^{k+1},M^{k+1}|S_d^k,S_0,I_d,I_0)]=\text{arg}\underset{T_{0d}^{k+1},M^{k+1}}{\text{max}}[\text{ln}p(S_d^k,S_0|T_{0d}^{k+1},M^{k+1})+\text{ln}p(I_d,I_0|T_{0d}^{k+1},M^{k+1})+\text{ln}p(T_{0d},M)]=\text{arg}\underset{T_{0d}^{k+1},M^{k+1}}{\text{max}}[\text{ln}p(S_d^k,S_0|T_{0d}^{k+1})+\text{ln}p(I_d,I_0|T_{0d}^{k+1},M^{k+1})+\text{ln}p(T_{0d}|M)+\text{ln}p(M)]$$ (10)

The first term on the right hand side represents conditional likelihood related to mapping the segmented soft tissue structures at days 0 and d, and the second term registers the intensities of the images while simultaneously estimating the probability tumor map. The third and fourth terms represent prior assumptions and constraints on the overall nonrigid mapping and tumor map.

Segmented Organ Matching

As discussed in the segmentation section, each object is represented by the zero level set of a higher dimensional level set Ψ. Assuming the objects vary during the treatment process according to a Gaussian distribution, and given that the different organs can be mapped respectively, we further simplifies the organ matching term as:

$$\text{ln}p(S_d^k,S_0|T_{0d}^{k+1})=\sum _{obj=1}^N\text{ln}p(S_{d-obj}^k,S_{0-obj}|T_{0d}^{k+1})=\sum _{obj=1}^N{\int }_{\mathbf{x}}\text{ln}\frac{1}{\sqrt{2\pi }{\sigma }_{obj}}\text{exp}\left[\frac{-{({\Psi }_{T_{0d}^{k+1}(S_{0-obj})}-{\Psi }_{S_{d-obj}^k})}^2}{2{\sigma }_{obj}^2}\right]d\mathbf{x}=\sum _{obj=1}^N-{\omega }_{obj}{\int }_{\mathbf{x}}{[{\Psi }_{T_{0d}^{k+1}(S_{0-obj})}-{\Psi }_{S_{d-obj}^k}]}^{2}d\mathbf{x}$$ (11)

with N represent the number of objects in each image, and ω_obj are used to weight different organs. When minimized w.r.t. $T_{0d}^{k+1}$, the organ matching term ensures the transformed day 0 organs and the segmented day d organs align over the regions [11].

Intensity Matching and Tumor Probability Map

In order to define the likelihood $p(I_d,I_0|T_{0d}^{k+1},M^{k+1})$, we assume conditional independence over the voxel locations x, as discussed in [18].

$$p(I_d,I_0|T_{0d}^{k+1},M^{k+1})=\prod _{\mathbf{x}}p(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x}))|M^{k+1}(\mathbf{x}))$$ (12)

Different from the work in [11] which only uses a single similarity metric, here we model the probability of the pair $p(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x}))|M^{k-1}(\mathbf{x}))$ to be dependent on the class of the pixel x. Each class is characterized by a probability distribution, denoted by p_N for the normal tissue and p_T for the tumor. Let M^k+1(x) be the tumor map which associates to x its probability to belong to a tumor. Thus, the probability distribution $p(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x}))|M^{k-1}(\mathbf{x}))$ can be defined as a mixture of the two class distribution:

$$p(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x}))|M^{k-1}(\mathbf{x}))=(1-M^{k+1}(\mathbf{x}))p_N(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x})))+M^{k+1}(\mathbf{x})p_T(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x})))$$ (13)

Normal Tissue Class

Across the treatment procedure, the tumor experiences a regression process. When the tumor shrinks, some part of the tumor returns to the intensity level around the normal tissue. Thereafter, the normal tissue in treatment day MR has two origins. One is the normal tissue in planning day MR, and the other comes from the tumor in planning day and returns to normal due to the tumor regression. We choose two different probability for these two types. The histograms of normal cervical tissue and tumor are shown in Fig.1. From clinical research[6] and Fig.1, we can see that the intensities of tumor are generally much higher than those of normal cervical tissue.

Fig. 1.

The histograms of normal cervix and tumor

Therefore, for the sake of simplicity, we characterize the normal tissue from the second type (used to be tumor but returned to normal later on) as areas with much lower intensity in treatment day MR[6]. We assume a voxel labeled as this part in day d MR can match any voxel in day 0 tumor with equal probability and use a uniform distribution. The remaining voxels are labeled as type 1 normal tissue (always normal since the planning day), which are modeled assuming a discrete Gaussian distribution across the corresponding voxel locations.

$$p_N(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x})))=\{\begin{array}{cc}1/c,& T_{0d}^{k+1}(I_0(\mathbf{x}))-I_d(\mathbf{x})>\Delta \\ \frac{1}{\sqrt{2\pi \sigma }}\mathit{\text{exp}}(-\frac{{|T_{0d}^{k+1}(I_0(\mathbf{x}))-I_d(\mathbf{x})|}^2}{2{\sigma }^2}),& \mathit{\text{otherwise}}\end{array}$$ (14)

where c is the number of voxels in the day 0 tumor, and Δ is the predetermined threshold used to differentiate the intensity of normal and tumor tissue.

Tumor Class

The definition of the tumor distribution is a difficult task. Similar to normal tissue, the tumor tissue in treatment day MR also has two origins. One is the tumor tissue in planning day MR, which represents the remaining tumor after the radiotherapy. We assume the voxel in the residual tumor can match any voxel in the initial tumor (day 0) with equal probability and use a uniform distribution. The other origin of tumor class in day d is from the normal tissue in planning MR, but the tissue grows into malignant tumor later on. This type is characterized with much higher intensity in day d image[6]. We assume that each normal voxel in the day 0 MR can turn into a tumor voxel with equal probability. Thus this type is also modeled using another uniform distribution, but with less probability, since the chance of this deterioration process is relatively small across the radiotherapy treatment.

$$p_T(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x})))=\{\begin{array}{cc}1/(V-c),& T_{0d}^{k+1}(I_0(\mathbf{x}))-I_d(\mathbf{x})<-\Delta \\ 1/c,& \mathit{\text{otherwise}}\end{array}$$ (15)

where V is the total number of voxels in MR image, c is the number of voxels in the day 0 tumor, and it’s the same c as in Eq.(14).

Non-negative Jacobian

We want the transformation to be non-singular hence we would like to have a large penalty on negative Jacobian for normal tissues[8]. Meanwhile, we simulate the tumor regression process by constraining the determinant of the transformation’s Jacobian at M(x) between 0 and 1.

$$p(T_{0d}(\mathbf{x})|M(\mathbf{x}))=(1-M(\mathbf{x}))p_N(T_{0d}(\mathbf{x}))+M(\mathbf{x})p_T(T_{0d}(\mathbf{x}))$$ (16)

where p_N and p_T are calculated using continuous logistic functions: (ε = 0.1)

$$p_N(T_{0d}(\mathbf{x}))=\frac{1}{V-c}{\left(\frac{1}{1+\mathit{\text{exp}}(-|J(T_{0d}(\mathbf{x}))|/\epsilon )}\right)}^2$$ (17)

$$p_T(T_{0d}(\mathbf{x}))=\frac{1}{c}[{\left(\frac{1}{1+\mathit{\text{exp}}(\frac{-|J(T_{0d}(\mathbf{x}))|}{\epsilon })}\right)}^2-{\left(\frac{1}{1+\mathit{\text{exp}}(\frac{1-|J(T_{0d}(\mathbf{x}))|}{\epsilon })}\right)}^2]$$ (18)

V and c are defined the same as above, and they are used to normalize the probability density functions. These constraints (p_N and p_T ) penalize negative Jacobians and p_T simulates the tumor regression process, thus reduce the probability of folding in the registration maps.

Tumor Map Prior

We assume that the tumor map arises from a Gibbs distribution. Many specific terms can be defined to describe the spatial configurations of different type of lesion. The previous detection approaches have a tendency to over-detect (to find more regions than the real ones)[13], hence in this paper, we use an energy restricting the total amount of abnormal pixels in the image.

$$p(M)=\frac{1}{Z}{e}^{-U(M)}=\frac{1}{Z}{e}^{-{\mathrm{\Sigma }}_{\mathbf{x}}M(\mathbf{x})}$$ (19)

Energy Function

Combine the above equations, we introduce:

$$E_{\mathit{\text{reg}}-\mathit{\text{det}}}(T,M)=-\text{ln}p(T_{0d}^{(k+1)},M^{(k+1)}|S_d^k,S_0,I_d,I_0)=\sum _{obj=1}^N{\omega }_{obj}{\int }_{\mathbf{x}}{[{\Psi }_{T_{0d}^{k+1}(S_{0-obj})}-{\Psi }_{S_{d-obj}^k}]}^{2}d\mathbf{x}-{\int }_{\mathbf{x}}\text{ln}p(I_d(\mathbf{x}),T_{0d}^{k+1}(I_0(\mathbf{x}))|M^{k+1}(\mathbf{x}))d\mathbf{x}-{\int }_{\mathbf{x}}\text{ln}p(T_{0d}(\mathbf{x})|M(\mathbf{x}))d\mathbf{x}+\sum _{\mathbf{x}}M(\mathbf{x})$$ (20)

$T_{0d}^{k+1}$ and M^k+1 are estimated using a conjugate gradient method to minimize Eq.(20). Eqs (9) and (20) run alternatively until convergence, then the soft tissue segmentation as well as the nonrigid registration and tumor detection in day d benefit from each other and can be estimated simultaneously.

3 Results

We tested our proposed method on 30 sets of MR data acquired from five different patients undergoing EBRT for cervical cancer at Princess Margret Hospital. Each of the patient had six weekly 3D MR images. A GE EXCITE 1.5-T magnet with a torso coil was used in all cases. T2-weighted, fast spin echo images (voxel size 0.36mm × 0.36mm × 5mm, and image dimension 512 × 512 × 38) were acquired. The clinician performed the bias field correction so that the intensities could be directly compared, and the Δ in Eq.(14) and (15) was chosen to be 150. The MR images were resliced to be isotropic. We adopted a ”leave-one-out” test so that all the tested images were not in their training sets.

Segmentation Results

Fig.2(a) shows an example of MR data set with extracted 3D bladder (green) and uterus (purple) surfaces using the proposed method. Fig.2(b)(c) show the segmentation of bladder and uterus from axial view respectively, compared with manual segmentation.

Fig. 2.

Segmentation Results. (a) An example of MR data set with automatically extracted bladder (green) and uterus (purple) surfaces. (b) Comparing bladder segmentations using the proposed algorithm (green) and manual segmentation (red). (c) Uterus results using the proposed algorithm (green) and manual segmentation (red).

We compare the experimental results from our algorithm with those obtained from the segmentation using traditional level set active shape models (without priors, and with shape priors only). To quantify the accuracy of our approach, we use mean absolute distance (MAD) and Hausdorff distance (HD) to evaluate the segmentation performance. While MAD is a global measure of the match of two surfaces, HD reflects their local similarities. Table 1 and 2 quantitatively analyze the segmentation results on bladder and uterus surfaces respectively. We observe that both MAD and HD decrease with the proposed method, which implies that our method has a consistent agreement with the manual segmentation.

Table 1.

Evaluation of Segmentation of Bladder

	MAD(mm)	HD(mm)
Without prior	7.61 ± 2.69	8.63 ± 3.18
With shape prior	3.28 ± 1.35	4.10 ± 1.41
Proposed method	0.82 ± 0.10	1.20 ± 0.15

Table 2.

Evaluation of Segmentation of Uterus

	MAD(mm)	HD(mm)
Without prior	10.57 ± 6.19	12.13 ± 5.88
With shape prior	6.11 ± 1.34	7.28 ± 1.21
Proposed method	1.26 ± 0.38	1.92 ± 0.75

Registration Results

The mapping performance of the proposed algorithm was also evaluated. For comparison, a conventional intensity-based FFD non rigid registration (NRR) [16] and rigid registration (RR) were performed on the same sets of real patient data. The corresponding difference images of the deformed bladder are shown in Fig.3.

Fig. 3.

Example of different transformations on the registration for one patient. (a) Difference image of bladder after rigid registration. (b) After nonrigid registration. (c) After the proposed method.

Organ overlaps between the ground truth in day d and the transformed organs from day 0 were used as metrics to assess the quality of the registration (Table 3). We also tracked the registration error percentage between the ground truth in day d and the transformed organs from day 0 for bladder and uterus, as shown in Table 4. The registration error is represented as percentage of false positives (PFP), which calculates the percentage of a non-match being declared to be a match. Let Ω_A and Ω_B be two regions enclosed by the surfaces A and B respectively, PFP is defined as follows:

$$\mathit{\text{PFP}}=\frac{\mathit{\text{Volume}}({\Omega }_B)-\mathit{\text{Volume}}({\Omega }_A\cap {\Omega }_B)}{\mathit{\text{Volume}}({\Omega }_A)}.$$

Table 3.

Evaluation of Registration: Organ Overlaps (%)

	Bladder	Uterus
Rigid Registration	65.76 ± 6.69	68.32 ± 5.18
Nonrigid Registration	80.34 ± 2.39	77.28 ± 3.84
Proposed Method	90.58 ± 1.95	87.80 ± 2.24

Table 4.

Evaluation of Registration Error: PFP (%)

	Bladder	Uterus
Rigid Registration	33.57 ± 5.34	30.92 ± 5.22
Nonrigid Registration	19.21 ± 3.41	21.78 ± 4.03
Proposed Method	9.36 ± 1.40	12.06 ± 1.96

From the Table 3 and Table 4, we found that the RR performed the poorest out of all the registrations algorithms, while the proposed method significantly outperformed the NRR at aligning segmented organs.

Detection Results

We compare the tumor images obtained using our method with the manual detection performed by a clinician. For the proposed method, we set a threshold for the tumor probability map M. Fig.4(b) shows the detection rely entirely on the histogram in Fig.1 alone, which can not provide a meaningful result. Fig.4(c)–(h) provide the comparison between the proposed method and the detection by an expert. The tumor binary images are obtained with the probability threshold 0.7, which presents all the voxels that have have an over 70% chance to be in the tumor. From our experiments, we have found that the detection results are not sensitive to the threshold. The thresholds between 0.5 and 0.85 give almost the same detection results.

Fig. 4.

Detection of cervical tumor for a patient. (a)3D surface of the detected tumor using out method. (b) Detection using the intensity information according to the histogram in Fig.1. (c)–(h): Comparison. Top: Tumor outlined manually by a clinician. Bottom: Detection results using our unified algorithm. Results from the first, third, and fifth week during the treatment are shown here.

From the experiments, we find that the tumor shape has a strong influence on the performance of the detection results. As a global trend, algorithms show more accurate detection on well-defined (e.g. Fig.4(c)(d)) than on ill-defined masses (e.g. Fig.4(e)(f)). Meanwhile, the tumor size has a weak influence on the detection performance. There is not a specific size where the algorithm performs poorly, i.e. the algorithm is not sensitive to the tumor size, as shown in Fig.4.

The detections of the algorithm are considered true positives if they overlapped with a ground truth (manual detection), and false positives otherwise. Free-response receiver operating characteristic (FROC) curves [13] are produced as validation of the algorithm. For performing the FROC analysis, a connected component algorithm is run to group neighboring voxels as a single detection. The algorithm achieved 90% sensitivity at 6.0 false positives per case. In comparison, the algorithm from [15] gave only 74% sensitivity at the same false positive rate. The corresponding FROC curves are shown in Fig.5. The proposed method shows better specificity and achieves a higher overall maximum sensitivity.

Fig. 5.

FROC curves, compared with the approach in [15]

Fig. 6 shows a representative slice from the planning day MRI with overlaid five treatment day tumor detection contours. It can be seen that the detected tumor appears in the same slice and the same location in the registered serial treatment images, and tumor regression process is clearlly visible from the contours. Using the proposed technique, we can easily segment the organs of interest, estimate the mapping between day 0 and day d, and calculate the location changes of the tumor for diagnosis and assessment, thus we can precisely guide the interventional devices toward the lesion during image guided therapy.

Fig. 6.

Mapping the detected tumor contours to planning day MRI

4 Conclusion

In this paper, a unified framework for simultaneous segmentation, nonrigid registration and tumor detection had been presented. In the segmentation process, the surfaces evolve according to the constraints from deformed contours and image gray level information as well as prior information. The constrained nonrigid registration part matches organs and intensity information respectively while taking the tumor detection into consideration. We define the intensity matching as a mixture of two distributions which describe statistically image gray-level variations for both pixel classes (i.e. tumor class and normal tissue class). These mixture distributions are weighted by the tumor detection map which assigns to each voxel its probability of abnormality. We also constraint the determinant of the transformation’s Jacobian, which guarantees the transformation to be smooth and simulates the tumor regression process. In the future, we plan to develop a system that incorporates a physical tumor regression model.