Shape-correlated Deformation Statistics for Respiratory Motion Prediction in 4D Lung

Abstract

4D image-guided radiation therapy (IGRT) for free-breathing lungs is challenging due to the complicated respiratory dynamics. Effective modeling of respiratory motion is crucial to account for the motion affects on the dose to tumors. We propose a shape-correlated statistical model on dense image deformations for patient-specic respiratory motion estimation in 4D lung IGRT. Using the shape deformations of the high-contrast lungs as the surrogate, the statistical model trained from the planning CTs can be used to predict the image deformation during delivery verication time, with the assumption that the respiratory motion at both times are similar for the same patient. Dense image deformation fields obtained by diffeomorphic image registrations characterize the respiratory motion within one breathing cycle. A point-based particle optimization algorithm is used to obtain the shape models of lungs with group-wise surface correspondences. Canonical correlation analysis (CCA) is adopted in training to maximize the linear correlation between the shape variations of the lungs and the corresponding dense image deformations. Both intra- and inter-session CT studies are carried out on a small group of lung cancer patients and evaluated in terms of the tumor location accuracies. The results suggest potential applications using the proposed method.

1. INTRODUCTION

4D image-guided radiation therapy (IGRT) in general is still in its early stage of development¹. The treatment of inoperable tumors in lung remains a therapeutic challenge. Increasing the radiation dose improves local control but requires high-precision techniques. There is widespread interest in linear accelerator-mounted cone-beam computed tomography (CBCT) systems for guiding radiation treatment, but respiratory motion adversely affects CBCT image quality and limits its accuracy. Improved localization accuracy will permit smaller treatment volumes, thereby enabling safe delivery of higher radiation doses. Our long-term goal of this research project is to improve the accuracy and reliability of CBCT guidance of radiation treatment in lung.

Non-rigid registration has been adopted to characterize the respiratory motion in the whole imaging space or within the major organ region^2–6. However, the regular non-rigid image registrations may not be usable due to serious reconstruction artifacts introduced by inevitable breathing motion. A model-based approach is to approximate a linear motion model of the breathing mechanism from simulation data and use the model to predict the motion of the target images. In practice, the simulation data used for training would be respiratory-correlated CT (RCCT images) obtained at the planning time and the target image used for on-line guidance could be CBCT, RCCT or megavoltage CT (MVCT), depending on different IGRT applications.

Recognizing the hysteresis and irregular breathing patterns, auxiliary devices, such as the pencil-beam navigators⁷ and the spirometer⁸, have been used to obtain extra modeling parameters as the surrogate for the motion estimation. Distinctive image features, such as high image contrast features, are potential surrogate candidates. In abdomen liver contour has been used as the surrogate for tracking the intra-hepatic tumors, which are hard to visualize in the CBCT scans⁹. In thorax air-filled lungs are the major sources of motion observation and most easily visible from the diaphragm region (the bottom of the lungs). Zhang et al.¹⁰ proposed to use the diaphragm position as the surrogate for estimating the CT respiratory motion and showed effective prediction results. Nevertheless a more comprehensive surrogate is needed to navigate the whole image deformations with better accuracy, especially in the superior regions of the lungs.

Considering its large coverage and high-contrast intensity feature, the shape of the lungs is commonly used to evaluate the motion estimation accuracy^{5, 10}. We propose to use the shape of the lungs as the surrogate to predicate patient-specific respiratory motion described by dense displacement field. We have carried out some preliminary studies on building a shape-guided statistic model, using a deformable medial representation for extracting lung shapes and multivariate linear regression for calculating correlation statistics¹¹. In this paper, we improve the method in both accuracy and efficiency by adopting a point-based surface correspondence method, which is designed to build shape models with good group-wise correspondence across the training samples. We also adopt the canonical correlation analysis to maximize the correlation between the shape and image deformations and reveal the relationship between different correlation methods in our application. Finally, prediction evaluations are demonstrated in intra-session RCCT studies of five patients and inter-session RCCT studies of 3 patients.

2. METHODOLOGY

Given the planning CT images at different phases within one breathing cycle, our method begins by calculating dense diffeomorphic image deformations from the end of expiration (EE) phase to all other phases (Sec.2.1). Meanwhile, shapes of the anatomical structures are automatically extracted from each phase image and the group wise surface correspondence is optimized via a point-based correspondence program (Sec.2.2). PCA are then applied to both the training shapes and the image deformations to reduce the dimensionality of the data (Sec.2.3). Finally by correlating the shape changes with the temporally corresponding image deformations, we compute a linear mapping which maps a shape model temporal change to a dense image deformation field (Sec.2.4). This linear correlation is then ready to apply to the lung shapes of target images to predict the corresponding image deformations. This relative computationally expensive training process can be done completely off-line. Given a target image at an arbitrary phase, we only need the extracted shape to estimate the corresponding image deformation using the trained statistics.

2.1. Motion description via diffeomorphic registration

Fluid based diffeomorphic image registration¹² is used to obtain the image deformation of all phases in one respiratory cycle. The registrations generate dense and smooth displacement vector fields for each phase, which are used to characterize the respiratory motion.

In a respiratory sequence, each image is denoted by I_t, where t is phase number that is typically from 0 to 9 for one complete breathing cycle, with approximately 0.5 seconds intervals. We use the end of expiration (EE) phase I₅, usually the middle phase, as the reference/fixed image due to its high reproducibility. For each image I_t with dimension of I × J × K, the corresponding dense deformation is denoted by the vector U_t = [u_t (0, 0, 0), …, u_t (i, j, k), …, u_t (I, J, K)]^T, where u_t (i, j, k) = [Δx, Δy, Δz] is the displacement vector field as the result of diffeomorphic registration with I₅. As a result of the choice of the reference phase, U₅ itself is zero.

Given the reference phase image and the displacement vector field, we can warp the reference image and get the deformed image or propagate any known anatomical structure contours from the reference image.

2.2. Shape modeling with group-wise correspondence

Due to high intensity contrast of the air filled lung regions in CT, the respiratory motions can be observed from the shape changes of the lungs (left lung and right lung) clearly, especially near the diaphragm regions. As the major source of motion in the imaging field, the geometrical variations of lungs are used as the surrogate for the temporally corresponding dense image deformation. Furthermore the high-contrast shape boundaries make the automatic segmentation feasible.

Binary lungs are extracted using a simple automated segmentation pipeline.¹³ First, a binary threshold is manually set for all the CT scans to extract the air-filled low intensity lung regions roughly. The higher intensity small vascular structures leave small holes in the rough segmentation. Then a 3D math-morphological “ball rolling” operator fills the remaining holes and remove unnecessary details in the connecting region of bronchi structures, airways and lungs.

The surface point distribution model (PDM) representing the shape of lungs is denoted by P_t = [p_t(0), …, p_t(N)]^T, where p_t(i) = [x, y, z] is the ith point on the surface and N is the number of surface points. The number of phases in a breathing cycle limits the training sample size to a small number. Considering the dimensionality of the displacement field, the shape correspondence condition therefore directly affects the effectiveness of the statistics in this high dimension low sample size (HDLSS) problem.

An state-of-art point-based surface modeling method^{14, 15} is adopted to construct compact shape models with group-wise shape correspondence. The method does not rely on any specific surface parameterization and topology. In this algorithm, surface points samples, defining the shape-to-shape correspondences, are modeled as sets of dynamic particles whose position is subject to entropy minimization. A fixed number of particles are automatically generated for all binary images at once with a quick optimization convergence. Examples of the PDM for the EE and EI (end of inspiration) phase are shown in Fig.1. In our experiments each pair of lungs contains 512 surface points.

Figure 1.

Segmentation and shape modeling. First three images from the left: the axial, coronal and sagittal views of the binary segmentation of the EE (white) on top of the EI (gray) phase; Two images on the right: The PDMs, composed of 512 points, shown on the surface of the lungs at EE phase and at EI phase with group-wise correspondence indicated by colors.

Comparing with the deformable m-rep model construction method used in our previous study¹¹, the point-based correspondence method has several advantages: 1) The algorithm explicitly optimizes on a group-wise correspondence measurement, which produce more reliable correspondence condition that is crucial for statistical analysis; 2) It handles multiple-objects shape and arbitrary topology, which makes the task of modeling two lungs at the same time much easier; 3) No initialization model is needed and the surface sampling density can be adjusted as needed. The computation of the point-based optimization is also efficient considering it generates all the training models at once. Users can also adjust the resolution of the surface sampling. Less dense sampling means shorter computation time. Our experiments suggest that 512 points are dense enough to represent the variation details of the lung shape for our prediction purpose.

2.3. PCA on shape and image deformation spaces

Both shape surrogates and the dense volume deformation representations are high dimensional. For 3D CT lung images with typical resolution of 512 × 512 × 100 the image deformation fields demand large storage. The dimension of 3D shape representation for lungs are also in the order of thousands. Dimension reduction is therefore necessary to carry out any computations involving covariance matrix between two large data matrices. Principal component analysis fits the task well. With good surface correspondences across the phases in one breathing cycle, we can carry out PCA on the 4D data, treating the shapes of each phase as sampling points in the shape space. Similarly for the image deformation, the correspondence of the displacement vectors at each voxel position (as a result of the image registration) allows the usage of PCA. With small number of training samples, it only takes 2–4 principal modes to cover 90% total variations. Removing noises in the data is shown to make notable differences in our application. We therefore only use those first 2–4 modes.

$${\mathbf{P}}_t={\mu }_{\mathbf{P}}+{\displaystyle \sum _{i=1}^{k_P}{\mathbf{x}}_i{\mathbf{e}}_i^P={\mu }_{\mathbf{P}}+X_t{\mathbf{E}}_P,}$$ (1)

$${\mathbf{U}}_t={\mu }_{\mathbf{U}}+{\displaystyle \sum _{i=1}^{k_U}{\mathbf{y}}_i{\mathbf{e}}_i^U={\mu }_{\mathbf{U}}+Y_t{\mathbf{E}}_U,}$$ (2)

where μ is the sample mean, e_i is the ith eigenvector, E is the matrix form of the eigenvectors, x_i/y_i is the corresponding coefficient or the projection score of the data in the ith eigendirection, and k is the number of eigenmodes. With the training sample consisted of N phases, N-1 eigenmodes at most for both shape deformation and image deformation spaces are obtained.

In Figure.2 we show the PCA results of one patient data set (10 phases) in one breathing cycle. Seperate PCA spaces for the shape deformations and the image deformations are calculated first from the 10 training samples. Then we project the training samples back to the first three principal modes to understand the distributions of the 10 phases in the PCA space. The three dimensional spaces are spanned by their first three principal components, whose score units are normalized by the standard deviations of each modes (1.0 of PC-1 means one standard deviation away from the mean along PC-1 direction). In the shape variation space, the first, second and the third mode takes up 76.2%, 15.5% and 3.6% total variations respectively. While in the image variation space, the first three modes take 67.1% 19.2% and 4.7% of the total variation. Strong linear correlation between the two spaces can be revealed from the first couple of modes that collects the majority sampling variation. For example, the data used in Figure.2 has the correlation coefficient of 0.98 between the first PC modes of the shape and image deformation spaces and 0.96 between the second PC modes.

Figure 2.

PCA results of shape and image deformation spaces from one sequence of RCCTs, where the number tagged with a point in space is the phase number in one breathing cycle. Left: Shapes of the 10 training phases projected back onto the space spanned by the first three principal components Right: Dense image deformations of 10 training phases project back onto the space spanned by the first three principal components. The Euclidean sample mean is marked by the solid red circle.

2.4. Shape-correlated image deformation statistics

2.4.1. Canonical Correlation Analysis

Canonical correlation analysis (CCA) has been recently used in biomedical image analysis fields, especially for predictive modeling of brain anatomical structures in neuroscience^16–18. Here we use CCA to maximize the correlation between the shape surrogate and it corresponding dense image deformation in the training sample.

After the PCA dimension reduction, the canonical correlation (CC) between the two multidimensional variables X_t and Y_t is maximized via the following canonical transformation. Given data matrices X = [X₀ − X̅, …, X_t − X̅, …, X₁ − X̅] and Y = [Y₀ − Y̅, …, Y_t − Y̅, …, Y₁ − Y̅] where X̅ and Y̅ are the mean vectors of the training sample, the canonical variables α and β is calculated by projecting the data onto the basis matrices A and B :

$$\alpha ={\mathbf{X}}^T\mathbf{A},\beta ={\mathbf{Y}}^T\mathbf{B}.$$ (3)

The canonical correlation basis A and B is calculated by solving the eigenvalue equations :

$$\{\begin{array}{c}{\mathbf{C}}_{XX}^{-1}{\mathbf{C}}_{XY}{\mathbf{C}}_{YY}^{-1}{\mathbf{C}}_{YX}\mathbf{A}={\rho }^2\mathbf{B}\\ {\mathbf{C}}_{YY}^{-1}{\mathbf{C}}_{YX}{\mathbf{C}}_{XX}^{-1}{\mathbf{C}}_{XY}\mathbf{A}={\rho }^2\mathbf{B},\end{array}$$ (4)

where C_XX and C_{Y Y} are the within-sets covariance matrices, and C_XY and C_{Y X} is the between-sets covariance matrix of X and Y. And the eigenvalues ρ² are the squared canonical correlations defined as $\rho =\frac{{\mathbf{A}}^T{\mathbf{C}}_{XY}\mathbf{B}}{\sqrt{{\mathbf{A}}^T{\mathbf{C}}_{XX}\mathbf{A}{\mathbf{B}}^T{\mathbf{C}}_{YY}\mathbf{B}}}$

To use the statistics for predicting the dense image deformation Û_t* of a target image at phase t*, the following steps are carried out:

1. Reparameterized the shape P_t* by $X_{t*}=(P_{t*}-{\mu }_{\mathbf{P}}){\mathbf{E}}_P^T$;

2. Calculate the CC representation of the shape α_t* = (X_t* − X̅)A ;

3. The canonical variate β_t* = α_t* ρ;

4. The CC representation of the displacement field Y_t* = β_t*B⁻¹ + Y̅ ;

5. Finally, the predicted displacement Û_t* = μ_U + X_t* E_U.

2.4.2. Relationship with MLR and PLS

There are alternatives for carrying out statistical prediction from the surrogate deformation to the target image deformation, such as multiple linear regression (MLR) and partial least square method (PLS). All three methods, trying to relate one group of data with the other, can be unified into the following generalized eigenproblem or two-matrix eigenproblem form¹⁹:

$$\mathbf{A}w=\lambda \mathbf{B}w.$$ (5)

MLR, CCA and PLS correspond to find the subspaces of minumum square error, maximum correlation and maxinum covariance respectively. The corresponding covariance matrixes used for A and B are shown in Table.1. In our application dimension reductions via PCA are first carried out to both data sets (X and Y) before calculating the correlation. C_xx and C_yy are therefore both scaled to identity matrix I, which makes all the three methods deliver equivalent results. One merit of the CCA method is that the correlation coefficients quantitatively indicate how strong the correlation is between the surrogate and the motion, which helps to understand how well the statistics model fit into the data.

Table 1.

Two-matrix problem foumular for MLR, CCA and PLS

	A	B
MLR	$\left(\begin{array}{cc}0& C_{xy}\\ C_{yx}& 0\end{array}\right)$	$\left(\begin{array}{cc}{C}_{xx}& 0\\ 0& \mathbf{I}\end{array}\right)$
CCA	$\left(\begin{array}{cc}0& C_{xy}\\ C_{yx}& 0\end{array}\right)$	$\left(\begin{array}{cc}{C}_{xx}& 0\\ 0& C_{yy}\end{array}\right)$
PLS	$\left(\begin{array}{cc}0& C_{xy}\\ C_{yx}& 0\end{array}\right)$	$\left(\begin{array}{cc}0& C_{xy}\\ C_{yx}& 0\end{array}\right)$

3. EXPERIMENTAL RESULTS

In our experiments the respiratory correlated CT (RCCT) data sets are provided by a 4-slice scanner (lightSpeed GX/i, GE Medical System), acquiring repeat CT images for a complete respiratory cycle at each couch position while recording patient respiration (Real-time Position Management System, Varian Medical Systems). The CT images are retrospectively sorted (GE Advantage 4D) to produce a series of 3D images at different respiratory time points. The training image sequences used for build the statistic model has the time resolution of 0.5 s and the CT slice thickness of 2.5 mm. We first carried out intra-session study, where the statistics are trained and tested within one sequence. In the inter-session study, we have secondary RCCT scan from the same patient to test the prediction results. The secondary RCCT sequence is imaged at a different day and has poor spatial and temporal resolutions and much smaller field of views.

3.1. Intra-session study: Leave-one-phase-out

Leave-one-phase-out (LOPO) study was carried out for each of the five patients. Namely, the statistics that used on each target phase image were trained from the other 9 phases from the same breathing cycle. For each patient the EE phase image was used as the reference image for registration, and the gross tumor volume (GTV) of the EE phase (contours provided by clinicians’ delineation) is also used as the reference GTV for validation.

In Fig.3 we show the LOPO prediction results on five patients (50 predictions in total). Standard boxplots are used for statistical comparison. We measured the Euclidian distance between the predicted displacement field and the “true” displacement field calculated from diffeomorphic registration. The 90% quantile surface distance accounts for large discrepancies. Given clinicians’ delineations on the GTV, we are able to measure the center of gravity (COG) errors of the predicted GTV. In order to better understand the scale of the tumor motion, we show the errors of without motion correction (static GTV), in which the reference GTV is used for all phases. We also plot the estimation errors when directly applying the image deformation to the GTV via the diffeomorphic registration used in training. The LOPO studies show that the prediction effectively corrects the tumor motion and is close to the results produced from direct image registration. The maximum error of 0.25cm surface distance is in fact within one voxel size, 0.1cm × 0.1cm × 0.25cm. In both patients, our shape-navigated statistical model largely reduces the estimation error from the static (no registration) method. We show an example of predicted tumor volume overlapped with the clinicians’ manual segmentations in Fig.4.

Figure 3.

LOPO study on 5 patients (50 predictions in each notched box). Left: The 99th percentile euclidian distance between the predicted displacement field and the “true” displacement field; Right: The euclidian distance on the center of gravity between the deformed GTV and the delineation truth.

Figure 4.

A median case (from the 50 predictions in terms of the COG error) example of predicted GTV in sequential 2D slices. Blue: predicted tumor voxels; Red: the manual delineations; Yellow: the overlapping volumes. This predicted GTV has the COG error of 1.9 mm and the Dice’s coefficient of 0.86.

3.2. Inter-session study

The LOPO studies in the last section are done within a single RCCT sequence, where both shape and image deformation statistics are fairly tight. While in the inter-session study, a RCCT sequence obtained on one day are used to predict the motion for another day. Although we assume that the breathing patterns of the same patient are repeatable in certain degree, it is not surprising to observe big variations between sequences taken years apart. Without extra known variability introduced to the statistics, our current model might not be able to handle large variations between the training and the test scan. In the following experiments, we have primary RCCT sequence and secondary RCCT sequences of the same patient taken with different imaging protocols. Besides the poorer resolution, the secondary RCCT sequences has much smaller imaging region and only four phase bins reconstructed. The coverage in the longitudinal direction of the second session is about 6 to 10 cm, centered around the tumor region, as shown in Figure.5. The four phases roughly sample the whole breathing cycle, including the EE and EI phases. The statistics trained from all ten phases of the primary session are used to predict the motion in the secondary session.

Figure 5.

Left: The second session EI phase image is shown on top of the first session EI phase Image: a coronal view, the intersection region is the darker region on the bottom part of the lungs; Right: The PDM of the partial lung shape, composed of 64 surface points, shown with the surface of the lungs at EE phase and at EI phase with group-wise correspondence.

A rigid registration based on the veritable bones was used to align the two sessions first. The pre-calculated displacement field and binary lungs at each phase from the first session were down-sampled and cropped to match the resolution and the size of the secondary scans. The correlation statistics modeling process are not much different than the intra-session study. With the same imaging modality, the same binary segmentation scheme can be directly carried out for the secondary scan. And we use the EE phase of the primary scan as the reference image for calculating the image deformation of the training set. The only notable difference for our inter-session study is that the shapes of the surrogate extracted from the target image are no longer the full set of lungs, but the bottom part of the lung region with a cut plane on top. Since our point-based correspondence algorithm does not depend on topology, the points on the top plane are not treated differently and therefore forced to be matched across the phases, which introduces artificial variations more or less. Also the real in-and-out-of plane motion variations during the breathing will not be constructed into the statistics due to the cut off. As a result, the statistical model in this setting is expected to be less robust challenged by the noises.

The preliminary tests were evaluated on 3 patients in terms of the tumor location errors (COG error), see Figure.6. Clinicians’ delineations on the tumors were provided on EI and EE phases of the secondary scan. The predicted image deformation propagated the tumor volume from the reference image (EE phase of the primary scan). The shown prediction results were able to correct some of the large tumor motions as large as 1.5 cm. The errors are in much bigger mm magnitude, partly due to the double-sized slice thickness we adopted to match the secondary scans.

Figure 6.

Inter-session study on 3 patients (each has 10 first-session images for training) : The COG errors of the predicted GTV are compared with the clinician’s delineations on EE and EI phases.

4. DISCUSSION AND CONCLUSION

4.1. Surrogates comparison

In earlier work of Zhang et.al¹⁰, the diaphragm position along the longitudinal axis was used as a distinctive one dimensional surrogate signal, which can be manually labeled in 3D. The one dimensional diaphragm surrogate might be capable of navigate the major breathing motion, but oversimplified motion model is not reliable towards accurate motion prediction, especially irregular breathing deformation with tumors located in various places in the lung. The further away the region is to the diaphragm, the less accurate the deformation might become. Especially in cases where the tumors are located in the upper part of the lung, the deformation around tumor might not correlated well with the diagram.

In our current method a lot more features are used by extracting the shape of the whole lung region, which therefore is designed to be more comprehensive and robust for the thorax motion prediction. In order to quantitatively understand the advantages of engaging the more complicated surrogate, we roughly compare the prediction results by replacing the shape model with the diaphragm signal that are manually recorded. Linear regression was computed to map the diaphragm position to the dense image deformation field. We carried out the LOPO studies using 4D RCCT sequences from two patients, in which the tumors are located in the neighborhood region of the diaphragms. The evaluations in terms of the prediction errors on the displacement field, lung boundaries and GTV are shown in Figure.7. As a local feature, the diaphragm position, is shown to be less capable of predicting the nonrigid deformation of the entire imaging field.

Figure 7.

Comparison of the surrogates on 2 patients (20 predictions in each notched box). Left: The 99th percentile euclidian distance between the predicted and true displacement fields are plotted to show the large discrepancies. Right: The center of gravity of the predicted GTV are compared with the clinician’s delineations.

4.2. Conclusion

The proposed shape-correlated deformation statistics is used to predict the patient-specific respiratory motion efficiently in both intra- and inter-session studies. The linear correlation between the shape and image deformation spaces is shown to be encouraging. In our modeling, we treat each phase in a breathing sequence as an independent sample. No temporal correlation among the phases is assumed, which makes the method applicable to irregular breathing patterns often seen in reality.

Large inter-session data sets need to be collected for further large-scale evaluation. Given 4D image sequences both at planing time and treatment time in different imaging modalities, such as Tomosynthesis and CBCT, our next goal includes estimating the respiratory motion if inter-session scans with different imaging modalities, which requires to extract shapes from the target images and maintain the correspondence with the training images. Also, it is possible to extend the methodology to inter-patient studies to include more training samples for more robust statistics.