Reduction of irregular breathing artifacts in respiration-correlated CT images using a respiratory motion model

Abstract

Purpose: Respiration-correlated CT (RCCT) images produced with commonly used phase-based sorting of CT slices often exhibit discontinuity artifacts between CT slices, caused by cycle-to-cycle amplitude variations in respiration. Sorting based on the displacement of the respiratory signal yields slices at more consistent respiratory motion states and hence reduces artifacts, but missing image data (gaps) may occur. The authors report on the application of a respiratory motion model to produce an RCCT image set with reduced artifacts and without missing data.

Methods: Input data consist of CT slices from a cine CT scan acquired while recording respiration by monitoring abdominal displacement. The model-based generation of RCCT images consists of four processing steps: (1) displacement-based sorting of CT slices to form volume images at 10 motion states over the cycle; (2) selection of a reference image without gaps and deformable registration between the reference image and each of the remaining images; (3) generation of the motion model by applying a principal component analysis to establish a relationship between displacement field and respiration signal at each motion state; (4) application of the motion model to deform the reference image into images at the 9 other motion states. Deformable image registration uses a modified fast free-form algorithm that excludes zero-intensity voxels, caused by missing data, from the image similarity term in the minimization function. In each iteration of the minimization, the displacement field in the gap regions is linearly interpolated from nearest neighbor nonzero intensity slices. Evaluation of the model-based RCCT examines three types of image sets: cine scans of a physical phantom programmed to move according to a patient respiratory signal, NURBS-based cardiac torso (NCAT) software phantom, and patient thoracic scans.

Results: Comparison in physical motion phantom shows that object distortion caused by variable motion amplitude in phase-based sorting is visibly reduced with model-based RCCT. Comparison of model-based RCCT to original NCAT images as ground truth shows best agreement at motion states whose displacement-sorted images have no missing slices, with mean and maximum discrepancies in lung of 1 and 3 mm, respectively. Larger discrepancies correlate with motion states having a larger number of missing slices in the displacement-sorted images. Artifacts in patient images at different motion states are also reduced. Comparison with displacement-sorted patient images as a ground truth shows that the model-based images closely reproduce the ground truth geometry at different motion states.

Conclusions: Results in phantom and patient images indicate that the proposed method can produce RCCT image sets with reduced artifacts relative to phase-sorted images, without the gaps inherent in displacement-sorted images. The method requires a reference image at one motion state that has no missing data. Highly irregular breathing patterns can affect the method’s performance, by introducing artifacts in the reference image (although reduced relative to phase-sorted images), or in decreased accuracy in the image prediction of motion states containing large regions of missing data.

INTRODUCTION

Respiration-correlated CT (RCCT) is widely used to obtain motion information of tumor and organs at risk for radiation treatment planning. RCCT is commonly used in commercial systems to define an internal target volume that encompasses the range of tumor motion in all images. An alternative approach is to determine a mean tumor position from RCCT and compute margins based on dosimetric considerations.¹ With RCCT it has become possible to account for respiratory motion effects on dose received by tumor and organs, using displacement vectors derived from deformable image registration to estimate the trajectory of each voxel and compute cumulative dose.^{2, 3} Investigations that incorporate motion compensation into optimization of intensity-modulated treatment plans have shown potential in reducing dose to nearby organs at risk.^{4, 5, 6, 7} All of these applications depend in varying degrees on the integrity of RCCT images.

In commercial systems, CT slices are acquired over several breathing cycles and are sorted based on a phase computed from the cyclical respiratory signal recorded by an external monitor, yielding a set of volume images at different motion states in the cycle. Phase-based sorting assumes repeatable breathing cycles, i.e., that the anatomy is at the same position for a given phase in every cycle, but in cases of normal breathing with varying cycle-to-cycle amplitude variations, it can lead to inconsistencies in the internal motion state and hence discontinuity artifacts in anatomy between slices (Fig. 1).^{8, 9, 10, 11} Sorting based on the displacement of the respiratory signal, or on tidal volume measured with spirometry, selects slices at consistent motion states and hence reduces artifacts, but there may be no acquired data for a given displacement at some slice positions, thus introducing gaps in the volume image.^{12, 13, 14, 15}

Figure 1.

Comparison of phase-based and displacement-based sorting. Plot of external RPM marker displacement vs time (black curve), and diaphragm position vs time (gray curve), in an irregular breathing example. Crosses indicate phase-based sorting at 0% phase; images selected at these points are at inconsistent respiratory states as indicated by the diaphragm variability. Horizontal line indicates the displacement-based sorting, which yields more consistent respiratory states between images but sometimes results in missing data (arrows).

Numerous methods have been proposed to reduce artifacts in RCCT image sets. Schreibmann et al. have used deformable image registration and interpolation of the deformation fields to derive images at intermediate inspiration and expiration states, starting from phase-sorted images at end expiration and end inspiration.¹⁶ The method assumes artifact-free images at end expiration and end inspiration, which is often not achievable with phase-based sorting. Ehrhardt et al. have used deformable image registration to replace an image containing artifacts at a given respiration state with one interpolated from neighboring images in the RCCT set.¹⁷ The method assumes that the neighboring images are relatively artifact-free, contain no gaps, and that the motion is in the same direction, i.e., inspiration or expiration. Zeng et al. have used a breathing index consisting of centroids of image slices to initially sort two reference volumes near end expiration and end inspiration, then iteratively refine the sorting using a motion model derived from deformable registration between the reference volumes.¹⁸ A limitation is that the method assumes that data exist at all couch positions for matching to the reference volumes.

Li et al. have examined phase-based sorting according to a combination of internal anatomical features.¹⁹ A limitation of the method is that the internal measures are not directly comparable among slices at different couch positions, which prevents the application of displacement-based sorting. Carnes et al. acquire cine images where successive couch positions overlap at one slice; the sorting procedure chooses images at adjacent couch positions that maximizes a normalized correlation coefficient of overlapping slices.²⁰ Johnston et al. sort images based on cross correlation of slices bordering adjacent couch positions.²¹ Gianoli et al. use K-means clustering to sort based on multiple reflective markers placed on the patient surface.²² A limitation of the above methods is that they do not address missing data at couch positions when the patient breathes shallowly.

In this study, we investigate the application of a respiratory motion model that we have previously developed²³ to produce an RCCT image set with reduced artifacts and without gaps.

METHODS

Input data consist of a cine CT image set (8-slice Lightspeed, GE HealthCare, Waukesha, WI) acquired while simultaneously recording respiration by monitoring abdominal displacement (Real-time Position Management RPM, Varian Medical Systems, Palo Alto, CA). Acquisition time per couch position is set to the patient’s respiration period (measured with RPM) plus 1 s, with gantry rotation period of 0.5 s. The time interval between consecutive images is the greater of either 1/20 of the respiration period or 0.25 s, thus yielding 20 or more repeat images at each couch position. Reconstructed CT slice thickness is 2.5 mm. The RPM data provide a tag of abdominal displacement (marker block position) and phase for each image. The slices are sorted into 10 phase bins (GE Advantage 4D) that provides a baseline for comparison to the proposed method.

The method consists of four processing steps (Fig. 2): (1) displacement-based sorting into 3D images; (2) selection of a reference image without gaps and application of deformable image registration between reference image and each of the other images; (3) generation of the motion model; and (4) generation of synthesized images at different abdominal displacements from the reference image and motion model. We describe each step in more detail.

Figure 2.

Flowchart of proposed method. Cine CT images are displacement sorted (upper left) to produce a set of images at different motion states, some containing gaps. One image without gaps is chosen from the set to be the reference image (middle left). Deformable registration is performed between the reference image and each of the other images, followed by a principal component analysis to produce a motion model (upper right). The motion model is used to deform the reference image, yielding a synthesized image without gaps at each of the motion states (lower right). For evaluation, the synthesized images are compared with the displacement-sorted images (lower left).

Displacement-based sorting

We use a displacement sorting scheme similar to that of Lu et al.¹³ A peak-and-valley finding algorithm is used to detect maxima and minima in the respiration trace (RPM displacement vs time), corresponding to end inspiration (EI) and end expiration (EE), respectively. This defines inspiration (from EE to EI) and expiration (from EI to EE) portions of the trace. The CT slices are sorted first into two groups corresponding to inspiration and expiration, and then sorted by RPM displacement within each group. Each group is divided into five displacement bins, each bin containing approximately equal numbers of CT slices. Bins 1–5 correspond to the expiration portion of the cycle, bins 6–10 to the inspiration portion. For each bin, the 3D image is constructed by choosing, at each CT couch position, the slice closest to the bin center in displacement. If at a couch position, there are no slices within the particular bin, slices with zero CT number are inserted (the multislice scanner yields 8 slices per couch position). This procedure yields 3D images at 10 displacement bins, referred to as motion states. Some of the images may contain gaps which are filled with slices containing zero-intensity pixels.

Deformable image registration

A 3D image at one of the motion states which does not have any gaps is chosen as a reference image. If more than one such 3D image exists, the one closest to end expiration, i.e., a quiescent state with minimal motion, is chosen. This minimizes artifacts in the image between couch positions, resulting from rapid changes in anatomy with displacement. For each of the remaining images, $I_B(\overrightarrow{x})$, deformable image registration (DIR) calculates the voxel-dependent displacement field $\overrightarrow{u}(\overrightarrow{x})$ which maps the image into the reference image $I_A(\overrightarrow{x})$. We use a modified version of a fast free-form algorithm²⁴ which minimizes the energy function $E(u)=\int w(I_B(\overrightarrow{x})){(I_B(\overrightarrow{x}+\overrightarrow{u})-I_A(\overrightarrow{x}))}^{2}d\overrightarrow{x}+\lambda \sum _{i=1}^3\int {|\nabla u_i|}^{2}d\overrightarrow{x}$ where the first term describes the similarity between the images, while the second term is a smoothing term of the displacement field. The weight factor w =0 when $I_B(\overrightarrow{x})=0$, otherwise w =1 ensures that zero-intensity voxels in the gaps do not contribute to the energy function. At each iteration of the minimization, the displacement field in the gap regions is linearly interpolated from nearest neighbor slices with nonzero pixel intensities above and below the gap. This ensures that the displacement field varies smoothly across the gap. Figure 3 shows one such example: the presence of a gap in one of the images causes excessive distortion of the displacement vector field (DVF) when applying the unmodified DIR method (without the weight factor), whereas the modified DIR (with weight factor) yields a smoothly varying DVF.

Figure 3.

Example of how the modified deformable image registration performs in the presence of gaps. (a) Overlay of end expiration (blue enhanced) and end inspiration (red) images before deformable image registration (DIR). Gap is present in the red enhanced image. (b) Image overlay after unmodified DIR. Red line segments indicate direction and magnitude of displacement vector field (DVF). DVF is perturbed and deformed (red enhanced) image is distorted in the vicinity of gap (black circles). (c) Image overlay after modified DIR, see text.

Generation of motion model

The motion model²³ uses a principal component analysis (PCA) to establish a relationship between the DVF and surrogate signal at each motion state: $u(t)=\mathit{Bs}(t)$ where $u(t)={[{\overrightarrow{u}}_1(t),\dots ,{\overrightarrow{u}}_m(t),\dots ,{\overrightarrow{u}}_M(t)]}^T$ is the DVF at motion state t, M is the total number of voxels, and $s(t)={[s_1(t),s_2(t)]}^T$ is the corresponding vector of two surrogate positions. In our model, $s_1(t)$ is given by the abdominal displacement obtained from the external RPM respiratory monitor at the current motion state t, relative to its position in the reference motion state, and $s_2(t)$ is the displacement at motion state t − 3 approximately 1/3 cycle prior, which distinguishes between the inspiration and expiration portions of the cycle. We use two principal components in deriving the motion model, which our prior studies have shown to adequately describe respiratory motion-induced deformation in the thorax.²³ The PCA serves to suppress artifacts and errors in the DVFs that are uncorrelated with surrogate motion.²⁵

Generation of synthesized images

Starting from the reference image, the surrogate signals s(t) of each motion state are input to the motion model $u(t)=\mathit{Bs}(t)$ to obtain the displacement field u(t) which deforms the reference image, yielding a synthesized image at motion state t.

Evaluation

We evaluate the method in a physical motion phantom, in an anthropomorphic software phantom, and with patient images.

A cine CT scan is acquired of a motion phantom (Quasar, Modus Medical Devices, London, ON), consisting of a 30 mm cube and a 20 mm sphere embedded in a cylindrical acrylic insert that moves inside a body phantom, and programmed to move according to a respiratory trace recorded from an irregularly breathing patient. The cine data are processed with the proposed method and the resultant RCCT images are compared with the phased-based RCCT images obtained with the vendor software (GE Advantage 4D).

The NURBS-based cardiac torso (NCAT) phantom is an anthropomorphic software phantom that provides a dynamic model of the human anatomy.²⁶ Organ shapes are constructed from nonuniform rational B-splines (NURBS) surfaces, based on the 3D Visible Human CT dataset. The software models respiratory motion using diaphragmatic contraction and chest expansion, and generates attenuation matrices that are used to produce volumetric CT image sets at user-specified intervals over the respiratory cycle. We use a 5 s respiratory period, 2 cm diaphragm excursion, 0.5 cm anterior–posterior (AP) chest excursion, and generate 10 motion states at 0.5 s intervals. The NCAT images are used to simulate the process as follows: The NCAT CT image at end expiration is rigidly registered to the patient’s phase-binned CT at end expiration such that the diaphragms are approximately aligned, so as to establish correspondence between NCAT CT and patient CT slice positions. Next the patient’s cine CT is binned, according to displacement. The resultant missing slices occurring in some of the motion states are simulated in the corresponding motion states in the NCAT images at the same relative positions along the axial direction. The simulated cine images and respiration trace are processed with the proposed method to yield synthesized images at 10 motion states. The synthesized images are compared with the originally generated NCAT images, which serve as ground truth. Model-based and phase-based RCCT methods are also compared in patient scans of the thorax and abdomen. Displacement-sorted images serve as an approximate ground truth.

To assess the ability of the technique to reduce artifacts in the predicted images, we use an approach based on the property of intensity derivatives at tissue boundaries in images: an intensity change at a boundary causes the derivative to have a peak. The sign of the peak value reflects the change of the tissue density (low to high or vice versa) when crossing the boundary. Following this idea, we take the intensity derivatives in the superior–inferior direction. Since, we are only interested in the amount but not in the direction of intensity change, we only preserve the absolute values (magnitude) of the derivatives. Afterward we take a sum of these values for each slice, normalize by the number of pixels, and plot the resultant filter response as a function of slice number. As a result, at slice locations without artifacts the filter responses for both phase-sorted and synthesized images should demonstrate similar patterns owing to anatomical variation, whereas at slice locations with artifacts, there should appear narrow (two-three slice wide) peaks in the phase-sorted images that are reduced or eliminated in the synthesized images. The filter response development uses the library of C++ classes from the National Library of Medicine Insight Segmentation and Registration Toolkit ITK v4.0.²⁷

RESULTS

Figure 4 shows the results obtained with the physical motion phantom. The top row shows phase-sorted images of the motion phantom at 10 motion states. Artifacts caused by variable amplitude resulting from a patient respiration trace are evident. The middle row shows displacement-sorted images of the same scan. Artifacts are visibly reduced but black regions (gaps) occur in the images where there are no CT slices available in the particular displacement bin (columns 4, 5, 6, 7, 10). The lower row shows the images from the motion model, which are synthesized from the reference image in column 3 of the middle row. One can see that artifacts are largely eliminated, with the exception of the top of images in columns 1 and 10, which is caused by the image deformation near the edge of the limited CT scan field-of-view.

Figure 4.

(a) Sagittal sections of phase-binned CT images of the physical phantom whose motion was programmed with a patient respiration trace. Distortion of spherical and cubic objects in some bins is evident. (b) Displacement-binned CT images. Distortion is reduced but gaps are evident in five bins. (c) Model-synthesized CT images, using the image in bin 3 of row (b) as a reference.

Figure 5 shows results from the NCAT phantom. The first column shows the original NCAT images for three motion states, the second column shows the same images with missing slices resulting from simulated displacement sorting with a patient respiration trace, the third column shows the model-synthesized images from a reference image at the 25% motion state, and the fourth column is a difference image (original minus model-synthesized). The best agreement between model-synthesized and original images occurs for the 5% motion state, whose displacement-sorted image has no missing slices. The mean and maximum discrepancy within the lungs, obtained by applying deformable registration between the model-synthesized and original images, is 1 and 3 mm, respectively. There is progressively more discrepancy in the 55% (mean 1.5 mm, maximum 4 mm) and 75% (mean 2 mm, maximum 5 mm) motion states, respectively, which correlates with the larger number of missing slices (two and four, respectively) in the displacement-sorted images.

Figure 5.

Sagittal sections of NCAT phantom at (a) 05%, (b) 55%, and (c) 75% motion states. First column shows originally generated CT images with spherical tumor in the right lung. Second column shows the same images containing gaps simulated from displacement-based sorting of a patient respiration trace. Third column shows model-synthesized images. Rightmost column shows difference image (original minus model-synthesized) where blue regions are positive (CT numbers in original image are larger) and red regions are negative (CT numbers in model-synthesized image are larger).

In order to further understand the relationship between the method’s accuracy and the amount of missing data, we perform a set of experiments in which increasing numbers of gaps are inserted into the 5% motion state image (near end inspiration), and in some experiments also into neighboring motion state images. The model-synthesized images are computed from the reference image at 55% (near end expiration). The tumor motion extent between the original images at 5% and 55% is 16 mm. The experiments are: (1) no gaps in any images serving as a baseline for comparison [Fig. 6a]; (2) one gap overlapping the region containing the spherical tumor [Fig. 6b]; (3) two contiguous gaps [Fig. 6c]; (4) three contiguous gaps [Fig. 6d]; (5) four contiguous gaps (not shown); (6) five contiguous gaps [Fig. 6e]; (7) two separate gaps [Fig. 6f]; (8) three separate gaps [Fig. 6g]; (9) four separate gaps (not shown); (10) five separate gaps [Fig. 6h]; (11) one gap at 2 motion states (5% and 15%); (12) one gap at 3 motion states (95%, 5%, and 15%); (13) one gap at 4 motion states (95%, 5%, 15%, and 25%); (14) one gap at 5 motion states (85%, 95%, 5%, 15%, and 25%); (15) three separate gaps in images at 3 motion states; (16) three separate gaps in images at 5 motion states; (17) three contiguous gaps in images at 3 motion states; (18) three contiguous gaps in images at 5 motion states. Figure 6i shows the mean and 90th percentile discrepancy between original and model-synthesized images at the 5% motion state, calculated inside the lungs in the gap centered on the tumor [red outline in Fig 6a]. From these results we make the following observations. Gaps in a single image (experiments 2–10) result in a modest to moderate increase in discrepancy, from a mean of 1.5 mm with no gap (experiment 1) up to 2.3 mm with 3 or more contiguous gaps (experiments 4–6). Contiguous gaps (experiments 2–6) have a slightly larger effect on discrepancy than separate gaps (experiments 7–10). A single gap occurring over multiple images (experiments 12–14, mean discrepancy up to 2.8 mm) has a slightly larger effect than multiple gaps in a single image. Multiple contiguous gaps over multiple images (experiments 17 and 18) have a large effect, with mean discrepancy up to 4.5 mm (90th percentile 8 mm) for 3 contiguous gaps over 5 images. The latter corresponds to nearly half the image data missing in the lungs.

Figure 6.

Experiments in NCAT phantom. All panels show the image at 5% motion state near end inspiration. (a) No gaps in any images; red outlines indicate volume-of-interest (VOI) for calculating discrepancy between model-synthesized and original images. (b) One gap overlapping the region containing the spherical tumor; (c)–(e) two, three and five contiguous gaps, respectively; (f)–(h) two, three and five separate gaps. (i) Mean (columns) and 90th percentile (error bars) discrepancy in VOI between model-synthesized and original images for each of the experiments described in the text.

Figure 7 shows a patient example. The phase-sorted image near end inspiration (a) shows several irregular breathing artifacts (arrows), while the displacement-sorted image (b) has gaps caused by missing slices. The model-synthesized image (c) eliminates both artifacts and missing slices. A red-blue overlay of the synthesized and displacement-sorted images (d) shows that the synthesized image closely reproduces the motion state geometry of the displacement-sorted image. There are small discrepancies visible in the lung airway tree.

Figure 7.

Example application of the proposed method in a patient case. (a) Phase-sorted image at 10% phase. Arrows indicate artifacts caused by irregular breathing. (b) Displacement-sorted image at 10% motion state with missing slices. (c) Model-synthesized image. (d) Overlay of synthesized (blue enhanced) and displacement-sorted images (red enhanced).

Figure 8 shows images of the same patient at 4 motion states. Upper row shows phase-sorted images with artifacts (arrows) at phases 0% (end inspiration), 40%, 70%, and 90%. Model-synthesized images (lower row) at the corresponding motion states have visibly reduced artifacts.

Figure 8.

Comparison of phase-sorted (upper row) and model-synthesized (lower row) images at four motion states, for the same patient as in Fig. 7. Arrows indicate artifacts in the phase-sorted images.

Figure 9 illustrates the ability of the derivative-based technique to localize artifacts by comparing the filter responses from phased-sorted and synthesized images. One can see that using the filter response function, the artifacts in the phased-sorted images appear as narrow sharp peaks in the filter response. In the synthesized images these peaks are diminished or eliminated, confirming reduction or elimination of the artifacts.

Figure 9.

Patient example of phase-sorted image with artifacts (top left), model-synthesized image (left bottom), and corresponding plots of filter response vs CT slice (right). Horizontal lines indicate correlations in the locations of image artifacts and sharp narrow peaks in the filter response.

Figure 10 shows the filter responses from both phased-sorted and model-synthesized images for six thoracic patient cases. The arrows indicate narrow peaks corresponding to artifacts in the phase-sorted images. In all cases the peaks are suppressed in the filter response of the synthesized images, indicating that the artifacts have effectively been removed.

Figure 10.

Plot of filter responses vs CT slice for six patient cases, comparing phase-binned (red) and model-synthesized (black) images. Arrows indicate peaks in the red curves, caused by breathing artifacts in the phase-binned images.

DISCUSSION AND CONCLUSIONS

We have proposed a new method for generating an RCCT image set from a cine CT scan, which addresses the problems of artifacts with phase-sorted methods and missing CT slices with displacement-sorted methods. It introduces several new methodologies: A modified deformable image registration of displacement-sorted images to ensure continuity of the displacement field across missing CT slices, application of PCA to ensure continuity of voxel trajectories between motion states, and use of a reference image with minimal artifacts to generate images at other motion states.

The method requires a reference image at one motion state that has no missing slices and is relatively artifact free. It is chosen from one of the displacement-sorted images, from which images at other motion states are synthesized. In cases where there is more than one displacement-sorted image with no missing slices, the motion state closest to end expiration (superiormost diaphragm position) is selected. This image is likely to have the fewest residual motion artifacts, owing to the relatively low motion velocity near end expiration. Cases of highly irregular breathing can affect the method’s performance. In each displacement-sorted bin and at each couch position, slices corresponding to the displacement closest to the bin center are chosen. Since there are a limited number of images to choose from within a bin, there can be variations in the displacements, and thus respiration states of the chosen slices, between couch positions [Fig. 11a]. This can result in artifacts in the displacement-sorted image [Fig. 11b], although the artifacts will be smaller than for phase-sorted images. If the affected image is chosen as the reference image, then the images at other motion states will contain the same artifacts, since they are synthesized from the reference image. One way to reduce these artifacts in some circumstances is to refine the image selection, by averaging the displacements of the selected slices and selecting a new set of slices closest to this average displacement. Alternatively, one may choose as reference a different displacement-sorted image with smaller displacement variations between couch positions, if one is available.

Figure 11.

(a) Plot of respiratory signal (abdominal displacement) vs time. Symbols with the same color denote CT slices occurring within the same displacement-based bin. Horizontal line indicates bin midpoint of the reference motion state, and cyan squares indicate points selected nearest the line at each couch position to generate the reference CT image (b). Fluctuations in the displacement of the selected points [arrows in (a)] result in artifacts in the reference image [arrows in (b)].

A second limitation, observed in the evaluation of NCAT phantom images, is that spatial accuracy is diminished in the prediction of motion states whose displacement-sorted images contain a large amount of missing data (Figs. 56). This is a consequence of the reduced amount of image data available to guide the deformable image registration. The discrepancy is such that the method underestimates the actual motion. It is interesting to note that in cases where there are large amounts of missing data in a single image, the method still performs reasonably well (Fig. 6). This is because the method in effect interpolates not only from neighboring slices in the same image by means of the modified deformable image registration but also interpolates from images in neighboring motion states by means of the motion model derived from PCA. Large regions of missing data occurring over neighboring images will result in the lowest accuracy, because both types of interpolation (across slices and images) are compromised.

A third limitation is that in cases of unusually irregular breathing, there may be no images without gaps at any motion state. Figure 12 shows an example case. The respiration trace acquired during the RCCT indicates that the patient misunderstood the instructions to breathe normally and instead held breath three times during the early part of the scan [Fig. 12a], resulting in missing data in all 10 motion states. The problem can be addressed by reducing the number of bins (in this case, 8 bins) for the displacement sorting. Figures 12b, 12c, 12d, 12e shows four of the eight displacement-sorted images. Note that the coarser binning, in addition to decreasing the temporal resolution of the RCCT, may introduce discontinuity artifacts [Fig. 12d], for the same reasons as discussed above with Fig. 11. Overlays of the model-synthesized and displacement-sorted images [Figs. 12f, 12g, 12h, 12i] show good agreement with reduced artifacts in the synthesized images. We point out that out of 24 patient cases studied to date, only one case exhibited missing data in all 10 displacement-sorted images, suggesting that the likelihood of its occurrence is small.

Figure 12.

Example in which displacement sorting into 10 bins yields no gap-free images. (a) Respiration (RPM) trace of a patient during a respiration-correlated CT scan. Quasi-flat portions in the early part of the trace indicate instances of momentary breath-hold. (b)–(e) Displacement-sorted images at four motion states following displacement sorting into eight bins. Arrow in (d) indicates discontinuity artifacts. (f)–(i) Overlay of model-synthesized (blue enhanced) and displacement-sorted (red enhanced) images.

An alternative approach is to use the method to replace only the missing data in the displacement-sorted images. That is, CT slices from the model-synthesized images are inserted into the CT slices with missing data in the corresponding displacement-sorted images. Figure 13 shows an example displacement-sorted image with gaps, the corresponding model-synthesized image, the displacement-sorted image with gaps replaced by synthesized data, and an overlay of the latter two images. As can be seen, a discontinuity artifact is introduced in one of the gaps. The discontinuity is a consequence of the fact that the model-synthesized image (constructed from two principal components) may differ from the displacement-sorted image with multiple gaps, as previously shown (Fig. 5). The choice of whether to use all-synthesized images or only replace missing data will depend on the clinical application and requires further investigation. On the one hand, we have seen that the model tends to underestimate the actual motion extent in an RCCT set containing many gaps (Fig. 6). On the other hand, replacement of only missing data may introduce discontinuity artifacts, not only in the tumor and other organs in the images but also in the deformation fields and motion trajectories derived from those images.

Figure 13.

Example of replacing gaps in displacement-sorted images with data from model-synthesized images. (a) Displacement-sorted image at 5% motion state with gaps, (b) model-synthesized image at the same motion state, (c) displacement-sorted images with model-synthesized data replacing gaps (arrow indicates area of discontinuity), (d) Overlay of images from (b) and (c).

The proposed method has several potential applications. The first is the improvement in the quality of widely used RCCT images for radiotherapy planning, specifically to obtain motion information of tumor and organs at risk. The method also yields a motion model of the patient that is applicable to 4D treatment planning, that is, it describes a motion trajectory of each voxel that is related to a motion surrogate in the images, such as the diaphragm or implanted fiducials. Such a model can predict deformations from images for which deformable image registration is not possible, such as fluoroscopy, cine radiographs acquired during treatment delivery, or respiration-correlated cone-beam CT reconstructed from sparse projections.²⁵ By using displacement-sorted images as input, the displacement fields derived from the deformable image registration contain fewer artifact-related errors, thus yielding a more reliable motion model.