Introduction of a pseudo demons force to enhance deformation range for robust reconstruction of super‐resolution time‐resolved 4DMRI

Abstract

Purpose

The purpose of this study was to enhance the deformation range of demons‐based deformable image registration (DIR) for large respiration‐induced organ motion in the reconstruction of time‐resolved four‐dimensional magnetic resonance imaging (TR‐4DMRI) for multi‐breath motion simulation.

Methods

A demons‐based DIR algorithm was modified to enhance the deformation range for TR‐4DMRI reconstruction using the super‐resolution approach. A pseudo demons force was introduced to accelerate the coarse deformation in a multi‐resolution (n = 3) DIR approach. The intensity gradient of a voxel was applied to its neighboring (5 × 5 × 5) voxels with a weight of Gaussian probability profile (σ = 1 voxel) to extend the demons force, especially on those voxels that have little intensity gradience but high‐intensity difference. A digital 4DMRI phantom with 3–8 cm diaphragmatic motions was used for DIR comparison. Six volunteers were scanned with two high‐resolution (highR: 2 × 2 × 2 mm³) breath‐hold (BH) 3DMR images at full inhalation (BHI) and full exhalation (BHE) and low‐resolution (lowR: 5 × 5 × 5 mm³) free‐breathing (FB) 3DMR cine images (2 Hz) under an IRB‐approved protocol. A cross‐consistency check (CCC) (BHI→FB←BHE), with voxel intensity correlation (VIC) and inverse consistency error (ICE), was introduced for cross‐verification of TR‐4DMRI reconstruction.

Results

Using the digital phantom, the maximum deformable magnitude is doubled using the modified DIR from 3 to 6 cm at the diaphragm. In six human subjects, the first 15‐iteration DIR using the pseudo force deforms 200 ± 150% more than the original force, and succeeds in all 12 cases, whereas the original demons‐based DIR failed in 67% of tested cases. Using the pseudo force, high VIC (>0.9) and small ICE (1.6 ± 0.6 mm) values are observed for DIR of BHI&BHE, BHI→FB, and BHE→FB. The CCC identifies four questionable cases, in which two cases need further DIR refinement, without missing true negative.

Conclusions

The introduction of a pseudo demons force enhances the largest deformation magnitude up to 6 cm. The cross‐consistency check ensures the quality of TR‐4DMRI reconstruction. Further investigation is ongoing to fully characterize TR‐4DMRI for potential multi‐breathing‐cycle radiotherapy simulation.

1. Introduction

Respiration‐induced tumor motion is one of the major uncertainties in radiotherapy of a thoracic or abdominal tumor and commonly occurring breathing irregularities have not been fully accounted for in the current radiotherapy planning process.^{1, 2} Attempts to incorporate tumor motion variation and statistics in treatment planning have been reported using the probability density function of tumor motion^{3, 4, 5} or a motion margin formula.⁶ Owing to lack of dynamic volumetric imaging, two‐dimensional (2D) cine magnetic resonance imaging (MRI) and 2D portal imaging have been applied to assess tumor motion PDF and its applicability in treatment planning to compensate for the tumor motion.^{7, 8, 9} As a tumor may not move within a plane,¹⁰ 2D cine MR imaging may provide incomplete motion data for a mobile tumor and its nearby organs at risk (OARs) for motion‐compensated planning. The 2D portal or fluoroscopic imaging requires implanted fiducials near a mobile tumor as its surrogates. Conventional respiratory‐correlated (RC) 4D computed tomography (4DCT) and recently developed 4DMRI^{11, 12, 13, 14} only provide a single‐breath composite image series that contain little motion variation, and the single‐breath snapshot of tumor/OAR motions may not represent the patient respiratory motion during treatment.¹⁵

Time‐resolved (or dynamic) volumetric four‐dimensional magnetic resonance imaging (TR‐4DMRI) has the potential to provide multi‐breath motion simulation with breathing irregularities for treatment planning.^{16, 17} However, because of slow physical MR relaxation and large clinical field of views (FOV), it seems impractical to directly acquire TR‐4DMRI images with adequate spatiotemporal resolution.^{16, 18, 19, 20} In the development of MR‐guided radiotherapy, a hybrid approach was reported to generate TR‐4DMRI based on RC‐4DMRI.^{21, 22} This method built a patient‐specific motion model based on RC‐4DMRI using the principal component analysis (PCA) and deformable image registration (DIR) and then reconstructed volumetric TR‐4DMRI with the guidance of 2D cine MR images. As the a priori motion model may be limited by RC‐4DMRI image quality, motion range, and motion pattern, the reconstructed TR‐4DMRI provides an approximation to tumor motion.²² Recently, a super‐resolution (SR) approach was reported to reconstruct TR‐4DMRI by combining two complementary image sets with high‐spatial resolution at breath‐hold (BH) and high‐temporal resolution (3D cine) in free‐breathing (FB) using DIR, providing multi‐breath images with motion irregularities.²³ This SR‐based method reconstructs TR‐4DMRI by deforming the BH image using low‐resolution 3D cine MR images as guidance without assuming periodic motion, allowing imaging of irregular motion. To reconstruct SR‐based TR‐4DMRI reliably and accurately under clinical conditions, the ability to deform large magnitudes of respiratory motion is critical because lung or liver tumor motion can be up to 4 cm in free‐breathing (FB),^{10, 24} breathing irregularities may cause greater motions,^{1, 15} and breath‐hold (BH) can be further away from the FB motion range.²⁵

Deformable image registration has been extensively studied in the past decade for radiation therapy applications, including patient‐specific motion modeling, organ contour propagation, radiation dose mapping, and adaptive radiation therapy.^{26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37} Among many algorithms, the intensity‐based demons algorithm is useful for single‐modality registration,^{28, 38} such as TR‐4DMRI reconstruction.²³ The demons‐based DIR has been widely implemented in programs, such as MATLAB^® (Mathwork, Inc.), ITK (Insight Segmentation and Registration Toolkits),³⁹ and graphical processing unit based codes.^{32, 35, 36} However, this algorithm has a limited deformable range and often requires other methods to boost its deformable range.⁴⁰ Reports on prostate, head and neck, and lung patients have studied 1–2 cm deformation magnitude,^{28, 32, 34, 35, 36, 41} which is at the low range of FB motion (1–4 cm)^{10, 24} and larger BH displacement (6 cm or greater) between full inhalation (BHI) and full exhalation (BHE).²⁵ Therefore, it is necessary to explore and enhance the DIR dynamic range and robustness to ensure reliable image quality of TR‐4DMRI for multi‐breath motion assessment.

To enhance the deformation range, we introduced a pseudo demons force and applied it to the coarse alignment, followed by the fine‐tuning DIR using the original demons forces in a multi‐resolution approach. A separate displacement vector field (DVF) was used at each resolution level and combined to create the composite DVF via vector operation, prior to reconstructing the TR‐4DMRI image. To demonstrate the enhancement of DIR deformation range, the diaphragm dome alignments were measured in the reconstructed TR‐4DMRI with or without the pseudo demons force. In addition, we introduced a cross‐consistency check (CCC) to cross‐verify the quality of TR‐4DMRI reconstruction by comparing the similarity between two reconstructed images using two opposite‐extreme BH images. Several T1‐mapped 4D images created using the 4D XCAT phantom⁴² and six subject images (BHI, BHE, and a 3D cine series) acquired under an IRB‐approved protocol were studied to evaluate the new demons pseudo force in TR‐4DMRI reconstruction.

2. Methods and materials

2.A. General description of super‐resolution time‐resolved 4DMRI approach

A T1‐weighted turbo field echo pulse sequence with TE/TR of 1/4 ms, a flip angle of 15°, an acceleration factor of 4 in SENSE parallel imaging, and a factor of 0.8 in partial Fourier approximation, was used to acquire 3D cine low‐resolution FB images (lowR: 5 × 5 × 5 mm³ at 2 Hz) and 3D high‐resolution BH image (highR: 2 × 2 × 2 mm³) using a 3 T MR scanner (Ingenia, Philips Healthcare). The same field of view (FOV) was used for both the low‐ and high‐resolution MR images.

Prior to registration, the 3D cine FB and 3D static BH image intensities were normalized, and the low‐resolution FB images were interpolated to have the same image matrix size as the BH image. The region of interest (ROI) was defined by the union of the two body contours from the moving and static images. A three‐level multi‐resolution DIR was applied with separated DVFs to reconstruct the TR‐4DMRI images (2 × 2 × 2 mm³ and 2 Hz). The registration log file recorded the number of iterations and associated time of DIR at each level of resolution. Additional detailed MR scanning conditions can be found in a previous report.²³

2.B. Conventional and modified demons force field

In the demons algorithm,³⁸ image gradient (${\overrightarrow{\mathrm{\nabla }}}_{\mathbf{m}}$) from moving image (m) was used as the driving force to move the underlying voxels while applying a Gaussian filter thereby “diffusing” nearby voxels to regulate anatomic integrity. A counter demons force (${\overrightarrow{\mathrm{\nabla }}}_{\mathbf{s}}$) from the static image (s) was introduced to accelerate the deformation.²⁸

$$\overrightarrow{f}=\left(m-s\right)\times \left(\frac{{\overrightarrow{\mathrm{\nabla }}}_{\mathrm{m}}}{{|{\overrightarrow{\mathrm{\nabla }}}_{\mathrm{m}}|}^2+{\alpha }^2{\left(\mathrm{m}-\mathrm{s}\right)}^2}+\frac{{\overrightarrow{\mathrm{\nabla }}}_{\mathrm{s}}}{{|{\overrightarrow{\mathrm{\nabla }}}_{\mathrm{s}}|}^2+{\alpha }^2{\left(\mathrm{m}-\mathrm{s}\right)}^2}\right)$$ (1)

where $\overrightarrow{f}$ is the total demons force, related to voxel intensity gradient (VIG: $\overrightarrow{\mathrm{\nabla }}$) and voxel intensity difference (VID: |m‐s|), and α is a normalization factor (0.2–0.7).

A pseudo force was introduced to allow the VIG of a voxel to affect the neighboring voxels within a Gaussian cube (gCube: 5 × 5 × 5 voxels) with a weight of the Gaussian distribution profile (σ = 1). The pseudo force ($p\overrightarrow{f}$) at a voxel (i,j,k∈ROI) was calculated by moving the gCube ($G_{n,m,l}$; n,m,l = −2,−1,0,1,2) and integrating the forces from the surrounding 124 neighboring voxels (n,m,l∈gCube). The pseudo force direction was determined by DVF directions (${\overrightarrow{T}}_{n,m,l}$) of the voxels (i,j,k), and its amplitude was determined by the VIG, VID, and Gaussian profile weight within the gCube (n,m,l). The pseudo force ($p\overrightarrow{f}$) at the point (i,j,k) and the enhanced force ($e\overrightarrow{f}$) are expressed as:

$$p\overrightarrow{f}(i,j,k)=w_{x,y,z}·{|m-s|}_{i,j,k}·\sum _{n,m,l}^{\mathit{gCube}}{\overrightarrow{T}}_{n,m,l}·G_{n,m,l}(i,j,k)$$ (2)

$$e\overrightarrow{f}=\overrightarrow{f}+\beta ·|\frac{{\overrightarrow{f}}_{max}}{p{\overrightarrow{f}}_{max}}|·p\overrightarrow{f}$$ (3)

where ${\overrightarrow{T}}_{n,m,l}$ denotes the accumulated force at the point (n,m,l), $w_{x,y,z}$ (z = SI = 1, y = AP = 0, and x = LR = 0) is the directional factor, and $\mathrm{\beta }·|\frac{{\overrightarrow{f}}_{\mathit{max}}}{p{\overrightarrow{f}}_{\mathit{max}}}|$ is the normalization factor of the p $\overrightarrow{f}$. In this study, $w_{x,y,z}=1$ and β = 1 were used for simplicity to have an isotropic weight and to balance the contributions from $\overrightarrow{f}$ and p $\overrightarrow{f}$, respectively. It is worthwhile to emphasize that the pseudo force strength received by a voxel is proportional to its own VID value, as indicated in Eq. (2).

2.C. Applying multiple DVFs in multi‐resolution DIR

When a separated DVF was applied at each of the multi‐resolution levels, a large deformation was separated into smaller deformation components, thereby reducing the amount of the deformation in each step. The final DVF was calculated from the individual DVF using vector operation. The pseudo demons force (Eqs. 2 and 3) was only applied in the initial low‐resolution fast‐matching alignment, whereas the original demons forces [Eq. (1)] was used in the remaining two levels of resolutions. Fig. 1(a) illustrates how the pseudo force field and the multi‐DVF were implemented using the stopping criteria of VID <10⁻⁴ in TR‐4DMRI image reconstruction. In detail, the following demons parameters were used in the multi‐resolution DIR:

1. Low resolution (8 × 8 × 8 mm³): $\overrightarrow{f}\left(+p\overrightarrow{f}\right)$ (fast matching); DVF₀(0→1);

2. Middle resolution (4 × 4 × 4 mm³): $\overrightarrow{f}$ onlzy (gross tuning); DVF₁(1→2);

3. High resolution (2 × 2 × 2 mm³): $\overrightarrow{f}$ only (fine tuning); DVF₂(2→3).

Figure 1.

Workflow for reconstruction of time‐resolved four‐dimensional magnetic resonance image (TR‐4DMRI) using enhanced demons DIR and cross‐consistency check (CCC). (a) Multi‐resolution DIR with the pseudo demons force and composite displacement vector field (DVF) to reconstruct TR‐4DMRI and (b) cross‐consistency check for reconstruction of TR‐4DMRI images by deforming two breath‐hold images at full inhalation (BHI) and full exhalation (BHE) to the same 3D cine images in free‐breathing (FB). The inverse consistency error (ICE) is calculated on three occasions. [Color figure can be viewed at wileyonlinelibrary.com]

The final DVF was calculated by the vector operation using the following formula:

$$DVF\left(0\to N\right)=\sum _{i=0}^{N-1}DV{F}_i\left(i\to i+1\right)$$ (4)

For a fair comparison, the same number of iterations (n = 15) was applied at each of the 3‐resolution levels of DIR, while only to the first low‐resolution level had different demons forces: the pseudo force and original force [Fig. 1(a)]. The intermediate images were saved, and the match of the diaphragm was evaluated with normalization to the full displacement (1.0) between the two images. The BHI and BHE images were used to reconstruct two 4DMRI for the comparison [Fig. 1(b)]. The voxel intensity correlation (VIC), defined as the correlation of two voxel intensity arrays (representing two images), was used to assess image alignment quality (0–1.0). Both visual VID image check and quantitative VIC measurements were evaluated.

2.D. Cross‐consistency check (CCC) of DIR

A cross‐consistency check (CCC) was introduced between two SR‐based TR‐4DMRI images reconstructed from two opposite BH images, denoted as BHI→FB←BHE [Fig. 1(b)]. A stopping criterion of ∆$\overline{\mathit{VID}}$<0.0001 was applied. If the DIR was accurate, identical TR‐4DMRI images deformed from BHI and BHE would be expected, namely,

$$DV{F}^{\mathit{BHI}}·BHI=DV{F}^{\mathit{BHE}}·BHE$$ (5)

where the two highR BH images were deformed to the lowR FB image from two opposite directions. To quantitate the similarity between two images, the CCC was defined as the ratio of the VIC of the two reconstructed images to the mean VIC of each reconstruction:

$$CCC=2\times \frac{VIC(DV{F}^{\mathit{BHI}}·BHI,DV{F}^{\mathit{BHE}}·BHE)}{VIC(DV{F}^{\mathit{BHI}}·BHI,FB)+VIC(DV{F}^{\mathit{BHE}}·BHE,FB)}$$ (6)

$$whereVIC=\frac{\mathrm{cov}(I_m,I_s)}{{\sigma }_{I_m}·{\sigma }_{I_s}}$$ (7)

The I_m and I_s denote the voxel intensity of the moving and static images, respectively, and the cov is the covariance and σ is the standard deviation. The CCC should be near unity because two deformed images should be similar, represented by similar VICs. Therefore, the CCC may serve as an indicator for the quality of the TR‐4DMRI reconstruction, whereas CCC <1.0‐ σ or CCC >1.0 + σ (possible outliers) implies a potential problem in one or both reconstructed images and therefore requires further examination. The σ is the standard deviation for the TR‐4DMRI image series. For the CCC evaluation with the use of pseudo demons force, a total of 24 DIRs was conducted and compared.

2.E. Inverse consistency error in CCC DIR approach

As shown in Fig. 1(b), the Inverse consistency error (ICE) was calculated in the following three deformation processes: two DIRs for 4DMRI reconstruction (BHI & FB and BHE & FB), and a DIR between the two reconstructed 4DMRI images ($DV{F}^{\mathit{BHI}}·BHI=DV{F}^{\mathit{BHE}}·BHE$). The operator “⇄” is for inverse DIR operation (→ forward DIR and ← backward DIR). If V=DVF ^F and U=DVF ^B represent the forward and backward DVF, respectively, then the ideal DIR should have |U ∘ V| = |V ∘ U| = 0. The ICE ^F and ICE ^B can be defined as^{34, 43}:

$$IC{E}^F=|U\circ V|andIC{E}^B=|V\circ U|$$ (8)

The ICE can be expressed as

$$ICE=\frac{IC{E}^F+IC{E}^B}{2}=\frac{|U\circ V|+|V\circ U|}{2}$$ (9)

As shown in Fig. 1(b), for each volunteer subject, 12 DIR were performed to compute the ICE values and a total of 72 DIR were performed and compared.

2.F. Verification of enhanced DIR using T1w‐mapped 4D XCAT phantom

A digital anthropomorphic phantom, 4D XCAT (version 2),^{11, 21, 44} was used to simulate 10 respiratory cycles, which was controlled by the motion curves of the diaphragm and chest wall in the superior‐inferior (SI) and anterior‐posterior (AP) directions, respectively. The intensity of the 4D XCAT phantom was first mapped to produce MRI‐like images, based on clinical T1w MRI. Ten 10‐phase 4D MRI images with SI motion range of 2–8 cm and AP motion range of 0.3–1.2 cm (at the skin) were simulated at a respiratory cycle of 5 s. The full exhalation and full inhalation images with a voxel size of 2 × 2 × 2 mm³ were used to simulate 3D cine MRI by down‐sampling them to 5 × 5 × 5 mm³. Similar DIR evaluations were performed as described below in human subjects. The achievable deformation magnitudes for the original demons force and enhanced demons force were evaluated using the 4D XCAT phantoms. A spherical tumor with a 3.0 cm diameter (Φ = 3.0 cm) and 5.0 cm motion range near the left diaphragm (motion range of 6.0 cm) was generated. The tumor alignment between the reconstructed TR‐4DMRI and the ground truth was evaluated based on the center of mass, volume and DICE similarity index.

2.G. Verification of enhanced DIR in six healthy volunteer study

Six healthy volunteers with both arms up were scanned using a FOV of the thorax and upper abdomen under an IRB‐approved protocol. Two BH images (BHI and BHE) were acquired at full inhalation and full exhalation. Using the image pairs of BHI and BHE of the six subjects, we down‐sampled the images to mimic the low‐resolution FB images and performed 96 DIRs between the BH and FB pairs at the specified conditions, including separate DIRs at each resolution level (x4, x2 and ×1 voxel sizes) with a fixed iteration (n = 15) with or without the pseudo force, and full DIR with a stopping criterion. The diaphragm dome was primarily used to quantitate the enhanced DIR performance and comparison with the original demons algorithm. By deforming both BHI and BHE to two selected extreme FB images, we reconstructed two TR‐4DMRI images for each of 24 cases and compared their similarity using the VIC and CCC.

3. Results

3.A. Enhanced demons force and its application in multi‐resolution optimization

A graphical representation of the voxel‐based magnitudes of the original demons force [Eq. (1)] and enhanced demons force [Eqs. (2) and (3)], together with the VID image between two MRI images are shown in Fig. 2. With the pseudo force ($p\overrightarrow{f}$), the enhanced force is boosted around high gradient voxels where high VIDs due to organ motion are present, such as the diaphragm and heart. The enhanced demons force boosts the initial coarse alignment at the diaphragm and heart in comparison to the conventional demons‐based DIR process.

Figure 2.

Graphical view of the enhanced demons force (intensity = strength) in two volunteer subjects: the original demons force (left), the difference image with voxel intensity difference (VID) (middle), and the enhanced force (right). The voxel with non‐zero force will be deformed in the direction of the voxel intensity gradient and a Gaussian filter is applied to the displacement vector field in each iteration to regulate the anatomy integrity throughout the optimization.

3.B. Evaluation of modified demons DIR using T1w‐mapped 4D XCAT digital phantom

Figure 3 illustrates the enhancement of the deformation range in 15 iterations at each resolution level using the pseudo demons force, compared with the original demons force. The pseudo force helps to achieve >60% deformation magnitude in 15 iterations, while original demons force can only achieve <20% when the deformation magnitude is >3 cm. The modified demons DIR with the pseudo force can deform beyond 6 cm deformation magnitude.

Figure 3.

Illustration of the enhancement of the deformation range of the modified demons with the pseudo force (blues) comparing to the original demons force (reds) in the digital T1‐mapped 4D XCAT phantom. The diaphragm domes were used for comparison (full alignment =1.0). 15 iterations were applied to each of the three‐level resolutions. The pseudo force was only applied in the first coarse alignment (blues). The results are the average of DIR from high‐resolution full inhalation (BHI) or full exhalation (BHE) to the low‐resolution images down‐sampled from the opposite BH images.

Figure 4 provides a graphical illustration of the enhanced deformation ability of the pseudo demons force on the T1w‐mapped XCAT phantoms, which succeeded in deforming more than 6 cm displacement at the diaphragm, whereas the original demons force failed at >3 cm displacement. The abdominal misalignment, primarily caused by missing tissue in the full inhalation moving images and the low‐contrast tissue, can be improved by local DIR refinement using the DIR stopping criteria.

Figure 4.

Demonstration of enhanced deformation ability at the diaphragm using the pseudo demons force vs original demons force in the first coarse alignment with 15 iterations. The diaphragm alignment is much improved using the pseudo force, compared with using the original force. The alignment of the low‐contrast tissue needs further local refinement at fine resolutions. Note that in the inferior image, the alignment from full exhalation to full inhalation was improved after more iterations with the stopping criteria. (a) 3cm and (b) 6cm displacements at the diaphragm. [Color figure can be viewed at wileyonlinelibrary.com]

The tumor alignment error in the deformed XCAT phantom with 5.0 cm tumor motion (Φ = 3 cm) and 6.0 cm diaphragm motion is 1.3 mm, the tumor volume ratio is 1.02, and the DICE similarity index is 0.84 in reference to the ground truth, suggesting an acceptable image fidelity. Further DIR refinement within the lung can further improve the DICE index to 0.88.

3.C. Comparison between DIR with original and enhanced demons forces in human subjects

Figure 5 demonstrates the results of multi‐resolution DIR with and without the pseudo force in the first low‐resolution level. The deformation enhancement at the diaphragm in level 1 by the pseudo demons force is 208 ± 126% (range: 25%–433%). This fast matching enhances the overall DIR performance, though the pseudo force is only applied in level 1. Five out of six subject images deformed more than half of the full diaphragm displacement in the first resolution level, except for subject 3 (~30% vs 10% or 20%). The enhance formation range observed in human subjects is consistent with the digital phantom study, as discussed above (3.B).

Figure 5.

Demonstration of the enhanced deformation range of deformable image registration (DIR) with the pseudo demons force between two extreme breath‐holds (BHI and BHE) in six volunteers. The full diaphragm displacement (28‐56 mm) is normalized to 1.0. The intermediate DIR results with 15 iterations at each resolution level are shown in red (with the original forces) and blue (with the pseudo force). The pseudo force was applied in the first coarse resolution achieving 70% alignment (dark blue) vs 25% alignment otherwise (dark red) on average. Nearly 100% alignment (light red) is achieved with initial pseudo force boost vs 65% alignment otherwise (light blue).

Figure 6 illustrates two visual CCC examples of TR‐4DMRI that resulted from opposite BH images. The diaphragm alignments are almost identical, visually confirming their validity with only mild differences in the VID images. Table 1 tabulates VIC and CCC values among original and deformed BHI and BHE with FB images. It is worthwhile to point out that the CCC value is quite sensitive to mild misalignment as shown in volunteer 3, for which the diaphragm alignment was not complete. This is a patient image specific and Fig. 5 illustrates a similar result. The accuracy of the high‐contrast organ is high, including the diaphragm, heart, and stomach, as shown in Fig. 6. At the diaphragm dome, the accuracy of the alignment is 0.8 ± 1.6 mm.

Figure 6.

Visual representation of cross‐consistency check (CCC) of the time‐resolved four‐dimensional magnetic resonance image (TR‐4DMRI) of two volunteers (A: v1 and B: v2) using two opposite breath‐hold images at full exhalation (BHE) and full inhalation (BHI). Almost identical diaphragm alignment is achieved between the two deformed images. In (b), the BHE is very close to the full exhalation FB, illustrating that the residual image intensity difference is primarily due to different image resolutions. [Color figure can be viewed at wileyonlinelibrary.com]

Table 1.

Cross‐consistency check (CCC) based on voxel intensity correlation (VIC) in the reconstruction of time‐resolved four‐dimensional magnetic resonance image (TR‐4DMRI) at two extreme free‐breathing (FB) phases near the ends of inhalation (BHI) and expiration (BHE). The initial and final displacements (d) of the diaphragm from breath‐hold (BH) to the FB are shown

Subject	FB phasea	BHI					BHE					Cross consistencyc
		d (mm)		VIC		ICE (mm)b	d (mm)		VIC		ICE (mm)	between 2 TR‐4DMRI
		Pre	Post	Pre	Post		Pre	Post	Pre	Post		VIC	CCC	ICE (mm)
1	E	58	0	0.54	0.86	1.66	2	2	0.79	0.89	1.52	0.88	1.01	0.03
	I	24	0	0.65	0.90	1.77	28	4	0.71	0.88	2.74	0.90	1.02	0.09
2	E	42	0	0.65	0.93	1.46	8	0	0.80	0.94	1.29	0.94	1.01	0.26
	I	20	2	0.73	0.94	1.73	18	2	0.82	0.93	1.01	0.95	1.01	0.29
3	E	66	0	0.31	0.80	2.43	16	0	0.73	0.87	1.60	0.80	0.95	0.25
	I	40	0	0.56	0.87	1.90	18	2	0.55	0.87	1.32	0.86	0.99	0.16
4	E	30	6	0.60	0.91	2.07	8	0	0.72	0.93	1.33	0.95	1.03	0.29
	I	28	2	0.69	0.93	1.40	10	0	0.74	0.93	1.88	0.96	1.03	0.14
5	E	40	0	0.62	0.88	0.98	8	0	0.83	0.90	0.89	0.91	1.02	0.08
	I	34	0	0.66	0.89	1.90	4	0	0.83	0.90	2.58	0.92	1.02	0.12
6	E	48	0	0.62	0.90	0.69	2	0	0.79	0.91	0.66	0.91	1.00	0.17
	I	32	0	0.68	0.91	1.49	14	0	0.74	0.91	1.05	0.92	1.01	0.18
Average		39	0.8	0.61	0.90	1.6	11	0.8	0.75	0.91	1.5	0.91	1.01	0.17
St Dev		14	1.8		0.04	0.5	8	1.3		0.02	0.6	0.04	0.02	0.08

^aTwo extreme cases from an FB series: exhalation (E) and inhalation (I).

^bICE—inverse consistent error [Eq. (9)].

^cThe CCC is defined in Eq. (6). If a DIR result has CCC<1‐SD or CCC>1 + SD, its quality should be visually checked. VIC is defined in Eq. (7).

4. Discussion

4.A. The pseudo demons force for enhanced DIR deformation range

The pseudo demons force provides a substantial boost to the gradient‐based force field in the fast‐marching alignment. The initial low‐resolution DIR alignment allows the pseudo force to exert a “long distance” impact because of larger voxels. It extends the influence of the image gradient at a voxel to the 124 neighboring voxels with a weighting factor following Gaussian distribution profile. This helps to deform voxels around a voxel with high VIG and VID, similar to the role of the Gaussian filter to the DVF, but up‐front. This pseudo force within a Gaussian cube can increase the mean deformation up‐front with reasonable anatomic integrity, unlike the Gaussian filter applied at the end of each iteration, which pulls back the highly deformed voxels to regulate the anatomic integrity by Gaussian smoothing. Therefore, the pseudo force accelerates the deformation and enhances the deformable range.

However, the pseudo force should only be applied in the initial coarse alignment within the region of interest to avoid excess deformation, causing distortion artifacts. The benefit of using the pseudo force in low‐resolution DIR is that it will reduce computation burden and maintain the “long‐distance” impact. In the high‐resolution DIR, the original demons force provides additional fine‐tuning to complete the DIR. In addition, using a separate DVF at each resolution level and combining multi‐DVF for the final deformation enhances the deformation range, almost equivalent to multi‐DIR processes. As the final DVF is computed prior to image transformation by vector operation using Eq. (4), it avoids repeating image interpolation and thus preserves the image resolution.

In practice, the BHI images may appear outside of the FB range, as a subject tends to inhale more for a breath‐hold, while the BHE is likely within FB range. A 6.6 cm deformation magnitude was observed in volunteer 3 (Table 1). Using the digital phantom, we have demonstrated that the application of the pseudo force deforms >60% magnitude within 15 iterations (Fig. 3). The modified demons DIR achieved accurate alignment on the diaphragm, heart, and stomach with high‐contrast (Figs. 4 and 6, and Table 1). Therefore, the enhanced deformation range makes the reconstruction of TR‐4DMRI a robust process, although in practice, we would try to acquire a BH in the middle of FB motion range. This is the first time to an image has been successfully deformed on such a large scale using the enhanced demons algorithm.

4.B. Cross‐consistency check for TR‐4DMRI reconstruction

To implement the SR‐based TR‐4DMRI technique for a clinical application requires not only a robust DIR method but also an effective indicator of the DIR quality are necessary. The CCC index [Eq. (6)] establishes a self‐consistency check of two independent results so that the results can be automatically cross‐validated by deforming two BH images from opposite directions. The results (shown in Fig. 6 and Table 1) suggest that the CCC index is effective in catching DIR misalignment. In the twelve cases in Table 1, the CCC caught four possible incomplete DIR cases and two were confirmed and then improved with refinement by tightening the stopping criteria. Note that in free‐breathing, the motion range is usually less than 4 cm, therefore, the success rate of reconstruction should be higher. As a comparison, the ICE index could be unreliable if the deformable range is limited. For instance, the ICE may grossly underestimate the error when a DIR deforms only half way forward but can fully deform backward. In fact, the ICE cannot distinguish the results using the original demons and enhanced demons. On the contrary, the CCC is designed to cross‐verify two independent deformations resulting from two opposite directions (BHI and BHE): the probability of a coincidental alignment is small and can be ignored.

In this study, three scenarios of the ICE assessment were also evaluated, as shown in Fig. 1(b) and Table 1. Between the two BHs and FB images, the ICE is about the same (1.6 ± 0.6 mm), whereas it is reduced to 0.17 ± 0.08 mm for the two reconstructed 4DMRI images, about 10 times smaller. This is because the two resultant 4DMRI images are so similar that little deformation is needed, resulting in a small ICE value. This indicates that ICE is dependent on both the DIR method and the initial deformation magnitude. By comparison, the CCC index is independent of deformation magnitude and therefore a useful cross‐verification index of DIR quality.

4.C. Limitation of the pseudo demons force and future direction

Currently, an isotropic Gaussian cube is applied in the pseudo force implementation. Given the knowledge that the respiratory motion is mostly occurring in the superior‐inferior (SI) direction, an anisotropic Gaussian cube with a different Gaussian distribution profile could be used to bias the deformation in the SI direction. Therefore, an up‐front, directional, and long‐distance impact can be established, ensuring a continuous initial deformation. In addition, the weight factor (β in Eq. (3)) could be used to alter the contribution from the pseudo force; when β = 0, Eq. (3) contains only the original demons force. This is an ongoing study for further improvement of the demons DIR algorithm.

Although the pseudo force enhanced the alignment at the diaphragm, which has the highest tissue contrast and largest gradient at the interface, it may not be useful to enhance the alignment of low‐contrast tissues. Therefore, the improvement of low‐contrast tissue alignment is still a challenge and it is currently under investigation. Combining the efforts of both pseudo and original demons forces, the image quality of reconstructed TR‐4DMRI can be reliably ensured. In fact, compared with CT images, T1w MR images provide superior soft‐tissue contrast other than the lungs, facilitating the DIR‐based reconstruction and verification.

Although this study provides an effective approach to increase DIR deformable range beyond 6 cm, in practice we would recommend minimizing the deformation between EOE and EOI using audio or visual breath coaching for BH acquisition. Therefore, two BHI and BHE images within the FB breathing motion range can be acquired, facilitating TR‐4DMRI reconstruction. In general, the smaller the initial deformation magnitude, the less uncertainty would be.

Last, we are recruiting patients for a TR‐4DMRI study under IRB‐approved clinical trials, so that we can evaluate the tumor motion, together with the motions of its surrounding organs. The enhanced deformation range using the pseudo demons force can assist to ensure the robustness of this SR‐based TR‐4DMRI technique to produce high‐quality multi‐breath images for motion‐compensated radiotherapy treatment planning.

5. Conclusion

The introduction of the pseudo demons force in the coarse DIR alignment, together with the multi‐DVF approach, enables the demons algorithm to increase the deformation range (up to 6 cm) for the reconstruction of super‐resolution TR‐4DMRI. In both a digital phantom and six human subjects, the enhancement of the deformation range of the demons DIR is demonstrated at the high‐contrast tissues, while the improvement of low‐contrast tissue is under an ongoing investigation. The cross‐consistency check has the potential to serve as the quality assurance indicator of the TR‐4DMRI reconstruction for clinical applications.

Conflict of interest

Memorial Sloan Kettering Cancer Center has a master research agreement with Philips Healthcare. Part of this work was presented in the AAPM'2017 annual meeting.