Automatic assessment of average diaphragm motion trajectory from 4DCT images through machine learning

Abstract

To automatically estimate average diaphragm motion trajectory (ADMT) based on four-dimensional computed tomography (4DCT), facilitating clinical assessment of respiratory motion and motion variation and retrospective motion study. We have developed an effective motion extraction approach and a machine-learning-based algorithm to estimate the ADMT. Eleven patients with 22 sets of 4DCT images (4DCT1 at simulation and 4DCT2 at treatment) were studied. After automatically segmenting the lungs, the differential volume-per-slice (dVPS) curves of the left and right lungs were calculated as a function of slice number for each phase with respective to the full-exhalation. After 5-slice moving average was performed, the discrete cosine transform (DCT) was applied to analyze the dVPS curves in frequency domain. The dimensionality of the spectrum data was reduced by using several lowest frequency coefficients (f_v) to account for most of the spectrum energy (Σf_v²). Multiple linear regression (MLR) method was then applied to determine the weights of these frequencies by fitting the ground truth—the measured ADMT, which are represented by three pivot points of the diaphragm on each side. The ‘leave-one-out’ cross validation method was employed to analyze the statistical performance of the prediction results in three image sets: 4DCT1, 4DCT2, and 4DCT1 + 4DCT2. Seven lowest frequencies in DCT domain were found to be sufficient to approximate the patient dVPS curves (R = 91%−96% in MLR fitting). The mean error in the predicted ADMT using leave-one-out method was 0.3 ± 1.9 mm for the left-side diaphragm and 0.0 ± 1.4 mm for the right-side diaphragm. The prediction error is lower in 4DCT2 than 4DCT1, and is the lowest in 4DCT1 and 4DCT2 combined. This frequency-analysis-based machine learning technique was employed to predict the ADMT automatically with an acceptable error (0.2 ± 1.6 mm). This volumetric approach is not affected by the presence of the lung tumors, providing an automatic robust tool to evaluate diaphragm motion.

Introduction

Personalized tumor-motion assessment has been widely implemented in the radiotherapy clinic using respiration-correlated four-dimensional computed tomography (4DCT) (Keall et al 2006, Li et al 2012). Patient-specific motion can be taken into account to apply a suitable motion management method in treatment simulation, planning and delivery. A widely applied approach is to define internal tumor volume (ITV) based on the union of clinical tumor volume (CTV) in all phase CT images (Ehler et al 2009, Kang et al 2010, van Dam et al 2010) or the overlaid CTV in the maximum intensity projection (MIP) image (Underberg et al 2005, Muirhead et al 2008, Ehler et al, 2009). The CTV is often formed by adding a margin of 0–8 mm to the gross tumor volume. To spare more normal tissue or to achieve a prescribed radiation dose without violating clinical dose constraints to nearby organs at risk, the ITV can be reduced by using a compression belt to inhibit the diaphragm motion (Lee et al 2004, Lovelock et al 2014), by respiratory gating to irradiate the tumor within the 30%–70% respiratory phase (Saw et al 2007, Nelson et al 2010), or by tracking the tumor motion in real time to achieve the most conformal dose delivery.

The diaphragm is the primary muscle responsible for respiratory motion and its movement is often used as an internal surrogate for respiration-induced tumor motion in the lung, liver and pancreas. In fluoroscopic imaging, the diaphragmatic dome is visible due to the large difference in tissue density at the diaphragm–lung interface. High correlations (0.94–0.98 and 0.98 ± 0.02) have been reported between the diaphragm and tumor motion in lung (Cervino et al 2009) and liver patients (Yang et al 2014). Reports have shown that diaphragm motion can be used as a surrogate for tumor motion without implanted fiducials (Li et al 2009c, Lin et al 2009, Dhou et al 2015). In cine megavoltage electronic portal imaging during beam-on time, initial study has shown the feasibility of extracting volumetric treatment images based on 4DCT-based motion modeling (Mishra et al 2014). In cone-beam CT (CBCT) imaging, projection images can be utilized by combining deformable image registration and principal component analysis (PCA) to estimate the tumor position with the diaphragm as the major anatomic landmark (Zhang et al 2007, Li et al 2010a, 2010b, Li et al 2011). In other CBCT studies, an automatic method was developed to detect the diaphragm motion (Siochi 2009, Chen and Siochi 2010, Dhou et al 2015). In 4DCT reconstruction, the diaphragm can be used as an internal surrogate for respiratory binning. In respiratory motion modeling, the mean diaphragm position can be accurately estimated from the lung volume change within the rib cage (Li et al 2009a, 2009b). Both the diaphragm and carina have been used as internal anatomic landmarks to predict lung tumor motion (Spoelstra et al 2012). Therefore, establishing the average diaphragm motion trajectory (ADMT), which approximates the volumetric-equivalent piston position within the rib cage (Li et al 2009b), is a useful step forward to predict tumor motion. In particular, this method could be useful in the clinic for estimating the motion of lesions located near the diaphragm, such as inferior lung lesions or superior liver lesions.

Machine learning, the use of mathematical and statistical algorithms to extract knowledge efficiently and adaptively from large-scale data, is the enabling arsenal behind many successes in the ‘big data’ era (Murphy 2012, Wang and Summers 2012). It has been applied to radiation oncology in recent years for treatment assessment (El Naqa et al 2009, Spencer et al 2009, Naqa et al 2010), treatment planning (Zhang et al 2009) and tumor motion prediction (Ruan and Keall 2010). In order to effectively extract useful information, it is essential to have an appropriate data collection, effective data representation, and automatic data processing tools.

Dimensionality reduction, one of the most important unsupervised learning methods, can remove redundant and trivial data, promote data visualization, and resolve over-fitting problems. One widely-applied methodology is the PCA to encode the respiratory motion, where the space spanned by the leading eigenvectors is employed to represent the original data. In respiratory motion studies, PCA has been utilized to simplify the problem set in various approaches (Yan et al 2013, Zhang et al 2013, Mishra et al 2014, Wilms et al 2014, Dhou et al 2015). Another methodology is manifold learning: projecting a manifold in higher dimensional space to a lower dimensional space while preserving the local neighborhood (Wachinger et al 2012, Usman et al 2013).

In this work, machine learning techniques such as Fourier transform based dimensionality reduction (unsupervised learning), multiple linear regression (MLR) (supervised learning), and ‘leave-one-out’ evaluation approach (cross-validation) have been used to successfully estimate ADMT. This was done by generating differential volume per slice (dVPS) of the lung, followed by a dimensionality-reduction procedure based on discrete cosine transform (DCT) to use low-frequency components and approximate the measured dVPS curves. The dimension reduction achieved by using low order Fourier coefficients to represent R² > 80% of the innate ‘periodic’ respiratory motion is an actual manifold learning approach. The ground truth of the mean diaphragm motion was then measured from 4DCT to train a prediction model through MLR (Hastie et al 2011). After the set of weighting factors in the MLR model was determined, the diaphragm motion of each individual patient was predicted by plugging patient-specific dVPS frequencies to the estimated regression model. The leave-one-out cross validation method was utilized to evaluate the accuracy of motion predictions.

Methods and materials

Patient 4DCT image data and prediction verification/test schemes

In this study, 22 4DCT images from 11 lung cancer patients were acquired with 2 scans for each patient: one was acquired at simulation (4DCT1) and the other, during the second week of treatment (4DCT2). The clinical study, approved by Institution Review Board, was conducted between January 2011 and August 2012. Amplitude-based binning with motion modeling (Hertanto et al 2012) was used for 4DCT image reconstruction with voxel size of 0.97 × 0.97 × 2.50 mm³. Three groups of 4DCT images were used in the following manner: (1) 4DCT1 group (M = 11), (2) 4DCT2 group (M = 11), and (3) 4DCT1 and 4DCT2 combined group (M = 22). In each group, the leave-one-out cross validation procedure was performed to predict the motion of every 4DCT image using the MLR model trained from the other M-1 images. In each prediction, the left and right hemispheres of the diaphragm were analyzed separately, and two series of predictions were performed for each case.

Preparation of dVPS from 4DCT

The dVPS was defined as the difference of lung volume per slice of any respiratory phase with reference to the full exhalation phase. Between two phases of respiration, dVPS shows the lung volume change per slice: chest wall motion results in anterior–posterior (AP) volume change from the lung apex to the reference diaphragm while diaphragm motion results in superior–inferior (SI) volume change primarily from the reference to current diaphragm position. Figure 1 shows a simplified lung-model case and a patient case, illustrating that diaphragm motion information is contained in dVPS. Therefore, the motion information of the diaphragm is enriched in the dVPS curves and ready to be extracted and quantified.

Figure 1.

Demonstration of diaphragm position information in the differential volume per slice (dVPS): (A) a model lung case with three phases (FE, 1 and 2) and (B) a patient lung case (#2). The superior and inferior edges of the ‘signature peak’ in dVPS reflect the diaphragm positions in the reference (FE: full-exhalation) and current phase, respectively. In both cases, the superior edges of the peak are aligned while the inferior edges are different due to different motions in different dVPS.

The dVPS curves for the left and right lung were calculated separately based on patients’ 4DCT. An automatic tool in 4D clinical multimodal imaging processing (4DCMIP) system (Li et al 2014, Wei et al 2014, Wei and Li 2015) was utilized to generate a pair of dVPS curves for each patient in four steps: (1) automatic segmenting the left and right lungs, separated by the medial plane, (2) calculating the lung volume per slice for each respiratory phase, (3) calculating the differential volume, or dVPS, using full exhalation as the reference phase, and (4) smoothing the dVPS using 5-slice moving average. The dVPS curves, which contain a large ‘signature’ peak around the diaphragm due to different diaphragm positions at different respiratory phases, were used as patient-specific inputs for diaphragm motion prediction. To procure the ground truths for the ADMT, three pivot points on the diaphragm were averaged to represent the mean position of the diaphragm (z₀) on each lateral side. The 3 points were the most inferior points at the anterior and posterior sides and the most-superior point at the diaphragm dome. The average diaphragm positions approximate the volume-equivalent position (Li et al 2009b).

DCT of dVPS and dimension reduction for machine learning

To mathematically express the dVPS curve (V) as a function of slice number, the DCT, which is the Fourier transform of even functions, was applied. The complete V vector may contain a very large number of elements, such as 100 (n = 100) in the frequency domain; the high dimensionality produces over-fitting problems. Consequently, as discussed in the Introduction, dimensionality reduction is necessitated before rigorous supervised, semi-supervised or unsupervised learning techniques can be effectively conducted (Bishop 2007). Only low-frequency components in the DCT domain were used to approximate the dVPS vector with the major information preserved.

Vector V was first transformed to a zero-mean vector V = V − mean (V), prior to DCT:

$$F_V=\mathrm{DCT}(V).$$ (1)

So F_v(j = 1), the dc coefficient, is 0 because the mean value of V is nullified. Parseval’s theorem dictates the following equality of energy (E):

$$E=\sum _{j=1}^n{F}_V^2(j)=\sum _{i=1}^n{V}^2(i).$$ (2)

Due to the presence of periodicity of breathing patterns, spatial and temporal redundancies (intensity similarity for spatially and temporally adjacent voxels) and high-frequency noises in dVPS, the leading ac coefficients, i.e., F_V (j) for j = 2 to k + 1, where k (≪ n) is the new lower dimensionality, carrying most energy of F_V and thus most energy of V according to equation (2) (Empirical evidence is presented in figure 3 of the next section). Consequently, instead of processing the much higher n-dimensional vector V, one can use R_k = (F_V (2), F_V (3), …, F_V(k + 1)) to represent V in the frequency domain without compromising the information carried by V. The reduced dimension k can be adaptively determined according to the following criterion:

$$\sqrt{E({\mathbf{R}}_k)/E}>90\%,\mathrm{where}E({\mathbf{R}}_k)=\sum _{i=1}^k{\left|{\mathbf{R}}_k(i)\right|}^2.$$ (3)

Figure 3.

The differential volume-per-slice (dVPS) curves of the left (red, dashed line) and right lung (green, solid line) between full inhalation (FI) and full exhalation (FE). The data are averaged with 5-slice moving average. The three pivot points on the diaphragm are also flagged in the chart with slice number. The top dVPS (Patient 2) is common for all patients, except for Patient 1, which is shown in the bottom dVPS. The bottom chart is an outlier showing two uncommon features on the left side: (1) heart interference and (2) negative dVPS (patient 1).

MLR for motion modeling

We hypothesized that the displacement value z₀ is a linear function of dVPS vector V:

$$z_0=L(V),$$ (4)

where L(·) is a certain linear function. Taking the DCT transform of both sides, and noticing that DCT (z₀) = z₀ and DCT is a linear transform, we thus have the following equation:

$$z_0=\mathit{DCT}\left[L(V)\right]=w_0+\sum _{i=1}^n\left[w_i·F_v(i)\right],$$ (5)

where n is the dimensionality of dVPS vector V.

The value n in equation (5) is in the order of 100 with the same number of corresponding Fourier coefficients w_i. To effectively estimate these high dimensional parameters, a training data size in the order of hundreds is needed. Owing to limited training sets and noises in the training data, overfitting in determining w_i’s would likely to occur. Therefore dimensionality reduction is necessitated to overcome this problem. As summarized in equation (3), the original large number n can be significantly reduced to k (k ≪ n). This methodology is in fact a manifold learning approach, one of the two leading methodologies in dimensionality reduction. To yield real time performance, instead of using existing manifold methods such as Isomap, local linear embedding (LLE) and Laplacian eigen map (Wachinger et al 2012), by taking advantage of the energy loading property in the DCT domain, we applied the MLR to estimate the optimal weights to generate motion data from low ac coefficients. In consequence, only the lowered k-D space w = (w₀, w₁, …, w_k) is needed. The MLR formula for w is as below:

$$\mathit{w}={(R^{+}z)}^T,$$ (6)

where $R=\left(\left(1,{\mathbf{R}}_k^1\right);\left(1,{\mathbf{R}}_k^2\right);\dots ;\left(1,{\mathbf{R}}_k^N\right)\right)$, and R⁺ the pseudo-inverse. Mathematical derivations of equation (6) can be found in the appendix.

In summary, the number k of lowest AC coefficients in the DCT domain and their frequency terms (R_k) were dynamically identified from the training 4DCT dVPS data sets based on equation (3). The regression weight vector w was estimated from the measured diaphragm positions using MLR method equation (6). The new z₀ is predicted from w from the measured R_k, as shown in figure 2(a). The machine learning algorithm was implemented in MATLAB and integrated to our in-house 4DCMIP system (Li et al 2014, Wei et al 2014, Wei and Li 2015). A screen shot of the system to evaluate the number of DCT components for dimensionality reduction is depicted in figure 2(b).

Figure 2.

(a) Workflow of the prediction of average diaphragm motion trajectory (ADMT). (b) The graphical user interface (GUI) of our MATLAB-based 4D clinical multimodal image processing (4DCMIP) system corresponding to the prediction of ADMT.

Results

Dimensionality reduction using DCT

Figure 3 shows two examples of the dVPS curves: one typical and one outlier. The outlier differs by possessing an extra peak superior to the diaphragmatic peak and a negative volume value at the most superior side in the dVPS curve, given the current 4DCT training pool. Having the two features together is unique and cannot be found in the other 21 dVPS curves. Using DCT method, the original dVPS curves can be approximated by a DCT function with different numbers of low-frequency components. Figure 4 shows the plot of R² of DCT (dVPS) based regression versus the number of low-frequency components. A transition point of the plots occurs below 7 components and any further increase of the number of low-frequency components only provides marginal improvement to the regression-based fitting (R²). Therefore, we set R = 0.9 (R² = 0.81) as the threshold (figure 4) and chose the seven lowest-frequency components to approximate the dVPS function, with R-values ranging from 0.91 to 0.96.

Figure 4.

Fitting of dVPS (differential volume-per-slice curves) using DCT (discrete cosine transformation) with different number of ac components. The seven largest Fourier coefficients represent the original dVPS curve with R² > 0.81 (or R > 90%).

Using DCT-7 function to predict ADMT

After the 7 lowest frequency components of the dVPS were determined by DCT analysis, MLR learning from the ground truth of the ADMT was proceeded to determine the 8 weight vectors (w_i, i = 0, 1, 2…7). Figure 5 shows the results of trajectory fitting for both left and right sides of the diaphragm. The quality of MLR-based fitting between the measured and estimated values is quantified by R². When comparing the weight vectors, it is interesting to note that the outlier has a distinguished vector set (w_i, i = 0, 1, 2…7).

Figure 5.

The multiple linear regression (MLR) fitting of the measured diaphragm motion (z₀) using a DCT function with 7 major Fourier components. The R² is given for each fitting graph greater than 0.81. The number of patient-phase (x-axis) is the accumulative phase number among 11 patients (each contains 9 phases, excluding the reference phase at full exhalation with zero motion). The y-axis is in unit of slice (slice thickness = 2.5 mm). Some ADMT variations can be seen between 4DCT1 (A), (B) and 4DCT2 (C), (D), and the left (A), (C) and right (B), (D) behave differently.

Learning and predicting diaphragm motion

Table 1 shows the errors of motion-trajectory prediction within the 4DCT1 or 4DCT2 image groups, and figure 6 shows the error distribution. Generally, the error is smaller with narrower distribution for the right, compared to the left side, and for 4DCT2 compared to 4DCT1. For all predictions, when M-1 4DCT images are used to predict the motion trajectory in the excluded image, the average error is −0.1 ± 1.9 mm. Within a single breathing cycle, the error frequently occurs around extreme cases such as full inhalation and full exhalation (figure 5). When the patient pool is increased by combining 4DCT1 and 4DCT2, the prediction is slightly improved, as shown in table 2 and figure 7. Tables 1 and 2 also contain two test results using the 11-patient training images and the results are in line with those from the leave-one-out verification.

Table 1.

The error of automatic prediction of absolute diaphragm position in 4DCT images.

Patient	Motion error, 4DCT1 (mm)				Motion error, 4DCT2 (mm)
	Left		Right		Left		Right
	Mean	STD	Mean	STD	Mean	STD	Mean	STD
1	−9.7a	3.4	−1.8	1.4	0.4	1.6	−0.1	0.6
2	−0.4	1.0	−1.6	0.9	−0.6	1.3	−1.1	1.3
3	0.7	1.4	1.5	0.8	0.6	1.6	−1.7	1.0
4	4.0	1.8	−1.6	0.8	0.7	1.3	−0.9	0.9
5	−2.2	0.7	0.8	1.5	−4.1	1.1	0.7	0.9
6	0.5	1.2	0.3	1.0	1.2	1.1	1.1	1.1
7	2.3	2.7	2.0	2.0	2.7	3.1	1.5	0.6
8	−0.9	1.3	−0.3	0.7	−0.3	2.6	0.0	1.8
9	2.5	2.4	2.1	2.5	−0.7	1.0	2.3	3.1
10	−0.6	0.8	1.3	0.7	2.9	0.8	0.6	1.2
11	1.3	1.8	−5.8	5.4	−1.3	2.2	−2.9	2.0
Mean	−0.2	1.7	−0.3	1.6	0.1	1.6	0.0	1.3
STD	3.6		2.3		1.9		1.5

^aThis outlier is due to interference from the heart, which produces a second dVPS peak near the diaphragmatic peak. In addition, the breathing phase on the chest wall for this scan has unusual behavior, with negative elevation during inhalation process; the other 4DCT did not have similar

Figure 6.

The distribution of prediction errors for left and right hemi-diaphragms among 11 patients. The 10-to-1 predictions for the left (A) and the right (B) using 4DCT1 images, and the 10-to-1 predictions for the left (C) and right (D) using 4DCT2 images.

Table 2.

The prediction results based on all 4DCT images of the 11 patients using leave-one-out cross verification.

Patient-Image	Motion error, 4DCT 1 and 2 (mm)
	Left		Right
	Mean	STD	Mean	STD
1-1	−9.3a	3.3	0.1	0.6
2-1	−0.9	0.3	−1.6	0.8
3-1	1.0	1.2	1.4	0.8
4-1	3.8	1.8	−1.4	0.7
5-1	−2.1	0.6	1.7	1.8
6-1	−2.4	1.9	0.3	0.9
7-1	0.4	1.3	−1.2	0.6
8-1	−1.1	1.0	0.1	1.0
9-1	1.9	1.8	1.9	1.8
10-1	-0.6	0.9	1.4	0.6
11-1	−0.4	2.0	−1.9	2.5
1-2	0.2	2.2	−0.8	0.4
2-2	−0.2	1.4	−0.9	0.9
3-2	−0.6	1.4	−2.1	1.1
4-2	1.7	1.6	−1.6	1.0
5-2	−3.5	0.8	−0.8	0.9
6-2	3.3	2.3	1.1	1.4
7-2	1.7	3.0	3.0	1.2
8-2	−0.6	2.3	−0.4	1.3
9-2	−0.4	1.9	1.9	3.4
10-2	2.6	1.0	0.4	1.3
11-2	2.2	3.0	−0.8	1.2
Mean	−0.1	1.7	0.0	1.2
STD	2.8		1.4

^aThis outlier is due to the interference from the heart, which produces a second dVPS peak near the diaphragmatic peak. When the outlier was excluded, the mean error is 0.33 ± 31.9 mm.

Figure 7.

The distribution of prediction errors for left and right hemi-diaphragm among 22 patients. The 10-to-1 predictions in the left (A) and right (B) combining the results from 4DCT1 and 4DCT2, and the 21-to-1 predictions in the left (C) and right (D) using 4DCT1 and 4DCT2 images.

Prediction interference from the presence of tumor

Among the 11 lung cancer patients studied, the presence of tumor(s) does not seem to affect the ADMT prediction. As an extreme case, figure 8 shows the image for patient 7 with 12 large tumors with various sizes, locations and motions. The prediction error is relatively high, due to the interference of the tumor around the diaphragm.

Figure 8.

An extreme case (patient 7) with the presence of multiple large lung tumors.

Discussion

Machine learning and prediction

In this study, machine learning techniques were employed to assess motion trajectories of the diaphragm in lung cancer patients. The DCT-based expression and dimensionality reduction was used to approximate the original hyper-dimensional dVPS curves, which were extracted from patient-specific 4DCT images automatically using our 4DCMIP system. By exploiting the spatial and temporal redundancies in 4DCT images, the seven lowest-frequency components in DCT domain were found to be sufficient to approximate the original dVPS function at high fidelity (R > 90%). From this low dimensional space, the weights of the MLR model were estimated using the training image set.

It is also noteworthy that the eight weights (from 0 to 7) are similar on ipsilateral side, while different between the left and right side of the diaphragm. The difference in the movements in the two lateral sides was also observed visually. This suggests that although the diaphragm is a single anatomical entity, the left and right hemispheres can move differently. The anatomical connection between the diaphragm and other organs, such as the esophagus and heart, may play a role for the different motion behaviors between the left and right sides of the diaphragm.

The DCT provides low-pass filtering for dimension reduction, while very high frequencies are usually due to noises in the dVPS vector and thus dropped. The efficacy demonstrated in our study is probably because the spatial and temporal redundancies for most signals are present ubiquitously: the data adjacent in the spatial and temporal neighborhoods are similar to each other. Therefore, using the 90% cut-off criterion is a reasonable approximation based on the signal compression standard (Bishop 2007). There is another rigorous Karhumen–Loeve transform, also known as PCA, which is more computationally intensive, on the order of O(n³). The DCT based on fast Fourier transform (FFT) is more efficient, in the order of O(n log n), where n is the dimensionality of the dVPS vector. Intensive studies have concluded that for most signals, the bases learned by PCA are almost identical to the DCT bases (Bishop 2007), which justifies the extensive use of DCT in medical imaging and signal, image and video processing. Real time performance is made possible by use of the FFT.

The PCA-based dimensionality reduction has been successfully applied in manifold learning in motion modeling together with deformable image registration using 4DCT or 4DCBCT (Zhang et al 2007, Li et al 2009c, Zhang et al 2010, Dhou et al 2015). A lung motion model can be established by using 2 to 3 eigenvectors to represent the largest modes of DVF variation. The dimensionality reduction method used in this study, however, is to approximate the original dVPS with R > 90% to quantify dVPS and facilitate ADMT prediction with machine learning. Dhou et al (2015) has shown using 4D CBCT prior to treatment should be helpful to minimize motion variation between simulation and treatment. The ADMT estimation is based on simulation 4DCT. Although this study does not directly address this issue, we are investigating to update ADMT at treatment using optical surface imaging (Li et al 2015) based on the expandable piston respiratory model (Li et al 2009b) that predicts the ADMT by separating lung extension (SI) and expansion (AP) motions.

Predicting the ADMT

The ground truth of diaphragm position and motion is established by calculating the average coordinates of three points on each side of the diaphragm. After all the respiratory phases are analyzed, the ADMT is obtained. This method is based on previous work, where six pivot points were averaged to represent the volumetrically-equivalent diaphragm position (Li et al 2009b). In this study, the ground truth is used for training the predictive model and making subsequent prediction. As shown in figure 3, the 3 pivot points at the anterior, posterior and dome of the diaphragm, spread around the peak in the dVPS form. The mean SI coordinate of these points is around the diaphragmatic peak.

The quality of this machine-learning-based method is generally higher for right hemi-sphere of the diaphragm, as shown in figures 5–7. This could be the result of interference from the heart on the left side, which makes it difficult to find the most-inferior point on the anterior side. We also observed that the quality for the second 4DCT set acquired two weeks later at treatment is better than that for the first 4DCT set acquired at simulation. This may be due to different breathing behaviors of the patients at different times: patients tend to be more relaxed and able to breathe more naturally after being through some of treatment; on the contrary, at simulation, patients tend to be nervous, which may cause a more unnatural muscle action, altering their breathing behavior from the norm. The outlier (patient 1, left) is due to the opposite motion in the chest wall (full-exhalation with higher chest volume) and the interference from the heart.

The presence of lung tumors, as shown in figure 8, does not affect the prediction results substantially. The lung images of patient 6 and 7 show multiple large tumors that occupy up to a third of the total lung volume on one side or both. For small lung tumors, their volumes are negligible, so they cause little alteration to dVPS. For large tumor, they contribute to dVPS mostly at the superior and inferior edges, where the images at different respiratory phases may have the largest volume difference, generating various small peaks superior to the diaphragm characteristic peak in the dVPS with high frequencies. Owing to the initial moving average and the DCT low-pass filter (dimension compression with 7 lowest frequencies), some of the tumor-induced high frequencies are likely filtered out. Therefore, this DCT-MLR machine learning process based on the dVPS is insensitive to the presence of lung tumors.

Advantages, applications, and limitation of this approach

The advantages of this machine-learning method can be summarized as follows: (1) the method predicts the average diaphragm motion automatically without user intervention. From lung segmentation, dVPS generation, to motion prediction, all are performed automatically. This could be a scalable approach to process large-scale medical data to gain deeper insights into the diaphragm motion. (2) The prediction uncertainty is within one slice thickness (2.5 mm > 1σ = 1.9 mm), which is clinically acceptable. The improvement is seen in the 4DCT1 + 4DCT2 group, where another 4DCT from the same patient is used as the training set, resulting in slightly improved prediction results. This suggests the importance of selecting representative training images and we are currently investigating this avenue to improve the prediction outcomes. (3) This method can be used to facilitate the motion estimation and therefore can help to assess the motion variation between different 4DCT scans for potential adaptation. In figure 4, the motion variations of the diaphragm are clearly shown. (4) This approach could be generalized to process respiratory-correlated 4D magnetic resonance imaging (4DMRI) images. Moreover, it could be useful to process sagittal or coronal cine 2DMRI images in real time and quantify the 2D ADMT value on the fly. This information could be used as an internal respiratory surrogate to index the cine 2DMRI images for reconstructing respiratory-correlated 4DMRI. The lung internal motion (lung vessels or tumor) in relationship with the diaphragm could be investigated, in addition to the correlation with external markers (Koch et al 2004).

The limitation of this machine learning method is the inherent ~5% (1 out of 22) chance of producing an outlier (mean ± 2σ) in the motion prediction. In addition, we have found that when the test patient has motion range outside of the training range, the prediction error may increase. In general, the larger the training dataset is the more robust the prediction outcomes. This study only includes 11 patients and 22 4DCT images, which is limited in representing the diversity of patient anatomy and respiratory motion, therefore, further investigation is needed to render our approach more comprehensive. Last, we recommend to applying same 4D image reconstruction method in both training and testing datasets, avoiding different level of motion artifacts caused by common breathing irregularities.

Conclusion

The motion trajectory for the left and right sides of the diaphragm can be predicted accurately with an average uncertainty of 1.6 mm (1σ), which is smaller than the 4DCT slice thickness (2.5 mm). Because of possible interference from cardiac motion, the prediction has a higher uncertainty on the left side while it is more accurate for the right side and increasing the number of 4DCT image sets of the same patient pool only provides marginal improvement. This method in predicting diaphragm motion could be potentially useful in the clinic for inferior lung lesions and superior liver lesions. Further investigations are needed to make this machine-learning method a comprehensive tool by including more patients into the training pool.

Appendix

Due to the energy loading in low frequency ac coefficients as summarized in equation (3), by discarding spectral components higher than k, equation (5) was changed to a linear equation where we can invoke a MLR process to estimate diaphragm position (z₀) based on the R_k array. The equation for the MLR is formulated as:

$$z_0=w_0+\sum _{i=1}^k[w_i·{\mathit{R}}_k(i)]$$ (A1)

Or

$$z_0=w\circ (1,{\mathit{R}}_k),$$ (A2)

where the weight vector w = (w₀, w₁, …, w_k); the symbol ∘ is the inner product operator.

By stacking all z₀’s as the vector z:

$$\mathit{z}={(z_0^1,z_0^2,\dots ,z_0^N)}^T,$$ (A3)

where the superscript ^T is the transpose operator. Furthermore, the corresponding $R_k^j$’s for all phases and all patients (j ranges from 1 to N, the total number of diaphragm positions in all motion trajectories of all patients) are put together as the matrix R

$$R=\left(\right(1,{\mathit{R}}_k^1);(1,{\mathit{R}}_k^2);\dots ;(1,{\mathit{R}}_k^N\left)\right).$$ (A4)

Equation (5) is represented by the following vector equation:

$$\mathit{z}=\mathit{w}\circ R=R{\mathit{w}}^T.$$ (A5)

The MLR then dictates that the weight vector (w) can be estimated by minimizing the sum of squared errors:

$$\mathit{w}={(R^{+}\mathit{z})}^{\mathit{T}},$$ (A6)

where R⁺ is the pseudo-inverse matrix of R, i.e., (R^T R)⁻¹ R^T. The computation of the pseudo-inverse matrix is expensive, in (Wilms et al 2014), some careful considerations were provided to render this computation more efficient. In our work, we used the well-known celebrated MATLAB’s backslash operator for this purpose: from equation (A5), w is then the mathematically equivalent (R\ z)^T. Based on our tests, with the same output, the backslash operator, the QR factorization based method, is faster than the method based on equation (A6) by order of magnitude. On our Dell Precision M6700 laptop with CPU Intel Core i7-3740QM @2.70GHz with 16GB RAM, the MLR procedure based on equation (A6) need 0.126 s, whereas the backslash operator based method took only 0.013 s (with one less order of magnitude), which makes it possible to deliver real time motion prediction during treatment time.