Adaptive respiratory signal prediction using dual multi-layer perceptron neural networks

Purpose.

To improve the prediction accuracy of respiratory signals by adapting the multi-layer perceptron neural network (MLP-NN) model to changing respiratory signals. We have previously developed an MLP-NN to predict respiratory signals obtained from a real-time position management (RPM) device. Preliminary testing results indicated that poor prediction accuracy may be observed after several seconds for irregular breathing patterns as only a set of fixed data was used in one-time training. To improve the prediction accuracy, we introduced a continuous learning technique using the updated training data to replace one-time learning using the fixed training data. We carried on this new prediction using an adaptation approach with dual MLP-NNs rather than single MLP-NN. When one MLP-NN was performing prediction of the respiratory signals, another one was being trained using the updated data and vice versa. The predicted performance was evaluated by root-mean-square-error (RMSE) between the predicted and true signals from 202 patients’ respiratory patterns each with 1 min recording length. The effects of adding an additional network, training parameter, and respiratory signal irregularity on the performance of the new predictor were investigated based on four different network configurations: a single MLP-NN, high-computation dual MLP-NNs (U1), two different combinations of high- and low-computation dual MLP-NNs (U2 and U3). The RMSEs using U1 method were reduced by 34%, 19%, and 10% compared to those using MLP-NN, U2 and U3 methods, respectively. Continuous training of an MLP-NN based on a dual-network configuration using updated respiratory signals improved prediction accuracy compared to one-time training of an MLP-NN using fixed signals.

1. Introduction

Dynamic tracking of a moving target with a radiation beam in radiation therapy requires prediction of the real-time target location ahead of beam delivery, as it takes time for the computer to calculate multi leaf collimator shapes and radiation output after the location of the moving target is identified. A number of potentially feasible prediction methods (Sharp et al 2004, Ren et al 2007, McCall and Jeraj 2007, Murphy and Pokhrel 2009, Verma et al 2011, Lee et al 2012, Guan et al 2013, Liu et al 2014, Bukovsky et al 2015, Sun et al 2017) have been reviewed. Among these methods, autoregressive moving average (ARMA) model (Sharp et al 2004, Ren et al 2007, McCall and Jeraj 2007), Kalman filtering method (Lee et al 2012, Guan et al 2013, Liu et al 2014), and artificial neural network (ANN) (Murphy and Pokhrel 2009, Bukovsky et al 2015, Sun et al 2017) were most commonly used. The major advantages of ARMA method were the real-time prediction and update of model parameters (Verma et al 2011). However, the ARMA model basically assumed the linear relationship between the pre- and post-respiratory signals. Hence, this method would not be suitable for the non-linear respiratory signals. The Kalman filter model required that two equations (the state equation and the observation equation) be established at the beginning. However, the initial parameters of these two equations were difficult to determine. On the other hand, the multi-layer perceptron neural network (MLP-NN) method could obtain an effective prediction performance for non-linear respiratory signals (Verma et al 2011, Sun et al 2017). The study by Verma et al (2011) showed that the ANN model achieved better prediction performance compared to the Kalman filtering method. However, the respiratory signals often change with time based on emotional and physical conditions in clinical situations. If the predictors remain unchanged, the prediction accuracy may decrease after a certain time. Consequently, it is crucial to adjust the predictors in the predictive process.

A few previously studies reported the respiration-induced motion prediction with real-time adaptability of the predictors (Ruan 2010, Tolakanahalli et al 2015). However, these models, such as the autoregressive and kernel density estimation models, might be challenged by their poor prediction accuracy due to simple model structures. Few studies (Murphy and Pokhrel 2009, Bukovsky et al 2015) investigated the adaptability of predictors using the neural network due to the fact that the practical requirement for real time was typically difficult to meet. Furthermore, these studies always trained the models using newly measured signals before carrying on new signal prediction (Murphy and Pokhrel 2009, Bukovsky et al 2015). This approach was referred to as a sliding window approach (Bukovsky et al 2015). For example, Bukovsky et al (2015) restricted the maximum epochs (ME) to 8 and used small training signals (TS) (90 and 180) at each step. Murphy and Pokhrel (2009) used only two hidden layer neurons and restricted the ME to 200 with only one set TS at each step. However, this kind of method may not optimize the loss function against the mass data set at each step due to small TS and ME. Hence, its prediction performance might become poor with data not used for training, i.e. the error was small using the training data set but large using the testing set. As such, use of a single MLP-NN may have challenges of both prediction accuracy and efficiency using real-time signals.

In this study, we introduced a new method using dual MLP-NNs to accurately predict respiratory signals. The core idea of this new prediction framework is to continuously train the parameters in MLP-NN using on-line acquired signals by introducing dual MLP-NNs. That is, when one MLP-NN is performing the prediction of the respiratory signals, another MLP-NN is being trained using the updated respiratory signals.

Two kinds of MLP-NN method were used in this study, the high-computation MLP-NN (HC-MLP-NN) and the low-computation MLP-NN (LC-MLP-NN). The HC-MLP-NN was a conventional MLP-NN which was also discussed in our previous study (Sun et al 2017). The LC-MLP-NN was an HC-MLP-NN with some restrictions (the ME and TS) to meet the practical requirement of real time prediction. Three different structures were investigated using the dual MLP-NNs method: dual MLP-NNs with both high-computations (U1), dual MLP-NNs with two different combinations of high- and low-computations (U2), MLP-NN with low-computations (U3). In the U2 method, an HC-MLP-NN was used to fully train the prediction model and an LC-MLP-NN was utilized to train the prediction model in real time during the fully training time of HC-MLP-NN. In the U3 method (also called the sliding window approach method), an LC-MLP-NN would be trained continuously to predict all the respiratory signals. In this study, two new methods (U1 and U2), were evaluated by calculating the root-mean-square-error (RMSE) between the predicted and true signals using 202 patients’ respiratory signals. Their performances were compared to that using the MLP-NN and sliding window method (the U3 method).

2. Materials and methods

2.1. Data acquisition

The respiratory patterns for 202 patient cases were obtained using a real-time position management (RPM) system (Varian Medical Systems, Palo Alto, CA) (Yan et al 2006). Each pattern was saved with 2 min recording length (30 Hz sampling rate). The initial 1 min recorded signals were used for training and the remaining 1 min recorded signals were used for testing.

2.2. Prediction process

The scheme for the prediction process in this study was outlined in figure 1. As explained in more detail in our previous publication (Sun et al 2017), the breathing signals collected before the selected cut-off time K, regarded as the TS (1 min recorded length), were first smoothed using Savitzky-Golay (S-G) algorithm (Savitzky and Golay 1964, Sun et al 2017) and then used for training ANN model. The collected breathing signals after K (1 min recording length) were used as testing signals for evaluating the proposed ANN model (MLP-NN, U1, U2 and U3). Besides, in order to keep the originality of raw data, the testing signals would not be smoothed.

Figure 1.

Flow chart of the MLP-NN prediction process.

The data samples between t = 1 and t = H, referred as training input data, were utilized to predict the target position at a specific time t = H + M (prediction target). This process would be repeated to predict the next target position. The data samples between t = K − H − M + 1 and t = K−M would be utilized to predict the final target position in the training data set. S(K + 1, K + 2, …, K + L) were utilized as the testing signals for evaluating the goodness of the trained ANN model (any of MLP-NNs with and without online training).

The similar process used in the training would be applied to predict the target position S′(K + H + M) using the data samples between t = K + 1 and t = K + H as inputs to the trained ANN. Here, S′(K + H + M) = g(S(K + 1),…, S(K + H)), where the function g() was an MLP-NN predictor and S′(K + H + M) was the (H + M)-signal forward predicted signal for the respiratory signal at t = K. This process was also repeated to predict the next target position. The final predicted signal S′(K + L) was based on the input data samples between t = K + L−H − M + 1 and t = K + L − M. Finally, the predicted signals S′(K + H + M), …, S′(K + L) and the true positions S(K + H + M), …, S(H + L) were compared.

2.3. The three methods

The sketch of the U1 training algorithm developed in this study is shown in figure 2. The U1 used two HC-MLP-NNs so that the alternative performance mechanism of predicting respiration position and updating the HC-MLP-NN parameters during training over the prediction time is possible. A three-layer MLP-NN with back-propagation error-minimizing algorithm was used as the HC-MLP-NN and was explained in more details elsewhere (Sun et al 2017). As shown in figure 3, the time intervals of T1, T2, …, T11 represented the different periods of time. The HC-MLP-NN 1, trained using the real respiratory signals collected during T1 period (the first TS indicated by blue color), was used to predict the respiratory signals (the first predicted signals) during T3 period (red color). After T4 (green color) and T5 periods (blue color), the HC-MLP-NN 2 started to be trained using the real signals collected during T5 period. In this process, the TS would be updated by removing the real signals during T4 period and adding new real respiratory signals (the length of this period was the same as T4) and the HC-MLP-NN 2 would be trained in T6 (black color) period. It should be noted that the weights and biases of the two HC-MLP-NNs in U1 would be initialized randomly at each fully trained process. Then the trained HC-MLP-NN 2 was used to predict the respiratory signals during T7 period (red color). After predicting the respiratory signals during T7 period using the trained HC-MLP-NN 2, the HC-MLP-NN 1 would again be utilized to predict the respiratory signals during T11 (red color) period as it has been trained using the real respiratory signals during T9 period (blue color) in the same way. The respiratory signals would be predicted continuously with this alternate prediction and training approach until the end of the prediction. It should be noted that a single MLP-NN method in this study would predict the all signals (T3, T7, T11…) after it was trained using the T1 period signals.

Figure 2.

Schematic illustration of prediction algorithm for U1.

Figure 3.

Schematic illustration of prediction algorithm for (a) U2 and (b) U3 training algorithms.

As shown in figure 3(a), the U2 method used an HC-MLP-NN (one MLP-NN of the U2) to fully train and subsequently train the prediction model while used an LC-MLP-NN (another MLP-NN of the U2) to update the model during the calculation time periods (T2, T6, T10, ….). Here, the time intervals of T_Train, T_Train1, … represented different time periods used for the first training and the subsequent training of the HC-MLP-NN. Besides, the teeth on the LC-MLP-NN in figure 3 illustrated real respiratory signals and T_real represented the training time period of the LC-MLP-NN. In particular, the HC-MLP-NN was used only to predict the respiratory signal at t = t_Train (The first predicted signal) by using the real respiratory signals collected during T_Train period (The first TS). Then the weights and biases of the trained model were transferred to the LC-MLP-NN. The LC-MLP-NN would be trained using the recent TS (TS of the LC-MLP-NN) during the time period of T2 with every new measured signal data prior to each new signal prediction. After t_Train1, the HC-MLP-NN would be fully trained using real signals during T_Train1 period and the weights and biases of the new model would be transferred to the LC-MLP-NN again. In this process, the TS would be updated by removing the real signals between t = t₁ and t = t_CLT and adding the new real respiratory signals (the length of this period would be the same as T2) and the LC-MLP-NN would be trained during the T6 period. This process will repeat during the entire prediction process until the end of prediction. It should be noted that the weights and biases of the HC-MLP-NN in U2 would be initialized randomly at each trained and fully trained process. As shown in figure 3(b), the LC-MLP-NN in the U3 method, would be trained using with every new measured signal data prior to each new signal prediction, would be used to predict all respiratory signals.

2.4. Evaluation methods

The parameters utilized in our previous study (Sun et al 2017) were used to test all four algorithms (MLP-NN, U1, U2, and U3): the time ahead of prediction was 500 ms (Sun et al 2017), the input layer neurons were 25, the hidden layer neurons were 8, the initial learning rate was 0.01, the threshold value for the cost function of MLP-NN was 0.01, and the ME of LC-MLP-NN for the U2 and U3 was 8.

RMSE between the true and predicted signals was calculated for assessment. The RMSE was the RMSEs from all predicted data samples and was calculated as following:

$$RMSE=\sqrt{\frac{1}{L-M-H+1}\sum _{t=H+M+K}^{K+L}(S^{\prime }(t)-S(t){)}^2}.$$

Here, S(t) is the true value and S′(t) is predicted value. Confidence interval (CI) based on bootstrapping technique was utilized as statistic method to evaluate the difference in this study. The percentage of CI was 95% in this study. The bootstrap sample number was 5000.

2.5. Breathing pattern variability

We introduced the respiratory variability metric (VM) to evaluate the performances of single MLP-NN, U1, U2 and U3 methods using different breathing patterns (King et al 2009, Sun et al 2017). More details were explained elsewhere (Cai et al 2007, Huang et al 2010, Sun et al 2017). All patient data were divided into two groups (irregular and regular) with equal number of data points below and above the selected variability value v = 0.11.

3. Results

3.1. Computational efficiency and prediction performance

The computational cost and real-time prediction capability of the four methods were investigated. For the U1 and U2 methods, the maximum calculating time of HC-MLP-NN model training using 1800 TS was 0.96 s which could not meet the requirement of real time prediction. However, the trained model could predict the respiratory signals in real time when it finished training. Hence, 1 s was used as both HC-MLP-NN model training time (T2, T6, T10… in figure 3 and T_Train, T_Train1, T_Train2… in figure 4(a)) and predictive time (T3, T7, T11 … in figure 3). For the U2 and U3 methods, the maximum calculation times of LC-MLP-NN with 90, 180 and 360 TS (Bukovsky et al 2015) were 0.031 s, 0.033 s and 0.082s using MATLAB2017b software on a typical personal computer (i7 HQ, 16 G), respectively. Hence, 180 was selected as the TS of the LC-MLP-NN.

Figure 4.

One patient’s (a) total respiratory curve and the prediction performance comparison between (b) U1 and MLP-NN, (c) U1 and U2, (d) U1 and U3.

Table 1 lists both the RMSEs and their corresponding CIs calculated between the true and predicted signals for all 202 patient cases using four different methods. The details of the ARMA method used in this study could be found in other study (White et al 2011). The Akaike’s information criterion methods (Hannan 1980, Pukkila et al 1990, White et al 2011) were applied to investigate the hyper-parameters (autoregressive and moving average model order) of the ARMA model using the grid search method and the 1440 TS (80% of the 1800) for each patient (the range was [1~5]). The maximum calculating time of the hyper-parameters tuning using 1440 TS was 11 s which could meet the requirement of real time (less than the time of 360 TS). Besides, for all the patients, the calculating time of the ARMA model training using 1800 TS was less than 0.33 s. Hence, 0.33s was used as the training time of the ARMA model. The U1 method reduced RMSE by 14.4%, 11.7%, 33.9% and 51.9% compared to the MLP-NN, U2, U3 and ARMA methods, respectively. The CI (95%) was also reduced to (0.110~0.117) using the U1 method from (0.126~0.138), (0.122~0.136), (0.162~0.182) and (0.223~0.250) using the MLP-NN, U2, U3 and ARMA methods, respectively.

Table 1.

Performance of four different MLP-NN methods.

Method	RMSE (cm)	CI (cm)
MLP-NN	0.132	0.126~0.138
U1	0.113	0.110~0.117
U2	0.128	0.122~0.136
U3	0.171	0.162~0.182
ARMA	0.235	0.223~0.250

Figure 4 illustrated an example of one patient’s total respiratory signals (curve) and the corresponding prediction performance comparison between the U1 and other three methods. As shown in figure 4(a), the respiratory signals between S(1) and S(K) were utilized to train the proposed ANN models while the signal positions between S(K + H + M) and S(K + L) were applied to assess the performance of all prediction methods. Moreover, the signal positions between S(K) and S(K + H + M) were used to predict the first signal. As shown in figures 4(b), the prediction errors were substantially larger using MLP-NN compared to U1. The prediction performance of U2 was comparable to that of U1, except for a few prediction positions. Besides, the prediction errors of U3 were large at nearly all the testing period.

The averages of RMSE using four prediction methods in different predictive periods were tested and results were shown in figure 5. The RMSEs using the MLP-NN method became worse with time. In contrast, the RMSEs using the U1, U2 and U3 methods were stable.

Figure 5.

The averages of RMSE using single MLP-NN, U1, U2 and U3 methods in each predictive period.

3.2. Effect of breathing pattern irregularity

Table 2 demonstrated both RMSE and corresponding CI values calculated between the true signals and predicted signals for all 202 patient cases including both regular and irregular groups using four different prediction methods. The U1 method achieved the best performance in both regular and irregular groups. RMSEs were reduced by 10% (from 0.110 to 0.099) for the regular group and 17.5% (from 0.22 to 0.16) for the irregular group using the U1 compared to the MLP-NN method. Besides, the RMSEs were reduced by 9.1% (from 0.110 to 0.100) for the regular group and 19.1% (from 0.157 to 0.127) for the irregular group using the U1 compared to the U3 method. The ranges of CIs were also improved from (0.109~0117) using the U1 method to (0.132~0.147), (0.118~0.138) and (0.155~0.228) using the MLP-NN, U2 and U3 methods, respectively.

Table 2.

Performance of four different methods for both regular and irregular groups.

Method	Group	RMSE (cm)	CI (cm)
Single MLP-NN	RE	0.110	0.104 ∼ 0.115
	IR	0.154	0.145 ∼ 0.164
U1	RE	0.099	0.096 ∼ 0.102
	IR	0.127	0.122 ∼ 0.133
U2	RE	0.100	0.096 ∼ 0.104
	IR	0.157	0.148 ∼ 0.169
U3	RE	0.129	0.123 ∼ 0.134
	IR	0.213	0.200 ∼ 0.231

RE: Regular group (variability < 0.11 cm). IR: Irregular group (variability > 0.11 cm).

The relationship between respiratory VM and the difference of RMSEs between the U1 and other three methods for all datasets is shown in figure 6. Here, MLP-U1, U2-U1 and U3-U1 represented the difference of RMSEs between MLP-NN and U1, U2 and U1, U3 and U1, respectively. The overall differences of prediction accuracy between the U1 and other three methods (MLP-NN, U2 and U3) became larger with increasing VM.

Figure 6.

Relationship between respiratory VM and the differences of RMSEs between the U1 and other three methods.

4. Discussion

All four prediction methods (MLP-NN, U1, U2 and U3) for predicting respiratory signals were developed, tested and compared. As shown in table 1, substantial improvement of prediction accuracy was accomplished using the U1 method by continuously training and updating the parameters in MLP-NN using on-line acquired signals with dual MLP-NNs.

The effectiveness of the dual MLP-NN approach is shown in figure 5 as tested using respiratory signals from 202 patient cases in this study, where the overall differences of prediction accuracy between the MLP-NN and the proposed methods (U1 and U2) became larger over time. This was because the differences between the old training data and the current breathing signals became larger as recording time increased. On the contrary, both U1 and U2 methods could prevent prediction accuracy from getting worse by updating the training data of each MLP-NN with time.

For all four methods discussed in this study, the averages of RMSE and CI from the regular group were better than those from the irregular group. However, our results indicated that U1 method substantially improved prediction performance in the irregular group mainly due to that the TS used in the U1 method were continuously updated. However, although the prediction model using the U3 method could be updated in real time, the performance was obviously worse than that using the U1 method, especially in the irregular group. This may be because the U3 method could not optimize the loss function against the mass data set at each step due to the two main parameters, TS and ME. For the TS parameter, a value of 180 was used in this study which might roughly cover only a single or two of breathing wave forms and might be insufficient to yield the general characteristics of the current respiratory motion. However, the maximum calculation times for the LC-MLP-NN using a range of TS values of 90, 180 and 360 were 0.031 s, 0.033 s and 0.082 s, respectively. Hence, if we used a value more than 180 for the TS, the calculation time would not meet the requirement of real-time. For the ME parameter, if the restriction of the ME was more than 8, the calculation time could not meet the requirement of real time. Hence, we selected 8 as the ME used in this study. These could be also the reasons why the U2 method got the best performance in the regular group but obtained poor performance in the irregular group. For the U2 and U3 methods, both expanding TS and reducing ME may be possible solutions for improving the prediction performance and meeting the real-time requirement. Reducing ME may be a solution to reduce the calculation time and provide extra time for the expansion of TS. Besides, using small ME may help avoid the overfitting problem (i.e. early-stopping). However, too small ME may lead to an underfitting problem. Moreover, the computational cost discussed in this study mainly focused on the adaptation approaches and did not depend on the ANN architecture. Although the technique presented in our study works well and is feasible for real-time adaptive prediction, other prediction models (Bukovsky et al 2015, Teo et al 2018) based on the deep learning techniques may also be investigated for the proposed real-time adaptation approaches in the future studies.

It should be noted that this study was based on the respiratory signals obtained from the RPM devices, which could be different from the actual respiratory motion of the internal organ.

5. Conclusion

A novel idea using dual MLP-NNs was developed using continuously updated training data to predict respiratory signals. The results demonstrated that the implementation of this novel idea in the U1 method substantially improved the prediction accuracy compared to the other three methods for a long-time period prediction (1 min was tested in this study). The U1 method could potentially be utilized as a valuable tool to predict respiratory motion for dynamic tracking of moving targets during radiation therapy.