Respiratory motion estimation of the liver with abdominal motion as a surrogate

Abstract

Background: Respiratory‐induced motion (RIM) causes uncertainties in localizing hepatic lesions, which could lead to inaccurate targeting during interventions. One approach to mitigate the problem is respiratory motion estimation (RME), in which the liver motion is estimated by measuring external signals called surrogates.

Methods: A learning‐based approach has been developed and validated to estimate the RIM of hepatic lesions. External markers placed on the human's abdomen were chosen as surrogates. Accordingly, appropriate motion models (multivariate, Ridge and Lasso regression models) were designed to correlate the liver motion with the abdominal motion, and trained to estimate the superior–inferior (SI) motion of the liver. Three subjects volunteered for 6 sessions of such that liver images acquired by magnetic resonance imaging (MRI) were recorded alongside camera‐tracked external markers.

Results and conclusions: The proposed machine learning approach was validated in MRI on human subjects and the results show that the approach could estimate the respiratory‐induced SI motion of the liver with a mean absolute error (MAE) accuracy below 2 mm.

1. INTRODUCTION

Percutaneous image‐guided interventions such as biopsy and tumor ablation are commonly used during the diagnosis and treatment of liver cancer.1, 2 Medical imaging modalities (such as computed tomography (CT), ultrasound (US), magnetic resonance imaging (MRI), etc.) are widely used during such interventions. In fact, high‐quality and real‐time medical images can significantly improve the overall diagnosis and treatment of cancer patients.3 However, the two requirements are usually contradictory. For instance, MRI offers high image quality (contrast and spatial resolution) at a low update rate, whereas ultrasound offers the opposite.4 For liver interventions, the soft tissue contrast in MRI makes it the preferable modality.4, 5 Apart from the imaging modality used, accurate targeting of hepatic lesions is vital to successfully complete the interventions.6 The most common cause of inaccurate targeting is internal organ motion. In particular, one of the main causes of internal organ motion is respiration.7

1.1. Respiratory‐induced motion

Respiratory‐induced motion (RIM) mainly affects the organs in the abdominal and thoracic regions (such as lungs, liver, diaphragm, etc.).8, 9 In fact, RIM of the liver could range from 8 to 25 mm in one direction during shallow breathing.7, 10

Thus, if RIM is not handled together with slow imaging modalities, locating the exact lesion motion will be uncertain, which implies inaccurate targeting of the inserted needle during percutaneous interventions. In fact, this uncertainty and inaccurate targeting cause significant damage to healthy tissues, insufficient treatment and recurrences during ablation, and misdiagnoses during biopsy.1, 11

A common solution to RIM is breath‐holding, which requires the patient to hold their breath for approximately 20 seconds, such that treatments occur only while the breath is held.12 The main disadvantages of breath‐holding are an increase in the intervention time, inconsistent lesion location between breath holds13 and that patients might feel uncomfortable during breath holds or cannot hold their breath for sufficient time.14

1.2. Respiratory motion estimation

An alternative solution to deal with RIM is respiratory motion estimation (RME). RME estimates the actual internal motion of interest by measuring external signals, so called surrogate data, that do not directly measure the actual internal motion of interest but have a strong correlation with it and can be easily measured.9 The actual internal motion of interest is referred to as motion data in this article, and is usually medical imaging data acquired at a low update rate.15 Moreover, RME depends on deriving a motion model that mathematically describes the relation between the motion data and surrogate data. This relationship is represented by a set of parameters that are determined during the training phase by the fitting method, which is typically a supervised learning algorithm. As shown in Figure 1, RME consists of two phases: a training phase during which the motion and surrogate data are fed to a fitting method to train the motion model offline, and a prediction phase in which the surrogate data are fed to the motion model to generate motion estimates.

Figure 1.

An overview of RME. The surrogate data and motion data (which represent the actual internal motion of interest) are acquired at the same time. The fitting algorithm trains the motion model using a supervised learning algorithm such that motion estimates could be predicted by using only the surrogate data

RME is utilized in applications where it is not possible or feasible to directly acquire the actual internal motion of interest with a tolerable temporal resolution.11 RME can also be used to minimize motion‐induced artifacts by adjusting the reconstruction of the acquired images.13 Moreover, the motion estimates can be implemented as gate signals during respiratory gating. In such a manner, the RIM of the lesions is constantly scanned during normal breathing, while the radiation is delivered during a specific window of the breathing cycle that is indicated by the gate signal.8

1.3. Surrogate data

MR navigators are the most common surrogates for MRI in RME.11, 13 The navigators are convenient for MR applications since they do not require extra equipment or installation. However, the main disadvantages of an MR navigator are that it is a 1‐dimensional signal11 and that it might slightly increase the acquisition time.4 Spirometery is another example of a surrogate signal that relies on measuring the volume of the air flow in and out of the lungs.16 The main advantage of spirometers is that they provide simple 1‐dimensional signals that have a high correlation to the respiratory motion.17 However, studies have shown that spirometers have a noticeable drift due to air leakage,13, 18, 19 as well as a noticeable discomfort to patients.16 In the same context, respiratory bellows are as common as spirometers but are tapered around the subject's thorax, measuring its motion due to respiration.20 The bellows share the same advantages and disadvantage as spirometers. Furthermore, to get an accurate measurement, the bellows have to be tightly stretched around the patient's thorax, which is uncomfortable for the patient.13

Accelerometers are also used as surrogate signals22 during percutaneous interventions1 and have been tested for their MRI compatibility.13 Nevertheless, they are usually not preferred due to their poor MRI compatibility and low correlation to respiratory motion.22 Surrogates could also rely on other medical imaging modalities. For example, high‐resolution US has been studied to measure the diaphragm motion while using low‐resolution MRI as motion data.4 However, US is still not MRI‐compatible unless the US is wired with optical fibres, which makes it expensive and fragile.

Finally, optical tracking is another choice of surrogate data. By using an infrared (IR) or a digital camera, markers placed on the patient's chest or abdomen are tracked. Many studies have shown that the patient's thorax motion has a good correlation with respiration.15, 16, 21, 22, 23 For example, Beddar et al. used the Real‐time Position Management System (RPM, Varian Medical Systems, Palo Alto, CA), which had a reflective marker placed on the human's abdomen and tracked using an IR camera, and also showed that the motion is well correlated with the internal motion of the liver.21 Vedam et al. studied the same effect but for internal motion of the diaphragm,24 and Ernst et al. studied the correlation between multiple external IR‐reflective LEDs and the internal liver motion of a swine15; both obtained similar correlation results to Beddar et al. 21 Henningsson et al. and Wasza et al. stated that using external markers is advantageous due to its high temporal and spatial resolution,25, 26 whereas Wilms et al. elaborated that by using optical tracking, multiple markers could be advantageous since the dimensionality of the system will increase without adding extra cost to the setup.27

In this context, the motion data are MRI‐acquired liver motion. Thus, the surrogate should operate safely inside the MRI scanner, and should not cause additional discomfort to the patient. As a result, for RME of the liver in MRI, optical tracking is a suitable surrogate since it is MRI‐compatible, drift‐free28 and multi‐dimensional.27 Moreover, the tracked markers are not tightened around the patient's abdomen but stuck on the body, thus saving space and causing less discomfort to the patient.

1.4. Contribution and objectives

The purpose of this study is to develop and validate an RME approach that can estimate the motion of a moving target in the liver to improve targeting during percutaneous image‐guided interventions. The paper focuses on RME of the liver in which the motion data are MRI‐acquired liver motion while the surrogate data are external markers. As shown in Table 1, most of the previous research in RME of the liver has focused on other types of surrogate data (such as MRI, bellows, accelerometers, etc.) apart from external markers. Furthermore, studies that used external markers as a surrogate for RME of the liver usually validated their approaches on either phantoms or animal subjects. In this context, the suggested approach was validated on human subjects by utilizing state‐of‐the‐art supervised learning fitting methods (namely linear, Ridge and Lasso).

Table 1.

Summary of related work on RME of the liver

Related work	Motion data	Surrogate data	Fitting methods	Experimental validation
Beddar et al.,21	CT	External markers	Linear regression	Human subjects
Odille et al.,29	MRI	Bellows	Image registration & reconstruction	Motion phantom & human subjects
White et al.,30	MRI	MRI	Linear regression	Human subjects
Nguyen et al.,31	CT	MRI	Population‐based deformable registration	Human subjects
Hinkle et al.,32	CT	External markers	Image registration & reconstruction	Motion phantom
Ernst et al.,15	Xray	External markers	Support vector regression	Porcine subject
Rijkhorst et al.,33	MRI	MRI	Linear regression	Human subjects
Buerger et al.,34	MRI	MRI	B‐spline smoothing	Human subjects
Preiswerk et al.,4	MRI	US	Kernel smoothing	Human subjects
Chen et al.,13	MRI	Acc.a & Bellows	Linear & Ridge regression	Motion phantom & Human subjects
Abayazid et al.,1	EMb	Acc.	RAkELc	Motion phantom

^aAccelerometers

^bElectromagnetic sensor

^cRandom k‐Labelset.

The objectives of this paper are to choose appropriate motion models and fitting algorithms that have a strong correspondence between the surrogate and motion data, and to validate and assess the designed motion models and the suggested RME framework by conducting in vivo experiments.

2. METHODS AND MATERIALS

2.1. Motion model

2.1.1. Analysis

Following McClelland et al.,11 the motion model is analyzed according to

• how the motion is represented in the model,

• how the motion data are related to the surrogate data,

• how the data are fitted.

The liver motion is represented by the displacements of the upper border of the liver and the abdominal motion is represented by the tracked displacements of the markers. Additionally, if the model is specific to the patient and session, linear representations are sufficient. As a result, linear fitting methods were chosen in this context. The most common method is multivariate linear regression (MVR).11, 35 Other methods are modifications of MVR, such as Ridge regression,36, 37 principal component regression,38 etc. With that said, the motion model is formulated as follows:

$$y_i\approx f(x_i)={\beta }_0{x}_{i0}+{\displaystyle \sum _{j=1}^p}{x}_{ij}{\beta }_j,$$ (1)

such that y _i represents the motion data, β = [β ₀,β ₁,…,β _p] are the parameters/coefficients of the motion model, x _i = [x _i1,x _i2,…,x _ip] are the model features perceived from the surrogate data, p is the number of parameters and x _i0 is the dummy unit intercept to account for the bias in the model. Furthermore, the parameters β are determined using the fitting method. In this context, MVR was implemented and extended with shrinkage methods (Ridge and Lasso regression). Shrinkage methods are used when multiple surrogate data are highly correlated with each other.11

2.1.2. Multivariate linear regression

The main objective of MVR is, given a set of N training data {(x ₁,y ₁), (x ₂,y ₂), …,(x _N,y _N)}, to estimate the set of parameters β using the ordinary least squares (OLS) method by minimizing the cost function J(β):

$$J(\beta )=\underset{\text{Fitting penalty}}{\underbrace{{\displaystyle \sum _{i=1}^N}{(y_i-f(x_i))}^2}}$$ (2)

over all the training data.35 Thus, MVR obtains a set of parameters β that provide the optimal fit to the training data. However, MVR can be challenging when dealing with multiple features since the greater the number of features, the more MVR tries to overfit the training data35 by capturing the noise and idiosyncratic characteristics of the signal rather than the physical model. Overfitting might also occur since the features are highly correlated, considering that they are extracted from multiple markers.11 Thus, MVR has to be extended to penalize for the number of parameters β.39

2.1.3. Shrinkage methods

Ridge regression is a shrinkage method that extends the cost function of MVR such that the cost function is written as follows40:

$$J_R(\beta )=\underset{\text{Fitting penalty}}{\underbrace{J(\beta )}}+\underset{\text{Shrinkage penalty}}{\underbrace{\lambda {\displaystyle \sum _{j=1}^p}{\beta }_j^2}},$$ (3)

such that λ denotes the shrinkage parameter to be tuned (λ ≥ 0). If λ is zero then the equation reduces to MVR. If λ is too big then the model is optimized to minimize the parameters rather than to fit the model, which will underfit the data. The greater the shrinkage parameter, the more the parameter β shrinks towards zero. Thus, there has to be a trade‐off in selecting the shrinkage coefficient. The parameter λ is tuned by validating the motion model over a set of validation data and choosing the parameter λ that results in the smallest prediction error (as stated in Section 2.1.5).

Lasso regression41 is another shrinkage method that penalizes the absolute size of the coefficients as follows:

$$J_L(\beta )=\underset{\text{Fitting penalty}}{\underbrace{J(\beta )}}+\underset{\text{Shrinkage penalty}}{\underbrace{\lambda {\displaystyle \sum _{j=1}^p}|{\beta }_j|}}.$$ (4)

The main difference between Ridge and Lasso is that Ridge shrinks the parameters towards zero while Lasso has the ability to shrink the parameters to exactly zero.39 Thus, Lasso has the ability to cancel out features that are not needed if the shrinkage parameter λ is relatively large. In other words, if the features are highly correlated, Lasso selects a set of fewer features and cancels out all the others. However, Lasso is more computationally expensive than Ridge because there is no closed‐form solution for its optimization problem.

2.1.4. Model selection

The acquired dataset has to be split into 3 segments: training data, validation data and test data,35 such that the training data are used to fit the models, the validation data are used to select the features and shrinkage parameter while the test data are used to evaluate the performance of the models. The proportions of test, validation and training data are chosen to be 50%, 20% and 30% respectively. Note that there is no general rule for choosing the ratios between the 3 data segments.35

2.1.5. Feature and shrinkage parameter selection

The features are extracted from the surrogate data and are selected such that they are simple, interpretable and have a good fit to the data. The features could be numeric values of the input signals (the positions of the markers) or polynomial representations of the input signals (quadratic, cubic or higher order).35 In the context of RME, the features can be the derivative(s) of the input signals1, 15 or a signal indicating inhalation/exhalation.11

For feature selection, the prediction error of models with varying complexity are compared as shown in Figure 2 such that when using the validation data, a minimum value for the validation error will correspond to the optimal features. Moreover, as shown in Figure 2, the shrinkage parameter with the lowest prediction error in the validation data is selected. Features and the shrinkage parameter should not be selected on the basis of the training data because the training data will result in overfitting, yielding a lower training error that is not intuitive, as shown in Figure 2.

Figure 2.

Feature and shrinkage parameter selection: the figure represents the procedure for selecting the features (left) and shrinkage parameter λ (right) for the chosen regression algorithms. It also shows that the selection should be based on the validation data

2.2. In vivo experiment

2.2.1. Overview

In this section, the suggested approach was validated by conducting experiments on human subjects. Figure 3(A) presents an overview of the experimental setup. The presented study involved two markers placed on the subject's abdomen and tracked using a digital camera outside the MRI cage. A multichannel spine coil (Esaote SpA, Genoa, Italy) was placed around the subject's thorax. The surrogate and motion data were acquired simultaneously at different rates. Subsequently, the 3 motion models were trained offline.

Figure 3.

Hardware setup. A, Overview of the setup of the in vivo experiments. B, The subject is in a supine position with hands behind the head. The subject was adjusted such that the liver is centred. C, Two markers were placed on the subject's abdomen. The markers are placed 5 cm to the right of the subject's umbilicus. Marker M2 coincides horizontally with the subject's umbilicus and marker M1 is 5 cm above M2. D, The camera is placed at the door of the MRI cage, which is approximately 2 m from the centre of the MRI bore, facing the subject

2.2.2. Measurement protocol

Three healthy subjects (2 males and 1 female) participated in the experiments. The subjects were not obese and their ages were in the range 20–28 years. Each subject signed informed consent before the start of the experiment. Each subject was subjected to two sessions of 3 minutes each. Because of the short scanning duration and the small magnetic field, the experiment involved no risk to the subjects. According to Dutch legislation no medical ethical approval was necessary for this specific research project due to its explorative nature and minimal risks. MRI safety of the subjects was safeguarded by an MRI screening procedure. The experiments were conducted at the University of Twente (Enschede, the Netherlands). During the scan, each subject was requested to breathe normally (without any previous training) and to try not to move during the scan.

2.2.3. Hardware setup

The MRI system used was a 0.25 T open‐bore system (G‐scan Brio, Esaote SpA, Genoa, Italy)(as shown in Figure 3(B)). The subject was placed on the MRI table in a supine position with his/her hands behind his/her head. The subject was positioned such that the centre of the field of view was approximately at the border of the liver.

Two 2 cm diameter 3D printed plastic spherical markers were placed on the subject's abdomen as shown in Figure 3c. A preliminary experiment was conducted to track the markers in several locations such that the locations with the highest range of motion (excursion) were chosen.

An industrial camera (MVBlueFox3, Matrix Vision GmbH, Oppenweiler, Germany) was chosen with a suitable lens (25 mm, 1.4 focal length). The camera was placed outside the MRI cage (approximately 2 m from the centre of the MRI bore) facing the subject (as presented in Figure 3(D)).

2.2.4. Data acquisition

The MRI sequence is a 2D balanced steady‐state free precession (2D HYCE S), imaging plane = sagittal, slice thickness = 15 mm, repetition time = 7 ms, echo time = 3.5 ms, reconstructed resolution = 1.5 mm × 1.5 mm, flip angle = 40° , field of view = 38 × 38 cm², temporal resolution approximately 1 fps. The camera acquired frames at 10 fps with a spatial resolution of 0.15 mm. The region of interest of the camera was 20 × 20 cm². The time stamp of each acquired frame from the MRI and the camera was recorded for temporal alignment and synchronization.

2.3. Post‐processing

2.3.1. Workflow

The workflow is shown in Figure 4. The MRI and camera images were segmented offline and the liver and abdominal motion were extracted. Moreover, the extracted signals were processed and split into 3 datasets (training, validation and test data). The training and validation data were utilized to train the motion model while the test data of the abdominal motion were fed to the motion model. Consequently, the estimated liver motion was evaluated and compared against the actual liver motion.

Figure 4.

Workflow of the suggested approach. Firstly, the sagittal MRI liver images and camera‐acquired abdominal motion were acquired simultaneously. The acquired images were segmented and the liver and marker motion were extracted. Additionally, the acquired data are processed and split for designing and evaluating the motion model

2.3.2. Image processing

The video streams were segmented offline. Each DICOM frame from the MRI was segmented using a suitable threshold value (for each volunteer) together with a series of morphological operations to detect the upper border of the liver. Only the SI motion was detected since the acquired spatial resolution (1.5 mm × 1.5 mm) was too low to detect the respiratory‐induced anterior–posterior (AP) motion (which is approximately 2 mm7). The segmented SI positions were computed relative to the reference frame as shown in Figure 5(A). In a similar manner, each frame from the camera was segmented using a suitable threshold value (for each volunteer) together with a series of morphological operations to detect the markers. The markers' SI and AP centroid positions were computed relative to their relative frame as shown in Figure 5(B).

Figure 5.

Post‐processing: the figure represents the acquired frames from MRI and camera and the segmented output. The markers were segmented from the camera while the liver's border was segmented from the MRI. The corresponding superior–inferior (SI) and anterior–posterior (AP) axes of the markers and the liver are presented at each segmented frame. According to the defined frames, inhalation corresponds to a negative SI and AP for the markers and positive SI and AP for the liver

2.3.3. Signal processing

After image segmentation, the markers' positions (SI and AP positions of the two markers) and the liver's SI position were processed. A suitable low‐pass filter was designed for the two signals. The signals were resampled in order to obtain the same sample rate. Temporal synchronization was performed in two steps. Firstly, the two signals were aligned according to their relative time stamps. Secondly, the two signals were aligned using correlation analysis to correct for the time delay.

2.4. Model evaluation

To evaluate the performance of the models used, the mean absolute error (MAE) and the adjusted coefficient of determination (adjusted R squared, $R_{\text{adj}}^2$) are calculated as follows:

$$\text{MAE}=\frac{{\sum }_{i=1}^N|Y_i-{Ŷ}_i|}{N}$$ (5)

$${\mathrm{R}}_{\text{adj}}^2=1-\frac{\text{RSS}.(n-1)}{\text{TSS}.(n-k)}$$ (6)

where N is the number of test data and k is the number of coefficients (β) including the intercept. Y _i and ${Ŷ}_i$ are the actual (ground truth) and estimated liver motion from the test data respectively. Moreover, RSS and TSS are the residual sum of squares and total sum of squares respectively, and are calculated as follows:

$$\text{RSS}={\displaystyle \sum _{i=1}^N}{(Y_i-{Ŷ}_i)}^2$$ (7)

$$\text{TSS}={\displaystyle \sum _{i=1}^N}{(Y_i-\overline{Y})}^2$$ (8)

where $\overline{Y}$ is the mean value of Y. Note that $R_{\text{adj}}^2$ is an indication of the goodness of fit of the model, which could be any value less than or equal to 1 such that $R_{\text{adj}}^2$ closer to 1 indicates a better fit.39 Furthermore, MAE is used to obtain a quantitative result of the predicted response. The smaller MAE is, the smaller is the test error.

3. RESULTS

3.1. Power density analysis

Figure 6 represents the normalized power density of the markers and the liver (SI) signals for all the sessions. Comparing the plots in the same row allows the assessment of intrafractional variations in each patient, while the columns allow the assessment of the variations between patients. As shown in Figure 6, the liver and marker signals were dominated by the breathing frequencies.42 The figure also shows spikes at low frequencies (smaller than 0.1 Hz), which indicate the presence of noise or drift in positions. It can also be seen that each subject had a different breathing pattern, thus the model has to be patient‐specific. Moreover, Figure 6 shows that the breathing patterns are not identical for each subject over the two sessions, which implies that the model should be updated for each session (intrafractionally). Note that according to the obtained data, the signals were not affected by other organ‐induced motion such as the heartbeat (operating over 1 Hz42).

Figure 6.

Power density analysis: the figure shows the power density (periodogram) of the liver SI motion (black solid line) and the markers' motion (gray dotted line) for all the subjects. The figure shows that the signals are dominated by breathing frequencies below (0.5 Hz). However, there are spikes at low frequencies (below 0.1 Hz)

3.2. Motion and surrogate data analysis

Figure 7 and Table 2 present the processed data from the subjects. Figure 7 presents the mean (μ) and the standard deviation (σ) plots of the peak‐to‐trough (PTT) motion for the liver's SI motion and the markers' SI and AP motion. Moreover, Table 2 presents the values of the plots shown in Figure 7. As shown in Figure 7 and Table 2, the motion

Figure 7.

Motion data analysis: mean μ and standard deviation σ of peak‐to‐trough (PTT) motion for the liver's SI motion and the markers' SI and AP motion. The mean and standard deviations are summarized in Table 2

Table 2.

Motion data analysis: mean μ and standard deviation σ (both in mm) of the peak‐to‐trough motion for the liver SI motion and the markers' SI and AP motion

Subject, Session	SI (μ ± σ)	Marker 1, SI (μ ± σ)	Marker 1, AP (μ ± σ)	Marker 2, SI (μ ± σ)	Marker 2, AP (μ ± σ)
Subject 1, session 1	11.8 ± 1.6	0.5 ± 0.1	3.7 ± 0.6	0.5 ± 0.2	3.3 ± 0.6
Subject 1, session 2	10.0 ± 1.8	0.4 ± 0.1	3.3 ± 0.7	0.4 ± 0.1	2.9 ± 0.7
Subject 2, session 1	11.1 ± 1.1	1.3 ± 0.2	2.5 ± 0.4	1.3 ± 0.2	2.5 ± 0.4
Subject 2, session 2	10.3 ± 1.2	1.2 ± 0.2	2.5 ± 0.3	0.2 ± 0.1	2.8 ± 0.4
Subject 3, session 1	19.2 ± 3.0	1.6 ± 0.3	11.7 ± 2.9	1.6 ± 0.3	11.7 ± 2.9
Subject 3, session 2	21.3 ± 2.1	1.9 ± 0.3	13.9 ± 1.8	3.2 ± 0.4	11.5 ± 1.5

data showed the range of motion expected from previous studies.7 The greatest PTT motion of the liver and markers was observed in subject 3 while the other two subjects had approximately similar ranges of motion to each other. The maximum mean PTT was 21.3 mm for the liver's SI motion, 13.9 mm and 11.7 mm for the two markers' AP motion and 1.9 mm and 3.2 mm for the two markers' SI motion (all observed in subject 3). The minimum mean PTT was 10.0 mm for the liver's SI motion (observed in subject 1), 0.4 mm and 0.2 mm for the two markers' SI motion (observed in subjects 1 and 2 respectively) and 2.5 mm for the two marker's AP motion (observed in subject 2). In all subjects, the markers' AP motion had a greater range than their SI motion. The tables also demonstrate the variations in breathing magnitudes between the subjects and between sessions, thus confirming that the motion models should be patient‐specific.

3.3. Feature selection analysis

Nine types of features have been chosen as shown in Table 3. In order to choose the optimal feature type, each one was validated using the validation data as explained in Section 2.1.5 and Figure 2. Moreover, Figure 8 shows the normalized MAE (NMAE) against the type of feature for the training and validation data. The type of feature is directly proportional to the model complexity. Thus, the higher the feature type the higher the model order and thus the complexity. As shown in Figure 8, choosing more than one marker improved the estimation accuracy. However, marker 2 (type 2) outperformed marker 1 (type 1). Moreover, choosing a more complex model (higher order) did not improve the performance of the model on the test data. Thus, the optimal feature selected was type 3. Finally, Figure 8 also demonstrates that the selection of the feature type should be based upon validated data and not training data.

Table 3.

Feature selection: Type 1 corresponds to the least complex and type 9 to the most complex

Type	Description
Type 1	Marker 1
Type 2	Marker 2
Type 3	Marker 1 and marker 2, first‐order polynomial
Type 4	Marker 1 and marker 2, second‐order polynomial
⋮	⋮
Type 9	Marker 1 and marker 2, seventh‐order polynomial

Figure 8.

Feature selection: the figure shows the normalized MAE (NMAE) against the feature type. The plot shows that feature type #3 has the smallest NMAE and thus it is the selected one

3.4. Shrinkage parameter analysis

The shrinkage parameters (for Lasso and Ridge) were selected by validating the model performance using the validation data. Ridge and Lasso models had the same type of feature selected in Section 3.3 (type 3). Figure 9 shows the NMAE against the shrinkage parameter λ for Ridge and Lasso. Figure 9 shows the expected results as explained in Section 2.1.5 and Figure 2. However, due to the limited resolution acquired for the liver data, the feature and shrinkage analysis were more challenging to tune. According to Figure 9, the shrinkage parameter was minimal at the values (0,0.2] and (0,0.4] for Ridge and Lasso respectively. Thus, the average values of λ were taken. These values were 0.1 and 0.2 for the Ridge and Lasso algorithms respectively.

Figure 9.

Shrinkage parameter selection: the figure shows the normalized MAE (NMAE) against the shrinkage parameter λ for both Ridge and Lasso regression. The plot shows that the minimum NMAE is at approximately 0.1 for Ridge and 0.2 for Lasso

3.5. Estimation accuracy

To evaluate the designed models, the estimated liver SI motion $Ŷ$ obtained from the regression model in equation (1) was compared to the true values of the liver SI motion Y obtained from MRI. Feature type 3 (see Table 3) was chosen for the 3 fitting methods (MVR, Ridge and Lasso) and the shrinkage parameter λ was 0.1 and 0.2 for Ridge and Lasso respectively.

Figure 10 shows the plots of $Ŷ$ compared to Y using the test data acquired from the 6 sessions conducted using MVR, Ridge and Lasso. Tables 4 and  5 present the performance measures (MAE and $R_{\text{adj}}^2$ respectively) of the estimated liver SI motion $Ŷ$ that are shown in Figure 10. Each row in Figure 10 and Tables 4 and 5 corresponds to one session (out of 6). Thus, comparing the plots and values of the same row allow the assessment of the 3 designed models. In addition, each column in Figure 10 and Tables 4 and 5 corresponds to one regression method (out of 3), which allows the assessment of each regression model throughout the sessions.

Figure 10.

Evaluation results: the figure shows the estimated values of the liver SI motion ( $Ŷ$) compared with the actual liver SI test data Y against time. Each row of the figure indicates the subject (S) and session number (Se) while each column shows the regression algorithm used. The MAE, standard deviation and $R_{\text{adj}}^2$ of the graphs are summarized in tables 4 and  5 respectively

Table 4.

Evaluation results: MAE and standard deviation σ (both in mm) of the estimated liver SI motion using MVR, Ridge and Lasso

	MVR	Ridge	Lasso
Subject, Session	MAE ±σ	MAE ±σ	MAE ±σ
Subject 1, session 1	1.5 ± 1.2	1.5 ± 1.1	1.2 ± 1.1
Subject 1, session 2	1.0 ± 0.8	1.0 ± 0.7	1.0 ± 0.7
Subject 2, session 1	1.4 ± 0.8	1.4 ± 0.8	1.0 ± 0.7
Subject 2, session 2	0.8 ± 0.7	0.8 ± 0.7	0.8 ± 0.7
Subject 3, session 1	1.5 ± 1.3	1.6 ± 1.3	1.5 ± 1.2
Subject 3, session 2	1.9 ± 1.9	1.8 ± 1.9	1.8 ± 1.9

Table 5.

Evaluation results: $R_{\text{adj}}^2$ of the estimated liver SI motion using MVR, Ridge and Lasso

	MVR	Ridge	Lasso
Subject, Session	$R_{\text{adj}}^2$	$R_{\text{adj}}^2$	$R_{\text{adj}}^2$
Subject 1, session 1	0.6	0.6	0.7
Subject 1, session 2	0.7	0.7	0.8
Subject 2, session 1	0.7	0.7	0.8
Subject 2, session 2	0.8	0.8	0.8
Subject 3, session 1	0.8	0.8	0.8
Subject 3, session 2	0.7	0.8	0.8

As shown in Figure 10, the 3 algorithms tracked the ground truth values of the liver motion accurately without lagging or deviation. However, subject 3 had an observable deviation in both sessions during the first 20 seconds. As shown in Table 4, subject 3 had the highest MAE for the 3 regression methods while subject 2 had the smallest. Moreover, as shown in Tables 4 and  5, as a general trend, Lasso outperformed MVR and Ridge in 3 of the 6 conducted sessions, while Lasso and Ridge performed equally in the other 3.

4. DISCUSSION

The data acquired from the markers and the MRI showed that breathing signals vary between patients, and consequently the models designed were patient‐specific. The acquired data were not affected by other organ‐induced motion such as the heart (operating over 1 Hz42).

This is because the temporal resolution of the MRI acquisition (1 fps) would not capture such high frequencies. However, for the markers, even though the temporal resolution is able to capture heartbeats, the markers' signals were not affected. The liver SI motion ranged between 10.0 mm and 21.3 mm, which is consistent with previous studies on the liver.10, 21 The marker motion was dominant in the AP (ranging between 2.5 mm and 13.9 mm) rather than the SI direction (ranging between 0.2 mm and 3.2 mm). The results also indicated a strong linear relationship between the abdominal motion and the liver's SI motion. Furthermore, higher order models did not improve the overall fit of the data. The results show that using only one marker would result in a lower accuracy than using two markers, confirming the results of Ernst et al. 15 Moreover, the location of the marker affects the estimation accuracy: the closer the marker was to the umbilicus the higher the estimation accuracy. The results also illustrated the accuracy of the estimated SI liver motion; the MAE ranged between 0.8 mm and 1.9 mm for the 3 regression models. The results demonstrated performances consistent with previous research studies on the liver using external markers,15, 24 ultrasound4 and accelerometers as a surrogate.13 Additionally, Lasso generally outperformed MVR and Ridge in the overall estimation accuracy. However, the spatial resolution of the acquired MRI liver images prevented a more detailed evaluation of the 3 models.

5. CONCLUSION

An RME approach to estimate the liver SI motion due to respiration was developed and evaluated. External markers placed on the human's abdomen were tracked using a digital camera. Abdominal motion tracking was chosen as surrogate data due to its MRI compatibility, comfort for the patient, high temporal and spatial resolution and high correlation with the liver motion. The motion data (liver) and the surrogate data (markers) were used to fit supervised learning regression models that were subsequently used to estimate the motion data based on the surrogate data. The supervised learning models in RME were assessed by analyzing the features and shrinkage parameters. The suggested approach was validated by in vivo experiments that showed that the markers succeeded in estimating the liver motion with a good accuracy (below 2 mm).

The proposed approach was developed with hepatic percutaneous interventions in mind. During interventions, the clinical operator relies on offline MRI images that are acquired prior to intervention. Thus, the respiratory‐induced motion remains uncertain and the operator relies on his/her experience to roughly estimate the motion. Our workflow is essential to guide the operator during intervention. First, with the subject in the MRI room prior to intervention, the motion and surrogate data are acquired simultaneously. Once the training phase is completed (i.e., the motion model is formulated), the subject is requested to leave the MRI room, and the motion data are estimated based solely on the tracked abdominal motion. With the advancements in adjustable MRI dockable tables,43 we envision the subject remaining on the MRI table with the markers attached. The markers could also be detached and reattached if their locations are marked with a temporary marker. With that said, the motion estimates could exploited as a feedback to the clinical operator as either raw data, a gated signal8 or synthetic MRI images.4 The motion estimate could also be fed back to a surgical robot for motion compensation. We envision a surgical robot for percutaneous interventions that corrects its reference motion trajectory by the estimated respiratory‐induced motion.

Our proposed approach was tested in an open‐bore MRI system in which the sagittal plane of the subject was clear, as shown in Figure 3(B). Tracking the markers from the sagittal plane is not essential. For instance, if close‐bore MRI is used, the transverse plane could also be used since this plane is clearer. In a more general case, the 3D motion of the markers could be tracked by using stereo cameras. Additionally, multiple cameras could be used for more accurate tracking, while IR cameras could be used under sensitive light conditions. In fact, with the technological advancements in computer vision and motion capture, several tracking options are commercially available and could be augmented in our approach.44

In this study, a low‐field MRI system was used to acquire the motion data, which resulted in a low temporal (1 fps) and spatial resolution (1.5 mm × 1.5 mm) that limited the overall evaluation of the proposed framework. The most crucial improvement to the proposed framework is using an MRI scanner that operates at a higher field strength. Acquiring images at a higher temporal resolution can significantly affect the performance since more data samples can be acquired. Moreover, acquiring images at a higher spatial resolution allows the investigation of other liver motions (AP and lateral motions) that were not investigated due to the low spatial resolution and the relatively small motion in these directions. Obtaining higher spatial resolution of the motion data will result in a finer tuning of feature and shrinkage parameters since the minimum values of these parameters are sharper.

As illustrated in the results, the differences between the 3 regression models were not significant. Thus, using a more complex regression algorithm would not improve the estimation accuracy significantly.27 However, since shrinkage methods might outperform MVR in the case of high dimensional data, using more than two markers will increase the system dimensionality and thus might improve the estimation accuracy in Ridge and Lasso. In fact, multiple markers have been used in previous studies, which indicated that their use will increase the system dimensionality and might capture more complex breathing patterns.15, 27

Finally, it is necessary to test the proposed approach on a wider set of subjects with more varied breathing patterns, age range and weight. The robustness of our proposed approach should also be evaluated on unhealthy subjects and sedated subjects.

Fahmi S, Simonis FFJ, Abayazid M. Respiratory motion estimation of the liver with abdominal motion as a surrogate. Int J Med Robotics Comput Assist Surg. 2018;14:e1940 10.1002/rcs.1940