Adversarial Bayesian Optimization for Quantifying Motion Artifact Within MRI

Abstract

Subject motion during an MRI sequence can cause ghosting effects or diffuse image noise in the phase-encoding direction and hence is likely to bias findings in neuroimaging studies. Detecting motion artifacts often relies on experts visually inspecting MRIs, which is subjective and expensive. To improve this detection, we develop a framework to automatically quantify the severity of motion artifact within a brain MRI. We formulate this task as a regression problem and train the regressor from a data set of MRIs with various amounts of motion artifacts. To resolve the issue of missing fine-grained ground-truth labels (level of artifacts), we propose Adversarial Bayesian Optimization (ABO) to infer the distribution of motion parameters (i.e., rotation and translation) underlying the acquired MRI data and then inject synthetic motion artifacts sampled from that estimated distribution into motion-free MRIs. After training the regressor on the synthetic data, we applied the model to quantify the motion level in 990 MRIs collected by the National Consortium on Alcohol and Neurodevelopment in Adolescence. Results show that the motion level derived by our approach is more reliable than the traditional metric based on Entropy Focus Criterion and manually defined binary labels.

1. Introduction

Due to the relatively long scanning time required for MRI sequences, subject motion creates one of the most common artifacts in MR imaging that results in blurring and ghosting in the image [1]. Fast and accurate quantification of motion artifacts within MRIs, therefore, becomes critical in most clinical settings as it generates real-time feedback on whether a re-scan is needed while the subject is still in the scanner, thereby reducing the need for scheduling additional scanning sessions [2]. Moreover, in large-scale neuroimaging studies, such quantification can be used to minimize the risk of motion-related confounders biasing findings by either confining analysis to MRIs with minimum motion artifacts or controlling for the level of motion artifact in statistical models [3,4].

In most existing neuroimaging studies, detection of motion artifacts in T1-weighted MRI relies on experts visually reviewing scans and labeling an MRI as either motion-free or with motion artifacts [5]. Such binary (or categorical) labeling is often subjective and inaccurate, especially when the scans are from multiple sites with various imaging protocols. An alternative to manual labeling is to use Entropy Focus Criterion (EFC) [6] to quantify the severity of motion artifact, but this metric is only suitable for detecting large-scale motions. To improve the quantification, prior works have tried to automate motion detection based on machine learning models. Lorch et al. [2] proposed a framework based on Random Forests to detect head and respiratory motion in MRIs, but their framework was only evaluated on synthetic data. Other works [7,8] designed a convolutional neural network to classify image patches and then used a special voting mechanism to determine the presence of motion artifact in the entire MRI. However, these classification models always assume availability of accurate ground-truth labels, which, as mentioned above, are difficult and expensive to produce for large multi-site studies. Moreover, these models only provide binary predictions, which cannot quantify detailed severity of the artifact. To resolve these issues, we aim to develop a regression framework to quantify the fine-grained level of motion artifact in an MRI via a convolutional neural network.

We are specifically interested in designing a training scheme to learn the regression network based on binary (presence or absence of artifact) and potentially noisy ground-truth labels derived by manual labeling. To generate training data with accurate and continuous labels (level of artifact), we injected synthetic artifacts induced by rigid motion into motion-free MRIs. To do so, we first developed a novel method called Adversarial Bayesian Optimization (ABO) to infer the distribution of motion parameters, i.e., the range of rotation and translation in 3D, underlying the motion observed in the real data set. We then generated a simulated data set with synthetic artifacts controlled by the estimated motion range, so that the regressor network could be trained in a traditional supervised fashion. We tested the proposed framework to quantify the severity of motion artifact in 990 T1-weighted MRIs from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) [9]. Results indicate that the proposed ABO could accurately estimate the range of motion parameters underlying a given data set. The regressor trained based on this estimated range more accurately quantified the level of motion in MRIs compared to traditional metrics based on EFC.

2. Methods

Let ${\mathcal{X}}_c$ be a set of motion-free MRIs and ${\mathcal{X}}_m$ be a set of MRIs with various level of motion, our goal is to derive a function q, such that for an MRI $x\in {\mathcal{X}}_m$, y = q(x) quantifies the level of motion. To train the regressor q without ground-truth y, a typical approach is to rely on data simulation. Here, we leverage the simulator function $\widehat{\mathit{\text{x}}}=s(\mathit{\text{x}};\mathit{\tau })$ proposed in [10]. The simulator can inject “k-space” motion artifacts controlled by a set of parameters τ (encoding rotation and translation of rigid motion) into a motion-free image $\mathit{x}\in {\mathcal{X}}_c$ to generate a synthetic motion-corrupted MRI $\widehat{\mathit{x}}\in {\widehat{\mathcal{X}}}_m$. As the ground-truth motion of $\widehat{\mathit{x}}$ is known (related to τ), one can learn q using traditional supervised training.

However, due to the variability in imaging protocols and characteristics of the study cohorts, one typically does not know the range of motion in ${\mathcal{X}}_m$, so the model trained on the simulated set ${\widehat{\mathcal{X}}}_m$ might not generalize well. To resolve this issue, we propose the following 2-step training strategy (Fig. 1): (1) infer the underlying distribution of motion parameters p(τ) associated with the real motion artifacts in ${\mathcal{X}}_m$; and (2) train the regressor q on a simulated data set ${\widehat{\mathcal{X}}}_m$, where τ (and also y) for each simulated MRI is sampled from p(τ).

Fig. 1.

Model Architecture: Step 1. Adversarial Bayesian Optimization (ABO) is used to infer the distribution of motion parameters underlying the real data set with motion artifacts ${\mathcal{X}}_m$; Step 2. A regressor is trained on a simulated data set, whose motion artifacts are sampled from the distribution estimated in Step 1.

2.1. Inferring Distribution of Motion Parameters via Adversarial Bayesian Optimization

Let τ = {τ₁, …, τ₆} be the amount of rotation and translation of the simulated motion in the 3 spatial directions, and let θ = {θ₁, …, θ₆} be the maximum transformation in those 6°. We then assume the transformation parameters follow independent uniform distributions $p(\tau ;\theta )={\prod }_{k}p\left({\tau }_k;{\theta }_k\right)={\prod }_k\mathcal{U}\left(-{\theta }_k,{\theta }_k\right)$. As such, p(τ; θ) can be used to characterize a data set with various level of motion artifacts.

We now aim to infer the maximum transformation θ associated with ${\mathcal{X}}_m$. We do so by adversarially training a discriminator network D with parameters ϕ, such that it can not distinguish the real data ${\mathcal{X}}_m$ from simulated data ${\widehat{\mathcal{X}}}_m$, which are generated by injecting synthetic motion (sampled from p(τ; θ)) into motion-free MRIs ${\mathcal{X}}_c$. This objective function can be written as

$$\underset{\mathit{\theta }}{\text{max}}\underset{\mathit{\varphi }}{\text{min}}{\mathbb{E}}_{\mathit{\text{x}}~{\mathit{\text{X}}}_m}\left[\text{log}{D}_{\mathit{\varphi }}(\mathit{x})\right]+{\mathbb{E}}_{\mathit{\tau }~p(\mathit{\tau };\mathit{\theta }),\mathit{x}~{\mathcal{X}}_c}\left[\text{log}\left(1-D_{\mathit{\varphi }}(s(\mathit{x};\mathit{\tau }))\right)\right].$$ (1)

As typically performed in adversarial learning, this loss function can be alternatively optimized with respect to θ and ϕ. To update ϕ given θ, we first simulate ${\widehat{\mathit{X}}}_m$ from X_c according to p(τ; θ) and then train the discriminator D to differentiate ${\widehat{\mathit{X}}}_m$ from X_m. To update θ given a trained discriminator D, we optimize θ so that the discriminator cannot distinguish between real and simulated data. We do so by maximizing the following function to fool the discriminator.

$$f(\mathit{\theta })={\mathbb{E}}_{\mathit{\tau }~p(\mathit{\tau };\mathit{\theta }),\mathit{x}~{\mathcal{X}}_c}\left[\text{log}\left(1-D_{\mathit{\varphi }}(s(\mathit{x};\mathit{\tau }))\right)\right].$$ (2)

However, a challenge in the optimization of Eq.(2) is that the gradient of f(θ) is hard to evaluate (due to the k-space transformation of x in s(x; τ)), so traditional back-propagation can not be directly applied. Here, we resort to Bayesian optimization to optimize Eq. (2).

Bayesian Optimization.

[11] is a machine-learning-based approach for finding the maximum of f without estimating its gradient. To do so, Bayesian optimization formulates a probability distribution over the functional f(·) as a Gaussian Process and repeatedly evaluates function values at various points. Let Θ = [θ¹, …, θ^t] be the first t evaluated points, and f = [f(θ¹), …, f(θ^t)] be their functional values. Let k(·,·) denote the covariance function [12] associated with the Gaussian Process. Then the functional value at an unobserved point θ* follows a normal distribution $f\left({\mathit{\theta }}^{*}\right)~\mathcal{N}\left(\mu \left[{\mathit{\theta }}^{*}\right],{\sigma }^2\left[{\mathit{\theta }}^{*}\right]\right)$, where

$$\mu \left[{\mathit{\theta }}^{*}\right]=K\left({\mathit{\theta }}^{*},\Theta \right)K{(\Theta ,\Theta )}^{-1}\mathit{f}$$ (3)

$${\sigma }^2\left[{\mathit{\theta }}^{*}\right]=K\left({\mathit{\theta }}^{*},{\mathit{\theta }}^{*}\right)-K\left({\mathit{\theta }}^{*},\Theta \right)K{(\Theta ,\Theta )}^{-1}K\left(\Theta ,{\mathit{\theta }}^{*}\right),$$ (4)

Here, K(Θ, Θ) is a t×t matrix with each element (i, j) recording k(θⁱ, θ^j), and likewise K(θ*, Θ) is a t × 1 vector. Given such a probability distribution, an acquisition function is used to select the next evaluation point with the highest chance to get closer to the global maximum. Here we use the Upper Confidence Bound (UCB) [12] as the acquisition function

$$\text{arg}\underset{{\mathit{\theta }}^{*}}{\text{max}}\text{UCB}\left({\mathit{\theta }}^{*}\right)=\mu \left[{\mathit{\theta }}^{*}\right]+\beta \sigma \left[{\mathit{\theta }}^{*}\right];$$ (5)

That is, the UCB favors regions with either high expected values μ[θ*] or high uncertainty (for exploring scarcely evaluated regions). As such, we repeatedly use Eqs. (3), (4), and (5) to evaluate new points until reaching a certain number of iterations. Lastly, we embed the Bayesian optimization strategy into the adversarial training scheme, resulting in the Adversarial Bayesian Optimization (see Algorithm 1).

2.2. Training a Regressor for Quantifying Level of Motion

We now simulate the data set ${\widehat{\mathcal{X}}}_m$ to train the regressor q. To generate a simulated $\widehat{\mathit{\text{x}}}$ and its corresponding ground truth y, we first randomly select $\mathit{x}\in {\mathcal{X}}_c$ and sample y from $\mathcal{U}(0,1)$. Then we construct $\widehat{\mathit{x}}=s(\mathit{x};y\mathit{\theta })$, i.e. an image with motion artifact at level y, and the maximum possible motion is guaranteed to be θ. Next, we train q on the simulated image and label pairs. As we intend to use q to rank the motion level across images, we design the training loss function as a combination of MSE loss and a rank loss [13]. Given two training images ${\widehat{\mathit{x}}}_1$, ${\widehat{\mathit{x}}}_2$ with y₁ < y₂, the objective function is

$$\text{arg}\underset{q}{\text{min}}{\left(y_1-q\left({\widehat{\mathit{x}}}_1\right)\right)}^2+{\left(y_2-q\left({\widehat{\mathit{x}}}_2\right)\right)}^2+\text{ReLU}\left(q\left({\widehat{\mathit{x}}}_1\right)-q\left({\widehat{\mathit{x}}}_2\right)\right),$$ (6)

which encourages the predictions to follow the correct rank order.

3. Experimental Setup

Data Set.

We used the T1-weighted MRI from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) [9,14], which acquired MRIs at 5 different sites across the US (denoted in the following as Site A through Site E). Up to year 6 of the study, all MRIs went through manual quality control, and 159 MRIs were labeled as ‘unusable’ due to motion artifacts, which were regarded as the real motion data set ${\mathcal{X}}_m$. To construct ${\mathcal{X}}_c$, we first collected the 831 MRIs from all subjects that were labeled as ‘usable’ by manual quality control¹. Considering imperfect human labeling, we selected 200 MRIs with lowest Entropy Focus Criterion (EFC) as ${\mathcal{X}}_c$. All MRIs analyzed in this study had a resolution of 256×256×146 or 256×256×160 but were downsized to the size of 128 × 128 in the axial plane and cropped in the z-direction to reserve 128 axial slices containing the brain region.

Validation.

To test if the proposed ABO can accurately imply the range of motion parameters associated with a data set, we applied it to simulated data sets with known ground-truth θ. We did so by setting all the 6 parameters {θ₁…θ₆} to an equal number and varied this number in the range of {0.1, 0.2, 0.5, 1, 2, 4}. We then ran the optimization using a search range of [0, 5] for each motion parameter, initialized θ with 5 searches in that range, and performed additional search 45 times with β = 2.58 in the UCB, which decayed by a factor of 0.95 after the 5^th iteration. After this validation, we applied ABO to infer θ associated with the real motion set ${\mathcal{X}}_m$.

Next, given the estimated θ, we simulated ${\widehat{\mathcal{X}}}_m$ containing 200 synthetic MRIs with ground-truth y. We additionally simulated 200 MRIs as a separate validation data set. To train the regressor q, we designed a simple feed-forward convolutional neural work containing 4 double convolutional layers of feature-channel dimension increasing from 32 to 256 with ReLU activations and max-pooling layers in between. Each convolutional layer used a kernel size of 3 × 3, stride of 1, and zero padding. Adam optimizer was used with a learning rate of 1e-4. After training, we applied q to the validation set to measure the prediction accuracy of y based on R² coefficients. We also compared the proposed loss function in Eq.(6) with the vanilla regression simply based on the MSE loss. Finally, we applied q to quantify the level of motion for the real data in ${\mathcal{X}}_m$ and all usable baseline scans of NCANDA. The code for ABO and regression is available at https://github.com/anastasb/MRI-Motion-Artifact-Quantification.

4. Results

Results on Adversarial Bayesian Optimization.

Figure 2 shows the inference of θ for synthetic data set ${\widehat{\mathcal{X}}}_m$ with known ground-truth θ and for real data ${\mathcal{X}}_m$. When ${\widehat{\mathcal{X}}}_m$ was constructed by ground-truth θ = 0.2, all 6 estimated parameters converged around the ground-truth value with diminishing variance (Fig. 2(a)). The results replicated when we tested the optimization on ${\widehat{\mathcal{X}}}_m$ with a larger ground-truth θ = 4 (Fig. 2(b)). The first 6 columns in Fig. 2(d) indicate that the proposed ABO approach consistently recovered the correct range of motion θ for data sets with various ground-truth θ.

Fig. 2.

(a–c). Estimated range of motion θ along iterations of Adversarial Bayesian Optimization (ABO) for synthetic and real data sets. (d) Distribution of the 6 estimated motion parameters over the last 5 iterations of ABO for synthetic (blue) and real (orange) data sets.

Next, we applied ABO to the real motion data set ${\mathcal{X}}_m$ to infer its underlying motion parameters θ (Fig. 2(c)), and the estimated parameters converged around 0.5. We also observe from Fig. 2(d) that the distribution of the 6 estimated parameters (over the last 5 iterations) for the real set ${\mathcal{X}}_m$ was comparable to the distribution for the simulated set ${\widehat{\mathcal{X}}}_m$ with ground-truth θ = 0.5. We therefore set θ = 0.5 for all 6 motion parameters for the real motion set ${\mathcal{X}}_m$.

Results on Motion Level Regression.

After training q on a simulated set ${\widehat{\mathcal{X}}}_m$ with θ = 0.5, we tested it on the hold-out set of 200 simulated test images. Figure 3 shows that the predicted motion level generally aligned with the ground truth (R² = 0.52), although the prediction seemed to be overly sensitive for small motion artifacts (e.g., < 0.2). However, this result was still more accurate than the vanilla regression model that only incorporated the MSE loss but without using the ranking loss (R² = 0.43).

Fig. 3.

Inference of motion level on the validation set.

Finally, we applied the trained regressor to estimate the level of motion for all 990 MRIs. Figure 4 displays MRIs from each site with the highest and lowest motion score. We then show the estimated scores for both usable MRIs and MRIs with motion across 5 sites in Fig. 5(c), and compared them with the scores defined by traditional metrics for motion quantification based on EFC (Fig. 5(a)+(b)). Note that the scale of motion level was incommensurate across approaches. We observe that the EFC-based scores exhibited a large variance across sites, which was significantly larger than the group difference between the usable and motion groups. This finding rendered the EFC metrics unsuitable for multi-site studies. On the other hand, the cross-site variance was largely reduced by our approach.

Fig. 4.

MRIs from each site with the highest and lowest motion score derived by our regression model.

Fig. 5.

Motion scores for the 990 MRIs of NCANDA defined by (a) EFC, (b) adjusted EFC (EFC computed over voxels with positive intensities), and (c) our model prediction. Our model results in the largest group separation in the scores for Sites A, B, and D, and has less variance across sites.

Compared to the EFC-based scores, the scores derived by the regressor network had a significantly larger separation between the usable and motion groups for sites A, B, and D (p < 0.01 two-way ANOVA). This indicates our approach could better quantify motion artifacts within MRIs. Note, although we expected a significant separation between the two groups, we did not regard the binary group assignment as ground-truth due to errors in manual quality control. This was supported by Fig. 6 showing two example MRIs defined as ‘usable’ in quality control but received high motion scores by our model. A follow-up visual inspection concluded that there were substantial motion artifacts (e.g., ringing effects) within these MRIs. Therefore, the overlap in the motion scores between the usable and motion groups across all sites potentially reflected the ambiguity and imperfection of manual labeling. Such noisy manual labels might prohibit effective training of supervised binary classification and supported that our regression-based framework might be more effective in quantifying motion artifacts.

Fig. 6.

Examples of MRIs which manual quality control labeled as usable but our estimator gave a high motion score.

5. Conclusion

We presented a framework for quantifying the severity of artifact induced by subject motion in T1-weighted MRIs. Being a part of the framework, we proposed Adversarial Bayesian Optimization, a novel approach for performing adversarial training when the generator component (i.e., the motion simulator in our context) is non-differentiable. The proposed motion-quantification framework can be used either as a first or second rater in quality control workflows. Here we used it as a second rater and found some instances of motion artifact which escaped manual quality control (Fig. 6). Our future research direction is to develop a method to improve the quality of MRIs with motion artifacts. In other words, given a motion-corrupted MRI, we aim to reconstruct the corresponding motion-free MRI. For that, the method presented herein was a prerequisite.