Unsupervised Deep Learning with Convolutional Neural Networks for Static Parallel Transmit Design: A Retrospective Study

Abstract

Purpose:

To mitigate B₁⁺ inhomogeneity at 7T for multi-channel transmit arrays using unsupervised deep learning with convolutional neural networks (CNNs).

Methods:

Deep learning parallel transmit (pTx) pulse design has received attention, but such methods have relied on supervised training and did not use CNNs for multi-channel B₁⁺ maps. In this work, we introduce an alternative approach that facilitates the use of CNNs with multi-channel B₁⁺ maps while performing unsupervised training. The multi-channel B₁⁺ maps are concatenated along the spatial dimension to enable shift-equivariant processing amenable to CNNs. Training is performed in an unsupervised manner using a physics-driven loss function that minimizes the discrepancy of the Bloch simulation with the target magnetization, which eliminates the calculation of reference transmit RF weights. The training database comprises 3824 2D sagittal, multi-channel B₁⁺ maps of the healthy human brain from 143 subjects. B₁⁺ data were acquired at 7T using an 8Tx/32Rx head coil. The proposed method is compared to the unregularized magnitude least-squares (MLS) solution for the target magnetization in static pTx design.

Results:

The proposed method outperformed the unregularized MLS solution for root-mean-square error and coefficient-of-variation and had comparable energy consumption. Additionally, the proposed method did not show local phase singularities leading to distinct holes in the resulting magnetization unlike the unregularized MLS solution.

Conclusion:

Proposed unsupervised deep learning with CNNs performs better than unregularized MLS in static pTx for speed and robustness.

1. Introduction

Ultra-high field (UHF) MRI offers higher signal-to-noise ratio (SNR) (1,2) with potential for better spatiotemporal resolutions and has found extensive use in a numerous applications including functional neuroimaging (3,4). However, the usage of higher static magnetic fields (B₀) leads to a radiofrequency (RF) magnetic (B₁⁺) field with a lower wavelength, which then becomes comparable to the size of the imaging targets (5). This results in greater non-uniformity of the flip angle distributions at UHF (6). Image artifacts are caused by this non-uniformity and must be eliminated for better image evaluation. B₁⁺ fields are subject-dependent and therefore subject-specific correction is required (7,8).

Parallel transmission (pTx) has been instrumental for handling B₁⁺ inhomogeneities at UHF (9–13). Prior to scanning, many pTx methods solve an optimization problem to generate the associated RF pulse for a target magnetization magnitude, which builds on a magnitude least-squares (MLS) objective function (3,9,14,15). However, since this calculation is time-consuming and needs to be done when the patient already lies in the scanner, there is interest in speeding up this process (16). One such approach is the use of subject-independent methods, such as the universal pulses (17). In this setup, the optimization problem is solved for a target magnetization over an existing database of B₁⁺ maps from distinct subjects. Thus, this strategy aims to produce sufficient B₁⁺ homogeneity for the training set of subjects, but its performance may be suboptimal compared to subject-specific methods (18). Hybrid methods have been proposed to improve this strategy by taking subject-specific information into account while using universal pulses as a starting point for optimization (19,20). More recently, machine learning methods have been proposed as an alternative to these optimization techniques (21–23). A SAR-efficient method uses kernelized ridge regression to learn B₁⁺ shimming weights (21). In another work, a fully-connected neural network is proposed to find B₁⁺ shimming weights (22). However, fully-connected neural networks require more tunable parameters than convolutional neural networks (CNNs) of similar depth and also cannot handle different input sizes. Therefore, CNNs are used in a subject-specific pulse design that predicts 2DRF pulses for a single channel B₁⁺ map (23). Other deep learning (DL) methods for indirectly enhancing pTx RF pulse design include generative adversarial networks to predict B₁⁺ distributions in the head following displacement (24). Multi-channel B₁⁺ maps have been utilized for a classification type approach with CNNs (25) for better RF pulse initialization. Additionally, reinforcement learning has been applied to RF pulse design (26). However, to the best of our knowledge, there are no works that use CNNs to directly output RF pulses with multi-channel B₁⁺ maps as inputs. Furthermore, the aforementioned methods have been trained using supervised learning, where a reference pTx RF waveform was calculated using computationally costly optimization algorithms across the whole training database.

In this work, we propose a strategy for unsupervised deep learning for pTx using CNNs with multi-channel B₁⁺ maps as input. We evaluate its feasibility in static pTx design at 7T. The proposed method outperforms the traditional unregularized MLS method quantitatively in terms of root-mean-square error (RMSE), and coefficient-of-variation (CoV). Our method has comparable excitation energy demand. Furthermore, the proposed method improves upon unregularized MLS in reducing nulls in the flip angle maps.

2. Methods

Imaging Data and Pre-processing:

B₁⁺ maps were obtained at University Hospital (Universitätsklinikum) in Erlangen, Germany with approval from the local Ethical Review Board. In vivo brain imaging was performed on 143 healthy subjects utilizing a 7T MAGNETOM Terra whole-body MR system (Siemens Healthcare, Erlangen, Germany) with an 8Tx/32Rx head coil (Nova Medical, Wilmington, MA), which led to 3824 2D sagittal slices that were used for the training database.

Several standard pre-processing steps were applied to B₁⁺ maps to preserve lower and peripheral regions (including the upper cervical spine, skull, nose, and jaw) and excluding erroneous measurements which correspond to low SNR, such as in the presence of air, or by phase differences or motion between the prepared and unprepared gradient echo sequences acquired during B₁⁺ mapping (20). First, a binary mask was generated for each sagittal slice by interpolating it ${\mathit{W}}_{\mathit{i}\mathit{n}\mathit{d}}$ to a target FOV with a normalized threshold of 0.5. Then, a universal binary mask, ${\mathit{W}}_{\mathit{u}\mathit{n}\mathit{i}}$, was obtained to eliminate possible outlier effect from B₁⁺ maps by thresholding the sum of the individual masks with a normalized threshold of 0.4, and are binary matrices representing all voxels (20). Concurrently, the B₁⁺ maps were spatially interpolated such that all subjects have the same FOV = 156×176 mm² and in-plane resolution = 4×4 mm². Finally, B₁⁺ maps were normalized by the 98.5^th percentile among all B₁⁺ maps from all subjects to avoid possible outlier effects (20).

Proposed Processing for Multi-channel B₁⁺ Maps and Unsupervised Training:

We propose to concatenate multichannel B₁⁺ maps, $B_1^{+}\left(x,y,c\right)$, along a spatial (y) dimension to yield 2D data, $B_{1c}^{+}\left(x,y\right)$, transforming the problem for shift-equivariant processing, amenable to CNNs. Once this input is generated, the real and imaginary parts of the complex maps are given as different channels leading to 2 input channels. The network itself is a feed-forward CNN, depicted in Figure 1. Convolutions and max-pool operations use 5×5 and 2×2 kernels, respectively. ReLU is utilized for activation. The real and imaginary parts of the complex shimming weights for each coil are obtained at the output of this network.

Figure 1.

Flowchart of the proposed method. B₁⁺ maps of different coils are concatenated in the y dimension for shift- equivariant processing. Real and imaginary parts are concatenated in the channel dimension, as usual. The neural network layers and corresponding layer output sizes are depicted.

The network is trained in an unsupervised manner with RMSE loss:

$$j=\frac{1}{m}{\sum }_{i=1}^m\sqrt{\frac{{\Vert \left|{\text{A}}^{\left(i\right)}{\text{b}}^{\left(\mathit{i}\right)}\right|-a^{\left(\mathit{i}\right)}\Vert }_{W^{\left(\mathit{i}\right)}}^2}{N^{\left(i\right)}}},$$ [1]

where $i$ denotes the training sample, $m$ is the number of datasets in the database, $A^{\left(i\right)}$ is the system matrix generated via the $B_1^{+}$ maps for each transmit coil at each spatial location vectorized as column vectors, $a^{\left(i\right)}$ is the target flip angle map, the diagonal matrix $W^{\left(i\right)}$ obtained by elements-wise multiplication of $W_{ind}^{\left(i\right)}$ and $W_{uni}$, $N^{\left(i\right)}$ is the number of elements in $W^{\left(i\right)}$, and ${\text{b}}^{\left(i\right)}$ is the $B_1^{+}$ complex shimming weights for each coil, i.e. the output of the neural network. The weighted norm ${\Vert \left|{\text{A}}^{\left(i\right)}{\text{b}}^{\left(i\right)}\right|-a^{\left(i\right)}\Vert }_{{\text{W}}^{\left(i\right)}}^2$ is defined as ${\left(\left|{\text{A}}^{\left(i\right)}{\text{b}}^{\left(i\right)}\right|-{\mathit{a}}^{\left(i\right)}\right)}^T{W}^{\left(i\right)}\left(\left|{\text{A}}^{\left(i\right)}{\text{b}}^{\left(i\right)}\right|-a^{\left(i\right)}\right)$.

The database was randomly split into 80% training, 10% validation, and 10% testing. The CNN was trained using stochastic gradient descent (SGD) with a learning rate of 10⁻³ with batch size of 1 over 100 epochs. All training and testing were performed using PyTorch and processed on a workstation with the following specifications: Intel E5–2640V3 CPU (2.6GHz and 256GB memory) and an NVIDIA Tesla V100 GPU with 32GB memory. Implementation of the proposed method will be provided online (https://imagine.umn.edu/research/software).

B₁⁺ Shimming Experiments:

Target flip angle maps, $a^{\left(i\right)}$, were obtained by multiplying the $W^{\left(i\right)}$ with desired flip angle α. Hard constraints were not considered. Soft constraints or regularization terms were also not included to avoid confounding factors due to tuning of weight hyperparameters. Thus, only the unregularized MLS objective function was used. For this study, the target flip angle was arbitrarily chosen as α=5.7°, lending itself to the small flip angle regime.

For comparison, the unregularized magnitude least-squares (MLS) optimization problem (9), formulated as ${\text{min}}_{{\text{b}}^{\left(i\right)}}{\Vert \left|{\text{A}}^{\left(i\right)}{\text{b}}^{\left(i\right)}\right|-a^{\left(i\right)}\Vert }_{{\text{W}}^{\left(i\right)}}^2$, was solved separately for each slice $i$ from the database to find the complex shimming weights for 100 distinct initializations with magnitudes ranging from 5.7/√8 to 5.7*√8, and with phases spanning −π to π, along with the default CP initialization in the vendor-supplied software implemented in MATLAB. Additionally, to provide a fair assessment of inference times, a PyTorch version of MLS was also implemented based on the MATLAB code.

The methods were quantitatively evaluated for each slice using

$$\text{RMSE}=\sqrt{\frac{{\Vert \left|{\text{A}}^{\left(i\right)}{\text{b}}^{\left(i\right)}\right|-a^{\left(i\right)}\Vert }_{{\text{W}}^{\left(i\right)}}^2}{N^{\left(i\right)}}}$$ [2]

$$\text{Coefficient of Variation }\left(\text{CoV}\right)=\frac{std\left(\left|A^{\left(i\right)}{b}^{\left(i\right)}\right|\right)}{\mathit{\text{mean}}\left(\left|A^{\left(i\right)}{b}^{\left(i\right)}\right|\right)}$$ [3]

and energy demand

$$E_{\mathit{\text{total}}}=\frac{{\text{b}}^H\text{b}}{a^2}$$ [4]

Statistical differences in RMSE, CoV, and $E_{\mathit{\text{total}}}$ were assessed for normality using a one-sample Kolmogorov-Smirnov test with a significance level of 5%. Subsequently, a paired t-test was used for Gaussian distributions, while a Wilcoxon signed-rank test was employed for non-Gaussian distributions, both with a significance level of P < 0.05.

Additionally, an analysis of the null problem in flip angle maps (21,27) was also made. In order to detect nulls, i.e. areas of severely low flip angle, the following steps were applied: 1) A binary mask was generated by thresholding the normalized flip angle maps above level t_d, which was empirically set to 0.6 (Supporting Information Figure S1). 2) Difference images were calculated between this mask and the normalized target magnetization, which is 1 in all the pixels of interest and 0 otherwise. 3) Image opening, which performs image erosion followed by dilation, was applied to remove spurious pixels in the difference image. 4) The number of nonzero pixels in this image was used as a surrogate for the number of pixels corresponding to nulls, with 0 corresponding to no nulls in the flip angle maps.

Finally, to test the robustness and performance of the proposed training strategy to the size of the training database, a stress test was conducted. To this end, the model’s performance was evaluated using varying numbers of training samples on an approximately logarithmic scale {1, 2, 6, 15, 35, 86, 211, 514, 1254, 3059}. These training samples corresponded to {1, 1, 1, 1, 2, 3, 9, 21, 48, 115} subjects, respectively.

3. Results

Figure 2 depicts best, median, and worst-case flip angle maps, assessed through the RMSE metric, for both the MLS and the proposed method. Figure 2a shows the best, median and worst cases for MLS, along with the corresponding DL images. These have RMSE values of 0.49°, 0.94°, and 1.52° for MLS, and 0.53°, 0.61°, and 0.46° for DL, respectively. The corresponding CoV values are 0.08, 0.17, and 0.27 for MLS and 0.09, 0.11, and 0.08 for DL. Similarly, Figure 2b shows the best, median and worst cases for DL, along with the corresponding MLS images. The corresponding RMSE values are 0.88°, 1.26°, and 0.63° for MLS, and 0.33°, 0.71°, and 1.11° for DL, respectively, while the CoV values are 0.12, 0.16, and 0.11 for MLS and 0.06, 0.13, and 0.20 for DL. We also note that the inference times for MLS and proposed DL method, both implemented in PyTorch, among test samples were 129.47±93.25 and 0.75±0.04 ms for single initialization, respectively. The training process itself, over the dataset of 3059 samples, required approximately 11 minutes for 100 epochs for the proposed method.

Figure 2.

Example of best, median, and worst-case flip angle maps (based on the RMSE metric) for the MLS (a) and proposed method (b). The corresponding results for each image are also provided below for comparison.

Figure 3 shows violin plots for the RMSE, CoV, and energy demand metrics. The average RMSE, CoV, and energy demand across test subjects for the proposed method and the best cases for all initializations from MLS for α=5.7° are 0.94°±0.24° (mean ± std), 0.14±0.03, and 17.22±9.89 for MLS, and 0.71°±0.20°, 0.13±0.04 and 17.82±5.84 for the proposed method, respectively. The differences in RMSE, CoV, and energy demand are statistically significant (P < 0.05). We note that the energy demand was not Gaussian-distributed, hence the Wilcoxon signed-rank test was used. Further evaluation of the correlations among these quantitative measures, including COV, RMSE, and energy demand, for both MLS and proposed method are provided in Supporting Information Figure S2 using a correlation matrix plot.

Figure 3.

CoV, RMSE, and energy demand results for MLS and the proposed method are shown in the violin plots. According to these plots, the proposed method performs better than the MLS method in terms of RMSE and CoV, highlighting improved field homogeneity, while having similar energy consumption. * indicates that results are significant with P < 0.05.

Figure 4 depicts representative images that highlight an instance of the nulls appearing in flip angle maps generated using the MLS technique. The proposed method with unsupervised training is able to mitigate this issue. Quantitative assessment reveals that a null is observed with a 13.84% and 4.70% frequency for MLS and the proposed method, respectively, among testing datasets.

Figure 4.

Representative images for MLS and proposed methods showing the null problem in the MLS method. The difference images are produced by subtracting the target magnetization, as described in (2), from the solution obtained using a threshold. This figure shows that the proposed method successfully finds complex shimming parameters so that there are no significantly large nulls in the resulting image. The green circles on the obtained MLS solution point the null which does not exist in the proposed method. The proposed method is close to the desired magnetization with CoV=0.11, RMSE=0.60° compared to the MLS solution with CoV=0.26, RMSE=1.47°.

Figure 5 illustrates results of the stress test conducted on the proposed method by varying the number of training samples on an approximately logarithmic scale. While the network underperforms with a very limited number of samples as expected, its performance starts to improve with as few as 86 samples, corresponding to 3 subjects. With only 514 training samples, corresponding to 21 subjects, it matches the performance of the full database consisting of 3059 samples from 115 subjects. These findings highlight the efficiency of the proposed DL training in this scenario, as it achieves noteworthy results with a substantially reduced number of training samples, making it a promising solution even in resource-constrained scenarios.

Figure 5.

A stress test to evaluate the performance of the neural network trained on different numbers of samples on an approximately logarithmic scale. Interestingly, the network starts performing reasonably with as few as 86 samples and achieves comparable performance to the full database with only 514 samples.

4. Discussion

In this study, we proposed an unsupervised deep learning method with CNNs for multi-channel pTx design. We demonstrated its feasibility through static B₁⁺ shimming at 7T, employing a slice-by-slice shimming technique (28,29), which offered an additional degree of freedom compared to regional approaches (30). Our method enhanced the uniformity of flip angle profiles quantitatively, displaying favorable results compared to the unregularized MLS technique in terms of RMSE and CoV metrics while maintaining a comparable energy consumption level.

DL methods have received attention for speeding up pTx design while maintaining subject-specific processing. However, to the best of our knowledge, previous methods for directly estimating RF pulses did not handle multi-channel B₁⁺ maps when using CNNs. A previous work that used neural networks with multi-channel maps relied on fully-connected neural networks (22), which require more parameters and cannot work across different input sizes, as opposed to CNNs. On the other hand, previous DL approaches that used CNNs for RF pulse estimation have used single coil B₁⁺ maps (23), and concatenation along the channel dimension with B₀ maps. Here, building on our insights from multi-slice image reconstruction (31,32), we concatenated multichannel B₁⁺ maps along a spatial dimension, which enabled shift-equivariant processing suitable for CNNs. We also note that multi-coil B₁⁺ input has been used for a classification-type task previously (25).

Supervised training has been popular for applications of DL to pTx (21–24). However, this necessitates additional computational resources and time to find reference RF waveforms. In this study, this extra computation is eliminated via unsupervised learning, speeding up training. Furthermore, supervised training implicitly ties the performance of the trained network to the conventional method used to generate the reference RF waveform. In particular, generating a reference label in this study involves solving a nonconvex optimization problem based on B₁⁺ maps, specifically the MLS method. Consequently, supervised training would learn to predict the MLS solution, albeit using a CNN for improved efficiency. Thus, the performance of supervised learning would be inherently limited by the MLS solutions. On the other hand, unsupervised learning bypasses this issue by directly minimizing the difference to target magnetization through a loss function. By avoiding reliance on MLS solutions as reference, unsupervised learning holds potential for superior performance in B₁⁺ shimming. This is similar to the use of DL in image reconstruction when acquisition of ground-truth data is infeasible, in which unsupervised methods outperform other conventional methods such as compressed sensing or parallel imaging (32–34). This strength of unsupervised learning is best exemplified in its ability to tackle the null problem associated with MLS (27). Better results are obtained using the proposed unsupervised learning, whose performance is not limited by the conventional approach that is used to generate a reference RF waveform, since the latter itself suffers from this issue. Conversely, it is worth noting that while unsupervised learning incorporates physics information in the loss function, it does not solve an explicit objective function as the MLS algorithm, making it potentially more challenging to interpret, particularly for failure modes.

One of the drawbacks of the proposed method is the need for a large training database. In vivo acquisition of such data containing more than a thousand slices is time-consuming and resource-intensive. While the results of the stress test on the number of training samples suggest fewer datasets may be sufficient, this may still put constraints on implementing similar DL techniques for new acquisition schemes. We note that synthetic B₁⁺ maps have also been proposed in the literature, which would eliminate the curation of B₁⁺ map databases (23,35). We also note newer efforts for circumventing the acquisition of B₁⁺ maps in the pTx pipeline by estimating it from B₁⁻ maps (36). The synergistic combination of our approach with such techniques warrants further investigation.

During initial experiments, several different network architectures of varying depths, including residual networks and U-nets, were studied. However, their performance did not substantially differ from the simple feed-forward CNN employed in this study. As a result, the current CNN architecture was chosen due to its expedited training speed and inherent simplicity. 100-to-200 epochs were determined to be sufficient for training (Supporting Information Figure S3). Opting for the lower number reduced the training time and resources.

Our current implementation does not have explicit constraints on power, voltage, or SAR. Among these, voltage constraint is the simplest to enforce through the addition of an appropriate nonlinear activation at the network output, such as sigmoid or ReLU6, as in (37). However, power or SAR constraints are substantially more difficult. To the best of our knowledge, inclusion of hard quadratic constraints with DL methods is an open question, although progress has been made in the broader artificial intelligence community (38–43). One step in this direction would be to use a soft constraint on power or SAR by incorporating an additional term into the loss function. However, the performance and robustness will substantially depend on tuning the weight for this soft constraint term. Nonetheless, hard constraints on SAR or power remain an open question that is critical for pTx problems.

The results in the article used a target α=5.7°, which is in the small flip-angle regime, where the forward operator is described by a linear system matrix A. To make the model applicable to higher flip angles, a Bloch simulation-based system function needs to be implemented in the loss function. There are existing publicly available implementations in PyTorch (44) and in MATLAB (45), which will be investigated in future studies. We also note that the incorporation of Bloch simulation-based methods in training would increase the total computational time. Thus, further studies are needed for the large flip angle regime.

While our study concentrated on single slice acquisitions, there is also a demand for shimming in non-selective volumetric or multi-slice scenarios (46). To this end, we adopted our method to both of these setups by replacing the 2D CNNs with their 3D counterparts. In the first case of volumetric 3D B₁⁺ shimming (Supporting Information Figure S4), MLS shows better RMSE and CoV compared to DL, though the difference is non-significant (P=0.63 for RMSE and 0.06 for CoV, Wilcoxon signed-rank test). Furthermore, DL has significantly improved energy demand compared to MLS (P < 0.05, Wilcoxon signed-rank test). In the second case of multi-slice imaging with 2 slices, where 1313 training samples are available, proposed DL significantly improves on MLS in all metrics (Supporting Information Figure S5). Addressing the challenges associated with insufficient training database size (47) and fine-tuning the CNN model parameters may lead to further improvements for 3D B₁⁺ shimming scenarios, though this was beyond the scope of our study.

Throughout this study, testing was retrospectively performed on in vivo data that was acquired previously. This allowed us to test the methods thoroughly on a large database and isolate cases with adverse failures, such as the null issue in the MLS method, instead of a small number of examples acquired post-training. Thus, an inherent limitation of our study is the lack of new acquisitions to compare the shimming performance of DL and MLS prospectively.

The pTx design pipeline presents time-consuming challenges beyond just RF pulse design, notably in B₁⁺ mapping, potentially B₀ mapping, ROI selection, masking, and sequence preparation. Our proposed approach tackles only the RF pulse design part, offering enhanced speed and generalizability, and may potentially find applications for real-time pTx design (48).

We note that static B₁⁺ shimming was used to establish proof-of-concept. In this setting, optimization-based methods are still relatively fast and accurate. Thus, the benefits of DL approaches may not be as pronounced. Nonetheless, further investigations for more complicated pTx pulse designs are warranted to fully harness the potential of our approach. Future studies will aim to extend the work to more complicated pTx pulse designs and incorporate additional constraints, such as SAR or peak power.

5. Conclusion

The proposed unsupervised deep learning strategy for CNNs with image domain concatenated inputs enables fast pTx pulse design, outperforming the unregularized MLS method in terms of homogeneity for static B₁⁺ shimming in the 2D small flip angle regime without constraints in this proof-of-principle study.

Supplementary Material

Supinfo

Supporting Information Figure S1. The percent null generation results for threshold values ​​on the flip angle maps are shown for both MLS and the proposed method. A threshold for both methods was chosen to be 0.6, indicated by the green dashed line in the figure. The dashed black line indicates that the MLS method produces nulls in the image even though the threshold is set to the desired flip angle. This is because some flip angle maps generated by the MLS method overestimate the desired flip angle.

Supporting Information Figure S2. Correlation matrix plot for the quantitative measures, revealing the correlations between various variable pairs. Non-relevant correlations are not shown, and the upper triangular section represents the significance level of these correlations (r: Pearson correlation coefficient).

Supporting Information Figure S3. The figure illustrates the training and test loss, indicating that the model does not exhibit signs of overfitting.

Supporting Information Figure S4. Performance of our approach for 3D B₁⁺ shimming. In this setting, the MLS method performs better than our approach in terms of CoV and RMSE, but our method has significantly better energy demand. Note that in this case, the training database size comprises 98 subjects, which is lower than the size indicated in our stress test (Fig. 5). Thus, if 3D shimming is the target application, further performance gains may be achieved for the DL approach using a larger training database, which warrants further investigation. * and NS indicates that results are significant with P < 0.05 and not significant, respectively (The distributions were assessed for normality. All distributions were found to be non-Gaussian and the Wilcoxon signed-rank test was employed for all of them).

Supporting Information Figure S5. Performance of our approach for multi-slice B₁⁺ shimming for 2 slices using a 3D CNN. In this setting, the proposed method outperforms MLS in terms of RMSE, CoV, and energy. * indicates that results are significant with P < 0.05 (The distributions were assessed for normality. CoV and energy distributions were found to be non-Gaussian and the Wilcoxon signed-rank test was employed for these distributions. RMSE distributions were found to be Gaussian and paired t-test was employed).