In silico GABA+ MEGA-PRESS: Effects of signal-to-noise ratio and linewidth on modeling the 3 ppm GABA+ resonance

Abstract

To investigate the GABA+ modeling accuracy of MEGA-PRESS GABA+-edited MRS data with various spectral quality scenarios, the influence of varying signal-to-noise ratio (SNR) and linewidth on the model estimates was quantified. MEGA-PRESS data from 46 volunteers were averaged to generate a template MEGA-PRESS spectrum, which was modeled and quantified to generate a GABA+ level ground truth. This spectrum was then manipulated by adding 427 combinations of varying artificial noise levels and line broadening, mimicking variations in GABA+ SNR and B₀ homogeneity. GABA+ modeling and quantification was performed with 100 simulated spectra per condition using automated routines in both Gannet 3.0 and Tarquin. The GABA+ estimation error was calculated as the relative deviation to the quantified GABA+ ground truth levels to assess the accuracy of GABA+ modeling. Finally, the accordance between the simulations and different in vivo scenarios was assessed. The GABA+ estimation error was smaller than 5% for all GABA+ SNR values with creatine linewidths lower than 9.7 Hz in Gannet 3.0 or unequal 10.6 Hz in Tarquin. The standard deviation of the GABA+ amplitude over 100 spectra per condition varied between 3.1 and 17% (Gannet 3.0) and between 1 and 11% (Tarquin) over the in vivo relevant GABA+ SNR range between 2.6 and 3.5. GABA+ edited studies might be realized for voxels with low GABA+ SNR at the cost of higher group-level variance. The accuracy of GABA+ modeling had no relation to commonly used quality metrics. The Tarquin algorithm was found to be more robust against linewidth changes than the fitting algorithm in Gannet.

1 |. INTRODUCTION

At 3 T, J-difference-edited ¹H MRS has become a common tool to investigate the major inhibitory neurotransmitter γ-aminobutyric acid (GABA) noninvasively.¹ To this end, Mescher-Garwood PRESS² (MEGA-PRESS) is increasingly used. A J-difference-edited MRS experiment usually consists of two sub-acquisitions (‘ON’ and ‘OFF’), differing in the frequency of an editing pulse, which is applied to differentiate signals from different spin systems according to their coupling networks. In a GABA-edited experiment, the ON pulse editing frequency is set to 1.9 ppm to refocus the evolution of the 3 ppm (GABA) signal, whereas an editing frequency off-resonance of GABA, eg 7.5 ppm, is chosen in the OFF experiment to let the coupling evolve freely. These experiments are usually repeated several hundred times to increase the signal-to-noise ratio (SNR). After averaging, the difference spectrum between ON and OFF experiments is calculated. Since the signals from Cr are identical in the two halves of the experiment, they cancel out upon subtraction, rendering the underlying 3 ppm GABA resonance visible. The described editing scheme co-edits homocarnosine and macromolecules³; therefore, the quantified 3 ppm signal is commonly termed GABA+.

GABA+-edited MEGA-PRESS has been frequently applied both in clinical and in behavioral studies. Measured GABA+ concentrations correlate with functional tasks/imaging.^4,5 Altered GABA+ concentrations are observed in healthy aging^6–8 as well as in various diseases including ADHD⁹ and Tourette syndrome.¹⁰ Further changes measured via GABA+-edited spectroscopy were reported for ALS¹¹ and hepatic encephalopathy.¹²

Due to the low concentrations of GABA (1.3–1.9 mmol/L) in the brain,¹³ strong signal overlap and the effort of implementing difference editing, the in vivo quantification remains technically challenging. Various sources of variance have been identified, including hardware influences, eg scanner drift^14,15 or differences in sequence implementations between vendors.^16,17 Therefore, limitations exist regarding the minimal spectroscopic voxel size, the number of acquired transients and the required statistical power for the analysis.¹⁸ In neurophysiological diseases showing regional specific effects, it remains challenging to create suitable voxel dimensions to reach sufficient SNR in acceptable time. Large voxel volumes up to 27 mL are the default recommendation for obtaining sufficient SNR¹. This approach sacrifices much needed anatomical specificity for measurement reliability and increases the risk of unknown partial volume effects. In some studies the voxel size was reduced to the target area,¹⁹ but this resulted in substantially higher inter-subject variance as compared with the usual ~12% in typical GABA+-edited investigations.¹⁶ Another relevant parameter in MRS is the shim quality, as broad lines hinder the resolution of neighboring resonances and frequency precision is elementary for spectral editing. B₀ field homogenization is of particular concern when investigating frontal or deep brain structures, as these tend to be close to areas of strong susceptibility gradients.

Unknown errors resulting from data modeling are another challenge in MRS. Usually, Cramer-Rao lower bounds (CRLBs)²⁰ or the GABA+ amplitude normalized standard deviation (SD) of the fit residual, as implemented in Gannet,²¹ are used to provide a measure of uncertainty of the metabolite modeling. Outlier criteria based on CRLB, SNR, or linewidth, may also be used to judge whether the data have been modeled reliably.^8,12,22 The major effect of SNR on the reliability of the fitting is commonly known and has been investigated in various studies.^23–25 Yet, the deviation of the model from real metabolite levels remains unclear. To judge the reliability of an MRS method, several approaches have been proposed: measuring the reproducibility via a test-retest approach,^26,27 manipulation of simulated datasets to create various spectral qualities,^28,29 and most recently a correlation analysis of specific metabolites between different spectroscopy techniques acquired in the same voxel.³⁰

The aim of this study was to investigate the impact of varying SNR and linewidth on modeling of GABA+. To this end, a MEGA-PRESS spectrum with a ground truth GABA+ peak area was manipulated to achieve various SNR levels and linewidths. Afterwards, spectra were quantified with two quantification tools for GABA+-edited MRS. The deviations of the modeled GABA+ area from the ground truth GABA+ area were calculated to determine minimal requirements of SNR and linewidth. Finally, the accordance in terms of GABA+ fit error, group variance, and number of rejected spectra between the simulations and two in vivo GABA+-edited scenarios was assessed.

2 |. MATERIAL AND METHODS

In a first part, different SNR levels and linewidths were added to a template MEGA-PRESS spectrum, and GABA+ levels were derived by modeling the spectra with two different analysis software tools. In a second part, different in vivo scenarios were analyzed to assess the agreement between the simulations and the in vivo GABA+ modeling.

2.1 |. MEGA-PRESS simulations

2.1.1 |. Creation of the template MEGA-PRESS spectrum

A template MEGA-PRESS spectrum with known metabolite concentrations (acquired on 3 T Siemens scanners, number of excitations (N_EX) = 256, T_r/T_e = 2000\68 ms, bandwidth = 4000 Hz, V = 27 mL, 4096 datapoints) was created as the mean of datasets of 48 volunteers. The spectra originated from a 27 mL cubic voxel in the medial parietal lobe. This dataset is the outcome of a recent ‘Big GABA’ multi-site study,¹⁶ which is available to the public from the NITRC data repository (http://www.nitrc.org/projects/biggaba). For each individual dataset, all 256 averages were frequency and phase corrected using spectral registration³¹ using the MATLAB-based (MathWorks, Natick, MA) toolbox Gannet 3.0,²¹ without applying zero-filling. The final number of datasets in the template after visual inspection was 46 subjects after the removal of two subjects due to severe frequency drift. The template ON and OFF spectra (Figure 1A) for the simulations were baseline-corrected and normalized by the water reference amplitude of each dataset. This approach was chosen over a fully simulated approach to achieve a more in vivo realistic template spectrum with high SNR, sufficient linewidth, and a well-behaved baseline. The creatine linewidth of the template spectrum was 7.9 Hz. The models of each quantification tool of the 3 ppm GABA+ resonance of the template are shown in Figure 1B, and the model results from each tool were used as the ground truth GABA+ levels for the corresponding tool (details in the quantification paragraph). The ground truth is subsequently used to calculate an estimation error for each quantification tool.

FIGURE 1.

Template spectrum and spectral manipulation. A, Template ON/OFF spectra (average of the 46 datasets) and resulting difference spectrum (magnified by a factor of 10). B, 3 ppm GABA+ model of the artificial difference spectrum without noise or line broadening (Gannet 3.0 left, Tarquin right). C, Mean creatine linewidth for all conditions. D, Mean GABA+ SNR for all conditions. Mean GABA+ SNR and mean creatine linewidths measured with FID-A are shown in C and D

2.1.2 |. In silico manipulation of the template spectrum

The template MEGA-PRESS spectrum was the basis for all subsequent SNR and line broadening manipulations, which were performed with the MATLAB-based toolbox FID-A.³² In an in vivo measurement, poor B₀ field homogeneity would lead to broad linewidth and poor SNR at once. To mimic these effects, two steps were applied. In a first step, seven different levels of exponential line broadening between 0 and 6 Hz, were applied to the template spectrum to mimic differences in B₀ homogeneity. Subsequently, 61 different SNR levels between 0 and 30 times the N-acetyl-aspartate (NAA) signal height at 2.01 ppm were created. The NAA signal height was determined as the maximum between 1.9 and 2.1 ppm in the real part of the frequency domain. Next, Gaussian-distributed noise was multiplied by the factor determined by the NAA signal height, and added to the real and imaginary parts of the template in the time domain to create the noisy data.

By intentionally applying the SNR level changes in the second step, all spectra preserve the broadened linewidth (mimicking a poor B₀ field homogeneity) (Figure 1C) and possess roughly the same GABA+ SNR (Figure 1D). In total, 100 spectra were simulated for each of the combinations of noise and line broadening levels.

For visualization and unrelated to the used quantification tools, the creatine linewidth (Figure 1C) and GABA+ SNR (Figure 1D) were calculated with FID-A for each combination. The creatine linewidth was determined as the FWHM of the Lorentzian fit between 2.85 and 3.1 ppm. The GABA+ SNR was defined as the ratio of the maximum height of the real GABA+ peak between 2.7 and 3.2 ppm and the noise SD between 9 and 10 ppm. The colormap visualizes the mean value of the calculated creatine linewidth (Figure 1C) or GABA+ SNR (Figure 1D) for a specific spectral manipulation. These calculations allow a direct comparison with literature, as creatine linewidth and GABA+ SNR are usually reported.

Example raw 3 ppm GABA+ resonances for different combinations are shown in Figure 2. As expected, the line broadening broadens the GABA+ resonance and increases the GABA+ SNR in the template spectrum (first column).

FIGURE 2.

Example raw 3 ppm GABA+ resonances. Frequency- and phase-corrected 3 ppm GABA+ resonances without apodization or zero-filling for representative conditions. All seven linewidth (LW) levels of creatine (rows) and 13 representative noise levels (columns) are shown

2.2 |. MEGA-PRESS in vivo study

All in vivo data were acquired on a clinical whole-body 3 T MRI (Siemens MAGNETOM Skyra A TIM system, Siemens Healthcare, Erlangen, Germany) using a 20-channel head coil for receive, and the body coil for transmit. Fourteen healthy volunteers (five female; age (mean ± SD) 26.3 ± 2.2 years) were recruited. The study was performed in accordance with the Declaration of Helsinki in its current version³³ and approved by the local internal review board (517R). All participants gave written informed consent prior to the examination. The in vivo study was focused on the thalamus, which is a challenging anatomical location regarding spectral quality and anatomical specificity of the MRS voxel.

2.2.1 |. Structural image

In each participant, a high-resolution 3D anatomical T₁-weighted magnetization-prepared gradient echo (MP-RAGE) scan (T_R/T_E = 1950/4.6 ms; isotropic resolution 1 mm; 176 transversal slices) was performed aligned to the anterior commissure-posterior commissure line after a scout scan. Sagittal and coronal images were reconstructed online for optimal localization of the spectroscopic voxels.

2.2.2 |. MEGA-PRESS acquisition

Two voxels with different SNR levels based on the voxel size and varying creatine linewidths were selected from another study. A small voxel of 2 mm × 2 mm × 2 mm = 8mL volume centered on the left thalamus and a larger voxel with 30 mm (AP) × 35 mm (LR) × 25 mm (HF) = 26.25 mL including the whole basal ganglia region are shown in Supporting Information Figure S1 N_EX = 256, T_R/T_E = 2050/68 ms, bandwidth = 1200 Hz, and 2048 datapoints. In comparison to the template, T_R was slightly longer and a different bandwidth as well as number of datapoints were chosen. A Siemens GRE Brain shim procedure followed by manual shimming was performed to achieve a water linewidth less than 15 Hz for the small voxel and more than 20 Hz for the large voxel, as indicated by the inline interactive display on the console.

2.3 |. Data processing, quantification, and spectral quality metrics

Two different modeling algorithms, Gannet 3.0 and Tarquin, were used to model and quantify the 3 ppm GABA+ resonance of all difference spectra for the in silico and in vivo data. Both quantification tools are commonly used for GABA+ quantification, and approximate the entire 3 ppm signal using simplified models, but do not attempt to separate out the co-edited homocarnosine or macromolecule signals. The analysis was limited to these two approaches, because there is currently no consensus on the optimal appropriate modelling of GABA-edited J-difference spectra using linear-combination modelling algorithms such as LCModel.³⁴ Additionally, both tools are freely available, and allow fully automated processing to reduce effects of user interaction. In the simulations, the estimation error (GABA+_error) of the model was defined as the rounded absolute value of the relative difference between the modeled GABA+ resonance in the template spectrum for each quantification tool, ie the ground truth GABA+_gt, and the modeled GABA+ resonances of the manipulated spectra GABA+_Area for each quantification tool:

$$\text{GABA}{+}_{\text{error}}(\%)=\left|\frac{\text{GABA}{+}_{\text{Area}}*100}{\text{GABA}{+}_{\text{GT}}}-100\right|$$

GABA+ areas that were calculated to be outside of a range between ±3 SD = 36% from the ground truth GABA+ area were rejected.¹⁶ in vivo spectra were quantified using GABA+/creatine ratios. The two different data processing and quantification pipelines for Gannet 3.0 and Tarquin are described in detail below.

2.3.1 |. Gannet 3.0

The analysis with the MATLAB-based toolbox Gannet 3.0,²¹ which is specifically developed for GABA+ quantification, included spectral registration for frequency and phase correction of the individual transients, automated rejection of corrupted transients, zero-filling to 32 768 data points and 3 Hz exponential line broadening. The combined GABA-Glx model included parameters to model a single Gaussian peak for the 3 ppm GABA+ and a double Gaussian peak for the co-edited 3.75 ppm Glx resonances, as well as baseline parameters with linear, sine, and cosine terms in the difference spectrum. Effects from choline subtraction artifacts are reduced by down-weighting the signal and the model between 3.16 and 3.285 ppm. GABA+ fit errors as a measurement of the quality of the individual fit are given (as percentages) as the ratio of the SD of the residual over the fitting range (2.79–3.55 ppm) and the GABA+ peak area. In vivo, the GABA+/Cr ratios are multiplied by a correction factor of 0.5, assuming half of the edited signal to originate from macromolecules and homocarnosine.³

In a second analysis, the impact of changes in the modeling parameters was investigated. Therefore, the default parameters of Gannet 3.0 modeling (fitting range 2.79–4.1 ppm, GABA bounds 2.79–3.2 ppm, and the upper and lower bounds −40 and −200 Hz for the half width at half maximum parameter of the peak) were changed to a fitting range of 2.7–4.2 ppm, GABA bounds of 2.7–3.2 ppm, and the upper and lower bounds of-10 and −800 Hz for the half width at half maximum parameter to allow the adaption to broader linewidths.

2.3.2 |. Tarquin

Data pre-processing with FID-A

The MATLAB-based toolbox FID-A was used for individual frequency and phase correction of the individual transients of the in vivo data using the spectral registration algorithm,³¹ and automated rejection of corrupted transients, both of which are not implemented in Tarquin itself. Alignment and rejection parameters were chosen analogously to the Gannet 3.0 implementation. No additional zero-filling or line broadening was applied. No separate frequency or phase correction was applied for the in silico data, as the ON and OFF sub-spectra were assumed to be aligned. The processed spectra were analyzed with Tarquin³⁵ as described in the following.

Fitting with Tarquin

Analysis with Tarquin³⁵ included residual water removal by Hankel Singular Value Decomposition, automatic phasing and referencing of the 2.01 ppm NAA signal. Modelling was performed with an internally calculated MEGA-PRESS basis set via nonlinear least squares fitting in the time domain. This basis set models the GABA+ resonance as two separate Gaussian peaks scaled as one proton each, ie as a pseudo-doublet at 2.95 and 3.04 ppm. The first 10 points of the time domain data are truncated during the initial fitting, as suggested in the documentation of Tarquin. Afterwards, the differences between the metabolite signal of the fit extrapolated to t = 0 and the full data is used to estimate the baseline. To allow a direct comparison of the modeling, a GABA+ fit error was calculated analogously to Gannet 3.0. In vivo, the GABA+/Cr ratios were multiplied by a macromolecule correction factor of 0.5, similarly to Gannet 3.0.

Spectral quality metrics

To analyze the results of the simulations, several spectral quality metrics were included in the analysis: (i) the mean of GABA+_error calculated over the 100 simulated spectra per combination as the difference from the known GABA+ model amplitude was examined as a unique characteristic of the present study—which is usually unknown—to determine the reliability of the GABA+ modeling; (ii) the fit error (GABA+_fit) as a common quality metric, some sort of which is usually provided by each tool; (iii) the standard deviation (GABA+_SD) of the GABA+ quantification of the 100 simulated spectra per combination. GABA+_SD reflects a measure of variance of the modeling introduced by the underlying SNR and linewidth changes. Higher GABA+_SD implies increasing susceptibility to SNR and linewidth changes of the modeling approach. Finally, the number of rejected spectra was considered as a measure of severe outliers in the modeling due to SNR and linewidth changes. GABA+ areas that were calculated to be outside of a range between ± 3 SD = 36% from the ground truth GABA+ area were rejected.¹⁶ For in vivo studies, these outliers are driven not exclusively by the factors mentioned above, but also by patient movement or frequency drift, as well as biological GABA variance and voxel placement inconsistencies not considered in this study. Additionally, the Pearson correlation coefficient r between the quantified 3 ppm GABA+ amplitudes for Gannet 3.0 and Tarquin was calculated across all parameter combinations.

The diagrams (Figures 3–6 and Supplementary Figure 2) are designed as follows. The y axis represents the 61 possible GABA+ SNR levels, the x axis shows the seven possible creatine linewidth values in Hz, and the color map displays the analyzed spectral quality metric.

FIGURE 3.

Quality metrics of 3 ppm GABA+ default modeling with Gannet 3.0. Each metric is analyzed for all 427 conditions with 100 spectra per condition. A, Mean GABA+_error. B, Mean GABA+_fit. C) GABA+_SD. D, Number of rejected spectra (GABA+_rejects)

FIGURE 6.

Correlation coefficient r between the model estimates of Gannet 3.0 and Tarquin. The correlation analysis is performed between the 100 spectra of each condition. Areas with strong (r > 0.5), medium (r > 0.3), and small (r > 0.1) correlations are marked. A, Default Gannet 3.0 model and Tarquin. B, Broad Gannet 3.0 model and Tarquin

3 |. RESULTS

3.1 |. In silico quantification

The four analyzed quality metrics for the modeling with Gannet 3.0 are shown in Figure 3. Over almost the entire GABA+ SNR range, GABA+_error is lower than 5% for creatine linewidths smaller than 9.7 Hz (Figure 3A) and remains between 5 and 10% for creatine linewidths between 9.7 and 10.6 Hz. For creatine linewidths larger than 10.6 Hz, GABA+_error exceeds 10%. GABA+_fit and GABA+_SD across all 100 simulated spectra are mainly affected by changes in GABA+ SNR (Figure 3B and 3C). GABA+_fit is smaller than 5% for GABA+ SNR larger than 7.2, and remains between 5 and 15% for GABA+ SNR in the range of 7.2 to 2.5. For GABA+ SNR smaller than 2.5, the fitting error is between 15 and 20%, which is only exceeded for creatine linewidths larger than 12.6 Hz. For GABA+ SNR larger than 4.3 the fitting error increases by approximately 1% over the whole range of creatine linewidths, while for GABA+ SNR smaller than 4.3 it increases by approximately 2% over the whole range of creatine linewidths. GABA+_sd smaller than 5% is observed for GABA+ SNR larger than 11.1, while it ranges between 5 and 15% for GABA+ SNR in the range of 11.1 to 5.4. For GABA+ SNR smaller than 5.4 an SD ranging from 15 to 22% is visible (Figure 3C). Fewer than 10% of the data were rejected for GABA+ SNR larger than 5.4, while up to 30% of the data were rejected for GABA+ SNR ranging from 5.3 to 3.2. Up to 60% of the data were rejected for GABA+ SNR smaller than 3.2.

Figure 4 illustrates the four analyzed quality metrics for the modeling with Tarquin. Over the whole GABA+ SNR range GABA+_error smaller than 5% is observed (Figure 4A). A deviation of larger than 5% is observed for creatine linewidth = 11.6 Hz and GABA+ SNR between 3.6 and 3.2. As with Gannet 3.0, GABA+_fit and GABA+_SD are mainly affected by changes in GABA+ SNR (Figure 4B and 4C). GABA+_fit ranges from 6 to 30% for GABA+ SNR levels between 92.3 and 5.4. For GABA+ SNR smaller than 5.4 it has a range from 30 to 104%. For GABA+ SNR larger than 3.6 the fitting error increases by approximately 5% over the whole range of creatine linewidths, while for GABA+ SNR smaller than 5.4 it increases by approximately 14% over the whole range of creatine linewidths. For GABA+ SNR larger than 5.4 a GABA+_SD smaller than 5% is observed, while it ranges between 5 and 19% for GABA+ SNR smaller than 5.4. For GABA+ SNR smaller than 2.1 GABA+_SD ranges from 15 to 18%. Fewer than 5% of the data were rejected for GABA+ SNR larger than 2.5, while up to 17% of the data were rejected for GABA+ SNR smaller than 2.5 (Figure 4D).

FIGURE 4.

Quality metrics of 3 ppm GABA+ modeling with Tarquin. Each metric is analyzed for all 427 conditions with 100 spectra per condition. A, Mean GABA+_error. B, Mean GABA+_fit. C, GABA+_Sd. D, GABA+_rejects

The analyses for the broader model in Gannet 3.0 are summarized in Figure 5. Compared with the default Gannet 3.0 model the range with linewidths larger than 9.7 Hz and GABA+ SNR lower than 3.2 shows lower deviations from the ground truth (Figure 5A). However, GABA+_fit increases for the same conditions (Figure 5B). GABA+_SD (Figure 5C) and the number of rejects (Figure 5D) are generally lower compared with the default Gannet 3.0 model.

FIGURE 5.

Quality metrics of 3 ppm GABA+ broad modeling with Gannet 3.0. Each metric is analyzed for all 427 conditions with 100 spectra per condition. A, Mean GABA+_error. B, Mean GABA+_fit. C, GABA+_Sd. D, GABA+_rejects

While comparing Figures 3, 4 and 5 the reader should be aware of the differences in the colormap’s scale in the A panels.

The correlation coefficient r between the model estimates with Gannet 3.0 and Tarquin is presented in Figure 6. Clusters of combinations with strong (r > 0.5), medium (r > 0.3), and small (r > 0.1) correlation are observable. The default Gannet 3.0 model shows a strong correlation for GABA+ SNR larger than 5.4 and creatine linewidth smaller than 11.6 Hz, while only very weak correlations between the two methods result for GABA+ SNR smaller than 2.5. The remaining clusters appear to have a medium correlation (Figure 6A). The model estimates between the broader Gannet 3.0 model and Tarquin appear to have a higher correlation. The strong correlation holds for GABA+ SNR larger than 3.6 almost for the whole linewidth range. Medium correlations were found for GABA+ SNR smaller than 3.6 for creatine linewidth smaller than 11.6 Hz (Figure 6B).

In another analysis we excluded the predefined rejection criterion of ±3 SD = 36% for the GABA+ area. The GABA+_error for this analysis is shown in Supplementary Material 2 for all fitting algorithms. The Gannet 3.0 GABA+_error increases for GABA+ SNR smaller than 2.1 and ranges between 8 and 15% for all creatine linewidths. It additionally increases to the same values for GABA+ SNR between 3.6 and 2.1 for creatine linewidths larger than 9.7 Hz. The GABA+_error shows almost no changes for the analysis with Tarquin.

3.1.1 |. In vivo quantification

Based on the defined outlier criteria, data from the small voxel were excluded for three participants, while all data from the large voxel remained in the analysis. In the Big GABA dataset, three participants were excluded for the modeling with Gannet, and one was excluded for the modeling with Tarquin.

Figure 7 illustrates the mean spectra and their SD for the small (Figure 7A) and the large in vivo voxel (Figure 7C) extracted from Gannet 3.0 (note that the preprocessing parameters for Gannet 3.0 and FID-A are similar). The mean and SD of the fits and the residual of both quantification tools are presented in Figure 7B and 7D for the small and the large voxel, respectively. In addition, the mean baseline is shown. The lower SNR of the small voxel becomes apparent in a higher SD of the mean spectra, fits, and residuals.

FIGURE 7.

In vivo spectra of the small (A, B) and large (C, D) voxels. Mean and individual spectra, as well as fit, baseline, and residual, are indicated by solid lines. Spectra, fit, and residual SDs are indicated by the shaded areas. The left column contains the mean and individual spectra of both voxels extracted from the Gannet 3.0 preprocessing, as well as the voxel positioning in the thalamus. The right column shows the fitting results of the quantification tools

In the small voxel, the following characteristics become apparent: Gannet 3.0 appears to have a smaller SD for the fits, while the mean GABA+ fits are comparable in height. The model of the GABA+ is notably broader for Tarquin due to the double Gaussian model. The SD of the residual is higher for Tarquin as no line broadening was applied. The mean baseline is more variable for Tarquin compared with Gannet 3.0, while Gannet shows a systematic decline in the baseline between 2.9 and 3.1 ppm.

For the large voxel, Gannet 3.0 has a smaller SD for the fits than Tarquin. The SD of the residual is low for both quantification tools and does not feature clear residual metabolite peaks in the 3 ppm GABA+ peak area. For the large voxel, the mean baseline is relatively flat compared with the small voxel. Still, Tarquin shows a higher degree of variation in the baseline estimate.

When comparing the broader Gannet 3.0 model with the default model, almost no changes are visible in the mean fits and residuals for both voxels.

Figure 8 shows the quantification results and quality metrics of the in vivo measurements. The estimated GABA+/Cr values were systematically higher with Tarquin (Figure 8A). The coefficient of variance (CoV) is lower for the Gannet 3.0 default model than for Tarquin in the small voxel (Gannet 3.0 default 29%; Gannet 3.0 broad 35%; Tarquin 30%), while it is higher for both Gannet 3.0 models than for Tarquin in the large voxel (Gannet 3.0 default 25%; Gannet 3.0 broad 20%; Tarquin 14%). Nevertheless, the introduction of a broader model in Gannet 3.0 seems to be beneficial for spectra with broader linewidth.

FIGURE 8.

In vivo GABA+ quantification. A, Distribution of GABA+-to-Cr ratios for the two voxels and two quantification tools. Successful quantifications (n) and the CoV are reported. B, Distribution of fit errors for the two voxels and two quantification tools. C, Distribution of GABA+ SNR for the two voxels. D, Distribution of creatine linewidths for the two voxels. Dots indicate individual measurements

For both voxels, the fitting error and its SD are higher for Tarquin than for Gannet 3.0 (Figure 8B). GABA+ SNR (Figure 8C) and creatine linewidth (Figure 8D) are higher in the larger voxel.

For the small voxel, no significant correlation was found between the GABA+/Cr ratios determined by both Gannet 3.0 models and Tarquin (r = 0.18; p = 0.65) (<FIG 9>Figure 9A), while a significant correlation was found for the large voxel (Gannet 3.0 default versus Tarquin, r = 0.54, p < 0.05, and Gannet 3.0 broad versus Tarquin, r = 0.58, p < 0.05) (Figure 9B). Similar to the simulations, the agreement between Tarquin and Gannet is higher for the broad model.

FIGURE 9.

In vivo GABA+-to-Cr ratio correlation between the model estimates of Gannet 3.0 and Tarquin. A, Correlation analysis for the small voxel with the default Gannet 3.0 model and Tarquin. B, Correlation analysis for the large voxel with the default Gannet 3.0 model and Tarquin. C, Correlation analysis for the small voxel with the broad Gannet 3.0 model and Tarquin. D, Correlation analysis for the large voxel with the broad Gannet 3.0 model and Tarquin. E, Correlation analysis for the small voxel with the default Gannet 3.0 model and broad Gannet 3.0 model. F, Correlation analysis for the large voxel with the default Gannet 3.0 model and broad Gannet 3.0 model

3.1.2 |. Comparing in vivo and in silico data

Table 1 compares the in silico with the in vivo measurements. The intention is to take the calculated GABA+ SNR and creatine linewidth of an in vivo dataset and to use these values in combination with the simulations to predict the underlying GABA+_error. Both datasets are largely congruent with the simulations in the measures of fit error and rejects; however, the precise numbers are only comparable to an order of magnitude. The variance (SD for in silico and CoV for in vivo) differs strongly between the in silico and the in vivo results. For the ‘Big GABA’ voxel, which was underlying the in silico data, an even better agreement was found.

TABLE 1.

Comparison between the GABA+ quantification of the simulations (in silico) and the in vivo measurements including the Big GABA data as well as the data measured in the present study. The data distribution is reported as SD for the simulations and as CoV for the in vivo measurements The estimation error from the different voxels was predicted from the estimation error of the simulations for a specific creatine linewidth and GABA+ SNR

			Estimation error [%]		Fit error [%]		Distribution [%]		Rejects [%]
	Creatine linewidth [Hz]	GABA+SNR	in silico	in vivo	in silico	in vivo	in silico (SD)	in vivo (CoV)	in silico	in vivo
Big GABA	7.9	24.4
Gannet default	–	–	0.3	–	3.1	4.0	2.7	10.4	9	6.25
Tarquin	–	–	0.1	–	6.0	5.7	1.4	14.7	0	2
Gannet broad	–	–	0.3	–	5.2	4.5	3.1	8.5	0	8.33
Small voxel	9.5	3.6
Gannet default	–	–	7.3	–	11.8	19.3	19.8	29.4	27	21
Tarquin	–	–	0.1	–	11.5	26.8	10.8	30	0	21
Gannet broad	–	–	1.8	–	13.8	19.9	17.5	35.5	13	21
Large voxel	10.4	8.4
Gannet default	–	–	7.1	–	5.6	8.6	11.4	25.2	7	0
Tarquin	–	–	3.6	–	7.7	11.4	4.1	13.6	0	0
Gannet broad	–	–	3.6	–	6.4	8.9	9.9	20.4	0	0

4 |. DISCUSSION

This study analyzed the influence of different SNR and linewidths of MEGA-PRESS spectra in order to identify recommendations for GABA spectroscopy studies.

In the first part, the effect of diminishing spectral quality on MEGA-PRESS spectroscopy, mimicked by artificial SNR and line broadening manipulations, was investigated in 427 different noise and creatine linewidth combinations with 100 simulated spectra for each possible combination. The GABA+ peaks of the difference spectra were modeled with two commonly used quantification tools (Gannet 3.0 and Tarquin). GABA+_error was calculated as the deviation from the known 3 ppm GABA+ resonance area, and common quality metrics (fit error, SD, and rejects) were determined.

All evidence combined, reliable GABA+ modeling is possible for GABA+ SNR > 3.2 and creatine linewidth < 9.7 Hz, which possess only 14% of the GABA+ SNR of the ‘gold standard’ 27 mL/10 min acquisition proposed in the literature.³⁶ The estimation error (ie the deviation from the known GABA+ amplitude) does not exceed 5% for Gannet 3.0 or Tarquin for these scenarios. However, for GABA+ SNR in the range of 5.4 to 2.5, GABA+_sd increases by 5% for Gannet 3.0 and 11% for Tarquin. Therefore, studies with lower GABA+ SNR are hampered by a loss of statistical power to detect small GABA+ changes with low effect sizes. Finally, GABA+ SNR changes at constant creatine linewidths are more critical to the GABA+ quantification than changes in the creatine linewidth at constant GABA+ SNR. For Tarquin, an estimation error of less than 5% was measured for creatine linewidths less than 11.6 Hz, while it ranges from 5 to 10% for Gannet 3.0 for the same scenarios. This suggests that the Tarquin algorithm may be less susceptible to linewidth change than the Gannet 3.0 algorithm, likely because the two-Gaussian model in Tarquin provides additional degrees of freedom. Thus, the choice of the quantification algorithm should be especially considered during investigation of regions that are susceptible to linewidth changes—eg the frontal cortices and thalamus.

First, GABA+_fit is not related to GABA+_error. Therefore, the real deviation from the ground truth cannot be judged by the fit quality³⁰ or other commonly used quality metrics for MRS. This fact should be considered carefully during the determination of the best algorithm to use for fitting the GABA+ resonance. Second, reliable GABA+ modeling is highly dependent on the quantification tool. GABA+_error exhibits differences between tools. GABA+_error > 5% was rarely seen for Tarquin, while it is passed by Gannet 3.0 for creatine linewidths greater than 10.6 Hz. Third, creatine linewidths affected GABA+_error only for very high GABA+ SNR values, while the effects on GABA+_fit, GABA+_SD, and the number of rejects were negligible. This implies that B₀ field homogeneity is, within commonly encountered ranges, not that critical a criterion for accurate modeling of the GABA+ resonance. SNR needs to be considered more carefully, which is in agreement with literature.¹⁸ Fourth, GABA+_SD differs between the quantification tools, with Tarquin presenting lower GABA+_SD than Gannet 3.0. This implies that the group variance of in vivo measurements—apart from biological variation within an in vivo cohort or variance introduced by changes in the voxel positioning—depends on the quantification algorithm used. Therefore, comparably small effect sizes could be obscured by variations introduced by the quantification tool. Despite the differences in the quality metrics, a strong correlation was found between the GABA+ model estimates of the two quantification tools for reasonable GABA SNR > 4.3 and creatine linewidth less than 11.6 Hz. A medium correlation was found over a relatively broad range of conditions. This indicates that variation in the model estimates of one model algorithm is similarly present for the other algorithm in this range. This also explains the low correlation for the high-SNR regions, where no considerable variation in the model estimates of either tool exists, which therefore leads to low correlation values for these conditions.

The additional analysis with the broader model in Gannet 3.0 indicates that the model choice has to be appropriate with regards to the data. Finally, the results illustrate that the commonly used quality metrics do not allow conclusions about the real GABA+_error.

In the second part of this study, 14 in vivo datasets were acquired in two thalamic voxels of different sizes, as this region is known to be complex for MRS. Afterwards, these spectra, as well as the data from the multi-site study,¹⁶ on which the template spectrum was based, were compared with the simulations. Additionally, an estimated GABA+_error was determined. As there is a good agreement between the in vivo and simulation datasets, the simulation may be used as an indicator either to judge the data quality of an already conducted study, or to classify a voxel conducted in a pilot measurement during study design. Future investigations should be executed to clarify, if the apparent differences in the data distribution between the in silico and the in vivo datasets could be interpreted as a variation solely attributed to biological variation.

Another possible contributor to the differences is the increased variability of the baseline in the in vivo study, as the baseline was not changed in the simulation. This additionally explains differences in the performance of each quantification tool regarding the simulations and the in vivo datasets. Furthermore, the slight differences in the acquisition parameters might explain the differences between the simulations and the thalamus voxels. Both aforementioned explanations are underlined by the good agreement between the simulations and the quality measures of the ‘Big GABA’ study,¹⁶ which was the foundation for the template. Additionally, it would be of interest to investigate the threshold noise level where significant differences remain visible. Therefore, the simulation space could be extended with different amplitudes of the GABA+ peak in the template spectrum. This approach would require a more sophisticated template spectrum, as proposed here.

Furthermore, the absence of correlation between the GABA+/Cr ratios of the small voxels reported by Gannet 3.0 and Tarquin indicates that the variation introduced by the modelling approach must be considered when comparing different studies. This is especially true for the differences in the baseline model of each algorithm. In addition, the GABA+/Cr ratios were found to be almost twice as high for Tarquin as for Gannet 3.0, which has also been found in a recent study.³⁷ This may be due to the proton scaling factor used in Tarquin to generate the basis set for the pseudo-doublet. However, the differences in the baseline models do not explain this difference. It also emphasizes the necessity for standardization, open source analysis practices, and transparent data reporting of MRS studies to make results more accessible and comparable between sites and tools.

Our results regarding the impact of GABA+ SNR on the group-level variance tally with a recent study on the design of GABA+-edited studies.¹⁸ Both studies indicate a reduction in quality gain, in terms of lower GABA+_SD, for higher numbers of averages. The GABA+ SNR of the large voxel in the present study is comparable to that of a 27 mL voxel with 320 averages, while the small voxel is comparable to 27 mL with 96 averages. As a conclusion of the in silico and in vivo measurements, GABA+-edited data reaches reliable quantification and reasonable group-level variance for 27 mL voxels with averages ranging from 128 and 210 compared with the commonly used 320.These values are only valid for similar T_R values and MEGA-PRESS ‘detectable’ GABA concentrations. The cut-off value in units of GABA+SNR determined from the simulations ranges between 7.2 and 5.4.

The present study was set up without macromolecule suppression. Under the assumption that 50% of the GABA+ signal can be attributed to macromolecules,³ the fitting behavior would presumably be similar to the template spectrum with half of the GABA signal. Therefore, the expected behavior for a certain SNR level can be drawn from Figures 3, 4, and 5. However, this remains speculative and should be investigated in future studies.

By comparing the results for the small in vivo voxel and the simulations with similar GABA+ SNR values, further conclusions can be drawn. The CoV is considerably higher for low GABA+ SNR and depends on the quantification tool. Finally, the number of rejected spectra increases for lower GABA+ SNR, which must be considered in studies with small voxels. Our results indicate that GABA+-edited data exhibiting a fit error larger than 15% lead to reliable spectra. Data rejection criteria should not exclusively be defined by GABA+ SNR or fit errors, but rather by considering the SD of the quantified GABA signal. Assuming GABA concentrations to be similar within one group (considering biological variability and pathologies to be consistent within that group), data quality estimation and outlier detection should be supported by GABA+_SD. However, the necessity of objective rejection criteria is enforced by the analysis in the Supporting Information (Figure S2), which revealed the effect of severe outliers on the calculated GABA+_errors.

Aside from the general knowledge gained about the impact of SNR and linewidth on GABA+-edited data, the present study provides insights on the performance of different quantification tools. The simulation approach presented in this work could be used to determine the performance of other quantification tools. Most notably, there is no widely adopted consensus on how to perform linear-combination modelling of GABA-edited difference spectra, for example with the widely used LCModel algorithm.³⁴ The approach recommended in the LCModel manual is to use a limited basis set (GABA, Glu, Gln, GSH, NAA, NAAG). While frequently being encountered in the literature, this is certainly not an appropriate model, since it does not account at all for the co-edited signals underlying the 3 ppm GABA peak (including homocarnosine, and the poorly characterized component from lysine-containing macromolecules). The resulting fit is therefore insufficient, resulting in considerable residuals. The approach presented in this manuscript may guide investigations into potential ways of addressing this unresolved question, eg including a homocarnosine basis function,³⁸ imposing soft constraints on parametrized MM signals,³⁹ including a measured (metabolite-nulled) macromolecule MEGA-PRESS background, or absorbing the co-edited signals into a highly flexible baseline spline.

Further studies on the performance of the common quantification tools are needed to fully understand the differences between the analytic methods, which could be achieved by using benchmark datasets collected in the ‘Big GABA’ multi-site studies.¹⁶

5 |. CONCLUSION

This study suggests that GABA+-edited studies might be realized for voxels with low GABA+ SNR at the cost of a higher group-level variance. Within the tested range the effect of decreasing GABA+ SNR at constant linewidth was found to be more critical than the effect of increasing linewidth at constant GABA+ SNR. Additionally, no relation between commonly reported quality metrics and GABA+ modeling accuracy was found. Furthermore, the adaption of the model to the data quality was found to be beneficial for the model estimates. Group variance induced by different quantification tools introduces additional unknown uncertainty, which might obscure group differences with small effect sizes in GABA+.

Abbreviations:

CoV

coefficient of variance

CRLB

Cramer-Rao lower bound

GABA

γ-aminobutyric acid

GABA+_error

GABA+ estimation error

GABA+_fit

GABA+ fit error

GABA+_SD

GABA+ standard deviation

MEGA-PRESS

Mescher-Garwood PRESS

NAA

N-acetyl-aspartate

N_EX

numberof excitations

SD

standard deviation

SNR

signal-to-noise ratio