Big GABA II: Water-Referenced Edited MR Spectroscopy at 25 Research Sites

Abstract

Accurate and reliable quantification of brain metabolites measured in vivo using ¹H magnetic resonance spectroscopy (MRS) is a topic of continued interest. Aside from differences in the basic approach to quantification, the quantification of metabolite data acquired at different sites and on different platforms poses an additional methodological challenge. In this study, spectrally edited γ-aminobutyric acid (GABA) MRS data were analyzed and GABA levels were quantified relative to an internal tissue water reference. Data from 284 volunteers scanned across 25 research sites were collected using GABA+ (GABA + co-edited macromolecules) and macromolecule-suppressed GABA editing. The unsuppressed water signal from the volume of interest was acquired for concentration referencing. Whole-brain T₁-weighted structural images were acquired and segmented to determine gray matter, white matter and cerebrospinal fluid voxel tissue fractions. Water-referenced GABA measurements were fully corrected for tissue-dependent signal relaxation and water visibility effects. The cohort-wide coefficient of variation was 17% for the GABA+ data and 29% for the MM-suppressed GABA data. The mean within-site coefficient of variation was 10% for the GABA+ data and 19% for the MM-suppressed GABA data. Vendor differences contributed 53% to the total variance in the GABA+ data, while the remaining variance was attributed to site- (11%) and participant-level (36%) effects. For the MM-suppressed data, 54% of the variance was attributed to site differences, while the remaining 46% was attributed to participant differences. Results from an exploratory analysis suggested that the vendor differences were related to the unsuppressed water signal acquisition. Discounting the observed vendor-specific effects, water-referenced GABA measurements exhibit similar levels of variance to creatine-referenced GABA measurements. It is concluded that quantification using internal tissue water referencing is a viable and reliable method for the quantification of in vivo GABA levels.

1. Introduction

In vivo ¹H magnetic resonance spectroscopy (MRS) allows noninvasive measurement of brain metabolite concentrations, but it does so only in a relative manner. Measurements usually rely on an internal reference signal and assumptions about the concentration of the reference compound. Common reference signals include the CH₃ singlets of the metabolites creatine (Cr) and N-acetylaspartate (NAA), or the unsuppressed brain tissue water signal from the same volume. Opinion in the field suggests that there is no reference signal that is optimal in all applications, and discussion is ongoing about the relative merits of each (Alger, 2010; Mullins et al., 2014).

The theory and empirical feasibility of the absolute quantification of metabolites as measured by MRS is well established (Barker et al., 1993; Christiansen et al., 1993; Danielsen and Henriksen, 1994; Ernst et al., 1993; Hennig et al., 1992; Kreis et al., 1993a; Thulborn and Ackerman, 1983). Later work has further refined these approaches, particularly with respect to using brain tissue water as an internal concentration reference (Gasparovic et al., 2018, 2006; Gussew et al., 2012; Knight-Scott et al., 2003). The typical procedure for using tissue water as an internal reference is to acquire an unsuppressed water signal using the same MRS acquisition protocol as used for the water-suppressed metabolite acquisition in a voxel co-localized to the volume of interest. With proper assumptions about certain properties of the metabolite and water signals in the various tissue compartments in the volume of interest, one may infer absolute metabolite concentrations from the acquired metabolite and reference signals. This is supported by the well-characterized properties of MR-visible water in the brain and its high concentration/large signal. On the other hand, using an endogenous metabolite signal, such as Cr, as a reference to derive metabolite ratios avoids the need for a separate water acquisition and may reduce error propagation that arises during more involved signal scaling procedures, but possibly at the expense of lower signal quality. Additionally, Cr is confined to brain tissue, whereas water is in brain and cerebrospinal fluid (CSF), making accurate corrections for CSF when using water as a concentration reference particularly important. At present, while strong opinions exist on the matter, both metabolite and water referencing have advantages and disadvantages (Jansen et al., 2006), and either approach is defensible. Indeed, the reliability of each approach has been shown to be similar (Bogner et al., 2010; Saleh et al., 2016), although in relatively small studies.

It is important to note that the concentration and relaxation properties of water, Cr and NAA can change in disease (Grasso et al., 2002; Huang et al., 2001; Kantarci et al., 2000; Laule et al., 2004; Rackayova et al., 2017), aging (Marjańska et al., 2017; Neeb et al., 2006; Reyngoudt et al., 2012) and brain development (Kreis et al., 1993b; Tkáč et al., 2003). Phantom replacement, scanning a phantom of a reference compound of known concentration for comparison to in vivo measurements (Buchli and Boesiger, 1993; Duc et al., 1998; Michaelis et al., 1993; Soher et al., 1996), can – with careful attention to differences in B₀/B₁ inhomogeneities, amplifier transmitter/receiver gains and RF coil loading factors – also be used to determine in vivo concentrations in absolute units. This method is technically challenging, involving additional experiments before or after the scan session, and is not commonly used given the difficulties of constructing a phantom with electric conductivity similar to human tissue and the extra time that is required for scanning the phantom (Jansen et al., 2006). An alternative approach is the ERETIC (electronic reference to access in vivo concentrations) method (Barantin et al., 1997; Zoelch et al., 2017), which relies on a synthetic RF reference signal. This approach is also challenging and requires specialized hardware. For all its limitations, internal concentration referencing remains the most practicable and widely used approach in in vivo ¹H MRS.

In addition to the nuances of different quantification methodologies, systematic differences in acquisition implementation and system hardware will have an impact on quantitative outcomes. This makes comparing MRS measurements collected across different sites and on different platforms non-trivial. If multi-site and multi-platform MRS studies are to be maximally useful, particularly in the era of “big data” (Bearden and Thompson, 2017; Miller et al., 2016; Van Essen et al., 2013), then the systematic effects on measurement variance must be assessed, understood and accounted for. This would then be followed by strategies for standardizing data acquisition, data processing and metabolite quantification methods.

The authors recently acquired a large multi-vendor, multi-site dataset, the purpose of which was to study the sources of variance in γ-aminobutyric acid (GABA) measurements collected by edited MRS. In the first paper describing this dataset (Mikkelsen et al., 2017), quantification was performed relative to the total Cr signal in the edit-OFF spectrum. In the current paper, GABA was quantified relative to brain tissue water, which additionally required accounting for individual differences in voxel tissue composition. In particular, this investigation aimed to determine whether quantification relative to water increases or decreases total variance (compared to Cr referencing) and discuss the impact of site- and vendor-related differences in structural image segmentation.

2. Methods

A fuller description of the acquisition and data processing methodology can be found in the original publication (Mikkelsen et al., 2017). Relevant details for this study, especially regarding quantification and tissue segmentation, are reported below.

2.1. Data collection

Data were acquired at 25 independent research sites, with each site contributing 5–12 datasets collected from consenting adult volunteers (cohort total: 284). Participants at each site were 18–35 years old, ~50% female and had no known neurological or psychiatric illnesses. Site-by-site participant demographics are provided in Table 1 in Mikkelsen et al. (2017). Scanning was conducted in accordance with ethical standards set by the institutional review board (IRB) at each site, including the sharing of anonymized data. Anonymized data files were shared securely with and analyzed by the co-authors at the Johns Hopkins University School of Medicine with local IRB approval.

Table 1.

Hardware and acquisition parameters used to collect 3D T₁-weighted structural images at each site.

Site ID	Tx/Rx hardware	Voxel resolution (mm3)	TE/TI/TR (ms)	Scan time (m:ss)	Flip angle (deg)	Slices	FOV (mm2)	Matrix size	Acceleration (factor)
G1	Body coil/32-ch head coil	0.94 × 0.94 × 1	2.68/600/7.42	4:07	10	226	256 × 256	256 × 256	ASSET (2)
G2	Body coil/8-ch head coil	0.9 × 0.9 × 1	2.73/650/6.24	2:54	8	180	256 × 256	256 × 256	ARC (2)
G3	Body coil/32-ch head coil	1 × 1 × 1	2.6/500/6.4	4:37	11	180	256 × 256	256 × 256	ASSET (2)
G4	Body coil/8-ch head coil	1 × 1 × 1	2.98/450/6.89	9:35	12	192	256 × 256	256 × 256	None
G5	Body coil/32-ch head coil	0.5 × 0.5 × 0.8	2.1/450/7.09	5:39	12	232	256 × 256	512 × 512	None
G6	Body coil/8-ch head coil	1 × 1 × 2	2.66/400/6.24	6:22	12	124	240 × 240	240 × 240	None
G7	Body coil/8-ch head coil	1 × 1 × 1	3.2/450/8.2	4:30	12	176	256 × 256	256 × 256	ARC (2)
G8	Body coil/8-ch head coil	1 × 1 × 1	4.17/450/10.19	5:27	12	180	256 × 256	256 × 256	ARC (2)
P1	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P2	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P3	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P4	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P5	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	170	256 × 256	256 × 256	SENSE (2)
P6	Body coil/8-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P7	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P8	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	204	256 × 256	256 × 256	SENSE (2)
P9	Body coil/32-ch head coil	1 × 1 × 1	3.1/865/6.9	7:10	8	176	256 × 256	256 × 256	SENSE (2)
S1	Body coil/32-ch head coil	1 × 1 × 1	2.52/900/1900	4:18	9	176	250 × 250	256 × 256	GRAPPA (2)
S2	Body coil/32-ch head coil	1 × 1 × 1	2.85/1050/1900	5:43	9	176	256 × 256	256 × 256	GRAPPA (2)
S3	Body coil/20-ch head/neck coil	1 × 1 × 1	1.77/900/1900	4:05	8	160	256 × 256	128 × 256	GRAPPA (2)
S4	Body coil/64-ch head coil	1 × 1 × 1	4.11/1000/2000	3:36	12	160	256 × 256	256 × 256	GRAPPA (2)
S5	Body coil/12-ch head coil	1 × 1 × 1	4.6/900/1950	4:01	9	176	192 × 256	192 × 256	GRAPPA (2)
S6	Body coil/32-ch head coil	1 × 1 × 1	2.26/900/1900	4:26	9	192	256 × 256	256 × 256	GRAPPA (2)
S7	Body coil/32-ch head coil	1 × 1 × 1	3.03/900/2300	5:21	9	192	256 × 256	256 × 256	GRAPPA (2)
S8	Body coil/64-ch head coil	1 × 1 × 1	3.02/900/1900	4:01	9	160	256 × 256	256 × 256	GRAPPA (2)

2.2. Data acquisition

GABA-edited MEGA-PRESS data (Mescher et al., 1998; Rothman et al., 1993) were collected at 3T at each site using a standard scan protocol. The MRI vendor breakdown was: eight GE; nine Philips; eight Siemens. Both GABA+-edited and macromolecule- (MM-) suppressed GABA-edited acquisitions were performed (Edden et al., 2012b; Henry et al., 2001). Complete details of the edited MRS acquisitions, including site-to-site idiosyncrasies, can be found in the earlier paper. Briefly, the GABA+ acquisition parameters were: TE = 68 ms; ON/OFF editing pulses = 1.9/7.46 ppm; editing pulse duration = 15 ms. The MM-suppressed GABA acquisition parameters were: TE = 68 ms (Siemens) or 80 ms (GE and Philips); ON/OFF editing pulses = 1.9/1.5 ppm; editing pulse duration = 20 ms. Common parameters were: TR = 2000 ms; 320 averages; 30 × 30 × 30 mm³ medial parietal lobe voxel (Fig. 1A). Six outer-volume saturation bands were applied in GE acquisitions (as is standard for GE PRESS-based sequences) but not in Philips or Siemens acquisitions (except for site S3, which used six saturation bands). Water suppression bandwidth scan parameters (not reported in Mikkelsen et al., 2017) were 140 Hz for Philips, 50 Hz for Siemens and 150 Hz for GE. Although some degree of control over the slice-selective gradient polarity (and the assignment of the three slice-selective pulses to the three physical dimensions of the voxel) is possible on all three vendors, no attempt was made to standardize these across vendors, nor were sites explicitly instructed to standardize gradient polarities (this was left at the discretion of scanner operators).

Fig. 1.

(A) Representative MRS voxel placement on a T₁-weighted structural image and probabilistic partial volume voxel maps following tissue segmentation for one participant. Corresponding tissue fractions of gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF) are shown. (B) Vendor-mean GABA-edited difference spectra acquired by GABA+ and MM-suppressed GABA editing. The gray patches represent ±1 standard deviation. The associated sample sizes are shown in parentheses.

Unsuppressed water signal acquisitions were collected for internal tissue water referencing. For the GE and Philips MEGA-PRESS implementations, the water reference was automatically acquired as part of the water-suppressed metabolite scans. For GE, the reference was acquired at the end of the water-suppressed acquisitions; 16 water averages were acquired. For Philips, the reference was acquired in an interleaved manner as the water signal was also used for real-time center frequency correction (Edden et al., 2016); a single water average was acquired for every 40 water-suppressed acquisitions (8 averages in total). Acquiring a water reference on the Siemens platform requires running a separate scan with identical receiver and transmitter gains. For this, the Siemens MEGA-PRESS WIP was used, where the water suppression RF pulses were turned off but the water suppression gradients and editing pulses were left on (“Only RF off” option); 8 or 16 water averages (depending on acquisition parameters) were acquired. The TE/TR of these acquisitions were the same as the corresponding water-suppressed acquisitions. Water reference acquisitions were acquired with the transmitter frequency shifted to the water frequency (as opposed to 3.0 ppm for water-suppressed acquisitions) on Philips and Siemens. For GE, both reference and water-suppressed data were acquired with the transmitter at 2.68 ppm.

Whole-brain 3D T₁-weighted structural images were acquired for accurate voxel placement and partial volume tissue correction. Sequences used were fast spoiled gradient-echo imaging (FSPGR; GE) (Low et al., 1993) and magnetization-prepared rapid gradient-echo imaging (MPRAGE; Philips/Siemens) (Mugler and Brookeman, 1990) (see Table 1 for acquisition parameters). Site-standard structural imaging protocols were used, with less effort to standardize acquisitions than the MRS protocols. Imaging data were saved in DICOM (GE and some Siemens sites) or NIfTI format (Philips and some Siemens sites). DICOM files were converted into NIfTI format for voxel segmentation and tissue segmentation purposes (see Section 2.4) using SPM12 (https://www.fil.ion.ucl.ac.uk/spm/software/spm12/).

2.3. Data processing

MRS data were processed in Gannet (Edden et al., 2014) using the pipeline described in the earlier report (Mikkelsen et al., 2017). Unsuppressed water acquisitions were processed in the same manner as the water-suppressed acquisitions and averaged. Briefly, processing steps included: frequency-and-phase correction by spectral registration (Near et al., 2015) (water-suppressed data only); 3-Hz exponential line broadening; zero-filling to yield a nominal spectral resolution of 0.061 Hz/point; and fast Fourier transformation into the frequency domain. Quality control and quality metrics were conducted and calculated as before. The linewidth of the water reference was measured as the full-width at half-maximum (FWHM) of the modeled water signal (see Section 2.5). As an independent measure of spectral linewidth, NAA FWHM linewidth was also measured from a Lorentzian-model fit of the NAA signal in the OFF spectrum.

2.4. Voxel co-registration and tissue segmentation

MRS voxels were co-registered to each volunteer’s structural image using the GannetCoRegister module in Gannet (Harris et al., 2015), which produces binary voxel masks in individual structural space. Structural images were segmented into gray matter (GM), white matter (WM) and CSF probabilistic partial volume maps using the unified tissue segmentation algorithm in SPM12 (Ashburner and Friston, 2005), executed through the GannetSegment module (Harris et al., 2015). GM, WM and CSF voxel volume fractions were calculated by multiplying the whole-brain partial volume maps for each tissue type by the corresponding binary voxel mask, summing over the partial volume estimates within each tissue-segmented voxel and then dividing these totals by the total over all three tissue-segmented voxels.

2.5. Quantification

The 3.0 ppm edited GABA signal was modeled as described previously (Mikkelsen et al., 2017). The water spectrum was modeled between 3.8 and 5.6 ppm with a Gaussian-Lorentzian function with phase and linear baseline parameters using nonlinear least-squares fitting. GABA measurements were quantified in pseudo-absolute molality units (approximating moles of GABA per kg of solute water) and corrected for partial volume effects (Gasparovic et al., 2006) based on the following equation:

$$C_{\text{G}}=\frac{I_{\text{G}}}{I_{\text{W}}}\cdot \frac{H_{\text{W}}}{H_{\text{G}}}\cdot \frac{MM}{\kappa }\cdot \left\{\frac{{\sum }_i^{\text{GM,WM,CSM}}{f}_{i,\text{vol}}{\beta }_i\text{exp}\left(-\frac{T{E}_{\text{W}}}{T_{2\text{W},i}}\right)\left[1-\text{exp}\left(-\frac{T{R}_{\text{W}}}{T_{1\text{W},i}}\right)\right]}{\left(f_{\text{GM},\text{vol}}{\beta }_{\text{GM}}+f_{\text{WM},\text{vol}}{\beta }_{\text{WM}}\right)\text{exp}\left(-\frac{T{E}_{\text{G}}}{T_{2\text{G}}}\right)\left[1-\text{exp}\left(-\frac{T{R}_{\text{G}}}{T_{1\text{G}}}\right)\right]}\right\}\cdot C_{\text{W}}$$ (1)

where C_G is the GABA concentration in institutional units (i.u.) and C_W is the assumed molal concentration of MR-visible solute water (55.51 mol/kg); I_G and I_W are the GABA and water signal integrals, respectively; H_W and H_G are the number of ¹H protons that give rise to the water and 3.0 ppm GABA signals (both 2), respectively; MM is a correction factor for the contribution of the co-edited macromolecule signal in the GABA+ signal, assumed to be 0.45 for GABA+ acquisitions and 1 for the MM-suppressed acquisitions; and κ is the editing efficiency, assumed to be 0.5. TE_G, TE_W, TR_G and TR_W are the echo and repetition times of the GABA-edited and water acquisitions, respectively. T_1W,i is the longitudinal relaxation time of water in GM (assumed to be 1331 ms), WM (assumed to be 832 ms) (Wansapura et al., 1999) or CSF (assumed to be 3817 ms) (Lu et al., 2005); T_2W,i is the transverse relaxation time of water in GM (assumed to be 110 ms), WM (assumed to be 79.2 ms) (Wansapura et al., 1999) or CSF (assumed to be 503 ms) (Piechnik et al., 2009). T_1G and T_2G are the longitudinal and transverse relaxation times of GABA, assumed to be 1310 and 88 ms, respectively (Edden et al., 2012a; Puts et al., 2013). The tissue-dependent water contents β_i (as defined in Ernst et al., 1993) are assumed to be 0.78, 0.65 and 0.97 for grey matter, white matter and CSF, respectively. f_i,vol is the volume fraction of GM, WM or CSF in the MRS voxel. Note that this equation equates to the molality equation in Gasparovic et al. (2006), with additional terms to account for MM fraction and editing efficiency that are particular to edited MRS of GABA.

Fit quality for the water peak model (the fit error) was assessed by normalizing the standard deviation (SD) of the model fit residuals to the amplitude of the modeled signal (Edden et al., 2014). This metric, the degree to which the measured signal cannot accurately be modeled as a Gaussian-Lorentzian, captures eddy current artifacts and some aspects of sub-optimal shimming.

To examine whether systematic effects on the variance of the GABA+ and MM-suppressed GABA data were attributed to the water acquisition, water-referenced Cr measurements were also quantified. The 3.0 ppm Cr signal in the OFF spectrum was modeled as described in the original publication. The longitudinal and transverse relaxation times of Cr were assumed to be 1350 and 154 ms, respectively (Mlynárik et al., 2001). MM and κ were not applied. Finally, the degree of association between participants’ water-referenced, tissue-corrected GABA values and their previously quantified GABA/Cr values as reported in Mikkelsen et al. (2017) was examined. It should be noted that these measurements are not independent (the GABA integral being a common factor) and, therefore, a strong correlation was expected.

2.6. Exploratory analysis

The results revealed systematically higher water-referenced GABA+ measurements from the Siemens sites as compared to the GE and Philips measurements (see Section 3). This level of variation was not apparent in the Cr-referenced GABA+ measurements that were previously reported (Mikkelsen et al., 2017). To reconcile this, an unplanned exploratory analysis was conducted in which the Siemens GABA measurements were referenced to a water signal acquired by a separate unsuppressed short-TE PRESS acquisition. These separate data were collected alongside the MEGA-PRESS data for the purpose of studying conventional MRS data acquired across the sites. The water reference data from this dataset enabled the unplanned exploratory analysis reported here. This acquisition was acquired at TE/TR = 35/2000 ms from a voxel in the same location as the MEGA-PRESS acquisition. Concentrations were quantified according to Eq. (1) without additional correction for any amplifier gain differences as it was assumed that the gain would not have changed between the PRESS and MEGA-PRESS acquisitions.

2.7. Statistical analysis

Linear mixed-effects models were fit to the water-referenced GABA data in R (version 3.5.2; R Core Team, 2018) using the lme4 package (Bates et al., 2015) and maximum likelihood for parameter estimation. An unconditional model (Eq. (1) in Mikkelsen et al., 2017) was fit to calculate variance partition coefficients (VPCs) to estimate the proportion of total variance attributed to vendor-, site- and participant-related effects. Secondary, conditional linear mixed-effects models (Eq. (5) in Mikkelsen et al., 2017) were also fit to the data to assess the impact of NAA linewidth, f_GM,vol, age and sex, and to test the association with GABA/Cr measurements. Goodness-of-fit was calculated as a log-likelihood statistic. Significance testing was performed using chi-square likelihood ratio tests, which were bootstrapped 2,000 times using parametric bootstrapping (Halekoh and Højsgaard, 2014). Effects were tested in the following order: vendor and site; NAA linewidth and f_GM,vol; age and sex. If an effect was significant, the relevant variable was retained in the next model; if not, it was removed. Unconditional linear mixed-effects models were also fit to the voxel tissue fractions to test for site and vendor effects. Post-hoc pairwise comparisons were corrected for multiple comparisons using the Holm-Bonferroni method (Holm, 1979). A p-value less than 0.05 was considered significant.

3. Results

GABA+ data from seven volunteers and MM-suppressed GABA data from 19 volunteers were removed from further analysis following quality control of the MRS data (largely due to excessive lipid contamination). All MM-suppressed GABA data from site G3 were excluded as consistent, excessive center frequency offsets (approximately –0.1 ppm on average) resulted in extremely small or absent GABA signals. One further dataset was removed because the unusually small water reference signal indicated an acquisition error. Vendor-mean GABA-edited difference spectra are shown in Fig. 1B.

Fig. 2A shows the GABA+ values arranged by site and by vendor. Mean ± 1 SD (standard error of the mean, SEM) GABA+ measurements were 3.32 ± 0.42 (0.04) i.u. for GE, 3.32 ± 0.36 (0.04) i.u. for Philips and 4.29 ± 0.49 (0.05) i.u. for Siemens. Siemens values were on average 29% higher than the GE (p_holm < 0.001) and Philips (p_holm < 0.001) values. The cohort-wide average was 3.61 ± 0.61 (0.04) i.u. Coefficients of variation (CVs) were 12.7%, 10.8% and 11.3% for GE, Philips and Siemens, and 16.9% across all sites and vendors. The mean within-site CV was 9.5%. Fig. 2B shows the MM-suppressed GABA values arranged by site and by vendor. Mean ± 1 SD MM-suppressed GABA measurements were 3.22 ± 1.02 (0.12) i.u. for GE, 3.52 ± 1.20 (0.12) i.u. for Philips and 3.60 ± 0.59 (0.07) i.u. for Siemens. Siemens MM-suppressed values were on average 12% higher than the GE (p_holm > 0.9) and only 2% higher than Philips (p_holm > 0.9) values. The cohort-wide average was 3.46 ± 1.00 (0.06) i.u. CVs were 31.7%, 34.0% and 16.4% for GE, Philips and Siemens, and 28.8% across all sites and vendors. The mean within-site CV was 18.7%. GM, WM and CSF fractions are displayed in Fig. 3. Across the cohort, the average (and CV of) f_GM,vol, f_WM,vol and f_CSF,vol was 0.59 ± 0.04 (6.9%), 0.28 ± 0.04 (14.0%) and 0.13 ± 0.05 (36.7%), respectively. Values of GABA+, f_GM,vol, f_WM,vol and f_CSF,vol for each site are listed in Table 2.

Fig. 2.

(A) Water-referenced GABA+ and (B) MM-suppressed GABA measurements fully corrected for partial volume effects, displayed by site and by vendor. The boxes shaded with lighter colors represent ±1 standard deviation and the darker boxes represent the 95% confidence interval. The solid white lines denote the mean, while the dashed white lines denote the median. Sites are colored by vendor (GE sites in green, Philips sites in orange, Siemens sites in blue).

Fig. 3.

Gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF) voxel tissue fractions, displayed by site and by vendor. GM = gray fill; WM = white fill; CSF = black fill. The red lines denote the mean. Sites are colored by vendor (GE sites with a green background, Philips sites with an orange background, Siemens sites with a blue background).

Table 2.

Quantification, tissue fractions and (TE = 68 ms) water data quality metrics, displayed by site and by vendor (shown as mean ± 1 standard deviation (CV)).

Site ID	GABA+ (i.u.)	MM-s GABA (i.u.)	Cr (i.u.)	fGM,vol	fWM,vol	fCSF,vol	Water fit error (%)	Water linewidth (Hz)
G1	3.33 ± 0.20 (6.1%)	4.40 ± 0.61 (13.9%)	11.69 ± 1.00	0.60 ± 0.05	0.26 ± 0.04	0.14 ± 0.03	0.95 ± 0.09	9.43 ± 0.48
G2	3.70 ± 0.26 (7.1%)	4.04 ± 0.78 (19.3%)	11.19 ± 0.61	0.56 ± 0.02	0.29 ± 0.03	0.15 ± 0.03	0.57 ± 0.11	9.98 ± 0.73
G3	2.92 ± 0.26 (9.0%)	DE	10.16 ± 0.69	0.60 ± 0.03	0.29 ± 0.02	0.11 ± 0.03	0.43 ± 0.09	9.46 ± 0.76
G4	3.31 ± 0.39 (11.7%)	2.95 ± 0.93 (31.6%)	10.54 ± 0.52	0.60 ± 0.03	0.28 ± 0.02	0.12 ± 0.04	0.42 ± 0.06	9.34 ± 0.40
G5	2.96 ± 0.20 (6.9%)	2.23 ± 0.54 (24.1%)	10.33 ± 0.41	0.65 ± 0.05	0.26 ± 0.01	0.09 ± 0.04	0.55 ± 0.11	9.79 ± 0.63
G6	3.22 ± 0.59 (18.2%)	DNA	9.65 ± 0.76	0.54 ± 0.04	0.32 ± 0.02	0.15 ± 0.04	0.65 ± 0.07	9.80 ± 0.92
G7	3.44 ± 0.38 (11.1%)	2.83 ± 0.84 (29.7%)	11.28 ± 0.55	0.57 ± 0.05	0.23 ± 0.02	0.19 ± 0.06	0.47 ± 0.11	8.72 ± 0.88
G8	3.46 ± 0.41 (11.7%)	2.81 ± 0.43 (15.2%)	11.10 ± 0.49	0.57 ± 0.03	0.23 ± 0.05	0.21 ± 0.05	0.41 ± 0.09	8.40 ± 0.42
AUGE	3.32 ± 0.42 (12.7%)	3.22 ± 1.02 (31.7%)	10.78 ± 0.90	0.58 ± 0.05	0.27 ± 0.04	0.15 ± 0.05	0.56 ± 0.19	9.37 ± 0.84
P1	3.30 ± 0.37 (11.1%)	3.47 ± 0.33 (9.6%)	11.46 ± 0.78	0.60 ± 0.03	0.27 ± 0.04	0.13 ± 0.04	0.47 ± 0.07	8.78 ± 0.53
P2	3.18 ± 0.25 (7.8%)	3.16 ± 0.71 (22.4%)	11.55 ± 0.71	0.56 ± 0.03	0.29 ± 0.02	0.15 ± 0.04	0.43 ± 0.07	8.74 ± 0.40
P3	3.29 ± 0.26 (7.8%)	3.66 ± 0.88 (24.1%)	11.24 ± 0.94	0.58 ± 0.02	0.29 ± 0.03	0.13 ± 0.03	0.84 ± 0.11	9.02 ± 0.43
P4	3.48 ± 0.58 (16.5%)	2.05 ± 0.65 (31.7%)	11.86 ± 0.56	0.59 ± 0.02	0.26 ± 0.02	0.15 ± 0.03	0.32 ± 0.06	8.78 ± 0.38
P5	3.03 ± 0.25 (8.1%)	2.61 ± 0.32 (12.3%)	11.32 ± 0.75	0.63 ± 0.03	0.27 ± 0.02	0.11 ± 0.03	0.75 ± 0.08	9.06 ± 0.24
P6	3.56 ± 0.27 (7.7%)	4.07 ± 1.42 (34.9%)	13.19 ± 1.19	0.57 ± 0.02	0.25 ± 0.03	0.18 ± 0.05	0.67 ± 0.17	8.71 ± 0.53
P7	3.10 ± 0.22 (7.0%)	2.87 ± 0.55 (19.3%)	11.88 ± 0.47	0.63 ± 0.03	0.27 ± 0.03	0.10 ± 0.03	0.69 ± 0.11	10.21 ± 0.62
P8	3.59 ± 0.33 (9.1%)	5.70 ± 0.40 (7.1%)	13.84 ± 0.54	0.61 ± 0.04	0.28 ± 0.03	0.11 ± 0.04	0.40 ± 0.04	9.03 ± 0.29
P9	3.31 ± 0.22 (6.7%)	4.10 ± 0.40 (9.6%)	11.13 ± 0.93	0.59 ± 0.02	0.28 ± 0.04	0.12 ± 0.03	0.50 ± 0.06	8.77 ± 0.31
All Philips	3.32 ± 0.36 (10.8%)	3.52 ± 1.20 (34.0%)	11.95 ± 1.17	0.59 ± 0.04	0.27 ± 0.03	0.13 ± 0.04	0.56 ± 0.19	9.01 ± 0.62
S1	4.17 ± 0.23 (5.5%)	3.39 ± 0.56 (16.5%)	14.76 ± 0.84	0.57 ± 0.03	0.30 ± 0.03	0.12 ± 0.05	0.39 ± 0.08	9.40 ± 0.72
S2	4.74 ± 0.39 (8.1%)	3.76 ± 0.52 (13.7%)	17.24 ± 0.28	0.55 ± 0.02	0.33 ± 0.02	0.12 ± 0.03	0.38 ± 0.03	9.22 ± 0.36
S3	3.87 ± 0.41 (10.6%)	3.75 ± 0.76 (20.1%)	14.86 ± 0.98	0.56 ± 0.03	0.33 ± 0.04	0.11 ± 0.04	0.30 ± 0.06	8.63 ± 0.39
S4	4.28 ± 0.48 (11.1%)	3.22 ± 0.55 (17.1%)	14.05 ± 0.57	0.61 ± 0.03	0.29 ± 0.02	0.10 ± 0.04	0.33 ± 0.05	8.81± 0.31
S5	4.46 ± 0.72 (16.2%)	3.73 ± 0.63 (17.0%)	14.45 ± 1.60	0.59 ± 0.05	0.31± 0.04	0.10 ± 0.07	0.42 ± 0.11	9.45 ± 0.92
S6	4.08 ± 0.17 (4.2%)	3.86 ± 0.37 (9.5%)	14.57 ± 0.42	0.59 ± 0.04	0.30 ± 0.03	0.11± 0.03	0.44 ± 0.08	9.12 ± 0.28
S7	4.43 ± 0.34 (7.8%)	3.55 ± 0.52 (14.7%)	13.88 ± 0.96	0.58 ± 0.03	0.28 ± 0.03	0.14 ± 0.04	0.46 ± 0.08	8.98 ± 0.39
S8	4.50 ± 0.50 (11.0%)	3.46 ± 0.61 (17.6%)	15.52 ± 0.61	0.58 ± 0.03	0.29 ± 0.03	0.13 ± 0.02	0.33 ± 0.07	8.84 ± 0.36
All Siemens	4.29 ± 0.49 (11.3%)	3.60 ± 0.59 (16.4%)	14.73 ± 1.19	0.58 ± 0.04	0.30 ± 0.04	0.12 ± 0.04	0.38 ± 0.10	9.04 ± 0.57
Overall	3.61 ± 0.61 (16.9%)	3.46 ± 1.00 (28.8%)	12.42 ± 1.94	0.59 ± 0.04	0.28 ± 0.04	0.13 ± 0.05	0.51 ± 0.19	9.14 ± 0.70

CV, coefficient of variation; DE, data excluded; DNA, data not acquired; MM-s, MM-suppressed.

Water fit errors and linewidths and Cr measurements (for TE = 68 ms data) are displayed in Fig. 4, with site- and vendor-averaged values given in Table 2. On average, fit errors were small, and lower for Siemens (0.38%) compared to GE (0.56%) and Philips (0.56%). Water linewidths were similar across the vendors, with Philips (9.01 ± 0.62 Hz) and Siemens (9.04 ± 0.57 Hz) showing slightly smaller linewidths compared to GE (9.37 ± 0.84 Hz). Average water-referenced Cr measurements were 14.6 ± 1.2 i.u. for GE, 16.2 ± 1.6 i.u. for Philips and 20.0 ± 1.6 i.u. for Siemens, based on TE = 68 ms data. The Siemens MM-suppressed acquisition, also acquired at TE = 68 ms, gave an average Cr measurement of 21.7 ± 2.1 i.u. Cr measurements from MM-suppressed acquisitions acquired at TE = 80 ms were 16.9 ± 1.2 i.u. for GE and 18.1 ± 1.7 i.u. for Philips, 14% higher on average.

Fig. 4.

Quality metrics and water-referenced Cr measurements (from the TE = 68 ms data), displayed by site and by vendor. (A) water fit error; (B) water linewidth; (C) Cr measurements fully corrected for partial volume effects. The boxes shaded with lighter colors represent ±1 standard deviation and the darker boxes represent the 95% confidence interval. The solid white lines denote the mean, while the dashed white lines denote the median. Sites are colored by vendor (GE sites in green, Philips sites in orange, Siemens sites in blue).

3.1. Linear mixed-effects analyses

A summary of the linear mixed-effects analyses on the GABA+ measurements is given in Table S1. The unconditional linear mixed-effects model showed that vendor and site effects contributed significantly to the total amount of variance in the data: χ²(1) = 28.36, p_boot = 0.001 and χ²(1) = 28.89, p_boot < 0.001, respectively. Based on the calculated VPCs, 53.6% of the variance was accounted for by vendor-level differences, while 10.7% was accounted for by site-level differences. The remaining proportion of variance (35.7%) was attributed to individual differences in participants (see Table 3). The same model applied to MM-suppressed GABA showed that site effects contributed significantly to the total variance: χ²(1) = 131.53, p_boot = 0.001, but vendor did not: χ²(1) < 0.1, p_boot = 0.65 (see Table S2). The corresponding VPCs were 53.6% (site level) and 46.4% (participant level) (see Table 3).

Table 3.

Summary of variance partition analyses. Cr-referenced values are reproduced from Mikkelsen et al., 2017.

	GABA+ (i.u.)	MM-suppressed GABA (i.u.)	GABA+/Cr	MM-suppressed GABA/Cr
Vendor	53.6%	-	8.2%	-
Site	10.7%	53.6%	19.7%	50.4%
Participant	35.7%	46.4%	72.1%	49.6%

The voxel tissue fractions exhibited significant site-related effects for f_GM,vol [χ²(1) = 56.33, p_boot = 0.001], f_WM,vol [χ²(1) = 46.77, p_boot < 0.001] and f_CSF,vol [χ (1) = 47.22, p_boot < 0.001], but only f_WM,vol showed an additional vendor-related effect [χ²(1) = 4.08, p_boot = 0.01]. Corresponding VPCs for f_GM,vol were: vendor < 0.1%; site = 31.4%; participant = 68.6%. For f_WM,vol these were: vendor = 12.8%; site = 24.9%; participant = 62.3%. For f_CSF,vol these were: vendor = 1.8%; site = 28.5%; participant = 69.7%. Pairwise comparisons showed that, at the vendor level, the Siemens f_WM,vol values were significantly higher than the GE (p_holm = 0.001) and Philips (p_holm = 0.003) values.

Based on the conditional linear mixed-effects analyses, neither GABA+ nor MM-suppressed GABA levels were significantly impacted by the effects of NAA linewidth [χ²(5) = 4.65, p_boot = 0.21; χ²(3) = 1.19, p_boot = 0.61]. GABA+ levels did, however, show a relationship with f_GM,vol [χ²(5) = 10.28, p_boot = 0.02], whereas MM-suppressed GABA levels did not [χ²(3) = 5.50, p_boot = 0.08]. There were no significant effects of age [χ²(5) = 1.19, p_boot = 0.75; χ²(3) = 0.85, p_boot = 0.69] or sex [χ²(5) = 1.56, p_boot = 0.67; χ²(3) = 2.34, p_boot = 0.35] on the GABA+ or MM-suppressed data, respectively. As shown in Fig. 5, the water-referenced and Cr-referenced measurements were strongly related, for both GABA+ and MM-suppressed acquisitions [χ²(5) = 257.2, p_boot < 0.001; χ²(3) = 495.7, p_boot < 0.001] as expected.

Fig. 5.

Scatterplots illustrating the relationship between (A) water-referenced GABA+ measurements and GABA+/Cr ratios and (B) water-referenced MM-suppressed GABA measurements and MM-suppressed GABA/Cr ratios. Individual measurements are color-coded by vendor (GE in green, Philips in orange, Siemens in blue). The black regression line shows the relationship between GABA+/Cr and water-referenced GABA+ over the entire dataset. Additional color-coded regression lines are shown for each site. R² values (i.e., the effect sizes) are also displayed.

3.2. Exploratory analysis

Using the Siemens short-TE water reference brought down the Siemens GABA+ values to an average of 3.65 ± 0.44 i.u. (an 18% reduction), reducing the discrepancy with the other vendors to 10% (boxplots plotted in Fig. S1). Corresponding VPCs were: vendor = 12.0%; site = 24.2%; participant = 63.8%, with vendor and site effects remaining significant: χ²(1) = 3.92, p_boot = 0.01 and χ²(1) = 32.77, p_boot < 0.001, respectively. The short-TE-referenced Siemens Cr measurements were also reduced on average (by 18%, to 17.1 ± 2.1 i.u.) compared to the initial analysis, again closer to the GE and Philips Cr measurements (Fig. S2).

4. Discussion

In this second paper describing a large multi-vendor, multi-site GABA-edited MRS dataset, it has been shown that water-referenced GABA measurements, including tissue correction based on variously acquired T₁-weighted structural images, can be applied across sites and vendors with relatively low levels of variance. Water-referenced quantification shows very similar levels of performance to Cr referencing, as reported previously (Mikkelsen et al., 2017), with the notable exception of an additional vendor-related effect.

4.1. Water vs. Cr referencing

One objective of this study was to compare quantitative outcomes of water and Cr referencing. Within site, water- and Cr-referenced GABA+ measurements both show variance of 9.5%. Levels of site-related variance are also similar (mean within-vendor CV: 11.6% vs. 11.3%, respectively). The major difference between the water- and Cr-referenced results was the systematic effect of vendor in the water-referenced data.

It is not clear why the water-referenced GABA+ values from the Siemens data were larger than the estimates from the other two vendors. This substantial vendor-related difference was not observed in the Cr-referenced data reported previously (Mikkelsen et al., 2017). Higher GABA+ values suggest a lower-than-expected water signal. Referencing to a short-TE PRESS water acquisition attenuated the discrepancy somewhat, suggesting that the Siemens MEGA-PRESS WIP water reference signal is most likely at issue. It is notable that the water fit errors were lower in the Siemens data, suggesting that the water signal is closer to a Gaussian-Lorentzian lineshape than the other vendors, and that the data undergo differing degrees of preprocessing (e.g., downsampling from the analog-to-digital converter sampling rate to the specified acquisition rate), with potentially different dynamic range performance – the water reference signal is ~10,000 times larger than the GABA signal and acquired with the same receiver gains. MM-suppressed data on Siemens were acquired with TE = 68 ms, compared to TE = 80 ms on the other vendors, a decision made in response to the TE-independent timing of editing pulses in that sequence. This clear difference in acquisition approach (which can only really be avoided by further sequence standardization; see, e.g., Saleh et al., 2019) means the MM-suppressed data offer little insight on the vendor water signal scaling effect. At this stage, it has not been possible to isolate the cause of this discrepancy, and communication with Siemens experts has not yielded a conclusive explanation.

Aside from this vendor effect, it was clear that the variation in the water-referenced GABA+ measurements was similar to the GABA+/Cr measurements. This suggests that the reliability of the two referencing strategies is comparable, consistent with previous smaller studies (Bogner et al., 2010; Saleh et al., 2016). This is perhaps surprising as several additional corrections were performed to obtain the water-referenced values, which introduces more sources of error into the quantification.

4.2. Water referencing approach

Quantification in this study used a best-practice approach, whereby the water reference signal was corrected for partial volume effects and relaxation. It has been shown that failing to account for these effects will lead to quantification errors (Gasparovic et al., 2006, 2018). These errors can be particularly pronounced at longer TEs (Gasparovic et al., 2006) or when there is large variability in tissue compartment fractions across cohorts (Harris et al., 2015; Mato Abad et al., 2014; Mikkelsen et al., 2016; Tal et al., 2012). The relatively low level of variance in the present dataset suggests that incorporating image-based voxel segmentation into the quantification routine did not add significant variance into the data.

Nonetheless, subtle differences in quantification and tissue correction methodologies can have important consequences on reported results. For instance, the linear relationship between GABA+ levels and age, seen when applying a simple CSF tissue correction, can largely be attributed to the dependency of GABA+ levels on tissue composition (Maes et al., 2018; Porges et al., 2017). Additionally, the units of measurement of water-referenced metabolite concentrations, and the information content of the values, will depend on the quantification approach used. Metabolite concentrations have been reported in molar, molal and institutional units (Gasparovic et al., 2018; Jansen et al., 2006; Knight-Scott et al., 2003; Kreis et al., 1993a). Interpretation of stated concentrations, and particularly comparisons across studies, can be challenging. In this study, measurements were reported in institutional units of molality.

Not every source of variance was captured in the statistical analysis. In the statistical model used here, participant-level variance not only accounts for true biological differences between individuals but also measurement error. One systematic site-level factor was the diversity of the T₁-weighted structural imaging protocols, leading to heterogeneity in T_1-weighted contrast and image signal-to-noise ratio, which lead to small but significant site-to-site differences in voxel segmentation. There is an extensive literature on the successes and limitations of image segmentation (Clark et al., 2006; Eggert et al., 2012; Klauschen et al., 2009), and while segmentation algorithms aim to be robust against the effects of imaging parameters, segmentation remains a challenging undertaking. The substantial tissue differences in water T₂, particularly given the medium TE of the water acquisitions (68 ms), and in GABA and MR-visible water concentrations, suggest that accurate segmentation is important for reproducible water-referenced quantification. It should also be noted that the MRS voxel masks that are generated for tissue segmentation purposes correspond to the nominal excited volume, which, because of acquisition parameters, voxel placement and chemical shift displacement errors, will not necessarily be centered on the excited signals of interest. In this study, the Philips and Siemens voxel masks were centered on the GABA and water reference signals, whereas the GE voxel masks centered on the 2.68 ppm signals (see Section 2.2). This variation will have introduced a degree of error in the water-referenced GABA measurements.

4.3. Macromolecular contamination

In quantifying the 3.0 ppm GABA+ signal in this work, a fixed signal fraction of co-edited macromolecule contribution to the peak was assumed, an assumption that may not be valid for studies in neurological and psychiatric disorders. As presented previously (Mikkelsen et al., 2017) for Cr-referenced values, MM-suppressed measures showed greater relative variance than GABA+ measures. The degree of similarity between Cr-referenced and water-referenced measurements was similar for the macromolecule-suppressed and GABA+ datasets.

4.4. Towards further standardization

As highlighted in (Mikkelsen et al., 2017), it is likely that differences in acquisition sequence contribute to variance at the level of both vendor (e.g., factors inherited from vendor PRESS sequences) and site (e.g., differences in implementation of MEGA editing), which might be removed/attenuated by further standardization. In a parallel piece of work, we have implemented a cross-vendor standardized MEGA-PRESS sequence on GE, Philips, Siemens and Canon systems (Saleh et al., 2019), which addresses some of these issues.

4.5. Conclusion

In summary, GABA+ levels were quantified using brain tissue water as an internal concentration reference across 25 sites and low levels of within-site variance were observed. This level of variability is similar to that seen for GABA+ measurements quantified relative to Cr. Given the concern that observed effects might be driven by changes in the reference signal, it is often helpful to quantify both water- and metabolite-referenced measurements. Study-specific expectations of reference signal stability (e.g., between-group differences in clinical populations or known changes in water content and signal relaxation) might suggest one concentration reference a priori. That said, the present results do not show a clear reason to prefer one reference signal for MRS quantification, and it can be concluded that water-referenced measurements of GABA+ are sufficiently reliable to be applied in multi-site studies.

Appendix

A subset of the data presented in this work has been made available on the NITRC portal in the “Big GABA” project repository (https://www.nitrc.org/projects/biggaba/) and is distributed freely under a non-commercial Creative Commons license. Community members are encouraged to make use of this resource for developing and optimizing new MRS methods. This data resource can also serve as a normative dataset against which clinical data may be compared or for quality assurance purposes.