Multiparameter mapping of relaxation (R1, R2*), proton density and magnetization transfer saturation at 3 T: A multicenter dual‐vendor reproducibility and repeatability study

Abstract

Multicenter clinical and quantitative magnetic resonance imaging (qMRI) studies require a high degree of reproducibility across different sites and scanner manufacturers, as well as time points. We therefore implemented a multiparameter mapping (MPM) protocol based on vendor's product sequences and demonstrate its repeatability and reproducibility for whole‐brain coverage. Within ~20 min, four MPM metrics (magnetization transfer saturation [MT], proton density [PD], longitudinal [R1], and effective transverse [R2*] relaxation rates) were measured using an optimized 1 mm isotropic resolution protocol on six 3 T MRI scanners from two different vendors. The same five healthy participants underwent two scanning sessions, on the same scanner, at each site. MPM metrics were calculated using the hMRI‐toolbox. To account for different MT pulses used by each vendor, we linearly scaled the MT values to harmonize them across vendors. To determine longitudinal repeatability and inter‐site comparability, the intra‐site (i.e., scan‐rescan experiment) coefficient of variation (CoV), inter‐site CoV, and bias across sites were estimated. For MT, R1, and PD, the intra‐ and inter‐site CoV was between 4 and 10% across sites and scan time points for intracranial gray and white matter. A higher intra‐site CoV (16%) was observed in R2* maps. The inter‐site bias was below 5% for all parameters. In conclusion, the MPM protocol yielded reliable quantitative maps at high resolution with a short acquisition time. The high reproducibility of MPM metrics across sites and scan time points combined with its tissue microstructure sensitivity facilitates longitudinal multicenter imaging studies targeting microstructural changes, for example, as a quantitative MRI biomarker for interventional clinical trials.

We developed and optimized a quantitative acquisition technique, which enables the fast high‐resolution mapping of multiple parameters on standard clinical MRI hard‐ and software. A travelling heads study on five healthy volunteers demonstrated its excellent reproducibility and comparability across six different sites with MRI scanners from two different vendors. The low inter‐site bias (<5%) for multicenter trials and the low intra‐scan variances (between 4 and 10% for R1, PD, and MT) suits it ideally for multi‐center longitudinal studies—such as clinical interventional studies.

1. INTRODUCTION

Multicenter clinical studies using magnetic resonance imaging (MRI) increase the size and diversity of the study population (Van Horn & Toga, 2009). Quantitative MRI (qMRI) can improve comparability across time points and sites (Cercignani, Dowell, & Tofts, 2018; Weiskopf et al., 2013). Moreover, qMRI metrics allow the tissue microstructure, such as myelin, iron, and the neuropil, to be indirectly probed, by separating disparate contrast mechanisms unlike standard contrast‐weighted MRI (Edwards, Kirilina, Mohammadi, & Weiskopf, 2018; Weiskopf, Mohammadi, Lutti, & Callaghan, 2015). Thus, qMRI has been applied in several multicenter studies (Weiskopf et al., 2013) and clinical trials, providing imaging biomarkers in the development of new outcome measures (Deoni et al., 2008; Filippi & Agosta, 2007; Freund et al., 2019; Horsfield et al., 2003; Mahajan & Ontaneda, 2017; Willcocks et al., 2016).

The multiparameter mapping (MPM) method (Helms, Dathe, Kallenberg, & Dechent, 2008; Weiskopf et al., 2013) provides quantitative high‐resolution maps (magnetization transfer saturation [MT], proton density [PD], longitudinal and effective transverse relaxation [R1, R2*]) sensitive to myelin and iron content in the brain within clinically feasible scan times (~20 min). MPM has been applied in a wide range of studies to investigate microstructural tissue properties in gray and white matter and has proven its potential to provide quantitative imaging biomarkers (Callaghan et al., 2014; Freund et al., 2013; Grabher et al., 2015; Seif et al., 2018; Weiskopf et al., 2013; Ziegler et al., 2018).

A previous multicenter study applying the MPM technique showed reproducibility of the quantitative maps using custom‐made sequences on the same MRI scanner model (Siemens Trio, software version VB17) (Weiskopf et al., 2013). Moreover, MPM has been applied in longitudinal studies to investigate microstructural brain changes induced by spinal cord injury (SCI) (Freund et al., 2013; Grabher et al., 2015; Villiger et al., 2015). Based on the reported inter‐site comparability and sensitivity to brain changes, the MPM protocol is currently being applied as an MRI outcome measure in an SCI clinical trial. More specifically, the multicenter, multinational, placebo‐controlled, phase‐II clinical trial NISCI (www.nisci-2020.eu) is using MPM to investigate the safety and preliminary efficacy of intrathecal anti‐Nogo‐A [NG101] in patients with acute SCI (Kucher et al., 2018). However, implementing a qMRI protocol in multicenter studies such as the NISCI trial requires careful coordination. Considerations involve how differences in scanner hardware and software can influence the MRI outcome measures and potentially lead to conflicting results.

This study therefore aimed (a) to optimize the MPM protocol and processing pipeline based on the vendors' product sequences customized for clinical trials (rather than custom sequences) and (b) to test the protocol across different MRI scanner types in the form of a traveling heads study. In this article we report the scan‐rescan repeatability and inter‐site comparability of the MPM protocol across six different clinical sites involved in the NISCI trial.

2. METHODS

2.1. Subjects and sites

The study was conducted on six 3 T MRI systems with different hardware and software (Table 1). Four scanners were manufactured by Siemens Healthineers (Erlangen, Germany) and two by Philips Healthcare (Eindhoven, Netherlands). Five healthy subjects (2 female, 3 male, aged 32.4 ± 6.0 years [mean ± SD]) were scanned twice each (i.e., scan‐rescan) at each site with an average inter‐scan interval of 2 hr between measurements. Informed written consent was obtained from each subject prior to each scan, and all sites obtained local ethical approval. Local radiographers, who were also involved in the NISCI study, where possible, performed the scans.

TABLE 1.

MRI scanning sites and main hardware and software specifications

Site	MRI vendor	MRI system	Number of channels of RF receive head coil	MRI software version
BCN	Siemens	Verio	32	VD13A
BSL	Siemens	Prisma	20	VE11C
HD	Siemens	Verio	16	VB19A
NOT	Philips	Achieva	16	5.3.0
ZH	Siemens	Skyra	16	VE11B
SNS	Philips	Achieva	16	5.1.7

Note: BCN: Clinica Creu Blanca, Barcelona, Spain; BSL: Radiology Department, University Hospital Basel, Basel, Switzerland; HD: Spinal Cord Injury Center, Univerisity Hospital, Heidelberg, Germany; NOT: Radiology, Swiss Paraplegic Center, Nottwill, Switzerland; ZH: Spinal Cord Injury Center, University Hospital Balgrist, University of Zurich, Zurich, Switzerland; SNS: Laboratory for Social and Neural Systems Research, Zurich Center for Neuroeconomics, University of Zurich, Zurich, Switzerland.

2.2. MRI acquisition

The MPM protocol was implemented based on product sequences available on the manufacturer's clinical MRI systems. Three‐dimensional (3D) data acquisition was composed of three multi‐echo spoiled gradient echo scans (i.e., fast low angle shot [FLASH] sequences on Siemens scanners and multi‐echo fast field echo [mFFE] sequences on Philips scanners) with MT, T1, and PD contrast weightings. Additional reference scans for bias correction using the hMRI‐toolbox (RRID: SCR_007037) (Tabelow et al., 2019) included mapping of the radio‐frequency (RF) transmit (B1⁺) and receive fields (B1⁻) on both vendor scanner platforms. The total acquisition time was 18:45 min on the Siemens scanners and 23:58 min on the Philips scanners.

Generally, the parameters of the multi‐echo gradient echo sequences were chosen with the aim of keeping scan time short (~20 min) and to achieve an ~1 mm isotropic resolution with a high image quality. A protocol previously used in a study on spinal cord injury served as the starting point for the protocol optimization, since it allowed successful monitoring of longitudinal injury related changes (Freund et al., 2013). The total number of acquired echoes, repetition time (TR), and maximum echo time (TE) were reduced to further shorten the scan time, while still allowing for reliable mapping of R2* in subcortical areas with typically high R2* values. The spacing between the different echoes was determined by using a high readout bandwidth of 480 Hz/pixel, in order to minimize chemical shift artifacts and achieve a high number of echoes for improved signal decay modeling. The excitation flip angles of the T1‐ and PD‐weighted gradient echo sequences were based on the median nominal Ernst angle for brain tissue, scaled by a factor of 0.4142 and 2.4142, respectively, in order to minimize noise propagation into the maps of the brain (Dathe & Helms, 2010). In the MT‐weighted gradient echo sequence, a small constant flip angle of 6° was chosen to control its systematic influence on the MT maps (Helms, Dathe, Weiskopf, & Dechent, 2011). The parameters of the MT saturation pulse were limited to the vendors' default product sequence settings (Table 2), since changes would have required pulse sequence programming and would not have been feasible for a clinical study. MT values were harmonized across vendor platforms in the postprocessing. The minimum TR for MT‐weighted sequences on the Philips platforms was driven by specific absorption rate (SAR) constraints. The RF spoiling characteristics differ between vendors' sequence implementations (Table 2), which was accounted for in the postprocessing.

TABLE 2.

Different MRI parameters on the Philips and Siemens scanners

Acquisition parameter	Siemens	Philips
Minimum TE and ΔTE (ms)	2.46	2.40
Maximum TE (ms)	14.76	14.40
Acquisition resolution (mm3)	1.1 × 1.0 × 1.1	1.0 × 1.0 × 1.0
FoV slice resolution (%)	91.0	78.5
FoV phase (%)	87.50	93.75
Slice pF (%)	75.0	62.5
Phase R	2	2
Slice R	1.00	1.25
PD & T1 TA (min)	3:38	3:37
Spoiling phase increment (°)	50*	150*
MT TA (min)	6:37	9:40
MT TR (ms)	37	48
MT pulse angle (°)	500*	220*
MT pulse length (ms)	10*	8*
MT pulse off res. freq. (kHz)	1.2*	1.0*
MT pulse shape	Gaussian*	Gauss filtered sinc*
MT pulse bandwidth (Hz)	192*	300*

Note: Default settings (mostly unchangeable) marked with *.

Abbreviations: FoV, field of view; MT, magnetization transfer; PD, proton density; pF, partial Fourier; R, parallel speedup factor; T1, longitudinal relaxation; TA: acquisition time; TE, echo time; TR, repetition time.

The acquisition protocols shared the following common parameters across all platforms (see Table 2 for differing parameters): TR of PD‐ and T1‐weighted contrasts: 18 ms; flip angles for MT, PD, and T1 weighted contrasts: 6°, 4°, 25°, respectively; six equidistant echoes (for TE, see Table 2); 1 mm isotropic reconstruction voxel size; readout (RO) field of view (FoV): 256 mm; base resolution: 256 pixels in the RO direction; 176 slices; readout in the head‐foot direction, inner phase encoding loop in the left–right (“slice”) direction, outer phase encoding loop in the anterior–posterior direction (“phase”); RO bandwidth: 480 Hz/pixel; elliptical k‐space coverage; parallel imaging speedup factor of 2 in the slow phase encoding direction (comprehensive list of parameters in Supplement 1).

The B1⁺ field mapping methods differed across vendors and sites. At Siemens sites, vendor‐supplied sequences were used. At three sites (BCN, HD, ZH) a rather slow (2:14 min) implementation “rf map” was applied. It was based on spin‐echo and stimulated echo acquisitions and is similar to the customized sequence by Lutti, Hutton, Finsterbusch, Helms, and Weiskopf (2010). However, it used a 2D gradient echo readout instead of a 3D echo‐planar imaging (EPI) readout. At another Siemens site (BSL), we used a faster implementation (“tfl_b1map”) (12 s) utilizing a gradient echo sequence with ultrafast turbo‐FLASH readout (available from version VE11 onwards) (Chung, Kim, Breton, & Axel, 2010). At the Philips sites (NOT, SNS) a vendor implementation of the actual flip angle imaging (AFI) technique was used (Yarnykh, 2007, 2010), which acquires spoiled gradient echo signals with two alternating TRs (3:36 min total scan time). The B1⁺ mapping acquisition parameters are detailed in Supplement 1.

The high‐resolution data was corrected for apparent sensitivity changes due to head motion between the acquisitions of the three differently weighted volumes, as implemented in the hMRI‐toolbox (Papp, Callaghan, Meyer, Buckley, & Weiskopf, 2016; Tabelow et al., 2019). To this end, low‐resolution 3D spoiled gradient echo volumes were acquired twice: once with the RF head coil and once with the body coil, with acquisition times of 10 s per coil. The ratio provided a relative net RF receive field sensitivity (B1⁻) map of the head coil (Papp et al., 2016; Tabelow et al., 2019). The acquisition was optimized for speed by using a low isotropic spatial resolution of 4 mm, short TE (2–3 ms) and a low flip angle of 6° (no partial Fourier, no parallel imaging speedup). The acquisition of the head and body coil volume pair was repeated before each of the three MPM contrasts (MT, PD, and T1). On the Philips platform, the sensitivity estimate and correction was performed in addition to the pre‐scan procedure (multi‐channel RF coil sensitivity normalization; “CLEAR”), since the built‐in procedure typically acquires the sensitivity maps once and does not dynamically update them between scans (Papp et al., 2016).

2.3. Data quality control

The acquisition parameters of each scan (as stored in the DICOM header) were manually checked post hoc against standard settings to detect inconsistencies in the data acquisition. Throughout the data processing pipeline, intermediate data volumes, segmentations, and parameter maps were systematically checked visually, especially to detect misregistration or erroneous scaling of quantitative maps.

2.4. Estimation of parameter maps

The MPM data were processed using a customized version of the hMRI‐toolbox (Tabelow et al., 2019), based on versions v0.1.1‐beta and v0.1.2‐beta, (available at https://github.com/tleutritz-cbs/hMRI-Toolbox/tree/BaudrexelRFspoiling) within the SPM12 framework (revision 7357; FIL, London, UK; RRID: SCR_007037) in MATLAB (version R2017b; Mathworks, Natick, MA; RRID: SCR_001622) on GNU/Linux computers with x86_64 architecture.

The main processing steps included the data conversion, calculation of quantitative maps, and reproducibility analyses. A collection of scripts encompassing all the following steps and a simplified ROI analysis within one subject is available on Github (https://github.com/tleutritz-cbs/MPM_quality).

DICOM images were first converted to NIfTI volumes using the hMRI‐toolbox converter and a comprehensive set of meta‐data in JSON files were stored for further processing (Tabelow et al., 2019). The Philips DICOM images were converted to NIfTI by applying scaling factors available from private tags, to enable quantitative evaluation of the data (Chenevert et al., 2014). The DICOM converter within the hMRI‐toolbox and within SPM12 (since version r7487) was adapted to take these scaling factors into account. Alternatively, Philips scanners have the opportunity to save data proportional to the MR signal within DICOMs as well. This could be achieved with a special setting within the reconstruction parameters (Density of Transverse Ordered Spins [DOTS]), but needs manual intervention for each session in “Gyrotest” mode. Unfortunately, this mode and the respective settings are not applicable within clinical trials. Furthermore, because the typical signal intensities of the Philips data differed widely from the default dynamic range threshold settings for the hMRI‐toolbox (originally designed for use with Siemens data), the settings for processing Philips data were changed as follows: MT‐threshold = 15 and A‐threshold = 10⁸. Customized parameter files with these modifications were loaded before further processing with the “Configure toolbox” module of the hMRI‐toolbox (Tabelow et al., 2019). The deletion of intermediate processing files (“cleanup”) was switched off. This enabled visual checks throughout the pipeline and the reuse of tissue segmentations for quality measures.

For optimal segmentation and registration of volumes, we first applied auto‐realignment as implemented in the hMRI‐toolbox (Tabelow et al., 2019). The first MT‐weighted echo was aligned to the PD‐weighted canonical template within SPM. Additional masking was applied to avoid segmentation issues due to noise outside the head. The masking was based on a python script for quality assessment within the Connectome project (Alfaro‐Almagro et al., 2018).

R1, PD, R2*, and MT maps were estimated from the multi‐echo data in combination with the B1⁺ and B1⁻ measurements using the “Create hMRI maps module” in the hMRI toolbox (Papp et al., 2016; Tabelow et al., 2019), based on the rational approximation of the signal equation presented by Helms, Dathe, and Dechent (2008). The R2* maps were calculated based on all echoes acquired across all contrasts using the ESTATICS estimation scheme (Weiskopf, Callaghan, Josephs, Lutti, & Mohammadi, 2014). To correct for R2*‐ and the remaining B1⁻‐related bias in the PD maps, the signal was extrapolated to TE = 0 (Ellerbrock & Mohammadi, 2018) and receive sensitivities were additionally corrected using a data based bias estimation according to UNICORT (Tabelow et al., 2019; Weiskopf et al., 2011). At one site (BSL), the B1⁻ maps suffered from excessive noise levels and could not be used for the MPM estimation. They had inadvertently been acquired with too high of a flip angle. Instead, B1⁻ was solely estimated using a data driven method based on unified segmentation and bias field correction within SPM (Ashburner & Friston, 2005; Tabelow et al., 2019). This is similar to UNICORT for B1⁺ correction (Tabelow et al., 2019; Weiskopf et al., 2011), but applied to PD maps only. All PD maps were calibrated to a value of 69 p.u. in WM (exceeding tissue probability of 95%) according to Tofts (2003).

To account for imperfect RF spoiling, we applied a voxel‐wise correction to the applied flip angles (after correction for B1^+/− inhomogenieties) depending on the applied phase increment (Table 2) according to the polynomial coefficients reported by Simon Baudrexel, Nöth, Schüre, and Deichmann (2017). The implementation of the method by Baudrexel et al. is available on github.com within a branch of the hMRI‐toolbox: https://codeload.github.com/tleutritz-cbs/hMRI-toolbox/zip/BaudrexelRFspoiling

3. HARMONIZATION OF MT MAPS

While the semi‐quantitative MT maps are largely insensitive to variations in local R1 values and B1⁺ fields (Callaghan et al., 2014; Helms, Dathe, & Dechent, 2008), they depend on the MT pulse used in the sequence (Table 2). The product sequences did not allow the user to precisely control the characteristics of the MT saturation pulses, thus a rescaling of MT maps was implemented to harmonize MT maps across manufacturers. The proposed harmonization also accounts for systematic differences in TR and measured R1 due to incidental MT by the excitation pulse (Olsson, Wirestam, Lätt, & Helms, 2020).

The estimated MT values (MT_orig) from Philips scanners were linearly scaled to minimize the difference with the target MT values across pixels in brain tissue (ZH site arbitrarily served as reference):

$$M{T}_{\mathit{\text{scaled}}}=aM{T}_{\mathit{\text{orig}}}+b$$ (1)

with two empirical parameters a and b, accounting for (a) the transferred saturation that is mainly driven by the saturation of the bound pool (i.e., power of the saturation pulse) and (b) a shift by direct saturation of the free water pool observed at frequency offsets <2 kHz (Helms, Dathe, & Dechent, 2008). Restriction to brain voxels was achieved by using gray matter (GM) and white matter (WM) masks determined by SPM unified segmentation (Ashburner & Friston, 2005) of the reference maps (ZH site). The GM/WM tissue probability masks were then set to a threshold of 99% to increase specificity to brain voxels. Cerebrospinal fluid (CSF) was explicitly excluded from the fitting procedure because it can exhibit direct saturation (offset) effects that would likely differ from tissue due to a much longer T2 and the absence of MT. In order to preserve the overall contrast, combined GM and WM masks were used for fitting. The resulting individual fitting parameters, over all subjects and scans from Philips sites, were used to estimate fixed scaling constants (a, b) by calculating the median of all fitted values. These two fixed parameters were then applied according to Equation (1) to all MT maps from Philips sites.

3.1. Analysis of inter‐ and intra‐site reproducibility

To determine intra‐ and inter‐site reproducibility of the MPM metrics, coefficients of variance (CoV) within and between sites were calculated voxel‐wise for each parameter map. To assess systematic bias, mean parameter values were additionally compared between sites. For the voxel‐based analysis, all quantitative maps were warped into common MNI space using DARTEL (Ashburner, 2007) as implemented within the hMRI toolbox. All subject data (including scan and rescan) from all sites was used to create the DARTEL template.

The intra‐site CoV was determined voxel‐wise as the SD (σ_intra) of the quantitative maps estimated from scan and rescan data over the mean (μ_intra) of both maps at a single site:

$${\mathrm{CoV}}_{\text{intra}}\left(\mathrm{vx},\text{site},\text{subj},\mathrm{MPM}\right)={\mathrm{\sigma }}_{\text{intra}}/{\mathrm{\mu }}_{\text{intra}}$$ (2)

with vx being the voxel number, site being the site where the data were acquired, subj being the subject identifier, and MPM being the mapped quantitative parameter. This represents the precision of MPM metrics within the same subject and site.

The inter‐site CoV was determined voxel‐wise as the SD (σ_intra) over the mean (μ_intra) across all scans for a specific subject, comprising bias observed at individual sites:

$${\mathrm{CoV}}_{\text{inter}}\left(\mathrm{vx},\text{subj},\mathrm{MPM}\right)={\mathrm{\sigma }}_{\text{inter}}/{\mathrm{\mu }}_{\text{inter}}$$ (3)

The site‐specific relative bias Δ was defined as the voxel‐wise ratio between μ_intra of the respective site and the mean of all μ_intra across all sites:

$$\mathrm{\Delta }\left(\mathrm{vx},\text{site},\text{subj},\mathrm{MPM}\right)={\mathrm{\mu }}_{\text{intra}}/\text{mean}\_\text{across}\_\text{sites}\left({\mathrm{\mu }}_{\text{intra}}\right)–1$$ (4)

Tissue probability maps for GM and WM from both (scan and rescan) MT maps were averaged per subject across all sites in order to provide a unified mask across all sites for better inter‐site comparison. These averages were then set to a threshold of 95% tissue probability, to serve as GM or WM tissue masks, respectively. Additional ROIs were extracted from FSL (version 5.0.9; FMRIB, Oxford, UK; RRID: SCR_002823) brain atlases (Desikan et al., 2006; Hua et al., 2008). The following ROIs were chosen because reference values were available (Weiskopf et al., 2013) or because significant pathological changes due to spinal cord injury have previously been observed within these ROIs (Freund et al., 2013; Freund, Rothwell, Craggs, Thompson, & Bestmann, 2011; Grabher et al., 2015; Villiger et al., 2015): caudate nucleus (CN), corpus callosum (CC), GM and WM of thalamus and cerebellum, cerebral peduncles, corticospinal tract (CST), hippocampi, as well as primary sensory (S1) and motor (M1) cortices, respectively. GM and WM masks and the additional ROIs (conjunct with the GM/WM mask, respectively) were used in the further ROI analysis.

The CoV_intra and CoV_inter for a specific ROI was determined by calculating the root‐mean‐square (RMS) value of the respective CoV measure across all N voxels within the ROI as follows:

$$x_{\mathrm{RMS}}=\sqrt{{\sum }_{}^{}\left({x_{vx}}^2\right)/N}$$ (5)

Analogously, summary CoV values across sites were also determined by the RMS across sites. The systematic bias for a specific site was determined by calculating the RMS value of the bias Δ across all voxels within the GM and WM ROIs.

To assess systematic differences introduced by the acquisition protocols, which differed somewhat between manufacturers (e.g., different RF pulses or spoiling characteristics), the MPM data were reordered into three different groups: (a) data from all Siemens sites; (b) all Siemens data excluding the data from the BSL site, due to poor quality of B1⁻ maps; (c) data from all Philips sites. For easier assessment of these three groups, summary measures of CoV_intra, CoV_inter, and bias Δ were calculated as the RMS value across the GM and WM masks, subjects and sites, respectively.

4. RESULTS

4.1. Data quality control

The DICOM header consistency check found the following minor deviations from the planned acquisition protocols. A single data set at site HD was acquired with minor TE differences (< 4.1%), which were corrected by MPM estimation procedures (i.e., estimation at TE = 0). In addition, partial Fourier was set to 6/8 instead of 1 for low‐resolution scans for B1⁻ in the same data set, which also occurred occasionally at other sites (NOT, ZH). This may have impacted the effective resolution and signal‐to‐noise ratio (SNR). However, this was not recognized as being detrimental to the B1⁻ mapping measurements. At one site (BSL), the B1⁻ maps suffered from excessive noise levels due to incorrect flip angle settings (23° instead of 6°). These data could not be used for the B1⁻ estimation. Instead, a data driven method was used (see above). A data set of a single subject was acquired with a 1.1 mm in‐plane resolution at one site (BCN). Moreover, for a single scan the parallel acquisition was switched off, which may have affected the effective resolution, SNR, and introduced motion artifacts due to the longer scan time.

Overall the protocol check demonstrated good adherence to the planned acquisition protocol (only about 2.4% of the approximately 12,000 checked acquisition parameters differed).

The visual checks of intermediate processing results were used to optimize the processing pipeline, for example, introducing realignment and head masking as described in the methods.

4.2. Harmonization of MT maps

The MT values obtained on the Philips scanners were harmonized using a linear model (Equation (1)). The coefficient of determination R² was in the range of 0.81–0.91 for fitting Equation (1) across GM/WM ROIs. Fitting parameters varied between subjects and sites (Figure 1). The median of the scaling factor a was a = 0.7895 and for the offset b = −0.0807. Rescaling of the Philips data using these two fixed scaling factors reduced the inter‐site bias Δ after harmonization (Figure 2).

FIGURE 1.

Range of fitted parameter values across subjects for MT harmonization of Philips scanner data: (a) scaling factor a, and (b) offset b. Box = interquartile range; bold horizontal line = median, feathers = data range of values across subjects

FIGURE 2.

Relative inter‐site bias of the MT values across subjects per site (a) without and (b) with rescaling of MT values. Box = interquartile range; bold horizontal line = median, feathers = data range of values across subjects. Cross = outlier, which exceeds the range of (q1–1.5 × [q3−q1]; q3 + 1.5 × [q3−q1]), where q1 and q3 are the 25th and 75th percentiles of the sample data, respectively

4.3. Inter‐ and intra‐site reproducibility and relative bias

The MT, PD, R1, and R2* maps showed a distinct GM/WM contrast and different anatomical structures in the brain. For example, the cortex, cerebellum, midbrain structures, basal ganglia, thalamus, optic radiation, and ventricles could be visualized (see Figure 3 for maps of a representative subject; see Figure 4 for quantitative ROI analyses). The RMS average of intra‐site scan‐rescan CoV for GM and WM was between 8 and 10% for harmonized maps of MT, ~7% for R1, and 4% for PD. It was higher for R2* with a CoV of up to 16% (Figures 5 and 10; see Figure 6 for spatial distribution). The inter‐site CoV showed a pattern similar to the intra‐site CoV (Figure 6, Figure 7, Figure 5, and Figure 8), indicating a good alignment of measures across sites. Average inter‐site biases were between 0.8 and 4.8% for GM and WM for MT, PD, and R1 maps in the whole brain and rose to 9.8% for R2* maps (Figure 9 and Figure 10).

FIGURE 3.

Mean of parameter maps shown for subject no. 5 scanned at all sites (axial slice through the center of the brain). The mean was calculated across the scan and rescan measurements

FIGURE 4.

Distribution of (a) MT, (b) PD, (c) R1, and (d) R2* mean values in regions of interest (ROI) across subjects and sites. Please note that PD values are scaled to 69 p.u. for the mean WM value by the PD map estimation process (Tabelow et al., 2019; Tofts, 2003). Box = interquartile range; bold horizontal line = median, feathers = data range of values across subjects. Cross = outlier, which exceeds the range of (q1–1.5 × [q3−q1]; q3 + 1.5 × [q3−q1]), where q1 and q3 are the 25th and 75th percentiles of the sample data, respectively

FIGURE 5.

Distribution of (a) MT, (b) PD, (c) R1 and (d) R2* intra‐site CoV in regions of interest (ROI) across subjects within each site. Box = interquartile range; bold horizontal line = median, feathers = data range of values across subjects. Cross = outlier, which exceeds the range of (q1–1.5 × [q3−q1]; q3 + 1.5 × [q3−q1]), where q1 and q3 are the 25th and 75th percentiles of the sample data, respectively

FIGURE 10.

(a) Inter‐ and (b) intra‐site CoV, and (c) bias in a subgroup of sites, which is aggregated across all subjects/sites in the corresponding subgroup and the whole brain (GM and WM ROIs)

FIGURE 6.

Intra‐site CoV of parameter maps shown for Subject 5 scanned at all sites (axial slice through the center of the brain). The intra‐site CoV was calculated across the scan and rescan measurements

FIGURE 7.

Inter‐site CoV of parameter maps shown for all subjects scanned at all sites (axial slice through the center of the brain). The inter‐site CoV was calculated over all six sites within the study

FIGURE 8.

Distribution of (a) MT, (b) PD, (c) R1, and (d) R2* inter‐site CoV in regions of interest (ROI) across subjects and sites. Box = interquartile range; bold horizontal line = median, feathers = data range of values across subjects. Cross = outlier, which exceeds the range of (q1–1.5 × [q3−q1]; q3 + 1.5 × [q3−q1]), where q1 and q3 are the 25th and 75th percentiles of the sample data, respectively

FIGURE 9.

Distribution of the relative bias of (a) MT, (b) PD, (c) R1, and (d) R2* across GM/WM ROIs and subjects for each site. Box = interquartile range; bold horizontal line = median, feathers = data range of values across subjects. Cross = outlier, which exceeds the range of (q1–1.5 × [q3−q1]; q3 + 1.5 × [q3−q1]), where q1 and q3 are the 25th and 75th percentiles of the sample data, respectively

Next, the contribution of differences across the MRI acquisition protocols to the CoVs and bias were assessed by analyzing subgroups of sites and comparing these to the whole data set comprising all sites (Figure 10). Considered subgroups were (a) all Siemens data, (b) all Siemens data without BSL site data, because of the differing processing schemes for B1⁻, and (c) all Philips data. Generally, the CoVs were in a similar range for all data from the different subgroups, except for the inter‐site CoV and bias of R2*. The highest CoVs were found for the R2* measures independent of the data subgroup. The intra‐site CoVs were slightly increased for the Philips subgroup (c) data set.

5. DISCUSSION

We implemented and compared quantitative multiparameter mapping protocols based on product pulse sequences from two different MRI manufacturers within a traveling heads study. Protocols were designed to achieve a high isotropic resolution close to 1 mm and total acquisition times of ~20 min, making them suitable for use in clinical trials targeting specific anatomical and microstructural metrics.

5.1. Accuracy of quantitative maps

The mean values of MT, PD, R1, and R2* (Figure 11) were generally in line with values published by Gracien et al. (2019); Gringel et al. (2009); Hagiwara et al. (2018); Krauss et al. (2015); Weiskopf et al. (2013). Relaxation (R1, R2*) values deviated by 0.3–10.9% from the values reported in Weiskopf et al. (2013) in GM, WM, caudate nucleus (CN), and the corpus callosum (CC). PD values differed by 3.7–13.6% from those reported in Krauss et al. (2015), considering the same ROIs. For R1, and R2* we achieved deviations between 0.3 and 21.2%, and between 1.5 and 3.8% for PD compared with Gracien et al. (2019), taking only GM and WM mean values into account. MT values differed between 11 and 22% from values reported in Gringel et al. (2009), whereas the deviation was only about 10% from Hagiwara et al. (2018). Both reference studies (Gringel et al., 2009; Hagiwara et al., 2018) used pulse settings comparable to those applied here.

FIGURE 11.

Mean values of parameter maps across subjects and sites for (a) MT, (b) PD, (c) R1, and (d) R2*. ROIs: Intracranial gray (GM) and white matter (WM), a sphere in the head of caudate nucleus (CN), a sphere in the corpus callosum (CC). Reference values marked by symbols (Gracien et al., 2019; Gringel et al., 2009; Hagiwara et al., 2018; Krauss, Gunnarsson, Andersson, & Thunberg, 2015; Weiskopf et al., 2013)

Quantitative PD and R1 maps generally rely on B1^+/− field correction, which was carried out with vendor sequences in this study, compared with reference studies. No direct comparison in regard to the actual accuracy of the applied sequences with custom‐made sequences was attempted. Any degree of erroneous field correction influences R1 maps by a factor of two, due to quadratic dependence on the actual flip angle. For example, the custom‐made B1 mapping applied in the study by Weiskopf et al. (2013) is accurate with a total error of less than ca. 3% (Lutti et al., 2010), contributing to an error rate of about 6% in R1 maps. In addition, the correction of imperfect RF spoiling relies on accurate flip angles and will enhance errors as well, which should be accounted for in the series of error propagation. The calibration of PD maps to a fixed value of 69 p. u. in WM (Tofts, 2003) might introduce a bias in these maps, not reflecting pathologic changes in WM. Moreover, it is known that R1 and PD values are affected by inadvertent magnetization transfer effects, which depend on the specifics of the RF pulse configuration and power (Teixeira, Malik, & Hajnal, 2019).

Closed‐source filter settings at Philips sites could not be controlled and thus might also have influenced the SNR of the data. Approximation to a TE of 0 reduced R2* biases, but might have introduced additional noise sources. Furthermore, residual deviations may have been driven by methodological differences (Stikov et al., 2015) or instrumental differences but also inter‐individual biological variation (Figure 11), since different cohorts were studied. Additionally, the approaches varied in B1^+/− field correction methods, treatment of incomplete RF spoiling biases, and different aspects of data processing, which may explain some differences to reference values. We did not account for the slight differences in field strengths between the two vendors (2.89 T for Siemens, 3.00 T for Philips MRI), which may have led to a small bias of ~1.4% in T1 relaxation (Rooney et al., 2007).

However, this study went beyond previous multicenter studies using quantitative mapping (Deoni et al., 2008; Gracien et al., 2019; Lee, Callaghan, Acosta‐Cabronero, Lutti, & Nagy, 2019; Weiskopf et al., 2013) by including six sites, two different manufacturers, and four different MRI scanner models. By including diverse hardware and software, this study captures the typical situation in multicenter clinical trials better and provides a more realistic reference for the performance of MPM in multicenter studies using widely available product sequences and methods.

5.2. Handling of protocol deviations

Inadvertent deviations from the standard protocols may occur in large complex imaging studies. Here, they included slight variations in echo times or spatial encoding settings. The flexibility of the hMRI processing toolbox and the intrinsic advantages of quantitative MRI allowed for a principled correction. For example, differing TE values were addressed by the fitting of R2* rather than relying on R2*‐weighted images and also by extrapolating the signal to TE = 0 for calculation of MT, R1, and PD maps. Other deviations mainly influenced the acquisition time and only to a certain degree the resulting measurements (e.g., differing signal‐to‐noise ratio). To help avoid incorrect settings of protocol parameters in upcoming multicenter studies, our traveling heads study was used to optimize standard operating procedures. The CoVs may be further reduced by selecting a subgroup of Siemens sites (excluding the data acquired with poor B1⁻ maps due to a fault in the flip angle settings and required adjustments of the processing).

5.3. Intra‐ and inter‐site CoV and bias

The proposed MPM acquisition achieved a high inter‐site comparability with a low inter‐site bias of less than 5% in the quantitative maps. Similarly, the intra‐ and inter‐site CoV were in a range of 5–10% for R1 and MT maps. Thus, the observed CoV was ~3 times lower than the trauma‐related effect sizes shown in longitudinal studies of spinal cord injury 12 months after injury, which range from 17 to 20% for R1, and 14% for MT (Grabher et al., 2015). Hence, the presented MPM protocol is expected to reliably detect and characterize the trauma‐related effects in longitudinal multicenter studies. Despite the harmonization of MT values across vendor platforms the intra‐ and inter‐site CoVs remained slightly elevated for the subgroup of Philips sites (Figure 10), also influencing the overall results across all sites (Figure 5 and Figure 8). This might also be related to the actual implementation of RF spoiling correction, which is equally applied to T1‐, PD‐, and MT‐weighted multi‐echo data. Nevertheless, the inter‐site bias for all sites was decreased from 15.7 to 4.8% as a result of MT harmonization. Considering the calibration of PD maps to 69 p. u. in WM, the low CoVs and bias might be underestimated.

A higher inter‐site CoV of 11.6–17.8% was observed for R2* maps, which was partly driven by the higher intra‐site CoV for R2* maps and systematic differences between the two manufacturers (Figure 10). This includes field strength and maximum TE (14.76 ms; Table 2), which is not optimal for estimating R2* in GM or WM. We also attribute the intra‐ and inter‐site differences in R2* to the relatively poor reproducibility and performance of shimming routines. Further studies should be performed to elucidate the vendor differences observed in our study, which should also include simulation of vendor sequences. Another source of variability of R2* might arise from magnetic field inhomogeneities, which could be corrected with the approach by S. Baudrexel et al. (2009).

6. CONSIDERATIONS

The current implementation of spoiling corrections, based on the vendor's spoiling schemes (Simon Baudrexel et al., 2017) within the hMRI‐toolbox, is still limited to vendor specific phase increments and might not be applied to the MT‐weighted multi‐echo data, where different spoiling schemes are applied.

The differences in MT pulses between manufacturers resulted in ~20% difference in MT saturation values, which were harmonized in the post‐processing by a linear rescaling of the MT values. This reduced the inter‐site bias considerably (Figure 2). This traveling heads study may serve as a reference for the rescaling of MT values in future multi‐vendor studies. Because of rescaling, MT values may not be comparable to previous studies, for example, due to differences between custom‐made sequences with optimized MT pulse scheme (Weiskopf et al., 2013), and vendor‐based sequences and MT pulses (Table 2).

The small sample size of five volunteers in a narrow age range may not represent the general population and its variability. Thus, care should be taken when extrapolating these results to different patient or subject groups, for example, populations of elderly patients. However, we believe that most of the study characteristics are fundamental and will be only modulated by the population studied. For instance, larger head motion will lead to general increases in CoV, which will add to the characteristics described here.

Due to the short time gap of 2 hr between the measurements, the scan‐rescan experiment mimics, but cannot fully capture, the variance components in long term longitudinal studies (e.g., instrumental deterioration, long term physiological fluctuations, hardware/software changes). Thus, we would consider the intra‐site CoV as an approximation and a lower limit of variability in longitudinal studies.

We only used vendor‐based sequences to make the MPM approach widely accessible. However, we included specialized vendor sequences for acquisition of the B1⁺ mapping reference data, which may depend on the software baseline. In case of the B1⁺ field mapping on the Philips platform, this required clinical science keys (CSK) (option 047). Alternatives for Philips scanners without this CSK would be the use of vendor sequences for double angle B1⁺ mapping methods (Boudreau et al., 2017) or the use of data driven postprocessing correction methods such as UNICORT (Weiskopf et al., 2011)

The correction of biases related to B1⁻ field inhomogoneities consisted of two main steps. Sensitivity maps acquired between all multi‐echo gradient echo sequences were used to correct for apparent sensitivity changes due to head motion between the acquisitions, that is, motion between different contrasts within the acquisition scheme of MPM (Papp et al., 2016). Since the method assumes the body coil RF receive sensitivity field to be uniform for calibration, the PD maps will be affected by any body coil sensitivity inhomogeneities. Thus, in the second step an additional correction was applied using a data driven bias estimation analogous to UNICORT and as implemented in the hMRI‐toolbox (Tabelow et al., 2019; Weiskopf et al., 2011). This reduced the inter‐ and intra‐site CoV further (data not shown)

Additionally, we improved the standard processing pipeline in the hMRI‐toolbox (Tabelow et al., 2019) by applying a head mask to reduce segmentation errors, as well as an additional implementation of correction for imperfect RF spoiling (Simon Baudrexel et al., 2017), reducing measurement biases.

7. CONCLUSIONS

This study investigated scan‐rescan and inter‐site reproducibility of the multiparameter mapping (MPM) approach implemented, at 3 T MRI, with Philips and Siemens vendor sequences. The aim of the study was to generally enable and additionally improve the comparability of multicenter studies. The 1 mm resolution MPM maps showed high repeatability and comparability across different testing sites. The measurements were comparable, as reflected by a low inter‐site bias (below 5%) and highly reproducible for quantitative maps of MT, R1, and PD. Intra‐site coefficients of variation for these measures ranged between 4 and 10% and up to 18% for R2* maps. Quantitative MRI parameters were in good agreement with previously reported studies (Weiskopf et al., 2013), with small deviations on the order of 0.3–10.9%. Since we used only vendor product sequences for the data acquisition, and the open source hMRI‐toolbox (www.hMRI.info; Tabelow et al. (2019)) for processing, the approach can be readily applied in quantitative MRI single‐ and multisite studies.

CONFLICT OF INTEREST

The Max Planck Institute for Human Cognitive and Brain Sciences and Wellcome Centre for Human Neuroimaging have institutional research agreements with Siemens Healthcare. NW was a speaker at an event organized by Siemens Healthcare and was reimbursed for the associated travel expenses.

Supporting information

Appendix S1. Supporting Information

Leutritz T, Seif M, Helms G, et al. Multiparameter mapping of relaxation (R1, R2*), proton density and magnetization transfer saturation at 3 T: A multicenter dual‐vendor reproducibility and repeatability study. Hum Brain Mapp. 2020;41:4232–4247. 10.1002/hbm.25122

DATA AVAILABILITY STATEMENT

The anonymized data that support the findings of this study are available from the corresponding author upon reasonable request. However, the personal, non‐anonymized MRI data cannot be made available, since their sharing is constrained by ethical and European GDPR regulations.