Stability of MRI metrics in the advanced research core of the NCAA-DoD concussion assessment, research and education (CARE) consortium

Abstract

The NCAA-DoD Concussion Assessment, Research, and Education (CARE) consortium is performing a large-scale, comprehensive study of sport related concussions in college student-athletes and military service academy cadets. The CARE “Advanced Research Core” (ARC), is focused on executing a cutting-edge investigative protocol on a subset of the overall CARE athlete population. Here, we present the details of the CARE ARC MRI acquisition and processing protocol along with preliminary analyzes of within-subject, between-site, and between-subject stability across a variety of MRI biomarkers. Two experimental datasets were utilized for this analysis. First, two “human phantom” subjects were imaged multiple times at each of the four CARE ARC imaging sites, which utilize equipment from two imaging vendors. Additionally, a control cohort of healthy athletes participating in non-contact sports were enrolled in the study at each CARE ARC site and imaged at four time points. Multiple morphological image contrasts were acquired in each MRI exam; along with quantitative diffusion, functional, perfusion, and relaxometry imaging metrics. As expected, the imaging markers were found to have varying levels of stability throughout the brain. Importantly, between-subject variance was generally found to be greater than within-subject and between-site variance. These results lend support to the expectation that cross-site and cross-vendor advanced quantitative MRI metrics can be utilized to improve analytic power in assessing sensitive neurological variations; such as those effects hypothesized to occur in sports-related-concussion. This stability analysis provides a crucial foundation for further work utilizing this expansive dataset, which will ultimately be freely available through the Federal Interagency Traumatic Brain Injury Research Informatics System.

Introduction

Mild traumatic brain injury (mTBI) is a major public health issue in the United States (U.S.) and worldwide. From a broad perspective, sport-related concussion (SRC), a sub-category of mTBI, has been the focus of dramatically increased attention across the clinical, research, and lay communities (McCrory et al., 2013, 2009, 2005; DeKosky et al., 2010; Kelly 1999; Langlois et al., 2006; Centers for Disease Control and Prevention 2007).

While previous studies have improved best practices in the evaluation and monitoring of SRC (McCrea et al., 2003; Guskiewicz et al., 2003), its management remains a significant challenge in the sports medicine community. These challenges stem from a variety of factors, including a lack of objective diagnostic tools, under-reporting of injuries and symptoms, and operational constraints of the competitive environment.

Large prospective studies have sought to improve patient management by clarifying the time course of clinical recovery after SRC (McCrea et al., 2003, 2005, 2009, 2010). Though these studies have shed light on the underlying nuances of SRC progression, no objective bio-marker of physiological recovery yet exists. In particular, biomarkers that relate the acute physiological effects of SRC on brain structure and function remain elusive. It is thus difficult to identify the window of cerebral vulnerability that extends beyond the point of clinical recovery (McCrory et al., 2017). This period of vulnerability is of substantial importance, as it may point to a physiologically compromised brain state that potentially increases the risks of repetitive injury.

The NCAA-DoD Grand Alliance: Concussion Assessment, Research and Education (CARE) Consortium is a large-scale, multi-site study of the natural history of concussion in both sexes across multiple sports. The published mission of this study, which is studying collegiate athletes and military service academy cadets, is to “gain a better understanding of the neurobiopsychosocial nature of concussive injury and recovery in order to ultimately enhance the safety and health of our student-athletes, service members, youth sports participants, and the broader public (CARE 2015).”

The CARE project includes a Clinical Science Core (CSC) and an Advanced Research Core (ARC). While the CSC is focused on expansive longitudinal clinical testing of subjects, the ARC is focused on advanced biomarker assessment in athletes participating in high impact collision sports with a higher incidence of concussion and head impact exposure. Magnetic resonance imaging (MRI), is one of the key components of the CARE ARC biomarker assessment project.

While the primary goal of the CARE MRI study is to develop and validate imaging biomarkers of recovery following SRC, the the present study focused on a more fundamental effort to determine the stability and reproducibility of key MRI metrics in non-injured individuals, against which to reliably measure the effects of concussion on brain structure and function. Following a detailed introduction to the CARE ARC MRI protocol, an analysis the stability of targeted quantitative MRI biometrics is presented. The results of this stability analysis are summarized so as to inform future retrospective analyzes that will utilize the data when it is publicly available in the Federal Interagency Traumatic Brain Injury Research (FIBIR) database.

The Transforming Research and Clinical Knowledge in TBI (TRACK-TBI) project is another large TBI study that is leveraging MRI biomarkers (Yue et al., 2013). The CARE MRI protocol was modeled after the TRACK-TBI MRI protocol, with additional modalities and application in very different cohorts of injured and non-injured NCAA student-athletes. The imaging biometric stability work presented here is also an extension of recently published TRACK-TBI stability analyzes focused primarily on diffusion imaging (Palacios et al., 2016). The present CARE MRI stability analysis expands this approach to a number of other MRI parameters and also leverages longitudinal control imaging sessions acquired under the ARC study protocol to perform within-subject stability analyzes. The presented study therefore serves as a novel exploration of quantitative MRI biometric stability, which may be of value to the broader neuroimaging community.

Methods

Study design

Under the CARE MRI protocol, concussed athletes undergo advanced, multi-modal MRI studies at the following time points: within 48 hours post-injury, the point at which the concussed athlete is cleared to begin return to play progression (RTP), 7 days post-RTP, and 6 months post-injury. For each concussed athlete, two matched control athletes (contact sport control and non-contact sport control) undergo the same MRI protocol at the same time points. While this subject population allows for consideration of concussion and head impact exposure, only non-contact sport athletes (without concussion or head impact exposure) were included in this study of stability. Physiological changes will induce longitudinal biometric variation within these control subjects. This variance serves as a background above which any potential biomarker effect must rise.

Each MRI exam includes anatomical and quantitative acquisitions tailored for assessment of mTBI. The imaging protocol is similar to the TRACK-TBI protocol, including T₁, T₂ FLAIR, and ${\text{T}}_2^{*}$ weighted structural imaging contrasts, along with diffusion tensor imaging (DTI) and resting state functional MRI (rs-fMRI) quantitative acquisitions. To this protocol, CARE ARC adds arterial spin-labeling (ASL) for quantitative perfusion assessment, and ${\text{T}}_2^{*}$ mapping for quantitative assessment of iron content and calcification. This protocol has been deployed to the four CARE ARC study sites (UCLA, UNC, VT, UW), spanning Siemens Trio and GE MR750 3T MRI systems.

Thirty subjects were imaged at multiple time points for within-subject stability measurements in the first 18 months of the CARE project. Thirty-five subjects were imaged at least once at each site and were utilized for between-subject variance measurements. Table 1 reports the number of non-contact sport control subjects at each site whose images were used in this analysis. Numbers below 30 in the top section and 35 in the lower section in Table 1 indicate subjects which were excluded by quality control processes; primarily due to the presence of motion artifacts.

Table 1.

Numbers of subjects for each imaging acquisition that were included in the stability analysis at each imaging site

Site	T1	T2 FLAIR	${\text{T}}_2^{*}$	Diff.	BOLD	ASL
Within-Subject
UCLA	6	6	5	6	4	6
UNC	10	11	10	11	12	9
UW	6	6	5	6	6	5
VT	6	7	5	7	7	NA
Totals	28	30	25	30	29	20
Between-Subject
UCLA	6	6	5	6	8	6
UNC	11	11	11	12	14	11
UW	6	6	5	6	6	5
VT	7	7		7	7	NA
Totals	30	30	26	31	35	22

The ASL pulse sequence was not available at the VT MRI system

Beyond the non-contact control athletes, a set of two “traveling human phantom” subjects were scanned on two occasions at each site in the study. The resulting data were utilized to estimate the site-to-site variation of the acquisition protocol (Table 2).

Table 2.

Hardware and software used for MRI acquisitions at each site

Site	Vendor	Model	Software	Coil	T1	T2 FLAIR	${\text{T}}_2^{*}$	Diff.	BOLD	ASL
UCLA	Siemens	Trio	VB17	12, 32-ch	S1	S2	S3	S4	S5	S6
UNC	Siemens	Trio	VB17	12, 32-ch	S1	S2	S3	S4	S5	S6
UW	GEHC	MR750	DV25	32-ch	G1	G2	G3	G4	G5	G6
VT	Siemens	Trio	VB17	8-ch	S1	S2	S3	NA	S5	S6

Each acquisition is labeled according to ‘GX’ for GE site and ‘SX’, which is used to distinguish the protocols outlined in Tables 3 and 4, which provide respective acquisition protocols utilized on the Siemens and GE vendor platforms

Informed consent was obtained from all individual participants included in the study. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Imaging protocol

T₁ weighted imaging

T₁ weighted images were acquired for gray/white matter segmentation, registration of images to standard space, and volumetrics. Some recent work has shown changes in brain volume within two weeks of injury (Jarrett et al., 2016). On the Siemens platform, a 3D Magnetization Prepared Rapid Gradient Echo (3D MP-RAGE) acquisition was used, while on the GE platform, a 3D BRAin VOlume (BRAVO) acquisition was utilized. In both cases, a 3D fast gradient recalled echo acquisition is coupled with an initial inversion pulse to enhance gray/white matter contrast. Acquisition parameters are shown in Tables 3 and 4.

Table 3.

Acquisition parameters for each deployed imaging protocol on the Siemens Trio platform

Label	S1	S2	S3	S4	S5	S6
Contrast	MPRAGE	T2 FLAIR	T2*	Diff	BOLD	ASL
Plane	Sagittal	Sagittal	Axial	Axial	Axial	Axial
Slices	176	80	56	60	45	36
Matrix	256 × 256	256 × 256	384 × 240	90 × 90	70 × 70	64 × 64
TE (ms)	2.98	390.00	10.00†	98.00	27.00	13.00
TR (ms)	2300.0	5000.0	45.0	7800.0	2250.0	3204.6
TI (ms)	900	1800	NA	NA	NA	700
FOV (cm)	25.6	25.6	25.6 × 16.0	24.3	24.5	25.6
Thickness (mm)	1.0	2.0	3.0	2.7	3.5	4.5
Volumes	1	1	1	54	265	109
b-Value	NA	NA	NA	1000	NA	NA
(s/ mm2)				2000
b-Directions	NA	NA	NA	30	NA	NA
Label Time (ms)	NA	NA	NA	NA	NA	1800.0
Label Delay (ms)	NA	NA	NA	NA	NA	1600.0

All times are in ms. Labels are in accordance with the site definitions identified in Table 2.

^†:Biopolar multi-echo ${\text{T}}_2^{*}$ sequence acquired 4 echoes with an echo spacing of 4.75 ms. The 1st echo is indicated in the table

Table 4.

Acquisition parameters for each deployed imaging protocol on the GEHC MR 750 platform

Label	G1	G2	G3	G4	G5	G6
Contrast	BRAVO	T2 FLAIR	T2*	Diff	BOLD	ASL
Plane	Sagittal	Sagittal	Axial	Axial	Axial	Axial
Slices	164	160	70	50	36	36
Matrix	256 × 256	256 × 256	384 × 240	80 × 80	64 × 64	128 × 128
TE (ms)	2.91	65.65	13.53†	67.00	33.00	10.54
TR (ms)	6.62	6500.00	58.70	5250.00	2000.00	4632.00
TI (ms)	450.0	1913.0	NA	NA	NA	1525.0
FOV (cm)	25.6	25.6	25.6 × 16.0	24.3	25.6	25.6
Thickness (mm)	1.0	2.0	3.0	3.0	4.0	3.0
Volumes	1	1	1	64	180	2
b-Value	NA	NA	NA	1000	NA	NA
(s/ mm2)				2000
b-Directions	NA	NA	NA	30	NA	NA
Label Time (ms)	NA	NA	NA	NA	NA	1450.0
Label Delay (ms)	NA	NA	NA	NA	NA	1520.0

All times are in ms. Labels are in accordance with the site definitions identified in Table 2.

^†:${\text{T}}_2^{*}$ sequence acquired 5 echoes with an echo spacing of 11 ms. The 1st echo is indicated in the table

T₂ FLAIR weighted imaging

T₂ weighted images acquired using fluid-attenuated inversion recovery (FLAIR) improve the conspicuity of white matter lesions due to severe TBI (Ashikaga et al., 1997). To investigate the presence of subtle lesions with mTBI, the ARC MRI protocol includes a T₂ FLAIR acquisition utilizing variable flip angle schemes within extended echo train fast-spin-echo sequences. On the GE and Siemens platforms these sequences are respectively Cube and SPACE, with parameters included in Tables 3 and 4.

${\text{T}}_2^{*}$ weighted imaging and ${\text{T}}_2^{*}$ mapping

The CARE-MRI 3D gradient recalled echo ${\text{T}}_2^{*}$ weighted protocol acquires multiple echo times that can be used for quantitative ${\text{T}}_2^{*}$ map computations. The longest echo time image of the acquisition set is additionally utilized as a conventional ${\text{T}}_2^{*}$ weighted image. These acquisitions are very sensitive to micro-hemorrhages and have been show useful in more severe cases of mTBI (Liu et al., 2016). The parameters for these sequences are presented in Tables 3 and 4.

Diffusion weighted imaging

Diffusion weighted imaging (DWI) is a well-studied MRI modality for quantitative assessments of mTBI. DWI is sensitive to variations in white matter integrity and is therefore a probe of axonal injuries that could be a source of post-injury symptoms (Shenton et al., 2012). A number of studies have shown that diffusion-tensor (DTI) processing of DWI data can detect differences in injured groups compared to control populations (Eierud et al., 2014; Dodd et al., 2014; Bigler 2013); primarily in mean diffusivity (MD) and fractional anisotropy (FA). However, the polarities of these observed effects are conflicting (Dodd et al., 2014), necessitating further investigations with larger cohorts and more sophisticated diffusion modeling.

The CARE DWI approach utilizes multiple elevated diffusion weightings that enable computation of diffusion kurtosis imaging (DKI) metrics (Lu et al., 2006), in addition to standard DTI metrics. DKI provides increased sensitivity microstructural changes and have shown preliminary advantages in analyzes of concussed cohorts (Lancaster et al., 2016). Though DKI parameters can be computed using the CARE MRI data, this analysis focuses only on the two core DWI metrics that have seen the most attention in mTBI: FA and MD. The parameters for these diffusion sequences are presented in Tables 3 and 4.

Resting state functional

Functional MRI (fMRI) is a time-series acquisition that measures blood oxygenation level dependent (BOLD) contrast. BOLD is sensitive to neurovascular coupling and neuronal activity, specifically arising from changes in cerebral blood flow (CBF) and cerebral blood volume (CBV). In the context of mTBI, neurotrauma can cause changes in synchronous neuronal activation, degrade the structural integrity of cerebral microvasculature, and alter cerebrovascular reactivity. In combination with the effects of mTBI on CBF, fMRI can probe of physiological changes after SRC. Numerous studies have identified fMRI signal alterations potentially caused by mTBI, predominantly focused on longer term changes in activation patterns (Mayer et al., 2015).

Like TRACK-TBI, the CARE protocol utilizes resting state fMRI (rs-fMRI), thus enabling a broad survey of functional activation metrics and network analyzes. The details of the rs-fMRI acquisitions are included in Tables 3 and 4. Though a wide variety of metrics could be explored using this functional data, we focus on the stability of two voxel-wise rs-fMRI metrics: REgional HOmogeneity (REHO) of BOLD signal fluctuations (Zang et al., 2004) and Fractional Amplification of Low-Frequency Fluctuations (FALFF) (Zou et al., 2008).

Arterial spin labeling

Modulation of the cerebrovascular complex is a substantial factor in physiological reactions to mTBI (Tan et al., 2014). Arterial spin labeling (ASL) is as an MRI modality that can estimate cerebral blood flow (CBF) by tagging inflowing blood using radiofrequency pulses (Detre et al., 1992). Recent longitudinal ASL experiments on concussed athletes have shown trends in decreased CBF which agree with results from animal models (Meier et al., 2015; Wang et al., 2016; Ginsberg et al., 1997; Muir et al., 1992; Yamakami & McIntosh 1989).

Within the CARE MRI protocol, vendor and software variations between sites necessitated deployment of two different ASL approaches. Further, one site (VT) did not have license to utilize an ASL sequence. On the GE MRI platform (UW), a three dimensional (3D) pseudo-continuous ASL (3D-pCASL) was utilized. On the Siemens MRI Trio platform (UNC, UCLA), the only commercially available ASL acquisition was a two-dimensional (2D) pulsed ASL (2D-pASL). Compared to 2D-pASL, 3D-pCASL providdes improved signal-to-noise ratio (SNR) and acquisition efficiency (Chen et al., 2011; Vidorreta et al., 2013; Wu et al., 2007). Despite these differences, both techniques are capable of providing estimates of relative CBF (rCBF). The details of the deployed ASL sequences are provided in Tables 3 and 4.

Processing pipelines

For reproducibility and transparency, detailed processing commands employed in each pipeline are offered in Supplemental Material to this manuscript. A summary of the implemented processing pipelines is presented in the following sections.

T₁ weighted

In native space, SPM 12 (Penny et al., 2011) was used to segment gray matter, white matter, and cerebrospinal fluid. Skull stripping was performed by multiplication with the union of these masks. The skull stripped brain was then registered to the MNI-152 template’s skull stripped T₁ weighted image (Mazziotta et al., 2001) using the FMRIB’s Linear and Nonlinear Image Registration Tools (FLIRT and FNIRT in FSL) (Jenkinson et al., 2012). Transformation parameters for this registration were saved for ensuing registration operations. To account for system gain differences that can vary on a per-exam basis, the final T₁ weighted MNI-space skull-stripped image data was scaled and self-normalized to have an average intensity of unity.

T₂ FLAIR

To align T₂ FLAIR datasets to MNI space, a 6 degree of freedom registration, FLIRT, was performed in native-space to map the T₂ FLAIR and the T₁ weighted datasets. The transformation arising from this registration was then applied sequentially with the aforementioned nonlinear transformation computed for registration of the T₁ weighted image to MNI-space. Like the T₁ weighted images, MNI-space skull-stripped brain volume intensity was scaled and self-normalized.

${\text{T}}_2^{*}$

To compute ${\text{T}}_2^{*}$, a weighted linear least squares fit was performed using the natural logarithm of the image intensity for each echo time for each voxel. The slope of this regression was utilized to estimate the ${\text{T}}_2^{*}$ relaxation rate. This regression was implemented using customized C++ software.

The last echo time collected for each dataset was also utilized as single ${\text{T}}_2^{*}$ weighted image.

The same processing pipeline as described for T₂ FLAIR registration to MNI-space was applied to the ${\text{T}}_2^{*}$ maps and ${\text{T}}_2^{*}$ weighted images. Only ${\text{T}}_2^{*}$ weighted images were scaled and self-normalized.

Diffusion

The diffusion weighted imaging protocol included an extra acquisition with no diffusion gradients (b = 0) and reversed phase encoding polarity. This dataset, coupled with the b = 0 image from the diffusion weighted acquisition, was input into the “topup” function of FSL (Andersson et al., 2003). The resulting distortion correction map was applied to the entire DTI dataset. The brain was then extracted using the FSL brain extraction tool and the dataset corrected for eddy currents using the FSL “eddy” algorithm (Andersson & Sotiropoulos 2016).

A weighted fitting procedure in FSL calculated FA and MD maps (Jenkinson et al., 2012). Resulting FA maps were registered to MNI-space using FLIRT (affine registration) and FNIRT (non-linear registration) paired with the MNI-space FA template (Mazziotta et al., 2001). The resulting transformation was also applied to the MD map.

Functional

Resting state fMRI data was processed using standard functions available with AFNI (Cox 1996) and FSL (Jenkinson et al., 2012). Temporal outliers were removed with AFNI’s “3dDespike”. The time series first image was registered to the T₁ weighted anatomical image through boundary-based registration in FSL’s “epi_reg” for use later in the pipeline. Slice timing correction was applied using quintic interpolation through AFNI’s “3dTshift”. Motion correction was achieved through the use of AFNI’s “3dvolreg” with cubic interpolation. The registration calculated for the first image to the T₁ weighted image was concatenated to the transformation from the T₁ weighted image to MNI-space with FSL’s “convert_xfm” function, and applied to the motion corrected time series with FSL’s “applywarp” function.

Nuisance regression utilized a number of standard regressors. A time series of motion parameters and their first temporal derivatives were generated with AFNI’s “1d_tool.py”; white matter and CSF time series were generated using AFNI’s “3dmaskave” with MNI-space masks. Regression was performed in two steps. First AFNI’s “3dDeconvolve” function was run with the following inputs: the six motion parameters and their derivatives, up to third order polynomials, white matter and CSF signals, and the option of “x1D_stop” to create outputs for the use of “3dTproject” for nuisance regression, which was run as the second step.

Images were smoothed to an effective 10 mm full width at half maximum Gaussian point-spread using AFNI’s “3dBlurToFWHM.” This normalizes smoothness across datasets from different equipment and is standard in multi-site functional imaging studies (Friedman et al., 2006).

The fALFF functional metric was calculated on the smoothed data set utilizing AFNI’s “3dRSFC” function with a bandpass filter of 0.01 to 0.10 Hz. REHO was calculated on the smoothed data using the AFNI function of “3dREHO.”

Arterial spin labeling

Blood flow was estimated using the relative CBF (rCBF) parameter, computed using the commercial software algorithms on the respective MRI platforms. The resulting maps were transformed to MNI-space through linear registration (FSL FLIRT) of baseline (untagged) dataset to the T₁ weighted dataset followed by an application of the transformation connecting the T₁ weighted image to MNI space. This sequence of transformations was then applied to the rCBF maps as well.

Coefficient of variation calculation

Coefficients of variation (CVs) were computed across individual subject exam sessions, across the imaging sites in traveling human phantoms, and across subjects. CVs represent the proportion of variation relative to the mean value of the metric. These CVs illustrate the relative stability of the computed metrics for repeated measurements on individual subjects, repeated measurements across each of the imaging sites, and different subject measurements within an imaging site. Given the diversity of contrasts considered in this work, this relatively simplistic method allows for the comparison of stability between computed metrics.

The algorithm for computing CVs utilized standard AFNI functions. Repeated measurements were concatenated using AFNI’s “3dTcat.” The mean and standard deviation were calculated on the concatenated data set using “3dTstat.” The ratio of the standard deviation to the mean, the final CV, was calculated using AFNI’s “3dCalc.” Averaging CVs across subjects or sites included concatenating the computed CVs with “3dTcat” and averaging with “3dTstat.”

To compute CVs across individual subject exam sessions, processed datasets for each metric in MNI-space were used to compute the average and standard deviation across all imaging time points for each subject. Individual CVs were computed for each subject and then averaged across all subjects.

Computation of CVs across imaging sites was performed by using processed datasets for each metric in MNI-space to compute the average and standard deviation across all sites for both of the traveling human phantoms. Between-site CVs were then generated averaging the CVs across both human phantoms.

To compute CVs across all subjects, processed data-sets for each modality in MNI-space for the first imaging time point were used to compute the average and standard deviation across all subjects at each imaging site. Between-subject CVs were then generated for each site. The final between-subject CV was then computed as the average of the between-subject CVs from each site.

Region of interest processing

The brain was subdivided into 143 regions of interest for quantitative analysis of CVs across distinct brain regions. These regions of interest were drawn from the Harvard-Oxford atlas of cortical and subcortical gray matter (Desikan et al., 2006), along with the Johns Hopkins atlas of white matter tracks (Mori et al., 2005). A joint atlas was generated as the union of the intersection of the MNI-space gray matter mask and the Harvard-Oxford gray matter atlas with the intersection of the MNI-space white matter mask and the Johns Hopkins white matter atlas. This union of intersections ensured that each voxel was assigned to at most one region of interest. Mean CVs were extracted from each region of interest in this atlas in the within-subject, between-subject, and between-site data sets. This data is provided in tabular form in the Online Supplement to this study.

Results

Figures 1, 2, 3, 4, 5, 6, 7, 8, and 9 show the mean images and within-subject, between-site, and between-subject CV maps for normalized T₁ weighted, normalized T₂ FLAIR, normalized ${\text{T}}_2^{*}$ weighted, quantitative ${\text{T}}_2^{*}$, MD, FA, REHO, fALFF, and rCBF, respectively. Table 5 shows the average CVs for each contrast and comparison within gray/white matter and CSF regions of interest.

Fig. 1.

T1 weighted image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 2.

FLAIR image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 3.

T2* weighted image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row. The black arrowhead in left column represents between-site CV changes that are due to reception coil differences between sites. The white arrowhead shows between-subject CV changes in the frontal lobe, which are due to variations in sinus-cavity geometry and composition across subjects

Fig. 4.

Computed T2* image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 5.

Mean diffusivity image mean in mm²/s, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 6.

Fractional anisotropy image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 7.

Regional homogeneity of low BOLD fluctuations image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 8.

Fractional amplitude of low frequency fluctuations image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Fig. 9.

Relative cerebral blood flow image mean, first row, within-subject CV, second row, between-site CV, third row, and between-subject CV, fourth row

Table 5.

Average CVs across all gray mattter, white matter, deep gray matter, and cerebrospinal fluid ROIs for within subject, between site, and between subject comparisons

	T1	T2 FLAIR	${\text{T}}_2^{*}-\text{\hspace{0.17em}Wt}$	${\text{T}}_2^{*}$	MD	FA	REHO	fALFF	rCBF
Gray Matter
Within Subject	0.052	0.060	0.059	0.100	0.089	0.140	0.052	0.031	0.190
Between Site	0.106	0.157	0.171	0.193	0.149	0.175	0.177	0.076	0.227
Between Subject	0.151	0.156	0.175	0.284	0.218	0.329	0.171	0.081	0.732
Deep Gray
Within Subject	0.043	0.046	0.048	0.094	0.040	0.064	0.051	0.029	0.224
Between Site	0.049	0.095	0.203	0.172	0.071	0.086	0.149	0.054	0.257
Between Subject	0.058	0.074	0.102	0.222	0.101	0.145	0.144	0.066	0.547
White Matter
Within Subject	0.039	0.040	0.040	0.050	0.035	0.029	0.050	0.026	0.554
Between Site	0.039	0.069	0.108	0.082	0.032	0.055	0.134	0.048	−3.579
Between Subject	0.038	0.051	0.066	0.115	0.069	0.080	0.134	0.06	0.819
CSF
Within Subject	0.055	0.100	0.058	0.197	0.132	0.158	0.061	0.037	1.035
Between Site	0.123	0.402	0.203	0.372	0.377	0.221	0.159	0.058	−0.548
Between Subject	0.176	0.342	0.191	0.493	0.259	0.332	0.188	0.086	0.503

Cases where between subject CVs are lower than between site CVs are bolded, and cases where the difference is statistically significant are underlined. Of note, regions where between site CVs are greater than between subject CVs are areas where the measurements are well known to have low signal-to-noise ratios. Negative CVs in rCBF measurements arise from measurement noise in regions of minimal blood flow where the tag images had greater intensity than control images

T₁ weighted

Mean images in Fig. 1 show clear distinction of gray matter, white matter, and CSF tissue compartments. As expected, the within-subject CV is very low, with observed tissue compartment differences resulting from the signal to noise ratio variations of these compartments. The between-site variations are larger, which is anticipated due to the differences between T₁ weighted pulse-sequence implementations on the different vendor platforms.

Between-subject measurements show the largest variation. This is consistent across many of the measured metrics. In this case, the variation is due to changes in detailed tissue composition from one subject to another. The natural variance of T₁ and T₂ values in normal brain tissue has been measured to be roughly 10% across most tissues for both relaxation rates (Just & Thelen 1988; Stevenson et al., 2000). Though the the CV seen here is due to contrast changes, rather than absolute T₁ values, the variance in T₁ values is a likely explanation for this observed between-subject variance.

T₂ FLAIR

Again, in Fig. 2, the mean T₂ FLAIR images show clear distinction of gray matter, white matter, and CSF tissue compartments. Also following the results of the T₁ weighted analysis, the within-subject CV is very low, with the between-site and between-subject measurements showing substantially larger variations. Again, the between-site variances are likely the result of pulse-sequence differences between vendors, while the between-subject variance is due to the aforementioned known 10% variance in T₂ values across healthy subjects.

${\text{T}}_2^{*}$

Mean ${\text{T}}_2^{*}$ images show some distinction of gray matter, white matter, and CSF tissue compartments in Fig 3. Following the trend of the previous results, the within-subject CV is very low. However, in this case the between-site variations show increased CV values, particularly within the center of the brain (black arrow). This is due to reception coil and post-processing differences between the sites, whereby some of the coils had improved signal reception from deeper areas of the brain. The between-subject CV does not show as substantial of an effect from the coil variations, due to the fact that between subject CVs were calculated on a site-by-site basis, then averaged across sites. Instead, a stronger CV is seen in the frontal lobe (white arrow). This region of increased between-subject CV is due to variations of sinus cavity geometry and composition (e.g. mucus) from subject to subject. Such variations change local magnetic field perturbations and consequently alter local signal integrity in ${\text{T}}_2^{*}$ weighted images.

Mean ${\text{T}}_2^{*}$ maps show some distinction of gray matter, white matter, and CSF tissue compartments in Fig. 4. The the within-subject CV is again very low for this quantitative metric. In this case, the between-subject variation is again larger than the between-site variation. This is due to natural variations of tissue $T_2^{*}$ values from subject to subject (Wansapura et al., 1999).

Diffusion

As shown in Fig. 5, CV values for MD are best for the within-subject measurements and worst for the between-subject measurements. With chaotic flow patterns in the CSF, high variance is seen in both the ventricles and along the pial surface.

Like MD, the CV values for FA are lowest within-subjects and worst for the between-subject measurements, seen in Fig. 6. These results indicate that between-site and cross platform systematic diffusion weighted imaging variations are lower than the natural variations between subjects. Additionally, FA metrics are shown to yield less variance in white matter where diffusion is more well defined.

Functional

The mean REHO map in Fig. 7 clearly shows more homogeneity within the gray matter, which is expected. Like all other metrics, values for REHO are best for the within-subject measurements. In the gray matter compartments, REHO shows very similar CV in between-subject and between-site measurements. In the white matter compartments, which are not typically studied in fMRI analyzes, REHO shows slightly improved stability in the between-site analysis.

Like REHO, the mean fALFF map shows an increase of power in low frequency BOLD oscillations within the gray matter, as expected in Fig 8. Like diffusion metrics, CV values for fALFF are consistently the lowest for the within-subject measurements and worst for the between-subject measurements.

Arterial spin labeling

As expected, the mean rCBF map shows substantially increased flow in the gray matter in Fig. 9. Consequently, the within-subject CV is far better in the gray matter than the other tissue compartments. The between-site CVs are far less than the between-subject CVs in the gray matter compartments. Even with substantial differences between the ASL acquisitions on the two vendor platforms, the subject-specific variations are more substantial than the quantitative platform-specific performance differences.

Region of interest results

Tables providing numerical CV results for all metrics across all ROIs are included as Online Supplements. Figure 10 provides a scatter plot summary of these ROI CV values and displays trends that have been consistently revealed in this work. The following trends are apparent in this plot: relative CBF shows the most systematic variation across all analysis groups, diffusion metrics are more consistent in white matter and yield CVs of less than 15% in within subject gray matter ROIs, and local fMRI metrics (REHO and fALFF) show strong stability in gray matter ROIs. The fMRI metrics also show excellent stability in the white matter, although most other metrics show similar levels of stability (with the exception of rCBF).

Fig. 10.

Coefficients of variation across all subjects in brain regions of interest for each modality

Table 5 quantitatively illustrates the observed trends in this stability analysis. For all of the metrics, within-subject CV is the lowest of the analysis groups. In addition, for most of the metrics, between-subject CVs are the largest source of variance. The exceptions to this trend are highlighted in boldface. Of these exceptions, only the underlined cases exhibit statistical significance. Only one of these cases (${\text{T}}_2^{*}$ weighted values in the deep gray compartment) is outside of the CSF compartment group. As previously discussed, this result is not surprising, due to the variations of reception coil performance between the different sites. The deep gray compartment will be most effected by this variance. The CSF compartment consistently shows very high CVs, and is clearly not a good compartment for comparison of the metrics analyzed in this work.

Discussion

Coefficients of variation illustrate different levels of variability between scanning sessions within a subject, between platforms, and between subjects. These differences offer insight into the levels of precision available for evaluating concussion exposure and recovery related changes in the imaging data acquired as part of the ARC study.

For each analyzed metric, coefficients of variation are lowest within repeated subjects. This is expected, as anatomical changes over the time scale of the repeated measurements are minimal, and identical hardware is used for each measurement within individual subjects. Despite these fixed parameters, physiologic factors can contribute to the non-zero coefficient of variation within the repeated measures. Such physiological variance is, in fact, the focal point of interest in the larger CARE MRI study. Variance within subjects across time points will be utilized to track the natural history of injury recovery, potential changes associated with exposure to sub-concussive impacts, and the natural variation of the imaged biomarkers associated with participation in high-level athletics.

Between-site coefficients of variation changed substantially for different imaging metrics. Generally, quantitative images (${\text{T}}_2^{*}$, MD, FA, REHO, and fALFF) show lower levels of variation than contrast weighted images (T₁, T₂ FLAIR, ${\text{T}}_2^{*}$). This difference is expected, given the variations of pulse sequences employed by the different scanner vendors.

Significant differences exist in the utilized approaches to ASL blood-flow estimation on the GE and Siemens platforms. This led to increased coefficient of variation in computed rCBF maps, which is particularly visible in regions of low CBF; e.g. white matter. The majority of the ASL datasets were acquired on the Siemens platform with the 2D-pASL approach; which demonstrated much higher variance than the 3D-pCASL approach. As a specific example, we point to an ROI based in the left postcentral gyrus. The CV of subjects in the GE platform group (N=5) in this ROI was 0.24. In the Siemens platform group (N=17), the CV in this ROI was 0.80. However, even with higher performing 3D-pCASL approach, the rCBF CV values are substantially higher than those of the other metrics analyzed in this study. Preliminary analyzes of ASL metrics in mTBI have shown very large effect sizes that survive this relatively large systematic variation in the measurement (Wang et al., 2016).

Between-subject coefficients of variation are generally found to be higher than all other analyzed data groups, with the single exception of ${\text{T}}_2^{*}$ weighted imaging.

To our knowledge, this is the first stability analysis of a wide variety of MRI metrics spanning locations, vendors, and subjects. The unique nature of the CARE consortium’s longitudinal non-contact sport control population enabled this broad MRI biometric stability analysis. In addition, the deployment of a human phantom to the imaging sites allowed for a more intricate analysis of site to site and cross-vendor differences.

The statistical approach in this study closely followed the TRACK-TBI stability analysis (Palacios et al., 2016). However, the data acquisition approaches for these two stability assessments were different. The TRACK-TBI approach utilized traveling phantom and diffusion-phantom acquisitions to look carefully at diffusion metric stability. This allowed for a limited study of within-subject and between-site variations. With the addition of a longitudinal non-injured control cohort, the current study was able to more carefully examine between-subject variations as well. However, the within-subject and between-site MD and FA stability results presented here are in very close agreement with those reported by the TRACK-TBI study, with white-matter ROIs CVs ranging around a few percent (Palacios et al., 2016). With respect to diffusion metrics, the ARC MRI stability analysis shows that the between-subject FA and MD variations are decisively larger, at roughly twice the CV seen within subjects or between sites.

Conclusions

This study represents a fundamental effort by the CARE Consortium to determine the stability and reproducibility of key MRI metrics in multi-center TBI studies, which is critical to reliably measuring and interpreting the effects of concussion on brain structure and function. The cumulative results of this study have established stability expectations for a broad set of advanced MRI metrics utilized in analyzes of mTBI and neuroimaging in general. This analysis relied upon acquisitions collected on normal controls scanned at longitudinal time points along with traveling human phantom assessments at each of four study imaging sites. Statistical analysis of metric data derived from these datasets has identified an important trend in the data stability: between-subject instability matches or dominates between-site instability. This finding is of substantial importance, in that it provides confidence in ability to perform collective analysis of all advanced mTBI MRI biomarkers from carefully controlled imaging studies spanning multiple imaging sites and multiple imaging vendors.