The Minimal Preprocessing Pipelines for the Human Connectome Project

Abstract

The Human Connectome Project (HCP) faces the challenging task of bringing multiple magnetic resonance imaging (MRI) modalities together in a common automated preprocessing framework across a large cohort of subjects. The MRI data acquired by the HCP differ in many ways from data acquired on conventional 3 Tesla scanners and often require newly developed preprocessing methods. We describe the minimal preprocessing pipelines for structural, functional, and diffusion MRI that were developed by the HCP to accomplish many low level tasks, including spatial artifact/distortion removal, surface generation, cross-modal registration, and alignment to standard space. These pipelines are specially designed to capitalize on the high quality data offered by the HCP. The final standard space makes use of a recently introduced CIFTI file format and the associated grayordinates spatial coordinate system. This allows for combined cortical surface and subcortical volume analyses while reducing the storage and processing requirements for high spatial and temporal resolution data. Here, we provide the minimum image acquisition requirements for the HCP minimal preprocessing pipelines and additional advice for investigators interested in replicating the HCP’s acquisition protocols or using these pipelines. Finally, we discuss some potential future improvements for the pipelines.

Introduction and Rationale

The Washington University-University of Minnesota Human Connectome Project Consortium (WU-Minn HCP) (Van Essen et al., 2012a) is charged with bringing data from the major MRI neuroimaging modalities, structural, functional, and diffusion, together into a cohesive framework to enable cross-subject comparisons and multi-modal analysis of brain architecture, connectivity, and function. Specifically, the imaging modalities include T1-weighted (T1w) and T2-weighted (T2w) structural scans, resting-state and task-based functional MRI scans, and diffusion-weighted MRI scans. Additionally, the HCP is committed to making these complex datasets publicly available and easy to use. Although the HCP will make the unprocessed NIFTI data available for all usable scans, we anticipate that many investigators will prefer to use the outputs of the minimal preprocessing pipelines, developed by members of the WU-Minn HCP in collaboration with members of the MGH/UCLA HCP. Investigators in the HCP consortia have acquired extensive experience with the state-of-the-art imaging data being generated by the HCP though their involvement in the 2-year-long piloting phase (I). The WU-Minn HCP Phase II data are being acquired on a customized Siemens Skyra 3T scanner, using advanced pulse sequences that are described elsewhere in this special issue (Ugurbil et al.; Smith et al.; Sotiropoulos it et al., THIS ISSUE). Many of the HCP datasets are qualitatively different from standard neuroimaging data, having higher spatial and temporal resolutions and differing distortions. As a result, these data must be processed differently to achieve optimal results. Users of unprocessed data must deal with complex and esoteric issues that are particularly relevant for these acquisitions, such as the correction of gradient nonlinearity distortion in images that were acquired with oblique slices relative to the scanner’s coordinate system. These and many other preprocessing considerations have already been resolved in the minimally preprocessed data. Additionally, the minimal preprocessing pipelines include steps, such as field map distortion correction, which are widely accepted to be beneficial (Cusack et al., 2003; Jezzard and Balaban, 1995) but often neglected in practice. Finally, the minimal preprocessing results are available in standard volume and combined surface and volume spaces to enable easier comparisons across different studies, both within the HCP consortium and outside of it. By taking care of the necessary spatial preprocessing once in a standardized fashion, rather than expecting each community user to repeat this processing, the minimal preprocessing pipelines will both avoid duplicate effort and ensure a minimum standard of data quality. Moreover, for users working with HCP data, the preprocessing pipelines will make it easier to report findings in a common space and replicate analytic efforts. We anticipate that most investigators will be more interested in the analyses that they can do with the HCP data, and less interested in repeating or refining the complicated preprocessing required to best capitalize on these cutting edge data.

The overall goals of the six minimal preprocessing pipelines are 1) to remove spatial artifacts and distortions; 2) to generate cortical surfaces, segmentations, and myelin maps; 3) to make the data easily viewable in the Connectome Workbench visualization software (Marcus et al, THIS ISSUE); 4) to generate precise within-subject cross-modal registrations; 5) to handle surface and volume cross-subject registrations to standard volume and surface spaces; and 6) to make the data available in the CIFTI format in a standard “grayordinates” space (see below). While achieving these goals, the minimal preprocessing pipelines are designed to minimize the amount of information actually removed from the data. Further preprocessing, such as, significant spatial smoothing, temporal filtering, nuisance regression, or motion censoring (scrubbing), all remove significant amounts of information. Moreover, the preferred strategies for using these techniques remain subjects of debate (Carp, 2011; Fox et al., 2009; Murphy et al., 2009; Power et al., 2011, 2012; Saad et al., 2012; Turner, 2012). Thus, these forms of preprocessing fall outside the scope of “minimal preprocessing,” and are largely covered elsewhere (e.g. Smith et al. THIS ISSUE, Barch et al THIS ISSUE). While the HCP consortium may make some choices and suggestions on these steps that remove information, many investigators will presumably prefer to make different choices. Additionally, there is no consensus as to the best way to analyze these imaging modalities after preprocessing, and new analysis methods will continue to be developed. Thus, while the HCP will provide data that have undergone additional analyses, we expect many users will be interested in downloading the minimally preprocessed data as the starting point for their own analyses.

The CIFTI File Format and Grayordinates, a Combined Cortical Surface and Subcortical Volume Coordinate System

Standard volume-based neuroimaging analyses will be easy to carry out using the outputs of the minimal preprocessing pipelines (e.g. Barch et al THIS ISSUE). However, we note that such analyses will waste many of the potential benefits offered by the high resolution HCP data for greater accuracy in spatial localization, both within individuals and across subject groups. It is now well established (Anticevic et al., 2008; Fischl et al., 2008; Frost and Goebel, 2012; Tucholka et al., 2012; Van Essen et al., 2012b) (Smith et al THIS ISSUE) that it is beneficial to analyze cortical neuroimaging data with surface-constrained methods. The fundamental reason is that the convoluted cortical sheet is most easily manipulated and analyzed as a 2D surface. 2D geodesic distances along the surface, rather than 3D Euclidean distances within the volume, are most neurobiologically relevant in many circumstances and are most suited to the geometry of the cerebral cortex. This insight has important consequences for image processing, especially spatial smoothing and intersubject registration. For example, cortical areas are spaced farther apart across the surface than they are in the volume because of the cortical convolutions. Functionally distinct areas may be separated by only a few millimeters in the volume across sulcal banks or gyral blades. Therefore, spatial smoothing constrained to the cortical surface will not mix signals from distinct cortical areas as much as 3D volumetric smoothing—not to mention avoiding mixing in nuisance signals from CSF or white matter (Jo et al., 2007). Additionally, surface-based intersubject registration needs only to align data within the plane of the surface (2D), whereas volume-based intersubject registration must also attempt to align the plane of the surface itself (3D). Thus, surface-based alignment of data across subjects is a somewhat easier problem to solve and yields results that achieve greater overlap (Anticevic et al., 2008; Fischl et al., 2008; Fischl et al., 1999b; Frost and Goebel, 2012; Tucholka et al., 2012; Van Essen et al., 2012b).

In contrast to cortical data, subcortical data generally come from deep grey matter structures and nuclei that exhibit 3D geometry, which is different from the 2D cerebral cortical sheet whose thickness varies only modestly. (Note: the cortical thickness of the cerebellum is much less than that of the cerebral cortex. It cannot be automatically segmented using the pipelines presented here, but in the future, new methods may be able to perform cerebellar surface reconstruction with the high-resolution HCP datasets. For now the cerebellum is treated as a volumetric structure, but future surface-based analysis of the cerebellum is supported within the existing CIFTI file format, Connectome Workbench, and its commandline algorithms). 3D subcortical structures are best analyzed in 3D volumetric space, both with regard to spatial smoothing and to cross subject comparisons. We use FreeSurfer to automatically segment many subcortical grey matter structures (Fischl et al., 2002). Much like we constrain cortical grey matter analyses to the surface, we can limit subcortical grey matter analyses to these automatically segmented parcels in each individual. We can then constrain 3D volumetric spatial smoothing to occur only within parcel boundaries, for example avoiding mixing signals from the third ventricle into the medial thalamus. Such parcel-constrained volumetric smoothing should in principle reduce the amount of noise being mixed into the grey matter structures of interest. Additionally, while nonlinear volume registration does an excellent job of aligning subcortical parcels (Andersson et al., 2007; Klein et al., 2009), we can constrain cross-subject comparisons of subcortical data to occur only within the subcortical parcels of each subject. This precise correspondence is achieved by resampling the sets of voxels in each of the subjects’ individually defined subcortical parcels to a standard set of voxels in each atlas parcel (see fMRISurface Pipeline below).

The Connectivity Informatics Technology Initiative (CIFTI) file format was created to support a variety of connectome-specific data representations, including combinations of cortical grey matter data modeled on surfaces and subcortical grey matter data modeled in volumetric parcels (http://www.nitrc.org/plugins/mwiki/index.php/cifti:ConnectivityMatrixFileFormats). Because gray matter can be modeled as either cortical surface vertices or subcortical voxels, the more general term “grayordinates” is used to describe the spatial dimension in this combined coordinate system. When right and left standard cortical surface meshes and a set of standard subcortical volume parcels are used to create a CIFTI grayordinates space, it is said to be a standard grayordinates space (Figure 1). Such a standard CIFTI grayordinates space has more precise spatial correspondence across subjects than volumetrically aligned data and is the desired endpoint of the HCP minimal preprocessing functional pipelines. The HCP rfMRI and tfMRI timeseries data are provided in this space (and also as NIFTI files in MNI volume space).

Figure 1.

Components of the CIFTI grayordinates spatial dimension and standard space. Left and right cerebral cortices contribute about 30k surface vertices each (less than the 32k vertex standard meshes, because the non-cortical medial wall is masked with standard ROIs). Additionally, 19 subcortical grey matter structures combine to contribute about 30k volume voxels. In total, there are 91,282 grayordinates corresponding to all of the grey matter sampled at a 2mm average vertex spacing on the surface and as 2mm voxels subcortically. The minimal preprocessing pipelines guarantee that each subject will have 91,282 maximally aligned grayordinates, facilitating cross-subject comparisons of multiple data modalities.

CIFTI was also developed to enable a more compact representation of high spatial and temporal resolution MRI data. Generation of a Dense Connectome (a matrix describing the connectivity from each grey matter point to all other grey matter points) is one instructive example. Early on in the HCP we realized that the generation of such a matrix would be to be extremely memory intensive and would require huge storage capacity. For example, the 2mm MNI space brain mask distributed with FSL (Smith et al., 2004) contains 228,483 voxels. A dense connectome with this spatial dimension would need 228,483 × 228,483 × 4bytes = ~195GB per modality, per analysis method, per subject. A key insight was the realization that all of grey matter could be represented at a 2mm resolution using only 91,282 grayordinates (Figure 1), including ~30,000 surface vertices (average inter-vertex spacing of 2mm) for each hemisphere and ~30,000 2mm subcortical grey matter voxels producing a CIFTI dense Connectome = 91,282 × 91,282 × 4bytes = 31GB. Combining only surface and volume grey matter data saves 84% of the space while still representing all of the grey matter to grey matter connectivity at the original 2mm resolution. Similarly for a single 1200 timepoint resting state run (14.4 minutes at 720ms TR), the CIFTI dense timeseries, consisting of grayordinates by time is 91,282 × 1200 = 0.41GB versus 228,483 × 1200 = 1.02GB for the masked volume timeseries, saving 60% of the space. The benefits are even larger when one considers that rectangular NIFTI volumes are often not masked at all in memory, storing many voxels outside the brainmask, and the same resting state run unmasked is 104 × 90 × 72 × 1200 = 3.01GB, with a CIFTI dense timeseries now providing an 86% savings in uncompressed disk space and, more importantly, memory. Thus, the CIFTI file format is important for reducing the computational and storage demands of high spatial and temporal resolution data. These considerations will only grow in importance as imaging resolutions continue to increase (e.g. with higher field strength). CIFTI also combines the left and right cerebral hemispheres and subcortical parcels into a single file, thereby simplifying file management for combined surface and volume analyses and visualization.

CIFTI files contain a single 2D matrix. One dimension of the matrix always represents the spatial domain (often the standard set of grayordinates) and the other dimension may represent something else. For instance, in the dense timeseries file type, the rows represent the spatial dimension, and the columns represent timepoints (Figure 2). In the dense Connectome file type, both matrix dimensions are spatial (grayordinates) (Figure 3). It is also possible to represent other types of data in grayordinates space, including ICA component spatial maps or task fMRI z-statistic contrast maps (Figure 4) in dense scalar file, or parcellation labels in a dense label file. A CIFTI file can be valid even if it contains only surface data from just one cerebral hemisphere, or the cerebellum, or just volume data. Within the HCP’s standard CIFTI grayordinates space, datasets from different subjects have the same spatial dimension, allowing straightforward grayordinate-wise comparisons across subjects. For the HCP data, these spatial correspondences have been achieved separately using the registration methods best suited to aligning each domain, i.e., nonlinear surface registration for the cortical structures and nonlinear volume registration for the volume structures (see HCP Structural Pipelines below).

Figure 2.

A CIFTI dense timeseries. The rows are grayordinates and represent the spatial dimension of the timeseries, whereas the columns are timepoints. Reading along a row produces a timecourse (green) for a given spatial point, and a different spatial point produces a different timecourse (red). The columns represent the spatial pattern across the entire brain at a given timepoint (yellow) and can be also visualized. A different spatial pattern at a second timepoint is also shown (cyan).

Figure 3.

An example of a CIFTI dense connectome. Both dimensions of this 2D matrix are grayordinates. Rows of the matrix represent seed points for functional connectivity, and the columns represent the spatial map of the connectivity. If one clicks on a point on the surface (green), Connectome Workbench finds the appropriate row in the dense connectome matrix, and displays the connectivity values on the appropriate cortical and subcortical structures. A different point (cyan) shows a different connectivity pattern.

Figure 4.

A CIFTI dense scalar file. CIFTI dense scalar files can contain any kind of scalar data, but in this case, two task fMRI contrast maps are shown (z-statistics). Each contrast corresponds to a different column in the schematic matrix. The first task contrast, Places – AVG Categories (yellow), is from a working memory task that used a number of categories of visual stimuli, including places. The second task contrast, Story – Math (cyan), is from a language task that contrasted listening to a story to doing math problems. For more details, see Barch et al THIS ISSUE.

Although the cortical surface and subcortical volume data are stored in a single 2D matrix, the CIFTI header contains information describing the structure that each row and column of the matrix belong to. For example, inside the HCP’s 2mm standard grayordinates space, row 1000 of the matrix would correspond to vertex 2152 of the left hemisphere surface (not vertex 1000, because vertices in the medial wall are excluded) and row 91000 would correspond to voxel (3.9,−1.4, 10.6) of the right thalamus. CIFTI files may contain whatever space the user chooses, however. Connectome Workbench (Marcus et al THIS ISSUE), the surface and volume data visualization platform customized for HCP data, can use the information in the CIFTI header to display the contents of the data matrix on the correct structures and in the correct spatial locations, as illustrated in Figures 2–4.

CIFTI is a new file format that is currently only supported natively by Connectome Workbench and its commandline utilities. While we encourage other neuroimaging software platforms to implement CIFTI compatibility, there are a variety of ways that these neuroimaging platforms can already make use of CIFTI data, benefiting from existing Connectome Workbench commandline file conversion utilities. If users need only to perform an analysis that treats each grayordinate independently, without reference to the spatial relationships between grayordinates, one can convert a CIFTI file to a NIFTI-1 file (by wrapping the grayordinates spatial dimension into the volumetric x, y, and z dimensions and keeping time as the 4^th NIFTI-1 dimension). Examples where this is useful include ICA analysis of a CIFTI dense timeseries in FSL’s melodic, or element-wise task fMRI analyses with FSL’s contrast_mgr or flameo executables. Similarly for generic analyses, CIFTI files can be converted to GIFTI external binary files for reading into Matlab. If knowledge of the spatial relationships needs to be preserved in the external software (an example is FSL’s film_gls tool for task fMRI analysis), a CIFTI file can be split into GIFTI surface data and NIFTI volume data, analyzed in separate parts, and then recombined into a CIFTI file. Finally, in Connectome Workbench’s commandline utilities we have prioritized developing CIFTI compatible algorithms that require knowledge of the spatial relationships in grayordinates data, such as spatial smoothing and resampling from one CIFTI standard space to another. We also provide flexible math commands that can be used evaluate mathematical expressions on CIFTI files. FSL is expected to include full CIFTI support in the future, but will provide surface analysis support via GIFTI files in the short-term (e.g. Barch et al THIS ISSUE).

During the piloting phase of the HCP grant (first two years) both acquisition (Ugurbil et al THIS ISSUE) and preprocessing methods were improved. One of these improvements was to incorporate preprocessing steps that capitalize on the advantages of CIFTI grayordinates datasets. In particular, we avoid unconstrained volume-based smoothing and cross subject averaging, instead using the more constrained methods described above. These methodological changes enable analyses that depend critically on accurate spatial localization, for example multi-modal cortical areal parcellation in which one attempts to estimate areal boundaries, measuring the areal extent of task-based activation, or measuring the areal extent resting state functional connectivity (Glasser et al., 2012a; Glasser et al., 2011). More conventional neuroimaging analyses that focus on peak activation coordinates or use spheres of a given radius as connectivity seeds may not require these improvements in spatial localization. Moreover, these methodological improvements reduce the need for spatial smoothing to attempt to account for alignment inaccuracies across subjects in group studies (Turner, 2012). We contend that it is generally better to avoid smoothing group data, in order to retain as much spatial detail as possible, though individual data may need to be smoothed (in a grayordinates-constrained way) to increase the signal/contrast of interest relative to high spatial frequency noise (e.g. myelin maps: (Glasser and Van Essen, 2011)). Continued improvements in cross-subject registration, such as driving registration with maps of properties more directly tied to the locations of cortical areas (e.g. myelin maps, task activations, and/or functional connectivity) (Robinson et al., 2013a; Robinson et al., 2013b) should accentuate this principle of better cross-subject cortical areal alignment, requiring even less spatial smoothing.

The above conceptual improvements in methodology served as guiding principles during the development of the minimal preprocessing pipelines presented here. We chose or developed methods that are fully automated and robust, so that large amounts of data can be processed without user intervention. Additionally, we used methods that cause minimal blurring and achieve accurate cross-modal alignment to enable use cases focused on fine details in the data. This will likely be an ongoing effort throughout the duration of the HCP, as the pipelines presented here will presumably undergo additions and improvements over time (see Future Directions below). Before discussing the pipelines themselves, we describe aspects of the HCP data acquisitions that are relevant to developing or running the HCP minimal preprocessing pipelines, but are not covered elsewhere in this issue (Ugurbil et al. THIS ISSUE; Smith et al. THIS ISSUE; Sotiropoulos et al. THIS ISSUE).

HCP Structural Acquisitions

The HCP structural acquisitions include high resolution T1-weighted (T1w) and T2-weighted (T2w) images (0.7mm isotropic) for the purpose of creating more accurate cortical surfaces and myelin maps than are attainable with lower resolution data (Glasser and Van Essen, 2011). While high spatial resolution was a primary goal, we also optimized the contrast parameters (flip angle (FA) and inversion time (TI) for the T1w scans, and echo time (TE) for the T2w scans). The major change from Glasser and Van Essen (2011) was to lengthen TE in the T2w scans to improve intracortical contrast for myelin detection. Additionally, B₀ fieldmaps, B₁−, and B₁+ maps are acquired for the purpose of correcting readout distortion in the T1w and T2w images and to enable future correction of intensity inhomogeneity by explicitly modeling the B₁− receive and B₁+ transmit fields. 32 minutes are spent acquiring the main T1w and T2w structural images, and 8 minutes are spent acquiring the auxiliary scans on the HCP’s custom 3T Siemens Skyra (Ugurbil et al. THIS ISSUE) using a 32-channel head coil. Two separate averages of the T1w image are acquired using the 3D MPRAGE (Mugler 3rd and Brookeman, 1990) sequence with 0.7mm isotropic resolution (FOV=224 mm, matrix=320, 256 sagittal slices in a single slab), TR=2400 ms, TE=2.14 ms, TI=1000 ms, FA=8°, Bandwidth (BW)=210 Hz per pixel, Echo Spacing (ES) =7.6 ms, with a non-selective binomial (1:1) water excitation pulse (a pair of 100 µs hard pulses with 1.2 ms spacing) to reduce signal from bone marrow and scalp fat, phase encoding undersampling factor GRAPPA=2, 10% phase encoding oversampling (anterior - posterior) to avoid nose wrap-around, asymmetric readout (i.e. omitting the initial 20% of the total samples before echo) along superior - inferior direction (z) with dwell time of 7.4 µs (used for readout distortion correction with FSL’s FUGUE (Jenkinson et al., 2012)). Two separate averages of the T2w image are acquired using the variable flip angle turbo spin-echo sequence (Siemens SPACE (Mugler et al., 2000)) with 0.7mm isotropic resolution (same matrix, FOV, and slices as in the T1w), TR=3200 ms, TE=565 ms, BW=744 Hz per pixel, no fat suppression pulse, phase encoding undersampling factor GRAPPA=2, total turbo factor=314 (to be achieved with a combination of turbo factor and slice turbo factor, when available), echo train length of 1105 echoes, 10% phase encoding oversampling (anterior - posterior) to avoid nose wrap-around, readout along superior - inferior direction with dwell time of 2.1µs (for readout distortion correction with FUGUE).

All auxiliary scans are acquired at 2mm isotropic resolution. The B₀ fieldmap is acquired by a dual-echo gradient echo sequence with delta TE=2.46ms. The B₁⁻ receive field can be calculated from proton density weighted images acquired from a FLASH sequence once with the 32 channel head receive coil and once with the body receive coil (similar to Siemens pre-scan normalize). The B₁⁺ transmit field can be calculated from an actual flip-angle imaging (AFI) acquisition (Yarnykh, 2007) with TR1/TR2 = 20/120ms (target flip angle=50°).

HCP Functional and Diffusion Acquisitions

The functional and diffusion acquisitions are described in detail in other papers of this special issue (Ugurbil et al.; Smith et al.; Sotiropoulos et al., THIS ISSUE), but a number of points relevant to the HCP pipelines are noted here as well. The functional data are acquired at 2mm isotropic, which is of unusually high spatial resolution for whole-brain coverage at 3T. Although the spatial specificity of BOLD fluctuations measured at 3T is around 4mm FWHM ((Parkes et al., 2005) Ugurbil et al. THIS ISSUE), there are benefits of sampling the data more finely when the goal is to accurately map grey matter BOLD signals onto the cortical sheet. From a purely geometrical perspective, higher resolution scans provide the potential to decrease induced cross-sulcal and cross-gyral timeseries correlation. Figure 5 uses a simple volume to surface mapping model to illustrate this effect by comparing 4mm, 3mm, 2.5mm, and 2mm fMRI acquisition resolutions and showing that the induced correlations between surface vertices that are far apart across the surface but close together in the volume are mostly eliminated at 2mm isotropic. Thus, with high-resolution data it becomes possible to assign the BOLD signal more specifically to opposing banks of sulci (Figure 6). This effect will become even more beneficial at 7T, where the point-spread-function of the BOLD fluctuations is closer to 2mm FWHM ((Shmuel et al., 2007) Ugurbil et al. THIS ISSUE). Higher resolution also helps limit the partial volume effects of other noise sources such as CSF, large veins, and white matter.

Figure 5.

A comparison of the geometrical effects of various fMRI acquisition resolutions on folded cortex averaged across the Q1 20 unrelated HCP subjects all on the same color scale. A, 4mm isotropic, B, 3mm isotropic, C, 2.5mm isotropic, D, 2mm isotropic. When mapping fMRI data of a given resolution to the midthickness surface, larger voxels can artificially increase the correlation between geodesically distant surface vertices, as larger voxels can span both sides of a sulcus or a thin gyral blade of white matter. The measure being displayed is calculated for each vertex by finding the set of vertices that could potentially share a voxel's data with the selected vertex during volume to surface mapping with trilinear interpolation. For each of the vertices in this set, we compute the expected correlation to the selected vertex that would result from mapping a white noise timeseries by trilinear interpolation, aligned such that a voxel center is located midway between the two vertices. We then multiply by the geodesic distance between the vertices in order to upweight the measure for the vertices that are far apart across the surface. Finally the maximum of the measure is taken over the set of vertices found to be in range. This model is a significant simplification of the problem, however, as it does not account for the variable pointspread function of BOLD at 3T or the effects of larger draining veins. The model also uses a simple trilinear interpolation volume to surface mapping method instead of the more complicated ribbon-based method used in the pipelines below. This model does, however, point to the potential benefits of higher acquisition resolution in reducing the induced cross-sulcal and cross gyral correlations between vertices that are far apart along the surface (and thus potentially lying in different cortical areas), but close together in the volume.

Figure 6.

Illustration of the spatial specificity of the high resolution HCP fMRI acquisition and minimal preprocessing pipelines in the left (top) and right (bottom) hemispheres of one HCP subject. Two resting state ICA components are shown, one involving the POS2 and retrosplenial cortex on the left and the other involving the peripheral visual cortex on the right. These two components exist on opposite sides of the parieto-occipital sulcus and have very distinct functional connectivity patterns. With the 2mm isotropic acquisition used here, the component spatial maps fall nicely between the white (green) and pial (cyan) surfaces that define the grey matter ribbon on either side of the sulcus.

Fast TR sampling using multiband pulse sequences has several benefits (HCP fMRI: TR=720ms, using a multiband factor of 8, FA=52°--reduced from 90° to match the Ernst angle, maximizing SNR). Multiband enables higher spatial resolution acquisitions, requiring large numbers of slices for whole-brain coverage, while still keeping the TR low. It reduces the need for slice timing correction, as all slices in each volume are acquired much closer together than in typical fMRI acquisitions (TR ~ 2.5s). Thus, in the HCP pipelines, no slice timing correction is employed. Additionally, it provides more robustness against the effects of rapid head movement, since three volumes are acquired in the time it typically takes to acquire one standard TR volume. A rapid movement may directly corrupt only one of these volumes, potentially leaving two intact, though some motion effects may be more persistent. Fast TR acquisitions require a different strategy for accurate registration to structural data. Because the multiband timeseries images have reduced tissue contrast (due to incomplete T1 relaxation), HCP pipelines use the single-band reference image with full tissue contrast as the image for registration to the structural data.

To reduce the signal loss and distortions from the high-resolution acquisition and to minimize the TE (33ms in our scans) and echo train length, we acquire a balanced number of volumes with left-right and right-left phase encoding directions using an asymmetric acquisition matrix, rather than the more typical anterior-posterior or posterior-anterior phase encoding directions with a symmetric acquisition matrix. The geometrical dimensions of the gradient echo EPI images are 2mm isotropic resolution (FOV: 208mm × 180mm, Matrix: 104 × 90 with 72 slices covering the entire brain). These left-right and right-left phase encoding directions also have the benefit of reducing the chance that the fMRI data would alias in the phase encoding direction or move outside the FOV because of distortion combined with head motion. To more rapidly measure the b0 field for correction EPI distortions, we acquire two spin echo EPI images with reversed phase encoding directions (60 seconds total for 3 pairs of images). (Note that we refer to these images as spin echo fieldmaps, though they measure the field by reversing the phase encoding direction, which is a very different mechanism from standard field maps that use a phase difference calculated from two different TEs). These spin echo EPI images have the same geometrical, echo spacing (0.58ms in our scans), and phase encoding direction parameters as the gradient echo fMRI scans. These images enable accurate correction for spatial distortions in the fMRI images so that these images can be precisely aligned with the structural images. Two of these spin echo EPI fieldmapping pairs are acquired in each functional session, for added robustness with respect to acquisition errors and subject movement, along with one set of B₁− receive field-mapping images (with identical parameters to those described in the structural session).

While the diffusion acquisition is covered in detail in (Ugurbil et al.; Sotiropoulos it et al., THIS ISSUE), a few points deserve mention here. Very high-resolution acquisitions (1.25mm isotropic) were chosen, as this will be critical for accurately mapping connectivity between cortical grey matter regions, as opposed to just localizing major fascicles in deep white matter. To obtain these high-resolution images, it was beneficial (in terms of TE) to use a Stejskal-Tanner (monopolar) diffusion-encoding scheme. This encoding scheme, together with the customized 100 mT/m gradient set in the HCP’s Skyra, achieves sufficient SNR at 1.25mm isotropic resolution and a diffusion-weighting of up to b=3000 s/mm² (Sotiropoulos it et al., THIS ISSUE). However, the use of a monopolar Stejskal-Tanner diffusion scheme and the more powerful gradients increases eddy current-induced distortions, which require the use of a sophisticated approach for correction (Andersson et al., 2012), (Sotiropoulos et al., THIS ISSUE). Correction for EPI and eddy-current-induced distortions in the diffusion data is based on manipulation of the acquisitions so that a given distortion manifests itself differently in different images (Andersson et al., 2003). Two phase-encoding direction-reversed images for each diffusion direction are acquired. To ensure better correspondence between the phase-encoding reversed pairs, the whole set of diffusion-weighted (DW) volumes is acquired in six separate series. These series are grouped into three pairs, and within each pair the two series contain the same DW directions but with reversed phase-encoding (i.e. a series of M_i DW volumes with RL phase-encoding is followed by a series of M_i volumes with LR phase-encoding, i=[1,2,3]). These diffusion-weighted images are corrected for B₀ and eddy current distortions as described in the diffusion preprocessing pipeline below. In addition to the diffusion-weighted images, one set of B₁− receive fieldmapping images, with identical parameters to those described in the structural session, is acquired.

HCP Pipelines: Minimum Acquisition Requirements

A number of investigators have asked questions such as: “How do I acquire data like the HCP?” and “What data do I need in order to use the HCP pipelines?” Though many studies will not require, or be in a position to acquire, data using protocols identical to those in the HCP, it is worth describing the minimum acquisition requirements for the HCP pipelines, along with suggestions for best acquisition practices. For all of the following, a 32-channel head coil will be very beneficial in providing the SNR necessary to emulate HCP scanning parameters. For the structural pipelines (upon which the functional and diffusion pipelines depend), high-resolution T1w and T2w scans are both required for surface reconstruction. While 1mm isotropic is acceptable, higher resolutions (0.8mm or 0.7mm) appreciably improve surface reconstructions and myelin maps ((Glasser et al., In Press), see below), particularly in thin regions of cortex or regions with thin gyral blades of white matter (e.g. primary somatosensory cortex or early visual cortex, see Figure 13). The T2w image is required to make accurate pial surfaces that exclude dura and blood vessels, which are isointense to grey matter in the T1w image (Figure 14). The T2w image is also used to make myelin maps based on the T1w/T2w ratio (Glasser and Van Essen, 2011). A regular gradient echo fieldmap (magnitude and phase difference) of relatively high resolution (e.g. 2mm isotropic) is recommended to correct the structural images for readout distortion, which differs between the T1w and T2w images because of their differing readout bandwidth (van der Kouwe et al., 2008). However, in practice the relative distortion between the two images is small (≤1mm) even in high susceptibility regions (e.g. inferior temporal cortex and orbitofrontal cortex). For the T1w image, some degree of fat insensitive excitation (Howarth et al., 2006) is recommended to prevent surface reconstruction errors. Nevertheless, off-resonance fat saturation should be avoided for either the T1w or T2w images to preserve contrast for myelin content by not modifying magnetization transfer effects (Glasser et al., In Press). In addition, vendor implemented receive bias field corrections, like Siemens’s pre-scan normalize, must be matched between the T1w and T2w images (either on for both or off for both).

Figure 13.

The benefits (stars) of a high resolution (0.7mm isotropic) T1w image for reconstructing the white matter surface in thin, heavily myelinated cortical regions such as the early visual cortex in the calcarine sulcus (A) and the early somatosensory cortex in the central sulcus (B). The red surface contour is the output of standard recon-all on the downsampled 1mm isotropic T1w data and the green surface contour is after adjustment with the original 0.7mm isotropic T1w data. In general, the white matter surface is placed too superficially in these regions (Type A error, Glasser and Van Essen 2011).

Figure 14.

The benefits of using a high resolution (0.7mm) T2w image to assist in pial surface placement. The blood vessels and dura are close to isointense with grey matter in the T1w image (A), however they are clearly of different intensity in the T2w image (B), allowing them to be easily excluded from the grey matter ribbon. The red surface is from the T1w image whereas the cyan surface is after cleanup with the T2w image.

For the functional pipelines, a fieldmap is required, because any neuroimaging analysis that aims for precise cross-modal registration between functional and structural (or other data modalities) will require EPI distortion correction. In general, either standard gradient echo fieldmaps or spin echo EPI fieldmaps can be acquired, though spin echo EPI field maps can be acquired more quickly with less chance of motion corruption. The geometrical parameters (FOV, matrix, phase encoding direction, resolution, number of slices) and echo spacing must be matched between the gradient echo EPI fMRI timeseries and the spin echo EPI fieldmaps. A single band reference scan also needs to be saved (as a separate series) together with the multi-band timeseries when using a multi-band sequence to serve as the reference for motion correction and for more accurate EPI to T1w registration. Higher spatial (≤2.5mm) and temporal (≤1s) resolutions are recommended for the reasons discussed above. For resting state fMRI scans, longer contiguous runs are preferable to shorter discontinuous runs, for temporal denoising purposes (Smith et al.; THIS ISSUE).

For the diffusion pipelines, the directions of the diffusion-sensitizing gradients should be distributed across the whole sphere to allow for better correction of eddy-current distortions and subject movement (Sotiropoulos et al., THIS ISSUE). In addition, and for estimating EPI distortions as well, pairs of diffusion-weighted volumes acquired with reversed phase-encoding directions are needed. Ideally, all volumes are acquired in phase-encoding reversed pairs, as this further helps in robustly estimating eddy-current distortions, but the minimum requirement is that at least a few (one in theory) b₀ pairs are acquired in this way. A Stejskal-Tanner (monopolar) diffusion-encoding scheme with multiband acceleration and phase encoding along the left-right direction are recommended to allow reduction of the TE (more SNR) and TR (for the collection of a larger number of unique diffusion directions (Sotiropoulos it et al., THIS ISSUE)). High spatial resolution (≤1.5mm), larger numbers of unique directions (≥128), and multiple q-space shells with moderately high b_max (e.g. 3000 s/mm²), depending on the employed resolution and the available SNR) will be required to take advantage of improvements in diffusion fiber orientation modeling and tractography that are being generated by the HCP. Investigators using standard Siemens Skyra or Trio equivalent scanners will likely need to sacrifice spatial resolution and/or maximum diffusion weighting to achieve high quality results, as the 100mT/m gradients of the HCP Skyra remain an important contributor to the improved diffusion SNR allowing a 1.25mm isotropic resolution with b_max=3000 s/mm².

HCP Pipelines

As shown in Figure 7, the six minimal preprocessing pipelines include three structural pipelines (PreFreeSurfer, FreeSurfer, and PostFreeSurfer), two functional pipelines (fMRIVolume and fMRISurface), and a Diffusion Preprocessing pipeline. Figure 7 also shows the overall workflow for preprocessing and data analysis in the HCP. We provide a brief high-level description of all the pipelines, followed by a detailed description of each, including the rationale for choices made in the pipelines and descriptions of novel methods used in them.

Figure 7.

An overview of the HCP Minimal Preprocessing Pipelines. The HCP Structural Pipelines are run first, and then either the HCP Functional Pipelines or the Diffusion Preprocessing Pipeline can be run. Afterwards, resting state, task, and diffusion analysis can proceed.

HCP Pipelines: Overview

The main goals of the first structural pipeline, PreFreeSurfer, are to produce an undistorted “native” structural volume space for each subject, align the T1w and T2w images, perform a B₁ (bias field) correction, and register the subject’s native structural volume space to MNI space. Thus, there are two volume spaces in HCP data (overview in Figure 8): 1) The subject’s undistorted native volume space (rigidly “aligned” to the axes of MNI space), which is where volumes and areas of structures should be measured and where tractography should be performed, as this space is the best approximation of the subject’s physical brain. 2) Standard MNI space is useful for comparisons across subjects and studies, particularly of subcortical data, which is more accurately aligned by nonlinear volume registration than cortical data is. The FreeSurfer pipeline is based on FreeSurfer version 5.2, with a number of enhancements. The main goals of this pipeline are to segment the volume into predefined structures (including the subcortical parcels used in CIFTI), reconstruct white and pial cortical surfaces, and perform FreeSurfer’s standard folding-based surface registration to their surface atlas (fsaverage). The final structural pipeline, PostFreeSurfer, produces all of the NIFTI volume and GIFTI surface files necessary for viewing the data in Connectome Workbench, along with applying the surface registration (to the Conte69 surface template (Van Essen et al., 2012b)), downsampling registered surfaces for connectivity analysis, creating the final brain mask, and creating myelin maps. There are three surface spaces in HCP data: the native surface mesh for each individual (~136k vertices, most accurate for volume to surface mapping), the high resolution Conte69 registered standard mesh (~164k vertices, appropriate for crosssubject analysis of high resolution data like myelin maps) and the low resolution Conte69 registered standard mesh (~32k vertices, appropriate for cross-subject analysis of low resolution data like fMRI or diffusion). These spaces are also shown in Figure 8. The 91,282 standard grayordinates (CIFTI) space is made up of a standard subcortical segmentation in 2mm MNI space (from the Conte69 subjects) and the 32k Conte69 mesh of both hemispheres (Figure 1). Following completion of the structural pipelines, the functional or diffusion pipelines may run.

Figure 8.

An overview of the volume and surface spaces produced by the HCP pipelines. Each space has specific uses, which are listed.

The first functional pipeline, fMRIVolume, removes spatial distortions, realigns volumes to compensate for subject motion, registers the fMRI data to the structural, reduces the bias field, normalizes the 4D image to a global mean, and masks the data with the final brain mask. Standard volume-based analyses of the fMRI data could proceed from the output of this pipeline. Care is taken in the fMRIVolume pipeline to minimize the smoothing from interpolation, and no overt volume smoothing is done. The main goal of the second functional pipeline, fMRISurface, is to bring the timeseries from the volume into the CIFTI grayordinates standard space. This is accomplished by mapping the voxels within the cortical grey matter ribbon onto the native cortical surface, transforming them according to the surface registration onto the 32k Conte69 mesh, and mapping the set of subcortical grey matter voxels from each subcortical parcel in each individual to a standard set of voxels in each atlas parcel. The result is a standard set of grayordinates in every subject (i.e. the same number in each subject, and with spatial correspondence) with 2mm average surface vertex and subcortical volume voxel spacing. These data are smoothed with surface (novel algorithm, see below) and parcel constrained smoothing of 2mm FWHM to regularize the mapping process. The output of these pipelines is a CIFTI dense timeseries that can be used for subsequent resting-state or task fMRI analyses. The diffusion preprocessing pipeline does the following: normalizes the b₀ image intensity across runs; removes EPI distortions, eddy-current-induced distortions, and subject motion; corrects for gradient-nonlinearities; registers the diffusion data with the structural; brings it into 1.25mm structural space; and masks the data with the final brain mask.

HCP Pipelines: PreFreeSurfer

The PreFreeSurfer pipeline (Figure 9) begins with the correction of MR gradient-nonlinearity-induced distortions. Because of tradeoffs made in the design of the custom HCP Skyra, the head is ~5cm above isocenter, making the effects of gradient nonlinearities more pronounced in HCP Skyra images relative to those from standard 3T scanners, especially in the frontal lobe (Figure 10). All images used in structural processing (the T1w, T2w, and the field map magnitude and phase) must be corrected for gradient nonlinearity distortion. We did not use the Siemens online gradient distortion correction method because it is not available for all sequences, and the gradient distortion correction must be done on all images for them to align in cross-modal registrations. For the correction of the distortion, the magnetic field generated by each gradient coil is modeled by a spherical harmonic expansion (specific to the SC72 gradients in the 3T Connectome scanner). The correction is then done with a customized version of the gradient_nonlin_unwarp package available in FreeSurfer (Jovicich et al., 2006). The customized version calculates an FSL-format warpfield that represents the spatial distortion of the image by using a proprietary Siemens gradient coefficient file (available on the scanner used to acquire the images) and the mm coordinate space of the image (including the rotation between image matrix space and scanner axes – that is, the oblique portion of the sform, where the sform is the matrix that relates the voxel coordinates to the mm coordinate space of the scanner, as defined by the NIFTI standard). This warpfield can then either be concatenated with other transforms or applied to the image using spline interpolation (which causes less blurring than trilinear interpolation). While this correction is required for a scanner like the HCP Skyra, which has more significant gradient nonlinearity, it may not be needed for standard scanners (e.g. Siemens Trio) with gradients that are more linear over a larger FOV and where the head position is closer to isocenter. Thus, this correction can be turned off if desired.

Figure 9.

The steps of the PreFreeSurfer pipeline. T1w and T2w images are processed through the same steps in parallel up until cross-modal registration. A fieldmap is used to remove readout distortion in the T1w and T2w images, which slightly improves their alignment and resulting myelin maps. The PreFreeSurfer pipeline also produces the Native and MNI Volume Spaces, which will be used in later pipelines and analyses.

Figure 10.

Effect of gradient nonlinearity distortion on the T1w image. Panel A is the T1w image before gradient distortion correction, whereas Panel B is the T1w image after gradient distortion correction. The red lines in both panels are from a rough tissue segmentation generated with FSL's FAST on the gradient distortion corrected image shown in Panel B. The ellipse highlights the region of maximum distortion, which is as much as the width of the grey matter ribbon in some places.

Subsequently, any repeated runs of T1w and T2w images are aligned with a 6 degrees of freedom (DOF) rigid body transformation using FSL’s FLIRT (Jenkinson et al., 2002; Jenkinson and Smith, 2001) and averaged (any number of averages are supported). Note that for the HCP, only T1w and T2w images judged to be “good” or “excellent” in overall quality (quality control procedures described in Marcus et al THIS ISSUE) were used for processing. For many subjects, only a single scan was used for the T1w and/or T2w. For greater robustness, the images are internally cropped to a smaller FOV to remove the neck (150mm in z in humans,) using FSL’s automated robustfov tool, and aligned with a 12 DOF (affine) FLIRT registration to the MNI space templates. This alignment allows the MNI space brain mask to be applied prior to the final registration to mask out any residual shifted scalp fat signal. The final transformation to the output average space is done with spline interpolation to minimize blurring.

Next, the average T1w and T2w images are aligned to the MNI space template (with 0.7mm resolution for the HCP data) using a rigid 6 DOF transform, derived from a 12 DOF affine registration. The 6 DOF transform aligns the AC, the AC-PC line and the inter-hemispheric plane, but maintains the original size and shape of the brain. The goal of this alignment step is to get the images in roughly the same orientation as the template, for convenience of visualization. This step also aligns the coordinate space axes to those of the MNI template, removing any rotation between the mm coordinate space and the image matrix (i.e. it removes the oblique components of the NIFTI sform), as oblique sforms are not consistently handled across imaging software platforms. Similar to the procedure for averaging across runs, robustfov is used to make sure this registration is robust and the transform is applied with spline interpolation. After this “acpc alignment” step, a robust initial brain extraction is performed using an initial linear (FLIRT) and non-linear (FNIRT) registration of the image to the MNI template. This warp is then inverted and the template brain mask is brought back into the acpc-alignment space. We found that this method of bringing the atlas brain mask to the individual’s space outperforms other common brain extraction methods like FSL’s BET (Smith, 2002), albeit at the cost of increased processing time. This initial brain mask is used to assist the final registrations to MNI space and to assist FreeSurfer with its brain mask creation process.

The final step in creating the subject’s undistorted native volume space is removing readout distortion (van der Kouwe et al., 2008), which differs between the T1w and T2w images due to their differing readout dwell times (and thus different bandwidths). This distortion is fairly subtle in comparison to gradient nonlinearity or EPI distortion, but is similar to EPI distortion in that it is greatest in regions with high B₀ inhomogeneity due to magnetic susceptibility differences (orbitofrontal cortex and inferior temporal cortex especially). Thus, readout distortion can be corrected by the same means as EPI distortion, i.e. using a fieldmap as the distortion field and scaling it according to the readout dwell time. For fieldmap preprocessing, a standard gradient echo fieldmap, having two magnitude images (at two different TEs) and a phase difference image, is converted into fieldmap (in units of radians per second) using the fsl_prepare_fieldmap script. Then the mean magnitude and fieldmap images are corrected for gradient nonlinearity distortion (just as for the T1w and T2w images). The fieldmap magnitude image is warped according to the readout distortion and registered, separately, to the T1w and T2w images using 6 DOF FLIRT. The fieldmap is then transformed according to these registrations and used to unwarp the T1w and T2w images, removing the differential readout distortion present in them. Although the T1w and T2w images being corrected are of higher resolution than the field map, the B₀ inhomogeneity is smoothly varying and is accurately corrected with this method (see Figure 11). The readout-distortion-corrected T1w image, which now has all spatial distortions removed from it, represents the native volume space for each subject. The undistorted T2w image is cross-modally registered to the T1w image using FLIRT’s boundary-based registration (BBR) cost function (Greve and Fischl, 2009) with 6 DOF, bringing it into the native volume space as well. Boundary-based registration seeks to align intensity gradients across tissue boundaries rather than minimizing a cost function over the entire set of image intensities. This registration strategy uses the same contrast mechanism that one uses to evaluate image alignment by eye. If a fieldmap is not available for the structural images, the pipeline has the option to run without it and only perform FLIRT BBR cross-modal registration of the T2w to the T1w, without removing readout distortion.

Figure 11.

Precise alignment of T2w (A) and T1w (B) images after correction for readout distortion. The coronal slice above is in the same anatomical region as van der Kouwe et al (2008) Figure 4, which illustrates the effects of readout distortion. White matter (green) and pial (blue) surface contours from the FreeSurfer pipeline are in the same locations relative to the anatomy in both T2w and T1w images. The ellipses indicate the locations where readout distortion would be present in uncorrected images.

Once the T1w and T2w images are in the same space, the same intensity inhomogeneity correction approach used for myelin mapping (Glasser and Van Essen, 2011) is used to correct the T1w and T2w images for B₁− bias and some B₁+ bias. As described previously (Rilling et al., 2011), the bias field F is estimated from the square root of the product of the T1w and T2w images after thresholding out non-brain tissues. This method works because the contrast × in the T1w MPRAGE and 1/x in the T2w SPACE within grey and white matter essentially cancel after multiplication, whereas the bias field F does not (Approximation 1).

$$\sqrt{T1w*T2w}\cong \sqrt{(x*F)*(\frac{1}{x}*F)}=F$$ (1)

After dilating the thresholded bias-field estimate (F) to fill the FOV, the bias field is then smoothed with a sigma of 5mm. This method avoids problems encountered in algorithms like FSL’s FAST (Zhang et al., 2001) and MINC’s nu_correct (Sled et al., 1998) that use white matter homogeneity as the basis for bias field correction. Just as grey matter is inhomogeneous in both T1w and T2w images due to myelin content (Fischl et al., 2004; Glasser and Van Essen, 2011), white matter is also inhomogeneous. Because grey and white matter inhomogeneities are not closely correlated, using inhomogeneities in white matter intensity to correct for those in grey matter is likely to introduce errors (Glasser and Van Essen, 2011).

After bias field correction of the T1w and T2w images, the T1w image is registered to MNI space with a FLIRT 12 DOF affine and then a FNIRT nonlinear registration, producing the final nonlinear volume transformation from the subject’s native volume space to MNI space. The outputs of the PreFreeSurfer pipeline are organized into a folder (by default called T1w) that contains the native volume space images and a second folder (by default called MNINonLinear) that contains the MNI space images. In addition to working robustly for a number of human datasets (HCP Pilot datasets, Conte69 datasets (Glasser and Van Essen, 2011; Van Essen et al., 2012b), and HCP Phase II datasets), the PreFreeSurfer pipeline also works robustly on chimpanzee and macaque datasets, if given appropriate chimpanzee and macaque templates and brain dimensions (Glasser et al., 2012b). The pipeline is thus not strictly tied to MNI space, and would work with any other standard space, provided that the appropriate template files are generated at the desired output resolution (e.g. the acquired resolution of the data for the high resolution templates).

HCP Pipelines: FreeSurfer

After the PreFreeSurfer pipeline, the FreeSurfer pipeline runs (Figure 12). FreeSurfer’s recon-all pipeline (Dale et al., 1999; Fischl et al., 2001; Fischl et al., 2008; Fischl et al., 2002; Fischl et al., 1999a; Fischl et al., 1999c; Ségonne et al., 2005) is an extensively tested and robust pipeline optimized for 1mm isotropic data; however, we fine-tuned several aspects of this pipeline specifically for the higher resolution HCP Phase II data. A current limitation with recon-all is that it cannot handle images of higher than 1mm isotropic resolution or structural scans of greater than 256 × 256 × 256 voxels. The 0.7mm isotropic resolution structural scans we acquire exceed these limits, and must therefore be downsampled to 1mm with spline interpolation prior to launching recon-all. Thus, most processing within recon-all (aside from final surface placement, see below) is conducted in a 1mm isotropic downsampled (RAS) space, which we refer to as FreeSurfer space. The input T1w image to the FreeSurfer pipeline is the native volume space output from the PreFreeSurfer pipeline—the distortion- and bias-corrected T1w image. This T1w image is normalized to a standard whole brain mean intensity to ensure FreeSurfer’s conversion to an 8-bit datatype is robust. Early on, we found that the HCP Phase II T1w data was not robustly registered using the linear registration within FreeSurfer that precedes FreeSurfer’s brain extraction. This registration is now aided using the initial brain mask generated in PreFreeSurfer to ensure that FreeSurfer’s internal brain mask is correct.

Figure 12.

The steps of the FreeSurfer Pipeline. FreeSurfer's recon-all forms the basis of most of the pipeline, but it is interrupted at certain steps to improve the robustness of the brain extraction, to fine tune the T2w to T1w registration, and to more accurately place the white and pial surfaces with high-resolution 0.7mm T1w and T2w data. These inputs are fed back into recon-all at each stage.

Aside from these changes, FreeSurfer’s recon-all is allowed to run normally, stopping at the point where final white matter surfaces are generated (almost until the end of the autorecon2 step). Important steps that are performed prior this stage include automated segmentation of the T1w volume into a variety of structures (Fischl et al., 2002), plus tessellation and topology correction of the initial white matter surface (Dale et al., 1999). The resultant white matter surface is generated using a segmentation of the 1mm downsampled T1w image. For final white matter surface placement, we make use of the full 0.7mm resolution of the HCP T1w images. The requisite FreeSurfer volume and surface files are brought into the 0.7mm native volume space, the high resolution T1w volume is intensity normalized, and the white matter surface position is adjusted based on intensity gradients in the 0.7mm T1w image. This adjustment corrects for regions in which the white matter surface was placed too superficially in the grey matter because of partial volume effects at 1mm (Figure 13). Additionally, the T2w to T1w registration is fine-tuned using FreeSurfer’s BBRegister algorithm (Greve and Fischl, 2009), which gave slightly more accurate results than FLIRT’s BBR implementation, most likely because of the higher quality surfaces used in the FreeSurfer BBR registration (the surface used in the PreFreeSurfer pipeline is based on a simple FSL FAST segmentation (Zhang et al., 2001)). The corrected white matter surfaces are then brought back into FreeSurfer space and recon-all processing continues (note that the surfaces are not retessellated so they do not have as many vertices as native 0.7mm images would produce). Spherical inflation of the white matter surface (Fischl et al., 1999a), registration to the fsaverage surface template based on cortical folding patterns (Fischl et al., 1999c), and automated segmentation of sulci and gyri (Desikan et al., 2006) are among the steps done during this recon-all stage.

Just before the pial surfaces are generated, recon-all is stopped again and pial surfaces are obtained with an improved algorithm that uses both the high-resolution T1w and T2w images. First, T1w and T2w images are normalized to a standard mean white matter intensity. As noted above, because residual intensity inhomogeneity after PreFreeSurfer bias-field correction is not necessarily correlated between grey and white matter, we do not use the FreeSurfer white matter specific intensity normalized image for pial surface generation. Instead, we carry out a greymatter specific intensity normalization of the T1w image as follows. Initial pial surfaces are generated from the high-resolution PreFreeSurfer bias-field corrected T1w image using relaxed Gaussian parameters (including intensities 4 sigmas above and below the grey matter mean intensity, versus the standard setting of 3 sigmas). This tends to include large amounts of dura and blood vessels (See Figure 14) and may cause the pial surface not to properly follow sulcal fundi, but it reduces the probability of excluding lightly myelinated grey matter. To remove the dura and blood vessels, the pial surface is eroded using the T2w image. Both dura and blood vessels are very different in intensity from grey matter in the T2w image, though they are close to isointense in the T1w image (Figure 14). This initial pial surface is used together with the white matter surface to define an initial grey matter ribbon of voxels whose centers lie between the two surfaces. The T1w image is smoothed using a ribbon constrained approach and a sigma of 5mm, and then the original image is divided by this smoothed image (after removing the mean). These operations effectively perform spatial highpass filtering within the grey matter ribbon, removing the low frequency effects of differences in myelin content across the image while keeping the high frequency effects of opposing cortical pial surfaces and radial intracortical contrast. This spatially highpass filtered T1w image is used to regenerate the pial surface from scratch with much more restrictive Gaussian parameters (2 sigmas above and below the grey matter mean intensity). The T2w surface is again used to remove any dura and blood vessels, and the result is the final pial surface. The pial surface now appropriately follows the contours in the volume, but does not exclude lightly myelinated grey matter. The benefits of these operations are evident in the group average Conte69 myelin maps presented previously (Glasser and Van Essen, 2011), especially in the very lightly myelinated medial prefrontal and anterior cingulate cortices (Figure 15). After this pial surface generation step, recon-all then continues to completion. The final steps include surface and volume anatomical parcellations and morphometric measurements of structure volumes and surface areas.

Figure 15.

The Conte69 (Glasser and Van Essen 2011) myelin maps before (A) and after (B) correcting a systematic Type B artifact where the pial surface was being placed too deep. In this artifact, lightly myelinated superficial cortical layers were being excluded in many subjects by the default recon-all algorithm (in FreeSurfer versions 4.5, 5.0, and 5.1) leading to artifactually higher myelin map values within the ellipse. Use of grey-matter specific normalization and the T2w surface eliminate the problem.

HCP Pipelines: PostFreeSurfer

The first task of the PostFreeSurfer pipeline (Figure 16) is to take the outputs of FreeSurfer that are in FreeSurfer proprietary formats and convert them to standard NIFTI and GIFTI formats. Additionally, these data are returned to the native volume space from the FreeSurfer 1mm RAS space. The white, pial, spherical, and registered spherical surfaces are all converted to GIFTI surface files; thickness, curvature, and sulc are all converted to GIFTI shape files. The three FreeSurfer cortical parcellations are converted to GIFTI label files and the three full subcortical volume parcellations are converted to NIFTI label files. One of these volume parcellations, the wmparc, is binarized, dilated three times and eroded twice to produce an accurate subject-specific brain mask of grey and white matter, which serves as the final brain mask that is used in any subsequent functional or diffusion processing. The goal of the erosion and dilation process is to fill in most holes (from sulci) inside the mask while maintaining a tight fit to the brain. One more dilation than erosion is used to ensure that when the mask is downsampled (e.g. to the fMRI resolution of 2mm) partial volume brain voxels are not left out of the mask. Additionally, the structural transforms for T1w and T2w images (which were generated in the PreFreeSurfer and FreeSurfer steps) are all combined into a single transform and applied to the averaged T1w and T2w images, bringing them into the native volume and MNI spaces with a single spline interpolation. From this base set of files, a mid-thickness surface is created by averaging the white and pial surfaces. Inflated and very inflated surfaces are created from the mid-thickness. These files are organized into a “spec” (specification) file, which lists relevant files grouped by data type and can be immediately loaded into Connectome Workbench for viewing the standard set of anatomical data. While the surface maps are made available as single hemisphere GIFTI files, the spec files list CIFTI files that combine each scalar or label map across left and right hemispheres, which allows them to be manipulated concurrently as single Connectome Workbench layers.

Figure 16.

An overview of the PostFreeSurfer pipeline. In addition to generating a final brain mask and myelin maps, it converts data to NIFTI and GIFTI formats and creates Connectome Workbench spec files in the surface spaces shown in Figure 8 above.

The above surface coordinate files all exist in the native volume space, and the surface topology is that of the native mesh. Together, these files represent the native volume, native surface mesh space. This space is mainly useful for individual analysis of myelin content maps (discussed below) and quality control of the surface reconstructions, and the data reside in the T1w/Native folder. The next space produced by the PostFreeSurfer pipeline is the native mesh, MNI volume space, in which all anatomical surface files are transformed using the native to MNI nonlinear volume transformation. This space is most useful for accurate volume to surface mapping of individual subject’s fMRI timeseries. As with the previous surface/volume space, a spec file is produced containing the standard set of anatomical data and the data reside in the MNINonLinear/Native folder.

PostFreeSurfer’s next step is to register the individual subject’s native-mesh surfaces to the Conte69 population-average surfaces, which have correspondence between left and right hemispheres (Van Essen et al., 2012b). This registration involves concatenation of the standard FreeSurfer registration (to the fsaverage mesh, and represented by the registered sphere, ‘sphere.reg’) with the group fsaverage to Conte69 atlas registration (Van Essen et al., 2012b). This combined surface registration is applied to all surfaces and surface data files, bringing them into correspondence with the 164k_fs_LR standard surface mesh. This mesh has an average vertex spacing of around 0.9mm on the midthickness, and it has an appropriate sampling resolution for analyzing high-resolution anatomical data such as myelin maps. A spec file listing the standard set of anatomical data resides along with the data in the MNINonLinear root folder.

The average vertex spacing of 0.9mm for the 164k_fs_LR atlas mesh represents an oversampling for fMRI data acquired at 2mm isotropic resolution. Since fMRI and connectivity datasets tend to have a large second temporal or spatial dimension, many surface datasets sampled on the 164k_fs_LR mesh would be prohibitively large. Instead, we use a downsampled surface space, the 32k_fs_LR space, which has an average vertex spacing of around 2mm on the midthickness surface. Although this surface does not track the anatomical contours in the volume as accurately as the native meshes, it represents the shape of the subject’s cerebral cortex fairly accurately for the purposes of visualization and surface-constrained analysis. Because of the smaller number of vertices, this lower resolution surface is more amenable to functional or connectivity analyses once data have been mapped onto the surface using the more accurate native mesh (see fMRISurface). A spec file listing the standard set of anatomical data resides along with the data in the MNINonLinear/fsaverage_LR32k folder. Finally, this 32k_fs_LR mesh is transformed from MNI space back to native volume space to enable tractography visualization using these meshes to occur in native volume space. A spec file containing the standard set of anatomical data resides along with the data in the T1w/fsaverage_LR32k folder.

The capability for bi-directional resampling between the native, high-resolution (164k_fs_LR), and low-resolution (32k_fs_LR) meshes is useful for a variety of purposes (e.g., viewing myelin maps generated at high resolution on the low-resolution map or viewing task-fMRI generated at low resolution on a high-resolution myelin map). We implemented a novel adaptive barycentric surface resampling approach in Connectome Workbench commandline utilities that provides greater accuracy than the methods previously available. The goal of the new method is to minimize blurring during resampling while ensuring that all data from the source mesh contributes to the result on the target mesh, even when the meshes are of significantly different resolution, or when a mesh has highly variable vertex spacing. The method first computes both the barycentric weights of the target sphere vertices in the source sphere triangles (forward weights), and the barycentric weights of the source sphere vertices in the target sphere triangles (backward weights). Next, it rearranges the backwards weights by target sphere vertex, so that they can be used in the same manner as the forward weights. These converted backward weights have an advantage when downsampling because there are as many of them as for the equivalent upsampling, meaning that all data on the source sphere is used in the resampling. However, they are inadequate on their own for upsampling, because some vertices could have no converted backward weights to get data from. (This is the converse of the problem of downsampling by a simple barycentric algorithm, where some vertices of the high-resolution mesh go unused.) The algorithm builds a set of adaptive weights by going through every target sphere vertex and comparing the two lists of weights. If the converted backward weights for the target vertex contain a source vertex that the forward weights do not, then the converted backward weights are used at that vertex, otherwise the forward weights are used. After generating these adaptive weights, the algorithm applies an area correction step using midthickness surfaces on the source and target meshes to derive the vertex areas. (See the surface smoothing algorithm described below for additional methodological details.) In this area correction step, the method multiplies the adaptive weights by the vertex area of their target vertex, normalizes them by the sum at their source vertices, then multiplies by the source vertex areas. Application of these weights to scalar data is straightforward, but for label data, a popularity technique is used: at each vertex, sum the weights by which label value they contribute to, then use the label value that has the highest weight sum.

Additionally, the PostFreeSurfer pipeline produces an accurate volume segmentation of the cortical ribbon based on the signed distance function of the white and pial surfaces. The pipeline also sets up the 2mm standard CIFTI grayordinates space for the fMRISurface pipeline. Finally, the PostFreeSurfer pipeline produces surface myelin maps and ribbon volume myelin maps using the methods described in (Glasser and Van Essen, 2011) together with the improvements described in (Glasser et al., In Press). Briefly, the T1w/T2w ratio in the voxels between the white and pial surfaces is mapped onto the midthickness surface. The principal difference from the original method (Glasser and Van Essen, 2011) is that less artifact correction of the myelin maps is needed given the various enhancements to surface reconstruction and positioning accuracy described in the FreeSurfer pipeline above. For quality control and evaluation of the structural pipelines, the myelin maps serve as a useful end point where errors anywhere along the preprocessing stream are likely to be evident. Subtle errors in registration or surface placement will produce clearly deleterious effects on the myelin maps (e.g. the example shown in Figure 15). Indeed, many of the refinements described above were developed to fix errors in processing that negatively impacted the myelin maps. Thus, the consistently high quality myelin maps in individual subjects produced at the end of the PostFreeSurfer pipeline indicate that the preceding steps of the pipeline generally work well (Figure 17).

Figure 17.

The unsmoothed myelin maps from an individual HCP subject.

Although the bias field correction method used in PreFreeSurfer is adequate for surface reconstruction, the myelin maps generated from the HCP Skyra data have more variable residual bias than those previously generated from Siemens Trios or a GE scanner (Glasser et al., 2012b; Glasser and Van Essen, 2011). This is likely because the positioning of the head is ~ 5cm above isocenter and because of the smaller (customized 56 cm) body transmit coil, making it more difficult to achieve the kind of B₁ transmit homogeneity found in other whole body scanners. We use a model of the expected low spatial frequency distribution of T1w/T2w intensities across the surface to correct the residual bias field present in the image. We take the previous group average myelin maps on the Conte69 mesh (e.g. from (Glasser and Van Essen, 2011)) and bring them onto each individual’s native mesh using the inverse of the surface registration from the PostFreeSurfer pipeline. Taking the difference between the group myelin map and the individual map, and smoothing this difference a substantial amount (e.g. a sigma of ~14mm) creates a field that can be subtracted from the individual myelin map, replacing the low spatial frequency content with that of the group map. The group map is considerably more robust, as it has been shown to be highly stable across scanners and acquisitions (Glasser and Van Essen, 2011) and shows a very similar distribution to that seen with other myelin mapping methods (Cohen-Adad, 2013; Sereno et al., 2012). Such low frequency correction does not affect the discriminability of neighboring cortical areas with differing myelin contents, whose boundaries exist at a much higher spatial frequency than a sigma of ~14mm (FWHM=~33mm). Figure 18 shows the effects of this correction on an HCP subject with a particularly large residual bias field. Both the original and the corrected, MyelinMap_BC, myelin maps will be included in the HCP data releases from Q2 onward.

Figure 18.

The effect of residual bias field correction on a particularly inhomogeneous HCP subject. A) Original Myelin Map. B) Reference group average myelin map (Conte69, from Glasser and Van Essen 2011). C) Estimated residual bias field. D) Bias corrected myelin map. The artifactual anterior-posterior very low spatial frequency gradient of T1w/T2w intensity is removed and the corrected myelin map now resembles the reference. All maps are displayed on the subject’s inflated surface.

HCP Pipelines: fMRIVolume

The fMRI volume pipeline (Figure 19) is in many ways like the PreFreeSurfer pipeline, solving analogous problems. It requires completion of the HCP structural pipelines (PreFreeSurfer, FreeSurfer, and PostFreeSurfer). As in the PreFreeSurfer pipeline, the first step is correction of gradient-nonlinearity-induced distortion, using the same methods as in the PreFreeSurfer pipeline. This step is optional but is strongly recommended for scanners with significant gradient nonlinearity. Next is realignment of the timeseries to correct for subject motion, which is accomplished with a 6 DOF FLIRT registration of each frame to the single-band reference image. An extra registration step is avoided by using the single-band reference image as the registration target, because its greater anatomical contrast is needed in the EPI to T1w registration. Additionally, the motion parameters are output in a twelve column text file with the following format: <x translation in mm> <y translation in mm> <z translation in mm> <x rotation in degrees> <y rotation in degrees> <z rotation in degrees> <derivative of×translation> <derivative of y translation> <derivative of z translation> <derivative of×rotation> <derivative of y rotation> <derivative of z rotation>. A demeaned and linearly detrended motion parameter file is provided as well for nuisance regression.

Figure 19.

The steps of the fMRIVolume Pipeline. All images undergo gradient distortion correction. Motion correction is to the single-band reference (SBRef) image. The SBRef image is used for EPI distortion correction and is registered to the T1w image. The spin echo EPI images have reversed phase encoding directions and are used to calculate deviations from the B0 field caused by susceptibility. All transformations, including the MNI space transformation are concatenated and applied to the original fMRI timeseries in a single spline interpolation step. Finally, this MNI space timeseries is masked and its intensity is normalized to a 4D global mean.

EPI fMRI images contain significant distortion in the phase encoding direction. This distortion can be corrected with either a regular (gradient-echo) fieldmap or a pair of spin echo EPI scans with opposite phase encoding directions. Reversing the phase encoding direction reverses the direction of the B₀ field inhomogeneity effect on the images. The FSL toolbox “topup” (Andersson et al., 2003) can then be used to estimate the distortion field. The distorted spin echo EPI images are aligned with a 6 DOF FLIRT registration to the distorted gradient echo EPI single-band reference image. This registration is concatenated with the topup-derived distortion field, producing a nonlinear transform that can be used to undistort the fMRI images. The single-band reference image is corrected for distortions using spline interpolation and is then registered to the T1w image, first using 6 DOF FLIRT with the BBR cost function and using FreeSurfer’s BBRegister for fine tuning (Greve and Fischl, 2009). The result is an accurate registration between the fMRI data and the structural data, which is important for volume to surface mapping in the fMRISurface pipeline.

The final steps of the fMRIVolume pipeline include concatenating all of the transforms for each registration and distortion correction step into a single nonlinear transformation that can be applied in a single resampling step. Each fMRI timepoint has a unique transformation driven by its 6 DOF motion correction matrix, but preceding that is the gradient nonlinearity distortion correction and following it is the EPI distortion correction, the EPI to T1w registration, and the native volume to MNI nonlinear registration. Every frame from the raw timeseries is transformed, according to its unique nonlinear transformation, directly to MNI space with a single spline interpolation, minimizing interpolation-induced blurring. The bias field calculated from the structural images is removed, reducing the B₁ intensity bias (however, this bias correction is only approximate in its present form because it is estimated from the structural data that are acquired in a different session). The data are then masked by the final brain mask from the PostFreeSurfer pipeline and normalized to a 4D whole brain mean of 10000. The outputs of the fMRIVolume pipeline include a NIFTI volume timeseries in MNINonLinear/Results/fMRIName that is in 2mm MNI space and has undergone the processing described above, a single-band reference image in 2mm MNI space, a Jacobian file representing the intensity modulations of the distortion correction, and both original and demeaned/de-trended motion parameters. Conventional volume-based fMRI analyses can proceed using this dataset.

HCP Pipelines: fMRISurface

The goal of the fMRISurface pipeline (Figure 20) is to take a volume timeseries and map it to the standard CIFTI grayordinates space. The first step is a novel partial volume weighted ribbon-constrained volume to surface mapping algorithm, in which the white and pial surfaces are used to define which fMRI voxels are within the grey matter ribbon. For maximum accuracy, the native mesh surfaces (in MNI volume space) are used in this step, because they have not been resampled and thus follow the tissue contours in the volume most precisely. Some voxels will be partially inside the grey matter ribbon and partially outside of it. The intensity value assigned to each surface vertex is the weighted average of the voxels wholly or partially within the grey matter ribbon, with partial voxels weighted according to their partial volume inside the ribbon. At each vertex, the algorithm finds the triangles that contain the vertex on both the white and pial surfaces (the surfaces must have vertex correspondence) and connects these triangles with the quadrilaterals formed by connecting their edges, constructing a polyhedron. Then, it creates a grid of points inside each voxel, and tests each one to see if it is inside the polyhedron. If this test is ambiguous because the quadrilaterals are twisted, the algorithm counts the point as half inside if it falls on opposite sides of a quadrilateral depending on which diagonal is used to split the quadrilateral into two triangles. The weight used for a voxel is the number of grid points inside, plus half the number of grid points half inside the polyhedron.

Figure 20.

The steps of the fMRISurface pipeline, which maps cortical grey matter voxels onto standard surface vertices and subcortical grey matter voxels onto standard subcortical parcels. These combine to form a standard number of grayordinates that have been aligned according to the nonlinear surface and volume registrations respectively. This allows precise cross subject correspondence between data in the grayordinates standard space.

Additionally, voxels having a locally high coefficient of variation in the timeseries data (greater than 0.5 standard deviations above the mean coefficient of variation of other voxels in a 5mm sigma Gaussian neighborhood) are excluded from volume to surface mapping. The mask of all brain voxels not excluded by the coefficient of variation threshold is provided as a pipeline output (RibbonVolumeToSurfaceMapping/goodvoxels.nii.gz). The practical effect of excluding voxels with high coefficient of variation is to remove voxels that are near the edge of the brain parenchyma or contain large blood vessels. The deviation from the local neighborhood coefficient of variation is used instead of the absolute coefficient of variation because the SNR varies across the brain according to the receive coil sensitivity profile and any EPI signal dropouts. The coefficient of variation of the surface timeseries is markedly reduced (Figure 21) as a result of this process. Additionally, a notable gyral bias in high coefficient of variation is eliminated with this exclusion process.

Figure 21.

The coefficient of variation of the surface timeseries without (A) and with (B) exclusion of locally noisy voxels. While certain parts of cortex are very noisy due to signal loss from susceptibility effects, many other parts of cortex, particularly some gyral crowns, have locally large amounts of noise that is effectively removed with this technique.

The surface timeseries are resampled from the high-resolution native mesh to the registered downsampled 32k_fs_LR mesh, where the sampling resolution matches the original voxel resolution more closely, using the adaptive barycentric surface resampling algorithm described above. The medial wall vertices are masked according to the Conte69 left and right atlas masks, as the medial wall is not a defined grey matter structure. The fMRI timeseries are slightly smoothed (2mm FWHM) on the surface to match the subcortical parcel constrained process described below and to regularize them after the mapping process to match the 2mm average vertex spacing of the 32k_fs_LR mesh.

The smoothing is performed using a novel geodesic Gaussian surface smoothing algorithm. At each vertex, the algorithm finds all vertices within a geodesic distance of three times the Gaussian sigma and assigns them initial weights according to their geodesic distance and the Gaussian function, forming the smoothing kernel for that vertex. Geodesic distance is computed on the midthickness surface using both the direct paths to immediate neighbor vertices, and by finding triangles that share an edge and computing the shortest path across them between the two vertices not on that edge. In practice, this method adds minimal computational time and substantially reduces biases caused by the discrete directions of triangle edges, resulting in more accurate geodesic distance measurements.

Next, the smoothing algorithm applies a correction for differences in the triangle areas associated with each vertex. The area of each vertex is computed as one third of the sum of the surface areas of the triangles that contain the vertex. The smoothing algorithm first multiplies each geodesic Gaussian vertex weight within each smoothing kernel by the area of the kernel’s center vertex (the vertex that the geodesic distance was computed from). Then, in each kernel, it divides each vertex's weight by the sum of the weights at that vertex across all of the smoothing kernels. Finally it multiplies each vertex’s weight within each kernel by its vertex area. If an ROI is being used to exclude certain vertices’ data values from the smoothed output, the algorithm includes their weights in the vertex area correction, but removes them from the kernel when the final Gaussian weighted average is computed. This inclusion in the areal correction prevents vertices on the edge of an ROI from having a stronger influence on their neighbors than if an ROI is not being used.

The two area multiplication steps serve to minimize two different biases that can occur on irregular meshes. The first multiplication reduces a bias related to the effect of triangle area irregularity on the effective shape of the kernel. To understand the first bias, consider two scenarios of a center vertex and a neighboring vertex, each having the same vertex area, but in Scenario A there are many vertices with small areas around and in Scenario B there are only a few large vertices. In Scenario A, the sums of the weights without this multiplication step would be larger than in Scenario B, which after division, would result in a change in amount of influence between the two unchanged vertices in Scenarios A and B. With this first multiplication of the weights by the area of the center vertex, the weight sums used for normalization become approximations of the surface integral of Gaussian kernels, which are largely independent of local mesh density. The result is a very similar weight between the unchanged vertices between the two scenarios after division.

The normalization and second multiplication, by the vertex area of each weight within the kernel, reduces a second bias that overemphasizes values at vertices with smaller area. The effect of the second bias on smoothing with and without these corrections was tested on a subject's native mesh right midthickness surface with a sigma of 3mm. The vertex area metric was used as the test data, in order to exacerbate the bias of vertex area in uncorrected smoothing, as the small values at small vertices are given too much importance, decreasing the value of the surface integral. The surface integral before smoothing was 91,894, after smoothing without area correction it was 73,372, and after smoothing with area correction it was 91,890. The units of this integral are mm⁴, resulting from the unusual procedure of integrating vertex area over the surface, rather than merely summing it. While this is a fairly extreme case designed to cause the uncorrected method to show a very large difference in surface integral, it does demonstrate that the corrected method avoids this bias very effectively.

The geodesic Gaussian surface smoothing algorithm with vertex area correction described above is more analogous to the typical Gaussian smoothing that is applied to volume data that exists on a regular grid. It is more complex because surface distances are geodesic rather than Cartesian and because the vertices containing the data are not regularly spaced, causing biases that the algorithm must correct. Thus, it is different from previous algorithms that needed to smooth and then estimate smoothness iteratively (e.g. (Hagler Jr et al., 2006)). The end results are 2mm FWHM smoothed surface timeseries on a standard mesh in which the vertex numbers correspond to spatially matched locations across subjects.

For volume voxels, a process similar to resampling onto a standard mesh is used to produce a set of corresponding voxels across subjects. As a part of the FreeSurfer pipeline, the subcortical grey matter is segmented into the following left and right structures: nucleus accumbens, amygdala, caudate, cerebellum, ventral diencephalon, hippocampus, pallidum, putamen, and thalamus along with the midline brain stem (Figure 1). The pipeline takes as one of its inputs, provided by PostFreeSurfer, the modal-average subcortical parcellation of these structures across the Conte69 atlas subjects in 2mm MNI space. Although the subjects’ data have been nonlinearly registered to MNI space with FNIRT, allowing individual and atlas parcellations to be directly overlaid, there is not a one-to-one correspondence of the voxels in each subcortical parcel across subjects. To achieve this kind of correspondence, the timecourse in each atlas voxel of a parcel is assigned the value of the Gaussian weighted average (2mm FWHM to match the surface smoothing) of the nearby voxels in the individual subject’s parcel. This parcel-constrained atlas smoothing/resampling process is best illustrated by an example: A particular voxel is a part of the left thalamus in the atlas parcellation. To find its timecourse in an individual, the timecourses of all of the nearby voxels within the left thalamus of the individual parcellation are averaged with a 2mm FWHM Gaussian weighting. This strategy both achieves the desired smoothing of the data while ensuring that voxels that are not a part of the left thalamus (e.g. third ventricle voxels) are not included in the timeseries. The FNIRT nonlinear registration is relied upon to achieve a high degree of correspondence between the individual and atlas parcels. Occasionally, single atlas parcel voxels or single individual parcel voxels may be outside the Gaussian neighborhood (i.e. the distance at which the Gaussian has fallen close to zero). In these cases, the closest individual voxels are used to assign the value to the atlas voxel, effectively a dilation operation. Alternatively, if individual voxels are far from an atlas voxel, they will not contribute much to the resampled timeseries. In practice, these effects occur in only a few voxels in a few subjects, as the subcortical parcellation and nonlinear volume registration are quite accurate in matching individuals to the atlas. Thus, each individual subject’s timeseries is sampled onto a standard set of left and right hemisphere surface vertices and a standard set of subcortical grey matter voxels, forming the standard CIFTI grayordinates space. The surface timeseries from both hemispheres and the subcortical volume timeseries from each subcortical structure are combined in a single data matrix that has the dimensions grayordinates × time (e.g. Figure 2). This CIFTI dense timeseries is the final output of the fMRI minimal preprocessing pipelines and represents the most compact yet still complete representation of the grey matter timeseries data. Additionally, the two hemispheres’ surface timecourses and the parcel-constrained subcortical volume timecourses can be regenerated from the CIFTI dense timeseries using Connectome Workbench’s commandline utilities.

HCP Pipelines: Diffusion Preprocessing

The Diffusion Preprocessing pipeline (Figure 22) starts by intensity normalizing the mean b0 image across the six diffusion series (three with one phase encoding direction and the other three with the opposite phase encoding direction). These phase encoding direction reversed b0 pairs are used to estimate the EPI distortion. This is done using the “topup” tool (Andersson et al., 2003) in FSL5. The estimated distortion field is then fed into a Gaussian Process predictor (Andersson et al., 2012) that uses all the data together and additionally estimates the eddy-current induced field inhomogeneities and head motion for each image volume. All these distortions are corrected in a single resampling step using the “eddy” tool in FSL5 (Andersson et al., 2012), (Sotiropoulos it et al., THIS ISSUE). Eddy also produces a text file that includes the modeled motion parameters and the parameters of the eddy current fields. The first six columns contain the rigid motion parameters <x translation in mm> <y translation in mm> <z translation in mm> <x rotation in radians> <y rotation in radians> <z rotation in radians> and the last 10 contain the eddy current distortion parameters.

Figure 22.

The steps of the diffusion preprocessing pipeline. After b0 intensity normalization, the b0 images of both phase encoding directions are used to calculate the susceptibility-induced B₀ field deviations. Then the full timeseries from both phase encoding directions is used in the “eddy” tool for modeling of eddy current distortions and subject motion. Gradient distortion is corrected and the b0 image is registered to the T1w image using BBR. The diffusion data output from eddy are then resampled into 1.25mm native structural space and masked. Though not shown in Figure 22, the diffusion directions and the gradient deviation estimates are also appropriately rotated and registered into structural space.

The gradient nonlinearity correction warpfield is then calculated for the diffusion data to remove this spatial distortion (Jovicich et al., 2006), and the mean b0 image is distortion corrected. Additionally, the effects of gradient nonlinearity on the diffusion encoding magnitudes and directions are calculated (Bammer et al., 2003), (Sotiropoulos it et al., THIS ISSUE). The partial derivatives of the spatially-dependent magnetic field are used to calculate a gradient field tensor at each voxel, which maps “nominal” to actual gradient magnitudes and directions (Bammer et al., 2003). Using the gradient field tensor, the magnitude and direction of the diffusion-sensitizing gradients can be corrected at each brain voxel. This information is subsequently used for more accurate fiber orientation estimation.

It is worth noting that the gradient nonlinearity correction is done at a much later stage in the diffusion pipeline than in the fMRIVolume pipeline. Ideally the gradient nonlinearity distortion correction would be done simultaneously with the B₀ inhomogeneity distortion correction, as well as with the eddy current distortion correction and rigid-body motion correction, as all of these interact. However, we do not currently have tools that integrate all the distortion correction methods into a single process. Therefore it is necessary to choose the order in which the corrections are applied. Since the combined eddy-current and B₀ inhomogeneity distortions in the phase-encode reversed diffusion acquisitions are much larger than the amount of rigid rotation between images within the sequence, it is a better approximation to run the topup and eddy tools prior to the gradient nonlinearity correction within the diffusion pipeline. This causes the rigid rotation corrections to be approximate but allows the eddy-current and B0 inhomogeneity distortions to be corrected more accurately, as the original phase-encoding image axis direction is maintained and these distortions occur along this direction. (In contrast, the fMRIVolume pipeline deals with fMRI images where the magnitude of the rigid rotations and the B0 inhomogeneity distortion are more similar in areas without much signal loss. For simplicity this pipeline applies the gradient nonlinearity correction earlier, although in practice it is likely to make very little difference.) Future tools will hopefully fully integrate all sources of distortion, making it unnecessary to choose the order.

The final stage of the diffusion pipeline calculates the transform between the native diffusion space and the native structural space. The gradient nonlinearity corrected mean b0 image is registered to the T1w structural image using a 6 DOF boundary-based registration, first with the FLIRT BBR cost function (FSL5) and then with FreeSurfer’s BBRegister for fine-tuning (Greve and Fischl, 2009). The diffusion data output by eddy are then transformed according to both the gradient nonlinearity correction and b0 to T1w registration into 1.25mm native structural volume space in a single step. Finally, the data are masked with the final brain mask from PostFreeSurfer to reduce the file size. The diffusion gradient vectors are rotated according to the rotational component of the b0 to T1w registration so that they will point in the correct directions relative to the structural space diffusion data. The gradient field tensor containing the effects of the gradient nonlinearity on the diffusion weighting and direction in each voxel is also transformed using the b0 to T1w registration using a method that preserves the tensorial nature of this data (Alexander et al., 2001). The transformation of the diffusion image data and auxiliary data to structural space allows direct visualization of the diffusion data together with the structural space surfaces and images in Connectome Workbench. The structural space diffusion data can then be subsequently used for fiber orientation estimation with FSL’s multi-shell spherical deconvolution toolbox (bedpostx) (Jbabdi et al., 2012). However, fiber orientation estimation and tractography are not parts of the minimal preprocessing diffusion pipeline, and thus they are described elsewhere (Sotiropoulos et al., THIS ISSUE).

Future Directions for the HCP Minimal Preprocessing Pipelines

There are several specific areas in which the minimal preprocessing pipelines might be improved. One would be to use the acquired B₁− and B₁+ fields to correct the T1w and T2w images for receive and transmit inhomogeneities prior to myelin map generation, rather than relying on the myelin map normalization algorithm described in PostFreeSurfer. Another area of improvement is subcortical segmentation. Although FreeSurfer’s subcortical segmentation is highly reliable, in some areas it consistently makes mistakes (e.g. in the putamen near the insula and claustrum) and further improvements are possible. First, segmentation could be done on the full 0.7mm resolution data instead of being limited to 1mm isotropic, allowing more accurate definition of existing structures. Second, additional modalities could be used in subcortical segmentation in addition to the T1w image. For example, the T2w image shows iron-related contrast in a number of subcortical nuclei, such as the globus pallidus and red nucleus. The high-resolution diffusion data acquired by the HCP offers another potential source of information for subcortical segmentation. Use of higher resolution images with more contrast may allow additional subcortical structures to be defined, such as distinguishing the claustrum from the putamen and the extreme and external capsules.

Better cross-subject surface-based registration, using more than cortical folding patterns is another improvement to the minimal preprocessing pipelines that is likely to occur in the future. The goal would be to improve cross-subject alignment of cortical areas in the standard grayordinates space, as some cortical areas are not particularly well aligned based on cortical folding patterns alone, particularly in regions of high cross-subject folding variability (Glasser et al., 2012a; Van Essen et al., 2012b). Additionally, FreeSurfer’s current folding-based registration algorithm introduces more areal distortion of the cortical surfaces than is desired relative to its ability to align cortical areas in these variable regions (Van Essen et al., 2012b). Such excess distortion may make subsequent registrations based on other modalities more difficult than if these registrations were performed after a more gentle registration that focuses on aligning only the more consistent cortical folds. That is because distortions in regions of high folding variability do not necessarily bring the cortical areas into better register. Undoing previous distortions that occurred in the wrong direction will require more a permissive surface registration regularization than if the distortions had not occurred in the first place. Using myelin maps as a part of surface registration would be a first step towards a multi-modal surface registration approach that better aligns cortical areas (Robinson et al., 2013a; Robinson et al., 2013b), as myelin maps are more closely related to cortical areas than are cortical folding patterns. Additional information from resting state functional connectivity patterns, task fMRI activation patterns (Sabuncu et al., 2010), or diffusion tractography (Petrović et al., 2009) should be helpful in achieving better cross-subject alignment of cortical areas, and methods for incorporating this information are actively being pursued. The most significant future changes to the minimal preprocessing pipelines will likely include incorporation of multi-modal surface registration to achieve better alignment of cortical areas across subjects within the grayordinates space.

Finally, it will be necessary to incorporate additional 7T data from 200 of the HCP subjects that were scanned at 3T. Rather than separately running the minimal preprocessing pipelines on 7T data, it will make more sense to build a new pipeline that makes use of the existing 3T data. For example, after all distortion corrections are applied, it should be possible to align the 7T structural, functional, and diffusion data with the 3T structural data using a simple rigid body transform. 7T data could then be analyzed using the same surfaces and standard grayordinate space as the 3T data. If higher resolution T1w images are acquired at 7T, these could be used to further refine the cortical surface, allowing improved myelin mapping with the 3T data or measuring myelin content with another technique directly at 7T such as MP2RAGE or similar techniques (Marques et al., 2010; Van de Moortele et al., 2009). Although the above changes may improve the pipelines in the future, the existing pipelines will be released publicly (http://www.humanconnectome.org/) as a tarball of scripts, documentation, and example data to allow others to process their data with whatever components of the HCP minimal preprocessing pipelines they wish to use.

Highlights.

• *Multi-modal preprocessing pipelines for the Human Connectome Project
• *Description of CIFTI file format and Grayordinates coordinate system
• *Combined surface and volume neuroimaging analysis