Abstract
The human brain grows the most dramatically during the perinatal and early post-natal periods, during which pre-term birth or perinatal injury that may alter brain structure and lead to developmental anomalies. Thus, characterizing cortical thickness of developing brains remains an important goal. However, this task is often complicated by inaccurate cortical surface extraction due to small-size brains. Here, we propose a novel complex framework for the reconstruction of neonatal WM and pial surfaces, accounting for large partial volumes due to small-size brains. The proposed approach relies only on T1-weighted images unlike previous T2-weighted image-based approaches while only T1-weighted images are sometimes available under the different clinical/research setting. Deep neural networks are first introduced to the neonatal magnetic resonance imaging (MRI) pipeline to address the mis-segmentation of brain tissues. Furthermore, this pipeline enhances cortical boundary delineation using combined models of the cerebrospinal fluid (CSF)/GM boundary detection with edge gradient information and a new skeletonization of sulcal folding where no CSF voxels are seen due to the limited resolution. We also proposed a systematic evaluation using three independent datasets comprising 736 pre-term and 97 term neonates. Qualitative assessment for reconstructed cortical surfaces shows that 86.9% are rated as accurate across the three site datasets. In addition, our landmark-based evaluation shows that the mean displacement of the cortical surfaces from the true boundaries was less than a voxel size (0.532 ± 0.035 mm). Evaluating the proposed pipeline (namely NEOCIVET 2.0) shows the robustness and reproducibility across different sites and different age-groups. The mean cortical thickness measured positively correlated with post-menstrual age (PMA) at scan (p < 0.0001); Cingulate cortical areas grew the most rapidly whereas the inferior temporal cortex grew the least rapidly. The range of the cortical thickness measured was biologically congruent (1.3 mm at 28 weeks of PMA to 1.8 mm at term equivalent). Cortical thickness measured on T1 MRI using NEOCIVET 2.0 was compared with that on T2 using the established dHCP pipeline. It was difficult to conclude that either T1 or T2 imaging is more ideal to construct cortical surfaces. NEOCIVET 2.0 has been open to the public through CBRAIN (https://mcin-cnim.ca/technology/cbrain/), a web-based platform for processing brain imaging data.
Keywords: neonatal brain MRI, T1, pial surface, cortical thickness, deformation
Introduction
The human brain grows the most dramatically during the perinatal and post-natal periods (Knickmeyer et al., 2008), where the majority of cortical growth and folding partition the brain for various sophisticated functions (Alexander-Bloch et al., 2018). This critical time window of neurodevelopment is susceptible to developmental anomalies, such as pre-term birth or perinatal injuries. The effect of such impairments on early development, although documented through limited histological or qualitative clinical imaging studies (Battin et al., 1998; Kapellou et al., 2006; Ball et al., 2012, 2013, 2015; Guo et al., 2016; Kim et al., 2016, 2020; Lefevre et al., 2016), still remain far from being completely understood. Given vast gaps in knowledge of early brain development, it remains a paramount goal to accurately measure cortical morphological changes in neonates in order to elucidate post-natal brain development and neurodevelopmental outcomes in pre-term survivors (Habas et al., 2012; Wright et al., 2014).
Modern neuroimaging approaches have provided the ability to more accurately and non-invasively characterize and study brain morphology, such as through in vivo T1-weighted (T1w) and T2-weighted (T2w) magnetic resonance imaging (MRI). However, unlike MRI methods implemented to study the adult brain, MRI methods and processing frameworks for the neonatal brain are still under investigation in order to circumvent substantial challenges, such as opposite/reduced tissue contrast between WM and GM, large within-tissue intensity variations, and regionally heterogeneous image appearances that dynamically change in accordance with neural development. In particular, these challenges lead to significant difficulties in quantifying cortical morphology measurements, including cortical thickness, surface curvature, and sulcal depth, which demand accurate surface reconstruction methods. Conventional pipelines such as CIVET (MacDonald et al., 2000; Kim et al., 2005), FreeSurfer (Fischl, 2012), and Caret (Van Essen et al., 2001) are not suitable for neonatal brains, as they rely on established tissue contrast normally exhibited in adult brain MRIs. As such, several modifications (Hill et al., 2010; Leroy et al., 2011; Wright et al., 2015; Kim et al., 2016) have been incorporated into such pipelines to optimize the surface reconstruction of neonatal brains. However, these methods provide only the successful reconstruction of the inner cortical surface, while remaining incapable of robustly measuring cortical thickness, which is a crucial morphological characteristic of brain development (Raznahan et al., 2011a).
One of the main challenges in reconstructing the outer cortical border is the significant partial volume effects (Osechinskiy and Kruggel, 2012; Li et al., 2019) located in areas of the brain scan where only scattered cerebrospinal fluid (CSF) voxels are found within the deep and narrow sulcal fundi. To overcome this issue, previous studies in adult brain MRI mainly included one of the two following strategies: (1) generate a skeleton representing the medial surface of each sulcus (Mangin et al., 1995; Han et al., 2001, 2004; Kim et al., 2005; Li et al., 2012), or (2) expand the initial cortical surface mesh (placed in the GM/WM border) onto opposing gyral folds toward each other until either GM and CSF reach a boundary or they are close enough to each other (Dale et al., 1999). Such adult-specific methods are based on the assumption that the partial volumes of CSF can be accurately modeled, and the GM/CSF boundary can be clearly identified. However, these assumptions cannot be applied to neonatal brain scans, due to their limited image resolution, and smaller head sizes which exacerbate partial volume effects in sulcal CSF (see Figure 1).
To more accurately generate the outer cortical surface, Li et al. (2014) applied a deformable surface model to longitudinal scans where CSF could be identified on the images acquired from older infants (e.g., 1 year old and older). Such MRIs would be acquired longitudinally in a small number of neonates under a planned research setting. The developing Human Connectome Project (dHCP) pipeline (Schuh et al., 2017; Makropoulos et al., 2018a) proposed the expansion of the inner cortical surface to the outer rim based on a non-intersection constraint and a local edge detection. The partial volume effect was corrected using a draw-EM tissue segmentation approach with specially designed sulci detection and enhancement approach (Makropoulos et al., 2016). However, this surface reconstruction could be performed only when T2w images were available. A more recent pipeline (Zöllei et al., 2020) performed surface reconstruction for 0–2 years old infants using T1w images only. This method used a multiatlas-based segmentation combined with Bayesian inference to solve the fundamental issue of unobservable CSF volumes within a sulcus (Figure 1). However, when applied to T1w images in the dHCP dataset, the quality of the generated surfaces using this method was much worse compared to those generated using the pipeline conducted on T2w images (Makropoulos et al., 2018b).
Given that volumetric T2-weighted images are not frequently used in infants due to poor contrast (Zöllei et al., 2020), a robust neonatal surface generation pipeline solely dependent on T1w images is needed. In this study, we propose a complete pipeline that robustly reconstructs inner cortical and pial surfaces of neonatal brains, while computing sulcal depth and cortical thickness measurements. This approach requires only one type of image modality (T1w MRI) and includes an additional number of novel features, compared to our previous pipeline (Kim et al., 2016) and other approaches, that address the aforementioned challenges. First, our new pipeline integrates a deep learning-based tissue segmentation method using non-local 3D-Unet into the cortical morphometry framework to address for regionally varying tissue contrast in neonates. Our approach resulted in higher accuracy compared to other neonatal brain tissue segmentation methods (Weisenfeld and Warfield, 2009; Gousias et al., 2013; Wang et al., 2013, 2015; Makropoulos et al., 2014; Beare et al., 2016). Second, CSF volumes are skeletonized by combining CSF-PVs with sulcal GM, which approximately place outer cortical surfaces onto GM/CSF boundaries. Third, our pipeline incorporates the generated CSF skeleton with CSF/GM edges, which are detected by a radial gradient-based approach, in order to more accurately identify candidate locations of the pial surface. Unlike the dHCP pipeline (Schuh et al., 2017; Makropoulos et al., 2018a), where the gradient-based refinement for pial surface was performed by directly deforming the WM surface, the gradient-based refinement for pial surface in our design is performed within a short distance range after the GM/CSF boundary is first approximated and placed at the skeleton. Finally, we systematically perform qualitative and quantitative evaluations using multi-site neonatal datasets to investigate whether the proposed pipeline could overcome the limitations in the modules that might be suboptimal for regional analyses of poor-contrast neonatal cortex. We compute cortical thickness values of over 80,000 vertices sampled on the constructed surface. After registering individual surfaces to their age-matched template, the results suggest that the proposed cortical surface reconstruction can robustly provide measurements that characterize realistic neonatal brain morphological changes across the multi-site datasets.
Materials and Equipment
Subjects
Our first dataset comprised of 223 pre-term newborns [mean post-menstrual age (PMA) at birth = 28.1 ± 2.0 weeks; range 24–33 weeks], admitted to UCSF Benioff Children’s Hospital San Francisco between June 2006 and January 2016. Exclusion criteria included (i) clinical evidence of a congenital malformation or syndrome, (ii) congenital infection, and (iii) newborns too clinically unstable for transport to and from the MRI scanner. Parental consent was obtained for all cases following a protocol approved by the institutional Committee on Human Research. All patients were scanned post-natally as soon as they became clinically stable (PMA at scan: 31.7 ± 1.8 weeks; range 26–40 weeks), and 132 patients were re-scanned before discharge at late pre-term age (PMA at scan: 35.9 ± 2.0 weeks; range 32–44 weeks). Due to extremely severe motion artifact, 36 baseline and 25 follow-up scans were excluded. The final database included 187 baseline (PMA = 31.8 ± 1.8 weeks) and 107 follow-up scans (36.0 ± 2.0 weeks).
We also utilized two independent datasets collected from the University of Northern Carolina (UNC) and T1w images from the dHCP to cross-validate our pipeline. The UNC dataset includes 57 term born neonates (PMA = 39.2 ± 1.3 weeks) who were scanned at PMA of 40–45 weeks. The 492 T1w images (PMA = 39.2 ± 3.8 weeks) in the dHCP dataset, which were scanned at PMA of 29–45 weeks, were also included.
Image Acquisition
At UCSF, customized MRI-compatible incubators with specialized head coils were used to provide a quiet, well-monitored environment for neonates during the MRI scan, minimizing patient movement and improving the signal-to-noise ratio. Newborns enrolled between June 2006 and July 2011 (n = 95) were scanned on a 1.5-Tesla General Electric Signa HDxt system (GE Medical systems, Waukesha, WI, United States) using a specialized high-sensitivity neonatal head coil built within a custom-built MRI compatible incubator. T1-weighted images were acquired using sagittal 3-dimensional inversion recovery spoiled gradient echo (3D SPGR) (repetition time [TR] = 35 ms; echo time [TE] = 6.00 ms; inversion time of 0.00 ms; field of view [FOV] = 256 × 192 mm2; number of excitations [NEX] = 1.00; and flip angle [FA] = 35°), yielding images with 1 × 1 × 1 mm3 spatial resolution. Newborns enrolled between July 2011 and March 2015 (n = 128) were scanned on a 3-Tesla General Electric Discovery MR750 system in a different MR compatible incubator and using a specially designed (for 3T imaging) neonatal head coil. T1-weighted images were also acquired using sagittal 3D IR-SPGR (TR = minimum; TE = minimum; inversion time of 450 ms; FOV = 180 × 180 mm2; NEX = 1.00; FA = 15°), and were reformatted in the axial and coronal planes, yielding images with 0.7 × 0.7 × 1 mm3 spatial resolution.
For the UNC dataset, the imaging protocol parameters match those used by Kim et al. (2016). Neonates were scanned on a 3T Siemens Trio system. T1-weighted images were acquired using 3-dimensional magnetization-prepared radio-frequency pulses and rapid gradient-echo (MP-RAGE; TR = 2400 ms; TE = 3.16 ms; inversion time [TI] = 1200 ms; and FA = 8°), yielding images with 1 × 1 × 1 mm3 spatial resolution.
Methods
Pipeline Overview
Our new framework has extended the original NEOCIVET pipeline (henceforth NEOCIVET 1.0) (Kim et al., 2016) to include the reconstruction of neonatal pial surfaces (GM-CSF interface) as well as the improvement of existing features. The new pipeline begins with general data pre-processing, including denoising and intensity non-uniformity correction. Then, the brain is extracted using a deep 3D-Unet (Hwang et al., 2019) and registered to the MNI-NIH neonatal brain template1, with spatial resolution of 0.6 × 0.6 × 0.6 mm3 (isotropic). Different types of brain tissue (GM, WM, and CSF) are thereafter segmented by an advanced deep non-local 3D-Unet (Wang Z. et al., 2020). Individual templates (MRI + manually segmented tissue labels) utilized for the deep learning approach are then selected evenly across all PMAs. Next, the corpus callosum is segmented on the midline-plane and used to divide the WM into hemispheres. Instead of the original parametric deformable model, a marching-cube based framework is newly adopted to generate a triangulated mesh WM surface attached to the boundary between the GM and WM. This process results in the number of surface-meshes adapted to the individually varying brain size and folding area, addressing the issue of the complex optimization of the parameters needed in the previous parametric model (Kim et al., 2016). After resampling to a fixed number of 81,920 surface meshes (triangles) using the icosahedron spherical fitting, this surface is further fitted to the sharp edge of the GM-WM interface based on the image intensity gradient, which preserves the spherical topology of the cortical mantle. A CSF skeleton is then generated from the union of GM and CSFs. Pial surface is constructed by expanding the WM surface toward the skeleton as an intermediate pial surface. The intermediate pial surface further undergoes a fine deformation to identify actual edges of sulcal CSF volumes using an intensity gradient feature model. Finally, the cortical thickness is estimated based on the shortest distance between the white matter and pial surface, with a smooth kernel size of 10 mm (Vasung et al., 2016). Figure 2 presents the workflow of our proposed pipeline.
Brain Extraction
Brain extraction is essential for T1w based neonatal surface reconstruction. Due to the similar intensity between WM and CSF, a liberal brain mask may include more CSF, which would increase the risk of mis-segmenting WM and CSF. In this study, a deep convolutional neural network-based approach, i.e., 3D-Unet (Çiçek et al., 2016), is used for brain extraction. 3D-Unet has been demonstrated to perform better than most of the existing methods in brain-extraction (Hwang et al., 2019). In our study, a 3D-Unet consists of three contracting encoder layers to analyze the whole image and three successive expanding decoder layers to produce a full-resolution segmentation (Ronneberger et al., 2015). Each layer contains two 3 × 3 × 3 convolutions each followed by a rectified linear unit (ReLu), and then a 2 × 2 × 2 max pooling with strides of two in each dimension. The 3D-Unet was trained on 36 × 36 × 36 voxels image patches fragmented from template MRI images.
We include 24 templates (MRI + manually segmented brain masks) from all age periods (24–45 weeks of PMA) to capture cortical morphology that dramatically changes during perinatal development. The templates used for the training of the segmentation algorithm were selected from all three datasets, and evenly selected across various PMA groups (see details of template selection in Figure 3). Manual segmentation of brain tissue is also included in the 24 templates, which are utilized as training templates for brain tissue segmentations, as discussed in the following section (see illustration of all 24 manual segmentations overlaid on MRI images in Supplementary Figure 2).
Tissue Segmentation
Brain tissues are segmented into WM, GM, and CSF. Our previous work (Kim et al., 2016; Liu et al., 2019), despite successfully solving the issue of large morphological alterations, did not fully address the mis-segmentation between WM and CSF due to the confounding intensities between WM and CSF in neonatal MRI (Li et al., 2019) for neonatal brain (Figure 4). To solve this, a non-local 3D-Unet (Wang Z. et al., 2020) is applied for tissue segmentation in our pipeline. Conventional 3D-Unet (like the one used in brain extraction) applies small kernels in convolution operators to scan inputs and extracts local information. As complex biomedical image segmentation usually benefits from a wide range of contextual information, one has to develop new operators that perform the global information aggregation to recover more accurate details. Non-local 3D-Unet was designed by adding a self-attention block, which computes outputs at one position by attending to every position of the input, to both the encoder layer and the decoder layer to aggregate/capture the contextual information. Non-local 3D-Unet has been demonstrated to outperform most state-of-the-art methods in neonatal tissue segmentation (Wang Z. et al., 2020).
In our study, the non-local 3D-Unet consists of three encoder layers and three decoder layers. Each layer contains two 3 × 3 × 3 convolutions followed by a 2 × 2 × 2 max pooling with strides of two in each dimension, and an aggregated down/up-sampling self-attention block. The non-local 3D-Unet was trained on 36 × 36 × 36 voxel image patches fragmented from template MRI images. Figure 4 illustrates that compared to the multi-atlas joint-fusion tissue segmentation that was applied in our previous work, mis-segmentation between WM and CSF is largely addressed by the non-local 3D-Unet.
WM Surface Reconstruction
The WM/GM interface is extracted using the surface extraction tools employed in CIVET-2.1.0, with extensions to neonate brains. In the first stage, a new marching-cube approach is utilized, where the number of surface-meshes is adapted to fit the individualized brain size and morphometry. This could facilitate the parameter selection which is needed in the parametric deformable models (MacDonald et al., 2000; Kim et al., 2016), and avoid underfitting due to suboptimal parameters. Spherical topology of the surface is guaranteed by collapsing, in voxel space, an outer ellipsoid enclosing the WM mask onto this mask, while preserving the spherical topology of the ellipsoid. Automated mesh adaptation tools (in-house software within CIVET-2.1.0) are then used to coarsen the initial marching-cubes surface to a manageable size before inflating it to a sphere. Specifically, vertices are removed by merging two vertices of one edge, and connections between new vertices and other vertices are setup to form new triangles. This operation is only performed when the two vertices are topologically valid. Then, the brain mesh is geometrically smoothed iteratively until a convex blob is obtained by an inflation algorithm, and the coordinates are normalized to the unit sphere. The inflated sphere serves as a mapping between the icosahedral sphere to the inflated sphere for resampling the brain surface like an icosahedral sphere at 81,920 polygons, as required by our in-house surface-based registration tools (Robbins et al., 2004). The resampled surface is then fitted to the T1w image. During the mesh deformation, a self-proximity term is also added to prevent the polygon intersection (Kim et al., 2005). In the second stage, the resultant boundaries are deformed again to refine the fitting using an image gradient term (see details in Gradient-based surface refinement). Examples of such improvements with the gradient feature-based refinement are shown in Figure 5. This surface is registered to an age specific template using a spherical registration method and resampled into 81,920 meshes by the icosahedron resampling (Kim et al., 2016), more details can be found in section “Surface Registration to Age-Specific Templates.”
CSF Skeletonization
The purpose of the skeletonization is to identify the hypothetical location of CSF within the sulcal bank and fundus to permit deep surface penetration of the pial surface in sulci. Most previous methods have attempted to detect the combination of CSF voxels and CSF partial volumes buried in the sulcus, and skeletonize them (Han et al., 2004; Kim et al., 2005; Lerch and Evans, 2005; Im et al., 2006; Lee et al., 2006; Xue et al., 2007). However, given that CSF voxels or CSF partial volumes can hardly be seen within a number of sulci on neonatal MRI due to small brain size, indistinguishable sulcal banks, and limited image resolution, the aforementioned approaches failed to reliably capture sulcal skeletons. Therefore, we choose to identify the medial surface of the union of GM and CSF as an alternative to CSF skeletons. The skeleton may not necessarily locate in the middle part of the GM-CSF union, therefore we use a second model to refine the initial pial surface, which is deformed to the skeleton, based on image intensities alone. If there are intensity gradient in unobserved CSF in the deep basin area, the skeleton is not a necessity to detect the CSF, since the difference of gradient and surface refinement would ultimately be used to detect the CSF. However, the skeleton has a strong effect to detect CSF in non-gradient or invisible CSF of the sulcal basin area, where the GM is hypothesized to have the same thickness in two sulcal banks.
The main steps involved in skeletonization are (1) Finding external and internal boundaries. We define the external boundary of the GM-CSF union as the hull of the brain mask and the internal border as the GM/WM boundary that is obtained from the white surface. Voxels located at the external boundary are first selected as part of the skeleton that represents the gyral interface. We then skeletonize the remaining part of the GM-CSF union by thinning the aggregation of voxels in the GM-CSF union, leading to the generation of medial surfaces of sulci. (2) Initialization of catchment basins for watershed algorithms. The latter skeletonization is conducted using a homotopic erosion that preserves the initial topology, with a watershed algorithm embedded in the erosion process (Riviere et al., 2002). The first step in the watershed algorithm is to identify initial seed voxels for catchment basins to represent potential sulcal fundi. These seed voxels are determined by thresholding the Gaussian curvature for each voxel on the boundary of WM and GM-CSF union. (3) Recursive erosion of the GM-CSF union. Then, water rises from the catchment basins, which is driven by the mean curvature. Voxels on the edge of the GM-CSF union are removed if their mean curvatures are lower than a given threshold. This erosion is processed iteratively. A skeleton is generated when, by recursive erosions, the water reached ridges which correspond to the medial localization of cortical folds. (4) Pruning of the remaining voxels. Due to the excessive detection of initial seeds in the watershed algorithm, redundant skeleton branches may be created (Shen et al., 2011). Therefore, an additional step of pruning is applied to remove small non-significant secondary branches and construct a more reasonable skeleton of sulcation. Our pruning process is performed by classifying the remaining voxels as deletable points and undeletable points depending on their topological characterizations, i.e., how the voxels connect to their neighbors within the remaining voxels. Specifically, each remaining voxel is labeled as one of nine categories: interior voxel, isolated voxel, border voxel, curve voxel, curves voxel, surface voxel, surface-curve voxel, surfaces voxel, and surfaces-curve voxel. The definition of the nine classes can be found in Malandain et al. (1993) and Mangin et al. (1995). The surface voxel, surface-curve voxel, surfaces voxel, and surfaces-curve voxel are undeletable voxels; the isolated voxels are deletable voxels. For the all other categories, users are able to select which categories are deletable voxels (see “Parameters Selection” section). The deletable voxels are iteratively removed until all the remaining voxels are undeletable voxels. Figure 6 shows an illustration of skeletonization of the union of GM and CSF voxels.
Pial Surface Reconstruction
The generation of pial surfaces consists of two stages: First, the initial pial surface is obtained by iteratively expanding the WM surface toward the skeleton of GM/CSF union. The expansion path to this boundary is defined using Laplacian fields generated from the WM surface to the skeleton (Kim et al., 2005). The path defined by the Laplacian fields provides the one-to-one projection/mapping of all the vertices on the WM surface onto the deformed surface. Defining the point correspondence between the two surfaces in this manner helps avoid the topological errors (Kim et al., 2005). On the other hand, the skeleton generated from the GM/CSF union is not a precise representation but rather a hypothetical delineation of the true location. In regions where CSF voxels or CSF partial volumes are observed within the sulcal bank, pial surface reconstructions are based on not only the skeleton of GM/CSF union, but also the CSF segmentations. In the second stage, the pial surface is thus iteratively fine-tuned to fit the adjacent, more realistic GM-CSF boundary using gradient information derived from intensities of voxels proximal to the surface for each current iteration. To this end, we generate a gradient profile along the direction normal to the surface and converged the surface fitting to a position of the corrected local maximum gradient (see more details in section “Gradient-Based Surface Refinement”).
Gradient-Based Surface Refinement
Potential inaccuracy of the pial surface reconstruction might be generated when based solely on the skeleton of the GM/CSF union. These occurrences make sulcal CSF skeletonization more difficult, leading to a defective surface deformation toward the true cortical boundary. Therefore, we adapt, from CIVET-2.1.0, a supplementary model to improve the surface deformation, which is based solely on the image intensity gradients and is applied to both WM and pial surface fittings.
For a more accurate surface deformation, we strategically compute a directional gradient by analyzing the one-dimensional (1D) intensity profile along the radial lines perpendicular to the surface, at each vertex. This directional gradient is sampled equally in spatial points by a search step (e.g., 0.5 mm) along the surface-normal rays within a certain distance range, defined as search distance, both inward and outward (e.g., −5 mm to +5 mm). The intensity values are blurred along each radial line to remove small local extrema. Raying from the center located on a given surface vertex, we perform a search process to identify a suitable edge (maximum gradient) among all the sample points within the defined search distance. For each vertex, the intensity at the sample point where the maximum gradient among all the sample points occurs is recorded. In this manner, we could map such intensities across all vertices. By doing so, the local target intensity, which is used to compute the maximum intensity gradient, is allowed to vary over the entire surface. Then, surface-based diffusion blurring is applied to this intensity map by incorporating information from neighboring vertices. This blurring procedure is critical because the extracted gradient information along the surface-normal profile is occasionally erroneous due to motions, image artifacts, bias field, blood vessels, flawed surface normal, and a relatively small range of the search distance. This blurring process attenuates erroneous gradients or outliers, thereby leading to more robust convergence. Finally, the surface mesh is deformed using (1) forces of attraction to locations displaying maximum blurred gradient values; and (2) constraint of the vertex-wise spatial smoothness. Our proposed gradient-based deformation follows a multi-resolution approach to efficiently fit the initial surface to the location displaying the maximum gradient, while avoiding being trapped in local maxima. An illustration of surface refinement can be found in Figure 7. Parameters related to the multi-resolution approach are described in “Parameters Selection” in Supplementary Materials.
Surface Registration to Age-Specific Templates
We register individual surfaces to the template using SURFTRACC (Robbins et al., 2004). This tool first transposes individual sulcal depth maps into icosahedral spheres. It then iteratively searches for local optimal vertex correspondence between an individual and a template sphere based on a matching of a given feature [in this case, depth potential function (Boucher et al., 2009)]. Mapping the deformed meshes back to the original cortical surface coordinates allows a registration of the individual to the template surface. To address for cortical folding changes dramatically during perinatal development, we previously constructed age-specific surface templates representing four age ranges: 26–30, 31–33, 34–36, and 37–40 weeks PMA. Details are found in our previous study (Kim et al., 2016). For inter-subject analyses and group comparisons, any given subject is registered to its corresponding age template and then is ultimately transformed to the oldest template space by concatenating the sequence of transformations between age-specific templates. It is noted that neonatal brains with pathology or an injury like intraventricular hemorrhage, white matter injury or ventriculomegaly may under-develop or deviate from the normal development (Kim et al., 2020). Use of their PMA in finding their age-matched template may thus lead to a suboptimal registration result. Also, there may be cases where their PMA are not provided. To overcome this, we first register the given individual surface separately to each of all the four age-specific templates. We then identify the best-matched template based upon the similarity (using Pearson’s correlation) of each template’s sulcal depth pattern with the registered individual surface. To evaluate this, we compare the age of the template chosen by the proposed method with that matched with the subject’s PMA. We perform this comparison in pre-term neonates with injuries and those without.
Results
Surface Quality Assessment
The qualities of WM and pial surfaces were assessed by visually scoring the fitting accuracy of the reconstructed surfaces. The scoring was done by assigning an overall score from 1 to 3 to each of the individual pial surfaces. Score 1 indicated poor quality, in which the contour substantially deviated from the cortical boundary, and the deviation was distributed globally. This occurs mostly due to severe motion artifacts. Score 2 indicated cases where the contour was close to the cortical boundary, but contained several local flaws. This occurred in cases with minor motion artifacts, and failed segmentation in the full depth of narrow sulci fundi. Score 3 indicated a contour with accurate tracing of the cortical boundary, or with only focal and slight deviation, which was difficult to find without careful review (Figure 8). Note that we considered surfaces with minor issues as good quality (score 3), which may have resulted in more “good” cases in our evaluation than previous studies (Makropoulos et al., 2018b). Two independent raters performed the quality assessment (QA) for all three sets (JS and NT) respectively.
To quantitatively perform the surface QA, we further conduct a comprehensive landmark-based analysis of accuracy for outer cortical surfaces. An independent rater (JK) visually picked a number of landmark points located on the GM/CSF interface. These landmarks were considered as samples representing the “ground truth” surface boundary. The Euclidian distance from each landmark to either the pial surface or the intermediate pial surface (generated at stage 1 where no gradient information was used) was computed. Due to shallow cortical folding, fewer landmarks are selected in younger pre-term neonates such that we select 30 landmarks in the group of 26–31 weeks PMA, 50 in 32–34 weeks, 100 in 35–36 weeks, and 100 in 37–45 weeks. The landmarks were selected randomly, but evenly distributed across cortical lobes (i.e., include but not limit to frontal, parietal, temporal, and occipital lobes) and approximately evenly distributed across all the sulci (i.e., mainly distributed in several main sulci like central sulcus, lateral sulcus, cingulate sulcus, and parietal occipital sulcus) that showed a sufficient depth and an eminent shape relative to secondary branches. Due to a large amount of time required for this manual procedure, 20 neonates (randomly selected with PMA evenly distributed between the aforementioned 4 age groups) were selected from the three datasets.
Qualitative Evaluation Results
Figure 9 presents the results of the surface QA for three datasets. WM surfaces produced by the proposed method were rated as good/accurate (score 3) in 83.6% of scans (averaged by two raters). Pial surfaces were rated as accurate in 82.1% overall. For the UNC dataset, WM surface yielded 92.1%, and pial surface 89.5%. For the dHCP dataset, WM surface yields 92.2%, and pial surface 89.2%. These results indicated no visual mistakes on the majority of the constructed surfaces. The surfaces rated as fair (average across three datasets: 7.8% for WM surfaces; 9.7% for pial surfaces) showed regional flaws but were similar to the true cortical boundary (score 2). Only 2.8% of the WM surfaces and 3.4% of the pial surfaces (average across three datasets) displayed poor quality (score = 1).
Inter-rater comparison was also performed to determine how many surfaces are consistently given the same score between the two raters.” Results indicated that among 843 scans (across three datasets), 116 WM surfaces (13.8%) were rated differently, and 141 pial surfaces (16.7%) were rated differently, which suggest the surface QA was relatively consistent across raters.
Quantitative Evaluation Results
Analyzing the 20 randomly selected subjects, the landmark-based displacements for the intermediate pial surfaces and refined surfaces are shown in Supplementary Figure 3. Results exhibited that the mean displacement for intermediate surfaces (mean ± sd = 0.621 ± 0.049 mm) significantly decreases when they were refined using the gradient information (0.532 ± 0.035 mm; p < 0.0001, paired t-test).
Selection of Age-Specific Templates for the Surface Registration: Using Sulcal Depth Similarity vs. Post-menstrual Age
We evaluated here whether the template yielding the highest correlation of sulcal depth measurement with the given individual surface to register is concordant with the template chosen using the PMA of the given individual data. To this end, we constructed a confusion matrix to map their overlap (Supplementary Figure 4). Results indicated that overall 82.0% of our data showed overlap between the two methods. Most of the mis-matched templates (15.6%) were found in the nearest neighboring age groups. Among the rest of the mis-matched templates (2.4%), the majority were those with brain injury (75%), whose matched templates were identified to be younger than their PMA. The overlap was lower for the youngest and the oldest babies (65–69%) than other age groups (90–94%).
Surface Reconstruction in dHCP Measurements
To directly compare between the quality of surfaces reconstructed by our pipeline and by the dHCP pipeline, we randomly selected 12 images from the dHCP data set and a trained expert (ML) drew landmark points on the white matter and pial surfaces based on their T1w MRI. On average, 80 landmark points were placed on both of these surfaces, in both hemispheres (see Figure 10 for an example). We then computed the shortest distance between these points and our computed surfaces. Additionally, in the case of the dHCP dataset, we also computed distances between the manually drawn landmark points and the surfaces provided by the dHCP minimal processing pipeline. The mean and standard deviation of the absolute value of these measurements are included in Figure 11. The white matter surfaces tended to be comparable between the two pipelines, while in the case of the pial surfaces, our pipeline yielded lower distance with the landmarks taken from the T1w image. That was, our pipeline showed significant different placements of the pial surface from the dHCP solutions which extracted cortical surfaces mainly on the T2w image (p < 0.05).
Note that the surfaces extracted on T2 weighted images that may estimate CSF partial volume and pial surface differently compared to T1 weighted images. Figure 10 right panel shows that the same slice as the one showed in Figure 10 (middle) but with T2w modality.
Cortical Thickness Measurements
To evaluate the ability of our pial surface reconstruction in characterizing the hypothesized developmental trajectory, we measured cortical thickness by the shortest distance between pair of vertices on the WM and pial surfaces. The vertex-wise thicknesses were calculated in the native MRI space where the scan was acquired.
We mapped the individualized cortical thickness trajectory with age for both UCSF and dHCP datasets in whole brain and six anatomical lobes. We observed a very smooth overlap between the two datasets and the trajectory of cortical thickness changes along with aging (Figure 12B). Across the two datasets, by including birth age, dataset, and field strength as covariates, our pipeline identified increases in cortical thickness over time in almost all cortical regions (overall cortical thickness: +0.041 mm/week; p < 0.0001, t = 4.99; Figure 12A). A vertex-based mapping showed that regions involved in the cingulate cortex display the most rapid growth among all brain regions whereas brain regions in the inferior temporal lobe, exhibited the slowest cortical thickness growth. Figure 13 presents cortical thickness maps in 4 cases representing each of the PMA age group.
Comparison Between 3 and 1.5T
We further compared cortical thickness between images collected using 3 and 1.5T MRI for different cortical regions in UCSF dataset. Birth age, scan age and injury scores were included as covariates in the general linear model. Results along with FDR correction for multiple comparisons showed that 3T images showed a trend of thicker cortex than 1.5T images (corrected p values between 0.0009 and 0.0792 for different lobes; Supplementary Figure 5). This suggests that the cortical thickness measurement is affected by the scanner field strength.
Comparison With Literature
Cortical thickness measurements in the neonatal population have been reported in past literature and consisted of varying values, ranging between 1 and 2.2 mm (Xue et al., 2007; Moeskops et al., 2013, 2015; Li et al., 2015; Makropoulos et al., 2016, 2018b; Geng et al., 2017). One reason for variability in measurements can be due to differences in image acquisition, image segmentation, and surface reconstruction methods. Figure 14 maps the distribution of cortical thickness values measured from multiple studies in relation to their average PMA at scan.
Comparison to the NEOCIVET 1.0 Pipeline
Our new framework extended upon the NEOCIVET 1.0 pipeline (Kim et al., 2016) which was designed for neonatal WM surface (WM-GM interface) reconstruction and measurement of sulcal depths. All new features that are incorporated are summarized relative to the NEOCIVET pipeline in Table 1.
TABLE 1.
NEOCIVET 1.0 | NEOCIVET 2.0 | |
Brain extraction | BEaST | 3D-Unet |
Tissue segmentation | Multi-atlas Joint Fusion | Non-local 3D-Unet |
Training set for tissue segmentation | 27 pre-term neonates Equally sampled in 4 age-subgroups (30–40 weeks PMA) from the UCSF dataset | 24 pre-term and term neonates Equally sampled in 5 age-subgroups (26–45 weeks PMA) from three datasets. |
Surface deformation and parameterization | Deforming icosahedron surfaces with a fixed number of triangles | Marching-cube deformation with various numbers of triangles adapted to brain size and folding complexity. Subsequently, icosahedron resampling of the deformed surface |
Surface refinement in WM surface deformation | None | Gradient-based approach |
Pial surface reconstruction | None | Yes |
Measurements | Curvature, sulcal depth | Curvature, sulcal depth, gyrification index, cortical thickness |
Notably, to allow the new pipeline to better segment images scanned for old pre-term or term neonates exhibiting large variability of cortical folding across individuals, we add more manually segmented templates from old pre-term neonates and term neonates from the three datasets (PMA = 40–45 weeks; n = 7) in the library for tissue segmentation, which was not included in NEOCIVET 1.0. Finally, the new pipeline is able to extract the pial surface and measure cortical thickness, which is novel features added to the proposed pipeline.
We further compared the technical details between our pipeline and the dHCP pipeline, which can be found in Supplementary Materials.
Discussion
In this study, we propose a new NEOCIVET pipeline (NEOCIVET 2.0) for automatically reconstructing neonatal WM and pial surfaces. Our new framework extends the NEOCIVET 1.0 (Kim et al., 2016), which was designed for neonatal WM surface (WM-GM interface) reconstruction. The NEOCIVET 2.0 pipeline primarily aims to strategically address important issues in neonatal MRI processing: (1) spatiotemporally changing tissue contrast; (2) significant partial volume effects which interfere with modeling sulcation in regions where detected CSF voxels are scarce and scattered. These issues could lead to defective pial surface reconstructions and inaccurate cortical thickness measurements if pipelines designed for adult brain MRIs, e.g., CIVET (MacDonald et al., 2000; Kim et al., 2005), FreeSurfer (Fischl, 2012), and Caret (Van Essen et al., 2001), were utilized. Specifically, to optimize neonatal pial surface reconstruction, we incorporate special features: (1) Construction of skeletons from the union of gray matter and CSF; and (2) Generation of pial surfaces by expanding the WM surface mesh toward the skeleton surface, a process refined using gradient information.
Furthermore, although NEOCIVET 1.0 successfully addressed several substantial issues/challenges of neonatal brain MRI processing (e.g., similar tissue intensity between WM and CSF, reduced tissue contrast between WM and GM, large within-tissue intensity variations, and regionally heterogeneous image appearances that dynamically change along with neural development), our proposed framework included additional modules that further improved the quality of WM surface reconstruction. For instance, deep learning-based convolutional neural networks (Wang K. et al., 2020) were introduced into the pipeline to render more accurate brain extraction and tissue segmentation, and to address the mis-segmentation between WM and CSF in neonatal T1w images. No existing pipelines that process neonatal brain MRI and perform cortical morphometry have not integrated deep learning neural networks for segmentation into their frameworks. We also incorporated a new marching-cubes approach that provides adaptive surface meshes according to each individual’s cortical folding complexity. The constructed surface is then further fine-tuned using gradient information extracted from voxel intensities.
Our qualitative assessment showed that our method can reconstruct most of the surfaces analyzed with reasonable quality. Our landmark-based quantitative assessment also suggests that the proposed gradient-based refinement method combined with skeleton fitting of CSF/GM union can more accurately position the pial surface to a location that is within a significantly small distance from the visually recognized “true” boundary. Similarly, the quantitative assessment also showed that NEOCIVET 2.0 fitted WM surfaces significantly closer to the “ground truth” in comparison to NEOCIVET 1.0 pipeline. Given such high accuracy as demonstrated in the evaluations, NEOCIVET 2.0 successfully characterizes and documents the developmental trajectory of cortical folding and thickness growth in pre-term neonates. Furthermore, cross-validation using three independent datasets also suggests the generalizability of our proposed approach and the reproducibility of results using different datasets.
Skeleton or Non-skeleton
Due to large partial volume effects in neonatal MRIs, CSF voxels within deep and narrow sulci typically cannot be captured with relatively large MR image resolution. Two main approaches, one with skeleton and the other without, were designed to circumvent this issue (Li et al., 2019) for neonatal pial surface reconstruction. However, both approaches encounter several issues. For example, a non-skeleton approach relies on the deformation of the cortical surface from opposing gyri to meet near the implicitly medial surface with self-intersection constraints to generate the pial surfaces in undetectable CSFs. However, this may require the subjective selection of deformation and self-constraint related parameters that may determine when the deformation stops, which would influence the pial surface generation. On the other hand, the skeleton approach may have a lower accuracy in properly pinpointing the correct positions of CSF (Osechinskiy and Kruggel, 2012), as it does not use edge information. As a result, both methods can lead to unreliable extraction of the sulcal boundaries.
Our approach combined skeletonization and edge information to improve the accuracy of the extraction of the pial surface. Regardless of invisible CSF voxels in sulci, skeletonization creates a barrier that can constrain the surface deformation. Furthermore, it provides a location where the GM-CSF boundary can be found within a very short distance. In this respect, our pipeline was able to more reliably search and refine true GM/CSF edges.
There have also been attempts to overcome the limitations associated with skeleton approaches. For example, a weighted geometric distance has been applied to the neonatal pial surface reconstruction (Xue et al., 2007; Li et al., 2012, 2014) to solve this issue. This method, however, relies heavily on segmentation of CSF components, which can be severely limited in cases of large partial volume effects.
Quality Assessment of Surface Reconstruction
Landmark-based surface QA showed that the displacement of our reconstructed WM and pial surfaces to the “true” WM and pial borders ranged between 0.5 and 0.65 mm, which is quite small compared to the image resolution for scanning the UCSF and UNC datasets (0.6–1 mm). This QA method, which was first designed to evaluate the adult brain surface reconstructions (Han et al., 2004; Tosun et al., 2006; Xue et al., 2007), was only performed in one neonatal/infant study in the past (Zöllei et al., 2020). Their pipeline was also evaluated on T1 images from the dHCP dataset. Landmark displacements for cortical surfaces measured in their study range between 1.17 and 1.19 mm for WM surface and 0.8–0.91 mm for pial surface, respectively (Zöllei et al., 2020), which are much larger than that in the current study, implying the robustness of our proposed reconstruction method. Differences in the reported displacement values between our study and adult studies may stem from differences in image segmentation, image resolution, and brain size.
Growth Pattern in Cortical Thickness
Cortical thickness measurements (1.4 mm at 28 weeks of PMA to 2.2 mm at term equivalent) computed through our pipeline is quite similar to past studies (2.1 mm at term; e.g., Li et al., 2014) but different from others (1.1 mm at term) (Makropoulos et al., 2018b). This discrepancy may be explained by differences in thickness measurement methods, image modality (T1w vs. T2w), image segmentation, and surface reconstruction methods. Our results demonstrated a global increase of cortical thickness that positively correlated with PMA at scan. For the vertex-based ROI analysis, as in Figure 12B in the main text, we observed a very smooth overlap between UCSF and dHCP datasets, and the trajectory of cortical thickness changes along with aging are reasonably linear in both of the two datasets. In particular, brain regions corresponding to cingulate cortices grew the most rapidly. Similar results were reported in a past study which demonstrated that the cortical thickness is higher in the cingulate cortices and superior frontal cortex compared to other brain regions of term-born infants (Li et al., 2015), indicating that cortical thickness grows the most rapidly in these regions before birth. By contrast, brain regions in the temporal lobe grow the least rapidly. Cortical plate and subplate in these regions are found to grow more rapidly earlier in the 2nd trimester of gestation (Vasung et al., 2016).
Other studies have identified alterations of cortical thickness and cognition in adolescents and adults born pre-term, especially in the frontal cortices (Martinussen et al., 2005; Nagy et al., 2010; Nam et al., 2015). Thus, considering past studies combined with our results, regions near the cingulate cortices may be important biomarkers of cortical thickness changes in neurodevelopment, and these regions may be vulnerable to long-term alterations associated with prematurity or perinatal injuries that influence clinical outcome. Such alterations in cortical thickness development may be affected by both genetic and environmental influences (Lenroot and Giedd, 2008) and have been associated with intelligence, executive functions, working memory, perceptual skills, and internalizing and externalizing behavior (Lohaugen et al., 2009; Skranes et al., 2012; Zubiaurre-Elorza et al., 2012).
Ultimately, cortical thickness trajectories have important clinical relevance and can be useful biomarkers that help to predict neurodevelopmental and cognitive outcomes. The new pipeline is capable of mapping dynamic cortical thickness changes throughout the third trimester of gestation, an early and critical phase of development, which can further support the early diagnosis of pre-term neonates and, thereby, allow clinicians to determine when and what medical interventions may be appropriate.
Limitations and Future Directions
We applied our pipeline to T1w images, while many attempts in neonatal brain segmentation and surface reconstruction were conducted on T2w images or T1w–T2w modalities combined (Shi et al., 2010; Wang et al., 2014; Makropoulos et al., 2016, 2018b) due to the increased contrast of brain tissues in T2w images. It is important to optimize a method that considers T1w images, which are collected prevalently for various research aims. Our pipeline is the first proposed specifically for T1 images. Despite our new evaluation results where cortical thickness measurements vary depending on the image modality analyzed by each pipeline, it is difficult to conclude that whether T1 or T2 imaging is more ideal to construct cortical surfaces. Our pipeline would rather supplement the dHCP pipeline depending on the availability of T1 or T2 MRI data, since only one sequence may be available under the different clinical/research setting. In order to make our method more compatible with other group’s methods in terms of pial surface reconstruction and cortical thickness measurement, we aim to expand our pipeline to include T2w images in future studies. In addition, as seen in our finding in differences of cortical thickness due to different MRI field strengths, there has been a collection of evidence that various factors may bias cortical thickness measurements (Fortin et al., 2018). Such factors biasing cortical surface reconstruction and subsequently cortical thickness measurement include different acquisition parameters, imaging sequence type (T1 vs. T2), field strength, MRI machine and patient’s characteristics such as the presence of brain injuries and perinatal clinical risk factors that leads to delay in brain growth. Heterogeneity of these factors among the datasets analyzed in previous studies and our study along with the methodological difference may lead to the low consensus among studies shown in Figure 14. If multiple datasets were included in a single study, researchers must harmonize them before the statistical analysis. Recently, several harmonization approaches have been proposed to address this issues when cortical thickness from multiple dataset were used together (Fortin et al., 2018).
Many neonates analyzed in our study were scanned twice, but each scan was treated as independent images when extracting their brain surfaces. While our approach is appropriate for cross-sectional analyses, past studies have proposed the segmentation/surface reconstruction methods that make use of information about the growth rate found in longitudinally collected MR images (Dai et al., 2013; Wang et al., 2013; Li et al., 2014, 2015; Nie et al., 2014). This approach continues to be a research area of interest for future investigations aiming to improve cortical surface reconstruction.
Accurate reconstruction of neonatal brain WM and pial surfaces can be applicable to sampling cortical measurements of other modal MRI data which explain changes in different aspects of the brain. For example, diffusion tensor imaging is used to extract parameters related to WM integrity, such as fractional anisotropy and mean diffusivity. T1/T2 ratio myelin map (Glasser and Van Essen, 2011) is also shown to be a sensitive biomarker that predicts neonatal brain ages (Lewis et al., 2018; Liu et al., 2020) and cognitive outcomes (Moeskops et al., 2017). Using a boundary-based registration approach (Greve and Fischl, 2009) based on our reconstructed brain surfaces, we plan to compute these measurements within the cortex or on superficial WM, leading to more versatile applications that help clarify mechanisms involved in the early brain development.
Future studies also aim to elucidate the neurobiological and cellular basis for changes in cortical thickness trajectories. A possible explanation of such topological changes is through prematurity-related abnormalities in synaptic pruning, which refines developing neural circuits important for cognitive function and specialization (Knudsen, 2004; Raznahan et al., 2011b).
We have opened the NEOCIVET 2.0 to the public through CBRAIN2, a web-based platform that distributes, processes, and exchanges 3D/4D brain imaging data. Our pipeline can also be used to investigate brain morphological changes in neonates born with clinical conditions.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by UCSF Benioff Children’s Hospital. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin. Written informed consent was obtained from the individual(s), and minor(s)’ legal guardian/next of kin, for the publication of any potentially identifiable images or data included in this article.
Author Contributions
ML: investigation, formal analysis and writing–original draft. CL, SJ, and AE: methodology. SYK: writing–original draft. SHK, JS, NJ, MS, AB, and DX: writing–review and editing. NT, SY, TI, BP, KY, WS, and JK: data curation. AT: funding acquisition and writing–review and editing. HK: conceptualization, investigation, supervision, funding acquisition and writing–review and editing. All authors contributed to the article and approved the submitted version.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Funding. This study was supported by the National Institutes of Health grants (P41EB015922; U54EB020406; U19AG024904; U01NS086090; and 003585-00001). HK was funded by Baxter Foundation Faculty Fellowship Award, and BrightFocus Research Grant award (A2019052S).
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2021.650082/full#supplementary-material
References
- Alexander-Bloch A. F., Shou H. C., Liu S. Y., Satterthwaite T. D., Glahn D. C., Shinohara R. T., et al. (2018). On testing for spatial correspondence between maps of human brain structure and function. Neuroimage 178 540–551. 10.1016/j.neuroimage.2018.05.070 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ball G., Boardman J. P., Rueckert D., Aljabar P., Arichi T., Merchant N., et al. (2012). The effect of preterm birth on thalamic and cortical development. Cereb. Cortex 22 1016–1024. 10.1093/cercor/bhr176 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ball G., Pazderova L., Chew A., Tusor N., Merchant N., Arichi T., et al. (2015). Thalamocortical connectivity predicts cognition in children born preterm. Cereb. Cortex 25 4310–4318. 10.1093/cercor/bhu331 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ball G., Srinivasan L., Aljabar P., Counsell S. J., Durighel G., Hajnal J. V., et al. (2013). Development of cortical microstructure in the preterm human brain. Proc. Natl. Acad. Sci. U.S.A. 110 9541–9546. 10.1073/pnas.1301652110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Battin M. R., Maalouf E. F., Counsell S. J., Herlihy A. H., Rutherford M. A., Azzopardi D., et al. (1998). Magnetic resonance imaging of the brain in very preterm infants: visualization of the germinal matrix, early myelination, and cortical folding. Pediatrics 101 957–962. 10.1542/peds.101.6.957 [DOI] [PubMed] [Google Scholar]
- Beare R. J., Chen J., Kelly C. E., Alexopoulos D., Smyser C. D., Rogers C. E., et al. (2016). Neonatal brain tissue classification with morphological adaptation and unified segmentation. Front. Neuroinform. 10:12. 10.3389/fninf.2016.00012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boucher M., Whitesides S., Evans A. (2009). Depth potential function for folding pattern representation, registration and analysis. Med. Image Anal. 13 203–214. 10.1016/j.media.2008.09.001 [DOI] [PubMed] [Google Scholar]
- Çiçek Ö, Abdulkadir A., Lienkamp S. S., Brox T., Ronneberger O. (2016). “3D U-Net: learning dense volumetric segmentation from sparse annotation,” in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (Cham: Springer), 424–432. 10.1007/978-3-319-46723-8_49 [DOI] [Google Scholar]
- Dai Y., Shi F., Wang L., Wu G., Shen D. (2013). iBEAT: a toolbox for infant brain magnetic resonance image processing. Neuroinformatics 11 211–225. 10.1007/s12021-012-9164-z [DOI] [PubMed] [Google Scholar]
- Dale A. M., Fischl B., Sereno M. I. (1999). Cortical surface-based analysis - I. Segmentation and surface reconstruction. Neuroimage 9 179–194. 10.1006/nimg.1998.0395 [DOI] [PubMed] [Google Scholar]
- Fischl B. (2012). FreeSurfer. Neuroimage 62 774–781. 10.1016/j.neuroimage.2012.01.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fortin J.-P., Cullen N., Sheline Y. I., Taylor W. D., Aselcioglu I., Cook P. A., et al. (2018). Harmonization of cortical thickness measurements across scanners and sites. Neuroimage 167 104–120. 10.1016/j.neuroimage.2017.11.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Geng X. J., Li G., Lu Z. H., Gao W., Wang L., Shen D. G., et al. (2017). Structural and maturational covariance in early childhood brain development. Cereb. Cortex 27 1795–1807. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glasser M. F., Van Essen D. C. (2011). Mapping human cortical areas in vivo based on myelin content as revealed by T1-and T2-weighted MRI. J. Neurosci. 31 11597–11616. 10.1523/JNEUROSCI.2180-11.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gousias I. S., Hammers A., Counsell S. J., Srinivasan L., Rutherford M. A., Heckemann R. A., et al. (2013). Magnetic resonance imaging of the newborn brain: automatic segmentation of brain images into 50 anatomical regions. PLoS One 8:e59990. 10.1371/journal.pone.0059990 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greve D. N., Fischl B. (2009). Accurate and robust brain image alignment using boundary-based registration. Neuroimage 48 63–72. 10.1016/j.neuroimage.2009.06.060 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guo Y., Dong P., Hao S., Wang L., Wu G., Shen D. (2016). Automatic Segmentation of hippocampus for longitudinal infant brain MR image sequence by spatial-temporal hypergraph learning. Patch Based Tech. Med. Imaging (2016) 9993 1–8. 10.1007/978-3-319-47118-1_1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Habas P. A., Scott J. A., Roosta A., Rajagopalan V., Kim K., Rousseau F., et al. (2012). Early folding patterns and asymmetries of the normal human brain detected from in utero MRI. Cereb. Cortex 22 13–25. 10.1093/cercor/bhr053 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Han X., Pham D. L., Tosun D., Rettmann M. E., Xu C. Y., Prince J. L. (2004). CRUISE: cortical reconstruction using implicit surface evolution. Neuroimage 23 997–1012. 10.1016/j.neuroimage.2004.06.043 [DOI] [PubMed] [Google Scholar]
- Han X., Xu C. Y., Rettmann M. E., Prince J. L. (2001). Automatic segmentation editing for cortical surface reconstruction. Proceedings of the Medical Imaging 2001: Image Processing, Vol. 4322 San Diego, CA, 194–203. 10.1117/12.431082 [DOI] [Google Scholar]
- Hill J., Dierker D., Neil J., Inder T., Knutsen A., Harwell J., et al. (2010). A surface-based analysis of hemispheric asymmetries and folding of cerebral cortex in term-born human infants. J. Neurosci. 30 2268–2276. 10.1523/JNEUROSCI.4682-09.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hwang H., Rehman H. Z. U., Lee S. (2019). 3D U-Net for skull stripping in brain MRI. Appl. Sci. 9:569. 10.3390/app9030569 [DOI] [Google Scholar]
- Im K., Lee J. M., Lee J., Shin Y. W., Kim I. Y., Kwon J. S., et al. (2006). Gender difference analysis of cortical thickness in healthy young adults with surface-based methods. Neuroimage 31 31–38. 10.1016/j.neuroimage.2005.11.042 [DOI] [PubMed] [Google Scholar]
- Kapellou O., Counsell S. J., Kennea N., Dyet L., Saeed N., Stark J., et al. (2006). Abnormal cortical development after premature birth shown by altered allometric scaling of brain growth. PLoS Med. 3:e265. 10.1371/journal.pmed.0030265 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kim H., Lepage C., Maheshwary R., Jeon S., Evans A. C., Hess C. P., et al. (2016). NEOCIVET: towards accurate morphometry of neonatal gyrification and clinical applications in preterm newborns. Neuroimage 138 28–42. 10.1016/j.neuroimage.2016.05.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kim J. S., Singh V., Lee J. K., Lerch J., Ad-Dab’bagh Y., Macdonald D., et al. (2005). Automated 3-D extraction and evaluation of the inner and outer cortical surfaces using a Laplacian map and partial volume effect classification. Neuroimage 27 210–221. 10.1016/j.neuroimage.2005.03.036 [DOI] [PubMed] [Google Scholar]
- Kim S. Y., Liu M., Hong S.-J., Toga A. W., Barkovich A. J., Xu D., et al. (2020). Disruption and compensation of sulcation-based covariance networks in neonatal brain growth after perinatal injury. Cereb. Cortex 30 6238–6253. 10.1093/cercor/bhaa181 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Knickmeyer R. C., Gouttard S., Kang C. Y., Evans D., Wilber K., Smith J. K., et al. (2008). A structural MRI study of human brain development from birth to 2 years. J. Neurosci. 28 12176–12182. 10.1523/JNEUROSCI.3479-08.2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Knudsen E. I. (2004). Sensitive periods in the development of the brain and behavior. J. Cogn. Neurosci. 16 1412–1425. 10.1162/0898929042304796 [DOI] [PubMed] [Google Scholar]
- Lee J. K., Lee J. M., Kim J. S., Kim I. Y., Evans A. C., Kim S. I. (2006). A novel quantitative cross-validation of different cortical surface reconstruction algorithms using MRI phantom. Neuroimage 31 572–584. 10.1016/j.neuroimage.2005.12.044 [DOI] [PubMed] [Google Scholar]
- Lefevre J., Germanaud D., Dubois J., Rousseau F., De Macedo Santos I., Angleys H., et al. (2016). Are developmental trajectories of cortical folding comparable between cross-sectional datasets of fetuses and preterm newborns? Cereb. Cortex 26 3023–3035. 10.1093/cercor/bhv123 [DOI] [PubMed] [Google Scholar]
- Lenroot R. K., Giedd J. N. (2008). The changing impact of genes and environment on brain development during childhood and adolescence: initial findings from a neuroimaging study of pediatric twins. Dev. Psychopathol. 20 1161–1175. 10.1017/S0954579408000552 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lerch J. P., Evans A. C. (2005). Cortical thickness analysis examined through power analysis and a population simulation. Neuroimage 24 163–173. 10.1016/j.neuroimage.2004.07.045 [DOI] [PubMed] [Google Scholar]
- Leroy F., Mangin J. F., Rousseau F., Glasel H., Hertz-Pannier L., Dubois J., et al. (2011). Atlas-free surface reconstruction of the cortical grey-white interface in infants. PLoS One 6:e27128. 10.1371/journal.pone.0027128 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis J. D., Evans A. C., Tohka J., Group B. D. C. (2018). T1 white/gray contrast as a predictor of chronological age, and an index of cognitive performance. NeuroImage 173 341–350. 10.1016/j.neuroimage.2018.02.050 [DOI] [PubMed] [Google Scholar]
- Li G., Lin W., Gilmore J. H., Shen D. (2015). Spatial patterns, longitudinal development, and hemispheric asymmetries of cortical thickness in infants from birth to 2 years of age. J. Neurosci. 35 9150–9162. 10.1523/JNEUROSCI.4107-14.2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li G., Nie J. X., Wang L., Shi F., Gilmore J. H., Lin W. L., et al. (2014). Measuring the dynamic longitudinal cortex development in infants by reconstruction of temporally consistent cortical surfaces. Neuroimage 90 266–279. 10.1016/j.neuroimage.2013.12.038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li G., Nie J. X., Wu G. R., Wang Y. P., Shen D. G., Initi A. S. D. N. (2012). Consistent reconstruction of cortical surfaces from longitudinal brain MR images. Neuroimage 59 3805–3820. 10.1016/j.neuroimage.2011.11.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li G., Wang L., Yap P. T., Wang F., Wu Z., Meng Y., et al. (2019). Computational neuroanatomy of baby brains: a review. Neuroimage 185 906–925. 10.1016/j.neuroimage.2018.03.042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu M., Duffy B. A., Sun Z., Toga A. W., Barkovich A. J., Xu D., et al. (2020). “Deep learning of cortical surface features using graph-convolution predicts neonatal brain age and neurodevelopmental outcome,” in Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI): IEEE, Iowa City, IA, 1335–1338. 10.1109/ISBI45749.2020.9098556 [DOI] [Google Scholar]
- Liu M., Lepage C., Jeon S., Flynn T., Yuan S., Kim J., et al. (2019). “A skeleton and deformation based model for neonatal pial surface reconstruction in preterm newborns,” in Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019): IEEE, Venice, 352–355. 10.1109/ISBI.2019.8759183 [DOI] [Google Scholar]
- Lohaugen G., Martinussen M., Evensen K., Vangberg T., Haraldseth O., Dale A., et al. (2009). Regional cerebral cortical thinning and neuropsychological impairments in very low birth weight (VLBW) adolescents. NeuroImage 47:S110. 10.1016/S1053-8119(09)70985-1 [DOI] [Google Scholar]
- MacDonald D., Kabani N., Avis D., Evans A. C. (2000). Automated 3-D extraction of inner and outer surfaces of cerebral cortex from MRI. Neuroimage 12 340–356. 10.1006/nimg.1999.0534 [DOI] [PubMed] [Google Scholar]
- Makropoulos A., Aljabar P., Wright R., Huning B., Merchant N., Arichi T., et al. (2016). Regional growth and atlasing of the developing human brain. Neuroimage 125 456–478. 10.1016/j.neuroimage.2015.10.047 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Makropoulos A., Counsell S. J., Rueckert D. (2018a). A review on automatic fetal and neonatal brain MRI segmentation. Neuroimage 170 231–248. 10.1016/j.neuroimage.2017.06.074 [DOI] [PubMed] [Google Scholar]
- Makropoulos A., Gousias I. S., Ledig C., Aljabar P., Serag A., Hajnal J. V., et al. (2014). Automatic whole brain MRI segmentation of the developing neonatal brain. IEEE Trans. Med. Imaging 33 1818–1831. 10.1109/TMI.2014.2322280 [DOI] [PubMed] [Google Scholar]
- Makropoulos A., Robinson E. C., Schuh A., Wright R., Fitzgibbon S., Bozek J., et al. (2018b). The developing human connectome project: a minimal processing pipeline for neonatal cortical surface reconstruction. Neuroimage 173 88–112. 10.1016/j.neuroimage.2018.01.054 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Malandain G., Bertrand G., Ayache N. (1993). Topological segmentation of discrete surfaces. Int. J. Comput. Vis. 10 183–197. 10.1007/BF01420736 [DOI] [Google Scholar]
- Mangin J.-F., Frouin V., Bloch I., Régis J., López-Krahe J. (1995). From 3D magnetic resonance images to structural representations of the cortex topography using topology preserving deformations. J. Math. Imaging Vis. 5 297–318. 10.1007/BF01250286 [DOI] [Google Scholar]
- Martinussen M., Fischl B., Larsson H., Skranes J., Kulseng S., Vangberg T., et al. (2005). Cerebral cortex thickness in 15-year-old adolescents with low birth weight measured by an automated MRI-based method. Brain 128 2588–2596. 10.1093/brain/awh610 [DOI] [PubMed] [Google Scholar]
- Moeskops P., Benders M. J., Kersbergen K. J., Groenendaal F., De Vries L. S., Viergever M. A., et al. (2015). Development of cortical morphology evaluated with longitudinal MR brain images of preterm infants. PLoS One 10:e0131552. 10.1371/journal.pone.0131552 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moeskops P., Benders M. J. N. L., Pearlman P. C., Kersbergen K. J., Leemans A., Viergever M. A., et al. (2013). “Assessment of quantitative cortical biomarkers in the developing brain of preterm infants,” in Medical Imaging 2013: Computer-Aided Diagnosis. Event: SPIE Medical Imaging, Vol. 8670 Orlando, FL. 10.1117/12.2007829 [DOI] [Google Scholar]
- Moeskops P., Išgum I., Keunen K., Claessens N. H., Haastert I. C., Groenendaal F., et al. (2017). Prediction of cognitive and motor outcome of preterm infants based on automatic quantitative descriptors from neonatal MR brain images. Sci. Rep. 7:2163. 10.1038/s41598-017-02307-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nagy Z., Lagercrantz H., Hutton C. (2010). Effects of preterm birth on cortical thickness measured in adolescence. Cereb. Cortex 21 300–306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nam K. W., Castellanos N., Simmons A., Froudist-Walsh S., Allin M. P., Walshe M., et al. (2015). Alterations in cortical thickness development in preterm-born individuals: implications for high-order cognitive functions. NeuroImage 115 64–75. 10.1016/j.neuroimage.2015.04.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nie J., Li G., Wang L., Shi F., Lin W., Gilmore J. H., et al. (2014). Longitudinal development of cortical thickness, folding, and fiber density networks in the first 2 years of life. Hum. Brain Mapp. 35 3726–3737. 10.1002/hbm.22432 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Osechinskiy S., Kruggel F. (2012). Cortical surface reconstruction from high-resolution MR brain images. Int. J. Biomed. Imaging 2012:870196. 10.1155/2012/870196 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Raznahan A., Lerch J. P., Lee N., Greenstein D., Wallace G. L., Stockman M., et al. (2011a). Patterns of coordinated anatomical change in human cortical development: a longitudinal neuroimaging study of maturational coupling. Neuron 72 873–884. 10.1016/j.neuron.2011.09.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Raznahan A., Shaw P., Lalonde F., Stockman M., Wallace G. L., Greenstein D., et al. (2011b). How does your cortex grow? J. Neurosci. 31 7174–7177. 10.1523/JNEUROSCI.0054-11.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riviere D., Mangin J. F., Papadopoulos-Orfanos D., Martinez J. M., Frouin V., Regis J. (2002). Automatic recognition of cortical sulci of the human brain using a congregation of neural networks. Med. Image Anal. 6 77–92. 10.1016/S1361-8415(02)00052-X [DOI] [PubMed] [Google Scholar]
- Robbins S., Evans A. C., Collins D. L., Whitesides S. (2004). Tuning and comparing spatial normalization methods. Med. Image Anal. 8 311–323. 10.1016/j.media.2004.06.009 [DOI] [PubMed] [Google Scholar]
- Ronneberger O., Fischer P., Brox T. (2015). “U-net: convolutional networks for biomedical image segmentation,” in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, (Cham: Springer; ), 234–241. 10.1007/978-3-319-24574-4_28 [DOI] [Google Scholar]
- Schuh A., Makropoulos A., Wright R., Robinson E. C., Tusor N., Steinweg J., et al. (2017). “A deformable model for the reconstruction of the neonatal cortex,” in Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging, Melbourne, VIC, 800-803. 10.1109/ISBI.2017.7950639 [DOI] [Google Scholar]
- Shen W., Bai X. A., Hu R., Wang H. Y., Latecki L. J. (2011). Skeleton growing and pruning with bending potential ratio. Pattern Recognit. 44 196–209. 10.1016/j.patcog.2010.08.021 [DOI] [Google Scholar]
- Shi F., Fan Y., Tang S., Gilmore J. H., Lin W., Shen D. (2010). Neonatal brain image segmentation in longitudinal MRI studies. Neuroimage 49 391–400. 10.1016/j.neuroimage.2009.07.066 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Skranes J., Løhaugen G. C., Evensen K.a.I, Indredavik M. S., Haraldseth O., Dale A. M., et al. (2012). Entorhinal cortical thinning affects perceptual and cognitive functions in adolescents born preterm with very low birth weight (VLBW). Early Hum. Dev. 88 103–109. 10.1016/j.earlhumdev.2011.07.017 [DOI] [PubMed] [Google Scholar]
- Tosun D., Rettmann M. E., Naiman D. Q., Resnick S. M., Kraut M. A., Prince J. L. (2006). Cortical reconstruction using implicit surface evolution: accuracy and precision analysis. NeuroImage 29 838–852. 10.1016/j.neuroimage.2005.08.061 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Essen D. C., Drury H. A., Dickson J., Harwell J., Hanlon D., Anderson C. H. (2001). An integrated software suite for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 8 443–459. 10.1136/jamia.2001.0080443 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vasung L., Lepage C., Radoš M., Pletikos M., Goldman J. S., Richiardi J., et al. (2016). Quantitative and qualitative analysis of transient fetal compartments during prenatal human brain development. Front. Neuroanat. 10:11. 10.3389/fnana.2016.00011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang H. Z., Suh J. W., Das S. R., Pluta J. B., Craige C., Yushkevich P. A. (2013). Multi-Atlas segmentation with joint label fusion. IEEE Trans. Pattern Anal. Mach. Intell. 35 611–623. 10.1109/TPAMI.2012.143 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang K., Shou Q., Ma S. J., Liebeskind D., Qiao X. J., Saver J., et al. (2020). Deep learning detection of penumbral tissue on arterial spin labeling in stroke. Stroke 51 489–497. 10.1161/STROKEAHA.119.027457 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang L., Gao Y., Shi F., Li G., Gilmore J. H., Lin W., et al. (2015). LINKS: learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images. NeuroImage 108 160–172. 10.1016/j.neuroimage.2014.12.042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang Q., Wu G., Wang L., Shi P., Lin W., Shen D. (2014). “Sparsity-learning-based longitudinal MR image registration for early brain development,” in Proceedings of the International Workshop on Machine Learning in Medical Imaging. MLMI 2014. Lecture Notes in Computer Science, Vol. 8679 eds Wu G., Zhang D., Zhou L. (Cham: Springer; ), 1–8. 10.1007/978-3-319-10581-9_1 [DOI] [Google Scholar]
- Wang Z., Zou N., Shen D., Ji S. (2020). “Non-Local U-Nets for Biomedical Image Segmentation,” in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34 New York, NY, 6315–6322. 10.1609/aaai.v34i04.6100 [DOI] [Google Scholar]
- Weisenfeld N. I., Warfield S. K. (2009). Automatic segmentation of newborn brain MRI. Neuroimage 47 564–572. 10.1016/j.neuroimage.2009.04.068 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wright R., Kyriakopoulou V., Ledig C., Rutherford M. A., Hajnal J. V., Rueckert D., et al. (2014). Automatic quantification of normal cortical folding patterns from fetal brain MRI. Neuroimage 91 21–32. 10.1016/j.neuroimage.2014.01.034 [DOI] [PubMed] [Google Scholar]
- Wright R., Makropoulos A., Kyriakopoulou V., Patkee P. A., Koch L. M., Rutherford M. A., et al. (2015). Construction of a fetal spatio-temporal cortical surface atlas from in utero MRI: application of spectral surface matching. Neuroimage 120 467–480. 10.1016/j.neuroimage.2015.05.087 [DOI] [PubMed] [Google Scholar]
- Xue H., Srinivasan L., Jiang S. Z., Rutherford M., Edwards A. D., Rueckert D., et al. (2007). Automatic segmentation and reconstruction of the cortex from neonatal MRI. Neuroimage 38 461–477. 10.1016/j.neuroimage.2007.07.030 [DOI] [PubMed] [Google Scholar]
- Zöllei L., Iglesias J. E., Ou Y., Grant P. E., Fischl B. (2020). Infant FreeSurfer: an automated segmentation and surface extraction pipeline for T1-weighted neuroimaging data of infants 0-2 years. Neuroimage 218:116946. 10.1016/j.neuroimage.2020.116946 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zubiaurre-Elorza L., Soria-Pastor S., Junque C., Sala-Llonch R., Segarra D., Bargallo N., et al. (2012). Cortical thickness and behavior abnormalities in children born preterm. PLoS One 7:e42148. 10.1371/journal.pone.0042148 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.