Automatic labeling of cortical sulci for the human fetal brain based on spatio-temporal information of gyrification

Abstract

Accurate parcellation and labeling of primary cortical sulci in the human fetal brain is useful for regional analysis of brain development. However, human fetal brains show large spatio-temporal changes in brain size, cortical folding patterns, and relative position/size of cortical regions, making accurate automatic sulcal labeling challenging. Here, we introduce a novel sulcal labeling method for the fetal brain using spatio-temporal gyrification information from multiple fetal templates. First, spatial probability maps of primary sulci are generated on the templates from 23 to 33 gestational weeks and registered to an individual brain. Second, temporal weights, which determine the level of contribution to the labeling for each template, are defined by similarity of gyrification between the individual and the template brains. We combine the weighted sulcal probability maps from the multiple templates and adopt sulcal basin-wise approach to assign sulcal labels to each basin. Our labeling method was applied to 25 fetuses (22.9 - 29.6 gestational weeks), and the labeling accuracy was compared to manually assigned sulcal labels using the Dice coefficient. Moreover, our multi-template basin-wise approach was compared to a single-template approach, which does not consider the temporal dynamics of gyrification, and a fully-vertex-wise approach. The mean accuracy of our approach was 0.958 across subjects, significantly higher than the accuracies of the other approaches. This novel approach shows highly accurate sulcal labeling and provides a reliable means to examine characteristics of cortical regions in the fetal brain.

Introduction

In the human cerebral cortex, gyrification is an important index of neurodevelopment. As a result of gyrification, the human brain has a distinctive pattern of cortical folds and it has been hypothesized that the folding pattern has a close relationship with functional organization and cytoarchitectonic areas (Fischl et al., 2008; Amunts et al., 2000; Eickhoff et al., 2006)(Hasnain et al., 2001; Rademacher et al., 1993; Welker, 1990; Zilles et al., 1997). Based on their importance, the folding pattern of the human brain has been widely analyzed (Cykowski et al., 2008; Dubois et al., 2008; Habas et al., 2012; Im et al., 2017, 2011, 2010; Leroy et al., 2015; Levine and Barnes, 1999; Lohmann et al., 1999; Nakamura et al., 2007; Nordahl et al., 2007; Tarui et al., 2018).

While inter-subject sulcal shapes are considerably diverse, primary sulci have been used as neuroanatomical landmarks because their locations are spatially uniform across individuals (Lohmann et al., 2008). Thus assigning anatomical labels to sulci is a useful approach for localizing brain areas, and investigating quantitative features in the labeled sulcal regions enables computational analysis in functionally/anatomically defined regions of interest (Bondiau et al., 2005; Desikan et al., 2006; Fischl et al., 2004; Gousias et al., 2013; Hammers et al., 2003; Im et al., 2011; Lohmann and von Cramon, 2000; Mega et al., 2005; Shattuck et al., 2008; Toga and Thompson, 2001). For labeling individual brains, surface-based registration techniques are often used to align spatial information of brain regions of templates (anatomical labels or their probability maps) onto the individual brain (Fischl et al., 1999; Lyttelton et al., 2009; Robbins, 2004). Because the spatial distribution of the deep sulcal folds is uniform across adults, aligned spatial information is reliably fitted to the individual brain. Therefore, previous labeling methods using surface-based registration showed highly accurate labeling results in the adult brain (Fischl et al., 2004; Desikan et al., 2006; Im et al., 2011). However, compared to the adult brain, the fetal brain shows greater temporal dynamics in brain folding and growth. The number and size of primary sulci detected in magnetic resonance (MR) images increases from early to late gestational weeks (GW) following an ordered sequence at specific gestational ages (Garel et al., 2003, 2001). In addition to the chronology of sulcal appearance, it has been reported that primary sensory areas develop earlier than regions associated with higher order cognitive function such as lateral temporal, prefrontal and parietal lobes (Fischl et al., 2008; Fjell et al., 2015; Habas et al., 2012; Hill et al., 2010; Vuoksimaa et al., 2016). Thus, in contrast to adults, the fetal brain has a dynamic temporal pattern of growth that results in prominent changes in brain size, sulcal shape, and relative areal expansion of cortical regions. These distinctive and time-variant patterns of cortical folding and areal expansion may affect the performance of surface-based registration. The spatial information generated from the adult brain may not be well-aligned to the fetal brain, and could result in mislabeling. To avoid and overcome alignment errors, the spatial information generated from the fetal brain needs to be established and used in the labeling procedure. Another source of error could arise from the temporal variation of sulcal emergence. The use of spatial information from one specific stage of gyrification may produce incomplete labeling results in an individual with more sulci than the average population. In this case, it is possible for some sulcal regions in the individual brain to be inaccurately assigned to other sulcal labels. Therefore, it is necessary to consider the folding pattern over time which changes during gestation.

Here, we propose a new multi-template-based method to label primary sulci at the sulcal basin level (Multi-Template-based Basin-wise Labeling: MTBL) in the fetal brain using spatio-temporal gyrification information from fetal brain templates between 23 and 33 GW (Serag et al., 2012). We labeled the sulci in 25 typically developing (TD) fetuses (22.9 to 29.6 GW) and validated the labeling accuracy against manually assigned sulcal labels. As the first paper on sulci labeling in the fetal brain, we systematically assessed the MTBL against other approaches typically applied to the adult population. Specifically, the MTBL was compared to single-template methods, which do not consider the temporal dynamics and inter-subject variation of gyrification, as well as vertex-wise methods.

Methods

Data and image acquisition

A total of 25 TD fetuses were included in this study (male/female: 12/13; age: 26.2 ± 1.9GW [mean ± standard deviation], ranged from 22.9 to 29.6GW, maternal age: 18-45 years). TD subjects were identified by the following exclusion criteria: multiple gestation pregnancies, dysmorphic features on ultrasound examination, other brain malformations or brain lesions, known chromosomal abnormalities, other identified organ anomalies, known congenital infections, and any abnormality from the fetal MRI.

This study was approved by Institutional Review Boards at Boston Children’s Hospital (BCH) and Tufts Medical Center (TMC), and all participants signed a written informed consent. Fetal brain MR images were acquired on a Siemens 3T Skyra scanner (BCH) or Phillips 1.5 T scanner (TMC). The MR images were collected using a T2-weighted HASTE (Half-Fourier Acquisition Single-Shot Turbo Spin-Echo) sequence with 1 mm in-plane resolution, FOV = 256mm, time repetition = ~1.5s (BCH) or 12.5s (TMC), time echo = ~120ms (BCH) or 180ms (TMC), and slice thickness = 2~4mm. After localizing the fetal brain, the HASTE scans were acquired multiple times in different orthogonal orientations for reliable image processing and analysis (scan time for acquisition of volumetric images was up to 20 minutes).

MR image processing and surface reconstruction

We adopted the pipeline for fetal MR image processing and surface extraction used in our previous study (Im et al., 2017; Tarui et al., 2017). Motion artifact on individual fetal brain MR images was corrected by combining multiple 2D slices to create a motion-corrected coherent volume data set with isotropic resolution (0.75 mm) (Kuklisova-Murgasova et al., 2012). Then, we manually aligned the volume images along the anterior and posterior commissure axis using AFNI (https://afni.nimh.nih.gov/afni) (Cox, 2012). For the aligned fetal MR images, the cortical plate was segmented by a semi-automatic approach based on voxel intensities using FreeView (https://surfer.nmr.mgh.harvard.edu). We automatically segmented the cortical plate on the basis of the intensity ranges in each slice, and then we manually edited the automatic segmentation results on orthogonal views of MR images. In the inner volume of the cortical plate, spatial smoothing was performed with 1.5mm full-width at half-maximum (FWHM) kernel to minimize the noise of the semi-automated segmentation approach. Then, the hemispheric (left and right) triangular surfaces of the cortical plate were automatically extracted by a function “isosurface” in MATLAB 2016b (MathWorks Inc., Natick, MA). The surface models were geometrically smoothed using Freesurfer (https://surfer.nmr.mgh.harvard.edu), because shape measures are susceptible to noise and small geometric changes. The overview of the pipeline used in this section is shown in Supplementary Figure 1.

Automatic labeling pipeline

In this section, we introduce the entire pipeline of the MTBL for fetal brain sulcal labeling. The fetal brain template images from 23 to 33GW (http://brain-development.org/brain-atlases)(Serag et al., 2012) were used in this study and processed using the same methods described in the above section.

We generated spatial probability maps of each sulcus on the multi-template fetal brain surfaces. The temporal weights were calculated to determine the contribution level of each template for labeling the individual brain based on folding similarity between an individual and the template surfaces. Combining the probability maps and the temporal weights, each sulcal basin segmented on the cortical plate surface was labeled. For the comparison of the MTBL with other labeling methods, we additionally implemented three other methods which employed a single-template approach, a vertex-wise approach, or both approaches together. The overview of all these methods is shown in Figure 1, and each step in the pipeline is detailed in the following sections.

Figure 1. Overview of the pipeline.

Each step (underline) is described in a subsection of “Automatic labeling pipeline”. There are 4 different methods to assign sulcal labels on individual brain surface. The multi-template-based basin-wise labeling (the MTBL) method is proposed one in this study (black arrows).

Segmentation and identification of sulcal basins

For implementing the MTBL, the first step was to extract the sulcal basins on the cortical surface which are defined as locally the deepest region in the cortical sulci (Auzias et al., 2015; Im et al., 2010; Lohmann et al., 2008; Meng et al., 2014; Tarui et al., 2018). To automatically segment the sulcal basins on the surface models, mean curvature was computed as the angular deviation from a patch around each vertex on the surface models (Meyer et al., 2003), and surface area was measured as Voronoi region area of each vertex on cortical surface (Meyer et al., 2003). Mean curvature was smoothed by 10mm FWHM kernel (Im et al., 2016, 2013, 2010; Tarui et al., 2018). On the smoothed mean curvature map, a watershed flooding algorithm was employed to identify sulcal basins (Im et al., 2010; Rettmann et al., 2002). To prevent noisy sulcal basins generated by minor geometric variations in sulcal shape, a merging criteria was applied when two or more basins met at a ridge point (small gyrus buried in a sulcus, Figure 2A) (Auzias et al., 2015; Im et al., 2010; Meng et al., 2014). The criteria included three thresholds: area of the basin [ThrA], geodesic distance between the deepest points of two basins [ThrD] and ridge height [ThrR]. In this study, we accounted for the dramatic changes in brain size during the fetal stage by normalizing the thresholds based on individual brain size rather than constant thresholds which were optimized for the adult brain (Im et al., 2010, 2011). The effect of the geometrical features (mean curvature and surface area) used in the merging process of the watershed algorithm is associated with brain size (Im et al., 2008; Meng et al., 2014). When applying improper thresholds regardless of the brain size, the size of sulcal basins are relatively large or small compared to the brain size that may produce some possible errors. A small size basin in the boundary of a sulcus may be assigned as a different label which is adjacent to the sulcus. Large thresholds may merge sulcal basins that belong to different sulci. These errors with improper basin size may decrease labeling accuracy, thus, it is necessary to extract the proper size of sulcal basins in the fetal brain. We normalized the thresholds by scaling of the geometrical features (Auzias et al., 2015; Meng et al., 2014). ThrA and ThrD were normalized with regard to the individual whole cortical surface area and ThrR to whole brain averaged mean curvature. In this study, ThrA was set as 0.0004 * SurfaceArea; ThrD as 0.031 * SurfaceArea^0.542; and ThrR as 0.25 * AverageCurvature. Details to find optimal thresholds are described in Supplementary Methods - Merging thresholds for basin extraction and Supplementary Figure 2. Then, sulcal basins in both templates and individuals were extracted using the watershed flooding procedure with the normalized thresholds. The water flooding was terminated when the water reached the vertices with zero or higher mean curvature, and sulcal basins were defined strictly within sulcal regions. The segmented sulcal basins in five individual fetuses at increasing gestational age are shown in Figure 2B.

Figure 2. The watershed algorithm and sulcal basins.

(A) 2D schematic illustration of merging process. Red dots represent the deepest point of each basin. Merging process starts when two or more sulcal basins meet at the ridge point (yellow dot). The merging criteria is based on ridge height, basin area and distance between the deepest points. (B) Sulcal basins are mapped on the mean curvature map of left hemisphere of 5 subjects. The regions circumscribed by white lines represent sulcal basins, and the arrows indicate the junctions (red: calcarine and parieto-occipital sulci, yellow: precentral and superior frontal sulci, purple: precentral and inferior frontal sulci, and gray: postcentral and intraparietal sulci).

With normalized thresholds in the merging process, noise in the flooding process was eliminated and the basins were divided by gyri or buried gyri. Based on these characteristics, a sulcal basin was segmented as a part of a specific primary sulcus and assigned to a unique sulcal label. Therefore, we adopted basin-wise labeling to assign sulcal labels to each basin. However, there were some exceptions to their characteristics where two primary sulci are often connected without border structures (Ono et al., 1990), which is termed a junction. There are four junctions which are found in the human brain (Ono et al., 1990): calcarine+parieto-occipital sulci, postcentral+intraparietal sulci, precentral+superior frontal sulci, and precentral+inferior frontal sulci. In this study, junctional basins that span two sulci are identified by arrows in Figure 2B. Given the location of the junctional basins, the junctional basins may have 2 sulci. For these regions, it is inappropriate to apply the basin-wise labeling which assigns one label for each basin. Thus we provide an additional approach to divide them into two corresponding sulcal labels as described in detail next.

Spatial probability map of primary sulci on fetal brain templates

We generated spatial probability maps of the primary sulci from each fetal template surface, which when taken together represent the emergence, shape and distribution of primary sulci across all GW. In order to generate the spatial probability maps for the MTBL method, it is essential to manually define primary sulci on all templates. All sulcal basins on the templates were manually annotated to belong to one of 19 primary sulcal labels (Figure 3A, and Supplementary Figure 3). The primary sulcal labels were 1. Sylvian fissure, 2. Central sulcus, 3. Superior frontal sulcus, 4. Middle frontal sulcus, 5. Inferior frontal sulcus, 6. Precentral sulcus, 7. Postcentral sulcus, 8. Intraparietal sulcus, 9. Superior temporal sulcus, 10. Inferior temporal sulcus, 11. Occipito-temporal sulcus, 12. Collateral sulcus, 13. Orbital sulcus, 14. Olfactory sulcus, 15. Cingulate sulcus, 16. Subparietal sulcus, 17. Lateral occipital sulcus, 18. Calcarine sulcus, and 19. Parieto-occipital sulcus. In the fetal templates, the four junctions were detected at different GW. The junction of calcarine+parieto-occipital sulci was detectable on the 23 GW template (the earliest template used in this study), the junction of postcentral+intraparietal sulci was found on the 26 GW template, and the junctions of precentral+inferior frontal sulci and precentral+superior frontal sulci were found on the 27 GW template. Because of the merging criteria of the watershed procedure, the junctional basins often cannot be divided. In these cases in the templates, we manually delineated them into the two individual sulci following sulcal locations and directions (Ono et al., 1990). All the annotated and delineated sulcal labels on the templates, and description of dividing junctions are shown in Supplementary Figure 3.

Figure 3. Sulcal labels and probability maps on the templates.

(A) Sulcal labels manually assigned on 33 gestational week (GW) template. A colored region and its number correspond to each primary sulcus: 1. Sylvian fissure, 2. Central sulcus, 3. Superior frontal sulcus, 4. Middle frontal sulcus, 5. Inferior frontal sulcus, 6. Precentral sulcus, 7. Postcentral sulcus, 8. Intraparietal sulcus, 9. Superior temporal sulcus, 10. Inferior temporal sulcus, 11. Occipito-temporal sulcus, 12. Collateral sulcus, 13. Orbital sulcus, 14. Olfactory sulcus, 15. Cingulate sulcus, 16. Subparietal sulcus, 17. Lateral occipital sulcus, 18. Calcarine sulcus, and 19. Parieto-occipital sulcus. (B) Sulcal labels on 23, 28 and 33 gestational week (GW) templates. (C) Probability maps of Sylvian fissure (SF), central sulcus (CS), superior temporal sulcus (STS) and cingulate sulcus (CingS) of the 23, 28 and 33GW templates. Each label was smoothed with 10mm full-width-half-maximum kernel preserving maximum value as 1, and the smoothed labels served as probability map.

To consider the spatial pattern of sulci, we generated spatial probability maps (P) of each sulcal label on the templates (Figure 3B). By smoothing each sulcal label with a 10mm FWHM kernel and scaling the values to be between 0 and 1, we generated a pseudo-probability in each vertex (k) that had a probability of each sulcal label (l) in each template surface (t) (Figure 3C). Then, we aligned the spatial probability maps on the template surfaces to the individual’s surface using a surface registration technique (Robbins et al., 2004). We visually checked the reliability of the surface registration and confirmed that sulcal folding patterns were matched well between the template and individual cortical surfaces. The example of the surface registration performance is shown in Supplementary Figure 4.

Temporal weights for the templates

Since fetal brains show the dynamic changes of gyrification and there is variability of sulcal emergence across individuals, it may not be enough to select and use a single template from a specific GW for accurate sulcal labeling. Instead, the MTBL employed the spatial probability maps from all the templates with temporal weights to assign sulcal labels. The temporal weights were calculated by determining the similarity of gyrification between an individual and all template brains, and temporal weights were allocated to spatial probability maps from each of the templates. The weights indicated levels of contribution of each template to label fetal sulci.

To estimate the temporal weights, we first modeled the temporal changes of cortical folding complexity across the fetal brain templates. We quantify the folding complexity using the Gyrification Index (GI), which is defined as the areal ratio between the cortical surface and the external convex hull (Gaser et al., 2006; Harris et al., 2004; Schaer et al., 2006; Zilles et al., 1988). Because GI in the fetal brain increases nonlinearly (Clouchoux et al., 2012), a power function was fitted to GIs of the templates to model the temporal changes of gyrification:

$$y=a\cdot t^b+1$$ [Eq.1]

where y and t represented the GI and GW of each template, respectively. Assuming the initial brain is very smooth without any folding, we set the intercept as 1 (Figure 4A). Thus, the variables a and b in Eq. 1 were estimated by GIs of the templates. Goodness of fit (adjusted R² = 0.970) indicated higher reliability of the power function for fitting GI of the template surfaces than other functions such as polynomial (adjusted R² = 0.957) and exponential function (adjusted R² = 0.967). This fitting stage provided a curve representing the temporal change of GI in the fetal templates from 23 to 33GW. In order to standardize individual folding patterns on that of templates, we calculated t by inversely applying Eq.1 using the subject’s GI and called it gyrification age (Age_GI) (Figure 4A). Age_GI is a standardized GW of the individual folding pattern corresponding to the temporal changes of gyrification across the templates. Thus, we used Age_GI rather than GW to extract the appropriate information for each template. To consider the variations in timing of sulcal emergence within an individual, we allocated a temporal weight for each template based on Age_GI. Using a Gaussian function centered at the Age_GI, the temporal weight of each template (w_t) was calculated as follows:

$$w_t=e^{-(t-Ag{e}_{GI}{)}^2∕{\sigma }^2}$$ [Eq.2]

where t represented GW of the templates, and σ² indicated width of the Gaussian function, FWHM (Figure 4B). Eq.2 allocated the template closest to Age_GI to have the highest weight and with weights ranging from 0 to 1. FWHM is an important parameter of the MTBL to determine the level of contribution of each template. The wider the kernel, the more similar the weights distributed across the templates, whereas a narrow kernel would preferentially set high weights to the templates which were close to subject Age_GI (Figure 4B).

Figure 4. Estimation of gyrification age, and computation of temporal weights.

(A) Gyrification Index (GI) of the templates (blue dots) are fitted (red curve) by a power function. GI and GW of an individual surface (green pentagon) is plotted. The gyrification age of the individual is calculated by projecting the individual GI onto the curve (green arrows). Goodness of fit (adjusted R²) of the power function is 0.970. (B) Based on the gyrification age (green dotted line), a Gaussian kernel is generated with 1, 2, 3, and 4 gestational week (GW) of full-width-half-maximum (FWHM), and the temporal weights (gray dots) for each template of GW are computed. For example, red dots indicate that the different weights of the 26GW template from different FWHM.

In this paper, we tested four kernel widths $({\sigma }^2=\{{}^1∕_{4ln2},{}^4∕_{4ln2},{}^9∕_{4ln2},and{}^{16}∕_{4ln2}\})$, which represented FWHM = 1, 2, 3 and 4 GW, respectively (Figure 4B). Following the previous study that reported 1 or 2 weeks’ variations in chronology of gyrification (Garel et al., 2001), we expected the narrowest kernel could provide the most accurate labeling results.

Labeling sulcal basins and dividing junctions

For labeling sulci in each individual brain using the MTBL, we combined the temporal weights and spatial probability maps from all templates. The probability maps on template surfaces were aligned to each individual brain by a 2D surface-based registration technique (sphere-to-sphere warping method), proposed to search the optimal correspondence of vertices using fields of folding magnitude (Boucher et al., 2009; Robbins, 2004). Maximizing the folding similarities between two surfaces, the probability map of each template was transformed and mapped onto the individual’s cortical plate surfaces. Finally, each vertex (k) on the cortical plate surface had a corresponding 11-by-19 (the number of templates-by-sulcal labels) probability matrix (P), where P_{t,l} is the probability of k having label (l) as defined from template (t). Matrix P was cross-multiplied by the vector of weights (11-by-1), and then a weighted sum was computed for each label. Thus, each vertex had a final probability for each of the 19 sulcal labels. In order to assign a label for i-th basin, previously identified, the final probabilities of all vertices within i-th basin (B_i) were summed across labels (S_{i,l}), and the arg max_l S_{i,l} of this was the label assigned. For all basins in a subject, the procedure of the MTBL was described as an equation as follows,

$$S_{\{i,l\}}=\sum _{k\in B_i}\sum _{t=23}^{33}{w}_t\cdot P_{\{t,l,k\}}$$ [Eq.3]

This basin-wise labeling was an appropriate approach for most primary sulci in the fetal brain, since they appear as simple, small grooves on the smooth surface and are also isolated from each other. However, assigning one label to each basin may produce inaccurate results in cases of junctional basins. Since the junctional basins were located in the middle of two connected sulci, they might need to be separated and labeled as two adjacent sulci. The first step was to determine whether a labeled basin was junctional. We detected sulcal basins which were preliminarily assigned as a junctional sulcus and set them as a junctional basin. For a junctional basin (i), we calculated degree of adjacency between two junctional sulci (degree of adjacency [doa] = (max S_{i,l})/(Σ_l S_{i,l})). Assuming that the det of a junctional basin was lower than that of other labeled basins, i was classified as a junctional basin when det is lower than a threshold for dividing. We empirically determined the dividing threshold for our study as 0.7. Having identified the junctional basins to be re-labeled, a vertex-wise labeling approach was adopted to divide them. Eq.4 was applied to each vertex in the junctional basins, where the inner summation was constrained to the two adjacent sulci as follows:

$$S_{\{k,l\}}=\sum _{t=23}^{33}\sum _{l\in j}{w}_t\cdot P_{\{t,j,k\}}$$ [Eq.4]

where j represented one of a set of two adjacent sulci in the junctional labels (j = {{precentral, superior frontal}, {precentral, inferior frontal}, {postcentral, intraparietal}, {calcarine, parieto-occipital}}). Using this vertex-wise labeling approach, we finally divided junctions into two sulci by finding arg max_l S_{k,l}.

Experiments

We assessed the performance of the MTBL method by comparing it to two alternative approaches: a fully-vertex-wise approach, and a single template approach (Figure 1). The fully-vertex-wise approach is the independent labeling of each vertex (k) within any given basin area,

$$S_{\{k,l\}}=\sum _{t=23}^{33}{w}_t\cdot P_{\{t,j,k\}}$$ [Eq.5]

For each vertex, we assigned arg max_k S_{i,k} as a sulcal label. Like the MTBL, four FWHM (1 to 4 GW) were applied to the fully-vertex-wise approach. The single-template labeling approach used the spatial probability map from only a single template. We created two different single-template approaches. One employed the 33 GW template which contains all the primary sulci in the brain. The other approach used a template that was found by rounding the subject Age_GI up to the nearest template GW. For these single-template approaches, we set the temporal weight of the corresponding template to 1, and 0 for all remaining templates in Eq.3 and Eq.4. Both single-template methods were used with both basin- and fully-vertex-wise labeling approaches. Since there were four widths of weight function (FWHM = 1, 2, 3 and 4) in Eq.2 and two single-template methods were applied in each of the basin- and fully-vertex-wise approach, a total of twelve different labeling results for each subject were compared.

We also performed supplementary experiments to show the reliability of the approaches used in this paper - calculation of curvature, choice of smoothing kernel size of the probability map, and the estimation of Age_GI. (See Supplementary Methods - Basin extraction with different curvatures, Smoothing kernel sizes, and Errors of gyrification age, respectively).

Quantitative validation

The MTBL was applied to assign sulcal labels in 25 TD fetal subjects. The manual annotation of sulcal basins by a neuroradiologist (EY) and a neuroanatomist (LV) was used as gold standard for sulcal labeling. For junctional labels, the raters either manually divided them into two different sulci using SUMA (https://afni.nimh.nih.gov/Suma) or assigned a single label.

In order to quantitatively validate the accuracy of the labeling results, we adopted the Dice coefficients defined as the area overlap ratio between two sets of labeling results (Dice, 1945):

$$Dicecoefficient=2\cdot \frac{\mid A\cap B\mid }{\mid A\mid +\mid B\mid }$$ [Eq.6]

where A and B were the areas of the labeling results to be compared. The range of the Dice coefficient ranges from 0 to 1 with 0 indicating no overlapping and 1 reflecting identically overlapping labeling results, respectively. Inter-rater reliability of the manual annotations between two raters was assessed by the Dice coefficient. The labeling results by the MTBL were compared to the manual annotations using the Dice coefficients separately, and two Dice coefficients were averaged. In addition, labeling accuracies of the other methods (including single-template and vertex-wise methods) were calculated by the Dice coefficient.

To statistically evaluate the difference of the MTBL results from the single-template methods and fully-vertex-wise approaches, we compared labeling accuracies using two-way repeated measures Analysis of Variance (ANOVA). For each significant subset from ANOVA, post-hoc paired t-test was applied to determine statistically significant differences using Bonferroni-corrected p value. In addition, we statistically tested whether the labeling accuracies were biased by data properties such as imaging scanner and subject age. A permutation test (n= 10000) was performed to determine the difference of the labeling accuracies between the subjects from two scanners (20 subjects from Simens 3T and 5 subjects from Philips 1.5T scanner). To evaluate the association of our labeling accuracies with GW and Age_GI, Pearson’s correlation analysis was used.

Results

The average inter-rater reliability of manual annotations of the sulcal labels across subjects was 0.964 ± 0.016 (mean ± standard deviation, ranged from 0.935 to 0.993). Supplementary Table 1 showed the inter-rater reliability of each sulcus. The averaged labeling accuracies across the subjects showed that the accuracy of the MTBL was higher than that of single-template or vertex-wise methods (Table 1). The labeling accuracies of the subjects were not significantly different between two scanners (p= 0.543). Labeling accuracy was not significantly related to GW (r= 0.175, p= 0.402) or Age_GI (r= 0.242, p= 0.240) (Figure 5B).

Table 1.

Statistical comparisons of sulcal labeling accuracy between the proposed, single template, and vertex-wise methods.

Method	Multi-template				Single-template
	FWHM				Nearest	33 GW
	1	2	3	4
Basin-wise	0.958 ± 0.024	0.952 ± 0.027	0.946 ± 0.028	0.942 ± 0.028 a	0.934 ± 0.031 a	0.931 ± 0.031 a
Vertex-wise	0.948 ± 0.020*	0.941 ± 0.023* a	0.933 ± 0.024* a,b	0.928 ± 0.025* a,b,c	0.930 ± 0.030 a	0.920 ± 0.027 a

Data (Dice coefficient): mean and standard deviation across individuals. The four results of proposed method in this study, the MTBL (multi-template-based basin-wise labeling) were shown in row of Basin-wise and column of Multi-template.

^*:significantly lower than basin-wise approach (significance level of p= 0.008);

^{a, b, and c:}significantly lower than multi-template method with FWHM=1,2, and 3, respectively (significance level of p= 0.001).

Figure 5. The labeling results of left hemisphere of 5 subjects and age-related accuracy changes.

(A) labeling results. The parameters used in this figure are FWHM=1 and dividing threshold=0.7. The subjects and color codes of the sulci correspond to those of Figure 2 and Figure 3, respectively. (B) changes of the labeling accuracy by gestational week (yellow points) and gyrification age (light blue points). Our labeling accuracy has no significant age-related relationships (gestation week [r= 0.175, p= 0.402], gyrification age [r= 0.242, p= 0.240]).

The statistical analysis of labeling accuracies between basin-wise and vertex-wise approaches indicated that the basin-wise approach yielded consistently higher accuracy (F= 4.912, p< 0.001). Because there were six post-hoc comparisons, we set the significance level as 0.008. The results of post-hoc paired t-test showed that the MTBL showed significantly superior accuracies at all kernel widths compared to the multi-template-based vertex-wise labeling method (p< 0.001 for FWHM= 1, 2, 3 and 4). No significant differences between basin- and vertex-wise approaches were found for both single-template methods (subject-specific template: p= 0.068 and 33GW template: p= 0.084).

In the comparison among the four labeling accuracies of the multi-template methods (the MTBL and vertex-wise approach) with different kernel size and two accuracies from single-template methods, a significant difference was observed (F= 6.719, p< 0.001). The highest labeling accuracy was found with the MTBL (0.958 ± 0.024) when using the narrowest kernel width (FWHM= 1). The labeling results with wider kernels showed relatively lower accuracies in both basin- and vertex-wise approaches. For 30 post-hoc tests, the Bonferroni-corrected p was set as 0.001. Interestingly, the accuracies obtained by the single-template method were consistently lower than the multi-template-based labeling method. In the post-hoc t-test for the basin-wise approach, the MTBL with the narrowest kernel width (FWHM= 1) showed significantly higher accuracies than those with widest kernel (FWHM= 4: p< 0.001) and single-template methods (subject-specific template: p< 0.001 and 33 GW template: p< 0.001). Generally, the trend in the vertex-wise labeling results was similar to the trend in the basin-wise labeling results. Using the vertex-wise labeling method, the labeling accuracy with the narrowest kernel width was significantly higher than that with the other kernels (FWHM= 2: p< 0.001, FWHM=3: p< 0.001, and FWHM= 4: p< 0.001) and single-template methods (subject-specific template: p< 0.001 and 33GW template: p< 0.001). For visual demonstration of the MTBL with 1 GW of FWHM kernel width which achieved the highest accuracies, the labeling results of the left hemisphere of five fetuses (the same subjects as in Figure 2) are displayed in Figure 5. All of the results with the same kernel size for 25 subjects are shown in Supplementary Figure 5.

To further validate the results of basin-wise labeling, the accuracy was measured and compared for each sulcus (Figure 6). High accuracies (> 0.95) were consistently achieved in the Sylvian fissure, central sulcus, middle frontal sulcus, superior temporal sulcus, occipito-temporal sulcus, collateral sulcus, and cingulate sulcus by the MTBL with any FWHM value of the weight function. Fairly high accuracies (> 0.85) were shown in inferior temporal sulcus, subparietal sulcus, and lateral occipital sulcus using the MTBL. Orbital and olfactory sulci showed different trends in the averaged accuracy. In the orbital sulcus, the MTBL showed lower accuracy (ranging from 0.670 to 0.748) than the single-template methods (subject-specific template: 0.855, and 33GW template: 1.000). Fluctuating accuracies were found for the olfactory sulcus depending on kernel widths applied. The accuracy was 0.917 when using the narrowest kernel width, whereas other kernel widths (FWHM= 3 and 4) extremely low labeling accuracy (0.478). The averaged accuracy of the MTBL with 4 different widths was higher in 15/19 sulci than that of single-template methods. In junctional sulci, fairly high accuracies (> 0.85) were achieved by the MTBL in postcentral sulcus, intraparietal sulcus, calcarine sulcus, parieto-occipital sulcus, and superior frontal sulcus. Relatively low accuracies were found in the inferior frontal sulcus (ranging from 0.810 to 0.814), and precentral sulcus (ranging from 0.843 to 0.847). The accuracies by the single-template method were similar or lower compared to the MTBL in all junctional sulci.

Figure 6. The accuracy of each sulcal label by proposed and single template methods.

The boxed sulcal labels indicate junctional sulci. The following abbreviations are used to refer to primary sulci - SF: Sylvian fissure, CS: central sulcus, SFS: superior frontal sulcus, MFS: middle frontal sulcus, IFS: inferior frontal sulcus, PreCS: precentral sulcus, PostCS: postcentral sulcus, IPS: intraparietal sulcus, STS: superior temporal sulcus, ITS: inferior temporal sulcus, OTS: occipito-temporal sulcus, Cols: collateral sulcus, OrbS: orbital sulcus, OlfS: olfactory sulcus, CingS: cingulate sulcus, SPS: subparietal sulcus, LOS: lateral occipital sulcus, CalsS: calcarine sulcus, and POS: parieto-occipital sulcus.

The sulcus-level accuracy of vertex-wise labeling was also measured and the results are shown in Supplementary Figure 6. Overall accuracies of vertex-wise labeling were slightly lower than those of basin-wise labeling. For small or late-emerged sulci such as middle frontal, subparietal, lateral occipital, occipito-temporal, orbital, and olfactory sulci, large difference were found between accuracies of basin- and vertex-wise labeling.

In the reliability experiments, we found no dramatic difference in labeling accuracies with different curvature calculation methods (Supplementary Table 2). But some sulci in Supplementary Figure 7 (central sulcus in Subject #2 and inferior temporal sulcus in subjects #5) were not detected using Freesurfer-based curvature estimation. The results from varying the smoothing kernel size of the probability map (Supplementary Table 3) and estimation of Age_GI (Supplementary Table 4) demonstrated that the selected parameters in this paper showed the highest accuracy.

Discussion

In this study, we proposed a novel method, the MTBL, for assigning sulcal labels in the fetal brain and applied it to 25 TD fetal subjects. Based on the ground truths from two experts, the MTBL demonstrated overall superior accuracy when compared to alternative approaches - single-template and vertex-wise frameworks. There are three primary aspects of the MTBL method that enabled more accurate labeling results in the fetal brain: temporal weights, the basin-wise approach and the dividing of junctions.

The MTBL used multiple templates between 23 and 33GW, which is the critical period for cortical folding (Chi et al., 1977; Garel et al., 2001; Habas et al., 2012). With the temporal weights, each spatial probability map had different contributions to labeling an individual brain. To calculate the temporal weights, we chose Age_GI rather than GW to best select the spatial probability maps of the templates because it is a standard measure of the degree of folding in an individual brain. Thus we center the temporal weight kernel on the subject's Age_GI to allocate the appropriate weights. We tested several widths for the weight function and the most accurate labeling results were achieved by the narrowest width (FWHM = 1), with 1 to 2 templates nearest to the Age_GI mainly contributing toward label assignment (Figure 4B). This result is consistent with our expectation that there are on average 1- to 2-weeks' variation in the detectability of primary sulci in 25% to 75% of the fetal population using MR images (Garel et al., 2001). Using a kernel to allocate temporal weights accommodates the different timing of gyrification across individuals. From both subject- and sulcus-wise comparisons, the MTBL showed more precise labeling results compared to the single-template methods, demonstrating that allocating the temporal weights based on Age_GI and kernel widths is effective for achieving superior labeling results in the fetal brain.

The orbital and olfactory sulci showed low and diverse accuracies depending on the width of the weight function. This may be due to the small size of these two sulci and small number of subjects as the Dice coefficient is more susceptible to errors in such cases. These sulci are characterized by small folds that may not be detectable on the surfaces. For instance, the olfactory sulcus has been reported to emerge in early GW (<25 weeks) (Chi et al., 1977; Garel et al., 2001), but was not frequently detected on both template and individual surfaces. It can be partially explained by the low resolution and motion artifacts of the fetal MR images which cannot capture structural details in small cortical folds. In addition, labeling with the template nearest to the GW of the subject gave the same accuracies as using the narrowest kernel for the calcarine, subparietal and lateral occipital sulci. Whereas the time of detection of these sulci are highly variable (the calcarine sulcus, Garel et al., 2001), or as yet unknown, the highest accuracy obtained by using nearest template suggests that these sulci emerge at a very specific GW. To confirm this and obtain greater reliability in labeling these sulci, more fetal brains and precise surface extraction with high resolution MR images are necessary in future studies.

In addition to allocating temporal weights in the MTBL method, we employed a basin-wise approach rather than a vertex-wise approach to avoid errors caused by inter-subject variability in sulcal shape, length and orientation (Keller et al., 2007; Ono et al., 1990). To minimize the errors, we extracted sulcal basins on the surfaces and labeled them using basin-wise weighted probabilities. Our results demonstrated the basin-wise approach to have higher labeling accuracy than the vertex-wise approach. This can also be confirmed by visual inspection (Supplementary Figure 8).

Dividing junctions is a challenging issue not only in the fetal brain but also in the adult brain. Some previous studies have proposed to divide two connected sulci using the shape of the junctions (Rettmann et al., 2005; Yang and Kruggel, 2009). Yang and Kruggel, 2009 generated a skeleton of a junctional basin and pruned the skeleton using neighborhood information such as length and connectivity. Rettmann et al., 2005 manually selected seed points in the boundary between sulci before applying a curve tracking algorithm to the points to separate the sulci. However, applying these methods to fetal subjects has limitations since the orientations of the sulci are not clear in the fetal brain compared to those in the adult brain. Therefore, we selectively used a vertex-wise approach to automatically divide junctions into two sulci. The potential to separate junctions is demonstrated both quantitatively and visually from the labeling results of the postcentral, intraparietal, calcarine, and parieto-occipital sulci. However, precentral, superior frontal, and inferior frontal sulci showed relatively lower accuracies than other primary sulci. One explanation for this is the precentral sulcus and its two vertically connected frontal sulci (superior and inferior) having greater morphological diversity between individuals, especially in the inferior part of the precentral junction (Germann et al., 2005; Keller et al., 2007; Ono et al., 1990). This might also result in the low inter-rater reliability in three junctional sulci (see Supplementary Table 1). Without reliable ground truths of the precentral and inferior frontal sulci, the performance of the MTBL in these sulci may not be reliable. However, to evaluate the performance of junction division in precentral and inferior frontal sulcal regions, we tested the longitudinal reliability of the boundary between these two sulci (see supplementary Methods). The boundary of the precentral and inferior frontal sulci junction in the fetal brain showed a similar pattern with the manually annotated boundary in the corresponding neonatal brain scanned 130 days later (Supplementary Figure 9). This longitudinal test demonstrates that the MTBL has the potential to divide junctions reliably in the fetal brain despite the low accuracies in precentral and inferior frontal sulci. To obtain high reliability when dividing junctions, more longitudinal subjects need to be tested in future studies.

In our previous studies, we employed spectral sulcal basin matching technique for sulcal pattern analysis in fetuses (Im et al., 2017; Ortinau et al., n.d.). Since the spectral matching technique finds correspondence of sulcal basins between individuals and templates, it could also be used for sulcus labeling. However, unlike the MTBL approach, it cannot divide junctional basins and assign two sulcal labels. Thus some differences can be seen between techniques, especially in the junctions (Supplementary Figure 10). Although it is enough to use the spectral matching technique for sulcal pattern similarity analysis, the MTBL can provide more precise labeling results.

In this paper, we tested for possible bias in labeling accuracy with properties of the data such as scanner and age effects. The 25 subjects were scanned on two different scanners (3T Siemens and 1.5T Philips), which could lead to variations in image intensity range and tissue contrast. Our cortical plate segmentation was not an automatic procedure, which is sensitive to image intensity and tissue contrast. Here, we visually revised the semi-automatic segmentation results based on anatomical knowledge of the fetal brain in each slice of the acquired MR images. Moreover, the folding measurements seem to be less sensitive to variations in tissue contrast and the definition of tissue boundaries in MR images than other volumetric measurements such as cortical thickness (Tarui et al., 2018). Our results showed that there was no significant difference in labeling accuracy among the subjects from each scanner. This has also been addressed in our previous study where the ultrafast T2 sequence between the scanners acquired data with similar image quality (Tarui et al., 2018). We also found that the labeling accuracy was not related to both GW or Age_GI. Consequently, these results demonstrated that the labeling accuracy of MTBL was not biased by different MRI scanners or the complexity and size of the brain as it changes during gestation. Furthermore, we examined the reliability of several approaches used in this study. Between curvature estimation methods, sulcal regions and the labeling accuracy were similar to each other. But there might be under- or over-extraction of the sulcal basins because merging thresholds are inherently dependent on the features used in watershed algorithm. Therefore, the merging thresholds should be optimized before labeling sulcal basins that are extracted using different curvature estimation methods. In future, the differences in basin extraction and labeling accuracy by curvature estimation methods should be further analyzed. In terms of the chosen smoothing kernel size for computing probability maps, and our estimation of Age_GI, reliability experiments demonstrated that these parameters were highly reliable.

In conclusion, we proposed a novel method for labeling sulci in the human fetal brain that accounts for the highly dynamic spatio-temporal folding process in this period of development. We also applied a basin-wise labeling approach and divided junctional sulci. We demonstrated the efficacy of MTBL to assign sulcal labels on 25 individual fetal brains with high accuracy. The approaches used in MTBL can be reference knowledge for labeling fetal sulci, and the labeling results will enable future studies on morphological analysis in each sulcus in the typical and atypical fetal brain.