A novel approach to multiple anatomical shape analysis: Application to fetal ventriculomegaly

Abstract

Fetal ventriculomegaly (VM) is a condition in which one or both lateral ventricles are enlarged, and is diagnosed as an atrial diameter larger than 10 mm. Evidence of altered cortical folding associated with VM has been shown in the literature. However, existing works use a single scalar value such as diagnosis or lateral ventricular volume to characterize VM and study its relationship with alterations in cortical folding, thus failing to reveal the spatially-heterogeneous associations. In this work, we propose a novel approach to identify fine-grained associations between cortical folding and ventricular enlargement by leveraging the vertex-wise correlations between their growth patterns in terms of area expansion and curvature. Our approach comprises three steps. In the first step, we define a joint graph Laplacian matrix using cortex-to-ventricle correlations. The joint Laplacian is built based on multiple cortical features. Next, we propose a spectral embedding of the cortex-to-ventricle graph into a common underlying space where its nodes are projected according to the joint ventricle-cortex growth patterns. In this low-dimensional joint ventricle-cortex space, associated growth patterns lie close to each other. In the final step, we perform hierarchical clustering in the joint embedded space to identify associated sub-regions between cortex and ventricle. Using a dataset of 25 healthy fetuses and 23 fetuses with isolated non-severe VM within the age range of 26–29 gestational weeks, our approach reveals clinically relevant and heterogeneous regional associations. Cortical regions forming these associations are further validated using statistical analysis, revealing regions with altered folding that are significantly associated with ventricular dilation.

1. Introduction

During the intrauterine life, the fetal brain undergoes drastic maturational changes (Benkarim et al., 2017). Cortical folding is one of the major processes that occur during this period and manifests in substantial increases in area expansion of the cortical plate and considerable changes in curvature due to the sulco-gyral formation, with which the brain acquires its highly convoluted morphology (Habas et al., 2012; Clouchoux et al., 2012; Wright et al., 2014). The timing of the emergence of the sulci and gyri in the fetal brain is considered an accurate marker of neurodevelopment (Garel et al., 2003; Studholme and Rousseau, 2014), and any deviations from the normative baseline in the degree and formational pattern of cortical folding might lead to adverse postnatal outcome.

In prenatal ultrasound examination, ventriculomegaly (VM) is the most frequent abnormal finding in the fetal brain, with around 1% incidence rate (Salomon et al., 2007; Huisman et al., 2012). VM is a condition in which one or both lateral ventricles are dilated, as shown in Fig. 1 A. In clinical practice, it is diagnosed as an atrial diameter larger than 10 mm at any gestational age (Cardoza et al., 1988) and is further categorized as mild (< 12 mm), non-severe (< 15 mm) and severe (> 15 mm) VM. When no other abnormalities are present, it is called isolated VM. Postnatal prognosis in VM fetuses is generally dependent on the presence of other abnormalities and the degree of ventricular enlargement (Griffiths et al., 2010). Fetuses with severe VM, either isolated or not, are at high risk of adverse neurodevelopmental outcome (Benkarim et al., 2018), whereas subjects with isolated non-severe VM (INSVM) are generally associated with good outcomes and only a small number with abnormal outcomes. For the latter subgroup, the atrial diameter is no longer a reliable prognostic marker (Melchiorre et al., 2009; Beeghly et al., 2010). Evidence of alterations in the cortical sheet associated with in utero VM has already been shown by studies in the literature. Based on volumetry, Kyriakopoulou et al. (2014) found that cortical gray matter was significantly enlarged in fetuses with isolated VM. Using curvature-based analysis, Scott et al. (2013) found reduced cortical folding in a small region around the parieto-occipital sulcus in fetuses with isolated mild VM. In (Benkarim et al., 2018), fetuses with isolated non-severe VM showed delayed cortical folding in the insula, the occipital lobe and the posterior part of the temporal lobe.

Fig. 1.

A: Cortical and ventricular surfaces of a 28 gestational weeks fetus with left VM. B: Regions of the lateral ventricle.

To study the associations between VM and cortical folding alterations, existing works either use diagnosis or ventricular volume to characterize this condition. Although ventricular volume captures the extent of enlargement and is more distinctive than the dichotomous information offered by diagnosis (Gholipour et al., 2012), a single scalar value might not be sufficient to provide all the important information related to ventricular enlargement (e.g., spatial information about the dilated ventricular regions). In this work, we aim to find associations between ventricular regions (see Fig. 1 B) and cortical folding by incorporating into our analysis the ventricular surfaces. For this purpose, we propose a novel approach based on manifold learning to jointly analyze the cortical and ventricular shapes using their growth patterns. The motivation of joint analysis of ventricle and cortex is because their development is highly associated, since neurons produced around the ventricle migrate towards the cortex (Clarke, 2007).

Manifold learning is a non-linear dimensionality reduction technique used to find a meaningful low-dimensional representation hidden in the original ambient space based on the relationships between the data. Approaches within this category such as Laplacian eigenmaps (Belkin and Niyogi, 2003) and diffusion maps (Coifman and Lafon, 2006) are spectral methods that start by building an affinity or similarity matrix from the original dataset, which is used to derive the graph Laplacian and the diffusion operator, respectively. Then, they proceed to solve the generalized eigenvalue problem to find the aforementioned low-dimensional representation. These approaches, however, were designed to work with a single dataset at a time. To work with multiple datasets simultaneously, manifold alignment can be used. Manifold alignment performs a simultaneous embedding of multiple datasets to unravel the relationships between them by finding a common underlying representation. The main challenge of these approaches is to establish correspondences between heterogeneous datasets in the ambient space. In our case, between the cortical and ventricular shapes. Several approaches have been proposed for problems with unknown (Wang and Mahadevan, 2009b; Baumgartner et al., 2013) and known correspondences (Zhai et al., 2010). Other works circumvent this problem using a two-step approach by embedding different datasets individually and then performing the alignment of the embedded data using, for example, Procrustes alignment (Wang and Mahadevan, 2008). Working with cortical surfaces, Lombaert et al. (2013) proposed a spectral approach for surface matching by individually embedding each surface and then finding the correspondences. This approach was further extended to embedding multiple surfaces in (Wright et al., 2015) to build a spatio-temporal cortical surface atlas of the fetal brain. These approaches however employ the spatial coordinates of the surfaces to perform the embedding. In our work, in contrast, we leverage the growth patterns of both cortical and ventricular shapes to jointly embed the data.

The main idea of our approach is to find a common underlying representation of the vertex-wise growth patterns of both cortical and ventricular surface shapes such that vertices with associated patterns from both anatomical surfaces can lie close to each other. In this way, regional associations can be conveniently found by identifying clusters containing vertices from both surfaces in the new latent space. The contributions of our work are threefold:

• We propose a novel approach for joint analysis of different anatomical shapes based on their growth patterns.

• We identify, for the first time, spatially fine-scaled associations related to in utero VM between ventricular surfaces and alterations in cortical folding.

• We use fusion of similarity matrices to capture associations based on multiple cortical features.

This work is an extension of our conference paper (Benkarim et al., 2018b). In this current work, we present an improved version of the proposed approach, considering the sign of the correlations to identify both positive and negative INSVM-related associations, provide a more extensive description of the methodology, evaluate our approach with both ipsilateral and contralateral associations, use more cortical and ventricular features, validate the obtained associations using statistical analysis, and include a thorough discussion of the results.

The remainder of this paper is organized as follows. Section 2 presents the details of our proposed approach. In Section 3 we describe the experimental setup and present results and validation. Section 4 is devoted to discussing the advantages and limitations of our approach and the potential clinical implications of our findings. Finally, Section 5 concludes the paper.

2. Methodology

In this section, we provide a description of the proposed approach to jointly analyze the relationships between cortical and ventricular shapes. The pipeline of our approach is shown in Fig. 2. In the first stage, we build one or more joint similarity matrices using cortex-to-ventricle correlations based on several features. Next, we fuse these single matrices into a common joint similarity matrix. Then, we propose a spectral embedding of the cortex-to-ventricle graph into a common underlying space, where its nodes are projected according to the joint ventricle-cortex growth patterns. In this low-dimensional joint ventricle-cortex space, associated growth patterns lie close to each other. Finally, hierarchical clustering is used to find these associations.

Fig. 2.

Pipeline of the proposed approach.

2.1. Joint spectral embedding

Given P subjects and their corresponding cortical and ventricular surfaces, each with N_c and N_v vertices respectively, the growth patterns x_i for each vertex are represented by:

$${\mathbf{x}}_i=\left[x_i^1,x_i^2,\dots ,x_i^P\right],$$ (1)

where $x_i^k$ is the feature (e.g., local surface area) of the kth subject at vertex i. Cortical and ventricular growth patterns can be represented using distinct features (e.g., we can use local area for ventricles while curvature for cortices). These features should ultimately be sensitive and able to capture the putative alterations in cortical folding related to the dilation of the lateral ventricles. The area of the ventricular surfaces increases dramatically with the enlargement of the lateral ventricles and local area can be considered to be an important feature in capturing this dilation. For the cortices, local area could, analogously, offer potential sensitivity in the detection of cortical folding deviations. Furthermore, curvature constitutes a different cortical feature that might provide complementary information to the aforementioned feature about ventricular dilation and gyrification, and might be specially susceptible to changes in the convolutions along the cortical surface.

Note that in order to build the growth patterns for a particular feature, surfaces from each shape need to have the same number of vertices and be in anatomical correspondence. In this study, growth patterns represent the anatomical variation in a cross-sectional dataset. Although it is preferable to use a longitudinal dataset to build the growth patterns, repeated in utero imaging is difficult due to both ethical and practical issues (Benkarim et al., 2017). Nevertheless, cross-sectional datasets are commonly used in the literature to build spatio-temporal atlases of the fetal brain (Habas et al., 2010; Serag et al., 2012; Wright et al., 2015).

In this work, we assume that there exists a common underlying representation for these heterogeneous growth patterns, x_i, such that vertices of associated regions from both surfaces can lie close to each other and, most likely, form dense clusters. Thus, we propose to find a shared representation of cortical and ventricular growth patterns by jointly projecting them onto a common space:

$$\begin{gathered} Y=\underset{Y}{\mathrm{argmin}}\sum _{i,j}{\Vert Y_i^c-Y_j^c\Vert }^2{S}_c(i,j)+\sum _{i,j}{\Vert Y_i^v-Y_j^v\Vert }^2{S}_v(i,j) \\ +\mu \sum _{i,j}{\Vert Y_i^c-Y_j^v\Vert }^2{S}_{cv}(i,j), \end{gathered}$$ (2)

where Y = [Y^c, Y^v]^T is the common latent representation with N = (N_c + Nv) rows such that the first N_c rows correspond to the embedded cortical growth patterns (i.e., Y^c ) and the remaining N_v rows belong to the ventricle (i.e., Y^v), T stands for matrix transpose, S_c and S_v are the intra-structure similarity matrices for cortices and ventricles respectively, S_cv is the similarity matrix between cortical and ventricular growth patterns, and μ is a trade-off parameter. Given two associated (i.e., high S_cv(i, j)) growth patterns, ${\mathbf{x}}_i^c$ and ${\mathbf{x}}_j^v$ from cortex and ventricle respectively, the third term in Eq. (2) enforces their projections (i.e., $Y_i^c$ and $Y_j^v$) to fall close to each other. This also occurs for similar growth patterns within the same shape (enforced by the first and second terms for cortices and ventricles, respectively).

Since we are interested in identifying associations between the growth patterns of both structures, similarity between the growth patterns is defined in terms of correlation. First, we build the inter-structure similarity matrix based on the absolute value of the correlations between the growth patterns of both surfaces as follows:

$$S_{cv}(i,j)=|\rho ({\mathbf{x}}_i^c,{\mathbf{x}}_j^v)|,$$ (3)

where ρ is Pearson’s correlation coefficient. Similarly, intra-structure similarity matrices (S_c and S_v) are built to capture within shape associations only using positive correlations. For the cortex, for example, we have:

$$S_c(i,j)=\max (\rho ({\mathbf{x}}_i^c,{\mathbf{x}}_j^c),0).$$ (4)

We use the absolute value of Pearson’s correlation coefficient for the inter-structure similarity matrix because we attempt to find both positive and negative associations between the cortical and ventricular shapes. While, for the intra-structure similarity matrices, we only encourage vertices to be close to each other in the new latent representation when their growth patterns are positively correlated.

Once we have our inter- and intra-structure similarity matrices, the joint similarity matrix $\mathcal{J}\in {ℝ}^{N\times N}$ is constructed by filling its block-diagonal with the intra-structure matrices and the off-diagonal with the inter-structure matrix as follows:

$$\mathcal{J}=\left(\begin{array}{cc}{S}_c& \mu S_{cv}\\ \mu S_{cv}^T& S_v\end{array}\right).$$ (5)

Then, the normalized graph Laplacian is computed using the joint similarity matrix:

$$L=I-D^{-1/2}\mathcal{J}{D}^{-1/2},$$ (6)

where D denotes the degree matrix of $\mathcal{J}$ (i.e., a diagonal matrix such that $D(i,i)={\sum }_j\mathcal{J}(i,j)$), and I is the identity matrix. Laplacian eigenmaps (Belkin and Niyogi, 2003) can then be used to solve Eq. (2) based on the joint graph Laplacian and find the common underlying space Y.

Eq. (2) can be only used with two different shapes. However, our approach can be generalized to work with more than two shapes by extending the joint similarity matrix defined in Eq. (5) to include the additional intra- and inter-stucture similarity matrices (Wang and Mahadevan, 2009a). Therefore, Eq. (2) can now be formulated as follows:

$$\begin{gathered} Y=\underset{Y}{\mathrm{argmin}}\sum _k\sum _{i,j}{\Vert Y_i^k-Y_j^k\Vert }^2{S}_k(i,j) \\ +\mu \sum _{k,l,k>l}\sum _{i,j}{\Vert Y_i^k-Y_j^l\Vert }^2{S}_{k,l}(i,j), \end{gathered}$$ (7)

where k and l are indices of the different shapes, S_k (i, j) is the similarity between the ith and jth growth patterns of kth shape (i.e., intra-structure similarity), and S_k,l(i, j) the similarity between the ith growth pattern of the kth shape and jth growth pattern of lth shape (i.e., inter-structure similarity).

2.2. Clustering-based associations

To discover the regional relationships induced by ventricular enlargement, the embedded growth patterns Y are grouped using agglomerative hierarchical clustering with unweighted average linkage (UPGMA) based on Euclidean distances. Initially, the algorithm considers all points in Y as individual clusters, and then proceeds iteratively by merging at each step the two clusters with the shortest distance until there is a single cluster that cointains all points. Given a pair of clusters, $\mathcal{A}$ and $\mathcal{B}$, UPGMA defines the distance between the clusters as the average of the (Euclidean) distances between all the points in both clusters:

$$d(\mathcal{A},\mathcal{B})=\frac{1}{\left|\mathcal{A}\right|\left|\mathcal{B}\right|}\sum _{u\in \mathcal{A},v\in \mathcal{B}}\Vert u-v{\Vert }_2,$$ (8)

where | · | denotes cardinality and $\Vert \cdot {\Vert }_2$ is the l₂-norm.

Associated regions are then identified by the clusters that gather vertices from both cortical and ventricular shapes. Hierarchical clustering was chosen instead of other approaches (e.g., K-means) since fetal cortical growth patterns have shown to follow a hierarchical organization (Xia et al., 2019). In their work, using a clustering approach with no hierarchical constraints, Xia et al. (2019) found that fetal cortical parcellations based on growth patterns form a hierarchical organization of the growth patterning, with emerging clusters confined within the boundaries of their ancestors. Using hierarchical clustering for our associations means that, as the number of clusters increases, regions with associations will produce other associations or shrink (and even disappear), but always preserve a spatial consistency as we traverse the hierarchy.

2.3. Multiple features for growth patterns

Using a single feature to describe cortical folding may be insufficient to accurately capture all the subtle folding alterations associated with ventricular enlargement. For the ventricles, although the area of the surfaces increases dramatically with the enlargement of the lateral ventricles and can be considered a reliable feature in capturing this dilation, other features, such as curvature, may capture different aspects of the dilation. We, therefore, propose to extend our approach to include multiple distinct features (e.g., local area, curvature). To do so, we use the similarity network fusion approach proposed by Wang et al. (2014). First, we build a similarity matrix based on the correlation of each of the cortical features with a given ventricular feature. Then, information from each individual similarity matrix is diffused in an iterative process until convergence. For the simple case of two similarity matrices (corresponding to two different cortical features), ${\mathcal{J}}_m,m\in \{1,2\}$, we first derive a dense P_m and a sparse W_m kernel matrix as follows:

$$P_m(i,j)=\{\begin{array}{ll}\frac{{\mathcal{J}}_m(i,j)}{2{\sum }_{k\ne i}{\mathcal{J}}_m(i,k)}\hfill & i\ne j\hfill \\ 1/2\hfill & \text{otherwise.}\hfill \end{array}$$ (9)

$$W_m(i,j)=\{\begin{array}{ll}\frac{{\mathcal{J}}_m(i,j)}{2{\sum }_{k\in {\mathcal{N}}_i}{\mathcal{J}}_m(i,k)}\hfill & j\in {\mathcal{N}}_i\hfill \\ 0\hfill & \text{otherwise,}\hfill \end{array}$$ (10)

where ${\mathcal{N}}_i$ denotes the neighborhood of the ith vertex in terms of the vertices with most correlated growth patterns. Fusion is then iteratively conducted for each of joint similarity matrices (i.e., ${\mathcal{J}}_1$ and ${\mathcal{J}}_2$) by updating each dense kernel based on its corresponding sparse kernel and the dense kernel of the other joint similarity matrix:

$$P_1^{t+1}=W_1{P}_2^t{W}_1^T,P_2^{t+1}=W_2{P}_1^t{W}_2^T,$$ (11)

where t indexes the dense kernel in the current iteration. In this way, the reliable similarity information is iteratively diffused across the two matrices. To work with multiple features, the diffusion process defined in Eq. (11) can be extended to n > 2 joint similarity matrices as follows:

$$P_m^{t+1}=W_m\left(\frac{{\sum }_{l\ne m}{P}_l^t}{n-1}\right)W_m^T,m=1,\dots ,n.$$ (12)

After convergence, the resulting dense kernels P_m are averaged to obtain the final joint similarity matrix:

$${\mathcal{P}}_f=\frac{1}{n}\sum _m{P}_m.$$ (13)

This fused similarity matrix, ${\mathcal{P}}_f$, is able to capture both common and complementary associations, and remove spurious and isolated correlations. Then, instead of $\mathcal{J}$, ${\mathcal{P}}_f$ is used to compute the joint Laplacian and project the growth patterns.

3. Experiments

In this section, we present the evaluation of our proposed approach in the identification of associations between cortical folding alterations and ventricular dilation, provide a comparison of the associations found using only a single feature and multiple cortical features after fusing the corresponding similarity matrices, and statistically assess these associations.

3.1. Dataset

In our experiments, we used a fetal brain MRI dataset of 25 controls and 23 subjects from a cohort within a research project on congenital isolated ventriculomegaly (8 subjects with left, 10 with right, and 5 with bilateral VM). Pregnant volunteers with healthy and normally grown fetuses of singleton pregnancies composed the control group and were prospectively enrolled specifically for the research purposes of this project. Approval was obtained for the study protocol from the Ethics Committee of the Hospital Clínic in Barcelona - Spain (HCB/2014/0484) and all patients gave written informed consent. Ages of the included subjects range between 26 to 29.3 gestational weeks. T2-weighted MR imaging was performed on a 1.5-T scanner (SIEMENS 105 MAGNETOM Aera syngo MR D13; Munich, Germany) with a 8-channel body coil. All images were acquired without sedation and following the American college of radiology guidelines for pregnancy and lactation. Half Fourier acquisition single shot turbo spin echo (HASTE) sequences were used with the following parameters: echo time of 82 ms, repetition time of 1500 ms, number of averaging = 1, 2.5 mm of slice thickness, 280 × 280 mm field of view and voxel size of 0.5 × 0.5 × 2.5 mm. For each subject, multiple orthogonal acquisitions were performed: 4 axial, 2 coronal and 2 sagittal stacks. Brain location and extraction from 2D slices was carried out for each of the stacks in an automatic manner using the approach proposed by Keraudren et al. (2014), followed by high-resolution 3D volume reconstruction with an isotropic voxel size of 0.75 mm using the method in (Murgasova et al., 2012).

3.2. Preprocessing

Reconstructed high-resolution images were segmented into the main tissues using the ensemble approach proposed in (Sanroma et al., 2016). Briefly, this method combines the initial probability maps produced by joint label fusion (Wang et al., 2013) and a regional learning-based approach based on support vector machines, calibrating their contributions by their corresponding weight maps, which are learned offline using a leave-one-out strategy. Ground-truth segmentations were obtained for the following structures: extracerebellar cerebro-spinal fluid (CSF), cortical gray matter (CoGM), white matter (WM), lateral ventricles (LV), cerebellum (CB) and brain stem (BS). For the ground-truth, first, 4 subjects were manually segmented by two expert raters. Then, the remaining subjects were segmented using the automatic method by Sanroma et al. (2016) and the automatic segmentations were manually corrected by the same expert raters. Segmentation performance in terms of Dice overlap on the 4 manually-segmented subjects for CSF, CoGM, WM, LV, CB and BS is 0.925 ± 0.019, 0.847 ± 0.032, 0.962 ± 0.015, 0.883 ± 0.025, 0.925 ± 0.011, and 0.896 ± 0.008, respectively.

For the cortices, we used the inner cortical surface instead of the outer surface since the boundary between the WM and the cortex is more stable and less prone to segmentation errors due to partial volume effects than the interface between the cortex and the CSF. Both WM and LV masks were smoothed using a 2 mm full width at half-maximum Gaussian kernel and cortical and ventricular surface meshes, for each hemisphere, were then reconstructed with the marching cubes algorithm (Lorensen and Cline, 1987). In order to establish vertex-wise correspondences between subjects, for each structure, surfaces were co-registered and thus were resampled to the same number of vertices per structure (Li et al., 2015; Xia et al., 2019), with 5000 and 1000 vertices for cortices and ventricles, respectively. The spherical demons approach proposed by Yeo et al. (2010) was used to co-register the cortices and ventricular surfaces were aligned using Deformetrica (Durrleman et al., 2014).

3.3. Experimental setup

Cortical growth patterns were estimated using three different features: local area (LA), mean curvature (MC) and curvedness index (CI). For the ventricles, we used LA and CI. Local area was computed as one third of the total area of adjacent triangles (Li et al., 2013). Mean curvature (defined as (k₁ + k₂)/2, where k₁ and k₂ are the principal curvatures) measures folding created without distortion, and curvedness index (i.e., $\sqrt{\left(k_1^2+k_2^2\right)/2}$) quantifies the degree of deviation from a flat plane (Koenderink and van Doorn, 1992).

Thus, we conducted 4 different experiments for each ventricular feature. For ventricular area, for example, we analyzed: 1) correlations between ventricular local area and cortical local area, 2) ventricular local area and cortical curvedness, 3) ventricular local area and mean curvature, and 4) fusing all three individual similarity matrices (i.e., using local area, mean curvature and curvedness index for the cortices and local area for the ventricles). These experiments were conducted for both ipsilateral and contralateral associations. For clustering, we used 2 to 30 clusters to illustrate the number of correlated regions identified and how they change with different numbers of clusters. The optimal associations between ventricles and cortices were determined by finding the most appropriate number of clusters using an internal validation index. The choice of the validation index is not trivial. In this work, the silhouette coefficient was chosen empirically after comparing with other indices such as elbow or Dunn indices, although no important differences were observed. Given a clustering configuration Π, the silhouette coefficient for the growth pattern i assigned to cluster A ϵ Π (i.e., i ϵ A ) is defined as:

$$s(i)=\frac{b(i)-a(i)}{max(a(i),b(i))},$$ (14)

where a(i) = d(i, A\{i}) and $b(i)={\min }_{B\in \mathrm{\Pi }\backslash A}d(i,B)$, with d(i, C) denoting the mean distance of the ith point to all points in cluster C ϵ Π. The optimal clustering (and, therefore, associations) was then chosen based on the number of clusters with the highest average silhouette coefficient over all cortical and ventricular vertices.

Associations corresponding to the optimal clustering were further validated using a general linear model (GLM). Since each association is composed of cortical and ventricular regions, we tested if our approach was able to accurately identify cortical regions with altered folding related to INSVM. Thus, we analyzed if these cortical regions were associated with ventricular volume using the following GLM:

$$F_c={\beta }_0+{\beta }_{1}GW+{\beta }_{2}STV+{\beta }_{3}VV,$$ (15)

where F_c corresponds to one of our features (e.g., mean curvature) averaged over all vertices in the cortical region belonging to some identified association, GW is gestational age in weeks, STV denotes supratentorial volume, and VV represents ventricular volume. Supratentorial volume was computed as a sum of the volumes of WM, CoGM and LV). Volumes for each of these tissues were computed based on the volumetric segmentations with an isotropic voxel size of 0.75 mm. Because the fetal brain becomes continuously more convoluted as growth proceeds, gestational age was incorporated into our model to control for the effect of age on the analyzed features. Furthermore, supratentorial volume was also included in our model as a covariate to account for scaling differences (Shimony et al., 2016) because our curvature-based features (i.e., mean curvature and curvedness index) depend on brain size (Rodriguez-Carranza et al., 2008).

3.4. Results

Ipsilateral associations between ventricular dilation and alterations in cortical folding identified by our approach in both left and right hemispheres are displayed in Figs. 3–4 for each of the single similarity matrices built based on ventricular local surface area and curvedness index, respectively. Associations found using the fused similarity matrices are shown in Fig. 5. For each similarity matrix, including the fused ones, associations are shown for different numbers of clusters, i.e., 3, 8, 15 and 20 clusters. Surfaces are displayed such that cortical and ventricular regions found to be associated are depicted with the same color code, while all other clusters that only contain vertices from a single shape (i.e., no association) are depicted in white. Note that, although surfaces in left and right hemispheres may have the same color, it does not indicate that there is an association between regions across hemispheres since each hemisphere was analyzed separately. This is only used for visualization purposes to illustrate, when possible, if we find similar associations bilaterally.

Fig. 3.

Identified ipsilateral cortex-ventricle associations based on correlations between ventricular local area and a) cortical curvedness, b) mean curvature and c) local area. Associations are color-coded, with white depicting no associations, and shown in lateral and medial views for each hemisphere separately.

Fig. 4.

Identified ipsilateral cortex-ventricle associations based on correlations between ventricular curvedness and a) cortical curvedness, b) mean curvature and c) local area. Associations are color-coded, with white depicting no associations, and shown in lateral and medial views for each hemisphere separately.

Fig. 5.

Ipsilateral associations found using the fused similarity matrices based on a) ventricular local area and b) curvedness. Associations are color-coded, with white depicting no associations, and shown in lateral and medial views for each hemisphere separately.

From these results, regardless of the cerebral hemisphere and the similarity matrix used to find the associations, we can observe that, in general, the posterior part of the lateral ventricles appears to be more associated with cortical regions than the anterior part (i.e., ventricular body and anterior horn). This gets more noticeable as the number of clusters increases. Moreover, the extension of ventricular and cortical regions is shrinked as the number of clusters increases and we obtain more localized and fine-grained associations (i.e., shared clusters), which emphasizes the strength of the associations remained.

With curvedness index describing cortical folding and ventricular local area, as shown in Fig. 3 a, we can highlight two associations common to both hemispheres: 1) between the posterior horn and posterior part of the occipital lobe (dark green), and 2) between the inferior horn and the medial part of the occipital lobe, including the parieto-occipital and calcarine sulci (in red). With mean curvature (see Fig. 3 b), the most consistent associations are between the posterior horn and the occipital lobe, in a large area in the left hemisphere whereas just a small region of this lobe appears to be associated with this horn in the right hemisphere, and the second association, which manifest bilaterally, is between the inferior horn and two different cortical regions: 1) a part of the parieto-occipital sulcus and 2) a small region around the superior temporal gyrus. In Fig. 3 c, when using local surface area for both shapes, we can see that associations between cortical and ventricular surfaces are mainly confined to the posterior horn and occipital lobe (dark green) and between the inferior horn and around the posterior part of the parietal lobe (in red). It is worth noting that, regardless of the cortical feature, the atrium is found in several associations, specially in the left hemisphere, although they are not consistent. When using curvedness index to characterize the ventricular growth patterns (see Fig. 4), we find fewer but stronger associations. The most recurrent association, irrespective of the cortical feature, is found between the posterior horn and the occipital lobe, with the exception of mean curvature in the left hemisphere, where the inferior horn is correlated with a small region of the central sulcus.

Still, using a single feature to describe the growth patterns might not be able to capture all putative associations or give rise to spurious ones. When using the fused similarity matrix, we obtain fewer associations if we compare with the results produced when using each cortical feature separately. This might suggest that fusion allows us to get rid of spurious and noisy associations. In Fig. 5, we can see that with increasing the number of clusters, ventricular regions forming associations are confined to the posterior part of the ventricle for both ventricular features. In particular, we have two associations common to both hemispheres when using ventricular LA: 1) between the posterior horn and the occipital lobe, with the right association including the whole calcarine sulcus and most of the parieto-occipital sulcus (dark green), and 2) between the inferior horn and the posterior part of the parietal lobe. Again, the right association also includes the superior temporal gyrus (in red). In the left hemisphere, analogously to the associations found with the individual cortical features, there are two additional associations: one in blue between the anterior horn and a small region in the medial part near the anterior cingulate gyrus, and another one depicted in green between the atrium and the superior part of the frontal lobe. With ventricular curvedness, only one single association is found (between the posterior horn and posterior part of the occipital lobe) across all clusters.

Fig. 6 displays the optimal associations in terms of the silhouette scores, as shown in Fig. 7, for each of the different similarity matrices used to obtain the embedding and each ventricular feature. For a better interpretation of these results, color codes assigned to each association with the names of the corresponding cortical and ventricular regions involved in the association are reported in Table 1. The best number of clusters, according to the silhouette coefficient, is around 12–15 clusters for all individual combinations of cortical and ventricular features, with 20 clusters achieving the highest silhouette score when using the fused similarity matrix based on local area and 13 clusters when using the fused matrix based on curvedness. It is worth noting here that silhouette scores might change even when associations remain the same, as other clusters are divided or merged within the same shape but not illustrated, since clusters with no associations are depicted in white.

Fig. 6.

Comparison of optimal ipsilateral associations, in terms of average silhouette coefficient, identified using single similarity matrices (i.e., curvedness index (CI), mean curvature (MC) and local area (LA)) and fused similarity matrix (FU) for both ventricular local area (top) and curvedness (bottom). Associations are color-coded, with white depicting no associations, and shown in lateral and medial views for each hemisphere separately.

Fig. 7.

Silhouette scores in the left (blue) and right (orange) hemispheres for the different single similarity matrices and the fused matrices built ipsilaterally based on ventricular LA (top) and CI (bottom).

Table 1.

Color codes for most common associations from the optimal clustering with the approximate regions from both cortical and ventricular shapes. Associations with blue and pink colors show no consistency.

	Ventricle	Cortex
	Posterior horn	Occipital lobe
	Inferior horn	Posteior part of parietal lobe
	Atrium	Frontal lobe
	Anterior horn	-
	Central part/body	-

Table 2 reports the p-values and the goodness-of-fit (R²) obtained with the GLM in Eq. (15) for each of the cortical regions within the associations corresponding to the optimal clustering as shown in Fig. 6. For ventricular LA and curvedness (i.e., CI) of the cortical region (depicted in red) composed of the lateral part of the occipital lobe and part of parietal lobe is significantly associated with ventricular volume in both hemispheres. No statistical significance is found with the other regions. With mean curvature and ventricular local area, we find four regions to be significantly associated with ventricular volume in the left hemisphere, while only one region shows statistical significance in the right hemisphere (red region including an area in the parieto-occipital sulcus and part of superior temporal gurys). We also found a significant differences provided by the association between mean curvature with ventricular curvedness (red region in the central sulcus). With local area for the cortices, there is one single region (coming from ventricular LA) showing statistical significance and only in the right hemisphere. No statistical significance was found with the cortical regions corresponding to the associations with ventricular CI. When using a single feature to characterize cortical folding, the identified associations do not give rise to cortical regions with significant altered folding. However, as reported in the second part of Table 2, fusing the different cortical features seems to accurately identify the cortical regions with altered folding related to ventricular enlargement. For ventricular LA, with the exception of the region in the superior part of the frontal lobe (depicted in green) in the left hemisphere, all remaining cortical regions are significantly associated with ventricular volume. Nonetheless, noteworthy is the fact that the average curvedness index of these regions shows no difference. The same occurs when using ventricular CI, the curvedness of the region in the occipital lobe (dark green) does not capture significant differences. This is better illustrated in Fig. 8, where we can see no clear difference between these regions in healthy controls and subjects with INSVM. This may suggest that this curvature feature does not capture accurate associations, which are then discarded during the fusion process. On the other hand, average mean curvature and total area of the remaining cortical regions (i.e., occipital lobe in dark green and the posterior part of the parietal lobe in red) are significantly associated with ventricular volume. Analogously, for ventricular LA, mean curvature and total area show significant differences in the occipital lobe, although the former is not significant in the right hemisphere.

Table 2.

P-values and goodness-of-fit using R² of ipsilateral associations captured by the different single similarity matrices (i.e., cortical curvedness index (CI), mean curvature (MC) and local area (LA)) with ventricular LA and CI, corresponding to the optimal clustering in terms of silhouette coefficient for left and right hemispheres. Each row is color-coded with the color corresponding to its association (see Table 1).

			Ventricular LA				Ventricular CI
			Left		Right		Left		Right
		Asso	p	R2	p	R2	p	R2	p	R2
Single	CI		0.381	0.804	0.267	0.729	0.147	0.561	0.144	0.528
			<10−5	0.729	<10−5	0.757	-	-	-	-
			0.297	0.770	-	-	-	-	-	-
			0.691	0.507	-	-	-	-	-	-
	MC		0.009	0.781	0.790	0.394	-	-	0.129	0.376
			<10−6	0.552	<10−4	0.587	0.026	0.594	-	-
			0.007	0.694	-	-	-	-	-	-
			-	-	0.718	0.170	-	-	0.126	0.115
			0.009	0.461	0.150	0.346	-	-	-	-
	LA		0.025	0.842	0.018	0.766	0.387	0.681	0.938	0.491
			0.102	0.825	<10−3	0.870	-	-	-	-
			0.181	0.897	-	-	-	-	-	-
			-	-	0.396	0.688	-	-	-	-
Fusion	CI		0.486	0.800	0.104	0.772	0.182	0.538	0.240	0.509
			0.158	0.758	0.110	0.797	-	-	-	-
			0.042	0.668	-	-	-	-	-	-
	MC		0.002	0.786	0.003	0.620	0.023	0.422	0.158	0.449
			<10−3	0.768	<10−4	0.724	-	-	-	-
			0.020	0.669	-	-	-	-	-	-
	LA		<10−3	0.818	0.002	0.803	0.012	0.708	<10−3	0.792
			0.004	0.874	0.012	0.834	-	-	-	-
			0.092	0.760	-	-	-	-	-	-

Fig. 8.

Scatter plots of each cortical feature versus gestational age (GA) in weeks for the ipsilateral associations found by the fused similarity matrix using ventricular local area (LA) and ventricular curvedness index (CI), with linear fits for healthy controls and subjects with left or right INSVM, depending on the hemisphere. Fetuses with bilateral ventricular enlargement appear as INSVM in both hemispheres. See Table 1 for color-coded associations.

All experiments were replicated for the study of contralateral associations, and few associations were found. Results are reported in the supplementary material. Most associations were confined to the inferior horn and the posterior part of the cortex. After fusion, we only found one association between the right inferior horn and the medial superior part of the parietal lobe in the left hemisphere, when using ventricular curvedness. Both curvature measures for this cortical region were statistical associated with INSVM. With ventricular local area, no associations were found.

4. Discussion

In this work, we have studied the associations of INSVM with folding alterations in the cortical sheets of fetal brains at mid-third trimester of gestation. During this period, rapid and complex maturational processes occur in the fetal brain that manifest in considerable cortical surface expansion and folding. To better capture INSVM-related deviations from normative cortical development, we proposed to incorporate the ventricular shapes along with the cortices in a common framework and model the relationships between both sets of shapes based on their growth patterns using multiple features. Multiple features were used to describe the growth patterns. This provided our approach with more sensitivity to detect subtle but important differences in both cortical folding and surface expansion. Furthermore, the heterogeneous in- formation coming from the different cortical features was merged to produce a common and more comprehensive representation of the growth patterns in order to capture complementary and reliable associations between ventricular enlargement and cortical folding.

As we have seen in the results, the three different cortical features produced distinct associations, except for the posterior horn and the occipital lobe association that was quite consistent among the different cortical and ventricular features in both left and right hemispheres. When using the fused similarity matrix, several noisy and spurious associations detected by the single similarity matrices individually were consequently discarded by our model. In fact, the majority of these noisy associations were not found to be significant in the statistical analysis.

4.1. Similarity matrices

Our approach requires building a joint similarity matrix that incorporates both intra-structure similarity matrices in its block-diagonal and the inter-structure similarity matrix in the off-diagonal. Each entry of the joint similarity matrix is derived from the correlations between its corresponding growth patterns. In this work, we proposed to only use the positive values of Pearson’s correlation coefficient for the intra-similarity matrices, while setting entries with negative correlations to zero. With this, we enforce within-shape clusters to have a similar tendency (e.g., all cortical vertices in the same cluster increase/decrease in mean curvature) and also induce sparsity in the similarity matrices, which is advantageous since it removes spurious correlations that might mislead the projection of the growth patterns. On the other hand, for the inter-structure similarity matrix (i.e., between ventricle and cortex), we used the absolute value of Pearson’s correlation coefficient between the growth patterns in order to identify both positive and negative correlations. As opposed to the intra-structure matrices, this one is a dense matrix. Given that ventricular area is always increased in fetuses with INSVM (i.e., ventricular enlargement), this means that we are interested in capturing associations with cortical regions with increased or decreased mean curvature, for example. Therefore, it might be the case that a ventricular region is correlated with two different cortical regions, each with a different sign. In this scenario, because both regions are not connected in their intra-similarity matrix, the joint spectral embedding step, according to which one is more strongly correlated, will project the growth patterns of this region closer to the ventricular region to desirably form a cluster. Consequently, the other cortical region is lost from this association. One possible approach to capture both cortical regions is to only use positive/negative correlation values to build the inter-structure similarity. In this case, experiments must be replicated to capture positive and then negative associations separately.

4.2. Fusion of multiple features

With the fusion of multiple similarity matrices derived from the different cortical features, we seek to find a more comprehensive set of associations by removing noisy correlations and capturing both common and complementary associations included in the different single similarity matrices. Indeed, our results show that most associations found when using the fused similarity matrix are significant, except for curvedness index and the cortical region in the superior part of the frontal lobe when using ventricular LA. This may indicate that the remaining cortical features (i.e., mean curvature and local area) capture more accurate correlations than the ones captured by curvedness index, although when used for the ventricles, this curvature descriptor provided a very consistent association. On the other hand, with single similarity matrices based on ventricular LA, few associations were found to be significant (e.g., only one out of four associations when using curvedness index), and they were highly inconsistent between hemispheres. With the fused similarity matrix, only one association (atrium with posterior part of the frontal lobe) was not present in both hemispheres, but was not found to be significant. Still, given the early inter-hemispheric asymmetries that appear in the fetal brain, it may be possible to find associations in one hemisphere that are not present in the other.

4.3. Regional associations and validation

Our approach relies on the correlation of the growth patterns to reveal the putative regional associations between the ventricular and cortical shapes. However, these associations do not guarantee to produce cortical regions with altered folding significantly associated with ventricular volume. In this work, these candidate cortical regions were further validated using statistical analysis. From these results, as we can see in Table 2, when gyrification is only characterized with a single feature, most associations recovered by our approach were not statistically significant, whereas with the fused similarity matrix (i.e., using all cortical features simultaneously), most associations were significant, with the exception of curvedness index. This suggests that similarity network fusion is advantageous in that it is able to unravel statistically significant associations while avoiding the non-significant associations found when analyzing each cortical feature individually. Moreover, the fact that curvedness index, as a descriptor of folding, was not found to be significantly associated with VM may imply that the fusion process is likely to be dominated by the individual similarity matrices corresponding to mean curvature and local area.

It is worth noting that prior to validating the associations using statistical analysis, a critical step in our proposed approach is the selection of the optimal number of clusters, which was chosen based on the silhouette coefficient. This might have a considerable impact on the final results. Associations found with a given number of clusters may not produce cortical regions that are significantly correlated with ventricular volume, but as the number of clusters increases these regions can shrink to smaller areas that may then show statistical significance.

4.4. Generalization to multiple shapes

In this work, we have applied our proposed approach to elucidate ipsilateral and contralateral associations between ventricular enlargement and cortical folding for each hemisphere separately, although contralateral associations were analyzed independently. In this case, however, a different approach, (i.e., jointly analyzing both ipsilateral and contralateral associations) could be adopted by extending the joint similarity matrix to incorporate an additional intra-structure similarity along with its corresponding inter-similarity matrices. For the left hemisphere, for instance, we would have to include the left ventricle and cortex (i.e., ipsilateral associations), and the right cortex (i.e., contralateral associations with the left ventricle).

Beyond our specific application, with our proposed approach we can incorporate more than two shapes to study their inter-relationships, e.g., inter-hemispheric asymmetries between the same structure, co-variance between distinct brain regions. The main challenge is to find and appropriate way to construct the inter-structure similarity matrices between the different sets of shapes. It is also important to note that for many shapes with large numbers of vertices, building a joint similarity matrix may be computationally prohibitive. Nonetheless, this problem can be easily tackled by finding the embedding for consecutive overlapping pairs of shapes and then aligning the growth patterns in the new representation.

4.5. Clinical insight

Rather than direct relationships between ventricular and cortical regions, where physical models would be more appropriate, associations found by our proposed approach could be interpreted from a different point of view. A single scalar value such as ventricular volume or diagnosis, because it simply characterizes global changes, might not be sensitive enough to capture differences in some cortical regions between healthy controls and fetuses in the INSVM cohort. Particularly, a global measure is blind to the location of dilation, cannot discern between subjects with ventricular dilation in the atrium or the posterior horn, nor can it provide information about the extent of enlargement in the aforementioned regions, even in the same ventricle. To overcome these limitations, the aim of our approach was to investigate fetal INSVM at a fine-grained level in order to capture the different changes in ventricular dilation both spatially and in magnitude, seeking to identify more subtle cortical folding alterations related to INSVM that otherwise would be overlooked. Along these lines, our results show that different ventricular regions are associated with distinct regions in the cortex, as shown by the regional associations displayed in Fig. 6 and assessed by their p-values and goodness-of-fit in Table 2. Our most important finding, therefore, is the spatially-varying nature of the relationship between the enlargement of the lateral ventricles and cortical folding, so far unexplored in the fetal MRI literature.

Although, in clinical practice, only the atrial diameter is used to perform the diagnosis, ventricular dilation is not simply confined to the atrium but rather it manifests across the whole ventricle. With the fused similarity matrix based on ventricular local area, according to the optimal clustering, the atrium (green region in Fig. 6) was found to be in one association in the left hemisphere, but this association was not significant. However, our results show that the posterior and inferior horns are highly associated with cortical folding alterations. In this context, the implications of our findings suggest that it may be important to expand the focus of diagnosis to consider other ventricular regions. This acquires even more relevance when considering the outcome of fetuses with INSVM. Although prognosis is generally good, there is a small number of fetuses with abnormal outcome, which speaks against the limitations of using a single measurement to characterize ventricular dilation. Elucidating the causes behind the un-favourable outcome of this subgroup is critical, and exploring the whole ventricular shapes might help improve prognosis. Beyond these ventricular regions, the corresponding cortical regions are also of great interest because they do show alterations in cortical folding and surface expansion and can then be used as potential prognostic biomarkers.

From the conducted experiments, we can highlight two ipsilateral associations: 1) the one involving the posterior horn and the occipital lobe, and 2) the one between the inferior horn and the posterior part of the parietal lobe. Although fewer associations were found in the contralateral analysis, from our results we can also underline the association between the inferior horn of the right ventricle and the superior medial part of the parietal lobe in the left hemisphere. In the study of fetal VM, existing works have mainly focused on finding cortical regions with altered folding (Scott et al., 2013; Benkarim et al., 2018). Several curvature-based folding descriptors of these cortical regions were found to be globally and ipsilaterally associated with ventricular volume in (Benkarim et al., 2018). The parieto-occipital sulcus confined within our first association (dark green) was also reported by Scott et al. (2013) to have significant changes in mean curvature under VM. In terms of the features, we found that the area of both cortical regions is increased in the INSVM cohort as illustrated by the scatter plots in Fig. 8. Since these regions are near the posterior part of the ventricles, we may expected them to be dilated as a consequence of the enlargement of the ventricles. The increase in mean curvature suggests more convex regions in fetuses with INSVM, hinting to a delayed cortical folding of these regions. Curvedness index, however, did not show any significant associations. This may be due to the fact that it is not able to capture the type of folding that these cortical regions undergo during the age range of our dataset, since it only quantifies the amount of deviation of the surface from a flat plane but not the direction.

In addition to the cortical regions with folding alterations, our work contributes with more information about fetal VM by also finding the ventricular regions that might have a significant impact in gyrification and, most importantly, providing the relationships between these ventricular and cortical regions.

4.6. Limitations and future directions

The main limitation of our approach is that correlations between growth patterns of the same shape are expected to be higher than between different shapes, specially if the growth patterns of the different shapes are represented with different features. This highlights the importance of the approach used to build the inter-similarity matrix in order to capture accurate associations. Moreover, this must be also considered when performing fusion of various similarity matrices because the sparse kernel we use to diffuse information between matrices is built based on the most similar growth patterns, which are more likely to be within the same shape (i.e., same intra-similarity matrix) and would, subsequently, neglect capturing associations between structures. On the other hand, fusing features that capture different directions of change in cortical folding may negatively impact the fusion process by losing the corresponding associations. Another limitation of our work is that the age range of our dataset is narrow and the morphological changes that occur in the cortex are not as prominent as in older fetuses. Using a dataset of older fetuses with more convoluted cortices may further demonstrate the potential of the proposed approach and show its effectiveness and robustness to capture more complex changes.

For future work, we can include other cortical features such as sulcal depth, cortical thickness and gyrification index (Zilles et al., 1988; Li et al., 2019) to possibly capture additional and more accurate associations. Other curvature measures to describe the ventricular growth patterns could also be of interest for the evaluation of the ventricular shapes, especially in specific regions such as the posterior horn and the arc formed by the inferior horn and the central part of the ventricle. Also, follow-up postnatal studies are an interesting direction of future work to validate the associations and the prognostic power of the cortical regions found to be altered in INSVM. Moreover, instead of a two-step approach where we fuse the similarity matrices that come from different features prior to performing the joint embedding, we could explore multiview approaches to carry out this process in a single step.

5. Conclusions

In this work, we have presented a novel approach to jointly analyze the relationships between multiple anatomical shapes based on their growth patterns using different features. Our approach seeks to find a common underlying representation of the growth patterns of the different anatomical shapes such that associated growth patterns lie close to each other in this low-dimensional embedded space and can be therefore recovered using hierarchical clustering. Spectral embedding is used to obtain this common representation from the joint similarity matrix, which can be built from a single feature or using multiple features and then fusing the corresponding similarity matrices to better capture the relationships between the different shapes. The proposed approach was evaluated on our in-house fetal dataset to study both ipsilateral and contralateral associations between fetal VM and cortical folding alterations. To the best of our knowledge, this is the first work that approaches the study of cortical folding in VM by incorporating both cortical and ventricular shapes to reveal spatially fine-grained associations between them. Our findings suggest that there is a strong ipsilateral relationship between ventricular dilation and cortical folding in INSVM fetuses. Moreover, our proposed approach is able to identify associations containing cortical regions with altered folding significantly associated with ventricular enlargement.