Predicting Infant Cortical Surface Development Using a 4D Varifold-based Learning Framework and Local Topography-based Shape Morphing

Abstract

Longitudinal neuroimaging analysis methods have remarkably advanced our understanding of early postnatal brain development. However, learning predictive models to trace forth the evolution trajectories of both normal and abnormal cortical shapes remains broadly absent. To fill this critical gap, we pioneered the first prediction model for longitudinal developing cortical surfaces in infants using a spatiotemporal current-based learning framework solely from the baseline cortical surface. In this paper, we detail this prediction model and even further improve its performance by introducing two key variants. First, we use the varifold metric to overcome the limitations of the current metric for surface registration that was used in our preliminary study. We also extend the conventional varifold-based surface registration model for pairwise registration to a spatiotemporal surface regression model. Second, we propose a morphing process of the baseline surface using its topographic attributes such as normal direction and principal curvature sign. Specifically, our method learns from longitudinal data both the geometric (vertices positions) and dynamic (temporal evolution trajectories) features of the infant cortical surface, comprising a training stage and a prediction stage. In the training stage, we use the proposed varifold-based shape regression model to estimate geodesic cortical shape evolution trajectories for each training subject. We then build an empirical mean spatiotemporal surface atlas. In the prediction stage, given an infant, we select the best learnt features from training subjects to simultaneously predict the cortical surface shapes at all later timepoints, based on similarity metrics between this baseline surface and the learnt baseline population average surface atlas. We used a leave-one-out cross validation method to predict the inner cortical surface shape at 3, 6, 9 and 12 months of age from the baseline cortical surface shape at birth. Our method attained a higher prediction accuracy and better captured the spatiotemporal dynamic change of the highly folded cortical surface than the previous proposed prediction method.

1. Introduction

Quantifying cortical morphological dynamics at an early postnatal stage of brain growth will help neuroscientists identify and characterize early neurodevelopmental disorders (Narr et al., 2007; Shaw et al., 2007; Lyall et al., 2014). More importantly, if one could learn to predict the normal cortical shape evolution for healthy infants, as well as for infant groups with specific brain disorders, one could learn to predict and distinguish normal from abnormal cortical development. However, prior to reaching this goal, a robust prediction model of cortical surface growth needs to be devised. Furthermore, noting that the cortex represents the abode of vital cognitive and decision-making brain functions as well as health-related behaviors, examining neurodiversity in both health and disease can be further propelled through developing efficient infant brain growth models that may help predict later changes in normal behavior, reasoning or learning abilities (Gabrieli et al., 2015).

On the other hand, modeling approaches for predicting the early postnatal human brain morphometry changes using longitudinal neuroimaging data are scarce –particularly for cortical dynamics prediction. Herein, we imply by cortical dynamics prediction the estimation of the spatiotemporal cortex shape deformation in the future (i.e., the evolution trajectories of the shape) using a set of available observations and measurements. Nie et al. developed the first mechanical cortical growth model (Nie et al., 2012) to simulate the dynamics of cortical folding from longitudinal MRI data in the first postnatal year, during which the cortical surface area increases by 76% (Lyall et al., 2014) and the gray matter volume increases by 149% (Gilmore et al., 2007). Although promising, this method requires the use of cortical surfaces at later timepoints of the same infant to guide the growth model and also gradually loses its accuracy as the number of data acquisition timepoints decreases. Ideally, one would expect to use the least number of input surfaces to accurately predict the development of the highly convoluted shape of the cortical surface.

Recently developed methods (Fletcher, 2013; Niethammer et al., 2011; Singh et al., 2013a,b) proposed various geodesic shape regression models to estimate diffeomeorphic (i.e., smooth and invertible) evolution trajectories; however, they were implemented for image time-series change tracking. A richer variant of these geodesic image regression methods was proposed in (Fishbaugh et al., 2014), where the regression scheme integrated surface shape information to improve 4D image deformation trajectories estimation. This method estimates both baseline image and surface through finding the optimal points and initial momenta that guide the image-surface deformation. One of its applications included the estimation of joint white matter surface and image deformation backwards in time from 20 months to 6 months. Although this model was able to extrapolate the future image-surface deformation, it required at least two observations for prediction. A geodesic shape regression in the framework of currents was developed in (Fishbaugh et al., 2013) to estimate subcortical structures at 6 months of age based on shapes from 9 to 24 months –which also requires more than one timepoint for what we refer to as backward-in-time prediction. A non-linear mixed effect dynamic prediction model was proposed in (Sadeghi et al., 2014) to estimate temporal change trajectories of radial diffusivity images derived from diffusion tensor imaging (DTI) of early brain development. However, it was limited to estimating region-level changes in 3D scalar fields (images) and required a predefined complex parametric form of the development trajectory.

Very recently, we proposed the first learning-based framework that predicts the dynamic postnatal cortical shape from a single baseline cortical surface at birth using a 4D diffeomorphic surface growth model rooted in the theory of currents (Rekik et al., 2015a). The developed prediction framework includes a training stage and a prediction stage. In the training stage, the proposed framework learns both geometric (vertices positions) and dynamic (diffeomorphic evolution trajectories) features of cortical surface growth for each infant using the available acquisition timepoints. We then estimate the mean empirical spatiotemporal atlas at the most commonly shared timepoints among the training subjects to simultaneously initialize the cortical surface shapes at all later timepoints for prediction. In the prediction stage, for each new subject, we refine this initialization by simultaneously moving vertices in the shapes to predict, based on how close the baseline cortical shape to the baseline cortical atlas. Once the positions of the baseline vertices are updated, they form together a virtual baseline shape, which is spatially close to the ground truth baseline cortical shape. Finally, retrieving the corresponding learnt smooth deformation trajectory for every vertex belonging to the constructed virtual shape predicts the cortical shape up to the last timepoint in the training dataset.

In this paper, we further improve the performance of the described framework by proposing two important variants. First, we use the varifold metric, which was demonstrated to perform better than the current metric in registering a pair of surfaces (Durrleman et al., 2014). We also extend the conventional varifold-based surface registration model into a spatiotemporal surface regression model to estimate the diffeomorphic temporal evolution trajectories of the baseline cortical surface shape. Second, we locally morph the baseline surface based on its topographic attributes, such as normal orientation or principal curvature sign, instead of solely relying on the most ‘similar’ vertices in only spatial closeness. Of note, the proposed method requires neither predefined parametric forms of the cortical developmental trajectories nor the guidance from the later time points of the same subject, in comparison with existing methods.

2. Varifold-based Learning of Longitudinal Shape Growth

2.1. Longitudinal varifold-based shape regression model and atlas building (training stage)

F. Almgren (Almgren, 1966) introduced the concept of varifolds in geometric measure theory which was further adapted to solve shape matching problems in (Charon and Trouvé, 2013; Durrleman et al., 2014). The varifold metric lays solid ground for multidimensional set of shapes deformation without requiring a point-to-point correspondence. Indeed, it does not require a poin-to-point surface correspondence (i.e. two surfaces to be matched can have a different number of vertices). Moreover, the varifold matching framework (Charon and Trouvé, 2013) is well rooted in a large diffeomorphic metric mapping (LDDMM) matching framework (Trouvé, 1998), which is mathematically well established and allows to perform statistics on the surface and its diffeomorphic deformation. Most importantly, the proposed framework is generic and the use of the varifold metric makes it robustly and easily extendable to multimodal imaging (e.g., white matter fibers (derived from DTI) encoded as 2D varifolds), thus one could effectively incorporate them into a varifold regression framework for multi-dimensional shapes without any correspondence. In addition, a key advantage of the varifold surface matching method over some conventional manifold matching methods (e.g. FreeSurfer (Fischl et al., 1999) and Spherical Demons (Yeo et al., 2010)) is that it directly operates on surfaces without the need to inflate them into spheres –a process that can distort surface metrics. We begin by reviewing the key ingredients for matching two shapes using the varifold metric as presented in (Charon and Trouvé, 2013; Durrleman et al., 2014).

Cortical surface representation using varifold metric

Encoding a surface S embedded in the ambient space E (here ℝ³) as a varifold refers to defining S as a collection of Grassmanians (non-oriented tangent planes at each of its vertices located at x), which holds its intrinsic local geometry. Each of these non-oriented tangent planes nests the non-oriented normals $\overleftrightarrow{n}(x)$ attached at each of its vertices at x, and they all together form the Grassman manifold G_d(E). In the case of surfaces, G_d(E) represents the quotient of the unit sphere 𝕊 in ℝ³ by two group elements {±Id_ℝ³}. This quotient space G_d(E) spans non-oriented vector elements $\overleftrightarrow{u}$ which are equivalent to u, u/|u| and –u/|u|.

More abstractly, a varifold surface is defined as a continuous linear form that integrates a vector field $\omega \in W:S(\omega )={\int }_{{\mathrm{\Omega }}_S}\omega (x,\overleftrightarrow{n(x)})|n(x)|dx$; where W is defined as a Reproducing Kernel Hilbert Space (RKHS) on the square-integrable space C₀(E × G_d(E)) and Ω_S denotes the surface shape domain. The reproducing kernel K_W on the space of varifolds is the tensor product of kernels on E and on G_d(E): K_W = k_e ⨂ k_t, where k_e denotes a positive continuous kernel on the space E (same as currents) and k_t denotes an additional linear continuous kernel of non-oriented unit vectors on the manifold G_d(E). In particular, for x, y ∈ E and $\overleftrightarrow{u},\overleftrightarrow{v}\in G_d(E)$, the varifold kernel is defined as $K_W((x,\overleftrightarrow{u}),(y,\overleftrightarrow{v}))=k_e(x,y){(\frac{u^{T}v}{|u||v|})}^2$ , where $k_e(x,y)=\mathit{\text{exp}}(-{|x-y|}^2/{\sigma }_e^2)$ is a Gaussian scalar kernel and σ_e denotes the scale under which shape details are regarded as noise. The space of varifolds is then defined as the dual space W^* (i.e., the space of linear mappings from W into ℝ). By the reproducing property, any varifold in W^* is defined as: $\omega (x,\overleftrightarrow{u})={\delta }_{(x,\overleftrightarrow{u})}(\omega )=<K_W((x,\overleftrightarrow{u}),·),\omega {>}_W$, where ${\delta }_{(x,\overleftrightarrow{u})}$ defines a Dirac varifold that acts on ω. A surface S with M meshes (triangles) is then approximated by the sum of Dirac varifolds parameterized by the positions x_i of the centers of its M meshes and their corresponding non-oriented normals $\overleftrightarrow{n_i}:S={\sum }_{i=1}^M{\delta }_{(x_i,\overleftrightarrow{n_i})}$ (Fig. 1). The Dirac varifold does not depend on the orientation given to each triangle. More importantly, the varifold space is endowed with a dot-product that produces a norm, which enables to define dissimilarity measures between pairs of shapes $S={\sum }_i{\delta }_{(x_i,\overleftrightarrow{n_i})}\text{ and }S\prime ={\sum }_j{\delta }_{(x_j^{\prime },\overleftrightarrow{n_j^{\prime }})}$:

$$<S,S\prime {>}_{W^{*}}=\sum _i\sum _j{k}_e(x_i,x_j^{\prime })\frac{{(n_i^T{n}_j^{\prime })}^2}{|n_i||n_j^{\prime }|}$$ (1)

Figure 1. Infant cortical surface representation, shooting and flowing as a varifold.

We visualize the varifold representation of the cortical surface in the left hemisphere as the sum of the Dirac delta varifolds $\delta (x_k,\overleftrightarrow{n_k})$ with x_k being the position of the center of the mesh k (triangle) and $\overleftrightarrow{n_k}$ as its nonr-oriented normal. The evolution trajectory is fully parameterized by the set of initial momenta attached to the optimal control points estimated at the baseline timepoint. Shooting these initial momenta onto a specific timepoint (e.g. 6 months) defines the momenta action p_k on each non-oriented target normal via a convolution with a deformation kernel k_V with σ_V as bandwidth.

Shooting and flowing varifolds onto each other for pairwise registration

As in (Durrleman et al., 2014), varifold pairwise registration between two surfaces S₀ and S₁ can be based on two elements: optimal control model (Allassonnière et al., 2005) and LDDMM (Trouvé, 1998). The first requires the definition of a control variable, which in the matching context, is represented by the velocity vector field v_t estimated along with the shape vertices’ positions x. This is achieved through solving a Hamiltonian system that parameterizes the geodesic diffeomorphism fully guided by a finite set of initial momenta p_k attached at control points c_k as introduced in (Durrleman et al., 2011). Interestingly, these control points do not necessarily belong to the shape as they are placed in shape regions that highly varied, thereby decoupling the deformation from the geometry of the object to deform. Hence, the geodesic deformation is rather determined by the complexity of anatomical changes in time (Fishbaugh et al., 2013; Durrleman et al., 2014). The sought initial system state 𝒮₀ = {c₀, p₀} is composed of the optimal control points c₀ = {c_0,k}_k=1,…,Nc and the initial momenta p₀ = {p_0,k}_k=1,…,Nc guiding the deformation of the ambiant 3D space. The initial system state is estimated through solving the dynamic system state: 𝒮̇(t) = F(𝒮(t)) with 𝒮(0) = 𝒮₀ that integrates the Euler-Lagrange equations:

$$\{\begin{array}{c}{\dot{c}}_k(t)={\sum }_{i=1}^{N_c}{k}_V(c_k(t),c_i(t))p_i(t)\hfill \\ {\dot{p}}_k(t)={\sum }_{i=1}^{N_c}{p}_k{(t)}^T{p}_i(t){\nabla }_1{k}_V(c_k(t),c_i(t))\hfill \end{array}$$ (2)

With initial conditions, c_k(0) = c_0,k and p_k(0) = p_0,k at the starting point t = 0. The initial set of control points c_k(0) are automatically placed every σ_e for initialization.

The first equation gives the derivative of the control point position c_k (i.e., velocity) and the second equation gives the derivative of the momentum p_k (acceleration). Now, we introduce the second ingredient which diffeomorphically flows the baseline varifold into the target one with a velocity v_t densely represented by the estimated set of initial momenta attached to the optimal control points. In the spirit of LDDMM framework, the time-varying velocity fields v_t is the solution of the following ordinary differential flow equation:

$$\{\begin{array}{c}\frac{d{\xi }_t(x)}{\mathit{\text{dt}}}=v_t◦{\xi }_t(x),t\in [0,1]\hfill \\ {\xi }_0={\mathit{\text{Id}}}_{{ℝ}^3}.\hfill \end{array}$$ (3)

where a particle at location x at time 0 will move to ξ(t,x) at later time t, where ξ(x,t) follows the integral curve of $\frac{\partial \xi (t,x)}{\partial t}$ starting with ξ(0,x) = x. If the velocity v(x,t) is sufficiently smooth then the solution for Eq. 3 is a diffeomorphism as demonstrated in (Joshi and Miller, 2000). A convenient choice for v_t is a RKHS V, densely spanned by a reproducing Gaussian kernel k_V which decays at a rate σ_V at which points in E move in a correlated manner. Thus, the velocity at any point x ∈ E is defined as the sum of scalar functions k_V located at a set of control points {c_k}_1,…,Nc convoluted with their deformation momenta $\{p_k\}:v(x,t)={\sum }_{k=1}^{N_c}{k}_V(x,c_k(t))p_k(t)$ and the kinetic energy of the total deformation induced by v(x,t) (i.e., regularity of the deformation) is defined as ${\int }_0^1{|v_t|}_V^2\mathit{\text{dt}}={\int }_0^1{\sum }_{i=1}^{N_c}{\sum }_{j=1}^{N_c}{p_i(t)}^{T}k(c_i(t),c_j(t))p_j(t)\mathit{\text{dt}}$.

Once the set of optimal control points are estimated along with the initial deformation momenta, we shoot the initial momenta onto the target timepoint, then we estimate the shape deformation trajectory ξ(t,x) via solving Eq. 3. This is achieved through a typical variational framework that minimizes the following energy functional to estimate the optimal (c₀,p₀):

$$E_{\mathit{\text{registration}}}=\frac{1}{2}{\int }_0^1{|v_t|}_V^2\mathit{\text{dt}}+\gamma {\parallel S_1-{\xi }_1·S_0\parallel }_{W^{*}}^2$$ (4)

The first energy term forces the warping trajectory to be smooth and the second makes the trajectory end close enough to the target surface. The parameter γ defines the trade-off between both of these terms. The estimation of (c₀,p₀) is posed as a single optimization problem and is solved by a single gradient descent (Durrleman et al., 2014).

Longitudinal varifold regression

We propose a natural extension of the original pairwise varifold registration model into a linear varifold regression model. We estimate the optimal initial deformation momenta and optimal control points by fitting the deforming baseline shape into a set of successive target shapes {S₀, …, S_N}. This is achieved through minimizing the updated energy functional:

$$E_{\mathit{\text{regression}}}=\frac{1}{2}{\int }_0^1{|v_t|}_V^2\mathit{\text{dt}}+\gamma \sum _{j\in \{1,\dots ,N\}}{\parallel S_j-{\xi }_{t_j}^{{\mathbf{P}}_0}·S_0\parallel }_{W^{*}}^2$$ (5)

The set of target shapes where we force the moving baseline shape to smoothly fit them at specific timepoints (in our case the imaging acquisition timepoints) is included in the second term. The initial state system 𝒮₀ fully determines the diffeomorphism ξ^p₀(t,x) geodesically linking all these shapes while accounting for their time difference. The superscript p₀ denotes that the temporal evolution trajectory ξ is spanned by the set of the estimated initial momenta.

Varifold-based geometric and dynamic features learning

In the training stage, we estimate a cortical surface growth scenario for each infant in our training dataset using the available MRI acquisition timepoints. We first register all the baseline surfaces of the training subjects into a common space. Then, for each warped baseline shape in this space, we estimate its temporal evolution trajectory. Both of these steps are achieved using the varifold regression model to link all subjects’ longitudinal shapes in space and time. This facilitates inter-subject comparison of deformation features estimated at any timpoint falling in the in-between obervations interval ]t_i, t_i+1[. At this point, we introduce the notion of a dynamic cloud 𝒞, which is composed of points q(x,t) = (x,ξ^p₀(x,t)) with x as a vertex belonging to any baseline shape S₀ in the training data and ξ^p₀(x,t) its temporal evolution trajectory. In particular, this geodesic trajectory is estimated through shooting the initial deformation momenta p₀ associated with the baseline shape S₀ onto the timepoint t, which bases the full recovery of the deformed shape at t through the flow equation and the deformation kernel k_V convolution with p₀(t). In other words, a point q(x,t) encodes the 3D position x of any baseline vertex at a specific timepoint t. Here, the 3D position of any baseline vertex x defines the geometric feature and the geodesic trajectory ξ^p₀(x,t), spanned by the initial momenta of its baseline surface, defines the dynamic feature of the learnt model. We will exploit both of these features to predict the evolution trajectory for a new baseline shape.

Cortical spatiotemporal atlas estimation

For each of the most commonly shared acquisition timepoints t_{i{i∈0,…,N}}, we build an empirical mean atlas 𝒜_i by computing the mean 3D position of the spatiotemporally aligned training subjects. We also include the simulated shape at a missing acquisition timepoint (transparent blue rectangle belonging to Infant N^trn, Fig. 2) using the varifold-based surface growth model for the atlas building if these shapes were acquired at ±1–month gap from the ground-truth shape (represented by the orange rectangle in Fig. 2). Indeed, at ±1–month gap, the varifold-based surface deformation model recovers neighboring information with good accuracy (mean surface distance error < 0.7mm) as for the current-based surface deformation model in (Rekik et al., 2015a). One could intuitively explain this by recalling the principle of the least action in a classical mechanical Lagrangian framework, which grounds the diffeomorphic geodesic surface deformation framework. This strategy allows us to include more data into building the temporal atlas {𝒜_t} with t ∈ {t₀, …, t_N} and to better capture inter-subject variability.

Figure 2. An overview of the proposed framework for learning dynamic cortical surface growth (training stage).

We estimate a smooth temporal trajectory for each of the baseline cortical shapes in the training dataset. A spatiotemporal atlas is built at the most commonly shared acquisition timepoints in the training subjects. We also include the estimated cortical surface if it is ±1–month distant from the ground truth in the atlas building process.

2.2. Prediction using topography-based local shape morphing and learnt features (testing stage)

To predict the evolution of the cortical surface for a new infant, the only information we need is the shape of the baseline cortical surface S₀ (at the first acquisition timepoint). Here we propose two different methods that exploit the topography of the baseline surface S₀ to select geometric and dynamic features from the cloud. The extracted features will define the temporal evolution of the cortical surface up to the last common acquisition timepoint in the training dataset. Both methods are based on the intuitive idea that vertices in the baseline ground truth shape dynamically behave in a way that is similar to their nearest and most similar neighboring vertices in the cloud. Originally, our hypothesis was inspired from the work of (Nie et al., 2012), which demonstrated (using a mechanical brain growth model) that the cortical regions with similar location and geometry are more likely to have similar growth dynamics. We further demonstrated this hypothesis on a randomly selected subject (reference baseline) by choosing both geometrically closest surface to it from the remaining baseline cortical surfaces in the cohort and geometrically most distant one. Then, we computed the average of the dot product between the initial momenta associated with each of these ‘closest’ and ‘most distant’ baseline surfaces and the reference baseline surface. Notably, Fig. 3 shows that the spatially geometrically closer surface to the reference baseline (with the mean surface distance error = 5.04mm) has most similar initial deformation momenta (i.e. larger average dot product = 0.9184) and the most distant one (with the mean surface distance error = 6.96mm) has a smaller average dot product (= 0.8952). Since the initial momenta fully lays out the future 4D deformation trajectory from baseline surface, this shows that geometrically closer baseline cortices have most similar dynamic behavior as they grow. Subsequently, we introduce the concept of a virtual shape that explores the learnt shape features to find the closest and most analogous shape in the cloud to the baseline shape S₀. The prediction framework for the spatiotemporal evolution of the ground truth surface S₀ is composed of two main steps:

Figure 3. Geometrically similar baseline infant cortical surfaces have a similar dynamic evolution.

After selecting a random reference baseline surface from our dataset (dark purple), we identify both baseline surfaces in our cohort that are closest (green) and most distant (red) from this reference. By computing the average dot product between the initial momenta vectors associated with the reference baseline and each of the selected surfaces, we show that the closest surface (B–green) has a more similar dynamic behavior (i.e. larger average dot product) to reference baseline than (A–red).

• Step 1: Virtual shape construction for simultaneous shape prediction at all later timepoints

First, we define a virtual shape S_virtual as the ensemble of baseline vertices from the cloud 𝒞 that are both spatially close and topographically similar to the ground truth baseline shape S₀ (Fig. 4). We initialize the virtual shape as the baseline atlas shape 𝒜₀ and also the shapes to predict {S̃_i} as the mean atlases 𝒜_i at the timepoint t_i. If the error distance between a vertex in the mean atlas shape 𝒜₀ and its corresponding baseline vertex in S₀ is smaller than ε, then we keep this vertex unmoved. Otherwise, we seek the closest and most topographically similar vertex from the cloud. For vertices in S_virtual having no closest vertex in the ε–range to the baseline shape, we propose two methods to construct the virtual shape S_virtual, each based on a specific topographic surface attribute:

1. Local directional search for the m–closest neighbors from the cloud that are most aligned with the baseline surface normals (Method 1): We update the position of a virtual vertex x that is within a Euclidean distance ε from S₀ (or > ε–far from S₀) by first selecting the set ${𝒱}_n^m(x)$ of m–closest vertices from the cloud to x and that are most aligned with the vertex normal direction n(x) as follows:

   • Define a set of vertices y from the cloud {y ∈ 𝒞; ∥x – y∥₂ < ε} and order it by increasing distance from the baseline vertex x.
   • If the Euclidean dot product n(x) · n(y) > 0 (i.e. same direction), then ${𝒱}_n^m(x)={𝒱}_n^m(x)\cup y$
   • Stop adding vertices from the cloud to ${𝒱}_n^m(x)$ when its cardinal becomes equal to m.

   Then, we compute the mean position x^new of the m-vertices in ${𝒱}_n^m(x)$ using Euclidean distance. To predict the evolution trajectory for the new vertex position x^new, we compute the mean of the estimated evolution trajectories ξ̄ from the set of their corresponding initial momenta estimated in the training stage. This mean spatially smoothed trajectory predicts the growth of the input baseline surface S₀ in ]t₀,t_N].
2. Local search for the m–closest neighbors from the cloud that share the same maximum principal curvature sign with the baseline surface (Method 2): We locally morph the initial virtual shape by exploring the topography of the closest points in the cloud and choosing the set ${𝒱}_{\kappa }^m(x)$ of m–closest ones that have the same maximum principal curvature κ sign as the baseline atlas vertex to move as follows:

   • Define a set of vertices y from the cloud {y ∈ 𝒞; ∥x – y∥₂ < ε} and order it by increasing distance from the baseline vertex x.
   • If κ(x) × κ(y) > 0 (i.e. same curvature sign), then ${𝒱}_{\kappa }^m(x)=V_{\kappa }^m(x)\cup y$.
   • Stop adding vertices from the cloud to ${𝒱}_{\kappa }^m(x)$ when its cardinal becomes equal to m.

   Then, as previously, we estimate the new vertex position x^new and its 4D evolution trajectory based on the initial momenta that are associated with vertices in ${𝒱}_{\kappa }^m(x)$.

Figure 4. Virtual shape construction and local morphing using spatially close and to-pographically similar points from the cloud.

(A) We visualize the overlap between the estimated baseline virtual shape S_virtual and both baseline surface S₀ and baseline atlas 𝒜₀ (used to initialize the virtual shape). The red dashed contours in the close-ups show that the local morphing processing (here using Method 2) brought the virtual shape closer to the baseline shape. (B) We overlay for one representative infant the ground truth cortical left hemisphere shapes S_i at later timepoints with the predicted shapes S̃_i from the estimated virtual shape.

The different steps for the simultaneous local morphing of the virtual shape and the shapes to predict using the cloud and the estimated atlases are described in Algorithm 1.

• Step 2: Estimation of a geodesic evolution of the cortical shape for a new subject

Once the set of shapes {S̃_i} are predicted at later timepoints, we minimize the energy Ẽ (Algorithm 1) to estimate the spatiotemporal deformation trajectory of the baseline shape S₀.

Algorithm 1.

Prediction of cortical surface shape evolution from a baseline shape

1:	INPUTS:
	The learnt mean atlases 𝒜ti
	The learnt cloud 𝒞
	The baseline ground truth shape S0
2:	Initialize Svirtual ← 𝒜0.
3:	Initialize S̃i ← 𝒜i for i ∈ {1, …, N}
4:	Initialize ε as the mean distance between S0 and 𝒜0 minus its standard deviation
5:	for every vertex x in the virtual shape Svirtual that is located outside the ε—neighborhood from S0 do
	Find the m—closest and topographically most similar vertices from the cloud using similarity metric (Method 1 or 2)
	Update its new position xnew (geometric feature) as the mean of the positions of its m—retrieved neighbors in 𝒞.
	Compute its evolution trajectory ξ̅(xnew, t)t∈[0,T] as the mean over the m retrieved 4D trajectories.
	Add the new created point qnew = q(xnew, ti) = (xnew, ξ̅(x, t)) to the cloud.
	Replace the position of the vertex x in the shape S̃i at timepoint ti with ξ̅(xnew, ti).
6:	end for
7:	Estimate the geodesic varifold-based baseline shape evolution using {S0, {S̃i}} by minimizing:
	$\tilde{E}={\int }_0^1{\parallel v_t\parallel }_V^2\mathit{\text{dt}}+\frac{1}{\gamma }{\sum }_i{\parallel {\tilde{S}}_i-{\tilde{\xi }}_{t_i}^v·S_0\parallel }_{W^{*}}$
8:	OUTPUT:
	Set of predicted surfaces {S̃i} at timepoints ti with i ∈ {0, …, N}
	Set of smooth temporal evolution trajectories ξ̅v for vertices in S0 for t ∈ [0, tN]

3. Results

3.1. Data and parameters setting

We used a leave-one-out cross validation method to evaluate the proposed framework on longitudinal inner cortical surfaces of 12 infants, each with 5 serial MRI scans acquired at around birth, 3 months, 6 months, 9 months and 12 months of age.

Image processing

All MR images at all the acquisition timepoints were preprocessed using an infant-specific framework developed in (Dai et al., 2013; Li et al., 2014b,c,d) including (1) the removal of the skull (Shi et al., 2012), followed by the removal of the cerebellum and brain stem by registering an atlas to each subject (Shen and Davatzikos, 2002; Wu et al., 2006); (2) intensity inhomogeneity correction using N3 method (Sled et al., 1998); (3) rigid alignment of each image to the age-specific infant brain atlas (Shi et al., 2011); (4) longitudinal infant tissue segmentation of infant brain MR images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) using a longitudinally consistent level-set-based segmentation method (Wang et al., 2013); and (5) filling the cortex insides and splitting the brain into left and right hemispheres.

Cortical surface reconstruction

For each segmented image, we reconstructed the inner cortical surface for each hemisphere using a deformable surface method (Li et al., 2012, 2014a). In particular, we corrected the topological and geometric defects in the WM and tesselated the cortical surface as a triangular mesh to guarantee a spherical topology for each hemisphere as proposed in (Li et al., 2012, 2014a). Ultimately, each cortical hemisphere was parcellated using the robust framework developed in (Li et al., 2014c).

4D varifold regression model parameters

We used the varifold-based geodesic shape regression model with parameters γ = 10⁻⁴, σ_e = 5, and σ_V = 25 (Durrleman, 2010; Durrleman et al., 2014).

3.2. Cortical shape prediction evaluation

We applied the proposed prediction framework for the right hemisphere data and the left hemisphere data separately. We then assessed the prediction accuracy for each hemisphere independently. For each hemisphere, we built five atlases using alternatively selected 11 different training subjects from the dataset, while leaving 1 subject out for testing to predict the inner cortical right and left hemispheres’ shapes at 3, 6, 9 and 12 months from the cortical surface shape at birth.

Spatiotemporal mean population atlas building for each hemisphere

We set the inter- and intra-subject cortical correspondences using the varifold-based shape regression model, so we can easily navigate from any subject at any timepoint to a different subject at a different timepoint. We then built spatiotemporal mean atlases at birth, 3, 6, 9 and 12 months. Each spatiotemporal atlas {𝒜₀,𝒜₁,𝒜₂,𝒜₃,𝒜₄} was estimated using 11 infants, while leaving 1 infant out for testing (Fig. 2).

Cortical shape prediction from the baseline cortical surface S₀

We implemented Algorithm 1 using the two proposed local baseline virtual shape morphing methods to predict baseline shape growth for each of the 12 testing infants at 3, 6, 9 and 12 months. The parameter ε was fixed as the mean distance between S₀ and 𝒜₀ minus its standard deviation. We chose m = 25 closest neighbors. We have also ran the original prediction framework proposed in (Rekik et al., 2015a) (referred to in all figures and tables as M_ref or Reference Method) where only spatial closeness was taken into account for virtual shape construction. For Reference Method, we chose the number of neighbors m = 8 that gave the best prediction results.

To compare the performance of the three approaches, we used two evaluation criteria: a vertex-wise distance between the predicted and the ground truth shapes and the surface Dice index D between the two shapes S and S′ $(D(S,S\prime )=\frac{2|S\cap S\prime |}{|S\cup S\prime |})$ for 36 anatomical regions of interest defined in (Desikan et al., 2006) and parcellated using the infant-specific method described in (Li et al., 2014c). We detail below the results for each evaluation criterion:

• Quantification of the alignment with anatomical boundaries of 36 ROIs in each hemisphere

Fig. 5 shows the mean Dice index in 36 ROIs, averaged across all subjects between the ground truth and the predicted cortical shapes for the right hemisphere (Fig. 5–top) and the left hemisphere (Fig. 5–bottom). Overall, method 2 (i.e. maximum principal curvature sign used as a similarity metric for virtual shape local morphing) achieves the best prediction results for the left hemisphere and performs as good as Method 1 (i.e. alignment with the normal direction used as a similarity metric for virtual shape local morphing) or slightly better for predicting the shape of the right hemisphere. Notably, the mean prediction error gradually decreases as the shape to predict becomes very distant in time from the baseline surface. For instance, it dropped from 88.00 ± 2.90% at 3 months to 75.26 ± 5.91% at 12 months for predicting the left hemispheric shape using Method 2. Nevertheless, it distinctly outperforms Reference Method where the prediction error peaks at 76.91 ± 5.15% at 3 months to eventually go down to 69.58 ± 6.22% at 12 months as well as Method 1 where the prediction error reaches its apex at 85.97 ± 2.79% and then slowly decreases to reach 73.85 ± 5.63% at 12 months. For right hemispheric shape prediction, the overall best performance is also achieved by Method 2 where the mean Dice index peaks at 88.59 ± 4.23% and is reduced to 74.64 ± 8.80% for predicting 12-month hemisphere shape.

Figure 5. Error bar plots for mean Dice index over all subjects between the ground truth and the predicted cortical surfaces for the right hemisphere (top) and the left hemisphere (bottom).

We compute the mean Dice index using the original method M_ref (Rekik et al., 2015a) and the two proposed Methods 1 (M1) and 2 (M2).

For a more localized analysis of our prediction results, we also overlay in Fig. 6-A and Fig. 7-A the mean Dice index map between the ground truth and the predicted surfaces in each of the 36 anatomical ROIs, respectively, averaged across 12 infants on the inflated left cortical hemisphere and the inflated right hemisphere at 3, 6, 9 and 12 months. Overall, Fig. 6-A–Method 2 demonstrates a better anatomical alignment than Fig. 6-A–Method 1 and Fig. 6-A–Reference Method, where the regional color gradually turns from light to dark purple with a very few exceptions (e.g. the overlap error peaks in the insula cortex at 6 and 9 months (54.92% and 60.4%)). Apart from these specific anatomical areas that are characterized by the complexity of their folding, particularly for the insula cortex which has the highest local gyrification index and an extremely folded and deeply buried structure (Li et al., 2014d), the local mean prediction error for other ROIs is very promising. Specifically, using Method 2, the left-hemisphere mean Dice index across the 12 subjects reaches its maxima at 95.75% in the rostral middle frontal gyrus at 3 months, 91.83% in the rostral anterior cingulate cortex at 6 months, 91.35% in the fusiform gyrus at 9 months and in the pars orbitalis at 12 months.

Figure 6. Prediction accuracy evaluation in 36 anatomical ROIs using the two proposed Methods 1 and 2 and Reference Method in the left hemisphere.

(Top) Mean Dice index between the ground truth and the predicted surfaces, averaged in each of the 36 anatomical ROIs, across 12 infants. (Bottom) Mean vertex-wise surface distance error between the ground truth and the predicted surfaces, averaged in each of the 36 ROIs, across 12 infants.

Figure 7. Prediction accuracy evaluation in 36 anatomical ROIs using the two proposed Methods 1 and 2 and Reference Method in the right hemisphere.

(Top) Mean Dice index between the ground truth and the predicted surfaces, averaged in each of the 36 anatomical ROIs, across 12 infants. (Bottom) Mean vertex-wise surface distance error between the ground truth and the predicted surfaces, averaged in each of the 36 ROIs, across 12 infants.

We notice similar local outperformance of Method 2 over Methods 1 and (Rekik et al., 2015a) for the right hemisphere. As time increases from 3 to 12 months the prediction accuracy slowly drops and it similarly partly aligns with the boundaries of the insula cortex at 6 and 9 months and the paracentral lobule at 12 months. Similarly to the left hemispheric prediction, the mean Dice index peaks in the rostral middle frontal gyrus at 3 months (94.39%), the rostral anterior cingulate cortex (91.75%) at 6 months and in the pars orbitalis at 12 months (88.92%). At 9 months, it peaks again in the rostral middle frontal gyrus (90.1%), further demonstrating the good performance of the proposed method.

• Spatial closeness to the ground truth surface

Tables 1 and 2 show the mean surface distance error between the ground truth and the predicted shapes respectively for the left and right hemispheres using Reference Method and the two proposed Methods 1 and 2. Clearly, the prediction method based on Method 2 shows a more promising performance in decreasing the mean surface distance prediction errors at later timepoints, compared with Method (Rekik et al., 2015a) and Method 1 for both the right hemisphere (Table 1) and the left hemisphere (Table 2). We also visualize in Fig. 6-B and Fig. 7-B the mean surface distance prediction error between the ground truth and the predicted surfaces in each of the 36 anatomical ROIs, respectively, averaged across 12 infants on the inflated left cortical hemisphere and the inflated right hemisphere at 3, 6, 9 and 12 months. With regard to this evaluation criterion, Method 2 also allows a better spatial closeness to the ground truth shape at different timepoints for most cortical regions. For instance, using Method 2, in the left hemisphere, the surface distance error reaches its minima at 0.64mm in the rostral middle frontal gyrus (3 months), 0.71mm in the insula cortex (6 months), 0.75mm in the caudal middle frontal gyrus (9 months), and 0.75mm in the middle temporal gyrus (12 months). On the other hand, the distance error reaches its maxima at 1.93mm in the isthmus-cingulate cortex at 3 months, in the postcentral gyrus at 6 months and in the transverse temporal cortex at 9 months, and peaks at 1.94mm in lateral orbitofrontal cortex at 12 months.

Table 1.

Mean surface error distance (mm) between the ground truth cortical surface and the predicted one for 12 infants in the left hemisphere.

Timepoint	Reference Method	Method 1	Method 2
3 months	0.831 ± 0.8	0.756 ± 0.741	0.74 ± 0.727
6 months	1.083 ± 1.022	0.991 ± 0.952	0.981 ± 0.949
9 months	1.191 ± 1.104	1.075 ± 1.016	1.059 ± 1.007
12 months	1.199 ± 1.243	1.096 ± 1.052	1.08 ± 1.039

Mean ± standard deviation between the predicted shape and the ground truth shape were computed using the Reference Method (Rekik et al., 2015a) and the two proposed Methods 1 and 2.

Table 2.

Mean surface error distance (mm) between the ground truth cortical surface and the predicted one for 12 infants in the right hemisphere.

Timepoint	Reference Method	Method 1	Method 2
3 months	0.885 ± 0.837	0.789 ± 0.768	0.756 ± 0.739
6 months	1.13 ± 1.056	1.053 ± 0.99	1.037 ± 0.976
9 months	1.186 ± 1.11	1.093 ± 1.021	1.068 ± 1.003
12 months	1.242 ± 1.115	1.149 ± 1.072	1.115 ± 1.05

Mean ± standard deviation between the predicted shape and the ground truth shape were computed using the Reference Method (Rekik et al., 2015a) and the two proposed Methods 1 and 2.

For the right hemisphere, the distance prediction error is slightly higher than for the left hemisphere. However, Method 2 still performs as good as Method 1 or slightly better, and remarkably both perform better than Reference Method as shown in Fig. 7-B. The lowest surface distance prediction error belongs to the rostral middle frontal gyrus (0.65mm, 3 months), the insula cortex (0.79mm, 6 months), the cuneus cortex (0.86mm, 9 months) and the parahippocampal gyrus (0.86mm, 12 months). Whereas the highest surface distance error reaches its maxima at 2.04mm in the isthmus-cingulate cortex at 3 months, in the postcentral gyrus at 6 months and in the transverse temporal cortex at 9 months, and peaks at 1.62mm in lateral occipital cortex at 12 months.

As we can see, the distance errors are quite small for both hemispheres, although they gradually increase from 3 to 12 months. We also observe regionally non-uniform error maps, especially for the surface distance error maps, which is most likely caused by the spatially variable inter-subject variations in terms of cortical folding and its development.

• Statistical significance of the proposed methods compared to method (Rekik et al., 2015a)

Although Method 2 overall performed better or at least as good as Method 1, no statistical significance between both methods for both left and right hemispheres shape prediction using the two evaluation criteria was found. However, when comparing Reference Method with Methods 1 and 2, the statistical difference is highly significant in the left hemisphere for the mean Dice index (p < 0.0001) and for the mean surface distance error (< 0.01 for Method 1, and < 0.005 for Method 2) at all ages. For the right hemisphere, comparing Reference Method with Methods 1 and 2, the statistical difference was also highly significant for the mean Dice index (p < 10⁻⁵) and for the mean surface distance error (< 0.01 for Method 1, and < 0.001 for Method 2) at all timepoints.

4. Discussion

In this article, we presented the first topography-based prediction model for dynamic cortical surface evolution in infants during the first postnatal year solely based on a single baseline cortical shape surface. Moreover, we extended the pairwise surface registration method using the varifold metric into a spatiotemporal diffeomorphic varifold regression model to learn both geometric and dynamic features of cortical surface shape growth for shape prediction at later timepoints. We then used the estimated initial momenta from the training data as dynamic features for simultaneously predicting cortical surface development at all later timepoints from a single timepoint. We also introduced the concept of “virtual shape” which represents the best candidate shape that is both spatially close and locally topographically similar to the baseline shape (the only available observation) for prediction. We initialized the virtual shape using the baseline estimated atlas then locally morphed it vertex-by-vertex to look more like the baseline shape. The local virtual shape morphing process explored the topography of the learnt cloud (distribution of baseline vertices and their associated initial momenta) based on two topographic attributes: normal direction (Method 1) and maximum principal curvature (Method 2).

Although the infant cortical shape is very challenging to model due to its highly convoluted foldings and dynamic growth (Li et al., 2014b,d), the proposed framework using Methods 1 and 2 showed promising prediction results that significantly outperformed our original proposed framework in (Rekik et al., 2015a), which was only based on spatial closeness criterion for local virtual shape morphing. In particular, Method 2 showed the best prediction performance with regard to both evaluation criteria (Dice overlap between the ground truth surface and the predicted shape in 36 anatomical ROIs, and surface distance error between them). However, despite the lack of statistical significance in prediction accuracy between using normal direction and maximum principal curvature sign, it is clear that the principal curvature is a more relevant attribute to use for local virtual shape morphing, as it is an important feature in the convoluted cortical surface in encoding the local direction of sulcal and gyral folds that marked previous work on the cortex oriented morphometry (Boucher et al., 2009). Notably, the proposed model also quantifies the speed at which the cortical shape grows as the predicted shapes can be used to guide the spatiotemporal geodesic deformation. Fig. 2 clearly captures the heterogeneous growth pattern of the inner cortical surface where different cortical regions grow at different speeds. This concurrently aligns with the observation on cortical development as reported in (Dubois et al., 2014, 2008; Nie et al., 2012; Li et al., 2014d).

Interestingly, we obtained better prediction results for the left hemisphere than for the right hemisphere. This may be explained by the fact that we estimated a diffeomorphic shape regression for each hemisphere independently through using the same deformation kernel standard deviation σ_V for both hemispheres, as we thought of the brain as one connected component. However, noting that left and right hemispheres are asymmetric and that the right hemisphere develops faster than the left one and also occupies more volume (Li et al., 2014b; Dubois et al., 2008), one could possibly improve the prediction accuracy by further exploring the varifold matching parameters or somehow incorporating the volume enclosed by the surface to deform into the varifold metric. Additionally, we also remark that the prediction error slowly increases from 3 to 12 months. This can be explained by the substantial growth of the cortical gyrification index with 16.1% in the first year of age as well as the drastic increase of the total brain volume of 109.7% as reported in (Li et al., 2014d; Dubois et al., 2014), both giving rise to more complex convoluted folds that are more challenging to accurately model and predict. Another alternative to improving the prediction of multiscale highlyfolded cortical regions (e.g. insula cortex) is to extend the varifold surface matching framework into a multiscale varifold matching method {this is one of insightful directions that we plan to explore in our future work.

5. Conclusion

We proposed the first varifold-based learning framework for predicting dynamic cortical surface evolution in infants based on cortical topographic attributes extracted at a single MRI acquisition timepoint. Undoubtedly, using surface attributes for local virtual shape morphing to jointly predict the cortical shape at later timepoints from the baseline shape has improved the prediction accuracy over the simple spatial closeness used as a selection criterion previously proposed in (Rekik et al., 2015a). This learning framework lays ground to not only learn how the cortex morphs but also to learn its dynamics, as it also predicts how fast each vertex in the cortical surface moves. This will enable us to predict a potential abnormality growth and its variation with respect to the normal predicted behavior using solely a single MR acquisition time point. In our future work, we would further boost up its performance by exploring additional morphological features (e.g., cortical thickness or local gyrification index) for virtual shape morphing and eventually jointly combining them to better predict the shape evolution in space, time and morphology. In addition, we may consider improving the shape growth framework by considering second order evolutions based on shape splines, which would also allow to account for any likely noise in the data as in (Trouvé and Vialard, 2012; Vialard, 2013). Furthermore, using an improved varifold-based surface registration/regression model as in (Rekik et al., 2015b) would contribute to increasing the robustness of the proposed shape prediction framework as well as its performance.