Consistent Reconstruction of Cortical Surfaces from Longitudinal Brain MR Images

Abstract

Accurate and consistent reconstruction of cortical surfaces from longitudinal human brain MR images is of great importance in studying subtle morphological changes of the cerebral cortex. This paper presents a new deformable surface method for consistent and accurate reconstruction of inner, central and outer cortical surfaces from longitudinal MR images. Specifically, the cortical surfaces of the group-mean image of all aligned longitudinal images of the same subject are first reconstructed by a deformable surface method driven by a force derived from the Laplace’s equation. And then the longitudinal cortical surfaces are consistently reconstructed by jointly deforming the cortical surfaces from the group-mean image to all longitudinal images. The proposed method has been successfully applied to both simulated and real longitudinal images, demonstrating its validity.

1 Introduction

The human cerebral cortex is a thin, highly folded sheet of gray matter with the thickness varying between 1 and 5 mm [1-3]. Reconstruction of cortical surfaces from brain MR images plays a vital role in studying structure and function of human brains, and many methods have been proposed in the literature [1-7]. However, most existing cortical surface reconstruction methods were designed for working on a single image. For studying longitudinal change of cortical structures, which is important to normal development, aging, and disease progression of human brains, it requires more accurate and consistent cortical surfaces reconstruction and representation, since longitudinal cortical changes are usually very subtle, especially in normal aging and Alzheimer’s disease. Therefore, applying these existing cross-sectional methods independently to reconstruction of cortical surfaces at each time point in a longitudinal imaging study may generate longitudinally-inconsistent cortical surfaces, due to the inconsistency of tissue segmentation, topology correction, and surface tessellation. Accordingly, efforts have been made toward the reconstruction of cortical surfaces from longitudinal images [8, 9], e.g., the longitudinal processing pipeline in FreeSurfer [8], in which cortical surfaces of the mean image of rigidly-aligned longitudinal images or the median image are used as initialization for each longitudinal image and then independently deformed at each time point.

This paper presents a new method for consistent reconstruction of inner, central, and outer cortical surfaces from longitudinal brain MR images. The inner cortical surface is the interface between white matter (WM) and gray matter (GM), and the outer cortical surface is the interface between GM and cerebrospinal fluid (CSF). The central cortical surface is defined as the layer lying in the geometric center of the cortex, approximately corresponding to the cytoarchitechtonic layer four [3, 4, 7]. For consistent reconstruction of cortical surfaces from longitudinal images, in our method, the cortical surfaces of a group-mean image of all non-rigidly aligned longitudinal images are first reconstructed using a deformable surface method, and then the cortical surfaces of the group-mean image are used as the initialization to reconstruct all longitudinal cortical surfaces simultaneously. To drive the deformable surfaces towards the target surfaces (inner, central and outer surfaces), a force derived from the Laplace’s equation [10] is adopted. Our method has been successfully applied to both simulated and real longitudinal images, demonstrating its validity.

Our proposed method has several advantages over exiting methods. First, temporal constraints are incorporated in longitudinal cortical surface reconstruction by jointly deforming cortical surfaces of all longitudinal images simultaneously in contrast to the existing methods which independently deform the cortical surfaces from the mean image to each time point [8, 9]. Second, a force derived from Laplace’s equation [10] is used to drive the deformable surface to help preserve the topology of the deformable surface. Third, the group-mean image is obtained by a nonlinear groupwise registration method in contrast to the rigid alignment used in the existing methods [8, 9]. At last, a longitudinally-consistent tissue segmentation method is adopted to facilitate the longitudinally consistent cortical surface reconstruction.

2 Methods

Given longitudinal brain MR images of a subject, our method for consistent cortical surface reconstruction consists of the following 4 major steps, as shown in Fig. 1. First, longitudinal images are preprocessed. Second, longitudinal images are groupwisely registered to obtain a group-mean image, and also their tissues are consistently segmented into WM, GM, and CSF. Third, the cortical surfaces of the group-mean image are reconstructed using a deformable surface method. Finally, the cortical surfaces of the group-mean image are warped to each longitudinal image and jointly deformed to reconstruct longitudinal cortical surfaces.

Fig. 1.

(a) The flow chart of the longitudinal cortical surface reconstruction method. (b) The flow chart of the cortical surface reconstruction for the group-mean image.

2.1 Preprocessing

The preprocessing procedure includes the following steps: (1) intensity inhomogeneity correction, (2) rigid registration of follow-up images onto the baseline image using FLIRT, (3) removing of non-brain tissues of the baseline images, and (4) masking of the brains of follow-up images using the brain mask of the baseline image.

2.2 Groupwise Registration and Consistent Longitudinal Tissue Segmentation

Considering the nonlinear longitudinal changes of brains, instead of averaging rigidly aligned images as did in existing methods [8, 9], a groupwise registration method [11] is adopted to obtain the group-mean image of longitudinal images, as well as the deformation fields from each longitudinal image to the group-mean image. To achieve longitudinally-consistent tissue segmentation, CLASSIC [12] is adopted to perform tissue segmentation on longitudinal images. To recover deep and narrow sulci, we use the ACE method [3] to modify the segmented GM volume to generate a no-more-than-one-voxel thick separation between opposite sulcal GM banks.

2.3 Cortical Surface Reconstruction for Group-Mean Image

To reconstruct cortical surfaces of the group-mean image, the inner cortical surface is reconstructed first in our method, and then the inner surface is deformed to reconstruct both central and outer surfaces. To obtain the topologically-correct inner surface, the topology of the WM volume is first corrected by a graph based method in [5] to ensure a spherical topology, and then the Marching cubes method is used to convert the boundary of the corrected WM volume to an explicit surface representation. Using a deformable surface method, this reconstructed rough inner surface is deformed under imposed forces to obtain the refined inner surface, as well as to obtain the reconstruction of the central and outer surfaces one-by-one. Since the deformable surface for cortical surface reconstruction from a single image can be considered as a special case of longitudinal cortical surface reconstruction, the general deformable surface will be detailed in Section 2.4. Fig. 2 shows an example of the reconstructed inner, central, and outer surfaces from an image by our method.

Fig. 2.

Reconstructed surfaces of an image. (a) Inner surface. (b) Central surface. (c) Outer surface. (d) Surfaces overlaid on a slice. Dark blue: inner; orange: central; light blue: outer.

2.4 Cortical Surfaces Reconstruction for Longitudinal Images

The cortical surfaces of the group-mean image are warped to each longitudinal image and further jointly deformed for longitudinal surface reconstruction using a deformable surface method. For longitudinal surface reconstruction with n time points, the deformable surfaces at 4D (3D spatial + 1D temporal) domain are parameterized as {x_t(u) = [x_t(u), y_t(u), z_t(u)]^T, u = (u₁, u₂) ∈ [0, 1] × [0, 1], t ∈ {1,…,n}}, which can be obtained by minimizing the following energy function:

$$E=\sum _{t=1}^n(\int \frac{1}{2}(\alpha \sum _{i=1}^2{\mid {\mathbf{x}}_{t,i}\mid }^2+\beta \sum _{i,j=1}^2{\mid {\mathbf{x}}_{t,ij}\mid }^2+\gamma {\mid {\mathbf{x}}_t^{\prime }\mid }^2)+E^{ext}\left({\mathbf{x}}_t\right)d\mathbf{u})$$ (1)

where parameters α and β control the tension and rigidity of surfaces, respectively. And the parameters γ controls the temporal smoothness of surfaces. x_t,i and x_t,ij denote the first and second partial derivative of x_t w.r.t. u_i, respectively. And ${\mathbf{x}}_t^{\prime }$ denotes the finite difference of x_t w.r.t. t. E^ext (x_t) is the external energy derived from the image at time t. The solution to the above energy minimization problem can be obtained by solving the following dynamic equation [4, 7]:

$${\mathbf{x}}_{t,\tau }\left(\tau \right)={\mathbf{F}}^{int}\left({\mathbf{x}}_t\left(\tau \right)\right)+{\mathbf{F}}^{ext}\left({\mathbf{x}}_t\left(\tau \right)\right)$$ (2)

where the internal force ${\mathbf{F}}^{int}\left({\mathbf{x}}_t\left(\tau \right)\right)=\alpha {\Delta }_u{\mathbf{x}}_t\left(\tau \right)-\beta {\Delta }_u\left({\Delta }_u{\mathbf{x}}_t\left(\tau \right)\right)+\gamma {\mathbf{x}}_t^{\prime \prime }$ and the external force F^ext(x_t(τ)) will be defined later. ${\Delta }_u={\partial }^2∕{(\partial u_1)}^2+{\partial }^2∕{(\partial u_2)}^2$ is the Laplacian operator to u, ${\mathbf{x}}_t^{\prime \prime }$ denotes the second order finite difference of x_t w.r.t. t. The first two terms in F^int(x_t(τ)) is the spatial regularizing force, and the third term is the temporal regularizing force, enforcing the temporal smoothness along the time t. Note that the deformable surface is treated as a function of surface evolution time τ. If only one time point exists, the temporal regularizing force will be 0 and the deformable surface can be used for surface reconstruction for the group-mean image as mentioned above.

The external force driving initial surfaces towards target surfaces is designed as:

$${\mathbf{F}}^{ext}\left({\mathbf{x}}_t\left(\tau \right)\right)=\text{G}\left({\mathbf{x}}_t\left(\tau \right)\right)\cdot {\mathbf{F}}^{GM}\left({\mathbf{x}}_t\left(\tau \right)\right)+(1-G\left({\mathbf{x}}_t\left(\tau \right)\right))\cdot {\mathbf{F}}^{NonGM}\left({\mathbf{x}}_t\left(\tau \right)\right)$$ (3)

where G(x_t(τ)) is the GM indicator function and F^GM(x_t(τ)) is a force activated inside of GM and derived from the Laplace’s equation [10]. F^NonGM(x_t(τ)) is a force activated outside of GM and defined as:

$${\mathbf{F}}^{NonGM}\left({\mathbf{x}}_t\left(\tau \right)\right)=D\left({\mathbf{x}}_t\left(\tau \right)\right)\cdot (2W\left({\mathbf{x}}_t\left(\tau \right)\right)-1)\cdot \mathbf{n}\left({\mathbf{x}}_t\left(\tau \right)\right)$$ (4)

where W(x_t(τ)) is the WM indicator function, and n(x_t(τ)) is the outward-oriented unit normal vector. D(x_t(τ)) is the force strength at vertex x_t(τ). For inner and outer surface reconstruction, D(x_t(τ)) is the distance along the direction of (2W(x_t(τ)) − 1)·n(x_t(τ)) to the WM/GM and GM/CSF interfaces, respectively. For central surface reconstruction, D(x_t(τ)) is set as the average distance along the direction of (2W(x_t(τ)) − 1)·n(x_t(τ)) to WM/GM and GM/CSF interfaces.

F^GM(x_t(τ)) is derived from Laplace’s equation of the GM [10], which is a second-order partial differential equation for a scalar field φ enclosed between two interfaces:

$$\Delta \phi =\frac{{\partial }^2\phi }{\partial x^2}+\frac{{\partial }^2\phi }{\partial y^2}+\frac{{\partial }^2\phi }{\partial z^2}=0$$ (5)

By setting the WM as the minimal value and the CSF as the maximum value, the Laplace’s equation is solved inside of the GM to obtain the harmonic function. The normalized gradient vector field of the harmonic function T(x_t(τ)) and the streamlines to both WM/GM and GM/CSF interfaces are computed for each point in GM. The Laplace’s equation establishes a one-to-one correspondence between WM/GM and GM/CSF interfaces and the streamlines of the harmonic function never intersect each other. This elegant property helps preserve the topology of the deformable surface. Denote the streamline lengths from a point x_t(τ) in GM to WM/GM and GM/CSF interfaces as L₀(x_t(τ)) and L₁(x_t(τ)), respectively. F^GM(x_t(τ)) for inner, central and outer surfaces reconstruction are respectively defined as:

$${\mathbf{F}}_{inner}^{GM}\left({\mathbf{x}}_t\left(\tau \right)\right)=-\mathbf{T}\left({\mathbf{x}}_t\left(\tau \right)\right)\cdot L_0\left({\mathbf{x}}_t\left(\tau \right)\right)$$ (6)

$${\mathbf{F}}_{central}^{GM}\left({\mathbf{x}}_t\left(\tau \right)\right)=\mathbf{T}\left({\mathbf{x}}_t\left(\tau \right)\right)\cdot (L_1\left({\mathbf{x}}_t\left(\tau \right)\right)-L_0\left({\mathbf{x}}_t\left(\tau \right)\right))$$ (7)

$${\mathbf{F}}_{outer}^{GM}\left({\mathbf{x}}_t\left(\tau \right)\right)=\mathbf{T}\left({\mathbf{x}}_t\left(\tau \right)\right)\cdot L_1\left({\mathbf{x}}_t\left(\tau \right)\right)$$ (8)

The central idea is that the direction of F^GM(x_t(τ)) should point toward the target surface and the magnitude of F^GM(x_t(τ)) should be directly proportional to the distance to the target surface. Fig. 3 illustrates the streamlines to GM/CSF interface and the force directions in GM.

Fig. 3.

Illustrations of the streamlines to GM/CSF interface and the force directions in GM. (a) Streamlines. (b) Zooming view of the yellow rectangular region in (a). (c) and (d) Force directions for central and outer surface reconstruction, respectively.

To prevent from self-intersection, a triangle-triangle intersection detection method [13] is used in triangle faces contained in a local sub-volume. Once self-intersection is detected, the deformation is cropped to a valid location. Fig. 4 shows an example of reconstructed longitudinal outer surfaces of a subject with color-coded cortical thickness, calculated as the closest distance between the inner and outer surfaces. We can observe the overall decline trend of the cortical thickness in aging.

Fig. 4.

An example of reconstructed longitudinal outer surfaces, color-coded by thickness.

3 Results

Data used in experiments are from ADNI [15]. The parameter β controls the rigidity of the deformable surface and is set as 0, as suggested in [4, 7]. Experimentally, α and γ are set as 0.25 and 0.1, respectively. In experiments, we also find that results are not sensitive to subtle changes of parameters.

Real Data

To evaluate the accuracy of the longitudinal inner and outer cortical surface reconstruction results, we compare the GM volume in the tissue-segmented image by CLASSIC [12] (denote as A) with the GM volume enclosed by the corresponding reconstructed inner and outer surfaces (denote as B). Three statistical values are calculated, including: (1) true positive: (A ⋂ B)/A, (2) false negative: (A ⋂ B̄)/A, (3) false positive: (Ā ⋂ B)/A, similar to the measurements adopted in [14]. To evaluate the accuracy of the longitudinal central cortical surface, we calculate the percentage of vertices of the reconstructed central surface falling outside the GM (non-GM vertices), as adopted in [7], since the central surface is converged to the inside of GM. The proposed method is applied to 10 normal healthy subjects, each with 4 longitudinal scans. The average true positive, false negative, false positive and percentage of non-GM vertices of 4 time points for each subject are shown in Fig. 5 (a). The above measurements are further compared to the values reported in the literature. Compared to those reported in [14] for validation of inner and outer surfaces, the average true positive of our method is around 0.77, which is higher than the value in [14] which is less than 0.70. And the average false negative is 0.23, compared to the value of around 0.35 in [14]. And the false positive of our method is 0.15, which is similar to the value in [14]. Compared to those reported in [7] for validation of central surfaces, the average percentage of non-GM vertices of the reconstructed central surfaces by our method is around 0.03, which is less than the value of 0.06 in [7]. Although different dataset are adopted in [7] and [14], we believe that this comparison can reflect the accuracy of our method to some degree.

Fig. 5.

(a) The average true positive, false negative, false positive and percentage of non-GM vertices of 4 time points for each subject. (b) The average distance errors of inner, central and outer surfaces of 4 time points for each subject compared to the “ground truth”.

Simulated Data

To further validate the inner and outer surfaces, we simulate images from the reconstructed inner and outer surfaces using the method in [14]. Briefly, the voxels inside the inner surface are labeled as WM, and the voxels between inner and outer surface are labeled as GM, and the voxels between skull and outer surface are labeled as CSF. The longitudinal image sequences are simulated from the corresponding reconstructed inner and outer surfaces. Then the inner and outer surfaces reconstructed from the simulated longitudinal images are compared to those original cortical surfaces, which are treated as the “ground truth”. The average distance errors between the two sets of surface are calculated and further averaged for 4 time points of each subject. To validate the central surfaces, we also simulate images from the reconstructed central surfaces using the method in [7]. Specifically, a thickness value following a Gaussian distribution with the mean 3.0mm and variance 1.0mm is generated for each point of the central surface, and all voxels inside the half thickness range are labeled as GM. Also, all voxels enclosed by GM are defined as WM, and all voxels between the skull and GM are labeled as CSF. The reconstructed central surfaces from the simulated longitudinal images are compared to those original central surfaces. The distance errors of inner, central, and outer surfaces of 10 simulated subjects are shown in Fig 5(b). The average distance errors of inner, central and outer surfaces are all around 0.6mm, indicating the accuracy of our method.

Longitudinal Thickness Changes

To test the capability of consistently capturing longitudinal cortical changes, we apply the proposed method on 15 normal healthy subjects and calculate the cortical thickness. The trajectory of the average cortical thickness for each subject is shown in Fig. 6 (a). For comparison, the results from our method without temporal constraint (by setting γ = 0) is shown in Fig. 6 (b). For quantitative comparison, a linear regression is performed on the longitudinal curves of average thickness of each subject, and the residuals are calculated. The average residuals of our method with and without temporal constraint are 0.01 ± 0.005mm and 0.024 ± 0.02mm, respectively. As we can see, the results with temporal constraint are more longitudinally consistent and smoother than those without temporal constraint.

Fig. 6.

The longitudinal change of average cortical thickness on 15 normal subjects. (a) Our method with temporal constraint. (b) Our method without temporal constraint.

Comparison with FreeSurfer

To further demonstrate the consistency of our method, we compare the trajectories of vertices on reconstructed longitudinal outer surface with those obtained from the longitudinal processing pipeline in FreeSurfer [8]. For quantitative evaluation, we calculate the ratio between the length of trajectory and the straight line distance of the first and the last time-point vertices in the longitudinal surfaces. Smaller ratio indicates smoother trajectory. Fig. 7 (a) and (b) show a comparison of the trajectory cropped from longitudinal surfaces obtained by our method and FreeSurfer, respectively. The distributions of ratios of whole surfaces are shown in Fig. 7 (c) and (d). Clearly, the trajectories from our method are much smoother than those obtained from FreeSurfer, since no temporal constraint is incorporated in longitudinal surface reconstruction in FreeSurfer.

Fig. 7.

Comparison of consistency of longitudinal surfaces between our method and FreeSurfer. (a) and (b) The trajectories color-coded by ratios obtained from our method and FreeSurfer, respectively. (c) and (d) The distribution of ratios by our method and FreeSurfer, respectively.

4 Conclusion

This paper presented a new method for consistent reconstruction of cortical surfaces from longitudinal brain MR images and experimental results demonstrate its validity. In future work, we will validate our proposed method using more longitudinal data.