Morphological Appearance Manifolds for Group-wise Morphometric Analysis

Abstract

Computational anatomy quantifies anatomical shape based on diffeomorphic transformations of a template. However, different templates warping algorithms, regularization parameters, or templates, lead to different representations of the same exact anatomy, raising a uniqueness issue: variations of these parameters are confounding factors as they give rise to non-unique representations. Recently, it has been shown that learning the equivalence class derived from the multitude of representations of a given anatomy can lead to improved and more stable morphological descriptors. Herein, we follow that approach, by approximating this equivalence class of morphological descriptors by a (nonlinear) morphological appearance manifold fit to the data via a locally linear model. Our approach parallels work in the computer vision field, in which variations lighting, pose and other parameters lead to image appearance manifolds representing the exact same figure in different ways.

The proposed framework is then used for group-wise registration and statistical analysis of biomedical images, by employing a minimum variance criterion to perform manifold-constrained optimization, i.e. to traverse each individual's morphological appearance manifold until group variance is minimal. The hypothesis is that this process is likely to reduce aforementioned confounding effects and potentially lead to morphological representations reflecting purely biological variations, instead of variations introduced by modeling assumptions and parameter settings.

1. Introduction

Computational anatomy has been a powerful tool for characterizing differences between normal and pathologic anatomies by analyzing their variations relative to a common template. Diffeomorphic shape transformations (Miller et al., 1993; Davatzikos et al., 1996; Thompson et al., 2000; Chetelat et al., 2002; Guo et al., 2005; Yushkevich et al., 2008; Taron et al., 2009) are first estimated to warp all anatomies to a template or vice versa; various descriptors are then derived to quantify their morphological characteristics.

Template transformations are often derived from image similarity measures, in the intensity-driven methods (Miller et al., 1993; Ashburner and Friston, 1999), either by employing feature or intensity differences or via mutual information (Christensen et al., 1993; Thompson and Toga, 1996; Woods et al., 1998; Rueckert et al., 1999; Johnson and Christensen, 2002; Bhatia et al., 2004; Avants et al., 2006). Topology is maintained by imposing smoothness constraints either via physical models (Christensen et al., 1993; Christensen and Johnson, 2001) or directly on the deformation field (Karacali and Davatzikos, 2004). Other approaches, such as (Thompson and Toga, 1996; Shen and Davatzikos, 2002; Wang et al., 2003), ensure biological correspondence through feature-based approaches by introducing biologically, anatomically and geometrically significance attributes in shape morphological representations.

Various morphometric analysis approaches have been presented in the literature, under the umbrella of computational anatomy: large deformation diffeomorphic metric mapping (LDDMM) (Joshi, 1998; Miller et al., 2002), deformation based morphometry (DBM) (Davatzikos et al., 1996; Ash-burner et al., 1998; Collins et al., 1998; Gaser et al., 1999; Cao and Worsley, 1999; Thirion and Calmon, 1999; Chung et al., 2001), voxel based morphometry (VBM) (Ashburner and Friston, 2000; Davatzikos et al., 2001; Baron et al., 2001; Bookstein, 2001; Chetelat et al., 2002), tensor based morphometry (TBM) (Thompson et al., 2000; Leow et al., 2006), depending on the aspects of the template transformation being measured. DBM, for instance, establishes group differences based on local deformation of anatomical structures through the Jacobian of the diffeomorphism, ignoring any potential residual between the warped template and the target shape. VBM, on the other hand, factors out global differences via a relatively less aggressive transformation, before analyzing anatomical differences captured by the residuals. VBM is, therefore, considered as complementary to DBM, since the former utilizes the information not represented by the transformation. The “modulated” VBM (Good et al., 2001) combines elements from DBM and VBM, albeit without a systematic treatment of the optimal way to balance template warping with residuals.

The inherent complexity of the problem poses a major challenge to these approaches. First, different parameters, the most important being the amount of regularization and the template, lead to different solutions when applied to the same exact anatomy. Second, anatomical correspondence may not be uniquely, or optimally, determined from intensity-based image attributes, which drive template warping algorithms. Third, exact anatomical correspondence may not exist at all due to anatomical variability across subjects. The aforementioned challenges lead to residual information that the transformation fails to capture. Even worse, this residual is inconsistent across different individuals, depending on how much they resemble the template.

To partially remedy this problem, some approaches have been proposed to use average anatomies as templates (Davis et al., 2004; Avants et al., 2006), to facilitate the template matching procedure. In most practical cases, considerable differences still persist between samples and the average brain. A very promising approach in this situation is group-wise registration (Durrleman et al., 2009; Bhatia et al., 2004; Twining et al., 2005; Allassonniere et al., 2007), which solves the problem to a certain extent, in the sense that instead of minimizing individual dissimilarity it minimizes combined cost. Bhatia et al. (2004), for instance, implicitly find the common coordinate system by constraining the sum of all deformations from itself to each subject to be zero. Davis et al. (2004) compute the most representative template image through a combined cost functional on the group of diffeomorphisms. Some researchers derived the combined cost functionals based on statistical models (Glasbey and Mardia, 2001; Durrleman et al., 2009; Allassonniere et al., 2007). Such group-wise registration based representations are, therefore, more consistent across the samples. However, regardless whether the template is chosen a priori or estimated from the data via an appropriate averaging process, it is still a single template. As a result, anatomies that fall close to the template are teated fundamentally differently by the template warping process, relative to anatomies that are further away. Although some methods used multiple templates - Sabuncu et al (2008) clustered images into different modes and Glasbey and Mardia (2001) generated different templates for each group of images of same fish, they either applied group-wise morphometric analysis on each group separately or connected different groups by simple affine registration. Moreover, other parameters of the registration system, including the smoothness level, still influence the obtained morphological representations.

In the approach presented herein, we follow and build upon the work of Makrogiannis et al. (2007); Baloch et al. (2007); Baloch and Davatzikos (2009), which uses a complete morphological descriptor of the form [Transformation, Residual]; any morphological information not captured by the transformation is captured by the residual, hence no morphological characteristic is discarded. An entire class of many anatomically equivalent descriptors is generated by varying parameters of the template transformation, as well as the template itself, all representing the same anatomy. The resultant anatomical equivalence class (AEC) is approximated herein by a morphological appearance manifold (MAM), containing all the combinations of [transformation, residual] that build an individual anatomy. Although one limitation of such representation is that such manifolds can be nowhere differentiable (Wakin, 2006), they can become differentiable by smoothing the images. Instead of directly smoothing the images prior to estimating a MAM, herein we follow a different approach: we estimate its local structure by fitting a hyperplane that approximates its tangent plane. In other words, we locally approximate the highly nonlinear manifold of an AEC by a locally linear manifold (hyperplane), and we keep updating this approximation as we move along the AEC. In a group-wise registration framework, among the infinitely many members of an AEC of each individual, a unique representation is selected according to the criterion that the group-wise variance is minimized. This problem is solved via a manifold-constrained optimization approach, which locally estimates the structure of the MAM via local PCA, and moves along the manifold to minimize the group variance. Intermediate steps of projection onto the manifold are necessary to guarantee that each individual's representation remains on its respective manifold, so that the person's anatomy is not distorted. Standard voxel-based analysis methods are then used to compare individuals and groups, however they are applied to the unique and optimal morphological signatures (OMS) obtained through this optimization.

In practice, it is impossible to sample all possible diffeomorphisms and respective residuals. Therefore, we have chosen to vary the two most important sources of variation in practice: the regularization weight, which determines how aggressive the template warping is; and the template, whose similarity with an individual anatomy can significantly affect the resultant morphological representation. By varying these two factors, we obtain an estimate of the structure of the MAM of each individual. An optimization approach then allows the representation of each individual anatomy to traverse its respective MAM, until a minimum variance criterion in the entire group is achieved (we will elaborate on this criterion later in the paper).

This paper builds upon the work in (Makrogiannis et al., 2007; Baloch et al., 2007; Baloch and Davatzikos, 2009), and presents several novel directions. First, unlike the work in (Makrogiannis et al., 2007; Baloch et al., 2007; Baloch and Davatzikos, 2009) which used a globally linear approximation, we use nonlinear estimation of the manifold by using a local linear approximation of its tangent plane. We demonstrate through examples that MAMs are highly nonlinear, which demonstrates that the previous work (Makrogiannis et al., 2007; Baloch et al., 2007; Baloch and Davatzikos, 2009) using linear approximations are likely to be insufficient, and which has motivated the use of nonlinear approximations. Second, the current study investigates the effect of varying the relative weight of the transformation and the residual, which is very important because it effectively investigates different distance metrics. Third, instead of finding a single optimal solution for the entire image, as has been done in the past (Makrogiannis et al., 2007; Baloch et al., 2007; Baloch and Davatzikos, 2009), we determine regionally optimal solutions. This is very important, since the optimal template or regularization parameter for one part of the anatomy is not necessarily the same as the optimal solution for another anatomical part. Fourth, much of the work in (Makrogiannis et al., 2007; Baloch et al., 2007; Baloch and Davatzikos, 2009) was based on tissue density maps (Ashburner and Friston, 2000; Davatzikos et al., 2001) (TDM), which are commonly used to quantify regional volumetrics and can be shown to result from the product of the Jacobian determinant and the spatially normalized images. The current study uses a broader analysis by considering the transformation and the residual jointly, thereby allowing us to examine all pertinent information provided by them. In fact, our experimental results show that the distance of TDM is highly correlated with distance of residuals alone, and therefore it tends to unfertilized information captured by the template warping. Depending on the shapes and the smoothness of the transformation, this information can be very important. Along the same lines, the current study investigates the effect of using L₁, instead of L₂ norm as distance metric. The former generally tends to be more robust to outliers, albeit more difficult to handle analytically. Finally, this paper provides an extensive set of experiments using both simulated and real datasets.

The approach described herein is akin to methods on image appearance manifolds that have been used in computer vision (Ham and Lee, 2007) to model variations in images that are caused by measurement parameters, such as illumination and pose. Such variations are confounding factors when, for example, one is interested in face recognition, as the same person appears different for different parameters. As in our work, learning such variations is important for determining robust parameters to be used for analysis and recognition.

Our work is also akin to the work in (Younes, 2007), which is known as the theory of metamorphosis. A fundamental difference of our approach, however, is that it uses a constrained optimization framework to restrict morphological representations of an individual to lie on his/her MAM. In contrast, metamorphosis is primarily concerned with modeling minimum energy paths from an individual to another, by jointly varying the residual and the transformation. These two approaches are therefore highly complementary, in that metamorphosis can be used in conjunction to the approach described herein to model differences between MAMs of different individuals, the latter the focus of our study.

2. Anatomical Equivalence Class Manifold

Computational anatomy involves characterizing anatomical differences between a subject S and a template T by mapping the template space Ω_T to the subject space Ω_S through a diffeomorphism h : Ω_T → Ω_S; x ⟼ h(x) by maximizing some similarity criterion between T and normalized subject S(h(x)). A zero residual mapping is usually very difficult if possible at all, resulting in a residual:

$$R_h\left(\mathbf{x}\right)≔T\left(\mathbf{x}\right)-S\left(h\left(\mathbf{x}\right)\right),\mathbf{x}\in {\Omega }_T$$ (1)

Our approach herein is to use a template warping algorithm that captures some of the morphological differences between the template and the target anatomies, in combination with the respective residual, which captures everything else. As in (Baloch et al., 2007), the transformation and the residual are combined into a concatenated descriptor; a weight μ is used herein to determine the relative weight of the two terms:

$${\mathcal{Q}}_h\left(\mathbf{x}\right)≔(\mu J_h,R_h),$$ (2)

where J_h is the Jacobian determinant of the transformation h. In (Lian and Davatzikos, 2008), a slightly different way of combining h(.) and R_h(.) was used, for quantifying patterns of local variations of sizes of biological tissues, as, for example, brain atrophy. Such local volumetric measurements can be quantified by TDM (Ashburner and Friston, 2000; Davatzikos et al., 2001), defined as follows:

$${\mathcal{Q}}_h\left(\mathbf{x}\right)≔J_h\left(\mathbf{x}\right)[T\left(\mathbf{x}\right)-R_h\left(\mathbf{x}\right)],\mathbf{x}\in {\Omega }_T,$$ (3)

That approach is convenient, in that it automatically scales the Jacobian determinant and the residual into a quantity that has a clear biological meaning: it reflects the amount of tissue present in a spatial location. However it is not necessarily the best way to combine these two entities.

For brevity, we refer to the morphological representation obtained via ${\mathcal{Q}}_h(.)$ as a complete morphological descriptor (CMD); the term “complete” stems from the fact that the residual completes the representation obtained via the transformation h(.) so that no morphological information is discarded.

${\mathcal{Q}}_h$, depends not only on the underlying anatomy but also on transformation parameters. An entire family of anatomically equivalent CMDs - they are equivalent to reconstruct the same anatomy, and form an equivalence class (the AEC) - may be generated by varying h. In practice, it is impossible to sample all possible transformations and residuals to characterize an AEC, nor is it desirable, as we would try to derive an AEC derived mostly from anatomically meaningless transformations. Therefore we concentrate on two important parameters for variations of h: h_λ,τ, where λ denotes the amount of regularization and τ denotes an intermediate template (Fig. 1). The main premise in using multiple intermediate templates is to characterize and remove a variability of the morphological descriptors that relates to lack of transitivity of the underlying transformations. In particular, if we were to go from a source to a target image via different intermediate steps, we should obtain the same final transformation (and residual). This is not guaranteed in practice, since deformable registration algorithms often converge to local minima. The resultant morphological descriptor then displays a variability that reflects not only true morphological variations, but also the aforementioned confounding effect. To capture such undesirable variations that arise from varying the template, we use many different intermediate templates, each producing its own morphological descriptor for a given anatomy. Since analysis eventually has to be carried out in a common space, we ultimately bring all the normalized anatomies to the template space Ω_T , however intermediate templates capture the variation we expect to see when the same anatomy is seen “via different templates”. Variations of the regularization parameter λ reflect variations observed by varying the degree of conformality of the transformation, with larger values of λ indicating very smooth transformations and large residuals, and λ = 0 indicating the most conforming transformation of the template under the assumptions of a respective deformable registration algorithm that seeks to minimize the residual. Fig. 2 shows the AEC obtained by varying intermediate templates and regularization parameters. Each AEC was obtained by continuously varying either τ along principal directions of space of deformation fields (Tang et al., 2009), or λ from small to large values. Since these measurements are embedded in a very high-dimensional space, for visualization purposes we calculated the first three principal components and projected onto them, so that 3D plots can be shown. The AEC in Fig. 2(b) clearly has a 1-D manifold structure, which is expected since it results from the variation of one parameter: λ. The AEC in Fig. 2(a) has a more complex structure, since variations of different templates cannot be captured by a small number of parameters.

Figure 1.

Morphological Appearance Manifold

Figure 2.

Manifold of varying (a) intermediate templates or (b,c) regularization parameters. (b,c) represent different images. Each sample on manifolds represents a CMD.

For tractability and notational simplicity, we combine confounding factors together to a parameter vector θ := (τ,λ); the corresponding definition of AEC becomes

$$\mathcal{A}\left(S\right)=\{\{\left({\mathcal{Q}}_{h_{\theta }}\left(\mathbf{x}\right)\right):S\left(h_{\theta }\left(\mathbf{x}\right)\right)=T\left(\mathbf{x}\right)-R_{h_{\theta }}\left(\mathbf{x}\right),\forall \mathbf{x}\in {\Omega }_T\},\forall \theta \in ϴ\}$$ (4)

This equation effectively says that “the AEC of a given individual comprises all possible ways of representing the morphology of that individual via a transformation of a template and a respective residual, obtained by varying the parameter vector Θ”. We approximate the AEC by a manifold, the morphological appearance manifold (MAM), which herein is swept as λ and τ are varied. (Other parameters can also be varied, at the expense of computational complexity.)

Finding optimal morphologic signatures (OMS) on these MAMs, a unique one for each anatomy, needs a suitable CMD feature and manifold approximation, which are the topics in the two following sections, respectively.

3. Feature Selection and Optimization Criterion

Recall that varying Θ = [θ₁, ..., θ_N] effectively allows every individual representation to slide along its own manifold, thereby leading to multiple ways of representing each individual. The goal of the optimization procedure presented in this section is to find a single optimal solution for each individual, i.e. to find a unique morphological representation for that individual. The optimality criterion is minimal group variance: each individual anatomy slides along its own manifold until the group variance is minimized, which is assumed to occur when the confounding effects of parameters and templates are removed; the remaining differences are presumed to better reflect true biological differences instead of modeling differences. Specifically,

$${ϴ}^{\ast }=\arg \underset{ϴ}{\min }\sum _{i,j=1}^N{\Vert {\mathcal{Q}}_{h_{{\theta }_i}}-{\mathcal{Q}}_{h_{{\theta }_j}}\Vert }_l^l$$ (5)

where ∥ · ∥_l is the l-norm distance criterion (l = 1 or 2), and ${\mathcal{Q}}_{h_{{\theta }_i}}$ is CMD, either of combined feature (2) or TDM (3). In our experiments later, the optimization (5) is solved by a greedy algorithm, that compute the optimal signature one subject by another through gradient direction until coverage.

The minimum variance criterion is motivated by the observation that Θ is a confounding factor, as it leads to different representations of the same exact anatomy. This type of variation clearly doesn't represent any true morphological difference, since the underlying anatomy is fixed, and therefore introduces artificial differences between different anatomies. For example, if we consider two anatomies only, our assumption is that if they are similar for some value of Θ , they must be similar anatomies. The respective vectors $\mathcal{Q}(.)$ might differ greatly for various values of Θ, even though the underlying anatomies are similar. It is reasonable then to use the minimum distance of the respective MAMs as a measure of distance between these two anatomies. Extending this to many anatomies in a group, we minimize the sum of pairwise square distances, which is equivalent to minimizing the group variance.

As mentioned in the introduction, TDMs have been used previously as a means to combine the Jacobian and the residual (Davatzikos et al., 2001; Good et al., 2001). By construction, the TDM offers an exact measurement of regional volumetrics of different tissues defined via segmentation, and has been used to quantify tissue growth or atrophy. However, TDM is not necessarily the best feature, since it is determined from pre-defined multiplicative relationship between the Jacobian and the residual that might or might not be the best way to capture morphological characteristics. In fact, it correlates significantly with the residual itself, as Fig. 3 shows (derived from the simulated shapes that will be discussed later in this paper). Minimizing the distance of TDMs may not optimally leverage upon the trade-off between the Jacobian and the residual, and might fail to find a solution that best reflects shape differences.

Figure 3.

Scatter plot between distance of the TDMs and distance of the residuals of 10000 groups of randomly selected samples

In general, we seek to find an optimal value for θ in (2) and then apply it in the criterion (5) to find a solution best reflecting morphological differences. Setting θ = 0 is equivalent to only minimizing distance of residuals and leads to aggressive, i.e. maximally conforming registration. Setting μ = ∞ is equivalent to only minimizing distance of Jacobians and leads to rigid transformations that leave large residuals. The best value of μ (which is between 0 ~ ∞) depends on the specific application. Herein, we consider a specific application of interest to many of our studies: estimating volumetric brain changes, which is commonly of interest to the study of neurodegenerative disease, such as Alzheimer's, and to studies of brain development.

3.1. Group difference detection

In order to investigate the relationship between the value of μ and our ability to detect morphological group difference, and therefore to determine a good range of μ, we used simulations. In particular, a group of simulated 2D shapes resembling a cortical convolution (sulcus) were used, since much of our interest is in the human brain; see Fig. 4. These shapes simulate different variations of cortical gray matter of the brain. Shapes in right column were obtained by shrinking the middle third of each of the ribbons of the left column. We used two groups of shapes: 1) centered - these shapes are roughly aligned around center location, which reflects a scenario in which a relatively good initial registration is available for different cortical sulci; 2) shifted - these shapes are misaligned by applying a random shift to each of them, which better reflects a more difficult scenario. These shapes were registered to 12 templates, using 37 different values of λ that is the level of smoothing on deformation field. The registration mechanism in these shapes was relatively simple, because the relative simplicity of the images: we used an elastic warping approach maximizing image similarity, in which λ was the weight of the elastic energy term and controlled the smoothness of the resultant deformation field. As customary in voxel-based analysis, the residuals were spatially smoothed by a Gaussian filter of standard deviation σ = 7 with voxel size as unit. In order to test whether the difference between these two groups could be identified, we applied the Hotelling T ² test (the t-test is commonly used when the measurements are scalar on each voxel, as in TDM). The logarithm of the minimum p-value was used as an indicator of significant group difference determined for a given μ. Note that the ground truth was known: it was atrophy introduced in the middle third of each ribbon.

Figure 4.

Simulated 2D shapes resembling sulci of the human brain. Some shapes (a) don't, and some shapes (b) do have thinning in the middle portion (middle third) of the ribbon

Fig. 5 shows the significance levels vs. the value of μ. Because the group difference in the centered shapes is mainly due to the shape thinning in their middle portion, a large value, μ = 5, yields best significance level. This indicates that when the starting point of the registration is very good (i.e. these folds are initially aligned and differ only by a very local deformation), aggressive registration is best for revealing group differences, since it is able to capture a great deal of shape detail. On the other hand, shifting the shapes effectively worsens the initialization of the registration, and therefore calls for a smaller value of μ = 3, which leaves much of the morphological detail to the residual. However, relatively stable significance levels were obtained for μ ∈ {2 ~ 6}. For example, the maximum of logarithm of minimum p-value for centered subjects is -8, still a good one. Therefore, we decide to choose one value for all experiments: μ = 3.

Figure 5.

The minimum p-value vs. the value of μ for simulated 2D subjects

3.2. Optimal weight for estimating longitudinal change

A second simulation was used to find a good value of μ for estimating longitudinal change, and in particular tissue atrophy. A real MR brain image was used as starting point, and a series of 12 scans were synthesized from the baseline scan, by simulating atrophy (shrinkage) in the superior temporal gyrus gradually, reaching up to 50% in the last scan (see Fig. 6). The thinking is that the MAM optimization framework will filter out the random “jitter” noise that is usually observed in serial scans, by shifting each scan along its MAM and finding the points that come closer to each other. Since this is the same brain over time, it is reasonable to expect that longitudinal scans should differ minimally, and therefore our optimization will achieve this goal, while still maintaining the longitudinal change, since each scan is constrained to lie on its manifold. We aimed to measure the accuracy of estimating longitudinal atrophy, as a function of μ. These images were registered to twelve templates and nine values of λ, detailed next. The registration method used in these experiments was HAMMER (Shen and Davatzikos, 2002), which was run at low, medium, and high-resolution. In each resolution, four different smoothness levels were used, controlled by the parameter λ. This resulted in overall 15 different levels of smoothness. Because tissue shrinkage was measured in this experiment, we computed the mean TDM of gray matter (the tissue in which atrophy was simulated) within the region of atrophy, after optimization was applied for different μ's. Linear curve-fitting was then applied over the 12 scans. The root of mean square error (MSE) was used as metric to find the best value of μ.

Figure 6.

The simulated atrophy at the superior temporal gyrus (right image).

Fig. 7 shows that the optimization can help reduce the MSE and the minimum MSE happens when μ = 1. Although this weight is different from the optimal weight for measuring group differences, this is expected, since these serial scans represented the same brain. One would then expect that the residual might be a more sensitive measure of longitudinal change, after the serial scans have been near-rigidly aligned. In contrast to this, inter-individual differences require a higher weight on the Jacobian determinant, since a more conforming deformation is necessary to register morphologies of different individuals.

Figure 7.

The root-MSE of linear curve fitting vs. the value of μ for simulated 3D MR images

4. Nonlinear Manifold Representation

In (Baloch et al., 2007; Baloch and Davatzikos, 2009), the AEC manifold ${\mathcal{Q}}_{h_{{\theta }_i}}$ was linearly approximated via its principal directions, computed via principal component analysis (PCA).

$${\mathcal{Q}}_{h_{{\theta }_i}}={\widehat{\mathcal{Q}}}_{h_{{\theta }_i}}+\sum _{j=1}^n{\alpha }_{ij}{V}_{ij}$$ (6)

where ${\widehat{\mathcal{Q}}}_{h_{{\theta }_i}}$ is the population mean, n is the number of eigen vectors that preserve 99% energy, and ${\alpha }_{ij}\in \left[{\alpha }_{ij}^{min}{\alpha }_{ij}^{max}\right]$ is the variation in the principal direction V_ij, which is originally represented by _i. However, the nonlinearity of the MAM (Fig. 2) calls for better approximations. In this paper, we sequentially construct the manifold by assuming linearity in a local neighborhood, and by updating the linear estimate as a point moves on the MAM. The same assumption is used for the popular nonlinear manifold learning methods ISOMAP (Tenenbaum et al., 2000) and LLE (Roweis and Saul, 2000), except herein we don't attempt to estimate the entire manifold, since we don't need it, but we only estimate the manifold around a trajectory that is traversed during the iterative solution to an optimization problem seeking the minimum variance points on a group of MAMs.

Under this assumption of local linearity, we apply PCA to represent the local neighborhood of the AEC manifold:

$${\mathcal{Q}}_{h_{{\theta }_i}}^{\left(\delta {\theta }_i^{center}\right)}\left({\theta }_i\right)={\widehat{\mathcal{Q}}}_{h_{{\theta }_i}}^{\left(\delta {\theta }_i^{center}\right)}+\sum _{j=1}^n{\alpha }_{ij}^{\left(\delta {\theta }_i^{center}\right)}{V}_{ij}^{\left(\delta {\theta }_i^{center}\right)}$$ (7)

where $\delta {\theta }_i^{center}$ is the local neighborhood around the center sample at ${\theta }_i^{center}$, and ${\alpha }_{ij}^{\left(\delta {\theta }_i^{center}\right)}\in \left[{\alpha }_{ij}^{\left(\delta {\theta }_i^{center}\right)min}{\alpha }_{ij}^{\left(\delta {\theta }_i^{center}\right)max}\right]$ is the variation in the principal direction $V_{ij}^{\left(\delta {\theta }_i^{center}\right)}$. This optimization problem is solved iteratively. During each iteration, the local linear approximation of the MAM is recomputed, using 10% of AECs on the manifold, centered on the current optimal point ${\mathcal{Q}}_{h_{{\theta }_i^{\ast }}}$:

$${\theta }_i^{center}={\theta }_i^{\ast }$$ (8)

and all principal directions were used to represent the local space. We refer to this optimization process as manifold-constrained optimization; see Fig. 8.

Figure 8.

Manifold-constrained optimization: the AEC of each individual resembles a manifold embedded in R^N, where N is the dimensionality of the measurement space. An optimal member of each person's AEC is found by locally approximating the structure of this manifold with PCA, and iteratively traversing the manifold until a certain criterion of minimum variance is met. This procedure removes variations that are introduced by confounding variables during the calculation of a template transformation.

However, setting the current point as the center for local MAM approximation tends to render the optimization vulnerable to local minima nearest to initial point. Increasing the neighborhood size may somewhat alleviate the problem, but also can lead to poor global linear approximation of a highly nonlinear manifold. We have found that it is very helpful to interleave the optimization steps described above with long steps, in which the center of each individual's local neighborhood is set so that the distance criterion (5) is optimized,

$${\theta }_i^{center}=\arg \underset{k}{\min }\sum _{j\ne i}{\Vert {\mathcal{Q}}_{h_{{\theta }_{ik}}}-{\mathcal{Q}}_{h_{{\theta }_j^{\ast }}}\Vert }_l^l$$ (9)

where ${\mathcal{Q}}_{h_{{\theta }_{ik}}}$ is the kth available sample on the i-th manifold.

Our algorithm can be summarized as four steps,

• 1)Select a sample on each MAM as initial optimal point. It could be the mean of MAM or any one sample on MAM.
• 2)Choose 10% of AECs, centered at current optimal point, to construct a local space using (7).
• 3)Compute the new optimal point in the local space based on (5). Update the optimal points using (9).
• 4)Repeat 2)-3) until converge.

The complexity of the proposed manifold-constrained optimization primarily depends on the length of data, sample number of each manifold, and the number of subjects in optimization. It takes only around 20 minutes to complete the manifold-constrained optimization of the hippocampus region of real 3D images used in Section 6.5. The dataset has the size 61 × 61 × 61 of each sample, 90 samples on each manifold, and 30 subjects. For the large dataset, the computational time is relatively small.

5. Spatially-varying optimization

Although in most of our experiments we run the optimization within relatively small 3D images or 2D shapes, practical applications call for optimization over the entire anatomy of interest, e.g. an entire brain or cardiac scan. This introduces complications. First, applying our optimization to 3D scans from a group of subjects would require an unacceptably large size of memory. For example, selecting the top 10 eigenvectors and using 4-byte float to store each value, the optimization over 100 subjects requires at least 256 × 256 × 210 × 100 × 10 × 4 = 55G bytes of memory. Clearly, this is not a feasible solution for standard computers or for reasonable sample sizes.

More importantly, the optimal solution generally tends to be spatially varying. For example, a template and regularization constant that is optimal for the superior frontal sulcus in the brain is not necessarily optimal for the inferior temporal sulcus, even within the same individual.

To overcome this fundamental limitation, but also to render the computations possible on regular computers, we solve the optimization problem regionally - we divide the whole image into small regions, optimize these regions separately, and combined OMSs of all regions as final OMS image. To avoid discontinuities close to the boundaries of different regions, we use overlapping subregions. This is a computationally efficient simplification of the more complex problem of applying continuity boundary conditions between different subregions, so that optimal solutions will vary smoothly throughout the image. Solving this latter problem is beyond the scope of the current work and one of our future directions of research.

6. Experimental results

To evaluate our optimization method, we applied it to five datasets. The first dataset has already been shown and was a group of simulated 2D ribbons with varying shapes and with a single focus of atrophy simulated randomly in half of them in the middle third of the ribbons: these were used in Section 3.1. The second is a group of 2D shapes also resembling the human brain and having two foci of atrophy: one in the (simulated) gray matter and one in the white matter. The third is a group of 2D shapes with two convolutions, each of which has a simulated atrophy region; this set was used to test the fact that different optimal Θ's are estimated for different regions, i.e. that a spatially varying optimization yields better results than a global optimization. The fourth is a group of real 3D MR brain images with simulated atrophy to be used as ground truth. The fifth is 3D MR brain images from normal elderly individuals and from patients with mild cognitive impairment (MCI). To investigate group differences, voxel-wise statistical analysis is applied using Hotelling's T ² test on the unoptimized and optimized morphological descriptors. The resultant p-value spatial maps are then compared against the known truth, when the latter is known. Gaussian filters are used to smooth the residual prior to the voxel-wise statistical analysis, as customary in voxel-based analysis.

6.1. Simulated 2D shapes with one convolution

As has already been described, the 2D simulated shapes represented the morphological variations we find in the cortex of the human brain. Half of them had thinning in the middle (central) third of the ribbon, and half didn't. The aim here was to determine whether the MAM-based optimization can better detect this subtle morphological difference between the two groups, in the presence of rather high morphological variability across shapes.

The optimal intermediate templates and optimal λ of three shapes are shown in Fig. 9. The different shapes have different optimal intermediate templates. The optimal λ of the three shapes are 8, 7 and 7, which represent medium smoothness of the transformation. This result is counter-intuitive at a first glance, and, however, is reasonable in a group-based registration because the optimal template for each subject must be somewhat different from the subject; see more discussion in Section 7.

Figure 9.

Original shapes (first row), optimal λ & intermediate templates (second row) and the global template (last in second row)

For a visual illustration of the shape alignment after use of different parameters, Fig. 10 shows the residual image variation across shapes before and after optimization. Using λ = 1 brings low variation for most regions, but yields large variation in middle third of the ribbon, which is where most of the morphological variability exists in these synthesized shapes; this is because the most aggressive transformation has trouble matching shape characteristics that are drastically different across individuals. With larger λ, the map of variation becomes more uniform, since alignment is rough and residual more spread-out. The variation of the optimized solutions sits somewhere in the middle of the two extremes of very aggressive and very rough alignment, implying that optimal performance is obtained via a relatively good alignment but with considerable residual. Importantly, variation in the left and right part of the ribbon is small, indicating that the optimal solution is the one that aligns that part of the image that can be matched reasonably well, and leaves the rest to the residual term of the morphological descriptor.

Figure 10.

Maps of standard deviation across subjects with (a) fixed regularization λ = 1, (b) fixed regularization λ = 35, (c) optimal fixed regularization λ = 12, (d) manifold mean, and (e) optimized signature.

The next experiment compares the p-values obtained by performing a two-group voxel-based analysis, which reflects our ability to detect the morphological abnormality (thinning in the middle portion of the ribbons). In Fig. 11(a), the optimized result is compared with the p-values obtained from the two-group comparison at different λ's. The optimal solution is actually better than the best possible solution obtained for λ = 12. We next performed a second experiment, which further tested the p-values obtained from the optimal solution with fixed-λ solutions, except we now allowed for the use of a different λ for each person. Even though it is not feasible in practice to know what λ to use for each person, we still performed this experiment to test the possibility that other combination(s) of λ for various shapes would give a better solution. Since the possible combination of λ values is huge, we performed random sampling of values of λ, and for each combination we calculated the minimum p-value obtained through the respective group comparison. Fig. 11(b) summarizes the results. The optimal solution was better than 98:53% of these random experiments, which indicates that the optimal solution is a very good one. (Out of so many random simulations, some should be expected to yield very low p-values just by chance or by virtue of producing solutions close to the optimal one, so the optimal solution is practically close to the best we can do in these experiments).

Figure 11.

Compare our solution with (a) fixing regularization for each shape and (b) 10000 groups of randomly selected samples.

Spatial maps of the p-value obtained from the two-group comparisons are shown in Figs. 13 and 14. The Jacobian determinant generally cannot find the region in which atrophy was present. For the centered shapes, which represent good initial registration, the region can be detected using the combination of the Jacobian determinant and the residual for strong regularization, because in this case the residual captures the group differences. But for shifted subjects, large λ doesn't work well, because the shapes are misaligned and therefore detection of the group difference requires at least some moderate registration. The optimal solution can further improve the detection (more significance p-value), especially when using local PCA for nonlinear approximation of the MAM.

Figure 13.

The P-maps of simulated centered shapes using (a) Jacobian determinant λ = 1, Combined Jacobian and residual with regularization (b) λ = 1, (c) λ = 35, and optimized combined Jacobian and residual using (d) global and (e-h) local PCA approximation, (d,e,g,h) L2-norm and (f) L1-norm distance criterion, (d-f) T1, (g) T2 and (h) T3 global template.

Figure 14.

The P-maps of simulated shifted shapes using (a) Jacobian determinant λ = 1, Combined Jacobian and residual with regularization (b) λ = 1, (c) λ = 35, and optimized combined Jacobian and residual using (d) global and (e-h) local PCA approximation, (d,e,g,h) L2-norm and (f) L1-norm distance criterion, (d-f) T1, (g) T2 and (h) T3 global template.

We used three different global templates shown in Fig. 12. Figs. 13 and 14 show that using a more convoluted global template (T3) can achieve better estimation. Our optimization is more robust when using more-convoluted global templates, T1 and T3. In general, these results show that selection of the final template can be important.

Figure 12.

Templates (a) T1, (b) T2 and (c) T3.

6.2. Simulated 2D shapes with smaller morphological variation

In the second simulated dataset we generated 2D convoluted shapes that had an image contrast that mimics the presence of two tissues (e.g. white and gray matter in the human brain). However, unlike our previous experiment that examined shapes with high variability, these shapes had very low morphological variability, which can be captured by the transformation alone, for the most part. We introduced 5% atrophy to one part of the ribbon, and another part of the interior tissue; these two regions are shown in Fig. 15. Because all shapes are similar to each other, in these experiments we only used a single template (which was similar to all these shapes), and constructed the MAMs by varying only λ. We used μ = 3, which was determined in Section 3.1.

Figure 15.

Ground truth for the simulated 2D shapes of low morphological variability. Atrophy of 5% was introduced in the two regions

The results are summarized in Fig. 16. In particular, the Jacobian determinant can indeed, as expected in this experiment, capture the group difference in these shapes reasonably well, albeit it is not the best approach. The combination with the residual for λ = 1 worked very well. On the other hand, λ = 35, which corresponds to a very rough registration, lead to a worse detection. The MAM-based solution worked well and was robust to the type of norm and to whether linear or nonlinear approximation of the MAMs was used. This experiment supports the fact that even when the most aggressive registration is able to achieve good matching and correspondence, in the presence of low morphological variability, the MAM optimization doesn't worsen performance, but in fact yields a slightly better localization of the abnormality in agreement with the ground truth.

Figure 16.

The P-maps for the shapes of Fig. 15 using (a) Jacobian determinant, Combined Jacobian and residual with (b) λ = 1, (c) λ = 35, (d) global PCA and (e-f) local PCA approximation, (e) L1-norm and (d,f) L2-norm distance criterion.

6.3. Simulated 2D shapes with two convolutions

In the third simulated dataset we generated 2D shapes that have two variable convolutions, by connecting two randomly-selected shapes in Fig. 4. Fig. 17(a) shows an example. The global template is shown in Fig. 17(b). For optimization, we used μ = 3, which was determined in Section 3.1. The main goal of this experiment was to investigate whether piece-wise optimization is better, compared to single optimization in the entire domain.

Figure 17.

(a) An example of shapes with two sulcus (b) Global template used

Fig. 18 summarizes the experimental results. Because the two convolutions may have different optimal template and regularization, optimizing over the entire image domain detected group difference in the region of one convolution only; see Fig. 18(c). Fig. 18(d) shows that piecewise optimization can help improve the detection of group difference regions in both convolution regions. However, two-piece based optimization also yields discontinuities between the two pieces. Using overlapping pieces helps reduce and remove the discontinuities, as shown in Fig. 18(e-f). The improvement was marginal beyond a two-region optimization, likely because these shapes had only two convolutions.

Figure 18.

The P-maps of simulated two-convolution shapes using (a) Jacobian determinant λ = 1, (b) Combined Jacobian and residual with λ = 1, (c) whole image optimization, piecewise optimization based on (d) two pieces, (e) four pieces, and (f) six pieces.

6.4. Simulated 3D data

For the fourth dataset, we simulated atrophy in three regions of a real MRI brain images: (a) Posterior Cingulate, (b) Hippocampus, (c) Superior temporal gyrus. Shrinkage of gray matter was introduced gradually over a period of 12 years, reaching up to 50%. These images were registered to six templates and nine values of λ (which results into 54 samples for each subject's MAM). To optimized the data, we used μ = 1, as discussed in Section 3.2. We compared three types of measurement: 1) the standard TDM, which is used by the RAVENS approach and by the modulated VBM and has been used in a variety of neuroimaging studies; 2) the CMD (Jacobian, residual) that was obtained for a highly deformable registration, which is the standard choice in the absence of the type of optimization pursued herein; 3) the OMS obtained via the proposed optimization.

Figs. 19 and 20 summarize the various results. The standard TDM generally fails to obtain a good estimate of the longitudinal atrophy, due to significant random temporal fluctuations. We attribute these fluctuations to registration error that is caused by registration algorithms being stuck at different local minima for each year in this series of scans. A representative example of failure to capture longitudinal atrophy is shown in Fig. 19(c), in which the TDM shows tissue density increase in a region that actually atrophied over time. Random longitudinal fluctuations also occur in the region without atrophy shown in Fig. 20. Our optimization significantly alleviates this problem and leads to longitudinally more stable results. Moreover, it seems to better estimate longitudinal atrophy in the regions in which it was simulated.

Figure 19.

Comparison of longitudinal profiles - the curve of mean TDM vs. years between most conforming CMD and OMS in the region of (a) Posterior Cingulate, (b) Hippocampus, (c) Superior temporal gyrus.

Figure 20.

Comparison of longitudinal profiles between most conforming CMD and OMS in the region without atrophy.

6.5. Real 3D data

We used MR brain images from 30 different persons, 15 of them were diagnosed with mild cognitive impairment (MCI) and the other 15 were normal elderly. Six templates and nine regularization were used to sample the MAMs. For optimization, we used μ = 3, determined from the simulated images as discussed in Section 3.1. Gaussian filters with standard deviation σ = 5 are used to smooth Jacobian and Residual prior to the statistical test, as customary in voxel-wise analysis.

We computed the p-map of unoptimized/optimized combined features to evaluate the estimated group differences, which are shown in Fig. 21. The Jacobian and the TDMs give patches of significance. The combined features tend to outline the hippocampus, which is actually the area in which we anticipated to find anatomical differences in these early MCI patients. Moreover, the combined features using local PCA tended to give a result that seems to emphasize the hippocampus most. We can only evaluate these results visually, as there is no ground truth for these datasets.

Figure 21.

(a) Global template and corresponding P-maps of 3D MR brain images using (b) Jacobian, (c) TDM with λ = 1. Combined features with (d) λ = 1, (e) λ = 6, and optimized combined features using (f) global and (g-h) local PCA approximation, and (g) L1-norm and (f,h) L2-norm distance criterion.

The p-map obtained by solving the piece-wise optimization over the entire image (by dividing the brain in 80 partitions) is shown in Fig. 22.

Figure 22.

P-maps of (a) TDM with λ = 1, (b) combined feature with λ = 1, and (c) optimized combined feature.

7. Conclusions and Discussion

In this paper, we proposed a framework for group-wise registration and morphological analysis of medical images, by building upon the work in (Makrogiannis et al., 2007; Baloch et al., 2007; Baloch and Davatzikos, 2009). The overall approach is based on the premise that a shape transformation alone is insufficient to represent the morphological characteristics of complex and variable anatomies. Combination of the transformation and the residual, the latter reflecting information that the transformation of the template cannot capture, leads to a non-uniqueness that is resolved by finding the group-wise minimum variance solution, constraining each shape to lie on its own MAM. We used local linear approximations of the manifold to follow the nonlinearity of a MAM. We employed a minimum variance criterion on selected complete morphological descriptor and performed manifold-constrained optimization, i.e. traversed each individual's MAM, to find the points on all MAMs that brought the group of morphological descriptors closest to each other. The proposed method can reduce the confounding variation in each MAM from noise, parameter and template selection, and better preserve and highlight the true variations that relate to true underlying morphological change. This is confirmed in our experimental results, where our approach improves the performance of estimating group difference for simulated and real datasets.

Our experimental results lead to many interesting conclusions, which are summarized next:

• The Jacobian determinant of the transformation is a relatively poor morphological descriptor, since it generally doesn't capture finer shape details. Although in theory a “perfect” transformation captures all morphological characteristics of an individual anatomy relative to the template, in practice this doesn't happen for many reasons, including difficulties in consistently defining anatomical correspondence across highly variable anatomies, local minima that cause registration algorithms to converge to poor solutions, and biases introduced by regularization constant and template selection. The combination of the Jacobian of a transformation with mid-level smoothness and the residual generally tends to perform better than the Jacobian or the residual alone.

• The optimal solutions obtained using the proposed optimization frame work are free of the need for a priori selection of regularization parameters or templates. Most importantly, there is no single optimal template or smoothness, but the optimal Θ is determined individually for each person. This is a defining characteristic of our approach, and is in sharp contrast with existing single-template or group-average template approaches, which use a common template, that is either pre-defined or estimated as part of group-wise registration.

• The optimal estimated templates are quite intriguing, at a first glance. In particular, when the original shape and global template look similar, the optimal intermediate template looks different from both. From the perspective of achieving a good registration, this is not reasonable, as a direct warping from the original shape to the template would give a good result. However from the perspective of a group-based optimality criterion, this optimal solution is quite reasonable. In particular, it implies that the optimal template corresponding to each individual anatomy must be somewhat different from that anatomy, as other members of the group also differ from their respective templates. If one were able to find one template that is similar to all anatomies, then intermediate templates would be similar as well, but this is not the case in reality, where no single template can be similar to all anatomies. This procedure effectively removes a bias that would otherwise be present in the data. If, for example, one were to find the optimal group-wise template, some shapes would be close to this average template, and some would be very different. A nearly-rigid transformation would suffice for the former (and would leave minimal residual), whereas an aggressive transformation with a great deal of residual would be necessary for the latter. This is an inherent bias, which is removed in our formulation by effectively “pushing away” the optimal template for each anatomy, so that a comparable amount of deformation and residual is measured for all members of the group. This is a fundamental characteristic of the solutions obtained via our formulation.

Many future directions are interesting to explore. In particular, we currently perform a large number of individual warpings in order to build the MAM. This is not necessary. One could only learn the vicinity of the manifold, while moving along it during the optimization. This means that for a tentative point on the manifold, i.e. for a tentative transformation and associate residual, regional (possibly random) perturbations of the transformation can provide samples that allow us to estimate the manifold locally and determine a direction along which to move. This procedure can be repeated until convergence. A second interesting and highly unexplored direction is the effect of different norms used in calculating group variance. We used L₁ and L₂ norms and found similar behavior, which is reassuring that the results don't change dramatically. However other types of norms might be more appropriate. In particular, the work of Younes and colleagues on metamorphosis (Younes, 2007) might provide more suitable measures of geodesic distance, albeit at the expense of significantly increased computational requirements. Finally, computationally-efficient ways to model high-dimensional transformations is very important in our approach, since we are exploring a large set of possible transformations. These directions are currently being explored in our laboratory.

Research Highlights.

1. The Jacobian determinant of the transformation is a relatively poor morphological descriptor, since it generally doesn't capture finer shape details.

2. The optimal solutions obtained using the proposed optimization framework are free of the need for a priori selection of regularization parameters or templates.

3. The optimal template for each subject in group-based optimal criterion must be somewhat different from the subject.