SMAC: Spatial multi-category angle-based classifier for high-dimensional neuroimaging data^☆

Abstract

With the development of advanced imaging techniques, scientists are interested in identifying imaging biomarkers that are related to different subtypes or transitional stages of various cancers, neuropsychiatric diseases, and neurodegenerative diseases, among many others. In this paper, we propose a novel spatial multi-category angle-based classifier (SMAC) for the efficient identification of such imaging biomarkers. The proposed SMAC not only utilizes the spatial structure of high-dimensional imaging data but also handles both binary and multicategory classification problems. We introduce an efficient algorithm based on an alternative direction method of multipliers to solve the large-scale optimization problem for SMAC. Both our simulation and real data experiments demonstrate the usefulness of SMAC.

Introduction

With advances in modern imaging technology, it is becoming increasingly prevalent to collect high-dimensional imaging data (e.g., magnetic resonance imaging [MRI]) in order to extract imaging biomarkers (or features) that are useful for various tasks, including disease detection, diagnosis, prognosis, and treatment, among many others (Chen et al., 1998; Lopez et al., 2009; Ramírez et al., 2009). For many diseases, such as Alzheimer's disease (AD) and breast cancer, it is expected that medical images contain clinically relevant information associated with their pathophysiology. A critical challenge is determining how to build a predictive model (or classifier) that can classify patients into clinically meaningful subgroups according to their imaging data. Such a model may improve the clinical care of these patients and possibly slow their disease progression.

In the current literature, there exist two groups of classification methods for imaging data, including feature-based analysis and image-based analysis. Feature-based analysis consists of (i) converting medical images into a set of features and (ii) building classifiers based on these extracted features. Standard feature extraction methods often extract some summary statistics (e.g., mean imaging intensity) in either segmented tumors or prefixed regions of interest (ROIs) in a template space. For example, Rusinek et al. (2004) used the partial volumes of the brain and cerebrospinal fluid (CSF) to classify AD versus normal control (NC), and Zhu et al. (2014) built a multi-category classifier using sparse linear discriminant analysis based on features extracted from 93 ROIs of both MRI and positron emission technology (PET) images. More examples of feature-based analysis can be found in Xu et al. (2000), Busatto et al. (2003), Colliot et al. (2008), and Yu et al. (2014). The major drawback of these feature-based methods is that they require knowledge of spatial segmentation to identify meaningful ROIs in order to extract informative, discriminating and independent features for the classification task.

Image-based analysis, however, uses raw imaging data across all grid points. Two key advantages of using raw imaging data include potential gain in classification accuracy and spatially interpretable coefficient maps of the classifiers in the original imaging space. The main challenges for image-based analysis include (i) high dimensionality, (ii) complex spatial information and (iii) noisy functional data. For example, a typical T1-weighted MR image of size 256 × 256 × 256 will yield a 16, 777, 216 dimensional space, and due to the inherent biological structure of the brain, these data also have complex spatial correlation and smoothness.

Many methods in the literature apply a pre-screening procedure to reduce the dimensionality of the imaging data, and build classifiers in the reduced imaging space. For example, Liu et al. (2012) applied the ensemble of multiple classifiers based on randomly selected patches of the MR images, and Hinrichs et al. (2011) built multiple kernel support vector machines based on 2000 to 250,000 features selected by voxel-wise t-tests. The pre-screening procedure can significantly reduce the computational cost in estimating the classifiers, but potentially loses important predictive information. On the other hand, many regularization techniques have been proposed to directly handle high-dimensional data, including imaging data as a special case (Grosenick et al., 2008, 2009; Yamashita et al., 2008; Van Gerven and Heskes, 2012). For instance, Yamashita et al. (2008) proposed a method by imposing L₂ norm regularization to logistic regression for classification of functional MRI data in various tasks; whereas Casanova et al. (2011) applied elastic-net penalized regression to distinguish between patients with AD versus NCs based on both gray matter and white matter segmentation maps. These regularization methods perform simultaneous estimation of coefficients across all voxels and select the predictive voxels. Since most standard regularization methods do not account for the spatial structure of imaging data, their resulting classifiers usually contain only isolated voxels; thus, it can be difficult to interpret the results. Moreover, standard sparsity penalties, such as L₁, can be sub-optimal for the high-dimensional prediction problems considered here, since the effect of high-dimensional imaging data on certain categories is often spatially clustered and non-sparse.

To effectively handle imaging data, it is critically important to utilize the spatial smoothness and correlation of imaging data in the construction of classifiers. For instance, Grosenick et al. (2013) proposed a spatial smoothing classifier based on the GraphNet penalty. Furthermore, Watanabe et al. (2014) developed a spatial support vector machine (SSVM) classifier based on the fused lasso (FL) and GraphNet penalties. These methods yield meaningful coefficient images and achieve good accuracy for binary neuroimage classification, but are not directly applicable to multi-category classification problems.

The aim of this paper is to develop a spatial multi-category angle-based classifier (SMAC) for high-dimensional imaging data. Compared with the existing methods in the literature, three major methodological contributions of this paper are as follows:

• The proposed SMAC not only utilizes the spatial structure of images, but also extends the angle-based classification framework recently developed by Zhang and Liu (2014) to perform simultaneous multi-category classification of imaging data.

• We use a hybrid of a generalized total variation (TV) penalty (Tibshirani et al., 2005) and a sparse L₁ penalty, namely an FL penalty, to identify spatially aggregated clusters that are important for discriminating different classes. Our methods are able to deliver competitive classification accuracy and interpretable imaging biomarkers.

• We have developed the SMAC package by using both MATLAB and Python. The MATLAB codes have been released through the websites “https://www.nitrc.org/projects/smac” and “https://github.com/BIG-S2/SMAC”. Our package includes a graphical user interface that is freely downloadable from the same websites. Our SMAC package can handle 1-dimensional (1-D) curves, 2-dimensional (2-D) surfaces, and 3-dimensional (3-D) volumes.

The rest of the paper is organized as follows. In Section 2, we introduce the SMAC framework and describe an optimization algorithm to efficiently estimate the model coefficients. We use two simulation experiments and the Alzheimer's Disease Neuroimaging Initiative (ADNI) data in Section 3 to examine the finite-sample performance of SMAC. In Section 4, we conclude with some discussion.

Methods and materials

Data structure

One important classification problem in the neuroimaging literature is to predict the disease status of patients based on their neurological images. The class label is denoted by a categorical response variable y, usually taking values of 1,2, …, K, indicating K different classes of interest. The covariate $\mathit{X}=\{x_d:d\in \mathcal{D}\}\in {\mathbb{R}}^p$ represents the observed imaging data, where $\mathcal{D}$ denotes the spatial structure of the image, which can be a 1-D curve, 2-D surface or 3-D volume, and d is a vector of length 1, 2 or 3, indicating the location of the underlying voxel in the image. Without loss of generality, we focus on 3-D real valued images in this paper, and use p as the dimension of the imaging data, which equals the total number of voxels in the image.

We use bold symbols to represent the imaging variables, such as X and use regular symbols for scalar and other non-image variables, such as y. The notation [n] represents the set {1, 2,…,n}.

Statistical classification framework

For a K-category classification problem, a statistical classifier builds a map from the covariate space ${\mathbb{R}}^p$ to the category space {1,…,K}. Given a new imaging observation X*, the classifier predicts the associated class label y* as ${\widehat{y}}^{\ast }$. To build the classifier, many statistical procedures can be fitted into the regularization framework of loss + penalty. A loss function l(·) is introduced to ensure the goodness of fit of the resulting model to the training data. Two groups of loss functions that are commonly used in the literature include likelihood-based and margin-based loss functions. Likelihood-based methods usually impose some assumption of probability distributions on the data and then establish the classification rule by solving some parametric statistical models. Examples of these methods include Fishers linear discriminant analysis (LDA) (Fisher, 1936) and logistic regression (Hastie et al., 2005). In contrast, margin-based methods solve the classification problems without imposing a strong distributional assumption on the data. Specifically, a margin-based method uses a functional margin as the input of the loss function l(·). The values of the functional margins are directly associated with the accuracy of the class label assignment. For binary classification with the class label W_y ∈ {±1} for y ∈ {1, 2}, one can obtain a function f(x) and use ${\widehat{W}}_y=\mathit{sign}(f(\mathit{X}))$ as a classification rule. In this case, the functional margin is defined as W_yf (X), indicating the correctness of the classification. Our proposed classifier belongs to margin-based methods.

When dealing with high-dimensional data, a regularization term is usually added to the loss function to prevent the models from over-fitting the training data. The choice of the regularization term is based on prior knowledge of the data structure and the properties of the specific penalty. For instance, the L₁ norm penalty can be utilized to learn the sparse structure of data (Tibshirani, 1996), and the L₂ type of penalties encourage continuous shrinkage in the estimation (Zou and Hastie, 2005). To choose the penalty term for handling the neuroimaging data, it is necessary to account for its high dimensionality and complex imaging structure. A desired penalty should encourage sparsity, while incorporating the spatial structure of the imaging data.

Binary large-margin classifiers

Many “off the shelf” classifiers are potential candidates for neuroimaging classification. Examples range from the very classical LDA (Fisher, 1936) and logistic regression (Hastie et al., 2005) to the recent machine learning techniques, such as the support vector machine (Boser et al., 1992) and boosting (Friedman et al., 2000). The choice of the classifier depends on the data structure and the goal of classification. However, there is no clear guideline about which classifier to choose in each complicated case. Liu et al. (2011) proposed a large-margin unified classifier (LUM), covering a rich family of classification methods, which allows us to tune our loss function within the rich LUM family to obtain a satisfactory solution. In this paper, we choose a special LUM loss function which has the following form:

$$l(u)=\{\begin{array}{cc}1-u,\hfill & \text{if}u<0;\hfill \\ e^{-u},\hfill & \text{if}u\ge 0.\hfill \end{array}\phantom{\}}$$ (1)

This special loss can be viewed as a hybrid of the support vector machine and AdaBoost, which allows us to maximize the separation margin and dynamically assign weights in “weak” learners (Freund and Schapire, 1997). We refer readers to the original paper for further details of the LUM loss.

Despite the potential improvement in classification performance when using LUM, this classifier was originally proposed to solve binary classification problems. The extension to multi-category cases requires additional effort. We address this issue in the following section.

Multi-category large-margin classifiers

To handle multi-category data, one simple approach is to conduct binary classification sequentially via the one-versus-one or one-versus-the-rest scheme in order to predict the class labels. These methods have been proven to be suboptimal when there is no dominating class (Liu and Yuan, 2011). Other classifiers solve the classification problem simultaneously by mapping covariates to a vector with the length equal to the total number of categories. Such classifiers can be found in Zhu and Hastie (2005), Zhu et al. (2009) and Liu and Yuan (2011). A sum-to-zero constraint on the predicted vector is usually applied to achieve desirable theoretical properties, but may increase the complexity of the corresponding optimization. Without this constraint, Zhang and Liu (2014) proposed a multi-category angle-based classifier (MAC) that can achieve the Fisher consistency and some other desirable properties.

For a K-category classification problem (K ≥ 2), MAC creates a map from the class labels y ∈ [K] to the vertices of a regular simplex in the (K – 1) -dimensional space, i.e.,

$$W_y=\{\begin{array}{cc}(K-1{)}^{-1∕2_{\xi }},\hfill & \text{if}y=1;\hfill \\ \frac{1+K^{1∕2}}{(K-1{)}^{3∕2}}\xi +{\left(\frac{K}{K-1}\right)}^{1∕2}{e}_{y-1},\hfill & \text{if}y\in [K]∕1,\hfill \end{array}\phantom{\}}$$ (2)

where $\xi \in {\mathbb{R}}^{K-1}$ is a vector with all elements being 1, and $e_y\in {\mathbb{R}}^{K-1}$ is a vector such that all elements are 0, except that the y-th component is 1. Note that for K = 2, it reduces to the traditional binary classification with labels W_y ∈ {±1}. Due to the property of the regular simplexes, the angles between any two projected class labels are equal, i.e., ∠(W_y, W_y′) = C_K for all y ≠ y′.

Instead of directly using the original class label y, MAC uses the projected class label W_y to solve the multi-category problem. In particular, we construct a function that maps the covariate X to the same K – 1 dimensional space, i.e., $f:{\mathbb{R}}^p\to {\mathbb{R}}^{K-1}$, and use the angle between f (X) and W_y for y ∈ [K] to determine the prediction rule, i.e.,

$$\widehat{y}=\underset{y}{\text{arg min}}\angle \left(W_y,f(\mathit{X})\right).$$

According to the “law of cosine”, this is equivalent to

$$\widehat{y}=\underset{y}{\text{arg}\max }\langle W_y,f(\mathit{X})\rangle ,$$ (3)

where ⟨·, ·⟩ denotes the inner product of two vectors. The inner product essentially plays the role of the functional margin in MAC, and the empirical risk minimization (ERM) is, therefore defined as follows:

$$\underset{f\in \mathcal{F}}{\text{min}}\left\{\sum _{i=1}^{n}l\left(\langle W_{y_i},f({\mathit{X}}_i)\rangle \right)+\lambda J(f)\right\},$$ (4)

where l(·) is the margin-based loss function defined by equation (1) and J(f) is the penalty term with the tuning parameter λ, which controls the strength of regularization.

Considering the specialty of voxel-based neuroimaging classification, we narrow the function space $\mathcal{F}$ to linear functions, so that the coefficients of f(·) are voxel-wisely matched with the structure of the imaging covariate X, i.e.,

$$\begin{array}{cc}\hfill f(\mathit{X})& ={\left(f_1(\mathit{X}),f_2(\mathit{X}),\dots ,f_{K-1}(\mathit{X})\right)}^T,\hfill \\ \hfill f_j(\mathit{X})& ={\beta }_{j,0}+x_{1{\beta }_{j,1}}+\dots +x_p{\beta }_{j,p}\text{for}j\in [K-1].\hfill \end{array}$$ (5)

Notice that β_j = (β_j,1, …, β_j,p)^T has a one-to-one correspondence with the imaging data X – (x₁ , …, x_p)^T. Thus, it can be also defined in the original imaging space of the covariates. In this case, we denote β_j as the coefficient image of the fitted classifier.

For a K-category classification problem, we have K – 1 coefficient images. In order to match the coefficient images with the K class labels, we denote the reconstructed coefficient images ${\beta }_y^{\ast }$, y ∈ [K] of the same dimension of β_j as follows,

$${\beta }_y^{\ast }=\sum _{j=1}^{K-1}{W}_{y,j}•{\beta }_j,$$ (6)

where W_{y, j} is the j-th element of the project class label W_y in Equation (2) and “ ● “ denotes the element-wise product.

Note that ${\beta }_y^{\ast }$ has the one-to-one correspondence with the class label y. Additionally, since ${\sum }_{y=1}^K{W}_y=0$ according to Equation (2), we have the sum-to-zero constraint on ${\beta }_y^{\ast }$ 's as well, i.e., ${\sum }_{k=1}^K{\beta }_y^{\ast }=0$. These properties ensure that the reconstructed coefficient images are comparable with the coefficient images obtained from other linear classification models with the sum-to-zero constraint, such as logistic regression.

Spatial smoothing regularization

The penalty term J(f) in problem (4) not only plays an important role of preventing the resulting classifier from over-fitting, but also helps to achieve some desired structure in the coefficient images. For imaging classification, unpenalized estimation often yields dense coefficients, but requires additional thresholding (or feature selection) to identify meaningful biomarkers. In contrast, the use of sparse penalties alone, such as lasso and the elastic net, leads to coefficient images with isolated voxels, which can be difficult to interpret. The use of spatial smoothing penalties not only captures the spatial smoothness in the image space, but also yields biologically interpretable coefficient images. For instance, Grosenick et al. (2013) proposed a spatial smoothing penalty, GraphNet, that incorporates the spatial structure in the elastic net penalization. However, the GraphNet penalty yields global smoothness in coefficient images, so it may be suboptimal in preserving sharp edges.

We introduce the generalized FL penalty (Tibshirani, 2011) to capture the spatial structure of imaging data. For an image $\mathit{I}=\{\mathit{I}(d)\in \mathbb{R}:d\in \mathcal{D}\}$, the discrete imaging intensities are evaluated at grid points d = (d₁, d₂, d₃)^T ∈ R³ in a compact set $\mathcal{D}$. The FL penalty is a weighted mixture of the L₁ and TV penalty on the imaging intensities. The L₁ penalty encourages both shrinkage and sparseness (Tibshirani, 1996); whereas the TV penalty regularizes the differences between the consecutive elements in the estimation. We denote the latter as the TV-I penalty. Its discrete formulation is defined as follows:

$$\text{TV}-\mathrm{I}(\mathit{I})=\sum _{d_1=1}^{D_1}\sum _{d_2=1}^{D_2}\sum _{d_3=1}^{D_3}\Vert \nabla {\mathit{I}}_{d_1,d_2,d_3}\Vert ,$$ (7)

where ∥ · ∥₁ denotes the L₁ norm, D₁, D₂ and D₃ respectively represent the total number of voxels along each dimension, and ∇ is the discrete differential operator such that ∇I_d1,d2,d3 = (∇₁I_d1,d2,d3, ∇₂I_d1,d2,d3, ∇₃I_d1,d2,d3)^T. Moreover, ∇₁I_d1,d2,d3 is defined as

$${\nabla }_1{\mathit{I}}_{d_1,d_2,d_3}=\{\begin{array}{cc}{\mathit{I}}_{d_1,d_2,d_3}-{\mathit{I}}_{d_1+1,d_2,d_3}\hfill & \text{if}1\le d_1\le D_1-1,\hfill \\ 0\hfill & \text{if}{d}_1=D_1,\hfill \end{array}\phantom{\}}$$

and ∇₂I_d1,d2,d3 and ∇₃I_d1,d2,d3 can be similarly defined.

The TV-I penalty penalizes the discrete gradient of the imaging function I(·). It encourages the spatial smoothness of I(·), while capturing its sharp edges. This property allows us to efficiently detect important blobs. However, in some cases, the TV-I penalty tends to yield images with block-wise constant blobs (Rudin et al., 1992), which might erase too many details. For this reason, we introduce the second-order TV penalty, denoted TV-II, which can capture blobs with a continuous change of intensity by imposing the regularization on the Hessian matrix of I(·), which encourages the gradual fade of I(·) in the space. The discrete formulation of TV-II is defined as follows:

$$\text{TV}-\mathrm{I}\mathrm{I}(\mathit{I})=\sum _{d_1=1}^{D_1-2}\sum _{d_2=1}^{D_2-2}\sum _{d_3=1}^{D_3-2}\Vert H\left({\mathit{I}}_{d_1,d_2,d_3}\right){\Vert }_1,$$ (8)

where H(I_d1,d2,d3) = (∇_m(∇_m′(I_d1,d2,d3)))_{1≤m,m′≤3} and ∥·∥₁ denotes the entry-wise L₁ norm of a matrix.

Note that the calculation of both gradient and Hessian operators can be represented as matrix multiplication on the vectorized images. In particular, the TV – I(I) in (7) can be represented as

$$\text{TV}-\mathrm{I}(\mathit{I})=\Vert \mathbf{D}\times \mathit{I}{\Vert }_1,$$

where D is the discrete derivative operator that contains the differencing operation along each of the 3 dimensions of the imaging domain.

Similarly, the TV-II penalty can be represented as

$$\text{TV}-\mathrm{I}\mathrm{I}(\mathit{I})=\Vert {\mathbf{D}}_{II}\mathbf{D}\times \mathit{I}{\Vert }_1,$$

where D_II = diag{D, D, D} is a diagonal block matrix, with 3 copies of matrix D representing the operations along each dimension.

For problem (4), we have K – 1 coefficient images for a K category classification problem and can denote β = (β₁, …, β_K–1)^T as the vector of all the imaging coefficients, as denoted in equation (5). The associated TV-I penalty is defined as

$$\text{TV}-\mathrm{I}(\beta )=\sum _{k=1}^{K-1}\text{TV}-\mathrm{I}({\beta }_k)=\sum _{k=1}^{K-1}\Vert \mathbf{D}\times {\beta }_k{\Vert }_1=\Vert {\mathbf{C}}_I\beta {\Vert }_1,$$

where C_I = [D, …, D] is K – 1 copies of the operator D. Similarly, we can define

$$\text{TV}-\mathrm{I}\mathrm{I}(\beta )=\Vert {\mathbf{C}}_{II}\beta {\Vert }_1,$$

where C_II = [D_IID,…, D_IID] is K – 1 copies of the matrix D_IID.

Finally, the EMR problem in (4) can be reformulated as follows:

$$\underset{\beta \in {\mathbb{R}}^{(K-1)(p+1)}}{\text{min}}\left\{\sum _{i=1}^{n}l\left(\langle W_{y_i},f({\mathit{X}}_i)\rangle \right)+\text{FL}(\beta )\right\},$$ (9)

where l(·) is the loss function in (1), f(·) is a system of linear functions defined in (5), and FL(β) = λ₁||β||₁ + λ||Cβ||₁ is the FL penalty, in which λ₁ and λ₂ are two non-negative tuning parameters and C = C_I for TV-I or C_II for TV-II.

Algorithm

The optimization in problem (9) is a mixture of smooth and non-smooth convex optimization. Many iterative proximal algorithms can be adopted here to solve this problem, such as ISTA and FISTA (Beck and Teboulle, 2009). However, the evaluation of the Lipschitz constant and the proximal operators can be computationally expensive in this case. Instead, we introduce an alternative direction method of multipliers (ADMM) (Boyd et al., 2011) algorithm to solve the optimization efficiently. A brief introduction of the ADMM is given in the Appendix A1.

Reformulation of ERM

We first reformulate the ERM (9) so that the ADMM algorithm can be applied smoothly. Note that the evaluation of the functional margins ⟨W_yi, f (X_i)⟩ consists of only linear operations. We construct a big matrix A, such that the inner product can be simplified as one matrix multiplication, i.e.,

$$\langle W_{y_i},f({\mathit{X}}_i)\rangle =\sum _{k=1}^{K-1}{W}_{y_i,k}\left(\langle {\mathit{X}}_i,{\mathit{\beta }}_K\rangle +{\beta }_{k,0}\right)={\mathbf{A}}_i,.\mathit{\beta }\text{for}i\in [n],$$ (10)

where A_i,. denotes the i-th row of the matrix A. The details for constructing such a matrix A can be found in Appendix A2.

The penalty term in (9) consists of a sum of two L₁ norms of vectors, and thus can be simplified as

$$\Vert \mathbf{B}\beta {\Vert }_1={\lambda }_1\Vert \beta {\Vert }_1+{\lambda }_2\Vert \mathbf{C}\beta {\Vert }_1,$$

where B^T = [λ₁ I,λ₂C^T]. With a little bit of adjustment to the notations, we use I to denote the identity matrix here. Furthermore, we reconstruct the differencing matrix C to a circulant matrix $\tilde{\mathbf{C}}$ by adding some additional rows, and define ${\tilde{\mathbf{B}}}^T=[{\lambda }_1\mathbf{I},{\lambda }_2{\tilde{\mathbf{C}}}^T]$ accordingly. Under this reformulation, the matrix ($\mathbf{I}+{\tilde{\mathbf{B}}}^T\tilde{\mathbf{B}}$) becomes a block circulant with a circulant block matrix and can be efficiently inverted by using the fast Fourier transform (FFT) (Chan et al., 1993).

For masked images, we introduce a recovering matrix R according to the masking matrix to recover the 3-D imaging structure with all the grid points in the space. A selection matrix M is then introduced to rule out the augmented rows added in $\tilde{\mathbf{B}}$ and force the regions outside the mask to zeros. Therefore, we have

$$\text{FL}(\beta )=\Vert M\tilde{B}R\beta {\Vert }_1.$$

The EMR is then reformulated as

$$\underset{\beta \in {\mathbb{R}}^{(K-1)(p+1)}}{\text{min}}\left\{\sum _{i=1}^{n}l\left((\mathbf{A}\beta {)}_i\right)+\Vert M\tilde{B}R\beta {\Vert }_1\right\}.$$

We further introduce some auxiliary constants and artificial variables to reformulate the problem in a desired form for the ADMM. This leads to our final ERM formulation as follows:

$$\underset{\beta ,{\mathrm{v}}_1,{\mathrm{v}}_2{\mathrm{v}}_3}{\text{min}}\sum _{i=1}^{n}l\left({\mathbf{v}}_{1i}\right)+\Vert {\mathbf{Mv}}_3{\Vert }_1$$ (11)

Subject to v₁ = Aβ, v₂ = Rβ, and ${\mathbf{v}}_3=\tilde{\mathbf{B}}{\mathbf{v}}_2$.

Specifically, we set ${\mathbb{X}}^T=[{\beta }^T,{\mathbf{v}}_3^T]$, ${\mathbb{Y}}^T=[{\mathbf{v}}_1^T,{\mathbf{v}}_2^T]$, $g_1(\mathbb{X})={\Vert {\mathbf{Mv}}_3\Vert }_1$ and $g_2(\mathbb{Y})={\sum }_{i=1}^{n}l({\mathbf{v}}_{1i})$., and denote

$${\mathbb{A}}_1=\left(\begin{array}{cc}\hfill \mathbf{A}\hfill & \hfill 0\hfill \\ \hfill \mathbf{R}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill \mathbf{I}\hfill \end{array}\right)\text{and}{\mathbb{A}}_2=\left(\begin{array}{cc}\hfill -\mathbf{I}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill -\mathbf{I}\hfill \\ \hfill 0\hfill & \hfill -\tilde{B}\hfill \end{array}\right),$$

and then the updating rules for the ADMM can be adopted smoothly for our problem.

Closed-form solutions for the subproblems

We first demonstrate the solution of the optimization in the $\mathbb{X}$ block, which contains the following two subproblems:

$${\beta }^{t+1}=\text{arg}\underset{\beta }{\text{min}}\left\{\Vert \mathbf{A}\beta -{\mathbf{v}}_1^t+{\mathbf{u}}_1^t{\Vert }_2^2+\Vert \mathbf{R}\beta -{\mathbf{v}}_2^t+{\mathbf{u}}_2^t{\Vert }_2^2\right\},$$ (12)

$${\mathbf{v}}_3^{t+1}=\text{arg}\underset{{\mathbf{v}}_3}{\text{min}}\left\{\Vert {\mathbf{Mv}}_3{\Vert }_1+\frac{\rho }{2}\Vert {\mathbf{v}}_3-\tilde{B}{\mathbf{v}}_2^t+{\mathbf{u}}_3^t{\Vert }_2^2\right\}.$$ (13)

Solution for β:

The optimization of β in (12) is a quadratic minimization problem, which has a closed-form solution:

$${\beta }^{t+1}={\mathbf{K}}^t-{\mathbf{A}}^T{\left(\mathbf{I}-{\mathbf{AA}}^T\right)}^{-1}{\mathbf{AK}}^t={\mathbf{K}}^t-{\mathbf{H}}_L{\mathbf{H}}_R^t,$$ (14)

where ${\mathbf{K}}^t={\mathbf{A}}^T({\mathbf{v}}_1^t-{\mathbf{u}}_1^t)+{\mathbf{R}}^T({\mathbf{v}}_2^t-{\mathbf{u}}_2^t)$ and ${\mathbf{H}}_R^t={\mathbf{AK}}^t$. Moreover, H_L = A^T(I – AA^T)⁻¹ is a fixed term across all iterations, so it can be precalculated.

Solution for v₃:

Problem 13 can be solved by a proximal algorithm, the solution of which is given by

$${\mathbf{v}}_3^{t+1}=\mathbf{M}\times {\text{Soft}}_{{\rho }^{-1}}\left(\tilde{B}{\mathbf{v}}_2^t-{\mathbf{u}}_3^t\right)-\left(\mathbf{I}-\mathbf{M}\right)\left(\tilde{B}{\mathbf{v}}_2^t-{\mathbf{u}}_3^t\right),$$ (15)

where Soft(·) is a component-wide soft thresholding operator (Parikh and Boyd, 2013), denoted by Soft_λ(v) = ((v_j – λ)₊ − (−v_j – λ)₊)_j, in which (x)₊ = max{x, 0}.

Next, we demonstrate the optimization of the $\mathbb{Y}$ block, which involves two variables v₁ and v₂. We apply the ADMM algorithms, and decompose it into the following two subproblems:

$${\mathbf{v}}_1^{t+1}=\text{arg}\underset{{\mathrm{v}}_1}{\text{min}}\left\{\sum _{i=1}^{n}l({\mathbf{v}}_{1i})+\frac{\rho }{2}\Vert \mathbf{A}{\beta }^{t+1}-{\mathbf{v}}_1+{\mathbf{u}}_1^t{\Vert }_2^2\right\},$$ (16)

$${\mathbf{v}}_2^{t+1}=\text{arg}\underset{{\mathrm{v}}_2}{\text{min}}\left\{\Vert \mathbf{R}{\beta }^{t+1}-{\mathbf{v}}_2+{\mathbf{u}}_2^t{\Vert }_2^2+\Vert {\mathbf{v}}_3^{t+1}-\tilde{\mathbf{B}}{\mathbf{v}}_2+{\mathbf{u}}_3^t{\Vert }_2^2\right\}.$$ (17)

Solution for v₁:

The optimization of v₁ in (16) can be solved component-wisely by applying the Newton-Raphson method, i.e.,

$${\mathbf{v}}_{1i}^{t+1}={\mathbf{v}}_{1i}^t-\frac{l^{\prime }\left({\mathbf{v}}_{1i}^t\right)+\rho \left({\mathbf{v}}_{1i}^t-({\mathbf{A}}_{i.}{\beta }^{t+1}+{\mathbf{u}}_{1i}^t)\right)}{l^{″}\left({\mathbf{v}}_{1i}^t+\rho \right)},\text{for}i=[n],$$ (18)

where l′(·) and l″(·) are the first- and second-order derivatives of the loss function l(·), which are given as follows:

$$l^{\prime }(u)=\{\begin{array}{cc}-1\hfill & \text{if}u<0\hfill \\ -e^{-u}\hfill & \text{if}u\ge 0\hfill \end{array}\phantom{\}}\text{and}{l}^{″}(u)=\{\begin{array}{cc}0\hfill & \text{if}u<0\hfill \\ e^{-u}\hfill & \text{if}u\ge 0\hfill \end{array}\phantom{\}}.$$

To ensure convergence, we need to conduct multiple iterations in every Newton step. In our implementation, we only perform 1 iteration, which has been shown to result in sufficiently good convergence in practice.

Solution for v₂:

The optimization of v₂ in (17) is a standard quadratic programming problem, which has a closed-form solution:

$${\mathbf{v}}_2^{t+1}={\left(\mathbf{I}+{\tilde{\mathbf{B}}}^T\tilde{\mathbf{B}}\right)}^{-1}\left\{\left(\mathbf{R}{\beta }^{t+1}+{\mathbf{u}}_2^t\right)+{\tilde{\mathbf{B}}}^T\left({\mathbf{v}}_3^{t+1}+{\mathbf{u}}_3^t\right)\right\}.$$

The direct inversion of the matrix $\mathbf{I}+{\tilde{\mathbf{B}}}^T\tilde{\mathbf{B}}$ may not be feasible due to the extra high dimensionality. We make use of its block circulant structure and solve the problem in v₂ by FFT at a cost of O(n log n) operations (Afonso et al., 2010). Specifically, we have

$${\mathbf{v}}_2^{t+1}=ifft\left(fft\left(\left(\mathbf{R}{\beta }^{t+1}+{\mathbf{u}}_2^t\right)+{\tilde{\mathbf{B}}}^T\left({\mathbf{v}}_3^{t+1}+{\mathbf{u}}_3^t\right)\right)÷fft\left({\mathrm{\Gamma }}_1\right)\right),$$ (19)

where fft and ifft denote the 3-D FFT and inverse FFT operators, respectively, “ ÷ “ denotes the element-wise division, and Γ₁, is the first column of matrix $\mathbf{I}+{\tilde{\mathbf{B}}}^T\tilde{\mathbf{B}}$.

A complete ADMM updating procedure is summarized in Algorithm 1. We list all the involved parameters in Table 1 for a convenient reference. The primal updates are discussed above. The dual updates are directly derived from the general updating rule of the ADMM algorithm. We conduct the primal and dual updates alternatively until the prespecified convergence criteria are satisfied. In particular, we check the relative change in the estimated coefficient β, and stop the algorithm if the total number of iterations exceeds a prespecified bound or the relative change is below a certain threshold, ε, i.e.,

$$\frac{\Vert {\beta }^{t+1}-{\beta }^t\Vert }{\Vert {\beta }^t\Vert }\le \epsilon .$$ (20)

Table 1.

List of parameters in Algorithm 2.4.2.

Parameter(s)	Description
β	Target variable in the optimization.
v1	Auxiliary variable, v1 = Aβ.
v2	Auxiliary variable, v2 = Rβ.
v3	Auxiliary variable, ${\mathbf{v}}_3=\tilde{\mathbf{B}}\beta$.
u1, u2, u3	Dual variables in the ADMM.
λ1, λ2	Penalty strength for L1 and TV-I/II respectively.
A	Matrix to compute functional margin, see equation (10).
Kt	Vector to be calculated for solving (14) ,${\mathbf{K}}^t={\mathbf{A}}^T({\mathbf{v}}_1^t-{\mathbf{u}}_1^t)+{\mathbf{R}}^T({\mathbf{v}}_2^t-{\mathbf{u}}_2^t)$.
$\tilde{\mathbf{B}}$	Augmented discrete operator for FL penalty, see Section 2.4.1 for details.
M	Selection matrix to rule out additional terms, see Section 2.4.1 for details.
R	Recovering matrix for masked images.
Γ1	The first column of matrix $\mathbf{I}+{\tilde{\mathbf{B}}}^T\tilde{\mathbf{B}}$.

Algorithm 1. ADMM algorithm for SMAC-I/II

$$\begin{array}{cc}\hfill & \mathbf{Initialize}\text{primal variables}\beta ,{\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_3\text{as}0.\hfill \\ \hfill & \mathbf{Initialize}\text{dual variables}{\mathbf{u}}_1,{\mathbf{u}}_2,{\mathbf{u}}_3\text{as}0.\hfill \\ \hfill & \mathbf{Set}t=0,\text{assign}{\lambda }_1,{\lambda }_2\ge 0.\hfill \\ \hfill & \text{Precompute}{\mathbf{H}}_L={\mathbf{A}}^T{(I-{\mathbf{AA}}^T)}^{-1}\hfill \\ \hfill & \mathbf{while}t\le t_{\mathit{max}}\mathbf{do}\hfill \\ \hfill & \text{Primal update}:\hfill \\ \hfill & {\beta }^{t+1}={\mathbf{K}}^t-{\mathbf{H}}_L{\mathbf{AK}}^t\left(\text{14a}\right)\hfill \\ \hfill & {\mathbf{v}}_3^{t+1}=\mathbf{M}\times {\text{Soft}}_{\frac{1}{\rho }}(\tilde{\mathbf{B}}{\mathbf{v}}_2^t-u_3^t)-(\mathbf{I}-\mathbf{M})(\tilde{\mathbf{B}}{\mathbf{v}}_2^t-{\mathbf{u}}_3^t)(\text{15a})\hfill \\ \hfill & {\mathbf{v}}_{1i}^{t+1}={\mathbf{v}}_{1i}^t-\frac{l^{\prime }\left({\mathbf{v}}_{1i}^t\right)+\rho ({\mathbf{v}}_{1i}^t-({\mathbf{A}}_{i.}{\beta }^{t+1}+u_{1i}^t\left)\right)}{l^{″}\left({\mathbf{v}}_{1i}^t\right)+\rho },\text{for}i=[n],(\text{18a})\hfill \\ \hfill & {\mathbf{v}}_2^{t+1}=\mathit{ifft}\left(\frac{\mathit{fft}\left(\right(\mathbf{R}{\beta }^{t+1}+{\mathbf{u}}_2^t)+{\tilde{\mathbf{B}}}^T({\mathbf{v}}_3^{t+1}+{\mathbf{u}}_3^t\left)\right)}{\mathit{fft}}({\Gamma }_1)\right)(\text{19a})\hfill \\ \hfill & \text{Dual update}:\hfill \\ \hfill & \begin{array}{c}{\mathbf{u}}_1^{t+1}=\mathbf{A}{\beta }^{t+1}-{\mathbf{v}}_1^{t+1}+{\mathbf{u}}_1^t\hfill \\ \hfill \\ {\mathbf{u}}_2^{t+1}=\mathbf{R}{\beta }^{t+1}-{\mathbf{v}}_2^{t+1}+{\mathbf{u}}_2^t\hfill \\ \hfill \\ {\mathbf{u}}_3^{t+1}={\mathbf{v}}_3-\tilde{\mathbf{B}}{\mathbf{v}}_2^{t+1}+{\mathbf{u}}_3^t\hfill \end{array}\hfill \\ \hfill & \hfill \\ \hfill & \text{Convergence criteria}:\hfill \\ \hfill & \mathbf{if}\Vert {\beta }^{t+1}-{\beta }^t\Vert ∕\Vert {\beta }^t\Vert >\epsilon \mathbf{then}\hfill \\ \hfill & t=t+1\hfill \\ \hfill & \mathbf{else}\hfill \\ \hfill & \begin{array}{c}\mathbf{break}\hfill \\ \mathbf{return}\beta ={\beta }^{t+1}\hfill \end{array}\hfill \\ \hfill & \mathbf{end}\mathbf{if}\hfill \\ \hfill & \mathbf{end}\mathbf{while}\hfill \end{array}$$

Simulation of synthetic data

To illustrate the finite sample performance of SMAC, we conducted simulation studies in both binary and multi-category cases.

Generation of the synthetic data

In Simulation I, we simulated 2 classes of images of size 20 × 20 × 10. The true signals for each class are denoted as θ₁ and θ₂ (see Fig. 1), where θ₁ has two ROIs and θ₂ has three ROIs. The discriminating region between the two classes is the ROI represented by the region of the black triangular prism in the center, which contains 75 voxels in total. The image intensities in the three ROIs are 0, 1 and 2, respectively.

Fig. 1.

True signals for two classes of images in Simulation I. The left panel is the true image of class 1: the transparent and yellow regions represent the voxel values of 0 and 1, respectively. The right panel is the true image of class 2: the transparent, yellow and black regions represent 0, 1 and 2, respectively.

In Simulation II, we considered classifying three classes of images. The image size is 32 × 32 × 4, and the true signals are θ₁, θ₂ and θ₃, which are graphically illustrated in Fig. 2. The image intensities are 0 in the black regions and 1 in the white regions. The discriminating regions among the three classes located in the first and second diagonal blocks are marked in the red boxes.

Fig. 2.

True signals for three classes of images in Simulation II. The three images are the top layer (z = 1) of the mean images for class 1, 2, and 3, respectively. White represents the voxel value of 1, and black represents 0. The four layers (z = 1,…,4) of the true image are identical within each class. The discriminating regions are marked in red boxes.

We generated noisy imaging samples by adding independent Gaussian noise at each voxel of the true signals, i.e., if the i-th image belongs to the k-th category, the associated noisy sample is given as

$${\mathit{X}}_i(t)={\theta }_k(t)+{\epsilon }_i(t)\text{for all}t\in \mathcal{D}\text{and}i\in [n],$$ (21)

where ${\epsilon }_i(t)\begin{array}{c}\hfill \underset{~}{\mathit{iid}}\hfill \\ \hfill \hfill \end{array}N(0,{\sigma }^2)$ represents the Gaussian noise. For both simulation studies, we set σ = 2 for all samples.

Application: classification of MRI images from ADNI data

For the real data applications, we analyzed data from the ADNI study, a large-scale multi-site study that has collected MRI and PET images, CSF, and blood biomarkers, among other patient data. In AD, the most common form of dementia, the affected individual progressively develops disabilities in memory, language, and behavior, and the disease eventually results in death. A key goal of the ADNI study is to develop more sensitive and accurate biomarkers for the early detection of AD. The participants in the ADNI study include cognitively NCs, individuals with amnestic mild cognitive impairment (MCI), and subjects with AD. More information about this study can be found at the ADNI website (http://adni.loni.usc.edu/).

Participants

In this paper, we used a subset of baseline T1-weighted images from the ADNI study. After removing images with low quality, we obtained a dataset consisting of 749 samples (209 NC, 361 MCI and 179 AD). Table 2 summarizes the demographic information of all the subjects in our data analysis.

Table 2.

Demographic Information of All Subjects in our Data Analysis. The unit for intracranial volume (ICV) is 10³ × cm³; for age and ICV, the means are reported, with standard deviations in parentheses.

	Male	Female	Age	ICV
NC	111	95	76.03 (4.95)	1.27 (0.12)
MCI	233	131	75.00 (7.38)	1.29 (0.14)
AD	98	81	75.50 (7.53)	1.27 (0.15)

Image acquisition and processing

All images were preprocessed by a standard procedure (Guo et al., 2014), including anterior commissure and posterior commissure correction, N2 bias field correction, skull-stripping, intensity inhomogeneity correction, cerebellum removal, segmentation, and registration. We generated RAVENS-maps for the whole brain, using the deformation field obtained during registration (Davatzikos et al., 2001) and obtained 749 images of size 128 × 128 × 128. Considering that the variability of age, gender and whole-brain volume among different subjects may affect the classification results, we first removed those factors by fitting linear regression models at each voxel, and then built the classification model based on the residual images of these linear models.

Results

Comparison, tuning parameter selection and cross-validation

The proposed SMAC is designed to handle whole-brain volumetric data and detect disease-related regions without any prior spatial knowledge. To evaluate the performance of SMAC in these two tasks, we compared our method with other classifiers that can handle high-dimensional whole-brain volumetric data without any pre-screening procedure, and which also have the ability to yield volumetric coefficient images in the same space of the covariates. Under this guidance for comparison, we chose the following classifiers for neuroimaging classification: logistic regression using elastic-net regularization (EN-LR) (Casanova et al., 2011), logistic regression with the GraphNet penalty (GN-LR) (Grosenick et al., 2013) and SSVM with an FL penalty (Watanabe et al., 2014). Since SSVM was originally designed only for binary problems, we did not include it in the multi-category problems. To distinguish between SMAC with TV-I and TV-II penalties, we respectively denote them as SMAC-I and SMAC-II.

All the methods mentioned above involve two tuning parameters, λ₁ and λ₂. For consistent comparison, we denoted λ₁ as the tuning parameter of the sparse penalty terms for all methods. In SSVM, SMAC-I and SMAC-II, we denoted λ₂ as the tuning parameter of the total variation terms, whereas in EN-LR and GN-LR, we defined λ₂ as the parameter of the L₂ norm penalty. We conducted a grid search to select the best pair of the two parameters across a 21 by 21 log-based grid for the synthetic data, i.e., λ₁ ⊗ λ₂ ∈ {0, 2⁻¹⁴, 2⁻¹³, …, 2⁵}^⊗2 and a smaller grid of λ₁ ⊗ λ₂ ∈ {0,2⁻¹³, 2⁻¹¹, …, 2³, 2⁵}^⊗2 for the real data.

For the analysis of the synthetic data, a data-rich scenario, we independently generated 30 training, 30 validation and 300 test samples for each class according to (21), which yielded 60 training, 60 validation and 600 test samples in Simulation I and 90 training, 90 validation and 900 test samples in Simulation II. We used the training samples to build models for each combination of λ₁ and λ₂, and evaluated the models on the validation samples to calculate the tuning classification accuracy and area under the curve (AUC) in the associated receiver operating characteristic (ROC) analysis. Based on the validation results, we picked the models with the highest classification accuracy. If ties occurred, we chose the models with highest AUC among them. If we still obtained multiple models, the one with a larger spatial penalty (λ₂) was selected as our final model. We applied the final model to the test samples to evaluate the classification performance. To validate the stability of the methods, we repeated the experiments for 50 iterations, and reported the means and standard deviations of the results.

For the real data analysis, we applied a stratified sampling on the whole dataset and split it into training (60%), validation (20%) and test (20%) sets, so that the proportions of NC, MCI and AD subjects were similar across the different sets. We used a validation and evaluation procedure that was similar to what we used in the simulation study. We repeated the above random split 30 times and recorded the means and standard deviations of the results.

Results from synthetic data analysis

Cross-validation and tuning results

The mean validation accuracy matrices from 50 iterations of the simulation studies are given in Fig. 3. In Simulation I (binary case), EN-LR yielded lower validation accuracy for most of the sparse estimation, i.e., λ₁ ∈ {2¹, …, 2⁵}. SSVM yielded higher tuning accuracies for the sparse and patched estimation, i.e., λ₁ ⊗ λ₂ ∈ {0,2⁻¹⁴,…, 2⁻⁸} ⊗ {2⁻⁵, …, 2⁻³}. GN-LR achieved very good validation accuracy when the sparsity and smoothness levels were relatively high, but yielded low accuracy when the sparsity level was too high, i.e., λ₁ ∈ {2⁴, 2⁵}. SMAC-I and SMAC-II achieved overall higher validation accuracy and were more sensitive to the change in tuning parameters. In particular, the SMAC methods were more sensitive to the penalty level of the total variation than the sparse term. This is mainly explained by the spatial smoothness assumption of the imaging data.

Fig. 3.

Validation accuracies for synthetic studies. The top row of 5 panels (from left to right) respectively correspond to the validation accuracy matrices of EN-LR, GN-LR, SSVM, SMAC-I and SMAC-II for the binary synthetic data. The bottom row of 4 panels (from left to right) respectively correspond to the validation accuracy matrices of EN-LR, GN-LR, SMAC-I and SMAC-II for the multi-category synthetic data. Each entry of the matrix is the tuning accuracy for the corresponding combination of λ₁ and λ₂. The vertical direction of the matrix represents the value of λ₁, from top to bottom being {0, 2⁻¹⁴, 2⁻¹³,…,2⁵}, and the horizontal direction represents λ₂, from left to right being {0, 2⁻¹⁴, 2⁻¹³,…,2⁵}.

The results of Simulation II are similar to those of Simulation I. The sparse method EN-LR yielded low validation accuracy for most combinations of the tuning parameters. GN-LR and SMAC achieved high accuracy under a relatively high sparsity level and a moderate smoothness penalty level, i.e., λ₁ ⊗ λ₂ ∈ {2⁻⁵, 2⁻⁴, 2⁻³}^⊗2.

Receiver operating characteristic (ROC) analysis and classification accuracy

The ROC analysis can simultaneously evaluate the true positive rate and the false positive rate for a binary classifier under different thresholds. The AUC numerically measures the performance of a classifier in the ROC analysis. When dealing with the multi-category cases, the ROC analysis can be implemented using the “one vs. the rest” strategy, i.e., transforming it into multiple binary problems. We conducted the ROC analysis for both binary and multi-category problems, randomly picked one result from the 50 iterations, and plotted the associated ROC curves; see Figs. 4 and 5. The numerical results for all iterations are summarized in Tables 3 and 4.

Fig. 4.

Receiver operating characteristic (ROC) analysis for the binary synthetic data based on 600 test samples.

Fig. 5.

Receiver operating characteristic (ROC) analysis for the multi-category synthetic data based on 900 test samples. Each panel represents the ROC curves evaluated using the “one-versus-the-rest” strategy.

Table 3.

Comparison of Results in the Classification of Binary Synthetic Data. Classification accuracy (ACC), true positive rate (TPR), true negative rate (TNR) and area under the ROC curve (AUC); values are reported as percentages; means from 50 iterations are reported, with standard deviations in parentheses.

Method	ACC	TPR	TNR	AUC
EN-LR	73.75 (5.24)	76.44 (7.00)	71.05 (8.50)	81.70 (5.99)
GN-LR	93.04 (1.75)	93.01 (1.94)	93.07 (1.92)	98.11 (0.79)
SSVM	92.23 (2.06)	94.11 (2.88)	90.35 (3.37)	97.91 (1.00)
SMAC-I	96.52 (1.21)	95.83 (2.16)	97.21 (1.30)	99.51 (0.31)
SMAC-II	96.17 (1.32)	95.54 (1.98)	96.79 (1.72)	99.39 (0.36)

Table 4.

Comparison of Results in the Classification of Multi-category Synthetic Data. Classification accuracy (ACC); area under the ROC curve (AUC1-3) values for classes 1, 2 and 3, respectively; values are reported as percentages; means from 50 iterations are reported, with standard deviations in parentheses.

Method	ACC	AUC1	AUC2	AUC3
EN-LR	64.89 (3.29)	63.96 (2.70)	86.41 (3.02)	86.54 (3.22)
GN-LR	92.17 (1.13)	90.74 (4.60)	99.55 (0.16)	99.23 (0.26)
SMAC-I	94.70 (1.10)	95.65 (1.42)	99.85 (0.06)	99.66 (0.13)
SMAC-II	94.19 (0.86)	95.85 (1.12)	99.71 (0.07)	99.64 (0.10)

In the binary classification example, SMAC-I achieved the highest classification accuracy of 96.52% and the largest AUC of 99.58%, followed by an accuracy of 96.17% and an AUC of 99.39% from SMAC-II. GN-LR and SSVM yielded accuracies of 93.04% and 92.23%, and AUC values of 98.11% and 97.91%, respectively. EN-LR achieved an accuracy of 73.75% and an AUC of 81.70%.

In the multi-category cases, SMAC-I and SMAC-II yielded the highest respective accuracies of 94.70% and 94.19%, as well as the largest AUC values in all three classes. GN-LR achieved an accuracy of 92.17%. EN-LR yielded an accuracy of 64.89% and had the smallest AUC values in all three classes.

In both simulation studies, the spatial methods were more stable in terms of the classification results and yielded smaller standard deviations among the 50 iterations. The sparse method EN-LR delivered sparse estimation consisting of isolated voxels, and thus yielded unstable models. In particular, its variable selection results varied a lot in different iterations.

Visualization and interpretation of coefficient images

We plotted all the coefficient images to illustrate the estimation and identification of those critical regions for classifying the samples. The plot of the coefficient images in Simulation I (see Fig. 6) reveals that EN-LR yielded a sparse coefficient image, consisting of isolated voxels; whereas all the other spatial penalized methods produced smooth coefficient images, clearly indicating the triangular discriminating region in the center. SSVM and SMAC-I both yielded clear boundaries between the predictive and irrelevant regions. SSVM contained many false positive voxels in the background; whereas SMAC-I had a “clean” background. GN-LR yielded smooth coefficient images with blurred boundaries around the triangular region and also contained some false positives in the background. SMAC-II yielded a similarly smooth coefficient image with many fewer false positives.

Fig. 6.

Estimated coefficient images obtained from five classification methods in Simulation I. The 5 panels display the coefficient images of EN-LR, SSVM, GN-LR, SMAC-I and SMAC-II. Each coefficient image is displayed in three respective directions: transverse, coronal and sagittal, from left to right. The center of all the images is located at (10, 10, 5).

In the multi-category example, we illustrated the reconstructed coefficient images (defined in Equation (6)) from SMAC-I/II and compared them with the penalized logistic regression methods (EN-LR and GN-LR). Since the three coefficients for each method summed to zero, we only displayed first two of them, i.e. ${\widehat{\beta }}_1$ and ${\widehat{\beta }}_2$ (see Fig. 7). The estimated coefficient images obtained from EN-LR consist of isolated voxels. GN-LR and SMAC-II yielded smooth patched estimations, but with a blurred boundary. The coefficient images from SMAC-I clearly captured the first and second diagonal block regions of the checkerboard image, which are the most critical regions for discriminating the three classes.

Fig. 7.

Estimated coefficient images obtained from four classification methods in Simulation II. The top panels are the respective coefficients from EN-LR and GN-LR, and the bottom panels are the coefficients from SMAC-I and SMAC-II. The first two coefficient images (β₁ and β₂) of each classifier are displayed. The coefficients from SMAC-I and SMAC-II are obtained using Equation (6). All the coefficient images are displayed in the transverse direction, centered at (16, 16, 1).

Accurately capturing the key discriminating regions is a requirement of a good image classifier. In both simulation studies, the sparsity-only classifier EN-LR underperformed due to the ignorance of the spatial structure. GN-LR and SMAC-II tended to yield smooth critical regions in which the imaging intensities continuously varied across voxels. SSVM and SMAC-I were able to capture the critical regions with clear boundaries. SMAC-I and SMAC-II achieved fewer false positives in the irrelevant regions, while the other methods contained either isolated or patchy false positives in the background. For these particular synthetic data, SMAC-I delivered the most competitive performance. This was mainly due to the assumption of patchy constant patterns in the discriminating regions. SMAC-II may have potential advantages when those regions have continuously varying intensities.

Model sensitivity on training sample size and noise level

To further analyze the stability of the proposed methods, we conducted a comprehensive sensitivity analysis on the sample size and noise level. In particular, the sample size analysis was done by repeating the experiment in Simulation I with the training sample size ranging from 10 to 100. The validation and test sample sizes were not changed, and the noise level remained the same for different sample sizes, i.e., ${\epsilon }_i(t)\begin{array}{c}\hfill \underset{~}{\mathit{iid}}\hfill \\ \hfill \hfill \end{array}N(0,4)$. A similar model selection procedure as that used in Simulation I was used, and we report the test results in Table 5 and Fig. 8. The noise level sensitivity analysis was done by fixing the training sample size (n = 30) and varying the standard deviation of the noise added to each voxel from σ = 1 to 4. The test results are summarized in Table 6 and Fig. 9.

Table 5.

Sample Size Sensitivity Analysis. Columns are different training sample sizes; classification accuracy (ACC); area under the ROC curve (AUC); values are reported as percentages; evaluation is based on 600 test samples.

Method	10	20	30	40	50	60	70	80	90	100
ACC
EN-LR	54.17	61.67	60.17	68.83	72.67	73.50	78.33	82.00	84.50	83.50
GN-LR	65.17	78.00	90.17	91.50	91.83	91.00	94.83	89.83	95.67	96.67
SSVM	59.33	82.50	92.67	93.50	93.50	90.67	93.67	94.83	94.67	93.67
SMAC-I	65.00	80.17	95.50	96.50	97.00	95.00	96.50	97.00	97.67	96.50
SMAC-II	55.00	89.67	96.83	96.17	96.83	96.00	91.00	95.50	95.67	97.33
AUC
EN-LR	56.69	65.97	64.51	78.84	81.12	79.63	85.88	90.72	92.77	92.92
GN-LR	71.45	86.44	96.59	97.43	97.23	97.39	98.91	97.13	99.09	99.48
SSVM	62.89	91.34	97.77	98.66	98.32	96.42	98.33	98.70	98.81	98.33
SMAC-I	77.50	90.15	99.42	99.65	99.73	98.98	99.71	99.69	99.76	99.63
SMAC-II	59.76	96.68	99.57	99.67	99.58	99.14	97.37	99.42	99.24	99.69

Fig. 8.

Classification results under different training sample sizes. The left panel displays the classification accuracy and right panel displays the area under the ROC curve.

Table 6.

Noise Level Sensitivity Analysis. Columns are different standard deviations of noise; classification accuracy (ACC); area under the ROC curve (AUC); values are reported as percentages; evaluation is based on 600 test samples.

Method	1.0	1.25	1.5	1.75	2.0	2.25	2.5	2.75	3.0	3.25	3.5	3.75	4.0
ACC
EN-LR	97.00	94.83	84.33	78.50	73.50	72.50	64.50	64.17	58.67	60.67	54.17	58.17	57.17
GN-LR	100.00	99.00	98.67	97.17	91.00	88.00	81.00	77.50	72.00	69.17	66.67	63.17	61.83
SSVM	99.67	98.83	96.50	93.50	90.67	84.83	80.50	76.33	72.50	68.83	65.33	57.00	56.00
SMAC-I	98.50	98.83	98.17	96.17	95.00	93.50	90.83	88.00	83.00	71.67	67.50	65.83	58.00
SMAC-II	99.33	98.67	97.00	98.00	96.00	94.33	89.33	80.83	76.17	82.00	69.17	69.33	70.83
AUC
EN-LR	99.97	98.81	92.90	87.89	79.63	80.64	69.03	67.56	61.13	63.61	54.55	60.72	58.42
GN-LR	100.00	99.96	99.91	99.65	97.39	95.42	89.43	85.46	80.61	76.38	73.31	69.15	67.66
SSVM	99.98	99.97	99.59	98.20	96.42	92.83	88.57	83.96	79.15	74.78	70.52	59.82	58.46
SMAC-I	99.84	99.94	99.87	99.31	98.98	97.86	96.75	95.19	91.30	78.25	74.06	73.44	62.56
SMAC-II	100.00	100.00	99.82	99.87	99.14	98.69	95.37	88.76	85.01	90.72	77.93	78.52	78.13

Fig. 9.

Classification results under different noise levels. The left panel displays classification accuracy and the right panel displays the area under the ROC curve.

From the sensitivity analysis, we conclude that the proposed SMAC methods can achieve high accuracies and AUCs with very limited training samples, e.g., n ≤ 50, and yield very competitive performance in the cases of mid-noise levels, e.g., σ ∈ (2,3).

Results from ADNI data

We conducted both binary and multi-category classification experiments using the ADNI data. In particular, we classified all possible pairs of the three classes as binary problems (NC vs AD, NC vs MCI and MCI vs AD) and identified AD, MCI and NC simultaneously as a three-category problem. The classification accuracies are presented in Tables 7 and 8. After we obtained the best tuning parameters from the 30 iterations of the three-way split, we refitted the model using all the data with the selected parameters and registered the coefficient images to the Montreal Neurological Institute (MNI)-152 template (Fonov et al., 2011). A plot of these coefficients in the orthogonal views is provided in Figs. 10 and 11.

Table 7.

Comparison of Results in the Binary Classification of MRI Data. Classification accuracy (ACC); area under the ROC curve (AUC); values are reported as percentages; means from 30 iterations are reported, with standard deviations in parentheses.

	NC vs AD		MCI vs AD		NC vs MCI
Method	ACC	AUC	ACC	AUC	ACC	AUC
EN-LR	86.84 (3.13)	93.33 (1.90)	69.22 (2.65)	66.31 (4.90)	70.35 (2.07)	73.33 (2.41)
GN-LR	86.23 (2.78)	95.95 (1.36)	65.67 (3.23)	68.15 (5.39)	68.55 (2.29)	76.69 (2.99)
SSVM	86.14 (3.41)	92.65 (3.02)	48.07 (5.47)	56.35 (4.21)	63.72 (0.00)	50.00 (0.00)
SMAC-I	89.12 (2.30)	93.92 (1.20)	69.13 (2.72)	67.24 (4.62)	70.68 (2.43)	75.31 (2.22)
SMAC-II	88.33 (3.32)	93.97 (1.44)	69.19 (3.59)	66.86 (4.64)	70.38 (2.51)	76.35 (2.48)

Table 8.

Comparison of Results for Simultaneously Classifying 3 Categories of MRI Data. Classification accuracy (ACC); area under the ROC curve (AUC1-3) with respective reference labels NC, MCI and AD; values are reported in percentages; means from 30 iterations are reported, with standard deviations in parentheses.

Method	ACC	AUC1	AUC2	AUC3
EN-LR	49.32 (3.18)	72.39 (3.15)	50.68 (4.66)	77.56 (4.44)
GN-LR	49.75 (2.55)	77.90 (2.99)	51.94 (4.83)	82.89 (1.45)
SMAC-I	53.22 (2.90)	81.01 (3.23)	50.31 (5.03)	77.53 (3.63)
SMAC-II	52.68 (4.20)	81.75 (3.85)	48.85 (4.79)	78.21 (3.67)

Fig. 10.

Estimated coefficient images obtained from five classification methods in the binary ADNI study. The five plots are the respective coefficient images from EN-LR, GN-LR, SSVM, SMAC-I and SMAC-II. Each coefficient is displayed in the views of coronal, sagittal and transverse planes. The slices are located at (0, – 17, 18).

Fig. 11.

Estimated coefficient images obtained from four classification methods in the multi-category ADNI study. The four rows of plots are the respective coefficient images from EN-LR, GN-LR, SMAC-I and SMAC-II. The first two coefficient images (β₁ and β₂) of each classifier are displayed. The coefficients from SMAC-I and SMAC-II are obtained using Equation (6). Each coefficient is displayed in the views of coronal, sagittal and transverse planes. The slices are located at (0, – 17, 18).

ROC analysis and classification accuracy

In the classification problem of NC vs AD, SMAC-I and SMAC-II achieved the highest two accuracies of 89.12% and 88.33% respectively. The other three methods yielded similar accuracies between 86% and 87%. In the classification of MCI vs AD and NC vs MCI, the overall accuracies were lower. This may be partially explained by the uncertainty involved in the cognitive test for identifying MCI and the heterogeneity within the MCI group. EN-LR and SMAC-I/II yielded accuracy values that were very close in these tasks. SSVM was outperformed by the other methods, and could not capture informative signals in the classification of NC vs MCI. Notice that GN-LR achieved the highest values of AUC in all three binary classification problems. This is explained by the merit of the logistic loss in terms of estimating the “soft” class label, i.e. the associated probability (Liu et al., 2011). SMAC-I/II also yielded very competitive AUC values (second best in all three problems).

For the simultaneous classification of NC, MCI and AD, the classification accuracies are lower than those for the binary cases. SMAC-I/II yielded higher accuracies (53.22% and 52.68%) compared to those achieved by EN-LR and GN-LR (49.32% and 49.75%). The AUC values for MCI (AUC2) were lower than the ones for NC and AD, which was consistent with the results in the binary cases. SMAC-II achieved the best and second best values for AUC1 and AUC3, indicating a better detection rate for NC and AD.

Clinically meaningful coefficient images

Different from our synthetic imaging data, the MRI images of human brains are much more complex. Due to heterogeneity across subjects and the potential bias in the registration process, the boundaries between the discriminating regions and the background may not be as sharp as they are in the synthetic data. The patchy patterns may not be a perfect assumption for this case, but still help the classifiers to recover the predictive regional signals. From the plots in Figs. 10 and 11, we can clearly see consistent patterns across the coefficient images from different spatial methods. The sparse method EN-LR delivers ultra-sparse estimation that is difficult to interpret biologically. Notice that, among all the spatial methods, SMAC-II can recover more smooth and patchy signals, while screening out the irreverent regions in the brain. This will make it easier to identify ROIs in the coefficient images of SMAC-II.

Comparing Figs. 10 and 11, we can see that for each spatial method, the effective regions of β in Fig. 10 and the first coefficient β₁ in Fig. 11 are relatively consistent, but the intensity values have the opposite signs, i.e., the regional effects are opposite. This is mainly because the positive class label in the binary problem is AD while in the multi-category problem, class label 1 is associated with NC.

By overlaying the coefficient images from SMAC-I/II on the MNI-152 ROI template, we are able to identify several significant discriminating regions, such as the frontal gyrus, hippocampus, and right fornix. Many papers in the existing literature have shown that these regions are potentially related to the development of MCI and AD. For instance, the hippocampal region is involved in memory processes that deteriorate with the development of AD. The structure of the hippocampus is altered by the degenerative processes associated with AD, and loss of the hippocampal volume occurs at a rate that is approximately two to four times faster in patients with AD than in age-matched healthy controls (West et al., 1994; Dubois et al., 2014).

Computational considerations

In the MATLAB implementation of our algorithms, most of the computation is realized through matrix operations. For moderate image sizes (e.g., total number of voxels less than 10⁴), our methods converge very fast compared to the others. For ultra-high-dimensional imaging data (e.g., total number of voxels greater than 10⁶), the matrix operations require more memory usage. Furthermore, for all the classifiers used in our comparison, the regularization parameters highly affect the convergence and computational cost of the algorithms. We ran all the programs on the same type of computer (Intel Xeon E5-2643 v3 @ 3.40 GHz) with the same random-access memory (8 GB DDR3 at 1600 MHz). All algorithms were set with the same maximum number of iterations (t_max = 1500) and convergence threshold (ε = 5 × 10⁻⁵) as defined in (20). We plotted the mean computational time from all 5 classifiers among 50 iterations in Simulation I (see Fig. 12). EN-LR required the shortest time for this classification problem. The variation in the computational time was very small. This was mainly due to the simplicity of the EN-LR model. SMAC-I required the second shortest computational time, followed by SMAC-II. This was because the second-order total variation involved computation of the discrete Hessian operators, which had larger sizes than the gradient operators in SMAC-I. GN-LR also yielded very competitive computational speed. SSVM was out-performed by the other classifiers in terms of the computational speed. This was mainly due to the splitting scheme in its ADMM algorithm and the heavy computational load in optimization involving the non-smoothing hinge loss.

Fig. 12.

Mean computational time for each method in Simulation I. In each plot, the vertical direction represents the value of λ₁, from top to bottom being {0,2⁻¹⁴, 2^–13,…,2⁵}, and the horizontal direction represents λ₂, from left to right being {0, 2⁻¹⁴, 2⁻¹³,…,2⁵}.

Discussion

In this paper, we propose a SMAC for neuroimaging classification. Our method achieves the desired spatial sparsity and smoothness in the coefficient images via imposing the FL penalty. It improves the accuracy in both binary and multi-category classification problems. Both the simulation studies and the real data application demonstrate the usefulness of the proposed method.

Numerous classification studies in the literature have used the ADNI data, but their data collection and evaluation procedures may vary significantly. A direct comparison of the results may not be a reasonable way to evaluate the methods. For example, Dukart et al. (2011) achieved 100% accuracy on the classification of NC vs AD, while we obtained 89.12% accuracy for the same problem. However, their study assessed only 13 NC and 21 AD subjects; whereas our study assessed the 749 participants in the ADNI study. Moreover, they used pre-computed ROI statistics from both MRI and fluorodeoxyglucose-PET images as predictors; whereas we directly classified the baseline MRI data and automatically extracted the regional information during the estimation procedure. An advantage of our proposed method is that we can handle imaging data with limited pre-processing, and still produce reasonably good results. This can be valuable when the prior knowledge of spatial segmentation is not available.

We introduce an efficient algorithm using ADMM to solve the corresponding large-scale optimization problem in our method. Specifically, we propose a novel splitting scheme in ADMM and reduce the complexity of optimization. As a result, our algorithm performs more efficiently than the ADMM algorithms in the existing literature, such as Ye and Xie (2011) and Watanabe et al. (2014). Moreover, the proposed algorithm is very flexible and can be applied to solve various other prediction problems within the loss + penalty framework. We have included the implementation of the squared error loss in our package, which allows users to perform spatial regularized high-dimensional regression analysis. Details about this extension are included in Appendix A3.

One potential limitation of the proposed method is the underlying assumption of the spatially clustered patterns in the true coefficient images. This is a reasonable assumption in most neuroimaging applications. However, if the overall predictive effect is scattered around most of the imaging space, this method can be inefficient due to the complexity of the optimization problem.

There are several possible interesting extensions of the proposed method for future exploration. For example, all linear classifiers are built based on the assumption that the images are perfectly aligned and the predictive regions are consistent across all the subjects within the same class. These assumptions can be violated in practice, both due to the non-negligible registration error and the heterogeneous structures within the population. The estimation and predictive performance can be highly affected by this issue. One possible future research direction is to handle this heterogeneity problem.

Appendix A1. Alternative Direction Method of Multipliers

The ADMM algorithm (Boyd et al., 2011; Mota et al., 2011) was developed to handle large-scale convex optimization problems with the following separable and constrained structure:

$$\underset{\mathbb{X},\mathbb{Y}}{\text{min}}{g}_1(\mathbb{X})+g_2(\mathbb{Y})\text{subject to}{\mathbb{A}}_1\mathbb{X}+{\mathbb{A}}_2\mathbb{Y}=0,$$ (22)

where $\mathbb{X}\in {\mathbb{R}}^p$ and $\mathbb{X}\in {\mathbb{R}}^q$ are unknown parameters, $g_1(\mathbb{X})$ and $g_2(\mathbb{Y})$ are two closed convex functions, and ${\mathbb{A}}_1\in {\mathbb{R}}^{m\times p}$ and ${\mathbb{A}}_2\in {\mathbb{R}}^{m\times q}$ represent m linear constraints on $\mathbb{X}$ and $\mathbb{Y}$, respectively. ADMM solves (22) by breaking them into smaller and simpler subproblems and solving them alternatively. Specifically, for the t + 1 iteration,

$$\begin{array}{c}\hfill {\mathbb{X}}^{t+1}=\underset{\mathbb{X}}{\text{arg min}}\left\{g_1(\mathbb{X})+\frac{\rho }{2}\Vert {\mathbb{A}}_1\mathbb{X}+{\mathbb{A}}_2{\mathbb{Y}}^t+{\mathbf{u}}^t{\Vert }_2^2\right\},\hfill \\ \hfill \hfill \\ \hfill {\mathbb{Y}}^{t+1}=\underset{\mathbb{Y}}{\text{arg min}}\left\{g_2(\mathbb{Y})+\frac{\rho }{2}\Vert {\mathbb{A}}_1{\mathbb{X}}^{t+1}+{\mathbb{A}}_2\mathbb{Y}+{\mathbf{u}}^t{\Vert }_2^2\right\},\hfill \\ \hfill \hfill \\ \hfill {\mathbf{u}}^{t+1}={\mathbb{A}}_1{\mathbb{X}}^{t+1}+{\mathbb{A}}_2{\mathbb{Y}}^{t+1}+{\mathbf{u}}^t,\hfill \end{array}$$

where ρ is the augmented Lagrangian parameter, u is a vector of dual variables, and ∥·∥₂ denotes the L₂ Euclidean norm. The choice of ρ affects the convergence rate of the algorithm (Boyd et al., 2011), and remains an open question in the literature. We implement our algorithm with ρ = 1, but it can be tuned in practice.

A2. The construction of matrix A

To simplify the inner product in the loss function, we need to construct a big matrix A to summarize all the linear operations. First, we denote

$$W_{y_i}={\left(W_{y_i,1},W_{y_i,2},\dots ,W_{y_i,K-1}\right)}^T\in {\mathbb{R}}^{K-1}$$

and construct a matrix ${\tilde{W}}_Y$ consisting of a stack of diagonal matrices, i.e.,

$${\tilde{W}}_Y=\left[W_{Y,1},W_{Y,2},\cdots ,W_{Y,K-1}\right],$$

where W_Y,l = diag{W_y1,l, … , W_yn,i} for l = 1, … K – 1. Moreover, we denote $\tilde{\mathit{X}}=\text{diag}\{\mathit{X},\cdots ,\mathit{X}\}$ as a matrix consisting of K – 1 copies of the original covariate matrix on the diagonal. In particular, the columns of 1's are added at the first column of the covariate matrix to include the intercepts in the computation. Then, we define $A={\tilde{W}}_Y\tilde{\mathit{X}}$, and it can be verified that

$$A\beta =\left[\begin{array}{c}\hfill \langle W_{y_1},f({\mathit{X}}_1)\rangle \hfill \\ \hfill ⋮\hfill \\ \hfill \langle W_{y_n},f({\mathit{X}}_n)\rangle \hfill \end{array}\right].$$

A3. Spatial regularized regression

The ADMM algorithm proposed in this paper is quite flexible and can be extended to solve other problems. In this section, we introduce the extension of our algorithm to solve a spatial regularized regression problem.

For the regression problem, the response variable y_i can be a continuous measure of a certain clinical index. Denote the covariate image as X_i. The regularized regression problem is given by

$$\widehat{\beta }=\text{arg min}\left\{\frac{1}{2}\sum _{i=1}^n({\mathrm{y}}_i-{\mathbf{X}}_i\beta {)}^2+FL(\beta )\right\}.$$

This is analogous to equation (9), by letting K = 2 and applying the square loss, i.e., l(u) = u². Both the first- and second-order total variations can be applied here.

We adopt the reformulation of equation (9) and construct a similarly constrained optimization to problem (11), i.e.,

$$\underset{\beta ,{\mathrm{v}}_1,{\mathrm{v}}_2{\mathrm{v}}_3}{\text{min}}\frac{1}{2}\sum _{i=1}^n(y_i-{\mathbf{v}}_{1i}{)}^2+\Vert \mathbf{M}{\mathbf{v}}_3{\Vert }_1$$

Subject to v₁ = Xβ, v₂ = Rβ, and ${\mathbf{v}}_3=\tilde{\mathbf{B}}{\mathbf{v}}_2$.

Here, all the variable are the same as those in the algorithm for SMAC, but with fixed K = 2. The only change is that the loss function part becomes the squared error loss and A in problem (11) becomes X. Thus, the solutions for β, v₂ and v₃ remain the same by setting A = X in (14).

The subproblem involving v₁ becomes the following:

$${\mathbf{v}}_1^{t+1}=\underset{{\mathrm{v}}_1}{\text{argmin}}\left\{\frac{1}{2}\Vert \mathbf{Y}-{\mathbf{v}}_1{\Vert }_2^2+\frac{\rho }{2}\Vert A{\beta }^{t+1}-{\mathbf{v}}_1+{\mathbf{u}}_1^t{\Vert }_2^2\right\},$$

where Y = [y₁,…,y_n]^T. This is a standard quadratic programming problem for which the solution is given by

$${\mathbf{v}}_1^{t+1}=\frac{1}{1+\rho }\left(\mathbf{Y}+\mathbf{A}{\beta }^{t+1}-{\mathbf{u}}_1^t\right).$$ (23)

Therefore, the whole algorithm can be summarized as Algorithm 2.

Algorithm 2. ADMM algorithm for spatial regularized regression.

$$\begin{array}{cc}\hfill & \mathbf{Initialize}\text{primal variables}\beta ,{\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_3\text{as}0.\hfill \\ \hfill & \mathbf{Initialize}\text{dual variables}{\mathbf{u}}_1,{\mathbf{u}}_2,{\mathbf{u}}_3\text{as}0.\hfill \\ \hfill & \mathbf{Set}t=0,\text{assign}{\lambda }_1,{\lambda }_2\ge 0.\hfill \\ \hfill & \mathrm{Set}A=\mathbf{X}\text{and}K=2.\hfill \\ \hfill & \text{Precompute}{\mathbf{H}}_L={\mathbf{A}}^T{(I-{\mathbf{AA}}^T)}^{-1}\hfill \\ \hfill & \mathbf{while}t\le t_{\mathit{max}}\mathbf{do}\hfill \\ \hfill & \mathit{Primal}\mathit{update}:\hfill \\ \hfill & \begin{array}{c}\hfill {\beta }^{t+1}={\mathbf{K}}^t-{\mathbf{H}}_L{\mathbf{AK}}^t\hfill \\ \hfill {\mathbf{v}}_3^{t+1}=\mathbf{M}\times {\text{Soft}}_{\frac{1}{\rho }}(\tilde{\mathbf{B}}{\mathbf{v}}_2^t-u_3^t)-(\mathbf{I}-\mathbf{M})(\tilde{\mathbf{B}}{\mathbf{v}}_2^t-{\mathbf{u}}_3^t)\hfill \\ \hfill {\mathbf{v}}_1^{t+1}=\frac{1}{1+\rho }(\mathbf{Y}+\mathbf{A}{\beta }^{t+1}-{\mathbf{u}}_1^t)\hfill \\ \hfill {\mathbf{v}}_2^{t+1}=\mathit{ifft}\left(\frac{\mathit{fft}\left(\right(\mathbf{R}{\beta }^{t+1}+{\mathbf{u}}_2^t)+{\tilde{\mathbf{B}}}^T({\mathbf{v}}_3^{t+1}+{\mathbf{u}}_3^t\left)\right)}{\mathit{fft}}({\Gamma }_1)\right)\hfill \end{array}\begin{array}{c}\left(14\mathrm{b}\right)\hfill \\ \left(15\mathrm{b}\right)\hfill \\ \left(23\mathrm{a}\right)\hfill \\ \left(19\mathrm{b}\right)\hfill \end{array}\hfill \\ \hfill & \mathit{Dual}\mathit{update}:\hfill \\ \hfill & \begin{array}{c}{\mathbf{u}}_1^{t+1}=\mathbf{A}{\beta }^{t+1}-{\mathbf{v}}_1^{t+1}+{\mathbf{u}}_1^t\hfill \\ {\mathbf{u}}_2^{t+1}=\mathbf{R}{\beta }^{t+1}-{\mathbf{v}}_2^{t+1}+{\mathbf{u}}_2^t\hfill \\ {\mathbf{u}}_3^{t+1}={\mathbf{v}}_3-\tilde{\mathbf{B}}{\mathbf{v}}_2^{t+1}+{\mathbf{u}}_3^t\hfill \end{array}\hfill \\ \hfill & \mathit{Convergence}\mathit{criteria}:\hfill \\ \hfill & \begin{array}{c}\mathbf{if}\Vert {\beta }^{t+1}-{\beta }^t\Vert ∕\Vert {\beta }^t\Vert >\epsilon \mathbf{then}\hfill \\ t=t+1\hfill \end{array}\hfill \\ \hfill & \mathbf{else}\hfill \\ \hfill & \begin{array}{c}\mathbf{break}\hfill \\ \mathbf{return}\beta ={\beta }^{t+1}\hfill \end{array}\hfill \\ \hfill & \mathbf{end}\mathbf{if}\hfill \\ \hfill & \mathbf{end}\mathbf{while}\hfill \end{array}$$