Scalar-on-Image Regression via the Soft-Thresholded Gaussian Process

SUMMARY

This work concerns spatial variable selection for scalar-on-image regression. We propose a new class of Bayesian nonparametric models and develop an efficient posterior computational aigorithm. The proposed soft-thresholded Gaussian process provides large prior support over the class of piecewise-smooth, sparse, and continuous spatially-varying regression coefficient functions. In addition, under some mild regularity conditions the soft-thresholded Gaussian proess prior leads to the posterior consistency for parameter estimation and variable selection for scalar-on-image regression, even when the number of predictors is larger than the sample size. The proposed method is compared to alternatives via simulation and applied to an electroen-cephalography study of alcoholism.

1. INTRODUCTION

Scalar-on-image regression has attracted considerable attention recently in both the frequentist and Bayesian literature. This problem is challenging for several reasons: the predictor is a two-dimensional or three-dimensional image where the number of pixels or voxels is often larger han the sample size; the observed predictors may be contaminated with noise; the true signal ay exhibit complex spatial structure; and most components of the predictor may have no effect on the response, and when they have an effect it may vary smoothly.

Regularized regression techniques are often needed when the number of predictors is much higher than the sample size. The lasso (Tibshirani, 1996) is a popular method for variable selection that employs a penalty based on the sum of the absolute values of the regression coefficients.However, most penalized approaches do not accommodate predictors with ordered components such as an image predictor. One exception is the fused lasso, which generalizes the lasso by pe-nalizing both the coefficients and their successive differences and thus ensures both sparsity and smoothness of the estimated effect. To incorporate the spatial structure of the predictors, Reiss & Ogden (2010) extended functional principal component regression to handle image predictors by approximating the coefficient function using B-splines. However, this method is not sensitive to sparsity or sharp edges. Wang et al. (2016) proposed a penalty based on the total variation of the regression function that yields piecewise smooth regression coefficients. The approach is focused primarily on prediction and does not quantify uncertainty in a way that permits statistical inference. Reiss et al. (2015) considered a wavelet expansion for the regression coefficients and conducted inference via hypothesis testing. Their approach requires that the image predictor has dimensions of equal size and moreover that the common size is a power of two. These strong assumptions are violated for our motivating application. Therefore, none of these methods are appropriate to detect a piecewise smooth, sparse, and continuous signal for scalar outcome and image predictor.

Scalar-on-image regression has been also approached from a Bayesian viewpoint. Goldsmith et al. (2014) proposed to model the regression coefficients as the product of two latent spatial processes to capture both sparsity and spatial smoothness of the important regression coefficients. They used an Ising prior for the binary indicator that a voxel is predictive of the response, and a conditionally autoregressive prior to smooth the nonzero regression coefficients. Use of an Ising prior for binary indicators was first discussed in Smith & Fahrmeir (2007) in the context of high dimensional spatial predictors and was also recently employed by Li et al. (2015) who use a Dirichlet process prior for the non-zero coefficients. In both Li et al. (2015) and Goldsmith et al. (2014), sparsity and smoothness are controlled separately by two independent spatial processes so the transitions from zero areas to neighboring nonzero areas may be abrupt, and computation is challenging because the Ising probability mass function does not have a simple closed form.

We propose an alternative approach for spatial variable selection in scalar-on-image regression by modeling the regression coefficients through soft-thresholding of a latent Gaussian process. The soft-thresholding function is well known for its relation with the lasso estimate when the design matrix is orthonormal (Tibshirani, 1996), and here we use it to specify a spatial prior with mass at zero. The idea is inspired by Boehm Vock et al. (2014) who considered Gaussian processes as a regularization technique for spatial variable selection. However their approach does not assign prior probability mass at zero for regression coefficients and is not designed for scalar-on-image regression. Unlike other Bayesian spatial models (Goldsmith et al., 2014; Li 65 et al., 2015), the soft-thresholded Gaussian process ensures a gradual transition between the zero and nonzero effects of the neighboring locations and provides large support over the class of spatially-varying regression coefficient functions that are piecewise smooth, sparse, and continuous. The use of the soft-thresholded Gaussian process avoids the computational problems posed by the Ising prior, and can be scaled to large datasets using a low-rank spatial model for the latent process. We show that the soft-thresholded Gaussian process prior leads to posterior consistency for both parameter estimation and variable selection under mild regularity conditions, even when the number of predictors is larger than the sample size.

The proposed method is introduced for a single image predictor and Gaussian responses mainly for simplicity. Extensions to accommodate multiple predictors and non-Gaussian responses are relatively straightforward. However, the theoretical investigation of the procedure for non-Gaussian responses is not trivial, and so we establish the theory for binary responses and the probit link function. The methods are applied to the data from an electroencephalography, EEG, study of alcoholism; see http://kdd.ics.uci.edu/datasets/eeg/eeg.data.html. The objective of our analysis is to estimate the relationship between alcoholism and brain activity. EEG signals are recorded for both alcoholics and controls at 64 channels of electrodes on subjects’ scalps for 256 seconds, leading to a high-dimensional predictor. The data have been previously described in Li et al. (2010) and Zhou & Li (2014). Previous literature has ignored the spatial structure of the electrodes shown in Fig. 1, which was recovered from the standard electrode position nomen-clature described by Fig. 1 of https://www.acns.org/pdf/guidelines/Guideline-5.pdf Our spatial analysis exploits the spatial configuration of the electrodes and reveals regions of the brain with activity that are predictive of alcoholism.

Fig. 1.

Standard electrode position nomenclature for 10–10 system

2. MODEL

2.1. Scalar-on-image regression

Let ${ℝ}^m$ be a m-dimensional vector space of real values. Suppose there are n subjects in the dataset and the data for the subject i consist of a scalar response variable, $Y_i\in {ℝ}^1$ a set of $p_n$ spatially-distributed image predictors, denoted $X_i={\left(X_{i,1},\dots ,X_{i,p_n}\right)}^{\text{T}}\in {ℝ}^{p_n}$ and other scalar covariates collected by $W_i={\left(W_{i,1},\dots ,W_{i,q}\right)}^{\text{T}}\in {ℝ}^q.$ Assume that ${\left\{W_i\right\}}_{i=1}^n$ are fixed design covariates. Here $X_{i,j}$ denotes the image intensity value measured at location $s_{j,}$ for j = 1,...,p_n. We assume that the set of locations $S={\left\{s_j\right\}}_{j=1}^{p_n}$ is a fixed subset of a compact closed region $\mathcal{B}\subseteq {ℝ}^d.\text{Let}\mathrm{N}(\mu ,\Sigma )$ denote a normal distribution with mean µ and variance-covariance matrix Σ or variance for one-dimensional case. We consider the following scalar-on-image regression model:

$$\left(Y_i|W_i,X_i,{\alpha }^{\text{v}},\mathit{\beta },{\sigma }^2\right)~\text{N}\left\{\sum _{k=1}^q{\alpha }_k{W}_{i,k}+p_n^{-1/2}\sum _{j=1}^{p_n}\mathit{\beta }\left(s_j\right)X_{i,j},{\sigma }^2\right\},i=1,\dots ,n,$$ (1)

where ${\alpha }^{\text{v}}={\left({\alpha }_1,\dots ,{\alpha }_q\right)}^{\text{T}}$ quantifies the effect of W_i and β(·) is a spatially-varying coefficient function defined on Ɓ. In practice, the normalizing scalar $p_n^{-1/2}$ can be absorbed into the image predictors; its role is to rescale the total effects of massive image predictors such that they are bounded away from infinity with a large probability, when p_n is very large. Scientifically, in brain imaging studies, the image predictors take values that measure the body tissue contrast or the neural activities at each spatial location and the number of image predictors, $p_n,$ is determined by the image resolution. Thus, the total effect of the image predictors reflects the total amount of the intensity in the brain signals, which should not increase to infinity as the image resolution increases. In model (1), the response is taken to be Gaussian and only one type of image predictor is included, although extensions of the modeling framework to non-Gaussian responses and multi-modality image predictor regression are straightforward.

2.2. Soft-thresholded Gaussian processes

To capture the characteristics of the image predictors and their effects on the response variable, the prior for $\mathit{\beta }(\cdot )$ should be sparse and spatial. That is, we assume that many locations have $\mathit{\beta }\left(s_j\right)=0,$ the sites with nonzero coefficients cluster spatially, and the coefficients vary smoothly in clusters of nonzero coefficients. To encode these desired properties into the prior, we represent $\mathit{\beta }(\cdot )$ as a transformation of a Gaussian process, $\mathit{\beta }(s)=g_{\lambda }\{\tilde{\mathit{\beta }}(s)\},$ where $g_{\lambda }$ is the transformation function dependent on parameter $\lambda \text{and}\tilde{\mathit{\beta }}(s)$ follows a Gaussian process prior. In this trans-kriging (Cressie, 1993) or Gaussian copula (Nelsen, 1999) model, the function $g_{\lambda }$ determines the marginal distribution of $\mathit{\beta }(s),$ while the covariance of the latent $\tilde{\mathit{\beta }}(s)$ determines $\mathit{\beta }{(s)}^{\prime }\text{s}$ dependence structure.

Spatial dependence is determined by the prior for $\tilde{\mathit{\beta }}(s).$ We assume that $\tilde{\mathit{\beta }}(\cdot )$ is a Gaussian process with mean zero and stationary covariance function $\text{cov}\left\{\tilde{\beta }(s),\tilde{\beta }\left(s^{\prime }\right)\right\}=\kappa \left(s-s^{\prime }\right)$ for some covariance function $k.$ Although other transformations are possible (Boehm Vock et al., 2014), we select $g_{\lambda }$ to be the soft-thresholding function to map $\tilde{\mathit{\beta }}(s)$ near zero to exact zero and thus give a sparse prior. Let

$$g\lambda (x)=\{\begin{array}{cc}0,& |x|\le \lambda ,\\ \text{sgn}(x)(|x|-\lambda ),& |x|>\lambda ,\end{array}$$ (2)

where $\text{sgn}(x)=1$ if x > 0, $\text{sgn}(x)=-1$ if x < 0, sgn(0) = 0. The thresholding parameter $\text{λ}>0$ determines the degree of sparsity. This soft-thresholded Gaussian process prior is denoted $\mathit{\beta }~\text{STGP}(\text{λ},\kappa ).$

3. THEORETICAL PROPERTIES

3.1. Notations and Definitions

We first introduce additional notation for the theoretical development and the formal definitions of the class of spatially-varying coefficient functions under consideration. We assume that all the random variables and stochastic processes in this article are defined on the same probability space, denoted $(\Omega ,\mathcal{F},\Pi ).\text{Let}{\mathbb{Z}}_{+}^d={\{0,1,\dots ,\}}^d\subset {ℝ}^d$ represent a space of d-dimensional non-negative integers. For any vector $v={\left(v_1,\dots ,v_d\right)}^{\text{T}}\in {ℝ}^d,\text{let}\Vert v{\Vert }_p={\left({\sum }_{l=1}^d{|v_l|}^p\right)}^{1/p}$ be the L_p norm for vector v for any $p\ge 1,\text{and let}\Vert v{\Vert }_{\infty }={\max }_{l=1}^d|v_l|$ be the supremum norm. For any $x\in R,\text{let}[x]$ be the smallest integer not smaller than x and $\lfloor x\rfloor$ be the largest integer not larger than x. Define the event indicator $I(\mathcal{A})=1.$ if event $\mathcal{A}\text{occurs},I(\mathcal{A})=0$ otherwise. For any $z={\left(z_1,\dots ,z_d\right)}^{\text{T}}\in {\mathbb{Z}}_{+}^d\text{define}z!={\prod }_{l=1}^d{\prod }_{k=1}^{kl}k$ and define $v^z={\prod }_{l=1}^d{v}_l^{z_l}$. For any real function f on Ɓ, let $||f|{|}_p=\{{\int }_B|f(s){|}^{P}ds{\}}^{1/p}$ denote the L_p norm for any p ≥ 1 and $||f{||}_{\infty }={\text{sup}}_{s\in \mathcal{B}}|f(s)|$ denote the supremum norm.

Definition 1. Denote by ${\mathcal{C}}^m(\mathcal{B})$ the set of differentiable functions of order m defined on Ɓ, $f(\cdot ),$ such that f(s) has partial derivatives

$$D^{\tau }f(s)=\frac{\partial \Vert \tau {\Vert }_{1}f}{s_1^{{\tau }_1}\cdots s_d^{{\tau }_d}}(s)=\sum _{\Vert \eta {\Vert }_1+\Vert \tau {\Vert }_1\le m}\frac{D^{\tau +\eta }f(t)}{\eta !}{(s-t)}^{\eta }+R_m(s,t),$$

where $\tau ={\left({\tau }_1,\dots ,{\tau }_d\right)}^{\text{T}}\in {\mathbb{Z}}_{+}^d,\eta \in {\mathbb{Z}}_{+}^d,t\in {ℝ}^d,$ given any point s₀ of Ɓ and any $ϵ>0$, there is a $\delta >0$ such that if s and t are any two points of Ɓ with $\parallel s-s_0{\parallel }_1<\delta$ and $\parallel t-s_0{\parallel }_1<\delta$, then $|R_m(s,t)|\le \parallel s-t{\parallel }_1^{m-\parallel \tau {\parallel }_1}\epsilon$. If $\parallel D^{\tau }f{\parallel }_{\infty }\le M<\infty$, then $|R_m(s,t)|\le (M\parallel s-t{\parallel }_1^{m+1})/(m+1)!$.

Denote by $\overline{\mathcal{R}}$ and $\partial \mathcal{R}$ the closure and the boundary of any set $\mathcal{R}\subseteq \mathcal{B}$

Definition 2. Define $\Theta =\{\mathit{\beta }(s):s\in \mathcal{B}\}$ to be the collection of all spatially-varying coefficient functions that satisfy the following conditions. Assume there exist two disjoint non-empty open sets ${\mathcal{R}}_{-1}\text{and}{\mathcal{R}}_1\text{with}{\overline{\mathcal{R}}}_1\cap {\overline{\mathcal{R}}}_{-1}=\varnothing$ such that

$$\mathit{\beta }(\cdot )\mathrm{is\; smooth\; over}{\overline{\mathcal{R}}}_{-1}\cup {\overline{\mathcal{R}}}_1,\mathit{\beta }(s)I\left(s\in {\overline{\mathcal{R}}}_{-1}\cup {\overline{\mathcal{R}}}_1\right)\in {\mathcal{C}}^{\rho }\left({\overline{\mathcal{R}}}_{-1}\cup {\overline{\mathcal{R}}}_1\right),\rho =\left[d/2\right].$$ (2.1)

$$\mathit{\beta }(s)=0\mathrm{for}s\in {\mathcal{R}}_0,\mathit{\beta }(s)>0\mathrm{for}s\in {\mathcal{R}}_1\mathrm{and}\mathit{\beta }(s)<0\mathrm{for}s\in {\mathcal{R}}_{-1},\mathrm{where}{\mathcal{R}}_0=\mathcal{B}-\left({\mathcal{R}}_{-1}\cup {\mathcal{R}}_1\right)\mathrm{and}{\mathcal{R}}_0-\left(\partial {\mathcal{R}}_1\cup \partial {\mathcal{R}}_{-1}\right)\ne \varnothing .$$ (2.2)

$$\mathit{\beta }(\cdot )\mathit{\text{is}}\mathit{\text{continuous}}\mathit{\text{over}}\mathcal{B}.\mathit{\text{That is}},\underset{s\to s_0}{\lim }\mathit{\beta }(s)=\mathit{\beta }\left(s_0\right),s_0\in \mathcal{B}.$$ (2.3)

Simply put, $\Theta$ is the collection of all piecewise smooth, sparse and continuous functions defined on Ɓ.

3.2. Large Support

One desired property for the Bayesian nonparametric model is to have prior support over a large class of functions. In this section, we show that the soft-thresholded Gaussian process has large support over $\Theta$. We begin with two appealing properties of the soft-thresholding function. All technical conditions are listed in the Appendix, as Conditions A1-A5.

Lemma 1. The soft-thresholding function g_λ(x) is Lipschitz continuous for any λ > 0, that is, for all $x_1,x_2\in ℝ,|g_{\lambda }\left(x_1\right)-g_{\lambda }\left(x_2\right)|\le |x_1-x_2|.$

Lemma 2. For any function ${\mathit{\beta }}_0\in \Theta$ and any threshold parameter λ₀ > 0, there exists a smooth function ${\tilde{\mathit{\beta }}}_0(s)\in {\mathcal{C}}^{\rho }(\mathcal{B})$ such that ${\mathit{\beta }}_0(s)=g_{{\lambda }_0}\left\{{\tilde{\mathit{\beta }}}_0(s)\right\}$.

Lemma 1 is proved directly by verifying the definition. The proof of Lemma 2 is not trivial, it requires a detailed construction of the smooth function ${\tilde{\mathit{\beta }}}_0(s)$. See the Appendix for details.

Theorem 1. For any function ${\mathit{\beta }}_0\in \Theta$ and $\epsilon >0$, the soft-thresholded Gaussian process prior $\mathit{\beta }~\text{STGP}\left({\lambda }_0,\kappa \right)$ satisfies $\Pi \left({\Vert \mathit{\beta }-{\mathit{\beta }}_0\Vert }_{\infty }<\epsilon \right)>0$, for any ${\lambda }_0>0$ and k that satisfy Condition A5.

Proof. By Lemma 2, for any threshold parameter λ₀ > 0 and a smooth function ${\tilde{\mathit{\beta }}}_0(s)\in {\mathcal{C}}^{\rho }(\mathcal{B})$ such that ${\mathit{\beta }}_0(s)=g_{{\lambda }_0}\left\{{\tilde{\mathit{\beta }}}_0(s)\right\}$. Since $\mathit{\beta }~\text{STGP}\left({\lambda }_0,\kappa \right)$, we have $\mathit{\beta }(s)=g_{{\lambda }_0}\{\tilde{\mathit{\beta }}(s)\}$ with $\tilde{\mathit{\beta }}(s)~\mathcal{G}\mathcal{P}(0,\kappa )$, By Lemma 1,

$$\Pi \left\{\underset{s\in \mathcal{B}}{\sup }|\mathit{\beta }(s)-{\mathit{\beta }}_0(s)|<\epsilon \right\}=\Pi \left[\underset{s\in \mathcal{B}}{\sup }|g_{{\lambda }_0}\{\tilde{\mathit{\beta }}(s)\}-g_{{\lambda }_0}\left\{{\tilde{\mathit{\beta }}}_0(s)\right\}|<\epsilon \right]\ge \Pi \left\{\underset{s\in \mathcal{B}}{\sup }|\tilde{\mathit{\beta }}(s)-{\tilde{\mathit{\beta }}}_0(s)|<\epsilon \right\}.$$

By Condition A5 and Theorem 4.5 of Tokdar & Ghosh (2007), ${\tilde{\mathit{\beta }}}_0(\cdot )$ is in the reproducing kernel Hilbert space of k, and then by Theorem 4 of Ghosal & Roy (2006), we have $\Pi \left\{\underset{s\in \mathcal{B}}{\sup }|\tilde{\mathit{\beta }}(s)-{\tilde{\mathit{\beta }}}_0(s)|<\epsilon \right\}>0$ which completes the proof. □

Theorem 1 implies that there is always a positive probability that the soft-thresholded Gaussian process concentrates in an arbitrarily small neighborhood of any spatially-varying coefficient function that has piecewise smoothness, sparsity and continuity properties. According to Lemma 2, for any positive ${\lambda }_1\ne {\lambda }_2$, there exist ${\tilde{\mathit{\beta }}}_1,{\tilde{\mathit{\beta }}}_2\in {\mathcal{C}}^{\rho }(\mathcal{B})$ such that ${\mathit{\beta }}_0(s)=g_{{\lambda }_1}\left\{{\tilde{\mathit{\beta }}}_1(s)\right\}=g_{{\lambda }_2}\left\{{\tilde{\mathit{\beta }}}_2(s)\right\}$. This implies that the thresholding parameter ${\lambda }_0$ and the latent smooth curve ${\tilde{\mathit{\beta }}}_0$ are not identifiable, but we can ensure that β₀ is identifiable by establishing the posterior consistency of parameter estimation.

3.3. Posterior Consistency

For i = 1,...,n, given the image predictor X_i on a set of spatial locations S and other covariates W_i, suppose the response Y_i is generated from the scalar-on-image regression model (1) with parameters a ${\alpha }_0^{\text{V}}\in {ℝ}^q$, ${\sigma }_0^2>0$ and ${\mathit{\beta }}_0\in \Theta$. We assume ${\alpha }_0^{\text{V}}$ and ${\sigma }_0^2$ are known for theoretical convenience; in practice it is straightforward to estimate them from the data. We assign a soft-thresholded Gaussian process prior for the spatially-varying coefficient function, i.e. $\mathit{\beta }~\text{STGP}(\lambda ,\kappa )$ for any given λ > 0 and covariance kernel k. In light of the large support property in Theorem 1, the following lemma shows the positivity of prior neighborhoods:

Lemma 3. Denote by ${\pi }_{n,i}(\cdot ;\mathit{\beta })$ the density function of $Z_{n,i}=\left(Y_i,W_i,X_i\right)$ in model (1) and suppose Condition A4 holds for $X_i$. Define ${\Lambda }_{n,i}\left(\cdot ;{\mathit{\beta }}_0,\mathit{\beta }\right)=\log {\pi }_{n,i}(\cdot ;\mathit{\beta })-\log {\pi }_{n,i}\left(\cdot ;{\mathit{\beta }}_0\right)$, $K_{n,i}\left({\mathit{\beta }}_0,\mathit{\beta }\right)=E_{{\mathit{\beta }}_0}\left\{{\Lambda }_{n,i}\left(Z_{n,i};{\mathit{\beta }}_0,\mathit{\beta }\right)\right\}$ and $V_{n,i}\left({\mathit{\beta }}_0,\mathit{\beta }\right)=\mathrm{var}{}_{{\mathit{\beta }}_0}\left\{{\Lambda }_{n,i}\left(Z_{n,i};{\mathit{\beta }}_0,\mathit{\beta }\right)\right\}$. There exists a set B with $\Pi (B)>0$ such that, for any $\epsilon >0$,

$$\underset{n\to \infty }{\lim \inf }\Pi \left[\left\{\mathit{\beta }\in B,n^{-1}\sum _{i=1}^n{K}_{n,i}\left({\mathit{\beta }}_0,\mathit{\beta }\right)<\epsilon \right\}\right]>0,n^{-2}\sum _{i=1}^n{V}_{n,i}\left({\mathit{\beta }}_0,\mathit{\beta }\right)\to 0,\mathit{\beta }\in B.$$

We construct sieves for the spatially-varying coefficient functions in $\Theta$ as

$${\Theta }_n=\left\{\mathit{\beta }\in \Theta :\Vert \mathit{\beta }{\Vert }_{\infty }\le p_n^{1/(2d)},\underset{s\in {\mathcal{R}}_1\cup {\mathcal{R}}_{-1}}{\sup }|D^{\tau }\mathit{\beta }(s)|\le p_n^{1/(2d)},1\le \Vert \tau {\Vert }_1\le \rho \right\},$$ (3)

where ρ is defined in Condition A1. By Lemmas A1-A5 in the Appendix, we can find an upper bound for the tail probability and construct uniform consistent tests in the following lemmas:

Lemma 4. If $\mathit{\beta }(s)~\text{STGP}\left({\lambda }_0,\kappa \right)$ with λ₀ > 0 and the kernel function k satisfies Condition A5, then there exist constants K and b such that for all n ≥ 1, $\Pi \left({\Theta }_n^C\right)\le K\exp \left(-b{p}_n^{1/d}\right)$.

Lemma 5. For any $\epsilon >0$ and $v_0/2<v<1/2$, there exist N, C₀, C₁ and C₂ such that for all n > N and all $\mathit{\beta }\in {\Theta }_n$, if ${\Vert \mathit{\beta }-{\mathit{\beta }}_0\Vert }_1>\epsilon$, a test function ${\Psi }_n$ can be constructed such that $E_{{\mathit{\beta }}_0}\left({\Psi }_n\right)\le C_0\exp \left(-C_2{n}^{2v}\right)$ and $E_{\mathit{\beta }}\left(1-{\Psi }_n\right)\le C_0\exp \left(-C_{1}n\right)$, where ${\upsilon }_0$ is defined in Condition A1.

Proofs of Lemmas 3–5 are provided in the Supplementary Material. These lemmas verify three important conditions for proving posterior consistency in the scalar-on-image regression based on Theorem A1 of Choudhuri et al. (2004). It follows that we have the following theorem:

Theorem 2. Denote the data by $D_n={\left\{Y_i,X_i,W_i\right\}}_{i=1}^n$. If Conditions A1-A5 hold, then for any ε > 0,

$$\Pi \left(\mathit{\beta }\in \Theta :{\Vert \mathit{\beta }-{\mathit{\beta }}_0\Vert }_1<\epsilon |D_n\right)\to 1,n\to \infty .$$ (4)

Theorem 2 implies that the soft-thresholded Gaussian process prior can ensure that the posterior distribution of the spatially-varying coefficient function concentrates in an arbitrarily small neighborhood of the true value, when the numbers of subjects and spatial locations are both sufficiently large. Given that the true function of interest is piecewise smooth, sparse and continuous, the soft-threshold Gaussian process prior can further ensure that the posterior probability of the sign of the spatially-varying coefficient function being correct converges to one as the sample size goes to infinity. The result is formally stated in the following theorem.

Theorem 3. Suppose the model assumptions, prior settings and regularity conditions for Theorem 2 hold. Then

$$\Pi \left[\text{sgn}\{\mathit{\beta }(s)\}=\text{sgn}\left\{{\mathit{\beta }}_0(s)\right\},\mathrm{for}\mathrm{all}s\in \mathcal{B}|D_n\right]\to 1,n\to \infty .$$ (5)

This theorem establishes the consistency of spatial variable selection. It does not require that the number of true image predictors is finite or less than the sample size. This result is reasonable in that the true spatially-varying coefficient function is piecewise smooth and continuous, and the soft-thresholded Gaussian process will borrow strength from neighboring locations to estimate the true image predictors. The Supplementary Material gives proofs of Theorems 2 and 3.

To apply the proposed model for the motivating dataset, we extend the proposed model (1) and Theorems 2–3 to analysis of the binary response variable using a probit model:

$$Y_i~\text{Bernoulli}\left({\pi }_i\right),{\Phi }^{-1}\left({\pi }_i\right)=\sum _{k=1}^q{\alpha }_k{W}_{i,k}+p_n^{-1/2}\sum _{j=1}^{p_n}\mathit{\beta }\left(s_j\right)X_{i,j},$$ (6)

where $\Phi (\cdot )$ is the standard normal cumulative distribution function.

Theorem 4. Assume the data D_n are generated from model (6) and prior settings are the same as Theorem 2. If Conditions A1–A5 and S1–S3 hold, then (4) and (5) hold under model (6).

Conditions S1–S3 and the proof of Theorem 4 are in the Supplementary Material.

4. POSTERIOR COMPUTATION

4.1. Model representation and prior specification

Next, we discuss the practical applicability of our method. We select a low-rank spatial model to ensure that computation remains possible for large datasets. We exploit the kernel convolution approximation of a spatial Gaussian process. As discussed in Higdon et al. (1999), any stationary Gaussian process V(s) can be written as $V(s)=\int K(s-t)w(t)dt$, where K is a kernel function and w is a white-noise process with mean zero and variance ${\sigma }_w^2$. Then the covariance function of $V(\cdot )$ is cov$\{V(s),V(s+h)\}=\kappa (h)={\sigma }_a^2\int K(s-t)K(s+h-t)dt$.

This representation suggests the following approximation for the latent process $\tilde{\mathit{\beta }}(s)\approx {\sum }_{l=1}^{L}K(s-t_l)a_l\delta$, where $t_1,\dots ,t_L\in {ℝ}^d$ are a grid of equally-spaced spatial knots covering Ɓ and δ is the grid size. Without loss of generality, we assume δ = 1 and K is a local kernel function. We use tapered Gaussian kernels with bandwidth ${\sigma }_h$, $K(h)=\exp \left\{-h^2/{\left(2{\sigma }_h\right)}^2\right\}I(h<3{\sigma }_h)$, so that $K(\parallel s-t_l\parallel )=0$ for s separated from t_l by at least $3{\sigma }_h$. Taking L < p knots and selecting compact kernels leads to computational savings, as discussed in Section 4.2.

The compact kernels K control the local spatial structure and the prior for the coefficients a = (a₁,..., a_L)^T controls broad spatial structure. Following Nychka et al. (2015), we assume that the knots t₁,...,t_L are arranged on an $m_1\times \cdots \times m_d$ array, and use $l~k$ to denote that knots t_l and t_k are adjacent on this array. We then use a conditionally autoregressive prior (Gelfand et al., 2010) for the kernel coefficients. The conditional autoregressive prior is also defined locally, with full conditional distribution

$$a_l|a_k,k\ne l~\text{N}\left(\frac{\vartheta }{n_l}\sum _{k~l}{a}_k,\frac{{\sigma }_a^2}{n_l}\right),$$ (7)

where n_l is the number of knots adjacent to the knot l, $\vartheta \in (0,1)$ quantifies the strength of spatial dependence, and ${\sigma }_a^2$ determines the variance. These full conditional distributions correspond to the joint distribution $a~\text{N}\left\{0,{\sigma }_a^2{(M-\vartheta A)}^{-1}\right\}$, where M is diagonal with diagonal elements (n₁...,n_L) and A is the adjacency matrix with (k, l) element equal 1 if k ~ l and zero otherwise.

Write ${\tilde{\mathit{\beta }}}^{\text{v}}={\left\{\tilde{\mathit{\beta }}\left(s_1\right),\dots ,\tilde{\mathit{\beta }}\left(s_p\right)\right\}}^{\text{T}}$. Denote by K the $p\times L$ kernel matrix with (j, l) element $K\left({\Vert s_j-t_l\Vert }_2\right)$, then ${\tilde{\mathit{\beta }}}^{\text{v}}~\text{N}\left\{0,{\sigma }_a^{2}K{(M-\vartheta A)}^{-1}{K}^{\text{T}}\right\}$ as a prior distribution. In this case, the $\tilde{\mathit{\beta }}\left(s_j\right)$ do not have equal variances which may be undesirable. Non-constant variance arises because the kernel knots may be unequally distributed, and because the conditional autoregressive model is non-stationary in that the variances of the a_l are unequal.

To stabilize the prior variance, define ${\tilde{K}}_{j,l}=K\left({\Vert s_j-t_l\Vert }_2\right)/w_j$ and let $\tilde{K}$ be the corresponding p × L matrix of standardized kernel coefficients, where w_j are constants chosen so that the prior variance for β_j is the same over j. We take w_j to be the square root of the j-th diagonal element of $K{(M-\vartheta A)}^{-1}{K}^T$, so the kernel functions depend on $\mathcal{\vartheta }$. By pulling the prior standard deviation σ_a out of the thresholding transformation we have an equivalent model representation of model (1) as

$$Y_i~\text{N}\left(W_i^{\text{T}}{\alpha }^{\text{v}}+p_n^{-1/2}{X}_i^{\text{T}}{\mathit{\beta }}^{\text{v}},{\sigma }^2\right),\mathit{\beta }\left(s_j\right)={\sigma }_a{g}_{\lambda }\{\tilde{\mathit{\beta }}\left(s_j\right)\},$$ (8)

where ${\tilde{\beta }}^{\text{v}}~\text{N}\left\{0,\tilde{K}{(M-\vartheta A)}^{-1}{\tilde{K}}^{\text{T}}\right\}$. After standardization, the prior variance of each $\tilde{\mathit{\beta }}\left(s_j\right)$ is one, and therefore the prior probability that $\tilde{\mathit{\beta }}\left(s_j\right)$ is nonzero is $2\Phi (-\lambda )$ for all j. This endows each parameter with a distinct interpretation: σ_a controls the scale of the nonzero coefficients, λ controls the prior degree of sparsity, and $\vartheta$ controls spatial dependence. With an additional set of conditions, we can show that model representation (8) when $L\to \infty$ to has a large prior support, thus leading to the posterior consistency and selection consistency. The Supplementary Material contains more details.

In practice, we normalize the response and covariates, and then select priors ${\alpha }^{\text{V}}~\text{N}\left(0,{10}^2{\text{I}}_q\right)$, ${\sigma }^2~\text{InvGamma}(0\cdot 1,0\cdot 1)$, ${\sigma }_a~\text{HalfNormal}(0,1)$, $\vartheta ~\text{Beta}(10,1)$, and $\lambda ~\text{Uniform}(0,5)$. Following Banerjee et al. (2004), we use a beta prior for $\vartheta$ with mean near one because only values near one provide appreciable spatial dependence. Finally, although our asymptotic results suggest that any λ > 0 will give posterior consistency, in finite samples posterior inference may be sensitive to the threshold so we use a prior rather than a fixed value.

4.2. Markov chain Monte Carlo Algorithm

For fully Bayesian inference on model (1), we sample from the posterior distribution using Metropolis-Hastings within Gibbs sampling. The parameters α^v, σ², and ${\sigma }_a^2$ have conjugate full conditional distributions and are updated using Gibbs sampling. The spatial dependence parameter $\vartheta$ is sampled using Metropolis-Hastings sampling using a beta candidate distribution with the current value as mean and standard deviation tuned to give acceptance rate around 0.4. The threshold λ is updated using Metropolis sampling with random-walk Gaussian candidate distribution with standard deviation tuned to have acceptance probability around 0.4. The Metropolis update for a_l uses the prior full conditional distribution in (7) as the candidate distribution which gives a high acceptance rate and thus good mixing without tuning.

To make posterior inference for the probit regression model (6), we can slightly modify the aforementioned algorithm by introducing an auxiliary continuous variable $Y_i^{*}$ for each response variable Y_i. We assume that $Y_i=I\left(Y_i^{*}>0\right)$ and $Y_i^{*}$ follows (1) with σ² = 1. The full conditional of $Y_i^{*}$ is a truncated normal distribution, which is straightforward to generate as a Gibbs sampling step in the posterior computation. The updating schemes for other parameters remain the same. For other types of non-Gaussian response models, we can directly use the Metropolis-Hastings sampling by modifying the likelihood function accordingly.

5. SIMULATION STUDY

5.1. Data generation

In this section we conduct a simulation study to compare the proposed methods with other popular methods for scalar-on-image regression. For each simulated observation, we generate a two-dimensional image X_i on the {1,...,m} × {1,...,m} grids with m = 30. The co-variates are generated following two covariance structures: exponential and with shared structure with the signal. The exponential covariates are Gaussian with mean E(X_ij) = 0 and $\mathrm{cov}\left(X_{i,j},X_{i,l}\right)=\exp \left(-d_{j,l}/{\vartheta }_x\right)$, where d_j,l is the distance between locations j and l and ${\vartheta }_x$ controls the range of spatial dependence. The covariates generated with shared structure with ${\mathit{\beta }}^{\text{v}}$ are $X_i={\tilde{X}}_i/2+e_i{\mathit{\beta }}^{\text{v}}$, where ${\tilde{X}}_i$ is Gaussian with exponential covariance with ${\vartheta }_x=3$ and $e_i~\text{N}\left(0,v^2\right)$. The continuous response is then generated as $Y_i~\text{N}\left(X_i^{\text{T}}{\mathit{\beta }}^{\text{v}},{\sigma }^2\right)$. For binary response as $Y_i~\text{Bernoulli}\left({\pi }_i\right)$ with ${\Phi }^{-1}\left({\pi }_i\right)=X_i^{\text{T}}{\mathit{\beta }}^{\text{v}}$. Both X_i and Y_i are independent for i = 1,...,n. We consider three true ${\mathit{\beta }}^{\text{v}}$ images: two sparse images plotted in Fig. 2, Five peaks and Triangle, and the dense image Waves with $\mathit{\beta }(s)=\left\{\cos \left(6\pi s_1/m\right)+\cos \left(6\pi s_2/m\right)\right\}/5$. We also compare sample size $n\in \{100,250\}$, spatial correlation ${\vartheta }_x\in \{3,6\}$, and error standard deviation $\sigma \in \{2,5\}$. For all combinations of these parameters considered we generate S = 100 datasets. We also simulate binary data with $\Pi \left(Y_i=1\right)=\Phi \left(X_i^{\text{T}}{\mathit{\beta }}^{\text{v}}\right)$ with n = 250 and exponential covariance with ${\vartheta }_x=3$.

Fig. 2.

True ${\mathit{\beta }}^v$ images used in the simulation study.

5.2. Methods

We fit our model with an m/2 × m/2 equally-spaced grid of knots covering {1,...,m} × {1,...,m} with bandwidth σ_h set to the minimum distance between knots. We fit our model both with λ > 0 where the prior is soft-thresholded Gaussian process with sparsity, and with λ = 0, in which case the prior is Gaussian process with no sparsity. For both models, we run the proposed Markov chain Monte Carlo algorithm 50,000 iterations with 10,000 burn-in, and compute the posterior mean of ${\mathit{\beta }}^{\text{v}}$. For the sparse model, we compute the posterior probability of a nonzero $\mathit{\beta }(s)$. We compare our method with the lasso (Tibshirani, 1996) and fused lasso (Tibshirani et al., 2005; Tibshirani & Taylor, 2011) penalized regression estimates

$$\begin{array}{l}{\widehat{\mathit{\beta }}}_{\text{L}}^{\text{v}}=\underset{{\mathit{\beta }}^{\text{v}}}{\text{argmin}}\left\{{\left(Y-X{\mathit{\beta }}^{\text{v}}\right)}^{\text{T}}\left(Y-X{\mathit{\beta }}^{\text{v}}\right)+\tilde{\lambda }\sum _j|\mathit{\beta }\left(s_j\right)|\right\},\\ {\widehat{\mathit{\beta }}}_{\text{FL}}^{\text{v}}=\underset{{\mathit{\beta }}^{\text{v}}}{\text{argmin}}\left\{{\left(Y-X{\mathit{\beta }}^{\text{v}}\right)}^{\text{T}}\left(Y-X{\mathit{\beta }}^{\text{v}}\right)+\tilde{\lambda }\sum _{j~k}|\mathit{\beta }\left(s_j\right)-\mathit{\beta }\left(s_k\right)|+\tilde{\gamma }\tilde{\lambda }\sum _j|\mathit{\beta }\left(s_j\right)|\right\}.\end{array}$$ (9)

The lasso estimate ${\hat{\mathit{\beta }}}_L^{\mathrm{v}}$ is computed using the lars package (Hastie & Efron, 2013) in R (R Core Team, 2017) and the tuning parameter $\tilde{\lambda }$ is selected using the Bayesian information criterion. The fused lasso estimate ${\widehat{\mathit{\beta }}}_{\text{FL}}^{\text{v}}$ is computed using the genlasso package (Arnold & Tibshirani, 2014) in R. The fussed lasso has two tuning parameters: $\tilde{\gamma }$ and $\tilde{\lambda }$. Due to computational considerations, we search only over $\tilde{\gamma }$ in {1/5,1, 5}. For each $\tilde{\gamma }$, $\tilde{\lambda }$ is selected using the Bayesian information criterion. The Supplementary Material gives results for each $\tilde{\gamma }$ and in the main text we report only the results for the $\tilde{\gamma }$ with the most precise estimates.

We also consider a functional principal component analysis-based alternative. We smooth each image using the technique of Xiao et al. (2013) implemented in the fbps function in R’s refund package (Crainiceanu et al., 2014), compute the eigen decomposition of the sample covariance of the smoothed images, and then perform principal components regression using the lasso penalty tuned via Bayesian information criterion. The Supplementary Material gives results using the leading eigenvectors that explain 80%, 90%, and 95% of the variation in the sample images and we report only the results for the value with most precise estimates in the main text.

Finally, we compare our approach with the Bayesian spatial model of Goldsmith et al. (2014) using $\mathit{\beta }\left(s_j\right)={\tilde{\alpha }}_j{\theta }_j$, where ${\tilde{\alpha }}_j\in \{0,1\}$ is the binary indicator that location j is included in the model, and θ_j € $\mathcal{R}$ is the regression coefficient given that the location is included. Both the ${\tilde{\alpha }}_j$ and ${\theta }_j$ have spatial priors; the continuous components θ_j follow a conditional autoregressive prior, and the binary components θ_j follow an Ising or autologistic prior (Gelfand et al., 2010) with full conditional distributions

$$\text{logit}\Pi \left({\tilde{\alpha }}_j=1|{\tilde{\alpha }}_l,l\ne j\right)=a_0+b_0\sum _{l~j}{\tilde{\alpha }}_l.$$ (10)

Estimating a₀ and b₀ is challenging because of the complexity of the Ising model (Møller et al., 2006), therefore Goldsmith et al. (2014) recommended selecting a₀ and b₀ using cross-validation over $a_0\in (-4,0)$ and $b_0\in (0,2)$. Due to computational limitations we select values in the middle of these intervals and set a₀ = −2 and b₀ = 1. The posterior mean of β(s) and the posterior probability of a nonzero β(s) are approximated based on 5,000 Markov chain Monte Carlo samples after the first 1,000 are discarded as burn-in.

5.3. Results

Tables 1 and 2 give the mean squared error for β^v estimation averaged over location, Type I error and power for detecting nonzero signals, along with the computing time. The soft-thresholded Gaussian process model gives the smallest mean squared error when the covariate has exponential correlation. Compared to the Gaussian process model, adding thresholding reduces mean squared error by roughly 50% in many cases. As expected, the functional principal component analysis method gives the smallest mean squared error in the shared-structure scenarios where the covariates are generated to have a similar spatial pattern as the true signal. Even in this case, the proposed method outperforms other methods which do not exploit this shared structure. Under the dense waves signal the non-sparse Gaussian process model gives the smallest mean squared error, but again the proposed method remains competitive.

Table 1.

Simulation study results for linear regression models. Methods are compared in terms of mean squared error for ${\mathit{\beta }}^V$, Type I error and power for feature detection. The scenarios vary by the true ${\mathit{\beta }}_0^v$ (Fig. 2), sample size n, similarity between covariates and true signal determined by τ, error variance σ, and spatial correlation range of the covariates ${\vartheta }_x$. Results are reported as the mean with the standard deviation over the 100 simulated datasets

					Mean squared error for βv, multiplied by 104
Signal	n	τ	σ	υx	Lasso	Fused lasso	FPCA	Ising	GP	STGP
Five peaks	100	0	5	3	326(5)	22(0)	36(0)	46(1)	27(0)	13(0)
	100	0	5	6	553(9)	22(0)	33(0)	44(1)	28(0)	13(0)
	100	0	2	3	102(1)	11(0)	24(0)	26(0)	15(0)	4(0)
	250	0	5	3	674(9)	13(0)	27(0)	52(1)	17(0)	6(0)
Triangle	100	0	5	3	289(4)	9(0)	17(0)	30(1)	19(0)	8(0)
	100	0	5	6	515(9)	8(0)	1.5(0)	28(1)	19(0)	8(0)
	100	0	2	3	73(1)	5(0)	12(0)	14(0)	10(0)	4(0)
	250	0	5	3	650(9)	6(0)	14(0)	34(0)	12(0)	6(0)
	100	2	5	3	1011(17)	7(0)	10(0)	27(1)	33(1)	13(1)
	100	4	5	3	1018(17)	6(0)	4(1)	32(1)	34(1)	14(1)
Waves	100	0	5	3	1260(13)	250(5)	188(7)	419(3)	48(1)	109(10)
	100	0	5	6	1639(17)	233(4)	126(3)	402(4)	51(1)	128(12)

					Type I error, %				Power, %
Signal	n	τ	σ	υx	Lasso	Fused lasso	Ising	STGP	Lasso	Fused lasso	Ising	STGP
Five peaks	100	0	5	3	10(0)	14(1)	0(0)	4(1)	17(0)	55(2)	4(0)	51(1)
	100	0	5	6	10(0)	37(1)	0(0)	7(1)	15(0)	80(1)	5(0)	58(1)
	100	0	2	3	79(0)	24(1)	0(0)	4(1)	28(0)	84(1)	4(0)	82(1)
	250	0	5	3	27(0)	19(1)	0(0)	4(1)	32(0)	77(1)	10(0)	71(1)
Triangle	100	0	5	3	10(0)	5(0)	0(0)	4(1)	23(1)	85(1)	9(0)	87(1)
	100	0	5	6	11(0)	8(1)	0(0)	4(1)	19(1)	91(1)	9(0)	86(1)
	100	0	2	3	9(0)	7(1)	0(0)	4(0)	42(1)	95(0)	5(0)	98(0)
	250	0	5	3	27(0)	5(0)	0(0)	4(0)	35(1)	92(1)	16(0)	96(1)
	100	2	5	3	11(0)	7(1)	0(0)	1(0)	17(0)	86(1)	8(0)	70(1)
	100	4	5	3	11(0)	2(1)	0(0)	3(1)	19(1)	84(1)	12(0)	73(1)

					Computing time, minutes
Signal	n	τ	σ	υx	Lasso	Fused lasso	FPCA	Ising	GP	STGP
Five peaks	100	0	5	3	0.02	16.77	5.40	27.61	4.81	11.28

Type I error: proportion of times estimating zero coefficients to be nonzero; Power: proportion of times estimating nonzero coefficients to be nonzero; FPCA, α%: functional principal component analysis approach (Xiao et al., 2013) using eigen vectors that explain α% of variation; Ising: Bayesian spatial variable selection with Ising priors (Goldsmith et al., 2014); GP: Gaussian process approach; STGP: soft-thresholded Gaussian process approach.

Table 2.

Simulation study results for binary regression models. Methods are compared in terms of mean squared error for ${\mathit{\beta }}^v$, Type I error and power for feature detection. The scenarios vary by the true ${\mathit{\beta }}_0^{\text{v}}$ see Fig. 2. Results are reported as the mean with the standard deviation over the 100 simulated datasets

	Mean squared error for βv, multiplied by 104
Signal	Lasso	FPCA, 80%	FPCA, 90%	FPCA, 95%	GP	STGP
Five peaks	11(2)	3(0)	6(2)	21(16)	1(0)	1(0)
Triangle	4(1)	1(0)	3(1)	8(4)	1(0)	0(0)
Waves	80(5)	18(4)	52(16)	38(8)	107(46)	191(138)

	Type I error, %		Power, %
Signal	Lasso	STGP	Lasso	STGP
Five peaks	3(0)	7(1)	15(0)	62(1)
Triangle	1(0)	4(0)	24(1)	91(1)

Type I error: proportion of times estimating zero coefficients to be nonzero; Power: proportion of times estimating nonzero coefficients to be nonzero; FPCA, α%: functional principal component analysis approach (Xiao et al., 2013) using eigen vectors that explain α% of variation; Ising: Bayesian spatial variable selection with Ising priors (Goldsmith et al., 2014); GP: Gaussian process approach; STGP: soft-thresholded Gaussian process approach.

For variable selection results, we only compare the proposed method with the fused lasso and the Ising model for a fair comparison, because the lasso does not incorporate spatial locations and other methods do not perform variable selection directly. The results show that the fused lasso has much larger type I errors in all cases and the Ising model has a very small power to detect the signal in each case. The proposed method is much more efficient than both the fused lasso and the Ising model for variable selection. For the computing time, the proposed method is comparable to the fused lasso and is faster than the Ising model.

6. ANALYSIS OF EEG DATA

Our motivating application is the study of the relationship between the electrical brain activity as measured through multi channel EEG signals and genetic predisposition to alcoholism. EEG is a medical imaging technique that records the electrical activity in the brain by measuring the current flows produced when the neurons are activated. The study comprises a total of 122 subjects: 77 alcoholic subjects and 45 non-alcoholic controls. For each subject, 64 electrodes were placed on their scalp and EEG was recorded from each electrode at a frequency of 256Hz. The electrode positions were located at standard sites, i.e., standard electrode position nomenclature according to American Electroencephalographic Society (Sharbrough et al., 1991). The subjects were presented with 120 trials under several settings involving one stimulus or two stimuli. We consider the multichannel average EEG across the 120 trials corresponding to a single stimulus. The dataset is publicly available at the University of California at Irvine Knowledge Discovery of Datasets. https://kdd.ics.uci.edu/databases/eeg/eeg.data.html.

These data have been previously analyzed by Li et al. (2010), Hung & Wang (2013) and Zhou & Li (2014). However, all the analyses ignore the spatial location of the electrodes on the scalp and used instead smooth based on their identification number, which ranges from 1 to 64 and is assigned arbitrarily relative to the electrodes’ position on the scalp. Our goal in this analysis is to detect the regions of brain which are most predictive of the alcoholism status; thus accounting for the actual position of the electrodes is a key component in our approach. In the absence of more sophisticated means to determine the electrodes’ position on the scalp, we consider a lattice design and assign a two-dimensional location to each electrode that matches closely the electrode’s standard position. Using the labels of the electrodes, we were able to identify only 60 of them. As a result our analysis will be based on the multichannel EEG from these 60 electrodes.

In accordance with the notation employed earlier, let Y_i be the alcoholism status indicator with Y_i = 1 if the ith subject is alcoholic and 0 otherwise. Furthermore, $\text{let}{X}_i=\{X_i\left(s_j;t\right):s_j\in {ℝ}^2,j=1,\dots ,60,t=1,\dots 256\}$ be the EEG image data for the ith subject which is indexed by a two-dimensional index accounting for the spatial location on the matching lattice design, s_j, and one-dimensional index for time, t.

We use a probit model to relate the alcoholism status and the multichannel EEG image: $Y_i|X_i,\mathit{\beta }~\text{Bernoulli}\left({\pi }_i\right)$) and ${\Phi }^{-1}\left({\pi }_i\right)={\sum }_{j=1}^{60}{\sum }_{k=1}^{256}{X}_i\left(s_j,t_k\right)\mathit{\beta }\left(s_j,t_k\right)$. The spatially-temporally varying coefficient function β quantifies the effect of the image on the response over time and is modeled using the soft-thresholded Gaussian process on spatial and temporal domain. We select a 5 × 5 square grid of spatial knots and 64 temporal knots, for a total of 1,600 three-dimensional knots. We initially fitted a conditional autoregressive model with a different dependence parameter $\vartheta$ for spatial and temporal neighbors (Reich et al., 2007), but found that the convergence was slow and that the estimates of both the spatial and temporal dependence were similar. Thus, we elected to use the same dependence parameter for all neighbors.

We evaluate the prediction performance of the proposed model using cross validation. We first calculate the posterior predictive probabilities that each test-set response is one and then apply the standard receiving operating characteristic curve plot which evaluates the classification accuracy over thresholds on the predictive probabilities. Figure 3 shows the receiving operating characteristic curve using leave-one-out cross validation. The results are compared with those of the lasso, functional principal component analysis and the soft-thresholding Gaussian process approach with thresholding parameter λ = 0. To facilitate computation for these methods, we thin the time points by two, leaving 128 time points. While no model is uniformly superior, the area under the curve corresponding to our approach is optimal among the alternatives we considered.

Fig. 3.

Receiving operating characteristic curves with the area under the curve, AUC, for the leave-one-out cross validation of the EEG data by six different methods: lasso (black solid, AUC = 0.789), functional principal component analysis using the leading eigen-vectors that explain 80% (red solid, AUC = 0.775), 90% (red dashes, AUC = 0.789), 95% (red dots, AUC = 0.777) of variations, Gaussian process approach (green solid, AUC = 0.770) and soft-thresholded Gaussian process approach (navy solid, AUC = 0.818)

The differences between the models are further examined in the estimated β functions plotted in Fig. 4, where we ignore the spatial location of the electrodes and plot them using their identification number. The lasso solution is nonzero for a single spatiotemporal location, while the functional principal component analysis and Gaussian process methods lead to non-sparse and thus uninterpretable β estimates. In contrast, the soft-thresholded Gaussian process based estimate is near zero for the vast majority of locations, and isolates a subset of the electrodes near time point 86 as the most powerful predictors of alcoholism.

Fig. 4.

Estimated spatial-temporal effects of the EEG image predictors by four different methods: Lasso, functional principal component analysis, Gaussian process and soft-thresholded Gaussian process. The Gaussian process and soft-thresholded Gaussian process estimates are posterior means.

Our analysis indicates that EEG measurements at time t = 86, which roughly corresponds to 0.336 fraction of second, are predictive of the alcoholism status. This observation is further confirmed by the plot of the posterior probability of nonzero $\mathit{\beta }\left(s_j,t\right)$‘s in Fig. 5a. This implies a delayed reaction to the stimulus, although this finding has to be confirmed with the investigators. To gain more insight into these findings, Figures 5b–5d focus on a particular time and display the posterior mean and posterior probability of a nonzero signal across the electrodes locations. They indicate that the right occipital/lateral region is the most predictive of the alcoholism status.

Fig. 5.

Summary of analysis of the EEG data by the soft-thresholded Gaussian process. Panel (a) plots the posterior probability of a nonzero β(s, t); each electrode is a line plotted over time t. The remaining panels map either the posterior probability of a nonzero β(s, t) or the posterior mean of β(s, t) at individual time points

7. DISCUSSION

The proposed method generates a few future directions that we consider to pursue. First, we plan to develop a more efficient posterior computation algorithm for analysis of voxel-level functional magnetic resonance imaging (fMRI) data, which typically contains 180,000 voxels for each subject. Any fast and scalable Gaussian processes approximation approach can be potentially applied to the soft-thresholded Gaussian process. For example, the recent ideas of nearest-neighbor Gaussian process approach by Datta et al. (2016) can be applied to our model. In addition, it is of great interest to perform joint analysis of datasets involving multiple imaging modalities, such as fMRi, diffusion tensor imaging and structural MRi. it is very challenging to model the dependence between the multiple imaging modality over space and to select the interactions between multiple modality image predictors in scalar-on-image regression. The extension of the soft-thresholded Gaussian process might solve this problem. The basic idea is to introduce hierarchical latent Gaussian processes and different types of thresholding parameters for different modalities, leading to an hierarchical soft-thresholded Gaussian process as the prior model for the effects of interactions.