A Semi-parametric Nonlinear Model for Event-Related fMRI

Abstract

Nonlinearity in evoked hemodynamic responses often presents in event-related fMRI studies. Volterra series, a higher-order extension of linear convolution, has been used in the literature to construct a nonlinear characterization of hemodynamic responses. Estimation of the Volterra kernel coefficients in these models is usually challenging due to the large number of parameters. We propose a new semi-parametric model based on Volterra series for the hemodynamic responses that greatly reduces the number of parameters and enables “information borrowing” among subjects. This model assumes that in the same brain region and under the same stimulus, the hemodynamic responses across subjects share a common but unknown functional shape that can differ in magnitude, latency and degree of interaction. We develop a computationally-efficient strategy based on splines to estimate the model parameters, and a hypothesis test on nonlinearity. The proposed method is compared with several existing methods via extensive simulations, and is applied to a real event-related fMRI study.

1. Introduction

The existence of nonlinearities in evoked responses in blood oxygen level-dependent (BOLD) fMRI, particularly in event-related designs, has been widely recognized in the literature (e.g., Vazquez and Noll, 1998; Buxton et al., 1998; Friston et al., 1998b, 2000; Miller et al., 2001; Soltysik et al., 2004, Wager et al., 2005). The extent of nonlinearity usually varies across brain regions and stimuli, and shorter intervals between stimuli lead to stronger nonlinearity than longer ones (Dale and Buckner 1997; Buckner 1998; Vazquez and Noll 1998; Liu and Gao 2000). These nonlinearities are believed to arise from nonlinearities both in the vascular response and at the neuronal level, and are commonly expressed as interactions among stimuli. Though the importance of adjusting for nonlinear interactions in estimating hemodynamic responses has been demonstrated (a compelling example is given in Wager et al. (2005)), reliable quantification of nonlinearity is challenging in practice. Two main types of nonlinear models for fMRI have been developed: the dynamical Ballon model (Buxton and Frank, 1997; Buxton et al., 1998; Mandeville et al., 1999) and the Volterra series based models (Friston et al., 1998b, 2000), the connection between which is established in Friston et al. (2000). These models are flexible in accommodating various interaction effects, but their implementation is often hampered by model complexity. For instance, the Volterra series models generally involve a large number of free parameters, which pose difficulty in obtaining stable estimates due to over-fitting and loss of power given limited available data. This motivates us to propose a parsimonious semi-parametric Volterra series model that enables efficient presentation and estimation of nonlinearities in this article.

The Volterra series model is an extension from the general linear model (GLM; Friston et al., 1995; Worsley and Friston, 1995), where the observed BOLD time series for each voxel is modeled as the linear convolution between the stimulus function and the unknown hemodynamic response function (HRF). The GLM assumes linear time invariant system, and thus is not applicable in the presence of significant deviation from expected linear system behavior. The Volterra series, a series of infinite sum of multidimensional convolutional integrals, is essentially a higher-order extension of linear convolutions. For simplicity, second-order Volterra series are most commonly used for characterizing pair-wise interactions between stimuli. Represented by two-dimensional spline bases in a fully nonparametric manner (Friston et al., 1998b), the second-order Volterra series is very flexible to accommodate a variety of nonlinear hemodynamic behaviors across different regions, stimuli and subjects. Moreover, under the spline representation, the extended GLM based on Volterra series is converted to a linear regression, the computation of which is straightforward. The ensuing parameter estimates, however, have large variances, especially when obtained from a single individual's data.

In Zhang et al. (2013), we proposed a semi-parametric HRF model within the GLM framework for multi-subject fMRI data. By assuming that for a fixed voxel and stimulus the HRFs share a common but unknown functional shape, and differ in magnitude and latency across subjects, this model allows for combining multi-subject data information for HRF estimation. Thus, the estimation efficiency can be significantly increased in contrast to analyzing each individual subject's data independently. We extend such “information borrowing” idea to the second-order Volterra series model. Specifically, in addition to using the semi-parametric HRF model, here we also assume that for a fixed voxel and a pair of stimuli, their associated second-order Volterra kernel has a common and unknown functional sphere, and differ in the extent of interaction across subjects. We develop a computationally-efficient strategy based on nonparametric spline expansions (Parker and Rice, 1985; Eubank, 1988; Wahba, 1990; De Boor, 2001; Ruppert et al., 2003) to estimate subject-specific and population-common characteristics. We also propose a hypothesis test on the sample average of second-order Volterra kernel estimates for assessing population interaction effect. Performance of the method is examined by both simulations and a real fMRI study.

Section 2 presents the new method: Section 2.1 introduces the semi-parametric model based on Volterra series; Section 2.2 describes a new spline-basis-based regularized estimation strategy for estimating the model parameters and discusses selection of functional basis and penalty parameter; and Section 2.3 develops a hypothesis test on nonlinearity. We then apply the proposed method to a real event-related fMRI study in Section 3.1 and compare the method with several existing methods via simulations in Section 3.2. Section 4 concludes.

2. Materials and Methods

2.1. Model

We adopt the standard massive univariate approach; since the same approach applies to each voxel, the subscript for voxel is omitted here. For subject i (i = 1, …, n), let y_i(t) for t = δ, …, T · δ be the observed fMRI time series of a given brain voxel, where δ is the experiment time unit when each fMRI scan is captured, usually ranging from 0.5 to 2 seconds. Also for subject i and stimulus k (k = 1, …, K), let υ_i,k(t) be the known stimulus function which equals 1 if the kth stimulus evoked at t(> 0) in the experimental design for subject i, and 0 otherwise. The Volterra series is an extension of the Taylor series representation of nonlinear system where the output of the nonlinear system depends on the past history of the input to the system. Friston et al. (1998b) proposed to use the second-order Volterra series to characterize nonlinearity in evoked hemodynamic responses as follows:

$$y_i(t)={\mathbf{d}}_i(t)\cdot {\mathit{\beta }}_i+\sum _{k=1}^K{\int }_0^m{h}_{i,k}(u)\cdot {\upsilon }_{i,k}(t-u)\mathit{\text{du}}+\sum _{k_1,k_2=1}^K{\int }_0^m{V}_{i,k_1{k}_2}(u_1,u_2)\cdot {\upsilon }_{i,k_1}(t-u_1)\cdot {\upsilon }_{i,k_2}(t-u_2){\mathit{\text{du}}}_1{\mathit{\text{du}}}_2+{\epsilon }_i(t),$$ (1)

where d_i(t) is a lower-order polynomial accounting for the low-frequency drift due to physiological noise or subject motion in the fMRI (Smith et al., 1999; Brosch et al., 2002; Luo and Puthusserypady, 2008); h_i,k(t) is the hemodynamic response function (HRF) corresponding to the kth stimulus for subject i; V_i,k1k2 (t₁, t₂) is the 2nd-order Volterra kernel function that models the interaction between the hemodynamic responses under stimuli k₁ and k₂ for subject i; m is a fixed constant defining the domain of the HRF; and ε_i(t) is the error term. Following a common practice in the literature, we adopt a 2nd-order polynomial for the drifting term d_i(t) = (1, t, t²) with parameters β_i = (β_i,0, β_i,1, β_i,2)′. Though it is possible to use higher order Volterra kernels, we focus on the second order for simplicity. The height, time to peak, and width of a HRF is commonly interpreted as magnitude, reaction time, and duration, respectively, of subjects' neuronal activity in response to stimuli. A typical HRF shape is shown in Figure 4(a), having onset at the stimulus-evoked time, reaching peak between 5 and 8 seconds, and declining afterward to the baseline (zero). Model (1) without the term of the 2nd-order Volterra kernel is the GLM (Friston et al., 1995). There is a vast literature on the estimation of the HRF h_i,k(t), including parametric methods (e.g., Worsley and Friston, 1995; Friston et al., 1998a; Glover, 1999; Henson et al., 2002; Riera et al., 2004; Lindquist and Wager, 2007; Lindquist et al., 2009) and nonparametric methods (e.g., Aguirre et al., 1998; Dale, 1999; Lange et al., 1999; Woolrich et al., 2004; Zarahn, 2002; Vakorin et al., 2007; Bai et al., 2009; Wang et al., 2011). Estimation of V_i,k1k2 (t₁, t₂) is more challenging than that of the HRF, because the Volterra kernel function, defined on the two-dimensional space, involves many more parameters, while the number of observations, T, for each subject is usually limited.

Figure 4. Simulated HRFs for Six Stimuli.

Model (1) can be viewed as a special case of linear functional models, with slope functions h_i,k and interaction functions V_i,k1k2. In the neuropsychological studies we consider, the underlying slope functions, the HRFs, vary across subjects in height, time to peak, and width. Therefore, the common practice of assuming identical parameter functions does not apply here. In fact, extracting subject-specific characteristics is often one of the main goals in multi-subject fMRI studies. To simultaneously model population-wide and subject-specific characteristics of brain activity, and to “borrow information” across subjects, we assume a semi-parametric form for both h and V:

$$h_{i,k}(t)=A_{i,k}\cdot f_k(t+D_{i,k}),$$ (2)

$$V_{i,k_1{k}_2}(t_1,t_2)=M_{i,k_1{k}_2}\cdot V_{k_1{k}_2}(t_1,t_2),$$ (3)

where A_i,k, D_i,k and M_i,k1k2 are unknown fixed parameters, representing magnitude and latency of brain's reaction to the kth stimulus, and intensity of the interaction between the k₁th and k₂th stimuli, respectively, for subject i; f_k(t) is the population average HRF corresponding to the kth stimulus, and V_k1k2 is the population average interaction function between the k₁th and k₂th stimuli. Model (3) assumes that the interaction pattern between hemodynamic responses of a given pair of stimuli is identical, but differs in intensity across subjects. No parametric assumption except for differentiability is imposed on f_k and V_k1k2. By assuming all the subjects have a common functional form of the HRFs and their interactions, Models (2) and (3) greatly reduce the number of parameters and also enable efficient information sharing across subjects. Note that Model (3) does not account for interaction effects on the onset and time to peak of hemodynamic responses, which are generally too complicated to be quantified for a two-dimensional function, whereas subject-specific interaction intensity is much easier to interpret. Model (2) was previously proposed in Zhang et al. (2013) in the context of GLM. When direct observations of h_i,k(t) are available, Model (2) is referred to as “shift and magnitude registration” by Ramsay and Silverman (2005). A similar shape-invariant model for longitudinal data analysis has been also discussed in Lindstrom (1995). In GLM, however, one need to address the additional challenge of deconvoluting h_i,k(t) from the observed time series.

2.2. Spline-based Estimation

We now develop a spline-basis-based regularized strategy to estimate the parameters in the proposed model. Assuming that the latency D_i,k is smaller than the experimental time unit, we use a first-order Taylor expansion to approximate Model (2), converting h_i,k(t) to a linear presentation in terms of subject-specific parameters A_i,k and D_i,k:

$$h_{i,k}(t)\approx A_{i,k}\cdot f_k(t)+C_{i,k}\cdot f_k^{(1)}(t),$$ (4)

where C_i,k = A_i,k · D_i,k. Then we represent f_k(t) by cubic B-spline bases: $f_k(t)={\sum }_{l=1}^L{a}_{kl}\cdot b_l(t)$, where the basis functions b_l(t) are chosen based on a partition Λ_q = (t₀ = 0, t₁, …, t_q = m) of the interval [0, m]. Selection of the knots Λ_q is discussed later. Given the boundary condition that h_i,k(0) = h_i,k(m) = 0, we let a_1k = a_Lk = 0.

Similarly, we represent the bivariate function V_k1k2 (t₁, t₂) by cubic spline bases:

$$V_{k_1{k}_2}(t_1,t_2)=\sum _{l_1{l}_2=1}^L{Z}_{k_1{k}_2{l}_1{l}_2}\cdot b_{l_1}(t_1)\cdot b_{l_2}(t_2).$$

It is known that nonlinearity disappears if events are spaced at least 5 seconds apart (Miezin et al., 2000), implying that V_k1k2 (t₁, t₂) = 0 for |t₁ − t₂| ≥ 5. Using this fact and cubic spline bases with equally-spaced knots, the number of free parameters can be reduced by letting Z_k1k2l1l2 = 0 for |l₁ − l₂| ≥ 4 + 5/m · (L − 2). This fact also indicates that some V_k1k2's, whose associated pairs of stimuli are always more than 5s apart in the experiment, equal zeros in the model. Moreover, in many event-related experiments, pairs of stimuli are separated at certain values, implying that some values of V_k1k2 (t₁, t₂) are not observable. In this case, because the spline bases b_l(t)'s only cover a short period of the domain [0, m], some coefficients Z_k1k2l1l2 are not observable and should not be included in the model, which can further reduce the number of free parameters.

Letting ℒ² = {(l₁, l₂) : 1 ≤ l₁, l₂ ≤ L; |l₁ − l₂| ≥ 4 + 5/m · (L − 2)} and = {(k₁, k₂) : there exists at least one (u₁, u₂) ∈ (0, m)² such that υ_i,k1 (t − u₁) = υ_i,k2 (t − u₂) = 1 for at least one subject i and |u₁ − u₂| < 5}. The nonlinear functional model (1) is transformed to the following bilinear model:

$$y_i(t)={\mathbf{d}}_i(t)\cdot {\mathit{\beta }}_i+\sum _{k=1}^K\sum _{l=2}^{L-1}{\omega }_{i,kl}\cdot {\rho }_{i,kl}(t)+\sum _{k=1}^K\sum _{l=2}^{L-1}{\varphi }_{i,kl}\cdot {\mathit{\varrho }}_{i,kl}(t)+\sum _{(k_1,k_2)\in {\mathcal{K}}^2}\sum _{(l_1,l_2)\in {\mathcal{L}}^2}{\nu }_{i,k_1{k}_2{l}_1{l}_2}\cdot {\psi }_{k_1{k}_2{l}_1{l}_2}(t)+{\epsilon }_i(t),$$ (5)

where ω_i,kl = A_i,k · a_kl, ϕ_i,kl = C_i,k · a_kl, ν_i,k1k2l1l2 = M_i,k1k2 · Z_k1k2l1l2, ${\rho }_{i,kl}(t)={\int }_0^m{b}_l(u)\cdot {\upsilon }_{i,k}(t-u)du$, ${\mathit{\varrho }}_{i,kl}(t)={\int }_0^m{b}_l(u)\cdot {\upsilon }_{i,k}(t-u)du$, and ${\psi }_{k_1{k}_2{l}_1{k}_2}(t)={\int }_0^m{\int }_0^m{b}_{l_1}(u_1)\cdot b_{l_2}(u_2)\cdot {\upsilon }_{i,k_1}(t-u_1)\cdot {\upsilon }_{i,k_2}(t-u_2){du}_1{du}_2$ are known functions. Here subject-specific parameters A_i,k, C_i,k, a_kl, M_i,k1k2, Z_k1k2l1l2 are not directly identifiable, but their products ω_i,kl, ϕ_i,kl and ν_i,k1k2l1l2 are unique. Therefore, the estimates of subject-specific HRFs and second-order Volterra kernels are still unique. Notations of the key parameters are listed in Table 1.

Table 1. Notations of key parameters.

parameter	description
di(t)	a vector of known time-varying covariates
βi	coefficients of di(t)
Ai,k	subject-specific magnitude of the kth HRF
Di,k	subject-specific latency of the kth HRF
Mi,k1k2	subject-specific degree of interaction between stimuli k1 and k2
akl	coefficients of the spline bases representing the kth common function fk(t)
Zk1k2l1l2	coefficients of the spline bases representing the 2nd-order Volterra Kernel Vk1k2 (t1, t2)
Ci,k	product Ai,k · Di,k
ωi,kl	product Ai,k · akl
ϕi,kl	product Ci,k · akl
νi,k1k2l1l2	product Mi,k1k2 · Zk1k2l1l2
ρi, kl (t)	known functions ${\int }_0^m{b}_l(u)\cdot {\upsilon }_{i,k}(t-u)du$
ϱi,kl(t)	known functions ${\int }_0^m{b}_l^{(1)}(u)\cdot {\upsilon }_{i,k}(t-u)du$
ψi,k1k2l1l2 (t)	known functions ${\int }_0^m{\int }_0^m{b}_{l_1}(u_1)\cdot b_{l_2}(u_2)\cdot {\upsilon }_{i,k_1}(t-u_1)\cdot {\upsilon }_{i,k_2}(t-u_2){du}_1{du}_2$

A standard approach to estimating parameters in a bilinear model is through minimizing the mean squared error (MSE) of fMRI time series via the alternating least squares (ALS) algorithm, an iterative optimizing procedure. Iterative procedures often lead to slow convergence and volatile estimates, particularly in the cases with a large number of parameters and low signal-to-noise ratio. Therefore, below we propose a new non-iterative estimation strategy based on regularization:

• Step 1. If the latency D_i,k is close to zero, parameters ϕ_i,kl's should be much smaller than ω_i,kl's and have little effect on estimating h_i,k. Given this, we first omit the term ϕ_i,kl · ϱ_i,kl(t) involving the first-order derivative of f_k in Model (5) and obtain parameter estimates β̂_i, ω̂_i,kl and ν̂_i,k1k2l1l2 for each subject i, by minimizing the penalized MSE (PMSE) of y_i(t),

  $${\text{PMSE}}_i=\sum _{t=\delta }^{T\cdot \delta }{\left[y_i(t)-{\mathbf{d}}_i(t)\cdot {\mathit{\beta }}_i-\sum _{k=1}^K\sum _{l=2}^{L-1}{\omega }_{i,kl}\cdot {\rho }_{i,kl}(t)-\sum _{k_1,k_2}\sum _{l_1,l_2}{\nu }_{i,k_1{k}_2{l}_1{l}_2}\cdot {\psi }_{i,k_1{k}_2{l}_1{l}_2}(t)\right]}^2+\lambda \left[\sum _k\int {\left(\sum _l{\omega }_{i,kl}\cdot b_l^{(2)}(u)\right)}^{2}du+\sum _{k_1,k_2}\int \int {\left(\sum _{l_1,l_2}{\nu }_{i,k_1{k}_2{l}_1{l}_2}\cdot b_{l_1}^{(1)}(u_1)\cdot b_{l_2}^{(1)}(u_2)\right)}^2{du}_1{du}_2\right].$$ (6)

• Step 2. Estimate f_k(t) and V_k1k2 (t₁, t₂) respectively by ${\widehat{f}}_k(t)={\sum }_{i=1}^n{\widehat{h}}_{i,k}(t)/n$ and ${\widehat{V}}_{k_1{k}_2}(t_1,t_2)={\sum }_{i=1}^n{\widehat{V}}_{i,k_1{k}_2}(t_1,t_2)/n$, where ${\widehat{h}}_{i,k}(t)={\sum }_{l=2}^{L-1}{\widehat{\omega }}_{i,kl}\cdot b_l(t)$ and ${\widehat{V}}_{i,k_1{k}_2}(t_1,t_2)={\sum }_{l_1{l}_2}{\widehat{\nu }}_{i,k_1{k}_2{l}_1{l}_2}\cdot b_{l_1}(t_1)\cdot b_{l_2}(t_2)$.

• Step 3. Given ${\widehat{a}}_{kl}={\sum }_{i=1}^n{\widehat{\omega }}_{i,kl}/n$ and ${\widehat{Z}}_{k_1{k}_2{l}_1{l}_2}={\sum }_i{\widehat{\nu }}_{i,k_1{k}_2{l}_1{l}_2}/n$ from Step 2, re-evaluate A_i,k, C_i,k and M_i,k1k2 through ordinary least square regression (OLS) of Model (5).

Step 1 is equivalent to estimating each subject's HRFs and the 2nd-order Volterra kernel in a fully nonparametric manner under spline-basis representations: $h_{i,k}(t)={\sum }_{l=2}^{L-1}{\omega }_{i,kl}\cdot b_l(t)$, and V_i,k1k2 (t₁,t₂) = Σ _l1,l2 ν_i,k1k2l1l2 ⋅ b_l1 (t₁) ⋅ b_l2 (t₂). The penalty in PMSE_i is used to regularize the roughness of the nonparametric estimates. The analytic minimizer of PMSE_i is essentially a Tikhonov-regularized regression estimator, because the MSE, the first term in (6), is quadratic of the parameters (β_i, ω_i,kl, ν_i,k1k2l1l2) and the penalty is quadratic of the parameters ω_i,kl and ν_i,k1k2l1l2. We believe the average of subjects' nonparametric HRF estimates can approximate the population mean HRF shape well in Step 2 for two reasons. First, the point-wise average of subjects' HRFs is close to the population mean HRF shape, if the underlying HRFs indeed follow the proposed semi-parametric model; second, empirically we found that though individual subject's nonparametric estimates may vary significantly in shape due to large data noise, the shape of their average is generally stable.

In the literature knot or basis selection is typically performed with direct observations of a single target function (Zhou and Shen, 2001), whereas in our study we need to estimate multiple h_i,k's and V_i,k1k2's simultaneously without any direct observations. For simplicity, we use equally-spaced knots for both h_i,k and V_i,k1k2, and select a set of bases from two choices—with knots separated by 1 and 1/2, respectively—by a ten-fold cross-validation (TFCV) procedure. Distinct from the standard approach, the TFCV here is carried out by dividing all subjects' fMRI data into ten time periods of equal length instead of ten sub-samples. Specifically, each time data in one period are removed, the model constructed based on the of rest data is used to predict the left-out data, and the overall prediction error summed up over ten periods is used as the criterion for knot selection.

As it for penalty parameter selection, available methods include ordinary cross-validation (OCV), generalized cross-validation (GCV; Wahba, 1990), GCV for functional data analysis by Reiss and Ogden (2007, 2009), restricted maximum likelihood (Wood, 2011), among many others. In our case, since penalty parameter selection confounds knot selection, the two are performed together by the modified TFCV above.

2.3. Hypothesis Testing on Nonlinearity

To detect deviation from the linear time-invariant system, we propose an easy-to-implement test on estimated V̂_k1k2 based on Hotelling's T-squared distribution. Under normality assumption of the error term or with long enough observation time T in Model (1), the estimates ν̂_i,k1k2 = (ν̂_i,k1k2l1l2, (l₁, l₂) ∈ ℒ²)' from Step 1 for each subject i approximately follows a normal distribution N(ν_ik1k2, Δ_i), where the variance covariance matrix Δ_i depends on convolutions ρ_i,kl(t), ϱ_i,kl(t) and ψ_i,k1k2l1l2 (t), and ${\sigma }_i^2=var({\epsilon }_i(t))$. Assuming that across population ν_i,k1k2 ∼ N(μ_k1k2, Λ), where μ_k1k2 denotes the parameters for the population mean interaction function V_k1k2, then the population-wise ν̂_i,k1k2 ∼ N(μ_k1k2, ϒ), where ϒ is the variance and covariance matrix of ν̂_i,k1k2 across population. Then the test of nonlinearity is reduced to test whether μ_k1k2 = 0.

To test the mean of independent and identically distributed multivariate (p-dimensional) Gaussian random variables, ${\mathbf{x}}_i\stackrel{i.i.d.}{\sim }\mathrm{N}(\mu ,\sum )$, it is standard to use the Hotelling's T-squared statistic, define by

$$(\overline{\mathbf{x}}-\mu )\prime {\mathbf{W}}^{-1}(\overline{\mathbf{x}}-\mu )\frac{n(n-p)}{(n-1)p},\text{with}\mathbf{W}=\sum _{i=1}^n({\mathbf{x}}_i-\overline{\mathbf{x}})\prime ({\mathbf{x}}_i-\overline{\mathbf{x}})/(n-1),$$

which follows an F distribution with degrees of freedom p and n − p. Based on this, we propose to test H₀ : μ_k1k2 = 0 vs. H_A : μ_k1k2 ≠ 0 by using the statistic

$${\mathcal{T}}^2=(\sum _{i=1}^n{\widehat{\mathit{\nu }}}_{i,k_1{k}_2}/n)\prime {\widehat{\Upsilon }}^{-1}(\sum _{i=1}^n{\widehat{\mathit{\nu }}}_{i,k_1{k}_2}/n),$$

where ϒ̂ is the sample variance-covariance matrix of ν̂_i,k1k2. We reject the null hypothesis if ${\mathcal{T}}^2>F_{p,n-p}^{1-\alpha }$, where $F_{p,n-p}^{1-\alpha }$ is the 1 − α percentile of an F distribution with degrees of freedom p and n − p. In practice, with many functional bases used to represent the kernel function V_k1k2, however, p can be even larger than n, or comparable to n, leading to close to singular ϒ̂ and thus low detection power. To address this issue, we use only a subset of (l₁, l₂) in ν̂_i,k1k2l1l2 to significantly reduce p. Specifically, we perform a test on equally spaced elements of ν̂_i,k1k2l1l2, given that V_k1k2 is smooth and ν_i,k1k2l1l2's corresponding to spatially-close regions usually have similar values. Simulations in Section 3.2 show that such a test has a high power with type I error preserved at the specified significance level.

3. Results

3.1. Real Data Example

Data

We analyze the fMRI data collected from the Monetary Incentive Delay (MID) Experiment, which measures subjects' brain activity related to reward and penalty processing (Knutson et al., 2000). In this experiment, 19 subjects (10 male, 9 female) between 22 and 25 years of age were recruited from a larger representative longitudinal community sample (Allen et al., 2007).

In the MID task, each participant completed a protocol comprised of 72 6-second trials involving either no monetary outcome (control/neutral task), a potential reward (reward task), or a potential penalty (penalty task). The fMRI scans were acquired at every 2s (TR), leading to T = 219 frames of data for each subject. In each trial, participants were first shown a cue shape for 500 ms (anticipation condition), then waited a variable interval of between 2500 and 3500 ms, and were shown a white target square lasting between 160 and 260 ms (response condition). The cue shape (circle, square or triangle) shown at the start of each trial signals the type of the trial (reward, penalty or no incentive) to be implemented, and the white target shown at the end of each trial indicates button press from the participants, who were also told that their reaction times would affect the amount of money they receive in the monetary reward trial or lose in the penalty trial. In total, there were six stimuli involved in the experiment: three signal stimuli for the three types of monetary outcomes and three response stimuli to which the participants were required to respond. The six stimuli are henceforth referred to as neutral signal, reward signal, penalty signal, neutral response, reward response, and penalty response. The order of trials in the protocol for each participant was randomized, with 25% of them control trials, 37.5% reward trials, and 37.5% punishment trials. During the experiment, we used a Siemens 3.0 Tesla MAGNETOM Trio high-speed magnetic imaging device at UVA's Fontaine Research Park to acquire fMRI data, with a CP transmit/receive head coil with integrated mirror. Two hundred twenty-four functional T2*-weighted Echo Planar images (EPIs) sensitive to BOLD contrast were collected per block, in volumes of 28 3.5-mm transversal echo-planar slices (1-mm slice gap) covering the whole brain (1-mm slice gap, TR=2000ms, TE=40ms, flip angle=90°, FOV= 192 mm, matrix= 64×64, voxel size= 3×3×3.5mm). More details of the experimental design, fMRI data acquisition and preprocessing can be found in Zhang et al. (2012).

Statistical Analysis and Discussion

We apply the proposed methods to four regions of interest (ROI): right putamen (2144 voxels), right amygdala (1587 voxels), right pallidum (1246 voxels), and right caudate (2504 voxels). These were determined structurally using the Harvard subcortical brain atlas, and were chosen for their likely involvement in affective neural processing based on previous studies (e.g., Knutson et al., 2000). For each voxel, we include in Model (1) six HRFs corresponding to the six stimuli. For each of the three tasks (neutral, reward and penalty), we use a 2nd-order Volterra kernel to characterize the interaction between the corresponding signal and response stimuli. Using the proposed non-iterative estimation strategy, we evaluate the HRFs and their interactions. Statistical significance of the nonlinear term is tested using the Hotelling's T-squared test in Section 2.3.

Figure 1 displays the heatmaps of P-values (P-values above 0.2 are not shown) of ROI voxels in testing interactions between signal and response stimuli. No significant interaction pattern is identified in right caudate and right amygdala, and thus the related results are omitted. There is almost no interaction between neutral signal and response stimuli across all the ROIs, which is intuitive, because neutral signal stimulus indicates that the following response is not required and does not affect any final gain. The most significant interaction is between monetary penalty signal and response stimuli, especially in right putamen and right pallidum. Table 2 summarizes the percentages of voxels identified to be significant in the test of interaction between reward and penalty stimuli in these two regions at different significance threshholds. We used the empirical Bayes approach by Efron (2008) to evaluate the false discovery rates of the multiple hypothesis testing. An alternative approach is to use Benjamini-Hochberg (BH) threshold (Benjamini and Hochberg, 1995) to control for the false discovery rate (FDR) at different rates. Since the signal and response stimuli are not closely presented with inter-stimulus-interval (ISI) ranging from 2.5s to 3.5s, the interaction effect is not as intense as those with ISIs no more than 1s. In addition, the power of detecting nonliearity is further diminished by the small sample size and large noise of fMRI data, and thus there are moderate FDRs in the multiple hypothesis tests of voxels. Nevertheless, a large proportion of voxels were still detected with significant interactions in the penalty task. In contrast, there is little interaction detected under the reward task. The reasons that interactions between negative signal and response stimuli are most prominent, and they are mainly in right putamen and right pallidum are two-fold. First, the putamen and pallidum are both regions of the basal ganglia, a subcortical network that is involved in, among other things, voluntary control of motor movements. Activation of these areas during signal presentation suggests preparatory motor activity in anticipation of the response cue. Second, such activation is more prominent in the penalty task is not surprising, given the large body of work in psychology indicating that individuals react more strongly to negative stimuli than to positive stimuli (e.g., Baumeister et al., 2001). For example, brains are generally more active under negative stimuli (Cacioppo et al., 1997) and negative interactions more strongly define our attitudes about relationships (e.g., Gottman, 1994; Hudson and Vangelisti, 1991).

Figure 1. Heat maps of P-values of voxels in ROIs.

P-values of nonlinear tests of interactions respectively between neutral, monetary reward, and monetary penalty signal and response stimuli in right putamen and pallidum. The P-values are presented in -log10 scale.

Table 2.

The percentages and associated false discovery rates (FDR, in parenthesis) of voxels identified in the ROIs by the test on nonlinearity at different significance levels.

Significance Level (FDR) (%)	Right Putamen		Right Pallidum
	Reward	Penalty	Reward	Penalty
5% FDR	7.4 (67.7)	20.7 (24.1)	5.6 (89.0)	13.8 (36.2)
10% FDR	18.1 (55.3)	33.4 (30.0)	12.4 (80.4)	23.5 (42.5)

To inspect the interaction effects, for each voxel with P-value smaller than 5% in right putamen and pallidum, we calculate the averaged 2nd-order Volterra kernel estimates across time and subjects, Σ_i Σ_t1 Σ_t2 V̂_i,k1k2 (t₁, t₂)/(n · m²), histograms of which are presented in Figures 2(a) and 2(c). To give a more explicit view of the detected nonlinearity, Figures 2(b) and 2(d) respectively show the estimated population mean V̂_k1k2 (t₁, t₂) of the voxel with the most significantly nonlinear behavior in the two regions. The color scale is arbitrary; light yellow is positive, and dark red is negative. Since intervals between consecutive stimuli in this experimental design are between 2 and 4s, nonzero values of V_k1k2 (t₁, t₂) only appear in the off-diagonal band where |t₁ − t₂| is between 2 and 4s, and the values at other points are not observable. The interactive effect of penalty tasks, especially in right putamen, tends to be negative. One possible explanation is that the signal stimulus prepares the subjects for the response, leading to less intensive reactivity when response stimulus is presented. Such a negative interaction effect was also reported in Friston et al. (1998b). In terms of data analysis, the magnitude of the HRF would be underestimated if significant nonlinearity in the underlying hemodynamic responses exists but is not taken into account in the estimation.

Figure 2.

Histograms of Σ_i Σ_t1 Σ_t2 V̂_i,k1k2 (t₁, t₂)/(n · m²) for modeling interactions between penalty signal and response stimuli of all voxels with P-value smaller than 5% in right putamen (a) and right pallidum (c). Estimated population average interaction function Σ_i V̂_i,k1k2 (t₁, t₂)/n between penalty signal and response stimuli of the voxel in right putamen (b) and right pallidum (d) with the smallest P-value.

Figure 3 displays the estimated population mean HRF f_k (dark line) and individual HRF h_i,k (broken lines) of several randomly selected subjects for the voxel in the right putamen that has the most significant interaction of the penalty task. The effect of “borrowing” information across subjects can be clearly seen here as f̂_k is much less variant than the ĥ_i,k's, though they share a similar shape, for each of the six stimuli. In general, the response stimuli evoked stronger and stabler activity across subjects than the signal stimuli, since subjects' response affected the ensuing monetary gain or losses. The mental activity caused by the signal stimulus has a large variation across subjects. Such findings are in keeping with previous work indicating that passive viewing or “resting” generally produces noisier data than those that require a response from subjects. One model suggests that this “noise” may be a product of interactions between individual differences in cognitive and affective styles with uncontrolled portions of the experiment (Coan et al., 2006). So while the response cue elicits the same motor response from everyone (and thus a more coherent neural response), passive cue viewing may elicit similar, but relatively less coherent mental actions.

Figure 3. Estimated HRFs of a voxel in right putamen with significant interactions in monetary penalty task.

The black lines are the estimated f_k while the three broken lines are the estimated h_i,k for three randomly selected subjects.

3.2. Simulations

Simulation Design

We conduct simulations to further examine the properties of the proposed semi-parametric model in HRF estimation and also to compare with four existing methods: the linear semi-parametric model for HRF without the 2nd-order Volterra kernels proposed by Zhang et al. (2013), referred to as the linear spline-based method; a parametric approach representing HRF by a linear combination of canonical HRF and its first derivative, called canonical method hereafter; nonparametric Tikhonov-regularized estimate with penalty parameter selected by generalized cross validation (Tik-GCV, Casanova et al., 2008); and nonparametric smooth finite impulse response (SFIR) method (Goutte et al., 2000).

We generate time series data using the experimental design identical to that in the MID experiment with six stimuli for n = 19 subjects and three interaction effects. The HRFs h_i,k(t) follow Model (2) with the population mean HRF f_k being a mixture of two gamma functions that have the same mathematical expression as the canonical HRF (Worsley and Friston, 1995):

$$f_k(t)=b_{1,k}^{a_{1,k}}\frac{t^{a_{1,k}-1}\cdot e^{-b_{1,k}t}}{\mathrm{\Gamma }(a_{1,k})}-c_k\cdot b_{2,k}^{a_{2,k}}\frac{t^{a_{2,k}-1}\cdot e^{-b_{2,k}t}}{\mathrm{\Gamma }(a_{2,k})},k=1,\dots ,6.$$ (7)

By assigning different values to the parameters, the six f_k's have distinct shapes. The parameters for the six HRFs are given in Table 3, and several simulated HRFs for each stimulus are displayed in Figure 4. The first two HRFs follow a canonical shape, but differ in the range of variation in latency. The third and fourth HRFs have distinct shapes from the canonical one, but still follow the proposed semi-parametric model. The last two HRFs violate the model assumption, having a large variation both in latency and magnitude. To mimic the MID experiment, we consider three types of nonlinearity, respectively characterized by three second-order Volterra kernels:

Table 3.

Parameters of the simulated HRFs h_i,k, where U(a, b) denotes uniform distribution defined on interval (a, b), and N(μ, σ²) denotes normal distribution with mean μ and variance σ².

HRF k	Aik	Dik	a1,i	a2,i	b1,i	b2,i	c
1	N(700, 3002)	U(-1.5,1.0)	6	16	1	1	1/6
2	N(500, 2002)	U(-1.0,1.0)	6	16	1	1	1/6
3	N(400, 1502)	U(0.0,4.0)	19	20	2	2	2/3
4	U(500, 1500)	U(1.0,4.0)	20	22	2	2	9/10
5	U(100, 500)	U(-3.0,0)	U(6,8)	U(15,18)	U(1,3)	U(1,3)	1/6
6	U(100, 500)	U(-2.0,1.0)	U(18,22)	U(9,25)	U(3,4)	U(3,4)	1/4

$$V_{1,4}(t_1,t_2)=8exp\{-\left|t_1/1500+t_2/1000\right|\},V_{2,5}=2exp\{-\left|t_1/1500-t_2/1000\right|\},V_{3,6}=0,$$

for |t₁ − t₂| ≤ 3.5 and t₁ ≤ 8, and the kernels equal zero at the rest of (t₁, t₂). These kernels are chosen such that their values are close to zero at the boundary of domain |t₁ − t₂| ≤ 3.5, beyond which very few observations are available. The associated subjects' intensities of interaction, M_i,14 and M_i,25 are generated from uniform distributions with range (−200, −100) and (−150, −100), respectively, to represent negative interactions observed in many practical cases.

The error terms ε_i = (ε_i(1), …, ε_i(T))′ are simulated from an autoregressive model of order 4 (AR(4)) with lag-1 correlation of 0.45 and lag-2 correlation of 0.35:

$${\epsilon }_i(t)=0.37{\epsilon }_i(t-1)+0.14{\epsilon }_i(t-2)+0.05{\epsilon }_i(t-3)+0.02{\epsilon }_i(t-4)+e_i(t),$$

where $e_i(t)\stackrel{i.i.d.}{\sim }\mathrm{N}(0,{\sigma }_i^2)$. To reflect the heteroscedastic variances across subjects, we let ${\sigma }_i^2$ vary across subjects, following Ga(2, 1/25) + 50 so that generated data have a weak signal-to-noise ratio. For each simulated example below, we first generate h_i,k, V_i,k1k2 for i = 1, …,n, k = 1, 2, …, 6 and (k₁, k₂) ∈ {(1, 3), (2, 4), (3, 6)}, and random second order polynomials d_i(t)β_i with β_i,0 ∼ U(−1, 1), β_i,1 ∼ U(−0.1, 0.1), β_i,2 ∼ U(−0.05, 0.05) for each i. Then based on these, y_i(t) is simulated given the design and the stimulus functions.

We use the root mean square error (RMSE) of subjects' HRF estimates and average relative errors (ARE) of the height (HR) of the estimated HRFs as the criterion for comparison:

$$e({\text{HR}}_k)=\frac{1}{n}\sum _{i=1}^n\frac{|{\text{HR}}_{i,k}-{\widehat{\text{HR}}}_{i,k}|}{{\text{HR}}_{i,k}},{\text{RMSE}}_k=\frac{1}{n}\sum _{i=1}^n\frac{\Vert h_{i,k}-{\widehat{h}}_{i,k}\Vert }{\Vert h_{i,k}\Vert },$$

where ‖ · ‖ is the L² norm.

Analysis and Results

We evaluated the type I and type II errors of the proposed hypothesis tests on nonlinearity, and showed the histograms of P-values in testing values of V_1,4, V_2,5, and V_3,6 in Figure 1. For zero interaction in the case of V_3,6, the histogram of P-values is close to be flat, indicating that the type I error of the test is preserved at the specified level. As shown in Figures 5(a) and 5(b), the test on V_1,4 has a power close to one with all the P-values strictly less than 1%. The test on V_2,5 though has a smaller power due to its smaller value, still detects nonlinearity 36 times out of 100 simulations with threshhold at 5%.

Figure 5. Histograms of P-values for testing nonzero V_1,4, V_2,5, and zero V_3,6 respectively.

Table 4 summarizes the ARE of HR and RMSE of the six HRFs obtained from the five methods, where the cubic-spline-based methods use knots equally separated by 2 unit time. Among these methods, the proposed nonlinear model generally performs the best with reasonably small errors both in estimating functional shape and HR, the linear spline model is the second best, followed by the SFIR and Tik-GCV, while the canonical method performs the worst, even when the underlying HRFs follow the canonical form (HRFs 1-2). This is not surprising given the proposed nonlinear model is the only method that accounts for the interactions between stimuli. However, in terms of estimating a single value HR, the nonlinear and linear models have comparable performance, though the former recovers the entire curve with a much smaller error. This is probably because with the large variation of the fMRI data, the variation of the maximum value of the HRF estimates under the linear and nonlinear models is comparable, though the locations of maximum can vary significantly. The underperformance of the canonical method, especially for HRFs 3-6, is likely due to the huge overall model fitting error coming from the misspecified functional shapes of the HRFs.

Table 4.

Mean AREs for estimating HR and RMSE of the simulated HRFs from the simulated example by different methods, where the spline-based methods use knots equally spaced by 1.

	HRF	Spline-Based Strategies		Can.	Tik-GCV	SFIR
	k	Nonlinear	Linear
RMSE	1	2.16	10.17	8.34	7.34	3.56
	2	1.87	4.18	9.38	4.50	2.93
	3	4.50	4.76	13.48	11.76	8.12
	4	2.16	3.67	12.16	7.08	4.18
	5	1.69	2.89	8.23	4.08	2.18
	6	1.84	2.64	10.56	5.17	2.37
e(HR)	1	3.41	3.62	29.87	5.09	29.87
	2	2.84	1.98	10.58	3.30	10.58
	3	6.29	10.02	6.33	32.45	6.33
	4	0.92	0.80	10.72	1.77	10.72
	5	0.63	0.80	7.07	1.09	7.07
	6	0.68	0.78	7.37	1.26	7.37

4. Discussion

We proposed a semi-parametric nonlinear characterization of hemodynamic responses for multi-subject fMRI data based on Volterra series. The new model is flexible to accommodate variation of brain activity across different stimuli and voxels, and allows “borrowing” information across subjects to increase estimation efficiency. Using first-order Taylor expansion and spline basis representation, the nonlinear model is converted to a bilinear one, for which we developed a fast non-iterative estimation strategy. Applying the proposed method to the event-related MID study, we identified deviation from the commonly assumed linear time-invariant system in various brain regions due to interactions between stimuli. Through Monte Carlo simulation, we also showed that the proposed method outperforms several existing methods for HRF estimation when the nonlinear effect is significant.

It is natural to extend the information-borrowing idea to spatial context, that is, information can be borrowed from neighboring voxels. In fact, spatial information has been taken into account in the pre-processing stage of fMRI data analysis, which usually involves spatial smoothing. Consequently, the fMRI time series at spatially-close voxels usually have similar values and the resulting parameter estimates for spatially-close voxels are very similar. In the analysis stage, it is possible to conduct another step of spatial smoothing over the estimates from the proposed model using existing methods in the literature. For example, Polzehl and Spokoiny (2000) developed a locally adaptive weights smoothing method for imaging denoising and enhancement in univariate situations where each data point associated with each image pixel/voxel can be well approximated by a local constant function depending only on the spatial location of the pixel/voxel. Li et al. (2011) extended this approach further and developed multiscale adaptive regression models for multi-subjects' vectors of image measurements. This method integrates imaging smoothing with spatial data analysis of the smoothed data. Arias-Castro et al. (2012) characterized the performance of non-local means and related adaptive kernel-based image denoising methods by providing theoretical bounds on the estimation errors of these methods, which depend on the number of observed pixels and the underlying imaging features. Readers are referred to Yue et al. (2010) for a more detailed overview of smoothing methods used in the neuroimaging literature.

A nontrivial number of parameters are usually required to characterize nonlinearity, which may substantially increase the variance of the estimates and thus reduce power of detecting activation when the sample size is small. On the other hand, when strong nonlinear effects present, our simulations show that estimation of the additional nonlinearity parameters does not undermine estimation of the HRFs, and in fact, ignoring them introduces large bias in the HRF estimates. Our approach to this bias-variance tradeoff is to limit the number of functional bases (and thus the number of free parameters) representing subject-specific HRFs and the 2nd-order Volterra kernel. Through simulations, we found that our approach is the most efficient when (1) the nonlinear effect is strong, and/or (2) the sample size is large, and/or (3) the number of parameters characterizing interactive effects is small. For example, in the MID application, only a small area of V_k1k2 was observed, which significantly reduced the number of free parameters. Consequently, the proposed nonlinear model performed well though three different types of interactions were modelled. More generally, in studies where a considerable number of pairs of interactions are modelled, estimation errors can still be reduced by utilizing the prior knowledge of the small domain of V_k1k2. As a practical guideline, we recommend to model nonlinearity only when the interaction effect is of interest, or is expected to be strong (e.g., in event-related designs with short ISIs).

In our estimation strategy, we only impose regularity on the 1st-order derivatives of the two arguments of V_i,k1k2 (t₁, t₂), without assuming high-order differentiability; estimation errors of the model may be further reduced by imposing a different roughness constraint. Moreover, different penalty parameters can be considered for roughness constraints on HRF and Volterra kernels.

We neglect the variation of interaction effect on response latency across subjects in our model for V_i,k1k2 for simplicity and easy interpretation. With sufficient data, it is possible to evaluate such subject-specific interaction effect on latency by, for instance, the following semi-parametric Volterra series model, V̂_i,k1k2 (t₁, t₂) = M_i,k1k2 · V_k1k2 (t₁, t₂ + L_i,k1k2) for t₂ > t₁, where L_i,k1k2 characterizes the subject-specific latency change. Similar to the estimation of the latency coefficient D_i,k in the HRF h_i,k(t), we can use a first-order Taylor expansion to approximate V_i,k1k2 (t₁, t₂) and simplify the estimation:

$${\stackrel{\sim }{V}}_{i,k_1{k}_2}(t_1,t_2)\approx M_{i,k_1{k}_2}\cdot V_{k_1{k}_2}(t_1,t_2)+M_{i,k_1{k}_2}\cdot L_{i,k_1{k}_2}\cdot V_{k_1{k}_2}^{(0,1)}(t_1,t_2),$$

where the superscript (0, 1) stands for the first order partial derivative on t₂. Based on spline representations of V_k1k2 and f_k, we can also use a non-iterative procedure to estimate V̂_i,k1k2 (t₁, t₂): first estimate f_k and V_k1k2 through the same Steps 1-2; then evaluate subject-specific parameters A_i,k, D_i,k, M_i,k1k2 and L_i,k1k2 by the OLS estimates given the estimated f_k and V_k1k2. We can impose Σ_i L_i,k1k2 = 0 to avoid identifiability issue. Under this restriction, if the interest is mainly on the extent of interaction, it is reasonable to use the model for V_i,k1k2 (t₁, t₂) proposed in the article, where subject-specific interaction effects on latency, with zero means, are incorporated into the error terms.

Higher-order, say 3rd-order, Volterra kernels can in principle be used for evaluating interactions between more than two stimuli. For the experiment with inter-stimulus interval larger than 2s, however, this may not be beneficial because: first, the ensuing model will be overly complicated; second, biologically high-order interactions most likely will be negligible in comparison to lower-order ones if the interval between non-consecutive stimuli is larger than 4s.

Highlights.

• Propose a semi-parametric nonlinear model based on Volterra series.

• The new model characterizes nonlinearity in fMRI.

• The new model improves efficiency by “borrowing” information across subjects.

• Propose a hypothesis test for nonlinearity under the new model.

• Develop a fast-to-compute algorithm for estimating Volterra kernels including HRF.