State Space Modeling of Time-Varying Contemporaneous and Lagged Relations in Connectivity Maps

Abstract

Most connectivity mapping techniques for neuroimaging data assume stationarity (i.e., network parameters are constant across time), but this assumption does not always hold true. The authors provide a description of a new approach for simultaneously detecting time-varying (or dynamic) contemporaneous and lagged relations in brain connectivity maps. Specifically, they use a novel raw data likelihood estimation technique (involving a second-order extended Kalman filter/smoother embedded in a nonlinear optimizer) to determine the variances of the random walks associated with state space model parameters and their autoregressive components. The authors illustrate their approach with simulated and blood oxygen level-dependent functional magnetic resonance imaging data from 30 daily cigarette smokers performing a verbal working memory task, focusing on seven regions of interest (ROIs). Twelve participants had dynamic directed functional connectivity maps: Eleven had one or more time-varying contemporaneous ROI state loadings, and one had a time-varying autoregressive parameter. Compared to smokers without dynamic maps, smokers with dynamic maps performed the task with greater accuracy. Thus, accurate detection of dynamic brain processes is meaningfully related to behavior in a clinical sample.

1. Introduction

1.1. Background and Study Motivation

Advances in connectivity mapping of functional neuroimaging data have significantly increased science and society’s understanding of the brain (Behrens & Sporns, 2012; Smith, 2012). Many of these advances concern data-driven connectivity analyses (Gates & Molenaar, 2012; Smith et al., 2011). One type of connectivity analysis is directed functional connectivity mapping, which aims to reveal the direction of relations between brain regions of interest (ROIs) based on statistical dependencies in the neural signal (Friston et al., 2013). It has been accomplished by means of several analysis techniques, particularly structural equation modeling (SEM) involving only contemporaneous directed connections, and vector autoregressive modeling (VAR) involving only lagged directed connections. Estimates of both contemporaneous and lagged directed connections can be obtained with structural VARs (Chen et al., 2011; Smith et al., 2012), and (extended) unified structural equation modeling (euSEM, cf. Gates et al., 2010; Gates et al., 2011; Kim et al., 2007). In order to streamline computation and aid interpretation, these analysis techniques often assume stationarity, implying that connectivity parameters are constant across the neuroimaging time series, but emerging evidence suggests that this assumption is not always an appropriate one (reviewed in Hutchison et al., 2013).

Researchers have generally used one of two approaches for detecting time-varying relations in connectivity maps. First, sliding windows show time-varying, or dynamic, relations in these maps. In general, they determine change in connectivity indices between brain regions of interest (ROIs) across equally-spaced sections – or windows – of the time series (for a description, see Franke et al., 2008). The indices of interest are usually derived from a correlation analysis, but parameters from other analyses (e.g., time-frequency and independent components) have also been used (Chang & Glover, 2010; Kiviniemi et al., 2011). This work shows that time-varying relations are present in resting state and task-related functional connectivity maps, and that some of the variation is systematic (e.g., as determined by clustering algorithms) within and between individuals (Allen et al., 2012; Betzel et al., 2012; Chang & Glover, 2010; Chang et al., 2013; Handwerker et al., 2012; Jones et al., 2012; Kiviniemi et al., 2011; Rack-Gomer & Liu, 2012; Sakoğlu et al., 2010; Tagliazucchi et al., 2012; Thompson et al., 2013).

Second, time-varying (structural) VARs and state space models with time-varying parameters enable model-based approaches to dynamic connectivity mapping. In general, they determine time-varying connectivity parameters (including Granger causality indices) between ROIs using sophisticated estimation techniques. Models for blood oxygen level-dependent (BOLD) functional magnetic resonance imaging (fMRI) data have primarily been estimated with recursive least squares (for a description, see Möller et al., 2001). State space models with time-varying parameters have been considered in, for instance, Milde et al. (2010) and Havlicek et al. (2011), while Primiceri (2005) presents a general discussion of time-varying structural VARs. Because of their direct relevance to the present model, these approaches will be further discussed below (see section 2.4.2). Models for electroencephalography (EEG) and magnetoencephalography (MEG) data have also been estimated with Kalman filters and smoothers (for a description, see Bar-Shalom & Fortmann, 1988), which provide more noise suppression than sliding windows and more stable estimates than recursive least squares (Milde et al., 2010; Vedel-Larsen et al., 2010). Importantly, work using time-varying VARs has similarly found that dynamic relations are present in task-related effective connectivity maps (Hemmelmann et al., 2009; Hu et al., 2012; Milde et al., 2010; Wacker et al., 2011).

Sliding windows and time-varying VARs indicate that the stationarity assumption does not hold for data-driven connectivity mapping, but these approaches have limitations. Sliding window approaches provide a coarse, piecemeal estimate of dynamic relations, with findings varying based on window length; for example, the signal-to-noise ratio is lower in short versus long windows (discussed in Hutchison et al., 2013). So far, time-varying VARs have only been used to estimate lagged dynamic relations using fixed (i.e., not freely estimated) variances for both invariant and time-varying parameters, while only the latter should have nonzero variances. Moreover, it is unclear whether Kalman filters similar to those that have been applied to EEG and MEG data are appropriate for BOLD data, for which dynamic contemporaneous relations are of greatest relevance due to the comparatively low temporal resolution of the BOLD signal (Beltz & Molenaar, 2015; Smith et al., 2011). Thus, questions remain concerning the presence of time-varying relations in data-driven connectivity maps. Do time-varying relations exist when contemporaneous and lagged connection parameters are estimated within the same model? This is a key question because both parameter types must be calculated within the same model in order to ensure accurate magnitude and direction of ROI relations (Gates et al., 2010; Kim et al., 2007). Furthermore, can arbitrary (i.e., freely estimated, without information about which relations are dynamic and how they vary in time) contemporaneous and lagged time-varying connection parameters in BOLD fMRI data be estimated with an optimized second-order extended Kalman filtering/smoothing approach? This is a key question because current Kalman filtering approaches do not freely estimate the variances of time-varying parameters (Havlicek et al., 2011), requiring that all relations be estimated as dynamic (e.g., Milde et al., 2010). This may bias results if both constant and dynamic relations are present in the time series.

It is important to address these methodological limitations because converging evidence suggests that dynamic relations in functional neuroimaging data reflect meaningful neural processes. Time-varying functional relations have been found with multiple neuroimaging modalities, including fMRI, EEG, and MEG in human beings and local field potential recordings in cats, suggesting that they are not a mere methodological byproduct of one signal type in human beings (e.g., Betzel et al., 2012; Chang & Glover, 2010; de Pasquale et al., 2010; Hemmelmann et al., 2009; Milde et al., 2010; Popa et al., 2009). In fact, recent work has demonstrated correspondence between dynamic brain relations measured by BOLD fMRI and EEG (e.g., Chang et al., 2013; Tagliazucchi et al., 2012). Moreover, time-varying relations have been linked to experimental conditions (e.g., caffeine intake), behavior (e.g., vigilance during an attention task), and disease states (e.g., Alzheimer’s Disease and schizophrenia), suggesting that they are validly reflecting brain-based processes (Jones et al., 2012; Rack-Gomer & Liu, 2012; Sakoğlu et al., 2010; Thompson et al., 2013). Finally, time-varying analysis approaches have the potential to increase understanding of systematic temporal changes in brain connectivity that were previously detected, but not necessarily interpreted in terms of neural network stationarity. For example, past work on the neural underpinnings of olfactory habituation implemented unique task paradigms in order to overcome the systematic decreases in brain activity that correspond to repeated presentations of an odorant (e.g., Karunanayaka et al., 2014), but advances in time-varying analyses would permit explicit modeling of such habituation effects.

1.2. Current Study

The goal of the current study was to validate exploratory state space models (SSMs) in simulated data and then to estimate the models for BOLD fMRI data, allowing for explicit modeling of both contemporaneous and lagged time-varying connection parameters without a priori information about which parameters are dynamic. SSMs, of which VARs are a special case (that do not include measurement models), also allow for dimension reduction based on principled statistical methods. Last but not least, to the best of our knowledge for the first time in neuroimaging an optimal raw data maximum likelihood method (for Gaussian series) or quasi-maximum likelihood method (for non-Gaussian series) was used, consisting of a second-order extended Kalman filter/smoother (sEKFS) embedded within a nonlinear optimizer. The sEKFS can be conceived of as acting as E-step and the nonlinear optimizer as M-step in a nonstandard EM-algorithm. This is an extension of our previous work (Beltz & Molenaar, 2015; Gates et al., 2010; Gates & Molenaar, 2012; Gates et al., 2011) regarding innovative connectivity and grouping procedures for BOLD fMRI data, procedures that are among the best in the field (as tested in Gates & Molenaar, 2012; Smith et al., 2011).

To accomplish our goal, we utilized BOLD fMRI data from nicotine-deprived cigarette smokers performing a verbal working memory task. These data are ideal for this methodological investigation because the nature of the sample and specificity of the task facilitated the interpretation of dynamic relations, when they were found. For instance, past work has shown differences between the brain activity of smokers and non-smokers during verbal working memory tasks, and the differences were modulated by nicotine deprivation (Sutherland et al., 2011; Xu et al., 2006).

2. Methods

2.1. Participants

Participants were 30 cigarette users (22 men, 8 women), aged 19 to 45 years; they were randomly selected from a sample of 118 individuals, who participated in one of two fMRI studies on smoking cue reactivity (Wilson et al., 2012, 2013). For both studies, participants had to report smoking an average of 15 to 40 cigarettes per day for the past 24 months, be right-handed, and pass an MRI safety screening.

2.2. Procedures

The testing procedures and task are outlined here and described in detail elsewhere (Nichols et al., 2014; Wilson et al., 2012, 2013). Eligible participants (as determined with a phone interview) came to the lab for a baseline session that included questionnaire and psychological task completion. Participants were cigarette-deprived for 12 hours before the neuroimaging session, as confirmed with carbon monoxide (CO) levels. During this session, they provided structural MRI data and fMRI data during multiple tasks, including the verbal working memory task.

2.3. Measures

2.3.1. Baseline Assessment

During the baseline assessment, basic demographic information and information regarding smoking patterns were assessed with standard forms. Participants also completed behavioral working memory tasks and questionnaires assessing a variety of constructs, only a subset of which are reported herein (for details, see Wilson et al., 2012).

2.3.2. Neuroimaging Assessment

Participants were scanned using a 3-Tesla head-only Siemens Allegra magnet (Siemens Corporation, New York, NY) equipped with a standard transmit/receive head coil. Prior to functional scanning, a 40 slice oblique-axial anatomical series (3.125 × 3.125 × 3.0 mm voxels) was acquired parallel to the anterior commissure-posterior commissure plane using a standard T2-weighted pulse sequence. Additionally, a high-resolution (1 × 1 × 1 mm voxels) three-dimensional structural volume was collected using a magnetization-prepared rapid gradient-echo sequence. Next, functional images were acquired in the same plane as the 40-slice anatomical series with coverage limited to the 38 center slices using a one-shot echo-planar imaging pulse sequence (TR = 2000 ms, TE = 25 ms, FOV = 20 cm, flip angle = 79°).

The functional images were acquired during an n-back task, which assesses verbal working memory. The task design is depicted in Figure 1 (see also Nichols et al., 2014). Participants completed several 36-s task blocks during which 12 (of 18) randomly-selected English letters individually appeared (each 500 ms) between fixation crosses (each 2500 ms). Participants completed control (0-back) and experimental (3-back) versions of the task. In the control version, participants pressed a button each time they saw the letter X. In the experimental version, participants pressed a button each time the current letter they saw matched a letter they saw exactly three items ago. In both versions, the letter the participants saw was a target (required a button press) 33% of the time. Each participant completed three 0-back task blocks, three 3-back task blocks, and two rest blocks (fixation cross appeared for 36 s).

Figure 1.

Control (0-back) and experimental (3-back) versions of the verbal working memory (n-back) task.

2.3.3. Task Performance

Task performance was assessed by accuracy and reaction time. Accuracy was the percent of correct responses, and reaction time was acquired for all responses. Both measures were collapsed across task conditions.

2.4. Data Analysis Plan

2.4.1. Data Preparation

Standard preprocessing for BOLD fMRI data was conducted. Functional images were corrected for head motion and slice timing, adjusted for drift within and between runs, and aligned to standard space using a transformation matrix generated from the co-registration of the structural images to Montreal Neurologic Institute (MNI) space with a six-parameter rigid-body automated registration algorithm. Functional images were then globally mean-normalized and smoothed using a three-dimensional Gaussian filter (8-mm full width at half maximum).

To identify regions of interest, a standard two-level random-effects general linear model (GLM) approach was implemented on a voxel-wise basis using the AFNI program 3dDeconvolve (Cox, 1996). For each participant, parameter estimates for each version of the n-back task were obtained from the GLM. A second-level paired t-test was then used to determine which brain regions exhibited an effect of memory load (i.e., 3-back > 0-back) using a voxel-wise significance threshold of p < 1 × 10⁻¹⁸ and a spatial extent threshold of 10 contiguous voxels; this yielded a corrected map-wise false positive rate of p < 0.001 according to Monte Carlo simulations conducted in AFNI AlphaSim (Cox, 1996). BOLD time series data for further analyses were extracted from each of these brain regions: dorsal anterior cingulate cortex (ACC), right and left dorsolateral prefrontal cortex (DLPFC), right and left lateral premotor cortex (LPM), and right and left inferior parietal lobule (IPL).

2.4.2. Model Specification

In what follows, matrices and column vectors are denoted by, respectively, upper case and lower case bold letters. Latent variables are denoted by Greek symbols. The apostrophe (‘) denotes transposition. The SSM with time-varying parameters for a p-variate observed BOLD time series y(t) with p-variate mean level μ is:

$$\begin{array}{l}\mathbf{y}(\mathrm{t})=\mathbf{\mu }+\mathbf{\Lambda }[\mathbf{\theta }(\mathrm{t})]\mathbf{\eta }(\mathrm{t})+\mathbf{\epsilon }(\mathrm{t})\\ \mathbf{\eta }(\mathrm{t}+1)=\mathbf{B}[\mathbf{\theta }(\mathrm{t})]\mathbf{\eta }(\mathrm{t})+\mathbf{\Gamma }[\mathbf{\theta }(\mathrm{t})]\mathbf{z}(\mathrm{t})+\mathbf{\zeta }(\mathrm{t}+1)\\ \mathbf{\theta }(\mathrm{t}+1)=\mathbf{\theta }(\mathrm{t})+\mathbf{\xi }(\mathrm{t}+1)\end{array}$$ (1)

η(t), ε(t), z(t), and θ(t) are, respectively, a latent q-variate state process, a p-variate residual measurement noise process, an s-variate measured fixed input sequence, and an r-variate vector process containing all unknown parameters. The q-variate process ζ(t) and the r-variate process ξ(t) are process noise series. It is minimally assumed that ε(t), ζ(t) and ξ(t) are zero mean, weakly stationary white noise processes (lacking sequential dependencies) that are mutually independent and have finite moments up to fourth order, thus guaranteeing that parameter estimates have minimum mean-squared error. To obtain maximum likelihood estimates it is additionally assumed that ε(t), ζ(t) and ξ(t) are Gaussian.

The first equation of (1) shows that the mapping of η(t) on y(t) given by the (p,q)-dimensional sequence of matrices Λ[θ(t)], t=1,…,T, depend upon the time-varying parameter-vector θ(t). The second equation describes the time evolution of the state process η(t); the (q,q)-dimensional autoregressive weights in B[θ(t)] depend upon θ(t) and therefore can be arbitrarily time-varying. Γ[θ(t)] is a sequence of (q,s)-dimensional matrices containing the arbitrarily time-varying regression coefficients of the state process on the external input. The third equation in (1) describes the time-dependent variation of the unknown parameters. The r-variate parameter process θ(t) obeys a random walk with Gaussian white noise innovations ξ(t).

2.4.2.1. Choice of additional assumptions

Several distinct additional assumptions can be entertained to arrive at identifiable instances of (1). Presently a so-called exploratory version of (1) is considered in which a) the matrix sequence Λ[θ(t)] has no a priori structure (save for the minimal possible constraints to arrive at identifiable exploratory models); b) the sequence of autoregressive weights B[θ(t)] consists of diagonal matrices; c) the covariance function of ζ(t) is diagonal: cov[ζ(t), ζ(t-u)′] = δ(u)diag-Ψ, where δ(u) is the Kronecker delta being zero except if u=0. The definition of the model is completed by assumptions about the covariance functions of ε(t) and ξ(t): cov[ε (t), ε (t-u)′] = δ(u)diag-Ξ and cov[ξ(t), ξ(t-u)′] = δ(u)diag-Φ. The assumption that ε(t) has diagonal covariance matrix is mandatory in order to obey the standard definition of measurement noise as being conditionally independent.

2.4.2.2. Model transformation

The exploratory model considered in this paper can a posteriori (after having been fitted to the data) be transformed into an equivalent confirmatory model by means of the following transformations: Λ[θ(t)]* = Λ[θ(t)]T, B[θ(t)]* = T⁻¹B[θ(t)]T, ζ(t)* = T⁻¹ζ(t), where T is an arbitrary nonsingular (q,q)-dimensional matrix (e.g., Goodwin & Sin, 1984). For instance, T can be chosen in such a way that Λ[θ(t)]* has simple structure, that is, each univariate component series of y(t) has a high (absolute value) regression coefficient on one component of η(t)* and vanishing regression coefficients on the remaining components. Notice that such transformations destroy the independence of the components of η(t)* in that B[θ(t)]* is no longer diagonal and ζ(t)* has non-diagonal covariance matrix.

2.4.2.3. Model interpretation

The state space model (1) is a causal model involving directed functional connectivity. It contains contemporaneous connections directed from the latent state process η(t) to the ROIs in y(t). Also, the evolution of the latent state process is described by a VAR and hence involves lagged directed functional connections. The difference with the definition of directed functional connectivity given by Friston et al. (2013), however, is that the state process involved in these directed connections is latent. To fully comply with Friston’s definition it is required to interpret the q-variate latent state process as the output of q indirectly measured ROIs. Such realistic interpretation of latent state processes now is the norm in psychometrics (cf. Borsboom, 2009). Further conjectures about the identity of each of the indirectly measured ROIs then are based on the pattern of its contemporaneous and lagged directed connections, either in the original state space model or in an equivalent transformed analogue thereof. In what follows we will interpret latent state processes in this realistic sense. In contrast, if a latent state process is not interpreted as the output of indirectly measured ROIs then the Friston et al. (2013) definition of directed functional connectivity may not apply.

2.4.2.4. Statistical analysis

To fit the state-space model with time-varying parameters to an observed multivariate time series, an augmented state process is defined, which consists of the original latent state process and the time-varying parameter process: x(t)′ = [η(t)′, θ(t)′]. (Implementation of the model fitting steps is presented below in section 2.4.3.) Then, using the augmented state process x(t), (1) is rewritten as the following nonlinear state-space model:

$$\begin{array}{l}\mathbf{y}(\mathrm{t})=\mathbf{h}[\mathbf{x}(\mathrm{t}),\mathrm{t}]+\mathbf{\epsilon }(\mathrm{t})\\ \mathbf{x}(\mathrm{t}+1)=\mathbf{f}[\mathbf{x}(\mathrm{t}),\mathrm{t}]+\mathbf{w}(\mathrm{t})\end{array}$$ (2)

The vector-valued nonlinear functions h[x(t),t] and f[x(t),t] consist of products of the entries of x(t). The (q+r)-dimensional innovations process w(t) is defined as the composition of the innovation processes ζ(t) and ξ(t): w(t)′ = [ζ(t)′, ξ(t)′]. That is, the form of h[x(t),t] in (2) is the p-dimensional column vector Λ[θ(t)]η(t). The corresponding form of f[x(t),t] is the (q+r)-dimensional column vector [η(t)′[B[θ(t)]′, θ(t)′]′. To illustrate with a simple model in which p=3 and q=1, and using simplified notation: x₁(t) = η(t), x₂(t)=λ₁(t), x₃(t)=λ₂(t), x₄(t)=λ₃(t), x₅(t)=b₁(t). Then h[x(t),t] = [x₂(t)x₁(t), x₃(t)x₁(t), x₄(t)x₁(t)]′, and f[x(t),t] = [x₅(t)x₁(t), x₂(t), x₃(t), x₄(t), x₅(t)]′.

In addition to the assumptions concerning Eq. (1) it is assumed that the initial condition in Eq. (2) is: x(0)′ = [η(0)′, θ(0)′] is Gaussian with zero mean for η(0) and means equal to the estimates obtained in the preliminary block-Toeplitz analysis (see below) for θ(0). The covariance matrix of x(0) is diagonal with large values for η(0) and the estimated standard errors obtained in the preliminary block-Toeplitz analysis for θ(0). Initial conditions also can be estimated according to the approach presented in Durbin and Koopman (2001). However, the effects of initial conditions decay exponentially and therefore are small for series of sufficient length. Moreover, we use the smoother which re-estimates the initial condition in the backward sweep.

Model (2) is fitted by means of the raw data likelihood method consisting of a new variant of the expectation maximization (EM) algorithm. The E-step is implemented by the sEKFS for estimation of the latent state process x(t), using filtering equations that are augmented by appropriate tensor terms (Eqs. 10.3.2-4 and 10.3.2-7 in Bar-Shalom et al., 2001). Because the nonlinearities in (2) are confined to products of the entries of x(t), the sEKFS is locally exact (derivatives with respect to the local state process of order higher than two are zero). The M-step consists of estimation of the variances of ε(t) and w(t) by maximizing the likelihood using constrained sequential quadratic optimization (Gill et al., 1998). The equations underlying the implementation of both the E- and M-steps are detailed in the Appendix, and the listing of a beta version of the implementation can be obtained from the Supplementary Materials.

Identification of time-varying parameters then proceeds as follows. If an estimated variance along the diagonal of cov[ξ(t), ξ(t-u)′] = δ(u)diag-Φ is zero, then the associated parameter is constant in time. In contrast, if an estimated variance of diag-Φ is relatively large then the associated parameter may vary substantially in time. The relation between process noise and temporal variation of parameters is, however, indirect (see NiedŸwiecki, 2000). It therefore is tested whether the time-dependent trajectory of each estimated parameter contains stretches which differ significantly from each other. That is, significant differences should not be limited to a single pair of nonadjacent time points but should involve distinct stretches each covering more than one adjacent time point. In this test the actual estimated time-varying standard errors are used as obtained from the sEKFS, together with the stringent criterion that pairs of stretches should differ from each other at least 3.1 standard errors.

2.4.2.5. Comparison with related approaches

Our estimation approach differs in a number of important respects from similar approaches (in, for instance, Havlicek et al., 2011; Hu et al., 2012; Milde et al., 2010). Firstly, use is made of the sEKFS instead of the first-order extended Kalman filter (Milde et al., 2010) or the cubature Kalman filter (Havlicek et al., 2011; Hu et al., 2012). The sEKFS is optimal in the sense that it provides an exact local approximation for the estimation of the extended state x(t) in (2). This is because all third- and higher-order derivatives in the sEKFS for (2) are zero. Direct comparison of the sEKFS with the first-order extended Kalman filter and the unscented filter (cf. Simon, 2006) shows that it outperforms the latter two approaches. Secondly, the variances of the process noise ξ(t+1) in the random walk for the parameter process θ(t), that is, the diagonal elements of Φ, are estimated by means of likelihood maximization. Each of these process noise variances is related to the estimation memory of the filter: the larger this variance is, the smaller is the estimation memory for this parameter (NiedŸwiecki, 2000). For constant parameters the estimation memory covers all data. Therefore the estimated innovation variances can be regarded as greatest lower bounds on the estimation memory. Previously these variances have been fixed at a small constant (Milde et al., 2010) or are derived from the local filtering results depending upon a fixed forgetting factor (Havlicek et al., 2011; Hu et al., 2012) as well as a fixed coefficient determining the degree of annealing (covariance resetting; Havlicek et al., 2011). Our approach allows for the possibility that variances associated with particular parameters are zero, indicating that these parameters are constant. Thirdly, time-varying structural VARs (Primiceri, 2005) also enable decompositions of contemporaneous and lagged dynamic connections. However, results thus obtained depend upon the ordering of univariate component series in the vector-valued observed series (Lütkepohl, 2005; Primiceri, 2005). In contrast, our approach to estimate contemporaneous and lagged dynamic connections is invariant under permutations of the order of the entries in y(t). Fourthly, regime shifting models (Olier et al., 2013) are optimal for tracking sudden transitions between otherwise stationary regimes (Chow & Zhang, 2013). Hence regime shifting state space models have different domains of application in comparison with our approach in which time-dependent parameter changes are assumed to be smooth (e.g., due to habituation).

In sum, our approach is optimal and timely, as it appears to be the first exploratory (data-driven) state space modeling approach to estimate smooth time-varying contemporaneous and lagged parameters simultaneously. This is important because brain networks measured by BOLD are best quantified with contemporaneous relations (Beltz & Molenaar, 2015; Smith et al., 2011), but the estimation of contemporaneous relations is biased without estimation of lagged connection parameters in the same network model (Gates et al., 2010).

2.4.3. Model Fitting

For each participant the dimension of the latent state process η(t) in (1) has been determined in a purely data-driven, exploratory way by means of the block-Toeplitz method described in Molenaar (1985). The dimensionality of each individual’s model was automatically determined by stepwise increases until the model had good fit to the data, according to the following criteria: the Confirmatory Fit Index (CFI), which had to be larger than .90, and the Non-Normed Fit Index (NNFI), which also had to be larger than .90 (cf. Brown, 2006). Both indices are derived from the likelihood ratio penalized for number of free parameters and were found to perform well in simulation studies (Gates & Molenaar, 2012). Estimates of the parameter vector thus obtained serve as starting values in the raw data likelihood fit of (1). The Expectation step in the likelihood maximization is obtained by means of the sEKFS; the Maximization step is obtained by means of NPSOL^ã (Gill et al., 2001), serving as a nonlinear optimizer of δ(u)diag-Ξ, δ(u)diag-Φ, and δ(u)diag-Ψ in (1). NPSOL maximizes the likelihood using a sequential quadratic programming method. Linear and/or nonlinear (in-)equality constraints are handled by a combination of penalty and Lagrange multiplier methods as described in Bertsekas (1999). In the present application only the soft inequality constraints are used that estimated variances should be non-negative. See also the Supplementary Materials.

2.4.4. Some Simulation Studies

As a preliminary test of the new EM algorithm, the results of some small-scale simulation studies are reported. The simulations are small-scale because the beta version of the implementation (obtainable from the additional materials) has not been optimized with respect to speed. Data were generated according to (1) with p=6 (six ROIs) and q=2 (bivariate state process). All parameter values in the simulation program correspond to the general pattern of parameter estimates obtained in our empirical application (presented below). In particular, the first column of Λ[θ(t)] has relatively high positive loadings on all ROIs while the second column only has loadings on the fourth, fifth and sixth ROI, the values of which are lower in absolute value than in the first column. Three data sets were generated, each consisting of 100 replications for T=200 TRs. Application of the EM algorithm is blind with respect to whether there are time-varying parameters and, if so, how these vary in time.

In the first data set there is no external input z(t). The loading (1,1) in Λ[θ(t)] linearly decreases from 1.0 at t=1 to 0.1 at t=200; all other parameters are constant. All parameter estimates are close to their true values (never exceeding the 95% confidence intervals about their true values). The algorithm correctly identifies that the loading (1,1) in Λ[θ(t)] is time-varying; Figure 2A depicts its average estimate across replications.

Figure 2.

SSM results of simulation studies, with each simulation containing 6 ROIs, 2 states, 200 measurements, and 100 replications. (A.) The SSM accurately recovered the time-varying contemporaneous loading for the first ROI on the first state (i.e., (1,1) in Λ[θ(t)]); the average across replicates is plotted, and estimates were within the 95% confidence intervals of the simulated values. (B.) The SSM accurately recovered the time-varying autoregressive component for the first state (i.e., (1,1) of B[θ(t)]); the average across replicates is plotted, and estimates were within the 95% confidence intervals of the simulated values. (C.) The SSM accurately recovered the time-varying direct effect of the external input on the first state (i.e., (1,1) of γ[θ(t)]); the average across replicates is plotted, and estimates were within the 95% confidence intervals of the simulated values.

In the second data set there is no external input z(t). The autoregressive parameter associated with the first component of the latent state process (element (1,1) of B[θ(t)]) decreases from 0.8 at t=75 to 0.1 at t=200; all other parameters are constant. Notice that at t=75 a kind of regime shift occurs involving a transition from a constant to a time-varying regime. The constant parameter estimates are close to their true values (never exceeding the 95% confidence intervals about their true values). The algorithm correctly detects that it is element (1,1) of B[θ(t)] that is partly time-varying. Figure 2B depicts its average estimate across replications.

In the third data set there is univariate external input z(t). The direct effect of the input on the first state process (element (1,1) of γ[θ(t)]) decreases from 0.7 at t=1 to 0.1 at t=50, after which it increases to 1.2 at t=200. The algorithm correctly identifies that element (1,1) of γ[θ(t)] is nonlinearly time-varying; Figure 2C depicts its average estimate across replications.

2.4.5. Task Performance

We also explored links between time-varying parameters and performance on the verbal working memory task. Specifically, we compared individuals with and without dynamic parameters on task accuracy and reaction time, using independent samples t-tests and Type I error of .05.

3. Results

3.1. General Characteristics of State Space Models (SSMs): Contemporaneous and Lagged Parameters

All participants had state space models of excellent fit, with CFIs and NNFIs equaling or exceeding .90. As stated above, Λ[θ(t)] had no a priori structure, so we determined its dimension for each participant by systematically increasing the number of states by one until both alternative fit indices were greater than or equal to .90. This resulted in models with three states for all participants, where states reflect independent patterns of dynamic coordination among ROIs.

Generally, all ROIs had substantial loadings (in Λ[θ(t)]) on the first state for all participants, and some ROIs had substantial loadings on the second and third states for subsets of participants. Table 1 shows the mean ROI loadings across participants on each state. All ROI loadings on the first state were significant and greater than or equal to .76. No ROI loadings on the second and third states were significant across all participants, even though loadings were significant for some individuals. Notice that the bilateral IPL had the highest average loadings on both the second and third states. Also, notice that the ACC and right LPM have some fixed loadings in order to attain an identifiable model solution. In sum, the first latent state process explained the most variance in the SSMs and constituted the dominant common network in this study, whereas the second and third states reflected participant-specific neural networks (that are discussed in greater detail below).

Table 1.

SSM results averaged across participants: Contemporaneous ROI loadings on each state and lagged weights describing each state’s evolution over time

		State 1	State 2	State 3
Contemporaneous loading (λ)	ACC	.85*	--	--
	R LPM	.85*	.17	--
	L DLPFC	.86*	.03	.14
	R IPL	.82*	.25	.22
	R DLPFC	.83*	.11	.10
	L LPM	.80*	.12	-.04
	L IPL	.76*	.19	.26
Autoregressive weight (β)		.79*	.63*	.60*

Note. ACC: anterior cingulate cortex; R LPM: right lateral premotor cortex; L DLPFC: left dorsolateral prefrontal cortex; R IPL: right inferior parietal lobule; R DLPFC: right dorsolateral prefrontal cortex; L LPM: left lateral premotor cortex; L IPL: left inferior parietal lobule.

^*p < .05

Each state also has an autoregressive weight (in B[θ(t)]), reflecting its evolution over time. Table 1 shows the means for these weights across participants. The average lagged parameter for each state was large and significant, but it was largest and least variable for the first state. Thus, the first state had the most slowly varying evolution over time, but the second and third states also had substantial lagged effects.

3.2. External Input Effects in SSMs

None of the parameters in γ[θ(t)], quantifying the direct lagged effects of external input on state processes, are time-varying. The external input had a significant direct effect on one or more state processes for 13 participants, with 11 participants showing significant effects on a single state process, and 2 participants showing significant effects on 2 state processes. It is particularly noteworthy that all these significant direct effects were only on the second and third states; the external input did not significantly impact the first state. Moreover, significant external input effects were primarily on states predominantly loading on the bilateral IPL, with 38% of the detected effects on a state loading on the left IPL and 27% of the effects on a state loading on the right IPL. Thus, the second and third states can be interpreted in terms of the lagged task effects they reflect for nearly half the sample.

3.3. Time-Varying SSM Parameters

Time-varying parameters were present in the SSMs of 12 participants. Table 2 lists the time-varying effects for each participant and defines the states containing time-varying parameters in terms of their ROI loadings and implications for brain function, with the bilateral LPM presumably contributing to task-related motor responses, the bilateral DLPFC to decision-making, and the bilateral IPL to language interpretation. Figure 3 shows plots of the time courses of all participants’ time-varying parameters listed in Table 2. Figure 4 integrates the information contained in Table 2 and Figure 3 by displaying dynamic brain networks for two exemplar participants.

Table 2.

Identification and interpretation of time-varying states

Participant	State	Time-varying parameters	Additional ROIs defining the state	Network identification
A	1	L IPL	All	General
	2	L LPM	L IPL	Left language interpretation & response
B	2	L DLPFC	R LPM	Decision-making & response
		L LPM	R DLPFC
C	1	ACC	All	General
D	1	ACC	All but R IPL	General
		R LPM
		R DLPFC
		L LPM
E	2	R DLPFC	L DLPFC	Language interpretation & decision-making
			R IPL
F	2	R LPM	R IPL	Language interpretation & response
		L LPM	L IPL
G	2	autoregression	R IPL	Language interpretation
			L IPL
H	3	L IPL		Left language interpretation
I	3	R DLPFC	R IPL	Language interpretation, decision-making, & response
		L LPM
		L IPL
J	1	L DLPFC	All	General
K	2	L LPM	L IPL	Left language interpretation & response
L	1	ACC	All	General

Note. ROI acronyms are defined in the Table 1 note.

Figure 3.

Time-varying parameters for all participants with dynamic SSMs, plotted across time; see also Table 2. The time courses of different parameters are depicted in different colors: black is the ACC, yellow is the R LPM, magenta is the L DLPFC, red is the R DLPFC, green is the L LPM, and blue is the L IPL. The time courses of different states are depicted in different line styles: solid lines are first state parameters, dashed lines are second state parameters, and dotted lines are third state parameters. Black lines with circles show the time course of a second state autoregression. The thick black line parallel to the y-axis represents the average standard deviation of the plotted parameters. The scale of the y-axis differs across participants to highlight dynamic parameters. ROI acronyms are defined in the Table 1 note.

Figure 4.

Time-varying state network maps for two exemplary participants. ROIs are orange and overlaid on a standard template brain; see also Nichols et al. (2014). Lines show which ROIs contribute to the latent state: Gray lines depict ROIs with stationary loadings, black lines depict ROIs with time-varying loadings, and line width reflects the magnitude of the loadings. (A.) Participant B (from Table 2) had a decision-making and response state (i.e., state 2) consisting of the bilateral LPM and bilateral DLPFC, with the both left hemisphere ROIs having time-varying loadings. (B.) Participant I (from Table 2) had a language interpretation, decision-making, and response state (i.e., state 3) consisting of the bilateral IPL, left LPM, and right DLPFC, with the left hemisphere ROIs and right DLPFC having time-varying loadings. ROI acronyms are defined in the Table 1 note.

Several conclusions can be drawn from these results. First, most time-varying parameters were contemporaneous (only participant G has a time-varying autoregressive parameter). Second, the left LPM was the ROI most often identified as time-varying. In fact, it was identified as time-varying twice as often – accounting for 30% of the dynamic parameters – as the next closest ROIs; the ACC, right DLPFC, and left IPL each accounted for 15% of the dynamic parameters. Third, the right IPL did not have time-varying loadings for any participant. Fourth, although there were similarities across participants, there were also marked individual differences in the presence and patterns of time-varying parameters. For instance, all 4 time-varying parameters for participant D markedly decreased across the time course, the 2 time-varying parameters for participant F spiked around time point 60 before plummeting, and the single time-varying parameter for participant K showed a slow, but steady increase across the time course until dropping off after time point 120. Moreover, the majority of the participants (7 of 12) had SSMs with a single time-varying parameter, but 3 participants had SSMs with 2 time-varying parameters, and 2 participants had SSMs with 3 and 4 time-varying parameters.

Fifth, time-varying parameters were present on all states. When a time-varying loading in Λ[θ(t)] is on the first state process then its interpretation is straight-forward because the first state represents a general, common network for all participants (see section 3.1). When a time-varying loading is on the second or third state, however, its interpretation depends upon the participation-specific pattern of loadings of those states. This is exemplified in Figure 4. For participant B (Figure 4A), the bilateral LPM and DLPFC significantly loaded on the second state, indicating that the state represents a decision-making and response network, and both left hemisphere ROIs have time-varying loadings in this network. For participant I (Figure 4B), the right DLPFC, left LPM, and left IPL have time-varying loadings on the third state, which also contains a notable right IPL loading, and thus, represents a language interpretation, decision-making, and response network. Furthermore, most participants had time-varying loadings associated with a single state, with only one (participant A) showing time-varying loadings on multiple states (i.e., a dynamic left IPL in the first, general state and a dynamic left LPM in the left language interpretation and response state).

3.4. Task Performance

Finally, we examined whether participants with and without time-varying parameters differed in verbal working memory task performance, focusing on task accuracy and reaction time. The groups differed on task accuracy, t(28) = −2.60, p = .02, with participants who had dynamic SSMs (M = .91, SD = .05) outperforming those who did not (M = .84, SD = .08). The groups did not differ in reaction time, though, t(28) = .07, p = .95, with both participants who had dynamic SSMs (M = 730ms, SD = 167ms) and those who did not (M = 725ms, SD = 223ms) responding in similar amounts of time.

4. Discussion

4.1. Interpretation and Significance of Findings

In this paper, we provided a description and first illustration of a new state space model (SSM)-based approach for detecting smoothly time-varying network connections in brain connectivity maps. The approach is exploratory and can be applied in a data-driven way without any a priori information. The obtained network connections are contemporaneous and lagged directed functional connections among directly and indirectly measured ROIs. This work fills three knowledge gaps concerning the presence and nature of time-varying connectivities. First, the model fit is innovative in at least one important respect: the process variances of random walks modeling the evolution of model parameters are estimated. In previous approaches these variances were fixed a priori at some small value, which leads to sub-optimal performance. Second, arbitrary time-varying parameters in BOLD fMRI data can be estimated by means of optimal (quasi-) maximum likelihood techniques involving locally exact second-order extended Kalman filtering/smoothing. Third, time-varying contemporaneous connections can be determined within the same model as lagged connections in a way that is independent of the ordering of univariate components in the vector-valued observed series. We provided technical details regarding these general conclusions, a listing of the beta version of the Fortran implementation that produced the results (see the Supplementary Materials), results from small simulation studies, and an application demonstrating feasibility to cigarette smokers’ verbal working memory BOLD data.

Alongside these general conclusions, there were several specific and novel findings. First, SSMs with three states were identified for all participants. The first state represented a general neural network, explaining the most variance in participants’ models and displaying a consistent pattern of loadings, with all seven ROIs contributing to it substantially and with similar orders of magnitude. The second and third states varied across participants, and thus, require participant-specific interpretations. For example, a state consisting of significant bilateral IPL loadings likely represents a language interpretation network in the context of the verbal working memory task participants were completing during data collection (see, e.g., participant G in Table 2). In fact, the bilateral IPL were the two ROIs with the greatest contributions to the second and third states of participants. Moreover, all three states had large autoregressive components, with the component for the first state being largest.

Second, the external input was related to brain function, with 43% of participants showing task-related effects in their SSMs. The external input consisted of the 3-back (most difficult) condition of the verbal working memory task, convolved with a double gamma function to approximate the hemodynamic response function. Effects of the external input were exclusively on participants’ second and third states. This is consistent with the large second and third state loadings on the bilateral IPL, and suggests that the bilateral IPL are hubs for neural networks underlying verbal working memory task performance in smokers.

Third, 40% of participants had time-varying parameters, and they were primarily associated with contemporaneous connections on the latent state processes. The loading on the left LPM was most frequently identified as being time-varying, a logical result because all participants were right-handed and the LPM subserves the motor responses that were variably elicited by the verbal working memory task. Because our estimation approach allows for arbitrary detection of time-varying parameters associated with both contemporaneous and lagged connections, it is particularly compelling that we found little evidence of dynamic lagged parameters. Thus, our work calls into question the use of time-varying vector autoregressive models (e.g., Hu et al., 2012; Milde et al., 2010), which do not permit estimation of time-varying contemporaneous relationships.

Fourth, the presence and pattern of time-varying ROIs showcases individual differences in the neural processes underlying behavior. Participants differed in the number of time-varying parameters in their maps, the specific ROIs or autoregressive components that were dynamic, the time course of the time-varying parameters, and the interpretation of the parameters based upon the state processes on which the parameters loaded. This highlights the importance of grouping procedures in connectivity mapping that allow for individual-level nuances in the maps (as presented in Gates & Molenaar, 2012).

Fifth, compared to those without dynamic relations, smokers with dynamic relations performed better on the verbal working memory task. Specifically, regular cigarette users who had dynamic patterns of brain connectivity provided more correct responses when identifying the letter X (0-back condition) and when the letter they were viewing was presented three items ago (3-back condition) than users who had stationary patterns of brain connectivity. This group difference likely reflects an important neural process because it is consistent with past work showing links between time-varying parameters and behavior (Rack-Gomer & Liu, 2012; Thompson et al., 2013), and it cannot be attributed to strategy use (e.g., as would be implied by a speed accuracy trade-off), since both smokers with and without dynamic patterns of brain connectivity had similar reaction times.

4.2. Suggestions for Future Research

Our approach can be straightforwardly generalized to multi-subject applications. In this way, loadings of latent state processes can be constrained to be equal or proportional across participants. For example, we could have forced participants to have equal right and left DLPFC loadings on the second latent state process in order to test its interpretation in terms of a “decision-making” network. Also the model can be implemented in a semi-confirmatory or confirmatory way instead of in an exploratory, data-driven way, as presented in this paper.

The meaning of dynamic connectivity maps for understanding brain processes underlying smoking behavior requires further investigation. Evidence reviewed above (section 1.1) suggests that dynamic processes are generally meaningful for behavior, but work within the domain of smoking – a model system for drug addiction – is lacking. We found that smokers with time-varying SSMs outperformed those without time-varying SSMs in verbal working memory task accuracy, but it is unclear how this is related to cigarette use. Are individuals with time-varying relations more likely to quit smoking? Do our findings generalize to tasks beyond working memory? Our study of cigarette users completing a well-characterized task provided valuable context for interpreting time-varying parameters, but future work could extend these methods to new samples and tasks in an attempt to predict smoking behavior.

4.3. Conclusions

We presented an innovative approach for estimating dynamic connectivity maps. Methodologically, we demonstrated for the first time that a novel data estimation technique embedded within second-order extended Kalman filtering/smoothing can be used to identify arbitrary time-varying contemporaneous and lagged relations in BOLD fMRI data. Substantively, we revealed a link between working memory and dynamic brain activity that has implications for smoking research. Future applications of this approach have the potential to provide insight into basic questions regarding brain function and applied questions regarding intervention.

Appendix

The nonlinear state-space model (2) in the main text to be fitted to the p-variate observed series y(t), t=1,2,…,T is:

$$\begin{array}{l}\mathbf{y}(\mathrm{t})=\mathbf{h}[\mathbf{x}(\mathrm{t}),\mathrm{t}]+\mathbf{\epsilon }(\mathrm{t})\\ \mathbf{x}(\mathrm{t}+1)=\mathbf{f}[\mathbf{x}(\mathrm{t}),\mathrm{t}]+\mathbf{w}(\mathrm{t})\end{array}$$

where x(t) is (q+r)-dimensional and consists of a concatenation of the state process η(t) and the parameter process θ(t) in the linear model (1) in the main text. The initial values θ(0) and var[θ(0)] are obtained by fitting the stationary version of the linear state-space model (1) in the text by means of either the block-Toeplitz approach (Molenaar, 1985) or the prediction error decomposition (Hamaker et al., 2005).

Let x(t|t′) denote the expectation of x(t) conditional on x(t′), where t′ = t or t-1. Define the following partial derivatives at x(t|t-1):

$$\begin{array}{l}{\mathbf{h}}_x(t)\equiv \frac{\partial }{{\partial }_x}\mathbf{h}[\mathbf{x},t]\\ {\mathbf{h}}_{xx}^i(t)\equiv \frac{{\partial }^2{h}^i[x(t),t]}{\partial x^2}\end{array}$$

Define the following partial derivatives at x(t|t):

$$\begin{array}{l}{\mathbf{f}}_x(t)\equiv \frac{\partial }{{\partial }_x}\mathbf{f}[\mathbf{x},t]\\ {\mathbf{f}}_{xx}^i(t)\equiv \frac{{\partial }^2{f}^i[x(t),t]}{\partial x^2}\end{array}$$

Let x(0|0)′ = [0′, θ(0)′] and P(0|0) = diag{I, var[θ(0)}. Let e_i have 1 at the i-th row and zeroes elsewhere and let Tr[.] denote the trace.

E-step. For t=1,…,T do:

$$\begin{array}{l}\mathbf{x}(t\mid t-1)=\mathbf{f}[x(t-1\mid t-1),t-1]+0.5{\sum }_{i=1}^{q+r}{\mathit{e}}_{i}Tr[{\mathbf{f}}_{xx}^i(t-1)\mathbf{P}(t-1\mid t-1)]\\ \mathbf{P}(t\mid t-1)={\mathbf{f}}_x(t-1)\mathbf{P}(t-1\mid t-1){\mathbf{f}}_x{(t-1)}^{\prime }+0.5{\sum }_{i=1}^{q+r}{\sum }_{j=1}^{q+r}{\mathit{e}}_i{\mathit{e}}_{j}Tr[{\mathbf{f}}_{xx}^i(t-1)\mathbf{P}(t-1\mid t-1){\mathbf{f}}_{xx}^j(t-1)\mathbf{P}(t-1\mid t-1)]+\mathit{var}[\mathbf{w}(t-1)]\\ \mathbf{y}(t\mid t-1)=\mathbf{h}[x(t\mid t-1),t]+0.5{\sum }_{i=1}^p{\mathit{e}}_{i}Tr[{\mathbf{h}}_{xx}^i(t)\mathbf{P}(t\mid t-1)]\\ \mathbf{S}(t)={\mathbf{h}}_x(t)\mathbf{P}(t\mid t-1){\mathbf{h}}_x{(t)}^{\prime }+0.5{\sum }_{i=1}^p{\sum }_{j=1}^p{\mathit{e}}_i{\mathit{e}}_{j}Tr[{\mathbf{h}}_{xx}^i(t)\mathbf{P}(t\mid t-1){\mathbf{h}}_{xx}^j(t)\mathbf{P}(t\mid t-1)]+\mathrm{\Xi }\\ \mathbf{W}(t)=\mathbf{P}(t\mid t-1){\mathbf{h}}_x(t)\mathbf{S}{(t)}^{-1}\\ \mathbf{P}(t\mid t)=[\mathrm{I}-\mathbf{W}(t){\mathbf{h}}_x(t)]\mathbf{P}(t\mid t-1){[\mathrm{I}-\mathbf{W}(t){\mathbf{h}}_x(t)]}^{\prime }+\mathbf{W}(t)\mathrm{\Xi }\mathbf{W}{(t)}^{\prime }\\ \mathbf{w}(t\mid t)=\mathbf{y}(t)-\mathbf{y}(t\mid t-1)\\ \mathbf{x}(t\mid t)=\mathbf{x}(t\mid t-1)+\mathbf{W}(t)\mathbf{w}(t\mid t)\end{array}$$

M-step. Maximize the log-likelihood L[w(t|t), t=1,…,T] with respect to var[ε (t), ε (t)′] = Ξ, var[ζ(t), ζ(t)′] = Ψ and var[ξ(t), ξ(t)′] = Φ:

$$\begin{array}{l}{max}_{\mathit{\Xi }\mathit{\Psi }\mathit{\Phi }}L[\mathbf{w}(t\mid t),t=1,\dots ,T]\\ L[\mathbf{w}(t\mid t),t=1,\dots ,T]=-0.5[\mathit{pTln}2\mathrm{\pi }+{\sum }_{t=1}^{T}ln\mid \mathbf{S}(t)\mid +{\sum }_{t=1}^T\mathbf{w}{(t\mid t)}^{\prime }\mathbf{S}{(t)}^{-1}\mathbf{w}(t\mid t)\mid ]\end{array}$$