Bayesian estimation of covariate assisted principal regression for brain functional connectivity

Abstract

This paper presents a Bayesian reformulation of covariate-assisted principal regression for covariance matrix outcomes to identify low-dimensional components in the covariance associated with covariates. By introducing a geometric approach to the covariance matrices and leveraging Euclidean geometry, we estimate dimension reduction parameters and model covariance heterogeneity based on covariates. This method enables joint estimation and uncertainty quantification of relevant model parameters associated with heteroscedasticity. We demonstrate our approach through simulation studies and apply it to analyze associations between covariates and brain functional connectivity using data from the Human Connectome Project.

1 Introduction

This paper reformulates covariate-assisted principal (CAP) regression of Zhao et al. (2021b) in the Bayesian paradigm. The approach identifies covariate-relevant components of the covariance of multivariate response data. Specifically, the method estimates a set of linear projections of multivariate response signals, whose variance is related to external covariates. In neuroscience, there is interest in analyzing statistical dependency between time-series of brain signals from distinct regions of the brain, which we refer to as functional connectivity (FC) (Lindquist 2008; Fornito and Bullmore 2012; Fornito et al. 2013; Monti et al. 2014; Fox and Dunson 2015). The brain signals underling FC are multivariate, and each brain activity is considered relative to others (Varoquaux et al. 2010) in analyzing FC, as this statistical dependency is related with behavioral characteristics (covariates). This paper develops a Bayesian approach to conducting supervised dimension reduction for the response signals, to analyze the association between external covariates and the FC characterized by the multivariate signals’ covariances.

Typically, the first step to analyze brain FC is to define a set of nodes corresponding to spatial regions of interest (ROIs), where each node is associated with its own time course of imaging data. Then the network connections (or an “edge” structure between the nodes) are subsequently estimated based on the statistical dependency between each of the nodes’ time course (van der Heuvel and Hulshoff Pol 2010; Friston 2011). FC networks have been inferred using Pearson’s correlation coefficients (Hutchison et al. 2013) and also with partial correlations in the context of Gaussian graphical models (Whittaker 1990; Hinne et al. 2014) summarized in the precision or inverse covariance matrix. In recent years, there has been a focus on subject-level graphical models where the node-to-node dependencies vary with respect to subject-level covariates. This line of research involves methods to estimate or test group-specific graphs (Guo et al. 2011; Danaher et al. 2014; Narayan et al. 2015; Peterson et al. 2015; Xia et al. 2015; Cai et al. 2016; Saegusa and Shojaie 2016; Lin et al. 2017; Tan et al. 2017; Xia and Li 2017; Durante and Dunson 2018; Xia et al. 2018) as well as general Gaussian graphical models for graph edges that allow both continuous and discrete covariates, estimated based on trees (Liu et al. 2010), kernels (Kolar et al. 2010; Lee and Xue 2018), linear or additive regression (Ni et al. 2019; Wang et al. 2022; Zhang and Li 2023). However, like other standard node-wise regression methods (e.g. Meinshausen and Buhlmann 2006; Peng et al. 2009; Kolar et al. 2010; Cheng et al. 2014; Leday et al. 2017; Ha et al. 2021) in Gaussian graphical models, these approaches focus on edge detection (ie estimation of the off-diagonal elements) rather than estimating the full precision or covariance matrix and do not explicitly constrain positive definiteness of precision or covariance matrices. Works on general tensor outcome regression (Li and Zhang 2017; Sun and Li 2017; Lock 2018) also do not generally guarantee the positive definiteness of the outcomes. While the problem of dimension reduction of individual covariances has been studied in brain dynamic connectivity analysis (Dai et al. 2020), problems in computer vision (Harandi et al. 2017; Li and Lu 2018; Gao et al. 2023) and brain computer interfaces (Davoudi et al. 2017; Xie et al. 2017) as well as multi-group covariance estimation (Flury 1984, 1986; Boik 2002; Pourahmadi et al. 2007; Hoff 2009; Franks and Hoff 2019), covariate information was not utilized in conducting dimension reduction, or it views the data at the group level, which does not account for subject-level heterogeneity in the brain networks. Gaussian graphical models have been applied to study brain connectivity networks in fMRI data (e.g. Li and Solea 2018; Zhang et al. 2020), however, the focus was on analyzing connectivity networks, without explicitly considering their relationship with subject-level covariates.

In this paper, in line with the covariance regression literatures (see, e.g. Engle and Kroner 1995; Fong et al. 2006; Varoquaux et al. 2010; Pourahmadi 2011; Hoff and Niu 2012; Fox and Dunson 2015; Zou et al. 2017; Zhao et al. 2021a,b, 2024), we will frame the problem of analyzing FC as modeling of heteroscedasticity, ie estimating a covariance function ${\mathbf{\Sigma }}_{\mathit{x}}=\text{var}[\mathit{Y}|\mathit{x}]$ across a range of values for an explanatory x-variable. In contrast to the approach developed in Zhao et al. (2021b) where each projection vector for ${\mathbf{\Sigma }}_i$ is estimated sequentially and in Franks (2022) where statistical inference is conducted conditionally on the estimated dimension-reduced subspace, the proposed framework allows coherent and simultaneous inference on all model parameters within the Bayesian paradigm.

One typical approach to associating brain FC with behavior is to take a massive univariate test approach that relates each connectivity matrix element with subject-level covariates (e.g. Woodward et al. 2011; Grillon et al. 2013). However, this “massive edgewise regression” lacks statistical power, as it (i) ignores dependencies among the connectivity elements; and (ii) involves quadratically increasing number of regressions that exacerbate the problem of multiple testing. On the other hand, multivariate methods such as principal component analysis (PCA) as considered in Crainiceanu and Punjabi (2011) consider the data from all ROIs at once, reducing the dimensionality of the original outcome to a smaller number of “networks” components, however, these common components may be associated with small eigenvalues, or the corresponding eigenvalues may not be associated with covariates.

The outcome data of interest are multivariate time-series resting-state fMRI (rs-fMRI) data in ${\mathbb{R}}^p$ measured simultaneously across the p ROIs (or parcels) defined based on an anatomical parcellation (Eickhoff et al. 2018) or “network nodes” (Smith et al. 2012) derived from a data-driven algorithm such as independent component analysis (ICA) (Calhoun et al. 2009; Smith et al. 2013). As in Seiler and Holmes (2017), we will apply the Bayesian CAP regression to data from the Human Connectome Project (HCP) (Van Essen et al. 2013) to compare short sleepers (ie $\le 6$ hours) with conventional sleepers (ie 7 to 9 hours) with respect to their FC.

2 Method

2.1 Covariance regression models

We consider n subjects, with subject-specific covariances for brain activity time series from p ROIs $\{{\mathbf{\Sigma }}_i\in {\mathbb{R}}^{p\times p},i=1,\dots ,n\}$. The space of valid covariance matrices ${\mathbf{\Sigma }}_i\in {\mathbb{R}}^{p\times p}$ is the space of symmetric positive definite (SPD) matrices, denoted as ${\text{Sym}}_p^{+}$ in this paper. The rs-fMRI time-series for a given subject i are drawn from a Gaussian distribution: ${\mathit{Y}}_{it}\sim N({\mathit{\mu }}_i,{\mathbf{\Sigma }}_i)$ with ${\mathbf{\mu }}_i\in {\mathbb{R}}^p$ and ${\mathbf{\Sigma }}_i\in {\text{Sym}}_p^{+}$. For centered data, the mean ${\mathit{\mu }}_i=0$, and the covariance ${\mathbf{\Sigma }}_i$ captures FC. Without loss of generality, we assume that the observed signal is mean-centered so that $\sum _{t=1}^{T_i}{\mathit{Y}}_{it}=0\in {\mathbb{R}}^p$ for each subject $(i=1,\dots ,n)$, as our focus is on FC characterized by the covariance between the brain signals. We observed ${\mathit{Y}}_{it}$ over T_i time points for each subject i $(i=1,\dots ,n)$ along with subject-level vectors of covariates ${\mathit{x}}_i\in {\mathbb{R}}^q(i=1,\dots ,n)$.

In this paper, instead of directly modeling the subject-specific covariances ${\mathbf{\Sigma }}_i=\text{cov}({\mathit{Y}}_{it})$ (as in Seiler and Holmes (2017); Fox and Dunson (2015); Zou et al. (2017)) in which most of the covariance heterogeneity may be unrelated with ${\mathit{x}}_i$, we aim to extract a lower dimensional component whose covariance heterogeneity is related with ${\mathit{x}}_i$. We will characterize this lower dimensional structure by a dimension reducing matrix $\mathbf{\Gamma }\in {\mathbb{R}}^{p\times d}$ where ${\mathbf{\Gamma }}^{\top }\mathbf{\Gamma }={\mathit{I}}_d$ (ie $\mathbf{\Gamma }$ is in a Stiefel manifold) with $d\le p$. Specifically, we consider a latent factor model for ${\mathit{Y}}_{it}$

$$\begin{array}{c}{\mathit{Y}}_{it}=\mathbf{\Gamma }{\mathbf{\Psi }}_i^{\frac{1}{2}}{\mathit{s}}_{it}+{\mathit{L}}_i{\mathit{ϵ}}_{it}\hfill \end{array}$$ (2.1)

with latent factors ${\mathit{s}}_{it}\sim N(0,{\mathit{I}}_d)$ and ${\mathit{ϵ}}_{it}\sim N(0,{\mathit{I}}_{p-d})$, of dimensions d and p—d, respectively, where

$${\mathbf{\Psi }}_i^{\frac{1}{2}}=\text{exp}\hspace{0.17em}\left(\text{diag}((\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i)/2)\right)$$ (2.2)

models the x-related heteroscedasticity along the projection directions $\mathbf{\Gamma }\in {\mathbb{R}}^{p\times d}$. In (2.2), $\text{diag}((\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i)/2)\in {\mathbb{R}}^{d\times d}$ is a diagonal matrix, where its diagonal elements are given by a linear predictor vector $(\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i)/2\in {\mathbb{R}}^d$. In (2.1), $\mathbf{\Gamma }\in {\mathbb{R}}^{p\times d}$ specifies the Principal Directions of Covariance (PDCs) of ${\mathit{Y}}_{it}$ related with ${\mathit{x}}_i$, whereas the other orthogonal components ${\mathit{L}}_i\in {\mathbb{R}}^{p\times (p-d)}$, which satisfy ${\mathit{L}}_i\perp \mathbf{\Gamma }$, are included to account for the “noise” directions and magnitudes of the heteroscedasticity that are unrelated with ${\mathit{x}}_i$.

In (2.2), the matrix $\mathit{B}=[{\mathit{\beta }}_0,\check{\mathit{B}}]\in {\mathbb{R}}^{d\times q}$ (where ${\mathit{\beta }}_0\in {\mathbb{R}}^d$ represents the intercept) is a regression coefficient matrix that relates ${\mathit{x}}_i\in {\mathbb{R}}^q$ (with its first element being 1) to the subject-level outcome covariance ${\mathbf{\Sigma }}_i$. Under model (2.1), the subject-level covariance is given by

$${\mathbf{\Sigma }}_i=\mathbf{\Gamma }{\mathbf{\Psi }}_i{\mathbf{\Gamma }}^{\top }+{\mathit{L}}_i{\mathit{L}}_i^{\top },$$ (2.3)

that decomposes the individual covariance matrices ${\mathbf{\Sigma }}_i$ into two components, covariates related and unrelated, a principal factor decomposition of ${\mathbf{\Sigma }}_i$. In (2.3), unlike the more general structure on ${\mathit{L}}_i{\mathit{L}}_i^{\top }$ whose variability is unrelated with ${\mathit{x}}_i$, the PDCs $\mathbf{\Gamma }$ serve as features (ie “subnetworks”) that we expect to be consistent across subjects. Along $\mathbf{\Gamma }$, model (2.2) incorporates subject-level random effects ${\mathit{z}}_i\sim N(0,\mathbf{\Omega })$ to capture additional heteroscedasticity not captured by ${\mathit{x}}_i$. In model (2.22), the diagonality of the d × d core tensor ${\mathbf{\Psi }}_i\in {\text{Sym}}_d^{+}$ is needed as an identifiability condition, since any non-diagonal SPD ${\tilde{\mathbf{\Psi }}}_i$ can be diagonalized by its normalized eigenvectors $\tilde{\mathit{A}}\in {\mathbb{R}}^{d\times d}$ (assuming common eigenvectors $\tilde{\mathbf{A}}$ for ${\tilde{\mathbf{\Psi }}}_i$ across subjects), and $\mathbf{\Gamma }\tilde{\mathit{A}}\in {\mathbb{R}}^{p\times d}$ can instead be used as the orthonormal dimension reduction matrix. While we impose the diagonality of ${\mathbf{\Psi }}_i$, we allow ${\mathit{z}}_i\sim N(0,\mathbf{\Omega })$, where $\mathbf{\Omega }$ may have off-diagonal elements that allow residual correlation in the the projected signals ${\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}\in {\mathbb{R}}^d$ beyond what is modeled by common covariates ${\mathit{x}}_i$.

Remark 2.1.

The covariance model (2.1) and (2.2) should be distinguished from the principal component (PC) regression that relates ${\mathit{x}}_i$ with the PCs ${\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}\in {\mathbb{R}}^d$, as our interest is in studying the association between the covariate ${\mathit{x}}_i$ with the variance of the components (ie heteroscedasticity), rather than with the components ${\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}\in {\mathbb{R}}^d$ themselves.

For a multivariate outcome signal ${\mathit{Y}}_{it}\in {\mathbb{R}}^p$Z${\mathbf{Y}}_{it}\in {\mathbb{R}}^p$ at time point t for subject i, Seiler and Holmes (2017) utilized a heteroscedesticity model, $\text{cov}({\mathit{Y}}_{it})=\mathit{B}{\mathit{x}}_i{\mathit{x}}_i^{\top }{\mathit{B}}^{\top }+{\sigma }^2{\mathit{I}}_p(t=1,\dots ,T_i)(i=1,\dots ,n)$, where the outcome covariance matrix ${\mathbf{\Sigma }}_i=\text{cov}({\mathit{Y}}_{it})$ is modeled by a quadratic function of $\mathit{B}{\mathit{x}}_i\in {\mathbb{R}}^p$, where $\mathit{B}\in {\mathbb{R}}^{p\times q}$ is the regression coefficient associated with ${\mathit{x}}_i\in {\mathbb{R}}^q$, and ${\sigma }^2>0$. However, this model is quite restrictive, as its outer product term $\mathbf{B}{\mathbf{x}}_i{\mathbf{x}}_i^{\top }{\mathbf{B}}^{\top }\in {\mathbb{R}}^{p\times p}$ is of rank 1, and the noise covariance term ${\sigma }^2{\mathit{I}}_p$ is diagonal with independent variances. On the other hand, model (2.3) identifies a covariate associated rank-d (where $d\ge 1)$ structure via $\mathbf{\Gamma }$ and allows a less restrictive noise covariance structure, which makes the covariance modeling with ${\mathit{x}}_i$ more flexible than that of Seiler and Holmes (2017). In particular, the outcome dimension reduction via $\mathbf{\Gamma }$ implicit in model (2.3) offers computational advantages through working with low dimensional (d-by-d) covariances (rather than full p-by-p covariances), that can be particularly advantageous when the number of within-subject time points (T_i) is relatively small compared to the signal dimension p. The general outer product approach proposed by Hoff and Niu (2012) replaces ${\sigma }^2{\mathit{I}}_p$ by a p × p SPD matrix, requiring a large number of parameters (that can scale quadratically in p). The approaches proposed in Fox and Dunson (2015); Zou et al. (2017) also similarly model the whole p × p matrix ${\mathbf{\Sigma }}_i$, which may make the interpretation challenging for large matrices (Zhao et al. 2021b).

Zhao et al. (2021b) considered CAP regression, $\text{var}({\mathit{\gamma }}^{(k)\top }{\mathit{Y}}_{it})=\text{exp}({\mathit{x}}_i^{\top }{\mathit{\beta }}^{(k)}),$ where the PDCs ${\mathit{\gamma }}^{(k)}\in {\mathbb{R}}^p(k=1,\dots ,d)$ are sequentially estimated subject to identifiability constraints ${\mathit{\gamma }}^{(k)\top }\overline{\mathbf{\Sigma }}{\mathit{\gamma }}^{(k)}=1$ (in which $\overline{\mathbf{\Sigma }}$ is a p × p covariance representative of the overall study population) and ${\mathit{\gamma }}^{(k)}\perp {\mathit{\gamma }}^{(k\prime )}$$(k\ne k\prime )$. However, under a sequential optimization framework, joint inference on the outcome projection matrix $\mathbf{\Gamma }=[{\mathit{\gamma }}^{(1)},\dots ,{\mathit{\gamma }}^{(d)}]\in {\mathbb{R}}^{p\times d}$ and the regression coefficient $\mathit{B}=[{\mathit{\beta }}^{(1)},\dots ,{\mathit{\beta }}^{(d)}]\in {\mathbb{R}}^{q\times d}$ is not straightforward, and thus, Zhao et al. (2021a,b, 2024) conducted bootstrap-based statistical inference only on the coefficients B, and not on $\mathbf{\Gamma }$. On the other hand, the proposed model (2.1), coupled with the core tensor model (2.2), further accounts for the additional heteroscedasticity in the projected outcomes by using subject-level random effets ${\mathit{z}}_i$ to relax the model assumption, while simultaneously modeling all the relevant parameters $(\mathbf{\Gamma },\mathbf{B},{\mathbf{\Psi }}_i,\mathbf{\Omega })$, allowing for more coherent downstream analysis that improves the model interpretability which we will discuss in Section 4.

2.2 Tangent space parametrization of dimension-reduced covariance

Due to the constraint ${\mathit{v}}^{\top }{\mathbf{\Sigma }}_i\mathit{v}\ge 0$ for all nonzero $\mathit{v}\in {\mathbb{R}}^p$, the space ${\text{Sym}}_p^{+}$ of covariance matrices $\left\{{\mathbf{\Sigma }}_i\right\}$ forms a curved manifold which does not conform to Euclidean geometry; for example, the negative of a SPD matrix and some linear combinations of SPD matrices are not SPD (Schwartzman 2016). Thus, analyzing ${\mathbf{\Sigma }}_i$ in the Euclidean vector space is not adequate to capture the curved nature of PDCs, and leads to a biased estimation of PDCs (Zhao et al. 2021b). However, ${\text{Sym}}_p^{+}$ is a Riemannian manifold under the affine-invariant Riemannian metric (AIRM) (Pennec et al. 2006), whose tangent space forms a vector space. We will use a Riemannian parametrization of SPD matrices in estimating the PDCs in this paper. A tangent space projection requires selection of a reference point that is close to ${\mathbf{\Sigma }}_i(i=1,\dots ,n)$ to be projected. A sensible reference point on ${\text{Sym}}_p^{+}$ is a mean of ${\mathbf{\Sigma }}_i(i=1,\dots ,n)$, denoted as $\overline{\mathbf{\Sigma }}\in {\text{Sym}}_p^{+}$. We will use the matrix whitening transport of Ng et al. (2016) to bring the covariances ${\mathbf{\Sigma }}_i(i=1,\dots ,n)$ close to ${\mathit{I}}_p$, by applying matrix whitening based on $\overline{\mathbf{\Sigma }}$. The resulting whitened covariances ${\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}{\mathbf{\Sigma }}_i{\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}$ would be close to the identity matrix ${\mathit{I}}_p$, at which we can construct a common tangent space for projection.

Remark 2.2.

Here we briefly review some relevant concepts of Riemannian geometry. Let $\mathit{A}\in {\text{Sym}}_p^{+}$, and $T_{\mathit{A}}({\text{Sym}}_p^{+})$ be the tangent space at A. Given two tangent vectors ${\mathit{X}}_1,{\mathit{X}}_2\in T_{\mathit{A}}({\text{Sym}}_p^{+})$ at A, the AIRM inner product is ${\langle {\mathit{X}}_1,{\mathit{X}}_2\rangle }_{\mathit{A}}=\text{tr}({\mathit{A}}^{-1}{\mathit{X}}_1{\mathit{A}}^{-1}{\mathit{X}}_2)$. Given $\mathit{X}\in T_{\mathit{A}}({\text{Sym}}_p^{+})$, there is a unique geodesic denoted as $\gamma (t)\in {\text{Sym}}_p^{+}$ such that $\gamma (0)=\mathit{A}$ and $\gamma \prime (0)=\mathit{X}$,

$$\gamma (t)={\hspace{0.17em}\text{Exp}\hspace{0.17em}}_{\mathit{A}}(t\mathit{X})={\mathit{A}}^{\frac{1}{2}}\hspace{0.17em}\text{exp}(t{\mathit{A}}^{-\frac{1}{2}}\mathit{X}{\mathit{A}}^{-\frac{1}{2}}){\mathit{A}}^{\frac{1}{2}}$$ (2.4)

that connects A to a point $\mathit{B}=\gamma (1)\in {\text{Sym}}_p^{+}$ when evaluated at t = 1. For $\mathit{X}\in T_{\mathit{A}}({\text{Sym}}_p^{+})$, the Exponential map, defined as ${\hspace{0.17em}\text{Exp}\hspace{0.17em}}_{\mathit{A}}(\mathit{X}):=\gamma (1)\in {\text{Sym}}_p^{+}$, projects the given X to a point $\mathit{B}\in {\text{Sym}}_p^{+}$, in such a way that the A and X distance on the tangent plane is the same as that between A and B on the manifold. The (AIRM) Log map, which is the inverse mapping of ${\hspace{0.17em}\text{Exp}\hspace{0.17em}}_{\mathit{A}}(\mathit{X})$, projects the point $\mathit{B}\in {\text{Sym}}_p^{+}$ back to the tangent vector,

$$\mathit{X}={\hspace{0.17em}\text{Log}\hspace{0.17em}}_{\mathit{A}}(\mathit{B})={\mathit{A}}^{\frac{1}{2}}\hspace{0.17em}\text{log}({\mathit{A}}^{-\frac{1}{2}}\mathit{B}{\mathit{A}}^{-\frac{1}{2}}){\mathit{A}}^{\frac{1}{2}}\hspace{0.17em}\in \hspace{0.17em}{T}_{\mathit{A}}({\text{Sym}}_p^{+}),$$ (2.5)

and we can re-express the geodesic (2.4) as $\gamma (t)={\hspace{0.17em}\text{Exp}\hspace{0.17em}}_{\mathit{A}}(t\hspace{0.17em}{\hspace{0.17em}\text{Log}\hspace{0.17em}}_{\mathit{A}}(\mathit{B})),t\in [0,1]$. The corresponding geodesic distance between A and B is $d(\mathit{A},\mathit{B})={\langle {\hspace{0.17em}\text{Log}\hspace{0.17em}}_{\mathit{A}}(\mathit{B}),{\hspace{0.17em}\text{Log}\hspace{0.17em}}_{\mathit{A}}(\mathit{B})\rangle }_{\mathit{A}}^{\frac{1}{2}}=\left|\right|\hspace{0.17em}\text{log}({\mathit{A}}^{-\frac{1}{2}}\mathit{B}{\mathit{A}}^{-\frac{1}{2}})|{|}_F,$ where $\left|\right|·|{|}_F$ is the Frobenius norm.

In this paper, for each dimension reducing matrix $\mathbf{\Gamma }\in {\mathbb{R}}^{p\times d}$, we will use $\overline{\mathbf{\Psi }}:={\mathbf{\Gamma }}^{\top }\overline{\mathbf{\Sigma }}\mathbf{\Gamma }\in {\text{Sym}}_d^{+}$, where $\overline{\mathbf{\Sigma }}$ is a fixed representative population level covariance, to “whiten” the individual level dimension-reduced covariances ${\mathbf{\Psi }}_i={\mathbf{\Gamma }}^{\top }{\mathbf{\Sigma }}_i\mathbf{\Gamma }\in {\text{Sym}}_d^{+}(i=1,\dots ,n)$ of model (2.3). Specifically, we will normalize ${\mathbf{\Psi }}_i$ by ${\overline{\mathbf{\Psi }}}^{-\frac{1}{2}}$ (where ${\overline{\mathbf{\Psi }}}^{-\frac{1}{2}}$ is computed based on the eigendecomposition of $\overline{\mathbf{\Psi }}={\mathbf{\Gamma }}^{\top }\overline{\mathbf{\Sigma }}\mathbf{\Gamma }$), so that the resulting individual “whitened” SPD ${\mathbf{\Psi }}_i^{\ast }:={\overline{\mathbf{\Psi }}}^{-\frac{1}{2}}{\mathbf{\Psi }}_i{\overline{\mathbf{\Psi }}}^{-\frac{1}{2}}={\mathbf{\Gamma }}^{\top }{\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}{\mathbf{\Sigma }}_i{\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}\mathbf{\Gamma }$$(i=1,\dots ,n)$ is close to the identity matrix ${\mathit{I}}_d$. We will parametrize these ${\mathbf{\Psi }}_i^{\ast }(i=1,\dots ,n)$ in the tangent space at ${\mathit{I}}_d$, by projecting ${\mathbf{\Psi }}_i^{\ast }$ at ${\mathit{I}}_d$ using the Log map,

$${\hspace{0.17em}\text{Log}\hspace{0.17em}}_{{\mathit{I}}_d}({\mathbf{\Psi }}_i^{\ast })=\text{log}({\mathbf{\Psi }}_i^{\ast })=\text{log}({\overline{\mathbf{\Psi }}}^{-\frac{1}{2}}{\mathbf{\Psi }}_i{\overline{\mathbf{\Psi }}}^{-\frac{1}{2}})\hspace{1em}(={\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)),$$ (2.6)

locally mapping the bipoint $\overline{\mathbf{\Psi }},{\mathbf{\Psi }}_i\in {\text{Sym}}_d^{+}\times {\text{Sym}}_d^{+}$ to an element in the tangent space at ${\mathit{I}}_d$. For notational convenience, in (2.6) let us denote the Log map, $\text{log}({\overline{\mathbf{\Psi }}}^{-\frac{1}{2}}{\mathbf{\Psi }}_i{\overline{\mathbf{\Psi }}}^{-\frac{1}{2}})$ given $\overline{\mathbf{\Psi }}$, as ${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)\in {\mathbb{R}}^{d\times d}$, which is no longer linked by the positive definiteness constraint (Pervaiz et al. 2020) and forms a vector space. Then, treating ${\mathbf{\Psi }}_i$ as a local perturbation of $\overline{\mathbf{\Psi }}$ in tangent space, we model ${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)$ in (2.6) by a linear model of the form,

$${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)=\text{diag}(\tilde{\mathit{B}}{\mathit{x}}_i+{\tilde{\mathit{z}}}_i)$$ (2.7)

where the linear predictor $\tilde{\mathit{B}}{\mathit{x}}_i+{\tilde{\mathit{z}}}_i\in {\mathbb{R}}^d$ lies in (unrestricted) Euclidean vector space. Upon parametrizing ${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)$ (with appropriate priors on $\tilde{\mathit{B}}$ and ${\tilde{\mathit{z}}}_i\sim N(0,\mathbf{\Omega })$), we will re-map these covariate-parametrized objects ${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)$ in (2.7) to the original space in ${\text{Sym}}_d^{+}$, by first taking Exponential map, $\text{Exp}({\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i))=\text{exp}({\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i))$ (ie taking (2.4) at t = 1 and $\mathit{A}={\mathit{I}}_d$) and then translating it back to the base point ${\mathbf{\Gamma }}^{\top }\overline{\mathbf{\Sigma }}\mathbf{\Gamma }$ through “de-whitening” with $\overline{\mathbf{\Psi }}={\mathbf{\Gamma }}^{\top }\overline{\mathbf{\Sigma }}\mathbf{\Gamma }$, yielding

$${\mathbf{\Psi }}_i=\text{exp}({\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)){\mathbf{\Gamma }}^{\top }\overline{\mathbf{\Sigma }}\mathbf{\Gamma }$$ (2.8)

which completes our parameterization of the core tensor ${\mathbf{\Psi }}_i$ in (2.3). To define the mapping (2.6), we select $\overline{\mathbf{\Sigma }}$ to represent an estimate of the Euclidean average of ${\mathbf{\Sigma }}_i$. Among examined estimators in previous works (Dadi et al. 2019; Pervaiz et al. 2020) this choice of $\overline{\mathbf{\Sigma }}$ showed stable performance across various scenarios. We set $\overline{\mathbf{\Sigma }}=\frac{1}{n}\sum _{i=1}^n{\widehat{\mathbf{\Sigma }}}_i$, where ${\widehat{\mathbf{\Sigma }}}_i=\frac{1}{T_i}\sum _{l=1}^{T_i}{\mathit{Y}}_{it}{\mathit{Y}}_{it}^{\top }$.

2.3 Posterior inference

2.3.1 Prior and likelihood specification

We perform posterior inference on the tangent space parameterized model (2.7), which will be mapped to parametrization (2.2). Let $\mathcal{D}$ represent the observed data and $\mathbf{\Psi }$ denote the collection ${\left\{{\mathbf{\Psi }}_i\right\}}_{i=1}^n$, and let ${\mathit{Y}}_i=\{{\mathit{Y}}_{i1},\dots ,{\mathit{Y}}_{i{T}_i}\}$. The posterior of parameters $(\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })$ can be expressed as the the product of a prior and the likelihood,

$$p(\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega }|\mathcal{D})\hspace{1em}\propto \hspace{1em}p(\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })\hspace{0.17em}\prod _{i=1}^{n}p\left({\mathit{Y}}_i|\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega }\right).$$ (2.9)

The covariate relevant component likelihood for subject i under (2.1) is

$$\begin{array}{cc}\hfill & p({\mathit{Y}}_i|\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })\hfill \\ \hfill & \hspace{1em}\propto \hspace{0.17em}|\mathbf{\Gamma }{\mathbf{\Psi }}_i{\mathbf{\Gamma }}^{\top }+{\mathit{L}}_i{\mathit{L}}_i^{\top }{|}^{-T_i/2}\hspace{0.17em}\text{exp}(-\frac{1}{2}\sum _{t=1}^{T_i}{\mathit{Y}}_{it}^{\top }(\mathbf{\Gamma }{\mathbf{\Psi }}_i^{-1}{\mathbf{\Gamma }}^{\top }+{\mathit{L}}_i{({\mathit{L}}_i^{\top }{\mathit{L}}_i)}^{-1}{\mathit{L}}_i^{\top }){\mathit{Y}}_{it})\hfill \\ \hfill & \hspace{1em}\propto \hspace{0.17em}|{\mathbf{\Psi }}_i{|}^{-T_i/2}\hspace{0.17em}\text{exp}(-\frac{1}{2}\sum _{t=1}^{T_i}{\mathit{Y}}_{it}^{\top }\mathbf{\Gamma }{\mathbf{\Psi }}_i^{-1}{\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it})\hspace{0.17em}|{\Xi }_i{|}^{-T_i/2}\hspace{0.17em}\text{exp}(-\frac{1}{2}\sum _{t=1}^{T_i}{\mathit{Y}}_{it}^{\top }{\mathit{L}}_i{\Xi }_i^{-1}{\mathit{L}}_i^{\top }{\mathit{Y}}_{it})\hfill \\ \hfill & \hspace{1em}\propto \hspace{0.17em}|{\mathbf{\Psi }}_i{|}^{-T_i/2}\hspace{0.17em}\text{exp}(-\frac{1}{2}\sum _{t=1}^{T_i}\text{tr}({\mathit{Y}}_{it}^{\top }\mathbf{\Gamma }{\mathbf{\Psi }}_i^{-1}{\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}))\hfill \\ \hfill & \hspace{1em}\propto \hspace{0.17em}|\hspace{0.17em}\text{exp}({\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)){|}^{-T_i/2}\hspace{0.17em}\text{exp}(-\frac{1}{2}\sum _{t=1}^{T_i}\text{tr}({\mathit{Y}}_{it}{\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}\mathbf{\Gamma }{\left(\text{exp}({\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i))\right)}^{-1}{\mathbf{\Gamma }}^{\top }{\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}{\mathit{Y}}_{it}^{\top }))\hfill \end{array}$$ (2.10)

where the last line follows from the tangent-space parametrization (2.8) of ${\mathbf{\Psi }}_i$. Equation (2.10) indicates that the likelihood is in the form of a Gaussian likelihood of transformed responses,

$${\mathbf{\Gamma }}^{\top }{\overline{\mathbf{\Sigma }}}^{-\frac{1}{2}}{\mathit{Y}}_{it}\sim N(0,\hspace{0.17em}\hspace{0.17em}\text{exp}({\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i))=N(0,\hspace{0.17em}\hspace{0.17em}\text{exp}(\text{diag}(\tilde{\mathit{B}}{\mathit{x}}_i+{\tilde{\mathit{z}}}_i))$$ (2.11)

and no attempt will be made to estimate the parameters ${\mathit{L}}_i$ in (2.1) unrelated with ${\mathit{x}}_i$.

We specify the prior $p(\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })=p(\mathbf{\Gamma },\tilde{\mathit{B}},\mathbf{\Omega })p(\mathbf{\Psi }|\mathbf{\Gamma },\tilde{\mathit{B}},\mathbf{\Omega })$ in (2.9) as

$$\begin{array}{c}\propto \hspace{0.17em}p(\mathbf{\Gamma })p(\tilde{\mathit{B}})p(\mathbf{\Omega })\hspace{0.17em}\text{exp}\hspace{0.17em}\left\{-\frac{1}{2}\sum _{i=1}^n({\overrightarrow{\varphi }}_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)-\tilde{\mathit{B}}{\mathit{x}}_i{)}^{\top }{\mathbf{\Omega }}^{-1}({\overrightarrow{\varphi }}_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)-\tilde{\mathit{B}}{\mathit{x}}_i)-\frac{n}{2}\text{log}\hspace{0.17em}|\mathbf{\Omega }|\right\},\hfill \end{array}$$ (2.12)

using independent priors $p(\mathbf{\Gamma },\tilde{\mathit{B}},\mathbf{\Omega })=p(\mathbf{\Gamma })p(\tilde{\mathit{B}})p(\mathbf{\Omega })$ and a conditional prior on $\mathbf{\Psi }={\left\{{\mathbf{\Psi }}_i\right\}}_{i=1}^n$ given $(\mathbf{\Gamma },\tilde{\mathit{B}},\mathbf{\Omega })$ based on ${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)=\text{diag}(\tilde{\mathit{B}}{\mathit{x}}_i+{\tilde{\mathit{z}}}_i)$. In (2.12), ${\overrightarrow{\varphi }}_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)\in {\mathbb{R}}^d$ denotes the vector of the diagonal elements of ${\varphi }_{\overline{\mathbf{\Psi }}}({\mathbf{\Psi }}_i)\in {\mathbb{R}}^{d\times d}$. For $\tilde{\mathit{B}}\in {\mathbb{R}}^{d\times q}$, we use a mean zero matrix Gaussian prior with element-wise standard deviation ${\sigma }_{{\tilde{B}}_{jk}}>0$. For $\mathbf{\Omega }\in {\text{Sym}}_d^{+}$, which we decompose into $\text{diag}(\mathit{\omega })\tilde{\mathbf{\Omega }}\text{diag}(\mathit{\omega })$, we use an unit-scale half-Cauchy distribution (Gelman 2006; Polson and Scott 2012) on each element of the standard deviation vector $\mathbf{\omega }\in {\mathbb{R}}^d$ (allowing for the possibility of extreme values) and a Lewandowski-Kurowicka-Joe (LKJ) prior (Lewandowski et al. 2009) on the correlation matrix $\tilde{\mathbf{\Omega }}$ with hyperparameter $\eta >0$ (specifying the amount of expected prior correlations). For $\mathbf{\Gamma }\in {\mathbb{R}}^{p\times d}$, we use a matrix angular central Gaussian (MACG) (Chikuse 1990; Jupp and Mardia 1999) with hyperparameter $\mathbf{\Phi }\in {\text{Sym}}_p^{+}$. An orthonormal random matrix $\mathbf{\Gamma }$ is said to be distributed as a MACG (with parameter $\mathbf{\Phi }$) if $\mathbf{\Gamma }\stackrel{d}{=}\mathit{U}{({\mathit{U}}^{\top }\mathit{U})}^{-1/2}$, where $\mathit{U}\in {\mathbb{R}}^{p\times d}$ follows a p × d matrix normal distribution, whose density is

$$f_{\mathit{U}}(\mathit{U})={(2\pi )}^{-pd/2}|\mathbf{\Phi }{|}^{-d/2}\hspace{0.17em}\text{exp}(\text{tr}(-{\mathit{U}}^{\top }{\mathbf{\Phi }}^{-1}\mathit{U}/2)).$$ (2.13)

If the row covariance $\mathbf{\Phi }={\mathit{I}}_p$, then the prior on U encodes no spatial information. In our illustrations, we employed flat priors on $\mathbf{\Gamma }$ and the correlation matrix $\tilde{\mathbf{\Omega }}$ (with $\mathbf{\Phi }={\mathit{I}}_p$ and η = 1, respectively), and weakly informative priors on $\tilde{\mathit{B}}$, using ${\sigma }_{{\tilde{B}}_{jk}}^2={2.5}^2$.

2.3.2 Posterior computation via polar expansion

A Markov chain Monte Carlo (MCMC) sampling for $\mathbf{\Gamma }$ from the posterior (2.9) is challenging due to the restriction that $\mathbf{\Gamma }$ is in a Stiefel manifold. We will use polar expansion to transform the orthonormal parameter $\mathbf{\Gamma }$ to an unconstrained object $(\mathit{U})$ to work around this restriction. Generally, “parameter expansion” of a statistical model refers to methods which expand the parameter space by introducing redundant working parameters for computational purposes (Jauch et al. 2021). By polar decomposition (Higham 1986), any arbitrary matrix $\mathit{U}\in {\mathbb{R}}^{p\times d}$ can be decomposed into two components,

$$\mathit{U}={\mathbf{\Gamma }}_{\mathit{U}}{\mathit{S}}_{\mathit{U}},$$ (2.14)

where the first component ${\mathbf{\Gamma }}_{\mathit{U}}:=\mathit{U}{({\mathit{U}}^{\top }\mathit{U})}^{-1/2}\in {\mathbb{R}}^{p\times d}$ is an orthonormal (rotation) matrix, and the second ${\mathit{S}}_{\mathit{U}}:={({\mathit{U}}^{\top }\mathit{U})}^{1/2}\in {\mathbb{R}}^{d\times d}$ is a symmetric nonnegative (stretch tensor) matrix.

Using a MACG prior on $\mathbf{\Gamma }$ with prior on U in (2.13) allows for posterior inference on U (rather than directly on $\mathbf{\Gamma }$). By employing the polar expansion of ${\mathbf{\Gamma }}_{\mathit{U}}$ to U in (2.14), we “parameter expand” an orthonormal ${\mathbf{\Gamma }}_{\mathit{U}}$ to an unconstrained U. This expanded parameter maintains the same model likelihood $p(\mathcal{D}|{\mathbf{\Gamma }}_{\mathit{U}},\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })$ as in (2.10). However, the prior $p({\mathbf{\Gamma }}_{\mathit{U}},\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })$ in (2.12) expands to $p(\mathit{U},\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega })$ under parametrization (2.14), leading to the corresponding posterior expansion from $p({\mathbf{\Gamma }}_{\mathit{U}},\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega }|\mathcal{D})$ in (2.9) to $p(\mathit{U},\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega }|\mathcal{D})$. Using MCMC, we first approximate samples from the expanded posterior $p(\mathbf{U},\mathbf{\Psi },\tilde{\mathbf{B}},\mathbf{\Omega }|\mathcal{D})$, then conduct the polar decomposition (2.14) to obtain the samples from the posterior of ${\mathbf{\Gamma }}_{\mathit{U}}$, which can be verified via a change of variable from U to ${\mathbf{\Gamma }}_{\mathit{U}}$. Specifically, given a Markov chain $\{{\mathit{U}}_s,{\mathbf{\Psi }}_s,{\tilde{\mathit{B}}}_s,{\mathbf{\Omega }}_s\}$ with a stationary distribution proportional to $p(\mathit{U},\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega }|\mathcal{D})$, we approximate the posterior of $\mathbf{\Gamma }$ by $\left\{{\mathbf{\Gamma }}_s\right\}$ where ${\mathbf{\Gamma }}_s={\mathit{U}}_s{({\mathit{U}}_s^{\top }{\mathit{U}}_s)}^{-1/2}$ for each s, yielding approximate samples from $p(\mathbf{\Gamma },\mathbf{\Psi },\tilde{\mathit{B}},\mathbf{\Omega }|\mathcal{D})$.

In this paper, approximate the posterior distribution of parameters $(\mathbf{U},\mathbf{\Psi },\tilde{\mathbf{B}},\mathbf{\Omega })$ using an adaptive Hamiltonian Monte Carlo (HMC) sampler (Neal 2011) with automatic differentiation and adaptive tuning, implemented in Stan (2023). Consequently, we obtain HMC posterior samples of $(\mathbf{\Gamma },\mathbf{\Psi },\mathit{B},\mathbf{\Omega })$. The mapping between B and $\tilde{\mathit{B}}$ is given in Supplementary Materials S1. As in any PCA-type analysis, there is a sign non-identifiability of $\mathbf{\Gamma }$; the non-identifiability of matrix $\mathbf{\Gamma }$ up to random sign changes for each component. That is, the component vector ${\mathit{\gamma }}^{(k)}$ and $-{\mathit{\gamma }}^{(k)}$ correspond to the same direction. We can align the posterior samples $\left\{{\mathit{\gamma }}_s^{(k)}\right\}$. For the first post-warmup sample ${\mathit{\gamma }}_1^{(k)}$, let $j_1=\text{arg}{\text{max}}_j(|{\gamma }_{j,1}^{(k)}|)$. For $s\ge 2$, we compared the sign of ${\gamma }_{j_1,s}^{(k)}$ with that of ${\gamma }_{j_1,1}^{(k)}$, and if the signs disagreed, we multiplied ${\mathit{\gamma }}_s^{(k)}$ by –1. The aligned ${\mathit{\gamma }}_s^{(k)}$’s were used to construct the credible intervals of ${\mathit{\gamma }}^{(k)}$. In Sections 3 and 4, we employed a burn-in of 700 steps, during which Stan optimizes tuning parameters for the HMC sampler. After burn-in, we ran HMC for an additional 1300 steps to generate 1300 post-warmup samples. Convergence was assessed by examining traceplots of random parameter subsets.

Unlike ICA, where the order of the extracted components is relatively arbitrary, the components ${\mathit{\gamma }}^{(k)\top }{\mathit{Y}}_{it}(k=1,\dots ,d)$ in (2.1) specified by $\mathbf{\Gamma }=[{\mathit{\gamma }}^{(1)},\dots ,{\mathit{\gamma }}^{(d)}]\in {\mathbb{R}}^{p\times d}$ can be ranked based on the sample variance of the expected log-variance $E[\text{log}\hspace{0.17em}{\check{\Psi }}_i^{(k)}|\mathcal{D}]$ they explain across observations $i=1,\dots ,n$, where $\text{log}\hspace{0.17em}{\check{\Psi }}_i^{(k)}={\mathit{x}}_i^{\top }{\mathit{\beta }}^{(k)}(k=1,\dots ,d)$; here we exclude subject-level random effects $z_i^{(k)}\in \mathbb{R}$ to quantify only covariate-associated heteroscedasticity. Specifically, we sort the d estimated components in decreasing order of the magnitude of the sample variance $V^{(k)}=\sum _{i=1}^n{\left\{E[\text{log}\hspace{0.17em}{\check{\Psi }}_i^{(k)}|\mathcal{D}]-\frac{1}{n}\sum _{i=1}^{n}E[\text{log}\hspace{0.17em}{\check{\Psi }}_i^{(k)}|\mathcal{D}]\right\}}^2(k=1,\dots ,d)$ of the expected log-variance $E[\text{log}\hspace{0.17em}{\check{\Psi }}_i^{(k)}|\mathcal{D}]$ attributable to ${\mathit{x}}_i$.

2.3.3 Determination of the number d of the components

We propose to use a selection criterion based on the Watanabe-Akaike Information Criterion (WAIC) (Watanabe 2010) which can be used to estimate the expected log posterior. Given a fixed d, we compute the log pointwise predictive density (LPPD) of the dimension reduced model, penalized by the WAIC effective degrees of freedom, ${\widehat{r}}_{\mathit{waic}}$ (e.g. Gelman et al. (2014)). Specifically, we select the dimensionality d of the covariate-assisted outcome projection, which maximizes the expected deviance between two models in the projected outcome space: one incorporating covariate-explained heteroscedasticity ${\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}\sim N(0,{\check{\mathbf{\Psi }}}_i=\text{exp}(\text{diag}(\mathit{B}{\mathit{x}}_i)))$, and the other without heteroscedasticity ${\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}\sim N(0,\overline{\mathbf{\Psi }}={\mathbf{\Gamma }}^{\top }\overline{\mathbf{\Sigma }}\mathbf{\Gamma })$. The expected deviance (scaled by –2) is estimated by

$$-2\hspace{0.17em}\sum _{i=1}^n\sum _{t=1}^{T_i}\frac{1}{S}\sum _{s=1}^S\hspace{0.17em}\text{log}\hspace{0.17em}{R}^{(s)}+2{\widehat{r}}_{\mathit{waic}},$$ (2.15)

where ${\widehat{r}}_{\mathit{waic}}=\sum _{i=1}^n\sum _{t=1}^{T_i}\frac{1}{S}\sum _{s=1}^S{\left(\text{log}\hspace{0.17em}{R}^{(s)}-\frac{1}{S}\sum _{s=1}^S\hspace{0.17em}\text{log}\hspace{0.17em}{R}^{(s)}\right)}^2,$ in which $R^{(s)}=\frac{p({\mathbf{\Gamma }}^{(s)\top }{\mathit{Y}}_{it}|{\check{\mathbf{\Psi }}}_i^{(s)})}{p({\mathbf{\Gamma }}^{(s)\top }{\mathit{Y}}_{it}|{\overline{\mathbf{\Psi }}}^{(s)})}$, ie the posterior ratio of the two models with vs. without covariate-explained heteroscedasticity, computed using the MCMC posterior parameter samples $(s=1,\dots ,S)$. If the covariates ${\mathit{x}}_i$ are predictive of the covariances $\mathbf{\Gamma }{\mathbf{\Sigma }}_i\mathbf{\Gamma }$ along all PDCs $\mathbf{\Gamma }\in {\mathbb{R}}^{p\times d}$ of rank d, then the corresponding expected log posterior, $E\left[\text{log}\hspace{0.17em}p({\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}|{\check{\mathbf{\Psi }}}_i)\right]$, will be large. However, for a too large rank d, the covariates may not predict the covariances $\mathbf{\Gamma }{\mathbf{\Sigma }}_i\mathbf{\Gamma }$ in all posited directions $\mathbf{\Gamma }$, leading to a smaller expected log posterior ratio, $E\left[\text{log}\hspace{0.17em}\frac{p({\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}|{\check{\mathbf{\Psi }}}_i)}{p({\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}|\overline{\mathbf{\Psi }})}\right]=E\left[\text{log}\hspace{0.17em}p({\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}|{\check{\mathbf{\Psi }}}_i)\right]-E\left[\text{log}\hspace{0.17em}p({\mathbf{\Gamma }}^{\top }{\mathit{Y}}_{it}|\overline{\mathbf{\Psi }})\right]$, compared to that with the optimal projected outcome dimension d. Considering the ratio is crucial for making this criterion comparable across different d’s, and we select d that minimizes this expected deviance. In Supplementary Materials S2, we demonstrate the validity of this criterion in selecting the correct number of covariate-relevant heteroscedasticity components.

3 Simulation illustration

3.1 Simulation setup

For each unit (subject) i, we simulate a set of outcome signals ${\mathit{Y}}_{it}\in {\mathbb{R}}^p(t=1,\dots ,T_i)$$(i=1,\dots ,n)$ from a Gaussian distribution with mean zero and p × p unit-specific covariance ${\mathbf{\Sigma }}_i$. We vary $n\in \{100,200,300,400\},\hspace{0.17em}{T}_i\in \{10,20,30\}$, and $p\in \{10,20\}$. We use model (2.3) to generate ${\mathbf{\Sigma }}_i\in {\mathbb{R}}^{p\times p}$, where the core SPD ${\mathbf{\Psi }}_i=\text{exp}(\text{diag}(\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i))\in {\text{Sym}}_d^{+}$ with d = 2, where ${\mathit{x}}_i={(1,x_{i1},x_{i2},x_{i3},x_{i4})}^{\top }\in {\mathbb{R}}^q$, is defined based on the subject-level linear predictors $\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i$,

$$\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i=\left(\begin{array}{ccccc}0.1& 0.4& -0.5& 0.5& -0.5\\ 0.1& -0.3& 0.4& -0.4& 0.4\end{array}\right)\left(\begin{array}{c}1\\ x_{i1}\\ x_{i2}\\ x_{i3}\\ x_{i4}\end{array}\right)+{\mathit{z}}_i={\mathit{\beta }}_0+\left(\begin{array}{c}(x_{i1}\hspace{0.17em}{x}_{i2}\hspace{0.17em}{x}_{i3}\hspace{0.17em}{x}_{i4}){\mathit{\beta }}^{(1)}\\ (x_{i1}\hspace{0.17em}{x}_{i2}\hspace{0.17em}{x}_{i3}\hspace{0.17em}{x}_{i4}){\mathit{\beta }}^{(2)}\end{array}\right)+{\mathit{z}}_i$$

of dimension d = 2, where ${\mathit{\beta }}_0={(0.1,0.1)}^{\top }\in {\mathbb{R}}^2$ is the intercept vector, ${\mathit{\beta }}^{(1)}={(0.4,-0.5,0.5,-0.5)}^{\top }$ and ${\mathbf{\beta }}^{(2)}={(-0.3,0.4,-0.4,0.4)}^{\top }$ are the regression coefficients for ${(x_{i1},x_{i2},x_{i3},x_{i4})}^{\top }\in {\mathbb{R}}^{q-1}$. We generate covariates $x_{i1}\stackrel{\mathit{iid}}{\sim }\text{Bernoulli}(0.5)$ and $x_{i2},x_{i3},x_{i4}\stackrel{\mathit{iid}}{\sim }N(0,1^2)$, and the subject-specific random effects ${\mathit{z}}_i\stackrel{\mathit{iid}}{\sim }N(0,\mathbf{\Omega })$, where $\mathbf{\Omega }=\left(\begin{array}{cc}{\omega }_{11}& {\omega }_{12}\\ {\omega }_{12}& {\omega }_{22}\end{array}\right)=\left(\begin{array}{cc}{0.5}^2& 0.1\\ 0.1& {0.5}^2\end{array}\right)$, to define ${\mathbf{\Psi }}_i$.

For each simulation run, we use the von Mises–Fisher distribution to randomly generate an orthonormal basis matrix $[\mathbf{\Gamma },\mathit{L}]\in {\mathbb{R}}^{p\times p}$ for ${\mathit{Y}}_{it}\in {\mathbb{R}}^p$, and its subcomponent $\mathit{L}\in {\mathbb{R}}^{p\times (p-d)}$ is further transformed by subject-specific orthonormal matrices ${\mathit{A}}_i\in {\mathbb{R}}^{(p-d)\times (p-d)}$, each randomly generated from the von Mises–Fisher distribution. Then, the “noise” covariance components ${\mathit{L}}_i{\mathit{L}}_i^{\top }=\mathit{L}{\mathit{A}}_i\hspace{0.17em}\text{exp}(\text{diag}({\mathit{ϵ}}_i)){\mathit{A}}_i^{\top }{\mathit{L}}^{\top }\in {\mathbb{R}}^{p\times p}$ are specified by generating ${\mathit{ϵ}}_i\in {\mathbb{R}}^{p-d}$ with each element ${ϵ}_{ij}\stackrel{\mathit{iid}}{\sim }N(0,{0.5}^2)$, whereas $\mathbf{\Gamma }\hspace{0.17em}\text{exp}(\text{diag}(\mathit{B}{\mathit{x}}_i+{\mathit{z}}_i)){\mathbf{\Gamma }}^{\top }\in {\mathbb{R}}^{p\times p}$ specify the “signal” components. For each simulation run, we compute the base covariance $\overline{\mathbf{\Sigma }}$ that we use for tangent-space parametrization of model (2.3) as the sample marginal covariance on the training sample.

To investigate the robustness of the method against model misspecification, we further consider the case where there are no common eigenvectors $\mathbf{\Gamma }$ across subjects. We consider subject-level random perturbation using the subject-level rotation matrices $\mathit{R}({\theta }_i)=\left(\begin{array}{cc}\hspace{0.17em}\text{cos}({\theta }_i)& -\text{sin}({\theta }_i)\\ \hspace{0.17em}\text{sin}({\theta }_i)& \hspace{0.17em}\text{cos}({\theta }_i)\end{array}\right)$ with random angles ${\theta }_i\stackrel{\mathit{iid}}{\sim }\text{Unif}[-\pi /10,\pi /10](i=1,\dots ,n)$, and use $\mathbf{\Gamma }\mathit{R}({\theta }_i)\in {\mathbb{R}}^{p\times d}(i=1,\dots ,n)$ in place of $\mathbf{\Gamma }$ in generating the responses in (2.1), referred to as “model misspecification” cases.

3.2 Evaluation metric

We run the simulation 50 times. For each simulation run, we compute, as evaluation metrics, the absolute cosine similarity $1-|\langle {\widehat{\mathit{\gamma }}}^{(k)},{\mathit{\gamma }}^{(k)}\rangle |$ for the loading coefficient vectors (where a value close to 0 indicates the proximity) and the root mean squared error (RMSE) $\left|\right|{\widehat{\mathit{\beta }}}^{(k)}-{\mathit{\beta }}^{(k)}\left|\right|/\sqrt{4}$ (k = 1, 2) for the regression coefficient vectors, as well as the RMSE for the elements of the random effect covariance matrix $\mathbf{\Omega },\hspace{0.17em}\left|\right|{({\widehat{\omega }}_{11},{\widehat{\omega }}_{12},{\widehat{\omega }}_{22})}^{\top }-{({\omega }_{11},{\omega }_{12},{\omega }_{22})}^{\top }\left|\right|/\sqrt{3}$, where the notation $\widehat{·}$ represents the posterior mean of $·$. While we conduct the model estimation using the tangent space parameterization (2.7) with $\tilde{\mathit{B}}$, the results are mapped to the original parametrization with B in (2.3). This approximately amounts to shifting the intercept vector ${\mathit{\beta }}_0:={({\beta }_0^{(1)},{\beta }_0^{(2)})}^{\top }\in {\mathbb{R}}^2$ by the diagonal elements of $\text{log}({\mathbf{\Gamma }}^{\top }{\overline{\mathbf{\Sigma }}}^{-1}\mathbf{\Gamma })\in {\mathbb{R}}^{2\times 2}$ (see Supplementary Materials S1). We report the estimation performance for ${\mathit{\beta }}_0$ by reporting RMSE $\left|\right|{\widehat{\mathit{\beta }}}_0-{\mathit{\beta }}_0\left|\right|/\sqrt{2}$, under the original parametrization with B. Additionally, to assess whether the constructed credible intervals provide reasonably correct coverage for the true values of the parameters, we evaluate the posterior credible intervals of the model parameters (${\mathit{\gamma }}^{(k)},\hspace{0.17em}{\mathit{\beta }}^{(k)},\hspace{0.17em}\mathbf{\Omega }$) with respect to the frequentist’s coverage proportion. Specifically, for each simulation run, we estimate the posterior distribution of the parameters and calculate the 95% posterior credible intervals for the parameters, and then evaluate how often the credible intervals contain the true parameter values. We used a random initialization of the Markov chains in our posterior sampling.

3.3 Simulation results

In Fig. 1, as sample sizes ($n,T)$ increase, the estimation performance tends to improve overall. Particularly when the sample sizes are relatively small (e.g. $n=100,T=10$), the improvement tends to depends on the magnitude of the covariate effects on the outcome projection component, as performance for parameters for the first component (${\mathit{\gamma }}^{(1)}$ and ${\mathit{\beta }}^{(1)}$) tends to be slightly better than those for the second components (${\mathit{\gamma }}^{(2)}$ and ${\mathit{\beta }}^{(2)}$), reflecting stronger covariate effects on the first projection component. The number of subjects (n) and time points (T) both influence performance; increasing T enhances estimation by providing more subject-level information for accurate estimates of subject-specific random effects and their covariance $\mathbf{\Omega }$, and accordingly population-level parameters ${\mathit{\gamma }}^{(k)}$ and ${\mathit{\beta }}^{(k)}$. The p = 10 cases reported in Supplementary Materials S3 show qualitative similar results to those for the p = 20 cases.

Fig. 1.

The model parameter estimation performance for p = 20 case, for the loading coefficient vectors ${\mathit{\gamma }}^{(k)}$ (k = 1, 2), elements of the random effect covariance matrix $\mathbf{\Omega }$, regression coefficients ${\mathit{\beta }}^{(k)}$ (k = 1, 2), and intercept ${\mathit{\beta }}_0$, averaged across 50 simulation replications, with varying $n\in \{100,200,300,400\}$ and $T\in \{10,20,30\}$.

In terms of coverage probability, the results in Table 1 for both p = 10 and 20 cases indicate that the “actual” coverage probability is reasonably close to the “nominal” coverage probability of 0.95, particularly with larger sample sizes (e.g. n = 400, T = 30) for the regression coefficients ${\mathit{\beta }}^{(k)}$. Overall, the results in Table 1 suggest that the Bayesian credible intervals exhibit reasonable frequentist coverage, providing estimates of the parameter uncertainty that aligns with the desired coverage level. In Supplementary Materials S4, we further examine the model’s performance under misspecification: 1) when excluding the random effect component ${\mathit{z}}_i$; and 2) when there are no common “signal” eigenvectors across subjects. Without the random effect, estimation performance remains comparable in terms of bias, but the coverage of 95% credible intervals tends to underestimate uncertainties, particularly for the regression coefficients ${\mathit{\beta }}^{(k)}$. The absence of common covariate-related eigenvectors introduces bias in estimating ${\mathit{\beta }}^{(k)}$, leading to lower coverage levels of the credible intervals than nominal. The average computation time (on a MacBook running M3 Max with 96 GB unified memory) was about 0.8 hours $(\text{SD}=0.16)$ for obtaining 1300 posterior samples on n = 400 subjects with T = 30 time points and p = 20.

Table 1.

The proportion of time that 95% posterior credible intervals contain the true values of the projection loading vectors ${\mathbf{\gamma }}^{(k)}$ (k = 1, 2), regression coefficients ${\mathit{\beta }}^{(k)}$ (k = 1, 2), and elements of $\mathbf{\Omega }$, averaged across 50 simulation replications, with varying $n\in \{100,200,300,400\}$ and $T\in \{10,20,30\}$.^a

		p = 10					p = 20
n	T	${\mathit{\gamma }}^{(1)}$	${\mathit{\gamma }}^{(2)}$	${\mathit{\beta }}^{(1)}$	${\mathit{\beta }}^{(2)}$	$\mathbf{\Omega }$	${\mathit{\gamma }}^{(1)}$	${\mathit{\gamma }}^{(2)}$	${\mathit{\beta }}^{(1)}$	${\mathit{\beta }}^{(2)}$	$\mathbf{\Omega }$
100	10	0.89	0.90	0.93	0.90	0.91	0.86	0.86	0.86	0.84	0.88
	20	0.85	0.85	0.91	0.87	0.93	0.90	0.89	0.92	0.87	0.94
	30	0.88	0.87	0.91	0.90	0.91	0.87	0.89	0.88	0.88	0.88
200	10	0.90	0.88	0.95	0.92	0.94	0.90	0.93	0.96	0.94	0.89
	20	0.92	0.91	0.97	0.92	0.93	0.91	0.92	0.93	0.94	0.93
	30	0.89	0.89	0.96	0.88	0.94	0.89	0.89	0.90	0.89	0.89
300	10	0.88	0.88	0.90	0.90	0.88	0.90	0.91	0.96	0.92	0.89
	20	0.90	0.86	0.90	0.84	0.91	0.92	0.90	0.96	0.92	0.90
	30	0.91	0.90	0.91	0.90	0.91	0.91	0.91	0.94	0.92	0.91
400	10	0.91	0.89	0.96	0.92	0.85	0.90	0.93	0.93	0.92	0.90
	20	0.94	0.91	0.96	0.96	0.87	0.92	0.91	0.94	0.92	0.93
	30	0.93	0.92	0.96	0.95	0.89	0.93	0.91	0.92	0.92	0.92

^aCoverage was computed for each entry, then averaged within components (${\mathit{\gamma }}^{(k)},\hspace{0.17em}{\mathit{\beta }}^{(k)}$ and $\mathbf{\Omega }$) and across the simulation replications (rounded to two significant digits).

4 Application

In this section, we applied the Bayesian CAP regression to data from HCP. As in Seiler and Holmes (2017), we used the rs-fMRI data from HCP 820 subjects and examined the associations between rs-fMRI and sleep duration. Each subject underwent 4 complete 15-min sessions (with $\text{TR}=750$ ms, corresponding to 1200 time points per session for each subject), and each 15-min run of each subject’s rfMRI data was preprocessed according to Smith et al. (2013). We focused on the first session which is about a typical duration for rs-fMRI studies. We also applied the proposed method to the other three sessions to examine the sensitivity and reliability of this regression (see Supplementary Materials S6, where the covariate-related FC exhibits a high level of consistency across all 4 scanning sessions, with the intra-cluster correlation coefficient value of 0.84, 0.72, 0.84 and 0.83, for the 4 identified network components in terms of the log-variance).

We used a data-driven parcellation based on spatial ICA with p = 15 components (ie using p = 15 data-driven “networks nodes;” see Fig. 2 for their most relevant axial slices in MNI152 space) from the HCP PTN (Parcellation + Timeseries + Netmats) dataset, where each subject’s rs-fMRI timeseries data were mapped onto the set of ICA maps (Filippini et al. 2009). We refer to Smith et al. (2013) for details about preprocessing and the ICA time series computation. We conduct inference on the association between the FC over these IC network nodes (Smith et al. 2012) and sleep duration, gender and their interaction.

Fig. 2.

Fifteen independent components (ICs) from spatial group-ICA constituting a data-driven parcellation with 15 components (“network nodes”), provided by the HCP PTN dataset, represented at the most relevant axial slices in MNI152 space. According to Seiler and Holmes (2017), these IC networks correspond to default network (Net15), cerebellum (Net9), visual areas (Net1, Net3, Net4, and Net8), cognition-language (Net2, Net5, Net10, and Net14), perception-somesthesis-pain (Net2, Net6, Net10, and Net14), sensorimotor (Net7 and Net11), executive control (Net12) and auditory (Net12 and Net13).

As in Seiler and Holmes (2017), we classified the subjects into two groups: a group of 489 conventional sleepers (average sleep duration between 7 and 9 hours each night) and a group of 241 short sleepers (average equal or less than 6 hours each night). This yielded a total of 730 participants to compare FC (over the IC networks in Fig. 2) between short and conventional sleepers. Since the time series are temporally correlated, we inferred the equivalent sample size of independent samples. We computed the effective sample size (ESS) defined by Kass et al. (1998), $\text{ESS}=\underset{i\in \{1,\dots ,n\},j\in \{1,\dots ,p\}}{\min }\left(\frac{T_i}{1+2\sum _{s=1}^{\infty }\text{cor}(Y_{i1}^{(j)},Y_{i,1+s}^{(j)})}\right),$ where $Y_{it}^{(j)}$ is the data at time t of the jth network node for subject i, following a conservative approach taking the minimum over all p components and n subjects as the overall estimator. Based on the estimated ESS, we performed thinning of the observed timeseries data, subsampling $T_i=T=\text{ESS}=34$ time points for each subject. The resulting outcome data, ${\mathit{Y}}_{it}(i=1,\dots ,n)(t=1,\dots ,T)$, were then mean-removed per each subject (so that $\sum _{i=1}^T{\mathit{Y}}_{it}=0\in {\mathbb{R}}^{15}$ for each i), and we focused on the association between their covariances ${\mathbf{\Sigma }}_i$ and covariates.

We used the WAIC criterion (2.15) to identify d = 4 projection components. The models’ WAIC values over the range of d = 1 to 6 were –227.9, –397.6, –520.4, –602.7, –573.4, and –358.4, where the minimizer was the d = 4 case. The parameters ($\mathbf{\Gamma }$, B and $\mathbf{\Omega }$) with d = 4 are summarized by their posterior means and 95% credible intervals, reported in Supplementary Materials S4. The expected value of the log Deviation from Diagonality (DfD) was 0.60, suggesting a moderate departure from the diagonality of ${\mathbf{\Psi }}_i$ assumed in (2.2), but the deviation is not overly pronounced.

Under model (2.1), for a linear contrast vector $\mathit{\delta }\in {\mathbb{R}}^q$, we can define the log covariance “contrast” map due to a $\mathit{\delta }$-change in the covariates $\mathit{x}\in {\mathbb{R}}^q$, which corresponds to $\mathbf{\Gamma }\left(\text{diag}(\mathit{B\delta })\right){\mathbf{\Gamma }}^{\top }\in {\mathbb{R}}^{p\times p}$ (see Supplementary Materials S7), where $\mathit{B}\in {\mathbb{R}}^{d\times q}$ is the regression coefficient matrix in (2.2). Specifically, the diagonal elements of this contrast matrix $\mathbf{\Gamma }\left(\text{diag}(\mathit{B\delta })\right){\mathbf{\Gamma }}^{\top }$ can be extracted and exponentiated. This represents the response signals’ variance ratio (VR) corresponding to a $\mathit{\delta }$-change in the covariates. For the four contrasts derived from the SleepDuration × Gender interaction, the left two column panels in Fig. 3 present the response signals’ variance ratio, contrasting (i) short vs. conventional sleeper among male; (ii) short vs. conventional sleeper among female; (iii) male vs. female among short sleeper; and (iv) male vs. female among conventional sleeper.

Fig. 3.

The response signals’ variance ratio (posterior means and 95% credible intervals), corresponding to the four contrasts formed by the Gender-by-SleepDuration interaction. The 95% credible intervals that do not include the variance ratio of 1 are highlighted in red. The sets (“parcel sets”) of network nodes whose signals’ variances are expected to change in the same impact directions due to the corresponding contrasts are indicated in the last column panels, for the Short vs. Conventional sleeper contrasts in the top row, and for the Male vs. Female contrasts in the bottom row.

In Fig. 3, the nodes, or “parcels,” whose VR values were identified (based on 95% credible intervals) to be significantly different from 1, were all with VR > 1. The third column panels of Fig. 3 indicate the nodes whose signals’ variances are expected to change in the same direction, for the Short vs. Conventional sleeper contrasts in the top row panel, and for the Male vs. Female contrasts in the bottom row panel.

For each $\mathit{\delta }$ contrast, we can infer the $\mathit{\delta }$ contrasts’ impact on the connectivity by 95% credible intervals on the $p(p+1)/2$ connectivity elements of the contrast matrix $\mathbf{\Gamma }\left(\text{diag}(\mathit{B\delta })\right){\mathbf{\Gamma }}^{\top }$. The first column panels in Fig. 4 display the covariance elements identified to be significant, whereas the second column panels display the posterior mean of the matrix elements of $\mathbf{\Gamma }\left(\text{diag}(\mathit{B\delta })\right){\mathbf{\Gamma }}^{\top }$, where each row panel corresponds to each $\mathit{\delta }$ contrast in the covariates. The results from the statistical significance maps in Fig. 4 indicate that, overall, there are more substantial connectivity differences between Short and Conventional sleepers (the first two row panels), compared to the cases when we compare Male vs. Female (the last two row panels), and there were slightly more pronounced Short vs. Conventional sleepers differences among Males (the first row panel) than among Females (the second row panel). While there were several identified connectivity differences between Male vs. Female among Short sleepers, there were no statistically significant Male vs. Female differences among Conventional sleepers.

Fig. 4.

The statistical significance map (the left column panels) and the posterior mean (the right column panels) of the log covariance contrast $\mathbf{\Gamma }\left(\text{diag}(\mathit{B\delta })\right){\mathbf{\Gamma }}^{\top }$ for each of the four covariate contrasts $\mathit{\delta }$, derived from the SleepDuration × Gender interaction.

One conventional method for analyzing group ICA data involves initially computing subject-level Pearson correlations between the ICs, which are then Fisher z-transformed. This process is performed on $(p(p-1)/2=)$ 105 pairs of correlations (calculated from 15 ICs), while we conduct the element-wise log transformation on the p = 15 diagonal elements. A total of 120 element-wise linear regressions were then conducted on SleepDuration, Gender and their interaction, and P-values were corrected for multiplicity using the Benjamini–Hochberg (BH) (Benjamini and Hochberg 1995) procedure to control the false discovery rate (FDR) at 0.05. The patterns of the connectivity differences, implied by each $\mathit{\delta }$-contrast, from this mass-univariate approach are presented in Supplementary Material S10, which were similar to the results from Bayesian CAP in Fig. 4. However, compared to the results from Bayesian CAP, far fewer statistically significant elements (13 vs. 77, out of 480 elements) were identified.

While the CAP regression formulation of Zhao et al. (2021b) also alleviates the multiplicity issue and thus can improve statistical power, inference is limited to the association between covariates and the projected outcome components, making it challenging to interpret covariates’ impacts in measured ROIs directly. Therefore, the approach is not directly comparable with the proposed approach here. In Supplementary Materials S8, we display the similarity (similarity between –1 and 1, with 0 indicating orthogonal) of the estimated projection directions from CAP (Zhao et al. 2021b) (in their first four leading components) and those from the proposed Bayesian latent factor model, which shows positive association for each projection direction with the similarity at least 0.4. We also report the CAP regression coefficients (with 95% bootstrap confidence intervals) for each estimated projected outcome component.

According to the meta analysis in Smith et al. (2009), the identified Parcel Set contrasting the Short vs. Conventional sleeper in Fig. 3 mainly correspond to visual areas (network nodes N1, N3, N4, N8), auditory areas (N12, N13) and sensory motor (N11). Curtis et al. (2016) found that self-reported sleep duration primarily co-varied with FC in auditory, visual, and sensorimotor cortices. Specifically, shorter sleep durations were associated with increased FC among auditory, visual, and sensorimotor cortices (these regions roughly correspond to the network nodes N1, N3, N4, N8, N12, N13, and N11), and decreased FC between these regions and the cerebellum (N9). These positive and negative associations found in Curtis et al. (2016) are consistent with the results in the contrast maps presented in Fig. 4 which contrast Short vs. Conventional sleepers.

5 Discussion

Extending the frequentist approach developed in Zhao et al. (2021b) under a probabilistic model (2.1), coupled with a geometric formulation of the dimension-reduced covariance objects ${\mathbf{\Psi }}_i$ in (2.3), the proposed Bayesian method provides a framework to conduct inference on all relevant parameters simultaneously, that produces more interpretable results regarding how the covariates’ effects are expressed in the ROIs. Furthermore, the outcome dimension reduction approach avoids the need to work with subject-specific full p-by-p sample covariance matrices, which can suffer from estimation instability when the number of time points (volumes) is not large (which is typically the case for fMRI signals). Generally, the CAP formulation of Zhao et al. (2021b) allows for a more targeted and efficient analysis by identifying the specific components of the outcome data relevant to the association between covariates and FC.

Although the computational burden and complexity associated with working with the full p-by-p sample covariance matrix can be significantly alleviated by reducing the dimensionality of the outcome data, the method is generally not suitable to be run in very high-dimensional outcome data, such as voxel-level data, and is better suited for intermediate spaces, such as those produced by ICA or an anatomical parcellation. Overfitting might occur due to the large number of parameters in the estimation of the outcome projection matrix $\mathbf{\Gamma }$. Future work will apply prior distributions on the dimension reducing matrix $\mathbf{\Gamma }$ as well as on the covariate effect parameters B that promote sparsity, for improved estimation and interpretation in higher dimensional spaces.

As in Zhao et al. (2021a,b, 2024), the assumption that we make in conducting the inference is partially common eigenvectors of the covariance structure (Wang et al. 2021), in which the covariance is decomposed into shared and unique components, where the shared components captures the information related to the covariates. Future endeavors will explore strategies to mitigate concerns related to model misspecification by addressing heterogeneity in these shared components across subjects. We have conducted preliminary thinning of the observed multivariate time-series to achieve an effective sample size, involving subsampling to eliminate temporal dependencies. Subsequent investigations will refine this approach to delve into individual differences in dynamic FC (e.g. Zhang et al. 2020; Bahrami et al. 2022), incorporating dimension reduction models that account for both between-subject heterogeneity in spatial patterns and within-subject temporal correlation through state-space modeling of latent factors. This will facilitate a deeper exploration of associations between covariates and FC.

A main challenge in modeling covariance matrices is the positive definiteness constraint. Unlike a mean vector where a link function can act element-wise, the positive-definiteness on a covariance matrix is a constraint on all its entry (Pourahmadi 2011). One approach is to transform the problem into an unconstrained estimation problem through a transformation such as Cholesky decomposition, although this requires natural ordering information. Alternative way is to consider a more fundamental geometric formulation, that views individual covariances as elements on a (nonlinear) manifold. A more global transformation (compared to an entry-wise transformation) such as matrix log-transformation then maps individual covariances to a tangent space, allowing for unconstrained operations. However, a global log-transformation poses interpretability challenges, as it generally alters the covariate’s impact directions with respect to the measured ROIs. Our geometry-based CAP approach focuses on identifying relevant eigenvectors, while simultaneously estimating eigenvalues-by-covariates associations through a linear model in a tangent space. By assuming and identifying relevant eigenvectors $\mathbf{\Gamma }$ that align with the covariates’ impact directions, the global log transformation maintains their orientation regarding the covariates’ effects, thus the estimated pairwise covariance contrasts preserve their interpretability as covariate-induced pairwise connectivity differences.

Yet another important challenge is the high dimensionality, as the number of covariance elements increase quadratically in the response variable’s dimension. Generally, CAP regression of Zhao et al. (2021b), and its extension developed here, is useful if there is no need to model the generation of the entire observations, and one is only interested in isolating the data into a potentially low-dimensional representation in which they exhibit certain desired characteristics such as maximizing the model likelihood associated with ${\mathbf{x}}_i$. Such supervised dimension reductions can generally mitigate the curse of dimensionality in covariance modeling.

Supplementary material

Supplementary material is available at Biostatistics Journal online.

Funding

This work was supported by National Institutes of Health (NIH) grant 5 R01 MH099003. Data were provided by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

Conflict of interest statement

None declared.

Data availability

The code used in this paper is accessible at the following GitHub repository: https://github.com/syhyunpark/bcap