Covariance Assisted Multivariate Penalized Additive Regression (CoMPAdRe)

Abstract

We propose a new method for the simultaneous selection and estimation of multivariate sparse additive models with correlated errors. Our method called Covariance Assisted Multivariate Penalized Additive Regression (CoMPAdRe) simultaneously selects among null, linear, and smooth non-linear effects for each predictor while incorporating joint estimation of the sparse residual structure among responses, with the motivation that accounting for inter-response correlation structure can lead to improved accuracy in variable selection and estimation efficiency. CoMPAdRe is constructed in a computationally efficient way that allows the selection and estimation of linear and non-linear covariates to be conducted in parallel across responses. Compared to single-response approaches that marginally select linear and non-linear covariate effects, we demonstrate in simulation studies that the joint multivariate modeling leads to gains in both estimation efficiency and selection accuracy, of greater magnitude in settings where signal is moderate relative to the level of noise. We apply our approach to protein-mRNA expression levels from multiple breast cancer pathways obtained from The Cancer Proteome Atlas and characterize both mRNA-protein associations and protein-protein subnetworks for each pathway. We find non-linear mRNA-protein associations for the Core Reactive, EMT, PIK-AKT, and RTK pathways. Supplementary Materials are available online.

1. Introduction

Additive models are a generalization of linear models in which a response is modeled as the sum of arbitrary smooth, non-linear functions of covariates (Hastie and Tibshirani 1986; Wood 2017). Likewise, multivariate linear regression generalizes classical linear regression to the setting where $Q$ potentially correlated responses are regressed on a common set of $p$ predictors (Izenman 2013). Multivariate generalizations of the additive model are less common in the literature, and the setting of interest in this article. To set notation, let $\mathit{Y}=\left\{{\mathit{y}}_1,\dots ,{\mathit{y}}_Q\right\}$ be a $(n\times Q)$ matrix in which the $Q$ responses are potentially correlated and have a common set of $p$ predictors in $(n\times p)$ matrix $\mathit{X}=\left\{{\mathit{X}}_1,\dots ,{\mathit{X}}_p\right\}$. Here, we model

$${\mathit{y}}_q={\beta }_{q0}+\sum _{j=1}^p{f}_{qj}\left({\mathit{X}}_j\right)+{\mathit{e}}_q\text{for}q=1,\dots ,Q$$ (1)

where the $n$ rows of $\mathit{e}=\left[{\mathit{e}}_1,\dots ,{\mathit{e}}_Q\right]$ are independently and identically $N\left({\mathit{\text{0}}}_Q,{\mathbf{\text{Σ}}}_{Q\times Q}^{-1}\right)$, and $\int f_{qj}\left(x\right)dx=0\forall (q,j)$ assumed for identifiability. Note that $f(\cdot )$ is indexed by response $q$ and covariate $j$. Just as in the classical single response setting of additive models, each response ${\mathit{y}}_q$ is represented by the sum of smooth, covariate-specific functions $f_{qj}$ for each ${\mathit{X}}_j$. In contrast to traditional settings, however, the residual error across the $\mathrm{Q}$ responses are correlated and related by precision matrix ${\mathbf{\text{Σ}}}_{Q\times Q}$.

In the single response setting, a variety of approaches have been developed for variable selection in additive models. The majority of these methods have been $L_1$-based penalized regression procedures for selecting between null and non-linear predictor effects (Lin and Zhang 2006; Ravikumar et al. 2009; Huang et al. 2010). Work has also been developed in a Bayesian context (Scheipl et al. 2012) and for models where estimated non-linear fits are piecewise constant with data-adaptive knots (Petersen et al. 2016). More recently, research has been extended to include the selection between null, linear, and non-linear predictor effects involving variants of a group lasso penalty (Chouldechova and Hastie 2015; Lou et al. 2016; Petersen and Witten 2019). These approaches often require pre-selection of a hyperparameter to favor linear versus non-linear fits. Multi-step algorithmic approaches have been developed as an alternative to favor parsimonious, interpretable linear fits when they sufficiently explain the association between predictor and response, with degree of preference towards linear fits controlled by a pre-specified hyperparameter (Tay and Tibshirani 2020).

There is also recent work on multivariate methods to perform simultaneous selection of regression coefficients and precision matrix elements to produce sparse solutions. For Gaussian multivariate linear regression, Rothman et al. (2010) and Yin and Li (2013) proposed joint $L_1$ penalties on regression coefficients and off-diagonal elements of the precision matrix. In a Bayesian paradigm, Bhadra and Mallick (2013), Ha et al. (2021) and Consonni et al. (2017) proposed hierarchical models with a hyper-inverse Wishart prior on the covariance matrix, and Deshpande et al. (2019) used optimization based on a multivariate spike-and-slab lasso (mSSL) prior to simultaneously select sparse sets of linear coefficients and precision elements.

There is some literature on methods for variable selection of non-linear functions in multivariate settings with non-independently distributed errors. Nandy et al. (2017) developed a method for the selection between null and smooth additive non-linear functions in the setting where errors are spatially dependent. The authors utilized the adaptive group lasso with an objective function that included a spatial weight matrix, establishing both selection consistency and convergence properties. In a multivariate context, Niu et al. (2020) developed a Bayesian variable selection procedure that, in the context of Gaussian graphical models, selected between null and smooth non-linear predictor effects for multiple responses while estimating the precision matrix among responses, provided that the precision matrix followed a decomposable graphical structure. This approach restricts all responses to have the same set of non-sparse predictors and does not distinguish between linear and non-linear predictor effects. To the best of our knowledge, the current literature does not encompass a multivariate approach that simultaneously selects among null, linear, and non-linear associations for each predictor-response pair while estimating an unconstrained dependence structure among responses.

To address this gap, we develop a computationally efficient solution to a sparse multivariate additive regression model, Covariance Assisted Multivariate Penalized Additive Regression (CoMPAdRe). CoMPAdRe employs a penalized spline basis representation (Demmler and Reinsch 1975) for $f_{qj}$, and jointly selects among null, linear, and smooth non-linear predictor effects for each response while simultaneously estimating a sparse precision matrix among responses. Our method enables the selection and estimation of linear and non-linear predictor effects to be conducted in parallel across responses. Through simulation studies, we show that incorporating estimated residual structure into the selection and estimation of predictor effects leads to gains in selection accuracy and in statistical efficiency. We use CoMPAdRe to study the associations between mRNA and protein expression levels for 8 breast cancer pathways derived from The Cancer Proteome Atlas (TPCA) that reveal several non-linear associations missed by other approaches (Li et al. 2013). Software for our method can be found at https://github.com/nmd1994/ComPAdRe along with examples for implementation.

2. Methods

To begin this section, we refer back to (1) and re-establish some notation. Let $\mathit{Y}$ be a $(n\times Q)$ matrix with sample size n and Q possibly correlated responses. Similarly, let $\mathit{X}$ be a $(n\times p)$ matrix of $p$ covariates; we assume all covariates are continuous for exposition but $\mathit{X}$ can also contain discrete covariates. Then, we model $y_q={\beta }_{q0}+\sum _{j=1}p{f}_{qj}\left(X_j\right)+e_{q}forq=1,\dots ,Q$ where, in contrast to the classical single response setting, the $n$ rows of $\mathit{e}=\left[{\mathit{e}}_1,\dots ,{\mathit{e}}_Q\right]$ are independently and identically $N\left({\mathit{0}}_Q,{\mathbf{\Sigma }}_{Q\times Q}^{-1}\right)$, and $\int f_{qj}\left(x\right)dx=0\forall (q,j)$ assumed for identifiability.

Additive models can be written using a number of possible non-linear functional representations for each $f_{qj}(\cdot )$. Some common choices are Gaussian processes or smoothing splines (Cheng et al. 2019; Wood 2017). Our multivariate framework is based on using the Demmler Reinsch parameterization of smoothing splines or O’ Sullivan penalized splines, which naturally partitions linear from non-linear components. In the next sub-sections, we introduce our notation and formulation starting with the notation of spline representation, the univariate additive model, and our variable selection processes before fully specifying our multivariate additive model selection approach.

2.1. Additive Models with Penalized Splines

A smoothing spline is a bivariate smoother for a single ($n\times 1$) response and covariate $\left\{\mathbf{y},{\mathit{X}}_j\right\}$ whose objective is described as follows: $\mathit{m}\mathit{i}\mathit{n}L\left(f_j\right)={\left\{y-f_j\left(X_j\right)\right\}}^T\left\{y-f_j\left(X_j\right)\right\}+{\lambda }_j\Vert f_j^{\prime }\Vert 2^2$. Smoothness is determined by tuning parameter ${\lambda }_j$, which controls the balance between interpolation and having a small second derivative; $f_j$ is a linear function when ${\Vert f_j^{″}\Vert }_2^2=0$ (Wang 2011). The optimal solution to the smoothing spline problem can be obtained by solving a representation of $f_j$ determined by a ($n\times k$) basis of splines ${\mathbf{\text{Φ}}}_j=\left({\varphi }_{1j}\mid {\varphi }_{2j}\mid \dots \mid {\varphi }_{kj}\right)$ with $k$ knots defined at unique values of ${\mathit{X}}_j:f_j=\sum _{k=1}^n{\varphi }_{kj}{\beta }_{kj}$. To further establish notation, let ${\mathit{\beta }}_j=\left({\beta }_{1j}\mid {\beta }_{2j}\mid \dots \mid {\beta }_{kj}\right)$ be a $(k\times 1)$ vector of coefficients, then $f_j={\mathbf{\Phi }}_j{\mathit{\beta }}_j$. Our objective is now min $L\left(f_j\right)={\left\{\mathit{y}-f_j\left({\mathit{X}}_j\right)\right\}}^T\left\{\mathit{y}-f_j\left({\mathit{X}}_j\right)\right\}+{\lambda }_j{\Vert f_j^{″}\Vert }_2^2={\left(\mathit{y}-{\mathrm{\Phi }}_j{\mathit{\beta }}_j\right)}^T\left(\mathit{y}-{\mathrm{\Phi }}_j{\mathit{\beta }}_j\right)+{\lambda }_j{\mathit{\beta }}_j^T{\mathbf{\Omega }}_j{\mathit{\beta }}_j$ where ${\mathrm{\Omega }}_{ij}=\int \varphi (z{)}_i^{″}\varphi {(z)}_j^{″}dz$. This is a standard penalized least squares problem, whose solution is ${\mathit{\beta }}_j={\left({\mathbf{\Phi }}_j^T{\mathbf{\Phi }}_j+{\lambda }_j{\mathbf{\Omega }}_j\right)}^{-1}{\mathbf{\Phi }}_j^T\mathit{y}$ (Wang 2011; Hansen 2019). For computational efficiency one can use ($k<n$) knots defined at certain quantiles of ${\mathit{X}}_j$, which would then be called penalized splines. We utilize O’Sullivan penalized splines for this article (Wand and Ormerod 2008).

Given the detailed description above, we define a penalized objective for an additive model where each function $f_j$ has a spline representation. We see that

$$\mathbf{\text{min}}{\left\{\mathit{y}-{\beta }_0-\sum _{j=1}^p{f}_j\left({\mathit{X}}_j\right)\right\}}^T\left\{\mathit{y}-{\beta }_0-\sum _{j=1}^p{f}_j\left({\mathit{X}}_j\right)\right\}+\sum _{j=1}^p{\lambda }_j\int f_j^{″}{\left(t_j\right)}^{2}d{t}_j$$

Conveniently, the optimal minimizer to this problem is an additive cubic spline model for which each $f_j$ is a cubic spline with knots defined at unique values of each ${\mathit{X}}_j$ (Wang 2011; Wood 2017). Note that we assume $\sum _{i=1}^n{f}_j\left(x_{ij}\right)=0$ for all $p$ in order to ensure an identifiable solution. To represent each $f_j$, we next outline the Demmler-Reinsch representation before redefining our additive model objective.

To form the Demmler-Reinsch representation of a smoothing (or penalized) spline, which separates linear and non-linear components of function $f_j$, we revisit spline basis ${\mathbf{\Phi }}_j$. Specifically, we note that $svd\left({\mathrm{\Phi }}_j\right)=U_j{D}_j{V_j}^T$, where ${\mathit{U}}_j$ is a $(n\times k)$ orthogonal matrix, ${\mathit{D}}_j$ is a diagonal $(k\times k)$ matrix with all non-negative values, and ${\mathit{V}}_j$ is a $(k\times k)$ orthogonal matrix. We then rewrite $f_j={\mathrm{\Phi }}_j{\mathit{\beta }}_j={\mathit{U}}_j{\mathit{D}}_j{\left({\mathit{D}}_j^2+{\lambda }_j{\mathit{V}}_j^T{\mathbf{\Omega }}_j{\mathit{V}}_j\right)}^{-1}{\mathit{D}}_j{\mathit{U}}_j^T={\mathit{U}}_j{\left(\mathit{I}-{\lambda }_j{\mathbf{\Omega }}_j\right)}^{-1}{\mathit{U}}_j^T$. Note that ${\mathbf{\Omega }}_j={\mathit{D}}_j^{-1}{\mathit{V}}_j^T{\mathbf{\Omega }}_j{\mathit{V}}_j{\mathit{D}}_j^{-1}$ is a $(k\times k)$ positive semi-definite matrix that can be diagonalized as $svd\left({\mathbf{\Omega }}_j\sim \right)=W_j{\mathbf{\Gamma }}_j{W_j}^T.{\mathit{w}}_j$ is a $(k\times k)$ orthogonal matrix and ${\mathbf{\Gamma }}_j$ is a $(k\times k)$ diagonal matrix with decreasing non-negative elements $\left({\gamma }_1\left|{\gamma }_2\right|\dots \left|0\right|0\right)$ whose last two diagonal entries are zero (Wood 2017; Hansen 2019). Given these representations, we now re-write $f_j={\mathit{U}}_j{\mathit{W}}_j{\left(\mathit{I}+{\lambda }_j{\mathbf{\Gamma }}_j\right)}^{-1}{\mathit{W}}_j^T{\mathit{U}}_j^T={\mathit{U}}_j{\left(\mathit{I}+{\lambda }_j{\mathbf{\Gamma }}_j\right)}^{-1}{{\mathit{U}}_j}^T\cdot {\mathit{U}}_j$ is the $(n\times k)$ Demmler-Reinsch basis with corresponding $(k\times 1)$ coefficient vector ${\mathit{\beta }}_j$. The final two columns of ${\mathit{U}}_j$ correspond to the intercept and linear components of $f_j$, and the other columns of ${\mathit{U}}_j$ correspond to non-linear components of $f_j$. The now diagonal smoothing penalty ${\mathbf{\Gamma }}_j$ leaves the 2 intercept and linear basis functions of ${\mathit{U}}_j$ unpenalized and the remaining basis functions increasingly penalized by their complexity (i.e. the values along the diagonal of ${\mathbf{\Gamma }}_j$) (Demmler and Reinsch 1975; Hansen 2019).

To summarize, we can represent the objective for $f_j$ as follows. Let ${\mathit{U}}_j=\left[\mathit{1},{{\mathit{U}}_j}^L,{{\mathit{U}}_j}^{NL}\right]$ denote the linear and non-linear components of basis functions ${\mathit{U}}_j$, respectively, with corresponding coefficients ${\mathit{\beta }}_j=\left[{\beta }_0,{{\beta }_j}^L,{{\mathit{\beta }}_j}^{NL}\right]$. Since ${\mathit{X}}_j$ is just an orthogonal rotation of ${{\mathit{U}}_j}^L$, we can trivially replace ${{\mathit{U}}_j}^L$ with ${\mathit{X}}_j$ and see:

minimize

$${\left\{\mathit{y}-\left({\beta }_0+{\mathit{X}}_j{{\beta }_j}^L+{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_j}^{NL}\right)\right\}}^T\left\{\mathit{y}-\left({\beta }_0+{\mathit{X}}_j{{\beta }_j}^L+{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_j}^{NL}\right)\right\}+{\lambda }_j{\mathit{\beta }}_j^T{\mathbf{\Gamma }}_j{\mathit{\beta }}_j$$

and can then rewrite our additive model objective with $j=1,\dots ,p$ covariates as:

minimize

$${\left\{\mathit{y}-{\beta }_0-{\sum }_{i=j}^p\left({\mathit{X}}_j{{\beta }_j}^L+{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_j}^{NL}\right)\right\}}^T\left\{\mathit{y}-{\beta }_0-{\sum }_{j=1}^p\left({\mathit{X}}_j{{\beta }_j}^L+{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_j}^{NL}\right)\right\}+{\sum }_{i=j}^p{\lambda }_j{\mathit{\beta }}_j^T{\mathrm{\Gamma }}_j{\mathit{\beta }}_j$$

To ensure identifiability constraints are satisfied, we omit all intercept basis functions for each $f_j$ and fit one global intercept (Lee et al. 2018). We now conclude this subsection by briefly describing how we can fit additive models as a linear mixed model; this detail becomes relevant during the estimation portion of our framework.

Estimating Additive Models as a Linear Mixed Model:

One can represent any $f_j={\sum }_{k=1}^n{\varphi }_{kj}\left(x\right){\mathit{\beta }}_{kj}$ with an associated measure controlling ‘wiggliness’ (i.e. ${\mathit{\beta }}_j^T{\mathbf{\Omega }}_j{\mathit{\beta }}_j$) in terms of a mixed effects model if they can separate linear and non-linear components of $f_j$, treating the linear components as fixed effects and non-linear components as random effects (Wood 2017). Given the Demmler-Reinsch representation, we can then represent a smoothing spline or penalized spline as follows: $\mathit{y}={\mathit{X}}_j{{\beta }_j}^L+{\mathit{U}}_j{{\mathit{\beta }}_j}^{NL}+\mathit{ϵ}$, where ${{\mathit{\beta }}_j}^{NL}\sim N\left(\mathit{0},\frac{{\mathbf{\Gamma }}_{NL}^{-1}}{{\lambda }_j}\right)$. ${\mathbf{\Gamma }}_{NL}$ represents a diagonal matrix of the positive non-zero values of $\mathbf{\Gamma }$, and $\mathit{ϵ}\sim N\left(\mathit{0},{\mathit{I}}_n{\sigma }^2\right)$. To incorporate an additive model into the mixed model framework, one can append the linear components for each function $f_j$ to the matrix of fixed effects and non-linear components of $f_j$ to the random effects design matrix while ensuring identifiability by restricting to a global intercept (Wood 2017).

2.2. Variable Selection for Additive Models

Given that additive models consider multiple covariates whose dimension $p$ may be high, approaches that perform variable selection can greatly aid in producing interpretable, computationally feasible solutions. To perform variable selection for functions $f_j$, in this article we adapt a smoothness-sparsity penalty that uses separate hyperparameters to control smoothness and selection of additive models. Adopting the above notation, the general form of the penalty is as follows:

$${\Vert \mathit{y}-{\sum }_{j=1}^p{f}_j\Vert }_n^2+{\sum }_{j=1}^{p}J\left(f_j\right),\text{where}{\Vert f_j\Vert }_n^2=\frac{1}{n}{\sum }_{i=1}^n{f}_{ij}^2\text{and}J\left(f_j\right)={\lambda }_1\sqrt{{\Vert f_j\Vert }_n^2+{\lambda }_2\int {\left(f_j^{″}\left(x\right)\right)}^{2}dx}.$$

Penalty $J\left(f_j\right)$ has two hyperparameters: ${\lambda }_1$ controls the level of sparsity while ${\lambda }_2$ controls smoothness via the same second-derivative penalty utilized in smoothing splines. Using the spline basis formulation for each $f_j$ previously described, the Method’s objective is $\mathbf{\text{min}}{\Vert \mathit{y}-{\beta }_0-{\sum }_{j=1}^p{\mathbf{\Phi }}_j{\mathit{\beta }}_j\Vert }_n^2+{\lambda }_1{\sum }_{j=1}^p\sqrt{\frac{1}{n}{\mathit{\beta }}_j^T{\mathbf{\Phi }}_j^T{\mathbf{\Phi }}_j{\mathit{\beta }}_j+{\lambda }_2{\mathit{\beta }}_j^T{\mathbf{\Omega }}_j{\mathit{\beta }}_j}$.

Computationally, this objective can be re-parameterized as a simple group lasso problem for fixed ${\lambda }_2$, ensuring the smoothness of the functional fit is accounted for in the selection process (Meier et al. 2009).

To illustrate how we perform linear and non-linear predictor selection, we first present a single response objective in this section before moving on to our full multivariate objective, where we induce sparsity in the precision matrix with $L_1$ penalization. More concretely, we utilize the Demmler-Reisch basis to separate linear and non-linear components of $f_j$, penalizing linear basis functions with an $L_1$ penalty and non-linear basis functions with an adapted version of the smoothness-sparsity penalty outlined above (Meier et al. 2009). We now note the following objective:

minimize

$$\begin{gathered} {\Vert \mathit{y}-{\beta }_0-\sum _{j=1}^p\left({\mathit{X}}_j{{\beta }_j}^L+{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_j}^{NL}\right)\Vert }_2^2+{\lambda }_1\sum _{j=1}^p{\Vert {{\beta }_j}^L\Vert }_1 \\ +{\lambda }_2\sum _{j=1}^p\sqrt{\frac{1}{n}{\left({{\mathit{\beta }}_j}^{NL}\right)}^T{\left({{\mathit{U}}_j}^{NL}\right)}^T{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_j}^{NL}+{\lambda }_{3j}{\left({{\mathit{\beta }}_j}^{NL}\right)}^T{\mathbf{\Gamma }}_{NL}^j{{\mathit{\beta }}_j}^{NL}} \end{gathered}$$

Linear selection is controlled by hyperparameter ${\lambda }_1$ while non-linear selection is controlled by ${\lambda }_2$. hyperparameter ${\lambda }_{3j}$ controls the degree of smoothness for covariate $j$’s non-linear basis functions; note that we use ${\text{Γ}}_{NL}^j$ because the first two diagonal components of $\mathrm{\Gamma }$, corresponding to linear basis functions, are zero (meaning linear components are left unsmoothed). This optimization problem will produce a selection of null, linear, or nonlinear effect for each proposed covariate $j=1,\dots ,p$, and in the univariate additive model settings we refer to this approach as Penalized Additive Regression, or PAdRe. In simulation studies, to show the benefit of multivariate modeling we will use this as a comparitor for out multivariate approach CoMPAdRe, which we will now describe.

2.3. Multivariate Objective and Model Fitting Procedure

Now transitioning to the multivariate additive model setting, suppose we observe a $(n\times Q)$ matrix $\mathit{Y}$ where each of our $Q$ responses are potentially correlated and contain a common $(n\times p)$ predictor set $\mathit{X}$. We begin by identifying a joint likelihood for our model. Let ${\mathit{y}}_q=\sum _{j=1}^{p}f\left({\mathit{X}}_{qj}\right)+{\mathit{e}}_q$ for $q=1,\dots ,Q$ where the $n$ rows of $\mathit{e}=\left[{\mathit{e}}_1,\dots ,{\mathit{e}}_Q\right]$ are independently and identically $N\left({\mathit{0}}_Q,{\mathbf{\Sigma }}_{Q\times Q}^{-1}\right)$. Then, we can rewrite ${\mathit{y}}_q={\beta }_0+\sum _{j=1}^p{\mathit{X}}_j{{\beta }_{qj}}^L+{{\mathit{U}}_j}^{NL}{{\mathit{\beta }}_{qj}}^{NL}+{\mathit{e}}_q$. Extending to Q responses, we concatenate notation and redefine $\mathit{X}=[\mathit{1},\mathit{X}]$, ${\mathit{\beta }}^L=[\mathit{1},{\mathit{\beta }}^L]$ and see: $\mathit{Y}\sim MVN\left(\mathit{X}{\mathit{\beta }}^L+{\mathit{U}}^{NL}{\mathit{\beta }}^{NL},{\mathbf{\Sigma }}_{Q\times Q}^{-1}\right)$ with the following negative log-likelihood: $g(\mathit{\beta },\mathbf{\Sigma })=tr\left[\frac{1}{n}{\left\{\mathit{Y}-\left(\mathit{X}{\mathit{\beta }}^L+{\mathit{U}}^{NL}{\mathit{\beta }}^{NL}\right)\right\}}^T\left\{\mathit{Y}-\left(\mathit{X}{\mathit{\beta }}^L+{\mathit{U}}^{NL}{\mathit{\beta }}^{NL}\right)\right\}\text{Σ}\right]-log|\mathbf{\Sigma }|$ where ${\mathit{U}}^{NL}$ is a concatenated $[n\times p(k-2)]$ matrix of non-linear basis functions, where each of p covariates has $k-2$ non-linear basis functions.

We now introduce penalties to our negative log-likelihood, completing our objective. Along with the penalties outlined in the previous section, we include an $L_1$ penalty on the off-diagonal elements of precision matrix ${\mathbf{\Sigma }}_{Q\times Q}$, similar to those utilized in the graphical lasso (Friedman et al. 2008). Our final model objective is demonstrated as follows:

minimize

$$\begin{gathered} g(\mathit{\beta },\mathbf{\Sigma })+\sum _{q=1}^Q{\lambda }_{1q}\sum _{j=1}^p{\Vert {\mathit{\beta }}_{qj}^L\Vert }_1+ \\ \sum _{q=1}^Q{\lambda }_{2q}\sum _{j=1}^p\sqrt{\left.\frac{1}{n}{\left({\mathit{\beta }}_{qj}^{NL}\right)}^T{\left({\mathit{U}}_{qj}^{NL}\right)}^T{\mathit{U}}_{qj}^{NL}{\mathit{\beta }}_{qj}^{NL}+{\lambda }_{3qj}{\left({\mathit{\beta }}_{qj}^{NL}\right)}^T{\mathrm{\Gamma }}_{NL}^{qj}{\mathit{\beta }}_{qj}^{NL}\right)} \\ +{\lambda }_4\sum _{q^{\prime }}{\Vert {\sigma }_{q^{\prime },q}\Vert }_1 \end{gathered}$$ (2)

To produce a solution to this non-convex objective, we break our problem down into a three step procedure in which each step, conditioned on the other steps, reduces to a simpler convex optimization problem. Our model fitting procedure also enables parallel computing in terms of responses $q$, ensuring a computationally scalable solution. We next describe this procedure, and how model-fitting is implemented, in more detail.

2.4. Model Fitting Procedure

Before outlining model fitting in Algorithm 1, we first describe a parameterization of precision matrix ${\mathbf{\Sigma }}_{Q\times Q}$ using a formulation that is based on the relationship between the precision matrix of a multivariate normal distribution and regression coefficients (Anderson 1984). Specifically, let $\{n\times (Q-1)\}$ matrix $E_{q^{\prime }}$ denote the vector of errors ${\left[{\mathit{e}}_{\mathit{1}},\dots ,{\mathit{e}}_{q-\mathit{1}},{\mathit{e}}_{q+\mathit{1}},\dots ,{\mathit{e}}_Q\right]}^T$ other than ${\mathit{e}}_q$, the error term for response $q$. We can then represent ${\mathit{e}}_q$ in terms of the following regression: ${\mathit{e}}_q={\mathit{E}}_{q^{\prime }}{\mathit{\alpha }}_q+{\mathit{ϵ}}_q$, where coefficient ${\mathit{\alpha }}_q=-\frac{{\mathbf{\Sigma }}_{q,E_{q^{\prime }}}}{{\sigma }_{q,q}},{\mathbf{\Sigma }}_{q,E_{q^{\prime }}}$ is a $((Q-1)\times 1)$ vector of partial correlations of response $q$ with the other $Q-1$ responses, and error term ${\mathit{ϵ}}_q\sim N\left(0,\frac{1}{{\sigma }_{q,q}}\right)$. Recent literature has extended this concept to the topic of variable selection for multi-layered Gaussian graphical models (mIGGM) (Ha et al. 2021). Their work, in the context of a multivariate regression, demonstrated that model fitting could be conducted as parallel single response regressions.

Specifically, in our context we can rewrite each response of $\mathit{Y}$ as follows:

$${\mathit{Y}}_q=\left(\mathit{X}{\mathit{\beta }}_{[,q]}^L+{\mathit{U}}^{NL}{\mathit{\beta }}_{[,q]}^{NL}\right)+{\mathit{Y}}_{[,-q]}{\mathit{\alpha }}_q-\mathit{X}{\mathit{\beta }}_{[,-q]}^L{\mathit{\alpha }}_q-{\mathit{U}}^{NL}{\mathit{\beta }}_{[,-q]}^{NL}{\mathit{\alpha }}_q+{\mathit{ϵ}}_q$$ (3)

where $[,q]$ denotes the column corresponding to response $q$ and $[,-q]$ refers to the set of all columns other than the column indexing response $q$. Given estimates of linear coefficients, non-linear coefficients, and precision matrix ${\mathbf{\Sigma }}_{Q\times Q}$, this formulation enables updating linear and non-linear selection and estimation steps as parallel single-response procedures, and reduces the selection of linear, non-linear, and precision components to simpler convex optimization problems when conditioned on previous estimates.

More specifically, Algorithm 1 breaks our overall model objective into 3 sequential steps given initial estimates of ${\mathit{\beta }}^L,{\mathit{\beta }}^{NL}$, and $\mathbf{\Sigma }$. Rewriting each response of $\mathit{Y}$ as seen above in (3), we first condition on (${\mathit{\beta }}^{NL},\mathbf{\Sigma }$) and update ${\mathit{\beta }}^L$ via $Q$ separate Lasso procedures; selected coefficients are then re-estimated via OLS. Then, conditioning on (${\mathit{\beta }}^L,\mathbf{\Sigma }$) we update selection of ${\mathit{\beta }}^{NL}$ via $Q$ separate group lasso procedures; both ${\mathit{\beta }}^L$ and ${\mathit{\beta }}^{NL}$ are then re-estimated with $Q$ separate linear mixed models. Finally, conditioning on $({\mathit{\beta }}^L,{\mathit{\beta }}^{NL})$ we update $\mathrm{\Sigma }$ with the graphical lasso. At each specific step of the algorithm, we note that all responses can be fit simultaneously (in parallel). We iterate among these three steps until $MSE\left(\hat{Y}\right)$ converges within a pre-specified tolerance. All tuning parameters are selected via cross-validation; full details on how cross-validation is implemented can be seen at the end of supplemental section S.1. Smoothness hyperparameters ${\lambda }_{3qj}$ are pre-specified with generalized cross-validation (GCV) marginally for every covariate-response combination; pre-fixing every ${\lambda }_{3qj}$ enables the selection of ${\mathit{\beta }}^{NL}$ to be reduced to $Q$ separate group lasso problems when conditioned on ${\mathit{\beta }}^L$ and $\mathbf{\Sigma }$. We note that CoMPAdRe does not pre-specify a preference for linear vs. non-linear fits; both linear and non-linear selection is performed for every covariate-response combination.

Algorithm 1:

Covariance-Assisted Multivariate Sparse Additive Regression (CoMPAdRe)

Result: Mean Components: Linear: ${\mathit{\beta }}^L$, Non-Linear: ${\mathit{\beta }}^{NL}$ & Estimated Precision $\mathbf{\Sigma }$

Require Initial estimates ${\mathit{\beta }}^L$, ${\mathit{\beta }}^{NL}$, and $\mathbf{\Sigma }$;

for $q=1,\dots ,Q$ do

Set ${\mathit{Y}}_q^{*}={\mathit{Y}}_q-{\mathit{U}}^{NL}{\mathit{\beta }}_{[,q]}^{NL}-{\mathit{Y}}_{[,-q]}{\mathit{\alpha }}_q+\mathit{X}{\mathit{\beta }}_{[,-q]}^L{\mathit{\alpha }}_q+{\mathit{U}}^{NL}{\mathit{\beta }}_{[,-q]}^{NL}{\mathit{\alpha }}_q$;

Solve ${\mathit{min}}_{{\mathit{\beta }}_{[,q]}^L}\Vert {\mathit{Y}}_q^{*}-\mathit{X}{\mathit{\beta }}_{[,q]}^L\Vert +{\lambda }_{1q}{\Vert {\mathit{\beta }}_{[,q]}^L\Vert }_1$;

end

Step 1.5: Linear Re-estimation with OLS;

Let ${X_{[,q]}}^{select}$, ${{\beta }_{[,q]}}^{select}$ denote the subset of covariates and coefficients selected as non-zero for response $\text{q}$;

for $q=\mathit{1},\dots ,Q$ do

${min}_{{\mathit{\beta }}_{[,q]}}\Vert Y_q^{*}-{X_{[,q]}}^{select}{{\beta }_{[,q]}}^{select}\Vert$

end

Step 2: Non-Linear Selection Update;

for $q=1,\dots ,Q$ do

Set ${\mathit{Y}}_q^{*}={\mathit{Y}}_q-\mathit{X}{\mathit{\beta }}_{[,q]}^L-{\mathit{Y}}_{[,-q]}{\mathit{\alpha }}_q+\mathit{X}{\mathit{\beta }}_{[,-q]}^L{\mathit{\alpha }}_q+{\mathit{U}}^{NL}{\mathit{\beta }}_{[,-q]}^{NL}{\mathit{\alpha }}_q$;

Solve ${\mathit{min}}_{{\mathit{\beta }}_{[,q]}^{NL}}\Vert {\mathit{Y}}_q^{*}-{\mathit{U}}^{NL}{\mathit{\beta }}_{[,q]}^{NL}\Vert +{\lambda }_{2q}\sum _{j=1}^p\sqrt{\frac{1}{n}{\left({\mathit{\beta }}_{[j,q]}^{NL}\right)}^T{\left({\mathit{U}}_{qj}^{NL}\right)}^T{\mathit{U}}_{qj}^{NL}{\mathit{\beta }}_{[j,q]}^{NL}+{\lambda }_{3qj}{\left({\mathit{\beta }}_{[j,q]}^{NL}\right)}^T{\text{Γ}}_{NL}^{qj}{\mathit{\beta }}_{[j,q]}^{NL}}$;

end

Step 2.5: Linear and Non-Linear Coefficient Estimation with Mixed Effects Model;

for $q=\mathit{1},\dots ,Q$ do

Set ${\mathit{Y}}_q^{\mathrm{*}}={\mathit{Y}}_q-{\mathit{Y}}_{[,-q)}{\mathit{\alpha }}_q+\mathit{X}{\mathit{\beta }}_{[,-q]}^L{\mathit{\alpha }}_q+{\mathit{U}}^{NL}{\mathit{\beta }}_{[,-q]}^{NL}{\mathit{\alpha }}_q$;

Let ${U_{[,q]}}^{n-select},{{\beta }_{[,q]}}^{n-select}$ denote the subset of basis functions and coefficients selected as nonzero and non-linear for response $\text{q}$;

Update estimation of ${{\beta }_{[,q]}}^{select}$, ${{\beta }_{[,q]}}^{n-select}$ with a linear mixed model

end

Step 3: Selection of Precision Elements;

Let ${\mathit{Y}}^{\mathrm{*}}=\mathit{Y}-X{\mathit{\beta }}^L-{\mathit{U}}^{NL}{\mathit{\beta }}^{NL}$;

Obtain an estimate of the empirical covariance and update precision matrix $\mathbf{\Sigma }$ using the graphical lasso (Friedman et al. 2008);

Stop when MSEY converges within tol

3. Simulation Study

Simulation Design:

We assess CoMPAdRe’s performance under settings with varying sample sizes, levels of residual dependence, and signal-to-noise ratios. We consider sample size $n=250$, number of responses $Q=10$, fix the number of covariates to either $p=10$ or $p=100$, and use the following signal-to-noise ratios: $\delta =(0.25,0.5,0.75,1,2)$. To induce residual dependence, we specify a Toeplitz structure to generate covariance matrices (in this case correlation matrices): ${\mathrm{\Sigma }}^{-1}={\left({\rho }^{\left|k-k^{\prime }\right|}\right)}_{k,k^{\prime }=1}^q$, where $\rho =(0.2,0.5,0.7,0.9)$ and higher values of $\rho$ correspond to more highly dependent responses. We present results for $p=100$ in the main body of this manuscript and present results from all other settings in supplemental section S.1. This section of the supplement includes simulation settings with alternative covariance structures as well as additional details for how tuning parameters were selected for each method. Each covariate ${\mathit{X}}_j$, for $j=1,\dots ,p$, is generated using $n$ draws from a random uniform distribution: Unif(−1,1). We consider the following functions for non-null associations between a covariate and a response, four types of nonlinear functions: $\left.{\mathit{f}}_{\mathit{1}}={\delta }^{\mathrm{*}}\left(1-\mathrm{e}\mathrm{x}\mathrm{p}\left(-2X_j\right)\right),{\mathit{f}}_{\mathit{2}}={\delta }^{\mathrm{*}}\left(X_j^2\right),{\mathit{f}}_{\mathit{3}}={\delta }^{\mathrm{*}}\left(X_j^3\right),{\mathit{f}}_{\mathit{4}}(\sigma =0.1)={\delta }^{\mathrm{*}}\frac{1}{\sqrt{2\pi }\sigma }\mathrm{e}\mathrm{x}\mathrm{p}\left(-\frac{X_j^2}{2{\sigma }^2}\right)\right)$, as well as linear functions: ${\mathit{f}}_{\mathit{5}}={\delta }^{\mathrm{*}}\left(X_j\right)$. A figure displaying the shapes of all functions along with additional simulations assessing function-specific selection and estimation performance across all methods considered is contained in supplemental section S.1.2.

For each simulated dataset, we generate the sparse set of predictors for each response as follows. We first randomly select 4 of the first 5 responses to have non-zero predictors. For each non-sparse response, we then randomly select 1 – 5 covariates to have any signal with that response. Each of these selected covariates is randomly assigned a function with probability 0.125 for $f_1,\dots ,f_4$ and 0.5 for $f_5$. Each response is then be generated as $Y_{iq}={\sum }_{j=1}^p{f}_{jq}\left(X_{ij}\right)+E_{iq};i=1,\dots ,n;q=1,\dots ,Q$, where ${\mathit{E}}_i={\left(E_{i1},\dots E_{iQ}\right)}^T\sim \mathit{M}\mathit{V}\mathit{N}\left({\mathit{0}}_Q,{\mathrm{\Sigma }}^{-1}\right)$. For each setting of sample size $n$, residual dependence level $\rho$, and signal-to-noise $\delta$ considered, we simulate 50 datasets in the manner outlined above.

Performance Assessment:

We compare CoMPAdRe to the following approaches: (1) PAdRe, (2) GAMSEL (Chouldechova and Hastie 2015), (3) Lasso (Tibshirani 1996), and (4) mSSL (Deshpande et al. 2019). These approaches can be divided as follows: single response approaches that marginally select linear and non-linear covariate associations for each response (1, 2), single response approaches that marginally select linear covariate associations ignoring the distinction between linear and non-linear fits (3), and multivariate approaches that simultaneously select linear associations and precision elements ignoring the distinction between linear and non-linear fits (4). Note that GAMSEL has a user-selected parameter for favoring linear vs. non-linear fits - we used the approach’s suggested default. All other tuning parameters across methods were selected via cross-validation. For selection accuracy, we report the true positive rate (TPR) and false positive rate (FPR) for null vs non-null signal. For estimation accuracy, we present the ratio of mean absolute deviation (MAD) between CoMPAdRe and method (1), (CoMPAdRe PAdRe), for estimating true function $\hat{f}$. Ratios of estimation accuracy between CoMPAdRe and other methods considered can be seen in supplemental section S.1. We also compare computational times across all approaches in this section of the supplement. After selection the marginal approach (1) is re-estimated using a linear mixed effects model as in CoMPAdRe.

Simulation Results:

Table 1 summarizes our key selection results, reporting the true positive rate (TPR) and false positive rate (FPR) for all five methods at different levels of residual dependence $\rho$ and signal-to-noise ratio $\delta$. A comprehensive breakdown of selection results by type of function (linear vs. non-linear) can be seen in supplemental section S.1. Similarly, Figure 1 visualizes key estimation accuracy results across levels of $\rho$ and $\delta$ for estimating overall true signal $\hat{f}$.

Table 1.

Summary of simulation results for settings where number of covariates $\rho =100$. Results are divided into true positive rate (TPR) and false positive rate (FPR), expressed as a percentage, for levels of residual dependence $\rho =(0.2,0.5,0.7,0.9)$ and signal-to-noise ratio $\delta =(0.25,0.5,0.75,1,2)$. Signal is divided into null vs. non-null signal and results are presented as the median with the interquartile range in parenthesis. 50 datasets were simulated for each setting and sample size was fixed to $n=250$ and the number of responses to $q=10$.

$\delta$		CoMPAdRe	PAdRe	GAMSEL	mSSL	Lasso		CoMPAdRe	PAdRe	GAMSEL	mSSL	Lasso
	$\rho =0.9$						$\rho =0.9$
0.25		77.0 (21.7)	11.1 (20.0)	11.1 (24.3)	27.0 (21.3)	11.1 (20.0)		0.2 (0.5)	< 0.1 (0)	< 0.1 (0)	< 0.1 (0)	< 0.1 (0)
0.5		87.5 (10.2)	40.8 (37.1)	55.8 (34.5)	71.4 (26.7)	40.8 (34.7)		0.3 (0.6)	< 0.1 (0.2)	0.7 (0.9)	< 0.1 (0)	< 0.1 (0.2)
0.75	TPR	93.3 (12.2)	69.6 (30.7)	83.3 (19.3)	73.0 (22.9)	64.0 (31.6)	FPR	0.4 (0.6)	0.1 (0.2)	1.5 (1.9)	< 0.1 (0)	0.1 (0.2)
1		92.9 (13.3)	76.9 (16.5)	91.7 (9.10)	75.0 (20.7)	66.7 (13.5)		0.2 (0.4)	0.1 (0.2)	1.5 (1.1)	< 0.1 (0)	0.1 (0.2)
2		97.1 (8.20)	85.1 (12.9)	> 99.8 (0)	77.8 (15.9)	72.7 (19.0)		0.4 (1.0)	0.1 (0.2)	2.5 (1.9)	< 0.1 (0)	0.1 (0.2)
	$\rho =0.7$						$\rho =0.7$
0.25		33.3 (20.8)	11.1 (24.5)	10.6 (25.0)	11.1 (12.3)	11.1 (24.5)		0.1 (0.3)	< 0.1 (0)	< 0.1 (0.1)	< 0.1 (0)	< 0.1 (0)
0.5		83.3 (20.0)	44.2 (34.6)	54.5 (23.8)	55.8 (28.1)	43.3 (33.3)		0.3 (0.5)	< 0.1 (0.2)	0.6 (1.0)	< 0.1 (0)	< 0.1 (0.2)
0.75	TPR	84.6 (13.7)	60.0 (27.1)	83.3 (17.5)	63.6 (22.5)	56.3 (22.9)	FPR	0.2 (0.3)	0.1 (0.2)	1.5 (1.7)	< 0.1 (0)	0.1 (0.2)
1		90.5 (15.1)	72.4 (16.3)	90.0 (18.2)	66.7 (19.4)	63.1 (22.0)		0.2 (0.4)	0.1 (0.2)	1.3 (1.4)	< 0.1 (0)	0.1 (0.2)
2		94.1 (10.8)	84.0 (15.7)	> 99.8 (0)	73.0 (26.2)	72.1 (24.8)		0.3 (0.6)	< 0.1 (0.3)	2.8 (1.8)	< 0.1 (0)	< 0.1 (0.3)
	$\rho =0.5$						$\rho =0.5$
0.25		15.4 (21.5)	10.6 (19.7)	12.5 (17.1)	11.1 (12.7)	10.6 (19.7)		< 0.1 (0.3)	< 0.1 (0)	< 0.1 (0)	< 0.1 (0)	< 0.1 (0)
0.5		72.7 (16.7)	44.9 (26.3)	56.9 (23.3)	33.3 (19.4)	44.4 (27.8)		0.3 (0.1)	0.1 (0.2)	0.6 (1.0)	< 0.1 (0)	0.1 (0.2)
0.75	TPR	84.0 (19.1)	64.2 (29.8)	81.8 (22.2)	65.2 (25.1)	61.5 (29.1)	FPR	0.2 (0.2)	0.1 (0.3)	1.6 (1.6)	< 0.1 (0)	0.1 (0.3)
1		88.9 (15.4)	78.2 (23.9)	89.4 (20.0)	66.7 (28.4)	69.2 (23.5)		0.2 (0.3)	< 0.1 (0.2)	1.5 (1.5)	< 0.1 (0)	< 0.1 (0.2)
2		92.9 (14.0)	84.0 (17.2)	> 99.8 (0)	72.7 (15.7)	75.0 (19.0)		0.2 (0.4)	0.1 (0.1)	3.0 (2.5)	< 0.1 (0)	0.1 (0.1)
	$\rho =0.2$						$\rho =0.2$
0.25		16.2 (24.3)	12.5 (19.5)	19.1 (19.8)	11.1 (12.2)	12.5 (19.5)		< 0.1 (0.1)	< 0.1 (0)	< 0.1 (0.1)	< 0.1 (0)	< 0.1 (0)
0.5		57.3 (20.2)	36.9 (27.5)	64.3 (23.2)	23.1 (20.8)	36.9 (25.0)		0.1 (0.3)	< 0.1 (0.2)	0.8 (1.3)	< 0.1 (0)	< 0.1 (0.2)
0.75	TPR	84.6 (17.3)	70.7 (28.9)	81.5 (21.5)	57.1 (18.6)	65.5 (27.5)	FPR	0.2 (0.4)	< 0.1 (0.2)	1.3 (1.4)	< 0.1 (0)	< 0.1 (0.2)
1		83.3 (17.9)	72.1 (23.9)	90.0 (20.0)	64.5 (26.1)	63.1 (28.0)		0.2 (0.4)	0.1 (0.2)	1.5 (1.1)	< 0.1 (0)	0.1 (0.2)
2		93.3 (11.1)	84.6 (20.2)	> 99.8 (0)	72.1 (21.8)	72.7 (21.3)		0.2 (0.6)	0.1 (0.2)	2.4 (1.6)	< 0.1 (0)	0.1 (0.2)

Fig. 1.

Boxplots of estimation accuracy results for non-linear signal $\widehat{f}$. Midpoint lines represent the median ratio of mean absolute deviation (MAD): $\left(\frac{\text{CoMPAdRe}}{\text{PAdRe}}\right)$ across 50 simulated datasets per setting. Settings considered vary based on signal-to-noise ratio $\delta$ and level of residual dependence $\rho$.

At moderate to high levels of residual dependence ($\mathit{\rho }=\mathit{0.7},\mathit{0.9}$), CoMPAdRe consistently demonstrates superior sensitivity for selecting significant covariates than other approaches while maintaining favorable false positive rates. The relative improvement of CoMPAdRe over PAdRe, an equivalent method in every way except ignoring the between-response correlation, demonstrates that the joint modeling resulted in improved variable selection. This difference was strongest at low signal-to-noise ratios ($\delta =0.25,0.5$). For example, in setting ($\delta =0.25,\rho =0.9$) CoMPAdRe had a significantly higher median true positive rate compared to the median rates of its competitors (CoMPAdRe = 0.770, PAdRe = 0.111, GAMSEL = 0.111, mSSL = 0.270, Lasso = 0.111). ComPAdRe had the highest sensitivity for all settings except for the highest signal-to-noise ($\delta =2$), for which GAMSEL had slightly higher sensitivity, but both GAMSEL and COMPAdRe near 1. GAMSEL, however, consistently reported a false positive rate approximately ∼ 10-fold higher than CoMPAdRe at higher signal to noise ratios $(\delta =[0.75,1,2\left]\right)$ across all levels of residual dependence $\rho$. This pattern persisted across all settings considered. For example, in settings where ($p=10,\delta =2$), GAMSEL showed an average median false positive rate of $\mathit{FPR}=0.122$ while CoMPAdRe showed an average median false positive rate of $\mathit{FPR}=0.011$ across all levels of residual dependence $\rho$.

At lower levels of residual dependence ($\mathit{\rho }=\mathit{0.2},\mathit{0.5}$), CoMPAdRe still generally exhibited superior sensitivity than competitors. The contrast between true positive rates at lower levels of signal to noise ($\delta =[0.25,0.5]$) was less stark than those seen at higher levels of $\rho$ across all methods. GAMSEL again had slightly higher sensitivity than CoMPAdRe at $(\delta =2)$ and $(\delta =1)$, but again accompanied by higher FPR. From additional simulations conducted to assess function-specific selection and estimation performance (supplemental section S.1.2), we see that linear selection approaches (Lasso and mSSL) consistently failed to select functions $f_2$ and $f_4$, even at high signal-to-noise ratios, while CoMPAdRe while allowing the potential of nonlinear associations did not appear to lose sensitivity for selecting covariates with linear associations.

In terms of estimation accuracy, CoMPAdRe outperformed method (1) when estimating overall signal $\hat{f}$ across both $\delta$ and $\rho$. In each setting of signal-to-noise $\delta$ the improved performance of CoMPAdRe relative to method (1) increased as level of residual dependence $\rho$ increased; this is evidenced by the downward linear trend in Figure 2 at each level of $\delta$. This trend persisted across all settings considered (see S.1). The only setting where CoMPAdRe didn’t show notable gains in estimation accuracy occurred at low signal-to-noise and levels of residual dependence: ($\delta =0.25$, $\rho =0.2$), ($\delta =0.25$, $\rho =0.5$). In these settings the ratio of estimation accuracy ($\frac{\text{CoMPAdRe}}{\text{PAdRe}}$) remained centered around 1. These results demonstrate the strong benefit in estimation accuracy from the joint modeling when strong inter-response correlation structure is present, but without substantial tradeoff when the responses have low levels of correlation. CoMPAdRe displayed similar performance relative to other approaches considered, with improvements in estimation accuracy increasing with higher levels of inter-response correlation. From additional function-specific simulations, CoMPAdRe demonstrated superior estimation accuracy to linear selection approaches, with results most evident for estimation of $f_1$ and $f_4$. Given simulation results that show the benefits of our approach, we now apply CoMPAdRe to protein-mRNA expression data obtained from The Cancer Protein Atlas project (TCPA).

Fig. 2.

Protein-Protein covariance networks for PI3K-AKT, EMT, DNA Damage, and Breast Reactive pathways. Blue edges indicate negative associations while red edges indicate positive associations. Edge thickness indicates the magnitude of the dependence between two corresponding proteins and node size is scaled relative to the strength and number of connections for a protein.

4. Analyses of Proteogenomics data in Breast Cancer

We applied CoMPAdRe to a proteogenomics dataset containing both protein and mRNA expression levels for 8 known breast cancer pathways obtained from The Cancer Protein Atlas project (TCPA). The central dogma of molecular biology states that genetic information is primarily passed from an individual’s DNA towards the production of mRNA (a process called transcription) before mRNA takes that information to a cell’s ribosomes to construct proteins (a process called translation) which carry out the body’s biological functions (Crick 1970). Furthermore, proteins are known to carry out biological functions in coordinated networks (De Las Rivas and Fontanillo 2010; Garcia et al. 2012). Here we consider a subset of TCPA data consisting of mRNA and protein data for $n=844$ subjects from 8 cancer-relevant biological pathways. For each pathway, we have data from 3–11 proteins and 3–11 mRNA transcripts. Our objective is three-fold: (1) to find which mRNA are predictive of protein expression in a particular pathway, (2) assess the shape of the relationship, whether linear or nonlinear, and (3) estimate the protein-protein networks accounting for mRNA expression. We will accomplish this by applying CoMPAdRe to each pathway, treating the $Q$ proteins as responses and $p\mathrm{m}\mathrm{R}\mathrm{N}\mathrm{A}$ transcripts as covariates. The joint approach will give us estimates of the $Q\times Q$ protein-protein network and, as demonstrated by our simulations, we expect this joint modeling will result in improved detection of mRNA transcripts predictive of protein abundances than if the proteins were modeled independently.

More information about each pathway can be found in (Akbani et al. 2014; Ha et al. 2018). As an initial step, covariates (mRNA expression levels) were centered and scaled by their mean and standard deviation. Just as in our simulation studies, we then formed B-spline bases by taking knots at the deciles for each covariate considered within a pathway.

Biological Interpretations:

Table 2 summarizes mRNA selection results for all pathways analyzed. A comprehensive table of mRNA selection results are contained in supplemental section S.2 along with visualizations of the shapes for selected nonlinear mRNA–protein associations. Among non-linear associations, we note that CDH1–$\beta$-catenin and CDH1–E-cadherin were selected for the Core Reactive Pathway, CDH1–E-cadherin for the EMT pathway, INPP4B–INPP4B for the PI3K-AKT pathway, and ERBB2–HER2PY1248 for the RTK pathway. We consistently found the same pattern for all non-linear mRNA–protein associations; protein expression increases with mRNA expression until seemingly hitting a plateau. Protein expression has been found to both be positively correlated with mRNA expression and to often plateau at high expression levels because of a suspected saturation of ribosomes, which would limit translation (Liu et al. 2016; van Asbeck et al. 2021). Therefore, identified non-linear mRNA–protein functional associations may provide insight into ‘saturation points’ beyond which increased mRNA expression ceases to result in an increase of protein abundance.

Table 2.

A summary of mRNA selection results for each pathway, divided into linear associations and non-linear associations. Names on the left side of the dash indicate mRNA biomarkers, while those on the right side of the dash indicate the corresponding matching protein. Lines ended by ‘...’ indicate the presence of one or more associations than those listed.

Pathway	Linear mRNA	Non-linear mRNA
Breast Reactive	GAPDH-GAPDH	-
Core Reactive	-	CDH1-$\beta$-catenin, ...
DNA damage	RAD50-RAD50, MRE11A-RAD50, ...	-
EMT	CDH1-$\beta$-catenin	CDH1-E-cadherin
PI3K - AKT	PTEN-PTEN, CDK1B-AKTPT308, ...	INPP4B-INPP4B
RAS-MAPK	YBX1-JNKPT183Y185, YBX1-YB1PS102	-
RTK	ERBB2-EGFRPY1068, EGFR-EGFRPY1068	ERBB2-HER2PY1248
TSC-mTOR	EIF4EBP1-X4EBP1PS65, ...	-

Pathway specific protein-protein networks:

Figure 2 displays protein-protein networks for the pathways detailed in mRNA selection. A visualization of protein-protein networks for all other pathways are detailed in supplemental section S.2. The most highly interconnected proteins, which we call hub nodes, include MYH11, E-cadherin, $\beta$-catenin, and EGFR. In particular, MYH11 showed strong associations with other proteins in the Breast Reactive pathway. MYH11 are smooth muscle myosin heavy chain proteins, which play an essential role in cell movement and the transport of materials within and between cells (Brownstein et al. 2018). While its exact function in breast cancer is not fully understood, MYH11 has been found to be downregulated in breast cancer tissues and to have been critical to empirically constructed indicators for survival prognosis in breast cancer (Zhu et al. 2020). Our results indicate that the role of MYH11 in breast cancer may be better understood through the biomarkers it was found to be highly associated with (CAVEOLINI, RBM15, GAPDH). We also note that previous literature has shown expression levels of E-cadherin and $\beta$-catenin to be strongly correlated with each other; reduced levels in both proteins are associated with poor survival prognosis in triple negative breast cancer (Shen et al. 2016).

5. Discussion

In this article, we introduced CoMPAdRe, a framework for simultaneous variable selection and estimation of sparse residual precision matrices in multivariate additive models. The approach simultaneously obtains a sparse estimate of the inter-response precision matrix and utilizes this association to borrow strength across responses in identifying significant covariates for each response, determining whether each selected covariate effect is linear or nonlinear. It also obtains an estimate of the residual precision matrix, which itself is a quantity of scientific interest in many settings including the protein-protein network modeling of our motivating example. Importantly, our approach allows different covariates to be selected for different responses, and the same covariate to have different functional relationships for different responses. The fitting procedure utilizes a regression approach to estimate and account for the residual correlation that simplifies the joint multivariate modeling into a series of single-response models to improve scalability and computational efficiency. We also note that the univariate special case of our method, which we call PAdRe, is a useful tool for doing linear and nonlinear variable selection in univariate additive model setting, as well.

We empirically demonstrated that CoMPAdRe achieves superior overall variable selection accuracy and statistical efficiency than marginal competitors that model each response independently, with the benefit more demonstrable as the inter-response correlation increases or the signal-to-noise levels decrease. This demonstrates the benefit of joint multivariate modeling, with the inherent borrowing of strength across responses resulting in not just better estimation accuracy, but also improved variable selection and determination of linearity or nonlinearity of the effects. The improved estimation accuracy was expected based on the principles of seemingly unrelated regression (SUR, Zellner (1963)), given that the sparsity prior on covariates implies different covariates for each response. However, our results also suggest that the joint modeling can substantially improve variable selection accuracy, as well. Future theoretical investigations would be interesting to evaluate and validate this result. Further, the improved performance for lower signal-to-noise ratios demonstrates that the efficiency gained by borrowing strength across responses may be especially important to detect more subtle signals or in the presence of higher noise levels. Our simulations also demonstrate that for covariates with nonlinear associations with responses, variable selection methods based on linear regression will tend to miss these variables, especially for certain nonlinear functional shapes. Thus, the variable selection framework we introduce enables identification of important predictive variables even when having a highly nonlinear association with responses.

While the CoMPAdRe method introduced here assumes unstructured inter-response correlation structure, it would be relatively straightforward to adapt to incorporate structured covariance matrices, for example if the responses are observed on some known temporal or spatial grid, or if some decomposable graph structure is known beforehand. This extension could bring the enhanced variable selection and estimation accuracy inherent to the joint modeling while accounting for known structure among the responses. We leave this work for future extensions. CoMPAdRe can also be modified to perform linear selection in a group-wise manner, which may aid interpretability in certain applications. This paper focused on variable selection and estimation, but it would also be of interest to obtain inferential quantities, including hypothesis tests for significant covariates or confidence measures for each selected variable as well as confidence bands for estimated regression functions and precision elements. We also leave this to future work. Finally, while as described in the supplement we have semi-automatic methods to estimate the tuning parameters that seem to work well, more rigorous and automatic approaches for selecting smoothing parameters in this setting may improve performance further, and will be investigated in the future. Software related to this article can be found at https://github.com/nmd1994/ComPAdRe along with examples for implementation.