Integrative Factor Regression and Its Inference for Multimodal Data Analysis

Abstract

Multimodal data, where different types of data are collected from the same subjects, are fast emerging in a large variety of scientific applications. Factor analysis is commonly used in integrative analysis of multimodal data, and is particularly useful to overcome the curse of high dimensionality and high correlations. However, there is little work on statistical inference for factor analysis based supervised modeling of multimodal data. In this article, we consider an integrative linear regression model that is built upon the latent factors extracted from multimodal data. We address three important questions: how to infer the significance of one data modality given the other modalities in the model; how to infer the significance of a combination of variables from one modality or across different modalities; and how to quantify the contribution, measured by the goodness-of-fit, of one data modality given the others. When answering each question, we explicitly characterize both the benefit and the extra cost of factor analysis. Those questions, to our knowledge, have not yet been addressed despite wide use of factor analysis in integrative multimodal analysis, and our proposal bridges an important gap. We study the empirical performance of our methods through simulations, and further illustrate with a multimodal neuroimaging analysis.

1. Introduction

Thanks to rapid technological advances, multiple types of data are now frequently collected for a common set of experimental subjects. Such a new data structure, often referred as multi-view, multi-source or multimodal data, is fast emerging in a wide range of scientific fields. Examples include multi-omics data in genomics, multimodal neuroimaging data in neuroscience, multimodal electronic health records data in health care administration, among others. Numerous empirical studies have found that, by combining diverse but usually complementary information from different types of data, an integrative analysis of multimodal data is often beneficial; see Uludag and Roebroeck (2014); Li et al. (2016); Richardson et al. (2016) for reviews and the references therein.

In view of the promise of multimodal data, a number of statistical methods have recently been developed for integrative analysis. An important class of such solutions is matrix or tensor factorization, which decomposes multimodal data into the components that capture joint variations shared across modalities, and the components that characterize modality-specific variations (Lock et al., 2013; Yang and Michailidis, 2015; Li and Jung, 2017; Lock and Li, 2018; Gaynanova and Li, 2019). Another class is canonical correlation analysis, which seeks maximum correlations between different data modalities through decomposition of the between-modality dependency structure (Li and Gaynanova, 2018; Shu et al., 2019). However, all these methods are unsupervised, in the sense that there is not a response variable involved. Li et al. (2018) recently proposed an integrative reduced-rank regression to model multivariate responses given multi-view data as predictors. Xue and Qu (2019) developed an estimating equations approach to accommodate block missing patterns in multimodal data. Their methods are supervised, but both focused on parameter estimation and variable selection instead of statistical inference.

It is of ubiquitous interest to study the predictive associations between responses and multimodal predictors. However, there are some unique characteristics of multimodal data that make the problem challenging. First, multimodal data are often high-dimensional. In plenty of applications, even a single modality contains more variables than the sample size. Second, some variables in multimodal data can be highly correlated. This can happen for the variables within a single modality, or for the related variables across multiple modalities as they often measure related features of the same subject. This phenomenon has been constantly observed, and is actually the base upon which those matrix or tensor factorization solutions are built (Lock et al., 2013). Such high correlations pose challenges when directly applying many standard high-dimensional methods, such as LASSO (Tibshirani, 1996), to multimodal data, as they usually require the predictors not to be highly correlated in order to achieve some desired statistical properties, e.g., the variable selection consistency. Finally, multimodal data pose new questions; for instance, how to quantify the contribution and statistical significance of one data modality conditioning on the other data modalities in the regression model.

Factor analysis is a well-known approach to both reduce high dimensionality and high correlations among the variables. For data with a single modality, Fan et al. (2013) employed a factor model to estimate a non-sparse covariance matrix. Kneip et al. (2011) proposed to include the latent factors as additional explanatory variables in a high-dimensional linear regression, and established the model selection consistency. Fan et al. (2016) proposed a factor-adjusted model selection method for a general high-dimensional M-estimation problem. They separated the latent factors from the idiosyncratic components to reduce correlations among the covariates, and showed that their method can reach the variable selection consistency under milder conditions than standard selection methods. Li et al. (2018) studied estimation of a covariance matrix of variables. They showed that leveraging on additional auxiliary variables can improve the estimation, when the auxiliary variables share some common latent factors with the variables of interest. For data with multiple modalities, Shen et al. (2013) proposed an integrative clustering method based on identifying common latent factors from multi-omics data. Zhang et al. (2019) developed an imputed factor regression model for dimension reduction and prediction of multimodal data with missing blocks. Despite these efforts, however, there is little work on statistical inference for supervised modeling of multimodal data. Moreover, there is no explicit quantification of the benefit of factor analysis in a multimodal regression setting, and many important inference-related questions remain unanswered.

In this article, we aim to bridge this gap. We consider an integrative linear regression model built upon the latent factors extracted from multimodal data. We show that this model alleviates high dimensionality and high correlations of multimodal data. Based on this model, we address three important questions: how to infer the significance of one data modality given the other modalities in the model; how to infer the significance of a combination of variables from one or more modalities; and how to quantify the contribution, measured by the goodness-of-fit, of one data modality given the others. When answering each question, we explicitly characterize both the benefit and the extra cost of factor analysis. First, by resorting to a relatively small number of latent factors, it effectively reduces the dimensionality and turns a high-dimensional test to a low-dimensional one when testing the significance of a whole modality. As a result, it enables us to derive a closed form for the limiting distribution of the test statistic; see Theorem 1. Second, our method can consistently estimate the support and nonzero components of the covariate coefficients in the regression model. More importantly, by using the decorrelated idiosyncratic components from factor analysis as the pseudo predictors, instead of the original highly correlated covariates, it requires much weaker conditions to reach the variable selection and estimation consistency; see Theorem 3. In addition, we show that, when there are enough variables in each modality so that the latent factors can be well estimated, the resulting estimation error can reach the minimax optimal rate, and under weaker conditions. Third, such an improvement in selection and estimation in turn benefits the inference of the significance of a linear combination of predictors, by requiring less stringent conditions to establish the limiting distribution of the test statistic; see Theorem 4. Finally, by leveraging on the latent factors shared across modalities, it enables us to obtain a closed-form measure of the variance of the response explained by one modality in addition to the others. Such a measure facilitates the quantification of the contribution of an individual modality.

Our proposal contributes on several fronts. Even though factor analysis has been widely used in multimodal data analysis, there has been no formal test developed to explicitly quantify the contribution and significance of an individual modality or a related set of variables across different modalities. Our proposal provides the first inferential tools to address those important questions. Moreover, our work is built on careful examination of the benefit and trade-off of factor analysis in regression. Compared to the existing literature, the proof techniques are much more involved than those of the standard setting when the design matrix is observed and fixed with a single data modality. The technical tools we develop here are not limited to our setting alone, but are applicable to general supervised high-dimensional factor models. We also remark that, although we focus on a linear factor regression model, most of the inference-related results we obtain can be extended to more general M-estimation problems such as a generalized linear model.

We employ the following notation throughout this article. For a vector $a\in {\mathcal{R}}^d$, let ‖a‖_∞ = max_j | a_j |, ${\Vert a\Vert }_1=\sum _{j=1}^d\left|a_j\right|$, ${\Vert a\Vert }_2={\left(\sum _{j=1}^d{a}_j^2\right)}^{1/2}$ denote its sup-norm, L₁-norm and Euclidean norm, respectively. For an index set $\mathcal{S}$, let a_S denote the subvector of a with indices in $\mathcal{S}$. In particular, let the subscript m denote the index set of the mth modality, and the subscript – m denote the index set of all other modalities. Let a^⊗2 = aa′ denote its outer product. Let supp(a) = {j : a_j ≠ 0} denote the support of a. For a square matrix $A=\left(a_{ij}\right)\in {\mathcal{R}}^{d\times d}$, let λ_min (A) and λ_max (A) denote its minimum and maximum eigenvalues. Let ${\Vert \mathit{\text{A}}\Vert }_{\infty }=\underset{ij}{\text{sup}}\left|a_{ij}\right|$, ${\Vert \mathit{\text{A}}\Vert }_1={\text{max}}_{1\le j\le d}\sum _{i=1}^d\left|a_{ij}\right|$, ‖A‖₂ =λ_max (A), ${\Vert A\Vert }_F={\left(\sum _{i,j}{a}_{ij}^2\right)}^{1/2}$ denote its element-wise sup-norm, L₁-norm, L₂-norm, and Frobenious norm, respectively. Let A_S denote the submatrix of A with row and column indices in $\mathcal{S}$. For a rectangular matrix $B=\left(b_{ij}\right)\in {\mathcal{R}}^{m\times n}$, let ${\Vert B\Vert }_{L_{\infty }}={\text{max}}_{1\le i\le m}\sum _{j=1}^n\left|b_{ij}\right|$. For two sequences a_n and b_n, write a_n = o(b_n) if a_n/b_n → 0, and a_n ≫ b_n if b_n/a_n → 0. For an integer M, let [M] = {1, …, M}. For a set $\mathcal{S}$, let | S | denote the number of elements in $\mathcal{S}$.

The rest of the article is organized as follows. We introduce the integrative factor regression model in Section 2, and describe the parameter estimation in Section 3. These results are mostly built upon the existing literature on factor analysis. Then we address the three questions, which to our knowledge have not been answered before. That is, we develop a test to evaluate the significance of an individual modality given the other modalities in Section 4, develop a test for a linear combination of predictors in Section 5, and derive a measure to quantify the contribution of an individual modality in Section 6. We present the simulations and a multimodal neuroimaging data example in Section 7. We relegate all technical proofs and some additional lemmas to the Supplementary Materials.

2. Integrative factor regression model

Suppose there are M modalities of variables. Let $x_m\in {\mathcal{R}}^{p_m}$ denote the vector of p_m random variables from the mth modality, and y denote the response variable. Let $x={\left(x^{\prime }{}_1,\dots ,x^{\prime }{}_M\right)}^{\prime }\in {\mathcal{R}}^p$, and $p=\sum _{m=1}^M{p}_m$. We assume x_m is driven by some latent factors in that x_m can be decomposed as

$$x_m={\Lambda }_m{f}_m+u_m,$$ (1)

where $f_m\in {\mathcal{R}}^{K_m}$ is the vector of K_m random latent factors, $u_m\in {\mathcal{R}}^{p_m}$ is the vector of random idiosyncratic errors of variables in the mth modality that are uncorrelated with f_m, and ${\Lambda }_m\in {\mathcal{R}}^{p_m\times K_m}$ is the loading matrix of x_m on the latent factors f_m. To avoid the identifiability issue on Λ_m and f_m, we adopt the usual assumption in the factor analysis literature by assuming that $\text{Var}\left(f_m\right)=I_{K_m}$, and ${\Lambda }_m{}^{\prime }{\Lambda }_m=D_m\in {\mathcal{R}}^{K_m\times K_m}$ is a diagonal matrix.

We also assume that $f={\left(f^{\prime }{}_1,\dots ,f^{\prime }{}_M\right)}^{\prime }\in {\mathcal{R}}^K$ is uncorrelated with $u={\left(u^{\prime }{}_1,\dots ,u^{\prime }{}_M\right)}^{\prime }\in {\mathcal{R}}^p$, where $K=\sum _{m=1}^M{K}_m$. Let $\Lambda =\text{diag}\left({\Lambda }_1,\dots ,{\Lambda }_M\right)\in {\mathcal{R}}^{p\times K}$ be the block diagonal matrix of the loading matrices from all modalities, and Σ_u = Var(u) be the covariance matrix of the idiosyncratic errors. In the factor analysis literature, e.g. Fan et al. (2013), it is often assumed that Σ_u is sparse, i.e., after removing the variations contributed by the latent factors, the correlations among the idiosyncratic components are weak. Therefore, the idiosyncratic u can be viewed as a decorrelated version of the original variables x.

We next employ a linear model to connect x_m with y, in that,

$$y=\sum _{m=1}^M{x}_m{}^{\prime }{\beta }_m^{*}+ϵ,$$ (2)

where ${\beta }_m^{*}\in {\mathcal{R}}^{p_m}$ is the true effect of x_m on the response y, and ϵ is an error uncorrelated of the covariates, with E(ϵ) = 0 and $\text{Var}(ϵ)={\sigma }_{ϵ}^2$.

Suppose we have n i.i.d. realizations of the data, $Y={\left(y_1,\dots ,y_n\right)}^{\prime }\in {\mathcal{R}}^n$, $X_m={\left(x_{1,m},\dots ,x_{n,m}\right)}^{\prime }\in {\mathcal{R}}^{n\times p_m}$, $X=\left(X_1,\dots ,X_M\right)\in {\mathcal{R}}^{n\times p}$, ${\beta }^{*}={\left({\beta }_1^{*\prime },\dots ,{\beta }_M^{*}{}^{\prime }\right)}^{\prime }\in {\mathcal{R}}^p$, and $ϵ={\left({ϵ}_1,\dots ,{ϵ}_n\right)}^{\prime }\in {\mathcal{R}}^n$. Then model (2) can be written as

$$Y=\sum _{m=1}^M{X}_m{\beta }_m^{*}+ϵ=X{\beta }^{*}+ϵ.$$

By the factor model (1), we have X_m = F_mΛ_m′+U_m, where $F_m={\left(f_{1,m},\dots ,f_{n,m}\right)}^{\prime }\in {\mathcal{R}}^{n\times K_m}$ is the matrix of the K_m factors in the mth modality pertaining to the n subjects, and $U_m={\left(u_{1,m},\dots ,u_{n,m}\right)}^{\prime }\in {\mathcal{R}}^{n\times p_m}$ is the matrix of idiosyncratic errors. Then, we have,

$$Y=F{\gamma }^{*}+U{\beta }^{*}+ϵ,$$ (3)

where $F=\left(F_1,\dots ,F_M\right)\in {\mathcal{R}}^{n\times K}$, ${\gamma }^{*}={\left({\beta }_1^{*\prime }{\Lambda }_1,\dots ,{\beta }_M^{*}{}^{\prime }{\Lambda }_M\right)}^{\prime }\in {\mathcal{R}}^K$, and $U=\left(U_1,\dots U_M\right)\in {\mathcal{R}}^{n\times p}$. We call model (3) an integrative factor regression model. In the remainder of this article, we aim to show that model (3) can benefit estimation, selection and inference about β*. The intuition is that, after the latent factors F are separated, the idiosyncratic error U can be treated as the pseudo predictors. Such a decorrelation eases selection of β*, which in turn benefits the inference on β*. In addition, the factor decomposition also serves as a dimension reduction tool, which enables us to derive some closed-form results in inference. We remark that, in model (3), the coefficient β* associated with U is the same as that associated with the original predictor X. It is the main object of interest in our inference, as its component ${\beta }_m^{*}$ reflects the effect of the mth modality x_m on the response y. Meanwhile, we treat γ* as a nuisance parameter. We also remark that, one does not necessarily have to perform factor decomposition for all data modalities. In practice, we first estimate the number of factors for each modality. For a particular modality that does not admit a factor structure, we can set the corresponding Λ_m = 0 and x_m = u_m in (1).

3. Estimation

To fit model (3), we first estimate the latent variables F and U, along with the number of latent factors K_m, using some well established methods in the factor analysis literature. We then estimate ${\beta }_m^{*}$ through a penalized least squares approach.

First, we estimate the latent variables F and U, we adopt the method in Bai and Li (2012) and Fan et al. (2013), by running principal components analysis (PCA) on each individual modality X_m. We then estimate F_m by $\sqrt{n}$ times eigenvectors corresponding to the largest K_m eigenvalues of X_mX_m′. Denote this estimator by F_m. We next estimate Λ_m by Λ_m = (1/n)X_m′F_m, and estimate U_m by U_m = X_m − F_mΛ_m′, accordingly. Fan et al. (2013) showed that F_m is a consistent estimator, up to a rotation, of F_m, under a pervasive condition that the latent factors should affect many variables; see Condition 2 and its discussion in Section 4. We remark that, there are alternative ways to estimate the factors. For instance, methods such as Ma (2013); Cai et al. (2013), and Lock et al. (2013) may also be applicable, as long as the resulting factor estimates are consistent.

Next, we determine the number of the latent factors K_m in each modality, which is usually unknown in practice. We use the method of Bai and Ng (2002) to estimate K_m by

$${\widehat{K}}_m=\underset{0\le k\le \tilde{M}}{\text{argmin}}\text{\hspace{0.17em}log}\left\{\frac{1}{n{p}_m}{\Vert X_m-n^{-1}{F}_{m_k}{F}_{m_k}{}^{\prime }{X}_m\Vert }_F^2\right\}+kg\left(n,p_m\right),$$ (4)

where $\tilde{M}$ is a pre-defined upper bound on K_m, $F_{m_k}$ is $\sqrt{n}$ times eigenvectors corresponding to the largest m_k eigenvalues of X_mX_m′, and g(n, p_m) is a penalty function that,

$$g\left(n,p_m\right)=\frac{n+p_m}{n{p}_m}\text{log}\left(\frac{n{p}_m}{n+p_m}\right),\text{\hspace{1em}or\hspace{1em}}g\left(n,p_m\right)=\frac{n+p_m}{n{p}_m}\text{log}\left(\text{min}\left\{n,p_m\right\}\right).$$

For both choices, Bai and Ng (2002) showed that ${\widehat{K}}_m$ is a consistent estimator of K_m under some regularity conditions. We make two additional remarks about K_m. First, in this article, we treat K_m as fixed, which is reasonable in numerous scientific applications. As K_m is related to the number of spiked eigenvalues of X_mX_m′, it is usually small. Second, following a common practice of the factor analysis literature, in our subsequent theoretical analysis, we treat K_m as known. All the theoretical results remain valid conditioning on that there is a consistent estimator ${\widehat{K}}_m$ of K_m.

Next, we estimate β* and γ*. We replace F and U with the corresponding estimators F = (F₁, …, F_M) and U = (U₁, …, U_M), and solve a penalized least squares problem,

$$(\widehat{\gamma },\beta )=\underset{(\gamma ,\beta )}{\text{argmin}}\frac{1}{2n}\sum _{i=1}^n{\left(y_i-F_i{}^{\prime }\gamma -U_i{}^{\prime }\beta \right)}^2+\lambda \sum _{j=1}^{p}p\left(\left|{\beta }_j\right|\right),$$ (5)

where p(·) is some general folded-concave penalty function, and λ is a tuning parameter. This class of penalty functions includes SCAD (Fan and Lv, 2011) and MCP (Zhang, 2010). It assumes that p(t) is increasing and concave in t ≥ 0, and has a continuous first derivative ṗ(t) with ṗ(0+) > 0. This optimization problem can be solved by standard proximal gradient descent algorithms (Parikh et al., 2014). We also briefly comment that, in our optimization (5), we do not impose the linear constraint that γ = Λ′β, mainly because both F and Λ are unknown and unidentifiable in our setting. Besides, as we later show in Theorem 3 that, even without this constraint, the estimator β from (5) achieves the minimax rate, and can consistently select the support and estimate the nonzero components of β*, as long as there are enough variables to estimate the latent factors well. We tune λ in (5) using the standard cross-validation method, following Fan and Lv (2011), while alternative criteria, e.g., the extended BIC (Chen and Chen, 2008), can also be used to tune λ.

Finally, we estimate ${\sigma }_{ϵ}^2$ by ${\widehat{\sigma }}_{ϵ}^2=n^{-1}\sum _{i=1}^n{\left(y_i-x_i{}^{\prime }\beta \right)}^2$, where β is the Lasso estimator of β* obtained by $\beta ={\text{argmin}}_{\beta }{(2n)}^{-1}\sum _{i=1}^n{\left(y_i-x_i{}^{\prime }\beta \right)}^2+{\lambda }_{ϵ}{\Vert \beta \Vert }_1$, where λ_ϵ is the tuning parameter. We show in the Supplementary Materials that ${\widehat{\sigma }}_{ϵ}^2$ is consistent to ${\sigma }_{ϵ}^2$. Actually, any consistent estimator of ${\sigma }_{ϵ}^2$ would suffice for the subsequent hypothesis testing procedures. Alternative methods such as scaled LASSO (Sun and Zhang, 2013), refitted cross-validation (Fan et al., 2012), or directly using the residuals from (5) can all be applied to estimate ${\sigma }_{ϵ}^2$ as well.

4. Hypothesis test of a whole modality

A crucial question in multimodal data analysis is to evaluate if a whole modality is significantly associated with the outcome, given other modalities in the model. For instance, in multi-omics analysis, it is of interest to test if DNA methylation correlates with the phenotypic traits related to genetic disorders given gene expression level (Richardson et al., 2016). In multimodal neuroimaging analysis, it is of interest to evaluate if functional imaging quantification for hypometabolism associates with the diagnosis of Alzheimer’s disease, given structural magnetic resonance imaging of brain atrophy measurement (Zhang et al., 2011). The challenge here is that even a single modality often contains many more variables than the sample size.

This is essentially a problem of testing a high-dimensional subvector of β* in a high-dimensional regression model. Related testing problems have been extensively studied for a single modality data. For example, Zhang and Zhang (2014); van de Geer et al. (2014); Javanmard and Montanari (2014) developed bias-corrected or de-sparsifying methods to test if a fixed-dimensional subvector of β* in a high-dimensional linear or generalized linear model equals zero. In particular, under a general M-estimation framework, Ning and Liu (2017) proposed a decorrelated score test for the same problem, i.e., to test if a subvector ${\beta }_S^{*}=0$. They first showed that their score test statistic has a closed-form limiting distribution when the dimension of the subset | S | is fixed. They then extended to the case where ${\beta }_S^{*}$ can be any arbitrary subvector of β* with | S | diverging and even when | S |> n. Built on a pioneering work by Chernozhukov et al. (2013), they showed that the distribution of the supremum of the decorrelated score functions can be approximated by a multiplier bootstrap approach. Consequently, they employed bootstrap simulations to obtain the critical values of the limiting distribution to form the rejection region. Our test differs from Ning and Liu (2017). When | S | diverges, the test of Ning and Liu (2017) no longer has a closed-form limiting distribution, and they had to resort to bootstrap for critical values. By contrast, we are able to obtain a closed-form limiting distribution for our test when | S | diverges. This is due to that, instead of using the observed likelihood, we perform factor decomposition on X_m first, then use the factor model as a dimension reduction tool to reduce a high-dimensional test to a fixed-dimensional one. Our method does pay the extra price that we need to estimate the latent factors to plug into the likelihood function. However, as we show later, this extra cost can be well controlled. We also numerically compare with Ning and Liu (2017) in Section 7.1. We show that our test is as powerful, and often more powerful than the test of Ning and Liu (2017).

Formally, for our multimodal analysis, we aim at testing the following pair of hypotheses:

$$H_0:{\beta }_m^{*}=0\text{\hspace{1em}versus\hspace{1em}}{H}_a:{\beta }_m^{*}\ne 0.$$ (6)

We perform factor decomposition on the mth modality following (1). Then,

$$x^{\prime }\beta =x_{-m}{}^{\prime }{\beta }_{-m}+f^{\prime }{}_m{\gamma }_m+u_m{}^{\prime }{\beta }_m,$$

where γ_m = Λ′_mβ_m. The null hypothesis ${\beta }_m^{*}=0$ implies that ${\gamma }_m^{*}=0$, where ${\gamma }_m^{*}={\Lambda }^{\prime }{}_m{\beta }_m^{*}$. Therefore, under the null hypothesis, testing ${\gamma }_m^{*}$ is the same as testing ${\beta }_m^{*}$. The difference is that ${\gamma }_m^{*}\in {\mathcal{R}}^{K_m}$ is a low-dimensional vector, while ${\beta }_m^{*}\in {\mathcal{R}}^{p_m}$ is high-dimensional. As such, the factor model plays the role of dimension reduction for our testing problem. Actually, directly testing ${\beta }_m^{*}$ is challenging, since the dimension of ${\beta }_m^{*}$ diverges with the sample size, and there is not a closed form for the limiting distribution of ${\beta }_m-{\beta }_m^{*}$, where β_m is an estimator of ${\beta }_m^{*}$. On the other hand, we note that, under the alternative hypothesis, the magnitude of ${\gamma }_m^{*}$ can be different from that of ${\beta }_m^{*}$. As such, the power of the test that is built on ${\gamma }_m^{*}$ can be different from the one that is built on ${\beta }_m^{*}$. We later study the local power property of the test based on ${\gamma }_m^{*}$ in detail.

Next, we develop a factor-adjusted decorrelated score test, and show that it is asymptotically efficient when the latent factors can be well estimated. Following Ning and Liu (2017), and based on the Gaussian quasi-likelihood, we define the decorrelated score function as

$$S\left(\beta ,{\gamma }_m\right)=\frac{1}{n{\sigma }_{ϵ}^2}\sum _{i=1}^n\left(y_i-f_{i,m}{}^{\prime }{\gamma }_m-z_i{}^{\prime }\beta \right)\left(f_{i,m}-W^{*\prime }{z}_i\right),$$

where $z_i={\left(x^{\prime }{}_{i,-m},u^{\prime }{}_{i,m}\right)}^{\prime }\in {\mathcal{R}}^p$, p_−m = p − p_m, and $W^{*}=\text{E}{\left(x_{i,-m}^{\otimes 2}\right)}^{-1}\text{E}\left(x_{i,-m}{f}^{\prime }{}_{i,m}\right)\in {\mathcal{R}}^{p_{-m}\times K_m}$, which is essentially the projection of the latent factors onto the linear space spanned by x_−m. Such a projection is needed to control the variability of the high-order terms in establishing the central limit theorem in Theorem 1 (Ning and Liu, 2017). In the high-dimensional setting, we need some sparsity condition on W*; see Condition 4, and solve a regularized problem to obtain its consistent estimator. We treat the score function as a function of γ_m. Under the null hypothesis, we propose to estimate S(β, γ_m) by,

$$\widehat{S}\left({\beta }_{-m},0\right)=\frac{1}{n{\widehat{\sigma }}_{ϵ}^2}\sum _{i=1}^n\left(y_i-x_{i,-m}{}^{\prime }{\beta }_{-m}\right)\left(f_{i,m}-W^{\prime }{x}_{i,-m}\right),$$

where f′_i,m is the ith row of the estimated latent factor matrix F_m, ${\widehat{\sigma }}_{ϵ}^2$ is any consistent estimator of ${\sigma }_{ϵ}^2$ that satisfies Condition 5 below, and ${\beta }_{-m}\in {\mathcal{R}}^{p_{-m}}$ and $W\in {\mathcal{R}}^{p_{-m}\times K_m}$ are obtained by solving the following optimization problems,

$$\left({\beta }_{-m},{\widehat{\gamma }}_m\right)=\underset{\left({\beta }_{-m},{\gamma }_m\right)}{\text{argmin}}\frac{1}{2n}\sum _{i=1}^n{\left(y_i-x_{i,-m}{}^{\prime }{\beta }_{-m}-f^{\prime }{}_{i,m}{\gamma }_m\right)}^2+{\lambda }_1{\Vert {\beta }_{-m}\Vert }_1,$$ (7)

$$W=\text{argmin}{\Vert W\Vert }_1,\text{\hspace{0.17em}such\hspace{0.17em}that}{\Vert \frac{1}{n}\sum _{i=1}^n{x}_{i,-m}\left(f^{\prime }{}_{i,m}-x^{\prime }{}_{i,-m}W\right)\Vert }_{\infty }\le {\lambda }_2.$$ (8)

We make a few remarks regarding our score function and compare it to Ning and Liu (2017). First, we need to estimate the latent factors in the decorrelated score function, while in the score function of Ning and Liu (2017), the covariates are fully observed. This introduces an additional layer of complexity when analyzing the statistical property of the score function. Later in Theorems 1 and 4, we carefully evaluate the extra cost of estimating the latent factors. Second, we need to involve γ_m in (7), even under the null hypothesis γ_m = 0. This is because, if γ_m is removed from (7), β_−m is no longer consistent to ${\beta }_{-m}^{*}$ under the alternative, which would in turn impact the power of the test. On the other hand, ${\widehat{\gamma }}_m$ is not used in constructing the test statistic, but only β_−m is. Finally, in order to consistently estimate W*, we need to solve a non-typical Dantzig problem (8), where the latent factors are replaced by their corresponding estimators.

Next, we compute the variance of the score function by using the Fisher information. By the sandwich formula, the information matrix is

$$I_{{\gamma }_m\mid {\beta }_{-m}}^{*}={\sigma }_{ϵ}^{-2}\left\{\text{E}\left(f_{i,m}^{\otimes 2}\right)-\text{E}\left(f_{i,m}{x}^{\prime }{}_{i,-m}\right)\text{E}{\left(x_{i,-m}^{\otimes 2}\right)}^{-1}\text{E}\left(x_{i,-m}{f}_{i,m}{}^{\prime }\right)\right\}\in {\mathcal{R}}^{K_m\times K_m},$$

which can be estimated by

$$I_{{\gamma }_m|{\beta }_{-m}}={\widehat{\sigma }}_{ϵ}^{-2}\left\{\frac{1}{n}\sum _{i=1}^n{f}_{i,m}^{\otimes 2}-W^{\prime }\left(\frac{1}{n}\sum _{i=1}^n{x}_{i,-m}{f}_{i,m}{}^{\prime }\right)\right\}.$$

Then, our test statistic is given by

$$T_n=\sqrt{n}{I}_{{\gamma }_m\mid {\beta }_{-m}}^{-1/2}S\left({\beta }_{-m},0\right).$$

We next show that, under the null hypothesis, the asymptotic distribution of T_n is $N\left(0,I_{K_m}\right)$. In other words, T_n is asymptotically efficient. We first begin with a set of conditions.

Condition 1. For m ∈[M], suppose ${\left\{{\left(f^{\prime }{}_{i,m},u^{\prime }{}_{i,m}\right)}^{\prime }\right\}}_{i=1}^n$ are i.i.d. uncorrelated sub-Gaussian random vectors with zero mean. That is, E(f_i,m) = 0, E(u_i,m) = 0, and E(f_i,mu_i,m′) = 0. Moreover, $\text{E}\left\{\text{exp}\left(t{\alpha }^{\prime }{f}_{i,m}\right)\right\}\le \text{exp}\left(C{\Vert \alpha \Vert }_2^2{t}^2/2\right)$, and $\text{E}\left\{\text{exp}\left(t{\alpha }^{\prime }{u}_{i,m}\right)\right\}\le \text{exp}\left(C{\Vert \alpha \Vert }_2^2{t}^2/2\right)$, for some constant C. In addition, for all k ∈ [K_m], $x_{i,-m}{}^{\prime }{w}_k^{*}$ are i.i.d. sub-Gaussian such that $\text{E}\left\{\text{exp}\left(t{x}_{i,-m}{}^{\prime }{w}_k^{*}\right)\right\}\le \text{exp}\left(C{t}^2/2\right)$, where $w_k^{*}$ is the kth column of W*. Additionally, ${\left\{{ϵ}_i\right\}}_{i=1}^n$ are i.i.d. sub-Gaussian with zero mean, and ϵ_i is uncorrelated with (f′_i,m, u′_i,m)′ for all m ∈[M].

Condition 2. For m ∈[M], suppose 0 < c ≤ λ_min (Λ_m′Λ_m/p_m) ≤ λ_max (Λ_m′Λ_m/p_m) ≤ C < ∞, for some positive constants c and C.

Condition 3. For m ∈[M], s, t ∈[p_m], i, j ∈[n], suppose $\text{E}\left[p_m^{-1/2}{\left\{u_{i,m}{}^{\prime }{u}_{j,m}-\text{E}\left(u_{i,m}{}^{\prime }{u}_{j,m}\right)\right\}}^4\right]\le C$, and $\text{E}{\Vert p_m^{-1/2}{\Lambda }_m{}^{\prime }{u}_{i,m}\Vert }_2^4\le C$. Moreover, ${\Vert {\Lambda }_m\Vert }_{\infty }\le C$, ${\lambda }_{\text{min}}\left({\Sigma }_{u_m}\right)>c$, ${\Vert {\Sigma }_{u_m}\Vert }_1\le C$, where ${\Sigma }_{u_m}=\text{Var}\left(u_m\right)$, and ${\text{min}}_{s,t\in \left[p_m\right]}\text{\hspace{0.17em}Var}\left(u_{i,m_s}{u}_{i,m_t}\right)>c$.

Condition 4. Let $s_w^{*}={\text{max}}_{k\in \left[K_m\right]}\left|\text{supp}\left(w_k^{*}\right)\right|$. Suppose $s_w^{*}\text{\hspace{0.17em}log}\left(p_{-m}\right)\left\{1\vee \left(n^{1/4}/\sqrt{p_m}\right)\right\}=o\left(n^{1/2}\right)$, and $[s_{-m}^{*}\{\sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}+1/\sqrt{p_m}\}]\cdot \sqrt{\text{log}\left(p_{-m}\right)}\left\{1\vee \left(n^{1/4}/\sqrt{p_m}\right)\right\}=o$(1).

Condition 5. Suppose ${\widehat{\sigma }}_{ϵ}^2={\sigma }_{ϵ}^2+O_P$(1).

Condition 6. Suppose $0<c\le {\lambda }_{\text{min}}\left(I_{{\gamma }_m|{\beta }_{-m}}^{*}\right)$.

Condition 1 is a typical sub-Gaussian assumption for high-dimensional problems. Condition 2 is the pervasive condition, and is common in factor analysis (Fan et al., 2013). It requires that the latent factors affect a large number of variables. This is reasonable for a variety of multimodal data. For instance, in multi-omics data, some genetic factors are believed to impact both gene expression and DNA methylation, and in multimodal neuroimaging, some neurological factors affect both brain structures and functions. Condition 3 imposes some technical requirements on the loading matrix and idiosyncratic component. Together, Conditions 2 and 3 ensure that f_i,m and u_i,m can be consistently estimated by the PCA method (Fan et al., 2013). Condition 4 is a sparsity condition on W* and ${\beta }_{-m}^{*}$, which requires $s_w^{*}$ and $s_{-m}^{*}$ to be much smaller than n. Under such a condition, W* and ${\beta }_{-m}^{*}$ can both be consistently estimated, even if the latent factors are unknown; see Lemmas 6 and 8 in the Supplementary Materials for more details. We remark that this sparsity assumption on W* is weaker and more flexible than requiring both $\text{E}{\left(x_{i,-m}^{\otimes 2}\right)}^{-1}$ and E(x_i,−mf′_i,m) are sparse. Condition 5 ensures the estimator of ${\sigma }_{ϵ}^2$ is consistent. Condition 6 ensures the information matrix is invertible. We also remark that, if there is no factor in the mth modality, Condition 1 reduces to the sub-Guassian assumption on x_i,m, and Conditions 2 and 3 are no longer needed for that modality.

We next obtain a closed-form limiting distribution for the test statistic T_s.

Theorem 1. Suppose Conditions 1–6 hold. Suppose ${\lambda }_1\asymp \sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}+1/\sqrt{p_m}$, and ${\lambda }_2\asymp \sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}\left\{1\vee \left(n^{1/4}/\sqrt{p_m}\right)\right\}$. Then, under $H_0:{\beta }_m^{*}=0$, it holds that

$$T_n\stackrel{D}{\to }N\left(0,I_{K_m}\right).$$

By Theorem 1, we reject the null hypothesis if $n{\left\{\widehat{S}\left({\beta }_{-m},0\right)\right\}}^{\prime }{I}_{{\gamma }_m\mid {\beta }_{-m}}^{-1}\widehat{S}\left({\beta }_{-m},0\right)>{\chi }_{\alpha }^2\left(K_m,0\right)$, where ${\chi }_{\alpha }^2\left(K_m,0\right)$ is the α-upper quantile of the χ² -distribution with K_m degrees of freedom.

We next explicitly discuss the benefit and the extra cost of our factor-based test when compared with Ning and Liu (2017). The main difference is that, through latent factors, we obtain a closed-form limiting distribution and do not have to resort to bootstrap. The price we pay mainly lies in Condition 4 and the choices of λ₁ and λ₂. Actually, the extra term $1/\sqrt{p_m}$ appearing in both Condition 4 and λ₁, λ₂ reflects the estimation error caused by using f_i,m to estimate ${\beta }_{-m}^{*}$. The term $n^{1/4}/\sqrt{p_m}$ is due to the same reason for estimating $w_k^{*}$. Therefore, the choices of the tuning parameters λ₁ and λ₂ need to be adjusted accordingly, by taking into account such extra estimation errors.

We further consider three scenarios. First, when p_m ≫ n, both $1/\sqrt{p_m}$ and $n^{1/4}/\sqrt{p_m}$ are dominated by $\sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}$. Therefore, the estimation errors of β_−m and W reach the optimal oracle rate, i.e. the best rate as if the latent factors were known; see Lemmas 6 and 8 in the Supplementary Materials. In this case, using the factor estimates actually does not incur any extra cost. The reason is that many variables are used to estimate the latent factors, and its estimation error is so small that it would not affect the inference on ${\beta }_m^{*}$. Second, when p_m = o(n), the estimation errors of β_−m and W would be greater than the optimal rate. However, the central limit theorem still holds, given proper choices of λ₁ and λ₂, and more stringent sparsity conditions on ${\beta }_{-m}^{*}$ and W* in Condition 4. Third, in a special case where variables in all modalities are driven by exactly the same latent factors, even we perform the hypothesis test on the mth modality, we could use variables from all different modalities to estimate the latent factors. Then, the terms $1/\sqrt{p_m}$ and $n^{1/4}/\sqrt{p_m}$ become $1/\sqrt{p}$ and $n^{1/4}/\sqrt{p}$, respectively, which are naturally dominated by $\sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}$. In this case, the optimal oracle rate is again attained. Such a result can be viewed as a blessing of the dimensionality for the factor model. In summary, our method is most suitable for testing the significance of a modality containing many variables, or for multimodal data with a large number of variables driven by some common latent factors.

Next, we study the power of the proposed test under the local alternative $H_{a_n}:{\beta }_m^{*}=b_{m_n}$, where $b_{m_n}$ is a sequence converging to 0 as n → ∞. Since we use the latent factors to transform the test on ${\beta }_m^{*}$ to the one on ${\gamma }_m^{*}$, we show that the local power of the test depends on $c_{m_n}={\Lambda }_m{}^{\prime }{b}_{m_n}$. We consider the following parameter space under the local alternative, $\mathcal{N}=\left\{{\beta }^{*}:{\beta }_m^{*}=b_{m_n},\left|\text{supp}\left({\beta }_{-m}^{*}\right)\right|=s_{-m}^{*},\text{\hspace{0.17em}where\hspace{0.17em}}{s}_{-m}^{*}\ll n\right\}$. The next theorem gives the limiting distribution of Q_n = T_n′T_n uniformly for all ${\beta }^{*}\in \mathcal{N}$ under the local alternative.

Theorem 2. Suppose the conditions of Theorem 1 hold. Suppose ${\lambda }_1\asymp \sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}+1/\sqrt{p_m}$, ${\lambda }_2\asymp \sqrt{\left(\text{log\hspace{0.17em}}{p}_{-m}\right)/n}\left\{1\vee \left(n^{1/4}/\sqrt{p_m}\right)\right\}$, ${\Vert b_{m_n}\Vert }_2=o(1/\sqrt{\text{log\hspace{0.17em}}n})$, and ${\Vert c_{m_n}\Vert }_2=o(1/\sqrt{\text{log\hspace{0.17em}}n})$. Then, under the $H_{a_n}$, it holds uniformly for all ${\beta }^{*}\in \mathcal{N}$ that

$$\underset{x>0}{\text{sup}}\left|\text{Pr}\left(Q_n\le x\right)-\text{Pr}\left\{{\chi }^2\left(K_m,h_{m_n}\right)\le x\right\}\right|\to 0,$$

where $h_{m_n}=n{c}_{m_n}{}^{\prime }{I}_{{\gamma }_m\mid {\beta }_{-m}}^{*}{c}_{m_n}$.

Since $h_{m_n}\asymp n{\Vert c_{m_n}\Vert }_2^2$, Theorem 2 implies that the local power of our test essentially depends on ${\Vert c_{m_n}\Vert }_2$. If we let ${\Vert c_{m_n}\Vert }_2=C{n}^{-{\varphi }_{{\gamma }_m}}$, the local power is to exhibit some transition behavior depending on the value of ${\varphi }_{{\gamma }_m}$, which is summarized in the next corollary.

Corollary 1. Suppose the conditions of Theorem 2 hold. Then,

1. $\underset{n\to \infty }{\text{lim}}\underset{{\beta }^{*}\in \mathcal{N}}{\text{sup}}\underset{x>0}{\text{sup}}\left|\text{Pr}\left(Q_n\le x\right)-\text{Pr}\left({\chi }^2\left(K_m,0\right)\le x\right)\right|\to 0$, if ${\varphi }_{{\gamma }_m}>1/2$;

2. $\underset{n\to \infty }{\text{lim}}\underset{{\beta }^{*}\in \mathcal{N}}{\text{sup}}\underset{x>0}{\text{sup}}\left|\text{Pr}\left(Q_n\le x\right)-\text{Pr}\left({\chi }^2\left(K_m,h\right)\le x\right)\right|\to 0$, if ${\varphi }_{{\gamma }_m}=1/2$;

3. $\underset{n\to \infty }{\text{lim\hspace{0.17em}inf}}\underset{{\beta }^{*}\in \mathcal{N}}{\text{sup}}\text{\hspace{0.17em}Pr}\left(Q_n>x\right)=1$, if ${\varphi }_{{\gamma }_m}<1/2$;

where $h=\underset{n\to \infty }{\text{lim}}n{c}_{m_n}{}^{\prime }{I}_{{\gamma }_m|{\beta }_{-m}}^{*}{c}_{m_n}$ in (b), and (c) holds for any x > 0.

We make some remarks. First, Corollary 1 shows that, the local power is to converge to the type I error if ${\varphi }_{{\gamma }_m}>1/2$; to a non-central χ² -distribution if ${\varphi }_{{\gamma }_m}=1/2$; and to 1 if ${\varphi }_{{\gamma }_m}<1/2$. Such a transition behavior is analogous to the classical local power results, which showed that the root-n local alternative is the transition point of the local power (van der Vaart, 2000). Second, the existing debiased method (van de Geer et al., 2014) and the decorrelated method (Ning and Liu, 2017) only established the root-n local power results when the dimension of the parameters being tested is fixed. Moreover, even though Ning and Liu (2017) utilized a multiplier bootstrap method to extend their test from a single parameter to arbitrarily many parameters, they only studied the local power when testing a single parameter. Our local power result differs in that we allow the dimension of ${\beta }_m^{*}$ to grow with n, whereas we fix the dimension of ${\gamma }_m^{*}$. Finally, Theorem 2 shows that the local power depends on the magnitude of ${\gamma }_m^{*}$, or $c_{m_n}={\Lambda }_m{}^{\prime }{b}_{m_n}$. This is again due to that we transform the test of ${\beta }_m^{*}$ to that of ${\gamma }_m^{*}$. Therefore, the power of our test depends on the relation between the loadings and where the alternative hypothesis occurs.

Next, we give some specific examples to further illustrate the power behavior of our proposed test. To simplify the discussion, we set K_m = 1.

Example 1. Let Λ_m = (D_L, 1, …, 1)′ and $b_{m_n}={\left(D_S{n}^{-1/2},0,\dots ,0\right)}^{\prime }$, where D_L and D_S are two constants. In this case, ${\Vert c_{m_n}\Vert }_2=D_S{D}_L{n}^{-1/2}$, and thus the power of our test depends on the product D_SD_L. On the contrary, even one had known apriori that the alternative only occurs at the first coordinate, and performs a debiased or decorrelated test on that coordinate, its local power depends on D_S. Therefore, when D_L is large and D_S is small, our testing method gains power. On the other hand, when D_L is small and D_S is large, the alternative methods may be more powerful.

Example 2. Let Λ_m = (1, 1, …, 1)′, and $b_{m_n}={\left(C_1{n}^{-2/3},C_2{n}^{-2/3}\dots ,C_L{n}^{-2/3},0,\dots ,0\right)}^{\prime }$. In this case, if L ≫ n^1/6, ${\Vert c_{m_n}\Vert }_2\gg n^{-1/2}$, then the power of our test converges to 1. On the contrary, if one performs a debiased or decorrelated test on each element of ${\beta }_m^{*}$, there is no power. Our testing method gains power in this example too.

Example 3. Let Λ_m = (0, 1, …, 1)′, and $b_{m_n}={\left(c_n,0,\dots ,0\right)}^{\prime }$. In this case, no matter how large c_n is, our testing method has no power to detect the alternative, because $c_{m_n}=0$.

As we have seen in these examples, when transforming the test from ${\beta }_m^{*}$ to ${\gamma }_m^{*}$, our method does not necessarily lose power, but can gain power in some situations. For instance, in Example 1, the variables with large loadings on the latent factors have nonzero coefficients, while in Example 2, many variables with nonzero loadings have nonzero coefficients. In such cases, our test gains power. On the other hand, in Example 3, the product of the loadings and the nonzero coefficients is small, then our test has little power.

5. Hypothesis test of linear combinations of predictors of one or more modalities

Another important question in multimodal data analysis is to test if some linear combinations of predictors, within the same modality or across different modalities, is significantly correlated with the response. This is because multimodal data often measures different aspects of related quantities. For instance, in multi-omics studies, expression data measures how genes are expressed, methylation data measures how DNA molecules are methylated, and both data may be related to the same set of genes. In multimodal neuroimaging analysis, brain structures, functions, and chemical constituents of the same brain regions are often measured simultaneously. As such, it is of great scientific interest to test if various measurements on a particular gene or brain region are associated with the outcome.

Shi et al. (2019) considered a similar testing problem in a high-dimensional generalized linear model for a single modality data. They derived the corresponding partially penalized likelihood ratio test, score test and Wald test, and showed that the three tests are asymptotically equivalent. They allowed the dimension of the model to grow with the sample size, as long as the dimension of the subvector being tested and the number of linear combinations are smaller than the sample size. Our method differs from Shi et al. (2019) in several ways. Shi et al. (2019) treated the design matrix X as fixed, while we treat X as i.i.d. random realizations from some distributions. More importantly, we do not directly use the observed X, but instead perform a factor decomposition and use the decorrelated idiosyncratic components as the pseudo design matrix. We explicitly show in Theorem 3 that such a factor-adjusted step leads to less stringent conditions to reach the variable selection and estimation consistency. Moreover, since variable selection consistency is needed to correctly calculate the variance of the test statistic, as shown in Theorem 4, our method also requires less stringent conditions to establish the limiting distribution of the test statistic. Moreover, our model concerns with data with multiple modalities, instead of a single modality as in Shi et al. (2019). High correlations are commonly observed in multimodal data, and as such the factor-adjusted decorrelation step becomes essential. Relatedly, Zhu and Bradic (2018) proposed a test for a linear combination of predictors under a unimodal linear regression model. Even though they did not restrain the size or the sparsity of the model, they only considered a single linear combination, and required the eigenvalues of the covariate covariance matrix Var(x) to be bounded. By contrast, both Shi et al. (2019) and we consider jointly testing multiple linear combinations of predictors, and we do not require the eigenvalues of Var(x) to be bounded. We further numerically compare with Shi et al. (2019) and Zhu and Bradic (2018) in Section 7.2.

Formally, we consider testing the following pair of hypotheses:

$$H_0:A{\beta }_T^{*}=b\text{\hspace{1em}versus\hspace{1em}}{H}_a:A{\beta }_T^{*}\ne b,$$ (9)

where $A\in {\mathcal{R}}^{r\times t}$, $b\in {\mathcal{R}}^r$, ${\beta }_T^{*}\in {\mathcal{R}}^t$ is a subvector of β*, and T ⊂[p] is a low-dimensional index set with | T |= t < n. This simultaneously tests r linear combinations of ${\beta }_T^{*}$, with r < n. We next develop a factor-adjusted Wald test.

To construct the test statistic, we first consider a penalized least squares problem,

$$\left({\widehat{\gamma }}_a,{\beta }_a\right)=\underset{(\gamma ,\beta )}{\text{argmin}}\frac{1}{2n}\sum _{i=1}^n{\left(y_i-F_i{}^{\prime }\gamma -U_i{}^{\prime }\beta \right)}^2+{\lambda }_a\sum _{j\cancel{\in }T}p\left(\left|{\beta }_j\right|\right).$$ (10)

This is essentially the same as (5), except that, instead of penalizing all variables in β, we do not penalize β_j for j ∈T. This is to avoid introducing bias when estimating ${\beta }_j^{*}$ for j ∈T, which is needed for Theorem 4. A similar idea was also adopted in Shi et al. (2019).

Given β_a, our factor-adjusted Wald test statistic is given by

$$T_w={\left(A{\beta }_{a,T}-b\right)}^{\prime }{\left(A{\Omega }_T{A}^{\prime }\right)}^{-1}\left(A{\beta }_{a,T}-b\right)/{\widehat{\sigma }}_{ϵ}^2,$$

where β_a,T is the sub-vector of β_a with indices in T, Ω_T is the first T rows and columns of

$${\Omega }_{T\cup {\widehat{S}}_a}=n{\left(\begin{array}{cc}{U}_T{}^{\prime }{U}_T& U_T{}^{\prime }{U}_{{\widehat{S}}_a}\\ U_{{\widehat{S}}_a}{}^{\prime }{U}_T& U_{{\widehat{S}}_a}{}^{\prime }{U}_{{\widehat{S}}_a}\end{array}\right)}^{-1},$$

${\widehat{S}}_a=\left\{j\in T^c:{\widehat{\beta }}_{a,j}\ne 0\right\}$, and ${\widehat{\sigma }}_{ϵ}^2$ is any consistent estimator of ${\sigma }_{ϵ}^2$. In this test statistic, ${\widehat{S}}_a$ plays a critical role in calculating the variance of Aβ_a,T − b. In fact, ${\widehat{S}}_a$ needs to be consistent to $S_a=\left\{j\in T^c:{\beta }_j^{*}\ne 0\right\}$ in order for the variance to be valid. Such a consistency is guaranteed by Theorem 3.

We next present a set of regularity conditions.

Condition 7. Suppose c ≤ λ_min{E(u^⊗2)} ≤ λ_max{E(u^⊗2)} ≤ C for some postive constants c and C.

Condition 8. Suppose ${\Vert \text{E}{\left(u_{T\cup S_a}^{\otimes 2}\right)}^{-1}\Vert }_{L_{\infty }}\le C$.

Condition 9. Suppose ${\Vert \text{E}\left(u_{{\left(T\cup S_a\right)}^c}{u}_{T\cup S_a}^{\prime }\right)\left\{\text{E}{\left(u_{T\cup S_a}^{\otimes 2}\right)}^{-1}\right\}\Vert }_{L_{\infty }}\le C$.

Condition 10. Suppose $d_n=\text{min}\left\{\left|{\beta }_j^{*}\right|:{\beta }_j^{*}\ne 0\right\}/2\gg {\lambda }_a\gg {\delta }_n$, where ${\delta }_n=\sqrt{(\text{log\hspace{0.17em}}p)/n}\left\{1\vee \left(n^{1/4}/\sqrt{p_{\text{min}}}\right)\right\}$, p_min = min_{m ∈[M]} p_m, and λ_aṗ(d_n) = o(δ_n), where ṗ is the first derivative.

We first note that Conditions 7–9 are imposed on u, instead of on x. Since u can be viewed as the residual of x after the latent factors are removed, the correlations among the variables in u are much weaker than those in x. In particular, Conditions 7 and 8 are needed to avoid singularity of E(u^⊗2) and $\text{E}\left(u_{T\cup S_a}^{\otimes 2}\right)$. Condition 9 is the well-known irrepresentable condition, which is necessary for establishing the variable selection consistency. Shi et al. (2019) required such a condition to hold for the Gram matrix X′X, which essentially requires the correlations among X must be small. This condition hardly holds for multimodal data. By contrast, we only impose such a condition on E(u^⊗2), which requires the idiosyncratic components not to be highly correlated. This condition is well accepted in the factor model literature. Indeed, when an exact factor model is assumed, E(u^⊗2) is a diagonal matrix, then Condition 7 naturally holds.

We now establish the variable selection and estimation consistency of the estimator β_a in (10), which is essential for deriving the asymptotic distribution of T_w.

Theorem 3. Suppose Conditions 1–3 and 7–10 hold. Then there exists a solution (${\widehat{\gamma }}_a$, β_a) of (10) such that, with probability tending to 1, the following results hold:

1. (sign consistency) sign(β_a) = sign(β*);

2. (L_∞ consistency) ${\Vert {\beta }_{a,T\cup S_a}-{\beta }_{T\cup S_a}^{*}\Vert }_{\infty }=O_P\left({\delta }_n\right)$;

3. (L₂ consistency) ${\Vert {\beta }_{a,T\cup S_a}-{\beta }_{T\cup S_a}^{*}\Vert }_2=O_P\left(\sqrt{t+S_a}{\delta }_n\right)$, where s_a =| S_a |;

4. (asymptotic expansion) $\sqrt{n}\left({\beta }_{a,T\cup S_a}-{\beta }_{T\cup S_a}^{*}\right)=n^{-1/2}{K}_n^{-1}{U}^{\prime }{}_{T\cup S_a}ϵ+o_P$(1), where $K_n=(1/n)U_{T\cup S_a}{}^{\prime }{U}_{T\cup S_a}$, if we further have that p_min ≫ n^3/2, and $\sqrt{n}{\lambda }_a\dot{p}\left(d_n\right)=o$(1).

We again explicitly examine the benefit and the extra cost of our factor-based test compared with Shi et al. (2019). The main difference is that we obtain the variable selection and estimation consistency under much weaker conditions than Shi et al. (2019). The price we pay lies in δ_n, which reflects the convergence rates in (b) and (c) of Theorem 3. Particularly, the component $n^{1/4}/\sqrt{p_{\text{min}}}$ in δ_n is due to the factor estimation. We consider two scenarios. First, when all data modalities have a large number of variables, i.e. p_min ≫ n^1/2, then ${\delta }_n=\sqrt{(\text{log\hspace{0.17em}}p)/n}$, which makes the convergence rates in (b) and (c) to be minimax optimal. This is because when there are enough variables to estimate the latent factors well, the extra factor estimation error becomes so small that it would not affect the estimation error on β_a. Second, when one modality has only a small number of variables, i.e. p_m = o(n^1/4) for some m ∈[M], estimating the latent factors in that modality becomes challenging, and the resulting estimation error would slow the convergence of β_a. In this case, one possible alternative solution is to skip factor decomposition for that particular modality, but directly use X_m in (10) and solve

$$\left({\widehat{\gamma }}_a,{\beta }_a\right)=\underset{(\gamma ,\beta )}{\text{argmin}}\frac{1}{2n}\sum _{i=1}^n{\left(y_i-X_{i,m}{}^{\prime }{\beta }_m-F_{i,-m}{}^{\prime }\gamma -U_{i,-m}{}^{\prime }{\beta }_{-m}\right)}^2+{\lambda }_a\sum _{j\cancel{\in }T}p\left(\left|{\beta }_j\right|\right),$$

where X_i,m′, F_i,−m′ and U′_i,−m denote the ith row of X_−m, F_−m and U_−m, respectively. Finally, we note that the variable selection and estimation consistency of β in (5) is directly implied by Theorem 3 if we treat T as the empty set.

Next, we study the asymptotic distribution of our test statistic T_w, and show that it can be uniformly approximated by a χ² -distribution under both H₀ and H_a. We need two more regularity conditions.

Condition 11. Suppose ${\Vert h_n\Vert }_2=O(\sqrt{r/n})$, and λ_max{(AA′)⁻¹} ≤ C for some constant C, where $h_n=A{\beta }_T^{*}-b$.

Condition 12. Suppose $r^{1/4}{n}^{-1/2}\text{E}{\left|u^{\prime }{}_{T\cup S_a}{\Sigma }_{u,T\cup S_a}^{-1}{u}_{T\cup S_a}\right|}^{3/2}\to 0$, where ${\Sigma }_{u,T\cup S_a}^{-1}$ is the inverse of the submatrix of Σ_u with rows and columns in T ∪ S_a.

Condition 11 regulates the local alternative h_n and avoids singularity of AA′. Condition 12 is a Lyapunov condition to ensure the asymptotic normality of ${\beta }_{a,T\cup S_a}$, which is the key to establish the χ² -approximation.

Theorem 4. Suppose the conditions of Theorem 3 and Conditions 11 and 12 hold, p_min ≫ n^3/2, $\sqrt{n}{\lambda }_a\dot{p}\left(d_n\right)=o$(1), and t + s_a = o(n^1/3). Then it holds that

$$\underset{x}{\text{sup}}\left|\text{Pr}\left(T_w\le x\right)-\text{Pr}\left\{{\chi }^2\left(r,v_n\right)\le x\right\}\right|\to 0,$$

where $v_n=n{h}_n{}^{\prime }{\left(A{\Omega }_T{A}^{\prime }\right)}^{-1}{h}_n/{\sigma }_{ϵ}^2$, Ω_T is the the submatrix of ${\Sigma }_{u,T\cup S_a}^{-1}$ with rows and columns in T.

By Theorem 4, we reject $H_0:A{\beta }_T^{*}=b$ if $T_w>{\chi }_{\alpha }^2(r,0)$, where ${\chi }_{\alpha }^2(r,0)$ is the α-upper quantile of the χ² -distribution with r degrees of freedom. The limiting distribution we establish in Theorem 4 is the same as the classical Wald test result for a low-dimensional linear regression model (Shi et al., 2019).

We also remark that, the requirement of p_min ≫ n^3/2 in Theorem 4 ensures that the latent factors in each modality can be well estimated. Therefore, the extra factor estimation error would not affect the limiting distribution of T_w. This condition is more stringent than that of p_min ≫ n^1/2, which guarantees the minimax optimal rate of estimation in Theorem 3. This is because hypothesis testing is a more challenging task than estimation.

Finally, write ${\Vert {\left(A{\Omega }_T{A}^{\prime }\right)}^{-1/2}{h}_n\Vert }_2=C{n}^{-{\varphi }_v}$ for some constant C > 0, and let $\tilde{\mathcal{N}}=\left\{{\beta }^{*}:{\Vert A{\beta }_T^{*}-b\Vert }_2=O(\sqrt{r/n}),t+s_a=o\left(n^{1/3}\right)\right\}$. Theorem 4 implies the following corollary regarding the local power of the proposed test. Its proof is similar to that for Corollary 1 and is omitted.

Corollary 2. Suppose the conditions of Theorem 4 hold. Then,

1. $\underset{n\to \infty }{\text{lim}}\underset{{\beta }^{*}\in \mathcal{N}}{\text{sup}}\underset{x>0}{\text{sup}}\left|\text{Pr}\left(T_w\le x\right)-\text{Pr}\left({\chi }^2(r,0)\le x\right)\right|\to 0$, if ϕ_v > 1/2;

2. $\underset{n\to \infty }{\text{lim}}\underset{{\beta }^{*}\in \mathcal{N}}{\text{sup}}\underset{x>0}{\text{sup}}\left|\text{Pr}\left(T_w\le x\right)-\text{Pr}\left({\chi }^2(r,v)\le x\right)\right|\to 0$, if ϕ_v = 1/2;

3. $\underset{n\to \infty }{\text{lim\hspace{0.17em}inf}}\underset{{\beta }^{*}\in \mathcal{N}}{\text{sup}}\text{\hspace{0.17em}Pr}\left(T_w>x\right)=1$, if ϕ_v < 1/2;

where $v=\underset{n\to \infty }{\text{lim}}{v}_n$ in (b), and (c) holds for any x > 0.

6. Quantification of contribution of a single modality

In addition to testing the significance of a whole data modality, it is of equal interest to quantify the amount of contribution of a modality conditioning on other data modalities in the regression model. As an example, in heritability analysis, the goal is to evaluate the contribution of genetic effects to the phenotype in addition to the environmental effects (Lynch et al., 1998). Motivated by the proportion of the response variance explained in the classical linear regression, we propose a measure of the contribution of a single data modality in our integrative factor regression model.

Let $x_{-m}\in {\mathcal{R}}^{p-p_m}$ denote the subvector of $x\in {\mathcal{R}}^p$ excluding $x_m\in {\mathcal{R}}^{p_m}$, and $X_{-m}\in {\mathcal{R}}^{n\times \left(p-p_m\right)}$ denote the submatrix of $X\in {\mathcal{R}}^{n\times p}$ excluding $X_m\in {\mathcal{R}}^{n\times p_m}$. To evaluate the contribution of x_m, our key idea is to compare the goodness-of-fit of regressing y on x to that of regressing y on x_−m. Toward that end, under model (2), we define

$${\sigma }_{m\mid -m}^2=\text{Var}\left(x_m{}^{\prime }{\beta }_m^{*}\mid x_{-m}\right).$$

Next, we present a proposition regarding ${\sigma }_{m\mid -m}^2$, where statements (a) and (b) show in two different ways that ${\sigma }_{m\mid -m}^2$ can be interpreted as the improvement of the goodness-of-fit, or equivalently, additional variance of the response explained, given by the mth modality in addition to all other modalities. This justifies why ${\sigma }_{m\mid -m}^2$ can be used to quantify the contribution of a single modality. To simplify the presentation, we only consider the case where p < n. We then discuss that such an interpretation of ${\sigma }_{m\mid -m}^2$ holds for p > n as well. Next, statement (c) shows that, if x_m and x_−m share some common factors, in that x_m = Λ_mf + u_m, and x_−m = Λ_−mf + u_−m, where $f\in {\mathcal{R}}^K$ we then have a closed-form expression for ${\sigma }_{m\mid -m}^2$. This expression holds true regardless of p < n or p > n, and thus provides a unified way of computing ${\sigma }_{m\mid -m}^2$ in practice.

Proposition 1. Suppose x follows a multivariate normal distribution and p < n. Let Y and Y_−m denote the predicted response by regressing y on x, and regressing y on x_−m, respectively, via least squares. Let ${\sigma }_y^2=\text{Var}(y)$. Then the following results hold:

1. ${\sigma }_{m\mid -m}^2=\text{E}{\Vert Y-Y_{-m}\Vert }_2^2/\left(n-p_{-m}\right)-{\sigma }_{ϵ}^2$;

2. ${\sigma }_{m\mid -m}^2={\sigma }_y^2\left(r^2-r_{-m}^2\right)$, where $r^2=1-\text{E}{\Vert Y-Y\Vert }_2^2/\left\{(n-p){\sigma }_y^2\right\}$, and $r_{-m}^2=1-\text{E}{\Vert Y-Y_{-m}\Vert }_2^2/\left\{\left(n-p_{-m}\right){\sigma }_y^2\right\}$;

3. ${\sigma }_{m\mid -m}^2={\beta }_m^{*}{}^{\prime }\left\{{\Lambda }_m{\left(I_K+{\Lambda }_{-m}{}^{\prime }{\Sigma }_{u_{-m}}^{-1}{\Lambda }_{-m}\right)}^{-1}{\Lambda }_m{}^{\prime }+{\Sigma }_{u_m}\right\}{\beta }_m^{*}$.

By Proposition 1(a), when regressing y using all but the mth modality, we have $\text{E}{\Vert Y-Y_{-m}\Vert }_2^2=\left(n-p_{-m}\right)\left({\sigma }_{ϵ}^2+{\sigma }_{m\mid -m}^2\right)$. On the other hand, when regressing y on all data modalities, we have $\text{E}{\Vert Y-Y\Vert }_2^2=(n-p){\sigma }_{ϵ}^2$. Therefore, from a goodness-of-fit perspective, ignoring x_m leads to a “worsened” prediction by an amount of ${\sigma }_{m\mid -m}^2$.

For Proposition 1(b), recall in the classical linear regression model, the adjusted R² measures the percentage of total variation in the response that has been explained by the predictors, and is defined as R² = 1 − {RSS / (n − p)}/{TSS / (n − 1)}, where RSS and TSS are the residual sum of squares and total sum of squares, respectively. Then, r² in Proposition 1(b) can be viewed as an “expected” percentage of total variation in the response explained, in that,

$$r^2=1-\frac{\text{E}(RSS)/(n-p)}{\text{E}(TSS)/(n-1)}=1-\frac{\text{E}{\Vert Y-Y\Vert }_2^2}{(n-p){\sigma }_y^2}.$$

As we show in the proof of Proposition 1, when using all but the mth modality, the “expected” percentage of total variation in the response explained is $r_{-m}^2=1-\left({\sigma }_{ϵ}^2+{\sigma }_{m\mid -m}^2\right)/{\sigma }_y^2$, where ${\sigma }_{ϵ}^2=\text{E}{\Vert Y-Y\Vert }_2^2/(n-p)$. On the other hand, when using all data modalities, the “expected” percentage of total variation in the response explained is $r^2=1-{\sigma }_{ϵ}^2/{\sigma }_y^2$. Therefore, using the mth modality improves the “expected” percentage of total variation in the response explained by an amount of ${\sigma }_{m\mid -m}^2/{\sigma }_y^2$.

We have so far justified ${\sigma }_{m\mid -m}^2$ in the setting where p < n. In the setting where p > n and the true model is sparse, in that there are only s variables associate with y with s < n, we can still use ${\sigma }_{m\mid -m}^2$ to quantify the contribution of an individual modality. This is because Proposition 1(a) and (b) continue to hold if we replace Y and Y_−m with Y and Y_−m, and replace (n − p) with (n − s), where Y denotes the predicted response by regressing y on the s true variables via least squares, and Y_−m is defined similarly but excluding the variables in the mth modality. In practice, of course, which subset are the true variables is unknown. However, Proposition 1(a) and (b) only provide conceptual justifications of ${\sigma }_{m\mid -m}^2$. We always resort to Proposition 1(c) to compute ${\sigma }_{m\mid -m}^2$, which holds regardless of p < n or p > n. Besides, it does not require the knowledge of the true variables, nor any extra variable selection step to identify them.

Next, we present a plug-in estimator of ${\sigma }_{m\mid -m}^2$ given the data. We first apply Bai and Ng (2002) in (4) to the concatenated data matrix $X=\left(X_1,\dots ,X_M\right)\in {\mathcal{R}}^{n\times p}$ to estimate the number of shared factors. We then apply PCA to obtain F. We next estimate Λ_m by Λ_m = (1/ n)X_m′F, and obtain U_m = X_m − FΛ_m′. We solve for β following (5). We then apply the thresholding method of Fan et al. (2013) to estimate Σ_u by Σ_u, whose (i, j)th element is ${\widehat{\sigma }}_{u,ij}^2=s\left(n^{-1}\sum _{\ell =1}^n{\widehat{U}}_{\ell i}{\widehat{U}}_{\ell j},\omega \right)$, s(x, ω) is a thresholding function, ω is the threshold, and U = (U₁, …, U_M). Let β_m denote the subvector of β with indices in the mth modality. By Theorem 3, β_m is a consistent estimator of ${\beta }_m^{*}$. By Theorem 3.3 of Fan et al. (2013), Λ_m and Λ_−m are two consistent estimators of the loading matrices. In addition, by Theorem 3.1 of Fan et al. (2013), Σ_u is a consistent estimator of Σ_u. Plugging all these estimators into Proposition 1(c) gives a consistent estimator of ${\sigma }_{m\mid -m}^2$.

We make two additional remarks about ${\sigma }_{m\mid -m}^2$. First, the closed-form expression of ${\sigma }_{m\mid -m}^2$ utilizes the factors commonly shared by x_m and x_−m. Indeed, such factors determine the correlations between x_m and x_−m. When no such common factors exist, x_m and x_−m are uncorrelated. In that case, $\text{Var}\left(x_m{}^{\prime }{\beta }_m^{*}\mid x_{-m}\right)=\text{Var}\left(x_m{}^{\prime }{\beta }_m^{*}\right)={\beta }_m^{*}{}^{\prime }{\Sigma }_{x_m}{\beta }_m^{*}={\beta }_m^{*}{}^{\prime }{\Sigma }_{u_m}{\beta }_m^{*}$. Therefore, the closed-form expression in Proposition 1(c) can be viewed as a more general form of this special case by taking the correlations between x_m and x_−m into account. Second, the computation of ${\sigma }_{m\mid -m}^2$ only requires to invert a sparse high-dimensional matrix ${\Sigma }_{u_{-m}}$ and a low-dimensional matrix $I_K+{\Lambda }_{-m}{}^{\prime }{\Sigma }_{u_{-m}}^{-1}{\Lambda }_{-m}$. If an exact factor model is further adopted such that Σ_u becomes a diagonal matrix, ${\sigma }_{m\mid -m}^2$ can be easily computed, as one only needs to invert a low-dimensional matrix. On the contrary, if one does not employ a factor model, then $\text{Var}\left(x_m{}^{\prime }{\beta }_m^{*}\mid x_{-m}\right)={\beta }_m^{*}{}^{\prime }\left({\Sigma }_{x_m}-{\Sigma }_{x_m,x_{-m}}{\Sigma }_{x_{-m}}^{-1}{\Sigma }_{x_{-m},x_m}\right){\beta }_m^{*}$, where ${\Sigma }_{x_m}=\text{E}\left(x_m^{\otimes 2}\right)$, ${\Sigma }_{x_m,x_{-m}}=\text{E}\left(x_m{x}_{-m}^{\prime }\right)$, ${\Sigma }_{x_{-m}}=\text{E}\left(x_{-m}^{\otimes 2}\right)$, and ${\Sigma }_{x_{-m},x_m}=\text{E}\left(x_{-m}{x}_m{}^{\prime }\right)$. Consequently, a large dense matrix ${\Sigma }_{x_{-m}}$ has to be inverted.

7. Numerical analysis

7.1. Test of a whole modality

We evaluate the empirical performance of the factor-adjusted score test of a whole modality proposed in Section 4. We generate M = 3 modalities, and consider two cases of x. Specifically, for each modality m = 1, 2, 3, x_m are n i.i.d. random samples generated from $N_{p_m}\left(0,{\Sigma }_m\right)$. For Case 1, ${\Sigma }_m={\Lambda }_m{\Lambda }_m{}^{\prime }+0.5I_{p_m}$, where each column of ${\Lambda }_m\in {\mathcal{R}}^{p_m\times K_m}$ is generated from $N_{p_m}(0,2)$, and the number of factors K_m = 2. For Case 2, the diagonal elements of Σ_m equal 1 and the off-diagonal elements equal 0.4. Accordingly, in Case 1, x_m indeed follows a factor model setup, and in Case 2, although x_m does not strictly follow a factor model, its covariance matrix has spiked eigenvalues. In both cases, we aim to test if the first modality x₁ is significantly associated with the response, i.e. $H_0:{\beta }_{11}^{*}=\dots ={\beta }_{1p_1}^{*}=0$. We then consider two types of alternatives. The first alternative is $H_{A1}:{\beta }_{11}^{*}=\dots ={\beta }_{1p_1}^{*}=\delta /p$, where δ is a sequence approaching zero. As such, there is a weak signal in each variable of x₁ and the overall signal is dense. The second alternative is $H_{A2}:{\beta }_{11}^{*}=\dots ={\beta }_{15}^{*}=\delta /5$, ${\beta }_{16}^{*}=\dots ={\beta }_{1p_1}^{*}=0$. As such, the overall signal is sparse, as all signals come only from the first 5 variables, whereas the rest do not associate with the response. For the other two modalities x₂ and x₃, we set ${\beta }_{21}^{*}=1$, ${\beta }_{22}^{*}=2$, ${\beta }_{23}^{*}=\dots ={\beta }_{2p_2}^{*}=0$, and ${\beta }_{31}^{*}=-1$, ${\beta }_{32}^{*}=-1$, ${\beta }_{33}^{*}=\dots ={\beta }_{3p_3}^{*}=0$. We generate the error ϵ as n i.i.d. samples from N(0, 0.5), and generate y based on model (2). We set p₁ = p₂ = p₃ = p/3. We consider two combinations (n, p) = (100, 600), and (200, 900). We compare our test with the score test of Ning and Liu (2017), where the critical values are obtained by bootstrap.

We report the proportion of rejections of H₀ by both tests out of 600 data replications as we vary the value of δ. When δ = 0, this gives the empirical size, and when δ > 0, it gives the empirical power of the two tests. Figures 1 and 2 report the results for Cases 1 and 2, respectively. In both cases, we see that our proposed test controls the Type I error at the nominal level of α = 0.05 when δ = 0. However, the test of Ning and Liu (2017) often yields an inflated size. This may be due to that their multiplier bootstrap method rejects the null hypothesis if the maximum of the decorrelated score functions of variables in that modality is greater than a threshold, and as such, it is easier to reject the null hypothesis. Moreover, our test achieves an as good or often a better power than the test of Ning and Liu (2017) as δ increases.

Fig. 1.

Empirical size and power of testing a whole modality in Case 1 for the factor-adjusted score test (solid line), and the score test of Ning and Liu (2017) (dashed line).

Fig. 2.

Empirical size and power of testing a whole modality in Case 2 for the factor-adjusted score test (solid line), and the score test of Ning and Liu (2017) (dashed line).

Next, we simulate data from the three examples as we discussed in Section 4 to further examine the performance of the proposed test. For all three examples, we generate M = 2 modalities with K_m = 1 factor in each modality, and set n = 100, p_m = 200, and α = 0.05. We aim to test the significance of the first modality. We generate x_m as n i.i.d. random samples from $N_{p_m}\left(0,{\Sigma }_m\right)$, where ${\Sigma }_m={\Lambda }_m{\Lambda }_m{}^{\prime }+0.5I_{p_m}$ for m = 1, 2. For the second modality, we always choose Λ₂ = (1, 1, 1, …, 1)′, and set its coefficients as ${\beta }_{21}^{*}=1$, ${\beta }_{22}^{*}=2$, ${\beta }_{23}^{*}=\dots ={\beta }_{2p_2}^{*}=0$. For the first modality, we choose different loadings and test different local alternatives. For Example 1, we choose Λ₁ = (20, 1, 1, …, 1)′, and the alternative $H_A:{\beta }_{11}^{*}=0.08$, ${\beta }_{12}^{*}=\dots ={\beta }_{1p_1}^{*}=0$. For Example 2, we choose Λ₁ = (1, 1, 1, …, 1)′, and the alternative $H_A:{\beta }_{11}^{*}=\dots ={\beta }_{18}^{*}=0.01$, ${\beta }_{19}^{*}=\dots ={\beta }_{1p_1}^{*}=0$. For Example 3, we choose Λ₁ = (0, 1, 1, …, 1)′, and the alternative $H_A:{\beta }_{11}^{*}=0.2$, ${\beta }_{12}^{*}=\dots ={\beta }_{1p_1}^{*}=0$. Table 1 reports the empirical size and power of the proposed factor-adjusted score test and the score test of Ning and Liu (2017) based on 600 data replications. For Example 1, thanks to the large loading of the first variable, the proposed test achieves a better power than the test of Ning and Liu (2017). For Example 2, the nonzero coefficients are spread out in eight covariates, and they all have loadings on the latent factor. Therefore, thanks to the cumulative effects, our method again achieves a better power to detect the alternative. For Example 3, while the nonzero coefficient ${\beta }_{11}^{*}$ is large, the corresponding loading is zero. Our test thus has no power in detecting such an alternative. These numerical results agree with our discussion in Section 4 on the local power of the proposed test.

Table 1.

Empirical size and power of testing a whole modality in Examples 1 to 3 for the factor-adjusted score test, and the decorrelated score test of Ning and Liu (2017).

	Factor-adjusted test		Test of Ning and Liu
	Size	Power	Size	Power
Example 1	0.06	0.50	0.05	0.40
Example 2	0.06	0.60	0.07	0.45
Example 3	0.06	0.07	0.05	0.35

7.2. Test of a linear combination of predictors

We next evaluate the empirical performance of the factor-adjusted Wald test of a linear combination of predictors proposed in Section 5. We again generate M = 3 modalities and two cases of x similarly as in Section 7.1, except that in the second case we increase the off-diagonal elements of Σ_m to 0.8 for m = 1, 2, 3. We set ${\beta }^{*}={\left({\beta }_1^{*}{}^{\prime },{\beta }_2^{*}{}^{\prime },{\beta }_3^{*}{}^{\prime }\right)}^{\prime }$, ${\beta }_1^{*}=(-2+\delta ,-1,0,\dots ,0{)}^{\prime }$, ${\beta }_2^{*}=(1+\delta ,2,0,\dots ,0{)}^{\prime }$, ${\beta }_3^{*}=(1+\delta ,1,0,\dots ,0{)}^{\prime }$, and aim to test the linear combination of the first variable in each modality that $H_0:{\beta }_{11}^{*}+{\beta }_{21}^{*}+{\beta }_{31}^{*}=0$ versus $H_A:{\beta }_{11}^{*}+{\beta }_{21}^{*}+{\beta }_{31}^{*}\ne 0$. The rest of the simulation setup is the same as that in Section 7.1. Since we only consider testing a single linear combination in this study, we compare our test with both the partially penalized Wald test proposed in Shi et al. (2019), and the test proposed in Zhu and Bradic (2018).

We again report the proportion of rejections of H₀ out of 600 data replications as we vary the value of δ. Figure 3 reports the results for both Cases 1 and 2. In both cases, we see that our factor-adjusted Wald test and the test of Shi et al. (2019) achieve a good control of the type I error at the nominal level α = 0.05 when δ = 0. But our test achieves a much improved power as δ increases. The main reason is that, in this example, the variables are highly correlated with each other. The factor adjustment alleviates such high correlations, and yields a better variable selection and estimation of the true regression coefficients, which in turn benefits the inference. Meanwhile, in Case 1, the test of Zhu and Bradic (2018) yields a type I error that is much larger than the nominal level. This is because, in this case, the variables are driven by the latent factors, which leads to a wide spectrum of the eigenvalues of the covariate covariance matrix. However, Zhu and Bradic (2018) required the eigenvalues to be bounded. In Case 2, the variables are not generated from a factor model. In this case, the test of Zhu and Bradic (2018) enjoys the best power. Nevertheless, it still suffers from an inflated type I error, which is around 0.09 and is about twice as large as the nominal level. Moreover, we recall that both our test and the test of Shi et al. (2019) can jointly test multiple linear combinations, while the test of Zhu and Bradic (2018) was designed to test a single linear combination.

Fig. 3.

Empirical size and power of testing a linear combination of predictors for the factor-adjusted Wald test (solid line), the Wald test of Shi et al. (2019) (dashed line), and the test of Zhu and Bradic (2018) (dotted line).

7.3. Multimodal neuroimaging analysis

We illustrate our methods with a multimodal neuroimaging analysis to study Alzheimer’s disease (AD). AD is an irreversible neurodegenerative disorder characterized by progressive impairment of cognitive and memory functions. It is the leading form of dementia in elderly subjects, and is the sixth leading cause of death in the United States. In 2018, AD affects over 5.5 million Americans, and without any effective treatment and prevention, this number is projected to almost triple by 2050 (Alzheimer’s Association, 2018). Tau is a hallmark pathological protein of AD, and is believed to be part of the driving mechanism of the disorder. It is present in the brains of both AD subjects and the elderly absent of dementia. Brain atrophy is another well known characteristic that differentiates between AD and normal aging. We study a dataset with n = 125 subjects. Each subject receives a positron emission tomography (PET) scan with AV-1451 tracer that measures accumulation of tau protein, as well as an anatomical magnetic resonance imaging (MRI) scan that measures brain grey matter cortical thickness. We map both types of images to a common brain atlas from Free Surfer, then summarize each PET and MRI image by a 58-dimensional vector, with each entry measuring the tau accumulation and cortical thickness of a particular brain region-of-interest (ROI), respectively. We remove some regions with quality issues for the PET images, which result in p₁ = 51 ROIs for PET, and p₂ = 58 ROIs for MRI, for each subject. In our integrative analysis, the tau and cortical thickness measurements form the two modalities x₁ and x₂. Memory score is a critical measure of cognitive decline for AD, and in our analysis, the memory score after removing potential age and sex effects is the response variable y.

We estimate the number of latent factors using the method of Bai and Li (2012), which concludes that there are ${\widehat{K}}_1=3$ factors in the tau modality and ${\widehat{K}}_2=1$ factor in the cortical thickness modality. We then estimate β* and γ* using (5) with a SCAD penalty, where the tuning parameter was chosen by cross-validation. We then apply the three methods we develop in this article. We first test the significance of the entire modality using the factor-adjusted score test in Section 4. The p-values are 4.9 × 10⁻⁷ and 1.2 × 10⁻³, for testing the significance of tau and cortical thickness modality, respectively. As such, both modalities are clearly significantly associated with the memory outcome. We then report the estimated non-zero coefficients from our integrative factor model and their corresponding brain regions in Table 2. We further carry out the factor-adjusted Wald test in Section 5 to evaluate if the identified regions are significantly correlated with the outcome in either modality. We report the corresponding p-values in Table 2 as well. Our findings agree with the AD literature. For instance, the ROI with the smallest p-value we found is inferior parietal lobe, which is one of brain regions that is known to be associated with progression from healthy aging to AD (Greene and Killiany, 2010). Another significant ROI is parahippocampal gyrus, and cortical thinning of this region has been identified as an early biomarker of AD (Echávarri et al., 2011; Krumm et al., 2016). Finally, we evaluate the contribution of each individual modality. If we include the tau modality x₁ in the model first, and add the cortical thickness modality x₂ next, we have ${\widehat{\sigma }}_1^2=\text{Var}\left(x_1{}^{\prime }{\beta }_1^{*}\right)=0.11$, and ${\widehat{\sigma }}_{2|1}^2=\text{Var}\left(x_2{}^{\prime }{\beta }_2^{*}\mid x_1\right)=0.18$. Correspondingly, ${\widehat{\sigma }}_1^2/{\widehat{\sigma }}_y^2=14\text{\%}$, and ${\widehat{\sigma }}_{2\mid 1}^2/{\widehat{\sigma }}_y^2=24\text{\%}$. In other words, the tau modality explains 14% total variation in the response, and adding the cortical thickness modality explains an additional 24% total variation. On the other hand, if we include the cortical thickness modality x₂ in the model first, and add the tau modality x₁ next, we have ${\widehat{\sigma }}_2^2=\text{Var}\left(x_2{}^{\prime }{\beta }_2^{*}\right)=0.19$, and ${\widehat{\sigma }}_{1|2}^2=\text{Var}\left(x_1{}^{\prime }{\beta }_1^{*}\mid x_2\right)=0.08$. Correspondingly, ${\widehat{\sigma }}_2^2/{\widehat{\sigma }}_y^2=25\text{\%}$, and ${\widehat{\sigma }}_{1|2}^2/{\widehat{\sigma }}_y^2=11\text{\%}$. In other words, the cortical thickness modality explains 25% total variation in the response, and adding the tau modality explains an additional 11% total variation. We also note that, the explained variation depends on which modality is already included in the model, and the total explained variations do not necessarily match if the two modalities enter the model in different orders.

Table 2.

The identified brain regions with the coefficient estimates and the corresponding p-values of the factor-adjusted Wald test for the significance of the brain regions.

		R.rostantcing	L.superiorparietal	L.inferiorparietal	L.middletemporal
Coefficient	β 1	0.07	0.12	0	0
	β 2	0	0	−0.18	−0.01
p-value		0.05	0.03	0.001	0.1
		L.parahippocampal	L.rostantcing	R.parstriangularis	R.superiortemporal
Coefficient	β 1	0	0	0	0
	β 2	0.07	0.06	0.01	0.12
p-value		0.02	0.03	0.14	0.03
		R.supramarginal	R.temppole
Coefficient	β 1	0	0
	β 2	0.11	0.04
p-value		0.02	0.05

8. Discussion

In recent years, high-dimensional inference has seen many fruitful results. Particularly, Javanmard and Lee (2020); Cai and Guo (2017); Zhu and Bradic (2017) have developed a family of flexible debiased methods to test the hypothesis of the form ${\beta }^{*}\in \mathcal{C}$, where $\mathcal{C}$ is a general set. By choosing different $\mathcal{C}$, these methods can handle a wide range of inference problems, and can potentially address the inference questions we target in this paper too. We view the proposed solution and the debiased method as two legitimate and complementary inferential approaches, each with its own strength and limitation. Specifically, the debiased method does not require the beta-min condition that is difficult to check in practice, but instead requires the eigenvalues of Var(x) to be bounded. Besides, it can test the linear combination of the whole p-dimensional parameters β*, while the existing work so far focuses on a single linear combination. By contrast, our proposal assumes the variables are driven by latent factors that satisfy a pervasive condition, and thus allows the largest eigenvalue of Var(x) to diverge. This may be desirable in the context of multimodal data analysis, as the predictor covariance matrix can have spiked eigenvalues. Nevertheless, our proposal requires the beta-min condition. In addition, we aim at testing multiple linear combinations, but require the number of linear combinations r and the number of involved parameters t in (9) to be smaller than the sample size. In conclusion, we believe our proposal offers a useful solution to address some important inference questions in multimodal data analysis. Meanwhile, developing the debiased counterpart provides an important alternative, and is warranted for future research.