Partial Quantile Tensor Regression

Abstract

Tensors, characterized as multidimensional arrays, are frequently encountered in modern scientific studies. Quantile regression has the unique capacity to explore how a tensor covariate influences different segments of the response distribution. In this work, we propose a partial quantile tensor regression (PQTR) framework, which novelly applies the core principle of the partial least squares technique to achieve effective dimension reduction for quantile regression with a tensor covariate. The proposed PQTR algorithm is computationally efficient and scalable to a large tensor covariate. Moreover, we uncover an appealing latent variable model representation for the PQTR algorithm, justifying a simple population interpretation of the resulting estimator. We further investigate the connection of the PQTR procedure with an envelope quantile tensor regression (EQTR) model, which defines a general set of sparsity conditions tailored to quantile tensor regression. We prove the root-$n$ consistency of the PQTR estimator under the EQTR model, and demonstrate its superior finite-sample performance compared to benchmark methods through simulation studies. We demonstrate the practical utility of the proposed method via an application to a neuroimaging study of post traumatic stress disorder (PTSD). Results derived from the proposed method are more neurobiologically meaningful and interpretable as compared to those from existing methods.

1. Introduction

Quantile regression (Koenker and Bassett, 1978) offers a valuable perspective for characterizing the associations between a response and covariates of interest. By formulating covariate effects on one or more quantiles of the response while allowing such effects to change across different quantile levels, quantile regression confers a flexible and robust view of how covariates influence different segments of the response distribution (e.g., outcomes from the middle range, or from low or upper tail). With these natural appeals, quantile regression has received increasing attention in real data analyses. Theory, inference, and computation for quantile regression in the typical data settings, where covariates can sensibly enter a quantile regression model in a vector form, have been well studied; see the monograph by Koenker (2005) for a comprehensive summary.

There are emerging scientific applications (e.g., neuroimaging studies) where the covariates of interest take the complex form of multidimensional arrays, formally termed as tensors. Quantile regression has the unique capacity to explore the effects of a tensor covariate on average outcomes as well as unusual outcomes that often attract scientific interests. For example, neuroimaging features indicative of severe symptoms of a mental disorder are of more diagnostic or therapeutic value than those explaining average symptoms. However, addressing quantile regression with tensor covariates (or predictors) faces nontrivial challenges not only from the large sizes of tensors (typically coupled with small sample sizes) but also posed by the substantive need to properly account for tensors’ inherent spatial structures. For example, a typical brain functional connectivity matrix generated from functional magnetic resonance imaging (fMRI) can be of size 300 × 300, including 90,000 elements. The connectivity matrix entries corresponding to connections between brain regions in various functional networks are naturally correlated, implying an intrinsic spatial structure that carries important scientific implications.

To handle a tensor covariate under quantile regression, intuitive approaches include regressing over a few features extracted from the tensor covariate, or vectorizing the whole tensor covariate and then applying existing high-dimensional quantile regression methods (Zou and Yuan, 2008; Wang et al., 2012; Zheng et al., 2013; Fan et al., 2014, among others). Similar strategies were applied in the early work on linear regression with tensor covariates (Worsley et al., 2004; Caffo et al., 2010, for example). However, these types of approaches have several limitations. That is, substituting the tensor covariate by its low-dimensional features may produce dubious results if the selected features are irrelevant to the response. Vectorizing the tensor covariate ignores the inherent spatial structure of tensors and thus may fail to provide interpretable results (see more discussions in Zhou and Li (2013)). Analyses with the vectorized tensor covariate, even after applying penalization, may still suffer from heavy computation burden and/or unstable results due to a large number of tensor elements coupled with a typically limited sample size.

In this work, we concern the quantile regression problem with the tensor covariate retained in its original tensor form. To address this problem, one possible direction is to approximate the tensor covariate coefficient by a low-rank tensor decomposition, such as the CANDECOMP/PARAFAC (CP) decomposition or Tucker decomposition. Such a strategy has been investigated under generalized linear regression (Zhou et al., 2013; Li et al., 2018; Guhaniyogi et al., 2017, for example) and also under quantile regression (Lu et al., 2020; Li and Zhang, 2021, for example). While utilizing a tensor decomposition can effectively retain the spatial structure of tensors and reduce the number of free parameters, current approaches are prone to various computational issues and challenges. Specifically, most existing approaches utilize the alternating block updating (ABU) algorithm (Zhou et al., 2013; Li et al., 2018; Lu et al., 2020; Li and Zhang, 2021; Zhou et al., 2021, for example) that optimizes the objective function with respect to each component of the decomposition cyclically. However, the number of free parameters in each decomposition component may still be greater than the sample size (Li et al., 2018; Lu et al., 2020; Li and Zhang, 2021, for example); thus additional penalty methods are required, incurring increased computational complexity and intensity (Lu et al., 2020; Li and Zhang, 2021, for example). In addition, using an ABU algorithm is generally not flexible to account for special structures of tensor predictors, such as symmetry. For example, when the tensor covariate is symmetric (e.g., functional connectivity matrix), it is not straightforward to adapt the ABU algorithm to generate exactly symmetric tensor coefficient, as pointed out by Zhang et al. (2022), and this may complicate the result interpretation.

To address the issues and limitations of current tensor-decomposition based methods, we propose to employ the partial least squares (PLS) technique (Wold, 1982; de Jong, 1993) that has emerged as a promising strategy for handling tensor covariates under linear regression. Under linear regression, the classical PLS approach, which implements supervised dimension reduction through projecting a vector covariate in a direction guided by the correlation between the response and the covariate, is viable even when the covariate dimension is much larger than the sample size. A bulk of literature has elucidated justifications and interpretations for the classical PLS based on the latent variable models (Helland, 1988, 1990, 2001) or nascent envelope models (Cook et al., 2013; Zhang and Li, 2017; Cook, 2018). To accommodate a tensor predictor, several authors (Bro, 1996; Eliseyev and Aksenova, 2013; Zhao et al., 2013) pioneered extending the classical PLS algorithm but did not provide the corresponding population interpretations. Zhang and Li (2017) formally developed a PLS algorithm under a linear tensor regression model and established a rigorous population interpretation for their tensor PLS estimator using the notion of envelope (Cook et al., 2010). Zhang and Li (2017)’s PLS algorithm serves to find a reduced tensor predictor via Tucker decomposition, while avoiding optimization settings involving excessive number of free parameters and a large number of iterations often required by existing approaches that use ABU algorithms. It can also seamlessly render a symmetric tensor coefficient estimate given a symmetric tensor predictor.

However, the utility of adapting the PLS strategy to the quantile regression setting with a tensor covariate remains unexplored. A main obstacle is that the PLS strategy operates upon a convenient linear relationship between regression coefficients and the covariance between response and covariates, which exists under linear regression but no longer holds under quantile regression. Dodge and Whittaker (2009) proposed a hybrid of PLS and quantile regression with a low-dimensional vector predictor by adopting an intuitive substitute for the covariance used by the classical PLS; however the interpretation and theoretical properties of the resulting estimators were not established.

Motivated by the important gaps discussed above, we propose a new regression framework, referred to as partial quantile tensor regression (PQTR), where the core principle of PLS is adapted to achieve dimension reduction and theoretically justified estimation for a quantile regression model including both a tensor covariate and a traditional low-dimensional vector covariate. Targeting the general tensor covariate, we propose to employ the quantile partial tensor covariance, which is generalized from the quantile partial covariance studied by Li et al. (2015) for vector predictors, to guide the dimension reduction in the presence of a tensor covariate. As shown in Sections 2 and 3, the choice of quantile partial tensor covariance facilitates algorithm development while enabling theoretical justification. Other types of quantile covariance, such as the one adopted by Dodge and Whittaker (2009), do not have similarly desirable capacities. The idea of integrating quantile partial tensor covariance into a PLS procedure, though intuitive, cannot be straightforwardly justified by extending the arguments for its linear regression counterpart. This is mainly due to lack of a clear relationship between quantile partial tensor covariance and the coefficients under quantile tensor regression. We tackle this difficulty by establishing a “pseudo” linear structure of the quantile partial tensor covariance in terms of the quantile tensor regression coefficients, which can sufficiently serve the need to justify the use of quantile partial tensor covariance. Moreover, for the first time, we reveal a population latent variable model for a PLS-type procedure in the quantile regression setting. Our finding confers a meaningful interpretation for the estimator resulted from the proposed PQTR procedure.

To gain a deeper insight regarding the utility of the proposed PQTR procedure, we further propose and study an envelope quantile tensor regression (EQTR) model closely tied to the concept of envelope (Cook et al., 2010) and a modified framework of sufficient dimension reduction (Cook, 1998). The formulation of the new EQTR model reflects a general set of sparsity conditions tailored to quantile tensor regression, and represents a novel adaptation of the envelope concept to quantile regression with tensor covariates. Ding et al. (2020) did precursor work on an envelope quantile regression model with a traditional vector covariate, proposing estimation based on the generalized method of moments (GMM). This approach, however, is not suitable for handling a tensor covariate due to potential scalability concerns and/or interpretative issues from vectorizing the tensor covariate. In this work, we delineate the connection between the proposed PQTR procedure and the new EQTR model via a population latent variable model representation. A useful finding is that PQTR can serve as a legitimate approach to providing scalable estimation of the EQTR model. Assuming the proposed EQTR model holds, we establish the $\sqrt{n}$-consistency of the estimator obtained from the proposed PQTR procedure. The proof involves addressing multiple challenging problems. These include using complex random matrix theory to characterize the factor matrices of the estimated reduced tensor, and dealing with a non-standard quantile regression problem where the tensor covariate is latent and unobserved. Such a problem is largely unexplored even in the special case with regular non-tensor covariates. Our asymptotic proof, which employs sophisticated empirical process arguments, may provide a general template to deal with similar estimation settings.

Our extensive simulation studies suggest that the PQTR estimator, as compared to some benchmark methods, can achieve better performance with much lower computational cost.

2. Partial Quantile Tensor Regression

2.1. Data and quantile tensor regression model

Let $Y$ denote a continuous response of interest, and $\mathit{Z}\in {\mathbb{R}}^{p_z}$ and $𝒳\in {\mathbb{R}}^{p_1\times \cdots \times p_m}$ denote a low-dimensional vector covariate and an $m\text{th}$-order tensor covariate, respectively. Throughout the paper, following Qi and Luo (2017), we will use capital bold letters $\mathit{A},\mathit{B},\mathit{X},\dots$ for matrices and calligraphic letters $𝒜,ℬ,𝒳,\dots$ for general tensors, and adopt standard notations for common tensor operations, which are reviewed in Section S0 of the Supplementary Materials.

Let $Q_Y(\tau \mid \mathit{X},\mathit{Z})=\text{inf}\{t:\text{Pr}(Y\le t\mid \mathit{X},\mathit{Z})\ge \tau \}$ be the $\tau \text{th}$ conditional quantile of $Y$ given $\mathit{X}$ and $\mathit{Z}$. Given $\mathit{X}$ and $\mathit{Z}$, a quantile tensor regression model may take the form,

$$Q_Y(\tau \mid \mathit{X},\mathit{Z})={\alpha }_0(\tau )+{\gamma }_0^{\mathsf{T}}(\tau )\mathit{Z}+\langle {ℬ}_0(\tau ),𝒳\rangle ,\tau \in \Delta ,$$ (1)

where $\Delta \subseteq (0,1)$ includes the quantile levels of interest. In model (1), ${\alpha }_0(\tau )\in \mathbb{R}$ is the intercept, ${\gamma }_0(\tau )\in {\mathbb{R}}^{p_z}$ is an unknown vector coefficient for $\mathit{Z}$, and ${ℬ}_0(\tau )\in {\mathbb{R}}^{p_1\times \cdots \times p_m}$ is an unknown $m\text{th}$-order tensor coefficient representing the effect of $𝒳$ on the $\tau \text{th}$ quantile of $Y$. Without loss of generality, we assume $E[\mathit{Z}]=0$ and $E[𝒳]=0$. For convenience, denote $\mathit{Z}={\left(1,{\mathit{Z}}^{\mathsf{T}}\right)}^{\mathsf{T}}$ and ${\tilde{\gamma }}_0(\tau )={\left({\alpha }_0(\tau ),{\gamma }_0{(\tau )}^{\mathsf{T}}\right)}^{\mathsf{T}}$.

Provided a sample consisting of i.i.d. $(Y_i,{\mathit{Z}}_i,{𝒳}_i)(i=1,\dots ,n)$, without concerning the dimensionality issue, one may estimate model (1) by minimizing the standard sample quantile regression loss function (Koenker, 2005),

$$\frac{1}{n}\sum _{i=1}^n{\rho }_{\tau }\left(Y_i-\alpha -{\gamma }^{\mathsf{T}}{\mathit{Z}}_i-\langle ℬ,{𝒳}_i\rangle \right),$$ (2)

with respect to $\alpha$, $\gamma$, and $ℬ$, where ${\rho }_{\tau }(w)=w\left\{\tau -\mathbf{I}\left(w<0\right)\right\}$ and $\mathbf{I}(\cdot )$ is the indicator function. Note that the minimizer of (2) can attain the typical root-$n$ convergence rate as $n$ goes to $\infty$. However, this optimization problem becomes infeasible when the number of unknown parameters, $1+p_z+\prod _{i=1}^m{p}_k$, exceeds the sample size $n$. This can occur even when each individual $p_k(k=1,\dots ,m)$ is not large. This serves as the main motivation of this work.

2.2. The proposed partial quantile tensor regression procedure

To conquer the dimensionality challenge for quantile tensor regression, our proposal is to adapt the core principle of PLS, which has demonstrated superb utility in achieving effective dimension reduction and accurate prediction in linear regression settings. Pertaining to a tensor covariate, Zhang and Li (2017) developed the Tensor Envelope PLS (TEPLS) algorithm for a linear tensor regression model of the form, $Y=\langle ℬ,𝒳\rangle +\epsilon$, where $\epsilon$ stands for the error term. TEPLS assumes a separable Kronecker covariance structure of $𝒳$, i.e. ${\Sigma }_{𝒳}=\text{var}(𝒳)={\Sigma }_m\otimes \cdots \otimes {\Sigma }_1$ with ${\Sigma }_k\in {\mathbb{R}}^{p_k\times p_k}$. The main line of the TEPLS algorithm is to first reduce $𝒳$ to $𝒯\doteq ⟦\mathit{X};{\mathit{W}}_1^{\mathsf{T}},\dots ,{\mathit{W}}_m^{\mathsf{T}}⟧\in {\mathbb{R}}^{d_1\times \cdots \times d_m}$, where ${\mathit{W}}_1,\dots ,{\mathit{W}}_m$ are factor matrices, and then regress $Y$ over $𝒯$. In the TEPLS algorithm, the initial cross covariance, $\mathrm{cov}(Y,𝒳)$, plays an instrumental role in deriving the factor matrices, ${\mathit{W}}_k\in {\mathbb{R}}^{p_k\times d_k}(k=1,\dots ,m)$, which are computed by maximizing a function of the deflated cross covariance term sequentially and independently of other factor matrices, without direct optimization. The total dimension of $𝒯,\prod _{k=1}^m{d}_k$, can be made sufficiently small if $d_k$ ’s are properly chosen. For example, for a $300\times 300$ second-order tensor covariate, if one sets $d_1=d_2=3$, then the TEPLS algorithm significantly reduces the total dimension of $𝒳$, 90,000, to 9, which is manageable with a small-to-moderate sample size. However, the TEPLS algorithm is not an appropriate tool to address the quantile tensor regression model (1) because the classical cross covariance, $\mathrm{cov}(Y,𝒳)$, does not contain sufficient information regarding the effects of $𝒳$ on quantiles of $Y$. In addition, TEPLS does not accommodate the low-dimensional vector covariate $\mathit{Z}$.

To address these obstacles, we propose to generalize the quantile partial covariance (Li et al., 2015) to guide the dimension reduction of $𝒳$ for the purpose of estimating ${ℬ}_0(\tau )$ while properly accounting for the effect of $\mathit{Z}$. More specifically, we define the quantile partial tensor covariance between the response $Y$ and the tensor covariate $𝒳$ given $\mathit{Z}$ as $\text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})=E\left[R(\tau )𝒳\right]$, where $R(\tau )={\psi }_{\tau }\left(Y-{\tilde{\gamma }}_{Y\mid \mathit{Z}}{(\tau )}^{\mathsf{T}}\mathit{Z}\right)$ with ${\psi }_{\tau }(w)=\tau -\mathbf{I}(w<0)$, ${\tilde{\gamma }}_{Y\mid \mathit{Z}}(\tau )={\left({\alpha }_{Y\mid \mathit{Z}}(\tau ),{\tilde{\gamma }}_{Y\mid \mathit{Z}}^{\mathsf{T}}(\tau )\right)}^{\mathsf{T}}$, and $\left({\alpha }_{Y\mid \mathit{Z}}(\tau ),{\gamma }_{Y\mid \mathit{Z}}(\tau )\right)=\underset{\alpha ,\gamma }{\text{arg}\text{min}}E\left[{\rho }_{\tau }\left(Y-\alpha -{\gamma }^{\mathsf{T}}\mathit{Z}\right)\right]$. Here $R(\tau )$ may be viewed as the $\tau \text{-th}$ quantile score of $Y$ after adjusting for the marginal quantile effect of $\mathit{Z}$. In Lemma 1, we derive an important result regarding the connection between the quantile partial tensor covariance $\text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})$ and the tensor covariate coefficient ${ℬ}_0(\tau )$.

Lemma 1

We assume that if ${\mathit{a}}^{\mathsf{T}}\text{vec}(𝒳)={\mathit{b}}^{\mathsf{T}}\mathit{Z}$ with probability one for two deterministic vectors $\mathit{a}$ and $\mathit{b}$, then $\mathit{a}=0$ and $\mathit{b}=0$. Suppose that the conditional density function of $Y$, given $(𝒳,\mathit{Z})$, denoted by $f_{Y\mid 𝒳,\mathit{Z}}$, is uniformly bounded away from 0 and $\infty$. Then under model (1), $\text{vec}\left(\text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})\right)=\mathit{V}(\tau )\text{vec}\left({ℬ}_0(\tau )\right)$, for $\tau \in \Delta$, where

$$\mathit{V}(\tau )=E\left[v_{\tau }(\mathit{Z},𝒳){\left(\text{vec}(𝒳)-E\left[v_{\tau }(\mathit{Z},𝒳)\text{vec}(𝒳){\mathit{Z}}^{\mathsf{T}}\right]E{\left[v_{\tau }(\mathit{Z},𝒳)\mathit{Z}{\mathit{Z}}^{\mathsf{T}}\right]}^{-1}\mathit{Z}\right)}^{\otimes 2}\right],$$

and $v_{\tau }(\mathit{Z},𝒳)={\int }_0^1{f}_{Y\mid 𝒳,\mathit{Z}}\left((1-u)\left({\tilde{\gamma }}_0{(\tau )}^{\mathsf{T}}\mathit{Z}+\langle {ℬ}_0(\tau ),𝒳\rangle \right)+u{\tilde{\gamma }}_{Y\mid \mathit{Z}}{(\tau )}^{\mathsf{T}}\mathit{Z}\right)du$. Furthermore, $\mathit{V}(\tau )$ is positive-definite.

Lemma 1 demonstrates that, after vectorization, $\text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})$ can be expressed as a “pseudo” linear transformation of ${ℬ}_0(\tau )$, which considerably differs from the simple linear relationship between the classical covariance and the coefficients under linear regression (Zhang and Li, 2017, Lemma 3.1–3.2). An immediate implication from Lemma 1 is that the quantile partial tensor covariance $\text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})$ would vanish if and only if all elements in ${ℬ}_0(\tau )$ are zero. A parallel result in Li et al. (2015) (see their Lemma 1) only infers about the equivalence between zero quantile partial covariance and zero quantile slope (or coefficient) under a simple linear quantile regression model with a single scalar covariate. In Lemma 1, we also derive the explicit form of $\mathit{V}(\tau )$, which is new in literature. The result suggests that $\mathit{V}(\tau )$ depends on the whole function ${ℬ}_0(\cdot )$, not just ${ℬ}_0(\tau )$, given its involvement of $f_{Y\mid 𝒳,\mathit{Z}}(\cdot )$. Such a subtle global dependence entails a mechanistic distinction between using quantile partial tensor covariance to guide dimension reduction versus using the traditional covariance, which warrants different lines of arguments for justification. This is reflected by some key technical results in Section 3 (see Proposition 5).

Motivated by Lemma 1, assuming a separable Kronecker covariance structure of $𝒳$, we propose a partial quantile tensor regression (PQTR) procedure, which mimics TEPLS while employing $𝒞(\tau )\doteq \text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})$ as the substitute for $\mathrm{cov}(Y,𝒳)$ to reduce $𝒳$ via a Tucker decomposition. The detailed procedure is described in Algorithm 1.

1 Given $\tau \in (0,1)$, calculate the partial quantile covariance $𝒞(\tau )=\text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})$. for $k=1$ to $m$ do

2 Standardize the mode-$k$ cross covariance matrix ${𝒞}_{(k)}(\tau )$:

$${𝒞}_{0k}(\tau )={𝒞}_{(k)}(\tau )\left({\Sigma }_m^{-1∕2}\otimes \cdots \otimes {\Sigma }_{k+1}^{-1∕2}\otimes {\Sigma }_{k-1}^{-1∕2}\otimes \cdots \otimes {\Sigma }_1^{-1∕2}\right)\in {\mathbb{R}}^{p_k\times (\prod _{j\ne k}{p}_j)}$$

for $s=1$ to $d_k$ do

4 3 Find ${\mathit{w}}_{sk}(\tau )\in {\mathrm{R}}^{p_k}$ that maximizing ${\mathit{w}}^{\mathsf{T}}{𝒞}_{(s-1)k}(\tau ){𝒞}_{(s-1)k}^{\mathsf{T}}(\tau )\mathit{w}$ subject to ${\mathit{w}}^{\mathsf{T}}\mathit{w}=1$ Deflate the cross covariance ${𝒞}_{sk}(\tau )={\mathit{Q}}_{sk}(\tau ){𝒞}_{0k}(\tau )$, where

$${\mathit{Q}}_{sk}(\tau )={\mathit{I}}_{p_k\times p_k}-{\Sigma }_k{\mathit{W}}_{sk}(\tau ){\left({\mathit{W}}_{sk}^{\mathsf{T}}(\tau ){\Sigma }_k{\mathit{W}}_{sk}(\tau )\right)}^{-1}{\mathit{W}}_{sk}^{\mathsf{T}}(\tau )\in {\mathrm{R}}^{p_k\times p_k}$$

is a projection onto the orthogonal subspace of $\text{span}({\Sigma }_k{\mathit{W}}_{sk}(\tau ))$, where

$${\mathit{W}}_{sk}(\tau )=({\mathit{w}}_{1k}(\tau ),\dots ,{\mathit{w}}_{sk}(\tau ))\in {\mathrm{R}}^{p_k\times s}.$$

end for

end for

7 6 5Reduce $\mathit{X}\in {\mathbb{R}}^{p_1\times \cdots \times p_m}$ to a tensor $𝒯(\tau )=⟦\mathit{X};{\mathit{W}}_1^{\mathsf{T}}(\tau ),\dots ,{\mathit{W}}_m^{\mathsf{T}}(\tau )⟧\in {\mathbb{R}}^{d_1\times \cdots \times d_m}$, where ${\mathit{W}}_k(\tau )={\mathit{W}}_{d_{k}k}(\tau )\in {\mathbb{R}}^{p_k\times d_k}$. Perform the quantile regression for $Y$ with respect to $\mathit{Z}$ and $𝒯(\tau )$, and obtain the estimation for coefficients $\alpha$, $\gamma$ and $𝒟(\tau )\in {\mathbb{R}}^{d_1\times d_2\times \cdots \times d_m}$. Obtain $ℬ(\tau )=𝒟(\tau );{\mathit{W}}_1(\tau ),\dots ,{\mathit{W}}_m(\tau )\in {\mathbb{R}}^{p_1\times \cdots \times p_m}$.

Algorithm 1 Partial quantile tensor regression (PQTR) procedure

The following are several remarks on the proposed PQTR procedure. First, since $𝒞(\tau )\doteq \text{qp}{\mathrm{cov}}_{\tau }(Y,𝒳\mid \mathit{Z})=\mathrm{cov}(R(\tau ),𝒳)$ given $E\{R(\tau )\}=0$, analogous to $𝒞=\mathrm{cov}(Y,𝒳)$ in TEPLS, we may regard Steps 1-5 as a minor variant of the corresponding steps in TEPLS with the response being $R(\tau )$. The separable Kronecker assumption has been widely used in tensor regression and high-dimensional covariance estimation literature (Hoff, 2011; Tsiligkaridis and Hero, 2013; Li and Zhang, 2017; Zhang and Li, 2017; Leng and Pan, 2018, among others). Of note, Step 3 in PQTR is equivalent to obtaining the eigenvector associated to the largest eigenvalue of ${𝒞}_{(s-1)k}(\tau ){𝒞}_{(s-1)k}^{\mathsf{T}}(\tau )$, which can be easily implemented by standard software, for instance, MATLAB or R. With a given $k$, Steps 3-4 of PQTR repeats only $d_k$ times to sequentially generate each column of ${\mathit{W}}_k(\tau )$. By this design, PQTR is more computationally efficient than existing work that adopts an ABU algorithm, which may involve hundreds of iterations. With a properly chosen small $d_k(k=1,\dots ,m)$, Step 6 is to perform trivial low-dimensional quantile regression. Moreover, Steps 2–4 in PQTR have an invariance property in the sense that they produce the same ${\mathit{W}}_{sk}(\tau )$ when $𝒳$ is multiplied by a constant because eigenvectors are invariant to matrix scaling. Finally, PQTR has an inherent capacity to accommodate a symmetric $𝒳$ with minor modifications. This is mainly because ${\mathit{W}}_k(\tau )$ generated from the resulting symmetric $𝒞(\tau )$ would be the same for all $k=1,\dots ,m$, leading to the symmetric $𝒯(\tau )$ at Step 5. Then, at Step 6 of PQTR, we regress $Y$ only over the non-redundant entries of the symmetric $𝒯(\tau )$, for example, lower triangular elements of a two-way $𝒯(\tau )$, and use the regression results to fill in all entries of $𝒟(\tau )$ by the symmetry constraint. The symmetric $𝒟(\tau )$ is then converted to a symmetric $ℬ(\tau )$ at Step 7.

2.3. Implementation of the PQTR procedure

Note that the PQTR procedure only requires the inputs of $𝒞(\tau )$, ${\Sigma }_k$ and $d_k(k=1,\dots ,m)$. Therefore, to implement PQTR based on a dataset, we only need to replace $𝒞(\tau )$ and ${\Sigma }_k$ in Algorithm 1 by their estimates and select proper $d_k$’s. More specifically, to obtain an estimate for $𝒞(\tau )$, we can first etimate ${\alpha }_{Y\mid \mathit{Z}}(\tau )$ and ${\gamma }_{Y\mid \mathit{Z}}(\tau )$ by $\left({\widehat{\alpha }}_{Y\mid \mathit{Z}}(\tau ),{\widehat{\gamma }}_{Y\mid \mathit{Z}}(\tau )\right)=\underset{\alpha ,\gamma }{\text{arg}\text{min}}\frac{1}{n}\sum _{i=1}^n{\rho }_{\tau }\left(Y_i-\alpha -{\gamma }^{\mathsf{T}}{\mathit{Z}}_i\right)$. This optimization problem can be solved as the regular quantile regression problem since ${\mathit{Z}}_i$, is a low-dimensional vector. Following the lines of Li et al. (2015), we can estimate $𝒞(\tau )$ by $𝒞(\tau )=\frac{1}{n}\sum _{i=1}^n{\psi }_{\tau }\left(Y_i-{\widehat{\alpha }}_{Y\mid \mathit{Z}}(\tau )-{\widehat{\gamma }}_{Y\mid \mathit{Z}}^{\mathsf{T}}(\tau ){\mathit{Z}}_i\right){𝒳}_i$. In addition, ${\Sigma }_k$ can be estimated by ${\Sigma }_k=n^{-1}\sum _{i=1}^n{𝒳}_{i(k)}{𝒳}_{i(k)}^{\mathsf{T}}$, up to a multiplicative constant, where ${𝒳}_{i(k)}$ is the mode-$k$ matricization of ${𝒳}_i(i=1,\dots ,n,k=1,\dots ,m)$ (Zhang and Li, 2017; Li and Zhang, 2017). We require that $n\prod _{j\ne k}{p}_j>p_k$ so that ${\Sigma }_k$ is invertible. By implementing Algorithm 1 with $𝒞(\tau )$ and ${\Sigma }_k$ replaced by $𝒞(\tau )$ and ${\Sigma }_k$ respectively, we then obtain the sample counterparts of ${\mathit{W}}_k(\tau )$ and $𝒯(\tau )$, denoted by ${\mathit{W}}_k(\tau )$ and $𝒯(\tau )\doteq ⟦\mathit{X};{\mathit{W}}_1^{\mathsf{T}}(\tau ),\dots ,{\mathit{W}}_m^{\mathsf{T}}(\tau )⟧$, respectively. Next, we obtain the estimators $\widehat{\alpha }(\tau ),\widehat{\gamma }(\tau )$ and $𝒟(\tau )$ from Step 6 and finally estimate ${ℬ}_0(\tau )$ by $ℬ(\tau )=⟦𝒟(\tau );{\mathit{W}}_1(\tau ),\dots ,{\mathit{W}}_m(\tau )⟧$ in Step 7.

Selecting proper $d_k$’s is a crucial part of the PQTR procedure. A natural way is to conduct cross-validation (CV) using the quantile regression loss function (2) as the criterion function. However, a CV method is typically computationally intensive. Therefore, we consider a eigenvalue-ratio (ER) approach, which is expected to be more computationally efficient. The key rationale is that Steps 3–4 of PQTR are expected to generate ${\mathit{W}}_k(\tau )$ associated with the eigenvalues of ${𝒞}_{0k}(\tau ){𝒞}_{0k}^{\mathsf{T}}(\tau )$ that are nonzero or much larger than the rest of eigenvalues. Consequently, we propose to choose $d_k$ by maximizing the ratio of two adjacent eigenvalues of ${𝒞}_{0k}(\tau ){𝒞}_{0k}^{\mathsf{T}}(\tau )$, following existing strategies in literature, for example, Lam and Yao (2012), Ahn and Horenstein (2013), and Fan et al. (2016). Specifically, for each $k$, we first generate, ${\lambda }_{k1},\dots ,{\lambda }_{l_{k\max }}$, the largest $l_{k\max }$ eigenvalues of ${𝒞}_{0k}(\tau ){𝒞}_{0k}^{\mathsf{T}}(\tau )$, where $l_{k\max }\le \lfloor {(n-p_z-1)}^{1∕m}\rfloor$ and $\lfloor x\rfloor$ denotes the largest integer that is smaller than $x$. We then choose the $d_k$ as ${\widehat{d}}_k={\text{arg max}}_{1\le l\le l_{k\max }}\{{\lambda }_{kl}∕{\lambda }_{k(l+1)}\}$. Based on our simulation studies, both CV and ER approaches perform well with realistic sample sizes, and it generally takes longer to implement the CV approach. Thus, in practice, one may use either ER or CV approach to select $d_k$’s when applying the proposed PQTR procedure, and the ER method may be preferred for the purpose of saving computational cost.

3. Population Interpretation of Partial Quantile Tensor Regression

3.1. Population latent variable model

It is critically important to understand how the proposed PQTR procedure can serve to address the quantile tensor regression model (1). To this end, in Propositions 1 and 2, we first show that PQTR induces a latent “orthogonal” decomposition of $𝒳$, and then identify a general mode-wise condition under which PQTR has an appealing population latent variable model representation. To better understand the general mode-wise condition and the population model, in Section 3.2, we propose a novel envelope quantile tensor regression (EQTR) model. In Section 3.3, we study the connection between the EQTR model and the general mode-wise condition, which entails an elegant population interpretation of the estimates obtained from the PQTR procedure.

Proposition 1

The PQTR procedure decomposes $𝒳$ as

$$𝒳=⟦(\tau );{\mathit{P}}_1^{\mathsf{T}}(\tau ),\dots ,{\mathit{P}}_m^{\mathsf{T}}(\tau )⟧+\mathscr{H}(\tau ),$$ (3)

where ${\mathit{P}}_k(\tau )={\left({\mathit{W}}_k^{\mathsf{T}}(\tau ){\Sigma }_k{\mathit{W}}_k(\tau )\right)}^{-1}{\mathit{W}}_k^{\mathsf{T}}(\tau ){\Sigma }_k\in {\mathbb{R}}^{d_k\times p_k}$,

$$\mathscr{H}(\tau )=\sum \_j=0^{\wedge }m-1\sum \_\sum _{l=1}^m{u}_l=j⟦𝒳;{\mathit{Q}}_1(\tau ,u_1),\dots ,{\mathit{Q}}_m(\tau ,u_m)⟧\in {\mathbb{R}}^{p_1\times \cdots \times p_m}.$$

Here, $u_l$ is some binary variable for $l=1,\dots m$, and ${\mathit{Q}}_k(\tau ,u_k)=(1-2u_k){\mathit{Q}}_k(\tau )+u_k{\mathit{I}}_{p_k\times p_k}$. with ${\mathit{Q}}_k(\tau )={\mathit{Q}}_{d_{k}k}(\tau )$ defined in POTR. Furthermore, $\mathscr{H}(\tau )$ and $𝒯(\tau )$ are uncorrelated as $\mathrm{vcov}\left(\mathscr{H}(\tau ),𝒯(\tau )\right)=0$.

Proposition 2

Under model (1), if ${ℬ}_0{(\tau )}_{(k)}\in \text{span}\left({\mathit{W}}_k(\tau )\right)$ for $k=1,\dots ,m$, the quantile tensor regression model (1) reduces to the latent variable models (3) and (4):

$$Q_Y(\tau \mid 𝒳,\mathit{Z})=Q_Y(\tau \mid 𝒯(\tau ),\mathit{Z})={\alpha }_0(\tau )+{\gamma }_0^{\mathsf{T}}(\tau )\mathit{Z}+\langle {𝒟}_0(\tau ),𝒯(\tau )\rangle ,$$ (4)

where ${𝒟}_0(\tau )=⟦{ℬ}_0(\tau );{\mathit{P}}_1(\tau ),\dots ,{\mathit{P}}_m(\tau )⟧\in {\mathbb{R}}^{d_1\times \cdots \times d_m}$.

Propositions 1 and 2 imply that $𝒳$ can be divided into two uncorrelated parts but only one part is related to the $\tau \text{th}$ conditional quantile of $Y$ through the latent variable $𝒯(\tau )$ given that ${ℬ}_0{(\tau )}_{(k)}(\tau )\in \text{span}\left({\mathit{W}}_k(\tau )\right)$. Implementing the proposed PQTR procedure first renders an approximation for $𝒯(\tau )$ (as indicated by Proposition 1), and subsequently provides estimates for ${𝒟}_0(\tau )$ under model (4) and then ${ℬ}_0(\tau )$ through Steps 6 and 7. The population interpretation of PQTR implied by the latent variable models (3) and (4) shares a similar spirit with the classical latent variable interpretation of PLS under linear regression (Helland, 1990).

3.2. Envelope quantile tensor regression model

To better understand the scenarios implied by the mode-wise condition ${ℬ}_0{(\tau )}_{(k)}\in \text{span}\left({\mathit{W}}_k(\tau )\right)$ which leads to the population latent variable models (3) and (4), we further propose and study an envelope quantile tensor regression (EQTR) model, which adapts the nascent envelope concept (Cook et al., 2010; Cook, 2018) under the quantile tensor regression model (1). Intuitively speaking, we formulate the new EQTR model by adapting the lines of sufficient dimension reduction (Cook, 1998) that divides the tensor covariate $𝒳$ into the material and immaterial parts. As shown in Section 3.3, ${ℬ}_0{(\tau )}_{(k)}\in \text{span}\left({\mathit{W}}_k(\tau )\right)(k=1,\dots ,m)$ holds under the EQTR model. This allows us to bridge the new EQTR model and the proposed PQTR procedure via the latent variable models (3) and (4).

To formulate the EQTR model, we propose to address the division of the material part and the immaterial part in a mode-wise manner as a special accommodation for the tensor covariate. Specifically, under model (1), we assume that there exists a series of $\tau$ -specific subspaces, ${\mathbb{S}}_{k,\tau }\subseteq {\mathbb{R}}^{p_k}(k=1,\dots ,m)$ with dimension ${\tilde{r}}_k\le p_k$, such that for $\tau \in \Delta$,

$$\mathrm{cov}\left(𝒳{\times }_k{\mathit{Q}}_{{\mathbb{S}}_{k,\tau }},𝒳{\times }_k{\mathit{P}}_{{\mathbb{S}}_{k,\tau }}\right)=0.$$ (E1)

$$Q_Y(\tau \mid 𝒳,\mathit{Z})=Q_Y(\tau \mid 𝒳{\times }_k{\mathit{P}}_{{\mathbb{S}}_{k,\tau }},\mathit{Z}).$$ (E2)

Here, ${\mathit{P}}_{{\mathbb{S}}_{k,\tau }}\in {\mathbb{R}}^{p_k\times p_k}$ is the orthogonal projection matrix onto ${\mathbb{S}}_{k,\tau }$, and ${\mathit{Q}}_{{\mathbb{S}}_{k,\tau }}$ is the orthogonal projection matrix onto the complement space of ${\mathbb{S}}_{k,\tau }$. When $m=1$ (i.e., vector covariate), these assumptions are the same as those adopted by Ding et al. (2020). Condition (E1) means that $𝒳$ can be decomposed into two uncorrelated mode-$k$ linear combinations, $𝒳{\times }_k{\mathit{Q}}_{{\mathbb{S}}_{k,\tau }}$ and $𝒳{\times }_k{\mathit{P}}_{{\mathbb{S}}_{k,\tau }}$. Condition (E2) implies $𝒳$ influences the $\tau \text{th}$ conditional quantile of $Y$ only through $𝒳{\times }_k{\mathit{P}}_{{\mathbb{S}}_{k,\tau }}$. These share the same spirit with the interpretations of model (3) and model (4).

To understand how conditions (E1) and (E2) are connected to the nascent envelope concept, in the following, we introduce the definitions of reducing subspace and envelope (Cook et al., 2010; Zhang and Li, 2017; Cook, 2018), and present Proposition 3, which reveals the envelope structure implied by (E1) and (E2).

Definition 1

A subspace $\mathbb{S}\subseteq {\mathbb{R}}^r$ is said to be a reducing subspace of $\mathit{M}\in {\mathbb{R}}^{r\times r}$ if $\mathbb{S}$ decomposes $\mathit{M}$ as $\mathit{M}={\mathit{P}}_{\mathbb{S}}\mathit{M}{\mathit{P}}_{\mathbb{S}}+{\mathit{Q}}_{\mathbb{S}}\mathit{M}{\mathit{Q}}_{\mathbb{S}}$, where ${\mathit{P}}_{\mathbb{S}}$ is the projection matrix onto $\mathbb{S}$ and ${\mathit{Q}}_{\mathbb{S}}$ the projection matrix to the complement space of $\mathbb{S}$. If $\mathbb{S}$ is a reducing subspace of $\mathit{M}$, we say that $\mathbb{S}$ reduces $\mathit{M}$.

Definition 2

Let $\mathit{M}\in {\mathbb{R}}^{r\times r}$ and let $\mathbb{S}\subseteq \text{span}(\mathit{M})$. Then the $\mathit{M}$ -envelope of $\mathbb{S}$, denoted by ${ℰ}_{\mathit{M}}(\mathbb{S})$, is the intersection of all reducing subspaces of $\mathit{M}$ that contain $\mathbb{S}$. For a matrix $\mathit{A}$ such that $\text{span}(\mathit{A})\subseteq \text{span}(\mathit{M})$, define ${ℰ}_{\mathit{M}}(\mathit{A})={ℰ}_{\mathit{M}}\left(\text{span}(\mathit{A})\right)$, where $\text{span}(\mathit{A})$ is the space spanned by the columns of $\mathit{A}$.

Definition 3

Let $𝒜\in {\mathbb{R}}^{p_1\times \cdots \times p_m}$ and $\text{vec}(𝒜)\in \text{span}(\mathit{M})$ for $\mathit{M}\in {\mathbb{R}}^{\left(\prod _{k=1}^m{p}_k\right)\times \left(\prod _{k=1}^m{p}_k\right)}$. The $\mathit{M}$ -tensor envelope of $𝒜$, denoted by ${ℰ}_{\mathit{M}}(𝒜)$, is the intersection of all reducing subspaces of $\mathit{M}$ that contain $\text{span}\left(\text{vec}(𝒜)\right)$, that is, ${ℰ}_{\mathit{M}}(𝒜)={ℰ}_{\mathit{M}}\left(\text{vec}(𝒜)\right)$.

Proposition 3

Under model (1), conditions 1 and 2 imply that for any $\tau \in \Delta$, ${ℬ}_0(\tau )=⟦𝒦(\tau );{\Gamma }_1(\tau ),\dots ,{\Gamma }_m(\tau )⟧$ and

$${\Sigma }_k={\Gamma }_k(\tau ){\Omega }_k(\tau ){\Gamma }_k^{\mathsf{T}}(\tau )+{\Gamma }_{0k}(\tau ){\Omega }_{0k}(\tau ){\Gamma }_{0k}^{\mathsf{T}}(\tau ),k=1,\dots ,m.,$$ (5)

where $𝒦(\tau )\in {\mathbb{R}}^{{\tilde{r}}_1\times \cdots \times {\tilde{r}}_m}$, ${\Gamma }_k(\tau )\in {\mathbb{R}}^{p_k\times {\tilde{r}}_k}$ denotes a semi-orthogonal basis of ${\mathbb{S}}_{k,\tau }$ such that ${\mathit{P}}_{{\mathbb{S}}_{k,\tau }}={\Gamma }_k(\tau ){\Gamma }_k^{\mathsf{T}}(\tau ),{\Gamma }_{0k}(\tau )\in {\mathbb{R}}^{p_k\times (p_k-{\tilde{r}}_k)}$ denotes a basis of the complement space of ${\mathbb{S}}_{k,\tau }$ such that ${\mathit{Q}}_{{\mathbb{S}}_{k,\tau }}={\Gamma }_{0k}(\tau ){\Gamma }_{0k}^{\mathsf{T}}(\tau )$, ${\Omega }_k(\tau )={\Gamma }_k^{\mathsf{T}}(\tau ){\Sigma }_k{\Gamma }_k(\tau )\in {\mathbb{R}}^{{\tilde{r}}_k\times {\tilde{r}}_k}$ and ${\Omega }_{0k}(\tau )={\Gamma }_{0k}^{\mathsf{T}}(\tau ){\Sigma }_k{\Gamma }_{0k}(\tau )\in {\mathbb{R}}^{(p_k-{\tilde{r}}_k)\times (p_k-{\tilde{r}}_k)}$, which are two symmetric positive-definite matrices.

Proposition 3 shows that ${\Sigma }_k$ can be decomposed into two orthogonal parts by projecting onto ${\mathbb{S}}_{k,\tau }$ and is connected with ${ℬ}_0(\tau )$ through ${\Gamma }_k(\tau )$, a semi-orthogonal basis of ${\mathbb{S}}_{k,\tau }$. From Proposition 3, we also see that ${ℬ}_0{(\tau )}_{(k)}\in {\mathbb{S}}_{k,\tau }$ and ${\Sigma }_k={\mathit{Q}}_{{\mathbb{S}}_{k,\tau }}{\Sigma }_k{\mathit{Q}}_{{\mathbb{S}}_{k,\tau }}+{\mathit{P}}_{{\mathbb{S}}_{k,\tau }}{\Sigma }_k{\mathit{P}}_{{\mathbb{S}}_{k,\tau }}$; hence ${\mathbb{S}}_{k,\tau }$ is a reducing subspace of ${\Sigma }_k$ that contains ${ℬ}_0{(\tau )}_{(k)}$. Therefore, we further set ${\mathbb{S}}_{k,\tau }={ℰ}_{{\Sigma }_k}\left({ℬ}_0{(\tau )}_{(k)}\right)$. According to Definition 2, such defined ${\mathbb{S}}_{k,\tau }$’s uniquely exist. Model (1), combined with conditions (E1) and (E2), confers the proposed EQTR model.

3.3. Connection between EQTR model and the PQTR procedure

In this subsection, we show that under the proposed EQTR model, the mode-wise sufficient condition in Proposition 2, ${ℬ}_0{(\tau )}_{(k)}\in \text{span}\left({\mathit{W}}_k(\tau )\right)$, can be satisfied, and thus PQTR can serve as a legitimate approach to providing scalable estimation of the EQTR model.

First, we note that, provided ${ℬ}_0{(\tau )}_{(k)}\in {\mathbb{S}}_{k,\tau }$ and ${\mathbb{S}}_{k,\tau }={ℰ}_{{\Sigma }_k}\left({ℬ}_0{(\tau )}_{(k)}\right)$, if we can show ${ℰ}_{{\Sigma }_k}\left({ℬ}_0{(\tau )}_{(k)}\right)\subseteq \text{span}\left({\mathit{W}}_k(\tau )\right)$, then ${ℬ}_0{(\tau )}_{(k)}\in \text{span}\left({\mathit{W}}_k(\tau )\right)$ follows. Following this line, we first establish the equivalence between $\text{span}\left({\mathit{W}}_k(\tau )\right)$ and ${ℰ}_{{\Sigma }_k}\left(𝒞{(\tau )}_{(k)}\right)$ in Proposition 4:

Proposition 4

Let $r_k$ be the dimension of ${ℰ}_{{\Sigma }_k}\left(𝒞{(\tau )}_{(k)}\right)$ for $k=1,\dots ,m$. Under model (1), for $k=1,\dots ,m$ and $\tau \in \Delta$, $\text{span}\left({\mathit{W}}_k(\tau )\right)={ℰ}_{{\Sigma }_k}\left(𝒞{(\tau )}_{(k)}\right)$ if $d_k\ge r_k$.

The result in Proposition 4 suggests that the PQTR procedure finds a semi-orthogonal basis of ${ℰ}_{{\Sigma }_k}\left(𝒞{(\tau )}_{(k)}\right)$.

The next step is to link ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$ and ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$. To this end, we utilize the result from Lemma 1, ${\mathit{V}}^{-1}(\tau )\text{vec}(𝒞(\tau ))=\text{vec}({ℬ}_0(\tau ))$. However, ${\mathit{V}}^{-1}(\tau )$, the matrix that bridges the tensor coefficient and the quantile partial tensor covariance, does not have a Kronecker structure, unlike its counterpart in the linear regression setting, ${\Sigma }_{𝒳}^{-1}$. This prevents us from attaining the mode-wide sufficient condition that ensures the nice population latent variable interpretation by straightforwardly adapting the method of Zhang and Li (2017). To tackle this difficulty, we propose a novel set of the regularity structural constraints tailored to quantile tensor regression.

Let $\mathit{U}(\tau )$ denote the semi-orthgonal basis of ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$, columns of which are eigenvectors of ${\Sigma }_{𝒳}$, and ${\mathit{Q}}_{\mathit{U}(\tau )}$ denote the projection matrix onto $\text{span}{\left(\mathit{U}(\tau )\right)}^{\perp }$. We impose the following conditions:

1. (i) $E\left[\left(\text{vec}{(𝒳)}^{\mathsf{T}},{\mathit{Z}}^{\mathsf{T}}\right)\mid \mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳),\mathit{Z}\right]$ is a linear function of ${(\text{vec}{(𝒳)}^{\mathsf{T}}\mathit{U}(\tau ),{\mathit{Z}}^{\mathsf{T}})}^{\mathsf{T}}$ and (ii) ${\mathit{Q}}_{\mathit{U}(\tau )}E\left[\text{vec}(𝒳){\mathit{Z}}^{\mathsf{T}}\right]=0$.

2. Suppose that $\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒞)$ does not contain zero elements.

Condition (E3)(i) mimics the condition in Proposition 2 of Cook and Zhang (2015) and plays a role in connecting $\mathit{V}(\tau )$-envelope with ${\Sigma }_{𝒳}$-envelope. In particular, condition (E3)(i) known as the linearity condition, has been widely considered in the literature of sufficient dimension reduction (Cook, 1998; Li and Wang, 2007) and envelop models for generalized linear model (Cook and Zhang, 2015). A sufficient condition for condition (E3)(i) is that ${\left(\text{vec}{(𝒳)}^{\mathsf{T}},{\mathit{Z}}^{\mathsf{T}}\right)}^{\mathsf{T}}$ follows an elliptically contoured distribution, for example, the multivariate normal distribution. Diaconis and Freedman (1984) and Hall and Li (1993) demonstrated that almost all projections of high-dimensional data are approximately normal, and thus this assumption is usually considered as mild. Condition (E3)(ii) means that the covariance between $𝒳$ and $\mathit{Z}$ is in ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$. A stronger version of condition (E3)(ii) is that $E\left[\text{vec}(𝒳){\mathit{Z}}^{\mathsf{T}}\right]=0$, implying that $𝒳$ and $\mathit{Z}$ are uncorrelated. This occurs in practice, for example, when $\mathit{Z}$ represents a randomized treatment indicator.

Condition (E4) is a crucial technical condition that leads to ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)\subseteq {ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$. In view of model (6) or (E2) and the definition of ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$, $\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳)$ may be regarded as latent variables that captures all the information regarding the association between the $\tau \text{th}$ conditional quantile of $Y$ and $𝒳$ and contains no redundant information. Since $\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒞)=E\left[R(\tau )\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳)\right]$, which reflects the association between $\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳)$ and the $\tau \text{th}$ conditional quantile of $Y$ after adjusting for the marginal effect of $\mathit{Z}$, it is reasonable to assume that all elements of $E\left[R(\tau ){\mathit{U}}^{\mathsf{T}}(\tau )\text{vec}(𝒳)\right]$ are nonzero; otherwise the elements of $\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳)$ corresponding to the zero elements of $E\left[R(\tau )\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳)\right]$ would be superfluous. Under conditions 1 and 2, we establish the relationship between ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$ and ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ in Proposition 5, which subsequently leads to the conclusion in Corollary 1 that justifies the latent variable model in Proposition 2.

Proposition 5

Suppose model (1) and conditions 1–2 hold, the assumptions of Lemma 1 are satisfied, and $d_k\ge r_k(k=1,\dots ,m)$. We have ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)\subseteq {ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$. Furthermore, let $f_{Y\mid \mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳),\mathit{Z}}(y)$ denote conditional density function of $Y$ given $\left(\mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳),\mathit{Z}\right)$. If $f_{Y\mid \mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳),\mathit{Z}}(y)=f_{Y\mid 𝒳,\mathit{Z}}(y)$ for all $y\in \mathbb{R}$, then ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)={ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$.

Corollary 1

Under the assumptions of Proposition 5, ${ℬ}_0{(\tau )}_{(k)}\in \text{span}\left({\mathit{W}}_k(\tau )\right)$.

Unlike the result for TEPLS under linear regression (Zhang and Li, 2017), the finding from Proposition 5 suggests that ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$ is usually not equal to but a subspace of ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$. A possible explanation for this unique phenomenon of PQTR is tied to the fact that ${\mathit{V}}^{-1}(\tau )\text{vec}(𝒞(\tau ))=\text{vec}({ℬ}_0(\tau ))$ but $\mathit{V}(\tau )$ involves $f_{Y\mid 𝒳,\mathit{Z}}(\cdot )$ that contains the information on ${ℬ}_0(\tau )$ across all $\tau \in (0,1)$; thus it is intuitive to perceive that ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ can be bigger than ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)$. A sufficient condition for ${ℰ}_{{\Sigma }_{𝒳}}\left({ℬ}_0(\tau )\right)={ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ is $f_{Y\mid \mathit{U}{(\tau )}^{\mathsf{T}}\text{vec}(𝒳),\mathit{Z}}(y)=f_{Y\mid 𝒳,\mathit{Z}}(y)$. A special case where this condition holds is the case where model (1) reduces to the standard linear tensor model with i.i.d. errors. Another special case is a scale-shift model $Y={\left(\langle {ℬ}^{\ast },𝒳\rangle +{\gamma }^{\mathsf{T}}\mathit{Z}\right)}^{ϵ}$, where $ϵ$ is independent of $(𝒳,\mathit{Z})$.

The result in Corollary 1 implies that the latent variable model representation of PQTR, given by (3)-(4), holds under the proposed EQTR model. In sum, the PQTR procedure finds ${\mathit{W}}_k(\tau )$, which is a semi-orthogonal basis of ${ℰ}_{{\Sigma }_k}\left({𝒞}_{(k)}(\tau )\right)$ that contains ${ℬ}_0{(\tau )}_{(k)}$, and subsequently yields estimates for ${\alpha }_0(\tau )$, ${\gamma }_0(\tau )$ and ${ℬ}_0(\tau )$ in model (1) through the latent variable model (4).

4. Asymptotic Properties of the Proposed Estimator

We study the asymptotic properties of the estimator obtained from the proposed PQTR procedure, denoted by $\theta (\tau )=\left(\widehat{\alpha }(\tau ),\widehat{\gamma }(\tau ),ℬ(\tau )\right)$. We first introduce necessary notations. Write $\theta (\tau )=\left(\alpha (\tau ),\gamma (\tau ),ℬ(\tau )\right)$ and ${\theta }_0(\tau )=\left({\alpha }_0(\tau ),{\gamma }_0(\tau ),{ℬ}_0(\tau )\right)$. We also define $L_2$-metric $d\left({\theta }_1,{\theta }_2\right)$ in the parameter space $\Theta (\tau )\subseteq \mathbb{R}\times {\mathbb{R}}^{p_z}\times {\mathbb{R}}^{\prod _{k=1}^m{p}_k}$ as $d\left({\theta }_1,{\theta }_2\right)=\sqrt{{({\alpha }_1-{\alpha }_2)}^2+{\Vert {\gamma }_1-{\gamma }_2\Vert }_2^2+{\Vert {ℬ}_1-{ℬ}_2\Vert }_F^2}$, where ${\Vert ℬ\Vert }_F\doteq \sqrt{\text{vec}{(ℬ)}^{\mathsf{T}}\text{vec}(ℬ)}$ stands for the Frobenius norm.

In Theorem 1, we establish the consistency and convergence rate of $\theta$.

Theorem 1

Suppose that regularity conditions $(C1)-(C6)$ in Section S2 of the Supplementary Materials and model (4) hold. If $d_k=r_k$, then $d\left(\theta (\tau ),{\theta }_0(\tau )\right)=o_p(1)$ and $\sqrt{n}d\left(\theta (\tau ),{\theta }_0(\tau )\right)=O_p(1)$ as $n\to \infty$.

Note that model (4) holds under the assumptions of Proposition 5. Therefore, by Theorem 1, we establish the $\sqrt{n}$-consistency of the estimator obtained from the PQTR procedure under the proposed EQTR model.

We also establish the consistency of the eigenvalue ratio method in Theorem 2

Theorem 2

Suppose the regularity conditions $(C1)-(C6)$ in Section S2 and condition (D1) in Section S7 of the Supplementary Materials hold. Under model (4), ${\widehat{d}}_k$ obtained from the eigenvalue ratio method (described in Section 2.3) equals $r_k$ with probability approaching to 1, i.e., $\text{Pr}({\widehat{d}}_k=r_k)\to 1$ as $n\to \infty$.

Proofs of Lemma 1, Propositions 1-5, and Theorems 1-2 are provided in Sections S4-S7 of the Supplementary Materials.

5. Simulation Studies

We evaluate the finite sample performance of the proposed method and compare it with some alternative methods via Monte Carlo simulations. We first consider scenarios with a second-order tensor covariate, $𝒳\in {\mathbb{R}}^{50\times 50}$, where the tensor covariate coefficient $ℬ(\tau )\in {\mathbb{R}}^{50\times 50}$ is the same for all $\tau$ (i.e., homogeneous scenarios) or vary across different $\tau$’s (i.e., heterogeneous scenarios).

For the homogeneous scenarios, we generate responses as $Y_i={\gamma }^{\mathsf{T}}{\mathit{Z}}_i+\langle ℬ,{𝒳}_i\rangle +{\Phi }_t^{-1}({\xi }_i)$, where ${\xi }_i$’s are i.i.d. standard uniform random variables, and ${\Phi }_t$ is the distribution function of Student’s $t$ distribution with 1 degree of freedom. For $ℬ$, as shown in Figures S1 and S2 in the Supplementary Materials, we consider frame and bi-circle shapes, which are realistic for mental health neuroimaging datasets. The frame shape represents a low-rank case of $ℬ$, while the bi-circle shape is a challenging high-rank case with rank 12, where ${𝒟}_0(\tau )$ includes 144 parameters. In this case, ${ℰ}_{{\Sigma }_{𝒳}}\left(ℬ(\tau )\right)={ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ and, $r_k$, the rank of ${ℰ}_{{\Sigma }_{𝒳}}\left(ℬ(\tau )\right)$, equals to ${\tilde{r}}_k$, the rank of $ℬ$, for $k=1,2$.

For the heterogeneous scenarios, we generate responses as $Y_i={\gamma }^{\mathsf{T}}{\mathit{Z}}_i+\langle ℬ({\xi }_i),{𝒳}_i\rangle +{\Phi }_t^{-1}({\xi }_i)$, where ${\xi }_i$’s are i.i.d. standard uniform random variables, and each element of $ℬ({\xi }_i)$ is increasing with respect to ${\xi }_i$. We consider two settings for $ℬ(\tau )$. In the first setting, $ℬ(\tau )={\Phi }^{-1}(\tau )\mathit{B}$ as illustrated in Figure S5 in the Supplementary Materials. In this case, $ℬ(\tau )$ is a matrix of the same rank and bases across different $\tau$’s but the values of its elements change with $\tau$. The first setting incurs ${ℰ}_{{\Sigma }_{𝒳}}\left(ℬ(\tau )\right)={ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ and $r_k={\tilde{r}}_k$. For the second setting, we set $ℬ(\tau )={\mathit{B}}_1\mathbf{I}\left(0<\tau <0.35\right)+{\mathit{B}}_2\mathbf{I}\left(0.35\le \tau <0.65\right)+{\mathit{B}}_3\mathbf{I}\left(0.65\le \tau <1\right)$ and plot the resulting $ℬ(\tau )$ in Figure 1 and S6 in the Supplementary Materials. The $ℬ(\tau )$ shown in Figure S6 in the Supplementary Materials is defined based on ${\{{\mathit{B}}_l\}}_{l=1}^3$ of the same rank 1 but different bases. The $ℬ(\tau )$ shown in Figure 1 corresponds to ${\{{\mathit{B}}_l\}}_{l=1}^3$ that have different ranks and bases, representing the most challenging setting. In this setting, ${ℰ}_{{\Sigma }_{𝒳}}\left(ℬ(\tau )\right)$ is not always equal to ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ because the rank of ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ is 3 but the rank of ${ℰ}_{{\Sigma }_{𝒳}}\left(ℬ(\tau )\right)$ is the rank of $ℬ(\tau )$, which is sometimes smaller than 3.

Figure 1:

Empirical averages of estimates for the heterogeneous tri-square-shaped tensor coefficients that are step functions of $\tau$ with $\tau =0.1,0.25,0.5,0.75$, and 0.9 when the covariance of the tensor covariate follows an envelope structure (Envelope) or a diagonal constant structure (Diagonal).

For both homogeneous and heterogeneous scenarios, we generate ${𝒳}_i$ with covariance ${\Sigma }_2\otimes {\Sigma }_1$, where ${\Sigma }_k(k=1,2)$ either follow the envelope structure in (5) or are diagonal-constant matrices which depict a reasonable spatial correlation pattern but do not follow the envelope structure in (5). The generation schemes for ${𝒳}_i$ and ${\mathit{Z}}_i$ are detailed in the Sections S1.1 and S1.2 of the Supplementary Materials.

In addition, we consider a scenario with a 3-way tensor covariate of dimension 16 × 16 × 16 , where the tensor coefficient is heterogeneous over $\tau$, as shown in Figure S8 in the Supplementary Materials. The tensor covariate ${𝒳}_i$, ${\mathit{Z}}_i$, and $Y_i$ are generated in the same manner as that for the heterogeneous setting with a 2-way tensor covariate.

We compare the performance of the following five methods:

Fix: the proposed PQTR procedure with $d_k$ fixed as the true rank of $ℬ$ in the homogeneous scenario and the true rank of $\mathit{B}$ or ${\cup }_{l=1,2,3}\text{span}({\mathit{B}}_l)$ (i.e., the largest possible rank of ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$) in the heterogeneous scenario;

ER: the proposed PQTR procedure with $d_k$’s estimated by the eigenvalue ratio method;

CV: the proposed PQTR procedure with $d_k$’s estimated by 5-fold CV;

CP: Li and Zhang (2021)’s CP-decomposition based method for quantile tensor regression that uses an ABU algorithm and adopts the fuse penalty.

TK: Lu et al. (2020)’s Tucker decomposition-based method for quantile tensor regression that uses an ABU algorithm with lasso-type penalty and does not accommodate a low dimensional vector predictor.

PCA: quantile regression with a reduced covariate vector obtained from applying the principle component analysis (PCA) to the vectorized tensor covariate.

Since the rank of ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞(\tau )\right)$ is unknown with real datasets, the method, Fix, serves as an oracle or benchmark version of the proposed method, while the methods, ER and CV, represent practical versions of the proposed PQTR. The fuse penalty used in CP was recommended and used in Li and Zhang (2021)’s numerical studies. Both CP and TK methods select the ranks from 1 to 3 and determine the tuning parameters by minimizing BIC as suggested by Li and Zhang (2021). To save the computational cost of TK, we assume the same rank for all modes when the tensor coefficient is 3-way. The initial values of CP and TK are generated from the L1-norm quantile regression (Li and Zhu, 2008) with the vectorized tensor covariates and the tuning parameter 0.1. All five methods are implemented in MATLAB. The MATLAB code for CP is obtained from the original implementation provided by Li and Zhang (2021), and the MATLAB code for TK is translated from the R code used by Lu et al. (2020). We evaluate these methods with sample sizes, 50, 100 and 200, which are realistic in neuroimaging studies, based on 100 replicates. The set of quantile levels of interest is specified as $\Delta =\{0.1,0.25,0.5,0.75,0.9\}$.

In Figures 1, and Figures S5, S6, and S10 in the Supplementary Materials, which correspond to the heterogeneous settings of $ℬ$ with either a two-way tensor covariate or a three-way tensor covariate, we present the empirical averages of the estimated tensor coefficients based on the five methods described above. These figures help assess the estimation biases of different methods by visualization. Figure S7 in the Supplementary Materials includes the box plots in the logarithmic scale that depict the Frobenius norm of the difference between the true tensor coefficients and the estimated tensor coefficients based on the six different methods, which provide meaningful quantification of estimation variability. Figure S8 in the Supplementary Materials presents the box plots of the computation time (in seconds) of the six different methods.

Based on the results for the heterogeneous scenarios shown by Figures 1, S5, S6, and S10, when the EQTR model holds, we note that the proposed methods, Fix, ER, and CV, can distinguish and capture the shapes of the true tensor coefficient across different $\tau$’s quite well for either 2-way or 3-way tensor coefficients. The empirical biases of Fix, ER and CV decrease as the sample size increases. ER seems to perform slightly better than CV in terms of estimation bias, while CV generally shows better estimation accuracy than ER (see Figure S7). In contrast, CP and TK yield noisy estimates across all $\tau$’s. As suggested by Figure S7, this may be caused by the presence of outlying estimates which may occur when CP and TK converge to some local optimums or fail to converge. Since the rank of the true tensor coefficient in the heterogeneous scenarios is always smaller than or equal to 3, the noisy results from CP and TK are not caused by assuming a rank lower than needed but are likely because the ABU optimization algorithm of CP or TK frequently converges to some local optimums provided a larger number of parameters to estimate. When the EQTR model does not hold, the average estimates based on the proposed methods, Fix, ER, and CV, may still capture the shapes of the true tensor coefficients given the relatively large sample size 200, and CV may outperform ER, particular in the three-way tensor coefficient case. In contrast, both CP and TK produce noisy estimates suffering from larger empirical biases and estimation variability. In all cases, the PCA approach tends to underestimate the magnitude of non-zero components of the tensor coefficient. It may completely miss capturing the shape of the true tensor coefficient when the sample size is small (e.g., $n=50$).

Based on the comparison of computation time in Figure S8, it is clear that the implementation of the proposed methods, Fix, ER, and CV, have a comparable speed to that of PCA but can be much faster than that of CP and TK. As expected, CV generally takes longer computation time as compared to ER.

Regarding the comparisons of the proposed methods with CP, TK, and PCA, we have very similar findings from the simulations in the homogeneous settings. We also consider TEPLS a benchmark method in the homogeneous cases. Figures S1-S2 show that TEPLS has comparable performance with PQTR in terms of recovering the shape of $ℬ(\tau )$. Figure S3 shows that TEPLS may yield smaller estimation errors than PQTR. This is expected as TEPLS directly utilizes the true linear relationship between the response and the tensor covariate while the proposed method assumes a more flexible relationship. However, TEPLS may sometimes generate unstable estimates as suggested by the many outlying points in the box-plots corresponding to TEPLS (see Figure S3). In Figure S4, it is shown that the proposed PQTR procedure demands similar amount of computing time as compared to TEPLS. More details are provided in Section S1.1 of the Supplementary Materials.

In summary, our simulation studies support both statistical and computational advantages of the proposed PQTR methods over the existing benchmark approaches.

6. Application

We apply the PQTR procedure to a neuroimaging dataset collected from a cohort of 98 female PTSD patients recruited by the Grady Trauma Project (Ressler et al., 2011). The goal of our analyses is to investigate the association between brain functional connectivity and severity of PTSD symptoms, measured by the PTSD symptom scale (PSS) total score, while adjusting for the effect of age. In our analyses, the response $Y$ corresponds to the PSS total score, the low-dimensional vector covariate $\mathit{Z}$ stands for age, and the tensor covariate $𝒳$ represents the functional connectivity matrix. Connectivity measures were derived from the rs-fMRI based on the Seitzman atlas which include 300 nodes across the brain. After removing 21 nodes with missing data for one or more participants, we analyze the connectivity matrix based on the remaining 279 nodes, which are grouped into functional modules. Before fitting the quantile tensor regression model (1), we apply the Fisher’s z-transformation to each entry of $𝒳$, which is bounded between $-1$ and 1 for being defined as a Pearson’s correlation coefficient. We also set the diagonal elements of $𝒳$ as zero to avoid infinite values after the Fisher’s z-transformation.

We implement the proposed PQTR procedure with $\Delta =\{0.25,0.4,0.5,0.6,0.75\}$. This allows us to examine how brain functional connectivity is associated with low $(\tau =0.25)$, middle $(\tau =0.4,0.5,0.6)$, and high $(\tau =0.75)$ ranges of PSS total score. As discussed in Section 2.2, to accommodate the symmetric nature of the functional connectivity matrix (and thus $ℬ(\tau )$), we slightly modify the proposed PQTR procedure. Specifically, we use the elements in the lower-triangular $𝒯(\tau )$ for the quantile regression conducted at Step 6, obtain the lower-triangular entries of $𝒟(\tau )$, and then symmetrically reflect the lower-triangular entries to impute the corresponding upper-triangular entries of $𝒟(\tau )$. We also compare the quantile regression results to the mean-based linear regression by TEPLS with necessary modifications, similar to those applied to PQTR, to include the lower-dimensional vector covariate for fair comparisons. We let $d_1=d_2$ for a two-way tensor (matrix) coefficient and employ the eigenvalue ratio method to save the computational cost. By the eigenvalue ratio method, $d_k(k=1,2)$ is selected as 1, 3, 3, 2, and 2 for PQTR with $\tau =0.25,0.4,0.5$, and 0.6 respectively, and as 2 for TEPLS.

In Figure 2, we present the standardized estimated tensor coefficients thresholded by 5 (i.e. setting as 0 if the magnitude is smaller than 5). The threshold 5 is chosen to yield reasonably sparse visualization of the estimated tensor coefficients. The dots in red (or blue) indicate the edges that may have significant positive (or negative) effects on quantiles or mean of PSS total score. These edges are also visualized in brain maps in Figure S10 in the Supplementary Materials. A counterpart of Figure 2 that adopts 4 as the thresholding cutoff is Figure S13 in the Supplementary Materials. From Figure 2, we observe a heterogeneous pattern in the effects of brain functional connectivity on different ranges of PSS total score. For example, the estimated coefficients for some edges within the cingulo opercular and default mode networks are negative corresponding to effects on lower quantiles (e.g. $\tau =0.25$) but not effects on upper quantiles (e.g. $\tau =0.75$). This suggests that the negative associations of these edges with PTSD symptom severity may mainly be present among patients with mild PTSD symptoms. For some edges within the somatomotor dorsal network or between the somatomotor dorsal area and the somatomotor lateral area, their coefficients are positive only at upper quantiles (e.g. $\tau =0.75$). An implication of this result is that stronger connectivity of these edges may be reflective of worse symptoms in patients with severe PTSD symptoms. These findings are generally consistent with those from some recent neuroimaging studies. For example, Zhang et al. (2020) reported that the connectivities within the somatomotor network is positively correlated with severity of PTSD symptoms, and Zandvakili et al. (2020) found that the connectivities in the default mode network have a negative relationship with PTSD symptoms.

Figure 2:

The heat map visualization of the standardized $ℬ(\tau )$ by ER thresholded at the cutoff 5, and the standardized estimated tensor coefficient based on mean tensor regression by TEPLS thresholded at the cutoff 5. The gray lines separate brain regions belong to different brain functional networks, which include: Auditory (Aud), Somatomotor Dorsal (SMd), Cingulo Opercular (CO), Default Mode (DMN), Fronto Parietal (FP), Somatomotor Lateral (SMI), Visual (Vis) and Salience (Sal).

In contrast, TEPLS, which adopts mean-based linear tensor regression, discovers little effects of connections within the somatomotor dorsal networks and default mode network on the PSS total score based on our dataset. In fact, no effect estimates by TEPLS pass the threshold 5 as shown in Figure 2. This may reflect diluted effects resulted from assuming constant effects across different ranges of PSS total score, which is inherited with linear regression modeling.

In addition, we apply the CP and TK methods to the PTSD dataset. As commented in Section 1, CP and TK, by using ABU algorithms, are not flexible in accounting for the symmetric structure of $𝒳$. In our analyses, we heuristically make the final coefficient estimate symmetric by specifying it as the average of the estimate obtained from CP (or TK) and its transpose. A similar approach was used by Zhang et al. (2022). Despite the adjustment, we find the results from CP or TK are difficult to interpret, and this is discussed in details in Section 1.4 of the Supplementary Materials.

As illustrated by this example, quantile tensor regression, as addressed by the proposed PQTR procedure, provides a flexible and robust tool to gain a detailed view regarding the association between brain functional connectivity and symptom severity of PTSD, which may not be attained by existing approaches.

7. Concluding remarks

This paper represents the first work that thoroughly studies the integration of PLS strategy with the effort to address quantile regression with a tensor covariate. In this work, we not only deliver an efficient algorithm (i.e. PQTR) to provide estimates scalable to a large tensor covariate but also uncover a meaningful way to interpret the resulting estimates. Moreover, we establish the connection between the latent variable model underlying the PQTR procedure and a new envelope model uniquely formulated for quantile tensor regression. The results contribute useful theoretical insights.

The proposed PQTR procedure bears some similarity with the principal component analysis/regression (PCA) in the sense that both are two-step procedures that carry out dimension reduction in the first step and regression at the second step. At the same time, they are several main distinctions. First, applying an existing PCA procedure to quantile tensor regression requires first vectorizing the tensor covariate $𝒳$. This can lead to interpretation issues as the inherent spatial structure of $𝒳$ is ignored. In contrast, PQTR circumvents such issues by retaining the original tensor form of $𝒳$. Secondly, PCA conducts unsupervised dimension reduction which only utilizes the covariance of covariates and ignores the information from the response (Cook, 2018), while the PQTR procedure implements supervised dimension reduction guided by the correlation between the response and the tensor covariate $𝒳$. As demonstrated by simulation studies, the supervised dimension reduction adopted by PQTR can lead to considerably improved empirical performance.

As enlightened by one referee, we note that the envelope estimation for ${ℰ}_{{\Sigma }_{𝒳}}\left(𝒞{(\tau )}_{(k)}\right)$ involved in the PQTR may be tackled by the Non-Iterative Enveloped Component Estimation (NIECE) procedure of Zhang et al. (2023), leading to a useful variant of PQTR. While NIECE has a principal component regression formulation, the resulting variant, however, is different from the regular application of PCA to tensor regression. This is because the quantile partial tensor covariance $𝒞(\tau )$ still plays a role in determining the envelopes in the former but is not involved in the latter. Delineating the connection between the proposed PQTR and the variant commented above merits future research.

For the tensor covariate, we assume $p_k$’s $(k=1,\dots ,m)$ are fixed and does not increase with the sample size $n$. This covers realistic tensor covariates such as those encountered in neuorimaging studies. When $p_k$’s diverge with the sample size $n$, one may consider imposing regularization to further promote sparsity, similarly to developing a sparse PLS method for linear regression (Chung and Keles, 2010, for example). As pointed out by one referee, one may also consider adapting the sieve estimation technique to allows for diverging $d_k$’s $(k=1,\dots ,m)$, thereby relaxing the low-rank tensor constraint imposed by the proposed method. These enlighten promising directions for future research. The work presented in this paper sets the critical foundation for such investigations.