Structured Matrix Completion with Applications to Genomic Data Integration

Abstract

Matrix completion has attracted significant recent attention in many fields including statistics, applied mathematics and electrical engineering. Current literature on matrix completion focuses primarily on independent sampling models under which the individual observed entries are sampled independently. Motivated by applications in genomic data integration, we propose a new framework of structured matrix completion (SMC) to treat structured missingness by design. Specifically, our proposed method aims at efficient matrix recovery when a subset of the rows and columns of an approximately low-rank matrix are observed. We provide theoretical justification for the proposed SMC method and derive lower bound for the estimation errors, which together establish the optimal rate of recovery over certain classes of approximately low-rank matrices. Simulation studies show that the method performs well in finite sample under a variety of configurations. The method is applied to integrate several ovarian cancer genomic studies with different extent of genomic measurements, which enables us to construct more accurate prediction rules for ovarian cancer survival.

1 Introduction

Motivated by an array of applications, matrix completion has attracted significant recent attention in different fields including statistics, applied mathematics and electrical engineering. The central goal of matrix completion is to recover a high-dimensional low-rank matrix based on a subset of its entries. Applications include recommender systems (Koren et al., 2009), genomics (Chi et al., 2013), multi-task learning (Argyriou et al., 2008), sensor localization (Biswas et al., 2006; Singer and Cucuringu, 2010), and computer vision (Chen and Suter, 2004; Tomasi and Kanade, 1992), among many others.

Matrix completion has been well studied under the uniform sampling model, where observed entries are assumed to be sampled uniformly at random. The best known approach is perhaps the constrained nuclear norm minimization (NNM), which has been shown to yield near-optimal results when the sampling distribution of the observed entries is uniform (Candés and Recht, 2009; Candés and Tao, 2010; Gross, 2011; Recht, 2011; Candes and Plan, 2011). For estimating approximately low-rank matrices from uniformly sampled noisy observations, several penalized or constrained NNM estimators, which are based on the same principle as the well-known Lasso and Dantzig selector for sparse signal recovery, were proposed and analyzed (Keshavan et al., 2010; Mazumder et al., 2010; Koltchinskii, 2011; Koltchinskii et al., 2011; Rohde et al., 2011). In many applications, the entries are sampled independently but not uniformly. In such a setting, Salakhutdinov and Srebro (2010) showed that the standard NNM methods do not perform well, and proposed a weighted NNM method, which depends on the true sampling distribution. In the case of unknown sampling distribution, Foygel et al. (2011) introduced an empirically-weighted NNM method. Cai and Zhou (2013) studied a max-norm constrained minimization method for the recovery of a low-rank matrix based on the noisy observations under the non-uniform sampling model. It was shown that the max-norm constrained least squares estimator is rate-optimal under the Frobenius norm loss and yields a more stable approximate recovery guarantee with respect to the sampling distributions.

The focus of matrix completion has so far been on the recovery of a low-rank matrix based on independently sampled entries. Motivated by applications in genomic data integration, we introduce in this paper a new framework of matrix completion called structured matrix completion (SMC), where a subset of the rows and a subset of the columns of an approximately low-rank matrix are observed and the goal is to reconstruct the whole matrix based on the observed rows and columns. We first discuss the genomic data integration problem before introducing the SMC model.

1.1 Genomic Data Integration

When analyzing genome-wide studies (GWS) of association, expression profiling or methylation, ensuring adequate power of the analysis is one of the most crucial goals due to the high dimensionality of the genomic markers under consideration. Because of cost constraints, GWS typically have small to moderate sample sizes and hence limited power. One approach to increase the power is to integrate information from multiple GWS of the same phenotype. However, some practical complications may hamper the feasibility of such integrative analysis. Different GWS often involve different platforms with distinct genomic coverage. For example, whole genome next generation sequencing (NGS) studies would provide mutation information on all loci while older technologies for genome-wide association studies (GWAS) would only provide information on a small subset of loci. In some settings, certain studies may provide a wider range of genomic data than others. For example, one study may provide extensive genomic measurements including gene expression, miRNA and DNA methylation while other studies may only measure gene expression.

To perform integrative analysis of studies with different extent of genomic measurements, the naive complete observation only approach may suffer from low power. For the GWAS setting with a small fraction of loci missing, many imputation methods have been proposed in recent years to improve the power of the studies. Examples of useful methods include haplotype reconstruction, k-nearest neighbor, regression and singular value decomposition methods (Scheet and Stephens, 2006; Li and Abecasis, 2006; Browning and Browning, 2009; Troyanskaya et al., 2001; Kim et al., 2005; Wang et al., 2006). Many of the haplotype phasing methods are considered to be highly effective in recovering missing genotype information (Yu and Schaid, 2007). These methods, while useful, are often computationally intensive. In addition, when one study has a much denser coverage than the other, the fraction of missingness could be high and an exceedingly large number of observation would need to be imputed. It is unclear whether it is statistically or computationally feasible to extend these methods to such settings. Moreover, haplotype based methods cannot be extended to incorporate other types of genomic data such as gene expression and miRNA data.

When integrating multiple studies with different extent of genomic measurements, the observed data can be viewed as complete rows and columns of a large matrix A and the missing components can be arranged as a submatrix of A. As such, the missingness in A is structured by design. In this paper, we propose a novel SMC method for imputing the missing submatrix of A. As shown in Section 5, by imputing the missing miRNA measurements and constructing prediction rules based on the imputed data, it is possible to significantly improve the prediction performance.

1.2 Structured Matrix Completion Model

Motivated by the applications mentioned above, this paper considers SMC where a subset of rows and columns are observed. Specifically, we observe m₁ < p₁ rows and m₂ < p₂ columns of a matrix A ∈ ℝ^p₁×p₂ and the goal is to recover the whole matrix. Since the singular values are invariant under row/column permutations, it can be assumed without loss of generality that we observe the first m₁ rows and m₂ columns of A which can be written in a block form:

$$A=\left[\begin{array}{cc}{m}_2& p_2-m_2\\ A_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]\begin{array}{c}{m}_1\\ p_1-m_1\end{array}$$ (1)

where A₁₁, A₁₂, and A₂₁ are observed and the goal is to recover the missing block A₂₂. See Figure 1(a) in Section 2 for a graphical display of the data. Clearly there is no way to recover A₂₂ if A is an arbitrary matrix. However, in many applications such as genomic data integration discussed earlier, A is approximately low-rank, which makes it possible to recover A₂₂ with accuracy. In this paper, we introduce a method based on the singular value decomposition (SVD) for the recovery of A₂₂ when A is approximately low-rank.

Figure 1.

Illustrative example with A ∈ ℝ^30×30, m₁ = m₂ = 10. (A darker block corresponds to larger magnitude.)

It is important to note that the observations here are much more “structured” comparing to the previous settings of matrix completion. As the observed entries are in full rows or full columns, the existing methods based on NNM are not suitable. As mentioned earlier, constrained NNM methods have been widely used in matrix completion problems based on independently observed entries. However, for the problem considered in the present paper, these methods do not utilize the structure of the observations and do not guarantee precise recovery even for exactly low-rank matrix A (See Remark 1 in Section 2). Numerical results in Section 4 show that NNM methods do not perform well in SMC.

In this paper we propose a new SMC method that can be easily implemented by a fast algorithm which only involves basic matrix operations and the SVD. The main idea of our recovery procedure is based on the Schur Complement. In the ideal case when A is exactly low rank, the Schur complement of the missing block, $A_{22}-A_{21}{A}_{11}^{†}{A}_{12}$, is zero and thus $A_{21}{A}_{11}^{†}{A}_{12}$ can be used to recover A₂₂ exactly. When A is approximately low rank, $A_{21}{A}_{11}^{†}{A}_{12}$ cannot be used directly to estimate A₂₂. For this case, we transform the observed blocks using SVD; remove some unimportant rows and columns based on thresholding rules; and subsequently apply a similar procedure to recover A₂₂.

Both its theoretical and numerical properties are studied. It is shown that the estimator recovers low-rank matrices accurately and is robust against small perturbations. A lower bound result shows that the estimator is rate optimal for a class of approximately low-rank matrices. Although it is required for the theoretical analysis that there is a significant gap between the singular values of the true low-rank matrix and those of the perturbation, simulation results indicate that this gap is not really necessary in practice and the estimator recovers A accurately whenever the singular values of A decay sufficiently fast.

1.3 Organization of the Paper

The rest of the paper is organized as follows. In Section 2, we introduce in detail the proposed SMC methods when A is exactly or approximately low-rank. The theoretical properties of the estimators are analyzed in Section 3. Both upper and lower bounds for the recovery accuracy under the Schatten-q norm loss are established. Simulation results are shown in Section 4 to investigate the numerical performance of the proposed methods. A real data application to genomic data integration is given in Section 5. Section 6 discusses a few practical issues related to real data applications. For reasons of space, the proofs of the main results and additional simulation results are given in the supplement (Cai et al., 2014). Some key technical tools used in the proofs of the main theorems are also developed and proved in the supplement.

2 Structured Matrix Completion: Methodology

In this section, we propose procedures to recover the submatrix A₂₂ based on the observed blocks A₁₁, A₁₂, and A₂₁. We begin with basic notation and definitions that will be used in the rest of the paper.

For a matrix U, we use U_[Ω1,Ω2] to represent its sub-matrix with row indices Ω₁ and column indices Ω₂. We also use the Matlab syntax to represent index sets. Specifically for integers a ≤ b, “a: b” represents {a, a + 1, ···, b}; and “:” alone represents the entire index set. Therefore, U_[:,1:r] stands for the first r columns of U while U_{[(m1+1):p1,:]} stands for the {m₁ + 1, …, p₁}^th rows of U. For the matrix A given in (1), we use the notation A_•1 and A_1• to denote ${[A_{11}^{\top },A_{21}^{\top }]}^{\top }$ and [A₁₁, A₁₂], respectively. For a matrix B ∈ ℝ^m×n, let $B=U\mathrm{\sum }{V}^{\top }={\sum }_i{\sigma }_i(B)u_i{v}_i^{\top }$ be the SVD, where Σ = diag{σ₁(B), σ₂(B), …} with σ₁(B) ≥ σ₂(B) ≥ ··· ≥ 0 being the singular values of B in decreasing order. The smallest singular value σ_min(m,n), which will be denoted by σ_min(B), plays an important role in our analysis. We also define $B_{max(r)}={\sum }_{i=1}^r{\sigma }_i(B)u_i{v}_i^{\top }$ and $B_{-max(r)}=B-B_{max(r)}={\sum }_{i\ge r+1}{\sigma }_i(B)u_i{v}_i^{\top }$. For 1 ≤ q ≤ ∞, the Schatten-q norm ||B||_q is defined to be the vector q-norm of the singular values of B, i.e. ${\Vert B\Vert }_q={({\sum }_i{\sigma }_i^q(B))}^{1/q}$. Three special cases are of particular interest: when q = 1, ||B||₁ = Σ_i σ_i (B) is the nuclear (or trace) norm of B and will be denoted as ||B||_*; when q = 2, ${\Vert B\Vert }_2=\sqrt{{\sum }_{i,j}{B}_{ij}^2}$ is the Frobenius norm of B and will be denoted as ||B||_F; when q = ∞, ||B||_∞ = σ₁(B) is the spectral norm of B that we simply denote as ||B||. For any matrix U ∈ ℝ^p×n, we use P_U ≡ U (U^⊤U)^† U^⊤ ∈ ℝ^p×p to denote the projection operator onto the column space of U. Throughout, we assume that A is approximately rank r in that for some integer 0 < r ≤ min(m₁, m₂), there is a significant gap between σ_r(A) and σ_r+1(A) and the tail ${\Vert A_{-max(r)}\Vert }_q={({\sum }_{k\ge r+1}{\sigma }_k^q(A))}^{1/q}$ is small. The gap assumption enables us to provide a theoretical upper bound on the accuracy of the estimator, while it is not necessary in practice (see Section 4 for more details).

2.1 Exact Low-rank Matrix Recovery

We begin with the relatively easy case where A is exactly of rank r. In this case, a simple analysis indicates that A can be perfectly recovered as shown in the following proposition.

Proposition 1

Suppose A is of rank r, the SVD of A₁₁ is A₁₁ = UΣV^⊤, where U ∈ ℝ^p₁×r, Σ ∈ ℝ^r×r, and V ∈ ℝ^p₂×r. If

$$\text{rank}([\begin{array}{cc}{A}_{11}& A_{12}\end{array}])=\text{rank}\left(\left[\begin{array}{c}{A}_{11}\\ A_{21}\end{array}\right]\right)=\text{rank}(A)=r,$$

then rank(A₁₁) = r and A₂₂ is exactly given by

$$A_{22}=A_{21}{(A_{11})}^{†}{A}_{12}=A_{21}V{(\mathrm{\sum })}^{-1}{U}^{\top }{A}_{12}.$$ (2)

Remark 1

Under the same conditions as Proposition 1, the NNM

$${\widehat{A}}_{22}=\underset{B}{argmin}{\Vert \left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& B\end{array}\right]\Vert }_{\ast }$$ (3)

fails to guarantee the exact recovery of A₂₂. Consider the case where A is a p₁ × p₂ matrix with all entries being 1. Suppose we observe arbitrary m₁ rows and m₂ columns, the NNM would yield Â₂₂ ∈ ℝ ^{(p₁−m₁)×(p₂−m₂)} with all entries being $\left(1\wedge \sqrt{\frac{m_1{m}_2}{(p_1-m_1)(p_2-m_2)}}\right)$ (See Lemma 4 in the Supplement). Hence when m₁m₂ < (p₁ − m₁)(p₂ − m₂), i.e., when the size of the observed blocks are much smaller than that of A, the NNM fails to recover exactly the missing block A₂₂. See also the numerical comparison in Section 4. The NNM (3) also fails to recover A₂₂ with high probability in a random matrix setting where $A=B_1{B}_2^T$ with B₁ ∈ ℝ^p₁×r and B₂ ∈ ℝ^p₂×r being i.i.d. standard Gaussian matrices. See Lemma 3 in the Supplement for further details. In addition to (3), other variations of NNM have been proposed in the literature, including penalized NNM (Toh and Yun, 2010; Mazumder et al., 2010),

$${\widehat{A}}^{PN}=\underset{Z}{argmin}\left\{\frac{1}{2}\sum _{(i_k,j_k)\in \mathrm{\Omega }}{(Z_{i_k,j_k}-A_{i_k,j_k})}^2+t{\Vert Z\Vert }_{\ast }\right\};$$ (4)

and constrained NNM with relaxation (Cai et al., 2010),

$${\widehat{A}}^{CN}=\underset{Z}{argmin}\{{\Vert Z\Vert }_{\ast }:\mid Z_{i_k,j_k}-A_{i_k,j_k}\mid \le t\text{for}(i_k,j_k)\in \mathrm{\Omega }\},$$ (5)

where Ω = {(i_k, j_k): A_ik,jk observed, 1 ≤ i_k ≤ p₁, 1 ≤ j_k ≤ p₂} and t is the tunning parameter. However, these NNM methods may not be suitable for SMC especially when only a small number of rows and columns are observed. In particular, when m₁ ≪ p₁, m₂ ≪ p₂, A is well spread in each block A₁₁, A₁₂, A₂₁, A₂₂, we have ||[A₁₁ A₁₂]||_* ≪ ||A||_*, [A₁₂]_* ≪ ||A||_*. Thus,

$${\Vert \left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& 0\end{array}\right]\Vert }_{\ast }\le {\Vert \left[\begin{array}{c}{A}_{11}\\ A_{21}\end{array}\right]\Vert }_{\ast }+{\Vert \left[A_{12}\right]\Vert }_{\ast }\ll {\Vert \left[\begin{array}{cc}{A}_{11}& A_{12}\\ A_{21}& A_{22}\end{array}\right]\Vert }_{\ast }.$$

In the other words, imputing A₂₂ with all zero yields a much smaller nuclear norm than imputing with the true A₂₂ and hence NNM methods would generally fail to recover A₂₂ under such settings.

Proposition 1 shows that, when A is exactly low-rank, A₂₂ can be recovered precisely by A₂₁(A₁₁)^† A₁₂. Unfortunately, this result heavily relies on the exactly low-rank assumption that cannot be directly used for approximately low-rank matrices. In fact, even with a small perturbation to A, the inverse of A₁₁ makes the formula A₂₁(A₁₁)^† A₁₂ unstable, which may lead to the failure of recovery. In practice, A is often not exactly low rank but approximately low rank. Thus for the rest of the paper, we focus on the latter setting.

2.2 Approximate Low-rank Matrix Recovery

Let A = UΣV^⊤ be the SVD of an approximately low rank matrix A and partition U ∈ ℝ^p₁×p₁, V ∈ ℝ^p₂×p₂ and Σ ∈ ℝ^p₁×p₂ into blocks as

$$U=\left[\begin{array}{cc}r& p_1-r\\ U_{11}& U_{12}\\ U_{21}& U_{22}\end{array}\right]\begin{array}{c}{m}_1\\ p_1-m_1\end{array},V=\left[\begin{array}{cc}r& p_2-r\\ V_{11}& V_{12}\\ V_{21}& V_{22}\end{array}\right]\begin{array}{c}{m}_2\\ p_2-m_2\end{array},\mathrm{\sum }=\left[\begin{array}{cc}r& p_2-r\\ {\mathrm{\sum }}_1& 0\\ 0& {\mathrm{\sum }}_2\end{array}\right]\begin{array}{c}r\\ p_1-r\end{array}$$ (6)

Then A can be decomposed as A = A_max(r) + A_−max(r) where A_max(r) is of rank r with the largest r singular values of A and A_−max(r) is general but with small singular values. Then

$$A_{max(r)}=U_{•1}{\mathrm{\sum }}_1{V}_{•1}^{\top }=\left[\begin{array}{cc}{m}_2& p_2-m_2\\ U_{11}{\mathrm{\sum }}_1{V}_{11}^{\top }& U_{11}{\mathrm{\sum }}_1{V}_{21}^{\top }\\ U_{21}{\mathrm{\sum }}_1{V}_{11}^{\top }& U_{21}{\mathrm{\sum }}_1{V}_{21}^{\top }\end{array}\right]\begin{array}{c}{m}_1\\ p_1-m_1\end{array},\text{and}{A}_{-max(r)}=U_{•2}{\mathrm{\sum }}_2{V}_{•2}^{\top }.$$ (7)

Here and in the sequel, we use the notation U_•k and U_k• to denote ${[U_{1k}^{\top },U_{2k}^{\top }]}^{\top }$ and [U_k1, U_k2], respectively. Thus, A_max(r) can be viewed as a rank-r approximation to A and obviously

$$U_{21}{\mathrm{\sum }}_1{V}_{21}^{\top }=\{U_{21}{\mathrm{\sum }}_1{V}_{11}^{\top }\}{\{U_{11}{\mathrm{\sum }}_1{V}_{11}^{\top }\}}^{-1}\{U_{11}{\mathrm{\sum }}_1{V}_{21}^{\top }\}.$$

We will use the observed A₁₁, A₁₂ and A₂₁ to obtain estimates of U_•1, V_•1 and Σ₁ and subsequently recover A₂₂ using an estimated $U_{21}{\mathrm{\sum }}_1{V}_{21}^{\top }$.

When r is known, i.e., we know where the gap is located in the singular values of A, a simple procedure can be implemented to estimate A₂₂ as described in Algorithm 1 below by estimating U_•1 and V_•1 using the principal components of A_•1 and A_1•.

Algorithm 1.

Algorithm for Structured Matrix Completion with a given r

Input A11 ∈ ℝm1×m2, A12 ∈ ℝ (p1−m1×m2, A21 ∈ ℝm1×(p2−m2). Calculate the SVD of A•1 and A1• to obtain A•1 = U(1)Σ(1)V(1)⊤, A1• = U(2)Σ(2)V(2)⊤. Suppose M, N are orthonormal basis of U11, V11. We estimate the column space of U11 and V11 by $\widehat{M}=U_{[:,1:r]}^{(2)},\widehat{N}=V_{[:,1:r]}^{(1)}$. Finally we estimate A22 as $${\widehat{A}}_{22}=A_{21}\widehat{N}{({\widehat{M}}^{\top }{A}_{11}\widehat{N})}^{-1}{\widehat{M}}^{\top }{A}_{12}.$$ (8)

However, Algorithm 1 has several major limitations. First, it relies on a given r which is typically unknown in practice. Second, the algorithm need to calculate the matrix divisions, which may cause serious precision issues when the matrix is near-singular or the rank r is mis-specified. To overcome these difficulties, we propose another Algorithm which essentially first estimates r with r̂ and then apply Algorithm 1 to recover A₂₂. Before introducing the algorithm of recovery without knowing r, it is helpful to illustrate the idea with heat maps in Figures 1 and 2.

Figure 2.

Searching for the appropriate position to truncate from r̂ = 10 to 1.

Our procedure has three steps.

1. First, we move the significant factors of A_•1 and A_1• to the front by rotating the columns of A_•1 and the rows of A_1• based on the SVD,

   $$A_{•1}=U^{(1)}{\mathrm{\sum }}^{(1)}{V}^{(1)\top },A_{1•}=U^{(2)}{\mathrm{\sum }}^{(2)}{V}^{(2)\top }.$$
   After the transformation, we have Z₁₁, Z₁₂, Z₂₁,

   $$Z_{11}=U^{(2)\top }{A}_{11}{V}^{(1)},Z_{12}=U^{(2)\top }{A}_{12},Z_{21}=A_{21}{V}^{(1)},Z_{22}=A_{22}.$$

   Clearly A and Z have the same singular values since the transformation is orthogonal. As shown in Figure 1(b), the amplitudes of the columns of $Z_{•1}={[Z_{11}^{\top },Z_{21}^{\top }]}^{\top }$ and the rows of Z_1• = [Z₁₁, Z₁₂] are decaying.

2. When A is exactly of rank r, the {r + 1, ···, m₁}^th rows and {r + 1, ···, m₂}^th columns of Z are zero. Due to the small perturbation term A_−max(r), the back columns of Z_•1 and rows of Z_1• are small but non-zero. In order to recover A_max(r), the best rank r approximation to A, a natural idea is to first delete these back rows of Z_1• and columns of Z_•1, i.e. the {r + 1, ···, m₁}^th rows and {r + 1, ···, m₂}^th columns of Z.

   However, since r is unknown, it is unclear how many back rows and columns should be removed. It will be helpful to have an estimate for r, r̂, and then use Z_21,[:,1:r̂], Z_{11,[1:r̂,1:r̂]} and Z_12[1:r̂,:] to recover A₂₂. It will be shown that a good choice of r̂ would satisfy that Z_{11,[1:r̂,1:r̂]} is non-singular and $\Vert Z_{21,[1:\widehat{r},1:\widehat{r}]}{Z}_{11,[1:\widehat{r},1:\widehat{r}]}^{-1}\Vert \le T_R$, where TR is some constant to be specified later. Our final estimator for r would be the largest r̂ that satisfies this condition, which can be identified recursively from min(m₁, m₂) to 1 (See Figure 2).

3. Finally, similar to (2),A₂₂ can be estimated by

   $${\widehat{A}}_{22}=Z_{21,[:,1:\widehat{r}]}{Z}_{11,[1:\widehat{r},1:\widehat{r}]}^{-1}{Z}_{12,[1:\widehat{r},;]},$$ (9)

   The method we propose can be summarized as the following algorithm.

   Algorithm 2.

   Algorithm of Structured Matrix Completion with unknown r

   Input: A11 ∈ ℝm1×m2, $A_{12}^{m_1\times (p_2-m_2)},A_{21}^{(p_1-m_1)\times m_2}$. Thresholding level: TR, (or TC). Calculate the SVD A•1 = U(1)Σ(1)V(1)⊤, A1• = U(2)Σ(2)V(2)⊤. Calculate Z11 ∈ ℝm1×m2, Z12 ∈ ℝm1×(p2−m2), Z21 ∈ ℝ(p1−m1)×m2 $Z_{11}=U^{(2)\top }{A}_{11}{V}^{(1)},Z_{12}=U^{(2)\top }{A}_{12},Z_{21}=A_{21}{V}^{(1)}.$ for s = min(m1, m2): −1: 1 do (Use iteration to find r̂) Calculate DR,s ∈ ℝ(p1−m1)×s (or DC,s ∈ ℝs×(p2−m2)) by solving linear equation system, $D_{R,s}=Z_{21,[:,1:s]}{Z}_{11,[1:s,1:s]}^{-1}(\text{or}{D}_{C,s}=Z_{11,[1:s,1:s]}^{-1}{Z}_{12,[1:s,:]})$ if Z11,[1:s,1:s] is not singular and ||DR,s|| ≤ TR (or ||DC,s|| ≤ TC) then r̂ = s; break from the loop; end if end for if (r̂ is not valued) then r̂ = 0. end if Finally we calculate the estimate as $${\widehat{A}}_{22}=Z_{21,[:,1:\widehat{r}]}{Z}_{11,[1:\widehat{r},1:\widehat{r}]}^{-1}{Z}_{12,[1:\widehat{r},:]}$$

It can also be seen from Algorithm 2 that the estimator r̂ is constructed based on either the row thresholding rule ||D_R,s|| ≤ T_R or the column thresholding rule ||D_C,s|| ≤ T_C. Discussions on the choice between D_R,s and D_C,s are given in the next section. Let us focus for now on the row thresholding based on $D_{R,s}=Z_{21,[:,1:s]}{Z}_{11,[1:s,1:s]}^{-1}$. It is important to note that Z_21[:,1:r] and Z_11,[1:r,1:r] approximate U₂₁Σ₁ and Σ₁, respectively. The idea behind the proposed r̂ is that when s > r, Z_21[:,1:s] and Z_11,[1:s,1:s] are nearly singular and hence D_R,s may either be deemed singular or with unbounded norm. When s = r, Z_11,[1:s,1:s] is non-singular with ||D_R,s|| bounded by some constant, as we show in Theorem 2. Thus, we estimate r̂ as the largest r such that Z_11,[1:s,1:s] is non-singular with ||D_R,s|| < T_R.

3 Theoretical Analysis

In this section, we investigate the theoretical properties of the algorithms introduced in Section 2. Upper bounds for the estimation errors of Algorithms 1 and 2 are presented in Theorems 1 and 2, respectively, and the lower-bound results are given in Theorem 3. These bounds together establish the optimal rate of recovery over certain classes of approximately low-rank matrices. The choices of tuning parameters T_R and T_C are discussed in Corollaries 1 and 2.

Theorem 1

Suppose  is given by the procedure of Algorithm 1. Assume

$${\sigma }_{r+1}(A)\le \frac{1}{2}{\sigma }_r(A)·{\sigma }_{min}(U_{11})·{\sigma }_{min}(V_{11}),$$ (10)

Then for any 1 ≤ q ≤ ∞,

$${\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 3{\Vert A_{-max(r)}\Vert }_q\left(1+\frac{1}{{\sigma }_{min}(U_{11})}\right)\left(1+\frac{1}{{\sigma }_{min}(V_{11})}\right)$$ (11)

Remark 2

It is helpful to explain intuitively why Condition (10) is needed. When A is approximately low-rank, the dominant low-rank component of A, A_max(r), serves as a good approximation to A, while the residual A_−max(r) is “small”. The goal is to recover A_max(r) well. Among the three observed blocks, A₁₁ is the most important and it is necessary to have A_max(r) dominating A_−max(r) in A₁₁. Note that A₁₁ = A_{max(r),[1:m1,1:m2]} + A_{−max(r),[1:m1,1:m2]},

$$\begin{array}{c}{\sigma }_r(A_{max(r),[1:m_1,1:m_2]})={\sigma }_r(U_{11}{\mathrm{\sum }}_1{V}_{11}^{\top })\ge {\sigma }_{min}(U_{11}){\sigma }_r(A){\sigma }_{min}(V_{11}),\\ \Vert A_{-max(r),[1:m_1,1:m_2]}\Vert =\Vert U_{12}{\mathrm{\sum }}_2{V}_{12}^{\top }\Vert \le {\sigma }_{r+1}(A).\end{array}$$

We thus require Condition (10) in Theorem 1 for the theoretical analysis.

Theorem 1 gives an upper bound for the estimation accuracy of Algorithm 1 under the assumption that there is a significant gap between σ_r(A) and σ_r+1(A) for some known r. It is noteworthy that there are possibly multiple values of r that satisfy Condition (10). In such a case, the bound (11) applies to all such r and the largest r yields the strongest result.

We now turn to Algorithm 2, where the knowledge of r is not assumed. Theorem 2 below shows that for properly chosen T_R or T_C, Algorithm 2 can lead to accurate recovery of A₂₂.

Theorem 2

Assume that there exists r ∈ [1, min(m₁, m₂)] such that

$${\sigma }_{r+1}(A)\le \frac{1}{4}{\sigma }_r(A)·{\sigma }_{min}(U_{11}){\sigma }_{min}(V_{11}).$$ (12)

Let T_R and T_C be two constants satisfying

$$T_R\ge \frac{1.36}{{\sigma }_{min}(U_{11})}+0.35\mathit{and}{T}_C\ge \frac{1.36}{{\sigma }_{min}(V_{11})}+\mathrm{0.35.}$$

Then for 1 ≤ q ≤ ∞, Â₂₂ given by Algorithm 2 satisfies

$$\begin{array}{r}{\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 6.5T_R\left(\frac{1}{{\sigma }_{min}(V_{11})}+1\right){\Vert A_{-max(r)}\Vert }_q\\ or{\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 6.5T_C\left(\frac{1}{{\sigma }_{min}(U_{11})}+1\right){\Vert A_{-max(r)}\Vert }_q\end{array}$$ (13)

when r̂ is estimated based on the thresholding rule ||D_R,s|| ≤ T_R or ||D_C,s|| ≤ T_C, respectively.

Besides σ_r(A) and σ_r+1(A), Theorems 1 and 2 involve σ_min(U₁₁) and σ_min(V₁₁), two important quantities that reflect how much the low-rank matrix $A_{max(r)}=U_{•1}{\mathrm{\sum }}_1{V}_{•1}^{\top }$ is concentrated on the first m₁ rows and m₂ columns. We should note that σ_min(U₁₁) and σ_min(V₁₁) depend on the singular vectors of A and σ_r(A) and σ_r+1(A) are the singular values of A. The lower bound in Theorem 3 below indicates that σ_min(U₁₁), σ_min(V₁₁), and the singular values of A together quantify the difficulty of the problem: recovery of A₂₂ gets harder as σ_min(U₁₁) and σ_min(V₁₁) become smaller or the {r + 1, ···, min(p₁, p₂)}^th singular values become larger. Define the class of approximately rank-r matrices (M₁, M₂) by

$${\mathcal{F}}_r(M_1,M_2)=\left\{A\in {ℝ}^{p_1\times p_2}:\begin{array}{l}{\sigma }_{min}(U_{11})\ge M_1,{\sigma }_{min}(V_{11})\ge M_2,\hfill \\ {\sigma }_{r+1}(A)\le \frac{1}{2}{\sigma }_r(A){\sigma }_{min}(U_{11}){\sigma }_{min}(V_{11})\hfill \end{array}\right\}.$$ (14)

Theorem 3 (Lower Bound)

Suppose r ≤ min(m₁, m₂, p₁ − m₁, p₂ − m₂) and 0 < M₁, M₂ < 1, then for all 1 ≤ q ≤ ∞,

$$\underset{{\widehat{A}}_{22}}{inf}\underset{A\in {\mathcal{F}}_r(M_1,M_2)}{sup}\frac{{\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q}{{\Vert A_{-max(r)}\Vert }_q}\ge \frac{1}{4}\left(\frac{1}{M_1}+1\right)\left(\frac{1}{M_2}+1\right).$$ (15)

Remark 3

Theorems 1, 2 and 3 together immediately yield the optimal rate of recovery over the class (M₁M₂),

$$\underset{{\widehat{A}}_{22}}{inf}\underset{A\in {\mathcal{F}}_r(M_1,M_2)}{sup}\frac{{\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q}{{\Vert A_{-max(r)}\Vert }_q}\asymp \left(\frac{1}{M_1}+1\right)\left(\frac{1}{M_2}+1\right)\text{for}0\le M_1,M_2<1,1\le q\le \infty .$$ (16)

Since U₁₁ and V₁₁ are determined by the SVD of A and σ_min(U₁₁) and σ_min(V₁₁) are unknown based only on A₁₁, A₁₂, and A₂₁, it is thus not straightforward to choose the tuning parameters T_R and T_C in a principled way. Theorem 2 also does not provide information on the choice between row and column thresholding. Such a choice generally depends on the problem setting. We consider below two settings where either the row/columns of A are randomly sampled or A is itself a random low-rank matrix. In such settings, when A is approximately rank r and at least O(r log r) number of rows and columns are observed, Algorithm 2 gives accurate recovery of A with fully specified tuning parameter. We first consider in Corollary 1 a fixed matrix A with the observed m₁ rows and m₂ columns selected uniformly randomly.

Corollary 1 (Random Rows/Columns)

Let A = UΣV^⊤ be the SVD of A ∈ ℝ^p₁×p₂. Set

$$W_r^{(1)}=\frac{p_1}{r}\underset{1\le i\le p_1}{max}\sum _{j=1}^r{U}_{ij}^2\text{and}{W}_r^{(2)}=\frac{p_2}{r}\underset{1\le i\le p_2}{max}\sum _{j=1}^r{V}_{ij}^2.$$ (17)

Let Ω₁ ⊂ {1, ···, p₁} and Ω₂ ⊂ {1, ···, p₂} be respectively the index set of the observed m₁ rows and m₂ columns. Then A can be decomposed as

$$A_{11}=A_{[{\mathrm{\Omega }}_1,{\mathrm{\Omega }}_2]},A_{21}=A_{[{\mathrm{\Omega }}_1^c,{\mathrm{\Omega }}_2]},A_{12}=A_{[{\mathrm{\Omega }}_1,{\mathrm{\Omega }}_2^c]},A_{22}=A_{[{\mathrm{\Omega }}_1^c,{\mathrm{\Omega }}_2^c]}.$$ (18)

1. Let Ω₁ and Ω₂ be independently and uniformly selected from {1, ···, p₁} and {1, ···, p₂} with or without replacement, respectively. Suppose there exists r ≤ min(m₁, m₂) such that

   $${\sigma }_{r+1}(A)\le \frac{1}{6}{\sigma }_r(A)\sqrt{\frac{m_1{m}_2}{p_1{p}_2}}.$$
   and the number of rows and number of columns we observed satisfy

   $$m_1\ge 12.5{rW}_r^{(1)}(log(r)+c),m_2\ge 12.5{rW}_r^{(2)}(log(r)+c),\mathit{for}\mathit{some}\mathit{constant}c>1.$$
   Algorithm 2 with either column thresholding with the break condition ||D_R,s|| ≤ T_R where $T_R=2\sqrt{\frac{p_1}{m_1}}$ or row thresholding with the break condition ||D_C,s|| ≤ T_C where $T_C=2\sqrt{\frac{p_2}{m_2}}$ satisfies, for all 1 ≤ q ≤ ∞,

   $${\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 29{\Vert A_{-max(r)}\Vert }_q\sqrt{\frac{p_1{p}_2}{m_1{m}_2}}\mathit{with}\mathit{probability}\ge 1-4exp(-c).$$
2. If Ω₁ is uniformly randomly selected from {1, ···, p₁} with or without replacement (Ω₂ is not necessarily random), and there exists r ≤ m₂ such that

   $${\sigma }_{r+1}(A)\le \frac{1}{5}{\sigma }_r(A){\sigma }_{min}(V_{11})\sqrt{\frac{m_1}{p_1}}$$
   and the number of observed rows satisfies

   $$m_1\ge 12.5{rW}_r^{(1)}(log(r)+c)\mathit{for}\mathit{some}\mathit{constant}c>1,$$ (19)
   then Algorithm 2 with the break condition ||D_R,s|| ≤ T_R where $T_R\ge 2\sqrt{\frac{p_1}{m_1}}$ satisfies, for all 1 ≤ q ≤ ∞,

   $${\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 6.5{\Vert A_{-max(r)}\Vert }_q{T}_R\left(\frac{1}{{\sigma }_{min}(V_{11})}+1\right)\mathit{with}\mathit{probability}\ge 1-2exp(-c).$$
3. Similarly, if Ω₂ is uniformly randomly selected from {1, ···, p₂} with or without replacement (Ω₁ is not necessarily random) and there exists r ≤ m₂ such that

   $${\sigma }_{r+1}(A)\le \frac{1}{5}{\sigma }_r(A){\sigma }_{min}(U_{11})\sqrt{\frac{m_2}{p_2}},$$
   and the number of observed columns satisfies

   $$m_2\ge 12.5{rW}_r^{(2)}(log(r)+c)\mathit{for}\mathit{some}\mathit{constant}c>1,$$ (20)
   then Algorithm 2 with the break condition ||D_C,s|| ≤ T_C where $T_C\ge 2\sqrt{\frac{p_2}{m_2}}$ satisfies, for all 1 ≤ q ≤ ∞,

   $${\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 6.5{\Vert A_{-max(r)}\Vert }_q{T}_C\left(\frac{1}{{\sigma }_{min}(U_{11})}+1\right)\mathit{with}\mathit{probability}\ge 1-2exp(-c).$$

Remark 4

The quantities $W_r^{(1)}$ and $W_r^{(2)}$ in Corollary 1 measure the variation of amplitude of each row or each column of A_max(r). When $W_r^{(1)}$ and $W_r^{(2)}$ become larger, a small number of rows and columns in A_max(r) would have larger amplitude than others, while these rows and columns would be missed with large probability in the sampling of Ω, which means the problem would become harder. Hence, more observations for the matrix with larger $W_r^{(1)}$ and $W_r^{(2)}$ are needed as shown in (19).

We now consider the case where A is a random matrix.

Corollary 2 (Random Matrix)

Suppose A ∈ ℝ^p₁×p₂ is a random matrix generated by A = UΣV^⊤, where the singular values Σ and singular space V are fixed, and U has orthonormal columns that are randomly sampled based on the Haar measure. Suppose we observe the first m₁ rows and first m₂ columns of A. Assume there exists $r<\frac{1}{2}min(m_1,m_2)$ such that

$${\sigma }_{r+1}(A)\le \frac{1}{5}{\sigma }_r(A){\sigma }_{min}(V_{11})\sqrt{\frac{m_1}{p_1}}.$$

Then there exist uniform constants c, δ > 0 such that if m₁ ≥ cr, Â₂₂ is given by Algorithm 2 with the break condition ||D_R,s|| ≤ T_R, where $T_R\ge 2\sqrt{\frac{p_1}{m_1}}$, we have for all 1 ≤ q ≤ ∞,

$${\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 6.5{\Vert A_{-max(r)}\Vert }_q{T}_R\left(\frac{1}{{\sigma }_{min}(V_{11})}+1\right)\mathit{with}\mathit{probability}at\mathit{least}1-e^{-\delta m_1}.$$

Parallel results hold for the case when U is fixed and V has orthonormal columns that are randomly sampled based on the Haar measure, and we observe the first m₁ rows and first m₂ columns of A. Assume there exists $r<\frac{1}{2}min(m_1,m_2)$ such that

$${\sigma }_{r+1}(A)\le \frac{1}{5}{\sigma }_r(A){\sigma }_{min}(U_{11})\sqrt{\frac{m_2}{p_2}}.$$

Then there exist unifrom constants c, δ > 0 such that if m₂ ≥ cr, Â₂₂ is given by Algorithm 2 with column thresholding with the break condition ||D_C,s|| ≤ T_C, where $T_C\ge 2\sqrt{\frac{p_2}{m_2}}$, we have for all 1 ≤ q ≤ ∞,

$${\Vert {\widehat{A}}_{22}-A_{22}\Vert }_q\le 6.5{\Vert A_{-max(r)}\Vert }_q{T}_C\left(\frac{1}{{\sigma }_{min}(U_{11})}+1\right)\mathit{with}\mathit{probability}at\mathit{least}1-e^{-\delta m_2}.$$

4 Simulation

In this section, we show results from extensive simulation studies that examine the numerical performance of Algorithm 2 on randomly generated matrices for various values of p₁, p₂, m₁ and m₂. We first consider settings where a gap between some adjacent singular values exists, as required by our theoretical analysis. Then we investigate settings where the singular values decay smoothly with no significant gap between adjacent singular values. The results show that the proposed procedure performs well even when there is no significant gap, as long as the singular values decay at a reasonable rate.

We also examine how sensitive the proposed estimators are to the choice of the threshold and the choice between row and column thresholding. In addition, we compare the performance of the SMC method with that of the NNM method. Finally, we consider a setting similar to the real data application discussed in the next section. Results shown below are based on 200–500 replications for each configuration. Additional simulation results on the effect of m₁, m₂ and ratio p₁/m₁ are provided in the supplement. Throughout, we generate the random matrix A from A = UΣV, where the singular values of the diagonal matrix Σ are chosen accordingly for different settings. The singular spaces U and V are drawn randomly from the Haar measure. Specifically, we generate i.i.d. standard Gaussian matrix Ũ ∈ ℝ^{p₁×min(p₁,p₂)} and Ṽ ∈ ℝ^{p₂×min(p₁,p₂)}, then apply the QR decomposition to Ũ and Ṽ and assign U and V with the Q part of the result.

We first consider the performance of Algorithm 2 when a significant gap between the r^th and (r + 1)^th singular values of A. We fixed p₁ = p₂ = 1000, m₁ = m₂ = 50 and choose the singular values as

$$\{\underset{r}{\underbrace{1,\cdots ,1}},g^{-1}{1}^{-1},g^{-1}{2}^{-1},\cdots \},g=1,2,\cdots ,10,r=4,12\text{and}20.$$ (21)

Here r is the rank of the major low-rank part A_max(r), $g=\frac{{\sigma }_r(A)}{{\sigma }_{r+1}(A)}$ is the gap ratio between the r^th and (r + 1)^th singular values of A. The average loss of Â₂₂ from Algorithm 2 with the row thresholding and $T_R=2\sqrt{p_1/m_1}$ under both the spectral norm and Frobenius norm losses are given in Figure 3. The results suggest that our algorithm performs better when r gets smaller and gap ratio g = σ_r(A)/σ_r+1(A) gets larger. Moreover, even when g = 1, namely there is no significant gap between any adjacent singular values, our algorithm still works well for small r. As will be seen in the following simulation studies, this is generally the case as long as the singular values of A decay sufficiently fast.

Figure 3.

Spectral norm loss (left panel) and Frobenius norm loss (right panel) when there is a gap between σ_r(A) and σ_r+1(A). The singular value values of A are given by (21), p₁ = p₂ = 1000, and m₁ = m₂ = 50.

We now turn to the settings with the singular values being {j^−α, j = 1, 2, …, min(p₁, p₂)}and various choices of α, p₁ and p₂. Hence, no significant gap between adjacent singular values exists under these settings and we aim to demonstrate that our method continues to work well. We first consider p₁ = p₂ = 1000, m₁ = m₂ = 50 and let α range from 0.3 to 2. Under this setting, we also study how the choice of thresholds affect the performance of our algorithm. For simplicity, we report results only for row thresholding as results for column thresholding are similar. The average loss of Â₂₂ from Algorithm 2 with $T_R\in \{c\sqrt{m_1/p_1},c\in [1,6]\}$ under both the spectral norm and Frobenius norm are given in Figure 4. In general, the algorithm performs well provided that α is not too small and as expected, the average loss decreases with a higher decay rate in the singular values. This indicates that the existence of a significant gap between adjacent singular values is not necessary in practice, provided that the singular values decay sufficiently fast. When comparing the results across different choices of the threshold, c = 2 as suggested in our theoretical analysis is indeed the optimal choice. Thus, in all subsequent numerical analysis, we fix c = 2.

Figure 4.

Spectral norm loss (left panel) and Frobenius norm loss (right panel) as the thresholding constant c varies. The singular values of A are {j^−α, j = 1, 2, …} with α varying from 0.3 to 2, p₁ = p₂ = 1000, and m₁ = m₂ = 50.

To investigate the impact of row versus column thresholding, we let the singular value decay rate be α = 1, p₁ = 300, p₂ = 3000, and m₁ and m₂ varying from 10 to 150. The original matrix A is generated the same way as before. We apply row and column thresholding with $T_R=2\sqrt{p_1/m_1}$ and $T_C=2\sqrt{p_2/m_2}$. It can be seen from Figure 5 that when the observed rows and columns are selected randomly, the results are not sensitive to the choice between row and column thresholding.

Figure 5.

Spectral and Frobenius norm losses with column/row thresholding. The singular values of A are {j⁻¹, j = 1, 2, …}, p₁ = 300, p₂ = 3000, and m₁, m₂ = 10, …, 150.

We next turn to the comparison between our proposed SMC algorithm and the penalized NNM method which recovers A by (4). The solution to (4) can be solved by the spectral regularization algorithm by Mazumder et al. (2010) or the accelerated proximal gradient algorithm by Toh and Yun (2010), where these two methods provide similar results. We use 5-fold cross-validation to select the tuning parameter t. Details on the implementation can be found in the Supplement.

We consider the setting where p₁ = p₂ = 500, m₁ = m₂ = 50, 100 and the singular value decay rate α ranges from 0.6 to 2. As shown in Figure 6, the proposed SMC method substantially outperform the penalized NNM method with respect to both the spectral and Frobenius norm loss, especially as α increases.

Figure 6.

Comparison of the proposed SMC method with the NNM method with 5-cross-validation for the settings with singular values of A being {j^−α, j = 1, 2, …} for α ranging from 0.6 to 2, p₁ = p₂ = 500, and m₁ = m₂ = 50 or 100.

Finally, we consider a simulation setting that mimics the ovarian cancer data application considered in the next section, where p₁ = 1148, p₂ = 1225, m₁ = 230, m₂ = 426 and the singular values of A decay at a polynomial rate α. Although the singular values of the full matrix are unknown, we estimate the decay rate based on the singular values of the fully observed 552 rows of the matrix from the TCGA study, denoted by {σ_j, j = 1, …, 522}. A simple linear regression of {log(σ_j), j = 1, …, 522} on {log(j), j = 1, …, 522} estimates α as 0.8777. In the simulation, we randomly generate A ∈ ℝ^p₁×p₂ such that the singular values are fixed as {j^−.8777, j = 1, 2, ···}. For comparison, we also obtained results for α = 1 as well as those based on the penalized NNM method with 5-cross-validation. As shown in Table 1, the relative spectral norm loss and relative Frobenius norm loss of the proposed method are reasonably small and substantially smaller than those from the penalized NNM method.

Table 1.

Relative spectral norm loss (||Â₂₂ – A₂₂||/||A₂₂||) and Frobenius norm loss (||Â₂₂ – A₂₂||_F/||A₂₂||_F) for p₁ = 1148, p₂ = 1225, m₁ = 230, m₂ = 426 and singular values of A being {j^−α: j = 1, 2, ···}.

	Relative spectral norm loss		Relative Frobenius norm loss
	SMC	NNM	SMC	NNM
α = 0.8777	0.1253	0.4614	0.2879	0.6122
α = 1	0.0732	0.4543	0.1794	0.5671

5 Application in Genomic Data Integration

In this section, we apply our proposed procedures to integrate multiple genomic studies of ovarian cancer (OC). OC is the fifth leading cause of cancer mortality among women, attributing to 14,000 deaths annually (Siegel et al., 2013). OC is a relatively heterogeneous disease with 5-year survival rate varying substantially among different subgroups. The overall 5-year survival rate is near 90% for stage I cancer. But the majority of the OC patients are diagnosed as stage III/IV diseases and tend to develop resistance to chemotherapy, resulting a 5-year survival rate only about 30% (Holschneider and Berek, 2000). On the other hand, a small minority of advanced cancers are sensitive to chemotherapy and do not replapse after treatment completion. Such a heterogeneity in disease progression is likely to be in part attributable to variations in underlying biological characteristics of OC (Berchuck et al., 2005). This heterogeneity and the lack of successful treatment strategies motivated multiple genomic studies of OC to identify molecular signatures that can distinguish OC subtypes, and in turn help to optimize and personalize treatment. For example, the Cancer Genome Atlas (TCGA) comprehensively measured genomic and epigenetic abnormalities on high grade OC samples (Cancer Genome Atlas Research Network, 2011). A gene expression risk score based on 193 genes, , was trained on 230 training samples, denoted by TCGA^(t), and shown as highly predictive of OC survival when validated on the TCGA independent validation set of size 322, denoted by TCGA^(v), as well as on several independent OC gene expression studies including those from Bonome et al. (2005) (BONO), Dressman et al. (2007) (DRES) and Tothill et al. (2008) (TOTH).

The TCGA study also showed that clustering of miRNA levels overlaps with gene-expression based clusters and is predictive of survival. It would be interesting to examine whether combining miRNA with could improve survival prediction when compared to alone. One may use TCGA^(v) to evaluate the added value of miRNA. However, TCGA^(v) is of limited sample size. Furthermore, since miRNA was only measured for the TCGA study, its utility in prediction cannot be directly validated using these independent studies. Here, we apply our proposed SMC method to impute the missing miRNA values and subsequently construct prediction rules based on both and the imputed miRNA, denoted by $\widehat{\text{miRNA}}$ for these independent validation sets. To facilitate the comparison with the analysis based on TCGA^(v) alone where miRNA measurements are observed, we only used the miRNA from TCGA^(t) for imputation and reserved the miRNA data from TCGA^(v) for validation purposes. To improve the imputation, we also included additional 300 genes that were previously used in a prognostic gene expression signature for predicting ovarian cancer survival (Denkert et al., 2009). This results in a total of m₁ = 426 unique gene expression variables available for imputation. Detailed information on the data used for imputation is shown in Figure 7. Prior to imputation, all gene expression and miRNA levels are log transformed and centered to have mean zero within each study to remove potential platform or batch effects. Since the observable rows (indexing subjects) can be viewed as random whereas the observable columns (indexing genes and miRNAs) are not random, we used row thresholding with threshold $T_R=2\sqrt{p_1/m_1}$ as suggested in the theoretical and simulation results. For comparison, we also imputed data using the penalized NNM method with tuning parameter t selected via 5-fold cross-validation.

Figure 7.

Imputation scheme for integrating multiple OC genomic studies.

We first compared $\widehat{\text{miRNA}}$ to the observed miRNA on TCGA^(v). Our imputation yielded a rank 2 matrix for $\widehat{\text{miRNA}}$ and the correlations between the two right and left singular vectors $\widehat{\text{miRNA}}$ to that of the observed miRNA variables are .90, .71, .34, .14, substantially higher than that of those from the NNM method, with the corresponding values 0.45, 0.06, 0.10, 0.05. This suggests that the SMC imputation does a good job in recovering the leading projections of the miRNA measurements and outperforms the NNM method.

To evaluate the utility of $\widehat{\text{miRNA}}$ for predicting OC survival, we used the TCGA^(t) to select 117 miRNA markers that are marginally associated with survival with a nominal p-value threshold of .05. We use the two leading principal components (PCs) of the 117 miRNA markers, ${\text{miRNA}}^{\text{PC}}={({\text{miRNA}}_1^{\text{PC}},{\text{miRNA}}_2^{\text{PC}})}^T$, as predictors for the survival outcome in addition to G. The imputation enables us to integrate information from 4 studies including TCGA^(t), which could substantially improve efficiency and prediction performance. We first assessed the association between {miRNA^PC, } and OC survival by fitting a stratified Cox model (Kalbfleisch and Prentice, 2011) to the integrated data that combines TCGA^(v) and the three additional studies via either the SMC or NNM methods. In addition, we fit the Cox model to (i) TCGA^(v) set alone with miRNA^PC obtained from the observed miRNA; and (ii) each individual study separately with imputed miRNA^PC. As shown in Table 2(a), the log hazard ratio (logHR) estimates for miRNA^PC from the integrated analysis, based on both SMC and NNM methods, are similar in magnitude to those obtained based on the observed miRNA values with TCGA^(v). However, the integrated analysis has substantially smaller standard error (SE) estimates due the increased sample sizes. The estimated logHRs are also reasonably consistent across studies when separate models were fit to individual studies.

Table 2.

Shown in (a) are the estimates of the log hazard ratio (logHR) along with their corresponding standard errors (SE) and p-values by fitting stratified Cox model integrating information from 4 independent studies with imputed miRNA based on the SMC method and the nuclear norm minimization (NNM); and Cox model to the TCGA test data with original observed miRNA (Ori.). Shown also are the estimates for each individual studies by fitting separate Cox models with imputed miRNA.

(a) Integrated Analysis with Imputed miRNA vs Single study with observed miRNA
	logHR			SE			p-value
	Ori.	SMC	NNM	Ori.	SMC	NNM	Ori.	SMC	NNM
	.067	.143	.168	.041	.034	.028	.104	.000	.000
${\text{miRNA}}_1^{\text{PC}}$	−.012	−.019	−.013	.009	.006	.012	.218	.001	.283
${\text{miRNA}}_2^{\text{PC}}$	.023	.018	−.005	.014	.009	.014	.092	.039	.725

(b) Estimates for Individual Studies with Imputed miRNA from the SMC method
	logHR				SE				p-value
	TCGA	TOTH	DRES	BONO	TCGA	TOTH	DRES	BONO	TCGA	TOTH	DRES	BONO
	.051	.377	.174	.311	.048	.069	.132	.117	.286	.000	.187	.008
${\text{miRNA}}_1^{\text{PC}}$	−.014	−.021	−.031	−.010	.011	.012	.014	.014	.207	.082	.030	.484
${\text{miRNA}}_2^{\text{PC}}$	.014	.045	−.021	.036	.016	.018	.022	.019	.391	.009	.336	.054

(c) Estimates for Individual Studies with Imputed miRNA from the NNM method
	logHR				SE				p-value
	TCGA	TOTH	DRES	BONO	TCGA	TOTH	DRES	BONO	TCGA	TOTH	DRES	BONO
	.082	.405	.361	.258	.037	.066	.114	.088	.028	.000	.002	.003
${\text{miRNA}}_1^{\text{PC}}$	−.045	.016	.055	−.008	.021	.026	.031	.023	.034	.544	.076	.721
${\text{miRNA}}_2^{\text{PC}}$	.008	−.086	−.043	.019	.026	.027	.034	.029	.758	.002	.201	.496

We also compared the prediction performance of the model based on alone to the model that includes both and the imputed miRNA^PC. Combining information from all 4 studies via standard meta analysis, the average improvement in C-statistic was 0.032 (SE = 0.013) for the SMC method and 0.001 (SE = 0.009) for the NNM method, suggesting that the imputed miRNA^PC from the SMC method has much higher predictive value compared to those obtained from the NNM method.

In summary, the results shown above suggest that our SMC procedure accurately recovers the leading PCs of the miRNA variables. In addition, adding miRNA^PC obtained from imputation using the proposed SMC method could significantly improve the prediction performance, which confirms the value of our method for integrative genomic analysis. When comparing to the NNM method, the proposed SMC method produces summaries of miRNA that is more correlated with the truth and yields leading PCs that are more predictive of OC survival.

6 Discussions

The present paper introduced a new framework of SMC where a subset of the rows and columns of an approximately low-rank matrix are observed. We proposed an SMC method for the recovery of the whole matrix with theoretical guarantees. The proposed procedure significantly outperforms the conventional NNM method for matrix completion, which does not take into account the special structure of the observations. As shown by our theoretical and numerical analyses, the widely adopted NNM methods for matrix completion are not suitable for the SMC setting. These NNM methods perform particularly poorly when a small number of rows and columns are observed.

The key assumption in matrix completion is the matrix being approximately low rank. This is reasonable in the ovarian cancer application since as indicated in the results from the TCGA study (Cancer Genome Atlas Research Network, 2011), the patterns observed in the miRNA signature are highly correlated with the patterns observed in the gene expression signature. This suggests the high correlation among the selected gene expression and miRNA variables. Results from the imputation based on the approximate low rank assumption given in Section 5 are also encouraging with promising correlations with true signals and good prediction performance from the imputed miRNA signatures. We expect that this imputation method will also work well in genotyping and sequencing applications, particularly for regions with reasonably high linkage disequilibrium.

Another main assumption that is needed in the theoretical analysis is that there is a significant gap between the r^th and (r + 1)^th singular values of A. This assumption may not be valid in real practice. In particular, the singular values of the ovarian dataset analyzed in Section 5 is decreasing smoothly without a significant gap. However, it has been shown in the simulation studies presented in Section 4 that, although there is no significant gap between any adjacent singular values of the matrix to be recovered, the proposed SMC method works well as long as the singular values decay sufficiently fast. Theoretical analysis for the proposed SMC method under more general patterns of singular value decay warrants future research.

To implement the proposed Algorithm 2, major decisions include the choice of threshold values and choosing between column thresholding and row thresholding. Based on both the-oretical and numerical studies, optimal threshold values can be set as $T_C=2\sqrt{p_2/m_2}$ for column thresholding and $T_R=2\sqrt{p_1/m_1}$ for row thresholding. Simulation results in Section 4 show that when both rows and columns are randomly chosen, the results are very similar. In the real data applications, the choice between row thresholding and column thresholding depends on whether the rows or columns are more \homogeneous”, or closer to being randomly sampled. For example, in the ovarian cancer dataset analyzed in Section 5, the rows correspond to the patients and the columns correspond to the gene expression levels and miRNA levels. Thus the rows are closer to random sample than the columns, consequently it is more natural to use the row thresholding in this case.

We have shown both theoretically and numerically in Sections 3 and 4 that Algorithm 2 provides a good recovery of A₂₂. However, the naive implementation of this algorithm requires min(m_1, m₂) matrix inversions and multiplication operations in the for loop that calculates ||D_R,s|| (or ||D_C,s||), s ∈ {r̂, r̂ + 1, ···, min(m₁, m₂)}. Taking into account the relationship among D_R,s (or D_C,s) for different s’s, it is possible to simultaneously calculate all ||D_R,s|| (or ||D_C,s||) and accelerate the computations. For reasons of space, we leave optimal implementation of Algorithm 2 as future work.