CONVERGENCE AND PREDICTION OF PRINCIPAL COMPONENT SCORES IN HIGH-DIMENSIONAL SETTINGS

Abstract

A number of settings arise in which it is of interest to predict Principal Component (PC) scores for new observations using data from an initial sample. In this paper, we demonstrate that naive approaches to PC score prediction can be substantially biased towards 0 in the analysis of large matrices. This phenomenon is largely related to known inconsistency results for sample eigenvalues and eigenvectors as both dimensions of the matrix increase. For the spiked eigenvalue model for random matrices, we expand the generality of these results, and propose bias-adjusted PC score prediction. In addition, we compute the asymptotic correlation coefficient between PC scores from sample and population eigenvectors. Simulation and real data examples from the genetics literature show the improved bias and numerical properties of our estimators.

1. Introduction

Principal component analysis (PCA) [19] is one of the leading statistical tools for analyzing multivariate data. It is especially popular in genetics/genomics, medical imaging, and chemometrics studies where high-dimensional data is common. PCA is typically used as a dimension reduction tool. A small number of top ranked principal component (PC) scores are computed by projecting data onto spaces spanned by the eigenvectors of sample covariance matrix, and are used to summarize data characteristics that contribute most to data variation. These PC scores can be subsequently used for data exploration and/or model predictions. For example, in genome-wide association studies (GWAS), PC scores are used to estimate ancestries of study subjects and as covariates to adjust for population stratification [24, 27]. In gene expression microarray studies, PC scores are used as synthetic “eigen-genes” or “meta-genes” intended to represent and discover gene expression patterns that might not be discernible from single-gene analysis [30].

Although PCA is widely applied in a number of settings, much of our theoretical understanding rests on a relatively small body of literature. Girshick [12] introduced the idea that the eigenvectors of sample covariance matrix are maximum likelihood estimators. Here a key concept in a population view of PCA is that the data arise as p-variate values from a distinct set of n independent samples. Later, the asymptotic distribution of eigenvalues and eigenvectors of the sample covariance matrix (i.e., the sample eigenvalues and eigenvectors) were derived for the situation where n goes to infinity and p is fixed [2, 13]. With the development of modern high-throughput technologies, it is not uncommon to have data where p is comparable in size to n, or substantially larger. Under the assumption that p and n grow at the same rate, that is p/n → γ > 0, there has been considerable effort to establish convergence results for sample eigenvalues and eigenvectors (see review [5]). The convergence of the sample eigenvalues and eigenvectors under the “spiked population” model proposed by Johnstone [18] has also been established[7, 23, 26]. For this model it is well known that the sample eigenvectors are not consistent estimators of the eigenvectors of population covariance (i.e., the population eigenvectors) [17, 23, 26]. Furthermore, Paul [26] has derived the degree of discrepancy in terms of the angle between the sample and population eigenvectors, under Gaussian assumptions for 0 < γ < 1. More recently, Nadler [23] has extended the same result to the more general γ > 0 using a matrix perturbation approach.

These results have considerable potential practical utility in understanding the behavior of PC analysis and prediction in modern datasets, for which p may be large. The practical goals of this paper focus primarily on the prediction of PC scores for samples which were not included in the original PC analysis. For example, gene expression data of new breast cancer patients may be collected, and we might want to estimate their PC scores in order to classify their cancer sub-type. The recalculation of PCs using both new and old data might not be practical, e.g. if the application of PCs from gene expression is used as a diagnostic tool in clinical applications. For GWAS analysis, it is known that PC analysis which includes related individuals tends to generate spurious PC scores which do not reflect the true underlying population substructures. To overcome this problem, it is common practice to include only one individual per family/sibship in the initial PC analysis. Another example arises in cross-validation for PC regression, in which PC scores for the test set might be derived using PCA performed on the training set [16]. For all of these applications, the predicted PC scores for a new sample are usually estimated in the “naive” fashion, in which the data vector of the new sample is multiplied by the sample eigenvectors from the original PC analysis. Indeed, there appears to be relatively little recognition in the genetics or data mining literature that this approach may lead to misleading conclusions.

For low dimensional data, where p is fixed as n increases or otherwise much smaller than n, the predicted PC scores are nearly unbiased and well-behaved. However, for high-dimensional data, particularly with p > n, they tend to be biased and shrunken towards 0. The following simple example of a stratified population with three strata illustrates the shrinkage phenomenon for predicted PC scores. We generated a training data set with n = 100 and p = 5000. Among the 100 samples, 50 are from stratum 1, 30 are from stratum 2 and the rest from stratum 3. For each stratum, we first created a p-dimensional mean vector μ_k (k = 1, 2, 3). Each element of each mean vector was created by drawing randomly with replacement from {−0.3, 0, 0.3}, and thereafter considered a fixed property of the stratum. Then for each sample from the kth stratum, its p covariates were simulated from the multivariate normal distribution MVN(μ_k, 4I), where I is the p×p identity matrix. A test dataset with the same sample size and μ_k vectors was also simulated. Figure 1 shows that the predicted PC scores for the test data are much closer to 0 compared to the scores from the training data. This shrinkage phenomenon may create a serious problem if the predicted PC scores are used to classify new test samples, perhaps by similarity to previous apparent clusters in the original data. In addition, the predicted PC scores may produce incorrect results if used for downstream analyses (e.g., as covariates in association analyses).

Fig 1.

Simulation results for p=5000 and n=(50,30,20). Different symbols represent different groups. White background color represents the training set and grey background color represents the test set. A) First 2 PC score plot of all simulated samples. B) Center of each group.

In this paper, we investigate the degree of shrinkage bias associated with the predicted PC scores, and then propose new bias-adjusted PC score estimates. As the shrinkage phenomenon is largely related to the limiting behavior of the sample eigenvectors, our first step is to describe the discrepancy between the sample and population eigenvectors. To achieve this purpose, we follow the assumption that p and n both are large and grow at the same rate. By applying and extending results from random matrix theory, we establish the convergence of the sample eigenvalues and eigenvectors under the spiked population model. We generalize Theorem 4 of Paul [26], which describes the asymptotic angle between sample and population eigenvectors, to non-Gaussian random variables for any γ > 0. We further derive the asymptotic angle between PC scores from sample eigenvectors and population eigenvectors, and the asymptotic shrinkage factor of the PC score predictions. Finally we construct estimators of the angles and the shrinkage factor. The theoretical results are presented in Section 2.

In section 3, we report simulations to assess the finite sample accuracy of the proposed asymptotic angle and shrinkage factor estimators. We also show the potential improvements in prediction accuracy for PC regression by using the bias adjusted PC scores. In section 4, we apply our PC analysis to a real genome-wide association study, which demonstrates that the shrinkage phenomenon occurs in real studies and that adjustment is needed.

2. Method

2.1. General Setting

Throughout this paper, we use ^T to denote matrix transpose, $\stackrel{p}{\to }$ to denote convergence in probability, and $\stackrel{a.s}{\to }$ to denote almost sure convergence. Let Λ = diag(λ₁, λ₂, …, λ_p), a p × p matrix with λ₁ ≥ λ₂ ≥ ··· ≥ λ_p, and E = [e₁, …, e_p], a p × p orthogonal matrix.

Define the p × n data matrix, X as [x₁, …, x_n], where x_j is the p-dimensional vector corresponding to the j^th sample. For the remainder of the paper, we assume the following:

Assumption 1

X = EΛ^1/2Z, where Z = {z_ij} is a p × n matrix whose elements z_ijs are i.i.d random variables with E(z_ij) = 0, $E(z_{ij}^2)=1$ and $E(z_{ij}^4)<\infty$.

Although the z_ijs are i.i.d, Assumption 1 allows for very flexible covariance structures for X, and thus the results of this paper are quite general. The population covariance matrix of X is Σ = EΛE^T. The sample covariance matrix S equals

$$\mathbf{S}={\mathbf{XX}}^T/n={\mathbf{E}\mathbf{\Lambda }}^{1/2}{\mathbf{ZZ}}^T{\mathbf{\Lambda }}^{1/2}{\mathbf{E}}^T/n.$$

The λ_ks are the underlying population eigenvalues. The spiked population model defined in [18] assumes that all the population eigenvalues are 1, except the first m eigenvalues. That is, λ₁ ≥ λ₂ ··· ≥ λ_m > λ_m+1 = ··· = λ_p = 1. The spectral decomposition of the sample covariance matrix is

$$\mathbf{S}={\mathbf{UDU}}^T,$$

where D = diag(d₁, d₂, …, d_p) is a diagonal matrix of the ordered sample eigenvalues and U = [u₁, …, u_p] is the corresponding p × p sample eigen-vector matrix. Then the PC score matrix is P = [p₁, p₂, …, p_n], where ${\mathbf{p}}_v^T={\mathbf{u}}_v^T\mathbf{X}$ is the vth sample PC score. For a new observation x_new, its predicted PC score is similarly defined as U^T x_new with the vth (PC) score equal to $q_v={\mathbf{u}}_v^T{\mathbf{x}}_{\mathit{new}}$.

2.2. Sample Eigenvalues and Eigenvectors

Under the classical setting of fixed p, it is well known that the sample eigenvalues and eigenvectors are consistent estimators of the corresponding population eigenvalues and eigenvectors [3]. Under the “large p, large n” framework, however, the consistency is not guaranteed. The following two lemmas summarize and extend some known convergence results.

Lemma 1

Let p/n → γ ≥ 0 as n → ∞.

1. When γ = 0,

   $$d_v\stackrel{a.s}{\to }\{\begin{array}{ll}{\lambda }_v,\hfill & \mathit{for}v\le m\hfill \\ 1,\hfill & \mathit{for}v>m;\hfill \end{array}$$ (1)
2. When γ > 0,

   $$d_v\stackrel{a.s}{\to }\{\begin{array}{ll}\rho ({\lambda }_v),\hfill & \mathit{for}v\le k\hfill \\ {(1+\sqrt{\gamma })}^2,\hfill & \mathit{for}v=k+1,\hfill \end{array}$$ (2)

   where k is the number of λ_v greater than $1/\sqrt{\gamma }$, and ρ(x) = x(1+γ/(x − 1)).

The result in ii) is due to Baik and Silverstein [7], while the proof of i) can be found in section (6.3). The result in i) shows that when γ = 0, the sample eigenvalues converge to the corresponding population eigenvalues, which is consistent with the classical PC result where p is fixed. The result in ii) shows that for any non-zero γ, d_v is no longer a consistent estimator of λ_v. However, a consistent estimator of λ_v can be constructed from Equation (2). Define

$${\rho }^{-1}(d)=\frac{d+1-\gamma +\sqrt{{(d+1-\gamma )}^2-4d}}{2}.$$

Then ρ⁻¹(d_v) is a consistent estimator of λ_v when ${\lambda }_v>1+\sqrt{\gamma }$. Furthermore, Baik et al. [6] have shown the $\sqrt{n}$-consistency of d_v to ρ(λ_v), and Bai and Yao [4] have shown that d_v is asymptotically normal.

Lemma 2

Suppose p/n → γ ≥ 0 as n → ∞. Let < ., . > be an inner product between two vectors. Under the assumption of multiplicity one,

1. if 0 < γ < 1, and the z_ijs follow the standard normal distribution, then

   $$\mid <{\mathbf{e}}_v,{\mathbf{u}}_v>\mid \stackrel{a.s}{\to }\{\begin{array}{ll}\phi ({\lambda }_v),\hfill & if{\lambda }_v>1+\sqrt{\gamma }\hfill \\ 0,\hfill & if1<{\lambda }_v\le 1+\sqrt{\gamma };\hfill \end{array}$$ (3)
2. removing the normal assumption on the z_ijs, the following weaker convergence result holds for all γ ≥ 0

   $$\mid <{\mathbf{e}}_v,{\mathbf{u}}_v>\mid \stackrel{p}{\to }\{\begin{array}{ll}\phi ({\lambda }_v),\hfill & if{\lambda }_v>1+\sqrt{\gamma }\hfill \\ 0,\hfill & if1<{\lambda }_v\le 1+\sqrt{\gamma }.\hfill \end{array}$$ (4)

   Here $\phi (x)=\sqrt{(1-\frac{\gamma }{{(x-1)}^2})/(1+\frac{\gamma }{x-1})}$.

The inner product between unit vectors is the cosine angle between these two. Thus, Lemma 2 shows the convergence of the angle between population and sample eigenvectors. For i), Paul [26] proved it for γ < 1; while Nadler [23] obtained the same conclusion for γ > 0 using the matrix perturbation approach under the Gaussian random noise model. We relax the Gaussian assumption on z and prove ii) for γ ≥ 0 in section 6.4. The result of ii) is general enough for the application of PCA to, for example, genome-wide association mapping, where each entry of X is a standardized variable of SNP genotypes, which are typically coded as {0, 1, 2}, corresponding to discrete genotypes.

2.3. Sample and Predicted PC Scores

In this section, we first discuss convergence of the sample PC scores, which forms the basis for the investigation of the shrinkage phenomenon of the predicted PC scores. For the sample PC scores, we have

Theorem 1

Let ${\mathbf{g}}_v^T={\mathbf{e}}_v^T\mathbf{X}/\sqrt{\mathbf{n}{\lambda }_v}$, the normalized v^th PC score derived from a corresponding population eigenvector, e_v, and ${\stackrel{\sim }{\mathbf{p}}}_v={\mathbf{p}}_v/\sqrt{d_v}$, the normalized v^th sample PC score. Suppose p/n → γ ≥ 0 as n → ∞. Under the multiplicity one assumption,

$$\mid <{\mathbf{g}}_v,{\stackrel{\sim }{\mathbf{p}}}_v>\mid \stackrel{p}{\to }\{\begin{array}{ll}\sqrt{1-\frac{\gamma }{{({\lambda }_v-1)}^2}},\hfill & if{\lambda }_v>1+\sqrt{\gamma }\hfill \\ 0,\hfill & if1<{\lambda }_v\le 1+\sqrt{\gamma }.\hfill \end{array}$$ (5)

The proof can be found in section 6.7. In PC analysis, the sample PC scores are typically used to estimate certain latent variables (largely the PC scores from population eigenvectors) that represent the underlying data structures. The above result allows us to quantify the accuracy of the sample PC scores. Note that here < g_v, p̃_v > is the correlation coefficient between g_v and p̃_v. Compared to Equation (3) in Lemma 2, the angle between the PC scores is smaller than the angle between their corresponding eigenvectors.

Before we formally derive the asymptotic shrinkage factor for the predicted PC scores, we first describe in mathematical terms the shrinkage phenomenon that was demonstrated in the Introduction. Note that the first population eigenvector e₁ satisfies

$${\mathbf{e}}_1=\underset{\mathbf{a}:{\mathbf{a}}^T\mathbf{a}=1}{\text{argmax}}E({({\mathbf{a}}^T\mathbf{x})}^2)$$

for a random vector x that follows the same distribution of the x_js. For the data matrix X, its first sample eigenvector u₁ satisfies

$${\mathbf{u}}_1=\underset{\mathbf{a}:{\mathbf{a}}^T\mathbf{a}=1}{\text{argmax}}\sum _{j=1}^n{({\mathbf{a}}^T{\mathbf{x}}_j)}^2.$$

Assuming that u₁ and the new sample x_new are independent of each other, we have

$$\begin{array}{l}E({({\mathbf{u}}_1^T{\mathbf{x}}_{\mathit{new}})}^2)=E(E({\mathbf{u}}_1^T{\mathbf{x}}_{\mathit{new}}{\mathbf{x}}_{\mathit{new}}^T{\mathbf{u}}_1^T\mid {\mathbf{u}}_1))=E({\mathbf{u}}_1^{T}E({\mathbf{x}}_{\mathit{new}}{\mathbf{x}}_{\mathit{new}}^T){\mathbf{u}}_1^T)\\ =E({\mathbf{u}}_t^T\mathbf{\sum }{\mathbf{u}}_1^T)\le {\mathbf{e}}_1^T\mathbf{\sum }{\mathbf{e}}_1=E({({\mathbf{e}}_1^T{\mathbf{x}}_{\mathit{new}})}^2).\end{array}$$ (6)

Since the ${\mathbf{u}}_1^T{\mathbf{x}}_j\text{s}(j=1,\dots ,n)$ follow the same distribution,

$$nE({({\mathbf{e}}_1^T{\mathbf{x}}_j)}^2)=E(\sum _{j=1}^n{({\mathbf{e}}_1^T{\mathbf{x}}_j)}^2)\le E(\sum _{j=1}^n{({\mathbf{u}}_1^T{\mathbf{x}}_j)}^2)=nE({({\mathbf{u}}_1^T{\mathbf{x}}_j)}^2).$$ (7)

By (6) and (7), we can show that

$$E({({\mathbf{u}}_1^T{\mathbf{x}}_{\mathit{new}})}^2)\le E({({\mathbf{e}}_1^T{\mathbf{x}}_{\mathit{new}})}^2)=E({({\mathbf{e}}_1^T{\mathbf{x}}_j)}^2)\le E({({\mathbf{u}}_1^T{\mathbf{x}}_j)}^2),$$

which demonstrates the shrinkage feature of the predicted PC scores. The amount of the shrinkage, or the asymptotic shrinkage factor, is given by the following theorem:

Theorem 2

Suppose p/n → γ ≥ 0 as n → ∞, ${\lambda }_v>1+\sqrt{\gamma }$. Under the multiplicity one assumption,

$$\sqrt{\frac{E(q_v^2)}{E(p_{vj}^2)}}\stackrel{n\to \infty }{\to }\frac{{\lambda }_v-1}{{\lambda }_v+\gamma -1}$$ (8)

where p_vj is the j^th element of p_v.

The proof is given in section 6.8. We call (λ_v − 1)/(λ_v + γ − 1), the (asymptotic) shrinkage factor for a new subject. As shown, the shrinkage factor is smaller than 1 if γ > 0. Quite sensibly, it is a decreasing function of γ and an increasing function of λ_v. The bias of the predicted PC score can be potentially large for those high dimensional data where p is substantially greater than n, and/or for the data with relatively minor underlying structures where λ_v is small.

2.4. Rescaling of sample eigenvalues

The previous theorems are based on the assumption that all except the top m eigenvalues are equal to 1. Even under the spiked eigenvalue model, some rescaling of the sample eigenvalues may be necessary with real data.

For a given data, let its ordered population eigenvalues Λ^* = {ζλ₁, …, ζλ_m, ζ, …, ζ}, where ζ ≠ = 1, and its corresponding sample eigenvalues ${\mathbf{D}}^{\ast }=\{d_1^{\ast },\dots ,d_n^{\ast }\}$. We can show that Equations (4), (8), and (5) still hold under such circumstances. However, ${\rho }^{-1}(d_v^{\ast })$ is no longer a consistent estimator of λ_v, because

$$d_v^{\ast }\stackrel{a.s}{\to }\zeta {\lambda }_v(1+\frac{\gamma }{{\lambda }_v-1})=\zeta \phi ({\lambda }_v).$$

To address this issue, Baik and Silverstein [7] have proposed a simple approach to estimate ζ. In their method, the top significant large sample eigenvalues are first separated from the other grouped sample eigenvalues. Then ζ is estimated as the ratio between the average of the grouped sample eigenvalues and the mean determined by the Marchenko-Pastur law [22]. To separate the eigenvalues, they have suggested to use a screeplot of the percent variance versus component number. However, for real data, we may not be able to clearly separate the sample eigenvalues in such a manner and readily apply the approach. Thus we need an automated method which does not require a clear separation of the sample eigenvalues.

The expectation of the sum of the sample eigenvalues when ζ = 1 is

$$E(\sum _{v=1}^p{d}_v)=E(\mathit{trace}(\mathbf{S}))=\mathit{trace}(E(\mathbf{S}))=\mathit{trace}(\mathbf{\sum })=\sum _{v=1}^p{\lambda }_v,$$

Thus, the sum of the rescaled eigenvalues is expected to be close to ( ${\displaystyle \sum _{v=1}^m}{\lambda }_v+p-m$). Let $r_v=d_v^{\ast }/\left({\displaystyle \sum _{v=1}^p}{d}_v^{\ast }\right)$ and d̂_v be a properly rescaled eigenvalue, then d̂_v should be very close to $r_v({\displaystyle \sum _{v=1}^m}{\lambda }_v+p-m)$. Note that $p/({\displaystyle \sum _{v=1}^m}{\lambda }_v+p-m)\to 1$ for fixed m and λ_v. Thus pr_v is a properly adjusted eigenvalue. However, for finite n and p, the difference between p and ( ${\displaystyle \sum _{v=1}^m}{\lambda }_v+p-m$) can be substantial, especially when the first several λ_vs are considerably larger than 1. To reduce this difference, we propose a novel method which iteratively estimates the ( ${\displaystyle \sum _{v=1}^m}{\lambda }_v+p-m$) and d̂_v.

1. Initially set d̂_v,0 = pr_v

2. For the l^th iteration, set λ̂_v,l = ρ⁻¹(d̂_v,l−1) for ${\widehat{d}}_{v,l-1}>{(1+\sqrt{\gamma })}^2$, and λ̂_v,l = 1 for ${\widehat{d}}_{v,l-1}\le {(1+\sqrt{\gamma })}^2$. Define k_l as the number of λ̂_v,ls that are greater than 1, and let

   $${\widehat{d}}_{v,l}=\left(\sum _{v=1}^{k_l}{\widehat{\lambda }}_{v,l}+p-k_l\right)r_v.$$
3. If ${\displaystyle \sum _{v=1}^{k_l}}{\widehat{\lambda }}_{v,l}+p-k_l$ converges, let

   $${\widehat{d}}_v={\widehat{d}}_{v,l}$$

and stop. Otherwise, go to step 2.

The consistency of d̂_v to ρ(λ_v) is shown in the following theorem.

Theorem 3

Let d̂_v be the rescaled sample eigenvalue from the proposed algorithm. Then, for ${\lambda }_v>1+\sqrt{\gamma }$ with multiplicity one,

$${\widehat{d}}_v\stackrel{p}{\to }\rho ({\lambda }_v)$$

Since ${\rho }^{-1}({\widehat{d}}_v)\stackrel{p}{\to }{\lambda }_v$, φ(ρ⁻¹(d̂_v,))² is a consistent estimator of φ(λ_v)². Combining this fact with Theorems 2 and 3, we can obtain the bias adjusted PC score $q_v^{\ast }$

$$q_v^{\ast }=q_v\frac{{\rho }^{-1}({\widehat{d}}_v)+\gamma -1}{{\rho }^{-1}({\widehat{d}}_v)-1}$$

and the asymptotic correlation coefficient between g_v and p̃_v

$$\sqrt{(1-\frac{\gamma }{{({\rho }^{-1}({\widehat{d}}_v)-1)}^2})}.$$

3. Simulation

First, we applied our bias adjustment process to the simulated data described in the Introduction. Our estimated asymptotic shrinkage factors are 0.465 and 0.329 for the first and second PC scores, respectively. The scatter plot of the top two bias adjusted PC scores is given in Figure 2. After the bias adjustment, the predicted PC scores of the test data are comparable to those of the training data. This indicates that our method is effective in correcting for the shrinkage bias.

Fig 2.

Shrinkage Adjusted PC scores of the data in Figure 1. Different symbols represent different groups. White background color represents the training set and grey background color represents the test set. A) plots of all simulation samples. B) Center of each group.

Next, we conducted a new simulation to check the accuracy of our estimators. For the jth sample (j = 1, …, n), its i^th variable was generated as

$$x_{ij}=\{\begin{array}{cc}{\lambda }_1{z}_{ij}& i=1\\ {\lambda }_2{z}_{ij}& i=1\\ z_{ij}& i>2\end{array}$$

where λ₁ > λ₂ > 1 and z_ij ~ N (0, 2²). Under this setting, λ₁ and λ₂ are the first and the second population eigenvalues. The first and second population eigenvectors are e₁ = {1, 0,…, 0} and e₂ = {0, 1, 0, …, 0} respectively. We set the standard deviation of z_ij to 2 instead of 1, which allows us to test whether the rescaling procedure works properly. We tried different values of γ and n, but set λ₁ and λ₂ to $4(1+\sqrt{\gamma })$ and $2(1+\sqrt{\gamma })$, respectively.

We split the simulated samples into test and training sets, each with n samples. We first estimated the asymptotic shrinkage factor based on the training samples. We then calculated the predicted PC scores on the test samples. To assess the accuracy of shrinkage factor estimator for each PC, we empirically estimated the shrinkage factor by the ratio of the mean predicted PC scores of the test samples to the mean PC scores of the training samples. That is, for the vth PC, the empirical shrinkage factor is estimated by $\sqrt{{\displaystyle \sum _{i=1}^n}{q}_{vi}^2/{\displaystyle \sum _{k=1}^n}{p}_{vk}^2}$. On the training samples, we also estimated the empirical angle between the sample and (known) population eigenvectors, as well as the empirical angle between PC scores from sample and population eigenvectors. The asymptotic theoretical estimates were also calculated. Tables 1 and 2 summarize the simulation results. Our asymptotic estimators provide accurate estimates for the angles and the shrinkage factor.

Table 1.

Cosine angle estimates of eigenvectors and PC scores based on 1000 simulations. “Angle” indicates the theoretical asymptotic cos (angle), “Estimate1” indicates the empirical cos(angle) estimator, ‘Estimate2” indicates the asymptotic cos(angle) estimator. For each estimator, each entry represents mean of 1, 000 simulation results with standard error in parentheses.

γ	n	PC 1			PC 2
		Angle	Angle Estimate1	Angle Estimate2	Angle	Angle Estimate1	Angle Estimate2
Eigenvectors
1	100	0.93	0.93(0.013)	0.91(0.027)	0.82	0.81(0.053)	0.80(0.052)
	200		0.93(0.009)	0.92(0.014)		0.81(0.030)	0.81(0.032)
20	100	0.70	0.69(0.037)	0.70(0.031)	0.51	0.50(0.053)	0.50(0.058)
	200		0.69(0.023)	0.70(0.022)		0.51(0.036)	0.51(0.041)
100	100	0.53	0.53(0.034)	0.53(0.031)	0.37	0.35(0.043)	0.35(0.047)
	200		0.53(0.024)	0.53(0.024)		0.36(0.029)	0.36(0.033)
500	100	0.38	0.38(0.029)	0.38(0.028)	0.25	0.24(0.033)	0.24(0.037)
	200		0.38(0.020)	0.38(0.020)		0.25(0.021)	0.25(0.024)
PC Scores
1	100	0.99	0.99(0.004)	0.98(0.016)	0.94	0.93(0.036)	0.91(0.048)
	200		0.99(0.003)	0.99(0.006)		0.94(0.019)	0.93(0.024)
20	100	0.98	0.97(0.083)	0.98(0.008)	0.89	0.86(0.105)	0.87(0.055)
	200		0.97(0.055)	0.98(0.005)		0.88(0.073)	0.88(0.036)
100	100	0.97	0.97(0.079)	0.97(0.009)	0.88	0.85(0.109)	0.86(0.060)
	200		0.97(0.058)	0.97(0.006)		0.86(0.076)	0.87(0.039)
500	100	0.97	0.96(0.084)	0.97(0.010)	0.87	0.83(0.117)	0.84(0.069)
	200		0.96(0.058)	0.97(0.007)		0.86(0.076)	0.86(0.038)

Table 2.

Shrinkage factor esimates based on 1000 simulation. “Factor” indicates the theoretical asymptotic factor, “Estimate1” indicates the empirical shrinkage factor estimator, ‘Estimate2” indicates the asymptotic shrinkage factor estimator. For each estimator, each entry represents mean of 1, 000 simulation results with standard error in parentheses.

γ	n	PC 1			PC 2
		Factor	Factor Estimate1	Factor Estimate2	Factor	Factor Estimate1	Factor Estimate2
1	100	0.88	0.88(0.017)	0.87(0.076)	0.75	0.75(0.044)	0.76(0.063)
	200		0.88(0.013)	0.87(0.054)		0.75(0.027)	0.75(0.044)
20	100	0.51	0.51(0.037)	0.51(0.038)	0.33	0.34(0.033)	0.32(0.038)
	200		0.51(0.025)	0.51(0.026)		0.34(0.022)	0.33(0.028)
100	100	0.30	0.30(0.024)	0.30(0.030)	0.17	0.17(0.019)	0.17(0.023)
	200		0.30(0.017)	0.30(0.023)		0.18(0.013)	0.17(0.017)
500	100	0.16	0.15(0.014)	0.16(0.020)	0.08	0.08(0.010)	0.08(0.013)
	200		0.15(0.010)	0.16(0.014)		0.08(0.007)	0.08(0.009)

Finally, we conducted simulation to demonstrate an application of the bias adjusted PC scores in PC regression. PC regression has been widely used in microarray gene-expression studies [9]. In this simulation, we let p = 5, 000, and our set up is very similar to the first simulation of Bair et al. [8]. Let x_ij denote the gene expression level of the ith gene for the jth subject. We generated each x_ij according to

$$x_{ij}=\{\begin{array}{cc}3+\epsilon & i\le g,j\le n/2\\ 4+\epsilon & i\le g,j>n/2\\ 3.5+\epsilon & i>g\end{array}$$

and the outcome variable y_j as

$$y_j=\frac{2}{g}\sum _{i=1}^g{x}_{ij}+{\epsilon }_y,$$

where n is the number of samples, g is the number of genes that are differentially expressed and associated with the phenotype, ε ~ N(0, 2²) and ε_y ~ N(0, 1). A total of eight different combinations of n and g were simulated. For the training data, we fit the PC regression with the first PC as the covariate and computed the mean square error (MSE). For the test samples with the same configuration of the training samples, we applied the PC model built on the training data to predict the phenotypes using the un-adjusted and adjusted PC scores. The results are presented in Table 3. We see that the MSE of the test set without bias adjustment is appreciably higher than that of the test set with bias adjustment, and the MSE of the test set with bias adjustment is comparable with the MSE of the training set.

Table 3.

Mean Square Error(MSE) of the PC regression based on gene-expression microarray data simulation with and without shrinkage adjustment. 1,000 simulation were conducted. Each entry in the table represents mean of the MSE with standard error in parentheses

n	g	Test Data without Adjustment	Test Data with Adjustment	Training Data
100	150	1.97(0.256)	1.70(0.284)	1.61(0.284)
100	300	1.63(0.230)	1.17(0.167)	1.12(0.158)
100	500	1.43(0.204)	1.07(0.157)	1.03(0.147)
100	1000	1.22(0.182)	1.03(0.148)	0.99(0.142)
200	150	1.73(0.159)	1.33(0.133)	1.30(0.131)
200	300	1.39(0.139)	1.08(0.105)	1.07(0.110)
200	500	1.24(0.131)	1.04(0.105)	1.01(0.101)
200	1000	1.10(0.114)	1.02(0.101)	1.00(0.101)

4. Real data example

Here we demonstrate that the shrinkage phenomenon appears in real data, and can be adjusted by our method. For this purpose, genetic data on samples from unrelated individuals in the Phase 3 HapMap study [http://hapmap.ncbi.nlm.nih.gov/] were used. HapMap is a dense genotyping study designed to elucidate population genetic differences. The genetic data are discrete, assuming the values 0, 1, or 2 at each genomic marker (also known as SNPs) for each individual. Data from CEU individuals (of northern and western European ancestry) were compared with data from TSI individuals (Toscani individuals from Italy, representing southern European ancestry).

Some initial data trimming steps are standard in genetic analysis. We first removed apparently related samples, and removed genomic markers with more than a 10% missing rate, and those with frequency less than 0.01 for the minor genetic allele. To avoid spurious PC results, we further pruned out SNPs that are in high linkage disequlibrium (LD) [11]. Lastly, we excluded 7 samples with PC scores greater than 6 standard deviations away from the mean of at least one of the top significant PCs (i.e., with Tracy-Widom (TW) Test p-value < 0.01) [24, 27]. The final dataset contained 178 samples (101 CEU, 77 TSI) and 100, 183 markers. We mean-centered and variance-standardized the genotypes for each marker [27]. The screeplot of the sample eigenvalues is presented in Figure 3. The first eigenvalue is substantially larger than the rest of the eigenvalues, although the TW test actually identifies two significant PCs. Figure 3 suggests that our data approximately satisfies the spiked eigenvalue assumption.

Fig 3.

Scree plot of the first 30 sample eigenvalues, CEU+TSI dataset

We estimated the asymptotic shrinkage factor and compared it with the following jackknife-based shrinkage factor estimate. For the first PC, we first computed the scores of all samples. Next, we removed one sample at a time and computed the (unadjusted) predicted PC score. We then calculated the jackknife estimate as the square root of the ratio of the means of the sample PC score and the predicted PC score. The jackknife shrinkage factor estimate is 0.319, which is close to our asymptotic estimate 0.325. Figure 4 shows the PC scores from the whole sample, the predicted PC score of an illustrative excluded sample, and its bias-adjusted predicted score. Clearly, the predicted PC score without adjustment is very biased towards zero, while the bias adjusted PC score is not.

Fig 4.

An instance with and without shrinkage adjustment, performed on Hapmap CEU(*) and TSI(+). “*” and “+” represent PC scores using all data. The 161^th sample was excluded from PCA, and PC score for it was predicted. The grey rectangle represents the predicted PC score without shrinkage adjustment and the grey circle represents the predicted PC score after the shrinkage adjustment

5. Discussion and conclusions

In this paper we have identified and explored the shrinkage phenomenon of the predicted PC scores, and have developed a novel method to adjust these quantities. We also have constructed the asymptotic estimator of correlation coefficient between PC scores from population eigenvectors and sample eigenvectors. In simulation experiments and real data analysis, we have demonstrated the accuracy of our estimates, and the capability to increase prediction accuracy in PC regression by adopting shrinkage bias adjustment. For achieving these, we consider asymptotics in the large p, large n framework, under the spiked population model.

We believe that this asymptotic regime applies well to many high dimensional datasets. It is not, however, the only model paradigm applied to such data. For example, the large p small n paradigm [1, 14], which assumes p/n → ∞, has also been explored. Under this assumption, Jung and Marron [20] have shown that the consistency and the strong inconsistency of the sample eigenvectors to population eigenvectors depend on whether p increases at a slower or faster rate than λ_v. It may be argued that for real data where p/n is “large,” we should follow the paradigm of Ahn et al. [1], Hall et al. [14]. However, for any real study, it is unclear how to test whether p increases at a faster rate than λ_v, or vice versa, making the application of Ahn et al. [1], Hall et al. [14] difficult in practice. Furthermore, the scenario where p and λ_v grow at the same rate is scientifically more interesting, for which we are aware of no theoretical results. In contrast, our asymptotic results can be straightforwardly applied. Further, our simulation results indicate that for p/n as large as 500, our asymptotic results still hold well. We believe that the approach we describe here applies to many datasets.

Although the results from the spiked model are useful, it is likely that observed data has more structure than allowed by the model. Recently, several methods have been suggested to estimate population eigenvalues under more general scenarios [10, 29]. However, no analogous results are available for the eigenvectors. In data analysis, jackknife estimators, as demonstrated in the real data analysis section, can be used. However, resampling approaches are very computationally intensive, and it remains of interest to establish the asymptotic behavior of eigenvectors in a variety of situations.

We note that inconsistency of the sample eigenvectors does not necessarily imply poor performance of PCA. For example, PCA has been successfully applied in genome-wide association studies for accurate estimation of ethnicity [27], and in PC regression for microarrays [21]. However, for any individual study we cannot rule out the possibility of poor performance of the PC analysis. Our asymptotic result on the correlation coefficient between PC scores from sample and population eigenvectors provides us a measure to quantify the performance of PC analysis.

For the CEU/TSI data, SNP pruning was applied to adjust for strong LD among adjacent SNPs. Such SNP pruning is a common practice in the analysis of GWAS data, and has been implemented in the popular GWAS analysis software Plink [28]. The primary goal of SNP pruning is to avoid spurious PC results unrelated to population substructures. Technically, our approach does not rely on any independence assumption of the SNPs. However, strong local correlation may affect eigenvalues considerably. Thus the value in SNP pruning may be viewed as helping the data better accord with the assumptions of the spiked population model. From the CEU/TSI data and our experience in other GWAS data, we have found that the most common pruning procedure implemented in Plink is sufficient for us to then apply our methods.

6. Proofs

Note that EΛ^1/2ZZ^T Λ^1/2E^T and Λ^1/2ZZ^T Λ^1/2 have the same eigenvalues, and E^T U is the eigenvector matrix of Λ^1/2ZZ^T Λ^1/2. Since eigenvalues and angles between sample and population eigenvectors are what we concerned about, without loss of generality (WLOG), in the sequel, we assume Λ to be the population covariance matrix.

6.1. Notations

We largely follow notations in Paul [26]. We denote λ_v(S) as the v^th largest eigenvalue of S. Let suffice A represent the first m coordinates and B represent the remaining coordinates. Then we can partition S into

$$\mathbf{S}=\left[\begin{array}{cc}{\mathbf{S}}_{AA}& {\mathbf{S}}_{AB}\\ {\mathbf{S}}_{BA}& {\mathbf{S}}_{BB}\end{array}\right]$$

We similarly partition the v^th eigenvector ${\mathbf{u}}_v^T$ into (u_A,v, u_B,v) and Z^T into [ ${\mathbf{Z}}_A^T,{\mathbf{Z}}_B^T$]. Define R_v as ||u_B,v|| and let ${\mathbf{a}}_v={\mathbf{u}}_{A,v}/\sqrt{1-R_v^2}$, then we get ||a_v|| = 1.

Applying singular value decomposition (SVD) to ${\mathbf{Z}}_B/\sqrt{n}$, we get

$$\frac{1}{\sqrt{n}}{\mathbf{Z}}_B={\mathbf{VM}}^{1/2}{\mathbf{H}}^T,$$ (9)

where M = diag(μ₁, …, μ_p−m) is a (p − m) × (p − m) diagonal matrix of ordered eigenvalues of S_BB, V is a (p − m) × (p − m) orthogonal matrix, and H is an n × (p − m) matrix. For n ≥ p − m, H has full rank orthogonal columns. When n < p−m, H has more columns than rows, hence it does not have full rank orthogonal columns. For the later case, we make H = [H_n, 0] where H_n is an n × n orthogonal matrix.

6.2. Propositions

We introduce two propositions for later use. The proofs of the 2 propositions can be found in section 6.5 and 6.6.

Proposition 1

Suppose Y is an n×m matrix with fixed m and each entry of Y is i.i.d random variable which satisfies the moment condition of z_ij in Assumption 1. Let C be an n × n symmetric non-negative definite random matrix and independent of Y. Further assume ||C|| = O(1). Then

$$\frac{1}{n}{\mathbf{Y}}^T\mathbf{CY}-\frac{1}{n}\mathit{trace}(\mathbf{C})\mathbf{I}\stackrel{p}{\to }0$$

as n → ∞

Proposition 2

Suppose y is an n dimensional random vector which follows the same distribution of the row vectors of Y and independent of S_BB. Let f(x) be a bounded continuous function on [ ${(1-\sqrt{\gamma })}^2,{(1+\sqrt{\gamma })}^2$] and f(0) = 0. Suppose F = diag(f(μ₁), …, f(μ_p−m)), where ${\{{\mu }_i\}}_{i=1}^{p-m}$ are ordered eigenvalues of M which is defined on (9), then

$$\frac{1}{n}{\mathbf{y}}^T{\mathbf{HFH}}^T\mathbf{y}-\gamma \int f(x){dF}_{\gamma }(x)\stackrel{p}{\to }0$$

as n → ∞, where F_γ(x) is a distribution function of Marchenko-Pastur law with parameter γ [22].

6.3. Proof of Part i) of Lemma 1)

6.3.1. When p is fixed

By the strong law of large numbers, $\mathbf{S}\stackrel{a.s}{\to }\mathbf{\Lambda }$. Since eigenvalues are continuous with respect to the operator norm, the lemma follows after applying continuous mapping theorem.

6.3.2. When p → ∞

For every small ε > 0, there exist p̃(n) and γ_ε such that p̃(n)/n → γ_ε > 0, λ_v(1 + γ_ε/(λ_v − 1)) < λ_v + ε for all v ≤ m, ${(1+\sqrt{{\gamma }_{\epsilon }})}^2<1+\epsilon$, and ${(1-\sqrt{{\gamma }_{\epsilon }})}^2>1-\epsilon$. For simplicity, we denote p̃(n) as p̃. Suppose Z_p̃ is a p̃ × n matrix that satisfies the moment condition of z_ij in Assumption 1. Define an augmented data matrix ${\stackrel{\sim }{\mathbf{X}}}^T={[{\mathbf{Z}}^T\mathbf{\Lambda },{\mathbf{Z}}_{\stackrel{\sim }{p}}^T]}^T$ and its sample covariance matrix S̃ = X̃X̃^T. Let S be a p × p upper left submatrix of S̃. We also let Ŝ be an (m + 1) × (m + 1) upper left submatrix of S̃. For v ≤ (m + 1), by the interlacing inequality (Theorem 4.3.15 of Horn and Johnson [15]),

$${\lambda }_v(\widehat{\mathbf{S}})\le {\lambda }_v(\mathbf{S})\le {\lambda }_v(\stackrel{\sim }{\mathbf{S}}).$$

Since ${\lambda }_v(\widehat{\mathbf{S}})\stackrel{a.s}{\to }{\lambda }_v,{\lambda }_v(\stackrel{\sim }{\mathbf{S}})\stackrel{a.s}{\to }{\lambda }_v(1+{\gamma }_{\epsilon }/({\lambda }_v-1))<1+\epsilon$ for v ≤ m, and ${\lambda }_v(\stackrel{\sim }{\mathbf{S}})\stackrel{a.s}{\to }{(1+\sqrt{{\gamma }_{\epsilon }})}^2<1+\epsilon$ for v = m + 1, we have

$${\lambda }_v-o(1)\le {\lambda }_v(\mathbf{S})<{\lambda }_v+\epsilon +o(1),\text{for}v\le m+1.$$

Thus,

$${\lambda }_v(\mathbf{S})\stackrel{a.s}{\to }{\lambda }_v,\text{for}v\le m+1.$$ (10)

Similarly by the interlacing inequality, we get

$${\lambda }_{\stackrel{\sim }{p}}(\stackrel{\sim }{S})\le {\lambda }_p(S){\le }_{m+1}(S).$$

Since ${\lambda }_{m+1}(S)\stackrel{a.s}{\to }1$, and ${\lambda }_{\stackrel{\sim }{p}}(\stackrel{\sim }{S})\stackrel{a.s}{\to }{(1-\sqrt{{\gamma }_{\epsilon }})}^2>1-\epsilon$, we conclude that

$${\lambda }_p(S)\stackrel{a.s}{\to }1.$$ (11)

The part i) of Lemma 1 follows by (10) and (11)

6.4. Proof of Part ii) of Lemma 2

Our proof of Lemma 2 (ii) closely follows the arguments in Paul [26]. From [26], it can be shown that

$$({\mathbf{S}}_{AA}+\frac{1}{n}{\mathbf{\Lambda }}_A^{1/2}{\mathbf{Z}}_A\mathbf{HM}{(d_v\mathbf{I}-\mathbf{M})}^{-1}{\mathbf{H}}^T{\mathbf{Z}}_A^T{\mathbf{\Lambda }}_A^{1/2}){\mathbf{a}}_v=d_v{\mathbf{a}}_v$$ (12)

and

$${\mathbf{a}}_v^T(\mathbf{I}+\frac{1}{n}{\mathbf{\Lambda }}_A^{1/2}{\mathbf{Z}}_A\mathbf{HM}{(d_v\mathbf{I}-\mathbf{M})}^{-2}{\mathbf{H}}^T{\mathbf{Z}}_A^T{\mathbf{\Lambda }}_A^{1/2}){\mathbf{a}}_v=\frac{1}{1-R_v^2}$$ (13)

where Λ_A = diag {λ₁, …, λ_m}.

6.4.1. When ${\lambda }_v>1+\sqrt{\gamma }$

We can show that

$$\langle {\mathbf{a}}_v,{\mathbf{e}}_{A,v}\rangle \stackrel{p}{\to }1$$ (14)

and

$$\frac{1}{n}{\mathbf{z}}_{Av}^T\mathbf{HM}{(d_v\mathbf{I}-\mathbf{M})}^{-2}{\mathbf{H}}^T{\mathbf{z}}_{Av}\stackrel{p}{\to }\{\begin{array}{ccc}\gamma \int \frac{x}{{({\rho }_v-x)}^2}{dF}_{\gamma }(x)& \text{for}& \gamma >0\\ 0& \text{for}& \gamma =0,\end{array}$$ (15)

where e_A,v is a vector of the first m coordinates of the v^th population eigen-vector e_v, ρ_v is ${\lambda }_v\left(1+\frac{\gamma }{{\lambda }_v-1}\right)$, and z_Av is a vector of v^th row of Z_A. The proofs can be found in 6.4.3. Note that e_v is a vector with 1 in its vth coordinate and 0 elsewhere. WLOG, we assume that 〈e_v, u_v〉 ≥ 0. Since $\langle {\mathbf{e}}_v,{\mathbf{u}}_v\rangle =\sqrt{1-R_v^2}\langle {\mathbf{e}}_{A,v},{\mathbf{a}}_v\rangle ,\langle {\mathbf{e}}_v,{\mathbf{u}}_v\rangle \stackrel{p}{\to }\sqrt{1-R_v^2}$. By (13) and (15), we can show that

$$\frac{1}{1-R_v^2}\stackrel{p}{\to }\{\begin{array}{ccc}1+{\lambda }_v\gamma \int \frac{x}{{({\rho }_v-x)}^2}{dF}_{\gamma }(x)& \text{for}& \gamma >0\\ 0& \text{for}& \gamma =0.\end{array}$$ (16)

From Lemma B.2 of [26],

$$\int \frac{x}{{({\rho }_v-x)}^2}{dF}_{\gamma }(x)=\frac{1}{{({\lambda }_v-1)}^2-\gamma }.$$ (17)

Thus

$$\sqrt{1-R_v^2}\stackrel{p}{\to }\{\begin{array}{ccc}\sqrt{(1-\frac{\gamma }{{({\lambda }_v-1)}^2})/(1+\frac{\gamma }{{\lambda }_v-1})}& \text{for}& \gamma >0\\ 0& \text{for}& \gamma =0.\end{array}$$ (18)

It concludes the proof of the first part of Lemma 2 ii).

6.4.2. When $1<{\lambda }_v\le 1+\sqrt{\gamma }$

Here we only need to consider γ > 0 because no eigenvalue satisfies this condition when γ = 0. We first show that $R_v\stackrel{p}{\to }1$, which implies ${\mathbf{u}}_{A,v}\stackrel{p}{\to }0$, hence $\langle {\mathbf{e}}_v,{\mathbf{u}}_v\rangle \stackrel{p}{\to }0$. For any ε > 0 and x ≥ 0, define

$${(x)}_{\epsilon }=\{\begin{array}{cc}x& \text{if}x>\epsilon \\ \epsilon & \text{if}x\le \epsilon \end{array}$$

and

$${\mathbf{G}}_{\epsilon }=\text{diag}(d_v/{({(d_v-{\mu }_1)}^2)}_{\epsilon },\dots ,d_v/{({(d_v-{\mu }_{p-m})}^2)}_{\epsilon }),$$

then by Propositions 1 and 2,

$$\frac{1}{n}{\mathbf{z}}_{Av}^T\mathbf{H}{\mathbf{G}}_{\epsilon }{\mathbf{H}}^T{\mathbf{z}}_{Av}\stackrel{p}{\to }\gamma \int \frac{x}{{({({\rho }_v-x)}^2)}_{\epsilon }}{dF}_{\gamma }(x)$$ (19)

By monotone convergence theorem,

$$\gamma \int \frac{x}{{({({\rho }_v-x)}^2)}_{\epsilon }}{dF}_{\gamma }(x)\stackrel{\epsilon \to 0}{\to }\gamma \int \frac{x}{{({\rho }_v-x)}^2}{dF}_{\gamma }(x)$$ (20)

RHS of (20) is

$${\int }_a^b\frac{\sqrt{(b-x)(x-a)}}{2\pi {(p_v-x)}^2}dx$$ (21)

where $a={(1-\sqrt{\gamma })}^2$ and $b={(1+\sqrt{\gamma })}^2$. Since (21) equals ∞ for any a ≤ ρ_v ≤ b, we conclude that

$$\frac{1}{n}{\mathbf{z}}_{Av}^T\mathbf{HM}{(d_v\mathbf{I}-\mathbf{M})}^{-2}{\mathbf{H}}^T{\mathbf{z}}_{Av}\stackrel{p}{\to }\infty$$ (22)

Therefore $R_v\stackrel{p}{\to }1$, which proves the second part of Lemma 2 ii).

6.4.3. Proof of (14) and (15)

Define ${\mathcal{R}}_v={\displaystyle \sum _{k\ne v}^m}\frac{{\lambda }_v}{{\rho }_v({\lambda }_k-{\lambda }_v)}{\mathbf{e}}_{A,k}{\mathbf{e}}_{A,k}^T$, = S_AA+S_AB(d_vI−S_BB)⁻¹ S_BA −(ρ_v/λ_v)Λ_A, α_v = |||| + |d_v − ρ_v|||||, and β_v = ||e_A,v||.

With the exactly same argument of [26], it can be shown that

$${\mathbf{a}}_v-{\mathbf{e}}_{A,v}=-{\mathcal{R}}_v{\mathcal{D}}_v{\mathbf{e}}_{A,v}+{\mathbf{r}}_v$$

where r_v = − (1 − 〈e_A,v, a_v〉)e_A,v − (a_v − e_A,v)+(d_v − ρ_v)(a_v − e_A,v). By Lemma 1 of [25], r_v = o_p(1), if α_v = o_p(1) and β_v = o_p(1).

When γ = 0, ${\mathbf{S}}_{AA}-({\rho }_v/{\lambda }_v){\mathbf{\Lambda }}_A\stackrel{p}{\to }0$ and the remainder of is

$${\mathbf{S}}_{AB}{(d_v\mathbf{I}-{\mathbf{S}}_{BB})}^{-1}{\mathbf{S}}_{BA}=\frac{1}{n}{\mathbf{\Lambda }}_A^{1/2}{\mathbf{Z}}_A\mathbf{HM}{(d_v\mathbf{I}-\mathbf{M})}^{-1}{\mathbf{H}}^T{\mathbf{Z}}_A^T{\mathbf{\Lambda }}_A^{1/2}.$$ (23)

Since $d_v\stackrel{a.s}{\to }{\lambda }_v$ and ${\mu }_1\stackrel{a.s}{\to }1$,

$$\left|\right|\mathbf{HM}{(d_v\mathbf{I}-\mathbf{M})}^{-1}{\mathbf{H}}^T\left|\right|\stackrel{a.s}{\to }1/({\lambda }_v-1)$$

By Proposition 1,

$$0\le (23)\le {\lambda }_1\frac{p{\mu }_1}{n(d_v-{\lambda }_1)}+o_p(1)=o_p(1),$$ (24)

hence = o_p(1).

When γ > 0, can be written as

$${\mathcal{D}}_v=[{\mathbf{S}}_{AA}-{\mathbf{\Lambda }}_A]+[{\mathbf{\Lambda }}_A^{1/2}(\frac{1}{n}{\mathbf{Z}}_A\mathbf{HM}{({\rho }_v\mathbf{I}-\mathbf{M})}^{-1}{\mathbf{H}}^T{\mathbf{Z}}_A-\frac{1}{n}\mathit{trace}(\mathbf{M}{({\rho }_v\mathbf{I}-\mathbf{M})}^{-1})\mathbf{I}){\mathbf{\Lambda }}_A^{1/2}]+[(\frac{1}{n}\mathit{trace}(\mathbf{M}{({\rho }_v\mathbf{I}-\mathbf{M})}^{-1})-\gamma \int \frac{x}{{\rho }_v-x}{dF}_{\gamma }(x)){\mathbf{\Lambda }}_A]+[({\rho }_v-d_v)\frac{1}{n}{\mathbf{\Lambda }}_A^{1/2}{\mathbf{Z}}_A\mathbf{HM}{({\rho }_v\mathbf{I}-\mathbf{M})}^{-1}{(d_v\mathbf{I}-\mathbf{M})}^{-1}{\mathbf{H}}^T{\mathbf{Z}}_A{\mathbf{\Lambda }}_A^{1/2}]$$ (25)

The first term of RHS is o_p(1) by the weak law of large number. The second and third terms are o_p(1) by Propositions 1 and 2. For the fourth term, ρ_v − d_v = o_p(1) and its remainder part is O_p(1). Therefore, = o_p(1). By combining the above results and = O_p(1) plus d_v − ρ_v = o_p(1), we prove the Equation (14).

For (15): When γ = 0, (15) can be proved by the exactly same way used to show (24). When γ > 0, $d_v\stackrel{a.s}{\to }{\rho }_v$, and ${\mu }_1\stackrel{a.s}{\to }{(1+\sqrt{\gamma })}^2<{\rho }_v$, hence $\left|\right|\mathbf{C}\left|\right|\stackrel{a.s}{\to }\frac{{(1+\sqrt{\gamma })}^2}{{({\rho }_v-{(1+\sqrt{\gamma })}^2)}^2}$. Therefore the result follows according to Propositions 1 and 2.

6.5. Proof of Proposition 1

Let μ₁ ≥ μ₂ ≥ ··· ≥ μ_n be the ordered eigenvalues of C, and c_ij be the (i, j)th element of C. Suppose y_s is the sth column of Y, and y_ij is the (i, j)th element of Y. We further define $\psi (s,s)=\frac{1}{n}{\mathbf{y}}_s^T\mathbf{C}{\mathbf{y}}_s-\frac{1}{n}\mathit{trace}(\mathbf{C})$ and $\psi (s,t)=\frac{1}{n}{\mathbf{y}}_s^T\mathbf{C}{\mathbf{y}}_t$ for s ≠ t. The conditional mean of ψ(s, s) given C is

$$\begin{array}{l}E(\psi (s,s)\mid \mathbf{C})=E(\frac{1}{n}\sum _{i,j}{c}_{ij}{y}_{is}{y}_{js}\mid \mathbf{C})-\frac{1}{n}\sum _{i=1}^n{\mu }_i\\ =\frac{1}{n}\sum _{i=1}^n{c}_{ii}E(y_{is}^2)+\frac{2}{n}\sum _{i<j}^n{c}_{ij}E(y_{is}{y}_{js})-\frac{1}{n}\sum _{i=1}^n{\mu }_i\\ =\frac{1}{n}\sum _{i=1}^n{c}_{ii}-\frac{1}{n}\sum _{i=1}^n{\mu }_i=0\end{array}$$ (26)

Thus, E(ψ(s, s)) = E(E(ψ(s, s)|C)) = E(0) = 0.

Next, the conditional variance of ψ(s, s) given C is

$$\begin{array}{l}\mathit{Var}(\psi (s,s)\mid \mathbf{C})=\frac{1}{n^2}\mathit{Var}(\sum _{i,j}{c}_{ij}{y}_{is}{y}_{js}\mid \mathbf{C})\\ =\frac{1}{n^2}\sum _{i,j,l,q=1}^n{c}_{ij}{c}_{lq}\mathit{Cov}(y_{is}{y}_{js},y_{ls}{y}_{qs})\\ =\frac{4}{n^2}\sum _{i,j=1}^n{c}_{ij}^2\mathit{Var}(y_{is}{y}_{js})\\ \le \frac{4\alpha }{n^2}\sum _{i,j=1}^n{c}_{ij}^2=\frac{4\alpha }{n^2}\mathit{trace}({\mathbf{C}}^2)=\frac{4\alpha }{n^2}\sum _{i=1}^n{\mu }_i^2\end{array}$$ (27)

where $\alpha =\mathit{max}(1,E(y_{is}^4)-1)$. Since ||C|| = O(1), ${\mu }_i^2\le {\left|\right|\mathbf{C}\left|\right|}^2=O(1)$. Therefore, Var(ψ(s, s)|C) ≤ O(1/n) and Var(ψ(s, s)) = Var(E(ψ(s, s)|C)) + E(Var(ψ(s, s)|C)) ≤ 0 + O(1/n) → 0 as n → ∞. By Chebyshev inequality, we can conclude that

$$\psi (s,s)\stackrel{p}{\to }0.$$

We can similarly show $\psi (s,t)\stackrel{p}{\to }0$, which we omit here.

6.6. Proof of Proposition 2

Consider an expansion

$$\begin{array}{l}\frac{1}{n}{y}^T{\mathbf{HFH}}^{T}y-\gamma \int f(x){dF}_{\gamma }(x)\\ =[\frac{1}{n}{y}^T{\mathbf{HFH}}^{T}y-\frac{1}{n}\mathit{trace}(\mathbf{F})]\\ +[\frac{1}{n}\mathit{trace}(\mathbf{F})-\gamma \int f(x){dF}_{\gamma }(x)]\\ =(a)+(b)\end{array}$$

We show that both (a) and (b) converge to 0 in probability.

1. Since ${\mu }_1\stackrel{a.s.}{\to }{(1+\sqrt{\gamma })}^2,{\mu }_{\mathit{min}(p-m,n)}\stackrel{a.s.}{\to }{(1-\sqrt{\gamma })}^2$, μ_k = 0 for k > min(p−m, n), and f(x) is continuous and bounded on [ ${(1-\sqrt{\gamma })}^2,{(1+\sqrt{\gamma })}^2$], there exists K > 0 such that sup_i |f (μ_i)| < K a.s. Let C = HFH^T, then trace(C) = trace(F). By Proposition 1, (a) = o_p(1).

2. Let F_p−m be an empirical spectral distribution of S_BB, then

   $$\frac{1}{n}\mathit{trace}(\mathbf{F})=\frac{p-m}{n}\int f(x){dF}_{p-m}(x),$$
   and $\int f(x){dF}_n(x)\stackrel{p}{\to }\int f(x){dF}_n(x)$ [5, 22]. Thus

   $$\frac{p-m}{n}\int f(x){dF}_{p-m}(x)\stackrel{p}{\to }\gamma \int f(x){dF}_{\gamma }(x),$$

   which shows that (b) = o_p(1).

Combining (a) and (b), we finish the proof.

6.7. Proof of Theorem 1

WLOG we assume 〈g_v, p̃_v〉 ≥ 0. Let e_v = {e_A,v, e_B,v}, then e_A,v is the vector with 1 in v^th coordinate and 0 elsewhere, and e_B,v is the zero vector. Since S_AAu_A,v + S_ABu_B,v = d_vu_A,v, we have

$$\begin{array}{l}<{\mathbf{g}}_v,{\stackrel{\sim }{\mathbf{p}}}_v>=\frac{1}{n\sqrt{d_v{\lambda }_v}}{\mathbf{e}}_v^T{\mathbf{XX}}^T{\mathbf{u}}_v\\ ={\mathbf{e}}_{A,v}^T{\mathbf{S}}_{AA}{\mathbf{u}}_{A,v}/\sqrt{d_v{\lambda }_v}+{\mathbf{e}}_{A,v}^T{\mathbf{S}}_{AB}{\mathbf{u}}_{B,v}/\sqrt{d_v{\lambda }_v}\\ =\frac{d_v}{\sqrt{d_v{\lambda }_v}}{\mathbf{e}}_{A,v}^T,{\mathbf{u}}_{A,v}=\sqrt{\frac{d_v}{{\lambda }_v}}{\mathbf{e}}_v^T{\mathbf{u}}_v\stackrel{p}{\to }\{\begin{array}{lll}\sqrt{(1-\frac{\gamma }{{({\lambda }_v-1)}^2})}\hfill & \text{for}\hfill & {\lambda }_v>1+\sqrt{\gamma }\hfill \\ \hfill 0\hfill & \text{for}\hfill & 1<{\lambda }_v\le 1+\sqrt{\gamma }.\hfill \end{array}\end{array}$$ (28)

6.8. Proof of Theorem 2

First, we show the square of the denominator converges to ρ(λ_v). Since $p_{vj}={\mathbf{u}}_v^T{\mathbf{x}}_j$, and $E(p_{vi}^2)=E(p_{vj}^2)$ for i ≠ j,

$$\begin{array}{l}E(p_{vj}^2)=\frac{1}{n}E(\sum _{j=1}^n{p}_{vj}^2)=\frac{1}{n}E(\sum _{j=1}^n{({\mathbf{u}}_v^T{\mathbf{x}}_j)}^2)\\ =E({\mathbf{u}}_v^T{\mathbf{XX}}^T{\mathbf{u}}_v/n)=E(d_v)\stackrel{a.s}{\to }\rho ({\lambda }_v)\end{array}$$ (29)

Next we show the square of numerator converges to φ(λ_v)²(λ_v − 1)+1. Define ${\mathbf{u}}_v^{\perp }:=\frac{1}{\sqrt{1-{({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2}}(I-{\mathbf{e}}_v{\mathbf{e}}_v^T){\mathbf{u}}_v$, then u_v can be expressed as

$${\mathbf{u}}_v=({\mathbf{u}}_v^T{\mathbf{e}}_v){\mathbf{e}}_v+\sqrt{1-{({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2}{\mathbf{u}}_v^{\perp }$$

Partition ${\mathbf{u}}_v^{\perp }=\left\{{\mathbf{u}}_{A,v}^{\perp },{\mathbf{u}}_{B,v}^{\perp }\right\}$. From (14), ${\mathbf{a}}_v\stackrel{p}{\to }{\mathbf{e}}_{A,v}$, therefore ${\mathbf{u}}_{A,v}^{\perp }\stackrel{p}{\to }0$ and ${\mathbf{u}}_{B,v}^{\perp T}{\mathbf{u}}_{B,v}^{\perp }\stackrel{p}{\to }1$ Since x_new and u_v are independent, we have

$$\begin{array}{l}E(q_v^2\mid {\mathbf{u}}_v)=E({({\mathbf{u}}_v^T{\mathbf{x}}_{\mathit{new}})}^2\mid {\mathbf{u}}_v)={\mathbf{u}}_v^{T}E({\mathbf{x}}_{\mathit{new}}{\mathbf{x}}_{\mathit{new}}^T\mid {\mathbf{u}}_v){\mathbf{u}}_v={\mathbf{u}}_v^T\mathrm{\Lambda }{\mathbf{u}}_v\\ ={({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2{\mathbf{e}}_v^T\mathrm{\Lambda }{\mathbf{e}}_v+(1-{({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2){\mathbf{u}}_v^{\perp T}\mathrm{\Lambda }{\mathbf{u}}_v^{\perp }+2{\mathbf{u}}_v^T{\mathbf{e}}_v\sqrt{1-{({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2}{\mathbf{e}}_v^T\mathrm{\Lambda }{\mathbf{u}}_v^{\perp }\\ ={({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2{\lambda }_v+(1-{({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2)({\mathbf{u}}_{A,v}^{\perp T}{\mathrm{\Lambda }}_A{\mathbf{u}}_{A,v}^{\perp }+{\mathbf{u}}_{B,v}^{\perp T}{\mathbf{u}}_{B,v}^{\perp })+2{\mathbf{u}}_v^T{\mathbf{e}}_v\sqrt{1-{({\mathbf{u}}_v^T{\mathbf{e}}_v)}^2}{\mathbf{e}}_{A,v}{\mathrm{\Lambda }}_A{\mathbf{u}}_{A,v}^{\perp }\stackrel{p}{\to }\phi {({\lambda }_v)}^2({\lambda }_v-1)+1.\end{array}$$ (30)

From (29) and (30),

$$\sqrt{\frac{E(q_v^2)}{E(p_{vi}^2)}}\to \sqrt{\frac{\phi {({\lambda }_v)}^2({\lambda }_v-1)+1}{\rho ({\lambda }_v)}}=\frac{({\lambda }_v-1)}{({\lambda }_v+\gamma -1)}.$$ (31)

6.9. Proof of Theorem 3

Since ρ⁻¹(pr_v) → λ_v for v ≤ k, WLOG we assume that k₀ = k, where k is the number of λ_v bigger than $1+\sqrt{\gamma }$. Set

$$h(x)=\sum _{v=1}^k{\rho }^{-1}(r_{v}x)+p-k-x$$ (32)

The first and second partial derivatives of h(x) are

$$\frac{\partial h(x)}{\partial x}=\frac{1}{2}\sum _{v=1}^k{r}_v+\frac{1}{2}\sum _{v=1}^k\frac{({xr}_v-(1+\gamma ))r_v}{\sqrt{{({xr}_v-(1+\gamma ))}^2-4\gamma }}-1$$ (33)

$$\frac{{\partial }^{2}h(x)}{\partial x^2}=2\sum _{v=1}^k\frac{-r_v^2\gamma }{{({({xr}_v-(1+\gamma ))}^2-4\gamma )}^{3/2}}<0,$$ (34)

so h(x) is a concave function of x given r_v. From the fact that ρ⁻¹(r_vp) > 1 for v ≤ k, we know h(p) > 0. Because of the concave nature of this function, h(x) = 0 has an unique solution τ on [p, ∞), which ${\displaystyle \sum _{v=1}^{k_l}}{\widehat{\lambda }}_{v,l}+p-m_l$ converges to. Thus d̂_v = τr_v. Define d̃_v = r_vω where $\omega ={\displaystyle \sum _{v=1}^k}{\lambda }_v+p-k$, and set d_v as the sample eigenvalue when σ² = 1. The sum of all d_v is

$$\sum _{v=1}^p{d}_v=\frac{1}{n}\mathit{trace}({ZZ}^T\mathrm{\Lambda })=\frac{1}{n}\sum _{i=1}^p\sum _{j=1}^n{\lambda }_i{z}_{ij}^2,$$ (35)

thus

$$E\left(\frac{{\displaystyle \sum _{v=1}^p}{d}_v}{\omega }\right)=\frac{{\displaystyle \sum _{v=1}^m}{\lambda }_v+p-k}{\omega }\to 1,\text{and}$$ (36)

and

$$\mathit{Var}\left(\frac{{\displaystyle \sum _{v=1}^p}{d}_v}{\omega }\right)=\frac{1}{n}\frac{{\displaystyle \sum _{v=1}^p}{\lambda }_v^2}{{\omega }^2}(E(z_{11}^4)-1)\to 0.$$ (37)

By (36) and (37)

$$\sum _{v=1}^p{d}_v/\omega =1+o_p(1).$$ (38)

Since d_v → ρ(λ_v) for v ≤ k,

$${\stackrel{\sim }{d}}_v=d_v\omega /\sum _{v=1}^p{d}_v=d_v(1+o_p(1))\stackrel{p}{\to }\rho ({\lambda }_v).$$ (39)

Now, we show that τ = ω + o_p(1). Plugging ω into h(x) and combining the fact that ρ⁻¹(d̃_v) = λ_v + o_p(1), we get

$$h(\omega )=\sum _{v=1}^k{\rho }^{-1}({\stackrel{\sim }{d}}_v)-\sum _{v=1}^k{\lambda }_v=o_p(1).$$ (40)

From the facts that h(x) is a continuous concave function, ω > p, and h(p) > 0, we conclude that

$$\omega =\tau +o_p(1).$$ (41)

Therefore

$${\widehat{d}}_v=r_v\tau =r_v(\omega +op(1))={\stackrel{\sim }{d}}_v+op(1)\stackrel{p}{\to }\rho ({\lambda }_v)$$ (42)

for v ≤ k, which concludes the proof.