Simultaneous dimension reduction and adjustment for confounding variation

Significance

With the advancement in high-throughput technologies, analyzing high-dimensional data has become a common task. Dimension reduction methods have been applied to visualize and identify dominant patterns in high-dimensional data. Confounding factors, commonly observed in high-throughput biological experiments, can affect the performance of these methods, and other downstream analysis. Here, we develop a method by coupling dimension reduction with the adjustment for confounder effects. Our method is able to capture the underlying patterns, as demonstrated by a human brain exon array dataset, a model organism ENCODE RNA sequencing dataset, and simulations.

Abstract

Dimension reduction methods are commonly applied to high-throughput biological datasets. However, the results can be hindered by confounding factors, either biological or technical in origin. In this study, we extend principal component analysis (PCA) to propose AC-PCA for simultaneous dimension reduction and adjustment for confounding (AC) variation. We show that AC-PCA can adjust for (i) variations across individual donors present in a human brain exon array dataset and (ii) variations of different species in a model organism ENCODE RNA sequencing dataset. Our approach is able to recover the anatomical structure of neocortical regions and to capture the shared variation among species during embryonic development. For gene selection purposes, we extend AC-PCA with sparsity constraints and propose and implement an efficient algorithm. The methods developed in this paper can also be applied to more general settings. The R package and MATLAB source code are available at https://github.com/linzx06/AC-PCA.

Dimension reduction methods, such as multidimensional scaling (MDS) and principal component analysis (PCA), are commonly applied in high-throughput biological datasets to visualize data in a low-dimensional space, identify dominant patterns, and extract relevant features (1–6). MDS aims to place each sample in a lower-dimensional space such that the between-sample distances are preserved as much as possible (7). PCA seeks the linear combinations of the original variables such that the derived variables capture maximal variance (8). One advantage of PCA is that the principal components (PCs) are more interpretable by checking the loadings of the variables.

Confounding factors, either biological or technical in origin, are commonly observed in high-throughput biological experiments. Various methods have been proposed to estimate the confounding variation, for example, regression models on known confounding factors (9) and factor models and surrogate vector analysis for unobserved confounding factors (10–15). However, limited work has been done in the context of dimension reduction. Confounding variation can affect PC-based visualization of the data points because it may obscure the desired biological variation, and it can also affect the loading of the variables in the PCs.

Here we extend PCA to propose AC-PCA for simultaneous dimension reduction and adjustment for confounding (AC) variation. We introduce a class of penalty functions in PCA, which encourages the PCs to be invariant to the confounding variation. We demonstrate the performance of AC-PCA through its application to a human brain development exon array dataset (4), a model organism ENCODE (modENCODE) RNA sequencing (RNA-Seq) dataset (16, 17), and simulated data. We also implemented AC-PCA with sparsity constraints to enable variable/gene selection and better interpretation of the PCs.

Results

AC-PCA in a General Form.

Let $X$ denote the $N\times p$ data matrix, where $N$ is the number of observations and $p$ is the number of variables/genes. $X$ is centered by column. Let $x_{(i)}$ denote the $i$th observation. Let $v$ denote a $p$-dimensional vector and $t_i=x_{(i)}\cdot v$ denote the projection induced by $v$. ${\sum }_{i=1}^N{t}_i^2={\sum }_{i=1}^N{(x_{(i)}\cdot v)}^2=v^T{X}^{T}Xv$ is proportional to the total variation after the projection and classical PCA seeks $v$ that maximizes it. The dimension extracted this way can be misleading if there is confounding variation (Results, A Motivating Example). Let $Y$ denote a $N\times l$ confounder matrix, representing $l$ confounders, and let $K=Y{Y}^T$. $Y$ is centered by column. We choose $Y$ so that $v^T{X}^{T}KXv$ represent the confounding variation in $t$. Because we are not interested in the subspace exhibiting the confounding variation, this suggests the following modification of PCA:

$$\begin{array}{c}\underset{v\in {ℝ}^p}{\text{maximize}\hspace{0.17em}}{v}^T{X}^{T}Xv-\lambda v^T{X}^{T}KXv\hfill \\ \text{subject to}\hspace{0.17em}{||v||}_2^2\le 1,\hfill \end{array}$$ [1]

where the tuning parameter $\lambda \ge 0$ controls the strength of regularization. If $\lambda =0$, this is classical PCA; when $\lambda$ is large enough, we are restricting the subspace to be orthogonal to the columns in $Y$. Denote $Z=X^{T}X-\lambda X^{T}KX$ and problem 1 can be solved directly by implementing eigendecomposition on $Z$. The confounder matrix $Y$ is user-defined, depending on the data structure and the assumptions on the confounding variation. We provide several examples below on how to choose $Y$. Methods for choosing $\lambda$ and for assessing the statistical significance of the extracted dimensions are presented in Materials and Methods.

In PCA, the loadings for the variables are typically nonzero. For better interpretation of the PCs, a sparse solution for $v$ can be achieved by adding ${\mathrm{\ell }}_1$ constraint:

$$\begin{array}{c}\underset{v\in {ℝ}^p}{\text{maximize}\hspace{0.17em}}{v}^T{X}^{T}Xv\hfill \\ \text{subject to}\hspace{0.17em}{v}^T{X}^{T}KXv\le c_1,{||v||}_1\le c_2,{||v||}_2^2\le 1,\hfill \end{array}$$ [2]

where $c_1$ is a constant depending on $\lambda$ and $c_2$ is the sparsity parameter.

A Motivating Example.

To motivate the problem, we first conducted PCA on a subset of samples from the human brain exon array dataset (4) (Fig. 1). The samples come from 10 brain regions in six donors and each donor tends to form a cluster. There are no clear patterns among the regions in the first 20 PCs (SI Appendix, Fig. S1). The variation of individual donors makes it challenging to extract the variation across brain regions. This dataset will be discussed in more detail later.

Fig. 1.

An example from the human brain exon array data, window 5. The labels represent the brain regions. Abbreviations for the brain regions are medial prefrontal cortex [MFC (F)], orbital prefrontal cortex [OFC (O)], ventrolateral prefrontal cortex [VFC (V)], dorsolateral prefrontal cortex [DFC (D)], primary motor cortex [M1C (M)], primary somatosensory cortex [S1C (S)], posterior inferior parietal cortex [IPC (P)], primary auditory cortex [A1C (A)], posterior superior temporal cortex [STC (T)], and inferior temporal cortex [ITC (I)]. Each color represents a donor. In some donors, samples from the left and/or right hemispheres are available. Samples in the right hemisphere are labeled in bold and italic.

For simplicity, we assume that there are no missing data. Let $X^{(i)}$ represent the $b\times p$ matrix for the gene expression levels of donor $i$, where $b$ is the number of brain regions and $p$ is the number of genes. By stacking the rows of $X^{(1)},\cdots ,X^{(n)}$, we obtain the $N\times p$ data matrix $X$, where $N=n\times b$. We propose the following objective function to adjust for the variation of donors:

$$\begin{array}{c}\underset{v\in {ℝ}^p}{\text{maximize}\hspace{0.17em}}{v}^T{X}^{T}Xv-{\displaystyle \frac{\lambda }{n}}{\displaystyle \sum _{i=1}^{n-1}}{\displaystyle \sum _{j=i+1}^n}{v}^T{(X^{(j)}-X^{(i)})}^T\times (X^{(j)}-X^{(i)})v\hfill \\ \text{subject to}\hspace{0.17em}{||v||}_2^2\le 1.\hfill \end{array}$$ [3]

Formula 3 is a special case of formula 1 (SI Appendix). The penalty term in formula 3 encourages the projection of the same region across donors to be similar. There are missing samples in the brain data and formula 3 can be modified to handle missing values (SI Appendix). More implementation details are discussed in Materials and Methods. The penalty can artificially induce the appearance of clusters. We provide guidelines on how to detect overcorrection in SI Appendix.

The Analysis of Variance Interpretation.

For a number of observations, assume that the samples can be divided into $K$ groups. $t_{jk}$ denotes the projection (induced by $v$) for the $j$th observation in group $k$, $j=1,\cdots ,n_k$. Let ${\overline{t}}_{\cdot k}$ and ${\overline{t}}_{\cdot \cdot }$ denote the group mean and grand mean, correspondingly. The total sum of squares $S{S}_T={\sum }_{k=1}^K{\sum }_{j=1}^{n_k}{(t_{jk}-{\overline{t}}_{\cdot \cdot })}^2$, the between-groups sum of squares $S{S}_B={\sum }_{k=1}^K{n}_k{({\overline{t}}_{\cdot k}-{\overline{t}}_{\cdot \cdot })}^2$, and the remaining sum of squares is represented by $S{S}_R$. We have $S{S}_T=S{S}_B+S{S}_R$. Consider the samples for a region $r$ in the brain data. Let the donor labels represent the groups, so we have $K=n$ and $n_k=1$, $\forall k$, because there are no replicates for a donor. Dropping the notation $j$, $S{S}_B^{(r)}={\sum }_{k=1}^n{(t_{rk}-{\overline{t}}_{r\cdot })}^2$. It can be shown that $nS{S}_B^{(r)}={\sum }_{k=1}^{n-1}{\sum }_{l=k+1}^n{(t_{rk}-t_{rl})}^2$. For samples in all of the regions, $S{S}_T={\sum }_{r=1}^b{\sum }_{k=1}^n{(t_{rk}-{\overline{t}}_{\cdot \cdot })}^2={\sum }_{r=1}^b{\sum }_{k=1}^n{(t_{rk})}^2$ because $X$ is centered. Let $S{S}_B^{\ast }\equiv {\sum }_{r=1}^{b}S{S}_B^{(r)}$. The objective function in formula 3 can be rewritten as $SS(\lambda )=S{S}_T-\lambda S{S}_B^{\ast }$, so it penalizes the donor-to-donor variation in the projected data. In general, the $Y$ matrix in formula 1 can be designed such that the penalty term represents the between-groups sum of squares, when the between-groups variation is undesirable.

Simulations.

We evaluated AC-PCA in simulations mimicking the brain data, and we set $b=10$, $p=400$ and $n=5$ in the simulations. We assumed that $X^{(i)}=\mathrm{\Omega }+{\mathrm{\Gamma }}^{(i)}+{ϵ}^{(i)}$, where $\mathrm{\Omega }$ is the low rank component shared among donors, ${\mathrm{\Gamma }}^{(i)}$ is the donor-specific component, and ${ϵ}^{(i)}$ is Gaussian noise. $𝚪$ is unknown and our goal is to capture the shared component $\mathrm{\Omega }$. Consider performing PCA on the pooled samples from multiple donors. When $𝚪=0$, the first several PCs can capture $\mathrm{\Omega }$. When $𝚪\ne 0$, the PCs can be affected by $𝚪$, as we have seen in the brain data. We further assumed that ${\mathrm{\Gamma }}^{(i)}={\mathrm{\Lambda }}_1^{(i)}+{\mathrm{\Lambda }}_2^{(i)}$: The donor’s effect is the same in all regions within a donor in ${\mathrm{\Lambda }}_1^{(i)}$, whereas it is different in ${\mathrm{\Lambda }}_2^{(i)}$. ${\mathrm{\Lambda }}_1^{(i)}$ causes samples from the same donor to cluster, whereas ${\mathrm{\Lambda }}_2^{(i)}$ allows for more complicated donor’s effect and we considered two settings (Materials and Methods): (i) only a subset of regions is affected (three random regions in each donor) and (ii) the latent structure in ${\mathrm{\Lambda }}_2^{(i)}$ is correlated with that in $\mathrm{\Omega }$. The results for one representative run and for 100 runs are shown in Fig. 2 and SI Appendix, Fig. S2. We compared AC-PCA with ComBat (9) and SVA (10, 14), where PCA was implemented after removing the confounder effects (SI Appendix). Compared with ComBat and SVA, both the projected data (“PC”) and the loading of genes (“PC loading”) in AC-PCA tend to be more correlated with that in the shared component $\mathrm{\Omega }$. To see how the other methods may fail, we calculated the correlations of the first two PCs with PC1 in ${\mathrm{\Lambda }}_1$ and ${\mathrm{\Lambda }}_2$ (SI Appendix, Figs. S3 and S4). ComBat adjusts for ${\mathrm{\Lambda }}_1$ well and cannot adjust for ${\mathrm{\Lambda }}_2$. This is as expected because ComBat assumes that the donor’s effect is similar in all regions within a donor, when using donor labels for the adjustment. Compared with ComBat, SVA adjusts for ${\mathrm{\Lambda }}_2$ better but not as well for ${\mathrm{\Lambda }}_1$. AC-PCA adjusts for both ${\mathrm{\Lambda }}_1$ and ${\mathrm{\Lambda }}_2$ well. Simulations for ${\mathrm{\Gamma }}^{(i)}={\mathrm{\Lambda }}_1^{(i)}$ or ${\mathrm{\Lambda }}_2^{(i)}$ alone and other settings are provided in SI Appendix.

Fig. 2.

Comparison of AC-PCA with PCA, ComBat (9), and SVA (10, 14) on simulated data. (A and C) Settings 1 and 2, one representative run. Each color represents a donor. (B) Setting 1, correlation with PCA on the shared component $\mathrm{\Omega }$. The correlations of the first two PCs in $\mathrm{\Omega }$ with the matched PCs in the three methods were calculated, and the distribution for 100 runs is shown. We calculated the Pearson’s correlation for the projected data (“PC”) and the Spearman’s rank correlation for the loading of genes (“PC loading”). The dot in the violin plot indicates the median of the distribution. (D) Setting 1, distribution of the distance between the left-out sample and the retained samples, the first two PCs.

In addition to identifying dominant patterns in the data, PCA has been used to detect abnormal samples, potentially caused by mislabeling. To see whether AC-PCA can detect a mislabeled sample, we performed leave-one-out cross-validation (CV): (i) We performed AC-PCA on the retained samples and used the eigenvectors to calculate the projection for the left-out sample, (ii) we calculated the distance of the left-out sample with each of the retained samples, and (iii) we iterated i and ii through all of the samples. In the first two PCs, the left-out sample tends to be closer to the retained samples with the same region label (Fig. 2D and SI Appendix, Figs. S5 and S6). The two distributions, same region label vs. different region labels, are well separated in the first two PCs, especially for PC1. For a left-out sample, by comparing its distances to samples that have the same region label vs. different region labels in the retained set, we are likely to identify whether it is mislabeled.

Application to the Human Brain Exon Array Data.

The human brain exon array dataset (4) includes the transcriptomes of 16 brain regions comprising 11 areas of the neocortex and 5 other regions. In the analysis, we used samples from $10$ regions in the neocortex. Primary visual cortex (V1C) was excluded from the analysis because the distinct nature of this area relative to other neocortical regions tended to compress the other 10 regions into a single cluster. We sorted the donors by age and defined nine time windows by grouping samples from every six donors (SI Appendix, Table S1). Samples within a time window are relatively homogeneous in time, except for window 4, in which the donor’s effect is likely driven by age (SI Appendix, Fig. S7). Samples in window 5 were used for demonstration in Fig. 1. In window 5, when we applied formula 3 to adjust for the confounding effects from individual donors, samples from the same neocortical region tended to cluster together, and we were able to recover the anatomical structure of neocortex (Fig. 3 A and B). ComBat and SVA were able to remove some donors’ effect, because samples no longer cluster by donors after the adjustment. However, no clear interregional patterns were identified by the two methods. One limitation of our method is that the age effect is not distinguished from the donor’s effect. It is challenging to distinguish between the two effects because the donor labels are highly confounded with age (SI Appendix, Fig. S7).

Fig. 3.

Application of AC-PCA to the human brain exon array data. (A) Comparisons of AC-PCA with ComBat (9) and SVA (10, 14), window 5. Abbreviations for the brain regions are medial prefrontal cortex [MFC (F)], orbital prefrontal cortex [OFC (O)], ventrolateral prefrontal cortex [VFC (V)], dorsolateral prefrontal cortex [DFC (D)], primary motor cortex [M1C (M)], primary somatosensory cortex [S1C (S)], posterior inferior parietal cortex [IPC (P)], primary auditory cortex [A1C (A)], posterior superior temporal cortex [STC (T)], and inferior temporal cortex [ITC (I)]. (B) Representative fetal human brain, lateral surface of hemisphere. MFC is not visible on the lateral surface. (C) The eigenvalues in the brain data (red cross) vs. permutated data (boxplot). The boxplots represent the distributions for 100 permutations. (D) Distribution of the Euclidean distance between all pairs of samples: one from the left-out donor and the other one from the retained samples. (E) Number of genes selected in the sparse PCs and the interregional variation explained by the regular PCs. Interregional variation is calculated to be the sum over the variance across regions among the individuals. (F) Expression levels for the genes with top loadings in windows 1 and 3. The top three genes are shown and each point represents the median over the individuals. (G) Conservation (dN/dS) scores for the genes in PC1 and PC2. “All” represents the total 17,568 genes in the exon array experiment. “All-5000” represents the 5,000 genes used in the analysis. “Essential” represents a list of $337$ genes that are functionally conserved and essential, obtained from the Database of Essential Genes (DEG) version 5.0 (18). “Window” is abbreviated as “W.”

We compared the eigenvalues in the brain data versus permutation by shuffling the region labels in each donor (Fig. 3C). The first three PCs are likely to be significant. When we shuffled all samples across donors, the trend was similar (SI Appendix, Fig. S8). We also compared the variance explained by the PCs in the brain data versus permutation, and the trend was similar (SI Appendix, Fig. S9). A parallel evidence for the significance of the PCs is achieved through CV. In addition to the leave-one-out CV presented in the simulation section, we considered leave-one-donor-out CV, where all samples within a donor are left out. We iterated through all donors and calculated the distance between all pairs of samples: one from the left-out donor and the other one from the retained samples. The CV result is consistent with the eigenvalue and variance results, because the left-out samples tend to be closer to the retained samples with the same region label in the first three PCs (SI Appendix, Figs. S10 and S11). When we combined the first three PCs, the Euclidean distance for the pair of samples with the same region label versus different region labels tended to be more separated, compared with that for each PC individually (Fig. 3D and SI Appendix, Figs. S10–S12). In fact, if we predict the region label of a testing sample by assigning it to the $k$-closest clusters of regions in the training set based on the first three PCs, the prediction accuracies are $32\%$($k=1$), $65\%$($k=2$), $80\%$($k=3$) for leave one donor out and $43\%$($k=1$), $67\%$($k=2$) and $85\%$($k=3$) for leave one sample out, whereas the expected accuracies for random guess are $10\%$($k=1$), $20\%$($k=2$), and $30\%$($k=3$). Because the confounding effect of the left-out donor cannot induce any bias in the clustering of regions, this result shows that the penalty in AC-PCA enables the learning of dimension with significantly reduced confounder influence. In simulation settings 1 and 2, the three criteria (eigenvalue, variance, and CV) give consistent estimates for the number of significant PCs, which equals to the number of latent factors in $\mathrm{\Omega }$ (SI Appendix, Figs. S5 and S6).

The visualization results for the other windows are shown in SI Appendix, Figs. S13 and S14. In summary, all methods performed reasonably well in windows 1 to 3; in windows 4 to 9, no clear interregional patterns were identified in ComBat and SVA, whereas the patterns identified by AC-PCA tend to agree with the regions’ physical locations. When PCA is performed separately for each donor and hemisphere, the gross structure tends to be consistent within a time window, but the pattern is distorted by a high level of noise (SI Appendix, Fig. S15). Next, we explored the temporal dynamics of the PCs in the brain data (SI Appendix, Fig. S16). The pattern is similar from windows $1$ to $5$, with PC1 representing the frontal-to-temporal gradient, which follows the contour of developing cortex (5), and PC2 representing the dorsal-to-ventral gradient. Starting from window $6$, these two components reversed order. The interregional variation explained by the first two PCs decreases close to birth (window $4$) and then increases in later time windows (SI Appendix, Fig. S17), similar to the “hourglass” pattern previously reported based on cross-species comparison and differential expression (19, 20).

We then implemented AC-PCA with sparsity constraints to select genes associated with the PCs. The number of genes with nonzero loadings are shown in Fig. 3E, along with the interregional variation explained in the regular PCs. Interestingly, the trends tend to be consistent: When the regular PC explains more variation, more genes are selected in the corresponding sparse PC. To produce more stringent and comparable gene lists, we chose the sparsity parameter such that $200$ genes are selected in each window. The overlap of gene lists across windows is moderate (SI Appendix, Fig. S18) and, as expected, the overlap with the first window decreases over time. The overlap between adjacent windows tends to be larger in later time windows, indicating that interregional differences become stable. Genes with the largest loadings demonstrate interesting spatial patterns (Fig. 3F and SI Appendix, Fig. S19). In windows 1 and 3, the top genes in PC1 follow the frontal-to-temporal gradient, whereas in PC2 they tend to follow the dorsal-to-ventral gradient. A brief overview of the functions of these genes is shown in SI Appendix, Table S2. We compared the gene level results between SVA and AC-PCA (SI Appendix, Fig. S19). In the later time windows, especially windows 5, 6, and 7, AC-PCA tends to select genes with larger interregional variation.

Finally, we demonstrate the functional conservation of the $200$ genes selected in PC1 and PC2. These genes tend to have low dN/dS scores for human versus macaque comparison, even lower than the complete list of all essential genes (Fig. 3G). In the human versus mouse comparison, we observed a similar trend (SI Appendix, Fig. S20). Parallel to the cross-species conservation, we also observed that these genes tend to have low heterozygosity scores, a measure of functional conservation in human (SI Appendix, Fig. S21).

Application to the modENCODE RNA-Seq Data.

The modENCODE project generates the transcriptional landscapes for model organisms during development (16, 17). In the analysis, we used the time-course RNA-Seq data for fly and worm embryonic development. For the fly data, samples were taken in 12 time windows during embryonic development: 0 to 2, 2 to 4, $\cdots$, 22 to 24 h; for the worm data, samples were taken every 30 min during embryonic development: $0,0.5,\cdots ,4,5,\cdots ,12$ h, where the sample from 4.5 h is missing, resulting in $24$ samples.

We first conducted PCA on fly and worm separately, as shown in Fig. 4A. Although the temporal patterns share some similarity, the projections for fly and worm are different. The genes with top loadings in fly have different temporal dynamics in worm, especially for PC2 (Fig. 4B). We also conducted PCA on fly and worm jointly, which reveals the variations of different species (Fig. 4C).

Fig. 4.

AC-PCA captures shared variation in fly and worm modENCODE RNA-Seq data, embryonic stage. (A) PCA on fly and worm separately. Each data point represents a time window (fly) or a time point (worm), in the unit of hours. (B) Expression levels of the top three genes in fly PCA. (C) PCA and AC-PCA on fly and worm jointly. (D) Expression levels of the top three genes in AC-PCA.

Let $X^{(f)}$ and $X^{(w)}$ represent the data matrices for fly and worm, correspondingly. Let $X_t^{(f)}$ and $X_t^{(w)}$ denote the data for time window $t$ and time point $t$, correspondingly. $X_{10}^{(w)}$ is missing. Let $X$ represent the data matrix for both species, by stacking the rows in $X^{(f)}$ and $X^{(w)}$. We propose the following objective function to adjust for the variation of species:

$$\begin{array}{c}\underset{v\in {ℝ}^p}{\text{maximize}\hspace{0.17em}}{v}^T{X}^{T}Xv-\lambda {\displaystyle \sum _{t=1}^{12}}{v}^T{(X_t^{(f)}-f(X^{(w)},t))}^T\times (X_t^{(f)}-f(X^{(w)},t))v\hfill \\ \text{subject to}\hspace{0.17em}{||v||}_2^2\le 1,\hfill \end{array}$$ [4]

where

$$f(X^{(w)},t)=\{\begin{array}{cc}\frac{1}{2}(X_{2t-1}^{(w)}+X_{2t+1}^{(w)}),\hfill & \text{if}t=5\hfill \\ \frac{1}{3}(X_{2t-1}^{(w)}+X_{2t}^{(w)}+X_{2t+1}^{(w)}),\hfill & \text{otherwise}\hfill \end{array}$$

Formula 4 is a special case of the general form (SI Appendix). To incorporate the difference in the length of the embryonic stages, we shrink the projection of $X_t^{(f)}$ toward the mean of $X_{2t-1}^{(w)}$, $X_{2t}^{(w)}$, and $X_{2t+1}^{(w)}$ after the projection. Li et al. (21) aligned fly and worm development based on stage-associated genes using the same dataset. We did not implement their alignment results, to keep the analysis unsupervised.

Formula 4 is able to capture the shared variation among fly and worm (Fig. 4C). The selected genes tend to have consistent and smooth temporal patterns in both species (Fig. 4D). PCA on fly and worm jointly cannot capture the direction of PC2 in AC-PCA, in which the gene expression levels peak in the middle embryonic stage. The other PCs in PCA are shown in SI Appendix, Fig. S22. After using ComBat with the species labels for the adjustment, PCA still cannot capture that direction (SI Appendix, Fig. S23).

Other Applications.

Two additional simulation examples are provided in SI Appendix, where we implemented other formulations of AC-PCA.

Discussion

Confounding variation can affect the performance of dimension reduction methods, and hence the visualization and interpretation of the results. In this study, we have proposed a general class of penalty functions in PCA for simultaneous dimension reduction and adjustment for confounding variation. In formula 1, we implemented linear kernel on $Y$ (i.e., $Y{Y}^T$) and other kernels can be used as well. The additive property of kernels enables a further generalization, through the combination of different kernels, to adjust for multiple types of confounders’ effects simultaneously.

The application of AC-PCA is not limited to transcriptome datasets. Dimension reduction methods have been applied to other types of genomics data for various purposes, such as feature extraction for methylation prediction (22), classifying yeast mutants using metabolic footprinting (23), classifying immune cells using DNA methylome (24), and others. AC-PCA is applicable to these datasets to capture the desired variation, adjust for potential confounders, and select the relevant features. AC-PCA can serve as an exploratory tool and be combined with other methods. For example, the extracted features can be implemented in regression models.

Materials and Methods

AC-PCA Adjusting for Variations of Individual Donors.

We treated the left and right hemispheres from the same donor as two different individuals when implementing formula 3. We implemented all methods (AC-PCA, PCA, ComBat, and SVA) separately for each time window. To formulate the penalty term in formula 3, we need to know the source of confounding variation. However, the donor labels are not necessarily required. Only the brain region labels (i.e., labels for the primary variables of interest) are needed, because we are penalizing every pair of samples in the same brain region (SI Appendix, SI Materials and Methods). The connection of formula 3 with canonical correlation analysis is shown in SI Appendix, SI Materials and Methods.

AC-PCA with Sparse Loading.

See SI Appendix, SI Materials and Methods.

Multiple PCs.

See SI Appendix, SI Materials and Methods.

Tuning $\lambda$.

When $\lambda$ increases from 0, the ratio $R(\lambda )=v^T{X}^{T}KXv/v^T{X}^{T}Xv$ will tend to decrease (SI Appendix, Fig. S24). If $v^T{X}^{T}KXv$ has the interpretation of the between-groups sum of squares, we choose the smallest $\lambda$ such that $R(\lambda )\le 0.05$ in the PCs that we are interested in, so the confounding variation is “small” compared with the total variation in the projected data. In other designs of $K$, $R(\lambda )$ can be greater than 1 and we choose the smallest $\lambda$ such that $R(\lambda )\le 0.05R(\lambda =0)$. We notice that the overall patterns captured by AC-PCA are robust to a wide range of $\lambda$s in simulations and real data analysis (SI Appendix, Fig. S25). In the human brain dataset, we fixed $\lambda$ = 5 for a better comparison over the time windows.

Significance of the PCs.

For a fixed $\lambda$, we evaluate the significance of the PCs by checking whether the eigenvalues and the variance explained by the PCs are significant. To achieve this, we compare the values in the original data vs. permutation, where we permutate the rows in $X$ and keep $Y$ the same.

Tuning ${\text{c}}_1$ and ${\text{c}}_2$.

See SI Appendix, SI Materials and Methods.

Simulations.

In setting 1, we considered $n=5$, $b=10$, and $p=400$. For the $i$th individual, $X^{(i)}=\mathrm{\Omega }+\alpha {\mathrm{\Gamma }}^{(i)}+{ϵ}^{(i)}$. The shared component $\mathrm{\Omega }=Wh.W=(w_1{w}_2)$ is a $b\times 2$ matrix, representing the latent structure of the shared variation. For visualization purpose, we assumed that it is smooth and has rank 2. Let $\mu =(1,\cdots ,b)\prime$ and $w_1$ is the normalized $\mu$, with mean 0 and variance 1. $w_2\sim \mathcal{N}(0,0.25\cdot \mathrm{\Sigma })$, where ${\mathrm{\Sigma }}_{ij}=\exp (-{(w_{i1}-w_{j1})}^2/4)$. $h$ is a $2\times p$ matrix and the rows in $h$ are generated from $\mathcal{N}(0,I_p)$. The donor-specific component ${\mathrm{\Gamma }}^{(i)}={\mathrm{\Lambda }}_1^{(i)}+{\mathrm{\Lambda }}_2^{(i)}$, where ${\mathrm{\Lambda }}_1^{(i)}=1r_i$ and ${\mathrm{\Lambda }}_2^{(i)}=B_i{s}_i$. The 1 represents a $b\times 1$ matrix with all 1s. $B_i$ is a $b\times 1$ matrix, in which three entries are generated from Uniform[0, 2], and the other entries are set to 0. $r_i$ and $s_i$ are $1\times p$ matrices, generated from $\mathcal{N}(0,I_p)$. $\alpha$ is a scalar indicating the strength of confounding variation, and we set $\alpha =2.5$. The entries in ${ϵ}^{(i)}$ are generated from $\mathcal{N}(0,0.25)$. In setting 2, the only difference from setting 1 is the term $B_i$. We first set $B_i$ to be equal to $w_1$ and then randomly pick and shuffle three entries. We implemented formula 3 for settings 1 and 2. More simulation results are shown in SI Appendix.

Data Preprocessing.

See SI Appendix, SI Materials and Methods.