Skip to main content

This is a preprint.

It has not yet been peer reviewed by a journal.

The National Library of Medicine is running a pilot to include preprints that result from research funded by NIH in PMC and PubMed.

bioRxiv logoLink to bioRxiv
[Preprint]. 2025 May 26:2025.05.21.655279. [Version 1] doi: 10.1101/2025.05.21.655279

Gene set optimization for cancer transcriptomics using sparse principal component analysis

H Robert Frost 1,*
PMCID: PMC12258712  PMID: 40661635

Abstract

A common approach for exploring pathway dysregulation in cancer involves the gene set or pathway analysis of tumor transcriptomic data. Unfortunately, the effectiveness of cancer gene set testing is limited by the fact that most gene set collections model gene activity in normal tissue, which can differ significantly from gene activity found within tumors. To address this challenge, we have developed a bioinformatics approach based on sparse principal component analysis (PCA) for optimizing existing gene set collections to reflect the pattern of gene activity in dysplastic tissue and have used this technique to optimize the Molecular Signatures Database (MSigDB) Hallmark collection for 21 solid human cancers profiled via bulk RNA-seq by The Tumor Genome Atlas (TCGA). Demonstrating the biological utility of our approach, the average survival association of gene set members is improved after optimization for nearly all cancer types and Hallmark gene sets.

1. Introduction

Cancer occurs when cells develop a relative growth advantage due to the dysregulation of pathways associated with cell proliferation, survival, or genome maintenance [1]. Understanding the process that leads to pathway alteration, and the impact that pathway dysfunction has on cell and tissue physiology, is therefore critical for an accurate understanding of cancer biology and development of therapeutic interventions. To characterize tumors in terms of biological pathways, researchers have leveraged pathway analysis, or gene set testing, techniques, some specifically optimized for cancer [2, 3], to analyze the large collections of cancer genomic data maintained by projects such as The Cancer Genome Atlas (TCGA) [4].

Although researchers have identified the pathways most frequently impacted by somatic alterations in cancer [1] and progress has been made developing cancer-specific pathway analysis methods that can integrate gene expression and somatic alteration data [2], existing approaches are limited by the misalignment between existing gene set collections and pathway activity within cancerous tissue. Most existing gene set collections, e.g., Gene Ontology [5], represent a tissue-agnostic model of normal gene activity. However, as cataloged by the Human Protein Atlas (HPA) [6] gene expression can differ substantially between different normal tissues and between healthy and malignant samples of the same tissue type [7].

We have previously explored the impact of tissue and cell type-specificity on gene set testing and developed approaches for computing normal tissue-specific [8] and cell type-specific[9] gene set weights, which can be used to select the most biologically relevant gene sets for a particular tissue type or perform p-value weighting after statistical analysis. Although these gene set weights can be used to control for normal tissue or cell type specificity [10], they fail to account for differences between healthy and cancerous tissues. Given the distinct pattern of gene activity found in healthy and cancerous tissue, a gene set testing approach that only accounts for normal tissue-specificity will still generate inflated error rates when applied to tumor expression data. To address the differences between healthy and neoplastic tissue and improve the accuracy of gene set testing on cancer transcriptomics data, we have developed an approach based on sparse principal component analysis (PCA) for customizing existing gene sets to match pattern of gene co-expression in tumors and have used this method to optimize the MSigDB Hallmark collection for 21 solid human cancers profiled by the TCGA. This optimization process and our evaluation approach are detailed in Section 2 with evaluation results in Section 3.

2. Data and Methods

2.1. Data sources

TCGA data was accessed via the UCSC Xena Datahub [11]. We downloaded and analyzed TPM-normalized RNA-sequencing and overall survival (OS) data for the 21 cancer cohorts listed in Table 1. We applied our gene set optimization method (as detailed in Section 2.2 below) to this cancer gene expression data to optimize the 50 gene sets in version 7.5 of the Molecular Signatures Database (MSigDB) Hallmark collection [12]. To evaluate the effectiveness of this optimization technique, we leveraged gene-level survival association statistics computed for the TCGA data by Uhlen et al. [6]. Specifically, these statistics (si,j for gene i and cancer type j) represent a signed log p-value where the p-value is generated via a Kaplan-Meir (KM) test of the association between the expression of gene i and overall survival for TCGA cohort j with sign based on the direction of the association (si,j>0 for favorable, si,j<0 for unfavorable).

Table 1:

The 21 analyzed TCGA cancer types and number of samples with expression data.

Abbrev (name) # Abbrev (name) # Abbrev (name) #

BLCA (bladder) 430 KIRC (kidney, clear cell) 607 PRAD (prostate) 551
BRCA (breast) 1,217 KIRP (kidney, papillary cell) 321 READ (rectal) 177
CESC (cervical) 309 LIHC (liver) 424 SKCM (melanoma) 472
COAD (colon) 512 LUAD (lung, adeno) 585 STAD (stomach) 407
GBM (gliobastoma) 173 LUSC (lung, squamous) 550 TGCT (testicular) 156
HNSC (head and neck) 546 OV (ovarian) 379 THCA (thyroid) 568
KICH (kidney, chromophobe) 89 PAAD (pancreatic) 182 UCEC (uterine) 583

2.2. Gene set optimization method

Our approach for cancer-specific gene set optimization leverages the empirical covariance structure of tumor gene expression data. Specifically, our approach seeks to identify one or more subsets of gene set members that are significantly co-expressed in tumors of a specific cancer type. This approach is motivated by the assumption that genes which play a biologically important role in a particular gene set/pathway will be positively co-expressed in the relevant normal tissue or tumor type [13]. To identify the co-expressed groups of genes in each set, we leveraged our Eigenvectors from Eigenvalues Sparse Principal Component Analysis (EESPCA) [14] method. The EESPCA method is based on the recently detailed formula for computing normed, squared eigenvector loadings of a Hermitian matrix from the eigenvalues of the full matrix and associated sub-matrices [15]. Importantly in this context, the sample covariance matrix Σˆ is Hermitian, which implies that squared normed PC loadings (v12 for the first PC) can be computed as a function of PC variances for the full data set and all of the leave-one-out variable subsets. Specifically, the normed squared loading for variable j in the first PC can be computed as: v1,j2=k=1p1(λ1(Σˆ)λk(Σˆj))/k=2p(λ1(Σˆ)λk(Σˆ)), where Σˆj represents the sample covariance sub-matrix with variable j removed and λk(Σ) is the kth eigenvalue of Σ. EESPCA is based on an approximate version of this formula, v˜1,j2=1λ1(Σˆj)/λ1(Σˆ), that can be efficiently computed for very large data matrices using the method of power iteration. Importantly, the ratio between the approximate and true eigenvector loadings, r1,j=v˜1,j2/v1,j2, provides information regarding the true sparsity structure of the population PCs and is used by the EESPCA method to generate sparse PC loading vectors. Relative to the state-of-the-art sparse PCA methods of Witten et al. [16], Yuan & Zhang [17] and Tan et al. [18], the EESPCA technique offers a two-orders-of-magnitude improvement in computational speed, does not require estimation of tuning parameters via cross-validation, and can more accurately identify true zero principal component loadings across a range of data matrix sizes and covariance structures.

To optimize membership for a specific gene set and TCGA cohort, the EESPCA method is used to compute the first three sparse PCs using the normalized RNA-seq counts for the set genes. The loadings of these sparse PCs are then used to create three different optimized versions of the target set by removing all genes that have a zero loading on the first PC, the first two PCs or the first three PCs. This approach supports the scenario where a set is comprised by up to three independent co-expressed blocks of genes.

2.3. Evaluation

To evaluate our cancer-specific gene set customization approach, we compared the si,j survival association statistics for genes in unoptimized and optimized gene sets. If the optimization process is preferentially retaining genes that play an important role in the biology of the target cancer, we expect that the average magnitude of the surival statistics will be higher for the optimized set than for the unoptimized set. If the m genes in the original set are split into mo in the optimized set and mr that are removed, this expectation can be stated mathematically as i=1mrsi,j<i=1m*si,j. When this inequality holds, we can conclude that the quality of the gene set improved after optimization, otherwise, it worsened. To assess the statistical significance of the change in survival statistics, directional Wilcoxon rank sum tests were performed comparing the survival statistics of retained vs removed genes. If the relevant directional test was marginally significant at α=0.05, the optimized set was deemed to be either ‘significantly improved’ or ‘significantly worse’. Overall, each optimization result was assigned to one of four distinct outcome classes: significantly improved, improved, worse, and significantly worse.

A related evaluation can be performed by comparing the magnitude of the survival statistics for cancer type j between a set optimized according to the expression data for cancer j (e.g., liver) and the same set optimized for a distinct cancer type k (e.g., breast). Our expectation in this case is that the average magnitude of the survival statistics will be larger when optimization was performed using expression data from the same cancer type than from a distinct cancer type, e.g., i=1m*si,j>i=1m*si,k.

3. Results

3.1. Proportion of gene set annotations retained after optimization

Figure 1 visualizes the proportion of gene set annotations retained after optimization for each of the 21 target TCGA cohorts and number of PCs. As expected, the number of genes retained after optimization increases as the number of PCs considered increases. The average proportion retained across all cohorts and Hallmark gene sets was 0.30 for 1 PC, 0.53 for 2 PCs, and 0.65 for 3 PCs. While the impact of optimization is similar for most cancer types, ovarian cancer (OV) is a noticable outlier with almost no genes removed when 2 or 3 PCs are considered, which implies that the higher order sparse PCs computed on the ovarian cancer expression data have very few zero values.

Figure 1:

Figure 1:

Proportion of Hallmark gene set annotations retained after optimization on the bulk RNA-seq data for each TCGA cohort using 1 to 3 sparse PCs.

3.2. Optimization outcomes for each TCGA cohort

Figure 2 visualizes the proportion of gene sets whose optimization results fall into one of the four outcome categories: significantly improved, improved, worse, and significantly worse (see Section 2.3 for details). Averaged across all of the cohorts and PC numbers, the average gene-level survival association significantly improved for 25.3% of the gene sets, improved for an additional 37.9%, worsened for 29.9% and significantly worsened for only 6.9%. Optimization using 3 PCs had the best average outcome (22.8% significantly improved, 44.2% improved, 27.3% worse, and 5.7% significantly worse). Importantly, optimization improves the survival association of annotated genes for well over half of the Hallmark gene sets for almost all of the TCGA cancer types (prostate (PRAD) and testicular (TGCT) cancers being the notable exceptions).

Figure 2:

Figure 2:

Proportions of the Hallmark gene sets by outcome class after optimization on the bulk RNA-seq data for each TCGA cohort using different numbers of PCs.

Figure 3 provides a similar outcome visualization for the case where gene sets optimized for one TCGA cohort are evaluated using survival statistics from distinct cohorts. As expected, the proportion of improved gene sets is lower in this case as compared to the scenario when survival statistics and expression data are based on the same cohort. Specifically, the average gene-level survival association significantly improved for 17.1% of the gene sets (vs 25.3%), improved for 39.5% (vs 37.9%), worsened for 35.1% (vs 29.9%) and significantly worsened for 8.3% (vs 6.9%). Surprisingly, our optimization method was still generally effective in this case which implies that the biologically relevant gene co-expression pattern is broadly similar across many solid human cancers. One notable outlier is liver cancer (LIHC), which is likely due to the fact that liver tissue has a relatively large number of highly tissue-specific genes [6].

Figure 3:

Figure 3:

Proportions of the Hallmark gene sets by outcome class after optimization on the bulk RNA-seq data from distinct TCGA cohorts.

4. Conclusions and future directions

To address the mismatch between public gene set collections, which are typically based on gene activity within normal tissues/cells, and the pattern of gene expression in cancer, we developed an optimization method based on the sparse PCA of tumor gene expression data. The results in this paper demonstrate that this approach can effectively retain genes whose expression is significantly associated with cancer survival. Importantly, for the optimization results based on 3 PCs for 21 solid cancers and 50 Hallmark gene sets, the mean survival association was improved in 63% of the cases with four times as many sets having a statistically significant increase in survival association strength than a significant decrease. Of course, not all cancer types and gene sets benefit from this optimization and evaluation based on the association with overall survival is only a proxy for general biological relevance. Given the heterogeneity of the results, the practical application of this type of approach would only use the optimized sets on non-TCGA data that saw a significant improvement on the relevant TCGA cancer type.

For the full-length version of this short paper, we plan to 1) extend the evaluation results to all MSigDB collections, 2) create public repository of optimized gene sets, 3) include gene set-specific results in addition to cancer type-specific results, and 4) explore other evaluation metrics (e.g., prognosis prediction using single-sample gene set statistics, recurrence-free survival vs. overall survival, association with non-survival endpoints, etc.).

Acknowledgments

I would like to thank Xingyu Zheng for her help evaluating an early prototype of this approach.

Funding

This work was funded by National Institutes of Health grants R35GM146586, R21CA253408, P20GM130454 and P30CA023108.

Footnotes

Conflict of interests

The authors have no conflicts of interest to declare.

References

  • [1].Vogelstein Bert, Papadopoulos Nickolas, Victor E Velculescu Shibin Zhou, Luis A Diaz Jr, and Kinzler Kenneth W. Cancer genome landscapes. Science, 339(6127):1546–58, Mar 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [2].Mutation Consequences and Pathway Analysis working group of the International Cancer Genome Consortium. Pathway and network analysis of cancer genomes. Nat Methods, 12(7):615–21, Jul 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Robert Frost H. Tissue-adjusted pathway analysis of cancer (tpac): A novel approach for quantifying tumor-specific gene set dysregulation relative to normal tissue. PLoS Comput Biol, 20(1):e1011717, Jan 2024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [4].Cancer Genome Atlas Research Network, Weinstein John N, Collisson Eric A, Mills Gordon B, Mills Shaw Kenna R, Ozenberger Brad A, Ellrott Kyle, Shmulevich Ilya, Sander Chris, and Stuart Joshua M. The cancer genome atlas pan-cancer analysis project. Nat Genet, 45(10):1113–20, Oct 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Ashburner Michael, Ball Catherine A., Blake Judith A., Botstein David, Butler Heather, Michael Cherry J., Davis Allan P., Dolinski Kara, Dwight Selina S., Eppig Janan T., Harris Midori A., Hill David P., Laurie Issel-Tarver Andrew Kasarskis, Lewis Suzanna, Matese John C., Richardson Joel E., Ringwald Martin, Rubin Gerald M., and Sherlock Gavin. Gene ontology: tool for the unification of biology. Nature Genetics, 25(1):25–29, May 2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [6].Uhlen Mathias, Zhang Cheng, Lee Sunjae, Evelina Sjöstedt Linn Fagerberg, Bidkhori Gholamreza, Benfeitas Rui, Arif Muhammad, Liu Zhengtao, Edfors Fredrik, Sanli Kemal, Kalle von Feilitzen Per Oksvold, Lundberg Emma, Hober Sophia, Nilsson Peter, Mattsson Johanna, Jochen M Schwenk Hans Brunnström, Glimelius Bengt, Tobias Sjöblom Per-Henrik Edqvist, Djureinovic Dijana, Micke Patrick, Lindskog Cecilia, Mardinoglu Adil, and Ponten Fredrik. A pathology atlas of the human cancer transcriptome. Science, 357(6352), Aug 2017. [DOI] [PubMed] [Google Scholar]
  • [7].Robert Frost H. Analyzing cancer gene expression data through the lens of normal tissue-specificity. PLoS Comput Biol, 17(6):e1009085, Jun 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Robert Frost H. Computation and application of tissue-specific gene set weights. Bioinformatics, Apr 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [9].Robert Frost H. Leveraging cell type-specificity for gene set analysis of single cell transcriptomics. bioRxiv, page 2024.09.25.615040, Sep 2024. [Google Scholar]
  • [10].Kamal Yasmin, Schmit Stephanie L, Hoehn Hannah J, Amos Christopher I, and Robert Frost H. Transcriptomic differences between primary colorectal adenocarcinomas and distant metastases reveal metastatic colorectal cancer subtypes. Cancer Res, 79(16):4227–4241, Aug 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [11].Mary J Goldman Brian Craft, Hastie Mim, Kristupas Repečka Fran McDade, Kamath Akhil, Banerjee Ayan, Luo Yunhai, Rogers Dave, Angela N Brooks Jingchun Zhu, and Haussler David. Visualizing and interpreting cancer genomics data via the xena platform. Nat Biotechnol, 38(6):675–678, June 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [12].Liberzon Arthur, Birger Chet, Helga Thorvaldsdóttir Mahmoud Ghandi, Mesirov Jill P, and Tamayo Pablo. The molecular signatures database (msigdb) hallmark gene set collection. Cell Syst, 1(6):417–425, Dec 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Segal Eran, Friedman Nir, Koller Daphne, and Regev Aviv. A module map showing conditional activity of expression modules in cancer. Nat Genet, 36(10):1090–8, Oct 2004. [DOI] [PubMed] [Google Scholar]
  • [14].Robert Frost H.. Eigenvectors from eigenvalues sparse principal component analysis. Journal of Computational and Graphical Statistics, 0(0):1–16, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [15].Denton Peter B., Parke Stephen J., Tao Terence, and Zhang Xining. Eigenvectors from eigenvalues. ArXiv Mathematics e-prints, arXiv:1908.03795v3, 2019. [Google Scholar]
  • [16].Witten Daniela M., Tibshirani Robert, and Hastie Trevor. A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics, 10(3):515–534, July 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [17].Yuan Xiao-Tong and Zhang Tong. Truncated power method for sparse eigenvalue problems. J. Mach. Learn. Res., 14(1):899–925, April 2013. [Google Scholar]
  • [18].Kean Ming Tan Zhaoran Wang, Liu Han, and Zhang Tong. Sparse generalized eigenvalue problem: optimal statistical rates via truncated rayleigh flow. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(5):1057–1086, 2018. [Google Scholar]

Articles from bioRxiv are provided here courtesy of Cold Spring Harbor Laboratory Preprints

RESOURCES