CorrAdjust unveils biologically relevant transcriptomic correlations by efficiently eliminating hidden confounders

Abstract

Correcting for confounding variables is often overlooked when computing RNA–RNA correlations, even though it can profoundly affect results. We introduce CorrAdjust, a method for identifying and correcting such hidden confounders. CorrAdjust selects a subset of principal components to residualize from expression data by maximizing the enrichment of “reference pairs” among highly correlated RNA–RNA pairs. Unlike traditional machine learning metrics, this novel enrichment-based metric is specifically designed to evaluate correlation data and provides valuable RNA-level interpretability. CorrAdjust outperforms current state-of-the-art methods when evaluated on 25 063 human RNA-seq datasets from The Cancer Genome Atlas, the Genotype-Tissue Expression project, and the Geuvadis collection. In particular, CorrAdjust excels at integrating small RNA and mRNA sequencing data, significantly enhancing the enrichment of experimentally validated miRNA targets among negatively correlated miRNA–mRNA pairs. CorrAdjust, with accompanying documentation and tutorials, is available at https://tju-cmc-org.github.io/CorrAdjust.

Graphical Abstract

Graphical Abstract.

Introduction

Confounder variables can lead to false positive and false negative calls in many types of statistical analyses of transcriptomic data. Often, such data contain known and latent variables that represent biological and technical factors and explain a significant fraction of gene expression variance [1–3]. Confounder correction is often overlooked when analyzing RNA–RNA correlations, a popular technique for the identification of co-expressed genes, gene prioritization, and even annotation of genes with poorly understood functions [4–6]. The methods for such correction are still in their infancy.

In differential gene expression analysis, confounders are routinely accounted for via their inclusion as covariates in generalized linear models such as DESeq2 [7] or edgeR [8]. The approaches that allow the estimation of hidden confounder factors in RNA-seq data include surrogate variable analysis (sva) [1] or removal of unwanted variation (RUV) [2]. In genome-wide association studies, principal components (PCs) are often used to adjust for the influence of ancestral heterogeneity when analyzing admixed populations [9, 10].

The existing methods for correcting transcriptomic correlation data can be broadly categorized into those that adjust for known variables and those that identify and remove latent factors. The most straightforward approach consists of residualizing known covariates from gene expression data with linear regression prior to computing the correlations, which is equivalent to computing partial correlations [11, 12]. A recently developed method named COBRA is also based on correcting for known variables: it decomposes the gene–gene covariance matrix into a linear combination of matrices, one per each confounder variable; the matrix associated with the intercept term represents the corrected covariances [13].

A common element of existing methods focused on identifying and removing hidden confounders is the estimation of latent variables through principal component analysis (PCA) or PCA modifications. In several such studies, “vanilla” PCs were directly residualized from the expression data [14–16]. Alternatively, PCA is applied only on a subset of genes with the lowest variance in an effort to maximize the likelihood of capturing unwanted variation instead of useful biological signals [17]. Yet another approach finds and removes hidden confounders after inferring a pattern from known covariates [18].

All these methods correct the expression data for a subset of “first N” PCs (or similar inferred variables). The choice of value for the parameter N critically influences the outcome. One approach removes all statistically significant PCs identified by the permutation procedure [14]. Other schemes estimate the number of components to eliminate by manually inspecting the distribution of correlations among positive controls (e.g., genes encoding ribosome subunits) and negative controls (e.g., a random set of genes) [15, 17], or by leveraging gene similarity matrices based on expression level correlations and Gene Ontology terms [16].

Previous studies used conventional machine learning metrics to evaluate pairwise correlations in the presence of reference sets of “positive” and “negative” gene pairs. These metrics include FDR (more widely known as “”) [14], sensitivity, specificity, and area under the receiver operating characteristic curve (ROC AUC) [11, 12]. However, these metrics are not well-suited for evaluating gene–gene correlations due to typically low numbers of significantly correlated gene pairs, imperfection of reference gene sets, and difficulty in interpreting gene-level findings when evaluating all pairs together (see Methods).

CorrAdjust addresses these gaps by detecting and eliminating hidden confounders in correlation analysis. Key to the method is a new enrichment-based metric specifically tailored for evaluating correlations. In what follows, we present the new method, apply it to the computation of mRNA–mRNA and miRNA–mRNA expression correlations, and compare its performance to other schemes through extensive benchmarks using data from three popular public repositories.

Methods

CorrAdjust: input data and problem formulation

In what follows, we will use the term “gene” to generically refer to transcripts, including protein-coding messenger RNAs (mRNAs) and microRNAs (miRNAs).

The input of CorrAdjust consists of two parts (Fig. 1A, left panel). The first part is a gene expression matrix with rows representing biological samples and columns representing genes, normalized in a way that allows gene expression comparisons across samples. Optionally, the expression matrix could comprise several experimental techniques (e.g., mRNA-seq and small RNA-seq) with each gene labeled by its “gene type” (e.g., mRNA and miRNA). The second part of the input is one or more of a priori-defined, ground truth collections of reference gene sets, which we call “reference collections”. We will refer to gene pairs in which both genes are jointly present in at least one common reference gene set as “reference pairs.” Examples of reference collections include Gene Ontology (GO) [19] for mRNA–mRNA pairs and TarBase [20] for miRNA–mRNA pairs. Reference pairs based on the GO collection comprise genes belonging to the same GO term. Reference pairs based on TarBase are formed by miRNAs and their experimentally validated target mRNAs.

Figure 1.

The outline of the CorrAdjust method. (A) Inputs to CorrAdjust are a gene expression matrix and reference collections of gene sets. On each iteration, gene pairs are ranked according to their correlation value. Each gene pair is labeled as “reference” (check mark) or “non-reference” (cross mark), depending on whether the genes jointly belong to at least one ground truth set from the reference collection. Next, CorrAdjust identifies the PC of the gene expression matrix, which, upon its residualization, maximizes the global enrichment score. The procedure is repeated until the convergence of the global enrichment score. (B) To compute the global enrichment score, the ranked gene pairs are first grouped into blocks, each centered around one gene (example genes 1 and 2 are highlighted). Hypergeometric distribution-based statistics are then computed for highly ranked pairs (rectangle with dashed borders) in each block, comparing the fraction of reference pairs between the highly ranked pairs and the rest of the block. Note that highly ranked gene pairs are defined based on the full table from step 2 of panel A. Then, the p-values corresponding to all genes are adjusted for multiple testing, log-transformed, and averaged into a global enrichment score.

We focus on the array of Pearson’s correlation coefficients computed between expression levels of all possible gene pairs formed by input genes (Fig. 1A, middle panel). In the case of multiple gene types, the analysis could be confined to correlations involving molecules of user-specified types only (e.g., only miRNA–mRNA correlations). The goal of CorrAdjust is to identify a subset of gene expression PCs (“confounders”) to adjust for, while maximizing the presence of reference pairs among highly correlated gene pairs. The optimization follows an iterative greedy approach, selecting the “best” PC at each iteration (Fig. 1A, right panel), which can result in a possibly non-consecutive set of PCs identified as confounders.

CorrAdjust: motivation for developing a novel metric

We rank the gene pairs in decreasing order of correlation, from most significant (highest rank) to least significant (lowest rank). The main metric of CorrAdjust determines whether the reference pairs are overrepresented among the highly ranked gene pairs. Note that in some cases, it may make sense to assign the highest ranks to the gene pairs with the most negative correlations and vice versa. One such example is the case of miRNA–mRNA gene pairs, where interacting miRNAs and mRNAs are expected to be anti-correlated.

Two critical observations motivated us to develop a new metric.

1. Typically, only a small subset of highly ranked gene pairs is relevant. Empirical evidence shows that most gene–gene correlations are centered around zero [4, 11, 12, 21, 22]. Thus, to capture an overrepresentation signal of reference pairs, it is essential to consider only the subset of the significant, highly ranked correlations.

2. Reference collections are not a perfect ground truth for co-expression studies. By design, not all reference pairs are expected to have correlated abundances at the transcript level. For example, two genes can belong to the same GO term because they interact at the protein level, which does not imply any correlation between the corresponding mRNAs.

Together, these points define our metric's key requirements. Based on the first observation, our metric rewards highly ranked reference pairs. Based on the second, our metric would not penalize lowly ranked reference pairs. Both aspects are naturally captured by precision-based metrics, which assess the fraction of reference pairs among highly ranked gene pairs.

We define the highly ranked gene pairs by taking the “top %” of the ranked pairs list. The value of is explicitly set by the user and is a hyperparameter. Doing so has the following key advantages. First, it does not depend on sample size, unlike thresholding which is based on correlation p-values. Second, correction for confounders often dramatically shrinks the distribution of correlations to zero (see Results), entangling the use of correlation threshold to define the set of significant gene pairs with uncontrolled changes in the set's cardinality.

The parameter controls the trade-off between precision and the number of gene pairs called significant. Indeed, it is closely related to the precision/recall trade-off since sensitivity is bounded from above by a multiple of (Lemma 1 in Supplementary Note S1). A small value of would decrease the number of considered gene pairs and, thus, sensitivity, but potentially increase precision by selecting only the most significant correlations. On the other hand, a large value of would select more pairs and increase sensitivity at the price of diluting precision by letting uncorrelated pairs through. Note that the maximum precision achieved by a no-skill model equals the fraction of reference pairs out of all pairs, which is determined by the reference collection being used. Thus, maximizing precision is a challenge in itself. On the contrary, one can always achieve a sensitivity of 1 by setting to 100%. Supplementary Note S2, Supplementary Fig. S1, and Supplementary Fig. S2 describe the sensitivity analysis for the parameter .

CorrAdjust: metric for evaluating correlations in the presence of reference collections

The computation of our metric comprises three steps. First, we rank gene pairs according to their correlation and mark the top % entries (Fig. 1A, middle panel). Second, for each gene, we compute overrepresentation scores by considering only gene pairs to which the gene belongs (Fig. 1B, left panel). Third, we average the overrepresentation scores from the previous step over all genes and all reference collections if more than one collection is available (Fig. 1B, right panel).

Let denote the absolute number of ordered gene pairs associated with the % cutoff, and denote the number of pairs containing the -th gene among the pairs (). By definition, , as the sum in the left part counts each gene pair twice. Let denote the number of reference pairs among the pairs. By analogy, we define as the total number of gene pairs containing the -th gene and as the number of reference pairs among the pairs. In the simplest case of only one gene type, for any , as each gene form pairs with all other genes. In the case of two or more gene types, values are not necessarily constant. For example, when miRNA–mRNA correlations are analyzed, equals the total number of mRNAs or miRNAs if represents miRNA or mRNA, respectively.

With , , , and values computed, we assess the statistical significance of the overrepresentation of reference pairs among highly ranked gene pairs using the hypergeometric test. A commonly used statistic in this scenario is enrichment, , which is defined as the ratio of over its expected value under the null hypergeometric model . An undesirable property of enrichment is the dependence of its range of possible values on distribution parameters, thus making enrichment incomparable between the different genes (represented by index ). As a more suitable alternative, we adapted a recently developed balanced precision () metric [23], which can be viewed as a non-linear monotonic transformation of enrichment (see Lemma 2 in Supplementary Note S1):

Here, , a constant defined by the reference collection only, and is the enrichment statistic defined above. It is easy to see that corresponds to , to , and to the maximum possible value of . Thus, balanced precision could be directly compared between different genes.

The above computations become unstable when is close to zero, i.e., when gene participates in a small number of pairs from the top % set. To tackle this problem, we use regularization inspired by the Bayesian beta-binomial model. Specifically, we add “pseudo” gene pairs to and gene pairs to (the latter is rounded during hypergeometric p-value computation). This procedure shrinks to 0.5 (or, equivalently, to 1) when is small, and the effect is negligible when is large. We completely skip computations for genes with  < 1 000 to avoid problems with statistical power. For example, when integrating fewer than 1 000 miRNAs with more than 1 000 mRNAs, we compute statistics only for values of  corresponding to miRNAs, not mRNAs. This is because each miRNA forms over 1 000 total miRNA–mRNA pairs, whereas each mRNA forms fewer than 1 000 miRNA–mRNA pairs.

Lastly, we adjust the hypergeometric p-values across all genes using the Benjamini–Hochberg procedure. We define the global enrichment score by averaging across all genes, similar to the approach used in Fisher's combined probability test. The -th p-value naturally combines and enrichment into a single number, while neither can be used as a score alone. When several reference collections are available, we compute separate scores for each collection and then average the scores at the final step.

As an alternative to the enrichment-based metric, CorrAdjust offers the option to select “balanced precision at ” (denoted ) – the computed over aggregated data (, , , ). This can be viewed as a scaled version of “precision at ” (denoted ), a metric widely used in information retrieval within the machine learning community [24]. A recent study used a very similar quantity, false discovery rate (FDR), to evaluate mRNA–mRNA correlations () [14]. We strongly recommend using the global enrichment score as the default option, as it is significantly more informative and interpretable (see Results). and are also vulnerable to Simpson's paradox; specifically, comparing ratios of aggregated values ( vs. ) instead of comparing individual ratios per gene ( vs. ) is a classical setup used to illustrate the paradox [25]. However, may prove valuable in scenarios not addressed in our study, such as cases where methylation and RNA-seq data are integrated and reference pairs represent promoters and their downstream genes, resulting in uniformly low values and an underpowered enrichment-based approach.

CorrAdjust: PC correction procedure

The goal of our confounder correction procedure is to identify and regress out major hidden factors that drive undesired correlations between non-reference gene pairs. Lemma 3 of Supplementary Note S1 shows that PCs can be derived as orthogonal vectors that maximize the change in the covariance matrix's Frobenius norm upon their residualization. This provides theoretical justification for leveraging PCs to correct the computed correlations.

We consider the transformation of gene expression matrix into matrix of PCs, where is the number of PCs (). We compute a partial correlation between expression levels of two genes controlling for a subset of  by regressing out separately from and , and computing Pearson's correlation on the residuals. The key observation is the mutual orthogonality of PCs, allowing to regress-out from columns of consecutively in arbitrary order.

We use a greedy approach to find a set of PCs to regress out to maximize the global enrichment score defined in the previous subsection. We begin by limiting our search space to include only PCs that capture significant variation. In the current implementation, we use the permutation approach from the sva v3.5 package [1] (“num.sv” function). Other techniques can also be used, e.g., the “scree plot” method [26]. In the first iteration of the greedy algorithm, we loop over all PCs and select the one () with the maximum global enrichment score after its residualization. In the -th iteration, we loop over the remaining PCs and select the one () with the maximum global enrichment score after its residualization from expression data with removed during the previous iterations.

The time complexity of CorrAdjust's greedy optimization scales approximately as with respect to the number of genes (computing and sorting correlations for all gene pairs), with respect to the number of considered PCs, and linearly as with respect to the number of samples. The required memory scales approximately as (most space is used for the correlation matrix).

CorrAdjust: training and validation sets of gene pairs

Once a reference collection is selected, the identities of the reference pairs emerge naturally. Despite PCA being independent of reference collections (unsupervised method), the procedure for selecting an optimal subset of PCs directly optimizes reference set enrichments and is supervised in nature. There are two elements that need to be evaluated to ensure generalizability and avoid overfitting: gene pairs and samples. CorrAdjust automatically partitions gene pairs into 50% training and 50% validation sets by selecting every other pair from the lexicographically sorted list of pairs, ensuring an even distribution of reference pairs between the two sets. The reason there is no test set for gene pairs is the impossibility of creating one that is fully independent of the training set, as the same samples are used for PCA and correlation computations. Training and test sets of samples are described in the next subsection.

The training gene pairs provide the enrichment scores that are used as the objective function during the greedy optimization. In parallel, we evaluate the model using validation gene pairs at each greedy iteration. In a typical scenario, the optimization trajectory represents a quickly ascending curve that eventually reaches a plateau with much slower growth (see Results). We reach the early stopping point (i.e., the final number of PCs to remove) when both of the following criteria are met:

• The validation score improves by less than 5% over two consecutive iterations.

• The validation score differs from the global maximum score by less than 50%.

As the above early-stopping procedure uses two thresholds (5% and 50%), we recommend that users always visually assess whether the automatically determined peak is adequate (e.g., in rare cases, it may stop too soon) and manually refine it if needed.

CorrAdjust: training and test sets of samples

We implemented an interface for using an optional test set of samples with CorrAdjust in the Python scikit-learn module style [27]. We use the training samples to derive gene-wise means for centering, PC loadings, and an optimal set of PCs. We then re-use these data to correct the test samples (if any) and evaluate their gene–gene correlations. We also implemented a training/test-friendly function for normalizing RNA-seq data using the median of ratios algorithm from DESeq2 [7] (vector of gene-wise geometric means is computed using the training set and reused with the test set).

CorrAdjust: gene permutations for negative control runs

We also devised and made available an automatic permutation-based procedure to confirm the absence of overfitting. Specifically, we break the real link between gene expression and reference collections by randomly shuffling gene names of the expression matrix. At the same time, the vector of gene–gene correlations, PCs, and reference collections remain unchanged. This procedure can be used to demonstrate that CorrAdjust does not detect any statistically significant enrichments after correction.

CorrAdjust: using the approach with samples from more than one condition

In some cases, input expression data comprise two or more groups (e.g., disease and control), and the user may be interested in performing differential correlation analysis [28, 29]. CorrAdjust includes built-in functionality to handle multiple user-defined sample groups. First, gene expression is centered separately in each group. Next, PCs are computed across all samples together. The standard CorrAdjust optimization procedure is then applied with a modified objective function. Specifically, correlations and global enrichment scores are computed for each sample group individually, and the scores are averaged across the groups. This approach ensures that the same PCs are residualized from each sample group, preventing the introduction of spurious differential correlations. At the same time, correlations are evaluated for each sample group independently, so CorrAdjust avoids removing real differential correlation signals.

CorrAdjust: software implementation

We provide an implementation of CorrAdjust within a Python module available on GitHub (https://tju-cmc-org.github.io/CorrAdjust). Although this paper focuses on gene expression data, the CorrAdjust implementation uses the traditional machine terminology of “samples” and “features” in its interface, enabling users to apply it to any dataset.

We use the scikit-learn [27] implementation of PCA. The code is highly optimized and parallelized using Numba [30] and SciPy [31] modules and could easily handle datasets with hundreds of samples and 20 000 genes.

One of the several output files that CorrAdjust produces is the confounder-corrected expression matrix. The matrix could be used as an input for any tool for network analysis, e.g., WGCNA [4], making it easy to integrate CorrAdjust into standard workflows of RNA-seq data analysis.

RNA-seq data

We downloaded deep sequencing datasets from three different projects: datasets from The Cancer Genome Atlas (TCGA) generated from cancer patients, datasets from the Genotype-Tissue Expression (GTEx) project generated from healthy donors, and datasets from the Geuvadis consortium generated from lymphoblastoid cell lines (LCLs) of healthy participants to the 1000 Genomes Project. Supplementary Table S1 shows a summary of the data and the full list of used samples and genes.

mRNA data. For TCGA and GTEx, we downloaded pre-mapped gene count tables from the GDC Data Portal (https://portal.gdc.cancer.gov; Data Release 32) and GTEx Portal (https://gtexportal.org/home/; GTEx Analysis V8), respectively. For Geuvadis, we downloaded raw sequencing reads in the FASTQ format (European Nucleotide Archive, PRJEB3366) and generated a read count matrix using STAR v2.7.10b [32] with GRCh38 reference genome and GENCODE v36 annotation. We selected only protein-coding genes for the downstream analysis.

Small RNA data. Small RNA-seq data are available only for the TCGA and Geuvadis samples. We downloaded raw FASTQ files for both projects and comprehensively profiled miRNA isoforms (isomiRs), tRNA-derived fragments (tRFs), rRNA-derived fragments (rRFs), Y RNA-derived fragments (yRFs), and their sequence variants using a slightly modified version of our recent pipeline [33]. Briefly, we used isoMiRmap [34] to map isomiRs, MINTmap [35] to map tRFs, and exhaustive exact mapping for rRFs and yRFs [33]. See Supplementary Methods for a detailed description of the mapping pipeline.

Summary of included datasets. For TCGA, we used 8 981 primary tumor samples across 32 cancer types with both mRNA-seq and small RNA-seq data available (17 962 datasets). For GTEx, we used 6 295 samples across 11 healthy tissues. For Geuvadis, we used 403 samples with both mRNA-seq and small RNA-seq data available (806 datasets). We specifically excluded Geuvadis datasets generated by laboratory “number six” as they used more sequencing cycles in small RNA-seq than the rest of the datasets. We also excluded datasets that corresponded to technical replicates.

Data normalization. We separately processed each TCGA cancer type, GTEx tissue type, and Geuvadis data. We randomly split samples into 50% training and 50% test sets. We kept only mRNAs with a median abundance ≥ 1 transcript-per-million (TPM) and small RNAs with a median abundance ≥ 10 reads-per-million (RPM). We normalized filtered raw read count matrices with the median of ratios algorithm [7], separately for mRNA-seq and small RNA-seq. Then, we combined mRNA and small RNA matrices (TCGA, Geuvadis) and winsorized abundances of each feature (mRNA or small RNA) using lower 5% and upper 5% expression values (winsorization helps mitigate the effect of outliers in linear regression and Pearson correlation computations). Finally, we applied the transform to the winsorized and normalized counts. We used only training samples to derive all the parameters used for filtering (median TPM or RPM), median of ratios normalization (gene-wise geometric means), and winsorization (expression quantiles).

Reference collections

mRNA–mRNA pairs. We downloaded reference gene sets in the GMT format from MSigDB v2023.2 [36]. We used all the gene sets from the Canonical Pathways and Gene Ontology (GO) subsets of MSigDB as reference collections for running CorrAdjust. In addition, we tested the models trained using the Canonical Pathways and GO on an independent reference collection TRRUST v2 [37], containing 795 gene sets of experimentally validated transcription factor targets. Supplementary Note S3, Supplementary Fig. S3, Supplementary Fig. S4, and Supplementary Fig. S5 describe the analysis showing that CorrAdjust is robust with respect to the number of gene sets and gene set sizes in the reference collections.

miRNA–mRNA pairs. We used two reference sets for miRNA–mRNA pairs: experimentally validated miRNA targets from TarBase v9.0 [20] and purely sequence-based predicted miRNA targets by RNA22 v2.0 [38]. In both cases, we used only the subset of canonical isomiRs [39].

Alternative confounder correction methods

We compared CorrAdjust with several other methods that also correct for hidden and known confounders.

sva_network. Parsana et al. [14] suggested residualizing the “first N” statistically significant PCs from gene expression data before computing correlations. At the time of running these benchmarks (October 2024), the publicly available implementation of this approach (“sva_network” function from the sva package) incorrectly centered the input data before running PCA. Because of this, we only used the sva package to estimate the number of PCs to regress out (“num.sv” function) and then used CorrAdjust for the actual residualization, mimicking sva_network behavior. We ran a PCA using the training samples and evaluated the corrected correlations using the test samples.

Linear regression on known confounders. We used multiple linear regression to residualize known variables from expression data. The variables included:

• sex, race, and age for the TCGA datasets;

• sex, age, RNA integrity number (RIN), and post-mortem interval (PMI) for the GTEx datasets;

• sex, population, and laboratory number for the Geuvadis datasets.

We fit regression models using training samples only. We discarded samples with missing information (TCGA only). We recomputed CorrAdjust scores on the remaining samples (while keeping PCs the same) to ensure comparability.

COBRA. We ran a Python implementation of the COBRA tool [13] from the netZooPy v0.10.6 package using the same known confounder variables. Since the tool does not provide an intuitive way of using training and test samples, we directly fit COBRA on the test samples.

Results

CorrAdjust is a novel method for identifying and correcting hidden confounder variables in transcriptomic correlation studies. It identifies a subset of confounder PCs in a greedy iterative manner. CorrAdjust inputs a gene expression matrix and one or more reference collections of gene sets (Fig. 1A, left panel). Each iteration of the method begins by ranking all gene–gene pairs according to the correlation of their expression values and marking the top % entries of the ranked pairs list as “highly ranked” (Fig. 1A, middle panel). Next, CorrAdjust identifies the strongest confounder among all PCs by maximizing the enrichment of gene pairs that are jointly present in one or more gene sets from the reference collection (the “reference pairs”) among the highly ranked pairs (Fig. 1A, right panel). The rationale behind uncovering hidden confounders among the PCs of the gene expression matrix is based on our theoretical proof that PCs are the factors with the greatest impact on the gene–gene covariance matrix upon their residualization (Lemma 3 in Supplementary Note S1).

Key to CorrAdjust is our novel metric for evaluating enrichment of reference pairs among the highly correlated pairs. To compute the metric, the ranked list of all gene pairs is first segmented into blocks, with the -th block corresponding to all gene pairs formed by the gene with index (Fig. 1B, left panel). Within each block, we use the hypergeometric distribution framework to quantify whether reference gene pairs are overrepresented among the highly ranked ones. Finally, p-values derived for each gene are adjusted for multiple testing, and negative logarithm values of adjusted p-values are averaged to generate a global enrichment score (Fig. 1B, right panel).

In the next two subsections, we will apply CorrAdjust to real RNA-seq data.

Case 1: correcting mRNA–mRNA correlations in whole blood datasets from GTEx

After filtering for low-expressed genes, the data contain expression levels of 10 021 protein-coding genes across 755 samples. We further split the samples into training (n = 378) and test (n = 377) sets and 50 205 210 gene pairs into equal training and validation sets. There are 19 statistically significant PCs according to the permutation test (see Methods).

We ran CorrAdjust using two reference collections: Canonical Pathways and Gene Ontology (GO). Fig. 2A shows optimization curves for the training set of samples, with the Y-axis showing the global enrichment score for Canonical Pathways, GO, and their average. The numbers on top of the curves show the identities of PCs selected at each iteration. After 3 rounds of greedy selection, the score reaches a plateau, and PC₁, PC₂, and PC₃ are selected for elimination. Canonical Pathways and GO scores follow a very similar pattern, suggesting that both collections capture a common enrichment signal. Importantly, scores computed using training and validation gene pairs are almost identical, showing a complete absence of overfitting with respect to gene pairs. Next, we evaluated the same trajectory of PC correction using the test set of samples (all gene pairs together). The test scores closely resemble training scores (Supplementary Fig. S6A).

Figure 2.

Application of CorrAdjust to mRNA–mRNA correlations computed over the GTEx whole blood dataset. (A) Optimization trajectory of CorrAdjust (training samples). Each iteration (X-axis) corresponds to the selection of one PC (number on top of the curves). Iteration 0 corresponds to uncorrected data. The Y-axis shows global enrichment scores for Canonical Pathways, Gene Ontology, and the average of these scores. (B and C) Volcano plots before and after CorrAdjust correction (test samples, all gene pairs, Canonical Pathways). Each marker represents one gene. The marker area stands for the number of highly ranked gene pairs involving the corresponding gene (top % approach, see “Methods”). The top 10 genes by adjusted p-value are annotated. (D and E) Kernel density estimation and cumulative distribution function of gene–gene correlations before and after CorrAdjust correction. Solid lines correspond to the pairs of mRNAs that are jointly present in at least one Canonical Pathway. (F) Correlations between identified confounder PCs and known covariates (training samples): post-mortem interval (PMI), RNA integrity number (RIN), sequencing depth, sex, and age group. (G and H) A representative gene pair with a high spurious correlation driven by a confounder (PMI), which disappears after residualizing PC₁ from the expression data. (I) Despite a strong impact on a gene–gene correlation, PMI does not distort differential expression analysis (female versus male donor comparison). The p-values were computed using Student’s t-test applied to normalized read counts (no PMI correction).

Figs 2B and C show gene-level Canonical Pathways enrichment scores before and after correction, respectively (test set of samples, all gene pairs). Each dot represents an individual gene and depicts its three main quantities:

• the Y-axis represents the adjusted p-value (a dot is colored in orange if the adjusted p-value is ≤ 0.05);

• the marker size represents the number of significantly correlated genes with the corresponding gene ( in the notation of Fig. 1B and “Methods”);

• the X-axis represents balanced precision.

As it is evident from the plots, the removal of PCs strongly increases pathway enrichments across many genes.

Fig. 2D shows the global impact of PC correction: gene–gene correlations are significantly shrunk toward zero. Together with increased enrichment scores, such shrinkage indicates the removal of a significant source of unwanted variance, driving high false positive correlations. Correlations of reference gene pairs shared by at least one Canonical Pathway (solid lines) are, on average, higher than for non-reference gene pairs (dashed lines). Fig. 2E provides more insight into the distribution's tails on a cumulative logarithmic scale. After correction, the difference between reference and non-reference gene pairs becomes noticeably more pronounced (note how the gap between solid and dashed lines increases toward higher correlations). The dotted horizontal gray line shows the used % threshold; before correction, it corresponds to the absolute correlation of 0.8 and becomes 0.55 after correction.

Some of the residualized PCs could be clearly interpreted (Fig. 2F and Supplementary Fig. S7). For example, PC₁ differentiates between samples obtained post-mortem (positive PMI) and pre-mortem (negative PMI), which was previously reported for the same GTEx whole blood dataset [40]. PC₂ and PC₃ also have statistically significant associations with known variables (PMI, RIN, sex, age), albeit with much weaker effect sizes. Below, we will show that adjusting for known confounders results in worse scores than CorrAdjust.

Fig. 2G shows an example of genes PUDP and PRSS36 with a high spurious correlation (Pearson's ) driven by a confounder (PMI). The correlation becomes insignificant when adjusted for PC₁ (Pearson's , Fig. 2H). This example is representative, as 97% of the 5 991 genes involved in pairs with uncorrected are significantly correlated with PC₁. After elimination of PC₁, the values of the absolute correlations dropped below 0.4 for more than half (53%) of such pairs.

Note that while PMI significantly distorts the gene–gene correlations, it does not necessarily have a comparably profound impact on differential expression analysis. For example, the comparison of uncorrected expression levels of the above two genes between female and male donors produces correct results (Fig. 2I): PUDP from the X chromosome is more expressed in females (Student's t-test ) consistently with reports of its escape from X chromosome inactivation [41], and PRSS36 from the autosomal chromosome 16 is not differentially expressed (. Testing the same associations while adjusting for PMI strengthens the PUDP p-value () but does not change the results qualitatively. This is counterintuitive but expected, as PMI distribution does not differ between males and females (Student's t-test ).

Case 2: correcting miRNA–mRNA correlations in LCL datasets from Geuvadis

For the second example, we ran CorrAdjust on 403 LCL samples for which both mRNA-seq and small RNA-seq datasets are available. This time, we used the following reference collections: the miRNA–mRNA pairs of TarBase [20] (experimentally validated miRNA targets) and the miRNA–mRNA pairs generated by RNA22 [38] (sequence-based computationally predicted miRNA targets). Since miRNAs downregulate their target genes, we assigned the highest ranks to the most negative miRNA–mRNA correlations.

CorrAdjust selects 8 PCs to be eliminated from the data: PC₁, PC₃–PC₆, PC₉, PC₁₀, and PC₁₂ (Fig. 3A). There is no evidence of overfitting both from the validation pairs (Fig. 3A) and the test samples (Supplementary Fig. S6B) perspectives. Strikingly, in the absence of any correlation correction, none of the miRNAs show a statistically significant overrepresentation of predicted targets among anti-correlated miRNA–mRNA pairs, for either the TarBase or the RNA22 reference sets. However, after correlation correction, most miRNAs show statistically significant enrichment: 78% of miRNAs for TarBase and 60% of miRNAs for RNA22. Figs 3B and C show volcano plots with TarBase miRNA target enrichments before and after the CorrAdjust application.

Figure 3.

Application of CorrAdjust to miRNA–mRNA correlations computed over the Geuvadis LCLs dataset. (A) Optimization trajectory of CorrAdjust (training samples). Each iteration (X-axis) corresponds to the selection of one PC (the number on top of the curves). Iteration 0 corresponds to uncorrected data. The Y-axis shows global enrichment scores for TarBase, RNA22, and the average of these scores. (B and C) Volcano plots before and after CorrAdjust correction (test samples, all pairs, TarBase). Each marker represents one miRNA. The marker area stands for the number of highly ranked miRNA–mRNA pairs involving the corresponding miRNA (top % approach, see “Methods”). The top 10 miRNAs by adjusted p-value are annotated. (D and E) Kernel density estimation of miRNA–mRNA correlations before and after CorrAdjust correction. Solid lines correspond to pairs of miRNAs and their experimentally validated targets from TarBase. (F) Correlations between identified confounder PCs and known covariates (training samples): mRNA sequencing depth, small non-coding RNA (sncRNA) sequencing depth, composition of sncRNA-omes, laboratory ID, sex, and population.

This dramatic improvement could also be noted at the level of whole miRNA–mRNA correlation distributions (Fig. 3D–E). Before the CorrAdjust correction, correlations between miRNAs and their experimentally validated targets (solid red line) tend to be more positive than for non-targets (dotted red line), contradicting the biological knowledge. The correction flips this pattern, putting experimentally validated pairs (solid green line) to the left of non-targets (dotted green line). Supplementary Fig. S8 shows intermediate states between uncorrected and fully corrected data. Adjusting for PC₁ removes the positive gap between the correlation distributions of miRNAs with their targets and non-targets, while adjusting for the subsequent PCs gradually creates the expected negative gap. Interestingly, PC₁ essentially represents % of isomiRs or % of rRFs in the small RNA-seq datasets (Fig. 3F), suggesting that the positive bias of miRNA–mRNA correlations in Fig. 3D is primarily driven by unwanted variance in the molecular composition of the small RNA datasets. The subsequent PCs have weak to moderate associations with both technical variables (sequencing depth, laboratory) and donor populations (Fig. 3F).

CorrAdjust outperforms other methods when correcting miRNA–mRNA correlations

We evaluated the ability of CorrAdjust and several competing methods to improve enrichments of TarBase and RNA22 targets among negative miRNA–mRNA correlations using 8 981 primary tumor samples spanning 32 TCGA cancer types. The competing methods included the sva_network method [14], correction for known factors with linear regression (see “Methods” for the list of factors), and correction for known factors with COBRA [13]. As a negative control experiment, we ran CorrAdjust on expression tables with shuffled mRNA names. We used % threshold for all datasets (Supplementary Note S2 describes the selection of ).

Fig. 4 shows the benchmarking results computed over test samples and all miRNA–mRNA pairs: the left panel compares CorrAdjust with hidden confounder correction methods, and the right panel compares it with known factor correction approaches. In virtually all cases, CorrAdjust outperforms the other techniques and shows a striking increase in global enrichment score (X-axis) compared to uncorrected correlations.

Figure 4.

Benchmarking results of CorrAdjust and alternative methods using miRNA–mRNA correlations (test samples, all pairs). (A) Methods for hidden confounders correction. (B) Methods for known covariates correction. The X-axis shows the global enrichment score (average of TarBase and RNA22). The columns to the right of the main plots show the number of PCs adjusted by CorrAdjust, the number of PCs adjusted by the sva_network approach, the number of adjusted known covariates, and the number of samples in the training set.

The median number of PCs regressed out by CorrAdjust is 8. Even though the sva_network approach removes significantly more PCs (median = 18), it results in lower than raw scores in 24/33 datasets (Fig. 4A). Known factor correction approaches also perform poorly, with linear regression and COBRA generating worse scores in the 16/33 and 21/33 datasets, respectively (Fig. 4B). The same trends persist with scores computed over training samples and training miRNA–mRNA pairs (Supplementary Fig. S9A–B), showing that the low performance of these methods is not due to poor generalizability from training to test samples. Interestingly, the sva_network “first N” approach shows non-stable behavior with respect to the number of PCs removed, so it cannot be improved by simply regressing out fewer (but predefined) number of PCs (see representative examples in Supplementary Fig. S9C–E).

Importantly, CorrAdjust training and test sample scores are extremely correlated (Spearman’s , ), as well as the training and validation miRNA–mRNA pair scores (, ). Additionally, all negative control experiments show near-zero scores even when evaluated with training samples and training gene pairs (Supplementary Fig. S9A). Taken together, there are no signs of overfitting when conducting supervised PC selection with CorrAdjust.

There is a positive correlation between the CorrAdjust enrichment score and training set sample size (Spearman’s , ), with the lowest scores corresponding to datasets with < 50 samples. At the same time, the correlation is not linear and diminishes as sample size increases: the quadratic term of the model is negative and statistically significant (), showing that an adequate sample size is required for a successful CorrAdjust application.

CorrAdjust outperforms other methods when correcting mRNA–mRNA correlations while eliminating fewer PCs

We next evaluated CorrAdjust and competing methods using mRNA–mRNA correlations and Canonical Pathways and GO as the reference collections (Fig. 5). This time, we also analyzed 6 295 datasets from 11 healthy tissues from the GTEx dataset (bottom panels in Fig. 5). We used % threshold for all these tests (see Supplementary Note S2).

Figure 5.

Benchmarking results of CorrAdjust and alternative methods using mRNA–mRNA correlations (test samples, all pairs). (A) Methods for hidden confounders correction. (B) Methods for known covariates correction. The top panels show TCGA and Geuvadis datasets, and the bottom ones show GTEx data. The X-axis shows the global enrichment score (average of Canonical Pathways and Gene Ontology). The columns to the right of the main plots show the number of PCs adjusted by CorrAdjust, the number of PCs adjusted by the sva_network approach, the number of adjusted known covariates, and the number of samples in the training set.

CorrAdjust again outperforms all competing methods when applied to the TCGA/Geuvadis (Fig. 5; top panels) and GTEx (Fig. 5; bottom panels) datasets. In almost all analyzed datasets, CorrAdjust marks only a few PCs as confounders (median = 1 PC). Interestingly, the differences between uncorrected and corrected values are not as high as in the miRNA–mRNA case. CorrAdjust enrichment score again positively and non-linearly correlates with the training set sample size (Spearman’s , ; negative quadratic term ), with the lowest scores corresponding to datasets with < 50 samples.

Correcting for known confounders with linear regression has very little impact on enrichment scores for most datasets (Fig. 5B). The sva_network and COBRA methods show significantly lower scores than uncorrected data in the vast majority of datasets, which is also the case with the training data (Supplementary Fig. S10).

Finally, we assessed whether the increase in the enrichment score after CorrAdjust correction generalizes to reference collections that were not used during model training. To do so, we evaluated the above models on TRRUST, a manually curated database of experimentally validated target genes of transcription factors. The enrichment scores again increased significantly upon CorrAdjust application in most datasets (Fig. 6A; Student’s t-test ). Moreover, the magnitude of the increase in enrichment score after the correction was highly correlated between the training reference collections (Canonical Pathways and GO) and TRRUST (Fig. 6B; Spearman’s , ). This confirms that the PCs identified and eliminated by CorrAdjust represent true confounding signals rather than artifacts specific to the reference collections used for training metric computation.

Figure 6.

Evaluation of CorrAdjust models trained using Canonical Pathways and Gene Ontology on an independent TRRUST reference collection. Each marker on both panels represents one TCGA cancer type, GTEx tissue, or the Geuvadis collection. (A) Global enrichment scores before and after CorrAdjust correction. The p-value in the top-right corner was computed using paired sample Student’s t-test. (B) Relative differences between CorrAdjust-corrected and uncorrected global enrichment scores computed using training reference collections (X-axis) and independent TRRUST collection (Y-axis). The top-right corner shows Spearman’s correlation and the associated p-value.

Discussion

We developed CorrAdjust, a PCA-based method for correcting hidden confounders in transcriptomic correlation analyses. Confounder PC identification is guided by a novel enrichment-based metric designed to assess whether highly correlated gene pairs are enriched in user-specified reference pair sets (Fig. 1). The metric is derived from individual gene scores, which can be visualized through volcano plots. These, along with other visualizations available in the CorrAdjust Python implementation, significantly enhance the interpretability of our approach (Figs 2 and 3). The corrected gene expression matrix produced by CorrAdjust can be used with any tools for downstream network analysis, such as WGCNA [4].

CorrAdjust clearly outperforms competitor methods for the correction of hidden and known confounders in the largest, to date, benchmark experiment. It includes over 25 000 human mRNA and small RNA sequencing datasets, spanning 32 cancer types from TCGA, 11 healthy tissues from GTEx, and LCLs from the Geuvadis consortium (Figs 4 and 5). Our collection covers all the GTEx tissues included in the previously reported mRNA–mRNA correlation benchmarks [11, 12, 14] (we are not aware of similar miRNA–mRNA studies). Consistently with the most recent report [12], correction for known covariates with linear regression has the least pronounced effect on correlations compared to the uncorrected data, while the sva_network method [14] performs worse than the uncorrected data for most datasets. The recently released COBRA tool [13] also shows scores mostly below the baseline.

At first sight, these data seem to differ from the results by Parsana et al., who reported significantly decreased FDR (equivalently, increased precision) with the GTEx data and Canonical Pathways collection upon sva_network correction [14]. The key difference between our analyses is that we evaluated raw and corrected correlations using the same number of gene pairs (% pairs with the highest absolute correlations in each case). On the other hand, Parsana et al. evaluated gene pairs under the same WGCNA cut-height, essentially using the same correlation cutoff. In Supplementary Note S4 and Supplementary Fig. S11, we show that while doing so improves precision, it is incorrect because it ends up considering orders of magnitude fewer gene pairs since the sva_network correction strongly shrinks the correlations toward zero.

We observed the most profound impact of CorrAdjust on the enrichment scores with miRNA–mRNA correlations, where the reference pairs comprise miRNAs and their target mRNAs (Fig. 4). Consistent with the existing literature [42–44], the baseline values for many of the datasets are very low. Geuvadis LCL data are the most extreme case, showing no miRNAs with a statistically significant overrepresentation of target mRNAs among the negative miRNA–mRNA correlations (Fig. 3B). After applying CorrAdjust, 78% of the miRNAs in LCLs became statistically significant (Fig. 3C), and the whole distribution of miRNA–mRNA correlations shifted to the biologically expected direction of negative correlations (Fig. 3D–E). Although we used only miRNA targets to compute the scores, the corrected expression data can be used to derive correlations between other sncRNA types (e.g., tRFs, rRFs, or yRFs) and mRNAs to get insights about the functions of these much less studied molecules [39]. We conjecture that the stronger effect of CorrAdjust in the miRNA–mRNA case is due to more confounders affecting expression values, as the data are integrated from two distinct sets of experiments (small-RNA- and mRNA-seq), which involve different library preparation protocols and separate sequencing runs. Consequently, we believe CorrAdjust might excel at integrating RNA expression with other omics data modalities, e.g., bisulfite sequencing, ATAC-seq, or ChIP-seq.

CorrAdjust differs from existing correlation correction methods by being reference-guided – it selects a subset of PCs in a supervised manner to distinguish between confounding and biological signals. In classical machine learning terms, CorrAdjust tackles a binary classification problem, assigning “reference” or “non-reference” labels to gene pairs based on a single feature: Pearson’s correlation. Note that biological samples are not explicitly referenced in this context, as their influence is inherently embedded within the correlations. To prevent overfitting and measure the generalizability of PC correction, we designed several specific methods. First, CorrAdjust automatically splits gene pairs (objects being classified) into 50% training and 50% validation sets, using the validation pairs for early stopping to determine the optimal number of PCs to adjust. We avoid the term “test set” for gene pairs, as the same samples are used for both PCA and correlations computation, thus compromising the independence of gene pairs. Second, CorrAdjust supports a test set of samples by re-using PC coefficients and identities derived from the training sample set. Next, CorrAdjust includes an automated negative control experiment option that shuffles gene names while preserving expression data (and, thus, PCs and correlations) and reference collection structures. Finally, the user can tune a model using one set of reference collections and evaluate it on independent collections (Fig. 6). We observed no overfitting with all the used datasets in the benchmark experiment using all four techniques.

CorrAdjust has several limitations:

• As with any correlation-based method, the main limitation of CorrAdjust is the requirement of a relatively large sample size; the benchmark runs showed reliable results when at least 50 samples were used.

• The confounders identified by CorrAdjust are represented by the PCs and do not explicitly carry an interpretation. Some of these PCs may be associated with known variables, and we encourage users to examine these associations (e.g., as shown in Figs 2F and 3F) to better understand what is being corrected. At the same time, many technical confounders are latent and/or hard to correct without using PCA. One example is a strong association of small RNA expression with the library preparation plate in TCGA [45]. Explicitly accounting for this variable is technically challenging, as there are numerous plates, each containing only a few samples of the same cancer type. Also, plate information is not available for non-TCGA data (Geuvadis, GTEx). Another example is the profound impact of RIN on both mRNA and small RNA expression [46, 47]. The RIN/expression dependency is non-linear (it is stronger when RIN is low) and, thus, is challenging to explicitly account for. Also, RIN is available only for a small subset of TCGA datasets and is not available for the Geuvadis data.

• CorrAdjust is specifically designed to correct gene–gene correlations, and its output should not be used with differential expression analysis tools, such as DESeq2 [7] or edgeR [8], as it can eliminate true biological differences in average gene expression levels between conditions of interest. Specialized tools, such as sva [1] of RUV [2], should be used instead.

• Another limitation of our analysis is the choice of reference collections. Here, we used Canonical Pathways, Gene Ontology, and TRRUST to evaluate mRNA–mRNA correlations, despite neither of the resources positioning as a database of reference gene pairs for transcriptome-based correlation analysis. We avoided using the database of gene pairs GIANT [48], as one of the main contributors to its gene–gene connectivity score is “vanilla” Pearson's correlation between expression levels computed over all Gene Expression Omnibus datasets, which might be heavily affected by common biases in transcriptomic analyses.

• Finally, CorrAdjust is developed and benchmarked using bulk sequencing data; its applicability to single-cell or spatial transcriptomics data is a direction of future research and would likely require using specialized correlation coefficients [49] to tackle data sparsity.

In conclusion, we believe confounder correction should be a standard practice for computing transcriptomic correlations. Such correction is especially vital when integrating different data modalities, such as the miRNA–mRNA case, as CorrAdjust increases the goodness of fit of correlations to experimentally validated data by orders of magnitude.

Supplementary data

Supplementary data are available at NAR online.

Conflict of interest

The authors declare that there are no conflicts of interests to disclose.

Funding

This study was supported by Thomas Jefferson University Institutional Funds (I.R.), the National Foundation for Cancer Research (NFCR) grant (I.R.), a Melanoma Research Institute of Excellence (MRIE) at Thomas Jefferson University grant (I.R.), and NIH grants R21CA280575 (I.R.) and R01HG012784 (I.R.). Funding to pay the Open Access publication charges for this article was provided by a mix of internal and grant funds.

Data availability

All of the analyzed datasets are in the public domain, and their identifiers are listed in Supplemental Table S1. The official documentation and tutorials for the CorrAdjust are available at https://tju-cmc-org.github.io/CorrAdjust. The source codes of CorrAdjust are available at https://github.com/TJU-CMC-Org/CorrAdjust and https://doi.org/10.5281/zenodo.15021849.