Comments on the Rank Product Method for Analyzing Replicated Experiments

Abstract

Breitling and colleagues [1] introduced a statistical technique, the rank product method, for detecting differentially regulated genes in replicated microarray experiments. The technique has achieved widespread acceptance and is now used more broadly, in such diverse fields as RNAi analysis, proteomics, and machine learning. In this note, we relate the rank product method to linear rank statistics and provide an alternative derivation of distribution theory attending the rank product method.

Introduction

In an influential paper, Breitling and colleagues [1] introduced a statistical technique for detecting differentially regulated genes in replicated microarray experiments. Their rank product method entails ranking expression levels within each replicate, then computing the product of the ranks across the replicates. The rank product is then compared to its sampling distribution under a permutation model for subsequent inference. The rank product method appears to be robust, with higher sensitivity and specificity than t-test methodology and desirable operating characteristics, as demonstrated in extensive numerical studies [2-5]. Although developed originally for microarrays, the rank product method has found widespread acceptance in diverse settings, e.g., RNAi analysis [6], proteomics [7], and machine learning model selection [8].

The purpose of this note is to provide an alternative method for establishing distributional properties of the rank product statistic, based on the classical notion of linear rank statistics [9]. This approach affords insight into theoretical properties of rank product method, and leads to useful extensions.

The rank product statistic

We briefly describe the rank product statistic. We start with expression levels for n genes from k replicate samples. Denote the expression level for the i^th gene in the j^th replicate by X_ij, where 1 ≤ i ≤ n, 1 ≤ j ≤ k. Next, rank the expression levels X_1j, X_2j, …, X_nj in each replicate j, forming R_ij = rank(X_ij), 1 ≤ R_ij ≤ n. [The ranking is such that genes with the smallest ranks are the “most interesting” from a biological perspective.] Then, Breitling's rank product statistic for the i^th gene is, up to a normalization constant, the product

$$R{P}_i=\prod _{j=1}^k{R}_{ij.}$$

Genes associated with sufficiently small RP values would be marked for further consideration, and Breitling and colleagues [1] posit a statistical formalism for such a determination. In particular, they propose a permutation approach to calculate the distribution of the RP_i under the null hypothesis that the X_ij are identically distributed (exchangeable) within each of the k independent replicates.

An alternative formulation

An equivalent statistic to RP_i is the monotone transformation

$$\text{log}(R{P}_i)=\sum _{j=1}^k\text{log}(R_{ij})$$

Monotonicity ensures that achieved significance levels of RP_i and log(RP_i) are identical. There are two key notions reflected in this transformation. First, since the k replicates are independent, log(RP_i) is the sum of k independent, identically distributed random variables under the null hypothesis. More fundamentally, the log transformation demonstrates that the rank product method engenders replacement of the ranks R_ij within each replicate by rank scores a_j(R_ij), where the score function here is given simply by a_j(i)=log(i), 1 ≤ i ≤ n, 1 ≤ j ≤ k.

Score functions

Probably the most common score function for linear rank statistics is the Wilcoxon rank score, a(i)=i, 1 ≤ i ≤ n. If this score function were used rather than the log function, then the resulting test statistic, entailing summation of ranks across the k replicates, is equivalent to Wise's genome scan meta-analysis (GSMA) statistic [10]. Wise and colleagues had invoked a clever inclusion/exclusion argument for deriving the exact null distribution of the GSMA statistic; Koziol and Feng [11,12] had used a more classical approach with probability generating functions for determination of the exact distribution of the GSMA statistic.

A second common choice for score function are the normal or van der Waerden scores [9]

$$\begin{array}{cc}a\left(i\right)={\Phi }^{-1}(\frac{i}{n+1}),& 1\le \mathrm{i}\le \mathrm{n},\end{array}$$

where Φ^-1 is the inverse of the standard normal distribution function. Within the family of univariate linear rank statistics, the resulting normal scores statistic is more powerful against location shift alternatives with normally distributed data than the Wilcoxon statistic [9]. In practice, one might expect the normal scores statistic to perform comparably to the t-test method.

Much of the richness and diversity of linear rank statistics arises from adoption of different score functions into the underlying construct. The rank product method is associated with a log score function, but other functions could easily be used, as we note later. At first glance, van der Waerden scores might be a reasonable choice if linear shifts in location of expression levels are expected. The log score function would seem more appropriate if shifts in the shape of the underlying distribution are also likely to occur.

Distribution theory

A generalized version of the rank product statistic is given by

$$\begin{array}{cc}{Y}_i=\sum _{j=1}^k{a}_j(R_{ij}),& 1\le \mathrm{i}\end{array}\le \text{n},1\le \mathrm{j}\le \mathrm{k}.$$

Note that under the null hypothesis of exchangeability within replicates, independence across replicates, the Y_i are identically distributed. Also, in practice, one would likely choose the score functions a_j(.) to be identical, all equaling a(.), say, though this is not necessary. For particular choices of a(.), exact distribution theory concerning Y_i is readily available through the notion of probability generating functions [6, 7]. Wilcoxon scores are particularly amenable to this construction; details are in [6, 7].

Alternatively, one could invoke a normal approximation to the distribution of Y_i, which is the sum of k independent random variables, identically distributed if all a_j(.) = a(.). In this regard, one could improve on the normal approximation via Edgeworth correction for skewness and kurtosis. Details are given in the Appendix. Note that weights could easily be incorporated into the construct for the Y_i: one might, for example, weight the replicates differently on a priori grounds.

A simple approximation for rank products

There is no need to invoke the formalism outlined above for the null distribution of the rank products, as there exists a remarkably simple approximation, which we now outline. First, note that R_ij/(n+1) is approximately uniformly distributed on the unit interval (0,1), the approximation improving as n (the number of genes) increases. Next, let U_j denote a uniform random variable [that is, U_j is uniformly distributed on (0,1)]. Then -log(U_j) has an exponential distribution on the positive real line with scale parameter 1, commonly denoted Exp(1). The key here is that the Exp(1) distribution is a particular case of a gamma distribution, namely, a Gamma(1,1) distribution. [The gamma distribution is a two-parameter continuous probability distribution. The two parameters are commonly referred to as the shape parameter k and the scale parameter θ, and the distribution is denoted Gamma(k,θ).] The sum of independent, identically distributed exponentials is also gamma distributed, with the same scale parameter, but an altered shape parameter: in our setting, $-\text{∑}_{j=1}^k\text{log}(U_j)$ has a Gamma(k,1) distribution.

How does all this relate to the rank product? Recall that we are interested in “sufficiently small” values of RP_i. We have the following steps:

$$\begin{array}{c}\text{Prob}({\text{RP}}_{\text{i}}\le \text{t})=\text{Prob}(\text{log}({\text{RP}}_{\text{i}})\le \text{log}(\text{t}))\\ =\text{Prob}(-\text{log}({\text{RP}}_{\text{i}})\ge -\text{log}(\text{t}))\\ =\text{Prob}(-\text{log}({\text{RP}}_{\text{i}})+\text{k}\ast \text{log}(\text{n}+\text{1})\ge -\text{log}(\text{t})+\text{k}\ast \text{log}(\text{n}+\text{1}))\\ \approx \text{Prob}(\text{Gamma}(\text{k},\text{1})\ge -\text{log}(\text{t})+\text{k}\ast \text{log}(\text{n}+\text{1})).\end{array}$$

That is, we may easily determine approximate critical values for RP_i by back transformation from the associated probability that a random variable distributed as Gamma(k,1) exceeds the cutoff value specified in the last equation above.

To investigate the adequacy of the gamma approximation, we examined the following cases: n=1000, 5000, 30000 (genes), and k=3, 5, 10 (replicates). For each of these combinations of (n,k), we generated 10000 independent RP values, transformed the values to -log(RP)+k*log(n+1), and compared the empirical distributions of the transformed values to their respective Gamma(k,1) approximations. We present probability plots of the empirical distributions in Figure 1. Clearly, agreement is excellent over the range of support, indicating that the empirical distributions of the RP values are indeed well-approximated by the corresponding gamma distributions. We caution that estimation of extreme tail probabilities of the RP statistics from either the permutation approach of Breitling and colleagues or the simulation approach utilized here is rather imprecise, even with moderately large permutation or simulation runs. Our preference for tail probabilities is the simple gamma approximation, which should be quite satisfactory in settings with reasonably large numbers of genes and replicates.

Figure 1.

Probability plots of the empirical distributions of the transformed RP statistics, based on 10000 simulations, versus corresponding quantiles of the approximating gamma distributions. A. n=10000 genes, k=3 replicates. B. n=10000 genes, k=5 replicates. C. n=10000 genes, k=10 replicates. D. n=5000 genes, k=3 replicates. E. n=5000 genes, k=5 replicates. F. n=5000 genes, k=10 replicates. G. n=30000 genes, k=3 replicates. H. n=30000 genes, k=5 replicates. I. n=30000 genes, k=10 replicates. Points should form approximately a straight line along the diagonal; departures from this straight line indicate departures from the specified distribution. In each instance, the approximating gamma distribution has scale parameter 1, and shape parameter equal to the number of replicates.

We note in passing that gamma probabilities are readily obtained in Excel, as both the gamma cumulative distribution function and inverse cumulative distribution function are built-in mathematical functions. Hence individuals utilizing Breitling's Excel template for calculation of rank products would not require additional specialized software for gamma calculations.

In summary, the underlying theory of linear rank statistics provides insight into distributional properties of the rank product method. Extensions of the rank product method involving other score functions are straightforward, and may afford enhanced power properties against particular alternatives of interest [9, 13]. A simple gamma approximation is available for the log-transformed rank product statistic, and should be quite satisfactory in practice.

Appendix

For simplicity, assume a_j(.)=a(.) for all j. Then, the Y_i are identically distributed, so it suffices to examine the distribution of Y₁. Y₁ is the sum of k independent, identically distributed random variables, so consideration can be further restricted to the distribution of Z=a(R₁₁) under the null hypothesis that all permutations of (R₁₁, R₂₁, …, R_I1) are equally likely. Moments of Z are easily found: the m^th moment is given by

$${\mu }_m^{\prime }=E\left(Z^m\right)=\left(1/n\right)\ast {\sum _{i=1}^n\left[a\left(i\right)\right]}^m$$

for any positive integer m. Closed form formulas are available for particular choices of a(.), e.g., Wilcoxon scores, otherwise, numerical calculation via spreadsheet is straightforward.

Next, consider the cumulants κ_m of Z. The cumulants of a distribution are closely related to the distribution's moments [14]. For example, if Z has an expected value μ=E(Z) and a variance σ²=E(Z-μ)², then these are the first two cumulants, that is, μ=κ₁ and σ²=κ₂. κ₃ and κ₄ are commonly referred to as the skewness and kurtosis respectively of the distribution of Z. Stuart and Ord [14, Sec. 3.14] give formulas for cumulants in terms of moments and conversely. It is advantageous to work with cumulants rather than moments because of the additivity property of cumulants in the independence setting: each cumulant of a sum of independent random variables is the sum of the corresponding cumulants of each of the random variables. Here, each cumulant of Y₁ is merely k * the corresponding cumulant of Z.

In addition, a distribution with cumulants κ_m can be readily approximated through an Edgeworth series representation [9]. Let W_j=(Z_1j-μ)/σ and S_k = k^1/2(Z̄ − μ) / σ Then the Edgeworth expansion for S_k is given by

$$Pr(S_k\le x)=\Phi (x)+k^{-1/2}{p}_1(x)\phi (x)+k^{-1}{p}_2(x)\phi (x)+\mathrm{\dots }$$

where

$$\begin{array}{c}{p}_1(x)=-\frac{1}{6}{\kappa }_3^2(x^2-1),\\ p_2(x)=-x(\frac{1}{24}{\kappa }_4(x^2-3)+\frac{1}{72}{\kappa }_3^2(x^4-10x^2+15))\end{array}$$

Φ and ϕ are the cumulative distribution function and density function respectively of the standard normal distribution, and, κ₃ and κ₄ are the skewness and kurtosis respectively of Z. Skewness is pronounced with non-symmetric score functions such as log(.), so skewness correction is appropriate in such instances.