Probabilities for separating sets of order statistics

Abstract

Consider a set of order statistics that arise from sorting samples from two different populations, each with their own, possibly different distribution functions. The probability that these order statistics fall in disjoint, ordered intervals and that of the smallest statistics, a certain number come from the first populations is given in terms of the two distribution functions. The result is applied to computing the joint probability of the number of rejections and the number of false rejections for the Benjamini–Hochberg false discovery rate procedure.

1. Introduction

Glueck et al. [1] gave explicit expressions for the probability that arbitrary subsets of order statistics fall in disjoint, ordered intervals on the set of real numbers. In this paper, we extend this work and consider two sets of real-valued, independent but not necessarily identically distributed random variables. We give expressions in terms of cumulative distribution functions for the probability that arbitrary subsets of order statistics fall in disjoint, ordered intervals and that of the smallest statistics, a certain number come from one set. We have been unable to find any previous papers on this topic. This problem is of interest in calculating probabilities for the Benjamini and Hochberg [2] multiple comparisons procedure.

2. A simple example

Consider the following simple example. Let X₁, X₂ ∈ [0, 1] be independent random variables. Denote by F_X1(x₁) and F_X2(x₂) the marginal cumulative distribution functions and by F_X1,X2(x₁, x₂) the joint cumulative distribution function of X₁ and X₂. Assume that the cumulative distribution functions are continuous. Let Y₁ = min{X₁, X₂} and let Y₂ = max{X₁, X₂} be the order statistics. For i = 1, 2, write the marginal cumulative distribution function of Y_i as F_Yi (y_i) and the joint cumulative distribution function as F_Y1,Y2(y₁, y₂), for y₁ ≤ y₂. This joint cumulative distribution function is also continuous [3, p. 10].

Choose numbers b₁ < b₂, b₁, b₂ ∈ (0, 1). We wish to find the probabilities

$$𝒜=\text{Pr}\{(y_1<b_1)\wedge (y_2>b_2)\},$$ (1)

$$\beta =\text{Pr}\{(y_1<b_1)\wedge (y_2>b_2)\wedge (x_1<b_1)\}$$ (2)

and

$$\gamma =\text{Pr}\{(y_1<b_1)\wedge (y_2>b_2)\wedge \neg (x_1<b_1)\}.$$ (3)

and express them in terms of the distribution functions F_X1 and F_X2. First, we will find the probabilities directly. So,

$$\begin{array}{cc}\beta \hfill & =\text{Pr}\{(x_1<b_1)\wedge (x_2>b_2)\}\hfill \\ \hfill & =F_{X_1}(b_1)[1-F_{X_2}(b_2)]\hfill \end{array}$$ (4)

and

$$\begin{array}{cc}\gamma \hfill & =\text{Pr}\{(x_1>b_2)\wedge (x_2<b_1)\}\hfill \\ \hfill & =[1-F_{X_1}(b_2)]F_{X_2}(b_1).\hfill \end{array}$$ (5)

Equations (4) and (5) follow directly from the independence of the random variables and the definition of the cumulative distribution functions. Since

$$\begin{array}{cc}\{(y_1<b_1)\wedge (y_2>b_2)\}\hfill & =\{(y_1<b_1)\wedge (y_2>b_2)\wedge (x_1<b_1)\}\hfill \\ \hfill & \cup \{(y_1<b_1)\wedge (y_2>b_2)\wedge \neg (x_1<b_1)\}\hfill \end{array}$$ (6)

and the union is disjoint, it follows that

$$𝒜=\beta +\gamma .$$ (7)

For a problem with more than two order statistics, the number of cases one needs to consider and the number of possible combinations of statistics, subsets, and bounds makes a direct approach impractical. An algorithmic approach to obtaining γ and β will allow the generalization to an arbitrary number of order statistics.

Using the assumption that the distribution functions are continuous, simple set operations, and the definition of distribution function, we obtain that the probability of the union (6) is

$$𝒜=\text{Pr}\{(y_1<b_1)\wedge \neg (y_2<b_2)\}$$ (8)

$$=\text{Pr}\{y_1<b_1\}-\text{Pr}\{(y_1<b_1)\wedge (y_2<b_2)\}$$ (9)

$$=F_{Y_1}(b_1)-F_{Y_1,Y_2}(b_1,b_2).$$ (10)

The cumulative distributions of the order statistics can be written [4],

$$F_{Y_1}(b_1)=F_{X_1}(b_1)F_{X_2}(b_1)+F_{X_1}(b_1)[1-F_{X_2}(b_1)]+[1-F_{X_1}(b_1)]F_{X_2}(b_1)$$ (11)

$$F_{Y_1,Y_2}(b_1,b_2)=F_{X_1}(b_1)[F_{X_2}(b_2)-F_{X_2}(b_1)]+[F_{X_1}(b_2)-F_{X_1}(b_1)]F_{X_2}(b_1)+F_{X_1}(b_1)F_{X_2}(b_1).$$ (12)

Then, substituting Equations (11) and (12) into Equation (10), we can write 𝒜 in terms of the distribution functions of X₁ and X₂,

$$\begin{array}{cc}𝒜=\hfill & F_{X_1}(b_1)[1-F_{X_2}(b_1)]+[1-F_{X_1}(b_1)]F_{X_2}(b_1)\hfill \\ \hfill & -F_{X_1}(b_1)[F_{X_2}(b_2)-F_{X_2}(b_1)]-[F_{X_1}(b_2)-F_{X_1}(b_1)]F_{X_2}(b_1)\hfill \end{array}$$ (13)

$$=F_{X_1}(b_1)[1-F_{X_2}(b_2)]+[1-F_{X_1}(b_2)]F_{X_2}(b_1).$$ (14)

We now interpret the terms in the sum in Equation (14). The term that includes F_X1(b₁) as a factor is the probability of an event in which x₁ < b₁ occurs, and the term that includes 1 − F_X1(b₂) as a factor is the probability of an event in which x₁ > b₂. Since b₁ < b₂, the two events are disjoint, and, consequently, Equation (7) follows again.

To summarize, we have expressed the probability in terms of the joint distribution of the order statistics, which was in turn written in terms of the distribution functions of the random variables. Finally, by recognizing terms that corresponded to a partition, we decomposed 𝒜 into a sum of β and γ, the two probabilities of interest.

3. General case

The logic used in this simple, two random variables example can be generalized to an arbitrary number of random variables. Consider a set of order statistics that arise from sorting samples from two different populations, each with their own, possibly different distribution function. We wish to find the probability that these order statistics fall in a given union of intervals and that of the smallest statistics, a certain number come from one population.

For this general case, we need to introduce some notation and definitions. Let X_i, i = 1,…m, be independent but not necessarily identically distributed real-valued random variables with values in the interval [0, 1] and continuous cumulative distribution functions F_Xi (x_i). Partition the set {X₁, X₂,…,X_m} into two subsets,

$$S_1=\{X_1,X_2,\dots ,X_n\},S_2=\{X_{n+1},X_{n+2},\dots ,X_m\}.$$ (15)

For example, one can consider measurements for males or females, or for two different populations of breast cancer, slow or fast growing. The order statistics Y₁, Y₂,…,Y_m are random variables defined by sorting the values of X_i. Thus Y₁ ≤ Y₂ ≤ … ≤ Y_m. Denote the realizations of the order statistics by y₁ ≤ y₂ ≤ … ≤ y_m.

The arguments of the joint cumulative distribution function of order statistics are customarily written omitting redundant arguments; thus for 1 ≤ e ≤ m let 1 ≤ n₁ < n₂ < ⋯ < n_e ≤ m, denote the indices of the order statistics of interest. The joint cumulative distribution function of the set {Y_n1, Y_n2,…,Y_ne}, which is a subset of the complete set of order statistics, is defined as

$$F_{Y_{n_1},\dots ,Y_{n_e}}(y_1,\dots ,y_e)=\text{Pr}(\{Y_{n_1}\le y_1\}\cap \{Y_{n_2}\le y_2\}\cap \cdots \cap \{Y_{n_e}\le y_e\}).$$ (16)

Suppose we are given s ≤ m disjoint intervals

$$(c_q,d_q),0=c_1<d_1<c_2<\cdots <c_s<d_s=1,$$ (17)

and integers

$$k_q\ge 0,{\displaystyle \sum _{q=1}^s{k}_q=m},$$ (18)

where k₀ = 0 and k_q is the number of order statistics that fall in the qth interval. Define $w_{q,1}=1+{\displaystyle {\sum }_{i=1}^{q-1}{k}_i,\text{ and }}{w}_{q,k_q}={\displaystyle {\sum }_{i=1}^q{k}_i}$ to be the subscripts of the largest and smallest order statistics, respectively, that fall in the qth interval. In the case when k_q = 1, we have w_q,1 = w_q,kq. Using this notation, the event that exactly k_q of the order statistics fall in the qth interval is

$$\{c_1<Y_{w_{1,1}}<\cdots <Y_{w_{1,k_1}}<d_1\mathrm{\wedge }\cdots \mathrm{\wedge }{c}_s<Y_{w_{s,1}}<\cdots <Y_{w_{s,k_s}}<d_s\},$$ (19)

or in a more compact notation (21) given below. Now let B be another random event. The following theorem gives the probability of this event intersected with the event (19), in terms of the cumulative distribution functions of the order statistics relative to the event B. This distribution function is defined by

$$F_{Y_{n_1},\dots ,Y_{n_e};B}(y_1,\dots ,y_e)=\text{Pr}(\{Y_{n_1}\le y_1\}\cap \{Y_{n_2}\le y_2\}\cap \cdots \cap \{Y_{n_e}\le y_e\}\cap B).$$ (20)

Contrary to the usual convention, we do not require that the indices of the order statistics in the cumulative distribution function (20) are sorted, because that would result in a complication of the notation in the next theorem (additional renumbering of the arguments).

THEOREM 1 Denote the event

$$E={\displaystyle \underset{q=1}{\overset{s}{\cap }}(\{c_q<Y_{w_{q,1}}\}\cap \{Y_{w_{q,k_q}}<d_q\})}.$$ (21)

Then

$$\begin{array}{cc}\text{Pr}(E\cap B)=\hfill & F_{Y_{w_{1,k_1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,k_s}};B}(d_1,d_2,\dots ,d_q)\hfill \\ \hfill & -{\displaystyle \sum _{q=1}^s{F}_{Y_{w_{1,k_1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,k_s}},Y_{w_{q,k_q}};B}(d_1,d_2,\dots ,d_q,c_q)}\hfill \\ \hfill & +{\displaystyle \sum _{\underset{r<t}{r,t=1}}^s{F}_{Y_{w_{1,k_1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,k_s}},Y_{w_{r,1},}{Y}_{w_{t,1};}B}(d_1,d_2,\dots ,d_q,c_r,c_t)}\hfill \\ \hfill ⋮& \hfill \\ \hfill & +{(-1)}^s{F}_{Y_{w_{1,1}},Y_{w_{1,k_1}},Y_{w_{2,1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,1}},Y_{w_{s,k_s}};B}(c_1,d_{1,}{c}_2,d_2,\dots ,c_s,d_s).\hfill \end{array}$$ (22)

Proof By standard set operations,

$$E={\displaystyle \underset{q=1}{\overset{s}{\cap }}\{c_q<Y_{w_{q,1}}\}\cap }{\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,k_q}}<d_q\}}$$ (23)

and

$${\displaystyle \underset{q=1}{\overset{s}{\cap }}\{c_q<Y_{w_{q,1}}\}=}{{\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,1}}\le c_q\}}}^C={\left({\displaystyle \underset{q=1}{\overset{s}{\cup }}\{Y_{w_{q,1}}\le c_q\}}\right)}^C,$$ (24)

where ^C denotes the complement. Therefore,

$$E\cap B={\left({\displaystyle \underset{q=1}{\overset{s}{\cup }}\{Y_{w_{q,1}}\le c_q\}}\right)}^C\cap F,$$ (25)

where the event F is defined by

$$F={\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,k_q}}<d_q\}}\cap B.$$ (26)

By the additivity of probability, it follows Equation from (25) that

$$\text{Pr}(E\cap B)=\text{Pr}(F)-\text{Pr}\left({\displaystyle \underset{q=1}{\overset{s}{\cup }}\{Y_{w_{q,1}}\le c_q\}\cap F}\right)=\text{Pr}(F)-\text{Pr}\left({\displaystyle \underset{q=1}{\overset{s}{\cup }}{A}_q}\right),$$ (27)

where A_q = {Y_wq,1 ≤ c_q} ∩ F. Using the additivity of probability again, we have

$$\text{Pr}\left({\displaystyle \underset{q=1}{\overset{s}{\cup }}{A}_q}\right)={\displaystyle \sum _{q=1}^s\text{Pr}(A_q)-}{\displaystyle \sum _{\underset{r<t}{r,t=1}}^s\text{Pr}(A_r\cap A_t)+\dots }$$ (28)

$$+{(-1)}^s\text{Pr}\left({\displaystyle \underset{q=1}{\overset{s}{\cap }}{A}_q}\right)$$ (29)

Now putting Equations (26)–(29) together and using the continuity of the cumulative distribution functions, we obtain

$$\begin{array}{cc}\text{Pr}(E\cap B)=\hfill & \underset{F_{Y_{w_{1,k_1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,k_s}};B}(d_1,d_2,\dots ,d_q)}{\underbrace{\text{Pr}\left({\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,k_q}}\le d_q\}\cap B}\right)}}\hfill \\ \hfill & -{\displaystyle \sum _{r=1}^s}\underset{F_{Y_{w_{1,k_1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,k_s}},Y_{w_{r,1}};B}(d_1,d_2,\dots ,d_q,c_r)}{\underbrace{\text{Pr}\left(\{Y_{w_{r,1}}\le c_r\}\cap {\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,k_q}}\le d_q\}\cap B}\right)}}\hfill \\ \hfill & +{\displaystyle \sum _{\underset{r<t}{r,t=1}}^s}\underset{F_{Y_{w_{1,k_1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,k_s}},Y_{w_{r,1}},Y_{w_{t,1}};B}(c_1{d}_1,d_2,\dots ,d_q,c_r,c_t)}{\underbrace{\text{Pr}\left(\{Y_{w_{r,1}}\le c_t\}\cap \{Y_{w_{t,1}}\le c_t\}\cap {\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,k_q}}\le d_q\}}\cap B\right)}}\hfill \\ \hfill ⋮& \hfill \\ \hfill & +{(-1)}^s\underset{F_{Y_{w_{1,1}},Y_{w_{1,k_1}},Y_{w_{2,1}},Y_{w_{2,k_2}},\dots ,Y_{w_{s,1}},Y_{w_{s,k_s}};B}(c_1,d_1,c_2,d_2,\dots ,c_s,d_s)}{\underbrace{\text{Pr}\left({\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,1}}\le c_q\}\cap }{\displaystyle \underset{q=1}{\overset{s}{\cap }}\{Y_{w_{q,k_q}}\le d_q\}\cap B}\right)}},\hfill \end{array}$$

which concludes the proof.

From now on assume that B is the event that exactly j elements of S₁ fall in the interval (0, y₁), for a given j ≤ n. This event is shown in Table 1. Thus, to compute the probability of interest, it is enough evaluate the cumulative distribution functions relative to the event B of the order statistic, given by Equation (20). An efficient method for the computation of cumulative distribution functions of order statistics from two populations was proposed by Glueck et al. [5]. Here we need a slight generalization, involving the event B, which requires a different proof.

Table 1.

Numbers of order statistics from the sets S₁ and S₂ in the interval (0, d₁) and outside the interval (0, d₁), in the event B.

	< d1	≥ d1
S1	j	n − j	n
S2	k1 − j	(m − n) − (k1 − j)	(m − n)
Total	k1	m − k1	m

THEOREM 2 Denote the index vector i = (i₀, i₁,…,i_e+1) and the summation index set

$$ℐ=\left\{\mathbf{i}:\begin{array}{c}\hfill 0=i_0\le i_1\le \cdots \le i_e\le i_{e+1}=m,\hfill \\ \hfill \mathit{\text{and}}{i}_a\ge n_a\mathit{\text{ for all }}1\le a\le k\hfill \end{array}\right\}.$$ (30)

Suppose that F_Xi (x) = F (x), for all 1 ≤ i ≤ n, and F_Xi (x) = G (x), for all n + 1 ≤ i ≤ m. Then the cumulative distribution function relative to the event B (20) is given by

$$F_{Y_{n_1},\dots ,Y_{n_e};B}(y_1,\dots ,y_e)={\displaystyle \sum _{\mathbf{i}\in ℐ}{\displaystyle \sum _{\mathbf{\lambda }}{\displaystyle \prod _{a=1}^{e+1}\frac{n!(m-n)!}{{\lambda }_a!(i_a-i_{a-1}-{\lambda }_a)!}\times {[F(y_a)-F(y_{a-1})]}^{{\lambda }_a}{[G(y_a)-G(y_{a-1})]}^{i_a-i_{a-1}-{\lambda }_a},}}}$$ (31)

where y₀ = 0, y_e+1 = 1, and λ = (λ₁, λ₂,…,λ_e+1) ranges over all integer vectors such that λ₁ = j and

$${\lambda }_1+{\lambda }_2+\cdots +{\lambda }_{e+1}=n,0\le {\lambda }_a\le i_a-i_{a-1}.$$ (32)

Proof Denote by A_i,λ the event that exactly i_a − i_a−1 of the random variables X_i fall in the interval (y_a−1, y_a), and exactly λ_a of those are elements of S₁. When a = 1, (y_a−1, y_a) = (y₀, y₁) = (0, y₁). If B occurs, λ₁ = j. Then from the binomial theorem,

$$\text{Pr}(A_{\mathbf{i},\mathbf{\lambda }})={\displaystyle \prod _{a=1}^{e+1}\frac{n!(m-n)!}{{\lambda }_a!(i_a-i_{a-1}-{\lambda }_a)!}{[F(y_a)-F(y_{a-1})]}^{{\lambda }_a}{[G(y_a)-G(y_{a-1})]}^{i_a-i_{a-1}-{\lambda }_a}}.$$ (33)

Since the events A_i,λ for different (i, λ) are disjoint, the result follows.

The only difference between Theorem 2 and the result by Glueck et al. [1] is the added condition λ₁ = j.

In the case of two random variables, we recover the same results as the direct method in Section 2. With m = 2, n = 1, s = 2, c₁ = 0, d₁ = b₁, c₂ = b₂, d₂ = 1, S₁ = {X₁}, S₂ = {X₁}, k₁ = 1, k₂ = 1, Y_w1,1 = Y_w1,k1 = Y₁, Y_w2,1 = Y_w2,k1 = Y₂, using Theorems 1 and 2 yields

$$(E\cap B)=\gamma ,$$ (34)

when j = 0, and

$$(E\cap B)=\beta ,$$ (35)

when j = 1.

In conclusion, for two sets of real-valued, independent but not necessarily identically distributed random variables, we have now given an expression for the probability that arbitrary subsets of order statistics fall in disjoint, ordered intervals and that of the smallest statistics, a certain number come from one set.

4. Concluding example

The methods of this paper can be used to calculating the joint probability of the number of rejections and the number of false rejection for the Benjamini–Hochberg [2] procedure. A rejection of a hypothesis for which the null holds is a false rejection. Given an false discovery rate α ∈ (0, 1), hypotheses H_i, i = 1,…,m, p-values X_i, and the corresponding order statistics for the p-values Y_i = X_(e) (the random variables X_i sorted in nondecreasing order X₍₁₎ ≤ X₍₂₎ ≤ ⋯ ≤ X_(m)), the procedure produces a nondecreasing sequence of numbers b_i = iα/m ∈ (0, 1), rejects the hypotheses H_(e), e = 1,…, k₁, such that k₁ is the largest number for which y_k1 ≤ b_k1, and accepts all others. For n ∈ {0, 1,…,m}, assume that the null holds for H₁, H₂,…, H_n and that the alternative holds for H_n+1, H_n+2,…,H_m. Let S₁ = {X₁, X₂,…,X_n} be the set of p-values that correspond to the null hypotheses, and S₂ = {X_n+1, X_n+2,…,X_m} be the set of p-values for which the alternative holds. Then j is the number of null hypotheses that are rejected, which is equal to the number of p-values corresponding to null hypotheses that fall in the interval [0, b_k1].

Under the assumption that the p-values for which the alternative holds have the same distribution, one can use the methods of this paper to find the joint distribution of j and k₁. For each value of k₁ and m, Glueck et al. [6] pointed out that the rejection regions for the Benjamini and Hochberg (1995) procedure can be decomposed into disjoint sets of events. These events correspond to certain numbers of order statistics falling into sets of intervals, defined by the numbers b_i. Details about the decomposition of the rejection regions into these events are given in Glueck et al. [6]. The general case is too complicated to detail here. However, as an example, we calculate the probabilities that with m = 2 hypotheses, and n = 1 null hypotheses, the Benjamini and Hochberg [2] procedure rejects k₁ = 1 hypotheses and that j, the number of false rejections, is either 0 or 1.

Suppose we wish to test m = 2 hypotheses. Specifically, we wish to test hypotheses about the location of the sample mean. We plan to conduct a two-sided test. We assume that we have two large populations, with known variances (both σ²), and that the variables of interest, say ε₁ and ε₂, are normally distributed, so that ε₁ ~ N(μ₁, σ²) and ε₂ ~ N(μ₂, σ²). We wish to test two hypotheses H₁: μ₁ = μ₀, and H₂: μ₂ = μ₀, with the alternative hypothesis for both populations the same, so H_A: μ = μ_A. We sample N_i random variables from each population, say ε_i1, ε_i2,…,ε_iNi. For convenience, we will assume that the random sample is of the same size for each hypothesis test, so N₁ = N₂ = N.

With

$${\overline{\epsilon }}_i=N^{-1}{\displaystyle \sum _{\delta =1}^N{\epsilon }_{i\delta }},$$ (36)

the test statistics are given by

$$Z_i={\left(\frac{\sigma }{\sqrt{N}}\right)}^{-1}({\overline{\epsilon }}_i-{\mu }_0),$$ (37)

and the two sided p-values are [7, p. 244]

$$X_i=\{\begin{array}{cc}2\mathrm{\Phi }(Z_i)\hfill & Z_i\le 0\hfill \\ 2[1-\mathrm{\Phi }(Z_i)]\hfill & Z_i>0\hfill \end{array},$$ (38)

where Φ is the cumulative distribution function of the standard normal (mean = 0 and variance = 1). Let ϕ be the probability density function of the standard normal.

Suppose that in truth, we have ε₁ ~ N (μ₀, σ²), so that the null holds for H₁, and ε₂ ~ N(μ_A, σ²), so that alternative holds for H₂. Define S₁ = {X₁}, and S₂ = {X₂}. Then the number of p-value for which the null holds, n = 1. For H₁, the hypotheses for which the null holds, the p-value has a uniform distribution on the interval [0, 1], so for x₁ ∈ [0, 1],

$$F_{X_1}(x_1)=x_1.$$ (39)

For H₂, the alternative holds. When we conduct the hypothesis test, we are unaware of the truth. We always calculate the p-value under the null. However, since the alternative actually holds,

$$\begin{array}{cc}\text{Pr}[Z_2\le z_2]\hfill & =\text{Pr}[\frac{{\overline{\epsilon }}_i-{\mu }_0}{\sigma /\sqrt{N}}\le z_2]\hfill \\ \hfill & =\text{Pr}[\frac{{\epsilon }_i-{\mu }_A}{\sigma /\sqrt{N}}\le z_2+\frac{{\mu }_0-{\mu }_A}{\sigma /\sqrt{N}}]\hfill \\ \hfill & =\mathrm{\Phi }[z_2+\frac{{\mu }_0-{\mu }_A}{\sigma /\sqrt{N}}].\hfill \end{array}$$ (40)

Finally,

$$\begin{array}{cc}{F}_{X_2}(x_2)\hfill & =\text{Pr}(X_2<x_2)\hfill \\ \hfill & =\text{Pr}(\{X_2<x_2\}\cap \{Z_2\le 0\})+\text{Pr}(\{X_2<x_2\}\cap \{Z_2>0\})\hfill \\ \hfill & =\text{Pr}(\{2\mathrm{\Phi }(Z_2)<x_2\})+\text{Pr}(\{2[1-\mathrm{\Phi }(Z_2)]<x_2\})\hfill \\ \hfill & =\text{Pr}\left(\right\{Z_2\le {\mathrm{\Phi }}^{-1}\left(\frac{x_2}{2}\right)\left\}\right)+1-\text{Pr}\left(\right\{Z_2\le {\mathrm{\Phi }}^{-1}\left(\frac{1-x_2}{2}\right)\left\}\right)\hfill \\ \hfill & =\mathrm{\Phi }[{\mathrm{\Phi }}^{-1}(x_2/2)+\frac{{\mu }_0-{\mu }_A}{\sigma /\sqrt{N}}]+1-\mathrm{\Phi }[{\mathrm{\Phi }}^{-1}(1-x_2/2)+\frac{{\mu }_0-{\mu }_A}{\sigma /\sqrt{N}}],\hfill \end{array}$$ (41)

where the last step follows by substitution from Equation (40).

Now, as a specific example, we fix μ₀ = 0, μ_A = 1, σ² = 1 α = 0.05. We wish to calculate the probability that k₁ = 1 and that j = 0 or j = 1. With c₁ = 0, d₁ = α/2, c₂ = α, d₂ = 1. This is the probability that of the two hypotheses, we reject exactly one, and it is H₁, the hypothesis for which the null holds. When j = 0, the rejection we make is of the hypothesis for which the alternative holds, and when j = 1, the rejection we make is of the null hypothesis, a false rejection.

We calculated the probability using our methodology, and by a simulation using a sample of 100,000 variables. Recall that k₁ is the number of order statistics that is less than b₁, and j is the number in Set 1, and less than b₁. The results are shown in Table 2.

Table 2.

Comparison of simulation and theory.

k1	j	Theory	Simulation	Difference
1	0	0.472982	0.47388	0.000898
1	1	0.00978051	0.0095	0.00028051

Note: Recall that k₁ is the number of hypotheses that were rejected, and j is the number of null hypotheses that were rejected. We had two hypotheses and one null hypothesis.

Notice that the simulation differs from the theory only in the fourth decimal place. The theory is exact. Software that implements this method in Mathematica is available from the authors upon request.