A Note on Monotonicity Assumptions for Exact Unconditional Tests in Binary Matched-pairs Designs

Summary

Exact unconditional tests have been widely applied to test the difference between two probabilities for 2×2 matched-pairs binary data with small sample size. In this context, Lloyd (2008, Biometrics 64, 716–723) proposed an E + M p-value, that showed better performance than the existing M p-value and C p-value. However, the analytical calculation of the E + M p-value requires that the Barnard convexity condition be satisfied; this can be challenging to prove theoretically. In this paper, by a simple reformulation, we show that a weaker condition, conditional monotonicity, is sufficient to calculate all three p-values (M, C and E + M) and their corresponding exact sizes. Moreover, this conditional monotonicity condition is applicable to non-inferiority tests.

1. Introduction

Lloyd (2008) considered exact unconditional tests for binary matched-pairs data when the sample size is small. A 2 × 2 table (as Table 1 in Lloyd (2008)) can summarize this type of data in four random cell counts X = (X₀₀, X₀₁, X₁₀, X₁₁) where X_ij, i, j = 0, 1, is the number of matched-pairs with responses i for Treatment 1 and j for Treatment 2. Vector X is assumed to have a multinomial distribution with probabilities π = (π₀₀, π₀₁, π₁₀, π₁₁). The problem of interest is to test the difference between success probabilities in the two groups, i.e., H₀: π₁₊ ≤ π₊₁ versus H₁: π₁₊ > π₊₁, where π₁₊ = π₁₁ + π₁₀ for Treatment 1 and π₊₁ = π₁₁ + π₀₁ for Treatment 2. This is equivalent to testing

$$H_0:{\pi }_{01}\ge {\pi }_{10}\text{versus}{H}_1:{\pi }_{01}<{\pi }_{10}.$$ (1)

To calculate p-values (M,C and E + M) and their exact sizes for testing (1), Lloyd (2008) used the Barnard Convexity Condition (BCC) to reduce the two-dimensional nuisance parameter space to one dimension. However, a rigorous proof that the BCC is satisfied is not always available. Therefore, numerical methods are often used to verify the BCC; this is the case for the E p-value when calculating the E + M p-value (Lloyd, 2008) and for the C p-value when calculating its exact size (Berger and Sidik, 2003).

In this paper, we show that the conditional monotonicity condition (CMC), a weaker condition than BCC, is sufficient for the dimension reduction in the calculation of p-values (M,C and E + M) and their exact sizes for testing (1) and that this dimension reduction technique can be extended to the non-inferiority case.

2. The BCC and the CMC

Denote observed values of X by (x₀₀, x₀₁, x₁₀, x₁₁) and let k = x₁₀ + x₀₁ be the number of discordant pairs and x = x₀₁. Following Lloyd (2008), we consider three statistics: McNemar’s test statistic, the likelihood ratio test statistic and the conditional sign test statistic. For any of the three test statistics above, denoted by T(x, k), the BCC (Lloyd, 2008) is satisfied if

$$T(x,k)\le T(x+1,k)\text{and}T(x,k)\ge T(x,k+1)$$ (2)

for all (x, k). That is, T(x, k) is nondecreasing in x for any fixed k, and is nonincreasing in k for any fixed x. Note that Berger and Sidik (2003) used a slightly different definition of the BCC: T̃(x₀₁, x₁₀) ≙ T(x₀₁, x₀₁ + x₁₀) is nondecreasing in x₀₁ and nonincreasing in x₁₀. It is easy to prove that this definition is stronger than Lloyd’s definition in (2).

The conditional monotonicity condition (CMC) is defined as

$$T(x,k)\le T(x+1,k),$$ (3)

which is only one of the constraints in (2), and thus, is a weaker condition than either Lloyd’s BCC or Berger and Sidik’s BCC. For example, while the test statistic $T(x,k)=(2x-k)/(k-\sqrt{k})$ satisfies the CMC, T(x, k) does not satisfy the BCC by either definition. Note that McNemar’s, the likelihood ratio and the conditional sign test statistics all satisfy the CMC.

3. The p-value calculation

Following Lloyd (2008), we reparameterize π₀₁ and π₁₀ by φ and η as φ:= π₁₀ + π₀₁ and η:= π₀₁/φ. Then the null space H₀ = {(π₀₁, π₁₀): π₀₁ ≥ π₁₀} = {(φ, η): 0 ≤ φ ≤ 1, η ≥ 1/2}. When the test statistic T(x, k) satisfies the CMC, let h(t, k) = sup{x: T(x, k) ≤ t}, which is also nondecreasing in t. We define h(t, k) to be -1 if the supremum does not exist. Using the definition of h(t, k), for a given observed statistic t_obs, the rejection set can be rewritten as {(x′, k′): T(x′, k′) ≤ t_obs} = {(x′, k′): 0 ≤ k′ ≤ n, 0 ≤ x′ ≤ h(t_obs, k′)}.

3.1 M p-value and C p-value

We adopt the notion that smaller values of test statistics favor H₁ and note that the probability mass function of (X, K) can be decomposed into P_{(φ, η)}(X = x, K = k) = b(k; n, φ)b(x; k, η) where b(x; n) is the binomial probability mass function (Lloyd, 2008). We also note that the M p-value is a special case of the C p-value; therefore, we focus our discussion on the C p-value.

Berger and Boos (1994) proposed a C p-value that confines the maximization within a well-behaved confidence interval in a general setting. Berger and Sidik (2003) adopted this C p-value definition for 2×2 matched-pairs data. Let I_γ be a 100(1 − γ)% confidence interval for φ (Berger and Sidik, 2003). Now we can reformulate the C p-value as

$$\begin{array}{l}C(x,k)=\underset{\phi \in I_{\gamma },\eta \ge 1/2}{sup}{P}_{(\phi ,\eta )}\{T(X,K)\le t_{\mathit{obs}}\}+\gamma \\ =\underset{\phi \in I_{\gamma },\eta \ge 1/2}{sup}\sum _{(x^{\prime },k^{\prime }):T(x^{\prime },k^{\prime })\le t_{\mathit{obs}}}{P}_{(\phi ,\eta )}(X=x^{\prime },K=k^{\prime })+\gamma \\ =\underset{\phi \in I_{\gamma }}{sup}\sum _{k^{\prime }=0}^n\left\{b(k^{\prime };n,\phi )\underset{n\ge 1/2}{sup}\sum _{x^{\prime }\le h(t_{\mathit{obs}},k^{\prime })}b(x^{\prime };k^{\prime },\eta )\right\}+\gamma \\ =\underset{\phi \in I_{\gamma }}{sup}\sum _{k^{\prime }=0}^{n}b(k^{\prime };n,\phi )\underset{n\ge 1/2}{sup}B\{h(t_{\mathit{obs}},k^{\prime });k^{\prime },\eta \}+\gamma \\ =\underset{\phi \in I_{\gamma }}{sup}\sum _{k^{\prime }=0}^{n}b(k^{\prime };n,\phi )B\{h(t_{\mathit{obs}},k^{\prime });k^{\prime },1/2\}+\gamma \end{array}$$ (4)

The last equality in (4) results from the fact that B(x; n, p) is nonincreasing in p for fixed x and n. Note that when γ = 0, the confidence-interval region is the null space H₀ and thus the C p-value becomes the M p-value which is defined as sup_{(φ, η)∈H0} P_(φ, _η){T (X, K) ≤ t_obs} (Suissa and Shuster, 1991) and equals ${sup}_{0\le \phi \le 1}{\sum }_{k^{\prime }=0}^{n}b(k\prime ;n,\phi )B\{h(t_{\mathit{obs}},k^{\prime });k^{\prime },1/2\}$. Thus, the maximization over two nuisance parameters (φ, η) in the M p-value and the C p-value can be reformulated into the maximization over the single parameter φ by using only the CMC property of the test statistic.

3.2 E + M p-value

The E + M p-value proposed by Lloyd (2008) consists of two steps. First the E p-value is defined as

$$E(x,k)=\underset{\eta \ge 1/2}{sup}{P}_{(\widehat{\phi },\eta )}\{T(X,K)\le t_{\mathit{obs}}\}=\sum _{k^{\prime }=0}^{n}b(k^{\prime };n,\widehat{\phi })B\{h(t_{\mathit{obs}},k^{\prime });k^{\prime },1/2\},$$ (5)

where φ̂ = k/n is both the restricted and unrestricted MLE of φ under H₀. Since the E p-value is not necessarily valid (for the definition of a valid p-value, see Berger and Boos, 1994), Lloyd (2008) further proposed a maximization step on the E p-value to obtain a valid p-value

$$(E+M)(x,k)-\underset{(\phi ,\eta )\in H_0}{sup}{P}_{(\phi ,\eta )}\{E(X,K)\le e_{\mathit{obs}}\},$$ (6)

where e_obs = E(x, k).

It is difficult to prove that the E p-value is non-increasing for k when x is fixed and thus the E p-value may not satisfy the BCC. Lloyd (2008) numerically verified the BCC for the E p-value. Our Theorem 1 guarantees that the E p-value satisfies the CMC, so the dimension reduction technique in (4) can be applied on the E p-value to obtain the E + M p-value.

3.3 Exact size of the p-values

Let P̃ (x, k) denote any of the M, C and E + M p-values. The exact size of P̃ is defined as

$$\text{size}(\alpha )=\underset{(\phi ,\eta )\in H_0}{sup}P\{\stackrel{\sim }{P}(X,K)\le \alpha \}+\underset{0\le \phi \le 1}{sup}\sum _{k^{\prime }=0}^{n}b(k^{\prime };n,\phi )B\{\stackrel{\sim }{h}(\alpha ,k^{\prime });k^{\prime },1/2\},$$

where h̃ (α; k) = sup{x′: P̃ (x′, k) ≤ α}. Berger and Sidik (2003) numerically checked the BCC for the C p-value when using the dimension reduction to calculate the exact size for the C p-value. By the following Theorem 1, we can show that all of the p-values (M, C and E + M) satisfy the CMC and thus the same dimension reduction technique in (4) can be applied to calculate their exact sizes.

Theorem 1

If a test statistic T(x, k) satisfies the conditional monotonicity condition, i.e., it is nondecreasing in x for any fixed k, then the associated p-values, M(x, k), C(x, k), E(x, k) and (E + M)(x, k), also satisfy the conditional monotonicity condition.

Proof

For a fixed k and x₁ ≤ x₂, let t_obs,1 = T (x₁, k), t_obs,2 = T (x₂, k), and then t_obs,1 ≤ t_obs,2 as T (x, k) is nondecreasing in x. Note that h(t, k) is also nondecreasing in t, then h(t_obs,1, k′) ≤ h(t_obs,2, k′) for any k′∈ {0, …, n}. Thus we have B{h(t_obs,1, k′); k′, 1/2} ≤ B{ h(t_obs,2, k′); k′, 1/2}. Therefore, M(x, k) and C(x, k) are nondecreasing in x for fixed k. Note that when k is fixed, φ̂ is also fixed; therefore, E(x, k) in (5) and (E + M)(x, k) in (6) are nondecreasing in x.

4. Extension to Non-inferiority Trials

In practice, the non-inferiority test of $H_0^{\ast }:{\pi }_{01}-{\pi }_{10}\ge \delta$ versus $H_1^{\ast }:{\pi }_{01}-{\pi }_{10}<\delta$ is of interest. Several test statistics have been proposed to test $H_0^{\ast }$; these include the score statistic (Nam, 1997; Tango, 1998) and the likelihood ratio statistic (Lloyd and Moldovan, 2008). To the best of our knowledge, rigorous proofs for the BCC of the score and the likelihood ratio statistics are not available in the literature, neither are the proofs for the BCC of the E p-value and the C p-value. Here, we can prove that both of these test statistics satisfy the weaker condition CMC (the proof for the likelihood ratio test statistic is given in the Appendix), and so do the E p-value and the C p-value. We applied the same transformations φ = π₀₁ + π₁₀ and η = π₀₁/φ. Note that φ has a lower bound δ under $H_0^{\ast }$. Following the derivation in Section 3, the M p-value for the non-inferiority test is M(x, k) = sup_φ∈(δ,1) Φ(x, k, φ), where

$$\mathrm{\Phi }(x,k,\phi )=\sum _{k^{\prime }=0}^{n}b(k^{\prime };n,\phi )B\{h(t_{\mathit{obs}};k^{\prime });k^{\prime },(\delta +\phi )/(2\phi )\}.$$

The C p-value is C(x, k) = sup_φ∈IγΦ(x, k, φ) + γ, where I_γ is the 100(1−γ)% confidence interval for φ in Sidik (2003). In Lloyd and Moldovan (2008), the E p-value was based on the restricted MLE φ̃ of φ. We define the E p-value to be E(x, k) = F(x, k, φ̂), where φ̂ = (1 − δ)k/n+δ; φ̂ is always within [δ, 1] and is a function of k. Thus, we obtain the CMC of the E p-value that ensures the M step required for the E + M p-value. The E + M p-values obtained from these two different E p-values are practically identical from our numerical evaluations. Similar to the superiority case, the CMC is sufficient to calculate p-values and their exact sizes for non-inferiority tests with one dimensional maximization.

Appendix

The CMC of likelihood ratio test statistic in non-inferiority trials

By using transformations φ = π₀₁+ π₁₀ and η = π₀₁/φ, the null space $H_0^{\ast }=\{({\pi }_{01},{\pi }_{10}):{\pi }_{01}-{\pi }_{10}\ge \delta \}$ becomes {(φ, η): δ ≤ φ ≤ 1, η ≥ (δ + φ)/2φ }, where 0 ≤ δ ≤ 1. The log likelihood function can be written as l = l(φ, η) = k log φ + (n − k) log(1 − φ) + x log η + (k − x) log(1 − η) + C, where C is independent of the parameters (φ, η). The restricted MLE φ̃ of φ satisfies ∂l/∂φ = 0 under π₀₁ − π₁₀ = δ, i.e.,

$$x/(\stackrel{\sim }{\phi }+\delta )+(k-x)/(\stackrel{\sim }{\phi }-\delta )-(n-k)/(1-\stackrel{\sim }{\phi })=0.$$ (A.1)

A simple algebra manipulation verifies that φ̃ is the greater of the two solutions for (A.1) and is no less than δ. Furthermore, by taking derivative on x for (A.1), we obtain ∂φ̃/∂x · (−U) = 1/(φ̃ − δ) − 1/(φ̃+ δ), where U = x/(φ̃+ δ)² +(k−x)/(φ̃ − δ)² +(n−k)/(1− φ̃)² is nonnegaive. Therefore, ∂φ̃/∂x ≤ 0.

Let (φ̂, η̂)={k/n, x/k} be the unrestricted MLE of (φ, η). Define D = l(φ̂, η̂) − l(φ̃, (δ + φ̃)/2φ̃), then the likelihood ratio statistic $Z_{LR}=\mathit{sign}(2x-k-n\delta )\sqrt{2D}$. With (A.1), we obtain d=∂D/∂x=log x(φ̃ − δ)/(k−x)(φ̃+ δ). And ∂δ/∂x = {1/(φ̃ − δ) − 1/(φ̃+ δ)}· ∂φ̃/∂x + 1/x + 1/(k − x). Here, ∂d/∂x ≥ 0 can be established if U ≥ x(k − x)/k{1/(φ̃ + δ)² + 1/(φ̃ − δ)² − 2/(φ̃ + δ)(φ̃ − δ)}, which is true by observing that x/(φ̃ + δ)² ≥ x(k − x)/k(φ̃ + δ)², and (k − x)/(φ̃ − δ)² ≥ x(k − x)/k(φ̃ − δ)².

Denote x₀ = (k + nδ)/2. When x = x₀, we have φ̂ = φ̃, so d(x₀) is zero. Note that d is nondecreasing in x. If x ≥ x₀, then d(x) ≥ 0 and thus D is nondecreasing, and so is Z_LR. If x < x₀, then d(x) ≤ 0 and thus D is nonincreasing, and then Z_LR is nondecreasing due to the negative sign of 2x − k − n. Therefore, Z_LR is nondecreasing in x for any fixed k.