Non-inferiority tests for clustered matched pair data

SUMMARY

Non-inferiority tests for matched-pair data where pairs are mutually independent may not be appropriate when pairs are clustered. The tests may require an adjustment to account for the correlation within a cluster. We consider the adjusted score and Wald-type tests, and a modification of Obuchowski’s method for non-inferiority and compare them with the non-inferiority test based on a method of moments estimate in terms of Type 1 error rate and power by simulations for a small cluster size under various correlation structures. In general, the score test adjusted by an inflation factor and the modified Obuchowski’s method perform as good as the test based on moments estimate in the accuracy of Type 1 error rates. The latter does not provide reasonably close Type 1 error rates to the nominal level when a number of clusters is 25 or smaller and a positive response rate for the standard procedure is 20% or lower. The adjusted score test, the method based on moments estimate and the modified test are comparable in power. The adjusted Wald-type test is too anti-conservative and we should caution use of the test. Since number of clusters is strongly related to the accuracy of empirical Type 1 error rate and power, it is very important to have a sufficiently large number of clusters in designing a clustered matched-pair study for non-inferiority.

1. INTRODUCTION

When the standard procedure is highly effective or accurate but it is expensive, toxic or invasive, researchers are interested in finding an inexpensive, less toxic or non-invasive new procedure which is no worse than a pre-specified amount in effectiveness or accuracy. A statistical test of significance for this research goal is called a non-inferiority test. McNemar’s statistic [1] for testing a difference between two treatments is not applicable for this situation. Lu and Bean [2] and Nam [3] have provided statistical methods to establish non-inferiority for matched-pair data when pairs are independent. However, these tests may not be appropriate for clustered matched pair data where pairs in a cluster are correlated. Durkalski et al [4] proposed a statistical method testing non-inferiority for clustered data using the method of moments [5] and examined the performance of the method for sparse data by simulations. The standard error calculated under the assumption of independent pairs is smaller than that under non-independent pairs when pairs in a cluster are positively correlated [6–11]. Eliasziw and Donner [12] adjusted McNemar’s test by utilizing an inflation factor to account for the underestimated variance resulting from clustering. We investigate the non-inferiority test statistics for clustered matched pair samples using a similar approach.

Section 2 provides notations and a review of non-inferiority tests, and presents an application of an inflation factor to adjust non-inferiority tests [2, 3] for clustered data. It also includes a modification of Obuchowski’s method [13] for non-inferiority. Section 3 gives evaluation of the performance of each of the non-inferiority tests in terms of the accuracy of empirical Type 1 error rates and power by simulations under the general correlation structure. Sections 4 and 5 contain a numerical example for an illustration and discussion.

2. NON-INFERIORITY TESTS

We are interested in establishing non-inferiority of a new procedure comparing with the standard procedure in clustered matched-pair studies. Consider a random sampling of K clusters from a population, where there are n_k units in the k^th cluster for k=1, 2,…, K. The new and standard procedures are administered to each unit. We have n_k pairs of matched samples (Y_1ik, Y_0ik) for i=1, 2, …, n_k in the k^th cluster. Y_1ik and Y_0ik are binary responses (i.e. 1 or 0 as respond or not-respond) of the new and standard procedures in the i^th pair and the k^th cluster. There are four possible pair types, i.e., (1, 1), (1, 0), (0, 1) and (0, 0). The matched observations and corresponding probabilities in the k_th cluster are shown in Table 1. We consider a case where the number of clusters is large and a number of units in each cluster is small. Assume a common difference between two procedures in proportions for all clusters, i.e., p_1•k − p_•1k = δ( or p_10k − p_01k = δ) for k=1, 2, …, K. We want to test the null hypothesis, H₀:δ = δ₀ (< 0) against an alternative, H₁ : δ > δ;₀. Letting t̂_k = p̂_10k − p̂_01k − δ₀ where p̂_10k = x_10k / n_k and p̂_01k = x_01k / n_k for every k, we have E(t̂_k) = 0 under H₀. Applying a method of moments estimate (Royal [5]), Durkalski et al [4] proposed a test statistic for non-inferiority of clustered matched pair data,

$$Z_{\mathit{\text{EV}}}={\displaystyle \sum _{k=1}^K\left(\frac{x_{10k}-x_{01k}}{n_k}-{\mathrm{\delta }}_0\right)/{\left\{{\displaystyle \sum _{k=1}^K{\left(\frac{x_{10k}-x_{01k}}{n_k}-{\mathrm{\delta }}_0\right)}^2}\right\}}^{1/2}.}$$ (1)

Table I.

Observations and probabilities for matched pairs in the k^th cluster

	Standard procedure
New procedure	1	0	sum
1	x11k(p11k)	x10k(p10k)	x1•k(p1•k)
0	x01k(p01k)	x00k(p00k)	x0•k(p0•k)
sum	x•1k(p•1k)	x•0k(p•0k)	nk

For a large number of clusters, the statistic (1) is approximately normal with mean zero and variance one under H₀. We reject H₀ against H₁ at α when Z_EV > z_(1-α) where z_(1-α) is the 100 × (1-α) percentile point of the standard normal distribution. Let the dot denote a sum over k, e.g., $n_{•}={\displaystyle \sum _{k=1}^K{n}_k}$ which is a total number of units across clusters. If all matched pairs are independent, we may pool K × 2 × 2 tables (Table 1) into a large single 2 × 2 table and apply the test statistics Z_LB (Lu and Bean [2]) or Z_N (Nam [3]) for non-inferiority:

$$Z_{\mathit{\text{LB}}}=(x_{10•}-x_{01•}-n_{•}{\mathrm{\delta }}_0)/{(x_{10•}+x_{01•}-n_{•}{\mathrm{\delta }}_0)}^{1/2}$$ (2)

or

$$Z_N=(x_{10•}-x_{01•}-n_{•}{\mathrm{\delta }}_0)/{\{n_{•}({\tilde{p}}_{10}+{\tilde{p}}_{01}-{\mathrm{\delta }}_0^2)\}}^{1/2}$$ (3)

where p̃₁₀ = p̃₀₁ = + δ and p̃₀₁ is the restricted maximum likelihood estimator of p₀₁ for given δ = δ₀ , i.e., p̃₀₁ = {−b+ (b² - 4ac)^1/2} / (2a) where a =2n_•, b = (2n_• + x_01• − x_10•)δ₀ − (x_10• + x_01•) and c = −δ₀(1 − δ₀)x_01•. The statistics (2) and (3) for testing non-inferiority have been used for data where all units are randomly sampled from the same population. If matched pairs are positively correlated in a cluster, the test statistics based on pooled data ignoring the positive correlation are inflated and p-values are distorted downward. It may falsely reject non-inferiority. A correlation coefficient between units in a cluster has been called the intraclass correlation coefficient. A simulation study by Durkalski et al [4] showed an actual Type 1 error rate of an unadjusted test statistic is larger than a nominal level and it is greater as an intraclass correlation coefficient increases. Our simulation indicates the same observation. The intraclass correlation coefficient may be estimated using the mean square error of analysis of variance for mixed model (Snedecor and Cochran [14] and Donner [6]). A variance of sample mean under the assumption of independent pairs can be adjusted by multiplying the variance inflation factor (or design effect) for clustered data, e.g., Kish [15], Cochran [16] and Cornfield [17]. This type of adjustment has been applied in analysis of data from random group sampling, community intervention trials, longitudinal studies and sample survey. Applying a similar procedure to adjust McNemar test statistic for non-independent matched pair data (Eliasziw and Donner [12] and Durkalski et al [18]), the test statistic for non-inferiority based on pooled data may be adjusted using an inflation factor, c̅ = 1+ (n_c − 1) ρ̂ where n_c is the adjusted mean number of discordant pairs and ρ̂ is a consistent estimator of the intraclass correlation coefficient ρ (Donner [6]), (See APPENDIX for n_c and ρ̂). The adjusted Z_LB and Z_N statistics testing H₀:p₁₀ − p₀₁ = δ₀ (< 0) against H₁:p₁₀ − p₀₁ > δ₀ for clustered matched pair data are

$$Z_{\mathit{\text{ALB}}}=Z_{\mathit{\text{LB}}}/{\{1+(n_c-1)\widehat{\mathrm{\rho }}\}}^{1/2}$$ (4)

and

$$Z_{\mathit{\text{AN}}}=Z_N/{\{1+(n_c-1)\widehat{\rho }\}}^{1/2}.$$ (5)

When n_c = 1 or ρ̂ = 0, an inflation factor is unity and no adjustment is needed. For a large number of clusters, the adjusted statistics are distributed asymptotically normal mean zero and variance one. As an alternative method to an adjusted McNemar’s statistic for testing the conventional null hypothesis (Eliasziw and Donner [12]), Obuchowski [13] presented a statistic using Rao and Scott’s formulation [19]. Denote that p̂_1•• = x_1••/n_•, p̂_•1• = x_•1•/n_• and p̅_1•• = {(x_1•• + x_•1•) / n_• + δ₀} /2, p̅_•1• = {(x_1•• + x_•1•) / n_• − δ₀}/2, p′ = (p_1••,p_•1•) and p̅′ = (p̅_1••, p̅_•1•). When a similar approach applied to a non-inferiority setting, we have

$$Z_o=({\widehat{p}}_{1••}-{\widehat{p}}_{•1•}-{\mathrm{\delta }}_0)/{\{\mathrm{v}\widehat{\mathrm{a}}\mathrm{r}{({\widehat{p}}_{1••}-{\widehat{p}}_{•1•}-{\mathrm{\delta }}_0)}_{p=\overline{p}}\}}^{1/2}$$ (6)

where vâr(p̂_1•• − p̂_•1• − δ₀)_p=p̅ {vâr(p̂_1••) + vâr(p̂_•1•) − 2 côv(p̂_1••,p̂_•1•)} _p=p̅,

$$\begin{array}{c}\mathrm{v}\widehat{\mathrm{a}}\mathrm{r}{({\widehat{p}}_{1••})}_{p=\overline{p}}=K{(K-1)}^{-1}\left\{{\displaystyle \sum _{k=1}^K{(x_{1•k}-n_k{\overline{p}}_{1••})}^2/n_{•}^2}\right\},\hfill \\ \mathrm{v}\widehat{\mathrm{a}}\mathrm{r}{({\widehat{p}}_{•1•})}_{p=\overline{p}}=K{(K-1)}^{-1}\left\{{\displaystyle \sum _{k=1}^K{(x_{•1k}-n_k{\overline{p}}_{•1•})}^2/n_{•}^2}\right\}\hfill \end{array}$$

and $\mathrm{c}\widehat{\mathrm{o}}\mathrm{v}{({\widehat{p}}_{1••},{\widehat{p}}_{•1•})}_{p=\overline{p}}=K{(K-1)}^{-1}\left\{{\displaystyle \sum _{k=1}^K(x_{1•k}-n_k{\overline{p}}_{1••})(x_{•1k}-n_k{\overline{p}}_{•1•})/n_{•}^2}\right\}$ (See Table 1 for notations ). For a large number of clusters, the statistic (6) is approximately a standard normal variate. We reject the null hypothesis at α when Z_O ≥ Z(1-α).

Denote v = (p₁₀ + p₀₁ − δ²), t̂ = (p̂_10• + p̂_01• − δ), and n_• = Kn̅ where $\overline{n}={\displaystyle \sum _{k=1}^K{n}_k/K}$. As n̅ → ∞, n_c / n̅ → f = p₁₀ + p₀₁ (APPENDIX). The variance of t̂ adjusted with an inflation factor is expressed as var(t̂) = v{1+ (n_c − 1)ρ} / (Kn̅) = (v / K){(1 − ρ) / n̅ + (n_c / n̅) ρ}. The above variance approaches to zero as K → ∞ while it converges to vfρ / K as n̅ → ∞ It suggests that a large number of clusters provides more strong impact than a larger cluster size does in precision of estimation of t and power of a test based on t̂. Note that we consider non-inferiority tests on clustered matched pair data with a large value of K and a small cluster size in this paper.

3. SIMULATIONS

We conducted a simulation study to assess the performance of statistics, Z_EV, Z_ALB, Z_AN and Z_O for testing non-inferiority of a new procedure compared with the standard one in empirical Type 1 error rate and power for clustered matched pair data. Pre-specified parameters are the number of cluster (K), the number of units in the k^th cluster (n_k), the probabilities of a positive response for the new and standard procedures (p₁ and p₀, respectively) and a negative value which is materially unacceptable difference between the new and standard procedures (p₁₀ − p₀₁ = δ₀ = −0.1). We applied a specified correlation structures similar to those by Obuchowski [13] and Durkalski et al [4]. The structure specified four within cluster correlations are: between the responses for the new procedure (r₁); between responses for the standard procedure (r₂); between responses for new and standard procedures on the same units (r₃); and between those responses of different units (r₄). We set r₁ = r₂=r and r₄=r/4 since r₄ is likely much smaller relative to r, and consider various values of r and r₃. Using Matlab subroutine, we generate a random vector from a (n_k × 2 )-variate normal distribution with mean 0 and variance-covariance ∑ where correlation coefficients (r, r₃ and r₄) are specified. The 1^st n_k variates are the outcome of n_k units with the kth cluster given the new procedure and the last n_k variates are those given the standard procedure. Denote clustered normal variates as z. Then, we generate a binomial response for each procedure. Define y as y=1 if z ≤ c or y=0 otherwise where c is a cut-off point which satisfies Pr(z ≤ c) = p. When the number of units per cluster (n_k) varies, we consider two kinds of distributions: the value of n_k is generated from a uniform distribution and also from a beta distribution. In our simulation study, 10,000 data sets are generated for each configuration. If the expected Type 1 error rate of a test is indeed a nominal 0.05 level, then a 95% confidence interval for α=0.05 is (0.046, 0.054) from 10,000 simulated data. Those empirical Type 1 error rates outside of the interval are shown with an asterisk sign (*). When there are no discordant pairs in a cluster in simulations, an intraclass correlation coefficient is not estimable and such a case is excluded in Type 1 error rate and power calculations. If undefined cases occur, then they are excluded from both numerator and denominator in Type 1 error rate and power computations and empirical coverage rate and power are adjusted accordingly.

Table II-A, B and C summarize simulated Type 1 error rates and power of tests for K=(100, 50, 25), r₁=r₂=(0, 0.1, 0.4, 0.6), r₃=0.5, δ₀ = −0.1 and δ₁=0 with n_k=2 units per cluster and those with uniformly as well as non-uniformly distributed n_k ≤5 per cluster, respectively. Table II-A for n_k=2 shows that Z_AN and Z_O tests provide empirical Type 1 error rates which are satisfactorily close to a nominal 0.05 level except those cases K≤50 and p₀=0.2. In these cases, Z_AN test is conservative while Z_O test is anti-conservative. The Z_EV test gives reasonably close Type 1 error rates but it does not give reliable Type 1 error rates for K ≤50 with p₀=0.2 or for K=25 with p₀=0.8. The performance of the Z_ALB test is always anti-conservative and less satisfactory in a comparison with the Z_EV, Z_AN and Z_O tests. When a cluster size is uniformly distributed, Table II-B for n_k ≤5 indicates that Z_AN and Z_O tests provide satisfactorily accurate Type 1 error rates except K ≤50 with p₀=0.2. A size of the Z_EV test is reasonably close to a nominal 0.05 level except K≤25, or K≤50 with p₀=0.2 and that of the Z_ALB test is generally not satisfactory. When a cluster size is distributed non-uniformly, Table II-C showed that the Z_O test gives a reasonably accurate Type 1 error rate unless K ≤50 with p₀=0.2, and the Z_AN test is satisfactory except those cases of K=25 or K=50 with p₀=0.8. The Z_EV test is satisfactory unless K=25 and p₀=0.2. In summary, the Z_EV, Z_AN and Z_O tests are generally robust and comparable unless a number of clusters is small and they perform clearly better than the Z_ALB test in the accuracy of Type 1 error rates to the nominal one.

Table II.

Table II-A. Empirical Type 1 Error rates and power: nk=2,r3=0.5,r4=r1/4,δ0=−0.1 (p1 − p0 = −0.1) and δ1 = p1 − p0 = 0. Asterisk indicates a value is outside the 95% confidence interval on the nominal level of 0.05.
			Type 1 error rate (%)				Power (%)
K	p0	r1=r2	ZEV	ZALB	ZAN	ZO	ZEV	ZALB	ZAN	ZO
100	0.8	0	4.8	5.3	4.7	4.6	90.8	91.6	90.6	90.5
		0.1	4.8	5.2	4.7	4.6	89.1	90.1	88.8	88.8
		0.4	4.9	5.3	4.8	4.8	84.2	85.2	89.6	83.7
		0.6	4.8	5.3	4.6	4.6	80.1	81.5	79.1	79.5
50	0.8	0	5.4	6.0*	4.9	4.9	68.2	70.6	66.7	66.1
		0.1	5.4	6.0*	5.1	4.9	65.7	68.6	64.1	64.0
		0.4	5.4	6.1*	5.2	5.2	59.3	62.3	57.7	58.1
		0.6	5.3	6.4*	5.1	5.2	54.6	57.9	53.3	54.1
25	0.8	0	5.9*	6.0*	5.3	5.1	44.5	45.3	42.1	41.1
		0.1	5.9*	5.8*	5.1	4.9	42.9	43.4	40.0	39.4
		0.4	5.9*	6.2*	5.0	5.2	39.1	39.2	35.4	36.3
		0.6	5.8*	6.4*	5.0	5.1	36.3	37.4	32.9	33.6
100	0.5	0	5.2	5.7*	5.3	5.1	78.5	80.0	78.7	78.0
		0.1	5.0	5.5*	5.1	4.9	76.0	77.2	46.1	75.6
		0.4	5.2	5.5*	5.2	5.0	69.3	70.6	69.3	68.5
		0.6	5.1	5.5*	5.1	4.9	64.6	65.	64.8	64.0
50	0.5	0	4.6	5.4	4.9	4.5*	53.0	55.9	54.2	52.7
		0.1	4.9	5.7*	5.1	4.8	50.8	53.8	51.8	50.6
		0.4	4.8	5.7*	4.9	4.7	45.1	48.3	45.7	44.5
		0.6	5.1	5.9*	5.2	5.0	41.8	44.6	42.0	41.1
25	0.5	0	5.5*	6.2*	5.4	4.8	33.1	35.7	33.0	30.6
		0.1	5.7*	6.4*	5.6*	5.0	31.6	34.2	31.5	29.1
		0.4	5.3	6.2*	5.2	4.7	28.4	31.4	28.5	26.4
		0.6	5.2	6.2*	5.3	4.7	26.2	28.9	26.7	24.9
100	0.2	0	5.1	5.4	4.5*	5.1	90.5	91.3	90.3	90.3
		0.1	5.3	5.7*	4.8	5.2	89.1	89.9	88.7	88.8
		0.4	5.6*	6.0*	4.9	5.5*	84.5	85.6	84.0	84.1
		0.6	5.7*	6.2*	4.9	5.6*	80.4	81.8	79.6	79.9
50	0.2	0	5.8*	5.9*	4.3*	5.3	67.6	69.9	66.2	65.8
		0.1	5.8*	5.8*	3.9*	5.1	65.5	68.0	63.8	63.7
		0.4	6.2*	6.4*	4.5*	5.7*	59.8	62.3	58.3	58.9
		0.6	5.9*	6.6*	4.5*	5.5*	54.8	58.2	53.4	54.2
25	0.2	0	6.7*	5.1	4.3*	5.8*	45.2	46.1	43.0	42.0
		0.1	7.2*	5.5*	4.6	6.2*	44.3	44.5	41.2	40.8
		0.4	7.3*	5.5*	4.3*	6.6*	39.6	40.1	36.1	36.9
		0.6	7.2*	6.2*	4.2*	6.4*	37.3	37.9	33.9	34.8

Table II-B. Empirical Type 1 error rates and power: nk ≤ 5 where nk is uniformly distributed, r3=0.5, r4=r1/4, δ0 = −01. (p1 − p0 = −0.1) and δ1 = p1 − p0 = 0. Asterisk indicates a value is outside the 95% confidence interval on the nominal level of 0.05.
			Type 1 error rate (%)				Power (%)
K	p0	r1=r2	ZEV	ZALB	ZAN	ZO	ZEV	ZALB	ZAN	ZO
100	0.8	0	5.0	5.5	4.9	4.8	91.1	97.6	97.2	97.2
		0.1	5.0	5.4	4.9	4.8	89.8	95.9	95.3	95.2
		0.4	4.9	5.3	4.7	4.7	84.3	87.6	86.3	86.3
		0.6	4.6	5.1	4.4*	4.4*	80.3	80.6	78.5	78.8
50	0.8	0	5.2	5.9*	5.0	4.8	69.2	83.0	80.9	79.9
		0.1	5.3	6.1*	5.0	4.8	66.9	78.4	75.9	75.2
		0.4	5.0	6.0*	4.9	4.9	59.9	66.5	63.0	62.2
		0.6	5.2	6.1*	5.0	5.1	55.2	58.6	54.5	54.6
25	0.8	0	6.1*	6.2*	5.5*	5.1	47.3	60.7	56.9	53.5
		0.1	5.9*	5.9*	5.0	5.0	45.7	56.4	52.4	49.5
		0.4	5.7*	5.9*	4.9	5.0	40.1	46.3	41.2	39.7
		0.6	5.9*	6.4*	5.1	5.0	36.0	40.9	35.0	34.4
100	0.5	0	5.4	5.8*	5.5*	5.3	78.8	90.3	89.9	89.4
		0.1	5.2	5.6*	5.2	5.0	76.4	85.9	85.4	84.6
		0.4	5.0	5.5*	5.0	4.8	68.9	72.4	71.2	70.0
		0.6	5.1	5.5*	5.1	4.9	63.9	64.8	63.4	62.5
50	0.5	0	5.2	6.0*	5.5*	5.1	54.1	69.0	67.5	65.3
		0.1	5.2	6.0*	5.4	5.1	51.2	63.2	61.5	59.1
		0.4	5.5*	6.4*	5.6*	5.4	44.7	49.4	47.4	44.8
		0.6	5.0	5.7*	5.1	4.8	40.6	42.0	40.0	38.2
25	0.5	0	5.5*	6.1*	5.4	4.9	35.8	48.2	54.9	41.1
		0.1	5.5*	6.2*	5.4	5.0	33.5	43.3	40.9	36.5
		0.4	5.6*	6.3*	5.5*	4.8	28.5	34.1	31.7	27.9
		0.6	5.2	6.2*	5.2	4.6	26.3	29.8	27.2	24.5
100	0.2	0	5.1	5.6*	4.6	5.1	90.9	97.4	97.1	97.0
		0.1	5.1	5.6*	4.7	5.1	89.4	95.8	95.1	95.1
		0.4	5.2	5.5*	4.4*	5.1	84.0	87.6	86.3	86.3
		0.6	5.4	5.9*	4.7	5.3	79.7	80.7	78.5	78.9
50	0.2	0	5.9*	6.0*	4.3*	5.3	68.8	82.2	80.4	79.4
		0.1	5.8*	5.8*	4.1*	5.2	66.9	77.8	75.6	74.7
		0.4	6.3*	6.6*	4.5*	5.6*	60.5	66.3	63.0	62.3
		0.6	6.7*	7.4*	5.1	6.1*	55.7	58.3	54.2	54.4
25	0.2	0	6.8*	5.8*	4.7	5.8*	47.2	60.7	56.8	53.2
		0.1	6.9*	5.4	4.4*	5.8*	45.5	56.7	52.4	49.6
		0.4	7.5*	5.5*	40*	6.5*	39.9	47.1	41.6	39.9
		0.6	7.6*	6.9*	4.5*	6.8*	36.1	40.8	34.8	34.3

Table II-C. Empirical Type 1 error rates and power: nk ≤ 5 where nk is non-uniformly distributed, r3=0.5, r4=r1/4, δ0 = −0.1. (p1 − p0 = −01) and δ1 = p1 − p0 = 0. Asterisk indicates a value is outside the 95% confidence interval on the nominal level of 0.05.
			Type 1 error rate (%)				Power (%)
K	p0	r1=r2	ZEV	ZALB	ZAN	ZO	ZEV	ZALB	ZAN	ZO
100	0.8	0	5.2	5.6*	5.2	5.1	97.1	98.2	98.0	98.0
		0.1	4.9	5.5*	5.1	4.9	95.9	97.0	96.7	96.7
		0.4	5.0	5.6*	5.0	4.9	89.9	91.2	90.1	90.3
		0.6	5.0	5.4	4.8	4.7	84.9	85.5	89.8	84.0
50	0.8	0	5.1	6.2*	5.6*	5.2	79.1	83.4	81.5	81.1
		0.1	5.4	6.2*	5.5*	5.2	76.0	79.8	77.7	77.1
		0.4	5.3	6.2*	5.4	5.2	66.7	69.4	66.2	65.6
		0.6	5.6*	6.6*	5.5*	5.3	59.2	61.9	58.4	58.3
25	0.8	0	5.2	7.0*	5.9*	5.2	55.1	61.4	57.0	54.7
		0.1	5.0	6.7*	5.6*	4.9	51.6	57.6	53.1	51.0
		0.4	5.5*	7.1*	5.8*	5.2	43.3	48.4	43.2	42.0
		0.6	5.5*	7.1*	5.6*	5.1	38.2	42.6	37.4	36.6
100	0.5	0	5.0	5.3	5.1	4.8	89.2	92.0	91.5	91.2
		0.1	4.9	5.5*	5.2	4.9	85.7	88.0	87.5	87.0
		0.4	4.8	5.5*	5.1	4.8	74.9	76.7	75.6	74.8
		0.6	5.2	5.7*	5.3	5.1	67.7	68.5	67.2	66.4
50	0.5	0	5.3	6.1*	5.7*	5.1	64.9	69.6	68.0	66.2
		0.1	5.1	6.0*	5.5*	5.0	60.2	64.3	62.8	60.9
		0.4	5.0	5.8*	5.3	4.7	49.4	52.3	50.3	48.3
		0.6	4.7	5.7*	5.0	4.6	43.6	46.2	44.3	42.6
25	0.5	0	5.0	6.4*	5.7*	4.8	42.5	48.8	46.3	42.3
		0.1	5.1	6.4*	5.8*	4.6	38.9	44.0	41.7	38.1
		0.4	5.0	6.6*	5.7*	4.7	31.0	35.9	33.4	29.9
		0.6	5.1	6.6*	5.7*	4.9	27.6	31.3	28.7	26.2
100	0.2	0	5.4	6.1*	5.1	5.3	96.7	97.7	97.4	97.3
		0.1	5.4	6.1*	5.2	5.4	95.4	96.4	96.0	95.9
		0.4	5.4	6.2*	5.1	5.5*	89.8	90.8	89.7	89.7
		0.6	5.6*	6.4*	5.3	5.7*	84.1	85.1	83.4	83.7
50	0.2	0	5.6	6.5*	5.3	5.4	79.8	83.8	81.7	81.2
		0.1	5.6	6.6*	5.6*	5.7*	76.9	80.5	78.6	78.0
		0.4	6.1	7.1*	5.5*	5.9*	66.1	68.9	65.7	65.2
		0.6	5.8	6.9*	5.2	5.8*	59.3	61.6	58.0	57.9
25	0.2	0	6.0*	7.6*	5.6*	5.7*	55.3	62.1	57.5	55.4
		0.1	6.2*	7.6*	5.6*	5.8*	52.3	58.2	53.5	51.7
		0.4	6.7*	7.8*	5.5*	6.4*	43.7	48.9	43.6	52.2
		0.6	6.7*	8.1*	5.5*	6.4*	38.8	43.6	38.1	37.3

Empirical power of each of the four tests is decreasing as an intraclass correlation coefficient increases, and it is greater as a number of clusters increases. Power is related to a response rate of the standard procedure p₀: it is greater as a departure of p₀ from 0.5 increases. Power is also inversely related to a size of the test. In assessment of power along with empirical Type 1 error rates, we may consider Z_EV, Z_AN and Z_O tests are competitive in power for n_k=2 and n_k ≤5. Although power ofZ_ALB method is slightly greater than those other tests, it does not imply that the Z_ALB test is more powerful since a Type 1 error rate of the Z_ALB test is always more anti-conservative and power is inversely related with Type 1 error rate. Simulations for those with various values of r₃, and r₄ =r₁/8 yield similar results in statistical properties.

We examine a relative importance of a number of clusters and cluster size for a fixed total number of units. Table III shows average empirical Type 1 error rates and power of a test over various values of p₀ and r₁. It demonstrates that, for a fixed total number of units (e.g., N=200), reliability of type 1 error and power of a non-inferiority test based on clustered matched pair data are more closely related with a number of clusters than a cluster size. Simulations show that a large number of clusters is pivotal to obtain accurate Type 1 error rate and good power of each of the four tests particularly when a cluster size is small.

Table III.

Average empirical Type 1 error rate and power of each test over P₀ = (0.8,0.5,0.2), r₁ = r₂ = (0,0.1,0.4,0.6),r₃ = 0.5 and r₄ = r₁ / 4 Those number in a doted bracket are Type 1 error rates and power for the total number of units N = k × n = 200.

Clusters		Average Type 1 error rate (%)			Average power (%)
K		n=2	n=4	n=8	n=2	n=4	n=8
100	ZEv	5.11	5.39	5.25	81.4	91.6	94.9
	ZALB	5.54	5.89	5.84	82.5	92.3	95.4
	ZAN	4.86	5.34	5.36	81.1	91.7	95.1
	ZO	4.98	5.33	5.18	81.0	91.5	94.8
50	ZEv	5.38	5.26	5.16	57.2	72.7	82.3
	ZALB	5.98	6.16	6.02	60.0	75.1	84.1
	ZAN	4.80	5.32	5.38	56.4	73.2	82.9
	ZO	5.05	5.12	4.97	56.1	72.3	81.7
25	ZEv	6.14	5.41	5.27	37.4	48.9	62.5
	ZALB	5.96	6.94	6.69	38.7	53.8	66.6
	ZAN	4.94	5.63	5.75	35.4	50.3	64.2
	ZO	5.37	5.13	4.85	34.7	47.8	61.2

4. AN EXAMPLE

Consider a diagnostic study for comparing two imaging techniques, i.e., positron emission tomography (PET) and single photon emission CT scan (SPECT), Neumann et al [20]. In 21 patients, there were 51 glands confirmed at surgery not to have hyperparathyroidism. The number of glands for each patient ranges from one to four with the average 2.4 glands. Consider a patient and glands in a patient as a cluster and units in a cluster, respectively. (See Table IV, Obuchowski [13]). PET provides good resolution images in detecting abnormal parathyroid but is expensive while SPECT is inexpensive and quickly performed but gives lower resolution. Weighting cost and time as well as accuracy, we may consider that SPECT is non-inferior to PET when the former must be no more than 10% lower than the latter in specificity, i.e., testing null against alternative hypotheses are H₀ : δ = −0.1 vs. H₁ : δ > −0.1 where p_10k − p_01k = δ for k=1, 2, …, 21. When a possible correlation within cluster is ignored, we have Z_LB=4.06 and Z_N=3.30 based on pooled data from (2) and (3). When they are adjusted by using an inflation factor based on discordant pairs, we have Z_ALB=3.31 (p=0.0005) and Z_AN=2.68 (p=0.004) from (4) and (5). Values of the adjusted statistics are smaller than those unadjusted ones. From (1) and (6), the statistic proposed by Durkalski et al [4] and the modified Obuchowski one are Z_EV=2.61 (p=0.005) and Z_O=2.62 (p=0.005), and two values are essentially identical. They are similar to the value of Z_AN. These three statistics are smaller than Z_ALB which is known to be anti-conservative in Section 3.

5. DISCUSSION

Lu and Bean [2] and Nam [3] have developed test statistics for non-inferiority when matched pairs are independent. The former is based on a Wald-type method and the latter on the likelihood score method. These test statistics are not applicable for clustered matched pair data. However, they can be adjusted properly using an inflation factor resulting from correlated pairs in a cluster. The adjusted statistics are appropriate for testing non-inferiority in clustered pair data. Durkalski et al [4] have presented a proper test statistic based on moment estimate. In addition, we also consider a modification of Obuchowski’s method [13]. We investigate four statistical methods applicable to analysis of clustered matched pair data for non-inferiority. For a large number of clusters, empirical Type 1 error rates of the test based on moments estimate, the adjusted score test by an inflation factor and a modified Obuchowski’s test for non-inferiority are satisfactorily close to a nominal level. When a number of clusters is small and a positive response rate for the standard procedure is low, the test based on moments estimate and the modified test are anti-conservative. The adjusted Wald-type test for non-inferiority is always anti-conservative. We may not recommend the Wald-type method for testing non-inferiority in clustered matched-pair studies. The adjusted score, moment based tests and the modified method are reasonably robust. The latter two tests are computationally simple and easy to apply while the former requires more computations but it yield information regarding an intra-class correlation coefficient in clustered matched-pairs. If tests ignoring the positive correlation within cluster do not reject non-inferiority, then tests adjusted by an inflation factor also do not reject the null hypothesis when the estimated correlation is positive. If an unadjusted test rejects the null, then the adjusted test may or may not reject the null and it is necessary to conduct the adjusted test for a statistical significance. The intra-class correlation coefficient involving an inflation factor for an adjustment may be also estimated using information from all pair types factor instead of those of discordant pairs only [12,18]. However, our empirical results indicate that this type of adjustment does not improve the accuracy of Type 1 error rates. When an intraclass coefficient is positive but close to zero, an estimated intraclass correlation coefficient can be negative due to sampling variations. We did not exclude such a case for empirical Type 1 error rate and power calculations. The exclusion may be resulted to bias of empirical error rate and power. For no intraclass correlation, a test without adjustment is appropriate. We employed the analysis of variance estimator of intraclass correlation coefficient to adjust Z_LB and Z_N test statistics in this paper. Although this is a commonly used estimator, there are also alternative estimators of the intraclass correlation coefficient. Donner [21] and Ridout et al [22] provide extensive reviews on estimators of intraclass correlation coefficient. The variance inflation factor is also called the design effect in sampling survey. Respondents in the same cluster are likely to be somewhat similar to one another. Selecting an additional member from the same cluster provides less information than would be a completely independent selection. The loss of effectiveness by cluster sampling, instead of simple random sampling is the design effect. The inflation factor (design effect) is the ratio of the variance of a statistic under use of cluster sampling to that under use of simple random sampling. The inflation factor is greater as an intraclass correlation increases. In cluster sampling, the clusters are treated as the sampling elements so analysis is done on a population of clusters. In stratified sampling, a random sample is drawn from each of strata and analysis is done on elements within strata. In this paper, we investigate non-inferiority tests for clustered matched pair data. Nam [24] studied noninferiority tests for stratified pair design. Liu et al [25] examined equivalence and non-inferiority when pairs are independent. For a small sample size, a test with a continuity correction leads to the test conservative. When a number of clusters is large, a continuity correction is not necessary. For a fixed total number of units, the accuracy of Type 1 error rate and power of a test is more closely related to a number of clusters than cluster size. In designing a clustered matched-pair study for non-inferiority, one should make a great effort to have a sufficiently large number of clusters. The asymptotic normality of a test statistic is based on the central limit theory and a large number of clusters is essential for a reliable p-value and a good power of the test.

APPENDIX

Define S_k = x_10k + x_01k for k=1, 2, …, K and K_d as the number of clusters at least one discordant pairs: $K_d={\displaystyle \sum _{k=1}^K{I}_k}$ where I_k=0 when S_k=0 and I_k=1 when S_k ≥1. The unadjusted and adjusted mean numbers of discordant pairs are $\overline{S}={\displaystyle \sum _{k=1}^K{S}_k/K_d}$ and $S_0=\overline{S}-\left\{{\displaystyle \sum _{k=1}^K{(S_k-\overline{S})}^2-(K-K_d){\overline{S}}^2}\right\}/\left\{K_d(K_d-1)\overline{S}\right\}$, e.g., [12, 23]. The mean squares between clusters and within clusters are expressed as $\mathit{\text{BMS}}=\left\{{\displaystyle \sum _{k=1}^K{(x_{10k}-S_k\overline{p})}^2/S_k}\right\}/K_d$ where $\overline{p}={\displaystyle \sum _{k=1}^K{x}_{10k}/{\displaystyle \sum _{k=1}^K{S}_k}}$ and $\mathit{\text{WMS}}={\displaystyle \sum _{k=1}^K(x_{10k}{x}_{01k}/S_k)/\left\{K_d(\overline{S}-1)\right\}}$ where Sk ≥1. An estimator of the intra-class correlation coefficient is ρ̂ = (BMS − WMS) / {BMS + (S₀ − 1)WMS}, e.g.,[12, 18, 23]. If S_k =0, then the k^th component of WMS is undefined and excluded from analysis. When all S_k’s are zero across clusters, the ρ̂ is not estimable.

An inflation factor resulting from intra-class correlation is written as c̅ = 1+(n_c − 1) ρ̂ where the adjusted mean number of discordant pairs is

$$n_c=S_0+K_d(S_0-\overline{S})=\overline{S}+{\widehat{\sigma }}^2/\overline{S}\text{where}{\widehat{\sigma }}^2=\left\{{\displaystyle \sum _{k=1}^K(S_k-\overline{S})-(K-K_d){\overline{S}}^2}\right\}/K_d.$$ (A1)

The 2^nd term of n_c is the variance-to-mean ratio which is a measure of a relative dispersion of S_k’s. Denote the mean cluster size as $\overline{n}={\displaystyle \sum _{k=1}^K{n}_k/K}$. The ratio of an adjusted mean number of discordant pairs to the mean cluster size is n_c / n̅ = S̅ / n̅ + σ̂² / (n̅S̅) from (A1). Assume p_10k = p₁₀ and p_01k = p₀₁ for every k. We have E(I_k) = Pr(S_k≥1) = 1 − Pr(S_k = 0) =1− (1− p₁₀ − p₀₁ )^n_k. When a cluster size is the same (n_k = n for every k , i.e., n̅ = n), we have $E(K_d)={\displaystyle \sum _{k=1}^{K}E(I_k)\to K}$ as n → ∞. Thus, n_c / n̅ → f = p₁₀ + p₀₁ as n → ∞. Similarly, we have n_c / n̅ → f as n̅ → ∞ when a cluster size varies.