A Weighted Rank-Sum Procedure for Comparing Samples with Multiple Endpoints

Summary

For comparing the distribution of two samples with multiple endpoints, O’Brien (1984) proposed rank-sum-type test statistics. Huang et al. (2005) extended these statistics to the general nonparametric Behrens-Fisher hypothesis problem and obtained improved test statistics by replacing the ad hoc variance with the asymptotic variance of the rank-sum statistics. In this paper we generalize the work of O’Brien (1984) and Huang et al. (2005) and propose a weighted rank-sum statistic. We show that the weighted rank-sum statistic is asymptotically normally distributed, permitting the computation of power, p-values and confidence intervals. We further demonstrate via simulation that the weighted rank-sum statistic is efficient in controlling the type I error rate and under certain alternatives, is more powerful than the statistics of O’Brien (1984) and Huang et al.(2005).

1. Introduction

Comparison of two or more samples with multiple endpoints is a common statistical problem in biomedical research. As an example, O’Brien (1984) described a randomized clinical trial of two therapies for the treatment of diabetes to investigate whether the experimental therapy yields better nerve function as measured by 34 electromyographic variables. Huang et al. (2005) gave another example of a clinical trial of Coenzyme Q₁₀ in treating early Parkinson’s disease to slow the functional decline of the disease, as indexed by a number of outcome measures, including mentation, motor and average daily living scales. Other examples can be found in Pocock, Geller and Tsiatis (1987), Shames et al. (1998), Tilley et al. (2000), and Li, Zhao and Paty (2001), to name a few.

Hotelling’s T² and the Bonferroni procedure are two popular approaches for comparing two multivariate samples. Hotelling’s T² is a global test statistic and makes no distinction between variables in their direction of change. The Bonferroni procedure assigns the Type I error for each variable and then tests the null hypothesis concerning each individual variable. Noting these drawbacks of the two methods, O’Brien (1984) proposed a nonparametric procedure, a rank-sum-type test, which is based on the rank of each individual variable among the combined observations from the two samples. Under the null hypothesis that the two multivariate samples have the same distribution, O’Brien’s (1984) rank-sum test statistic asymptotically is distribution-free and follows a standard normal distribution. Huang et al. (2005) noticed that under a more general null hypothesis in the Behrens-Fisher problem, e.g. Troendle (2002), O’Brien’s (1984) test statistics are no longer distribution-free and can substantially inflate the Type I error rate when used for testing the general Behrens-Fisher hypothesis. Subsequently Huang et al. (2005) provided a modification of O’Brien’s (1984) test by adjusting for the variances of the rank sums.

Generalizing O’Brien’s (1984) rank-sum test and the modified test of Hang et al. (2005), we propose a weighted-rank-sum statistic for testing the general nonparametric Behrens-Fisher hypothesis. The weights can be chosen to be constants emphasizing the importance of the individual variables, or they can be chosen to minimize the variance of the weighted-rank-sum statistic. Under mild conditions, the weighted-rank-sum statistic is asymptotically normally distributed, thus permitting the computation of power, p-values and confidence intervals. Simulation studies demonstrate that the weighted rank-sum statistic is efficient in controlling type I error and is more powerful than the statistics of O’Brien (1984) and Huang et al. (2005) for certain alternatives.

2. Weighted-Rank-Sum Statistics for the Behrens-Fisher Problem

Suppose our interest is to compare the distribution of two p-dimensional variables, X = (X₁, ···, X_p)’, and Y = (Y₁, ···, Y_p)’, representing the outcomes of p endpoints from subjects in, say, the standard therapy arm and the experimental therapy arm in a clinical trial, or the controls and cases in a case-control study, respectively. We assume that X and Y follow distributions F and G, with marginal distributions F_a and G_a of X_a and Y_a respectively, where a = 1, ···, p. Following Huang et al. (2005), we define

$${\theta }_a=\mathrm{Pr}(X_a<Y_a)-\mathrm{Pr}(X_a>Y_a),a=1,\cdots ,p,$$ (1)

and consider testing the null hypothesis

$$H_0:{\theta }_1=\cdots ={\theta }_p=0.$$ (2)

This is a nonparametric version of the Behrens-Fisher problem. The null space under H₀, {(F, G): θ₁ = ··· = θ_p = 0.}, is larger than the usual null space under $H_0,\{(F,G):{\theta }_1=\cdots ={\theta }_p=0.\}$. In a clinical trial setting θ_a can be viewed as a measure of the marginal treatment efficacy (corresponding to the ath endpoint) of the experimental therapy relative to the standard therapy, assuming that larger outcomes indicate better treatment results. Thus a larger positive value of θ_a indicates better treatment results with respect to the ath outcome variable for the experimental therapy than the standard.

2.1 Rank-sum type test statistics

Let x_i = (x_i1, ···, x_ip)’, i = 1, ···, m, be the outcomes for the ith subject from the X-sample and y_j = (y_j1, ···, y_jp)’, j = 1, ···, n, be the outcomes of the jth subject from the Y-sample, and write N = m + n. For the ath outcome variable, a = 1, ···, p, we combine the two samples and rank the N observations x_1a, ···, x_ma, y_1a, ···, y_na, and denote by R_xia and R_yja, the midrank of x_ia and y_ja, respectively. Then we observe $S_{xi}={\sum }_{a=1}^p{R}_{xia}$ for each subject, i = 1, ···, m, from the X-sample, and $S_{yj}={\sum }_{a=1}^p{R}_{yja}$ for each subject, j = 1, ···, n from the Y-sample, by summing up the ranks of the p variables. O’Brien (1984) suggested reducing the problem of comparing two multivariate distributions to one of comparing the rank sums between {S_xi, i = 1, ···, m} and {S_yj, j = 1, ···, n} using the usual two-sample t-tests. This yields a t-test statistic

$$T_1=\frac{{\stackrel{‒}{S}}_y-{\stackrel{‒}{S}}_x}{\widehat{\sigma }\sqrt{1∕m+1∕n}},T_2=\frac{{\stackrel{‒}{S}}_y-{\stackrel{‒}{S}}_x}{\sqrt{{\widehat{\sigma }}_x^2∕m+{\widehat{\sigma }}_y^{2}n}},$$ (3)

analogous to the usual two-sample t-test with equal variances and unequal variances, respectively, where ${\stackrel{‒}{S}}_x={\sum }_{i=1}^m{S}_{xi}∕m$, ${\stackrel{‒}{S}}_y={\sum }_{j=1}^n{S}_{yi}∕n$, ${\widehat{\sigma }}_x^2={\sum }_{i=1}^m{(S_{xi}-{\stackrel{‒}{S}}_x)}^2∕(m-1)$, ${\widehat{\sigma }}_y^2={\sum }_{j=1}^n{(S_{yj}-{\stackrel{‒}{S}}_y)}^2∕(n-1)$ and ${\widehat{\sigma }}^2=((m-1){\widehat{\sigma }}_x^2+(n-1){\widehat{\sigma }}_y^2)∕(N-2)$.

Huang et al. (2005) noticed that under the more restricted null hypothesis, $H_0^{\prime }$ : F = G, both T₁ and T₂ asymptotically follow the standard normal distribution. However, when F ≠ G these two statistics remain asymptotically normally distributed, but with nonunit variances. When used to test the Behrens-Fisher hypothesis H₀, these test statistics can substantially inflate the Type I error rate, as demonstrated in Huang et al. (2005). To make O’Brien’s (1984) test suitable for testing the null hypothesis H₀, Huang et al. (2005) derived the asymptotic variances of the two statistics and suggested using the following two modified test statistics for H₀:

$$T_{1a}=\frac{{\stackrel{‒}{S}}_y-{\stackrel{‒}{S}}_x}{\widehat{\sigma }\sqrt{{\widehat{h}}_1(1∕m+1∕n)}},T_{2a}=\frac{{\stackrel{‒}{S}}_y-{\stackrel{‒}{S}}_x}{\sqrt{{\widehat{h}}_2({\sigma }_x^2∕m+{\sigma }_y^2∕n)}},$$ (4)

where ${\widehat{h}}_1$ and ${\widehat{h}}_2$ are consistent estimates of

$$h_1=\frac{\sum _{a=1}^p\sum _{b=1}^p{(1+\lambda )}^2(c_{ab}+d_{ab}\lambda )}{\sum _{a=1}^p\sum _{b=1}^p(e_{ab}{\lambda }^3+(d_{ab}+2f_{ab}){\lambda }^2+(c_{ab}+2{\eta }_{ab})\lambda +{\xi }_{ab})},$$

and

$$h_2=\frac{\sum _{a=1}^p\sum _{b=1}^p{(1+\lambda )}^2(c_{ab}+d_{ab}\lambda )}{\sum _{a=1}^p\sum _{b=1}^p(d_{ab}{\lambda }^3+(e_{ab}+2{\eta }_{ab}){\lambda }^2+({\xi }_{ab}+2f_{ab})\lambda +c_{ab})},$$

respectively, with $\lambda =m∕n$, $c_{ab}=\mathrm{Cov}(G_a\left(X_a\right),G_b\left(X_b\right))$, $d_{ab}=\mathrm{Cov}(F_a\left(Y_a\right),F_b\left(Y_b\right))$, $e_{ab}=\mathrm{Cov}(F_a\left(X_a\right),F_b\left(X_b\right))$, $f_{ab}=\mathrm{Cov}(F_a\left(X_a\right),G_b\left(X_b\right))$, and ${\xi }_{ab}=\mathrm{Cov}(G_a\left(Y_a\right),G_b\left(Y_b\right))$, ${\eta }_{ab}=\mathrm{Cov}\left(G_a\left(Y_a\right),F_b\left(Y_b\right)\right)$.

Huang et al. (2005) further showed that, under the null hypothesis H₀, the two test statistics asymptotically follow the standard normal distribution and thus the Type I error rate can be controlled at significance level α by rejecting H₀ if the magnitude of the test statistic exceeds the critical value of Φ^-1(1 - α/2), where Φ is the standard normal distribution function.

2.2 Weighted rank-sum statistics

O’Brien’s (1984) rank-sum test and the modified version of Huang et al. (2005) gave equal weights to the rank of each individual variable. In many situations unequal weights are desirable so that different emphasis can be assigned to different variables. Moreover statistical optimization requires that the weights be proportional to the reciprocals of the variances of the variables to be combined, when the variables are mutually independent. Taking these arguments into consideration, we propose using weighted rank-sum statistics. Let w_a ≥ 0, a = 1, . . ., p, be a constant or a random variable. The weighted rank for the ith subject in the X-sample is defined as $R_{xi}={\sum }_{a=1}^p{w}_a{R}_{xia}$, i = 1, ···, m, and the weighted rank for the jth subject in the Y-sample is $R_{yj}={\sum }_{a=1}^p{w}_a{R}_{yja}$, j = 1, ···, n. Setting w_a = 1 for every a, a = 1, . . . , p leads to the test statistics considered by O’Brien (1984) or Huang et al. (2005). Moreover, if only a subset of the p variables are of interest, we can set the weights to be one for variables in the subset and zero for variables not in the subset.

Using the weighted rank sum, we propose the following two test statistics:

$$T_{w1}=\frac{{\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x}{\widehat{\sigma }\sqrt{{\widehat{h}}_{w1}(1∕m+1∕n)}},T_{w2}=\frac{{\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x}{\sqrt{{\widehat{h}}_{w2}({\widehat{\sigma }}_x^2∕m+{\widehat{\sigma }}_y^2∕n)}},$$ (5)

where ${\stackrel{‒}{R}}_x={\sum }_{i=1}^m{R}_{xi}∕m$, ${\stackrel{‒}{R}}_y={\sum }_{j=1}^m{R}_{yj}∕n$, ${\widehat{\sigma }}_x^2={\sum }_{i=1}^m{(R_{xi}-{\stackrel{‒}{R}}_x)}^2∕(m-1)$, ${\widehat{\sigma }}_y^2={\sum }_{j=1}^n{(R_{yj}-{\stackrel{‒}{R}}_y)}^2∕(n-1)$ and ${\widehat{\sigma }}^2=[(m-1){\widehat{\sigma }}_x^2+(n-1){\widehat{\sigma }}_y^2]∕(m+n-2)$ and ${\widehat{h}}_{w1}$ and ${\widehat{h}}_{w2}$ are consistent estimates (e.g. empirical estimates) of

$$h_{w1}=\frac{\sum _{a=1}^p\sum _{b=1}^p{(1+\lambda )}^2{w}_a{w}_b(c_{ab}+d_{ab}\lambda )}{\sum _{a=1}^p\sum _{b=1}^p{w}_a{w}_b[e_{ab}{\lambda }^3+(d_{ab}+2f_{ab}){\lambda }^2+(c_{ab}+2{\eta }_{ab})\lambda +{\xi }_{ab}]},$$

and

$$h_{w2}=\frac{\sum _{a=1}^p\sum _{b=1}^p{(1+\lambda )}^2{w}_a{w}_b(c_{ab}+d_{ab}\lambda )}{\sum _{a=1}^p\sum _{b=1}^p{w}_a{w}_b[d_{ab}{\lambda }^3+(e_{ab}+2{\eta }_{ab}){\lambda }^2+({\xi }_{ab}+2f_{ab})\lambda +c_{ab}]},$$

respectively, with c_ab, d_ab, e_ab, f_ab, ξ_ab and η_ab as for Eq. (4).

We have the following results.

Theorem 1

Under the null hypothesis H₀, T_w1andT_w2 both converge in distribution to the standard normal distribution as min{m, n} → ∞ and 0 < λ = lim{m/n} < ∞.

Therefore H₀ is rejected if |T_w1| or |T_w2| is larger than Φ^-1(1 - α/2); both tests asymptotically maintain the Type I error rate at the nominal level of α.

Proof of Theorem 1

Denote the indicator function by I_{·} on a set by I_{·}. Let Λ = (λ_ab), with ${\lambda }_{ab}=\mathrm{Cov}({\stackrel{‒}{R}}_{ya}-{\stackrel{‒}{R}}_{xa},{\stackrel{‒}{R}}_{yb}-{\stackrel{‒}{R}}_{xb})$ and let w = (w₁, ···, w_p)’.

$${\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x=\frac{m+n}{2mn}\sum _{a=1}^p{w}_a\sum _{i=1}^m\sum _{j=1}^n\{I_{\{x_{ia}<y_{ja}\}}-I_{\{x_{ia}>y_{ja}\}}\}=\sqrt{m+n}{w}^{\prime }U,$$

where

$$U=\frac{\sqrt{m+n}}{2mn}(\sum _{i=1}^m\sum _{j=1}^n\{I_{\{x_{i1}<y_{j1}\}}-I_{\{x_{i1}>y_{j1}\}}\},\cdots ,\sum _{i=1}^m\sum _{j=1}^n\{I_{\{x_{ip}<y_{jp}\}}-I_{\{x_{ip}>y_{jp}\}}\}{)}^{\prime }.$$

Note that U is a p-dimensional vector with each element being a U-statistic. It follows from standard asymptotic theory on U-statistics that, under the null hypothesis H₀, U converges to a p-variate normal distribution with mean 0 = (0, ···, 0)’ and variance-covariance matrix Δ = Cov(U) as min{m, n} → ∞ and 0 < m/n < ∞. Therefore $({\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x)∕\sqrt{w^{\prime }\Delta w}$ asymptotically follows a standard normal distribution under the null hypothesis H₀. Hence, it suffices to show that $\widehat{\sigma }\sqrt{{\widehat{h}}_{w1}(1∕m+1∕n)}$ and $\sqrt{{\widehat{h}}_{w2}({\widehat{\sigma }}_x^2∕m+{\widehat{\sigma }}_y^2∕n)}$ are consistent estimates of $\sqrt{w^{\prime }\Delta w}$, which can be derived following the proof of Theorem 1 and Theorem 2 in Huang et al. (2005).

3. Selection of Weights

The weights w can be chosen to meet practical needs, for example, to exclude some variables by setting the weights to be zero. In other situations the weights can be so determined that the variances of the weighted rank-sum is minimized at certain parameter values in the null or alternative hypothesis space. Here we search for the w that minimizes the variance V(w) of ${\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x$ under the null hypothesis. To this end, we first give the following definitions. For any a ∈ {1, ···, p}, define R_y(x_ia) to be the midrank of x_ia among {x_ia, y_1a, ···, y_na}, R_x(x_ia) the midrank of x_ia among {x_1a, ···, x_ma}, R_x(y_ja) the midrank of y_ja among {x_1a, ···, x_ma, y_ja}, R_y(y_ja) the midrank of y_ja among {y_1a, ···, y_na}. Let I_k be the identity matrix of order k, J_k be the column vector of order k whose elements are 1, and define $H_k=I_k-J_k{J}_k^{\prime }∕k$. Then following Huang et al. (2005), we can obtaine consistent estimates of ${\widehat{h}}_{w1}$ and ${\widehat{h}}_{w2}$ as

$${\widehat{h}}_{w1}=\frac{s^2}{mn}\times \frac{w^{\prime }(P^{\prime }P+U^{\prime }U)w}{w^{\prime }[{(P+Q)}^{\prime }(P+Q)+{(U+V)}^{\prime }(U+V)]w},$$

and

$${\widehat{h}}_{w2}=s^2\times \frac{w^{\prime }(P^{\prime }P+U^{\prime }U)w}{w^{\prime }\left[n^2{(P+Q)}^{\prime }(P+Q)m^2{(U+V)}^{\prime }(U+V)\right]w},$$

respectively, where P = (p_ia)_m×p with $p_{ia}=2R_y\left(x_{ia}\right)-2-n+n{\widehat{\theta }}_a$, Q = (q_ia)_m×p with q_ia = 2R_x(x_ia) - 1 - m, U = (u_ja)_n×p with $u_{ja}=2R_x\left(y_{ja}\right)-2-m-m{\widehat{\theta }}_a$, and V = (v_ja)_n×p with q_ja = 2R_y(y_ja)-1-n, where i = 1, ···, m, a = 1, p, j = 1, ···, n, and ${\widehat{\theta }}_a={\sum }_{i=1}^m{\sum }_{j=1}^n\{I_{\{x_{ia}<y_{ja}\}}-I_{\{x_{ia}>y_{ja}\}}\}∕\left(mn\right)$.

Hence, for T_w1 and T_w2, we have the estimated variances of ${\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x$,

$$\widehat{V_1\left(w\right)}=\frac{{(m+n)}^3}{m^2{n}^2(m+n-2)}\times \frac{w^{\prime }[M_x^{\prime }{H}_m{M}_x+M_y^{\prime }{H}_n{M}_y]w{w}^{\prime }(P^{\prime }P+U^{\prime }U)w}{w^{\prime }[{(P+Q)}^{\prime }(P+Q)+{(U+V)}^{\prime }(U+V)]w},$$

and

$$\widehat{V_2\left(w\right)}={(m+n)}^2\times \frac{w^{\prime }[\frac{M_x^{\prime }{H}_m{M}_x}{m(m-1)}+\frac{M_y^{\prime }{H}_n{M}_y}{n(n-1)}]w{w}^{\prime }(P^{\prime }P+U^{\prime }U)w}{w^{\prime }[n^2{(P+Q)}^{\prime }(P+Q)+m^2{(U+V)}^{\prime }(U+V)]},$$

where M_x = (R_xia) and M_y = (R_yja), the rank matrix for the X-sample and Y-sample, respectively.

The optimal weights w₁ (or w₂) are those that minimize the variance of ${\stackrel{‒}{R}}_y-{\stackrel{‒}{R}}_x$, and they can be estimated by

$${\widehat{w}}_1={\mathrm{argmin}}_{w^{\prime }{J}_p=1,w\ge 0}\widehat{V_1\left(w\right)},{\widehat{w}}_2={\mathrm{argmin}}_{w^{\prime }{J}_p=1,w\ge 0}\widehat{V_2\left(w\right)}.$$

The weights and their estimates can be computed only numerically, since there are no closed forms. Furthermore numerical results show that ${\widehat{w}}_1\approx {\widehat{w}}_2$. This is understandable since, as pointed out earlier, both $\widehat{\sigma }\sqrt{{\widehat{h}}_{w1}(1∕m+1∕n)}$ and $\sqrt{{\widehat{h}}_{w2}({\widehat{\sigma }}_x^2∕m+{\widehat{\sigma }}_y^2∕n)}$ are consistent estimates of $\sqrt{w^{\prime }\Delta w}$.

4. Simulation Study and Real Data Example

4.1 Simulation studies

In this section, we conduct a simulation study to evaluate the type I error rate and power of the proposed tests, T_w1 and T_w2, for comparison with those of O’Brien (1984), T₁ and T₂, and Huang et al. (2005), T_h1 and T_h2. To this end, we consider X = (x_i1, X_i2)’, i = 1, . . ., m, random samples from a bivariate normal distribution with mean (0, 0.5)′ and variance-covariance matrix $\left(\begin{array}{cc}\hfill 1\hfill & \hfill 0.8\hfill \\ \hfill 0.8\hfill & \hfill 1\hfill \end{array}\right)$, and Y = (Y_j1, Y_j2)’, j = 1,..., n, random samples from a bivariate normal distribution with mean (0, 0.5)’ and variance-covariance matrix $\left(\begin{array}{cc}\hfill 4\hfill & \hfill 7.2\hfill \\ \hfill 7.2\hfill & \hfill 16\hfill \end{array}\right)$. Clearly the null hypothesis holds with these two distributions, i.e., for any i and j, Pr(X_i1 < Y_j1)-Pr(X_i1 > Y_j1) = Pr(X_i2 < Y_j2) - Pr(X_i2 > Y_j2) = 0. We generate 10,000 replicates for each pair of m and n selected from {50, 100, 200}. For each replicate the optimal weights are estimated from the simulated data using the method described in Section 3. The simulated Type I error is the proportion of the null hypothesis H₀ being rejected at nominal significance level of 0.05 (two-sided).

The simulated power is obtained similarly under the same settings except that the mean vector of the X-samples is set to (0,-0.5)’, and the variance-covariance matrices are set to $\left(\begin{array}{cc}\hfill 1\hfill & \hfill 1.13\hfill \\ \hfill 1.13\hfill & \hfill 2\hfill \end{array}\right)$ for the X-sample and $\left(\begin{array}{cc}\hfill 2\hfill & \hfill 2.55\hfill \\ \hfill 2.55\hfill & \hfill 4\hfill \end{array}\right)$ for the Y-sample.

Table 1 presents the simulation results for the type I error and power. The results indicate that both the methods in the present paper and in Huang et al. (1984) effectively maintain the nominal type I error, with a minor discrepancy possibly due to variation in the simulation. In comparison, O’Brien’s (1984) tests produce inconsistent type I error rates, mostly inflated over the nominal significance level of 0.05. For example, with m = 200, n = 100, the empirical type I error rates of O’Brien’s tests are 0.101 and 0.059, while the tests in Huang et al. (2005) are 0.051 and 0.050, and the proposed two tests give 0.054 and 0.053, respectively. The power of the proposed tests is substantially higher than those of O’Brien (1984) and Huang et al. (2005). For example, with m = 100, n = 50, the power values are, respectively, 0.394 and 0.351 for O’Brien’ (1984) tests, 0.360 and 0.355 for the tests of Huang et al. (2005), and 0.567 and 0.563, for the proposed tests, more than 15% higher than other tests. It is worth noting that even when O’Brien’s (1984) tests produce smaller type I error (m = 50, n = 100 and m = 100, n = 200), the proposed tests still achieve a considerably higher power than other tests.

Table 1.

Type I error and power results under significance level, 0.05. (10,000 replicates)

m	n	T1	T2	Th1	Th2	Tw1	Tw2
Type I error rate
50	50	0.064	0.063	0.052	0.051	0.056	0.055
100	100	0.066	0.066	0.052	0.051	0.052	0.052
200	200	0.061	0.061	0.048	0.048	0.049	0.049
50	100	0.031	0.069	0.051	0.051	0.053	0.053
100	200	0.029	0.067	0.049	0.050	0.050	0.051
100	50	0.105	0.061	0.056	0.054	0.060	0.059
200	100	0.101	0.059	0.051	0.050	0.054	0.053
Power
50	50	0.311	0.311	0.316	0.315	0.512	0.511
100	100	0.534	0.533	0.535	0.535	0.734	0.734
200	200	0.825	0.825	0.825	0.825	0.933	0.933
50	100	0.375	0.431	0.436	0.433	0.637	0.635
100	200	0.664	0.716	0.717	0.716	0.866	0.865
100	50	0.394	0.351	0.360	0.355	0.567	0.563
200	100	0.641	0.591	0.596	0.593	0.790	0.788

4.2 An example

The role of certain growth hormones is one major objective of The Growth and Maturation in Children with Autism or Autistic Spectrum Disorder (ASD) Study (the Autism/ASD Study), a case-control study conducted by the Eunice Kennedy Shriver National Institute of Child Health and Human Development in 2002-2005; see Mills et al. (2007) for details of subject enrollment and data collection. The study enrolled eighty-one subjects, 75 boys and 6 girls, diagnosed as having autism/ASD, and eighty age-matched controls (59 boys and 21 girls). Blood samples were assayed for insulin-like growth factors (IGF-1, IGF-2), insulin-like growth factor binding protein (IGFBP-3), and growth hormone binding protein (GHBP), as well as for dehydroepiandrosterone (DHEA) and DHEA-sulphate (DHEAS).

To illustrate the proposed methods in comparison with other approaches, we exclude from the analysis data from the girls, due to their small sample size, and four boys in the case group who did not provide blood samples, thus yielding 71 cases and 59 controls in the analysis. We confine our attention to five hormones: insulin-like growth factor-1 (IGF-1), insulin-like growth factor 2 (IGF-2), IGF binding protein (IGFBP-3), growth hormone binding protein (GHBP), and dehydroepiandrosterone (DHEA). DHEA-sulphate (DHEAS) was not included in the analysis since its levels were undetectable in more than half of the subjects (Mills et al., 2007). To be investigated is whether the levels of a growth-related hormone, if any, differ between cases and controls.

We applied the proposed test and the tests of O’Brien (1984) and Huang et al. (2005) to the five growth-related hormone levels in cases and controls. The P-values were 2.86 × 10^-6 and 1.75 × 10^-6 for O’Brien’s tests and 6.28 × 10^-7 and 6.30 × 10^-7 for the two tests of Huang et al. In contrast the proposed two tests give P-values 1.93 × 10^-7 and 3.33 × 10^-7, respectively. For this example, the proposed method is shown to be more powerful than O’Brien’s method, but only slightly better than the tests of Huan et al. (2005). This is partly because the differences between cases and controls all fall into the same direction, that is, for each hormone, its level among cases are higher than among controls.

5. Discussion

For testing the Behrens-Fisher hypothesis, we proposed a weighted rank-sum test statistic that effectively maintains the type I error rate and possesses higher power than the tests of O’Brien (1984) and Huang et al. (2005). The optimal weights do not have closed forms and need to be estimated using available data.

All the tests discussed are nonparametric in nature and are “global” in the sense that they summarize the multi-dimensional data into one-dimensional statistics. Under the most restricted null hypothesis that the two multivariate distributions are identical these tests are asymptotically equivalent. However, under the less restrictive Behrens-Fisher hypotheses, they perform differently. It would be interesting to see how these test statistics behave under other hypotheses.

The proposed test gains its power by accumulating evidence across comparisons on each individual outcomes, but may still have low power in situations when the differences in the outcomes between the two samples, as measured by the θ_as, exist but fall into different directions (some differences are positive and some are negative or zero). In this regard, more robust tests, such as the one described in Yu et al. (2006), could serve as plausible alternatives.