Nonparametric versus parametric tests of location in biomedical research

The choice of statistical test has a profound impact on the interpretation of data. Understanding this choice is important for the critical evaluation of the biomedical literature. The question often arises on whether to use nonparametric or parametric tests. The t-test is the most widely used statistical test for comparing the means of two independent groups. This parametric test assumes that the data are distributed Normally, that samples from different groups are independent and that the variances between the groups are equal. The most commonly used nonparametric test in this situation is the Wilcoxon Rank Sum Test (WRST) and the closely related Mann-Whitney U-test. The WRST assumes that observations from the different groups are random samples (i.e. independent and identically distributed) from their respective populations and are mutually independent and that the observations are ordinal or continuous measurements. When there are k groups (treatments), the nonparametric test is Kruskal-Wallis test (KW), a generalization of the WRST. KW is the nonparametric equivalent to ANOVA. Using nonparametric tests instead of parametric tests begs two questions: 1) what happens if the nonparametric test is used when the parametric assumptions are met and 2) what happens when the parametric assumptions are not met?

To answer these questions one must first discuss the underlying goal of the study. Usually in biomedical applications one is interested in measures of location such as the mean. One can test if the treatment (experimental condition) has an effect (location shift) on the population under study. For example, one may be interested in the effect of treatment(s) on a specific measurement, say cell count, compared to the control. Data of this nature are often analyzed with the t-test, or if there are k > 2 groups, ANOVA. In the parametric case one tests for differences in the means among the groups. In the nonparametric equivalents the location statistic is the median.

The assumptions for the nonparametric test are weaker than those for the parametric test, and it has been stated that when the assumptions are not met, it is better to use the nonparametric test. However, real data are rarely exactly Normal ^1–3. Does this mean that one should never use the t-test? In many datasets seen in the biomedical sciences, there often exist several observations that differ from the others, the so-called outliers. One must also then consider what is the best summary statistic for central tendency. That is, there should be some concept of robustness to assess the properties of the estimators themselves. Robustness, in one sense, refers to the insensitivity of the estimator to outliers or violations in underlying assumptions. One concept of robustness is the breakdown point ⁴. The breakdown point is defined as fraction of data that can be arbitrary (corrupted) without making the estimator arbitrarily bad. For example, the sample mean is defined as: x₁+x₂+…+x_n/n. If we let any one of the observations (say x_n) get arbitrarily large, the mean will become arbitrarily large. This means that even if an investigator has only one large outlier, the mean is arbitrary. Thus the breakdown point for the mean is 0. The median, which is commonly used when data are skewed or there exists outliers, is defined as the central value in a distribution where above and below lie an equal number of values. Intuitively one can see that if we let a minority of observations go to infinity, the median will not be arbitrarily bad. The breakdown point of the median is ½, this is the highest breakdown point. From the point of robustness and breakdown point, the mean is a good estimator only if the data have zero outliers (no “heavy” tails), no skewness (symmetry of Normal distribution is kept), and there is unimodality. The median is more insensitive to these departures from Normality. Nonparametric methods such as the WRST and KW use the median and are thus robust in this sense.

If there exists departures from normality, it seems prudent, in the sense of robustness, to use the nonparametric test. However, one must consider the cost, in terms of power, of applying the nonparametric test when indeed the data are distributed normally and satisfy the other assumptions of the parametric test. With this comes the notion of Asymptotic Relative Efficiency (ARE). The ARE, simply defined, is how many more subjects are needed for the nonparametric test to have equivalent power as the parametric test for a fixed Type I error rate α. If the ARE=1, then the two tests have equal power for the same number of subjects. AREs < 1 indicate that the parametric test is more powerful and AREs > 1 indicate that the nonparametric test has more power. The ARE of the WRST versus the t-test when the underlying assumptions of the t-test are satisfied is 0.955 ^5–7. Similarly, KW versus ANOVA has an ARE of 0.955. However these nonparametric tests are much more powerful than their parametric counterparts when the underlying distributions are heavy tailed or have extreme skewness ^{5, 6, 8–10}. In some cases the ARE became infinite. Thus, there is minimal power loss associated with the nonparametric tests even when the data are distributed normally, while the power gains of these tests when normality is violated are substantial.

As the sample sizes become infinite, the parametric tests are robust to departures from Normality. However, because of cost and potential risks to humans and animals, many of the sample sizes in the biomedical literature are far from infinite. Thus it is prudent to examine the properties of these estimators when the sample size is small (<25 per group). The small sample properties of the WRST versus the t-test have been studied extensively ^{1, 5–9}. The WRST has been shown to be as powerful in small samples as the t-test under the location shift alternatives and can be much more powerful than the t-test under certain non-normality conditions ^{5, 6}. Monte Carlo experiments found that for tests of location shift, the WRST was the best test in almost all cases ⁸. Further, in some small sample Monte Carlo simulations the WRST was more powerful than the t-test even when the two samples were independent, identically Normally distributed⁸. The WRST had large power advantages over the t-test in small sample sizes for distributions that possessed extreme asymmetry or where there existed a point mass at 0¹. Moreover, under Normality conditions with small samples, ANOVA performed only slightly better than KW. However when the distributions were mixtures of Normals, exponential, or double-exponential, KW was substantially more powerful ¹⁰.

Data are often non-Normal in the biomedical sciences ^{1–3, 11} and the sample sizes are often small. In data where there exists skewness, extreme asymmetries, multimodality, or heavy tails, nonparametric tests such as WRST and KW offer a very satisfactory alternative to parametric tests, especially in small samples. Taken together, these results suggest that when the data are distributed normally and all of the other assumptions are met, there is relatively little loss in terms of power to use WRST or KW and there can be almost infinite gains when these assumptions are not met. Because of this, one should consider using the nonparametric test of location for the primary analysis.

Biography

Christina Ramirez Kitchen, Ph.D. is an Associate Professor of Biostatistics at the UCLA School of Public Health. Dr. Ramirez Kitchen graduated from the California Institute of Technology. Her main research interests include nonparametrics, high-dimensional algorithms, Bayesian hierarchical models and phylogenetics.