Fig. 1.

Evaluation of type I errors of $W_d^{\ast }$ and PERMANOVA permutation tests. Simulation under the null hypothesis results for comparison of $W_d^{\ast }$ (Wstar), PERMANOVA (Permanova), and distribution-based Welch ANOVA F (WelchF) tests are presented. In panel a, we evaluate the fraction of null hypotheses that have been rejected by each test at α=0.05. The subpanels of a correspond to simulated datasets with corresponding number of samples in the non-reference groups, with columns corresponding to the least dispersed and rows corresponding to the most dispersed sample. In panel b, the raw p values from $W_d^{\ast }$ test are plotted against those for the same data with Welch ANOVA F test. In panel c, we do the same for PERMANOVA p values and color the points by respective degree of heteroscedasticity in the simulated dataset