Table 3.
Type I error of six global tests of p-values when p-values are independent or strongly correlated (The nominal Type I error rate is 0.05 and the severe inflation of Type I error with simulated error rate > 0.1 is written in bold italic)
| Independent Uniform (0,1) | ||||||
|---|---|---|---|---|---|---|
|
n |
KS |
Inverse chi |
Inverse norm |
Tippett |
Wilcoxon |
Logit |
| 20 |
0.052 |
0.053 |
0.049 |
0.051 |
0.052 |
0.050 |
| 50 |
0.051 |
0.052 |
0.051 |
0.048 |
0.051 |
0.050 |
| 100 |
0.046 |
0.049 |
0.050 |
0.049 |
0.049 |
0.051 |
| 200 |
0.047 |
0.048 |
0.047 |
0.052 |
0.049 |
0.048 |
| 300 |
0.051 |
0.054 |
0.053 |
0.051 |
0.053 |
0.053 |
| 400 |
0.042 |
0.046 |
0.046 |
0.048 |
0.047 |
0.046 |
| 500 |
0.051 |
0.050 |
0.049 |
0.051 |
0.050 |
0.050 |
|
Correlated Uniform (0,1) | ||||||
|
n |
KS |
Inverse chi |
Inverse norm |
Tippett |
Wilcoxon |
Logit |
| 20 |
0.061 |
0.059 |
0.063 |
0.050 |
0.065 |
0.062 |
| 50 |
0.058 |
0.060 |
0.063 |
0.049 |
0.061 |
0.062 |
| 100 |
0.060 |
0.066 |
0.068 |
0.049 |
0.066 |
0.067 |
| 200 |
0.063 |
0.069 |
0.073 |
0.050 |
0.072 |
0.072 |
| 300 |
0.069 |
0.071 |
0.074 |
0.052 |
0.073 |
0.074 |
| 400 |
0.064 |
0.066 |
0.073 |
0.048 |
0.071 |
0.072 |
| 500 |
0.064 |
0.070 |
0.071 |
0.049 |
0.068 |
0.069 |
|
Correlated Uniform (0,1) ± 0.1Beta (5,1), | ||||||
|
n |
KS |
Inverse chi |
Inverse norm |
Tippett |
Wilcoxon |
Logit |
| 20 |
0.061 |
0.047 |
0.058 |
0.035 |
0.061 |
0.057 |
| 50 |
0.081 |
0.037 |
0.054 |
0.046 |
0.06 |
0.053 |
| 100 |
0.080 |
0.039 |
0.061 |
0.045 |
0.056 |
0.057 |
| 200 |
0.156
|
0.031 |
0.062 |
0.039 |
0.059 |
0.059 |
| 300 |
0.180
|
0.017 |
0.04 |
0.039 |
0.049 |
0.042 |
| 400 |
0.270
|
0.025 |
0.055 |
0.052 |
0.053 |
0.056 |
| 500 |
0.278
|
0.017 |
0.052 |
0.046 |
0.060 |
0.052 |
|
Correlated 0.5 Uniform (0,1) ± 0.5Beta (5,1), | ||||||
|
n |
KS |
Inverse chi |
Inverse norm |
Tippett |
Wilcoxon |
Logit |
| 20 |
0.239
|
0.007 |
0.009 |
0.035 |
0.013 |
0.009 |
| 50 |
0.652
|
0.002 |
0.022 |
0.025 |
0.02 |
0.022 |
| 100 |
0.911
|
0.001 |
0.018 |
0.023 |
0.018 |
0.019 |
| 200 |
0.998
|
0 |
0.021 |
0.034 |
0.025 |
0.021 |
| 300 |
1
|
0 |
0.029 |
0.033 |
0.039 |
0.027 |
| 400 |
1
|
0 |
0.027 |
0.024 |
0.034 |
0.023 |
| 500 |
1
|
0 |
0.031 |
0.024 |
0.036 |
0.025 |
|
Correlated 0.9 Uniform (0,1) ± 0.1Beta (6,3), | ||||||
|
n |
KS |
Inverse chi |
Inverse norm |
Tippett |
Wilcoxon |
Logit |
| 20 |
0.072 |
0.041 |
0.05 |
0.038 |
0.052 |
0.05 |
| 50 |
0.107
|
0.039 |
0.065 |
0.051 |
0.068 |
0.062 |
| 100 |
0.148
|
0.048 |
0.086 |
0.04 |
0.089 |
0.086 |
| 200 |
0.227
|
0.038 |
0.088 |
0.048 |
0.101 |
0.086 |
| 300 |
0.296
|
0.024 |
0.097 |
0.042 |
0.119
|
0.095 |
| 400 |
0.353
|
0.04 |
0.108
|
0.053 |
0.131
|
0.105
|
| 500 |
0.439
|
0.041 |
0.12
|
0.033 |
0.148
|
0.111
|
|
Correlated 0.5 Uniform (0,1) ± 0.5Beta (6,3), | ||||||
|
n |
KS |
Inverse chi |
Inverse norm |
Tippett |
Wilcoxon |
Logit |
| 20 |
0.421
|
0.01 |
0.038 |
0.025 |
0.063 |
0.029 |
| 50 |
0.837
|
0.008 |
0.089 |
0.019 |
0.147
|
0.071 |
| 100 |
0.989
|
0.008 |
0.16
|
0.024 |
0.253
|
0.122
|
| 200 |
1
|
0.001 |
0.268
|
0.023 |
0.411
|
0.207
|
| 300 |
1
|
0 |
0.392
|
0.022 |
0.581
|
0.32
|
| 400 |
1
|
0.004 |
0.492
|
0.031 |
0.679
|
0.422
|
| 500 | 1 | 0.003 | 0.551 | 0.024 | 0.761 | 0.486 |