Fig. 6.

Arbitrariness resulting from dichotomization. Two hypothetical effects that independently follow Gaussian distributions: $N\left(m_1,{\sigma }_1^2\right)$ (blue) and $N\left(m_2,{\sigma }_2^2\right)$ (red), respectively, where m₁ = 0.2;σ₁ = 0.1;m₂ = 0.4;σ₂ = 0.3. Under NHST the first effect would be considered statistically significant with a one-sided (or two-sided) p-value of 0.023 (or 0.045); in contrast, the second effect would not be viewed statistically significant given a one-sided (or two-sided) p-value of 0.091 (or 0.18). On the other hand, the difference between the two effects is not statistically significant with a one-sided p-value of 0.26; in fact, the second effect is more likely larger than the first one with a probability of 0.74.