Table 1.
“pr” stands for probability.
| True null effect (H0) | True positive effect (H1) | ||
|---|---|---|---|
| Pre-experiment probability of H0 and H1 | Long run of experiments | pr(H0) | pr(H1) |
| The conditional probability of having this data or more extreme data given that H0 is true | Single experiment | p-value | — |
| The conditional probability of having a significant test result given that H0 or H1 are true | Long run of experiments | Alpha level (α) Type I error False Positive False Alarm | Power = 1 −β True Positive Hit |
| The conditional probability of not having a significant test result given that H0 or H1 are true | Long run of experiments | 1 – α = Confidence level True Negative Correct Rejection | β = 1 – Power Type II error False Negative Miss |
| Post-experiment probability of H0 and H1 given a significant test result | Long run of experiments | FRP pr(H0|significant result) | TRP pr(H1|significant result) |
NHST textbooks typically only present rows 3 and 4 of this table (Alpha level, Power, Confidence level and Type II error). We follow the NHST view and deal with long run probabilities only. Note that the p value does not fit this view as it does not have any long run interpretation besides that it is a random variable (Murdoch et al., 2008). The most important variables are bolded, familiar signal detection categories are also provided. NHST does not deal with the concepts in italics.