Table 1.
Four Concepts under Hypothesis Testing and Binary Classification
| Concept | Hypothesis Testing | Binary Classification | |
|---|---|---|---|
| Binary question | Is the null hypothesis false? (unanswerable) | Does the instance have a label 1? | |
| Binary answer | 0 (no) | The null hypothesis is true (unobservable) | The instance has a label 0 |
| 1 (yes) | The null hypothesis is false (unobservable) | The instance has a label 1 | |
| Decision rule | A statistical test that inputs data and outputs a p value, which is compared against a user-specified significance level α | A trained classifier that inputs an unlabeled instance's feature values and outputs a predicted label | |
| Binary decision | 0 | Do not reject the null hypothesis | Label the instance as 0 |
| 1 | Reject the null hypothesis | Label the instance as 1 |