Table 1.

Overview of the main criteria for evaluating statistical methods in the four considered examples

Example	Evaluation criterion	Target value
A: testing and CI	Type 1 error	Close to and not greater than nominal value α
	Type 2 error	Low
	Coverage of (1–α) CI	Close to and not lower than nominal value 1–α
B: explaining	Mean coefficient values	Close to true values (low bias)
	Precision of coefficient estimation	High (low variance)
	Coverage of CI	Close to and not lower than nominal value 1–α
	Sensitivity of variable selection	High
	Specificity of variable selection	High
C: predicting	Prediction error on independent data	Low
	Accuracy measures	High
D: clustering	Agreement with true cluster structure	High
All settings	Stability	High
	Computational cost	Low
	Success of the computation (eg, ‘convergence’)	Yes

The last column indicates which values the considered evaluation criterion takes if the investigated method is good.