Table 1.

Classification accuracies and statistical significance for the primary tests on DEAP and SEED. Columns denoted by ${}^1$ were compared together via a paired t-test, and columns denoted by ${}^2$ were compared to chance (50%) via one-sample t-tests. Bolded p-values indicate statistical significance. Under randomized labels, t.r. k-f, t.r. bl., and s.r. k-f are short for trial-randomized k-fold, trial-randomized block-wise, and sample-randomized k-fold CV, respectively.

			True Labels			Random Labels
Dataset	Classifier	Classification	k-Fold ${}^{\mathbf{1}}$	Block ${}^{\mathbf{1}}$	${}^{\mathbf{1}}$	t.r. k-f ${}^{\mathbf{2}}$		t.r. bl. ${}^{\mathbf{2}}$		s.r. k-f ${}^{\mathbf{2}}$
DEAP	SVM	Pos–Neu	70.00	58.91	p < 0.001	63.28	p < 0.001	47.91	p = 0.027	50.05	p = 0.830
		Neg–Neu	65.90	53.75	p < 0.001	63.44	p < 0.001	51.43	p = 0.400	49.82	p = 0.533
		Pos–Neg	72.29	63.13	p < 0.001	62.70	p < 0.001	50.82	p = 0.512	49.84	p = 0.478
	LDA	Pos–Neu	69.44	59.63	p < 0.001	62.49	p < 0.001	50.16	p = 0.922	50.19	p = 0.487
		Neg–Neu	65.37	53.42	p < 0.001	64.57	p < 0.001	50.39	p = 0.793	50.14	p = 0.553
		Pos–Neg	71.83	62.81	p < 0.001	63.86	p < 0.001	50.06	p = 0.964	50.16	p = 0.378
	KNN	Pos–Neu	70.83	58.89	p < 0.001	65.07	p < 0.001	50.26	p = 0.823	50.17	p = 0.391
		Neg–Neu	67.45	54.01	p < 0.001	64.30	p < 0.001	50.03	p = 0.972	49.84	p = 0.371
		Pos–Neg	72.02	62.31	p < 0.001	63.80	p < 0.001	49.05	p = 0.370	50.21	p = 0.212
SEED	SVM	Pos–Neu	95.70	81.57	p < 0.001	83.86	p < 0.001	48.03	p = 0.484	49.63	p = 0.317
		Neg–Neu	88.89	57.95	p < 0.001	86.75	p < 0.001	42.88	p = 0.028	49.63	p = 0.480
		Pos–Neg	96.54	85.22	p < 0.001	83.83	p < 0.001	45.97	p = 0.143	49.71	p = 0.351
	LDA	Pos–Neu	95.10	83.80	p < 0.001	84.21	p < 0.001	41.35	p = 0.018	49.88	p = 0.699
		Neg–Neu	88.42	60.48	p < 0.001	84.79	p < 0.001	46.25	p = 0.278	50.06	p = 0.840
		Pos–Neg	95.85	87.86	p = 0.001	83.44	p < 0.001	45.44	p = 0.185	49.75	p = 0.288
	KNN	Pos–Neu	96.44	78.51	p < 0.001	91.52	p < 0.001	46.70	p = 0.560	50.21	p = 0.358
		Neg–Neu	90.94	51.73	p < 0.001	92.14	p < 0.001	48.17	p = 0.541	50.11	p = 0.723
		Pos–Neg	97.04	80.24	p < 0.001	91.53	p < 0.001	44.08	p = 0.217	49.83	p = 0.386