Table 2. Nested, 5-Fold Cross-Validation Statistics of Classification Models of AChE Inhibition at 1 μM Threshold for Seven Species^a.

Human AChE Training Set: 1813 Active/4075 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	0.93	0.84	0.84	0.83	0.85	0.87	0.70	0.70
ADA	0.91	0.80	0.81	0.80	0.82	0.85	0.64	0.64
BNB	0.84	0.73	0.71	0.75	0.75	0.76	0.51	0.51
kNN	0.92	0.84	0.81	0.87	0.85	0.84	0.71	0.71
LREG	0.91	0.82	0.82	0.81	0.84	0.86	0.67	0.67
RF	0.94	0.84	0.86	0.81	0.86	0.90	0.71	0.72
SVC	0.94	0.86	0.86	0.85	0.87	0.89	0.74	0.74
XGB	0.93	0.85	0.85	0.85	0.87	0.88	0.73	0.73
AttentiveFPc	0.75	0.76	0.72	0.83	0.75	0.74	0.43	0.43

Eel AChE Training Set: 2084 Active/5459 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	0.90	0.79	0.76	0.82	0.83	0.84	0.65	0.66
ADA	0.91	0.78	0.81	0.76	0.84	0.89	0.66	0.66
BNB	0.85	0.71	0.74	0.69	0.79	0.85	0.55	0.55
kNN	0.93	0.84	0.82	0.85	0.87	0.89	0.73	0.73
LREG	0.91	0.80	0.81	0.80	0.85	0.88	0.68	0.68
RF	0.94	0.84	0.86	0.81	0.88	0.92	0.74	0.74
SVC	0.93	0.84	0.83	0.84	0.87	0.89	0.73	0.73
XGB	0.93	0.83	0.84	0.82	0.87	0.90	0.73	0.73
AttentiveFPc	0.86	0.82	0.83	0.82	0.87	0.76	0.53	0.53

Rat AChE Training Set: 687 Active/1406 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	0.95	0.87	0.85	0.90	0.87	0.85	0.74	0.75
ADA	0.92	0.84	0.86	0.82	0.85	0.87	0.69	0.69
BNB	0.87	0.80	0.82	0.78	0.81	0.84	0.62	0.62
kNN	0.93	0.86	0.83	0.89	0.86	0.83	0.72	0.72
LREG	0.94	0.86	0.86	0.86	0.86	0.87	0.72	0.72
RF	0.94	0.87	0.87	0.87	0.87	0.87	0.74	0.74
SVC	0.93	0.87	0.85	0.88	0.87	0.86	0.73	0.73
XGB	0.94	0.87	0.87	0.87	0.87	0.87	0.74	0.74

Mouse AChE Training Set: 145 Active/368 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	0.95	0.89	0.96	0.83	0.92	0.98	0.83	0.83
ADA	0.93	0.84	0.86	0.83	0.88	0.91	0.74	0.75
BNB	0.89	0.79	0.82	0.78	0.84	0.88	0.66	0.67
kNN	0.90	0.85	0.82	0.90	0.88	0.86	0.75	0.76
LREG	0.94	0.86	0.84	0.88	0.88	0.89	0.76	0.76
RF	0.95	0.86	0.87	0.86	0.89	0.91	0.77	0.77
SVC	0.93	0.87	0.84	0.91	0.89	0.88	0.78	0.78
XGB	0.93	0.87	0.87	0.87	0.89	0.91	0.78	0.78

Cow AChE Training Set: 239 Active/457 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	0.94	0.85	0.89	0.81	0.85	0.89	0.70	0.70
ADA	0.92	0.87	0.88	0.86	0.87	0.87	0.73	0.73
BNB	0.93	0.87	0.89	0.85	0.87	0.89	0.73	0.74
kNN	0.92	0.88	0.86	0.90	0.87	0.84	0.74	0.74
LREG	0.94	0.89	0.90	0.88	0.89	0.89	0.77	0.77
RF	0.94	0.88	0.91	0.85	0.88	0.90	0.76	0.76
SVC	0.94	0.89	0.90	0.88	0.89	0.89	0.77	0.77
XGB	0.93	0.88	0.89	0.88	0.88	0.88	0.75	0.76

Ray AChE Training Set: 156 Active/307 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	0.95	0.89	0.88	0.91	0.89	0.87	0.77	0.77
ADA	0.93	0.87	0.92	0.83	0.88	0.93	0.75	0.76
BNB	0.89	0.83	0.81	0.85	0.82	0.80	0.65	0.65
kNN	0.92	0.88	0.92	0.85	0.89	0.92	0.77	0.78
LREG	0.94	0.89	0.91	0.87	0.89	0.91	0.78	0.78
RF	0.94	0.87	0.94	0.82	0.88	0.95	0.77	0.78
SVC	0.95	0.89	0.92	0.86	0.89	0.93	0.79	0.79
XGB	0.94	0.90	0.91	0.88	0.90	0.91	0.79	0.79

Mosquito AChE Training Set: 27 Active/72 Total Compounds
model	AUC	F1 score	precision	recall	accuracy	specificity	Cohen’s kappa	MCC
DLb	1	0.67	1	0.50	0.80	1	0.55	0.61
ADA	0.83	0.73	0.79	0.71	0.81	0.87	0.59	0.60
BNB	0.85	0.56	0.61	0.55	0.71	0.78	0.33	0.35
kNN	0.88	0.80	0.85	0.82	0.85	0.87	0.69	0.71
LREG	0.86	0.77	0.86	0.75	0.82	0.89	0.65	0.67
RF	0.88	0.81	0.88	0.78	0.85	0.91	0.70	0.72
SVC	0.87	0.75	0.71	0.82	0.81	0.80	0.60	0.61
XGB	0.82	0.70	0.83	0.64	0.79	0.89	0.55	0.58

^aDL = deep learning; ADA = AdaBoosted decision trees; BNB = Bernoulli naïve Bayes; kNN = K-nearest neighbors; LREG = LogisticRegression; RF = random forest; SVC = support vector classification; XGB = XGBoost.

^bStatistics for a 20% leave-out set.

^cStatistics for 5-fold cross-validation.