Skip to main content
. Author manuscript; available in PMC: 2020 Nov 30.
Published in final edited form as: Mol Pharm. 2019 Feb 26;16(4):1620–1632. doi: 10.1021/acs.molpharmaceut.8b01297

Table 3.

test set validation statistical significance using different scores. A. Rank normalized score (pairwise; Wilcoxon matched-pairs signed rank test), B. Rank normalized score (Unpaired Comparison; Mann-Whitney U test), C. ΔRNS (Unpaired Comparison; Mann-Whitney U test). Assay Central (AC); AdaBoost (ada); Bernoulli Naïve-Bayes (bnb); k-nearest neighbors (knn); Random Forest (rf); support vector classification (svc); Deep Neural Networks (DL).

A.
ada bnb DL knn rf svc Consensus AC
ada N/A 0.0006*** 0.1011 0.5457 0.1688 0.0894 0.0340* 0.0164*
bnb 0.0006*** N/A 0.0004*** 0.0491* 0.0035** 0.0008*** <0.0001**** 0.9888
DL 0.1011 0.0004*** N/A 0.0291* 0.0425* 0.5088 0.9218 0.0002***
knn 0.5457 0.0491* 0.0291* N/A 0.2768 0.0526 0.0457* 0.0526
rf 0.1688 0.0425* 0.0035** 0.2768 N/A 0.0039** 0.0011** 0.2076
svc 0.0894 0.0008*** 0.5088 0.0526 0.0039** N/A 0.5457 0.0008***
Consensus 0.0340* <0.0001**** 0.9218 0.0457* 0.0011** 0.5457 N/A <0.0001****
AC 0.0164* 0.9888 0.0002*** 0.0526 0.2076 0.0008*** <0.0001**** N/A
B.
ada bnb DL knn rf svc Consensus AC
ada N/A 0.2912 0.4071 0.6165 0.4803 0.3956 0.4803 0.2382
bnb 0.2912 N/A 0.0355* 0.3842 0.4932 0.0287* 0.0483* 0.8944
DL 0.4071 0.0355* N/A 0.1153 0.1106 >0.9999 0.8143 0.0587
knn 0.6165 0.3842 0.1153 N/A 0.9430 0.1106 0.2221 0.6753
rf 0.4803 0.4932 0.1106 0.9430 N/A 0.0890 0.1305 0.6456
svc 0.3956 0.0287* >0.9999 0.1106 0.0890 N/A 0.9593 0.0459*
Consensus 0.4803 0.0483* 0.8143 0.2221 0.1305 0.9593 N/A 0.0533
AC 0.2382 0.8944 0.0587 0.6753 0.6456 0.0459* 0.0533 N/A
C.
ada bnb DL knn rf svc Consensus AC
ada N/A 0.0025** 0.1448 0.6444 0.1151 0.0756 0.1524 0.0205*
bnb 0.0025** N/A <0.0001**** 0.0217* 0.2818 <0.0001**** <0.0001**** 0.6456
DL 0.1448 <0.0001**** N/A 0.0349* 0.0023** 0.4499 0.6159 <0.0001****
knn 0.6444 0.0217* 0.0349* N/A 0.2008 0.0263* 0.0534 0.0820
rf 0.1151 0.2818 0.0023** 0.2008 N/A 0.0012** 0.0022** 0.5364
svc 0.0756 <0.0001**** 0.4499 0.0263* 0.0012** N/A 0.3187 <0.0001****
Consensus 0.1524 <0.0001**** 0.6159 0.0534 0.0022** 0.3187 N/A 0.0001***
AC 0.0205* 0.6456 <0.0001**** 0.0820 0.5364 <0.0001**** 0.0001*** N/A