Table 5.
Comparison of our models’ performance on the five data sets with 90% labeled data with other models’ performance. Our four models are AE-MBO (autoencoder with MBO), BT-MBO (transformer with MBO), ECFP-MBO (extended-connectivity fingerprints with MBO), and Consensus-MBO, a model generating the consensus from our top two scoring methods for a given data set and percent of labeled data (more details in Section 2.6). Performance of our models is given as average ROC-AUC score over 10 labeled sets with standard deviation, and Consensus-MBO performance is given as the average over 10 trials with standard deviation.
| ROC-AUC Scores for 90% Labeled Data | |||||
|---|---|---|---|---|---|
| Model | Ames | Bace | BBBP | Beet | ClinTox |
| BT-MBO (Proposed) | 0.801 ± .013 | 0.839 ± .02 | 0.902 ± .034 | 0.831 ± .067 | 0.948 ± .048 |
| AE-MBO (Proposed) | 0.737 ± .017 | 0.818 ± .037 | 0.886 ± .029 | 0.785 ± .079 | 0.807 ± .085 |
| ECFP-MBO (Proposed) | 0.787 ± .019 | 0.855 ± .026 | 0.858 ± .041 | 0.842 ± .051 | 0.634 ± .060 |
| Consensus-MBO (Proposed) | 0.788 ± .015 | 0.833 ± .030 | 0.876 ± .030 | 0.750 ± .080 | 0.819 ± .015 |
| SSLP-FPs(C) [5] | 0.87 ± .022 | 0.872 ± .036 | 0.949 ± .016 | 0.797 ± .079 | 0.963 ± .044 |
| SSLP-FPs(CP) [5] | 0.861 ± .025 | 0.869 ± .029 | 0.953 ± .009 | 0.818 ± .075 | 0.939 ± .047 |
| SSLP-FPs(CPZ) [5] | 0.856 ± .023 | 0.868 ± .033 | 0.946 ± .015 | 0.827 ± .061 | 0.941 ± .035 |
| Auto-FPs [5, 59] | 0.887 ± .018 | 0.903 ± .022 | 0.969 ± .01 | 0.838 ± .059 | 0.95 ± .037 |
| ECFP2_512 [5, 48] | 0.867 ± .016 | 0.904 ± .024 | 0.913 ± .019 | 0.83 ± .06 | 0.831 ± .056 |
| ECFP2_1024 [5, 48] | 0.868 ± .016 | 0.905 ± .022 | 0.92 ± .015 | 0.839 ± .058 | 0.821 ± .058 |
| ECFP2_2048 [5, 48] | 0.869 ± .02 | 0.906 ± .024 | 0.919 ± .021 | 0.833 ± .063 | 0.833 ± .053 |
| ECFP4_512 [5, 48] | 0.865 ± .013 | 0.896 ± .023 | 0.908 ± .025 | 0.824 ± .061 | 0.801 ± .049 |
| ECFP4_1024 [5, 48] | 0.867 ± .016 | 0.901 ± .024 | 0.914 ± .024 | 0.833 ± .058 | 0.782 ± .052 |
| ECFP4_2048 [5, 48] | 0.872 ± .017 | 0.907 ± .023 | 0.916 ± .021 | 0.828 ± .066 | 0.784 ± .053 |
| ECFP6_512 [5, 48] | 0.861 ± .013 | 0.889 ± .026 | 0.9 ± .032 | 0.848 ± .062 | 0.77 ± .048 |
| ECFP6_1024 [5, 48] | 0.862 ± .013 | 0.893 ± .026 | 0.907 ± .029 | 0.833 ± .059 | 0.77 ± .054 |
| ECFP6_2048 [5, 48] | 0.864 ± .014 | 0.899 ± .024 | 0.911 ± .026 | 0.828 ± .063 | 0.75 ± .059 |