Table 1.
Accuracy metrics for agonist, antagonist, and binding activity cross-validation
| Activity | # chemicals | Model and parameters | Sensitivity | Specificity | Bal accuracy | Accuracy | ROC AUC | Score |
|---|---|---|---|---|---|---|---|---|
| Agonist | 1538 | Morgan kNN arithm k = 10 | 0.63 | 0.98 | 0.80 | 0.96 | 0.91 | 0.70 |
| Agonist | 1538 | Morgan kNN geom k = 2 | 0.40 | 0.99 | 0.70 | 0.96 | 0.73 | 0.49 |
| Agonist | 1538 | Morgan kNN exp k = 10 X = 1.5 | 0.69 | 0.97 | 0.83 | 0.96 | 0.92 | 0.73 |
| Agonist | 1538 | Morgan GkNN k = 10 X = 1 Y = 1 | 0.63 | 0.98 | 0.80 | 0.96 | 0.92 | 0.70 |
| Agonist | 1538 | Morgan GkNN k = 10 X = 1 Y = 3 | 0.66 | 0.97 | 0.82 | 0.96 | 0.92 | 0.72 |
| Agonist | 1538 | Morgan GkNN k = 10 X = 1.5 Y = 3 | 0.74 | 0.95 | 0.84 | 0.94 | 0.92 | 0.72 |
| Agonist | 1538 | Morgan GkNN k = 20 X = 1.5 Y = 5 | 0.75 | 0.95 | 0.85 | 0.94 | 0.91 | 0.73 |
| Antagonist | 1645 | Morgan kNN arithm k = 3 | 0.44 | 1.00 | 0.72 | 1.00 | 0.70 | 0.51 |
| Antagonist | 1645 | Morgan kNN geom k = 3 | 0.00 | 1.00 | 0.50 | 0.99 | 0.50 | 0.25 |
| Antagonist | 1645 | Morgan kNN exp k = 3 X = 1.5 | 0.44 | 1.00 | 0.72 | 1.00 | 0.70 | 0.51 |
| Antagonist | 1645 | Indigo kNN arithm k = 10 | 0.22 | 1.00 | 0.61 | 0.99 | 0.73 | 0.44 |
| Antagonist | 1645 | Indigo kNN geom k = 10 | 0.00 | 1.00 | 0.50 | 0.99 | 0.50 | 0.25 |
| Antagonist | 1645 | Indigo kNN exp k = 10 X = 1.5 | 0.44 | 1.00 | 0.72 | 0.99 | 0.73 | 0.53 |
| Antagonist | 1645 | Indigo GkNN k = 10 X = 3 Y = 7 | 0.56 | 0.98 | 0.77 | 0.98 | 0.73 | 0.55 |
| Antagonist | 1645 | Indigo GkNN k = 10 X = 5 Y = 15 | 0.56 | 0.98 | 0.77 | 0.98 | 0.73 | 0.55 |
| Binding | 1529 | Morgan kNN arithm k = 10 | 0.63 | 0.98 | 0.80 | 0.96 | 0.90 | 0.69 |
| Binding | 1529 | Morgan kNN geom k = 2 | 0.43 | 0.99 | 0.71 | 0.96 | 0.74 | 0.50 |
| Binding | 1529 | Morgan kNN exp k = 10 X = 1.5 | 0.69 | 0.97 | 0.83 | 0.95 | 0.90 | 0.71 |
| Binding | 1529 | Morgan GkNN k = 10 X = 1 Y = 1 | 0.63 | 0.98 | 0.80 | 0.96 | 0.90 | 0.69 |
| Binding | 1529 | Morgan GkNN k = 10 X = 1 Y = 3 | 0.66 | 0.97 | 0.82 | 0.95 | 0.90 | 0.70 |
| Binding | 1529 | Morgan GkNN k = 10 X = 1.5 Y = 3 | 0.73 | 0.94 | 0.84 | 0.93 | 0.90 | 0.70 |
| Binding | 1529 | Morgan GkNN k = 20 X = 1.5 Y = 5 | 0.75 | 0.95 | 0.85 | 0.94 | 0.89 | 0.71 |
“kNN arithm”, “kNN geom”, and “kNN exp” indicate the kNN models with the arithmetic, geometric, and exponential averaging, respectively. The cumulative score shown in the last column is the product of balanced accuracy, accuracy, and ROC AUC. Italic font indicates accuracy metric values that exceed those for the CERAPP consensus model