Table 2. Performance comparison of deep learning and surrogate in vitro assays. The first and second columns list the ChEMBL-IDs of the assay to be predicted and the surrogate in vitro assay, that serves as the predictor, respectively. The third column shows the biomolecular target which the assays intended to measure. The fourth and fifth columns report respectively the accuracies of the surrogate in vitro assay and of the computational method together with their 90% confidence intervals. The last column gives the p-values of proportion tests for difference in accuracy. Significant results are indicated in bold.
| Assay | Surrogate assay | Target | Surrogate assay accuracy | Deep learning accuracy | p-value |
| CHEMBL1909134 | CHEMBL1613777 | CYP450-2C19 | 0.54 [0.4136, 0.653] | 0.95 [0.9198, 0.9658] | 1.95 × 10–17 |
| CHEMBL1909200 | CHEMBL1614521 | ERK | 0.56 [0.4012, 0.7005] | 0.98 [0.9615, 0.9912] | 9.57 × 10–17 |
| CHEMBL1909136 | CHEMBL1614110 | CYP450-2D6 | 0.51 [0.3923, 0.6197] | 0.91 [0.8734, 0.9319] | 2.76 × 10–15 |
| CHEMBL1909135 | CHEMBL1614027 | CYP450-2C9 | 0.68 [0.5567, 0.7853] | 0.95 [0.9213, 0.9664] | 1.35 × 10–9 |
| CHEMBL1909138 | CHEMBL1614108 | CYP450-3A4 | 0.86 [0.8041, 0.9071] | 0.95 [0.9278, 0.9713] | 1.76 × 10–4 |
| CHEMBL1614310 | CHEMBL1614544 | Lamin A | 0.65 [0.5797, 0.7092] | 0.74 [0.657, 0.8105] | 7.83 × 10–2 |
| CHEMBL1737863 | CHEMBL1614255 | PMI | 0.62 [0.3869, 0.8105] | 0.75 [0.6504, 0.8337] | 2.77 × 10–1 |
| CHEMBL1614016 | CHEMBL1794352 | Luciferase | 0.86 [0.8086, 0.9043] | 0.88 [0.8108, 0.9238] | 7.53 × 10–1 |
| CHEMBL1794393 | CHEMBL1614512 | MKP-3 | 0.67 [0.4469, 0.8357] | 0.70 [0.5945, 0.7853] | 8.07 × 10–1 |
| CHEMBL1614355 | CHEMBL1614052 | NPSR1 | 0.70 [0.5913, 0.7947] | 0.70 [0.5549, 0.8126] | 1.00 × 100 |
| CHEMBL1614479 | CHEMBL1614052 | NPSR1 | 0.94 [0.835, 0.9791] | 0.91 [0.7858, 0.9647] | 7.31 × 10–1 |
| CHEMBL1614197 | CHEMBL1614087 | ROR | 0.74 [0.6572, 0.8058] | 0.71 [0.6045, 0.7994] | 6.54 × 10–1 |
| CHEMBL1614539 | CHEMBL1614052 | NPSR1 | 0.88 [0.724, 0.9531] | 0.79 [0.6393, 0.888] | 3.95 × 10–1 |
| CHEMBL1738575 | CHEMBL1614247 | NOD2 | 0.87 [0.6533, 0.9657] | 0.75 [0.6235, 0.8462] | 3.78 × 10–1 |
| CHEMBL1737868 | CHEMBL1738317 | ATAD5 | 1.00 [0.6555, 1] | 0.81 [0.7098, 0.8893] | 2.05 × 10–1 |
| CHEMBL1613806 | CHEMBL1613949 | PTPN7 | 0.92 [0.5975, 0.9956] | 0.69 [0.5598, 0.8046] | 1.61 × 10–1 |
| CHEMBL1614105 | CHEMBL1614290 | SUA1 | 0.96 [0.9267, 0.9814] | 0.92 [0.8715, 0.9575] | 1.20 × 10–1 |
| CHEMBL1963940 | CHEMBL1794352 | Luciferase | 1.00 [0.8076, 1] | 0.87 [0.775, 0.9344] | 1.15 × 10–1 |
| CHEMBL1741321 | CHEMBL1614110 | CYP450-2D6 | 0.99 [0.9889, 0.9956] | 0.83 [0.8184, 0.8352] | 5.80 × 10–164 |
| CHEMBL1741325 | CHEMBL1614027 | CYP450-2C9 | 0.99 [0.9839, 0.993] | 0.75 [0.7428, 0.762] | 1.82 × 10–198 |
| CHEMBL1741323 | CHEMBL1613777 | CYP450-2C19 | 0.99 [0.9822, 0.9911] | 0.77 [0.7602, 0.7789] | 1.20 × 10–204 |
| CHEMBL1741324 | CHEMBL1613886 | CYP450-3A4 | 1.00 [0.9925, 0.9971] | 0.74 [0.7328, 0.7522] | <1.0 × 10–250 |