Fig. 7. Top-performing model predictions compared against single-dose assays.
a Receiver operating characteristic (ROC) curves when ranking the 394 compound-kinase pairs in Round 2 using the pKd predictions either from the ensemble of the top-performing models (average predicted pKd from Q.E.D, DMIS_DK and AI Winter is Coming), or only from the Q.E.D model, compared against the experimental single-dose inhibition assays (the pairs with higher inhibition% are ranked first). The true positive activity class contains pairs with measured pKd > 6 (see Supplementary Fig. 15 for pKd > 7). The area under the ROC curve values are shown after the predictors (and the balanced accuracy is marked in the parentheses), and the diagonal dotted line shows the random predictor with an accuracy of AU-ROC = 0.50. b Precision-recall (PR) curves for the same activity classification analysis as shown in a. The area under the PR curve values are shown after the predictors and the horizontal dotted line indicates the random predictor with a precision of 0.64. Note: Round 2 Kd measurements were pre-selected to include mostly pairs with single-dose inhibition >80%, which makes Round 2 pairs optimal for systematic analysis of false positive predictions, and hence sensitivity (recall) and PPV (precision). However, these 394 pairs pre-selected for Kd profiling were less optimal for a comprehensive analysis of false negative predictions, and the evaluation of specificity. Source data are provided as a Source Data file54.