Table 2.
Cross-validation results on the PDBbind dataset.
| Model | PDBbind KIKD |
PDBbind KIKD (Unseen Scaffolds) |
PDBbind IC50 |
PDBbind IC50 (Unseen Scaffolds) |
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RMSE | MAE | PCORR | CI | RMSE | MAE | PCORR | CI | RMSE | MAE | PCORR | CI | RMSE | MAE | PCORR | CI | |
| ArkDTA (ours) | 1.2941 (0.0673) | 0.9618 (0.0375) | 0.7671 (0.0194) | 0.7898 (0.0073) | 1.3452 (0.0734) | 1.0231 (0.0440) | 0.7362 (0.0227) | 0.7717 (0.0108) | 1.2073 (0.0522) | 0.8536 (0.0288) | 0.7752 (0.0169) | 0.7366 (0.0040) | 1.1689 (0.0586) | 0.8368 (0.0340) | 0.7406 (0.0249) | 0.7122 (0.0119) |
| ArkDTA (Remove ) | 1.3005 (0.0650) | 0.9738 (0.0459) | 0.7656 (0.0210) | 0.7887 (0.0089) | 1.3561 (0.0654) | 1.0413 (0.0468) | 0.7335 (0.0169) | 0.7700 (0.0074) | 1.1931 (0.0360) | 0.8667 (0.0330) | 0.7791 (0.0118) | 0.7368 (0.0104) | 1.1899 (0.0564) | 0.8800 (0.0463) | 0.7306 (0.0273) | 0.7055 (0.0226) |
| ArkDTA (Freeze ESM) | 1.3457 (0.0396) | 1.0170 (0.0222) | 0.7512 (0.0089) | 0.7806 (0.0024) | 1.4059 (0.0645) | 1.0905 (0.0446) | 0.7147 (0.0215) | 0.7601 (0.0086) | 1.2561 (0.0415) | 0.9150 (0.0297) | 0.7540 (0.0165) | 0.7251 (0.0065) | 1.2401 (0.0326) | 0.9114 (0.0257) | 0.7066 (0.0146) | 0.6965 (0.0092) |
| DeepDTA | 1.3589 (0.0534) | 0.9761 (0.0405) | 0.7657 (0.0145) | 0.7879 (0.0077) | 1.3651 (0.0338) | 1.0354 (0.0517) | 0.7278 (0.0225) | 0.7687 (0.0119) | 1.2131 (0.0468) | 0.8794 (0.0291) | 0.7687 (0.0259) | 0.7349 (0.0124) | 1.1963 (0.0655) | 0.8721 (0.0439) | 0.7195 (0.0453) | 0.7071 (0.0160) |
| GraphDTA | 1.5511 (0.0401) | 1.1624 (0.0371) | 0.6641 (0.0123) | 0.7396 (0.0057) | 1.5934 (0.0579) | 1.2085 (0.0305) | 0.6268 (0.0271) | 0.7234 (0.0104) | 1.5515 (0.1243) | 1.1449 (0.1156) | 0.6557 (0.0432) | 0.6967 (0.0177) | 1.5438 (0.1372) | 1.1442 (0.1026) | 0.5969 (0.0577) | 0.6729 (0.0208) |
| TransformerCPI | 1.4982 (0.0375) | 1.1478 (0.0256) | 0.6716 (0.0171) | 0.7455 (0.0079) | 1.5008 (0.0551) | 1.1606 (0.0356) | 0.6519 (0.0236) | 0.7348 (0.0090) | 1.3465 (0.0332) | 0.9646 (0.0373) | 0.7190 (0.0120) | 0.7144 (0.0083) | 1.2905 (0.0761) | 0.9332 (0.0538) | 0.6816 (0.0399) | 0.6931 (0.0164) |
| MONN | 1.3283 (0.0440) | 0.9927 (0.0193) | 0.7563 (0.0137) | 0.7849 (0.0061) | 1.3534 (0.0822) | 1.0288 (0.0482) | 0.7343 (0.0264) | 0.7736 (0.0105) | 1.3994 (0.0417) | 1.0450 (0.0290) | 0.6725 (0.0286) | 0.6936 (0.0131) | 1.3242 (0.0388) | 0.9749 (0.0171) | 0.6414 (0.0270) | 0.6732 (0.0113) |
| HyperAttentionDTI | 1.4028 (0.0186) | 1.0888 (0.0192) | 0.7257 (0.0149) | 0.7683 (0.0061) | 1.4480 (0.0418) | 1.1399 (0.0351) | 0.6899 (0.0153) | 0.7510 (0.0067) | 1.2994 (0.0293) | 0.9816 (0.0265) | 0.7371 (0.0142) | 0.7233 (0.0047) | 1.2652 (0.0406) | 0.9594 (0.0332) | 0.6879 (0.0295) | 0.6962 (0.0049) |
| AttentionDTA | 1.2711 (0.0454) | 0.9410 (0.0297) | 0.7726 (0.0125) | 0.7908 (0.0063) | 1.3213 (0.0500) | 1.0074 (0.0341) | 0.7372 (0.0169) | 0.7716 (0.0063) | 1.2131 (0.0639) | 0.8420 (0.0425) | 0.7721 (0.0254) | 0.7446 (0.0136) | 1.2071 (0.0712) | 0.8456 (0.0437) | 0.7211 (0.0369) | 0.7151 (0.0171) |
| BACPI | 1.3594 (0.0362) | 1.0164 (0.0152) | 0.7427 (0.0139) | 0.7772 (0.0055) | 1.4343 (0.0660) | 1.0998 (0.0380) | 0.6974 (0.0320) | 0.7545 (0.0116) | 1.3583 (0.0577) | 1.0439 (0.0395) | 0.7140 (0.0285) | 0.7278 (0.0152) | 1.3798 (0.0652) | 1.0702 (0.0505) | 0.6507 (0.0385) | 0.7001 (0.0168) |
| IIFDTI | 1.3832 (0.0590) | 1.0513 (0.0594) | 0.7332 (0.0242) | 0.7701 (0.0123) | 1.4340 (0.0817) | 1.1169 (0.0662) | 0.6891 (0.0383) | 0.7474 (0.0166) | 1.3027 (0.0554) | 0.9326 (0.0436) | 0.7345 (0.0235) | 0.7116 (0.0173) | 1.2868 (0.0625) | 0.9261 (0.0446) | 0.6774 (0.0380) | 0.6827 (0.0179) |
Main cross-validation results for ArkDTA’s performance binding affinity prediction compared with its baselines and ablations. The results for both KIKD and IC50 subsets of the PDBbind dataset are based on mean and standard deviation of its five different test folds. Unseen Scaffolds refers to model’s performance evaluated on test instances where the ligands’ compound scaffolds do not overlap with those of the training instances in each fold. Best performance values are shown in bold.