Table 1: Evaluation of models trained on human PPIs.
We show performance of D-SCRIPT, PIPR (currently the best-performing sequence-based deep learning PPI prediction method), and two hybrid approaches: D-HYBRID refers to D-SCRIPT predictions augmented with PIPR predictions, and P-HYBRID refers to PIPR predictions augmented with D-SCRIPT predictions (STAR Methods). H. sapiens results are average performance over 5-fold cross validation, using 38,345 positive (and 10 times as many negative) PPIs for training for each fold. All other species were evaluated using a model trained on human data, and evaluated on 5,000 positive and 50,000 negative PPIs (2,000/20,000 for E. coli due to limited data). D-SCRIPT outperforms PIPR cross-species, though PIPR performs better on in-sample species (i.e. human cross-validation). The best performing method for each species and metric is bolded. AUPR: area under precision-recall curve; AUROC: area under receiver operating characteristic curve. See also Table S1.
Species | Model | AUPR | Precision | Recall | AUROC |
---|---|---|---|---|---|
M. musculus | PIPR | 0.526 | 0.734 | 0.331 | 0.839 |
D-SCRIPT | 0.580 | 0.818 | 0.346 | 0.833 | |
D-HYBRID | 0.609 | 0.820 | 0.355 | 0.838 | |
D. melanogaster | PIPR | 0.278 | 0.521 | 0.121 | 0.728 |
D-SCRIPT | 0.552 | 0.798 | 0.359 | 0.824 | |
D-HYBRID | 0.562 | 0.798 | 0.361 | 0.824 | |
C. elegans | PIPR | 0.346 | 0.673 | 0.142 | 0.757 |
D-SCRIPT | 0.548 | 0.840 | 0.306 | 0.813 | |
D-HYBRID | 0.559 | 0.841 | 0.308 | 0.814 | |
S. cerevisiae | PIPR | 0.230 | 0.398 | 0.085 | 0.718 |
D-SCRIPT | 0.405 | 0.706 | 0.223 | 0.789 | |
D-HYBRID | 0.417 | 0.708 | 0.225 | 0.789 | |
E. coli | PIPR | 0.308 | 0.629 | 0.131 | 0.675 |
D-SCRIPT | 0.571 | 0.791 | 0.520 | 0.863 | |
D-HYBRID | 0.588 | 0.793 | 0.394 | 0.863 | |
H. sapiens (5-fold cross validation) |
PIPR | 0.835 | 0.838 | 0.701 | 0.960 |
D-SCRIPT | 0.516 | 0.728 | 0.278 | 0.833 | |
P-HYBRID | 0.844 | 0.949 | 0.400 | 0.962 |