Table 3. Summary of benchmark (RB106, RB144 and RB198) results in terms of AUC (mean ± Std).
Benchmark [r] | RNABindR 2.0 | PSSM | Sequence-based control | aaRNA |
---|---|---|---|---|
RB106 | 0.81 | 0.776 ± 0.001 * | 0.803 ± 0.001 * | 0.8251 ± 0.0009 |
RB144 | 0.81 | 0.782 ± 0.001 * | 0.801 ± 0.002 * | 0.830 ± 0.001 |
RB198 | 0.80 | 0.7696 ± 0.0007 * | 0.7974 ± 0.0007 * | 0.8343 ± 0.0004 |
Benchmark [p] | RNABindR 2.0 | PSSM | Sequence-based control | aaRNA |
RB106 | 0.74 | 0.721 ± 0.119 ** | 0.735 ± 0.109 ** | 0.765 ± 0.116 |
RB144 | 0.74 | 0.723 ± 0.118 ** | 0.733 ± 0.111 ** | 0.778 ± 0.105 |
RB198 | 0.73 | 0.716 ± 0.114 ** | 0.738 ± 0.106 ** | 0.784 ± 0.103 |
The corresponding ROC plots and AUC distribution patterns are given in Figure 4 (residue-based evaluation) and Supplementary Figure S13 (protein-based evaluation), respectively. RNABindR 2.0 is the best-performing sequence-based method from various approaches evaluated in the work (16). Its reported performance is listed. Sequence-based control method represents three sequence features of the aaRNA, which are adapted from the work SRCPred (12). In Benchmark [r], AUCs were measured on a protein-residue basis, and reported AUCs are the average results of five repetitions of five-fold cross-validation. The average AUC of the aaRNA method is significantly greater than that of the PSSM or sequence-based control method using a t-test. In Benchmark [p], AUCs were individually calculated for each protein chain, and a paired Wilcoxon test was applied to check whether the distribution of the aaRNA AUC is shifted to the right relative to that of the PSSM and sequence-based control. The significance of differences between the alternative methods and aaRNA is indicated by * for P-values < 10-5 and ** for P-values < 10-10.