Table 5. Summary of the independent benchmark RB67 results in terms of MCC.
| Evaluation | Method | Accuracy | Specificity [+] (Precision) | Sensitivity (Recall) | F-measure | MCC | AUC |
|---|---|---|---|---|---|---|---|
| aaRNA | 0.882 | 0.437 | 0.494 | 0.464 | 0.399 | 0.842 | |
| BindN+ | 0.862 | 0.372 | 0.491 | 0.423 | 0.351 | 0.814 | |
| Residue- | RNAbindR 2.0 | 0.867 | 0.376 | 0.438 | 0.404 | 0.331 | 0.798 |
| based | Seq-CTRL | 0.886 | 0.443 | 0.401 | 0.421 | 0.358 | 0.811 |
| KYG | 0.804 | 0.274 | 0.542 | 0.364 | 0.284 | 0.780 | |
| DRNA | 0.842 | 0.298 | 0.392 | 0.339 | 0.254 | N/A | |
| OPRA | 0.843 | 0.301 | 0.403 | 0.345 | 0.261 | N/A | |
| aaRNA | 0.844 | 0.428 | 0.449 | 0.398 | 0.323 | 0.814 | |
| BindN+ | 0.828 | 0.377 | 0.463 | 0.397 | 0.301 | 0.780 * | |
| Protein- | RNAbindR 2.0 | 0.750 | 0.296 | 0.616 | 0.372 | 0.272 | 0.764 ** |
| based | Seq-CTRL | 0.797 | 0.355 | 0.488 | 0.372 | 0.286 | 0.787 *** |
| KYG | 0.769 | 0.298 | 0.505 | 0.349 | 0.240 | 0.716 **** | |
| DRNA | 0.795 | 0.319 | 0.397 | 0.331 | 0.229 | N/A | |
| OPRA | 0.797 | 0.242 | 0.259 | 0.203 | 0.116 | N/A |
Different predictors were compared in the same way as the RB44 benchmark. When tested on this up-to-date benchmark, the aaRNA got a superior performance than all others. Paired Wilcoxon tests on protein-averaged AUCs of aaRNA and other methods indicated significant differences (P* < 2e-4, P** < 2e-5, P*** < 7e-5 and P**** < 4e-9).