Table 1. Comparison of 3 Meta-BASIC versions and 12 components in LiveBench-8.
| EASY | Score | Hits | HARD | Score | Hits | ROC | Score |
|---|---|---|---|---|---|---|---|
| rdotb | 6002 | 67 | mBAS | 2306 | 34 | BasD | 92.0 |
| zdotb | 5971 | 68 | BasD | 2261 | 33 | rorfc | 92.0 |
| mBAS | 5964 | 68 | rorfc | 2233 | 32 | zmatc | 90.9 |
| BasP | 5958 | 67 | zdotb | 2225 | 32 | mBAS | 90.1 |
| BasD | 5951 | 68 | zdotc | 2213 | 32 | zorfc | 89.4 |
| rdotc | 5932 | 68 | BasP | 2157 | 31 | zdotc | 88.7 |
| zmatb | 5909 | 68 | zorfc | 2150 | 31 | zdotb | 87.9 |
| zdotc | 5908 | 68 | rorfb | 2144 | 31 | zmatb | 87.9 |
| zmatc | 5886 | 68 | rdotb | 2143 | 30 | rorfb | 87.7 |
| rorfc | 5880 | 68 | rdotc | 2125 | 31 | BasP | 87.6 |
| rorfb | 5875 | 66 | zmatc | 2063 | 30 | rdotc | 85.6 |
| zorfb | 5869 | 66 | zorfb | 2030 | 29 | zorfb | 83.5 |
| zorfc | 5861 | 68 | zmatb | 1996 | 30 | rmatc | 81.8 |
| rmatb | 5843 | 67 | rmatb | 1973 | 28 | rdotb | 79.5 |
| rmatc | 5822 | 68 | rmatc | 1902 | 27 | rmatb | 76.7 |
The targets are divided into EASY and HARD. The ‘Score’ columns for both categories show the total score obtained with the 3D-eval evaluation method. The ‘Hits’ columns indicate the number of correct hits produced by the methods. The specificity of the methods is evaluated in the ROC column. A higher ROC score indicates a higher reliability of the confidence score generated by each method. The details of the evaluation procedure are described on the LiveBench pages (http://bioinfo.pl/LiveBench/) and in related publications. Three tested Meta-BASIC versions are shown in boldface. There are three basic component methods dot, orf and mat. orf and dot use dot product calculation when comparing two vectors of the aligned profiles. mat uses vector times matrix times vector multiplication to compare the two vectors. Methods with names starting with the letter r return the raw score of the alignment while methods starting with the letter z return the Z-score transformation of the raw score. Methods with names ending with letter b use three PSI-Blast iterations to calculate the profile for each family while methods with names ending with the letter c use six PSI-Blast iterations. mBAS calculates the score by averaging the results obtained with all six methods which return Z-scores (zorfb, zorfc, zdotb, zdotc, zmatb, zmatc). The underlying raw scores cannot be easily compared with each other. BasP uses only zdotb and zmatb (three PSI-Blast iterations). BasD uses zdotc and zmatc (six PSI-Blast iterations) and it was selected as the current version of Meta-BASIC.