Table 1. Benchmarks against protein structural databases.
| Program | BAliBASE 3 | BAliBASE 3+fp | SABmark 1.65 |
| (Acc/Sn/PPV) | (Acc/Sn/PPV) | (Acc/Sn/PPV) | |
| AMAP | 0.70/0.62/0.83 | 0.73/0.61/0.80 | 0.57/0.43/0.46 |
| ClustalW | 0.66/0.63/0.62 | 0.59/0.63/0.53 | 0.38/0.44/0.30 |
| DIALIGN | 0.68/0.63/0.68 | 0.68/0.62/0.63 | 0.48/0.41/0.34 |
| FSA | 0.71/0.62/0.85 | 0.75/0.62/0.84 | 0.59/0.38/0.52 |
| FSA (–maxsn) | 0.73/0.68/0.76 | 0.74/0.68/0.72 | 0.52/0.45/0.39 |
| MAFFT | 0.74/0.71/0.71 | 0.68/0.71/0.61 | 0.44/0.49/0.35 |
| MUMMALS | 0.74/0.70/0.73 | 0.69/0.70/0.64 | 0.49/0.52/0.38 |
| MUSCLE | 0.70/0.67/0.66 | 0.63/0.66/0.57 | 0.40/0.46/0.32 |
| Probalign | 0.76/0.72/0.73 | 0.71/0.71/0.65 | 0.49/0.50/0.37 |
| ProbCons | 0.74/0.70/0.72 | 0.69/0.70/0.64 | 0.47/0.50/0.37 |
| T-Coffee | 0.72/0.67/0.71 | 0.67/0.67/0.63 | 0.45/0.46/0.35 |
| SeqAn::T-Coffee | 0.73/0.69/0.70 | 0.67/0.69/0.61 | 0.43/0.47/0.34 |
Comparisons of the accuracies (Acc), sensitivities (Sn) and positive predictive values (PPV) of FSA and other alignment methods on the BAliBASE 3 [24] and SABmark 1.65 [25] databases. Probalign has the highest accuracy on the commonly-used BAliBASE 3 dataset and FSA in default mode has superior accuracy on the BAliBASE 3+fp and SABmark 1.65 datasets (note that only FSA and AMAP explicitly attempt to maximize the expected accuracy). FSA has higher positive predictive values than any other program on all datasets and can additionally achieve high sensitivity when run in maximum-sensitivity mode. The BAliBASE 3+fp dataset, which mirrors BAliBASE 3 but includes a single non-homologous sequence in each alignment, was designed to test the robustness of alignment programs to incomplete homology. Traditional alignment programs, designed to maximize sensitivity, suffer greatly-increased mis-alignment when even a single non-homologous sequence is introduced; in contrast, FSA is robust to the non-homologous sequence and has an unchanged positive predictive value. Remarkably, FSA was the only tested program with a mis-alignment rate of <50% on the SABmark 1.65 dataset; the majority of the homology statements made by other programs were incorrect. Because the SABmark 1.65 dataset contains many sequences of only partial or even no homology, a method such as FSA which is robust to non-homologous sequence performs better under our accuracy criterion than a program such as MUMMALS despite the fact that MUMMALS has significantly-higher sensitivity on this dataset. The BAliBASE 3 dataset consisted of full-length sequences in all reference sets RV11, RV12, RV20, RV30, RV40 and RV50; we created the BAliBASE 3+fp dataset from the same reference sets by adding a single false-positive, a random sequence, to each alignment. The SABmark 1.65 dataset consisted of the Twilight Zone and Superfamilies datasets.