Table 7.
Global leaveout analysis for additional Category 2 entries—including only challenges where the correct answer was not in any training set
| Allen | Brouard | Dührkop | Ruttkies | Vaniya | Verdegem | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CFM_orig | CFM_re-train | CSI_IOKR_A | CSI_IOKR_AR* | CSI_IOKR_U | CSI:FID_orig | CSI:FID_leaveout* | MetFrag* | MetFrag+CFM* | MS−FINDER | MAGMa+ | MAGMa* | |
| Top 1 Neg. | 6 | 6 | 6 | 6 | 4 | 0 | 0 | 4 | 10 | 7 | 4 | 3 |
| Top 1 Pos. | 4 | 9 | 9 | 10 | 7 | 9 | 13 | 1 | 3 | 3 | 2 | 2 |
| Top 1 | 10 | 15 | 15 | 16 | 11 | 9 | 13 | 5 | 13 | 10 | 6 | 5 |
| Top 3 | 23 | 24 | 29 | 26 | 27 | 17 | 23 | 16 | 27 | 25 | 16 | 14 |
| Top 10 | 46 | 40 | 45 | 46 | 40 | 25 | 32 | 39 | 47 | 38 | 35 | 35 |
| Mean rank | 52.57 | 64.05 | 106.5 | 97.84 | 99.92 | 52.81 | 41.48 | 68.38 | 37.16 | 28.7 | 76.75 | 100.4 |
| Med. rank | 10 | 12.5 | 8 | 10 | 12 | 7 | 3 | 14.5 | 8 | 7.5 | 23.5 | 20.5 |
| Mean RRP | 0.863 | 0.872 | 0.849 | 0.856 | 0.837 | 0.891 | 0.91 | 0.863 | 0.878 | 0.738 | 0.832 | 0.811 |
| Med. RRP | 0.966 | 0.961 | 0.963 | 0.967 | 0.956 | 0.981 | 0.993 | 0.942 | 0.972 | 0.806 | 0.924 | 0.902 |
| Gold | 18 | 21 | 23 | 26 | 19 | 11 | 17 | 7 | 18 | 18 | 10 | 9 |
| F1 score | 628 | 654 | 735 | 691 | 632 | 403 | 557 | 484 | 707 | 594 | 462 | 434 |
| Medal Sc. | 79 | 94 | 105 | 98 | 91 | 59 | 87 | 50 | 95 | 85 | 46 | 46 |
n = 43 (negative) and n = 44 (positive)
The best value for selected statistics is marked in bold