Table 1.
Effectiveness of bm25 and the Lucene ranking algorithm on abstracts, full-text articles, and spans from full text.
| MAP | ||
| Ivory (bm25) | Ivory (Lucene) | |
| Abstract | 0.163 | 0.129 |
| Article | 0.146 (-11%)° | 0.235 (+82%)** |
| Span (max) | 0.240 (+47%)** | 0.206 (+60%)** |
| Span (sum) | 0.192 (+18%)* | 0.198 (+54%)** |
| P20 | ||
| Ivory (bm25) | Ivory (Lucene) | |
| Abstract | 0.322 | 0.293 |
| Article | 0.158 (-51%)** | 0.353 (+20%)* |
| Span (max) | 0.357 (+11%)° | 0.332 (+13%)° |
| Span (sum) | 0.314 (-3%)° | 0.317 (+8%)* |
| IP@R50 | ||
| Ivory (bm25) | Ivory (Lucene) | |
| Abstract | 0.110 | 0.090 |
| Article | 0.163 (+48%)° | 0.222 (+146%)** |
| Span (max) | 0.212 (+93%)** | 0.189 (+109%)** |
| Span (sum) | 0.149 (+36%)* | 0.159 (+77%)** |
For all metrics, relative improvements over baseline are shown; ** = statistically significant (p < 0.01); * = statistically significant (p < 0.05); ° = not significant.