Table 4.
(a) IR Results on epic-qa-dev for document recall: Passages vs. Documents. 5000 passages ranking vs. 5000 document rankings. | |||||||
---|---|---|---|---|---|---|---|
Index | reranking | Test EPIC-QA_docs |
|||||
R@500 | R@1K | R@2K | R@3K | R@4K | R@5K | ||
passages | yes | 0.6959 | 0.7979 | 0.8597 | 0.8693 | 0.8716 | 0.8716 |
documents | yes | 0.7041 | 0.7867 | 0.8342 | 0.8538 | 0.8655 | 0.8686 |
(b) IR Results on epic-qa-dev for passage recall: Passages vs. Documents. 50,000 passages ranking vs. 5000 document ranking. | |||||||||
---|---|---|---|---|---|---|---|---|---|
Index | Test EPIC-QA_passages |
||||||||
R@100 | R@500 | R@1K | R@2K | R@5K | R@1K0 | R@15K | R@30K | R@50K | |
pas | 0.2199 | 0.4645 | 0.582 | 0.6815 | 0.7577 | 0.8321 | 0.8676 | 0.9027 | 0.9082 |
docs | 0.0867 | 0.2234 | 0.3336 | 0.4345 | 0.6189 | 0.7442 | 0.7883 | 0.8422 | 0.8724 |
(c) IR Results on epic-qa-dev for nugget recall: Passages vs. Documents. 50,000 passages ranking vs. 5000 document ranking. | |||||||||
---|---|---|---|---|---|---|---|---|---|
Index | Test EPIC-QA_passages |
||||||||
N@100 | N@500 | N@1K | N@2K | N@5K | N@10K | N@15K | N@30K | N@50K | |
pas | 0.5988 | 0.7934 | 0.88 | 0.9134 | 0.941 | 0.9589 | 0.9626 | 0.9774 | 0.9774 |
docs | 0.3068 | 0.5156 | 0.6537 | 0.758 | 0.8661 | 0.921 | 0.942 | 0.9629 | 0.9644 |
(d) EPIC-QA evaluation results on epic-qa-dev: Passages vs. Documents. EPIC-QA metrics. 50,000 passages ranking vs. 5000 document ranking. | |||||||||
---|---|---|---|---|---|---|---|---|---|
NDNS@k | Test EPIC-QA - epicQA evaluation |
||||||||
pas |
doc2pas |
pas2sent |
|||||||
Partial | Relaxed | Exact | Partial | Relaxed | Exact | Partial | Relaxed | Exact | |
1,000 | 0.2463 | 0.2296 | 0.1823 | 0.1124 | 0.1031 | 0.0936 | 0.1786 | 0.1799 | 0.2072 |
5,000 | 0.2519 | 0.2347 | 0.1871 | 0.1299 | 0.1180 | 0.1084 | 0.2098 | 0.2113 | 0.2425 |
15,000 | 0.2541 | 0.2362 | 0.1886 | 0.1338 | 0.1213 | 0.1118 | 0.2250 | 0.2266 | 0.2601 |
30,000 | 0.2552 | 0.2371 | 0.1893 | 0.1357 | 0.1228 | 0.1133 | 0.2274 | 0.2290 | 0.2628 |