Table 2.
R-squared value for cost models and the area under the learning curve (ALC) for each method – Passive (PL) vs Active (AL), cost-agnostic vs cost-aware – in a 2-hour simulation study. Best results for each user are in boldface
| R2 |
ALC |
|||
|---|---|---|---|---|
| User A | User B | User A | User B | |
| PL: RANDOM | – | – | 0.621 | 0.620 |
| AL: Uncertainty (LC) | – | – | 0.595 | 0.590 |
| AL: CAUSE | – | – | 0.635 | 0.624 |
| Cost-CAUSE variants | ||||
| AL: NOWa (baseline) | 0.668 | 0.459 | – | – |
| AL: COUNTa | 0.791 | 0.530 | 0.650 | 0.650 |
| AL: COUNT_LEXICALa | 0.818 | 0.543 | 0.652 | 0.649 |
| AL: COUNT_LEXICAL_SYNTACTICa | 0.823 | 0.545 | 0.656 | 0.654 |
Abbreviations: AL, active learning; ALC, under the learning curve; LC, least confidence; PL, passive learning.
a
Results of 4 Cost-CAUSE variants were shown, which used cost models including different features to estimate annotation time. The text after colon shows the features in the cost model.