Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
Precision, recall and F-1 for aggregated AMT spans evaluated against the union of expert span labels, for all three P, I, and O elements.
| Precision | Recall | F-l | |
|---|---|---|---|
| Majority Vote | 0.903 | 0.507 | 0.604 |
| Dawid Skene | 0.840 | 0.641 | 0.686 |
| HMMCrowd | 0.719 | 0.761 | 0.698 |
| Interventions | Precision | Recall | F-l |
| Majority Vote | 0.843 | 0.432 | 0.519 |
| Dawid Skene | 0.755 | 0.623 | 0.650 |
| HMMCrowd | 0.644 | 0.800 | 0.683 |
| Outcomes | Precision | Recall | F-l |
| Majority Vote | 0.711 | 0.577 | 0.623 |
| Dawid Skene | 0.652 | 0.648 | 0.629 |
| HMMCrowd | 0.498 | 0.807 | 0.593 |
