Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
Inter-annotator agreement after annotation round 1, and round 2 for each annotation batch.
| Annotation Round 1 | Annotation Round 2 | |||||
|---|---|---|---|---|---|---|
| Batch 1 | Batch 2 | Batch 3 | Batch 1 | Batch 2 | Batch 3 | |
| Exact Agreement | 42.20% | 73.70% | 72.90% | 74.1% | 81.8% | 81.2% |
| Mismatch ID | 8.60% | 6.50% | 9.80% | 5.6% | 1.0% | 4.3% |
| Single - annotation | 49.20% | 19.80% | 17.30% | 20.3% | 17.2% | 14.5% |
