Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
Average agreements between the system and human annotators, all human annotators and subsets.
| Author/system | Kappa (average) |
|---|---|
| Human-system agreement average | 0.33 |
| Human-QSpec agreement average | 0.35 |
| Human-human agreement average | 0.40 |
| Average: annotators 1, 2 and 3 | 0.41 |
| Average: annotators 1, 2 and 4 | 0.40 |
| Average: annotators 1, 3 and 4 | 0.34 |
| Average: annotators 2, 3 and 4 | 0.38 |
