Table 4. Comparison of human and neural network annotation performance.
Annotator 1 | Neural network | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
NA (No. of annotations) | AN (Annotated number) | NA (No. of annotations) | AN (Annotated number) | |||||||||||||||
Annotator 2 | NA | 5,380 (74%) | 3,530 (5%) | Annotators | NA | 4,070 (58%) | 1,079 (15%) | |||||||||||
AN | 5,442 (7%) | 8,414 (11%) | AN | 443 (6%) | 1,425 (20%) | |||||||||||||
Precision | Recall | F1 Score | Kappa | Precision | Recall | F1 Score | Kappa | |||||||||||
0.71 | 0.61 | 0.64 | 0.57 | 0.56 | 0.72 | 0.64 | 0.50 |