Table 3:

Inter-annotator agreement (measured by Krippendorff’s Alpha); we calculated the IAA for each document over all eight annotators

	Iteration 1	Iteration 2	Iteration 3	Iteration 4	Main Annotation
μα	0.892	0.795	0.701	0.752	0.821
σα	0.050	0.156	0.056	0.103	0.072
minα	0.805	0.313	0.562	0.506	0.594
Lower Quartile	0.848	0.755	0.675	0.730	0.799
Median	0.896	0.846	0.701	0.789	0.844
Upper Quartile	0.929	0.919	0.741	0.820	0.864
maxα	0.961	0.960	0.809	0.868	0.930