Table 3.

Inter-annotator agreement (measured using Cohen’s kappa) between average volunteers (A) and two groups of experienced volunteers (E₁ and E₂). Region includes both country and state/territories as applicable.

	All NPI event types			Top 5 NPI event types
	A vs E1	A vs E2	E1 vs E2	A vs E1	A vs E2	E1 vs E2
Type	0.63	0.69	0.80	0.81	0.77	0.85
Type + Value	0.41	0.42	0.69	0.51	0.47	0.70
Date	0.50	0.61	0.73	0.60	0.69	0.76
Region	0.99	1.00	0.99	0.98	1.00	0.98
Restriction	0.36	0.43	0.74	0.74	0.58	0.69
Type + Date	0.44	0.53	0.70	0.51	0.59	0.72
Type + Value + Date	0.31	0.34	0.62	0.35	0.36	0.59
Type + Value + Date + Region	0.30	0.33	0.62	0.35	0.36	0.59
Type + Value + Date + Region + Restriction	0.26	0.29	0.61	0.34	0.35	0.59