Table 4.

Agreement level (Cohen’s kappa) between human annotators, and between the primary annotator and the models adapted by random sampling at iterations 5, 10 and 20.

Participant	Agreement between annotators	Agreement  (model, annotator 1) – iteration 5	Agreement (model, annotator 1) – iteration 10	Agreement (model, annotator 1) – iteration 20
PT01	0.584	0.498	0.541	0.616
PT02	0.748	0.736	0.768	0.753
PT03	0.600	0.868	0.862	0.895
PT04	0.593	0.705	0.766	0.828
PT09	0.727	0.856	0.888	0.943
PT10	0.548	0.665	0.721	0.785
PT16	0.844	0.954	0.953	0.963
PT18	0.859	0.8	0.78	0.789
PT20	0.793	0.937	0.942	0.944