Table 1.

Comparison of the inter-annotator agreement and classifier performance over four evaluation measures

	Annotator agreement	Stance classifier
No. of testing examples	6807	37, 048
Alpha	$67.7\%$	$45.8\%$
$\overline{F_1}(-,+)$	$74.6\%$	$60.3\%$
$Accuracy$	$77.2\%$	$59.5\%$
$Accuracy\pm 1$ ($-,+$)	$96.0\%$	$90.5\%$