. 2016 May 5;11(5):e0155036. doi: 10.1371/journal.pone.0155036

Table 5. Differences between the three sentiment classes (−,0,+).

The differences are measured in terms of Alpha, for the union of self- and inter-annotator agreements. The second column shows the relative difference between the Alpha_int (interval) and Alpha_nom (nominal) agreement measures. The third and fourth columns show the distances of the negative (−) and positive (+) class to the neutral class (0), respectively, normalized with the distance between them. The last row is the average difference, but without the low quality Albanian and Spanish, and the subsumed Emojis datasets (in bold). Only the numbers in bold do not support the thesis that sentiment classes are ordered.

Dataset	$\frac{A l p h a_{i n t} - A l p h a_{n o m}}{A l p h a_{n o m}}$	$\frac{A l p h a_{i n t} (-, 0)}{A l p h a_{i n t} (-, +)}$	$\frac{A l p h a_{i n t} (0, +)}{A l p h a_{i n t} (-, +)}$
Albanian	0.786	0.759	0.146
Bulgarian	0.193	0.781	0.547
English	0.200	0.686	0.647
German	0.174	0.702	0.513
Hungarian	0.156	0.880	0.665
Polish	0.263	0.685	0.592
Portuguese	0.277	0.500	0.476
Russian	0.134	0.693	0.854
Ser/Cro/Bos	0.225	0.706	0.635
Slovak	0.001	1.258	0.956
Slovenian	0.279	0.579	0.595
Spanish	-0.158	1.440	1.043
Swedish	0.129	0.806	0.788
Facebook(it)	0.190	0.675	0.715
DJIA30	0.020	0.714	0.687
Environment	0.271	0.559	0.440
Emojis	0.234	0.824	0.586
Average*	0.179	0.730	0.650