Table 5. Annotation guidelines of the most commonly studied hate speech datasets.

Dataset	Action	Target	Clarifications
Waseem	Attacks, seeks to silence, criticises, negatively stereotypes, promotes hate speech or violent crime, blatantly misrepresents truth or seeks to distort views on, uses a sexist or racial slur, defends xenophobia or sexism	A minority	(Inclusion) Contains a screen name that is offensive, as per the previous criteria, the tweet is ambiguous (at best), and the tweet is on a topic that satisfies any of the above criteria
Davidson	Express hatred towards, humiliate, insult*	A group or members of the group	(Exclusion) Think not just about the words appearing in a given tweet but about the context in which they were used; the presence of a particular word, however offensive, did not necessarily indicate a tweet is hate speech
Founta	Express hatred towards, humiliate, insult	Individual or group, on the basis of attributes such as race, religion, ethnic origin, sexual orientation, disability, or gender	N/A
HatEval	Spread, incite, promote, justify hatred or violence towards, dehumanizing, hurting or intimidating**	Women or immigrants	(Exclusion) Hate speech against other targets, offensive language, blasphemy, historical denial, overt incitement to terrorism, offense towards public servants and police officers, defamation

Notes.

Original wording from the publications or supplementary materials; action verbs grouped for easier comparison:

underlined

directly attack or attempt to hurt,

italic

promote hate towards.

N/A

no relevant descriptions found

^*Davidson et al. (2017) also gave annotators “a paragraph explaining it (the definition) in further detail”, which was not provided in their publication.

^**Basile et al. (2019) also gave annotators some examples in their introduction of the task (rather than the main guidelines, thus not included).