Study Level Coding Form
This coding form is for each unique study. Note that a study may be reported in multiple manuscripts (publications, technical reports, etc.). Also, some reports may include the results for distinct studies, such as evaluations in different cities. Our unit‐of‐analysis for the meta‐analysis is an independent study. No two studies should include any of the same participants. If there are multiple publications for the same study, use the most complete study as the primary study ID and all other related studies as cross reference IDs.
Identifiers
1.	Reference ID	studyid	|__|__|__|__|
2.	Other related references	crossref1	|__|__|__|__|
		corssref2	|__|__|__|__|
		corssref3	|__|__|__|__|
		corssref4	|__|__|__|__|
		corssref5	|__|__|__|__|
3.	Coder's initials	sinitials	|__|__|__|
4.	Creation date (mm/dd/yy)	sdate	|__|__|__|__|__|
5.	Modification date (mm/dd/yy)	sdatem	|__|__|__|__|__|
General Study Information
6.	Publication type	pubtype	|__|
	1.Book 2.Book chapter 3.Journal article (peer reviewed) 4.Journal article (not‐peer reviewed) 5.Thesis‐dissertation 6.Technical report 7.Conference paper 8.Government publication 9.Other (Specify): ______________
7.	Language type of study	language	|__|
	1. English
	2. German
	3. Persian
	4. Arabic
8.	Geographic location of study	location	|__|__|__|__|
	1.North America 2.South America 3.Europe 4.Africa 5.Asia 6.Oceania
9.	Years of data collection
	Year data collection started	datastart	|__|__|__|__|
	Year data collection ended	dataend	|__|__|__|__|
10.	Intervention type	inttype	|__|
	1. Online only
	2. Online and offline/mixed approach
	3. Offline only
11.	Researcher involvement	rinvolve	|__|
	1. Researcher initiated intervention
	2. Online platform‐initiated intervention
	3. Government initiated intervention
12.	Was this research funded by a grant or external agency	funding	|__|
	0. No
	1. Yes
	9. Cannot tell
Research Design
13.	Unit of assignment to conditions	uoa	|__|
	1.Individual 2.Incident (might include multiple comments) 3.Online platform 4.Online groups 5.Other 9. Cannot tell
14.	Methodological approach	method	|__|
	1. Qualitative
	2. Quantitative
	3. Mixed methods
15.	How subjects were assigned to condition (this is about assignment not sampling)	assign	|__|
	1.Randomly after matching, yoking, stratification, blocking, etc. 2.Randomly without matching 3.Regression discontinuity (quantitative cutting point defines groups) 4.Wait list control or other such quasi‐random procedures (e.g., alternating cases) 5.Quasi‐experimental, matched individual level 6.Quasi‐experimental, matched group level (e.g., classrooms) 7.Quasi‐experimental, statistical controls for baseline differences 8.Quasi‐experimental, no statistical controls for baseline differences 9.Quasi‐experimental, other 10.Quasi‐experimental, cohort design (historical controls)
16.	If random assignment or regression discontinuity design:	rndinteg	|__|
	1. Integrity of randomization or other assignment method maintained (no more than a few cases failed to end up in desired group)
	2. Failures of randomization or assignment occurred
	3. No information on integrity of assignment process
17.	[RISK OF BIAS ITEM] Is there any risk of selective outcome reporting bias, that is, is there any evidence that the authors have not reported findings for all variables measured as part of this study?	selectrepb	|__|
	1. Low Risk
	2. Some Concerns
	3. High Risk
18.	Study level coding notes	snotes
Comparison Level Coding Form This coding form is for each treatment/comparison contrast coded from a study. For most studies, you will only code this form once. However, some studies may have two or more treatment conditions or two or more comparison conditions. In the coding below, it is critical to indicate if any of the treatment/comparison contrasts for a study share sample participants. For example, a study might have two distinct treatments but only one comparison group. In this case, these comparisons share sample participants (i.e., the same comparison condition).
Identifiers
1.	Reference ID	studyid	|__|__|__|__|
2.	Condition ID	compid	|__|__|__|__|
3.	Coder's initials	cinitials	|__|__|__|
4.	Creation date (mm/dd/yy)	cdate	|__|__|__|__|__|
5.	Modification date (mm/dd/yy)	cdatem	|__|__|__|__|__|
6.	Treatment group label	txlabel	|__|__|__|__|__
7.	Control/comparison group label	cglabel	|__|__|__|__|__
Sample Information
8.	Treatment group sample size (at start of study before attrition; −99 if cannot tell)	ctxn	|__|__|__|__|__|
9.	Comparison group sample size (at start of study before attrition; −99 if cannot tell)	ccgn	|__|__|__|__|__|
10.	Mean or median age of sample (−99 if cannot tell)	meanage	|__|__|.__|
11.	Youngest age in sample (−99 if cannot tell)	minage	|__|__|
12.	Oldest age in sample (−99 if cannot tell)	maxage	|__|__|
13.	Sex distribution for this treatment/comparison contrast	sex	|__|
	1.100% Male 2.90–99% Male 3.75–89% Male 4.26–75% Male 5.11–25% Male 6.1–10% Male 7.0% Male 99. Unknown
14.	Percent of this condition that is represented by each of the following race/ethnic group (−99 if missing unknown):
	1. White	white	|__|__|__|.__|
	2. Black/African/Caribbean	black	|__|__|__|.__|
	3. Hispanic (non‐White)	hispanic	|__|__|__|.__|
	4. Asian	asian	|__|__|__|.__|
	5. Mixed/Multiple ethnic groups	mixed	|__|__|__|.__|
	6. Other	raceother	|__|__|__|.__|
Nature of Treatment Condition
15.	Type of intervention	inttype	|__|
	1.Online hate detection only 2.Server shutdowns 3.Deletion of social media accounts 4.Responding to online hate via counter‐narratives 5.Modifying hateful content 6.Countering “fake news” 7.Twitter “fact” check 8.Other (specify): _________
16.	Content of intervention	intcontent	|__|
	1.Everyday hate 2.Right‐wing extremist content 3.Islamist extremist content 4.Islamist extremist content    99. Cannot tell
17a.	Location of intervention	intlocate	|__|
	1.Websites 2.Text messaging applications 3.Online and social media platforms
17b.	If social media, which platform	platform	|__|
	1.Facebook 2.Instagram 3.TikTok 4.WhatsApp 5.Google 6.YouTube 7.Snapchat 8.Twitter 9.4chan 10.Gab 11.Other (specify): ______________
18.	Other elements of this condition:	txother
Nature of Comparison Condition
19.	Type of comparison condition	comptype	|__|
	1. No exposure
	2. Comparison exposure
	3. Other
	[Note: we will add to the list of options as we code studies.]
20.	Services or sanctions for the comparison condition	compother
Comparability of Conditions
21.	Were the conditions compared for baseline equivalence on any of the following, either statistically or descriptively? (0 = statistically; 1 = descriptively; 9 = cannot tell)
	1. Sex	basediff1	|__|
	2. Race	basediff2	|__|
	3. Age	basediff3	|__|
22.	RISK OF BIAS ITEM: Based on the above, is there a risk of selection bias, that is, that the groups were different at baseline?	selectbias	|__|
	1. Low risk
	2. High risk
	3. Unclear
23.	RISK OF BIAS ITEM: Is there a risk of general attrition bias for the primary outcome measure, that is, attrition in excess of 10%?	attrition1	|__|
	1. Low risk
	2. High risk
	3. Unclear
24.	RISK OF BIAS ITEM: Is there a risk of different attrition bias for the primary outcome measure, that is, meaningful differential attrition?	attrition2	|__|
	1. Low Risk
	2. Some Concerns
	3. High Risk
25. Notes about coding this comparison		cnotes
Outcome (Dependent Variable) Coding Form
Code each eligible outcome or dependent variable using the form below. Note that you should code this only once for a variable that is measured at multiple time points. That is, recidivism measured at 3‐, 6‐, and 9‐months is a single dependent variable. Code the characteristics of the measure using this form and the data for each measurement time point on the effect size forms.
Identifiers
1.	Reference ID	studyid	|__|__|__|__|
2.	Coder's initials	dvinitials	|__|__|__|
3.	Creation date (mm/dd/yy)	dvdate	|__|__|__|__|__|
4.	Modification date (mm/dd/yy)	dvdatem	|__|__|__|__|__|
5.	Outcome ID	dvid	|__|__|__|__|
6.	Dependent variable label	dvlabel	|__|__|__|__|
Characteristics of Variable
7.	Elements reported in this outcome measure irrespective of the type of incident and reporting source (check best one):	dvelem	|__|__|__|
	1. Global dichotomy or polychotomy (e.g., created, or consumed cyberhate, extremist content or non‐extremist content = yes/no)
	2. Summed dichotomous (e.g., sum of “yes/no” on list of specific behaviors)
	3. Frequency or rate, (count of incident; incidents per 1000 persons)
	4. Severity (seriousness rating or index), see this often with self‐report measures
	5. Event timing (e.g., days without content creation; time since last post, log on, video watch)
	6. Proportion or amount of time on extremist website, etc.
	7. Rating of amount of delinquency, severity, change, etc. This is similar to frequency but in rating form. (e.g., How often you did “x” behavior)
	8. More than one of above elements combined in composite measure
	9. Other
	99. Cannot tell
8.	Type of behavior represented by this measure (what's counted, irrespective of source of information and authors' label or description of the measure) check best one:	dvtype	|__|__|__|
	1. Content creation (e.g., production and authorship of original content such as making videos, writing blog posts, or uploading content)
	2. Transmission of hate speech (e.g., racist, homophobic, anti‐Semitic), not specifically restricted to extremist acts
	3. Consumption of cyberhate (e.g., watch videos, visit social media platforms, or read blogs without making accounts from self or observer's report)
	4. Collecting extremist content (e.g., organize links and content for either their personal use or to disseminate information to others who are active online
	5. Critics (e.g., comment on social media posts, submit reviews, and rate content)
	6. Joiners (e.g., those who maintain accounts but do not comment or post publicly available content)
	7. Other
	99. Cannot tell
9.	RISK OF BIAS ITEM: Person providing outcome data knows which condition the participant is in (i.e., is there a potential bias from the lack of blinding of the assessor?)	dvbias	|__|
	1. Low Risk
	2. Some Concerns
	3. High Risk
10.	Notes regarding this outcome measure	dvnotes

Effect Size Coding Form
Code all effect sizes of interest using the form below, coding each effect size separately (i.e., with a different copy of the form or record in the database). Indicate the study ID, comparison ID, and dependent variable ID. Give each effect size within a study a unique ID (i.e., 1, 2, 3…).
There are several ways to compute effect sizes using the different tabs. ONLY USE ONE METHOD per effect size. If you have the raw means and also a regression coefficient for the same outcome from a model that adjusts for baseline differences, these are two different effect sizes. The different effect size computation methods are:
1.Means and standard deviations 2.Means and standard errors 3.Frequency of failures in each condition 4.Proportion of failures in each condition 5.Logistic regression coefficient for treatment effect dummy code 6.OLS unstandardized regression coefficient 7.OLS standardized regression coefficient 8.Independent samples t test 9.Chi‐square test (2 by 2, df = 1) 10.Point‐biserial correlation coefficient 11.Phi correlation coefficient 12.Hand computation (e.g., using the online effect size calculator)
Identifiers
1.	Reference ID	studyid	|__|__|__|__|
2.	Coder initials	esinitials	|__|__|__|
3.	Creation date	esdate	|__|__|__|__|__|
4.	Modification date	esdatem	|__|__|__|__|__|
5.	Comparison ID	compid	|__|__|__|__|
6.	Outcome ID	dvid	|__|__|__|__|
7.	Effect Size ID	esid	|__|__|__|__|
Effect Size Information
8.	Direction of effect	esdirect	|__|
	1 = favors treatment
	2 = favors control
	3 = neither, exactly equal
	99 = cannot tell
9.	Type of effect size (i.e., baseline differences, first post treatment outcome measure, or a follow‐up measure)	estype	|__|
	1 = baseline (pretest)
	2 = posttest
	3 = follow‐up
10.	Effect reported as statistically significant by authors	essig	|__|
	0 = no
	1 = yes
	99 = cannot tell
11.	Timing of measurement (months captured by the measure from the point of assignment to conditions or diversion/formal processing; if reported in months, divide by 4.3; 8888 if not applicable; 9999 if missing)
	Mean	estime1	|__|__|__|__|
	Minimum	estime2	|__|__|__|__|
	Maximum	estime3	|__|__|__|__|
Effect Size Data
12.	Treatment group sample size for this effect size	estxn	|__|__|__|__|
13.	Comparison group sample size for this effect size	escgn	|__|__|__|__|
	Scaled outcome data
14.	Mean treatment group	esmtx	|__|__|__|__|.__|__|
15.	Mean comparison group	esmcg	|__|__|__|__|.__|__|
16.	Are the above means adjusted for baseline differences? 0 = no; 1 = yes; 99 = cannot tell)	esmadj	|__|
17.	Standard deviation treatment group	essdtx	|__|__|__|__|.__|__|
18.	Standard deviation comparison group	essdcg	|__|__|__|__|.__|__|
19.	Standard error treatment group	essetx	|__|__|__|__|.__|__|
20.	Standard error comparison group	essecg	|__|__|__|__|.__|__|
	Dichotomous outcome data
21.	Treatment group number successful	Estxn	|__|__|__|__|
22.	Comparison group number successful	Escgn	|__|__|__|__|
23.	Treatment group number failures	estxnf	|__|__|__|__|
24.	Comparison group number failures	escgnf	|__|__|__|__|
25.	Treatment group proportion of successes (only code this if raw frequencies are not available)	estxpf	|__|.__|__|__|__|__|
26.	Comparison group proportion of successes (only code this if raw frequencies are not available)	escgpf	|__|.__|__|__|__|__|
27.	Are the above frequencies or proportions adjusted for baseline differences? (1 = yes; 0 = no; 9 = cannot tell)	espadj	|__|
	Logistic regression
28.	Logistic regression coefficient (for treatment effect dummy)	eslgor	|__|.__|__|__|__|__|
29.	Standard error for logistic regression coefficient	esselgor	|__|.__|__|__|__|__|
30.	t test or z test for logistic regression coefficient	esolst	|__|.__|__|__|__|__|
31.	Odds ratio for treatment effect dummy (optional)	esor	|__|__|__|.__|__|__|
	OLS regression
32.	Unstandardized regression coefficient	esolsb	|__|.__|__|__|__|__|
33.	Standard regression coefficient	esolsbeta	|__|.__|__|__|__|__|
34.	Standard error of regression coefficient	esolsse	|__|.__|__|__|__|__|
35.	Standard deviation for dependent variable	essd	|__|.__|__|__|__|__|
	Other possible effect size data
36.	t test (comparing two‐sample means; not the t from a regression model)	est	|__|__|__|__|.__|__|
37.	p value from a t test (comparing two‐sample means; not the t from a regression model)	espfromt	|__|.__|__|__|__|__|
38.	Correlation coefficient point‐biserial (treatment versus comparison correlated with scaled variable)	esrpb	|__|.__|__|__|__|__|
39.	Correlation coefficient phi (treatment versus comparison correlated with a dichotomous variable)	esrphi	|__|.__|__|__|__|__|
40.	Chi‐square (treatment versus comparison correlated with a dichotomous variable, df must equal 1)	eschisq	|__|__|__|__|.__|__|
	Effect size computed by hand (e.g., using online calculator)
41.	Standardized mean difference effect size computed by hand (d‐type)	eshand	|__|__|.__|__|__|__|
42.	Variance for standardized mean different effect size computed by hand	eshandv	|__|__|.__|__|__|__|
43.	Computed effect size	escalc	|__|__|.__|__|__|__|
44.	Computed effect size standard error	escalcse	|__|__|.__|__|__|__|
	Effect size coding notes
45.	Page number where effect size data found	espage	|__|__|__|__|__|__|
46.	Notes about this effect size	esnotes