Risk of bias tool for impact evaluations
	Description	Question	Coding
ID		Study
	Unique study identification #Paper	Surname/year of first author of paper for effect size data extraction	For example, PES001Open answer
Research methods—study design and risk of bias	Design type	What type of study design is used?	1 = Randomised controlled trial (RCT) (random assignment to households/individuals) 2 = Cluster‐RCT 3 = RDD (quasiexperiment with discontinuity assignment) 4 = CBA (comparison group with baseline and endline data collection) 5 = Panel data, but no baseline 6 = Comparison group with endline data only 7 = Natural experiment 8 = Other
	Methods used for analysis	Which methods are used to control for selection bias and confounding?	1 = PSM 2 = Covariate matching 3 = DID 4 = IV‐regression 5 = Heckman selection model 6 = Fixed effects regression 7 = Other regression 8 = Randomised study
	Design and analysis method description	Briefly describe the study design and analysis method undertaken by the authors	Open answer
	Mechanism of assignment	1: Mechanism of assignment: was the allocation or identification mechanism able to control for selection bias?	1 = Yes, 2 = No, 8 = Unclear
	Mechanism of assignment	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Group equivalence	2: Group equivalence: was the method of analysis executed adequately to ensure comparability of groups throughout the study and prevent confounding?	1 = Yes, 2 = No, 8 = Unclear
	Group equivalence	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Spill‐overs, cross‐overs and contamination	3: Spill‐overs, cross‐overs and contamination: was the study adequately protected against spill‐overs, cross‐overs and contamination?	1 = Yes, 2 = No, 8 = Unclear
	Spill‐overs, cross‐overs and contamination	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Outcome reporting	4: Outcome reporting: was the study free from selective outcome reporting?	1 = Yes, 2 = No, 8 = Unclear
	Outcome reporting	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Analysis reporting	5: Analysis reporting: was the study free from selective analysis reporting?	1 = Yes, 2 = No, 8 = Unclear
	Analysis reporting	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Performance bias	6: Performance bias: was the process of being observed free from motivation bias?	1 = Yes, 2 = No, 8 = Unclear
	Performance bias	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Other bias	7: Other risks of bias: Is the study free from other sources of bias?	1 = Yes, 2 = No, 8 = Unclear
	Other bias	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)	Open answer
	Type of comparison group		1 = No intervention (business as usual) 2 = Other intervention 3 = Placebo control 4 = Pipeline (wait‐list) control
	Other intervention differentially received by comparison group	Describe any nonenvironmental comparison group intervention received which treatment group does not?	Open answer
	Unit of analysis	Are there any unit of analysis errors? (e.g., the unit of analysis is different from the unit of treatement allocation and authors do not correct for these unit of analysis differences)?	1 = Yes, 2 = No, 8 = Not clear, 9 = N/A
	Blinded participants	Blinding of participants?	1 = Yes, 2 = No, 9 = N/A
	Blinded observers	Blinding of outcome assessors?	1 = Yes, 2 = No, 9 = N/A
	Blinded analysts	Blinding of data analysts	1 = Yes, 2 = No, 9 = N/A
	Method used to blind	Describe method(s) used to blind	Open answer (including describe method of placebo control)