General	ID	EPPI ID
General	Study first author	Open answer
General	Time taken to complete assessment	Minutes
General	Design type: What type of study design is used?	1= RCT (random assignment to households/individuals) or quasi‐RCT	‐
		2= Cluster‐RCT (quasi‐RCT)
General	Methods used for analysis: Which methods are used to control for selection bias and confounding?	1 = Statistical matching (PSM, CEM, covariate matching)	‐
		2 = Difference in differences (DID) estimation methods
		3 = IV‐regression (2‐stage least squares or bivariate probit)
		4 = Heckman selection model
		5 = Fixed effects regression
		6 = Covariate adjusted estimation
		7 = Propensity weighted regression
		8 = Comparison of means
		9 = Other (please state)
General	Design and analysis method description	Open answer	Briefly describe the study design and analysis method undertaken by the authors
General	Study population	Open answer	Provide any details in the paper that describe how the study population was selected, covering:
			(a) How is the population selected? what is the sampling strategy to recruit participants from that population into the study?
			(b) What are the characteristics of that study participants?
			(c) Was this a pilot program aimed at being scaled up?
			(d) Were there specific factors of success or failure in the implementation?
General	Type of comparison group	1=No intervention (service delivery as usual)	Indicate type of comparison group
		2=Other intervention
		3=Pipeline (wait‐list) control (still service delivery as usual)
General	Type of comparison group (if other)	Open answer
General	Ethical clearance	Open answer	Provide any details of ethical research clearances granted. Report unclear if this information is not available
General	Study registration	Open answer	Provide any detials of study registration, inlcuding registry IDs, and so forth
1: Assignment mechanism—Assessment	Assignment mechanism: Was the allocation or identification mechanism random or as good as random?	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	(a) The authors describe a random component in sequence generation/randomization method (e.g., lottery, coin toss, random number generator) and assignment is performed for all units at the start of the study centrally or using a method concealed from participants and intervention delivery	Score “Yes” if all criterion (a), (b), (c) and (d) are satisfied
				Score "Probably Yes" if only criterion (a) and (b) are not satisfied OR if only criteria (c) is not satisfied
				Score “Unclear” if (d) is not satisfied because no balance table is reported
			(b) If public lottery is used for the sequence generation, authors provide detail on the exact settings and participants attending the lottery	Score "Probably No" if (d) is not satisfied because there is no balance table reported and there is evidence suggesting a problem in the randomization, such as baseline coefficients in a diff‐in‐diff regression table are very different or sample size is too small for the procedure used (using stratification when there are less than two units for each intervention and control group in each strata can lead to imbalance)
			(c) If a special randomization procedure is used to ensure balance, it is well described and justified given the study setting (stratification, pairwise matching, unique random draw, multiple random draws, etc.)
			(d) A balance table is reported suggesting that allocation was random between all groups including subgroup receiving different treatment within control or treatment groups (if the comparison is relevant for this assessment)	Score “No” if (d) is not satisfied because there are large imbalances concerning a large number of variables, providing evidence that the assignment was not random. If this is scored as no, use the NRS tool
1: Assignment mechanism—Justification	Assignment justification	Open answer	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)
2: Unit of analysis—Assessment	Unit of analysis: Is unit of analysis in cluster allocation addressed in SE calculation?	1=Yes, 2=No, 3=Not reported/unclear, 4=Not applicable	Score “Yes" if UoA = UoR OR if UoA != UoR and SEs are clustered at the UoR level OR data is collapsed to the UoR level
			Score "Not reported/unclear" if not enough information is provided on the way the SEs were calculated or what the unit of analysis is
			Score “Not applicable” if it is not a cluster RCT
			Score “No” otherwise
2: Unit of analysis—Justification	Method used to address differences between UoA and unit of data collection	Open answer
3: Selection bias—Assessment	Selection bias Was any differential selection into or out of the study (attrition bias) adequately resolved?	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	Score “Yes" if there is no attrition or attrition falls into the green zone and the study establishes that attrition is randomly distributed (e.g., by presenting balance by key characteristics across groups) AND if survey respondents were randomly sampled
			Score "Probably yes" if attrition falls into the green zone AND if survey respondents were randomly sampled
			Score "Unclear" if there is an attrition problem but no information provided on the relationship between attrition and treatment status, OR if there is not enough information on how the population surveyed was sampled
			Score "Probably no" if there is attrition which is likely to be related to the intervention OR there is some indication that the survey respondents were purposely sampled in a way that might have led the sampling to be different between treatment and control groups, or attrition falls into the yellow zone
			Score "No" if attrition falls into the red zone
3: Selection bias—Justification	Selection bias justification	Open answer	Justification for coding decision (Include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)
3: Confounding—Assessment	Confounding and group equivalence: Was the method of analysis executed adequately to ensure comparability of groups throughout the study and prevent confounding	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	(a) Baseline characteristics are similar in magnitude;	Score “Yes” if criterion (a) and (b) are satisfied;
			(b) Unbalanced covariates at the individual and cluster level are controlled in adjusted analysis;	Score "Probably yes" if (a) is not satisfied but (b) is satisfied and imbalances are small in magnitude OR if only (a) is satisfied.
			(c) Adjustments to the randomization were taken into account in the analysis (stratum fixed effects, pairwise matching variables)? (Bruhn & McKenzie, 2009)	Score “Unclear” if no balance table is provided or if imbalances are controlled for but they are very large in magnitude and assignment mechanism is not coded as "Yes" or "Probably yes"
				Score "Probably no" if (a) and (b) are not satisfied and the magnitude of imbalances are small
				Score “No” if (a) and (b) are not satisfied and the magnitude of imbalances are large and covariates are clear determinant of the outcomes
3: Confounding—Justification	Confounding justification	Open answer	Justification for coding decision (include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)
4: Deviations from intended interventions—Assessment	Deviations from intended interventions: Spill‐overs, cross‐overs and contamination: was the study adequately protected against spill‐overs, cross‐overs and contamination?	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	(a) There was no implementation issues that might have led the control participants to receive the treatment (implementer's mistake)	Score “Yes” if criterion (a), (b), (c) and (d) are satisfied;
			(b) The intervention is unlikely to spill‐over to comparisons (e.g., participants and non‐participants are geographically and/or socially separated from one another and general equilibrium effects are not likely) or the potential effects of spill overs were measured (e.g., variation in the % of unit within a cluster receiving the treatment)	Score "Probably yes" if there is no obvious problem but there is no information reported on potential risks related to spill overs, contamination, or survey effects in the control group OR if there were issues with spill‐overs but they were controlled for or measured
				Score “Unclear” if spill‐overs, cross‐overs, survey effects and/or contamination are not addressed clearly
			(c) There is no risk of contamination by external programs: the treatment and comparisons are isolated from other interventions which might explain changes in outcomes.	Score "Probably no" if any of the criterion (a), (b), (c) or (d) are not satisfied but the scale of the issue is not clear
			(d) There is nothing in the surveys that might have given the control participants an idea of what the other group might receive OR they did but there is no risk that this has changed their behaviours; AND the survey process did not reveal information to the control group that they did not have before (e.g., the study aims to measure increase in take up of a service or product that participants might not know about) Authors might put something in place in the design of the study that allows to control for that survey effect (e.g., a pure control with no monitoring except baseline end line)	Score “No” if any of the criterion (a), (b), (c) or (d) are not satisfied and happened at a large scale in the study
4: Deviations from intended interventions—Justification	Deviations justification	Open answer	Justification for coding decision (Include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)
			For example, intervention groups are geographically separated, authors use intention to treat estimation or instrumental variables to account for non‐adherence, and survey questions are not likely to expose individuals in the control group to information about desirable behaviours (“survey effects”)
5. Performance bias—Assessment	Performance bias: was the process of monitoring individuals unlikely to introduce motivation bias among participants?	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	(a) The authors state explicitly that the process of monitoring the intervention and outcome measurement is blinded and conducted in the same frequency for treatment and control groups, or argue convincingly why it is not likely that being monitored could affect the performance of participants in treatment and comparison groups in different ways (such as resulting in Hawthorne or John Henry effects).	Score “Yes” if either criterion (a) or (b) are satisfied;
				Score "Probably yes" if the study is based on data collected during a trial and there is no obvious issue with the monitoring processes but authors do not mention potential risks
				Score “Unclear” if it is not clear whether the authors use an appropriate method to prevent Hawthorne and John Henry Effects (e.g., blinding of outcomes and, or enumerators, other methods to ensure consistent monitoring across groups). Hawthorne effects may result where participants know that they are being observed and John Henry Effects may result from participant knowledge of being compared
			(b) The outcome is based on data collected in the context of a survey, and not associated with a particular intervention trial, or data are collected from administrative records or in the context of a retrospective (ex post) evaluation	Score "Probably no" if there was imbalance in the frequency of monitoring in intervention groups, which might have influenced participants' behaviours
				Score "No" if neither criterion (a) or (b) are satisfied
5. Performance bias—Justification	Performance bias justification	Open answer	Justification for coding decision (Include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)
6. Outcome measurement bias—Assessment	Outcome measurement bias: was the study free from biases in outcome measurement?	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	(a) Outcome assessors are blinded or the outcome measures are not likely to be biased by their judgement	Score “Yes” if criterion (a), (b), (c) and (d) are satisfied:
			(b) For self‐reported outcomes: respondents in the intervention group are not more likely to have accurate answers due to recall bias;	Score "Probably yes" if there is a small risk related to any of (a), (b), (c) or (d) and there is no more information provided to to justify the absence of bias OR if there was a high risk of bias but authors have either controlled it in their design or measured it with a placebo outcomes
			(c) For self‐reported outcomes: respondents do not have incentives to over/under report something related to their performance or actions, OR researchers put in place mechanisms to reduce the risk of reporting bias (researchers not strongly involved in the implementation of the program and it is clear that their answers to the survey will not affect what they receive in the future) OR authors have measured the risks of bias through falsification tests or measuring the effect on placebo outcomes in cases where there was a risk of reporting bias	Score “Unclear” if it there is a high risk related to any of (a), (b), (c) or (d) and there is no more information provided to to justify the absence of bias
				Score "Probably no" if there are high risk related to (a), (b), (c) or (d) and it is clear that authors were not able to control for this bias
				Score “No” if there is evidence of bias
			(d) Timing issue: the data collection period did not differ between intervention and comparison group, the baseline data is not likely to be affected by the beginning of the intervention or affects a small percentage of the study participants
6. Outcome measurement bias—Justification	Outcome measurement justification	Open answer	Justification for coding decision (Include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages).
7. Reporting bias—Assessment	Analysis reporting: was the study free from selective analysis reporting?	1= Yes, 2 = Probably Yes, 3 = Probably No, 4 = No, 8 = Unclear	(a) A preanalysis plan or trial protocol is published and referred to or the trial was pre‐registered or the outcomes were pre‐registered;	Score "Yes" if all the criterion (a), (b), (c), (d), and (e) are satisfied;
			(b) Authors report results corresponding to the outcomes announced in the method section (there is no outcome reporting bias);	Score "Probably yes" if all the conditions are met except (a), or if all the conditions are met but there is some element missing that could have helped understand the results better (e);
			(c) Authors report results of unadjusted analysis and intention to treat (ITT) estimation, alongside any adjusted and treatment‐on‐the‐treated/complier‐average‐causal‐effects analysis)	Score "Unclear" if there is not enough information to determine that there is an analysis missing;
			(d) Authors use the appropriate analysis method (use baseline data when available) and different treatment arms are differentiated in the analysis	Score "Probably no" if any of the criterion (b), (c) or (d) are not satisfied;
			(e) Authors have reported all the analysis which could help understand the results and no other bias is assessed as unclear due to the lack of an important analysis (e.g., a balance table or a subgroup analysis)	Score "No" if any of the criterion (b), (c) or (d) are not satisfied and there is evidence that the analysis results would be different because large imbalances were not controlled for, compliance was very low and ITT estimation was not reported or different treatment arms were pooled
8. Reporting bias—Justification	Analysis reporting justification	Open answer	Justification for coding decision (Include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages)
9. Other bias—Assessment	Other risks of bias Is the study free from other sources of bias?	1= Yes, 4 = No
9. Other bias—Justification	Other bias justification	Open answer	Justification for coding decision (Include a brief summary of justification for rating, mentioning your response to all sub questions, cite relevant pages). For example, information is collected using a different survey instrument in different intervention groups; measurement of the intervention received in unclear
10. Blinding—observers—Assessment	Blinding of participants?	1=Yes, 2=No, 8=unclear, 9= N/A	If there is no information, code NO. If there is information but it is ambiguous, code UNCLEAR
10. Blinding—observers—Assessment	Blinding of outcome assessors?	1=Yes, 2=No, 8=unclear, 9= N/A	If there is no information, code NO. If there is information but it is ambiguous, code UNCLEAR
10. Blinding—analysts—Assessment	Blinding of data analysts?	1=Yes, 2=No, 8=unclear, 9= N/A	If there is no information, code NO. If there is information but it is ambiguous, code UNCLEAR
10. Blinding—method(s)	Method(s) used to blind	Open answer (including describe method of placebo control), No 9= N/A	Describe method(s) used to blind
11. External validity—Assessment	External validity	Open answer	a) What do authors say about external validity?	Include all information that can help assess the external validity of the results