Table B2.
Provisional Tool For Effect Size Data Coding
| Description | Question | Coding | |
|---|---|---|---|
| ID | Unique study identification # | For example, AQC001 | |
| Unique effect size identification # | For example, AQC001_01, AQC001_02, AQC001_03, and so forth | ||
| First author—impact evaluation | Surname | Open answer | |
| Outcome for effect size (answer for all studies) | Outcome | Which outcome is being coded? |
1 = Productivity 2 = Income 3 = Nutrition or health 4 = Empowerment 5 = Other (specify) |
| Definition of outcome | Please provide the authors definition of the outcome (including description of the subgroup if relevant) | Open answer | |
| Exposure to intervention (in months) | How long is the intervention exposure itself? | # | |
| Evaluation period (in months) | The total number of months elapsed between offering an intervention and the point at which an outcome measure is taken post intervention, or as a follow‐up measurement. If <1 month, use decimals (e.g., 1 week would be 0.25) | # | |
| Comparison | What type of comparison group is used? |
1 = No intervention (service delivery as usual) 2 = Other intervention 3 = Pipeline (wait‐list) control (still service intervention the delivery as usual) |
|
| Describe Comparison Group | If answer above is (1) no intervention, type N/A, if (2) Other Intervention, list what control group is receiving, if (3) Pipeline control, report when the control group will receive the intervention in relation to the treatment group (e.g., 1 year later) | Free Text | |
| Counterfactual | How is the counterfactual chosen? | Free text (e.g., random control trial, propensity score matching, etc.)—Multiple codes are ok | |
| Subgroup analysis | Is this effect size data for a subgroup? |
0 = No 1 = Yes |
|
| Subgroup analysis description | If yes to question 2, which type of subgroup? | Open answer—this can include separate samples for gender, income, place of residence | |
| Source | Which page(s) contain the effect size data? | Open answer | |
| Data to be extracted | Which type of data to be extracted? |
1 = Continuous—means and SDs 2 = Continuous—mean difference and SD 3 = Dichotomous outcome—proportions 4 = Regression data—dichotomous outcome (e.g., logistic regression) 5 = Regression data—continuous outcome (e.g., linear regression) |
|
| Analysis type for this effect size | What type of analysis was used (Regression, 2SLS, ANCOVA, etc.)? Multiple codes are ok | Free text | |
| Effect size data (answer for all studies) | Sample size metric | Sample size unit of analysis |
1 = Individual 2 = Household 3 = Group (e.g., community org) 4 = Village 5 = Other 6 = Not clear |
| Treatment effect estimated | What treatment effect is estimated? |
1 = ITT 2 = ATET 3 = ATE 4 = LATE |
|
| Sample size (treatment) | Initial sample size treatment group | # | |
| Sample size (control) | Initial sample size control group | # | |
| Sample size (total) | Initial sample size total | # | |
| Observations (treatment) | Number of treatment observations after attrition/follow up | # | |
| Observations (control) | Number of control observations after attrition/follow up | # | |
| Observations (total) | Total number of control observations after attrition/follow up | # | |
| Outcome data—if continuous (Means and SDs) | Baseline outcome treatment | State result of baseline outcome for treatment group | # |
| SD Baseline outcome treatment | State SD of baseline outcome measure for treatment group | # | |
| Baseline outcome control | State result of baseline outcome for control group | # | |
| SD Baseline outcome control | State SD of baseline outcome measure for control group | # | |
| Outcome in treatment post intervention | State result of post intervention outcome for treatment group | # | |
| SD Outcome in treatment post intervention | State SD of post intervention outcome measure for treatment group | # | |
| Outcome in control post intervention | State result of post intervention outcome for control group | # | |
| SD Outcome in control post intervention | State SD of post intervention outcome measure for control group | # | |
| Outcome in treatment 1st follow up | State result of 1st follow up outcome measure for treatment group | # | |
| SD Outcome in treatment 1st follow up | State SD 1st follow up outcome measure for treatment group | # | |
| Outcome in control 1st follow up | State result of 1st follow up outcome measure for treatment group | # | |
| SD Outcome in control 1st follow up | State SD 1st follow up outcome measure for treatment group | # | |
| Outcome data—If continuous (Mean difference and SD/SE at follow up) | Mean difference at follow up | State mean difference | # |
| SD at follow up | State SD at follow up | # | |
| SE | State SE | # | |
| Outcomes data—if dichotomous (Proportions r) | Baseline number with outcome in treatment | State result of baseline outcome for treatment group | # |
| Proportion with outcome at baseline in treatment | State proportion with outcome at baseline in treatment | # | |
| Baseline number with outcome in control | State result of baseline outcome for treatment group | # | |
| Proportion with outcome at baseline in control | State proportion with outcome at baseline in control | # | |
| Number with outcome in treatment post intervention | State number with outcome post intervention for treatment group | # | |
| Proportion with outcome in treatment group post intervention | State proportion with outcome post intervention in control group | # | |
| Number with outcome in control post intervention | State number with outcome post intervention for control group | # | |
| Proportion with outcome in control group post intervention | State proportion with outcome post intervention in control group | # | |
| Number with outcome in treatment 1st follow up | State number with outcome at 1st follow up for treatment group | # | |
| Proportion with outcome in treatment group 1st follow up | State proportion with outcome at 1st follow up in treatment group | # | |
| Number with outcome in control 1st follow up | State number with outcome at 1st follow up for control group | # | |
| Proportion with outcome in control group 1st follow up | State proportion with outcome at 1st follow up in control group | # | |
| Regression data | Type of coefficient | What is the coefficient type? | 1 = raw 2 = standardised 3 = other |
| Coefficient | What is the coefficient estimate? | # | |
| Pooled SD of outcome | What is the pooled SD of the outcome? | # | |
| SE | What is the SE of the coefficient estimate? | # | |
| t test | What is the t statistic associated with the focal predictor? | # | |
| p value | What is the p value associated with the coded effect? | # |