Breisinger 2018.
Study characteristics | ||
Methods |
Study design: controlled trial (using regression discontinuity for allocation) How were missing data handled? It is assumed that missing data were excluded. The intended sample for the Takaful and Karama analysis components consisted of 7996 HHs. Of these, 1144 HHs could not be located based on the address data in the registration form. An additional 261 HHs were visited but no HH members could be located, and 70 HHs were not surveyed due to other reasons including declining to participate in the survey, no capable respondent being identified, being part of another HH already in the sample or the registrant having died. Considering only HHs for which there was no error in the location information, the overall response rate was 95.2%. Randomisation ratio: N/A Recruitment method: see sampling methodology. Sampled HHs were visited by an interviewer. Sample size justification and outcome used: outcome used NR. Justification: based on power calculations that the evaluation team conducted to determine the sample size required for the TKP impact evaluation, the study required 8016 HHs spread across 501 clusters (villages), with 16 HHs per cluster. Authors purposefully selected more than the 16 HHs per cluster knowing that there was a risk of being unable to locate all the HHs in the registrant sample. Sampling method: targeting for programme: to avoid inefficient targeting, the programme combined geographical targeting with a PMT mechanism. With respect to the geographical targeting, the programme was first launched in the poorest districts within the poorest governorates in Egypt. The PMT was used to identify the poor within the selected districts, based on selection criteria and a set cutoff score, based on the poverty line derived from Egypt's HH Income, Expenditure and Consumption Survey for 2012–2013. In addition to the PMT, both Takaful and Karama have other categorical selection criteria; Takaful requires that beneficiaries have children and Karama requires that beneficiaries be elderly or disabled (or both), or (added later) orphaned. PMT formula varies by region but the threshold is the same across all regions. Over time, since the programme started and across the 4 enrolment waves, the threshold has been changed. Sampling for the impact evaluation survey: "The sampling strategy for the TKP Impact evaluation was designed to provide a representative sample of Takaful and Karama HHs with Proxy Means Test scores near the thresholds for an RD [regression discontinuity] approach to impact analysis. Stratification for the sample selection was based primarily on region. For the Upper Rural region of Egypt, they stratified by governorate (9 governorates in Upper Rural). As such, they defined 14 strata: 5 for each region excluding Upper Rural and 9 strata for Upper Rural. They sampled VCs within the 14 region‐governorate strata using simple random sampling, where the number of clusters per stratum was proportional to the share of registrants in each stratum, and restricted selection to clusters in which there was a sufficient number of registrants near the threshold. The clusters were defined as Takaful‐only clusters (325) or mixed Takaful and Karama clusters (75). Within each village, 20 HHs were randomly selected for inclusion in the survey if they were within 600 points of the current Takaful threshold score of 4500 or 200 points of the Karama threshold of 7203. On average, they selected 10 eligible and 10 ineligible HHs, and for Takaful HHs, they weighted the probability of selection such that HHs within 200 points of the current threshold of 4500 were 2.5 times more likely, and HHs within 200–400 points of the cutoff were 1.5 times more likely to be selected than those that were 400–600 points from the cutoff. They selected more HHs than the 16 HHs per cluster that the power calculations suggested would be necessary, knowing that there was a risk of not being able to locate all the HHs in the registrant sample. Study aim or objective: objective of programme: Takaful and Karama is a conditional 5 cash transfer programme that seeks to provide income support to poor families with children (under 18 years of age), poor elderly (aged ≥ 65 years) and people with severe disability. Objective of impact evaluation: to provide rigorous evidence on the impacts of the programme on HH consumption, poverty and other measures of well‐being including child education, health and food security, and the prevalence of overweight and obesity in adult women. Study period: approximately 28 months. Programme start: March 2015; impact evaluation survey: 15 July to 30 August 2017. Takaful beneficiaries were in programme for about 11 months on average. Unit of allocation or exposure: HHs |
|
Participants |
Baseline characteristics Intervention or exposure: NR Control: NR Overall: NR Inclusion criteria: 1. Eligible governorates: those where the share of Takaful‐eligible HHs (HH eligibility defined as having a PMT score ≤ 4500) in the governorate was ≥ 0.5%. Therefore, governorates that had a very small share of eligible HHs in TKP were excluded, in part to help manage survey costs. The only exception was governorates in the Frontier region, where the share of eligible HHs was < 0.5%. These governorates were kept so that the Frontier region, which may have had different sources of poverty and food insecurity, was represented in the evaluation sample. 2. Eligible VCs for Takaful: those that had ≥ 7 Takaful eligible HHs. 3. Eligible VCs for Karama: those that had ≥ 6 Karama eligible HHs with a PMT score 7000–7400 and had ≥ 1 elderly (aged ≥ 65 years) or disabled member. Exclusion criteria: PMT‐score‐based targeting, if they met 1 of 6 exclusion criteria: owned a car, owned > 1 feddan of land, had a government job or pension, received transfers from abroad or had a formal private sector job with insurance. No children. Pretreatment: Tables A2.1–A2.6 showed that of 20 HH characteristics all but 1 showed a statistically significant difference for beneficiaries vs non‐beneficiaries. Therefore, the HHs on either side of the cutoff were similar and provided valid comparison groups. There were NO baseline data – the above related to (quote): "Finally, we check whether adding some HH characteristics to the specifications affects our impact estimates. Rather than include all of the variables in the PMT score (which would be endogenous and also highly correlated with the PMT score, which is a requirement for inclusion in the generalised IV and RD models), we include a subset of potentially exogenous HH characteristics (such as HH size, education level of the HH head) as controls in the specification." Attrition per relevant group: as there was no baseline and follow‐up of participants in this study, no attrition was reported. Authors reported that since only HHs for which there was no error in the location information were included, the overall response rate was 95.2%. Description of subgroups measured and reported: NR Total number completed and analysed per relevant group: total number of HHs surveyed: 6003; beneficiary 2190; non‐beneficiary 3813 Total number enrolled per relevant group: no baseline data. Total number of HHs surveyed: 6003; beneficiary 2190; non‐beneficiary 3813 (however, there was a potential error in the data (numbers in 'details' column of table 3.3.2 do not equal the numbers in 'number of HHs column') Total number randomised per relevant group: N/A |
|
Interventions |
Intervention characteristics Intervention or exposure
Control
|
|
Outcomes | Dietary diversity: HH DDS (0–12); mother's DDS; child 6‐ to 23‐month DDS; child 24‐ to 59‐month DDS Anthropometry: HAZ; wasted Morbidity: % children aged < 5 years who had diarrhoea in past 4 weeks; % children aged < 5 years who had fever in past 4 weeks Adverse outcomes: overweight |
|
Identification |
Sponsorship source: UK Foreign and Commonwealth Office (UK FCO) Country: Egypt Setting: poor HHs. Districts: the rollout phases were: first wave launched in the poorest 19 districts of 6 governorates in Upper Egypt (Suhag, Assiut, Luxor, Qena, Aswan and Giza); where poverty rate was ≥ 50%. Through the second wave, the programme expanded to districts where poverty rate was ≥ 30%. In the third wave, the programme was expanded further, covering districts where poverty rate was to ≥ 17.9%. Finally, fourth wave opened registration to all districts (MoSS biannual report, December 2016). Author's name: Clemens Breisinger Email: ifpri@cgiar.org Declarations of interest: NR Study or programme name and acronym: Takaful cash transfer programme and Karama cash transfer programme Type of record: report |
|
Notes | ||
Risk of bias | ||
Bias | Authors' judgement | Support for judgement |
Random sequence generation (Selection bias) | High risk | No randomisation done. Study approximated a controlled trial, which used a regression discontinuity design to define allocation to study groups (i.e. according to a threshold for inclusion in the intervention). |
Allocation concealment (Selection bias) | High risk | Allocation was not concealed. Study approximated a controlled trial, which used a regression discontinuity design to define allocation to study groups (i.e. according to a threshold for inclusion in the intervention). |
Baseline characteristics similar (Selection bias) | Unclear risk | Although technically there were no baseline measurements, it was indicated that (quote) "Finally, a key assumption for our estimation strategy is that the households just above and just below the threshold are similar in household characteristics. Annex II Tables A2.1–A2.6 provide evidence that this is indeed the case." |
Baseline outcome measurements similar (Selection bias) | High risk | No baseline data available. Although HH characteristics not related to the outcomes were the same, this might not be the case for the outcomes itself. |
Blinding of participants and personnel (Performance bias) | Low risk | Blinding was not possible but it is unlikely that lack of blinding caused a deviation in how the intervention was implemented. |
Blinding of outcome assessment (Detection bias) | High risk | Unclear if interviewers were blinded. Blinding was not possible. Outcomes were assessed based on data self‐reported by the participants. Participants were aware whether they had been approved to receive transfers or not, and this may have influenced their responses. |
Protection against contamination (Performance bias) | Low risk | Assignment was at HH level, based on the PMT score threshold. The threshold varied over time, after the programme started, so different HHs were enrolled over the different enrolment waves, so that some HHs that did not qualify for the intervention in 1 wave could qualify in the subsequent wave. However, the analyses were based on instrumental variables model, which takes into account the different thresholds over time. |
Incomplete outcome data (Attrition bias) | Low risk | All HHs surveyed were analysed. Quote: "Considering only households for which there was no error in the location information, the overall response rate was 95.2 percent." |
Selective outcome reporting (Reporting bias) | Unclear risk | Authors mentioned a protocol that was submitted to an ethics review board, but we were unable to access it. |
Other bias | Low risk | Misclassification bias of the exposure: low risk. Exposure assigned externally and confirmed with administrative data. Measurement bias: low risk. |