List randomization for soliciting experience of intimate partner violence: Application to the evaluation of Zambia’s unconditional child grant program

Abstract

Social scientists have increasingly invested in understanding how to improve data quality and measurement of sensitive topics in household surveys. We utilize the technique of list randomization to collect measures of physical intimate partner violence in an experimental impact evaluation of the Government of Zambia’s Child Grant Program. The Child Grant Program is an unconditional cash transfer, which targeted female caregivers of children under the age of five in rural areas to receive the equivalent of $24 USD as a bi-monthly stipend. The implementation results show that the list randomization methodology functioned as planned, with approximately 15% of the sample identifying 12-month prevalence of physical intimate partner violence. According to this measure, after four years, the program had no measurable effect on partner violence. List randomization is a promising approach to incorporate sensitive measures into multi-topic evaluations, however more research is needed to improve upon methodology for application to measurement of violence.

1. INTRODUCTION

Social scientists have increasingly invested in research to understand how to improve data quality and measurement in household surveys. In particular, there is ongoing innovation to better capture measurement of sensitive topics, including indicators of intimate partner violence (IPV), which are traditionally thought to be underreported. Survey respondents may under-report or answer untruthfully due to factors related to stigma, shame, social desirability or social pressures, and fear of retaliation or legal action, among others (Blair 2015; Palermo et al. 2014). Because of compounding influences and differences by respondent subpopulations, it is often difficult to understand in a meaningful way the directions and magnitude of bias in a particular sensitive indicator. Additionally, motivation for measurement innovation may stem from the desire to measure sensitive behaviors within the context of a multi-topic evaluation, where little time and resources are available to incorporate ethically challenging topics which are not the primary objective of the data collection.

List randomization (or list experiments), a technique originally used by political scientists in political polling and opinion surveys (Kuklinski et al. 1997a/1997b), has recently been applied to sexual behavior and health. In basic form, the method aggregates a response to a question of sensitive nature alongside responses to non-sensitive questions, thereby masking the specific sensitive response to the interviewer. By randomizing lists containing a sensitive item and lists containing non-sensitive items, in aggregate, researchers can then identify prevalence within a given population. If respondents’ believe their answer remains undisclosed to the interviewer, they may be more likely to report behaviors which are private in nature. List randomization has been shown to increase disclosure to sensitive items, however has also been critiqued for complexity and implementation challenges (Blair 2015). List randomization has recently been applied to solicit attitudes around female genital mutilation/cutting (FGM/C) in Ethiopia (De Cao & Lutz 2015), and sexual behavior including condom use, number of partners and unfaithfulness in Colombia and Uganda (Chong et al. 2013; Jamison et al. 2013). To our knowledge, this technique has never been applied to collect IPV.

We utilize list randomization to collect a measure of IPV in an experimental impact evaluation of the Government of Zambia’s Child Grant Program (CGP), an unconditional cash transfer. The CGP was initiated in 2010 with the overall goal of reducing extreme poverty and curbing the intergenerational transfer of poverty. The CGP was implemented by the Ministry of Community Development, Mother and Child Health (MCDMCH) and operated in Kalabo, Shangombo and Kaputa districts which represent the districts with the highest rates of mortality, morbidity, stunting and wasting among children aged zero to five years. In eligible areas, the CGP transfers a fixed cash sum of approximately US $24 transferred on a bi-monthly basis (US $12 per month equivalent, or approximately 27% of baseline household consumption), to the primary female adult in households with a child under the age of five. While there is evidence that the CGP met its basic poverty-reduction targets, as well as increased women’s financial standing and intra-household decision-making power specifically (Bonilla et al. 2017; Handa et al. 2015; Natali et al. 2016), there was specific interest in understanding potential for impacts on IPV.

Existing literature on cash transfers, primarily from Latin American suggests that cash can play a role in reducing IPV experienced by women in beneficiary households, through reductions in poverty-related stress and increases in women’s status within the household (Buller et al. 2016; Hidrobo et al. 2016; Hidrobo & Fernald 2013). Despite promising evidence, the relationship between cash transfers and IPV is not unambiguously negative—in particular there is a possibility of an increase in IPV in cases where men may seek to gain control over resources or re-assert power (Bloch & Rao 2002; Gibbs et al. 2017). However, to date there are only two quantitative impact evaluations from Africa examining impacts of cash transfers on IPV outcomes: in Kenya, a 12-month unconditional cash transfer is found to decrease 6-month experience of physical and sexual IPV across villages, however has mixed findings comparing within villages (Haushofer & Shapiro 2016); in South Africa a four-year cash transfer conditional on schooling is found to decrease physical IPV among females aged 13 to 20 years at baseline (Pettifor et al. 2016). One reason for lack of evidence is that implementers of already overstretched multi-topic surveys do not have the time or resources to devote to collecting additional indicators—particularly those of a sensitive nature which require additional ethical considerations and referral processes (WHO 2001).

2. METHODOLOGY

The evaluation of the CGP was designed as a cluster-randomized control trial (cRCT) with 90 clusters dived into treatment (45) and control (45). A baseline survey among a random sample of 2,515 beneficiary and non-beneficiary households was carried out in October-November of 2010. Four longitudinal follow-up surveys were subsequently collected after 24, 30, 36, and 48 months. The IPV list randomization was conducted in the 48-month wave which contained a module on women’s empowerment asked of the primary female (CGP beneficiary) in each household. We include in our sample 2,171 women age 15 to 49 among the full sample and 1,617 among the currently married or co-habiting sub-sample who answered the list randomization. Further, we restrict our analysis to females who have data at 48-month follow-up and who were living in study households since the start of the evaluation, for an attrition rate of 13% (p = 0.14 difference between treatment and control groups, calculated at the household level among a sample of baseline households with responses to the women’s empowerment module). Therefore, our estimation of program impacts relies on the successful randomization into treatment and control at baseline, which has been demonstrated elsewhere (AIR 2015). We show overall and differential attrition rates in Appendix Tables 1 and 2 and similar to other publications which have interrogated attrition for the sample of women answering the empowerment module, do not found attrition to be problematic for internal validity (Bonilla et al. 2017; Natali et al. 2016).

The list randomization was implemented as follows: Enumerators were instructed to ask either Panel A or Panel B based on the even/odd numbering of the last digit of the household identification number.

Panel A instructed to read: “Here is a list of four things that some people have done, and some people have not. Please listen to them and then tell me HOW MANY of them you have done or experienced in the last 12 months. Do not tell me which you have done, just tell me how many total. Wait until I have read the entire list to respond.” The list in Panel A included the following (non-sensitive) items:

1. Taken care of a sick relative who is unable to care for themselves

2. Gone to visit one of your child’s teachers to talk about their progress in school

3. Traveled to the district center to buy goods or sell items you produced

4. Attended the wedding celebration of a friend

   Panel B was identical to Panel A, however included a fifth item:

5. Been slapped, punched, kicked, or physically harmed by your partner

Thus, the list comparison is between the group of women randomized to answer the 4-item non-sensitive list, and the group randomized to answer the 5-item list containing the sensitive question about IPV. Subtracting the number of affirmative reported statements in the 5-item group from the 4-item group gives us an estimate of the percentage of women in our sample who report IPV. It is important to note that we ask about physical IPV only, and ask only three behaviorally specific outcomes: (slapped, punched and kicked), in contrast to the typical modified Conflict Tactics Scale implemented in the WHO Violence Against Women surveys and the Demographic and Health Surveys (DHS) (Ellsberg & Heise 2005; Hindin et al. 2008).

We estimate the impact of the CGP on IPV (total number of list outcomes reported), given by the coefficient on the interaction of the CGP treatment indicator and a binary indicator of whether the woman was asked the 5-item list containing the violence question. We run multivariate adjusted and unadjusted models, controlling for age of woman, highest grade completed, marital status, log of household size, and district of residence (baseline values of all covariates) and cluster standard errors at the primary sampling unit level.

3. RESULTS

Table 1 shows the descriptive results of outcome indicators and background characteristics by randomization to the “violence set (5-item list)” or “control set (4-item list).” Descriptive statistics show that the mean age at baseline of the women interviewed is 28.2 with an average of approximately grade four education completion. The majority of women (nearly 3/4 of the sample) were married or co-habiting at baseline, while 14% were divorced/separated or widowed and remaining 12% had never been married or had never cohabited. All covariates were balanced between those who were asked the violence set and control set, with the exception of household size among the married or co-habiting sample (Panel B).

Table 1:

Summary statistics among female recipients of the Child Grant Program by administration of list randomization

Panel A: All women (15–49 years)
	(1)	(2)	(3)
	Full sample	Violence set	Control set
	mean	mean	mean
List randomization (# items reported)	1.906	1.978***	1.832
Age (years, baseline)	28.18	28.01	28.36
Highest grade attained (baseline)	4.019	3.950	4.089
Divorced/separated/widowed (baseline)	0.143	0.135	0.152
Married/cohabiting (baseline)	0.742	0.746	0.738
Log household size (baseline)	1.846	1.842	1.850
Shang’ombo district	0.345	0.344	0.347
Kaputa district	0.316	0.318	0.313
Kalabo district	0.339	0.338	0.340
N	2,171	1,096	1,075
Panel B: Married or co-habiting women (15–49 years)
	(1)	(2)	(3)
	Full sample	Violence set	Control set
	mean	mean	mean
List randomization (# items reported)	1.885	1.959**	1.809
Age (years, baseline)	28.16	27.88	28.44
Highest grade attained (baseline)	3.932	3.957	3.906
Divorced/separated/widowed (baseline)	0.0489	0.0414	0.0565
Married/cohabiting (baseline)	0.906	0.910	0.902
Log household size (baseline)	1.877	1.864*	1.890
Shang’ombo district	0.363	0.367	0.359
Kaputa district	0.354	0.350	0.358
Kalabo district	0.283	0.284	0.283
N	1,617	821	796

Notes: Bivariate t-tests were run to test for differences in covariates at baseline between women asked the violence set (5-item group) and control question set (4-item group);

^***p<0.01;

^**p<0.05,

^*p<0.10

In total, respondents identified 1.9 list items which were true (affirmative answers), with means of 1.978 (SE=0.039) for the 5-item group (violence set) and 1.832 (SE 0.034) for the 4-item group (control set). This indicates that approximately 14.6% (1.978–1.832; SE=0.052) women experienced physical IPV in the last 12-months. This same percentage is 14.95% in the married or cohabiting sample (1.959–1.809; SE=0.0598). The percentage of women asked the 5-item scale was 50.5%, and paired with linkages to household identification numbers, shows enumerators were able to follow instructions regarding the random assignment to groups A and B, and thus the list randomization functioned as intended. However, the statistical validity of list randomization measures depends on a number of assumptions, including absence of design effects. We formally test for this possibility and discuss in the Appendix alongside Table A3, initial results suggest we cannot reject the null hypothesis of no design effects, and thus proceed with analysis under the assumption of no design effects (Blair & Imai 2012). The prevalence of 12-month physical IPV reported in the Zambian DHS 2013–14 among ever-married women age 15–49 is 21.2% (95% confidence interval: 20.0–22.5, CSO et al. 2014), indicating that our estimates are in the range expected according to national statistics. The lower incidence likely stems from the fact that DHS asks seven behaviorally specific questions related to physical IPV [1) pushed, shook, threw something at you, 2) Slapped, 3) Twisted your arm or pulled your hair, 4) Punched with fist or with something that could hurt you, 5) Kicked or dragged or beaten, 6) Strangled or burned on purpose, 7) Threatened you or attacked you with a knife, gun or other weapon].

Table 2 shows the result of the regression analysis with unadjusted (1 and 3) and adjusted models (2 and 4) predicting the number of list randomization questions women answered affirmative for in the full women’s sample and the married/cohabiting sub-group. We find no measurable significant impact of the CGP on IPV in any of the models as shown by the interaction term (Treatment*Asked violence set). Following Holbrook and Krosnick (2010) we run additional analysis with interactions between asked violence set and demographic indicators, and find no change in results (Appendix Table A4).

Table 2:

Impact of CGP on women’s report of intimate partner violence (age 15 – 49 years), using cross-sectional Ordinary Least Squares Regression at 48 months follow-up

	(1)	(2)	(3)	(4)
	All women	All women	Married or co-habiting women	Married or co-habiting women
Treatment (CGP participant)	0.019	0.012	−0.002	−0.011
	(0.112)	(0.111)	(0.125)	(0.124)
Asked violence set	0.134**	0.141**	0.124*	0.128*
	(0.067)	(0.066)	(0.066)	(0.066)
Treatment * Asked violence set	0.028	0.021	0.052	0.043
	(0.100)	(0.100)	(0.102)	(0.102)
Age (years, baseline)		−0.002		−0.001
		(0.004)		(0.005)
Highest grade attained (baseline)		0.022**		0.025**
		(0.009)		(0.010)
Divorced/separated/widowed (baseline)		0.048
		(0.074)
Log household size (baseline)		0.103		0.090
		(0.100)		(0.126)
Shang’ombo district	−0.582***	−0.555***	−0.518***	−0.476***
	(0.152)	(0.153)	(0.162)	(0.163)
Kaputa district	−0.594***	−0.585***	−0.535***	−0.519***
	(0.141)	(0.141)	(0.153)	(0.152)
Constant	2.210***	1.972***	2.188***	1.922***
	(0.122)	(0.247)	(0.142)	(0.288)
Observations	2,171	2,171	1,617	1,617
R-squared	0.056	0.060	0.043	0.047
Joint F-test Asked violence set, Treatment* Asked violence set (p-value)	0.015	0.011	0.017	0.017

Notes: Robust standard errors clustered at the village level in parentheses;

^***p<0.01,

^**p<0.05,

^*p<0.10

4. DISCUSSION

As interest grows in the potential of other economic-based interventions to reduce IPV, innovations in methodology are needed to include and test hypotheses in evaluations not directly set up to measure violence (World Bank, 2015). This could include instances were evaluations are overstretched, and resources cannot be dedicated to logistical necessities (e.g. same sex enumerators) or ethical standards (e.g. ensuring safe referral mechanisms, additional enumerator training). This could also include instances where there is no buy-in from stakeholders for fear of discovering adverse effects, or in situations where IPV is taboo and where low women’s status prevents or threatens credible responses to direct questioning. We demonstrate that list randomization is a credible method to implement in cases were direct elicitation is not possible.

There are several limitations to our analysis. First, the question we implemented to capture IPV was basic, and it is likely that we captured a lower bound of 12-month physical IPV. In addition, although we find no significant impacts on IPV—we cannot reject the possibility that our study lacked power to detect differences in this measure. Further, we were not able to implement a more complex survey design where we collected full standard IPV modules to validate the list experiment, for precisely the same reasons which motivate this paper. Finally, Glynn (2013) suggests methods such as the double list randomization in order to boost the power of the list experiment. These variations and validation efforts are needed to advance our understanding of potential for this method to capture IPV.

Future research efforts should expand on methodology available to measure IPV in non-invasive ways. For example, there are other techniques being developed for reducing bias, such as using non-verbal response cards to minimize stigma (Lindstrom et al. 2015) or validating sensitive measures using qualitative methods (Blattman et al. 2016). Although much more research is needed to improve the quality and reliability of measures, this effort has shown that it is possible to implement a list randomization for IPV in a large multi-topic survey—and that results can fill evidence gaps in understanding program impacts which would not have been possible without this methodological innovation.