A Behavior-Based Intervention That Prevents Sexual Assault: The Results of a Matched-Pairs, Cluster-Randomized Study in Nairobi, Kenya

Abstract

DESIGN

A cluster-randomized, matched-pairs, parallel trial of a behavior-based sexual assault prevention intervention in the informal settlements.

METHODS

Participants were primary school girls aged 10–16. Classroom-based interventions for girls and boys were delivered by instructors from the same settlements, at the same time, over six two-hour sessions. The girls’ program had components of empowerment, gender relations, and self-defense. The boys’ program promotes healthy gender norms. The control arm of the study received a health and hygiene curriculum. The primary outcome was the rate of sexual assault in the prior 12 months at the cluster level (school level). Secondary outcomes included the generalized self-efficacy scale, the distribution of number of times victims were sexually assaulted in the prior period, skills used, disclosure rates, and distribution of perpetrators. Difference-in-differences estimates are reported with bootstrapped confidence intervals.

RESULTS

Fourteen schools with 3,147 girls from the intervention group and 14 schools with 2,539 girls from the control group were included in the analysis. We estimate a 3.7% decrease, p=0.03 and 95% CI=(0.4%, 8.0%), in risk of sexual assault in the intervention group due to the intervention (initially 7.3% at baseline). We estimate an increase in mean generalized self-efficacy score of 0.19 (baseline average 3.1, on a 1–4 scale), p=0.0004 and 95% CI=(0.08, 0.39).

INTERPRETATION

This innovative intervention that combined parallel training for young adolescent girls and boys in school settings showed significant reduction in the rate of sexual assault among girls in this population.

Introduction

Background

Prevalence of sexual assault amongst adolescent girls varies depending upon a number of risk factors including age, geographic location, socioeconomic status, cultural gender norms, and women’s economic dependence on men. Because of the complexity of the contributing factors, interventions to reduce both the prevalence and incidence of violence against women and girls have utilized a wide range of approaches with variable results (Ellsberg et al., 2014). Most interventions have been evaluated in high-income countries with fewer studies in low- and middle-income countries (Ellsberg et al., 2014), and there is a paucity of data specifically addressing sexual assault among adolescents (Lundgren & Amin, 2015). In the informal settlements of Nairobi, Kenya, we have previously shown that approximately one in four high school girls experienced sexual assault in the preceding year, and in the majority of cases the perpetrator was known to the victim, most commonly a boyfriend (Sinclair et al., 2013). In two cross-sectional studies, we found that a six-week classroom-based girls’ empowerment and self-defense program successfully reduced the incidence of sexual assault by a factor of 38%–63% compared to a control group. However, these studies were limited by relatively small sample sizes, retrospective design, and lack of randomization (Sarnquist et al., 2016; Sarnquist et al., 2014; Sinclair et al., 2013).

There is increasing recognition that gender based violence (GBV) prevention efforts need to target boys and men in addition to girls and women (Devries et al., 2013; Ellsberg et al., 2014). We have previously shown that, in the same settlements in Nairobi, a parallel classroom-based educational intervention in adolescent boys successfully improved boys’ attitudes towards girls regarding gender stereotypes and that these changes were sustained one year later (Keller et al., 2015).

Based on our previous findings discussed above, and our belief that this problem is best tackled by addressing boys’ and girls’ needs simultaneously in the same communities, we conducted a trial to test the hypothesis that a six-week classroom-based girls’ empowerment program in parallel with a boys’ educational program would significantly reduce the incidence of sexual assault in the year after intervention compared to the group randomized to receive a standard of care (SOC) life skills class. The study design was a large-scale cluster-randomized controlled trial with primary school students clustered at the school level. This study addresses three key gaps in the current literature: (i) to address the methodological limitations of prior studies, (ii) to broaden the intervention to include both boys and girls taught in tandem, and (iii) to initiate the intervention at an earlier age, among primary school students rather than secondary school students, which may increase the effectiveness by reaching children before they enter the later teenage years, the ages of greatest risk for sexual assault.

Methods

Trial Design

This was a parallel-group, matched-pairs, cluster-randomized study conducted in Nairobi, Kenya from October 2013 to October 2014. Student-participants formed natural clusters at the school level. Clustering and matched pairs were done at the school level. This study used an “open-cohort” design: participants were allowed to enter and exit between baseline and final measurement; measurements were taken on all students within the school at time of measurement at baseline and follow-up. We follow most closely to the cluster-focused, intent-to- treat approach as described in Vuchinich et al. (2012). See “Statistical Methods for Primary and Secondary Outcomes” for discussion of how to interpret the results of an open-cohort, cluster-randomized trial (CRT). No important changes to trial design were made after commencement. This trial was not registered with a major registry.

Participants

Enrolled participants were adolescent girls and boys, attending 30 primary schools in the informal settlements of Nairobi, Kenya, who agreed to undergo the trainings. At baseline the participants were in classes 5, 6 and 7 and were in 6, 7 and 8 at follow-up – thus, though an open cohort design, many were the same participants at both times. Age of girls in this study ranged from 10 to 16 with an interquartile of 12 to 14. Boys in this study were of a similar age distribution as the girls. See Table 1 for baseline covariate information about the intervention and control groups. The schools came from the following informal settlements: Korogocho, Huruma, Dandora, Kibera, and Mukuru. The schools were selected by the implementing partner, Ujamaa-Africa, for the schools’ location in the informal settlements and their school administrators’ willingness to participate in a yearlong CRT of the classroom-based intervention. Schools were also selected so as to be naïve to the intervention, having never received the trainings before.

Table 1.

Self-reported covariates of girls at baseline. Denominator is 2,700 girls at 14 schools in control and 3,406 girls at 14 schools in intervention.

Covariate	Intervention	SOC
Area
Dandora	31%	31%
Huruma	7%	6%
Kibera	33%	24%
Korogocho	17%	24%
Mukuru	12%	15%
Class
5	28%	31%
6	35%	33%
7	33%	34%
8	3%	1%
Missing	1%	1%
Age
mean (years - approximate)	12.3	12.4
Skipped meals
Never	65%	65%
Rarely (1–2× in 4 weeks)	20%	18%
Sometimes (3–10× in 4 weeks)	11%	13%
Often (>10× in 4 weeks)	3%	3%
Missing	0%	0%
Alcohol use
Never	91%	93%
Occasionally	8%	6%
Weekly	1%	1%
Missing	0%	0%
Ever raped
No	90.2%	90.6%
Yes	9.8%	8.9%
Missing	0.0%	0.5%
Raped in previous year
No	92.7%	93.3%
Yes	7.3%	6.4%
Missing	0.0%	0.3%
GSES*
mean (scale 1–4)	3.1	3.1

^*Among non-missing responses.

Interventions

The intervention arm and SOC arm were both classroom-based curricula taught by instructors to students. Curricula for both interventions, IMPower for girls and 50:50 for boys, were developed specifically by No Means No World Wide (NMNW)—a U.S.-based NGO—and focused on the unique needs of younger adolescents in Nairobi. The intervention development process involved an extensive literature review, focus groups, and piloting dozens of classes among the target population.

Both arms were taught from January–March of 2014. The intervention included six two-hour sessions, followed up with booster training sessions within three months. The SOC did not have a refresher course as part of its curriculum design. All sessions had ratios of approximately 1 instructor to 15 students. Both male and female instructors were chosen through an intensive process that ensured that they were respected members of their respective communities and had a background in, and passion for, preventing sexual violence. All trainers received extensive instruction by expert facilitators, and participated in mock interviews and field-training exercises conducted outside of the study area. Trainers were required to pass a rigorous examination consisting of a written test, oral examination, and physical skills demonstration before becoming paid employees teaching the curriculum at intervention sites. New trainers were supervised by a more experienced trainer for their first year of teaching.

Girls’ Intervention

The goal of the IMPower intervention for girls was to empower the girls to avoid risky situations, advocate for themselves, and, if needed, defend themselves against an attack. Learning methods included role-plays, facilitated discussions, and verbal and physical skills practice. In Session I, rapport, definitions, and objectives were established. Session II focused on personal awareness, self-efficacy, boundaries, and assertive communication skills. Session III was an introduction to physical defense. Session IV reviewed verbal and physical skills and focused on specific strikes using bags and mitts. Session V focused on de-escalation and negotiation to avoid fighting and covered more advanced defense techniques, such as multiple or armed attackers. Session VI reviewed all previous sessions, and facilitators also encouraged women to share assault experiences. Survivors were linking to the Sexual Assault Survivors Anonymous program, which holds free weekly meetings in all the informal settlements where Ujamaa operates. This intervention is not radically different from the intervention examined in Senn et al (2015).

Boys’ Intervention

The boys’ intervention, “50:50”, was specifically designed for 10- to 13-year-old boys and focused on promoting gender equality and developing positive masculinity. As with the girls, sessions included role-plays, facilitated discussions, and verbal and physical skills practice. Session topics included developing awareness about gender interactions and negative gender roles, identifying emotions, and skill building around courage and the use of verbal interventions in harassment or assault situations.

Standard of Care

The standard of care group received a one-time 1.5- to 2-hour life skills class, taught by NMNW trainers, covering a wide range of topics such as hygiene, food safety, and personal rights. All school-aged children who attend school typically receive this curriculum. Thus, participants in the intervention arm likely will receive the SOC at some point in their educational careers.

Ethical Considerations

This intervention was a behavior modification program with a low risk of an increase in harm due to the intervention. Surveys were anonymous, so incidences of sexual assault were only identified if the participants decided to disclose to the trainers or other research staff. Ujamaa-Africa instructors and researchers are trained to link students who disclose sexual assault to organizations such as Médecins Sans Frontières and to programs and services provided by Ujamaa-Africa.

Approval for the study in Kenya was provided by the Kenyan National Commission for Science, Technology and Innovation (NACOSTI). The analysis provided by Stanford researchers received a nonhuman subject determination from the Stanford internal review board (IRB). Ujamaa-Africa obtained assent from all study participants.

Primary and Secondary Outcomes

Students in all intervention and SOC schools completed written baseline surveys during class in the first quarter of 2014, before beginning the training programs. The instructors read the questions to the students and then the students were asked to mark their responses. No unique identifying information was requested on the surveys in order to provide anonymity around these very sensitive questions. Once the surveys were completed, the individual student placed her survey into a locked ballot box (Gregson, Zhuwau, Ndlovu, & Nyamukapa, 2002). This level of anonymity meant that there was no way to link baseline surveys to outcome surveys for any particular adolescent. All schools, except one, completed outcome surveys in October 2014.

The primary outcome was change in self-reported incidence of sexual assault at the school level on an annualized basis. Though both boys and girls received training, outcomes were only measured on the girls in the study. At baseline the question was stated as: “In the last one year has anyone forced you against your will to have sex (penetration of your vagina, anus or mouth with a penis or another object)?” On the final survey the question was phrased: “Since you took the Self-Defense Classes has anyone forced you against your will to have sex (penetration of your vagina, anus or mouth with a penis or another object)?”

There were some inconsistencies on some surveys in reporting on sexual assault. For example: a follow-up question regarding a perpetrator may indicate there was an assault even though the girl reported no sexual assault in the prior period. We considered six possible ways of adjudicating inconsistencies: five of the definitions produced qualitatively the same conclusions; only one of the definitions differed slightly by producing a non-significant result. See the appendix for detailed discussion of how these inconsistencies were resolved and the accompanying sensitivity analyses.

Secondary outcomes included: the Generalized Self-Efficacy Scale (GSES), reported perpetrator, whether or not a victim disclosed the assault to anyone, and whom the victim disclosed to. The GSES was created to assess a general sense of perceived self-efficacy with the aim of predicting ability to cope with everyday hassles and ability to rebound from stressful life events. Positive correlations have been found with favorable emotions, dispositional optimism, and work satisfaction. Negative correlations have been found with depression, anxiety, stress, burnout, and health complaints (Gregson et al., 2002). The GSES has 10 questions, each answered on an ordinal scale of 1 to 4. We analyzed the mean score across the 10 questions. If fewer than 10 questions were answered, we used the mean of the answered questions, provided at least 7 were answered, as recommended by the GSES documentation. (Schwarzer, 2014)

Exploratory questions regarding rates of disclosure of sexual assault and to whom the girls disclosed were asked on the survey to assess changes in patterns of disclosure (Table 3). Girls were asked to report the number of times they had been assaulted in the prior period (Figure 2) and who assaulted them (see Table 4 in the appendix). We also asked if the girls used the skills learned in the intervention to prevent a sexual assault, and which type of skills they used in order to stop the assault.

Table 3.

Of those girls who indicated they disclosed their sexual assault, this table reports the distribution to whom the girls reported. The distributions at baseline were not substantially different.

Follow-Up Period
Who did you tell?	Intervention	SOC
Boyfriend	8%	11%
Any relative	17%	17%
Friend/neighbor	38%	48%
Authority figure	17%	9%
Other	20%	15%

Figure 2.

This figure shows the changes in distributions of number of incidences between baseline and follow-up surveys, and how these distributions either did not change much (SOC group) or reduced quite a bit in the lower counts (intervention group). As measured by the Pearson’s chi-squared statistic, the change in the intervention group is significant (p-value approximately 0.016).

Type of Randomization

Prior to assignment to intervention or control, matched pairs were created using information on number of girls in the school, number of boys in the school, academic performance, public versus private school, location, materials used to construct the school, and materials used for the floor. The characteristics and assignments of the matched-pairs design are summarized in Table 2. Each school had an equal probability of being assigned to the intervention. The algorithm is described in detail in the appendix.

Table 2.

Summary of school-level covariates at baseline, from the 28 schools that reported at both baseline and follow-up. GSES is the mean of 10 questions, each on an ordinal scale from 1–4.

Covariate	Intervention Schools	SOC Schools
Area (count)
Dandora	2	2
Huruma	3	2
Kibera	5	3
Korogocho	2	4
Mukuru	3	4
Number of girls in school (mean)	243	193
Ever raped (%)*	11.9%	8.1%
Raped in previous year (%)*	8.3%	6.3%
GSES (mean)*	3.1	3.1

^*Among non-missing responses.

Statistical Methods for Primary and Secondary Outcomes

As in all cluster-randomized trials (CRTs), statistical inference must take into account the impact of individual-level participants being more similar within a cluster than across clusters—that is, the intracluster correlation (ICC). The bootstrap method used for inference accounts for this correlation; see the appendix for a detailed discussion. The clustering also impacts how one should think about the meaning of the targeted estimand of the statistical inference. In this study, there is the additional complication in the study design that we used an open cohort—allowing study participants to exit and enter the study between the baseline and final survey periods.

Given the issues outlined above, one way to understand the estimates are as longitudinal changes on the cluster level—e.g., estimating the change in probability of sexual assault for a school a year after receiving the intervention. Note that open-cohort estimands are useful for answering the question, “What will happen to the rate of rape within a school a year after introducing the intervention, assuming natural turnover in the enrollment of the school?” Assuming that “community immunity” and other interference effects are minimal, a closed-cohort design would be better at answering the question, “What will happen to the probability of rape for a particular girl who received the intervention?”

Our study design and estimation procedure make use of a difference-in-differences estimator detailed in the appendix.

The pre-intervention questionnaires asked about rape in the previous year, but the post-intervention questionnaires asked about the period after the end of the trainings, a period of less than one year. This means that the girls were likely reporting on different exposure times (i.e., 12 months in the baseline, 9 months for the follow-up period). In order to account for this we adopted a bootstrap resampling testing methodology (5,000 resamples with resampling of girls done within school and time period) with an adjustment of the observed proportions in follow-up using a Poisson process approximation for the primary outcome. See the appendix for more details on the estimation procedure. All analyses were conducted using R version 3.1.1.

Methods for Additional Analyses

To assess changes in disclosure patterns, we compared rates of disclosure and to whom the girls disclosed. We also assessed the distribution of the number of times victims reported being assaulted in the prior period, as well as who forced them. The study was not powered to assess these questions, so these analyses should be considered exploratory analyses. We report these rates using observed rates and compare using Pearson’s chi-squared.

Several sensitivity analyses were performed, assessing different assumptions and approximations used in this analysis. We include a permutation-based analysis of the primary outcome which only makes use of the researcher-controlled randomization. See the appendix for more detail.

Results

Participant Flow Diagram (Counts for Primary Outcome)

Losses and Exclusions

After randomization, two schools refused to participate in the study. One school declined to participate citing another NGO’s concurrent activity in the school. No reason was given for the other school. The first school (approximately 100 girls) was assigned to the SOC arm of the study. The second school (approximately 20 girls) was assigned to the intervention. These two schools were both from Dandora. It was coincidental that these two schools had been matched together as part of the randomization process.

Academic administrators in a third school—an intervention school—were unable to allocate time for our trainers to administer the outcomes surveys. It appears the reason for the loss of this school’s follow-up was concern about time, rather than concern associated with the outcomes.

This was an open-cohort study, so individual student-participants were not tracked between baseline and follow-up surveys. Students could exit and enter the study between baseline and follow-up survey periods. While the number of student surveys completed in a given school tended to vary between baseline and follow-up periods, on average schools had 91% as many surveys at follow-up as they did at baseline. Three schools had more surveys at follow-up. Among the 28 schools providing complete paired data, four schools had 75% or fewer at follow-up (two SOC schools and two intervention schools).

Baseline Data

See the appendix for details of how many observations were used in each analysis.

Estimates for Primary and Secondary Outcomes

At baseline, the girls in the intervention schools reported an annualized rate of rape of 7.3%, while the girls in the SOC schools reported a rate of 6.4%. Across all surveys at baseline, there was a report of 6.9%. The point estimate for the study’s primary outcome, the risk difference in self-reported annualized rate of rape due to the intervention, was a reduction of 3.7%, with an associated p-value of 0.030. The 95% confidence interval generated using the Poisson weighted bootstrap was a reduction of (0.4%, 8.0%).

The point estimate of the change in mean GSES going from SOC to the intervention was an increase in GSES of 0.19 (baseline 3.1 on a 1–4 ordinal scale), with an associated p-value estimated to be 0.0004. The 95% confidence interval estimated from the bootstrap method was an increase between (0.08, 0.39).

Ancillary Analyses

As an exploratory analysis, we investigated how the intervention was related to disclosure patterns. If a girl reported being sexually assaulted, we asked if she had disclosed it to anyone. In the baseline period the intervention and SOC groups disclosed at nearly identical rates—63% in the SOC reported and 62% in the intervention group reported. At the follow-up period the two groups differed quite a bit—52% in the SOC and 65% in the intervention, with a p-value of 0.0510. The SOC reported at lower rates in the follow-up period, but the intervention group remained unchanged.

To further investigate reporting patterns, we asked the girls who indicated that they had disclosed their sexual assault to whom they reported. The distribution of whom they disclosed to did not meaningfully change between SOC and intervention, nor between baseline and follow-up (as assessed by a Pearson’s chi-squared test). Table 3 summarizes the observed distribution from the follow-up surveys.

To begin to quantify which kinds of sexual assaults are most impacted by the intervention, we investigated the number of times a girl was sexually assaulted in the prior period (Figure 2). At baseline, the SOC and intervention groups were quite similar in their distribution. In the follow-up period, the distributions were quite dissimilar. The intervention group shifted away from many singleton reports, leaving the distribution weighted to higher number encounters. This is counter to the SOC group, which had very little change in its distribution. This observed shifting, in only the intervention group, is consistent with the intervention having a high impact on reducing “one-time” situations, but perhaps having a lower impact on “high-risk” situations.

In terms of perpetrators, one-off incidences were more likely to be reported as being perpetrated by “Other” or “Friend/Neighbour” while two or more were more likely to be “Boyfriend” or “Any Relative.” A chi-square test of a difference in these distributions produces a p-value of 0.002.

In the intervention group at follow-up, 35% of girls reported using the skills learned in the trainings to stop a sexual assault. Of these girls, they reported using only verbal skills 37% of the time, only physical skills 23% of the time, and both verbal and physical skills 40% of the time.

Several sensitivity analyses were performed to assess the impact of assumptions used in this evaluation; see the appendix for details. The sensitivity analyses did not produce qualitatively different conclusions from those presented in this manuscript.

Discussion

Interpretation

This study evaluated the effect of an empowerment and self-defense training for girls, coupled with a gender-equality training for boys, on reducing sexual assault among the girls participating in the intervention. We also considered the effect the training had on the perpetrator mix, self-efficacy, and skills most frequently used by the girls to prevent sexual assault. We estimate a risk difference of 3.7% in the annualized rate of sexual assault (p=0.030) for girls in the schools that participated in the intervention as compared to the SOC. In addition, there was a significant increase in self-efficacy (0.19, p=0.0004). This cluster-randomized trial addresses the limitations of our previous quasi-experimental studies, while confirming our prior findings that this intervention significantly reduced sexual assault among adolescent girls (Sarnquist et al., 2014; Sinclair et al., 2013). Importantly, the current study demonstrated the effectiveness of the interventions in a younger age group with a lower initial rate of sexual assault.

There have been calls by the international community for high-quality evidence on the effectiveness of programs to prevent sexual violence (Ellsberg et al., 2014; Garcia-Moreno 2005; Michau et al. 2015). This study is a unique addition to this body of literature due to its strong methodological design (a cluster-randomized trial with a large number of clusters) and focus on younger adolescents in sub-Saharan Africa. If the intervention proves to be durable, receiving the intervention at a younger age may decrease the risk of sexual assault as adolescents move into the highest-risk late teen years as well as across their lifespan. Even at this young age, however, the baseline incidence of sexual assault was nearly 10%, suggesting that an even earlier age of intervention may be valuable.

Most of the previous studies on sexual assault (and broader GBV) prevention, as well as the current study, suggest that multi-prong approaches, especially those that include modules on shifting gender norms, are necessary. For example, the IMAGE trial took a multi-pronged approach and showed that a structural intervention focused on gender issues and HIV prevention, combined with a microfinance program, reduced intimate partner violence in a South Africa cohort (Pronyk et al., 2006). In the case of IMAGE, the structural intervention (“Sisters for Life”) had many elements in common with the intervention described here, including regular meetings with trainers to learn about topics such as gender roles and norms, domestic violence, and empowerment, although it was focused on a slightly older population (14–35 years of age) than our study. The 2014 SASA! study in Kampala, Uganda, showed that community mobilization with a focus on changing negative gender norms reduced both physical and sexual intimate partner violence (IPV) incidence as well as acceptance of such violence by both women and men across entire communities (Abramsky et al., 2014). That intervention has many elements in common with our intervention, including a rigorous selection and training process where community leaders are identified and empowered to make a difference in their own communities. It is significantly different, however, in that the curriculum for our study was tightly defined and codified, whereas SASA! encourages the creation of different interventions based on local community needs. Another major violence (and HIV) prevention study in Rakai, Uganda, entitled “SHARE,” which focused on reducing IPV and HIV incidence through a combination of HIV care and community mobilization to improve IPV-related behaviors, also showed a decrease in physical (but not emotional) IPV, as well as a decrease in HIV (Wagman et al., 2015). That study was also in an older population (aged 15–59), but it further supports the need for multi-pronged approaches and curriculum specific to changing gender norms and relationships.

An important finding was the increase in reported self-efficacy among the intervention girls. We measured self-efficacy because we hypothesized that it is an essential intermediary outcome on the pathway to the longer-term impact of reducing sexual assault. This hypothesis was underscored by the two theories that drove the creation of the intervention, as both social learning theory and the Health Belief Model include self-efficacy as a key component of behavior change (Bandura, 1977; Rosenstock, Strecher, & Becker, 1988). Our results show that our intervention was effective at improving self-efficacy, and they further support our hypothesis that increasing self-efficacy may be one effective mechanism to decrease sexual assault in these communities.

Limitations

This study made use of the planned expansion of Ujamaa-Africa’s program into new schools. The study was an add-on, meant to gain as much information as possible while minimally interfering with the natural development of the NGO and its mission. As a consequence there are several major limitations to this study.

Surveys were designed, field tested, translated into Kiswahili, and implemented within the timescale of two months, and were limited to no more than two pages and 30 minutes of classroom time. Thus, survey items were limited.

Also as a consequence of the constraints outlined above, the surveys were read to the class (unisex) by the instructors and completed individually by the girls and boys. The instructors were trained in how to administer questions in both English and Kiswahili. This large group format is suboptimal compared to the preferred one-to-one or small group interviewing. To ensure privacy and mitigate feelings of discomfort responding to sensitive questions, this survey did not collect uniquely identifying information at baseline nor on the final survey. The ballot box method was used, and individuals’ surveys could not be linked between baseline and follow-up periods. On the cluster level, however, the surveys represent longitudinal measurements.

Another challenge this study faces is that the course instructors (from both the intervention and SOC arms) were also tasked with deploying the survey to the same students they instructed. We believe this could have increased the potential for demand effects such as the intervention arm students having felt compelled to report they used the skills to prevent a rape, and also may asymmetrically change reporting patterns. For example, girls in the intervention arm may have felt more comfortable reporting incidences to instructors who demonstrated great care for preventing sexual assault, as compared to students who were taught the SOC by the same instructors.

Here we identify three limitations directly related to the experimental design. First, all measures were self-reported; we plan to measure biological markers such as pregnancy and sexually transmitted infections in future studies. Second, possible cross-contamination between schools and communities is possible, although distance was used in randomization to ensure that intervention and control schools were as geographically distant as possible. Third, the follow-up period was relatively short (nine months). Nonetheless, the large effect size and rigorous design of this study, as well as the focus on younger adolescents, support the further scaling and study of this intervention.

Generalizability

These interventions, IMPower and 50:50, were developed specifically to meet the needs of young adolescents in the informal settlements in Nairobi. Outcomes were only measured on girls, though boys are known to experience sexual assault (Mulawa et al., 2014). Thus, the generalizability of these findings may be limited to girls in other low-income country settings, and especially to high-risk areas, like these settlements. The interventions, however, have solid theoretical underpinnings in social learning theory and the Health Belief Model (Bandura, 1977; Rosenstock et al., 1988), drawing from empowerment, gender relations, and self-defense manuals and best practices from the United States, Israel, and Canada, as well as other areas of sub-Saharan Africa. The authors believe that these interventions can easily be adapted to other, somewhat similar settings, such as the informal settlements in South Africa, and likely, with more significant adaptation, would also be relevant in less impoverished settings and in other regions. The intervention curricula are catalogued in detailed, referenced manuals, which should support the process of adaptation to novel settings.

The intervention is tailored to delivery in a classroom environment. There are potential benefits to this delivery that would not be present in an individual-based training program (e.g., larger cross-section of student-age population, “group immunity”). It is also possible that there are instructor-level effects that may vary in different settings. For example, the Ujamaa-Africa instructors are highly specialized in delivering this curriculum and are chosen for their passion for preventing GBV; one might imagine that having primary school teachers provide the same curriculum may have a different impact. Other sexual assault prevention programs can be used for both in-school and out-of-school youth (Jewkes et al., 2014).. Out-of-school youth may be even more of a vulnerable population than in-school.

Conclusions

This study showed that this intervention can significantly reduce sexual violence in a highly susceptible population and confirms that the intervention is effective in younger adolescents in whom the prevalence of GBV is lower. While the results of the current study replicate and expand upon our earlier findings in the same region of Kenya, they need to be replicated and scaled in other settings and in other countries.

Figure 1.

Participant flow diagram for this study. See “Losses and Exclusions” for more discussion.

Appendix

Primary and Secondary Outcomes

Students in all intervention and standard of care (SOC) schools completed written baseline surveys during class in the first quarter of 2014, before beginning the training programs. The instructors read the questions to the students and then the students were asked to mark their responses. No unique identifying information was requested on the surveys. Once the surveys were completed, the individual student placed her survey into a locked ballot box (1). This level of anonymity means that there is no way to link baseline surveys to outcome surveys for any particular adolescent. All schools, except one, completed outcome surveys in October 2014.

Technically speaking, the primary outcome was change in self-reported incidence of sexual assault at the school level on an annualized basis. Though both boys and girls received training, outcomes are only measured on the girls in the study. Due to the sensitive nature of reporting such information, anonymous surveys were used. At baseline the question was stated as: “In the last one year has anyone forced you against your will to have sex (penetration of your vagina, anus or mouth with a penis or another object)?” On the final survey the question was phrased: “Since you took the Self-Defense Classes has anyone forced you against your will to have sex (penetration of your vagina, anus or mouth with a penis or another object)?”

There were some inconsistencies on some surveys in reporting on sexual assault. For instance, a girl may have answered that she had never been raped but also answered that she had been raped in the prior 12 months. The definition that was used for the primary outcome, rape in the prior year (or since the course), was as follows: If the question about rape in the prior year was answered yes or no, that answer was used. If the prior year answer was missing and the answer was no for being raped ever, then the prior year answer was set to no. Otherwise, we implemented a “preponderance” algorithm. There were four other questions on the survey that could be used to indicate a rape had occurred. If the answers to those questions implied an answer of yes or no to the main question, based on having at least two more answers implying one direction than in the other, we imputed that answer. Before imputation, 262 of 12,094 surveys had no answer to the yes/no question for the prior year. After imputation, only 17 surveys still had no answer. Of the 245 imputed values, 232 were no and 13 were yes. For the question about ever being raped, a similar procedure was used, except that the first step was that if the answer to the question about rape in the prior year was yes, then the answer for ever was marked as yes. Of 220 surveys with no answer initially, only 28 had no answer after imputation. Of the 192 imputed values, 163 were no and 29 were yes.

Secondary outcomes included the generalized self-efficacy scale (GSES), reported perpetrator, and whether or not a victim disclosed the assault to authorities. The GSES scale was created to assess a general sense of perceived self-efficacy with the aim of predicting ability to cope with everyday hassles and ability to rebound from stressful life events. Positive correlations have been found with favorable emotions, dispositional optimism, and work satisfaction. Negative correlations have been found with depression, anxiety, stress, burnout, and health complaints (2). The GSES has 10 questions, each answered on an ordinal scale of 1 to 4. We analyzed the mean score across the 10 questions. If fewer than 10 questions were answered, we used the mean of the answered questions, provided at least 7 were answered, as recommended by the GSES FAQ (3).

Exploratory questions were asked on the survey to assess changes in reporting patterns; questions regarding rates of reporting and to whom the girls reported were asked (Table 3). Girls were asked to report the number of times they had been assaulted in the prior period (Figure 2) and who had assaulted them (Table 4). We also asked if the girls used the skills learned in the intervention to prevent a sexual assault, and which type of skills they used in order to stop the assault.

Costs and cost-effectiveness were not recorded or calculated in a standardized way.

Type of Randomization

Prior to assignment to intervention or control, matched pairs were created using information on number of girls in the school, number of boys in the school, academic performance, public versus private, location, materials used to construct the school, and materials used for the floor.

The study’s statistician used a nonbipartite matching algorithm to find optimal matched pairs (4). The characteristics and assignments of the matched-pairs design are summarized in Table 2. A binary vector of length 16, representing the 16 matched pairs, was created using the sample() function in R. A 1 (intervention) or a 0 (SOC) was sampled for each of the 16 entries in the vector, with the probability of sampling a 1 being ½. This approach ensures that each school had an equal probability of being assigned to the intervention.

Statistical Methods for Primary and Secondary Outcomes

As in all cluster-randomized trials (CRTs) statistical inference must take into account the impact of individual-level participants being more similar within a cluster than across clusters—that is, the intracluster correlation (ICC). The clustering also impacts how one should think about the meaning of the targeted estimand of the statistical inference. In this study, there is the additional complication in the study design that we used an open cohort—allowing study participants to exit and enter the study between baseline and the final survey periods.

Given the issues outlined above, one way to understand our targeted estimand is the change (due to the intervention) in the probability of a randomly sampled girl reporting being raped in the prior 12 months. Note that this open-cohort estimand would be useful for answering the question, “What will happen to the rate of rape within a school a year after introducing the intervention, assuming natural turnover in the enrollment of the school?” Assuming that “community immunity” and other interference effects are minimal, a closed-cohort design would be better at answering the question, “What will happen to the probability of rape for a particular girl who received the intervention?”. The interpretation provided here is also imposed by the fact that the surveys were designed to be anonymous, making linking of baseline and follow-up surveys for a particular girl impossible.

Our study design and estimation procedure make use of a difference-in-differences estimator. Difference-in-differences estimators make use of two types of contrasts: (i) the difference between the baseline and follow-up period and (ii) the difference between the SOC group and the intervention group. The first contrasts help to reduce variation in the outcome by only examining the change through time. For example, when compared to all other schools in the study, a particular school may have a high rate of reported rape on the follow-up surveys—say 10%—but this may actually be a reduction from that particular school’s baseline survey—say 12%. By differencing baseline and follow-up survey information, we see that this hypothetical school actually decreased by 2%. One can think of this step as “accounting for variation at baseline,” reducing the importance of school-specific features in our estimation procedure. But these two contrasts alone may not be enough information. It may be the case that both the intervention group and the SOC group experienced increases in the reported rate of rape. This would not be surprising in our study population, as the girls are a year older at follow-up than at baseline and are likely aging into a more difficult period with higher rates of rape. The second contrast is to take the first two differences – (i) intervention follow-up minus intervention baseline and (ii) SOC follow-up minus SOC at baseline – and subtract (ii) from (i). When there is randomization, this contrast of contrasts isolates the effect of the intervention. One can think of there being two types of controls—the SOC and the baseline measurements—each serving to reduce different types of statistical noise.

The pre-intervention questionnaires asked about rape in the previous year, but the post-intervention questionnaires asked about the period after the end of the trainings, a period of less than one year. This means that the girls were likely reporting on different exposure times (i.e., 12 months in the baseline, 9 months for the follow-up period). In order to account for this, we adopted a bootstrap resampling testing methodology (5,000 resamples) with an adjustment of the observed proportions in follow-up using a Poisson process approximation for the primary outcome. In the original data and each resampled dataset, we calculated the observed proportion who answered yes for each school and time period. For the follow-up time periods, we calculated adjusted rates in each school by setting the proportion with no events equal to the probability of zero events from a Poisson process, e^−λτ where λ is the rate and τ (< 1 year) is the length of the time interval in years. We only used the answer to the simple yes/no question to calculate the rate, because we believed the answers to it were more trustworthy than the answers to the question about number of events. Using the calculated rate, $\widehat{\lambda }$, we then estimated the probability of no event during one year by $e^{-\widehat{\lambda }}$ and hence the probability of at least one event by $(1-e^{-\widehat{\lambda }})$. We then calculated the estimated change in probability at each school, the difference in changes for each matched pair, and then the mean of the difference across all pairs. In our primary analysis we used a value for τ of 9 months (0.75 years) in both arms. Please see “Ancillary Analyses” for a brief discussion of a sensitivity analysis performed related to this Poisson process approximation.

To reflect the clustered nature of the data, each bootstrap resample was created by first resampling with replacement among the matched pairs of schools and then resampling girls with replacement within school and time period (including girls with missing data for the variable of interest). Percentile bootstrap confidence intervals and p-values were calculated from the values of the mean difference of differences in the resamples. Resampling pairs provides estimates that correspond to random/mixed effects for schools. Resampling students only would correspond to an analysis with fixed effects, and yield less-variable results for the bootstrap replications. Note that the two schools that dropped out between allocation and the beginning of the trainings were in the same matched pair, so there was no loss of data because the outcomes were analyzed based on the difference in changes in matched pairs. The school that dropped out and the school matched to it do not provide data for the estimation of the effect in the primary analysis.

The issues above relating to follow-up interval do not arise for GSES since it is based on feelings at a specific point in time. We used the same bootstrap difference-of-differences test, but with mean scores rather than proportions and without the Poisson process adjustment.

Methods for Additional Analyses

To assess changes in reporting patterns, we compared rates of disclosure and to whom the girls disclosed. We also assessed the distribution of the number of times a girl reported being assaulted in the prior period, as well as who forced them. The study was not powered to assess these questions, so these analyses should be considered exploratory analyses. We report these rates using observed rates and compare using Pearson’s chi-squared.

As a form of sensitivity analysis, we also estimated the change in annual incidence of self-reported sexual assault due to the intervention using a generalized linear mixed model. The unit of observation in this model is a girl. The parameter of interest is the change, due to the intervention, in a girl’s probability of reporting being raped in the prior 12 months. The outcome is binary (i.e., sexually assaulted in the prior 12 months? yes/no), so a mixed-effects logistic model was used. Fixed effects were used to account for location (informal settlement) and age of girls. To account for clustering at the school level, a random effect for school affiliation was used in the model. To account for the matched-pairs structure of the design, a random effect was used. This approach allowed us to include girls from all 30 schools. However it was naïve in that it didn’t account for the shorter time period covered in follow-up.

Because the training in the intervention arm was longer than that in the SOC arm, it is possible that the length of time from the end of training to filling out the questionnaires was shorter in the intervention arm than in the SOC arm. A second sensitivity analysis was performed using our bootstrap/Poisson approach, but assuming that answers in the SOC arm covered 10 months while those in the intervention arm covered 9 months.

Results

Numbers Analyzed

For the control group there were 2,700 surveys at baseline and 2,539 at follow-up (ratio of 0.94) in the 14 schools paired with intervention schools at both baseline and follow-up. In the intervention group there were 3,406 at baseline and 3,147 at follow-up (ratio of 0.92). These provided the results for our primary analyses.

Effects for school and pair were included for the sensitivity analysis using mixed models, so we were able to use all of the surveys. For the mixed-model sensitivity analysis, in the control group there were 15 schools at both times with 2,827 surveys at baseline and 2,591 at follow-up; in the intervention group there were 15 schools with 3,529 surveys at baseline and 14 schools with 3,147 surveys at follow-up.

Ancillary Analyses

The distribution of perpetrators was not different at follow up between the intervention and control arms. We report the distribution of perpetrators from the follow-up surveys in Table 4. Though there were some changes from baseline to follow-up, these changes were not in a discernable pattern nor statistically significant.

As outlined in “Methods for Additional Analyses,” to assess the method of inference we also ran a generalized linear mixed-effects model as a contrast to the bootstrap with Poisson approximation. In a model with fixed effects for time period, intervention, the interaction of time and intervention (equivalent to our difference in differences), area (a.k.a. location or unplanned settlement), and age and random effects for pair and school, the p-value for the interaction was 0.033. Thus, the conclusion drawn from this model was qualitatively similar to our primary method discussed in “Statistical Methods for Primary and Secondary Outcomes,” though this model fails to account for the discrepancy in exposure periods between the baseline and follow-up survey questions.

We also performed a sensitivity analysis examining the impact of the Poisson process approximation. We considered the potential difference in exposure periods between the intervention and SOC arms in the interval between the end of training and the follow-up period. The SOC was quite short, one class. The intervention tended to last six months or more. Taken literally, the primary outcome question at follow-up asked if a rape had occurred after the end of trainings. This means there could have been a differential in exposure periods between intervention and SOC. Thus, we re-ran the analysis outlined in “Statistical Methods for Primary and Secondary Outcomes,” except using 9 months in the intervention and 10 months in the control. This sensitivity analysis produced a nearly identical estimate of the treatment effect, but the sensitivity analysis had a p-value of 0.064 as compared to 0.030 for the main analysis.

The final inference-related sensitivity analysis we performed was a permutation-based analysis of the treatment effect at the cluster level. Using only the researcher-directed randomization as the warrant for inference, we performed a matched-pairs Wilcoxon ranked-sign analysis of the difference-in-differences for the two primary outcomes: (i) raped in prior 12 months and (ii) GSES. We used both the Poisson-adjusted and unadjusted outcome rates for “raped in prior 12 months.” There were no qualitative differences in conclusions: effect sizes were unchanged and confidence intervals were slightly larger but not so large as to include the null-effect.

We also performed a sensitivity analysis of the primary outcome by examining several definitions of what constituted a report of rape. Some surveys provided contradictory responses—e.g., one response indicating the girl had never been raped, but another response from the same girl indicating she had been raped several times in the last year. We considered six possible ways of adjudicating these inconsistencies: five of the definitions produced qualitatively the same conclusions; only one of the definitions produced a non-significant result. The one non-significant result was the most “hair-trigger” definition, switching someone to indicating a sexual assault should any possible sub-question suggest this as a possibility, even if the main question and all the other sub-questions indicated no assault. It is likely this definition produced too many false-positives on both sides, introducing measurement error which attenuated the underlying signal.

Miscellaneous

Sample-Size Calculation

The planned study size was limited to 32 schools due to the NGO’s financial and logistical constraints. A power calculation was run using the CRTSize package in R (5). Given the primary outcome is binary, the n4props() function from CRTSize was deployed. Using an alpha level of 0.05, an allocation strategy of an equal number of intervention and SOC clusters, and the experimental group proportion of 0.05 versus the control group proportion of 0.07 (i.e., a risk difference of 0.02), we then assumed an intraclass correlation of 0.001 and an average cluster size of 200 girls. The assumed risk difference and intraclass correlation were chosen based on prior experience with the intervention. Under a one-tailed test, this provided an estimated power of 0.83 for this study. Under identical assumptions, a study with 14 clusters in the intervention group and 14 in the control group would have an estimated power of approximately 0.79.

Table 4.

Of those girls who reported being sexually assaulted in the prior period on the follow-up surveys, this table reports the distribution of perpetrators as identified by the girls. If multiple assailants, the girls were instructed to report the most recent assailant. Our “Other” category combined three original categories: “Stranger”, “Gangster”, and “Other”. Overall, “Other” was 67% “Stranger”, 6% “Gangster”, and 27% “Other” unspecified.

Follow-Up Period
Who forced you?	Intervention	SOC
Boyfriend	30%	32%
Any relative	10%	13%
Friend/neighbor	30%	32%
Authority figure	9%	5%
Other	21%	19%