Process and Outcome Evaluation of a Community Intervention for Orphan Adolescents in Western Kenya

Abstract

We conducted a 2-year pilot randomized controlled trial (N = 105) in a high HIV-prevalence area in rural western Kenya to test whether providing young orphan adolescents with uniforms, school fees, and community visitors improves school retention and reduces HIV risk factors. The trial was a community intervention, limited to one community. In this paper, we examined intervention implementation and its association with outcomes using longitudinal data. We used both quantitative and qualitative methods to evaluate the community-based model for orphan HIV prevention, with recommendations for future studies. Despite promising effects after 1 year, GEE analyses showed null effects after 2 years. Volunteer community visitors, a key element of the intervention, showed little of the expected effect although qualitative reports documented active assistance to prevent orphans' school absence. For future research, we recommend capturing the transition to high school, a larger sample size, and biomarker data to add strength to the research design. We also recommend a school-based intervention approach to improve implementation and reduce infrastructure costs. Finally, we recommend evaluating nurses as agents for improving school attendance and preventing dropout because of their unique ability to address critical biopsychosocial problems.

Introduction

HIV/AIDS continues to be one of the most serious challenges to global health, with 33.3 million people living with the disease in 2009 [1]. In Sub-Saharan Africa (SSA), where the vast majority of new cases and deaths occur, an estimated 15 million orphans have lost one or both parents [1]. Orphan youth are more likely to face malnutrition [2], school drop-out [3–5], poor psychosocial well-being [6] and earlier sexual debut [7]. Despite recognition of these vulnerabilities, challenges in caring for orphans persist, and orphan groups are only peripherally included in HIV prevention research.

In SSA, a surviving parent, grandparent, aunt or uncle are the main caregivers for orphans, while some exceptionally vulnerable children live in orphan-headed households [8]. Care in orphanages has been deemed expensive and cost ineffective [9], often resulting in worse outcomes for children due to high child-to-staff ratios, lack of social bonding and life skills, and stigma [8]. Although care by biological relatives is preferred, families in high prevalence, low resource communities have become overwhelmed by the numbers and needs of orphans, and remaining HIV-infected parents or other caregivers may themselves die before the orphan reaches adulthood.

We conducted a small 2-year pilot randomized controlled trial in a high HIV-prevalence area in rural, western Kenya. The trial was a community intervention, limited to one community, examining whether providing supports designed to keep orphan children in school could help reduce the children's risk for HIV. The school supports provided to those who were randomly assigned to the intervention group (about half of the study participants) included payment of school fees, school uniforms, school supplies, and services of community visitors. All households with a study participant in the trial (both intervention and control) received mosquito nets, blankets for all children in the household under age 16, and biweekly food supplements.

This trial tested a structural intervention, aimed to promote health by altering the context in which people function, eliminating barriers to health behaviors and facilitating the ability and motivation to make better health decisions [10]. Thus, the intervention focused on contextual factors that affect individual behaviors rather than the individual behavior itself [11]. The intervention is rooted in the Social Development Model [12], which suggests that connections to pro-social adults, peers, and institutions result in better adolescent health and behavioral outcomes. Simply attending school appears to reduce HIV risk factors, particularly among girls (for review, see Jukes, Simmons, & Bundy, 2008 [13]). Growing empirical evidence in SSA suggests that adolescent orphans are at higher risk for early sexual debut and HIV infection [7, 14–16]. School may be a particularly important social environment to prevent HIV among orphans, because it provides a sense of order and routine after the trauma of parental death, critical access to caring adults who can supervise behavior and foster positive norms, and hope for a better future through education.

Only a few studies have rigorously tested school support as a structural intervention to prevent HIV. In a randomized controlled trial of 328 primary schools in Western Kenya, Duflo and colleagues [17] found that providing student uniforms significantly decreased school dropout and marriage for both boys and girls, and pregnancy for girls when compared to providing teachers with comprehensive training to deliver the Kenyan HIV curriculum. In a more recent study, Baird and colleagues [18] experimentally tested a conditional cash transfer (CCT) program with young women (N = 2,893) in Malawi and found reductions in early marriage, teen pregnancy, and self-reported sexual behavior after 1 year of paying secondary school fees for the intervention group and providing a small monthly cash incentive if they stayed in school. In a study of orphan adolescent girls (N = 328), Hallfors et al. [19] tested experimentally whether comprehensive support could keep the girls in school and reduce HIV risk behaviors in rural eastern Zimbabwe. The intervention significantly reduced school dropout by 82% and marriage by 63%, and significantly reduced school absence after 2 years. Compared to controls, the intervention group also trended towards greater school bonding, better future expectations, more equitable gender attitudes, and greater concerns about the consequences of sex.

The current study sought to replicate a model that provided food supplements, mosquito nets, blankets, and school support to orphans in a nearby community. Specifically, we pilot tested the effects of the school support components on HIV risk factors in a small experimental study of orphan boys and girls. The model program (Reach Out) grew quickly over several years with US donor support, including its own primary care clinic and high school, but has never been evaluated. The Reach Out model incorporated volunteer community visitors (CVs) to monitor orphan health and well-being. We included CVs to monitor the intervention group orphans' school attendance, identify problems that would cause the child to miss school, and intervene. Unique to our study, registered nurses from the Moi University School of Nursing (about 83 miles away in Eldoret) supervised the CVs as research collaborators.

After 1 year of exposure to our Kenya intervention, the intervention showed promising findings, such as increased disagreement with early sex (p < .001) and increased bonding with adults in the family (p = .02); the intervention also showed marginally significant reductions in school dropout (p = .05); sexual debut (42% reduction; p = .07); and decreased support of wife beating (p = .07), using one-tailed tests for significance testing [20]. No negative effects were found.

The purpose of the present study is to examine intervention implementation and its association with outcomes using longitudinal data over three time points. We use both quantitative and qualitative methods to evaluate the community-based model for orphan HIV prevention, and we make recommendations for future studies.

Methods

Study Design and Participants

The study sample included 105 children aged 11–14 years at baseline, with one or both parents deceased from any cause, in 79 households in a rural community near Kisumu, identified by local leaders and verified by research staff. All participants were of Luo ethnicity. Consent forms were read and discussed with guardians in each household prior to signing. All protocols were approved by US and Kenya institutional review boards. We conducted stratified random assignment to study conditions by orphan household. First, we assigned the same study condition to all eligible participants in the household, as well as households in close proximity, to avoid the perception of relative deprivation among participants. Then, using baseline survey data, we stratified households by participant sexual debut (yes, no) and gender, and then generated random numbers for random assignment. Fifty-three students were assigned to the experimental and 52 to the control condition. The average age for both intervention and control group youth was 12.9 years at baseline (May 2008); the average grade was 6.04 for intervention and 5.73 for controls (NS); 58.5% of intervention students were girls as were 59.6% of controls (NS); 17% of both groups reported sexual debut [20].

Intervention

All study households, regardless of condition, were given twice monthly food supplements (modest amounts of maize, sugar, cooking oil, and water guard) and mosquito nets and blankets for the children in the home. Intervention group participants also received school uniforms, any necessary school fees, sanitary pads and underpants for girls who had started menses, and monitoring and assistance from a community visitor (CV). CVs were recruited using the following criteria: female gender, positive community reputation, willing to work as a volunteer without pay, and not having one's own child in the study. Primary school education is free except for tutoring fees and pre-high school exam fees. For high school, we had an agreement with the Reach Out Director that intervention students could attend the Reach Out High School on partial scholarships provided by a US foundation supporting orphan students.

Measures

Survey Questionnaire

The annual self-administered questionnaire included items about demographic and outcome variables, using audio computer-assisted self-interviews (ACASI) in either English or Luo. Items were programmed with skip patterns and students could refuse any item. We also obtained information from the schools about enrollment/dropout and reason for dropout, including marriage or pregnancy.

Key study variables were as follows:

• School Absence was measured as a self-reported variable characterizing absence during the past year. Perceptions of Caring Adults measured how much students felt that three key groups (Teachers, Adults in Family, Adults in Community) cared about them. School Dropout was collected from school staff, classmates, and others who could also inform if the participant had moved and enrolled at another school. Educational Aspiration was measured by the level of education the respondent would like to complete. Future Expectations was measured by questions about the respondent's perceived chances of completing secondary school and beyond; making enough money by age 30; and living to age 35.

• Gender Equity Index (alpha = .71) was comprised of five items, such as “Education is more important for boys than for girls,” with higher scores indicating greater disagreement on equitable attitudes. Wife Beating Endorsement is a count of yes responses to five items, such as “Is it okay for a husband to beat his wife if she argues with him?” Three scales measured attitudes towards sex: Disagree with Early Sex (alpha = .65), Waiting (for sex) Because of Values (alpha = .66), and Waiting Because of Consequences (alpha = .76); a single item also measured Disagree with Waiting for Sex Until Marriage. Sexual Debut was measured by the survey item “Have you ever had sexual intercourse?” (coded yes or no). Marital and Pregnancy Status were collected from participants, their families, knowledgeable school staff, classmates, and others to include all events known as of January 1, 2010.

• CV Scores Criteria for evaluating CV performance was developed for a similar study in Zimbabwe and modified for the Kenya study. The criteria addressed seven domains: knowledge of students, addressing needs of students, monitoring adverse events, monitoring school attendance, record keeping, networking, and favorable regard. Three Kenyan research staff members from Moi University, including the primary nurse supervisor, rated them separately and then discussed any discrepancies to determine a consensus rating. Higher CV ratings indicated better performance. We also compared CVs on the number of students assigned, the number of boys versus girls, CV education, and medical attention sought for assigned youth.

Analytic Method

The study was longitudinal with multiple survey measurements of the same students, resulting in a probable violation of the assumption that residuals of the model were independent. To address this lack of independence, we used Generalized Estimating Equations (GEE), which analyze correlated outcomes with reasonable statistical efficiency [21]. The models assessed study condition (intervention versus Control), time, and the condition by time interaction, and CV score controlling for age and gender. We hypothesized that the higher the CV rating, the stronger the association with beneficial study outcomes. Intervention students received their assigned CV score; control students received a 0 (no CV). Due to the small sample size, we used a statistical significance criterion of p ≤ .10, two-tailed test.

Qualitative data were derived from the nurse researchers' regular reports about program implementation, summaries from monthly conference calls with the Kenya team, and additional documentation from investigators and project staff. These data, observations, and reports were reviewed and analyzed to develop a detailed history of project implementation, including themes that would help explain the results.

Results

Table 1 shows the evaluation ratings and individual characteristics of the five CVs. Ten to 12 students were assigned to each CV, with the proportion of boys ranging from 20 to 80%. CV education ranged from Grade 4 to high school diploma with additional vocational training. The CV with the highest educational level had the highest performance score, while the CV with the lowest educational level had the lowest performance score; this relationship was mostly linear. All of the CVs helped some assigned youth to get medical attention. A few youth had chronic problems (e.g., epilepsy) that required repeated clinic visits (see, e.g., CV 5). Supervision by the nurse in weekly meetings helped standardize identification and treatment based on need (see more in qualitative results, below).

Table 1.

Characteristics and evaluation rating scores of the community visitors (CVs)

	CV1	CV4	CV3	CV2	CV5
Assigned students
Number of students	N = 10	N = 10	N = 12	N = 10	N = 11
Number of boys	2 (20%)	4 (40%)	4 (33%)	8 (80%)	4 (36%)
CV evaluation rates
Knowledge	3	3	4	4	5
Needs assessment	3	3	3	3	4
Crisis intervention	2	2	3	4	5
Attendance records	3	3	3	4	4
Performance record	2	3	3	3	4
Networking	3	3	4	4	5
Acceptance	3	3	4	3	5
Average	2.71	2.86	3.43	3.57	4.57
Medical treatment
Total number of clinic visits for assigned children	4	5	22	8	33

Table 2 presents the results of the multivariate GEE analyses, which show null effects after 2 years. Only one marginally significant (p = .09) program effect was found: the control group was more likely than the intervention group to believe they would graduate from college. CVs accounted for four independent effects: educational aspiration, college graduation expectation, wife beating endorsement (if she refuses sex), and waiting for sex until marriage attitude. Of these, only the college graduation item was in the hypothesized direction. Other items favored lower scoring or no CV. As a further check, we tested the model limited to intervention students only and found that effects by CV score were essentially the same.

Table 2.

GEE outcomes reflecting longitudinal exposure over 3 Time points, including baseline

Outcome	Experimental (%)			Control (%)			Intervention: Time × condition (p value)	CV (P value)	Gender: ref = male (p value)	Age (p value)
	Time 1 (N = 53)	Time 2 (N = 52)	Time 3 (n = 52)	Time 1 (N = 52)	Time 2 (N = 51)	Time 3 (N = 49)
School dropout	0	–	2 (3.8%)	0	–	6 (11.5%)	3.66 CI = 0.7 ~19.9 (0.13)	–	2.56 CI = 0.5 ~14.4 (0.28)	1.9 CI = 0.9 ~3.9 (0.07)*
Marriage or Pregnancy	0	–	3 (5.7%)	0	–	2 (3.8%)	0.67 CI = 0.1 ~4.2 (0.67)	–	–	1.73 CI= 0.8 ~4.0 (0.20)
Absenteeism	1.92	1.74	1.87	1.94	1.91	1.79	0.02 (0.90)	0.20 (0.26)	0.25 (0.16)	0.07 (0.34)
Perception of caring adults
Teachers care	4.00	4.07	4.11	3.78	3.74	4.19	−0.14 (0.67)	0.14 (0.41)	−0.04 (0.83)	0.17 (0.03)**
Adults in the family care	3.60	3.79	3.85	3.81	3.39	3.88	0.38 (0.12)	0.07 (0.75)	−0.15 (0.51)	−0.00 (0.99)
Adults in the community care	3.57	3.48	3.57	3.46	3.35	3.80	−0.17 (0.54)	−0.15 (0.47)	0.04 (0.87)	0.04 (0.67)
Educational aspiration	51 (96.2%)	51 (98.1%)	52 (100%)	46 (88.5%)	46 (90.2%)	46 (95.8%)	0.80 (0.36)	−1.81 (0.08)*	1.58 (0.02)**	−0.05 (0.90)
Future expectations
Graduate from secondary schools	42 (79.2%)	40 (76.9%)	42 (80.8%)	39 (75.0%)	34 (66.7%)	43 (87.8%)	−0.31 (0.29)	0.46 (0.14)	0.58 (0.09)*	0.07 (0.61)
Graduating from College	44 (83.0%)	43 (81.1%)	38 (73.1%)	40 (76.9%)	34 (65.4%)	41 (83.7%)	2.72 (0.09)*	1.04 (0.01)*	0.50 (0.15)	0.13 (0.37)
Good salary by age 30	28 (52.8%)	28 (53.9%)	35 (67.3%)	24 (46.2%)	29 (56.9%)	32 (65.3%)	−0.11 (0.68)	0.08 (0.81)	0.37 (0.20)	0.23 (0.04)**
Live to age 35	34 (64.2%)	34 (65.4%)	34 (65.4%)	31 (59.6%)	35 (68.6%)	34 (69.4%)	−0.23 (0.37)	0.33 (0.32)	0.27 (0.40)	0.41 (0.00)**
Gender Equity (α = .71)	2.26	2.79	2.77	2.44	2.85	2.86	0.05 (0.63)	−0.23 (0.28)	0.88 (< .00)***	0.13 (0.04)**
Wife beating: is it acceptable for a husband to beat his wife?
If she neglects the children	31 (58.5%)	31 (59.6%)	36 (69.2%)	30 (57.7%)	35 (68.6%)	32 (65.3%)	0.05 (0.85)	0.13 (0.68)	0.04 (0.84)	−0.22 (0.10)
If she argues with him	33 (62.3%)	21 (40.4%)	23 (44.2%)	30 (57.7%)	17 (33.3%)	18 (36.7%)	0.05 (0.84)	0.37 (0.29)	0.06 (0.83)	−0.10 (0.44)
If she refuses sex with him	5 (9.4%)	1 (1.9%)	1 (1.9%)	6 (11.5%)	7 (13.7%)	2 (4.1%)	−0.65 (0.37)	1.76 (0.04)**	−0.44 (0.36)	0.01 (0.98)
If she burns the food	9 (17.0%)	9 (17.3%)	6 (11.5%)	11 (21.2%)	10 (19.6%)	4 (8.2%)	0.33 (0.26)	0.74 (0.11)	0.23 (0.59)	0.08 (0.59)
For any reason	8 (15.1%)	3 (5.8%)	9 (17.3%)	5 (9.6%)	6 (11.8%)	5 (10.2%)	0.07 (0.87)	0.56 (0.11)	−0.10 (0.80)	0.32 (0.05)*
Attitudes
Disagree with early sex	3.79	4.31	4.21	4.04	3.79	4.14	0.15 (0.12)	−0.02 (0.92)	0.07 (0.61)	0.08 (0.14)
Disagree with waiting for sex until marriage	1.89	1.94	1.87	2.42	2.10	1.84	0.29 (0.13)	−0.37 (0.03)**	−0.22 (0.26)	−0.12(0.13)
Virginity because of values	1.05	1.06	0.98	1.05	1.09	1.00	−0.01 (0.72)	0.01 (0.78)	−0.04 (0.17)	−0.02 (0.13)
Virginity because of consequences	1.08	1.07	1.03	1.21	1.45	1.09	0.04 (0.21)	−0.03 (0.25)	−0.04 (0.29)	0.00 (0.86)
Sexual debut	9 (17.0%)	10 (19.2%)	12 (23.1%)	9 (17.3%)	17 (33.3%)	14 (28.6%)	−0.13 (0.66)	0.34 (0.50)	−0.07 (0.85)	0.31 (0.04)**

Intervention, CV, gender, and age effects include odds ratio and (confidence intervals) for school dropout and marriage. All others include GEE coefficient and (p value).

^*p ≤ .10;

^**p ≤ .05;

^***p ≤ .01, two-tailed test

Gender and age covariates showed important differences. Girls were significantly more likely to report more equitable gender attitudes and higher educational aspirations compared to boys; they were also marginally more likely to believe they would graduate from secondary school (Table 2). With advancing age, youth were significantly more likely to begin having sex, believe in gender equity, have higher future expectations, and believe teachers care about them. They were also marginally more likely to agree that it is acceptable for husbands to beat their wives “for any reason.”

Qualitative Observations

The study was delayed four months because of violence following the contested presidential elections which left 800 people dead and up to 600,000 people displaced [22, 23]. Violence and disruption were intense around the study site because a Luo candidate lost to the incumbent (of the Kikuyu tribe) under controversial circumstances. However, by June 2011 program staff commenced the project by distributing mosquito nets and blankets to all study households following baseline data collection. At the beginning, the neighboring Reach Out Director managed funds and monitored program implementation to establish the replication. He appointed a former Reach Out employee to purchase and distribute food for all households, and uniforms for the intervention group, along with CV supervision. A local village elder, who also served on the Reach Out Board, served as the community arbiter.

Seven community members applied for the CV position and were interviewed; five were selected based on previous experience, education, and proximity to children and schools. Moi University researchers provided a 1-day group training with pre- and post-testing. CV duties included weekly school visits and monthly home visits to monitor assigned children with documentation on standardized forms. Although recruited as volunteers, CVs were given a small monthly honorarium (about $13), reimbursement for travel, and supplies such as a notebook, pen, and umbrella. CVs visited the schools weekly and households at least monthly to identify factors which hindered school attendance. During the school visits, the CVs met with classroom teachers who provided class attendance records, academic performance, and comments about any behavior concerns. If a child missed more than 1 day of school, the CV investigated the reason, visiting the home if necessary.

The Director secured donated land for the project and determined that an office was needed to store the food and provide a space for staff to meet. During the first 8 months, food distribution was sporadic due to food shortages, rising prices, and logistical delays. Uniforms were also slow to be distributed. In December, the elder died after a brief hospitalization. With his death, there was a growing consensus among the community and the research teams that local leadership was needed to improve implementation. A former teacher with strong community credentials was selected to become the Program Director, supported by local leaders serving as a Board of Directors, establishing the program's independence from Reach Out. To further improve implementation, the local Director was to be responsible for food distribution and other services which accrued across both conditions, while the Moi University research team would implement school supports specific to the intervention group. This included supervision of the CVs, who began meeting weekly with the nurse researcher to discuss children who were absent and how to help.

The most common cause of school absence among the intervention group was poor health. If an intervention child was sick, the CV could use emergency funds to address problems resulting in absenteeism (e.g., to pay for a clinic visit and medication for malaria or parasites); the research nurse supervisor would first assess the child and determine whether the child should be taken to clinic. Malaria was very common, with over a dozen documented cases (almost 25%) among the intervention group in the 2 year period. Other health problems included menstrual pain, urinary infections, accidents that needed sutures and antibiotics, jiggers (infested biting insects), dog bite, vaginal infection, epilepsy, and anemia from parasitic worms. Oversight by the research nurses and their advice about health practices were much appreciated by community members who generally had very limited access to health care. Examples of reported illnesses were as follows:

He had some tonsillitis and malaria and was taken to [clinic] where he was given antibiotics and anti malarials. He missed 3 days of school. After completion of the dosage, [child] is now doing well and school is going on as always.

During the last visit, we met [child] who is an epileptic and has been growing weak, she is still weakly but is going on with her anti-malarial medication. She has shown some improvement since the last visit.

[Child] was bitten by a dog and had to get 5 anti-rabies injections which were so costly.

Other factors that prevented orphans from attending school included work:

It was reported that during this planting season, some of the guardians of the children send them to the shambas (farm fields) to plough before going to school. [Child] is one of such children. On the 20^th, she missed school because of the above reason and the CV talked to the guardian about it.

According to the CV [child] is being given too much work at home by the guardian and so gets tired even very early in the morning.

[Child] missed school on the 14^th of April because he was sent to look for cattle by the guardian.

In [household] we found [child] busy working and her very weak grandmother also working on their cotton shamba. The grandmother is not feeling well and her hands were swollen at the time. She says she has to work hard so as to get food for the children. [Child] has no clothing to put on at home and says she does not even have enough panties.'

Some teen orphans were also truant.

From [school], we have [child] who according to the teachers comments during the school visit, she had gotten into relationships with certain male members of the community. She has a close friend, [child] who is also one of the children in the intervention group, whom she associates with and both of them often miss school and in tend to be disobedient. [CV] has since talked to the two and also to the guardians after visiting their homes and she now reports improvement as per the class teachers.

[Child] missed one whole week of tuition (extra lessons), when he later reported back he told the class teacher that he had been sick but was sent back home to bring anything that would prove that he had been sick. At home, there was no evidence of him being sick and it was just a way of malingering school so he went to school the following week and we even paid for the tuition.

After the first year, six intervention students had completed grade 8. Of the six intervention students, three repeated grade 8, one went to the Reach Out High School with intervention support, and two others went to elite high schools with family support. Among the rest of the intervention group (N = 47), six repeated a primary school grade, and one girl dropped out to marry. Three control students repeated a primary school grade; two completed grade 8 and both went on to high school, supported by family.

After 2 years, 13 students in the intervention group (25%) had completed grade 8; five went to Reach Out and seven went to other high schools with intervention support; one went to a high school with family support. Six intervention students repeated a primary grade and two dropped out: one girl got married and one girl got pregnant. Five control students repeated a primary grade and three dropped out of primary school; seven (13%) had completed grade 8; one went to the Reach Out High School on scholarship (intervention crossover), three went to/continued high school with family support, and three dropped out of school.

There were other reports of crossover with control group students accessing services at the Reach Out health clinic. Control group households were aware that others were receiving school support and hoped that by their effort their children could also benefit. Interactions between the CVs and control group parents/guardians were unavoidable in this small tight-knit community.

During their home visits to the intervention children, the CVs would at time meet with parents of the control group and when asked of certain issues concerning their children, said it is the community's tradition not to avoid talking so much and in the moment of talking, they would at one point or another end up getting to know and even try to sort out issues concerning the Control group.

Parents of the control and intervention groups interacted and parents of the controls also decided to urge their children to work hard so that they can get into the intervention group.

From the moment the Control group learned that they did not get school support, they thought it was because they were not working hard in school so they said among themselves they should work hard so they can also be supported

One of the CVs also was at some point heard telling the teachers in school that they should urge all the children in the research to work hard in order to be sponsored even after the end of the project.

Discussion

Although initial outcomes appeared promising [20], the intervention failed to show significant effects after 2 years. A key element of the intervention, CVs, did not improve outcomes despite considerable involvement and assistance. These findings are in contrast to a school-based study of Zimbabwe orphan girls [19], which showed strong and significant reductions in school dropout, school absence, and early marriage after 2 years, as well as marginally significant improvements in posited mediators, using teachers in a similar role.

How then should we interpret these findings? First, we found that conducting a randomized controlled trial in a single tight-knit community was fraught with problems. The intervention offered students uniforms and an opportunity to continue into high school—benefits that were highly visible and deeply valued by families. Although the community welcomed the study and understood the concept of a randomized controlled trial, control group households knew what they were missing, despite our attempts to minimize perceived deprivation. So did staff, who were sometimes persuaded to make an exception, or at least provide hope that control group students could eventually receive the intervention.

Food and other support for orphan households were appreciated but sometimes became a source of strain (e.g., when not distributed on schedule). Appointing a local Director to manage food distribution improved implementation, but ultimately providing food may have undermined the experiment by bringing CVs and households across condition together. In contrast, we randomized schools to condition in the Zimbabwe study, reducing the potential for spillover.

Another factor that weakened the testing of the intervention was the selection of children by age rather than grade. At baseline, we sought to identify households in the community with orphans who were developmentally ready to transition to first sexual intercourse for HIV prevention. But orphans tend to be in lower grades than normal for their age [24], and few orphan participants made the transition to high school during the 2 years of the study. In contrast, we selected Zimbabwe orphan girls in the same upper primary grade across randomized primary schools, and almost all completed 1 year of high school within the first 2 years.

In the present study, only 13% of the control students were eligible for high school over 2 years. Without the higher dropout that accompanies transition to high school, it was difficult to see the effects of an intervention designed to keep children in school as a protective environment. Only two boys and one condition group girl dropped out while in primary school, along with two intervention group girls (all three girls left school to marry), suggesting that the intervention does little to prevent school dropout when tuition is free. Moreover, the economic offset of food supplements, mosquito nets, and blankets, may have helped control group households pay for primary school essentials, such as uniforms. As children moved into high school, greater differences in dropout emerged, but the sample size was too small to detect a significant effect.

At study close, we found that very few children in the study could remain in secondary school without continued assistance. Although the Kenyan government promised free secondary education in 2008, in reality it only subsidizes costs. Even with the government contribution, many poor families cannot manage the required fees, uniforms, and supplies, nor can they forego the opportunity cost of youth labor [25]. High schools are relatively few and can be selective. At the end of Grade 8, all students must take the Kenya Certificate of Primary Education (KCPE) examination; the score determines whether the child will be invited to attend one or more high school. Students with low scores may repeat Grade 8, leave school, or enter a polytechnic or vocational school. Polytechnic school is also costly but lasts only 2 years and boosts chances of employment compared to high school; a college degree is simply out of reach for the vast majority [25].

Given our experience, we offer a number of recommendations. First, randomized controlled trials are generally not suitable for small pilot behavioral intervention studies because of the low budget and short duration. Randomizing households in a single community should likewise be avoided, despite assurances from local leadership and attempts to reduce perceived deprivation. An alternative would have been to randomize communities to condition, but the infrastructure costs to launch multiple community-based intervention sites would be prohibitive. We prefer a school-based intervention because the infrastructure (buildings, staff, accountability processes) already exists. Baird and colleagues [18] reported that implementing their cash transfer and tuition intervention through the community was excessively costly and recommended implementation through the schools. Using cost effectiveness analyses, we found the school-based Zimbabwe orphan girl school support intervention was cost effective for day students (Miller et al. in preparation).

We also recommend that participants be selected by upper primary school grade rather than age, to capture the transition from primary to secondary school, particularly in countries that have free primary but not secondary education. The downside is that some children (and especially orphans) will be older than peers for grade 8 and may have already commenced sexual risk behavior. As we found in the present study, sexual initiation and related attitudes are age sensitive. The remedy is to collect HIV and HSV-2 (genital herpes) biomarker data at baseline and final follow-up to determine whether the intervention prevents these sexually transmitted diseases. HSV-2 is highly prevalent among sexually active women, and to a lesser extent men, in high HIV prevalence sub-Saharan regions (see, e.g., Amornkul et al. [26]). Biomarker data also greatly increases the reliability and validity of measurement compared to self-reported sexual behavior [14].

We also recommend that researchers carefully consider whether they want to include males in their sample. Both the Baird et al. [18] and Hallfors et al. [19] samples were limited to females because the most direct and objective outcomes (marriage and pregnancy, as well as HIV and other sexually transmitted infection) affect women at much earlier ages than men. In the present study, we also found important gender differences on conceptual mediators (education aspiration and gender equity attitudes), but our sample size was too small to examine gender moderation. Given the important policy ramifications of providing school support for girl but not boy orphans, it is essential that at least some studies include males. Those studies must have sufficient sample size (and sufficient duration) to detect key outcomes and should test for gender moderation effects.

In terms of the intervention, we believe that nurses may be particularly useful for implementing this intervention, given the considerable health problems facing adolescents in low-resource countries. The nurses were also able to successfully intervene in public health problems such as excessive child labor and family alcoholism. The nurse home visiting program [27] which has been studied extensively, found that registered nurses were effective in reducing negative outcomes for single mothers and their children, but trained paraprofessionals were not [28]. It may be that nurses have unique educational and experiential preparation to function effectively in this type of public health role. However, the role and cost of nurses must be carefully considered in cost benefit analyses, if positive intervention effects are found.

In summary, we were not able to detect intervention effects in our small pilot study after 2 years. Nevertheless, given our early positive findings, as well as positive effects from other similar intervention studies, we believe that further research is warranted. We do not recommend the community-based approach, however, because of the high infrastructure costs and political difficulties in implementation. For testing, it is critical to capture the transition to high school; biomarker testing should be included for measurement, along with a sufficiently large sample size of both boys and girls. Finally, nurses may be more potent than CVs in monitoring attendance and preventing school dropout, but cost effectiveness must be examined.