Process Evaluation of a Clinical Trial to Test School Support as HIV Prevention among Orphaned Adolescents in Western Kenya

Abstract

Orphaned adolescents are a large and vulnerable population in sub-Saharan Africa, at higher risk for HIV than non-orphans. Yet prevention of new infection is critical for adolescents since they are less likely than adults to enter and remain in treatment, and are the only age group with rising AIDS death rates. We report process evaluation for a randomized controlled trial (RCT) testing support to stay in school (tuition, uniform, nurse visits) as an HIV-prevention strategy for orphaned Kenyan adolescents. The RCT found no intervention effect on HIV/HSV-2 biomarker outcomes. With process evaluation, we examined the extent to which intervention elements were implemented as intended among the intervention group (N=412) over the 3-year study period (2012–2014), the implementation effects on school enrollment (0–9 terms), and whether more time in school impacted HIV/HSV-2. All analyses examined differences as a whole, and by gender. Findings indicate that school fees and uniforms were fully implemented in 94% and 96% of cases, respectively. On average, participants received 79% of the required nurse visits. Although better implementation of nurse visits predicted more terms in school, number of terms did not predict the likelihood of HIV/HSV-2 infection. Attending boarding school also increased number of school terms, but reduced the odds of infection for boys only. Four previous RCTs have been conducted in sub-Saharan Africa and only one found limited evidence of school impact on adolescent HIV/HSV-2 infection. Our findings add further indication that the association between school support and HIV/HSV-2 prevention appears to be weak or under-specified.

An estimated 35 million people are infected with HIV globally, including 2.1 million children and youth, aged 10–19 years (Idele et al., 2014). According to the World Health Organization, Sub-Saharan Africa (SSA) accounts for 25.8 million people living with AIDS in 2014 and almost 70% of new cases (AIDS.gov, 2016). Although incidence decreased among all age groups between 2000 and 2015 (Joint United Nations Programme on HIV/AIDS [UNAIDS], 2015a, 2015b), prevention efforts in SSA continue to be critical, particularly for adolescents. Infected SSA adolescents are significantly less likely than adults to enter, comply with, and maintain in treatment (Bakanda et al., 2011; Jani et al., 2011; Lamb et al., 2014; Nachega et al., 2009; Weigel et al. 2012). Further, adolescents aged 10–19 years are the only age group in which AIDS deaths have risen between 2001 and 2012 (United Nations Children’s Emergency Fund [UNICEF], 2016).

Orphaned adolescents are a large and vulnerable population in SSA, particularly in countries that have been hardest hit by HIV. For example, more than one in five (21%) of all children in Zimbabwe are orphans (i.e., less than 18 years with one or both parents deceased) (UNAIDS, UNICEF, & United States Agency for International Development [USAID], 2004), and among those aged 15–17 years, 41% are orphaned (Zimbabwe National Statistics Agency [ZIMSTAT] & ICF International, 2012). Further, Zimbabwean orphaned adolescent girls are significantly more likely than non-orphans to have experienced sexual debut, child marriage, and pregnancy, and to be HIV-infected (Luseno, Zhang, Rusakaniko, Cho, & Hallfors, 2015). A recent cash transfer study in Malawi found that orphans disproportionately fall to the lowest rungs of the economic ladder, comprising 15% of the child population but accounting for 66% of “ultra-poor” households (Luseno, Singh, Handa, & Suchindran, 2014; National Statistical Office & UNICEF, 2008). Recent estimates in Kenya suggest that 10% of Kenyan children have lost one or both parents, with a larger proportion (18.2%) in the western region formerly known as Nyanza Province, where HIV prevalence was among the highest in the country (Kenya National Bureau of Statistics, 2014).

Over the past decade, there has been a growing interest in determining whether keeping adolescents in school, especially adolescent girls, can help to mitigate the risk of HIV (Jukes, Simmons, & Bundy, 2008; Pettifor, MacPhail, Nguyen, & Rosenberg, 2012). The most common intervention approach has been cash transfer programs (Fiszbein & Schady, 2009). Behavioral economists theorized that providing the poor with school subsidies through various cash transfer schemes would improve socioeconomic status (SES), thereby providing better access to food and improved educational opportunities that would positively affect expectations for the future and preferences for healthier lifestyles (Fiszbein & Schady, 2009; Pettifor et al., 2012).

Cash transfer approaches typically provide cash to parents and, in some cases, youth, with or without the explicit requirement that youth stay in school (Luseno, Singh, et al., 2014). Other variations, such as the intervention described here, have supplied uniforms and paid tuition and exam fees directly to schools, testing the concept that because school is a structured, prosocial environment, extending school enrollment through high school would improve future expectations and health behaviors, including delaying sexual debut and reducing sexual infection risk (Cho et al., 2011; Hallfors, Cho, Rusakaniko, Iritani, Mapfumo, & Halpern, 2011; Hallfors et al., 2015). The studies cited above focused on orphans, who were expected to benefit more from school support, as well as from the structured setting of the school environment, than non-orphans, because of their greater vulnerability (Hallfors, Cho, Mbai, Milimo, & Itindi, 2012; Luseno et al., 2015).

Five experimental trials have tested the effects of conditional cash transfer schemes on HIV prevention in sub-Saharan Africa. Most have resulted in improved school enrollment and attendance for intervention group participants compared to controls, and lowered various HIV risk factors. But none have found main effect changes in HIV or HSV-2 (genital herpes) biological markers (Cho, Mbai, Luseno, Hobbs, Halpern, & Hallfors, 2017; Duflo, Dupas, & Kremer, 2015; Hallfors et al., 2015; Pettifor et al., 2016), although one study (Baird, Garfein, McIntosh, & Ozler, 2012) reported a significant effect on HIV biomarkers among girls enrolled in school at baseline. That cluster randomized trial provided a combination of modest conditional cash payments and high school tuition for young women (ages 13–22), most of whom were in school at baseline (Baird et al., 2012). About half of the original sample were tested for HIV and HSV-2 after 18 months, although none were tested at baseline. When the analysis was limited to school girls only, the investigators found a treatment effect on HIV biomarkers, but not on HSV-2 or HIV-related factors such as marriage, pregnancy, sexual debut, or unprotected sexual intercourse. A second study, in South Africa, enrolled 2,533 school girls, aged 14–17 years at baseline, randomized to the intervention or control group (Pettifor et al., 2016). Conditional on school enrollment, cash transfers were given to intervention group girls ($10/month) and their parents ($20/month). HIV and HSV-2 biomarkers were collected at baseline and endline. No intervention effects were found on HIV or HSV-2, nor were differences found on school enrollment, but the intervention group was less likely to report any sex partner in the past month, physical violence from a partner, and unprotected sex (past 3 months) compared to the control group.

A third large study (Duflo et al., 2015) randomly assigned 328 Kenya primary schools (comprising almost 20,000 boys and girls in Grade 6) to four conditions: no treatment control (82 schools); education subsidy (providing all sixth graders in 83 schools with school uniforms); enhanced training program (week-long preparation for 3 teachers at each of 83 schools to better implement the Kenya HIV prevention curriculum); and a joint program, that included both the education subsidy and the enhanced training program (80 schools). Investigators found no difference in HIV or HSV-2 biomarkers between the education subsidy intervention and control groups, although there were fewer pregnancies among the education subsidy group. The joint program, however, showed a marginally significant effect on female HSV-2 biomarkers (reducing HSV-2 by 2.3 percentage points; p<.10), but no effect for males.

The fourth study was a clustered randomized trial of 25 primary schools, including 328 orphan girls (ages 10–16 years) identified in Grade 6 (Hallfors et al., 2011). Intervention participants received school tuition, uniforms, and school supplies. After three years, the intervention group was significantly more likely to be enrolled in school, less likely to be married, and had better quality of life than controls (Miller, Hallfors, Cho, Luseno, & Waehrer, 2013). Given the findings, the control group was offered a partial intervention consisting of school fees. After five years (but not at baseline), HIV and HSV-2 biomarkers were collected. No difference on either HIV or HSV-2 infection was found, although the intervention group continued to be more likely to be in school and less likely to be married, and SES (measured as meals/day) was greater among the intervention group (Hallfors et al., 2015).

The fifth (and current) study was conducted in Siaya County in western Kenya, noted for its high HIV prevalence (National AIDS Control Council of Kenya, 2014). Study design was informed by earlier pilot work with process evaluation suggesting five improvements: randomization of schools rather than households, no treatment for the control group; selecting orphans in Grades 7 and 8 rather than by age; registered nurse visits rather than lay community visitors; and HIV/HSV-2 biomarkers at baseline and endline (Hallfors et al., 2012). The subsequent randomized controlled trial (RCT; parent study) incorporated all of these design modifications.

We hypothesized that the longer orphaned youth remained in school, the greater the positive effect on HIV-related behaviors: delayed sexual debut, marriage, and pregnancy; increased condom use; reduced transactional sex; and increased male circumcision. We expected positive impacts on risk factors would result in reduced incidence of new HIV/HSV-2 infection three years after baseline. We also expected improved health-related quality of life, due to the economic value of school support, the visits from the nurses, and the positive mental health effects of being in school and avoiding risk behaviors. The only behavioral HIV-related benefits found, however, were increased male circumcision and reduced transactional sex (Cho et al., 2017). We also found some quality of life differences between the two conditions, with the intervention group less likely to report problems with anxiety and depression and the ability to perform usual activities, but no group difference on the full 5-item quality of life scale (Jelsma, Hansen, De Weerdt, De Cock, & Kind, 2003). The present paper reports on findings from the process evaluation of this RCT.

A key research question for process evaluation is the extent to which intervention elements were implemented as intended. This question is particularly important in the present study as we attempt to understand mostly null effects. A second question is whether any of the elements are more or less important in predicting school enrollment. Third, we address the important conceptual question of whether longer or more comprehensive school attendance among the intervention group (holding constant the availability of school support) leads to reductions in HIV/HSV-2 incidence and whether outcomes differed by gender.

Methods

Study Design and Participants

We analyze data from intervention group participants in a clustered RCT, conducted in western Kenya from 2011–2014. Twenty-six primary schools in the area with at least 20 orphans in grades 7 and 8 were selected to draw the study sample, comprising 837 children of mostly Luo ethnicity, with one or both parents deceased from any cause. Primary schools were randomly assigned to study condition (13 control and 13 intervention), with 412 participants in the intervention group at baseline. Three years later at final data collection, one intervention participant had withdrawn from the study and three had died.

Human Subjects Protection

Study protocols were approved by IRBs at PIRE (USA) and Moi University (Kenya). Participation in the study required written parent/guardian permission and participant assent. When participants reached 18 years of age, researchers reviewed study risks, benefits, and procedures and obtained written consent from them as adults for continued participation.

Intervention

Intervention participants received school fees, uniforms, and nurse visits, which continued for three years (2012–2014), or until the student dropped out of school. For primary school students, we paid the Kenya Certificate of Primary Education (KCPE) exam fee at Grade 8. Once transitioned to secondary school, we paid fees each term (three/year), based on the school’s official fee schedule. We gave one primary school uniform to all students in Grades 7 and 8 in 2012. When students transitioned to secondary, we gave them a uniform for their new school. Nurse research assistants (RAs) were to visit the majority of intervention students monthly at local schools to monitor attendance and address problems leading to dropout. For students at remote schools, they were to communicate regularly with school personnel and see students once each term when paying fees. However, by the end of the first term in 2012, we recognized that monthly nurse visits were not feasible (see Implementation Analysis), and so changed the protocol to at least one contact each term for all intervention participants, with priority to those in school.

Measures

The implementation data used to respond to our research questions came from a variety of sources, primarily payment documentation, monthly staff reports, and staff logs. These data were merged with annual survey data from the parent RCT, collected using audio computer-assisted self-administered interviews on personal digital assistant devices. We used several validated instruments to compose the survey, which was implemented both in Kenya and Zimbabwe studies (Cho et al., 2015; Hallfors et al., 2011; Hallfors et al., 2015). At baseline (2011) and the final follow up (2014), we collected biological samples for HIV and HSV-2 testing (Cho et al., 2017; Luseno, Hallfors, et al., 2014), and here use manufacturer thresholds for HSV-2 infection.

Intervention elements: school fees, uniforms, and nurse visits

An implementation “dosage” score was calculated for each of the three intervention elements: school fees, uniforms, and nurse visits. The resulting variables responded to the first research question (degree of implementation fidelity) and comprised the independent variables in question 2 (key elements predicting continued school enrollment). Dated payment documentation, monthly field staff reports, and staff logs indicated whether fees and uniforms had been paid according to protocol, the number of nurse and/or accountant visits, as well as the school each student was attending, the type of school (primary, secondary, or vocational), and whether the student was boarding or not.

Binary measures (1,0) for fees and uniforms were calculated based on whether participants received these supports as per protocol or not. For example, if a participant received fees every term they remained in school and then dropped out, implementation fidelity was “1," even though no fees were paid after they dropped out of school. If tuition was not paid in one or more terms while they were attending school, the dosage coded would be “0.” For uniforms, most participants received two over the three years: one in primary school and one in high school. But some youth transitioned to high school in 2012 and thus would not have received a primary school uniform—only a high school uniform. In both cases, uniform implementation would be coded “1.”

Implementation dosage of nurse and/or accountant visits was coded as the percentage of visits that should have been received (0–9/9*100%) over three years. Staff reports also indicated whether any student issues had been identified during the visits, and the timing and circumstances of school transfer, absence, and dropout to determine implementation fidelity and to better understand obstacles and facilitators of intervention implementation. We used these data, along with payment and spreadsheet case data (described below), to conduct implementation analyses, which sought to further describe and interpret implementation findings within the local context.

Boarding, terms in school, and survey covariates

If students boarded during high school in one or more years from 2012–2014, this was also noted as both a continuous (0–3, depending on the number of years in boarding school) and binary (1,0) variable. Boarding was dichotomously coded for bivariate analysis (Table 1) and continuously for multivariate analyses in response to research questions 2 and 3. Conceptually, boarding offers more intensive exposure to the school environment, and we examined the impact of boarding as a potential protective factor for continued school enrollment and for the prevention of incident HIV/HSV-2 (research questions 2 and 3).

Table 1.

Characteristics of Intervention Group Sample by Gender

Characteristics	TOTAL N (%) or Mean (range)	Females N (%) or Mean (range)	Male N (%) or Mean (range)	Gender Difference Chi-square/T-test
Gender	410 (100%)	197 (48%)	213 (52%)	NA
Age at baseline	14.8 (11–20)	14.7 (11–20)	14.9 (12–19)	T=−1.24 (p=0.20)
Double vs. Single orphan	N=146 (35.6%)	N=61 (31.0%)	N=85 (39.9%)	F=34.0 (p= ≤.001)**
SES at baseline	2.2 (1–5)	2.3 (1–5)	2.1 (1–5)	T=2.63 (p=01)**
Ever Boarded vs Never Boarded	N=155 (38%)	N=61 (31%)	N=94 (44%)	F=7.34 (p=0.01)**
Total Terms in School	7.96 (0–9)	7.76 (0–9)	8.14 (0–9)	T=−1.86 (p=0.06)
Attended all 9 terms vs < 9	N=298 (72.7%)	N=134 (68%)	N=164 (77%)	F=5.04 (p=0.02)*
All Fees Paid Appropriately	N=387 (94%)	N=183 (92%)	N=204 (95%)	F=1.60 (p=0.21)
All Uniforms Received	N=395 (96%)	N=190 (96%)	N=205 (96%)	F=0.01 (p=0.91)
% of Nurse Visits	79% (22%– 100%)	77% (22%– 100%)	81% (44%– 100%)	T=−2.91 (p=0.004)**
Surveyed at Final Data Collection	N=372 (91%)	N=172 (87%)	N=200 (94%)	F=3.34 (p=0.07)
New Combined HIV/HSV-2 Incidence at Final data collection	N=111 (31.1%)	N=61 (37.7%)	N=50 (25.6%)	F=5.57 (p=0.02)*
Baseline HIV/HSV-2	N=23 (5.6%)	N=14 (7.1%)	N=9 (4.2%)	F=1.97 (p=0.16)
New HIV Incidence at Final data collection	N=2 (0.5%)	N=1 (0.6%)	N=1 (0.5%)	F=0.01 (p=0.92)
Baseline HIV	N=5 (1.2%)	N=2 (1.0%)	N=3 (1.4%)	F=0.21 (p=0.64)
New HSV-2 incidence at Final Data Collection	N=110 (30.8%)	N=60 (37%)	N=50 (25.6%)	F=5.07 (0.02)*
Baseline HSV-2	N=18 (4.4%)	N=12 (6.1%)	N=6 (2.8%)	F=3.18 (p=0.07)

^*p<.05;

^**p<.01

SES–Socio-economic status, measured as self-reported meals/day

HIV–Human Immunodeficiency Virus

HSV–2 Herpes Simplex Virus Type 2

Number of terms in school was the dependent variable in research question 2 analyses, and the independent variable in question 3 analyses in which HIV/HSV-2 biomarker data were the dependent variables. Number of terms was calculated from school enrollment and dropout data, field staff reports, and tuition payment documentation. U.S. researchers entered these data into a spreadsheet by year and by term for each participant over the course of the study. Kenyan field staff, who were well-acquainted with all participants from regular contact, then reviewed the spreadsheet case by case and entered any known missing data and made any needed corrections.

For the second and third research questions, the following survey data variables were entered as covariates in analysis models: gender, age, orphan status (double vs single), and SES (meals per day) at baseline. Meals per day measured responses to the question “How many meals do you have in a day?” and was selected as a more sensitive measure of SES for rural adolescent orphans rather than the more common asset index, which tends to reflect community-level infrastructural development (e.g., electricity) as well as involvement in the cash economy and ownership of durable goods (Bingenheimer, 2007; Howe, Hargreaves, Ploubidis, De Stavola, & Huttly, 2011).

Statistical Analyses

After calculating the implementation of participant-level intervention elements, we conducted bivariate analyses on these and all other variables (dependent, independent, and covariates) to compare gender differences, using T-tests for continuous variables and Rao-Scott Chi-square tests for categorical variables. We then conducted regression analyses to evaluate the impact of intervention components and boarding on the number of terms stayed in school (Question 2). Finally, logistic regression analyses were conducted to examine the impact of longer school enrollment (number of terms) and greater school exposure (boarding) on new incidence of HIV and HSV-2 infection at study endline, dropping cases found to be infected at baseline (Question 3). In all models, age, gender (total sample only), SES, and orphan status were included as covariates; analyses were also run stratified by gender. Since students were nested within schools at baseline, survey procedures in SAS (SAS Institute Inc., 2013) were used to account for clustering.

Results

Table 1 provides an overview of sample characteristics, with comparisons by gender. The average age of intervention participants at baseline was 14.8 years. About 36% were double orphans; more boys (40%) than girls (31%) had lost both parents. Girls reported more meals/day (SES) than boys. Boys, on average, completed more terms in school than girls, although results were significant only when dichotomized. All school fees were paid according to protocol for 94% of students, and 96% received requisite uniforms. On average, participants received about 79% of requisite nurse visits; boys got more visits than girls. Girls were more likely than boys to be newly infected with HSV-2 at endline.

Of the three intervention variables, only nurse visits was positively associated with number of terms enrolled in school for the total sample, as well as for both genders (Table 2, Question 2). Uniforms was negatively associated with number of terms, for the full sample only. In other words, students who did not get all their requisite uniforms were in school for more terms, on average, than those who did, although the proportion who did not get their uniforms was very small. Fees was positively associated with terms enrolled for boys, but negatively associated for girls. Boarding was positively associated with terms enrolled for the total sample, and for both genders. Of the covariates, age and double orphan were negatively associated with terms in school.

Table 2.

Regression Analyses: Number of Terms Enrolled in School

	TOTAL SAMPLE N=404		FEMALE N=194		MALE N=210
	Estimate	P-Value	Estimate	P-Value	Estimate	P-Value
Intercept	10.06	<0.0001**	10.61	0.002**	9.19	0.004**
Age	-0.33	0.0003**	−0.41	0.009**	−0.28	0.005**
Gender (ref=male)	−0.22	0.34	NA	NA	NA	NA
SES	0.04	0.73	0.17	0.32	−0.04	0.77
Double Orphan (1,0)	−0.60	0.007**	−0.44	0.08	−0.68	0.02*
Fees (1,0)	−0.04	0.89	−0.82	0.03*	0.63	0.08
Nurse Visits (%)	0.02	0.001**	0.02	0.002**	0.02	0.006**
Uniforms (1,0)	−0.90	0.04*	−0.53	0.49	−0.81	0.08
Boarding (range=0–3 years)	0.41	0.0001**	0.39	0.004**	0.42	0.008**

^*p<.05;

^**p<.01

SES–Socio-economic status, measured as self-reported meals/day

Number of terms enrolled, however, was not associated with HIV/HSV-2 incidence (Table 3, Question 2). Boarding was associated with lower odds of new infection, but for boys only. Gender was marginally associated with HIV/HSV-2 incidence (p=.07); the odds of infection were 73% higher for girls than for boys.

Table 3.

Logistic Regression Analyses: NEW HIV/HSV-2 INCIDENCE

	TOTAL SAMPLE		FEMALE		MALE
	N=357		N=162		N=195
	OR (CI)	P-value	OR (CI)	P-value	OR (CI)	P-value
Age	0.99 (0.75–1.31)	0.96	0.98 (0.77–1.25)	0.88	1.01 (0.65–1.57)	0.96
Gender (ref=male)	1.73 (0.96–3.13)	0.07	NA	NA	NA	NA
SES	0.86 (0.53–1.39)	0.49	0.82 (0.43–1.60)	0.53	0.90 (0.62–1.32)	0.56
Double Orphan (1,0)	1.35 (0.90–2.03)	0.13	1.31 (0.76–2.28)	0.30	1.40 (0.51–3.81)	0.49
Number of terms (range = 0–9)	1.04 (0.87–1.23)	0.65	1.01 (0.84–1.21)	0.95	1.10 (0.84–1.44)	0.46
Boarding (range=0–3)	0.81 (0.67–0.97)	0.03*	0.89 (0.65–1.22)	0.43	0.73 (0.54–0.98)	0.04*

^*p<.05

SES–Socio-economic status, measured as self-reported meals/day

HIV–Human Immunodeficiency Virus

HSV–2 Herpes Simplex Virus Type 2

Implementation Analyses (Question 1)

Data show faithful implementation of school fees and uniforms per protocol for the vast majority of participants. Regarding fees, 6% of students (N=23) failed to receive fees according to protocol, but in most cases these were for relatively small amounts. For example, nine students lacked payment of the Grade 8 KCPE exam fee, which cost about $12 USD, but no other fee payments were missed for this group. Of the remaining 14 students, most transferred to another school without notifying study staff. After one or two terms, staff found them and resumed payments. High school fees averaged about $360 USD/year. About 38% of students boarded for one to three years at a cost ranging from $300–$1,400/year, while day school fees ranged from $100–$500/year. Technical schools, with training in skills such as tailoring, mechanics, agriculture, and hair dressing, varied from $90–$1,300/year, as some schools included boarding.

Similarly, 96% of participants received uniforms according to protocols. Of the rest (N=15), two chose to receive school support from another donor, who may or may not have provided uniforms. Most of the others had moved far out of the study area. The cost of uniforms varied from about $15 for a primary school uniform to $50 for a high school uniform. A dedicated accountant was essential to ensure payment of both school fees and uniforms.

Although providing fees and uniforms was challenging, nurse visits proved even more difficult to implement. In Kenya, youth in Grade 8 take a standardized national test called the Kenya Certificate of Primary Education (KCPE). The student selects three high schools they would like to attend the following January. Their KCPE scores are sent to these schools, and the students then wait to see if one or more of their selected high schools invite them to attend. Those who do well academically apply to more elite secondary schools which are often long distances away and require boarding. Intervention participants attended either Grade 7 or 8 in 13 primary schools at baseline (fall 2011), and by January 2012, they began scattering to secondary schools, which numbered 150 by 2014.

With so many schools, it was challenging to locate students who had moved or were absent, and sometimes the nurse RA had to wait hours to see students because teachers would not release them from class or exams. In most instances, the nurse RA made every attempt to find a participant who was absent from, or had left, school to find out what the problem was, whether they wanted to return to school, or whether they might prefer to attend a vocational or technical school. Hiring a full-time accountant increased the likelihood that schools and students would be visited each term; the nurse and accountant worked closely together to share information.

Discussion

Although five sub-Saharan clinical trials have sought to answer the question of whether support to stay in school results in a reduction of HIV/HSV-2 infection, positive findings on biomarkers have been found in only one (Baird et al., 2012). The parent Kenya RCT, along with the rest (Pettifor et al., 2016; Hallfors et al., 2015; Duflo et al., 2015), found no impact of school support on HIV or HSV-2 infection when intervention and control groups were compared. With process evaluation, we sought to answer three questions limited to intervention group data: Was the intervention implemented with fidelity? Were any of the elements more or less important in predicting whether youth stayed in school? Did longer enrollment in school reduce the likelihood of HIV/HSV-2 infection?

First, we found that implementation fidelity for school fees and uniforms was high (94–96%). Our findings suggest it is possible to successfully implement a fees and uniforms scholarship program but costs may be prohibitive. Secondary school fees are expensive, particularly for a developing country. Students with higher scholastic ability typically must board when attending schools of higher quality than the local day schools. Uniforms are also costly, particularly at the secondary level. Administrative costs to monitor effective implementation must likewise be taken into account.

Nurse RA visits were difficult to implement faithfully. We employed university-educated registered nurses who did an outstanding job tracking and helping students under difficult field conditions. They also had other research-related duties (e.g., consent, data collection, HSV-2 disclosure and follow-up) that took time away from regular school visits to monitor and assess the needs of intervention students. Yet nurse and accountant visits were essential in discovering student transfers and ensuring that fees and uniforms were paid at the new school. Nurses were also uniquely able to assess and address health problems that could impact on school attendance and performance.

In terms of our second question, only nurse visits and boarding were consistently and positively associated with staying in school. Uniform and tuition provision showed an odd and inconsistent association with number of school terms, possibly due to the small proportion of students who missed out on any appropriate payments. Nonetheless, given our main dependent variable of interest, the impact of individual intervention elements on longer school enrollment ultimately only matters if longer schooling prevents HIV infection and other sexually transmitted diseases highly associated with HIV, such as HSV-2 (Ferrand et al., 2010; Freeman et al., 2006; Glynn, Biraro, & Weiss, 2009). Although HIV incidence was very low, given the high incidence of new HSV-2 infection between baseline and endline, we should have detected any existing association between staying in school and combined infection.

The duration of school attendance, however, had no effect on the combined HIV/HSV-2 biomarker variable (Question 3). This suggests that the concept of school support as HIV prevention, while compelling, has little evidence to support it, among either boys or girls. If adolescents who drop out of school are at higher risk of sexually transmitted infection than non-dropouts, then fewer terms in school should have resulted in higher HIV/HSV-2 infection. Moreover, in RCT analyses, the intervention decreased the odds of school dropout, but failed to reduce the odds of sexual risk behaviors, such as early sexual debut, marriage, or pregnancy, and it did not increase condom use (Cho et al., 2017). The intervention did, however, decrease the odds of transactional sex (odds ratio=.47; p=.03), but the number reporting this behavior was low (about 8% of the full study sample). It also increased the odds of male circumcision (odds ratio=1.66; p=.04) (Cho et al., 2017), which may eventually have an impact on HIV prevention.

As in our RCT, the Malawi study of a large sample of female adolescents showed few effects on behavioral risk factors, but that study did find an HIV biomarker impact when they limited the sample to girls enrolled in school at baseline. The authors, however, urged caution in interpretation, given the low event rate for biological outcomes (17 HIV positive cases of 799 in the control group vs three of 235 in the intervention group) (Baird et al., 2012). Further, biomarkers were not collected at baseline, only at the end of the two-year intervention trial. Thus, some participants may have already been infected before the intervention was implemented, through either perinatal or sexual transmission. In contrast, the parent Kenya RCT (Cho et al., 2017), as well as the South Africa study (Pettifor et al., 2016), collected baseline biomarker samples and only new incidence of HIV and HSV-2 infection were examined in the endline analyses; both found no association.

It is possible that the beneficial impact of school support takes longer to materialize, as students grow older, but a clear rationale for investing in this line of research is lacking. While each of the extant school support/conditional cash transfer studies found some impact on a few sexual risk factors, these factors were inconsistent across studies, suggesting no clear underlying mechanism that might eventually impact HIV infection, particularly among women. Most previous studies did not include males, but we found that boarding school attendance was protective against HSV-2 infection for males but not for females. Further, the RCT found an increase in circumcision, which we tentatively attribute to mobilizers working with schools to offer circumcision to male students (Cho et al., 2017). Because the intervention group was more likely to be in school, they may have enjoyed the benefit of circumcision mobilization through schools. Further examination of this link is warranted.

Our study has several limitations. We limited our analyses to intervention group data for process evaluation. Since the control group received no treatment and we had more specific and objective information about intervention group school attendance, the current design most effectively addressed our three research questions. In earlier multivariate analyses, we included KCPE scores to multivariate models as a marker for school achievement and ability, because these constructs could confound the independent effect of boarding. KCPE was a significant predictor for terms in school in all three models, but boarding was significant for the total sample only. On the other hand, KCPE was not significant in predicting HIV/HSV-2 infection and had little effect on any of the models. We ultimately dropped the variable because it was problematic (i.e., portions of the sample took the Grade 8 KCPE in different years, some never took it all before dropping out of school, and others had missing data).

Although overall study retention of adolescent orphans was very high (91%), youth who remained enrolled for all 9 terms had higher retention at endline data collection (99%) than those who remained in school for less than 9 terms (69%). However, study retention at endline in the parent RCT was equivalent by condition and the control group had significantly higher school dropout than the intervention group, yet no differences in HIV/HSV-2 infection by condition were found (Cho et al., 2017). Finally, our sample size was reduced for the biomarker analyses because it included only participants who were HIV and HSV-2 negative at baseline and who were again tested at endline. Nevertheless, given the large proportion of participants with new HSV-2 infection incidence at endline, and the adequate sample size for multivariate analysis, there appears to be no association between HIV/HSV-2 infection and number of terms in school among intervention students for whom we have observed school data.

In conclusion, while the intervention was reasonably well-implemented and provided generous support to participants over three years, no impact was found on HIV/HSV-2 infection in the full trial. Based on extant research and our own findings, the evidence for school support as an effective HIV prevention strategy appears to be weak, although helping youth to stay in school very likely will provide other health and economic benefits (Gakidou, Cowling, Lozano, & Murray, 2010; Psacharaopoulos & Patrinos, 2004; Wyndow, Li, & Mattes, 2013). While nurse visits positively affected school enrollment for orphaned youth and better implementation may have further increased continuous enrollment for some intervention participants, the point is moot since no association between length of school enrollment and sexually transmitted infection was found, despite high incident HSV-2 infection. For males, however, the parent study finding of increased circumcision among the intervention group and the present study’s finding of an association between boarding and lower HSV-2 infection both suggest promising areas of future research for school-based HIV prevention.